This branch introduces support for reading the "Windows-wide" Git
configuration from `%PROGRAMDATA%\Git\config`. As these settings are
intended to be shared between *all* Git-related software, that config
file takes an even lower precedence than `$(prefix)/etc/gitconfig`.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Between the libgit2 and the Git for Windows project, there has been a
discussion how we could share Git configuration to avoid duplication (or
worse: skew).
Earlier, libgit2 was nice enough to just re-use Git for Windows'
C:\Program Files (x86)\Git\etc\gitconfig
but with the upcoming Git for Windows 2.x, there would be more paths to
search, as we will have 64-bit and 32-bit versions, and the
corresponding config files will be in %PROGRAMFILES%\Git\mingw64\etc and
...\mingw32\etc, respectively.
Worse: there are portable Git for Windows versions out there which live
in totally unrelated directories, still.
Therefore we came to a consensus to use `%PROGRAMDATA%\Git\config` as the
location for shared Git settings that are of wider interest than just Git
for Windows.
On XP, there is no %PROGRAMDATA%, therefore we need to use
"%ALLUSERSPROFILE%\Application Data\Git\config" in those setups.
Of course, the configuration in `%PROGRAMDATA%\Git\config` has the
widest reach, therefore it must take the lowest precedence, i.e. Git for
Windows can still override settings in its `etc/gitconfig` file.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The contract for the stat() and lstat() function is:
> stat(): stats the file pointed to by path and fills in buf.
> lstat(): is identical to stat(), except that if path is a symbolic link,
> then the link itself is stat-ed, not the file that it refers to.
stat() should always return the statistics of the file or directory a
symbolic link is pointing to. The lstat() function is used to get the
stats for the symlink. Hence the check should not be there.
Signed-off-by: Loris Chiocca <loris@chiocca.ch>
This is needed so that `_wchdir()` can be used with drive root
directories, e.g. C:\ (`_wchdir("C:")` fails to switch the directory
to the root directory).
This fixes https://github.com/msysgit/git/issues/359 (in Git for Windows
2.x only, though).
Likewise, `readlink()`'s semantics require a trailing slash for symbolic
links pointing to directories. Otherwise all checked out symbolic links
pointing to directories would be marked as modified even directly after a
fresh clone.
This fixes https://github.com/git-for-windows/git/issues/210
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Symlinks on Windows have a flag that indicates whether the target is a file
or a directory. Symlinks of wrong type simply don't work. This even affects
core Win32 APIs (e.g. DeleteFile() refuses to delete directory symlinks).
However, CreateFile() with FILE_FLAG_BACKUP_SEMANTICS doesn't seem to care.
Check the target type by first creating a tentative file symlink, opening
it, and checking the type of the resulting handle. If it is a directory,
recreate the symlink with the directory flag set.
It is possible to create symlinks before the target exists (or in case of
symlinks to symlinks: before the target type is known). If this happens,
create a tentative file symlink and postpone the directory decision: keep
a list of phantom symlinks to be processed whenever a new directory is
created in mingw_mkdir().
Limitations: This algorithm may fail if a link target changes from file to
directory or vice versa, or if the target directory is created in another
process.
Signed-off-by: Karsten Blees <blees@dcon.de>
Implement symlink() that always creates file symlinks. Fails with ENOSYS
if symlinks are disabled or unsupported.
Note: CreateSymbolicLinkW() was introduced with symlink support in Windows
Vista. For compatibility with Windows XP, we need to load it dynamically
and fail gracefully if it isnt's available.
Signed-off-by: Karsten Blees <blees@dcon.de>
Implement readlink() by reading NTFS reparse points. Works for symlinks
and directory junctions. If symlinks are disabled, fail with ENOSYS.
Signed-off-by: Karsten Blees <blees@dcon.de>
If symlinks are enabled, resolve all symlinks when changing directories,
as required by POSIX.
Note: Git's real_path() function bases its link resolution algorithm on
this property of chdir(). Unfortunately, the current directory on Windows
is limited to only MAX_PATH (260) characters. Therefore using symlinks and
long paths in combination may be problematic.
Note: GetFinalPathNameByHandleW() was introduced with symlink support in
Windows Vista. Thus, for compatibility with Windows XP, we need to load it
dynamically and behave gracefully if it isnt's available.
Signed-off-by: Karsten Blees <blees@dcon.de>
MSVCRT's _wrename() cannot rename symlinks over existing files: it returns
success without doing anything. Newer MSVCR*.dll versions probably do not
have this problem: according to CRT sources, they just call MoveFileEx()
with the MOVEFILE_COPY_ALLOWED flag.
Get rid of _wrename() and call MoveFileEx() with proper error handling.
Signed-off-by: Karsten Blees <blees@dcon.de>
_wunlink() / DeleteFileW() refuses to delete symlinks to directories. If
_wunlink() fails with ERROR_ACCESS_DENIED, try _wrmdir() as well.
Signed-off-by: Karsten Blees <blees@dcon.de>
Symlinks on Windows don't work the same way as on Unix systems. E.g. there
are different types of symlinks for directories and files, creating
symlinks requires administrative privileges etc.
By default, disable symlink support on Windows. I.e. users explicitly have
to enable it with 'git config [--system|--global] core.symlinks true'.
The test suite ignores system / global config files. Allow testing *with*
symlink support by checking if native symlinks are enabled in MSys2 (via
'MSYS=winsymlinks:nativestrict').
Reminder: This would need to be changed if / when we find a way to run the
test suite in a non-MSys-based shell (e.g. dash).
Signed-off-by: Karsten Blees <blees@dcon.de>
The retry pattern is duplicated in three places. It also seems to be too
hard to use: mingw_unlink() and mingw_rmdir() duplicate the code to retry,
and both of them do so incompletely. They also do not restore errno if the
user answers 'no'.
Introduce a retry_ask_yes_no() helper function that handles retry with
small delay, asking the user, and restoring errno.
mingw_unlink: include _wchmod in the retry loop (which may fail if the
file is locked exclusively).
mingw_rmdir: include special error handling in the retry loop.
Signed-off-by: Karsten Blees <blees@dcon.de>
Dynamic loading of DLL functions is duplicated in several places.
Add a set of macros to simplify the process.
Signed-off-by: Karsten Blees <blees@dcon.de>
Git typically doesn't trust the stat.st_size member of symlinks (e.g. see
strbuf_readlink()). However, some functions take shortcuts if st_size is 0
(e.g. diff_populate_filespec()).
In mingw_lstat() and fscache_lstat(), make sure to return an adequate size.
The extra overhead of opening and reading the reparse point to calculate
the exact size is not necessary, as git doesn't rely on the value anyway.
Signed-off-by: Karsten Blees <blees@dcon.de>
Move S_IFLNK detection to file_attr_to_st_mode() and reuse it in fscache.
Implement DT_LNK detection in dirent.c and the fscache readdir version.
Signed-off-by: Karsten Blees <blees@dcon.de>
When obtaining lstat information for reparse points, we need to call
FindFirstFile() in addition to GetFileInformationEx() to obtain the type
of the reparse point (symlink, mount point etc.). However, currently there
is no error handling whatsoever if FindFirstFile() fails.
Call FindFirstFile() before modifying the stat *buf output parameter and
error out if the call fails.
Note: The FindFirstFile() return value includes all the data that we get
from GetFileAttributesEx(), so we could replace GetFileAttributesEx() with
FindFirstFile(). We don't do that because GetFileAttributesEx() is about
twice as fast for single files. I.e. we only pay the extra cost of calling
FindFirstFile() in the rare case that we encounter a reparse point.
Note: The indentation of the remaining reparse point code will be fixed in
the next patch.
Signed-off-by: Karsten Blees <blees@dcon.de>
With the new mingw_stat() implementation, do_lstat() is only called from
mingw_lstat() (with follow == 0). Remove the extra function and the old
mingw_stat()-specific (follow == 1) logic.
Signed-off-by: Karsten Blees <blees@dcon.de>
With respect to symlinks, the current stat() implementation is almost the
same as lstat(): except for the file type (st_mode & S_IFMT), it returns
information about the link rather than the target.
Implement stat by opening the file with as little permissions as possible
and calling GetFileInformationByHandle on it. This way, all link resoltion
is handled by the Windows file system layer.
If symlinks are disabled, use lstat() as before, but fail with ELOOP if a
symlink would have to be resolved.
Signed-off-by: Karsten Blees <blees@dcon.de>
GetFileAttributes cannot handle paths with trailing dir separator. The
current [l]stat implementation calls GetFileAttributes twice if the path
has trailing slashes (first with the original path passed to [l]stat, and
and a second time with a path copy with trailing '/' removed).
With Unicode conversion, we get the length of the path for free and also
have a (wide char) buffer that can be modified.
Remove trailing directory separators before calling the Win32 API.
Signed-off-by: Karsten Blees <blees@dcon.de>
POSIX semantics requires lstat() to fail with ENOTDIR when "[a]
component of the path prefix names an existing file that is neither a
directory nor a symbolic link to a directory".
See http://pubs.opengroup.org/onlinepubs/9699919799/functions/lstat.html
This behavior is expected by t1404-update-ref-df-conflicts now.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
7ebac8cb94 made launching of .exe
externals work when installed in Unicode paths. But it broke launching
of non-.exe externals, no matter where they were installed. We now
correctly maintain the UTF-8 and UTF-16 paths in tandem in lookup_prog.
This fixes t5526, among others.
Signed-off-by: Adam Roben <adam@roben.org>
If Git were installed in a path containing non-ASCII characters,
commands such as git-am and git-submodule, which are implemented as
externals, would fail to launch with the following error:
> fatal: 'am' appears to be a git command, but we were not
> able to execute it. Maybe git-am is broken?
This was due to lookup_prog not being Unicode-aware. It was somehow
missed in 2ee5a1a14a.
Note that the only problem in this function was calling
GetFileAttributes instead of GetFileAttributesW. The calls to access()
were fine because access() is a macro which resolves to mingw_access,
which already handles Unicode correctly. But I changed lookup_prog to
use _waccess directly so that we only convert the path to UTF-16 once.
Signed-off-by: Adam Roben <adam@roben.org>
On Windows XP3 in git bash
git clone git@github.com:octocat/Spoon-Knife.git
cd Spoon-Knife
git gui
menu Remote\Fetch from\origin
error: cannot spawn git: No such file or directory
error: could not run rev-list
if u run
git fetch --all
it worked normal in git bash or gitgui tools
In second version CreateProcess get 'C:\Git\libexec\git-core/git.exe' in
first version - C:/Git/libexec/git-core/git.exe and not executes (unix
slashes)
after fixing C:\Git\libexec\git-core\git.exe or
C:/Git/libexec/git-core\git.exe it works normal
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Since baaf233 (connect: improve check for plink to reduce false
positives, 2015-04-26), t5601 writes out a `plink.exe` for testing that
is actually a shell script. So the assumption that the `.exe` extension
implies that the file is *not* a shell script is now wrong.
The original idea to special-case `.exe` files was probably to
help performance, but since we are in a code path that involves spawning
a new process (which in and of itself is pretty slow on Windows anyway),
we pursue a better idea to improve performance elsewhere: we try to
convert scripts into builtins and to reduce the number of spawned
processes by adding more internal API calls.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
HOME initialization was historically duplicated in many different places,
including /etc/profile, launch scripts such as git-bash.vbs and gitk.cmd,
and (although slightly broken) in the git-wrapper.
Even unrelated projects such as GitExtensions and TortoiseGit need to
implement the same logic to be able to call git directly.
Initialize HOME in git's own startup code so that we can eventually retire
all the duplicate initialization code.
Signed-off-by: Karsten Blees <blees@dcon.de>
Lets keep the environment initialization and conversion section as lean as
possible and move recently added tweaks to setup_windows_environment().
This fixes the following potential problems:
* Prevent duplicate TZ variables if both TZ and MSYS2_TZ are set.
* Some of the higher level x* APIs from wrapper.c require a working
getenv(), using e.g. xstrdup() during initialization is dangerous.
* Slashifying the Windows TMP variable may break native Windows programs,
use POSIX TMPDIR instead.
* Properly slashify TMPDIR even if it is already set, and also if we only
have TEMP, but not TMP.
* Reduce complexity from O(n) to O(log n).
Signed-off-by: Karsten Blees <blees@dcon.de>
This developer should really, really have known better. The fact that we
are changing the environment in ways for which the MSVCRT is not
prepared for is bad enough. But then this developer followed the request
to re-enable nedmalloc -- despite the prediction that it would cause an
access violation, predicting it in the same message as the request to
re-enable nedmalloc, no less!
To paper over the issue until the time when this developer finds the
time to re-design the Unicode environment handling from scratch, let's
hope that cURL is the only library we are using that *may* set an
environment variable using MSVCRT's putenv() after we fscked the
environment up.
Note: this commit can serve as no source of pride to anyone, certainly
not yours truly. It is necessary as a quick and pragmatic stop gap,
though, to prevent worse problems.
Note: cURL manages to set the variable CHARSET when nedmalloc is *not*
enabled, without causing an access violation. In that case, it sets it
successfully to the value "cp" + GetACP() (hence it is our choice, too,
cURL may need it, Git does not):
https://github.com/bagder/curl/blob/aa5808b5/lib/easy.c#L157-L162
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
HANDLE is defined internally as a void *, but in many cases it is
actually guaranteed to be a 32-bit integer. In these cases, GCC should
not warn about a cast of a pointer to an integer of a different type
because we know exactly what we are doing.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The environment is modified in most surprising circumstances, and not
all of them are under Git's control. For example, calling
curl_global_init() on Windows will ensure that the CHARSET variable is
set, adding one if necessary.
While the previous commit worked around crashes triggered by such
outside changes of the environment by relaxing the requirement that the
environment be terminated by a NULL pointer, the other assumption made
by `mingw_getenv()` and `mingw_putenv()` is that the environment is
sorted, for efficient lookup via binary search.
Let's make real sure that our environment is intact before querying or
modifying it, and reinitialize our idea of the environment if necessary.
With this commit, before working on the environment we look briefly for
indicators that the environment was modified outside of our control, and
to ensure that it is terminated with a NULL pointer and sorted again in
that case.
Note: the indicators are maybe not sufficient. For example, when a
variable is removed, it will not be noticed. It might also be a problem
if outside changes to the environment result in a modified `environ`
pointer: it is unclear whether such a modification could result in a
problem when `mingw_putenv()` needs to `realloc()` the environment
buffer.
For the moment, however, the current fix works well enough, so let's
only face the potential problems when (and if!) they occur.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Outside of our Windows-specific code, the end of the environment can be
marked also by a pointer to a NUL character, not only by a NULL pointer
as our code assumed so far.
That led to a buffer overrun in `make_environment_block()` when running
`git-remote-https` in `mintty` (because `curl_global_init()` added the
`CHARSET` environment variable *outside* of `mingw_putenv()`, ending the
environment in a pointer to an empty string).
Side note for future debugging on Windows: when running programs in
`mintty`, the standard input/output/error is not connected to a Win32
Console, but instead is pipe()d. That means that even stderr may not be
written completely before a crash, but has to be fflush()ed explicitly.
For example, when debugging crashes, the developer should insert an
`fflush(stderr);` at the end of the `error()` function defined in
usage.c.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It is unlikely that we have an empty environment, ever, but *if* we do,
when `environ_size - 1` is passed to `bsearchenv()` it is misinterpreted
as a real large integer.
To make the code truly defensive, refuse to do anything at all if the
size is negative (which should not happen, of course).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When the rename function tries to move a directory it fails if the target
directory exists. It should check if it can delete the (possibly empty)
target directory and then try again to move the directory.
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: 마누엘 <nalla@users.noreply.github.com>
When shell scripts access a $TMP variable containing backslashes, they
will be mistaken for escape characters. Let's not let that happen by
converting them to forward slashes.
This fixes t7800 with MSys2.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
With MSys2, the "TZ" environment variable gets filtered out when calling
non-MSys2 executables. The reason is that Windows' time zone handling is
substantially different from the POSIX one.
However, we just taught Git for Windows' fork of the MSys2 runtime to
pass on the timezone in a different environment variable, MSYS2_TZ for
the sole purpose of Git being able to reinterpret it correctly.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
MSys2's strace facility is very useful for debugging... With this patch,
the bash will be executed through strace if the environment variable
GIT_STRACE_COMMANDS is set, which comes in real handy when investigating
issues in the test suite.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Windows paths are typically limited to MAX_PATH = 260 characters, even
though the underlying NTFS file system supports paths up to 32,767 chars.
This limitation is also evident in Windows Explorer, cmd.exe and many
other applications (including IDEs).
Particularly annoying is that most Windows APIs return bogus error codes
if a relative path only barely exceeds MAX_PATH in conjunction with the
current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the
infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG.
Many Windows wide char APIs support longer than MAX_PATH paths through the
file namespace prefix ('\\?\' or '\\?\UNC\') followed by an absolute path.
Notable exceptions include functions dealing with executables and the
current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as
well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...).
Introduce a handle_long_path function to check the length of a specified
path properly (and fail with ENAMETOOLONG), and to optionally expand long
paths using the '\\?\' file namespace prefix. Short paths will not be
modified, so we don't need to worry about device names (NUL, CON, AUX).
Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be
limited to MAX_PATH (at least not on Win7), so we can use it to do the
heavy lifting of the conversion (translate '/' to '\', eliminate '.' and
'..', and make an absolute path).
Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH
limit.
Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs
that support long paths.
While improved error checking is always active, long paths support must be
explicitly enabled via 'core.longpaths' option. This is to prevent end
users to shoot themselves in the foot by checking out files that Windows
Explorer, cmd/bash or their favorite IDE cannot handle.
Test suite:
Test the case is when the full pathname length of a dir is close
to 260 (MAX_PATH).
Bug report and an original reproducer by Andrey Rogozhnikov:
https://github.com/msysgit/git/pull/122#issuecomment-43604199
[jes: adjusted test number to avoid conflicts]
Thanks-to: Martin W. Kirst <maki@bitkings.de>
Thanks-to: Doug Kelly <dougk.ff7@gmail.com>
Signed-off-by: Karsten Blees <blees@dcon.de>
Original-test-by: Andrey Rogozhnikov <rogozhnikov.andrey@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite
slow. Windows operating system APIs seem to be much better at scanning the
status of entire directories than checking single files. A caching
implementation may improve performance by bulk-reading entire directories
or reusing data obtained via opendir / readdir.
Make the lstat implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.
Signed-off-by: Karsten Blees <blees@dcon.de>
At least for cross-platform projects, it makes sense to hide the
files starting with a dot, as this is the behavior on Unix/MacOSX.
However, at least Eclipse has problems interpreting the hidden flag
correctly, so the default is to hide only the .git/ directory.
The config setting core.hideDotFiles therefore supports not only
'true' and 'false', but also 'dotGitOnly'.
[jes: clarified the commit message, made git init respect the setting
by marking the .git/ directory only after reading the config, and added
documentation, and rebased on top of current junio/next]
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The code to open and test the second end of the pipe clearly imitates
the code for the first end. A little too closely, though... Let's fix
the obvious copy-edit bug.
Signed-off-by: Jose F. Morales <jfmcjf@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
According to the Linux open(2) man page, open() must return EISDIR
if a directory was attempted to be opened for writing. Our emulation
in mingw_open() does not get this right: it checks only for O_CREAT.
Fix it to check for a write request.
This fixes a failure in reflog handling, which opens files with
O_APPEND|O_WRONLY, but without O_CREAT, and expects EISDIR when the
named file happens to be a directory.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Most of these are battle-tested in msysgit and are needed to
complete what has been merged to 'master' already.
* sk/mingw-uni-fix-more:
Win32: enable color output in Windows cmd.exe
Win32: patch Windows environment on startup
Win32: keep the environment sorted
Win32: use low-level memory allocation during initialization
Win32: reduce environment array reallocations
Win32: don't copy the environment twice when spawning child processes
Win32: factor out environment block creation
Win32: unify environment function names
Win32: unify environment case-sensitivity
Win32: fix environment memory leaks
Win32: Unicode environment (incoming)
Win32: Unicode environment (outgoing)
Revert "Windows: teach getenv to do a case-sensitive search"
tests: do not pass iso8859-1 encoded parameter
