This branch introduces support for reading the "Windows-wide" Git
configuration from `%PROGRAMDATA%\Git\config`. As these settings are
intended to be shared between *all* Git-related software, that config
file takes an even lower precedence than `$(prefix)/etc/gitconfig`.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Between the libgit2 and the Git for Windows project, there has been a
discussion how we could share Git configuration to avoid duplication (or
worse: skew).
Earlier, libgit2 was nice enough to just re-use Git for Windows'
C:\Program Files (x86)\Git\etc\gitconfig
but with the upcoming Git for Windows 2.x, there would be more paths to
search, as we will have 64-bit and 32-bit versions, and the
corresponding config files will be in %PROGRAMFILES%\Git\mingw64\etc and
...\mingw32\etc, respectively.
Worse: there are portable Git for Windows versions out there which live
in totally unrelated directories, still.
Therefore we came to a consensus to use `%PROGRAMDATA%\Git\config` as the
location for shared Git settings that are of wider interest than just Git
for Windows.
On XP, there is no %PROGRAMDATA%, therefore we need to use
"%ALLUSERSPROFILE%\Application Data\Git\config" in those setups.
Of course, the configuration in `%PROGRAMDATA%\Git\config` has the
widest reach, therefore it must take the lowest precedence, i.e. Git for
Windows can still override settings in its `etc/gitconfig` file.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We no longer use any of MSVCRT's stat-functions, so there's no need to
stick to a CRT-compatible 'struct stat' either.
Define and use our own POSIX-2013-compatible 'struct stat' with nanosecond-
precision file times.
Note: Due to performance issues when using git variants with different file
time resolutions, this patch does *not* yet enable nanosecond precision in
the Makefile (use 'make USE_NSEC=1').
Signed-off-by: Karsten Blees <blees@dcon.de>
The contract for the stat() and lstat() function is:
> stat(): stats the file pointed to by path and fills in buf.
> lstat(): is identical to stat(), except that if path is a symbolic link,
> then the link itself is stat-ed, not the file that it refers to.
stat() should always return the statistics of the file or directory a
symbolic link is pointing to. The lstat() function is used to get the
stats for the symlink. Hence the check should not be there.
Signed-off-by: Loris Chiocca <loris@chiocca.ch>
fstat() is the only stat-related CRT function for which we don't have a
full replacement yet (and thus the only reason to stick with MSVCRT's
'struct stat' definition).
Fully implement fstat(), in preparation of implementing a POSIX 2013
compatible 'struct stat' with nanosecond-precision file times.
Signed-off-by: Karsten Blees <blees@dcon.de>
This is needed so that `_wchdir()` can be used with drive root
directories, e.g. C:\ (`_wchdir("C:")` fails to switch the directory
to the root directory).
This fixes https://github.com/msysgit/git/issues/359 (in Git for Windows
2.x only, though).
Likewise, `readlink()`'s semantics require a trailing slash for symbolic
links pointing to directories. Otherwise all checked out symbolic links
pointing to directories would be marked as modified even directly after a
fresh clone.
This fixes https://github.com/git-for-windows/git/issues/210
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Symlinks on Windows have a flag that indicates whether the target is a file
or a directory. Symlinks of wrong type simply don't work. This even affects
core Win32 APIs (e.g. DeleteFile() refuses to delete directory symlinks).
However, CreateFile() with FILE_FLAG_BACKUP_SEMANTICS doesn't seem to care.
Check the target type by first creating a tentative file symlink, opening
it, and checking the type of the resulting handle. If it is a directory,
recreate the symlink with the directory flag set.
It is possible to create symlinks before the target exists (or in case of
symlinks to symlinks: before the target type is known). If this happens,
create a tentative file symlink and postpone the directory decision: keep
a list of phantom symlinks to be processed whenever a new directory is
created in mingw_mkdir().
Limitations: This algorithm may fail if a link target changes from file to
directory or vice versa, or if the target directory is created in another
process.
Signed-off-by: Karsten Blees <blees@dcon.de>
Implement symlink() that always creates file symlinks. Fails with ENOSYS
if symlinks are disabled or unsupported.
Note: CreateSymbolicLinkW() was introduced with symlink support in Windows
Vista. For compatibility with Windows XP, we need to load it dynamically
and fail gracefully if it isnt's available.
Signed-off-by: Karsten Blees <blees@dcon.de>
Implement readlink() by reading NTFS reparse points. Works for symlinks
and directory junctions. If symlinks are disabled, fail with ENOSYS.
Signed-off-by: Karsten Blees <blees@dcon.de>
If symlinks are enabled, resolve all symlinks when changing directories,
as required by POSIX.
Note: Git's real_path() function bases its link resolution algorithm on
this property of chdir(). Unfortunately, the current directory on Windows
is limited to only MAX_PATH (260) characters. Therefore using symlinks and
long paths in combination may be problematic.
Note: GetFinalPathNameByHandleW() was introduced with symlink support in
Windows Vista. Thus, for compatibility with Windows XP, we need to load it
dynamically and behave gracefully if it isnt's available.
Signed-off-by: Karsten Blees <blees@dcon.de>
MSVCRT's _wrename() cannot rename symlinks over existing files: it returns
success without doing anything. Newer MSVCR*.dll versions probably do not
have this problem: according to CRT sources, they just call MoveFileEx()
with the MOVEFILE_COPY_ALLOWED flag.
Get rid of _wrename() and call MoveFileEx() with proper error handling.
Signed-off-by: Karsten Blees <blees@dcon.de>
_wunlink() / DeleteFileW() refuses to delete symlinks to directories. If
_wunlink() fails with ERROR_ACCESS_DENIED, try _wrmdir() as well.
Signed-off-by: Karsten Blees <blees@dcon.de>
Symlinks on Windows don't work the same way as on Unix systems. E.g. there
are different types of symlinks for directories and files, creating
symlinks requires administrative privileges etc.
By default, disable symlink support on Windows. I.e. users explicitly have
to enable it with 'git config [--system|--global] core.symlinks true'.
The test suite ignores system / global config files. Allow testing *with*
symlink support by checking if native symlinks are enabled in MSys2 (via
'MSYS=winsymlinks:nativestrict').
Reminder: This would need to be changed if / when we find a way to run the
test suite in a non-MSys-based shell (e.g. dash).
Signed-off-by: Karsten Blees <blees@dcon.de>
The retry pattern is duplicated in three places. It also seems to be too
hard to use: mingw_unlink() and mingw_rmdir() duplicate the code to retry,
and both of them do so incompletely. They also do not restore errno if the
user answers 'no'.
Introduce a retry_ask_yes_no() helper function that handles retry with
small delay, asking the user, and restoring errno.
mingw_unlink: include _wchmod in the retry loop (which may fail if the
file is locked exclusively).
mingw_rmdir: include special error handling in the retry loop.
Signed-off-by: Karsten Blees <blees@dcon.de>
Git typically doesn't trust the stat.st_size member of symlinks (e.g. see
strbuf_readlink()). However, some functions take shortcuts if st_size is 0
(e.g. diff_populate_filespec()).
In mingw_lstat() and fscache_lstat(), make sure to return an adequate size.
The extra overhead of opening and reading the reparse point to calculate
the exact size is not necessary, as git doesn't rely on the value anyway.
Signed-off-by: Karsten Blees <blees@dcon.de>
Move S_IFLNK detection to file_attr_to_st_mode() and reuse it in fscache.
Implement DT_LNK detection in dirent.c and the fscache readdir version.
Signed-off-by: Karsten Blees <blees@dcon.de>
When obtaining lstat information for reparse points, we need to call
FindFirstFile() in addition to GetFileInformationEx() to obtain the type
of the reparse point (symlink, mount point etc.). However, currently there
is no error handling whatsoever if FindFirstFile() fails.
Call FindFirstFile() before modifying the stat *buf output parameter and
error out if the call fails.
Note: The FindFirstFile() return value includes all the data that we get
from GetFileAttributesEx(), so we could replace GetFileAttributesEx() with
FindFirstFile(). We don't do that because GetFileAttributesEx() is about
twice as fast for single files. I.e. we only pay the extra cost of calling
FindFirstFile() in the rare case that we encounter a reparse point.
Note: The indentation of the remaining reparse point code will be fixed in
the next patch.
Signed-off-by: Karsten Blees <blees@dcon.de>
With the new mingw_stat() implementation, do_lstat() is only called from
mingw_lstat() (with follow == 0). Remove the extra function and the old
mingw_stat()-specific (follow == 1) logic.
Signed-off-by: Karsten Blees <blees@dcon.de>
With respect to symlinks, the current stat() implementation is almost the
same as lstat(): except for the file type (st_mode & S_IFMT), it returns
information about the link rather than the target.
Implement stat by opening the file with as little permissions as possible
and calling GetFileInformationByHandle on it. This way, all link resoltion
is handled by the Windows file system layer.
If symlinks are disabled, use lstat() as before, but fail with ELOOP if a
symlink would have to be resolved.
Signed-off-by: Karsten Blees <blees@dcon.de>
GetFileAttributes cannot handle paths with trailing dir separator. The
current [l]stat implementation calls GetFileAttributes twice if the path
has trailing slashes (first with the original path passed to [l]stat, and
and a second time with a path copy with trailing '/' removed).
With Unicode conversion, we get the length of the path for free and also
have a (wide char) buffer that can be modified.
Remove trailing directory separators before calling the Win32 API.
Signed-off-by: Karsten Blees <blees@dcon.de>
We introduced helper macros to simplify loading functions dynamically.
Might just as well use them.
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We introduced helper macros to simplify loading functions dynamically.
Might just as well use them.
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When a user is registered in a Windows domain, it is really easy to
obtain the email address. So let's do that.
Suggested by Lutz Roeder.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We do have the excellent GetUserInfoEx() function to obtain more
detailed information of the current user (if the user is part of a
Windows domain); Let's use it.
Suggested by Lutz Roeder.
To avoid the cost of loading Secur32.dll (even lazily, loading DLLs
takes a non-neglibile amount of time), we use the established technique
to load DLLs only when, and if, needed.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This topic branch introduces a highly-experimental feature allowing to
override stdin/stdout/stderr by setting environment variables e.g. to
named pipes, solving a problem in highly multi-threaded applications
where inheritable handles could cause blocked Git operations.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When we access IPv6-related functions, we load the corresponding system
library using the `LoadLibrary()` function, which is not the recommended
way to load system libraries.
In practice, it does not make a difference: the `ws2_32.dll` library
containing the IPv6 functions is already loaded into memory, so
LoadLibrary() simply reuses the already-loaded library.
Still, recommended way is recommended way, so let's use that instead.
While at it, also adjust the code in contrib/ that loads system libraries.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Accessing the Windows console through the special CONIN$ / CONOUT$ devices
doesn't work properly for non-ASCII usernames an passwords.
It also doesn't work for terminal emulators that hide the native console
window (such as mintty), and 'TERM=xterm*' is not necessarily a reliable
indicator for such terminals.
The new shell_prompt() function, on the other hand, works fine for both
MSys1 and MSys2, in native console windows as well as mintty, and properly
supports Unicode. It just needs bash on the path (for 'read -s', which is
bash-specific).
On Windows, try to use the shell to read from the terminal. If that fails
with ENOENT (i.e. bash was not found), use CONIN/OUT as fallback.
Note: To test this, create a UTF-8 credential file with non-ASCII chars,
e.g. in git-bash: 'echo url=http://täst.com > cred.txt'. Then in git-cmd,
'git credential fill <cred.txt' works (shell version), while calling git
without the git-wrapper (i.e. 'mingw64\bin\git credential fill <cred.txt')
mangles non-ASCII chars in both console output and input.
Signed-off-by: Karsten Blees <blees@dcon.de>
The `git_terminal_prompt()` function expects the terminal window to be
attached to a Win32 Console. However, this is not the case with terminal
windows other than `cmd.exe`'s, e.g. with MSys2's own `mintty`.
Non-cmd terminals such as `mintty` still have to have a Win32 Console
to be proper console programs, but have to hide the Win32 Console to
be able to provide more flexibility (such as being resizeable not only
vertically but also horizontally). By writing to that Win32 Console,
`git_terminal_prompt()` manages only to send the prompt to nowhere and
to wait for input from a Console to which the user has no access.
This commit introduces a function specifically to support `mintty` -- or
other terminals that are compatible with MSys2's `/dev/tty` emulation. We
use the `TERM` environment variable as an indicator for that: if the value
starts with "xterm" (such as `mintty`'s "xterm_256color"), we prefer to
let `xterm_prompt()` handle the user interaction.
The most prominent user of `git_terminal_prompt()` is certainly
`git-remote-https.exe`. It is an interesting use case because both
`stdin` and `stdout` are redirected when Git calls said executable, yet
it still wants to access the terminal.
When running inside a `mintty`, the terminal is not accessible to the
`git-remote-https.exe` program, though, because it is a MinGW program
and the `mintty` terminal is not backed by a Win32 console.
To solve that problem, we simply call out to the shell -- which is an
*MSys2* program and can therefore access `/dev/tty`.
Helped-by: nalla <nalla@hamal.uberspace.de>
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
HOME initialization was historically duplicated in many different places,
including /etc/profile, launch scripts such as git-bash.vbs and gitk.cmd,
and (although slightly broken) in the git-wrapper.
Even unrelated projects such as GitExtensions and TortoiseGit need to
implement the same logic to be able to call git directly.
Initialize HOME in git's own startup code so that we can eventually retire
all the duplicate initialization code.
Signed-off-by: Karsten Blees <blees@dcon.de>
This developer should really, really have known better. The fact that we
are changing the environment in ways for which the MSVCRT is not
prepared for is bad enough. But then this developer followed the request
to re-enable nedmalloc -- despite the prediction that it would cause an
access violation, predicting it in the same message as the request to
re-enable nedmalloc, no less!
To paper over the issue until the time when this developer finds the
time to re-design the Unicode environment handling from scratch, let's
hope that cURL is the only library we are using that *may* set an
environment variable using MSVCRT's putenv() after we fscked the
environment up.
Note: this commit can serve as no source of pride to anyone, certainly
not yours truly. It is necessary as a quick and pragmatic stop gap,
though, to prevent worse problems.
Note: cURL manages to set the variable CHARSET when nedmalloc is *not*
enabled, without causing an access violation. In that case, it sets it
successfully to the value "cp" + GetACP() (hence it is our choice, too,
cURL may need it, Git does not):
https://github.com/bagder/curl/blob/aa5808b5/lib/easy.c#L157-L162
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The environment is modified in most surprising circumstances, and not
all of them are under Git's control. For example, calling
curl_global_init() on Windows will ensure that the CHARSET variable is
set, adding one if necessary.
While the previous commit worked around crashes triggered by such
outside changes of the environment by relaxing the requirement that the
environment be terminated by a NULL pointer, the other assumption made
by `mingw_getenv()` and `mingw_putenv()` is that the environment is
sorted, for efficient lookup via binary search.
Let's make real sure that our environment is intact before querying or
modifying it, and reinitialize our idea of the environment if necessary.
With this commit, before working on the environment we look briefly for
indicators that the environment was modified outside of our control, and
to ensure that it is terminated with a NULL pointer and sorted again in
that case.
Note: the indicators are maybe not sufficient. For example, when a
variable is removed, it will not be noticed. It might also be a problem
if outside changes to the environment result in a modified `environ`
pointer: it is unclear whether such a modification could result in a
problem when `mingw_putenv()` needs to `realloc()` the environment
buffer.
For the moment, however, the current fix works well enough, so let's
only face the potential problems when (and if!) they occur.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Outside of our Windows-specific code, the end of the environment can be
marked also by a pointer to a NUL character, not only by a NULL pointer
as our code assumed so far.
That led to a buffer overrun in `make_environment_block()` when running
`git-remote-https` in `mintty` (because `curl_global_init()` added the
`CHARSET` environment variable *outside* of `mingw_putenv()`, ending the
environment in a pointer to an empty string).
Side note for future debugging on Windows: when running programs in
`mintty`, the standard input/output/error is not connected to a Win32
Console, but instead is pipe()d. That means that even stderr may not be
written completely before a crash, but has to be fflush()ed explicitly.
For example, when debugging crashes, the developer should insert an
`fflush(stderr);` at the end of the `error()` function defined in
usage.c.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It is unlikely that we have an empty environment, ever, but *if* we do,
when `environ_size - 1` is passed to `bsearchenv()` it is misinterpreted
as a real large integer.
To make the code truly defensive, refuse to do anything at all if the
size is negative (which should not happen, of course).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
On Windows >= Vista, not having an application manifest with a
requestedExecutionLevel can cause several kinds of confusing behavior.
The first and more obvious behavior is "Installer Detection", where
Windows sometimes decides (by looking at things like the file name and
even sequences of bytes within the executable) that an executable is an
installer and should run elevated (causing the well-known popup dialog
to appear). In Git's context, subcommands such as "git patch-id" or "git
update-index" fall prey to this behavior.
The second and more confusing behavior is "File Virtualization". It
means that when files are written without having write permission, it
does not fail (as expected), but they are instead redirected to
somewhere else. When the files are read, the original contents are
returned, though, not the ones that were just written somewhere else.
Even more confusing, not all write accesses are redirected; Trying to
write to write-protected .exe files, for example, will fail instead of
redirecting.
In addition to being unwanted behavior, File Virtualization causes
dramatic slowdowns in Git (see for instance
http://code.google.com/p/msysgit/issues/detail?id=320).
There are two ways to prevent those two behaviors: Either you embed an
application manifest within all your executables, or you add an external
manifest (a file with the same name followed by .manifest) to all your
executables. Since Git's builtins are hardlinked (or copied), it is
simpler and more robust to embed a manifest.
A recent enough MSVC compiler should already embed a working internal
manifest, but for MinGW you have to do so by hand.
Very lightly tested on Wine, where like on Windows XP it should not make
any difference.
References:
- New UAC Technologies for Windows Vista
http://msdn.microsoft.com/en-us/library/bb756960.aspx
- Create and Embed an Application Manifest (UAC)
http://msdn.microsoft.com/en-us/library/bb756929.aspx
[js: simplified the embedding dramatically by reusing Git for Windows'
existing Windows resource file, removed the optional (and dubious)
processorArchitecture attribute of the manifest's assemblyIdentity
section.]
Signed-off-by: Cesar Eduardo Barros <cesarb@cesarb.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The unsetenv code has no idea to update our environ_size, therefore
causing segmentation faults when environment variables are removed
without compat/mingw.c's knowing (MinGW's optimized lookup would try
to strcmp() against NULL in such a case).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
MSys2's strace facility is very useful for debugging... With this patch,
the bash will be executed through strace if the environment variable
GIT_STRACE_COMMANDS is set, which comes in real handy when investigating
issues in the test suite.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
With this topic branch, the PERL5LIB variable is unset to avoid external
settings from interfering with Git's own Perl interpreter.
This branch also cleans up some of our Windows-only config setting code
(and this will need to be rearranged in the next merging rebase so that
the cleanup comes first, and fscache and longPaths support build on
top).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Dynamic loading of DLL functions is duplicated in several places.
Add a set of macros to simplify the process.
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The "2>&1" notation in POSIX shells implies that stderr is redirected to
stdout. Let's special-case this value for the environment variable
GIT_REDIRECT_STDERR to allow writing to the same destination as stdout.
The functionality was suggested by Jeff Hostetler.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
