Commit Graph

525 Commits

Author SHA1 Message Date
Karsten Blees
abfc64760f Win32: add symlink-specific error codes
Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:44:06 +01:00
Karsten Blees
8d14118743 Win32: change default of 'core.symlinks' to false
Symlinks on Windows don't work the same way as on Unix systems. E.g. there
are different types of symlinks for directories and files, creating
symlinks requires administrative privileges etc.

By default, disable symlink support on Windows. I.e. users explicitly have
to enable it with 'git config [--system|--global] core.symlinks true'.

The test suite ignores system / global config files. Allow testing *with*
symlink support by checking if native symlinks are enabled in MSys2 (via
'MSYS=winsymlinks:nativestrict').

Reminder: This would need to be changed if / when we find a way to run the
test suite in a non-MSys-based shell (e.g. dash).

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:44:05 +01:00
Karsten Blees
82b70cd01c Win32: factor out retry logic
The retry pattern is duplicated in three places. It also seems to be too
hard to use: mingw_unlink() and mingw_rmdir() duplicate the code to retry,
and both of them do so incompletely. They also do not restore errno if the
user answers 'no'.

Introduce a retry_ask_yes_no() helper function that handles retry with
small delay, asking the user, and restoring errno.

mingw_unlink: include _wchmod in the retry loop (which may fail if the
file is locked exclusively).

mingw_rmdir: include special error handling in the retry loop.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:44:03 +01:00
Karsten Blees
2004aca8d1 Win32: simplify loading of DLL functions
Dynamic loading of DLL functions is duplicated in several places.

Add a set of macros to simplify the process.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:44:01 +01:00
Karsten Blees
8d479366da Win32: lstat(): return adequate stat.st_size for symlinks
Git typically doesn't trust the stat.st_size member of symlinks (e.g. see
strbuf_readlink()). However, some functions take shortcuts if st_size is 0
(e.g. diff_populate_filespec()).

In mingw_lstat() and fscache_lstat(), make sure to return an adequate size.

The extra overhead of opening and reading the reparse point to calculate
the exact size is not necessary, as git doesn't rely on the value anyway.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:43:58 +01:00
Karsten Blees
83005ac04b Win32: teach fscache and dirent about symlinks
Move S_IFLNK detection to file_attr_to_st_mode() and reuse it in fscache.

Implement DT_LNK detection in dirent.c and the fscache readdir version.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:43:58 +01:00
Karsten Blees
2599fdddfc Win32: let mingw_lstat() error early upon problems with reparse points
When obtaining lstat information for reparse points, we need to call
FindFirstFile() in addition to GetFileInformationEx() to obtain the type
of the reparse point (symlink, mount point etc.). However, currently there
is no error handling whatsoever if FindFirstFile() fails.

Call FindFirstFile() before modifying the stat *buf output parameter and
error out if the call fails.

Note: The FindFirstFile() return value includes all the data that we get
from GetFileAttributesEx(), so we could replace GetFileAttributesEx() with
FindFirstFile(). We don't do that because GetFileAttributesEx() is about
twice as fast for single files. I.e. we only pay the extra cost of calling
FindFirstFile() in the rare case that we encounter a reparse point.

Note: The indentation of the remaining reparse point code will be fixed in
the next patch.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:43:55 +01:00
Karsten Blees
b8000f8439 Win32: remove separate do_lstat() function
With the new mingw_stat() implementation, do_lstat() is only called from
mingw_lstat() (with follow == 0). Remove the extra function and the old
mingw_stat()-specific (follow == 1) logic.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:43:53 +01:00
Karsten Blees
beb451648e Win32: implement stat() with symlink support
With respect to symlinks, the current stat() implementation is almost the
same as lstat(): except for the file type (st_mode & S_IFMT), it returns
information about the link rather than the target.

Implement stat by opening the file with as little permissions as possible
and calling GetFileInformationByHandle on it. This way, all link resoltion
is handled by the Windows file system layer.

If symlinks are disabled, use lstat() as before, but fail with ELOOP if a
symlink would have to be resolved.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:43:51 +01:00
Karsten Blees
686c0e70a8 Win32: don't call GetFileAttributes twice in mingw_lstat()
GetFileAttributes cannot handle paths with trailing dir separator. The
current [l]stat implementation calls GetFileAttributes twice if the path
has trailing slashes (first with the original path passed to [l]stat, and
and a second time with a path copy with trailing '/' removed).

With Unicode conversion, we get the length of the path for free and also
have a (wide char) buffer that can be modified.

Remove trailing directory separators before calling the Win32 API.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:43:48 +01:00
Anton Serbulov
87360db061 mingw: fix getcwd when the parent directory cannot be queried
`GetLongPathName()` function may fail when it is unable to query
the parent directory of a path component to determine the long name
for that component. It happens, because of it uses `FindFirstFile()`
function for each next short part of path. The `FindFirstFile()`
requires `List Directory` and `Synchronize` desired access for a calling
process.

In case of lacking such permission for some part of path,
the `GetLongPathName()` returns 0 as result and `GetLastError()`
returns ERROR_ACCESS_DENIED.

`GetFinalPathNameByHandle()` function can help in such cases, because
it requires `Read Attributes` and `Synchronize` desired access to the
target path only.

The `GetFinalPathNameByHandle()` function was introduced on
`Windows Server 2008/Windows Vista`. So we need to load it dynamically.

`CreateFile()` parameters:
    `lpFileName` = path to the current directory
    `dwDesiredAccess` = 0 (it means `Read Attributes` and `Synchronize`)
    `dwShareMode` = FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE
                    (it prevents `Sharing Violation`)
    `lpSecurityAttributes` = NULL (default security attributes)
    `dwCreationDisposition` = OPEN_EXISTING
                              (required to obtain a directory handle)
    `dwFlagsAndAttributes` = FILE_FLAG_BACKUP_SEMANTICS
                             (required to obtain a directory handle)
    `hTemplateFile` = NULL (when opening an existing file or directory,
                            `CreateFile` ignores this parameter)

The string that is returned by `GetFinalPathNameByHandle()` function
uses the \\?\ syntax. To skip the prefix and convert backslashes
to slashes, the `normalize_ntpath()` mingw function will be used.

Note: `GetFinalPathNameByHandle()` function returns a final path.
It is the path that is returned when a path is fully resolved.
For example, for a symbolic link named "C:\tmp\mydir" that points to
"D:\yourdir", the final path would be "D:\yourdir".

Signed-off-by: Anton Serbulov <aserbulov@plesk.com>
2016-03-14 09:34:07 +01:00
Johannes Schindelin
9f4192ab75 mingw: ensure getcwd() reports the correct case
When switching the current working directory, say, in PowerShell, it is
quite possible to use a different capitalization than the one that is
recorded on disk. While doing the same in `cmd.exe` adjusts the
capitalization magically, that does not happen in PowerShell so that
`getcwd()` returns the current directory in a different way than is
recorded on disk.

Typically this creates no problems except when you call

	git log .

in a subdirectory called, say, "GIT/" but you switched to "Git/" and
your `getcwd()` reports the latter, then Git won't understand that you
wanted to see the history as per the `GIT/` subdirectory but it thinks you
wanted to see the history of some directory that may have existed in the
past (but actually never did).

So let's be extra careful to adjust the capitalization of the current
directory before working with it.

Reported by a few PowerShell power users ;-)

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:34:06 +01:00
Karsten Blees
cb309c4f2a compat/terminal.c: only use the Windows console if bash 'read -r' fails
Accessing the Windows console through the special CONIN$ / CONOUT$ devices
doesn't work properly for non-ASCII usernames an passwords.

It also doesn't work for terminal emulators that hide the native console
window (such as mintty), and 'TERM=xterm*' is not necessarily a reliable
indicator for such terminals.

The new shell_prompt() function, on the other hand, works fine for both
MSys1 and MSys2, in native console windows as well as mintty, and properly
supports Unicode. It just needs bash on the path (for 'read -s', which is
bash-specific).

On Windows, try to use the shell to read from the terminal. If that fails
with ENOENT (i.e. bash was not found), use CONIN/OUT as fallback.

Note: To test this, create a UTF-8 credential file with non-ASCII chars,
e.g. in git-bash: 'echo url=http://täst.com > cred.txt'. Then in git-cmd,
'git credential fill <cred.txt' works (shell version), while calling git
without the git-wrapper (i.e. 'mingw64\bin\git credential fill <cred.txt')
mangles non-ASCII chars in both console output and input.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:34:03 +01:00
Karsten Blees
0f3dba9c74 mingw: Support git_terminal_prompt with more terminals
The `git_terminal_prompt()` function expects the terminal window to be
attached to a Win32 Console. However, this is not the case with terminal
windows other than `cmd.exe`'s, e.g. with MSys2's own `mintty`.

Non-cmd terminals such as `mintty` still have to have a Win32 Console
to be proper console programs, but have to hide the Win32 Console to
be able to provide more flexibility (such as being resizeable not only
vertically but also horizontally). By writing to that Win32 Console,
`git_terminal_prompt()` manages only to send the prompt to nowhere and
to wait for input from a Console to which the user has no access.

This commit introduces a function specifically to support `mintty` -- or
other terminals that are compatible with MSys2's `/dev/tty` emulation. We
use the `TERM` environment variable as an indicator for that: if the value
starts with "xterm" (such as `mintty`'s "xterm_256color"), we prefer to
let `xterm_prompt()` handle the user interaction.

The most prominent user of `git_terminal_prompt()` is certainly
`git-remote-https.exe`. It is an interesting use case because both
`stdin` and `stdout` are redirected when Git calls said executable, yet
it still wants to access the terminal.

When running inside a `mintty`, the terminal is not accessible to the
`git-remote-https.exe` program, though, because it is a MinGW program
and the `mintty` terminal is not backed by a Win32 console.

To solve that problem, we simply call out to the shell -- which is an
*MSys2* program and can therefore access `/dev/tty`.

Helped-by: nalla <nalla@hamal.uberspace.de>
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:34:02 +01:00
Karsten Blees
e2570e3448 mingw: initialize HOME on startup
HOME initialization was historically duplicated in many different places,
including /etc/profile, launch scripts such as git-bash.vbs and gitk.cmd,
and (although slightly broken) in the git-wrapper.

Even unrelated projects such as GitExtensions and TortoiseGit need to
implement the same logic to be able to call git directly.

Initialize HOME in git's own startup code so that we can eventually retire
all the duplicate initialization code.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:33:59 +01:00
Johannes Schindelin
fbe95d202c mingw: HOT FIX: work around environment issues -- again
This developer should really, really have known better. The fact that we
are changing the environment in ways for which the MSVCRT is not
prepared for is bad enough. But then this developer followed the request
to re-enable nedmalloc -- despite the prediction that it would cause an
access violation, predicting it in the same message as the request to
re-enable nedmalloc, no less!

To paper over the issue until the time when this developer finds the
time to re-design the Unicode environment handling from scratch, let's
hope that cURL is the only library we are using that *may* set an
environment variable using MSVCRT's putenv() after we fscked the
environment up.

Note: this commit can serve as no source of pride to anyone, certainly
not yours truly. It is necessary as a quick and pragmatic stop gap,
though, to prevent worse problems.

Note: cURL manages to set the variable CHARSET when nedmalloc is *not*
enabled, without causing an access violation. In that case, it sets it
successfully to the value "cp" + GetACP() (hence it is our choice, too,
cURL may need it, Git does not):

	https://github.com/bagder/curl/blob/aa5808b5/lib/easy.c#L157-L162

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:33:57 +01:00
Johannes Schindelin
78293d7603 mingw: be *very* wary about outside environment changes
The environment is modified in most surprising circumstances, and not
all of them are under Git's control. For example, calling
curl_global_init() on Windows will ensure that the CHARSET variable is
set, adding one if necessary.

While the previous commit worked around crashes triggered by such
outside changes of the environment by relaxing the requirement that the
environment be terminated by a NULL pointer, the other assumption made
by `mingw_getenv()` and `mingw_putenv()` is that the environment is
sorted, for efficient lookup via binary search.

Let's make real sure that our environment is intact before querying or
modifying it, and reinitialize our idea of the environment if necessary.

With this commit, before working on the environment we look briefly for
indicators that the environment was modified outside of our control, and
to ensure that it is terminated with a NULL pointer and sorted again in
that case.

Note: the indicators are maybe not sufficient. For example, when a
variable is removed, it will not be noticed. It might also be a problem
if outside changes to the environment result in a modified `environ`
pointer: it is unclear whether such a modification could result in a
problem when `mingw_putenv()` needs to `realloc()` the environment
buffer.

For the moment, however, the current fix works well enough, so let's
only face the potential problems when (and if!) they occur.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:33:51 +01:00
Johannes Schindelin
783e3eb464 mingw: be more defensive when making the environment block
Outside of our Windows-specific code, the end of the environment can be
marked also by a pointer to a NUL character, not only by a NULL pointer
as our code assumed so far.

That led to a buffer overrun in `make_environment_block()` when running
`git-remote-https` in `mintty` (because `curl_global_init()` added the
`CHARSET` environment variable *outside* of `mingw_putenv()`, ending the
environment in a pointer to an empty string).

Side note for future debugging on Windows: when running programs in
`mintty`, the standard input/output/error is not connected to a Win32
Console, but instead is pipe()d. That means that even stderr may not be
written completely before a crash, but has to be fflush()ed explicitly.
For example, when debugging crashes, the developer should insert an
`fflush(stderr);` at the end of the `error()` function defined in
usage.c.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:33:49 +01:00
Johannes Schindelin
c23f8e5513 UTF-8 environment: be a little bit more defensive
It is unlikely that we have an empty environment, ever, but *if* we do,
when `environ_size - 1` is passed to `bsearchenv()` it is misinterpreted
as a real large integer.

To make the code truly defensive, refuse to do anything at all if the
size is negative (which should not happen, of course).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:33:48 +01:00
Alan Davies
ea9b12a2e1 winansi.c: Fix colourization on Cygwin pseudo terminals.
Git only colours the output and uses pagination if isatty() returns 1.
MSys and Cygwin emulate pseudo terminals via named pipes, meaning that
isatty() returns 0.

Commit 3adef8de55 fixed this for MSys
terminals, but not Cygwin.

The named pipes that Cygwin and Msys use are very similar.  MSys PTY pipes
are called 'msys-*-pty*' and Cygwin uses 'cygwin-*-pty*'.  This commit
modifies the existing check to allow both MSys and Cygwin PTY pipes to be
identified as TTYs.

Note that Pagination is still broken on Cygwin.  less.exe is spawned (as
seen in Process Explorer and using GIT_TRACE=1), but the output is not
being piped into it.

This partially fixes https://github.com/git-for-windows/git/issues/267

Signed-off-by: Alan Davies <alan.n.davies@gmail.com>
2016-03-14 09:33:46 +01:00
Karsten Blees
fb567dccee mingw: make isatty() recognize MSYS2's pseudo terminals (/dev/pty*)
MSYS2 emulates pseudo terminals via named pipes, and isatty() returns 0
for such file descriptors. Therefore, some interactive functionality
(such as launching a pager, asking if a failed unlink should be repeated
etc.) doesn't work when run in a terminal emulator that uses MSYS2's
ptys (such as mintty).

However, MSYS2 uses special names for its pty pipes ('msys-*-pty*'),
which allows us to distinguish them from normal piped input / output.

On startup, check if stdin / stdout / stderr are connected to such pipes
using the NtQueryObject API from NTDll.dll. If the names match, adjust
the flags in MSVCRT's ioinfo structure accordingly.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:33:45 +01:00
Cesar Eduardo Barros
54b5d9f2ca mingw: Embed a manifest to trick UAC into Doing The Right Thing
On Windows >= Vista, not having an application manifest with a
requestedExecutionLevel can cause several kinds of confusing behavior.

The first and more obvious behavior is "Installer Detection", where
Windows sometimes decides (by looking at things like the file name and
even sequences of bytes within the executable) that an executable is an
installer and should run elevated (causing the well-known popup dialog
to appear). In Git's context, subcommands such as "git patch-id" or "git
update-index" fall prey to this behavior.

The second and more confusing behavior is "File Virtualization". It
means that when files are written without having write permission, it
does not fail (as expected), but they are instead redirected to
somewhere else. When the files are read, the original contents are
returned, though, not the ones that were just written somewhere else.
Even more confusing, not all write accesses are redirected; Trying to
write to write-protected .exe files, for example, will fail instead of
redirecting.

In addition to being unwanted behavior, File Virtualization causes
dramatic slowdowns in Git (see for instance
http://code.google.com/p/msysgit/issues/detail?id=320).

There are two ways to prevent those two behaviors: Either you embed an
application manifest within all your executables, or you add an external
manifest (a file with the same name followed by .manifest) to all your
executables. Since Git's builtins are hardlinked (or copied), it is
simpler and more robust to embed a manifest.

A recent enough MSVC compiler should already embed a working internal
manifest, but for MinGW you have to do so by hand.

Very lightly tested on Wine, where like on Windows XP it should not make
any difference.

References:
  - New UAC Technologies for Windows Vista
    http://msdn.microsoft.com/en-us/library/bb756960.aspx
  - Create and Embed an Application Manifest (UAC)
    http://msdn.microsoft.com/en-us/library/bb756929.aspx

[js: simplified the embedding dramatically by reusing Git for Windows'
existing Windows resource file, removed the optional (and dubious)
processorArchitecture attribute of the manifest's assemblyIdentity
section.]

Signed-off-by: Cesar Eduardo Barros <cesarb@cesarb.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:33:43 +01:00
Johannes Schindelin
0ccf7cedec Error out when mingw_startup() *and* NO_UNSETENV are active
The unsetenv code has no idea to update our environ_size, therefore
causing segmentation faults when environment variables are removed
without compat/mingw.c's knowing (MinGW's optimized lookup would try
to strcmp() against NULL in such a case).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:33:39 +01:00
Johannes Schindelin
ce47e4dfbd Help debugging with MSys2 by optionally executing bash with strace
MSys2's strace facility is very useful for debugging... With this patch,
the bash will be executed through strace if the environment variable
GIT_STRACE_COMMANDS is set, which comes in real handy when investigating
issues in the test suite.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:33:35 +01:00
Karsten Blees
6e7f94da16 Win32: fix 'lstat("dir/")' with long paths
Use a suffciently large buffer to strip the trailing slash.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:30:09 +01:00
Johannes Schindelin
8c6794db08 Win32: support long paths
Windows paths are typically limited to MAX_PATH = 260 characters, even
though the underlying NTFS file system supports paths up to 32,767 chars.
This limitation is also evident in Windows Explorer, cmd.exe and many
other applications (including IDEs).

Particularly annoying is that most Windows APIs return bogus error codes
if a relative path only barely exceeds MAX_PATH in conjunction with the
current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the
infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG.

Many Windows wide char APIs support longer than MAX_PATH paths through the
file namespace prefix ('\\?\' or '\\?\UNC\') followed by an absolute path.
Notable exceptions include functions dealing with executables and the
current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as
well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...).

Introduce a handle_long_path function to check the length of a specified
path properly (and fail with ENAMETOOLONG), and to optionally expand long
paths using the '\\?\' file namespace prefix. Short paths will not be
modified, so we don't need to worry about device names (NUL, CON, AUX).

Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be
limited to MAX_PATH (at least not on Win7), so we can use it to do the
heavy lifting of the conversion (translate '/' to '\', eliminate '.' and
'..', and make an absolute path).

Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH
limit.

Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs
that support long paths.

While improved error checking is always active, long paths support must be
explicitly enabled via 'core.longpaths' option. This is to prevent end
users to shoot themselves in the foot by checking out files that Windows
Explorer, cmd/bash or their favorite IDE cannot handle.

Test suite:
Test the case is when the full pathname length of a dir is close
to 260 (MAX_PATH).
Bug report and an original reproducer by Andrey Rogozhnikov:
https://github.com/msysgit/git/pull/122#issuecomment-43604199

[jes: adjusted test number to avoid conflicts]

Thanks-to: Martin W. Kirst <maki@bitkings.de>
Thanks-to: Doug Kelly <dougk.ff7@gmail.com>
Signed-off-by: Karsten Blees <blees@dcon.de>
Original-test-by: Andrey Rogozhnikov <rogozhnikov.andrey@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:30:06 +01:00
Karsten Blees
a8183a019a fscache: load directories only once
If multiple threads access a directory that is not yet in the cache, the
directory will be loaded by each thread. Only one of the results is added
to the cache, all others are leaked. This wastes performance and memory.

On cache miss, add a future object to the cache to indicate that the
directory is currently being loaded. Subsequent threads register themselves
with the future object and wait. When the first thread has loaded the
directory, it replaces the future object with the result and notifies
waiting threads.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:30:04 +01:00
Karsten Blees
f8e108859f Win32: add a cache below mingw's lstat and dirent implementations
Checking the work tree status is quite slow on Windows, due to slow lstat
emulation (git calls lstat once for each file in the index). Windows
operating system APIs seem to be much better at scanning the status
of entire directories than checking single files.

Add an lstat implementation that uses a cache for lstat data. Cache misses
read the entire parent directory and add it to the cache. Subsequent lstat
calls for the same directory are served directly from the cache.

Also implement opendir / readdir / closedir so that they create and use
directory listings in the cache.

The cache doesn't track file system changes and doesn't plug into any
modifying file APIs, so it has to be explicitly enabled for git functions
that don't modify the working copy.

Note: in an earlier version of this patch, the cache was always active and
tracked file system changes via ReadDirectoryChangesW. However, this was
much more complex and had negative impact on the performance of modifying
git commands such as 'git checkout'.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:30:03 +01:00
Karsten Blees
bcd459dbb6 Win32: make the lstat implementation pluggable
Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite
slow. Windows operating system APIs seem to be much better at scanning the
status of entire directories than checking single files. A caching
implementation may improve performance by bulk-reading entire directories
or reusing data obtained via opendir / readdir.

Make the lstat implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:30:01 +01:00
Karsten Blees
eb83a6dab4 Win32: Make the dirent implementation pluggable
Emulating the POSIX dirent API on Windows via FindFirstFile/FindNextFile is
pretty staightforward, however, most of the information provided in the
WIN32_FIND_DATA structure is thrown away in the process. A more
sophisticated implementation may cache this data, e.g. for later reuse in
calls to lstat.

Make the dirent implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Define a base DIR structure with pointers to readdir/closedir that match
the opendir implementation (i.e. similar to vtable pointers in OOP).
Define readdir/closedir so that they call the function pointers in the DIR
structure. This allows to choose the opendir implementation on a
call-by-call basis.

Move the fixed sized dirent.d_name buffer to the dirent-specific DIR
structure, as d_name may be implementation specific (e.g. a caching
implementation may just set d_name to point into the cache instead of
copying the entire file name string).

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:30:00 +01:00
Karsten Blees
c89ab3ceef Win32: dirent.c: Move opendir down
Move opendir down in preparation for the next patch.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:29:58 +01:00
Karsten Blees
2f21c7b658 Win32: make FILETIME conversion functions public
Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-14 09:29:57 +01:00
Johannes Schindelin
525913d7ab When initializing .git/, record the current setting of core.hideDotFiles
This is on Windows only, of course.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:29:39 +01:00
Erik Faye-Lund
c3f6b19602 core.hideDotFiles: hide '.git' dir by default
At least for cross-platform projects, it makes sense to hide the
files starting with a dot, as this is the behavior on Unix/MacOSX.

However, at least Eclipse has problems interpreting the hidden flag
correctly, so the default is to hide only the .git/ directory.

The config setting core.hideDotFiles therefore supports not only
'true' and 'false', but also 'dotGitOnly'.

[jes: clarified the commit message, made git init respect the setting
by marking the .git/ directory only after reading the config, and added
documentation, and rebased on top of current junio/next]

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:29:38 +01:00
Johannes Schindelin
de8534754a mingw: let lstat() fail with errno == ENOTDIR when appropriate
POSIX semantics requires lstat() to fail with ENOTDIR when "[a]
component of the path prefix names an existing file that is neither a
directory nor a symbolic link to a directory".

See http://pubs.opengroup.org/onlinepubs/9699919799/functions/lstat.html

This behavior is expected by t1404-update-ref-df-conflicts now.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:29:16 +01:00
마누엘
3b677c3257 mingw: try to delete target directory before renaming
When the rename() function tries to move a directory it fails if the
target directory exists. It should check if it can delete the (possibly
empty) target directory and then try again to move the directory.

This partially fixes t9100-git-svn-basic.sh.

Signed-off-by: 마누엘 <nalla@hamal.uberspace.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:29:15 +01:00
Johannes Schindelin
adcc6f3e26 mingw: prepare the TMPDIR environment variable for shell scripts
When shell scripts access a $TMPDIR variable containing backslashes,
they will be mistaken for escape characters. Let's not let that happen
by converting them to forward slashes.

This partially fixes t7800 with MSYS2.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:29:14 +01:00
Karsten Blees
a882e426db mingw: factor out Windows specific environment setup
We will add more environment-related code to that new function
in the next patch.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-14 09:29:13 +01:00
Johannes Schindelin
e6dbd0b25a mingw: uglify (a, 0) definitions to shut up warnings
When the result of a (a, 0) expression is not used, MSys2's GCC version
finds it necessary to complain with a warning:

	right-hand operand of comma expression has no effect

Let's just pretend to use the 0 value and have a peaceful and quiet life
again.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-14 09:29:06 +01:00
Johannes Schindelin
9c48b59fe6 mingw: squash another warning about a cast
MSys2's compiler is correct that casting a "void *" to a "DWORD" loses
precision, but in the case of pthread_exit() we know that the value
fits into a DWORD.

Just like casting handles to DWORDs, let's work around this issue by
casting to "intrptr_t" first, and immediately cast to the final type.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-14 09:29:05 +01:00
Johannes Schindelin
2a47e5acb2 mingw: avoid warnings when casting HANDLEs to int
HANDLE is defined internally as a void *, but in many cases it is
actually guaranteed to be a 32-bit integer. In these cases, GCC should
not warn about a cast of a pointer to an integer of a different type
because we know exactly what we are doing.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-14 09:29:04 +01:00
Johannes Schindelin
b758277e34 mingw: avoid redefining S_* constants
When compiling with MSys2's compiler, these constants are already defined.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-14 09:29:04 +01:00
Johannes Schindelin
1cc3e80918 compat/winansi: support compiling with MSys2
MSys2 already defines the _CONSOLE_FONT_INFOEX structure.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-14 09:29:03 +01:00
Johannes Schindelin
6d510f63eb compat/mingw: support MSys2-based MinGW build
The excellent MSys2 project brings a substantially updated MinGW
environment including newer GCC versions and new headers. To support
compiling Git, let's special-case the new MinGW (tell-tale: the
_MINGW64_VERSION_MAJOR constant is defined).

Note: this commit only addresses compile failures, not compile warnings
(that task is left for a future patch).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-14 09:29:02 +01:00
Johannes Schindelin
4425e1eb56 nedmalloc: allow compiling with MSys2's compiler
With MSys2's GCC, `ReadWriteBarrier` is already defined, and FORCEINLINE
unfortunately gets defined incorrectly.

Let's work around both problems, using the MSys2-specific
__MINGW64_VERSION_MAJOR constant to guard the FORCEINLINE definition so
as not to affect other platforms.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-14 09:29:00 +01:00
Junio C Hamano
b7a6ec609f Merge branch 'jk/tighten-alloc' into maint
* jk/tighten-alloc: (23 commits)
  compat/mingw: brown paper bag fix for 50a6c8e
  ewah: convert to REALLOC_ARRAY, etc
  convert ewah/bitmap code to use xmalloc
  diff_populate_gitlink: use a strbuf
  transport_anonymize_url: use xstrfmt
  git-compat-util: drop mempcpy compat code
  sequencer: simplify memory allocation of get_message
  test-path-utils: fix normalize_path_copy output buffer size
  fetch-pack: simplify add_sought_entry
  fast-import: simplify allocation in start_packfile
  write_untracked_extension: use FLEX_ALLOC helper
  prepare_{git,shell}_cmd: use argv_array
  use st_add and st_mult for allocation size computation
  convert trivial cases to FLEX_ARRAY macros
  use xmallocz to avoid size arithmetic
  convert trivial cases to ALLOC_ARRAY
  convert manual allocations to argv_array
  argv-array: add detach function
  add helpers for allocating flex-array structs
  harden REALLOC_ARRAY and xcalloc against size_t overflow
  ...
2016-03-10 11:13:43 -08:00
Jeff King
8d5b3325e7 compat/mingw: brown paper bag fix for 50a6c8e
Commit 50a6c8e (use st_add and st_mult for allocation size
computation, 2016-02-22) fixed up many xmalloc call-sites
including ones in compat/mingw.c.

But I screwed up one of them, which was half-converted to
ALLOC_ARRAY, using a very early prototype of the function.
And I never caught it because I don't build on Windows.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-29 11:04:23 -08:00
Jeff King
50a6c8efa2 use st_add and st_mult for allocation size computation
If our size computation overflows size_t, we may allocate a
much smaller buffer than we expected and overflow it. It's
probably impossible to trigger an overflow in most of these
sites in practice, but it is easy enough convert their
additions and multiplications into overflow-checking
variants. This may be fixing real bugs, and it makes
auditing the code easier.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-22 14:51:09 -08:00
Jeff King
b32fa95fd8 convert trivial cases to ALLOC_ARRAY
Each of these cases can be converted to use ALLOC_ARRAY or
REALLOC_ARRAY, which has two advantages:

  1. It automatically checks the array-size multiplication
     for overflow.

  2. It always uses sizeof(*array) for the element-size,
     so that it can never go out of sync with the declared
     type of the array.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-22 14:51:09 -08:00
Patrick Steinhardt
3d1806487a config: rename git_config_set_or_die to git_config_set
Rename git_config_set_or_die functions to git_config_set, leading
to the new default behavior of dying whenever a configuration
error occurs.

By now all callers that shall die on error have been transitioned
to the _or_die variants, thus making this patch a simple rename
of the functions.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-22 10:23:55 -08:00