Commit Graph

70803 Commits

Author SHA1 Message Date
Johannes Schindelin
53c2be60e7 mingw: load system libraries the recommended way
When we access IPv6-related functions, we load the corresponding system
library using the `LoadLibrary()` function, which is not the recommended
way to load system libraries.

In practice, it does not make a difference: the `ws2_32.dll` library
containing the IPv6 functions is already loaded into memory, so
LoadLibrary() simply reuses the already-loaded library.

Still, recommended way is recommended way, so let's use that instead.

While at it, also adjust the code in contrib/ that loads system libraries.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:12 +02:00
Johannes Schindelin
72106c87c6 Windows: force-recompile git.res for differing architectures
When git.rc is compiled into git.res, the result is actually dependent
on the architecture. That is, you cannot simply link a 32-bit git.res
into a 64-bit git.exe.

Therefore, to allow 32-bit and 64-bit builds in the same directory, we
let git.res depend on GIT-PREFIX so that it gets recompiled when
compiling for a different architecture (this works because the exec path
changes based on the architecture: /mingw32/libexec/git-core for 32-bit
and /mingw64/libexec/git-core for 64-bit).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:12 +02:00
Johannes Schindelin
d22ba36522 mingw: Suppress warning that <commit>:.gitattributes does not exist
On Windows, a file name containing a colon is illegal. We should
therefore expect the corresponding errno when `fopen()` is called for a
path of the form <commit>:.gitattributes.

This fixes https://github.com/git-for-windows/git/issues/255.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:12 +02:00
Karsten Blees
7e2448dfbd compat/terminal.c: only use the Windows console if bash 'read -r' fails
Accessing the Windows console through the special CONIN$ / CONOUT$ devices
doesn't work properly for non-ASCII usernames an passwords.

It also doesn't work for terminal emulators that hide the native console
window (such as mintty), and 'TERM=xterm*' is not necessarily a reliable
indicator for such terminals.

The new shell_prompt() function, on the other hand, works fine for both
MSys1 and MSys2, in native console windows as well as mintty, and properly
supports Unicode. It just needs bash on the path (for 'read -s', which is
bash-specific).

On Windows, try to use the shell to read from the terminal. If that fails
with ENOENT (i.e. bash was not found), use CONIN/OUT as fallback.

Note: To test this, create a UTF-8 credential file with non-ASCII chars,
e.g. in git-bash: 'echo url=http://täst.com > cred.txt'. Then in git-cmd,
'git credential fill <cred.txt' works (shell version), while calling git
without the git-wrapper (i.e. 'mingw64\bin\git credential fill <cred.txt')
mangles non-ASCII chars in both console output and input.

Signed-off-by: Karsten Blees <blees@dcon.de>
2017-04-26 16:57:12 +02:00
Karsten Blees
c4d00e6530 mingw: Support git_terminal_prompt with more terminals
The `git_terminal_prompt()` function expects the terminal window to be
attached to a Win32 Console. However, this is not the case with terminal
windows other than `cmd.exe`'s, e.g. with MSys2's own `mintty`.

Non-cmd terminals such as `mintty` still have to have a Win32 Console
to be proper console programs, but have to hide the Win32 Console to
be able to provide more flexibility (such as being resizeable not only
vertically but also horizontally). By writing to that Win32 Console,
`git_terminal_prompt()` manages only to send the prompt to nowhere and
to wait for input from a Console to which the user has no access.

This commit introduces a function specifically to support `mintty` -- or
other terminals that are compatible with MSys2's `/dev/tty` emulation. We
use the `TERM` environment variable as an indicator for that: if the value
starts with "xterm" (such as `mintty`'s "xterm_256color"), we prefer to
let `xterm_prompt()` handle the user interaction.

The most prominent user of `git_terminal_prompt()` is certainly
`git-remote-https.exe`. It is an interesting use case because both
`stdin` and `stdout` are redirected when Git calls said executable, yet
it still wants to access the terminal.

When running inside a `mintty`, the terminal is not accessible to the
`git-remote-https.exe` program, though, because it is a MinGW program
and the `mintty` terminal is not backed by a Win32 console.

To solve that problem, we simply call out to the shell -- which is an
*MSys2* program and can therefore access `/dev/tty`.

Helped-by: nalla <nalla@hamal.uberspace.de>
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:12 +02:00
nalla
832fa892d6 mingw: explicitly fflush stdout
For performance reasons `stdout` is not unbuffered by default. That leads
to problems if after printing to `stdout` a read on `stdin` is performed.

For that reason interactive commands like `git clean -i` do not function
properly anymore if the `stdout` is not flushed by `fflush(stdout)` before
trying to read from `stdin`.

In the case of `git clean -i` all reads on `stdin` were preceded by a
`fflush(stdout)` call.

Signed-off-by: nalla <nalla@hamal.uberspace.de>
2017-04-26 16:57:12 +02:00
Karsten Blees
66e5973328 mingw: initialize HOME on startup
HOME initialization was historically duplicated in many different places,
including /etc/profile, launch scripts such as git-bash.vbs and gitk.cmd,
and (although slightly broken) in the git-wrapper.

Even unrelated projects such as GitExtensions and TortoiseGit need to
implement the same logic to be able to call git directly.

Initialize HOME in git's own startup code so that we can eventually retire
all the duplicate initialization code.

Signed-off-by: Karsten Blees <blees@dcon.de>
2017-04-26 16:57:12 +02:00
Johannes Schindelin
f877d46314 mingw: HOT FIX: work around environment issues -- again
This developer should really, really have known better. The fact that we
are changing the environment in ways for which the MSVCRT is not
prepared for is bad enough. But then this developer followed the request
to re-enable nedmalloc -- despite the prediction that it would cause an
access violation, predicting it in the same message as the request to
re-enable nedmalloc, no less!

To paper over the issue until the time when this developer finds the
time to re-design the Unicode environment handling from scratch, let's
hope that cURL is the only library we are using that *may* set an
environment variable using MSVCRT's putenv() after we fscked the
environment up.

Note: this commit can serve as no source of pride to anyone, certainly
not yours truly. It is necessary as a quick and pragmatic stop gap,
though, to prevent worse problems.

Note: cURL manages to set the variable CHARSET when nedmalloc is *not*
enabled, without causing an access violation. In that case, it sets it
successfully to the value "cp" + GetACP() (hence it is our choice, too,
cURL may need it, Git does not):

	https://github.com/bagder/curl/blob/aa5808b5/lib/easy.c#L157-L162

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:12 +02:00
Karsten Blees
e7dbb506d3 gettext: always use UTF-8 on native Windows
Git on native Windows exclusively uses UTF-8 for console output (both with
mintty and native console windows). Gettext uses setlocale() to determine
the output encoding for translated text, however, MSVCRT's setlocale()
doesn't support UTF-8. As a result, translated text is encoded in system
encoding (GetAPC()), and non-ASCII chars are mangled in console output.

Use gettext's bind_textdomain_codeset() to force the encoding to UTF-8 on
native Windows.

In this developers' setup, HAVE_LIBCHARSET_H is apparently defined, but
we *really* want to override the locale_charset() here.

Signed-off-by: Karsten Blees <blees@dcon.de>
2017-04-26 16:57:11 +02:00
Johannes Schindelin
6f89777b08 Avoid illegal filenames when building Documentation on NTFS
A '+' is not a valid part of a filename with Windows file systems (it is
reserved because the '+' operator meant file concatenation back in the
DOS days).

Let's just not use it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:11 +02:00
Johannes Schindelin
c06457ad99 git-gui (MinGW): make use of MSys2's msgfmt
When Git for Windows was still based on MSys1, we had no gettext, ergo
no msgfmt, either. Therefore, we introduced a small and simple Tcl
script to perform the same task.

However, with MSys2, we no longer need that because we have a proper
msgfmt executable. Plus, the po2msg.sh script somehow manages to hang
when run in parallel.

Two reasons to use real msgfmt.exe instead.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:11 +02:00
Johannes Schindelin
3de4dd3aef mingw: be *very* wary about outside environment changes
The environment is modified in most surprising circumstances, and not
all of them are under Git's control. For example, calling
curl_global_init() on Windows will ensure that the CHARSET variable is
set, adding one if necessary.

While the previous commit worked around crashes triggered by such
outside changes of the environment by relaxing the requirement that the
environment be terminated by a NULL pointer, the other assumption made
by `mingw_getenv()` and `mingw_putenv()` is that the environment is
sorted, for efficient lookup via binary search.

Let's make real sure that our environment is intact before querying or
modifying it, and reinitialize our idea of the environment if necessary.

With this commit, before working on the environment we look briefly for
indicators that the environment was modified outside of our control, and
to ensure that it is terminated with a NULL pointer and sorted again in
that case.

Note: the indicators are maybe not sufficient. For example, when a
variable is removed, it will not be noticed. It might also be a problem
if outside changes to the environment result in a modified `environ`
pointer: it is unclear whether such a modification could result in a
problem when `mingw_putenv()` needs to `realloc()` the environment
buffer.

For the moment, however, the current fix works well enough, so let's
only face the potential problems when (and if!) they occur.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:11 +02:00
Johannes Schindelin
8162bfdc08 mingw: be more defensive when making the environment block
Outside of our Windows-specific code, the end of the environment can be
marked also by a pointer to a NUL character, not only by a NULL pointer
as our code assumed so far.

That led to a buffer overrun in `make_environment_block()` when running
`git-remote-https` in `mintty` (because `curl_global_init()` added the
`CHARSET` environment variable *outside* of `mingw_putenv()`, ending the
environment in a pointer to an empty string).

Side note for future debugging on Windows: when running programs in
`mintty`, the standard input/output/error is not connected to a Win32
Console, but instead is pipe()d. That means that even stderr may not be
written completely before a crash, but has to be fflush()ed explicitly.
For example, when debugging crashes, the developer should insert an
`fflush(stderr);` at the end of the `error()` function defined in
usage.c.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:11 +02:00
Johannes Schindelin
9d4924ad2b UTF-8 environment: be a little bit more defensive
It is unlikely that we have an empty environment, ever, but *if* we do,
when `environ_size - 1` is passed to `bsearchenv()` it is misinterpreted
as a real large integer.

To make the code truly defensive, refuse to do anything at all if the
size is negative (which should not happen, of course).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:11 +02:00
Johannes Schindelin
815b211660 mingw: enable stack smashing protector
As suggested privately to Brendan Forster by some unnamed person
(suggestion for the future: use the public mailing list, or even the
public GitHub issue tracker, that is a much better place to offer such
suggestions), we should make use of gcc's stack smashing protector that
helps detect stack buffer overruns early.

Rather than using -fstack-protector, we use -fstack-protector-strong
because it strikes a better balance between how much code is affected
and the performance impact.

In a local test (time git log --grep=is -p), best of 5 timings went from
23.009s to 22.997s (i.e. the performance impact was *well* lost in the
noise).

This fixes https://github.com/git-for-windows/git/issues/501

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:11 +02:00
Cesar Eduardo Barros
4a073da22c mingw: Embed a manifest to trick UAC into Doing The Right Thing
On Windows >= Vista, not having an application manifest with a
requestedExecutionLevel can cause several kinds of confusing behavior.

The first and more obvious behavior is "Installer Detection", where
Windows sometimes decides (by looking at things like the file name and
even sequences of bytes within the executable) that an executable is an
installer and should run elevated (causing the well-known popup dialog
to appear). In Git's context, subcommands such as "git patch-id" or "git
update-index" fall prey to this behavior.

The second and more confusing behavior is "File Virtualization". It
means that when files are written without having write permission, it
does not fail (as expected), but they are instead redirected to
somewhere else. When the files are read, the original contents are
returned, though, not the ones that were just written somewhere else.
Even more confusing, not all write accesses are redirected; Trying to
write to write-protected .exe files, for example, will fail instead of
redirecting.

In addition to being unwanted behavior, File Virtualization causes
dramatic slowdowns in Git (see for instance
http://code.google.com/p/msysgit/issues/detail?id=320).

There are two ways to prevent those two behaviors: Either you embed an
application manifest within all your executables, or you add an external
manifest (a file with the same name followed by .manifest) to all your
executables. Since Git's builtins are hardlinked (or copied), it is
simpler and more robust to embed a manifest.

A recent enough MSVC compiler should already embed a working internal
manifest, but for MinGW you have to do so by hand.

Very lightly tested on Wine, where like on Windows XP it should not make
any difference.

References:
  - New UAC Technologies for Windows Vista
    http://msdn.microsoft.com/en-us/library/bb756960.aspx
  - Create and Embed an Application Manifest (UAC)
    http://msdn.microsoft.com/en-us/library/bb756929.aspx

[js: simplified the embedding dramatically by reusing Git for Windows'
existing Windows resource file, removed the optional (and dubious)
processorArchitecture attribute of the manifest's assemblyIdentity
section.]

Signed-off-by: Cesar Eduardo Barros <cesarb@cesarb.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:11 +02:00
Johannes Schindelin
7cdaf67d8f Tests: optionally skip redirecting stdin/stdout/stderr
There is a really useful debugging technique developed by Sverre
Rabbelier that inserts "bash &&" somewhere in the test scripts, letting
the developer interact at given points with the current state.

Another debugging technique, used a lot by this here coder, is to run
certain executables via gdb by guarding a "gdb -args" call in
bin-wrappers/git.

Both techniques were disabled by 781f76b1(test-lib: redirect stdin of
tests).

Let's reinstate the ability to run an interactive shell by making the
redirection optional: setting the TEST_NO_REDIRECT environment variable
will skip the redirection.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:10 +02:00
Johannes Schindelin
ba4219a0f4 Error out when mingw_startup() *and* NO_UNSETENV are active
The unsetenv code has no idea to update our environ_size, therefore
causing segmentation faults when environment variables are removed
without compat/mingw.c's knowing (MinGW's optimized lookup would try
to strcmp() against NULL in such a case).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:10 +02:00
Johannes Schindelin
9c705da498 Build Python stuff with MSys2
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:10 +02:00
Johannes Schindelin
aa89e295d5 Help debugging with MSys2 by optionally executing bash with strace
MSys2's strace facility is very useful for debugging... With this patch,
the bash will be executed through strace if the environment variable
GIT_STRACE_COMMANDS is set, which comes in real handy when investigating
issues in the test suite.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:10 +02:00
Sebastian Schuberth
12608f7bf8 Makefile: Set htmldir to match the default HTML docs location under MSYS
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
2017-04-26 16:57:10 +02:00
Sebastian Schuberth
c29227aaa4 MinGW: Use MakeMaker to build the Perl libraries
This way the libraries get properly installed into the "site_perl"
directory and we just have to move them out of the "mingw" directory.

Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
2017-04-26 16:57:10 +02:00
Johannes Schindelin
035ee9dea6 http: also treat config options sslCert and sslKey as paths
This is a companion patch to bf9acba (http: treat config options
sslCAPath and sslCAInfo as paths, 2015-11-23).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:09 +02:00
Johannes Schindelin
6ab28abe11 mingw: handle absolute paths in expand_user_path()
On Windows, an absolute POSIX path needs to be turned into a Windows
one.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:09 +02:00
Johannes Schindelin
7dedc9ae61 Merge 'long-paths' into HEAD 2017-04-26 16:57:09 +02:00
Johannes Schindelin
b546ef84a5 Merge 'fscache' into HEAD 2017-04-26 16:57:09 +02:00
Johannes Schindelin
79ab5f48ce Merge branch 'perl5lib'
With this topic branch, the PERL5LIB variable is unset to avoid external
settings from interfering with Git's own Perl interpreter.

This branch also cleans up some of our Windows-only config setting code
(and this will need to be rearranged in the next merging rebase so that
the cleanup comes first, and fscache and longPaths support build on
top).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:08 +02:00
Karsten Blees
3738cb2e63 Win32: fix 'lstat("dir/")' with long paths
Use a suffciently large buffer to strip the trailing slash.

Signed-off-by: Karsten Blees <blees@dcon.de>
2017-04-26 16:57:07 +02:00
Karsten Blees
809fe37577 Win32: support long paths
Windows paths are typically limited to MAX_PATH = 260 characters, even
though the underlying NTFS file system supports paths up to 32,767 chars.
This limitation is also evident in Windows Explorer, cmd.exe and many
other applications (including IDEs).

Particularly annoying is that most Windows APIs return bogus error codes
if a relative path only barely exceeds MAX_PATH in conjunction with the
current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the
infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG.

Many Windows wide char APIs support longer than MAX_PATH paths through the
file namespace prefix ('\\?\' or '\\?\UNC\') followed by an absolute path.
Notable exceptions include functions dealing with executables and the
current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as
well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...).

Introduce a handle_long_path function to check the length of a specified
path properly (and fail with ENAMETOOLONG), and to optionally expand long
paths using the '\\?\' file namespace prefix. Short paths will not be
modified, so we don't need to worry about device names (NUL, CON, AUX).

Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be
limited to MAX_PATH (at least not on Win7), so we can use it to do the
heavy lifting of the conversion (translate '/' to '\', eliminate '.' and
'..', and make an absolute path).

Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH
limit.

Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs
that support long paths.

While improved error checking is always active, long paths support must be
explicitly enabled via 'core.longpaths' option. This is to prevent end
users to shoot themselves in the foot by checking out files that Windows
Explorer, cmd/bash or their favorite IDE cannot handle.

Test suite:
Test the case is when the full pathname length of a dir is close
to 260 (MAX_PATH).
Bug report and an original reproducer by Andrey Rogozhnikov:
https://github.com/msysgit/git/pull/122#issuecomment-43604199

Note that the test cannot rely on the presence of short names, as they
are not enabled by default except on the system drive.

[jes: adjusted test number to avoid conflicts, reinstated && chain,
adjusted test to work without short names]

Thanks-to: Martin W. Kirst <maki@bitkings.de>
Thanks-to: Doug Kelly <dougk.ff7@gmail.com>
Signed-off-by: Karsten Blees <blees@dcon.de>
Original-test-by: Andrey Rogozhnikov <rogozhnikov.andrey@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:07 +02:00
Johannes Schindelin
eb8ae43f28 Win32: support long paths
Windows paths are typically limited to MAX_PATH = 260 characters, even
though the underlying NTFS file system supports paths up to 32,767 chars.
This limitation is also evident in Windows Explorer, cmd.exe and many
other applications (including IDEs).

Particularly annoying is that most Windows APIs return bogus error codes
if a relative path only barely exceeds MAX_PATH in conjunction with the
current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the
infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG.

Many Windows wide char APIs support longer than MAX_PATH paths through the
file namespace prefix ('\\?\' or '\\?\UNC\') followed by an absolute path.
Notable exceptions include functions dealing with executables and the
current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as
well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...).

Introduce a handle_long_path function to check the length of a specified
path properly (and fail with ENAMETOOLONG), and to optionally expand long
paths using the '\\?\' file namespace prefix. Short paths will not be
modified, so we don't need to worry about device names (NUL, CON, AUX).

Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be
limited to MAX_PATH (at least not on Win7), so we can use it to do the
heavy lifting of the conversion (translate '/' to '\', eliminate '.' and
'..', and make an absolute path).

Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH
limit.

Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs
that support long paths.

While improved error checking is always active, long paths support must be
explicitly enabled via 'core.longpaths' option. This is to prevent end
users to shoot themselves in the foot by checking out files that Windows
Explorer, cmd/bash or their favorite IDE cannot handle.

Test suite:
Test the case is when the full pathname length of a dir is close
to 260 (MAX_PATH).
Bug report and an original reproducer by Andrey Rogozhnikov:
https://github.com/msysgit/git/pull/122#issuecomment-43604199

[jes: adjusted test number to avoid conflicts]

Thanks-to: Martin W. Kirst <maki@bitkings.de>
Thanks-to: Doug Kelly <dougk.ff7@gmail.com>
Signed-off-by: Karsten Blees <blees@dcon.de>
Original-test-by: Andrey Rogozhnikov <rogozhnikov.andrey@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:07 +02:00
Doug Kelly
c6febd1e91 Add a test demonstrating a problem with long submodule paths
[jes: adusted test number to avoid conflicts, fixed non-portable use of
the 'export' statement]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:07 +02:00
Karsten Blees
c70cff3d40 fscache: load directories only once
If multiple threads access a directory that is not yet in the cache, the
directory will be loaded by each thread. Only one of the results is added
to the cache, all others are leaked. This wastes performance and memory.

On cache miss, add a future object to the cache to indicate that the
directory is currently being loaded. Subsequent threads register themselves
with the future object and wait. When the first thread has loaded the
directory, it replaces the future object with the result and notifies
waiting threads.

Signed-off-by: Karsten Blees <blees@dcon.de>
2017-04-26 16:57:06 +02:00
Karsten Blees
6c1a8941cf Win32: add a cache below mingw's lstat and dirent implementations
Checking the work tree status is quite slow on Windows, due to slow lstat
emulation (git calls lstat once for each file in the index). Windows
operating system APIs seem to be much better at scanning the status
of entire directories than checking single files.

Add an lstat implementation that uses a cache for lstat data. Cache misses
read the entire parent directory and add it to the cache. Subsequent lstat
calls for the same directory are served directly from the cache.

Also implement opendir / readdir / closedir so that they create and use
directory listings in the cache.

The cache doesn't track file system changes and doesn't plug into any
modifying file APIs, so it has to be explicitly enabled for git functions
that don't modify the working copy.

Note: in an earlier version of this patch, the cache was always active and
tracked file system changes via ReadDirectoryChangesW. However, this was
much more complex and had negative impact on the performance of modifying
git commands such as 'git checkout'.

Signed-off-by: Karsten Blees <blees@dcon.de>
2017-04-26 16:57:06 +02:00
Karsten Blees
52656cc0c3 add infrastructure for read-only file system level caches
Add a macro to mark code sections that only read from the file system,
along with a config option and documentation.

This facilitates implementation of relatively simple file system level
caches without the need to synchronize with the file system.

Enable read-only sections for 'git status' and preload_index.

Signed-off-by: Karsten Blees <blees@dcon.de>
2017-04-26 16:57:06 +02:00
Karsten Blees
8bebf69dbc Win32: make the lstat implementation pluggable
Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite
slow. Windows operating system APIs seem to be much better at scanning the
status of entire directories than checking single files. A caching
implementation may improve performance by bulk-reading entire directories
or reusing data obtained via opendir / readdir.

Make the lstat implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Signed-off-by: Karsten Blees <blees@dcon.de>
2017-04-26 16:57:06 +02:00
Karsten Blees
a59e778150 Win32: Make the dirent implementation pluggable
Emulating the POSIX dirent API on Windows via FindFirstFile/FindNextFile is
pretty staightforward, however, most of the information provided in the
WIN32_FIND_DATA structure is thrown away in the process. A more
sophisticated implementation may cache this data, e.g. for later reuse in
calls to lstat.

Make the dirent implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Define a base DIR structure with pointers to readdir/closedir that match
the opendir implementation (i.e. similar to vtable pointers in OOP).
Define readdir/closedir so that they call the function pointers in the DIR
structure. This allows to choose the opendir implementation on a
call-by-call basis.

Move the fixed sized dirent.d_name buffer to the dirent-specific DIR
structure, as d_name may be implementation specific (e.g. a caching
implementation may just set d_name to point into the cache instead of
copying the entire file name string).

Signed-off-by: Karsten Blees <blees@dcon.de>
2017-04-26 16:57:05 +02:00
Karsten Blees
5afe4e23d7 Win32: dirent.c: Move opendir down
Move opendir down in preparation for the next patch.

Signed-off-by: Karsten Blees <blees@dcon.de>
2017-04-26 16:57:05 +02:00
Karsten Blees
429ea008a2 Win32: make FILETIME conversion functions public
Signed-off-by: Karsten Blees <blees@dcon.de>
2017-04-26 16:57:05 +02:00
Johannes Schindelin
96257e4f9d mingw: unset PERL5LIB by default
Git for Windows ships with its own Perl interpreter, and insists on
using it, so it will most likely wreak havoc if PERL5LIB is set before
launching Git.

Let's just unset that environment variables when spawning processes.

To make this feature extensible (and overrideable), there is a new
config setting `core.unsetenvvars` that allows specifying a
comma-separated list of names to unset before spawning processes.

Reported by Gabriel Fuhrmann.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:04 +02:00
Johannes Schindelin
181700c80b Move Windows-specific config settings into compat/mingw.c
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:04 +02:00
Johannes Schindelin
cce713d087 Allow for platform-specific core.* config settings
In the Git for Windows project, we have ample precendent for config
settings that apply to Windows, and to Windows only.

Let's formalize this concept by introducing a platform_core_config()
function that can be #define'd in a platform-specific manner.

This will allow us to contain platform-specific code better, as the
corresponding variables no longer need to be exported so that they can
be defined in environment.c and be set in config.c

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:04 +02:00
Johannes Schindelin
a699e62923 mingw: include the full version information in the resources
This fixes https://github.com/git-for-windows/git/issues/723

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:57:03 +02:00
Johannes Schindelin
b2185ebf1a Start the merging-rebase to upstream/maint
This commit starts the rebase of b213911f46 to 49800c9407

This merging rebase was started not because upstream's `maint` branch
advanced (it did not), but to replace a couple of patches with newer
iterations, as sent to the upstream Git project independently. In
particular, those are:

- The patch series merged by fe73baf344
  (jeffhostetler/jeffhostetler/quick_add_index_entry) was replaced by
  the most recent iteration of 'jh/add-index-entry-optim' (b986df5c35)

- The Pull Request jeffhostetler/jeffhostetler/string_list_realloc
  merged via c775bdd100 was replaced by the latest iteration of
  'jh/string-list-micro-optim' (950a234cbd)

- The jh/memihash-opt patches merged by 2862058e9d were replaced by the
  newest iteration (41b3eb4a6b)

- The bug fix where difftool used a buffer after freeing it
  (d33e487771) was replaced by the one that made it into upstream
  (882add136f)

- The patch in the `coverity` series that tried to fix a resource leak
  in git-am (a5208164e2) was replaced by a better patch submitted by
  Ren_ Scharfe (ac8ce18d89)

- The patch in the `coverity` series that tried to fix a resource leak
  in the `handle_ssh_variant()` function (f07be76f51) has been dropped,
  as a different patch had been accepted into `pu` already

- The rebase-i-extra patches (e1be548aaf) were replaced by the latest
  iteration (1d3d10b9e15)

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 16:55:54 +02:00
Johannes Schindelin
bf1cd086d6 Merge branch 'coverity'
Coverity is a tool to analyze code statically, trying to find common (or
not so common) problems before they occur in production.

Coverity offers its services to Open Source software, and just like
upstream Git, Git for Windows applied and was granted the use.

While Coverity reports a lot of false positives due to Git's (ab-)use of
the FLEX_ARRAY feature (where it declares a 0-byte or 1-byte array at the
end of a struct, and then allocates a variable-length data structure
holding a variable-length string at the end, so that the struct as well as
the string can be released with a single free()), there were a few issues
reported that are true positives, and not all of them were resource leaks
in builtins (for which it is considered kind of okay to not release memory
just before exit() is called anyway).

This topic branch tries to address a couple of those issues.

Note: there are a couple more issues left, either because they are tricky
to resolve (in some cases, the custody of occasionally-allocated memory is
very unclear) or because it is unclear whether they are false positives
(due to the hard-to-reason-about nature of the code). It's a start,
though.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 12:24:06 +02:00
Johannes Schindelin
37dabece6d Merge branch 'drive-prefix'
This topic branch allows us to specify absolute paths without the drive
prefix e.g. when cloning.

Example:

	C:\Users\me> git clone https://github.com/git/git \upstream-git

This will clone into a new directory C:\upstream-git, in line with how
Windows interprets absolute paths.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 12:24:05 +02:00
Johannes Schindelin
f07be76f51 handle_ssh_variant: plug memory leak
The split_cmdline() function either assigns an allocated array to the argv
parameter or NULL. In the first case, we have to free() it, in the latter
it does no harm.

Reported via Coverity.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 12:24:04 +02:00
Johannes Schindelin
a5208164e2 am: close input stream even in case of an error
Reported via Coverity.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 12:24:04 +02:00
Johannes Schindelin
6b88dd4e15 submodule_uses_worktrees(): plug memory leak
There is really no reason why we would need to hold onto the allocated
string longer than necessary.

Reported by Coverity.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 12:24:04 +02:00
Johannes Schindelin
219dce8722 show_worktree(): plug memory leak
The buffer allocated by shorten_unambiguous_ref() needs to be released.

Discovered by Coverity.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 12:24:03 +02:00
Johannes Schindelin
272448d64f name-rev: avoid leaking memory in the deref case
When the `name_rev()` function is asked to dereference the tip name, it
allocates memory. But when it turns out that another tip already
described the commit better than the current one, we forgot to release
the memory.

Pointed out by Coverity.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-04-26 12:24:03 +02:00