Commit Graph

710 Commits

Author SHA1 Message Date
Johannes Schindelin
e8384c3a9e Merge pull request #1214 from rongjiecomputer/master
Implement pthread_cond_t with Win32 CONDITION_VARIABLE
2018-03-23 13:58:46 +01:00
Loo Rong Jie
a59494f17b Remove old code and macro-ize implementation
Signed-off-by: Loo Rong Jie <loorongjie@gmail.com>
2018-03-23 13:58:38 +01:00
Loo Rong Jie
c0398c3eaa Format to 80 cols
Signed-off-by: Loo Rong Jie <loorongjie@gmail.com>
2018-03-23 13:58:38 +01:00
Loo Rong Jie
e9ea6bbfdc Implement pthread_cond_t with Win32 CONDITION_VARIABLE
Win32 CONDITION_VARIABLE has better performance and is easier to
maintain.

Since CONDITION_VARIABLE is not available in Windows XP and below,
old implementation of pthread_cond_t is kept under define guard
'GIT_WIN_XP_SUPPORT'. To enable old implementation, build with
make CFLAGS="-DGIT_WIN_XP_SUPPORT".

Signed-off-by: Loo Rong Jie <loorongjie@gmail.com>

fast-forwarded.
2018-03-23 13:58:37 +01:00
Johannes Schindelin
e2bf92e4b1 mingw: when path_lookup() failed, try BusyBox
BusyBox comes with a ton of applets ("applet" being the identical
concept to Git's "builtins"). And similar to Git's builtins, the applets
can be called via `busybox <command>`, or the BusyBox executable can be
copied/hard-linked to the command name.

The similarities do not end here. Just as with Git's builtins, it is
problematic that BusyBox' hard-linked applets cannot easily be put into
a .zip file: .zip archives have no concept of hard-links and therefore
would store identical copies (and also extract identical copies,
"inflating" the archive unnecessarily).

To counteract that issue, MinGit already ships without hard-linked
copies of the builtins, and the plan is to do the same with BusyBox'
applets: simply ship busybox.exe as single executable, without
hard-linked applets.

To accommodate that, Git is being taught by this commit a very special
trick, exploiting the fact that it is possible to call an executable
with a command-line whose argv[0] is different from the executable's
name: when `sh` is to be spawned, and no `sh` is found in the PATH, but
busybox.exe is, use that executable (with unchanged argv).

Likewise, if any executable to be spawned is not on the PATH, but
busybox.exe is found, parse the output of `busybox.exe --help` to find
out what applets are included, and if the command matches an included
applet name, use busybox.exe to execute it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:57:23 +01:00
Johannes Schindelin
fda76cd9c6 mingw: explicitly specify with which cmd to prefix the cmdline
The main idea of this patch is that even if we have to look up the
absolute path of the script, if only the basename was specified as
argv[0], then we should use that basename on the command line, too, not
the absolute path.

This patch will also help with the upcoming patch where we automatically
substitute "sh ..." by "busybox sh ..." if "sh" is not in the PATH but
"busybox" is: we will do that by substituting the actual executable, but
still keep prepending "sh" to the command line.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:57:23 +01:00
Johannes Schindelin
b9e85f45de t5580: test cloning without file://, test fetching via UNC paths
It gets a bit silly to add the commands to the name of the test script,
so let's just rename it while we're testing more UNC stuff.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:57:07 +01:00
Johannes Schindelin
6371dbffd6 mingw: special-case arguments to sh
The MSYS2 runtime does its best to emulate the command-line wildcard
expansion and de-quoting which would be performed by the calling Unix
shell on Unix systems.

Those Unix shell quoting rules differ from the quoting rules applying to
Windows' cmd and Powershell, making it a little awkward to quote
command-line parameters properly when spawning other processes.

In particular, git.exe passes arguments to subprocesses that are *not*
intended to be interpreted as wildcards, and if they contain
backslashes, those are not to be interpreted as escape characters, e.g.
when passing Windows paths.

Note: this is only a problem when calling MSYS2 executables, not when
calling MINGW executables such as git.exe. However, we do call MSYS2
executables frequently, most notably when setting the use_shell flag in
the child_process structure.

There is no elegant way to determine whether the .exe file to be
executed is an MSYS2 program or a MINGW one. But since the use case of
passing a command line through the shell is so prevalent, we need to
work around this issue at least when executing sh.exe.

Let's introduce an ugly, hard-coded test whether argv[0] is "sh", and
whether it refers to the MSYS2 Bash, to determine whether we need to
quote the arguments differently than usual.

That still does not fix the issue completely, but at least it is
something.

Incidentally, this also fixes the problem where `git clone \\server\repo`
failed due to incorrect handling of the backslashes when handing the path
to the git-upload-pack process.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:57:02 +01:00
Johannes Schindelin
cd7c8e6765 mingw: try to create symlinks without elevated permissions
With Windows 10 Build 14972 in Developer Mode, a new flag is supported
by CreateSymbolicLink() to create symbolic links even when running
outside of an elevated session (which was previously required).

This new flag is called SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE and
has the numeric value 0x02.

Previous Windows 10 versions will not understand that flag and return an
ERROR_INVALID_PARAMETER, therefore we have to be careful to try passing
that flag only when the build number indicates that it is supported.

For more information about the new flag, see this blog post:
https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/

This patch is loosely based on the patch submitted by Samuel D. Leslie
as https://github.com/git-for-windows/git/pull/1184.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:56:55 +01:00
Johannes Schindelin
8f63abadba mingw: kill unterminated child processes on signals
Git for Windows' MSYS2 runtime was just adjusted to kill processes
gently, by injecting a thread that calls ExitProcess(). In case of
signals (such as when handling Ctrl+C in a MinTTY window), the exit code
is 128 + sign_no, as expected by Git's source code.

However, as there is no POSIX signal handling on Windows, no signal
handlers are called. Instead, functions registered via atexit() are
called. We work around that by testing the exit code explicitly.

This fixes the Git for Windows side of the bug where  interrupting `git
clone https://...` would send the spawned-off `git remote-https` process
into the background instead of interrupting it, i.e. the clone would
continue and its progress would be reported mercilessly to the console
window without the user being able to do anything about it (short of
firing up the task manager and killing the appropriate task manually).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:56:48 +01:00
Johannes Schindelin
62f68aab88 mingw: kill child processes in a gentler way
The TerminateProcess() function does not actually leave the child
processes any chance to perform any cleanup operations. This is bad
insofar as Git itself expects its signal handlers to run.

A symptom is e.g. a left-behind .lock file that would not be left behind
if the same operation was run, say, on Linux.

To remedy this situation, we use an obscure trick: we inject a thread
into the process that needs to be killed and to let that thread run the
ExitProcess() function with the desired exit status. Thanks J Wyman for
describing this trick.

The advantage is that the ExitProcess() function lets the atexit
handlers run. While this is still different from what Git expects (i.e.
running a signal handler), in practice Git sets up signal handlers and
atexit handlers that call the same code to clean up after itself.

In case that the gentle method to terminate the process failed, we still
fall back to calling TerminateProcess(), but in that case we now also
make sure that processes spawned by the spawned process are terminated;
TerminateProcess() does not give the spawned process a chance to do so
itself.

Please note that this change only affects how Git for Windows tries to
terminate processes spawned by Git's own executables. Third-party
software that *calls* Git and wants to terminate it *still* need to make
sure to imitate this gentle method, otherwise this patch will not have
any effect.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:56:48 +01:00
Johannes Schindelin
be9e762ce5 Merge branch 'drive-prefix'
This topic branch allows us to specify absolute paths without the drive
prefix e.g. when cloning.

Example:

	C:\Users\me> git clone https://github.com/git/git \upstream-git

This will clone into a new directory C:\upstream-git, in line with how
Windows interprets absolute paths.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:56:45 +01:00
Johannes Schindelin
dfbf2a9184 mingw: fix isatty() after dup2()
We newly handle isatty() by special-casing the stdin/stdout/stderr file
descriptors, caching the return value. However, we missed the case where
dup2() overrides the respective file descriptor.

That poses a problem e.g. where the `show` builtin asks for a pager very
early, the `setup_pager()` function sets the pager depending on the
return value of `isatty()` and then redirects stdout. Subsequently,
`cmd_log_init_finish()` calls `setup_pager()` *again*. What should
happen now is that `isatty()` reports that stdout is *not* a TTY and
consequently stdout should be left alone.

Let's override dup2() to handle this appropriately.

This fixes https://github.com/git-for-windows/git/issues/1077

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:56:39 +01:00
Johannes Schindelin
59635e43c7 mingw: ensure valid CTYPE
A change between versions 2.4.1 and 2.6.0 of the MSYS2 runtime modified
how Cygwin's runtime (and hence Git for Windows' MSYS2 runtime
derivative) handles locales: d16a56306d (Consolidate wctomb/mbtowc calls
for POSIX-1.2008, 2016-07-20).

An unintended side-effect is that "cold-calling" into the POSIX
emulation will start with a locale based on the current code page,
something that Git for Windows is very ill-prepared for, as it expects
to be able to pass a command-line containing non-ASCII characters to the
shell without having those characters munged.

One symptom of this behavior: when `git clone` or `git fetch` shell out
to call `git-upload-pack` with a path that contains non-ASCII
characters, the shell tried to interpret the entire command-line
(including command-line parameters) as executable path, which obviously
must fail.

This fixes https://github.com/git-for-windows/git/issues/1036

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:56:38 +01:00
Johannes Schindelin
4f8f181825 Merge pull request #994 from jeffhostetler/jeffhostetler/fscache_nfd
fscache: add not-found directory cache to fscache
2018-03-23 13:56:37 +01:00
Johannes Schindelin
dbd3abe0c8 Merge pull request #1003 from shoelzer/master
poll: Use GetTickCount64 to avoid wraparound issues
2018-03-23 13:56:28 +01:00
Johannes Schindelin
2a97b52d49 Merge 'misc-vs-fixes-extra' into HEAD
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:56:23 +01:00
Johannes Schindelin
cd1e308b02 Merge branch 'visual-studio'
This topic branch teaches the project generator to generate a Visual
Studio solution, ready to be opened in Visual Studio 2010 or later.

The idea, of course, is to let some automatic build job generate and
commit the project files with

	make MSVC=1 vcxproj

and then (force-)push to a special-purpose branch.

The major part of this branch thicket concerns itself not only with
generating the Visual Studio project files, but making sure that the
user can then run the test suite from a regular Git Bash (i.e. *not*
requiring a Git for Windows SDK), e.g. by running

	cd t
	prove --timer --jobs 15 ./t[0-9]*.sh

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:56:21 +01:00
Johannes Schindelin
4f1c11e19b mingw: make readlink() independent of core.symlinks
Regardless whether we think we are able to create symbolic links, we
should always read them.

This fixes https://github.com/git-for-windows/git/issues/958

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:56:16 +01:00
Johannes Schindelin
5f0b80ff0c Merge branch 'mingw-strftime'
This topic branch works around an out-of-memory bug when the user
specified a format via --date=format:<format> that strftime() does
not like.

Reported by Stefan Naewe.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:56:07 +01:00
Johannes Schindelin
fd3ef2677d Merge branch 'spawn-with-spaces'
This change lets us spawn .bat scripts whose paths contain spaces.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:55:56 +01:00
Johannes Schindelin
6e0f18c9e2 Merge pull request #773 from jeffhostetler/vs2015
Build with VS2015
2018-03-23 13:55:44 +01:00
Johannes Schindelin
6aa9333d3a Merge branch 'program-data-config'
This branch introduces support for reading the "Windows-wide" Git
configuration from `%PROGRAMDATA%\Git\config`. As these settings are
intended to be shared between *all* Git-related software, that config
file takes an even lower precedence than `$(prefix)/etc/gitconfig`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:55:39 +01:00
Johannes Schindelin
e6099362d6 Merge 'mingw-getcwd' into HEAD 2018-03-23 13:55:37 +01:00
Johannes Schindelin
87f46e422d Merge pull request #443 from kblees/kb/nanosecond-file-times-v2.5.3
nanosecond file times for v2.5.3
2018-03-23 13:55:34 +01:00
Johannes Schindelin
de53194188 Merge pull request #156 from kblees/kb/symlinks
Symlink support
2018-03-23 13:55:32 +01:00
Johannes Schindelin
249e9a8865 Merge 'fix-externals' into HEAD 2018-03-23 13:55:28 +01:00
Johannes Schindelin
87025a35bc mingw: allow absolute paths without drive prefix
When specifying an absolute path without a drive prefix, we convert that
path internally. Let's make sure that we handle that case properly, too
;-)

This fixes the command

	git clone https://github.com/git-for-windows/git \G4W

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:55:07 +01:00
Jeff Hostetler
37cafeaea7 fscache: remember not-found directories
Teach FSCACHE to remember "not found" directories.

This is a performance optimization.

FSCACHE is a performance optimization available for Windows.  It
intercepts Posix-style lstat() calls into an in-memory directory
using FindFirst/FindNext.  It improves performance on Windows by
catching the first lstat() call in a directory, using FindFirst/
FindNext to read the list of files (and attribute data) for the
entire directory into the cache, and short-cut subsequent lstat()
calls in the same directory.  This gives a major performance
boost on Windows.

However, it does not remember "not found" directories.  When STATUS
runs and there are missing directories, the lstat() interception
fails to find the parent directory and simply return ENOENT for the
file -- it does not remember that the FindFirst on the directory
failed. Thus subsequent lstat() calls in the same directory, each
re-attempt the FindFirst.  This completely defeats any performance
gains.

This can be seen by doing a sparse-checkout on a large repo and
then doing a read-tree to reset the skip-worktree bits and then
running status.

This change reduced status times for my very large repo by 60%.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:54:56 +01:00
Jeff Hostetler
56e3d74eb7 fscache: add key for GIT_TRACE_FSCACHE
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:54:56 +01:00
Johannes Schindelin
4494a4db6d poll: lazy-load GetTickCount64()
This fixes the compilation, actually, as we still did not make the jump to
post-Windows XP completely: we still compile with _WIN32_WINNT set to
0x0502 (which corresponds to Windows Server 2003 and is technically
greater than Windows XP's 0x0501).

However, GetTickCount64() is only available starting with Windows
Vista/Windows Server 2008.

Let's just lazy-load the function, which should also help Git for Windows
contributors who want to reinstate Windows XP support.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:54:38 +01:00
Steve Hoelzer
1c9108ba55 poll: Use GetTickCount64 to avoid wraparound issues
From Visual Studio 2015 Code Analysis: Warning C28159 Consider using
'GetTickCount64' instead of 'GetTickCount'.

Reason: GetTickCount overflows roughly every 49 days. Code that does not
take that into account can loop indefinitely. GetTickCount64 operates on
64 bit values and does not have that problem.

Signed-off-by: Steve Hoelzer <shoelzer@gmail.com>
2018-03-23 13:54:37 +01:00
Johannes Schindelin
6ee005a4de msvc: avoid debug assertion windows in Debug Mode
For regular debugging, it is pretty helpful when a debug assertion in a
running application triggers a window that offers to start the debugger.

However, when running the test suite, it is not so helpful, in
particular when the debug assertions are then suppressed anyway because
we disable the invalid parameter checking (via invalidcontinue.obj, see
the comment in config.mak.uname about that object for more information).

So let's simply disable that window in Debug Mode (it is already
disabled in Release Mode).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:54:31 +01:00
Johannes Schindelin
46a1b3df71 Merge 'misc-vs-fixes' into HEAD 2018-03-23 13:54:24 +01:00
Philip Oakley
54f95fdd24 msvc: add pragmas for common warnings
MSVC can be overzealous about some warnings. Disable them.

Signed-off-by: Philip Oakley <philipoakley@iee.org>
2018-03-23 13:54:21 +01:00
Johannes Schindelin
86dc23d5d6 terminal.c: guard the inclusion of inttypes.h
We do have a lovely Makefile option to state that that header file is
not available. Let's use it everywhere...

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:54:05 +01:00
Johannes Schindelin
e140160eaa obstack: fix compiler warning
MS Visual C suggests that the construct

	condition ? (int) i : (ptrdiff_t) d

is incorrect. Let's fix this by casting to ptrdiff_t also for the
positive arm of the conditional.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:54:05 +01:00
Johannes Schindelin
a46bd1995b mingw: abort on invalid strftime formats
On Windows, strftime() does not silently ignore invalid formats, but
warns about them and then returns 0 and sets errno to EINVAL.

Unfortunately, Git does not expect such a behavior, as it disagrees
with strftime()'s semantics on Linux. As a consequence, Git
misinterprets the return value 0 as "I need more space" and grows the
buffer. As the larger buffer does not fix the format, the buffer grows
and grows and grows until we are out of memory and abort.

Ideally, we would switch off the parameter validation just for
strftime(), but we cannot even override the invalid parameter handler
via _set_thread_local_invalid_parameter_handler() using MINGW because
that function is not declared. Even _set_invalid_parameter_handler(),
which *is* declared, does not help, as it simply does... nothing.

So let's just bite the bullet and override strftime() for MINGW and
abort on an invalid format string. While this does not provide the
best user experience, it is the best we can do.

See https://msdn.microsoft.com/en-us/library/fe06s4ak.aspx for more
details.

This fixes https://github.com/git-for-windows/git/issues/863

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:53:51 +01:00
Johannes Schindelin
264993b771 mingw: support spawning programs containing spaces in their names
The CreateProcessW() function does not really support spaces in its
first argument, lpApplicationName. But it supports passing NULL as
lpApplicationName, which makes it figure out the application from the
(possibly quoted) first argument of lpCommandLine.

Let's use that trick (if we are certain that the first argument matches
the executable's path) to support launching programs whose path contains
spaces.

This fixes https://github.com/git-for-windows/git/issue/692

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:53:34 +01:00
Johannes Schindelin
04bbd16932 compat/vcbuild: possibly reuse Git for Windows' SDK's NuGet
In Git for Windows' SDK, there is already a script to package Git for
Windows as a NuGet package, downloading nuget.exe if needed.

Let's just fall back to using that executable (if it is there) if
nuget.exe was not found in the PATH.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:53:17 +01:00
Jeff Hostetler
d16acfbd9b vs2015: remove todo list item from README_vs2015.txt
Remove todo list item for vs2015 build notes regarding
the third party DLLs.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2018-03-23 13:53:16 +01:00
Johannes Schindelin
2c282eb1c9 msvc: use libiconv
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:53:14 +01:00
Johannes Schindelin
17881c63a4 msvc: respect the quiet-by-default output
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-03-23 13:53:13 +01:00
Jeff Hostetler
e3f6de6b1b msvc: release mode PDBs and library DLLs
Install required third-party DLLs next to EXEs.

Build and install release mode PDBs for git
executables allowing detailed stack traces
in the event of crash.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2018-03-23 13:53:13 +01:00
Johannes Schindelin
cd6dce47ee msvc: fix detect_msys_tty()
The ntstatus.h header is only available in MINGW.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2018-03-23 13:53:12 +01:00
Jeff Hostetler
c00eafbc9a msvc: define ftello()
It is just called different in MSVC's headers.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2018-03-23 13:53:12 +01:00
Jeff Hostetler
1448e2d9a1 msvc: do not re-declare the timespec struct
VS2015's headers already declare that struct.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2018-03-23 13:53:11 +01:00
Johannes Schindelin
bf6b5413ba msvc: provide a main() wrapper similar to mingw_main()
The MINGW version of the main() wrapper gets away with declaring symbols
that were intentionally not exported. However, some of these symbols do
not actually exist in MSVC's UCRT.

So let's add an MSVC version of the main() wrapper that uses wmain() and
imports the UNICODE argv and environment. While at it, we pass our UTF-8
version of ARGV to the real main -- rather than overwriting __argv as is
done in the MINGW Version.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2018-03-23 13:53:11 +01:00
Jeff Hostetler
a22de0e1a2 msvc: do not pretend to support all signals
This special-cases various signals that are not supported on Windows,
such as SIGPIPE. These cause the UCRT to throw asserts (at least in
debug mode).

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2018-03-23 13:53:10 +01:00
Jeff Hostetler
dcee2231b7 msvc: mark a variable as non-const
VS2015 complains when using a const pointer in memcpy()/free().

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2018-03-23 13:53:10 +01:00