Commit Graph

506 Commits

Author SHA1 Message Date
Johannes Schindelin
5621f24dce Silence GCC's "cast of pointer to integer of a different size" warning
When calculating hashes from pointers, it actually makes sense to cut
off the most significant bits. In that case, said warning does not make
a whole lot of sense.

So let's just work around it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-31 18:24:15 +01:00
Johannes Schindelin
4646cc2993 mingw: avoid warnings when casting HANDLEs to int
HANDLE is defined internally as a void *, but in many cases it is
actually guaranteed to be a 32-bit integer. In these cases, GCC should
not warn about a cast of a pointer to an integer of a different type
because we know exactly what we are doing.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-31 18:06:33 +01:00
Johannes Schindelin
c557dee8da mingw: uglify pthread_mutex_init definition to shut up warning
When the result of a (a, 0) expression is not used, GCC now finds it
necessary to complain with a warning:

	right-hand operand of comma expression has no effect

Let's just pretend to use the 0 value and have a peaceful and quiet life
again.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-31 17:39:54 +01:00
nalla
caaae01999 git-wrapper: support git.exe to be in a spaced dir
When *Git for Windows* is installed into a directory that has spaces in
it, e.g. `C:\Program Files\Git`, the `git-wrapper` appends this directory
unquoted when fixing up the command line. To resolve this, just quote the
provided `execpath`.

Signed-off-by: nalla <nalla@hamal.uberspace.de>
2015-03-30 15:10:38 +01:00
Johannes Schindelin
be311e2381 git-wrapper: Allow git-cmd.exe to add only /cmd/ to the PATH
The idea of having the Git wrapper in the /cmd/ directory is to allow
adding only a *tiny* set of executables to the search path, to allow
minimal interference with other software applications. It is quite
likely, for example, that other software applications require their own
version of zlib1.dll and would not be overly happy to find the version
Git for Windows ships.

The /cmd/ directory also gives us the opportunity to let the Git wrapper
handle the `gitk` script. It is a Tcl/Tk script that is not recognized
by Windows, therefore calling `gitk` in `cmd.exe` would not work, even
if we add all of Git for Windows' bin/ directories.

So let's use the /cmd/ directory instead of adding /mingw??/bin/ and
/usr/bin/ to the PATH when launching Git CMD.

The way we implemented Git CMD is to embed the appropriate command line
as string resource into a copy of the Git wrapper. Therefore we extended
that syntax to allow for configuring a minimal search path.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-28 19:15:57 +01:00
Johannes Schindelin
4766c3db08 git-wrapper: optionally skip cd $HOME when configured via resources
We recently added the ability to configure copies of the Git wrapper to
launch custom command-lines, configured via plain old Windows resources.
The main user is Git for Windows' `git-bash.exe`, of course. When the
user double-clicks the `git bash` icon, it makes sense to start the Bash
in the user's home directory.

Third-party software, such as TortoiseGit or GitHub for Windows, may
want to start the Git Bash in another directory, though.

Now, when third-party software wants to call Git, they already have to
construct a command-line, and can easily pass a command-line option
`--no-cd` (which this commit introduces), and since that option is not
available when the user double-clicks an icon on the Desktop or in the
Explorer, let's keep the default to switch to the home directory if the
`--no-cd` flag was not passed along.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-28 19:04:24 +01:00
Johannes Schindelin
b072d688cf git-wrapper: make command-line argument skipping more robust
When we rewrite the command-line to call the *real* Git, we want to skip
the first command-line parameter. The previous code worked in most
circumstances, but was a bit fragile because it assumed that no fancy
quoting would take place.

In the next commit, we will want to have the option to skip more than
just one command-line parameter, so we have to be much more careful with
the command-line handling.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-28 19:04:21 +01:00
Johannes Schindelin
83553058ba git-wrapper: remove 'gui' and 'citool' handling
In the meantime, Git for Windows learned to handle those subcommands
quite well itself; There is no longer a need to special-case them in the
wrapper.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-28 19:04:19 +01:00
Johannes Schindelin
2ac10bd7c5 Let the Git wrapper replace cmd\gitk.cmd, too
In a push to polish Git for Windows more, we are moving away from
scripts toward proper binaries.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-28 19:04:17 +01:00
Johannes Schindelin
41c1e1f32c Git wrapper: allow overriding what executable is called
The Git wrapper does one thing, and does it well: setting up the
environment required to run Git and its scripts, and then hand off to
another program.

We already do this for the Git executable itself; in Git for Windows'
context, we have exactly the same need also when calling the Git Bash or
Git CMD. However, both are tied to what particular shell environment you
use, though: MSys or MSys2 (or whatever else cunning developers make
work for them). This means that the Git Bash and Git CMD need to be
compiled in the respective context (e.g. when compiling the
mingw-w64-git package in the MSys2 context).

Happily, Windows offers a way to configure compiled executables:
resources. So let's just look whether the current executable has a
string resource and use it as the command-line to execute after the
environment is set up. To support MSys2's Git Bash better (where
`mintty` should, but might not, be available), we verify whether the
specified executable exists, and keep looking for string resources if it
does not.

For even more flexibility, we expand environment variables specified as
`@@<VARIABLE-NAME>@@`, and for convenience `@@EXEPATH@@` expands into
the directory in which the executable resides.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-28 19:04:16 +01:00
Johannes Schindelin
1b1e072595 fixup! git-wrapper: add code to configure command-lines to be launched
The intention of this change was to make it easier for the Git for
Windows installer, or for power Git users, to change the command-line
launched when executing Git Bash. The idea was to allow reconfiguring
the Git Bash to run different terminals than MSys2's default, mintty.

However, the comments this commit got let no room for misunderstanding:
at least three developers who gained trust by being active in the Git
for Windows offered their vetoes.

RIP, resource editor.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-28 19:04:13 +01:00
Johannes Schindelin
c5cdad2f8d fixup! git-wrapper: allow the edit-res.exe copy to change icons, too
Instead of making it possible to change the .exe icon of git-bash.exe or
git-cmd.exe, this change was compared to adding the option to format USB
drives.

Let's get rid of this change, therefore, before real people are harmed.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-28 19:04:12 +01:00
Johannes Schindelin
580472680d git-wrapper: inherit stdin/stdout/stderr even without a console
Otherwise the output of Git commands cannot be caught by, say, Git GUI
(because it is running detached from any console, which would make
`git.exe` inherit the standard handles implicitly).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-28 19:03:31 +01:00
Johannes Schindelin
35e219dff2 git-wrapper: allow the edit-res.exe copy to change icons, too
As we already allow changing text resources, it is a very small step to
allow changing the `.exe` icon, too.

So let's just do it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-26 09:05:14 +01:00
Johannes Schindelin
304d297463 git-wrapper: add code to configure command-lines to be launched
To allow the Git wrapper to replace the `git-bash.bat` script (which
would always open the Win32 console, even if the user wants to use a
different terminal emulator), we need to offer a way to configure
*which* command-line to run.

We need to recompile the git-wrapper with the `-mwindows` flag to
declare that this is a GUI program (not a console program). Having to
recompile it anyway, let's put the new code in a conditionally compiled
section so that the builtins (for which we use the git-wrapper, too) do
not need to carry around that code.

To configure the command-line, we use a way that is very typical for
Windows: resources. Windows resources are data that are stored inside
.exe files, but can be changed *after* compilation. Therefore, this
facility is *exactly* what we want: we can easily copy the .exe to the
new name `git-bash.exe`, configure that executable to run the Bash, and
then copy it again to the new name `git-cmd.exe` and configure that
executable to run `cmd.exe` instead.

For even more flexibility, we expand environment variables specified as
`@@<VARIABLE-NAME>@@`, and for convenience `@@EXEPATH@@` expands into
the directory in which the executable resides.

Sadly, an executable cannot configure itself: the `.exe` file is locked
while the process is running. This means we have to have a separate
executable to edit the resources anyway, so let's just enhance the Git
wrapper *itself*: when copied to the new name `edit-res.exe`, it can
edit other copies of the Git wrapper like so:

	edit-res.exe git-cmd.exe command '@@COMSPEC@@ /K'

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-26 09:05:12 +01:00
Johannes Schindelin
d7ff2b2b16 git-wrapper: prepare for executing configurable command-lines
We are about to use the Git wrapper to call the Git Bash of Git for
Windows. All the wrapper needs to do for that is to set up the
environment variables, use the home directory as working directory and
then hand off to a user-specified command-line.

We prepare the existing code for this change by introducing flags to set
up the environment variables, to launch a non-Git program, and to use
the home directory as working directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-26 09:05:11 +01:00
Johannes Schindelin
9dbe2649f7 git-wrapper: support MSys2
The original purpose of the Git wrapper is to run from inside Git for
Windows' /cmd/ directory, to allow setting up some environment variables
before Git is allowed to take over.

Due to differences in the file system layout, MSys2 requires some
changes for that to work.

In addition, we must take care to set the `MSYSTEM` environment variable
to `MINGW32` or `MINGW64`, respectively, to allow MSys2 to be configured
correctly in case Git launches a shell or Perl script.

We also need to change the `TERM` variable to `cygwin` instead of
`msys`, otherwise the pager `less.exe` (spawned e.g. by `git log`) will
simply crash with a message similar to this one:

	1 [main] less 9832 cygwin_exception::open_stackdumpfile:
	Dumping stack trace to less.exe.stackdump

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-26 09:00:18 +01:00
Johannes Schindelin
4f5f8566a8 fixup! Let the Git wrapper serve as a drop-in replacement for builtins
Prepare to prefix the command-line with non-builtins. We are now using
the Git wrapper to call non-Git programs, too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-25 20:22:22 +01:00
Johannes Schindelin
3ee6eff6fd Merge pull request #38 from dscho/mingw-environment
Fix access violations when cloning/fetching via HTTPS
2015-03-21 10:43:19 +01:00
Johannes Schindelin
e9d0cf4178 Merge pull request #40 from nalla/git-terminal-prompt
mingw: Proper `git_terminal_prompt` with `xterm`
2015-03-21 10:43:19 +01:00
Johannes Schindelin
12e5c37398 mingw: be *very* wary about outside environment changes
The environment is modified in most surprising circumstances, and not
all of them are under Git's control. For example, calling
curl_global_init() on Windows will ensure that the CHARSET variable is
set, adding one if necessary.

While the previous commit worked around crashes triggered by such
outside changes of the environment by relaxing the requirement that the
environment be terminated by a NULL pointer, the other assumption made
by `mingw_getenv()` and `mingw_putenv()` is that the environment is
sorted, for efficient lookup via binary search.

Let's make real sure that our environment is intact before querying or
modifying it, and reinitialize our idea of the environment if necessary.

With this commit, before working on the environment we look briefly for
indicators that the environment was modified outside of our control, and
to ensure that it is terminated with a NULL pointer and sorted again in
that case.

Note: the indicators are maybe not sufficient. For example, when a
variable is removed, it will not be noticed. It might also be a problem
if outside changes to the environment result in a modified `environ`
pointer: it is unclear whether such a modification could result in a
problem when `mingw_putenv()` needs to `realloc()` the environment
buffer.

For the moment, however, the current fix works well enough, so let's
only face the potential problems when (and if!) they occur.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-21 10:43:10 +01:00
Johannes Schindelin
6432af8787 mingw: be more defensive when making the environment block
Outside of our Windows-specific code, the end of the environment can be
marked also by a pointer to a NUL character, not only by a NULL pointer
as our code assumed so far.

That led to a buffer overrun in `make_environment_block()` when running
`git-remote-https` in `mintty` (because `curl_global_init()` added the
`CHARSET` environment variable *outside* of `mingw_putenv()`, ending the
environment in a pointer to an empty string).

Side note for future debugging on Windows: when running programs in
`mintty`, the standard input/output/error is not connected to a Win32
Console, but instead is pipe()d. That means that even stderr may not be
written completely before a crash, but has to be fflush()ed explicitly.
For example, when debugging crashes, the developer should insert an
`fflush(stderr);` at the end of the `error()` function defined in
usage.c.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-21 10:43:10 +01:00
Johannes Schindelin
66d271482b UTF-8 environment: be a little bit more defensive
It is unlikely that we have an empty environment, ever, but *if* we do,
when `environ_size - 1` is passed to `bsearchenv()` it is misinterpreted
as a real large integer.

To make the code truly defensive, refuse to do anything at all if the
size is negative (which should not happen, of course).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-21 10:43:10 +01:00
nalla
24f9b07c71 mingw: Support git_terminal_prompt with more terminals
The `git_terminal_prompt()` function expects the terminal window to be
attached to a Win32 Console. However, this is not the case with terminal
windows other than `cmd.exe`'s, e.g. with MSys2's own `mintty`.

Non-cmd terminals such as `mintty` still have to have a Win32 Console
to be proper console programs, but have to hide the Win32 Console to
be able to provide more flexibility (such as being resizeable not only
vertically but also horizontally). By writing to that Win32 Console,
`git_terminal_prompt()` manages only to send the prompt to nowhere and
to wait for input from a Console to which the user has no access.

This commit introduces a function specifically to support `mintty` -- or
other terminals that are compatible with MSys2's `/dev/tty` emulation. We
use the `TERM` environment variable as an indicator for that: if the value
starts with "xterm" (such as `mintty`'s "xterm_256color"), we prefer to
let `xterm_prompt()` handle the user interaction.

To handle the case when standard input/output are redirected – as is the
case when pushing via HTTPS: `git-remote-https`' standard input and
output are pipes from/to the main Git executable – we make use of the
`MSYS_TTY_HANDLES` environment variable that was introduced to
fix another bug in MSys2-based Git: this environment variable contains
the Win32 `HANDLE`s of the standard input, output and error as originally
passed from MSys2 to the Git executable, enclosed within space
characters, skipping handles that do not refer to the terminal window
(e.g. when they were redirected). We will only use those handles when
that environment variable lists all three handles because then we can be
100% certain that we are running inside a terminal window, and that we
know exactly which Win32 handles to use to communicate with it.

Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: nalla <nalla@hamal.uberspace.de>
2015-03-21 10:43:09 +01:00
Johannes Schindelin
68a8e3108e Merge 'poll_inftim' into HEAD
This was originally 'pull request #330 from ethomson/poll_inftim' in
msysgit/git.

poll: honor the timeout on Win32

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-20 15:04:57 +01:00
Johannes Schindelin
d002525dd2 Merge pull request #34 from dscho/git-wrapper
Use msysGit's `git-wrapper` instead of the builtins
2015-03-20 15:04:55 +01:00
Johannes Schindelin
d818c047bc Merge pull request #122 from kblees/kb/long-paths-v2
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-20 15:04:47 +01:00
Johannes Schindelin
931ec5dd53 Merge remote-tracking branch 'kblees/kb/fscache-v4-tentative-1.8.5' into thicket-1.8.5.2 2015-03-20 15:04:45 +01:00
Johannes Schindelin
a8e39d4c40 Merge 'fix-externals' into HEAD 2015-03-20 15:04:35 +01:00
Johannes Schindelin
02e2dfebf8 Merge pull request #28 from dscho/tty-handles
Teach msys2-runtime to hand the tty through to Git
2015-03-20 15:04:27 +01:00
Johannes Schindelin
59967c3f79 Merge pull request #33 from dscho/manifest
Embed a manifest into git.exe
2015-03-20 15:04:26 +01:00
Johannes Schindelin
5d039ed525 Merge pull request #26 from dscho/msys2
Fixes required to build Git for Windows with MSys2
2015-03-20 15:04:23 +01:00
Edward Thomson
1d99025822 poll: honor the timeout on Win32
Ensure that when passing a pipe, the gnulib poll replacement will not
return 0 before the timeout has passed.

Not obeying the timeout (and merely returning 0) causes pathological
behavior when preparing a packfile for a repository and taking a
long time to do so.  If poll were to return 0 immediately, this would
cause keep-alives to get sent as quickly as possible until the packfile
was created.  Such deviance from the standard would cause megabytes (or
more) of keep-alive packets to be sent.

GetTickCount is used as it is efficient, stable and monotonically
increasing.  (Neither GetSystemTime nor QueryPerformanceCounter have
all three of these properties.)
2015-03-20 14:34:59 +01:00
Johannes Schindelin
7bfa170534 Let the Git wrapper serve as a drop-in replacement for builtins
Git started out as a bunch of separate commands, in the true Unix spirit.
Over time, more and more functionality was shared between the different
Git commands, though, so it made sense to introduce the notion of
"builtins": programs that are actually integrated into the main Git
executable.

These builtins can be called in two ways: either by specifying a
subcommand as the first command-line argument, or -- for backwards
compatibility -- by calling the Git executable hardlinked to a filename
of the form "git-<subcommand>". Example: the "log" command can be called
via "git log <parameters>" or via "git-log <parameters>". The latter
form is actually deprecated and only supported for scripts; calling
"git-log" interactively will not even work by default because the
libexec/git-core/ directory is not in the PATH.

All of this is well and groovy as long as hard links are supported.

Sadly, this is not the case in general on Windows. So it actually hurts
quite a bit when you have to fall back to copying all of git.exe's
currently 7.5MB 109 times, just for backwards compatibility.

The simple solution would be to install really trivial shell script
wrappers in place of the builtins:

	for builtin in $BUILTINS
	do
		rm git-$builtin.exe
		printf '#!/bin/sh\nexec git %s "$@"\n' $builtin > git-builtin
		chmod a+x git-builtin
	done

This method would work -- even on Windows because Git for Windows ships a
full-fledged Bash. However, the Windows Bash comes at a price: it needs to
spin up a full-fledged POSIX emulation layer everytime it starts.
Therefore, the shell script solution would incur a significant performance
penalty.

The best solution the Git for Windows team could come up with is to extend
the Git wrapper -- that is needed to call Git from cmd.exe anyway, and
that weighs in with a scant 19KB -- to also serve as a drop-in replacement
for the builtins so that the following workaround is satisfactory:

	for builtin in $BUILTINS
	do
		cp git-wrapper.exe git-$builtin.exe
	done

This commit allows for this, by extending the module file parsing to
turn builtin command names like `git-log.exe ...` into calls to the main
Git executable: `git.exe log ...`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-20 14:34:03 +01:00
Johannes Schindelin
b8e9daa455 Refactor git-wrapper into more functions
This prepares the wrapper for modifications to serve as a drop-in
replacement for the builtins.

This commit's diff is best viewed with the `-w` flag.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-20 14:34:02 +01:00
Johannes Schindelin
d4183729b2 mingw: Work around MSVCRT's isatty() not knowing about MSys2
MSys2 has a slightly different notion of what constitutes a tty than
the Microsoft C runtime. The former knows whether stdin/stdout/stderr
was redirected or not, while the latter looks for a Win32 Console.

In particular when we want to know whether to spawn a pager or not, we
would rather want to know what MSys2 thinks.

We are about to introduce a change to the msys2-runtime that sets an
environment variable MSYS_TTY_HANDLES to a list of Win32 handles that
correspond to stdin/stdout/stderr, respectively, *but skips* handles that
MSys2 does not think are terminals.

This commit handles that input to augment the isatty() function to return
1 also when MSYS_TTY_HANDLES contains the corresponding handle.

The only time when Git needs to know whether a Console is attached or not
is when winansi.c is asked to Do Its Thing, therefore we refrain from
overriding isatty there.

Note: this was an issue with MSys1-based Git for Windows, too, hidden by
the fact that Git for Windows used `cmd.exe` as a terminal -- which is
backed by a real Win32 Console. Had MSys1 used, say, rxvt as its default
terminal, the symptom would have been that "git log" does not spawn a
pager by default but instead outputs the entire history (without color
coding, too). In MSys2, the default terminal is mintty, therefore we
finally could not avoid to address the issue.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-20 14:33:24 +01:00
Johannes Schindelin
27db4f8930 Add Git for Windows' wrapper executable
On Windows, Git is faced by the challenge that it has to set up certain
environment variables before running Git under special circumstances
such as when Git is called directly from cmd.exe (i.e. outside any
Bash environment).

This source code was taken from msysGit's commit 74a198d:

https://github.com/msysgit/msysgit/blob/74a198d/src/git-wrapper/git-wrapper.c

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-20 14:33:24 +01:00
Cesar Eduardo Barros
08184097bf mingw: Embed a manifest to trick UAC into Doing The Right Thing
On Windows >= Vista, not having an application manifest with a
requestedExecutionLevel can cause several kinds of confusing behavior.

The first and more obvious behavior is "Installer Detection", where
Windows sometimes decides (by looking at things like the file name and
even sequences of bytes within the executable) that an executable is an
installer and should run elevated (causing the well-known popup dialog
to appear). In Git's context, subcommands such as "git patch-id" or "git
update-index" fall prey to this behavior.

The second and more confusing behavior is "File Virtualization". It
means that when files are written without having write permission, it
does not fail (as expected), but they are instead redirected to
somewhere else. When the files are read, the original contents are
returned, though, not the ones that were just written somewhere else.
Even more confusing, not all write accesses are redirected; Trying to
write to write-protected .exe files, for example, will fail instead of
redirecting.

In addition to being unwanted behavior, File Virtualization causes
dramatic slowdowns in Git (see for instance
http://code.google.com/p/msysgit/issues/detail?id=320).

There are two ways to prevent those two behaviors: Either you embed an
application manifest within all your executables, or you add an external
manifest (a file with the same name followed by .manifest) to all your
executables. Since Git's builtins are hardlinked (or copied), it is
simpler and more robust to embed a manifest.

A recent enough MSVC compiler should already embed a working internal
manifest, but for MinGW you have to do so by hand.

Very lightly tested on Wine, where like on Windows XP it should not make
any difference.

References:
  - New UAC Technologies for Windows Vista
    http://msdn.microsoft.com/en-us/library/bb756960.aspx
  - Create and Embed an Application Manifest (UAC)
    http://msdn.microsoft.com/en-us/library/bb756929.aspx

[js: simplified the embedding dramatically by reusing Git for Windows'
existing Windows resource file, removed the optional (and dubious)
processorArchitecture attribute of the manifest's assemblyIdentity
section.]

Signed-off-by: Cesar Eduardo Barros <cesarb@cesarb.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-20 14:33:23 +01:00
Karsten Blees
71463a023a Win32: fix 'lstat("dir/")' with long paths
Use a suffciently large buffer to strip the trailing slash.

Signed-off-by: Karsten Blees <blees@dcon.de>
2015-03-20 14:12:07 +01:00
Karsten Blees
6a2aba2d55 Win32: support long paths
Windows paths are typically limited to MAX_PATH = 260 characters, even
though the underlying NTFS file system supports paths up to 32,767 chars.
This limitation is also evident in Windows Explorer, cmd.exe and many
other applications (including IDEs).

Particularly annoying is that most Windows APIs return bogus error codes
if a relative path only barely exceeds MAX_PATH in conjunction with the
current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the
infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG.

Many Windows wide char APIs support longer than MAX_PATH paths through the
file namespace prefix ('\\?\' or '\\?\UNC\') followed by an absolute path.
Notable exceptions include functions dealing with executables and the
current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as
well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...).

Introduce a handle_long_path function to check the length of a specified
path properly (and fail with ENAMETOOLONG), and to optionally expand long
paths using the '\\?\' file namespace prefix. Short paths will not be
modified, so we don't need to worry about device names (NUL, CON, AUX).

Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be
limited to MAX_PATH (at least not on Win7), so we can use it to do the
heavy lifting of the conversion (translate '/' to '\', eliminate '.' and
'..', and make an absolute path).

Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH
limit.

Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs
that support long paths.

While improved error checking is always active, long paths support must be
explicitly enabled via 'core.longpaths' option. This is to prevent end
users to shoot themselves in the foot by checking out files that Windows
Explorer, cmd/bash or their favorite IDE cannot handle.

Test suite:
Test the case is when the full pathname length of a dir is close
to 260 (MAX_PATH).
Bug report and an original reproducer by Andrey Rogozhnikov:
https://github.com/msysgit/git/pull/122#issuecomment-43604199

Thanks-to: Martin W. Kirst <maki@bitkings.de>
Thanks-to: Doug Kelly <dougk.ff7@gmail.com>
Signed-off-by: Karsten Blees <blees@dcon.de>
Original-test-by: Andrey Rogozhnikov <rogozhnikov.andrey@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-20 14:12:07 +01:00
Karsten Blees
0e65aaf4e5 fscache: load directories only once
If multiple threads access a directory that is not yet in the cache, the
directory will be loaded by each thread. Only one of the results is added
to the cache, all others are leaked. This wastes performance and memory.

On cache miss, add a future object to the cache to indicate that the
directory is currently being loaded. Subsequent threads register themselves
with the future object and wait. When the first thread has loaded the
directory, it replaces the future object with the result and notifies
waiting threads.

Signed-off-by: Karsten Blees <blees@dcon.de>
2015-03-20 14:12:06 +01:00
Karsten Blees
ff5db09243 Win32: add a cache below mingw's lstat and dirent implementations
Checking the work tree status is quite slow on Windows, due to slow lstat
emulation (git calls lstat once for each file in the index). Windows
operating system APIs seem to be much better at scanning the status
of entire directories than checking single files.

Add an lstat implementation that uses a cache for lstat data. Cache misses
read the entire parent directory and add it to the cache. Subsequent lstat
calls for the same directory are served directly from the cache.

Also implement opendir / readdir / closedir so that they create and use
directory listings in the cache.

The cache doesn't track file system changes and doesn't plug into any
modifying file APIs, so it has to be explicitly enabled for git functions
that don't modify the working copy.

Note: in an earlier version of this patch, the cache was always active and
tracked file system changes via ReadDirectoryChangesW. However, this was
much more complex and had negative impact on the performance of modifying
git commands such as 'git checkout'.

Signed-off-by: Karsten Blees <blees@dcon.de>
2015-03-20 14:12:06 +01:00
Karsten Blees
24d3836325 Win32: make the lstat implementation pluggable
Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite
slow. Windows operating system APIs seem to be much better at scanning the
status of entire directories than checking single files. A caching
implementation may improve performance by bulk-reading entire directories
or reusing data obtained via opendir / readdir.

Make the lstat implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Signed-off-by: Karsten Blees <blees@dcon.de>
2015-03-20 14:12:05 +01:00
Karsten Blees
e447c08887 Win32: Make the dirent implementation pluggable
Emulating the POSIX dirent API on Windows via FindFirstFile/FindNextFile is
pretty staightforward, however, most of the information provided in the
WIN32_FIND_DATA structure is thrown away in the process. A more
sophisticated implementation may cache this data, e.g. for later reuse in
calls to lstat.

Make the dirent implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Define a base DIR structure with pointers to readdir/closedir that match
the opendir implementation (i.e. similar to vtable pointers in OOP).
Define readdir/closedir so that they call the function pointers in the DIR
structure. This allows to choose the opendir implementation on a
call-by-call basis.

Move the fixed sized dirent.d_name buffer to the dirent-specific DIR
structure, as d_name may be implementation specific (e.g. a caching
implementation may just set d_name to point into the cache instead of
copying the entire file name string).

Signed-off-by: Karsten Blees <blees@dcon.de>
2015-03-20 14:12:05 +01:00
Karsten Blees
c006b3cf03 Win32: dirent.c: Move opendir down
Move opendir down in preparation for the next patch.

Signed-off-by: Karsten Blees <blees@dcon.de>
2015-03-20 14:12:05 +01:00
Karsten Blees
64a7336529 Win32: make FILETIME conversion functions public
Signed-off-by: Karsten Blees <blees@dcon.de>
2015-03-20 14:12:05 +01:00
Adam Roben
5cca1ca912 Make non-.exe externals work again
7ebac8cb94 made launching of .exe
externals work when installed in Unicode paths. But it broke launching
of non-.exe externals, no matter where they were installed. We now
correctly maintain the UTF-8 and UTF-16 paths in tandem in lookup_prog.

This fixes t5526, among others.

Signed-off-by: Adam Roben <adam@roben.org>
2015-03-20 14:12:01 +01:00
Adam Roben
e53dd0cf82 Fix launching of externals from Unicode paths
If Git were installed in a path containing non-ASCII characters,
commands such as git-am and git-submodule, which are implemented as
externals, would fail to launch with the following error:

> fatal: 'am' appears to be a git command, but we were not
> able to execute it. Maybe git-am is broken?

This was due to lookup_prog not being Unicode-aware. It was somehow
missed in 2ee5a1a14a.

Note that the only problem in this function was calling
GetFileAttributes instead of GetFileAttributesW. The calls to access()
were fine because access() is a macro which resolves to mingw_access,
which already handles Unicode correctly. But I changed lookup_prog to
use _waccess directly so that we only convert the path to UTF-16 once.

Signed-off-by: Adam Roben <adam@roben.org>
2015-03-20 14:12:01 +01:00
Evgeny Pashkin
68d2ee0187 Fixed wrong path delimiter in exe finding
On Windows XP3 in git bash
git clone git@github.com:octocat/Spoon-Knife.git
cd Spoon-Knife
git gui
menu Remote\Fetch from\origin
error: cannot spawn git: No such file or directory
error: could not run rev-list

if u run
git fetch --all
it worked normal in git bash or gitgui tools

In second version CreateProcess get 'C:\Git\libexec\git-core/git.exe' in
first version - C:/Git/libexec/git-core/git.exe and not executes (unix
slashes)

after fixing C:\Git\libexec\git-core\git.exe or
C:/Git/libexec/git-core\git.exe it works normal

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2015-03-20 14:12:01 +01:00
마누엘
1f6151d1f2 mingw: Try to delete target directory first.
When the rename function tries to move a directory it fails if the target
directory exists. It should check if it can delete the (possibly empty)
target directory and then try again to move the directory.

Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: 마누엘 <nalla@users.noreply.github.com>
2015-03-20 14:11:54 +01:00