We already have a directory where we store files intended for use by
multiple test scripts. The same directory is a better home for the
test-binary-*.png files than t/.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The idea of copying README and COPYING into t/diff-lib/ was to step away
from using files from outside t/ in tests. Let's really make sure that
we use the files from t/diff-lib/ instead of other versions of those
files.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It is convenient to assume that everybody who wants to build & test Git
has access to a working `iconv` executable (after all, we already pretty
much require libiconv).
However, that limits esoteric test scenarios such as Git for Windows',
where an end user installation has to ship with `iconv` for the sole
purpose of being testable. That payload serves no other purpose.
So let's just have a test helper (to be able to test Git, the test
helpers have to be available, after all) to act as `iconv` replacement.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This helper is slightly more performant than the script with MSYS2's
Bash. And a lot more readable.
To accommodate t1050, which wants to compare files weighing in with 3MB
(falling outside of t1050's malloc limit of 1.5MB), we simply lift the
allocation limit by setting the environment variable GIT_ALLOC_LIMIT to
zero when calling the helper.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It is a bit strange, and even undesirable, to require Perl just to run
the test suite even when NO_PERL was set.
This patch does not fix this problem by any stretch of imagination.
However, it fixes *the* Perl invocation that *every single* test script
has to run.
While at it, it makes the source code also more grep'able, as the code
that unsets some, but not all, GIT_* environment variables just became a
*lot* more explicit. And all that while still reducing the total number
of lines.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Instead of relying on the presence of `make`, or `prove`, we might just
as well use our own facilities to run the test suite.
This helps e.g. when trying to verify a Git for Windows installation
without requiring to download a full Git for Windows SDK (which would use
up 600+ megabytes of bandwidth, and over a gigabyte of disk space).
Of course, it still requires the test helpers to be build *somewhere*,
and the Git version should at least roughly match the version from which
the test suite comes.
At the same time, this new way to run the test suite allows to validate
that a BusyBox-backed MinGit works as expected (verifying that BusyBox'
functionality is enough to at least pass the test suite).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This topic branch conflicts with the next change where we teach the
`test-tool run-command` to run the test suite, so let's merge it to
avoid those merge conflicts.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
BusyBox comes with a ton of applets ("applet" being the identical
concept to Git's "builtins"). And similar to Git's builtins, the applets
can be called via `busybox <command>`, or the BusyBox executable can be
copied/hard-linked to the command name.
The similarities do not end here. Just as with Git's builtins, it is
problematic that BusyBox' hard-linked applets cannot easily be put into
a .zip file: .zip archives have no concept of hard-links and therefore
would store identical copies (and also extract identical copies,
"inflating" the archive unnecessarily).
To counteract that issue, MinGit already ships without hard-linked
copies of the builtins, and the plan is to do the same with BusyBox'
applets: simply ship busybox.exe as single executable, without
hard-linked applets.
To accommodate that, Git is being taught by this commit a very special
trick, exploiting the fact that it is possible to call an executable
with a command-line whose argv[0] is different from the executable's
name: when `sh` is to be spawned, and no `sh` is found in the PATH, but
busybox.exe is, use that executable (with unchanged argv).
Likewise, if any executable to be spawned is not on the PATH, but
busybox.exe is found, parse the output of `busybox.exe --help` to find
out what applets are included, and if the command matches an included
applet name, use busybox.exe to execute it.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We need to merge it early, to avoid a conflict where we add another
`#include` to `compat/mingw.c`.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
By default, CreateProcess() does not inherit any open file handles,
unless the bInheritHandles parameter is set to TRUE. Which we do need to
set because we need to pass in stdin/stdout/stderr to talk to the child
processes. Sadly, this means that all file handles (unless marked via
O_NOINHERIT) are inherited.
This lead to problems in GVFS Git, where a long-running read-object hook
is used to hydrate missing objects, and depending on the circumstances,
might only be called *after* Git opened a file handle.
Ideally, we would not open files without O_NOINHERIT unless *really*
necessary (i.e. when we want to pass the opened file handle as standard
handle into a child process), but apparently it is all-too-easy to
introduce incorrect open() calls: this happened, and prevented updating
a file after the read-object hook was started because the hook still
held a handle on said file.
Happily, there is a solution: as described in the "Old New Thing"
https://blogs.msdn.microsoft.com/oldnewthing/20111216-00/?p=8873 there
is a way, starting with Windows Vista, that lets us define precisely
which handles should be inherited by the child process.
And since we bumped the minimum Windows version for use with Git for
Windows to Vista with v2.10.1 (i.e. a *long* time ago), we can use this
method. So let's do exactly that.
We need to make sure that the list of handles to inherit does not
contain duplicates; Otherwise CreateProcessW() would fail with
ERROR_INVALID_ARGUMENT.
While at it, stop setting errno to ENOENT unless it really is the
correct value.
Also, fall back to not limiting handle inheritance under certain error
conditions (e.g. on Windows 7, which is a lot stricter in what handles
you can specify to limit to).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The main idea of this patch is that even if we have to look up the
absolute path of the script, if only the basename was specified as
argv[0], then we should use that basename on the command line, too, not
the absolute path.
This patch will also help with the upcoming patch where we automatically
substitute "sh ..." by "busybox sh ..." if "sh" is not in the PATH but
"busybox" is: we will do that by substituting the actual executable, but
still keep prepending "sh" to the command line.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This topic branch conflicts with the next change that will change the
way we call `CreateProcessW()`. So let's merge it early, to avoid merge
conflicts during a merge (because we would have to resolve this with
every single merging-rebase).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This helps with minimal installations such as MinGit that refuse to
waste .zip real estate by shipping identical copies of builtins (.zip
files do not support hard links).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The CreateProcessW() function does not really support spaces in its
first argument, lpApplicationName. But it supports passing NULL as
lpApplicationName, which makes it figure out the application from the
(possibly quoted) first argument of lpCommandLine.
Let's use that trick (if we are certain that the first argument matches
the executable's path) to support launching programs whose path contains
spaces.
This fixes https://github.com/git-for-windows/git/issue/692
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Windows paths are typically limited to MAX_PATH = 260 characters, even
though the underlying NTFS file system supports paths up to 32,767 chars.
This limitation is also evident in Windows Explorer, cmd.exe and many
other applications (including IDEs).
Particularly annoying is that most Windows APIs return bogus error codes
if a relative path only barely exceeds MAX_PATH in conjunction with the
current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the
infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG.
Many Windows wide char APIs support longer than MAX_PATH paths through the
file namespace prefix ('\\?\' or '\\?\UNC\') followed by an absolute path.
Notable exceptions include functions dealing with executables and the
current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as
well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...).
Introduce a handle_long_path function to check the length of a specified
path properly (and fail with ENAMETOOLONG), and to optionally expand long
paths using the '\\?\' file namespace prefix. Short paths will not be
modified, so we don't need to worry about device names (NUL, CON, AUX).
Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be
limited to MAX_PATH (at least not on Win7), so we can use it to do the
heavy lifting of the conversion (translate '/' to '\', eliminate '.' and
'..', and make an absolute path).
Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH
limit.
Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs
that support long paths.
While improved error checking is always active, long paths support must be
explicitly enabled via 'core.longpaths' option. This is to prevent end
users to shoot themselves in the foot by checking out files that Windows
Explorer, cmd/bash or their favorite IDE cannot handle.
Test suite:
Test the case is when the full pathname length of a dir is close
to 260 (MAX_PATH).
Bug report and an original reproducer by Andrey Rogozhnikov:
https://github.com/msysgit/git/pull/122#issuecomment-43604199
Note that the test cannot rely on the presence of short names, as they
are not enabled by default except on the system drive.
[jes: adjusted test number to avoid conflicts, reinstated && chain,
adjusted test to work without short names]
Thanks-to: Martin W. Kirst <maki@bitkings.de>
Thanks-to: Doug Kelly <dougk.ff7@gmail.com>
Signed-off-by: Karsten Blees <blees@dcon.de>
Original-test-by: Andrey Rogozhnikov <rogozhnikov.andrey@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Windows paths are typically limited to MAX_PATH = 260 characters, even
though the underlying NTFS file system supports paths up to 32,767 chars.
This limitation is also evident in Windows Explorer, cmd.exe and many
other applications (including IDEs).
Particularly annoying is that most Windows APIs return bogus error codes
if a relative path only barely exceeds MAX_PATH in conjunction with the
current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the
infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG.
Many Windows wide char APIs support longer than MAX_PATH paths through the
file namespace prefix ('\\?\' or '\\?\UNC\') followed by an absolute path.
Notable exceptions include functions dealing with executables and the
current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as
well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...).
Introduce a handle_long_path function to check the length of a specified
path properly (and fail with ENAMETOOLONG), and to optionally expand long
paths using the '\\?\' file namespace prefix. Short paths will not be
modified, so we don't need to worry about device names (NUL, CON, AUX).
Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be
limited to MAX_PATH (at least not on Win7), so we can use it to do the
heavy lifting of the conversion (translate '/' to '\', eliminate '.' and
'..', and make an absolute path).
Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH
limit.
Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs
that support long paths.
While improved error checking is always active, long paths support must be
explicitly enabled via 'core.longpaths' option. This is to prevent end
users to shoot themselves in the foot by checking out files that Windows
Explorer, cmd/bash or their favorite IDE cannot handle.
Test suite:
Test the case is when the full pathname length of a dir is close
to 260 (MAX_PATH).
Bug report and an original reproducer by Andrey Rogozhnikov:
https://github.com/msysgit/git/pull/122#issuecomment-43604199
[jes: adjusted test number to avoid conflicts]
Thanks-to: Martin W. Kirst <maki@bitkings.de>
Thanks-to: Doug Kelly <dougk.ff7@gmail.com>
Signed-off-by: Karsten Blees <blees@dcon.de>
Original-test-by: Andrey Rogozhnikov <rogozhnikov.andrey@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
[jes: adjusted test number to avoid conflicts, fixed non-portable use of
the 'export' statement, fixed broken && chain]
Signed-off-by: Doug Kelly <dougk.ff7@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
If multiple threads access a directory that is not yet in the cache, the
directory will be loaded by each thread. Only one of the results is added
to the cache, all others are leaked. This wastes performance and memory.
On cache miss, add a future object to the cache to indicate that the
directory is currently being loaded. Subsequent threads register themselves
with the future object and wait. When the first thread has loaded the
directory, it replaces the future object with the result and notifies
waiting threads.
Signed-off-by: Karsten Blees <blees@dcon.de>
Checking the work tree status is quite slow on Windows, due to slow lstat
emulation (git calls lstat once for each file in the index). Windows
operating system APIs seem to be much better at scanning the status
of entire directories than checking single files.
Add an lstat implementation that uses a cache for lstat data. Cache misses
read the entire parent directory and add it to the cache. Subsequent lstat
calls for the same directory are served directly from the cache.
Also implement opendir / readdir / closedir so that they create and use
directory listings in the cache.
The cache doesn't track file system changes and doesn't plug into any
modifying file APIs, so it has to be explicitly enabled for git functions
that don't modify the working copy.
Note: in an earlier version of this patch, the cache was always active and
tracked file system changes via ReadDirectoryChangesW. However, this was
much more complex and had negative impact on the performance of modifying
git commands such as 'git checkout'.
Signed-off-by: Karsten Blees <blees@dcon.de>
Add a macro to mark code sections that only read from the file system,
along with a config option and documentation.
This facilitates implementation of relatively simple file system level
caches without the need to synchronize with the file system.
Enable read-only sections for 'git status' and preload_index.
Signed-off-by: Karsten Blees <blees@dcon.de>
Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite
slow. Windows operating system APIs seem to be much better at scanning the
status of entire directories than checking single files. A caching
implementation may improve performance by bulk-reading entire directories
or reusing data obtained via opendir / readdir.
Make the lstat implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.
Signed-off-by: Karsten Blees <blees@dcon.de>
Emulating the POSIX dirent API on Windows via FindFirstFile/FindNextFile is
pretty staightforward, however, most of the information provided in the
WIN32_FIND_DATA structure is thrown away in the process. A more
sophisticated implementation may cache this data, e.g. for later reuse in
calls to lstat.
Make the dirent implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.
Define a base DIR structure with pointers to readdir/closedir that match
the opendir implementation (i.e. similar to vtable pointers in OOP).
Define readdir/closedir so that they call the function pointers in the DIR
structure. This allows to choose the opendir implementation on a
call-by-call basis.
Move the fixed sized dirent.d_name buffer to the dirent-specific DIR
structure, as d_name may be implementation specific (e.g. a caching
implementation may just set d_name to point into the cache instead of
copying the entire file name string).
Signed-off-by: Karsten Blees <blees@dcon.de>
In December 2016 and January 2017, we revamped the Windows-specific
`isatty()` handling, replacing a hack by a more robust solution.
This patch is a follow-up we realized was necessary already in March
2017, but forgot to contribute to core Git yet.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When compiling Git with a runtime prefix (so that it can be installed
into any location, finding its libexec/ directory relative to the
location of the `git` executable), it is convenient to provide
"absolute" Unix-y paths e.g. for http.sslCAInfo, and have those absolute
paths be resolved relative to the runtime prefix.
This patch makes it so for Windows. It is up for discussion whether we
want this for other platforms, too, as long as building with
RUNTIME_PREFIX.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
On Windows, it is more common than on other platforms to "cross-compile"
32-bit binaries on a 64-bit machine. It is even possible to compile
32-bit binaries and 64-bit binaries in the same worktree, as the CFLAGS
differ, triggering a rebuild.
Except for the resource file, which does not need the CFLAGS. Let's work
around that.
This patch has been carried in Git for Windows for well over a year now,
and while it is not interesting to non-Windows platforms, it is time to
graduate to core Git.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This patch has been carried by Git for Windows for almost two years. It
makes loading system libraries safer.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This patch series contains a couple of fixes revolving around testing
an installed Git, via GIT_TEST_INSTALLED=/path/to/git.
The original motivation for these patches is that Git for Windows wants
to provide a version where the Unix shell scripts are interpreted by
BusyBox (to reduce the footprint on disk, mainly), and we want to verify
that this actually works, and is not perchance missing any Unix shell
tool that is present in the Git for Windows SDK but is missing from the
installed set of files.
While the BusyBox-based Git for Windows is not ready for prime time,
this here patch series is, and might be useful for packagers who want to
verify a similar scenario.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We newly handle isatty() by special-casing the stdin/stdout/stderr file
descriptors, caching the return value. However, we missed the case where
dup2() overrides the respective file descriptor.
That poses a problem e.g. where the `show` builtin asks for a pager very
early, the `setup_pager()` function sets the pager depending on the
return value of `isatty()` and then redirects stdout. Subsequently,
`cmd_log_init_finish()` calls `setup_pager()` *again*. What should
happen now is that `isatty()` reports that stdout is *not* a TTY and
consequently stdout should be left alone.
Let's override dup2() to handle this appropriately.
This fixes https://github.com/git-for-windows/git/issues/1077
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When git.rc is compiled into git.res, the result is actually dependent
on the architecture. That is, you cannot simply link a 32-bit git.res
into a 64-bit git.exe.
Therefore, to allow 32-bit and 64-bit builds in the same directory, we
let git.res depend on GIT-PREFIX so that it gets recompiled when
compiling for a different architecture (this works because the exec path
changes based on the architecture: /mingw32/libexec/git-core for 32-bit
and /mingw64/libexec/git-core for 64-bit).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When we access IPv6-related functions, we load the corresponding system
library using the `LoadLibrary()` function, which is not the recommended
way to load system libraries.
In practice, it does not make a difference: the `ws2_32.dll` library
containing the IPv6 functions is already loaded into memory, so
LoadLibrary() simply reuses the already-loaded library.
Still, recommended way is recommended way, so let's use that instead.
While at it, also adjust the code in contrib/ that loads system libraries.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In the bin-wrappers/* scripts, we already take pains to use `git.exe`
rather than `git`, as this could pick up the wrong thing on Windows
(i.e. if there exists a `git` file or directory in the build directory).
Now we do the same in the tests' start-up code.
This also helps when testing an installed Git, as there might be even
more likely some stray file or directory in the way.
Note: the only way we can record whether the `.exe` suffix is by writing
it to the `GIT-BUILD-OPTIONS` file and sourcing it at the beginning of
`t/test-lib.sh`. This is not a requirement introduced by this patch, but
we move the call to be able to use the `$X` variable that holds the file
extension, if any.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We really only need the test helpers in that case, but that is not what
we test for. So let's skip the test for now when we know that we want to
test an installed Git.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It makes very, very little sense to test the built git-sh-i18n when the
user asked specifically to test another one.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It really makes very, very little sense to use a different git
executable than the one the caller indicated via setting the environment
variable GIT_TEST_INSTALLED.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We really need to be able to find the test helpers... Really. This
change was forgotten when we moved the test helpers into t/helper/
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
For some reason, when being called via TortoiseGit the standard handles,
or at least what is returned by _get_osfhandle(0) for standard input,
can take on the value (HANDLE)-2 (which is not a legal value, according
to the documentation).
Even if this value is not documented anywhere, CreateProcess() seems to
work fine without complaints if hStdInput set to this value.
In contrast, the upcoming code to restrict which file handles get
inherited by spawned processes would result in `ERROR_INVALID_PARAMETER`
when including such handle values in the list.
To help this, special-case the value (HANDLE)-2 returned by
_get_osfhandle() and replace it with INVALID_HANDLE_VALUE, which will
hopefully let the handle inheritance restriction work even when called
from TortoiseGit.
This fixes https://github.com/git-for-windows/git/issues/1481
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When spawning child processes, we really should be careful which file
handles we let them inherit.
This is doubly important on Windows, where we cannot rename, delete, or
modify files if there is still a file handle open.
Sadly, we have to guard this test inside #ifdef WIN32: we need to use
the value of the HANDLE directly, and that concept does not exist on
Linux/Unix.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This topic branch teaches Git to accept UNC paths of the form
file://host/share/repository.git.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This topic branch brings support for choosing cURL's SSL backend at
runtime via http.sslBackend, based on patches already submitted to the
cURL project and backported to cURL 7.54.1 as used in Git for Windows'
SDK.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This backports f31d23a399 (Merge branch 'bw/config-h', 2017-06-24) from
upstream Git's `master` branch, plus a patch that seems to be required
to let the test suite pass.
This topic branch fixes problems when looking up aliases in worktrees.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
