Quite some time ago, a last plea to the XP users out there who want to
see Windows XP support in Git for Windows, asking them to get engaged
and help, vanished into the depths of the universe.
It is time to codify the ascent by the "silent majority" of XP users,
and mark the minimum Windows version required for Git for Windows as
Windows Vista.
This, incidentally, lets us use quite a few nice new APIs.
This also means that we no longer need the inet_pton() and inet_ntop()
emulation, and we no longer need to do the PROC_ADDR dance with the
`CreateSymbolicLinkW()` function, either.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Previously, we only ever declared a target Windows version if compiling
with Visual C.
Which meant that we were relying on the MinGW headers to guess which
Windows version we want to target...
Let's be explicit about it, in particular because we actually want to
bump the target Windows version to Vista (which we will do in the next
commit).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Windows Vista (and later) actually have a working poll(), but we still
cannot use it because it only works on sockets.
So let's detect when we are targeting Windows Vista and undefine those
constants, and define `pollfd` so that we can declare our own pollfd
struct.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When spawning child processes, we really should be careful which file
handles we let them inherit.
This is doubly important on Windows, where we cannot rename, delete, or
modify files if there is still a file handle open.
Sadly, we have to guard this test inside #ifdef WIN32: we need to use
the value of the HANDLE directly, and that concept does not exist on
Linux/Unix.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The recently introduced speedup of `git checkout` sadly introduced a
regression: after writing a file, the cached stat data is obviously
incorrect, yet we still used it!
This patch simply reverts the original speedup (as opposed to try to
hotfix it as https://github.com/git-for-windows/git/pull/1443 tries to
do), and marks it as a fixup commit so that the next merging-rebase will
drop the patch.
We will most likely want to speed up this code path, though, but may
find a better way, e.g. by invalidating the cached data cleverly. (This
will be a *little* tricky, though, as readdir() uses the data, and we
must ensure that we do not pull it under readdir()'s rag when
invalidating it, and we also must avoid re-caching for every written
file, as that would be *even* slower than fscache=false.)
This fixes https://github.com/git-for-windows/git/issues/1438 and
https://github.com/git-for-windows/git/issues/1442
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Take a hint from commit ea68b0ce9f (hash-object: don't use mmap() for
small files, 2010-02-21) and use read() instead of mmap() for small
packed-refs files.
This also fixes https://github.com/git-for-windows/git/issues/1410
(where xmmap() returns NULL for zero length[1], for which munmap() later
fails).
Alternatively, we could simply check for NULL before munmap(), or
introduce xmunmap() that could be used together with xmmap(). However,
always setting snapshot->buf to a valid pointer, by relying on
xmalloc(0)'s fallback to 1-byte allocation, makes using snapshots
easier.
[1] Logic introduced in commit 9130ac1e19 (Better error messages for
corrupt databases, 2007-01-11)
This was cherry-picked from upstream's `pu` branch so that the fix is
included in Git for Windows v2.16.0.
Signed-off-by: Kim Gybels <kgybels@infogroep.be>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In git reset --hard, unpack-trees() is called with oneway_merge().
oneway_merge calls lstat for each files in a repository.
It is bottleneck of git reset --hard, especially in large repository.
This patch improves time by using fscache.
In chromium repository, time of git reset --hard is changed like below.
I took 3 times stats in the repository.
master:
TotalSeconds: 21.0337971
TotalSeconds: 20.0046612
TotalSeconds: 20.6501752
Avg: 20.5628778333333
this patch:
TotalSeconds: 4.8552376
TotalSeconds: 4.8722343
TotalSeconds: 4.9268245
Avg: 4.88476546666667
Signed-off-by: Takuto Ikuta <tikuta@chromium.org>
Especially in huge code bases with fast-moving `master`, it can be
prohibitively expensive to calculate whether an upstream branch of
a local branch is ahead, behind or diverged.
This topic branch introduces a set of flags to avoid that computation
when we're not even interested in it to begin with.
This merge commit takes the feature early, therefore it is marked
experimental.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When I do git fetch, git call file stats under .git/objects for each
refs. This takes time when there are many refs.
By enabling fscache, git takes file stats by directory traversing and that
improved the speed of fetch-pack for repository having large number of
refs.
In my windows workstation, this improves the time of `git fetch` for
chromium repository like below. I took stats 3 times.
* With this patch
TotalSeconds: 9.9825165
TotalSeconds: 9.1862075
TotalSeconds: 10.1956256
Avg: 9.78811653333333
* Without this patch
TotalSeconds: 15.8406702
TotalSeconds: 15.6248053
TotalSeconds: 15.2085938
Avg: 15.5580231
Signed-off-by: Takuto Ikuta <tikuta@chromium.org>
Teach long (normal) status format to respect the --no-ahead-behind
parameter and skip the possibly expensive ahead/behind computation
between the branch and the upstream.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Teach "git status --short --branch" to respect "--no-ahead-behind"
parameter to skip computing ahead/behind counts for the branch and
its upstream and just report '[different]'.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Once upon a time, git-clone would refuse to write into a
directory that it did not itself create. The cleanup
routines for a failed clone could therefore just remove the
git and worktree dirs completely.
In 55892d2398 (Allow cloning to an existing empty directory,
2009-01-11), we learned to write into an existing directory.
Which means that doing:
mkdir foo
git clone will-fail foo
ends up deleting foo. This isn't a huge catastrophe, since
by definition foo must be empty. But it's somewhat
confusing; we should leave the filesystem as we found it.
Because we know that the only directory we'll write into is
an empty one, we can handle this case by just passing the
KEEP_TOPLEVEL flag to our recursive delete (if we could
write into populated directories, we'd have to keep track of
what we wrote and what we did not).
Note that we need to handle the work-tree and git-dir
separately, though, as only one might exist (and the new
tests in t5600 cover all cases).
Reported-by: Stephan Janssen <sjanssen@you-get.com>
Teach "git status" and "git commit" to accept "--no-ahead-behind"
and "--ahead-behind" arguments to request quick or full ahead/behind
reporting.
When "--no-ahead-behind" is given, the existing porcelain V2 line
"branch.ab +x -y" is replaced with a new "branch.ab +? -?" line.
This indicates that the branch and its upstream are or are not equal
without the expense of computing the full ahead/behind values.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This is to speed up git checkout for directory in very large repositories.
Taking file stats while directory traversing is faster than
stating to each files on windows.
`git checkout .` in master branch of chromium repositry, having 284659 files,
takes more than 18 seconds.
This patch improved the time to around 4 seconds on my SSD laptop.
Signed-off-by: Takuto Ikuta <tikuta@chromium.org>
Two parts of git-clone's setup logic check whether a
directory exists, and they both call stat directly the same
scratch "struct stat" buffer. Let's pull that into a helper,
which has a few advantages:
- it makes the purpose of the stat call more obvious
- it makes it clear that we don't care about the
information in "buf" remaining valid
- if we later decide to make the check more robust (e.g.,
complaining about non-directories), we can do it in one
place
Note that we could just use file_exists() for this, which
has identical code. But we specifically care about
directories, so this future-proofs us against that function
later getting more picky about actual files.
Extend stat_tracking_info() to return +1 when branches are not equal and to
take a new "enum ahead_behind_flags" argument to allow skipping the (possibly
expensive) ahead/behind computation.
This will be used in the next commit to allow "git status" to avoid full
ahead/behind calculations for performance reasons.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This is an old script which could use some updating before
we add to it:
- use the standard line-breaking:
test_expect_success 'title' '
body
'
- run all code inside test_expect blocks to catch
unexpected failures in setup steps
- use "test_commit -C" instead of manually entering
sub-repo
- use test_when_finished for cleanup steps
- test_path_is_* as appropriate
Back when this test was written, git-clone could not handle
a repository without any commits. These days it works fine,
and this comment is out of date.
At first glance it seems like we could just drop this code
entirely now, but it's necessary for the final test, which
was added later. That test corrupts the repository my
temporarily removing its objects, which means we need to
have some objects to move.
Git for Windows supports the core.longPaths config setting to allow
writing/reading long paths via the \\?\ trick for a long time now.
However, for that support to work, it is absolutely necessary that
git_default_config() is given a chance to parse the config. Otherwise
Git will be non the wiser.
So let's make sure that as many commands that previously failed to
parse the core.* settings now do that, implicitly enabling long path
support in a lot more places.
Note: this is not a perfect solution, and it cannot be, as there is
a chicken-and-egg problem in reading the config itself...
This fixes https://github.com/git-for-windows/git/issues/1218
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
From the documentation of said setting:
This boolean will enable fsync() when writing object files.
This is a total waste of time and effort on a filesystem that
orders data writes properly, but can be useful for filesystems
that do not use journalling (traditional UNIX filesystems) or
that only journal metadata and not file contents (OS X’s HFS+,
or Linux ext3 with "data=writeback").
The most common file system on Windows (NTFS) does not guarantee that
order, therefore a sudden loss of power (or any other event causing an
unclean shutdown) would cause corrupt files (i.e. files filled with
NULs). Therefore we need to change the default.
Note that the documentation makes it sound as if this causes really bad
performance. In reality, writing loose objects is something that is done
only rarely, and only a handful of files at a time.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
A ton of Git commands simply do not read (or at least parse) the core.*
settings. This is not good, as Git for Windows relies on the
core.longPaths setting to be read quite early on.
So let's just make sure that all commands read the config and give
platform_core_config() a chance.
This patch teaches tons of Git commands to respect the config setting
`core.longPaths = true`, including `pack-refs`, thereby fixing
https://github.com/git-for-windows/git/issues/1218
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Fix regression described in:
https://github.com/git-for-windows/git/issues/1392
which was introduced in:
b2353379bb
Problem Symptoms
================
When the user has a .gitignore file that is a symlink, the fscache
optimization introduced above caused the stat-data from the symlink,
rather that of the target file, to be returned. Later when the ignore
file was read, the buffer length did not match the stat.st_size field
and we called die("cannot use <path> as an exclude file")
Optimization Rationale
======================
The above optimization calls lstat() before open() primarily to ask
fscache if the file exists. It gets the current stat-data as a side
effect essentially for free (since we already have it in memory).
If the file does not exist, it does not need to call open(). And
since very few directories have .gitignore files, we can greatly
reduce time spent in the filesystem.
Discussion of Fix
=================
The above optimization calls lstat() rather than stat() because the
fscache only intercepts lstat() calls. Calls to stat() stay directed
to the mingw_stat() completly bypassing fscache. Furthermore, calls
to mingw_stat() always call {open, fstat, close} so that symlinks are
properly dereferenced, which adds *additional* open/close calls on top
of what the original code in dir.c is doing.
Since the problem only manifests for symlinks, we add code to overwrite
the stat-data when the path is a symlink. This preserves the effect of
the performance gains provided by the fscache in the normal case.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Make fscache_enabled() function public rather than static.
Remove unneeded fscache_is_enabled() function.
Change is_fscache_enabled() macro to call fscache_enabled().
is_fscache_enabled() now takes a pathname so that the answer
is more precise and mean "is fscache enabled for this pathname",
since fscache only stores repo-relative paths and not absolute
paths, we can avoid attempting lookups for absolute paths.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
The problem with not having, say, git-receive-pack.exe after a full
build is that the test suite will then happily use the *installed*
git-receive-pack.exe because it finds nothing else.
Absolutely not what we want. We want to have confidence that our test
covers the MSVC-built Git executables, and not some random stuff.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The idea of the `vcxproj` target is to generate .sln/.vcxproj files and
then commit them, to be used elsewhere. Typically, this is done in a
VSTS job whenever `master` changes. So there is little use in
initializing vcpkg and building all the dependencies: they are not
necessary here.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The `vcxproj` target does not, in fact, depend on MSVC being defined, so
let's just move it outside of that block.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We just introduced a way to build Git for Windows with MSVC on the
command line using vcpkg-generated, up-to-date dependencies. Let's bring
that convenience to the Visual Studio project, too.
(The previous method, fetching NuGet packages, is fraught with problems:
as C++ libraries have to be built for every architecture and for every
toolset, the NuGet packages which we would like to consume fell behind
and are not up-to-date with the current versions of the libraries, e.g.
cURL and OpenSSL. By using vcpkg we avoid that problem, always building
the newest dependency versions.)
The trick is to initialize the VCPKG system once, and then build Git's
dependencies using it. We do that by attaching a pre-build event to the
libgit project (which is now the base project on which all others
depend, therefore no other project is built in paralleli, side-stepping
issues with vcpkg being unprepared for being run in parallel).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The .vcxproj's text nodes do not actually need to URL-encode double
quotes. So let's not do that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It really does depend on libgit. It does not hurt to let it depend on
xdiff, and it makes the code simpler.
It is necessary to get this dependency chain right, because we will
introduce a change where the vcpkg system is initialized before building
libgit. The vcpkg system will then build the dependencies needed by Git
(and thereby make the include headers available):
As the vcpkg system cannot be run in parallel (it does not lock,
wreaking havoc with files being accessed and written at the same time,
letting the vcpkg processes stumble over each others' toes. We prevent
that by ensuring that only one project is built at first: libgit. And
this project's PreBuildEvent will be used to initialize vcpkg and build
all dependencies. Subsequently, the other projects can be built in
parallel.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
As we do not consume NuGet packages any longer, there is no sense to try
to point PATH to their unpacked .dll files, either.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Dependencies such as cURL and OpenSSL are necessary to build and run
Git. Previously, we obtained those dependencies by fetching NuGet
packages.
However, it is notoriously hard to keep NuGet packages of C/C++
libraries up-to-date, as the toolsets for different Visual Studio
versions are different, and the NuGet packages would have to ship them
all.
That is the reason why the NuGet packages we use are quite old, and even
insecure in the case of cURL and OpenSSL (the versions contain known
security flaws that have been addressed by later versions for which no
NuGet packages are available).
The better way to handle this situation is to use the vcpkg system:
https://github.com/Microsoft/vcpkg
The idea is that a single Git repository contains enough supporting
files to build up-to-date versions of a large number of Open Source
libraries on demand, including cURL and OpenSSL.
We integrate this system via four new .bat files to
1) initialize the vcpkg system,
2) build the packages,
4) set up Git's Makefile system to find the build artifacts, and
3) copy the artifacts into the top-level directory
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The option is deprecated now, and we better make sure that keeps saying
so until we finally remove it.
Suggested by Kevin Willford.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We used to install into $HOME/bin/, which wreaks havoc with installed
versions of Git for Windows (because $HOME/bin is *prepended* to the
PATH, hence would override `git.exe` in Git Bash).
Let's align the MSVC case with the non-MSVC case and install into
/mingw64/bin/ (or /mingw32/bin/ in 32-bit Git for Windows SDKs) instead.
Noticed by Derrick Stolee.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Teach read_directory_recursive() and add_excludes() to
be aware of optional fscache and avoid trying to open()
and fstat() non-existant ".gitignore" files in every
directory in the worktree.
The current code in add_excludes() calls open() and then
fstat() for a ".gitignore" file in each directory present
in the worktree. Change that when fscache is enabled to
call lstat() first and if present, call open().
This seems backwards because both lstat needs to do more
work than fstat. But when fscache is enabled, fscache will
already know if the .gitignore file exists and can completely
avoid the IO calls. This works because of the lstat diversion
to mingw_lstat when fscache is enabled.
This reduced status times on a 350K file enlistment of the
Windows repo on a NVMe SSD by 0.25 seconds.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
It was a bad idea to just remove that option from Git for Windows
v2.15.0, as early users of that (still experimental) option would have
been puzzled what they are supposed to do now.
So let's reintroduce the flag, but make sure to show the user good
advice how to fix this going forward.
We'll remove this option in a more orderly fashion either in v2.16.0 or
in v2.17.0.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The script assumes that we're in the top-level directory of the
checkout. That does not need to be true.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
