If multiple threads access a directory that is not yet in the cache, the
directory will be loaded by each thread. Only one of the results is added
to the cache, all others are leaked. This wastes performance and memory.
On cache miss, add a future object to the cache to indicate that the
directory is currently being loaded. Subsequent threads register themselves
with the future object and wait. When the first thread has loaded the
directory, it replaces the future object with the result and notifies
waiting threads.
Signed-off-by: Karsten Blees <blees@dcon.de>
Checking the work tree status is quite slow on Windows, due to slow lstat
emulation (git calls lstat once for each file in the index). Windows
operating system APIs seem to be much better at scanning the status
of entire directories than checking single files.
Add an lstat implementation that uses a cache for lstat data. Cache misses
read the entire parent directory and add it to the cache. Subsequent lstat
calls for the same directory are served directly from the cache.
Also implement opendir / readdir / closedir so that they create and use
directory listings in the cache.
The cache doesn't track file system changes and doesn't plug into any
modifying file APIs, so it has to be explicitly enabled for git functions
that don't modify the working copy.
Note: in an earlier version of this patch, the cache was always active and
tracked file system changes via ReadDirectoryChangesW. However, this was
much more complex and had negative impact on the performance of modifying
git commands such as 'git checkout'.
Signed-off-by: Karsten Blees <blees@dcon.de>
Add a macro to mark code sections that only read from the file system,
along with a config option and documentation.
This facilitates implementation of relatively simple file system level
caches without the need to synchronize with the file system.
Enable read-only sections for 'git status' and preload_index.
Signed-off-by: Karsten Blees <blees@dcon.de>
Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite
slow. Windows operating system APIs seem to be much better at scanning the
status of entire directories than checking single files. A caching
implementation may improve performance by bulk-reading entire directories
or reusing data obtained via opendir / readdir.
Make the lstat implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.
Signed-off-by: Karsten Blees <blees@dcon.de>
Emulating the POSIX dirent API on Windows via FindFirstFile/FindNextFile is
pretty staightforward, however, most of the information provided in the
WIN32_FIND_DATA structure is thrown away in the process. A more
sophisticated implementation may cache this data, e.g. for later reuse in
calls to lstat.
Make the dirent implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.
Define a base DIR structure with pointers to readdir/closedir that match
the opendir implementation (i.e. similar to vtable pointers in OOP).
Define readdir/closedir so that they call the function pointers in the DIR
structure. This allows to choose the opendir implementation on a
call-by-call basis.
Move the fixed sized dirent.d_name buffer to the dirent-specific DIR
structure, as d_name may be implementation specific (e.g. a caching
implementation may just set d_name to point into the cache instead of
copying the entire file name string).
Signed-off-by: Karsten Blees <blees@dcon.de>
Git for Windows ships with its own Perl interpreter, and insists on
using it, so it will most likely wreak havoc if PERL5LIB is set before
launching Git.
Let's just unset that environment variables when spawning processes.
To make this feature extensible (and overrideable), there is a new
config setting `core.unsetenvvars` that allows specifying a
comma-separated list of names to unset before spawning processes.
Reported by Gabriel Fuhrmann.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In the Git for Windows project, we have ample precendent for config
settings that apply to Windows, and to Windows only.
Let's formalize this concept by introducing a platform_core_config()
function that can be #define'd in a platform-specific manner.
This will allow us to contain platform-specific code better, as the
corresponding variables no longer need to be exported so that they can
be defined in environment.c and be set in config.c
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The changes in upstream/maint are actually not really that crucial for Git
for Windows, but we can use it as an excuse to fix the branch thicket: the
`visual-studio` branch clearly wants to be based on the
`jeffhostetler/vs2015` branch, as the latter contains important compile
fixes for Visual C, required to build inside Visual Studio.
While at it, we also move the Vagrant patches farther to the end, as they
are unlikely to be accepted upstream, so we'll try only after getting the
MSVC/VS patches in.
Also, let's reorder a couple other patches into more logical places.
This commit starts the rebase of 4638a925c0 to a274e0a036
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When Git's source code calls isatty(), it really asks whether the
respective file descriptor is connected to an interactive terminal.
Windows' _isatty() function, however, determines whether the file
descriptor is associated with a character device. And NUL, Windows'
equivalent of /dev/null, is a character device.
Which means that for years, Git mistakenly detected an associated
interactive terminal when being run through the test suite, which
almost always redirects stdin, stdout and stderr to /dev/null.
This bug only became obvious, and painfully so, when the new
bisect--helper entered the `pu` branch and made the automatic build & test
time out because t6030 was waiting for an answer.
For details, see
https://msdn.microsoft.com/en-us/library/f4s0ddew.aspx
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This merges the "Pre-release to test fixes for #981 and #987".
These commits were actually meant to land in `master` before merging the
Pull Requests; Better late than never.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Use ALLOC_GROW() macro when reallocing a string_list array
rather than simply increasing it by 32. This is a performance
optimization.
During status on a very large repo and there are many changes,
a significant percentage of the total run time was spent
reallocing the wt_status.changes array.
This change decreased the time in wt_status_collect_changes_worktree()
from 125 seconds to 45 seconds on my very large repository.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
In particular when local tags are used (or tags that are pushed to some
fork) to build Git, it is very hard to figure out from which particular
revision a particular Git executable was built.
Let's just report that in our build options.
We need to be careful, though, to report when the current commit cannot be
determined, e.g. when building from a tarball without any associated Git
repository.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It is common in corporate setups to have permissions managed via a
domain account. That means that the user does not really have to log in
when accessing a central repository via https://, but that the login
credentials are used to authenticate with that repository.
The common way to do that used to require empty credentials, i.e. hitting
Enter twice when being asked for user name and password, or by using the
very funny notation https://:@server/repository
A recent commit (5275c3081c (http: http.emptyauth should allow empty (not
just NULL) usernames, 2016-10-04)) broke that usage, though, all of a
sudden requiring users to set http.emptyAuth = true.
Which brings us to the bigger question why http.emptyAuth defaults to
false, to begin with.
It would be one thing if cURL would not let the user specify credentials
interactively after attempting NTLM authentication (i.e. login
credentials), but that is not the case.
It would be another thing if attempting NTLM authentication was not
usually what users need to do when trying to authenticate via https://.
But that is also not the case.
So let's just go ahead and change the default, and unbreak the NTLM
authentication. As a bonus, this also makes the "you need to hit Enter
twice" (which is hard to explain: why enter empty credentials when you
want to authenticate with your login credentials?) and the ":@" hack
(which is also pretty, pretty hard to explain to users) obsolete.
This fixes https://github.com/git-for-windows/git/issues/987
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Teach add_index_entry_with_check() and has_dir_name()
to see if the path of the new item is greater than the
last path in the index array before attempting to search
for it.
This is a performance optimization.
During checkout, merge_working_tree() populates the new
index in sorted order, so this change saves at least 2
lookups per file.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
This is a performance optimization.
Teach do_read_index() to call verify_hdr() using a thread
and allow SHA1 verification to run concurrently with the
parsing of index-entries and extensions.
For large index files, this cuts the startup time in half.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
For regular debugging, it is pretty helpful when a debug assertion in a
running application triggers a window that offers to start the debugger.
However, when running the test suite, it is not so helpful, in
particular when the debug assertions are then suppressed anyway because
we disable the invalid parameter checking (via invalidcontinue.obj, see
the comment in config.mak.uname about that object for more information).
So let's simply disable that window in Debug Mode (it is already
disabled in Release Mode).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This topic branch teaches the project generator to generate a Visual
Studio solution, ready to be opened in Visual Studio 2010 or later.
The idea, of course, is to let some automatic build job generate and
commit the project files with
make MSVC=1 vcxproj
and then (force-)push to a special-purpose branch.
The major part of this branch thicket concerns itself not only with
generating the Visual Studio project files, but making sure that the
user can then run the test suite from a regular Git Bash (i.e. *not*
requiring a Git for Windows SDK), e.g. by running
cd t
prove --timer --jobs 15 ./t[0-9]*.sh
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This branch fixes the problem where merge-recursive's add_cacheinfo()
expected refresh_cache_entry() always to return a valid cache entry (but
it does not do that e.g. when the file in the worktree is modified).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This topic branch brings the new, experimental builtin version of the
difftool into Git for Windows' master branch.
It still hands off to the legacy Perl script unless the feature flag is
flipped: only when the config setting difftool.useBuiltin is set to true
will `git difftool` actually use the experimental builtin. The idea is to
play it safe for the majority of users, but to allow heavy difftool users
to test early and to help make the builtin robust, before we actually
retire the Perl script.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The entire idea of generating the VS solution makes only sense if we
generate it via Continuous Integration; otherwise potential users would
still have to download the entire Git for Windows SDK.
So let's just add a target in the Makefile that can be used to generate
said solution; The generated files will then be committed so that they
can be pushed to a branch ready to check out by Visual Studio users.
To make things even more useful, we also generate and commit other files
that are required to run the test suite, such as templates and
bin-wrappers: with this, developers can run the test suite in a regular
Git Bash (that is part of a regular Git for Windows installation) after
building the solution in Visual Studio.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When building with Microsoft Visual C, we use NuGet to acquire the
dependencies (such as OpenSSL, cURL, etc). We even unpack those
dependencies.
This patch teaches the test suite to add the directory with the unpacked
.dll files to the PATH before running the tests.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We already unpack the NuGet packages in a certain place, via
compat/vcbuild/Makefile. Let's let Visual Studio use the very same place.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This file is not available in earlier MSVC versions, and it is not
necessary to include it with MSVC, anyway.
Signed-off-by: Philip Oakley <philipoakley@iee.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Microsoft flipped the Windows Safe Exception Handling default
in VS2013 so that zlib became unacceptable to certain OS versions
(Vista and subsequent 32-bit OS's) without the addition of
the option -SAFESEH:NO.
Provide a switch to disable the Safe Exception Handler when required.
The option ImageHasSafeExceptionHandlers for VS2013 is not available in
earlier versions, so use the SAFESEH:NO linker flag. See
https://msdn.microsoft.com/en-us/library/9a89h429.aspx for
further details.
This has only had limited testing due to the lack of a suitable system.
Helped-by: Yue Lin Ho <b8732003@student.nsysu.edu.tw>
Signed-off-by: Philip Oakley <philipoakley@iee.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Needed for: test-config; t-dump-split-index; t-dump-untracked-cache;
t-fake-ssh; t-sha1-array; t-submodule-config.
Plus a few spares.
Signed-off-by: Philip Oakley <philipoakley@iee.org>
Visual Studio takes the first listed application/library as the default
startup project [1].
Detect the 'git' project and place it the head of the apps list, rather
than the tail.
Export the apps list before libs list for both the projects and global
structures of the .sln file.
[1] http://stackoverflow.com/questions/1238553/
vs2008-where-is-the-startup-project-setting-stored-for-a-solution
"In the solution file, there are a list of pseudo-XML "Project"
entries. It turns out that whatever is the first one ends up as
the Startup Project, unless it’s overridden in the suo file. Argh.
I just rearranged the order in the file and it’s good."
"just moving the pseudo-xml isn't enough. You also have to move the
group of entries in the "GlobalSection(ProjectConfigurationPlatforms)
= postSolution" group that has the GUID of the project you moved to
the top. So there are two places to move lines."
Signed-off-by: Philip Oakley <philipoakley@iee.org>
Delete the duplicated GUID from the generation code for the Visual Studio
.sln project file.
The duplicate GUID tended to be allocated to test-svn-fe, which was then
ignored by Visual Studio / MSVC, and its omission from the build never
noticed.
Signed-off-by: Philip Oakley <philipoakley@iee.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Visual Studio has this very neat feature that you can get dependencies in
the form of NuGet packages, and even further: you can specify in a project
what NuGet packages it needs. These dependencies can then be fetched via
right-clicking the solution in the Solution Explorer and clicking the
"Restore NuGet Packages" entry.
This feature is so neat, in fact, that we want to support it in Git for
Windows. The idea is that we will be able to provide developers with a
checkout of the Git sources that can be built outside of the Git for
Windows SDK, using *only* Visual Studio.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The .sln/.vcproj files were used to define projects up until Visual
Studio 2008, but starting with Visual Studio 2010 the project
definitions are stored in .sln/.vcxproj files (which can also be used
by the MSBuild system).
Let's just copy-edit the generator of the .vcproj files to a new
generator that produces .vcxproj files directly, without forcing Visual
Studio to upgrade the project definitions.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
With the recent changes to allow building with MSVC=1, we now pass the
/OPT:REF option to the compiler. This confuses the parser that wants to
turn the output of a dry run into project definitions for QMake and Visual
Studio:
Unhandled link option @ line 213: /OPT:REF at [...]
Let's just extend the code that passes through options that start with a
dash, so that it passes through options that start with a slash, too.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Add an option for capturing the output of the make dry-run used in
determining the msvc-build structure for easy debugging.
You can use the output of `--make-out <path>` in subsequent runs via the
`--in <path>` option.
Signed-off-by: Philip Oakley <philipoakley@iee.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Git's test suite shows tons of breakages unless Git is compiled
*without* NO_ICONV. That means, in turn, that we need to generate
build definitions *with* libiconv, which in turn implies that we
have to handle the -liconv option properly.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
