Commit Graph

90032 Commits

Author SHA1 Message Date
Johannes Schindelin
9ef2ea46ac ci: rename the library of common functions
The name is hard-coded to reflect that we use Travis CI for continuous
testing.

In the next commits, we will extend this to be able use Azure DevOps,
too.

So let's adjust the name to make it more generic.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 18:25:12 +01:00
Johannes Schindelin
8c1b7d6abd tests: optionally skip bin-wrappers/
This speeds up the tests by a bit on Windows, where running Unix shell
scripts (and spawning processes) is not exactly a cheap operation.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 18:25:08 +01:00
Johannes Schindelin
a0e524d987 t0061: fix with --with-dashes and RUNTIME_PREFIX
When building Git with RUNTIME_PREFIX and starting a test helper from
t/helper/, it fails to detect the system prefix correctly.

This is the reason that the warning

	RUNTIME_PREFIX requested, but prefix computation failed. [...]

to be printed.

In t0061, we did not expect that to happen, and it actually did not
happen in the normal case, because bin-wrappers/test-tool specifically
sets GIT_TEXTDOMAINDIR (and as a consequence, nothing in test-tool wants
to know about the runtime prefix).

However, with --with-dashes, bin-wrappers/test-tool is no longer called,
but t/helper/test-tool is called directly.

So let's just ignore the RUNTIME_PREFIX warning.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 18:24:30 +01:00
Johannes Schindelin
afa65bfa01 tests: add t/helper/ to the PATH with --with-dashes
We really need to be able to find the test helpers... Really. This
change was forgotten when we moved the test helpers into t/helper/

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 18:24:26 +01:00
Johannes Schindelin
1085d510a6 Start the merging-rebase to 0d438c7f4e
This commit starts the rebase of 0d438c7f4e to 0d438c7f4e

The idea is that this is the last merging-rebase before rebasing onto
v2.20.0-rc0. Read: no code changes, just rearranging the commits into
a nice, logical commit topology.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 14:31:08 +01:00
Johannes Schindelin
26d0946668 Merge branch 'update-ready-for-upstream'
This closes https://github.com/git-for-windows/git/pull/1936

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:46:06 +01:00
Johannes Schindelin
cb5118b5e6 Merge 'test-git-installed-v2'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:41:01 +01:00
Johannes Schindelin
88837b692b Merge 'mingw-modernize-pthread_cond_t-v1'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:40:36 +01:00
Johannes Schindelin
dade23fdbd Merge 'http-ssl-backend-v2'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:40:29 +01:00
Johannes Schindelin
5113fc3927 Merge 'bw/config-h-v2'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:40:22 +01:00
Johannes Schindelin
b094daa875 squash! tests: explicitly use git.exe on Windows
tests: explicitly use `git.exe` on Windows

On Windows, when we refer to `/an/absolute/path/to/git`, it magically
resolves `git.exe` at that location. Except if something of the name
`git` exists next to that `git.exe`. So if we call `$BUILD_DIR/git`, it
will find `$BUILD_DIR/git.exe` *only* if there is not, say, a directory
called `$BUILD_DIR/git`.

Such a directory, however, exists in Git for Windows when building with
Visual Studio (our Visual Studio project generator defaults to putting
the build files into a directory whose name is the base name of the
corresponding `.exe`).

In the bin-wrappers/* scripts, we already take pains to use `git.exe`
rather than `git`, as this could pick up the wrong thing on Windows
(i.e. if there exists a `git` file or directory in the build directory).

Now we do the same in the tests' start-up code.

This also helps when testing an installed Git, as there might be even
more likely some stray file or directory in the way.

Note: the only way we can record whether the `.exe` suffix is by writing
it to the `GIT-BUILD-OPTIONS` file and sourcing it at the beginning of
`t/test-lib.sh`. This is not a requirement introduced by this patch, but
we move the call to be able to use the `$X` variable that holds the file
extension, if any.

Note also: the many, many calls to `git this` and `git that` are
unaffected, as the regular PATH search will find the `.exe` files on
Windows (and not be confused by a directory of the name `git` that is
in one of the directories listed in the `PATH` variable), while
`/path/to/git` would not, per se, know that it is looking for an
executable and happily prefer such a directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:38:18 +01:00
Johannes Schindelin
ce7b0f2e14 squash! tests: do not require Git to be built when testing an installed Git
tests: do not require Git to be built when testing an installed Git

We really only need the test helpers to be built in the worktree in that
case, but that is not what we test for.

On the other hand it is a perfect opportunity to verify that
`GIT_TEST_INSTALLED` points to a working Git.

So let's test the appropriate Git executable. While at it, also adjust
the error message in the `GIT_TEST_INSTALLED` case.

This patch is best viewed with `-w --patience`.

Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:38:15 +01:00
Johannes Schindelin
fe37aea812 squash! t/lib-gettext: test installed git-sh-i18n if GIT_TEST_INSTALLED is set
t/lib-gettext: test installed git-sh-i18n if GIT_TEST_INSTALLED is set

It makes very, very little sense to test the built git-sh-i18n when the
user asked specifically to test another one.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:38:12 +01:00
Johannes Schindelin
faa7cccbc9 squash! tests: respect GIT_TEST_INSTALLED when initializing repositories
tests: respect GIT_TEST_INSTALLED when initializing repositories

It really makes very, very little sense to use a different git
executable than the one the caller indicated via setting the environment
variable GIT_TEST_INSTALLED.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:38:09 +01:00
Johannes Schindelin
7801f037af squash! tests: fix GIT_TEST_INSTALLED's PATH to include t/helper/
tests: fix GIT_TEST_INSTALLED's PATH to include t/helper/

We really need to be able to find the test helpers... Really. This
change was forgotten when we moved the test helpers into t/helper/

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:38:05 +01:00
Johannes Schindelin
d64d6debd2 fixup! tests: fix GIT_TEST_INSTALLED's PATH to include t/helper/
In preparation for applying the latest iteration that was submitted to
the Git mailing list, let's revert this here patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:35:45 +01:00
Johannes Schindelin
69d262d0ec fixup! tests: respect GIT_TEST_INSTALLED when initializing repositories
In preparation for applying the latest iteration that was submitted to
the Git mailing list, let's revert this here patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:35:42 +01:00
Johannes Schindelin
c1f55c4fcd fixup! t/lib-gettext: test installed git-sh-i18n if GIT_TEST_INSTALLED is set
In preparation for applying the latest iteration that was submitted to
the Git mailing list, let's revert this here patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:35:39 +01:00
Johannes Schindelin
1f438ff32f fixup! tests: do not require Git to be built when testing an installed Git
In preparation for applying the latest iteration that was submitted to
the Git mailing list, let's revert this here patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:35:33 +01:00
Johannes Schindelin
2fb1d8b7cd fixup! tests: explicitly use git.exe on Windows
In preparation for applying the latest iteration that was submitted to
the Git mailing list, let's revert this here patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:35:05 +01:00
Loo Rong Jie
a0505d0a89 squash! Implement pthread_cond_t with Win32 CONDITION_VARIABLE
win32: replace pthread_cond_*() with much simpler code

The Win32 CONDITION_VARIABLE has better performance and is easier to
maintain, as the code is a lot shorter now (the semantics of the
CONDITION_VARIABLE matches the pthread_cond_t very well).

Note: CONDITION_VARIABLE is not available in Windows XP and below,
but the declared minimal Windows version required to build and run
Git for Windows is Windows Vista (which is also beyond its
end-of-life, but for less long than Windows XP), so that's okay.

Signed-off-by: Loo Rong Jie <loorongjie@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:31:31 +01:00
Johannes Schindelin
f7ca946ce4 fixup! Implement pthread_cond_t with Win32 CONDITION_VARIABLE
In preparation for applying the patch that actually made it to the Git
mailing list, let's revert this.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:28:49 +01:00
Johannes Schindelin
dba23ff759 fixup! Format to 80 cols
In preparation for applying the patch that actually made it to the Git
mailing list, let's revert this.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:28:32 +01:00
Johannes Schindelin
07100ba0e0 fixup! Remove old code and macro-ize implementation
In preparation for applying the patch that actually made it to the Git
mailing list, let's revert this.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:26:31 +01:00
Johannes Schindelin
850406a924 squash! http: when using Secure Channel, ignore sslCAInfo by default
http: when using Secure Channel, ignore sslCAInfo by default

As of cURL v7.60.0, the Secure Channel backend can use the certificate
bundle provided via `http.sslCAInfo`, but that would override the
Windows Certificate Store. Since this is not desirable by default, let's
tell Git to not ask cURL to use that bundle by default when the `schannel`
backend was configured via `http.sslBackend`, unless
`http.schannelUseSSLCAInfo` overrides this behavior.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:18:49 +01:00
Brendan Forster
a67fd95daf squash! http: add support for disabling SSL revocation checks in cURL
http: add support for disabling SSL revocation checks in cURL

This adds support for a new http.schannelCheckRevoke config value.

This config value is only used if http.sslBackend is set to "schannel",
which forces cURL to use the Windows Certificate Store when validating
server certificates associated with a remote server.

This config value should only be set to "false" if you are in an
environment where revocation checks are blocked by the network, with
no alternative options.

This is only supported in cURL 7.44 or later.

Note: originally, we wanted to call the config setting
`http.schannel.checkRevoke`. This, however, does not work: the `http.*`
config settings can be limited to specific URLs via `http.<url>.*`
(and this feature would mistake `schannel` for a URL).

Helped by Agustín Martín Barbero.

Signed-off-by: Brendan Forster <github@brendanforster.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:18:40 +01:00
Johannes Schindelin
18dab43c05 squash! http: add support for selecting SSL backends at runtime
http: add support for selecting SSL backends at runtime

As of version 7.56.0, curl supports being compiled with multiple SSL
backends.

This patch adds the Git side of that feature: by setting http.sslBackend
to "openssl" or "schannel", Git for Windows can now choose the SSL
backend at runtime.

This comes in handy on Windows because Secure Channel ("schannel") is
the native solution, accessing the Windows Credential Store, thereby
allowing for enterprise-wide management of certificates. For historical
reasons, Git for Windows needs to support OpenSSL still, as it has
previously been the only supported SSL backend in Git for Windows for
almost a decade.

The patch has been carried in Git for Windows for over a year, and is
considered mature.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:18:28 +01:00
Johannes Schindelin
69eadfb835 fixup! http: add support for selecting SSL backends at runtime
Revert this in preparation for applying the latest iteration of the
patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:15:51 +01:00
Johannes Schindelin
a36e5dca52 fixup! http: add support for disabling SSL revocation checks in cURL
Revert this in preparation for applying the latest iteration of the
patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:15:41 +01:00
Johannes Schindelin
17719e683f fixup! http: when using Secure Channel, ignore sslCAInfo by default
Revert this in preparation for applying the latest iteration of the
patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:15:14 +01:00
Johannes Schindelin
0154b9c065 squash! do_git_config_sequence(): fall back to git_dir if commondir is NULL
config: report a bug if git_dir exists without commondir

This did happen at some stage, and was fixed relatively quickly. Make
sure that we detect very quickly, too, should that happen again.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:13:19 +01:00
Johannes Schindelin
b8591e07fd fixup! do_git_config_sequence(): fall back to git_dir if commondir is NULL
In preparation for applying the latest iteration of bw/config-h that was
submitted to the Git mailing list, let's revert the original iteration.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-19 11:09:08 +01:00
Johannes Schindelin
82dacae3c2 Merge pull request #1933 from dscho/update-create-empty-bundle
Update `create-empty-bundle`
2018-11-16 16:52:09 +01:00
Johannes Schindelin
8d08b44ff9 Merge pull request #1932 from dscho/prepare-for-2.20.0-rc0
Prepare for v2.20.0-rc0
2018-11-16 16:48:55 +01:00
Jeff King
3f0700f40b squash! bundle: refuse to create empty bundle
TODO: change authorship to Peff!

bundle: dup() output descriptor closer to point-of-use

When writing a bundle to a file, the bundle code actually creates
"your.bundle.lock" using our lockfile interface. We feed that output
descriptor to a child git-pack-objects via run-command, which has the
quirk that it closes the output descriptor in the parent.

To avoid confusing the lockfile code (which still thinks the descriptor
is valid), we dup() it, and operate on the duplicate.

However, this has a confusing side effect: after the dup() but before we
call pack-objects, we have _two_ descriptors open to the lockfile. If we
call die() during that time, the lockfile code will try to clean up the
partially-written file. It knows to close() the file before unlinking,
since on some platforms (i.e., Windows) the open file would block the
deletion. But it doesn't know about the duplicate descriptor. On
Windows, triggering an error at the right part of the code will result
in the cleanup failing and the lockfile being left in the filesystem.

We can solve this by moving the dup() much closer to start_command(),
shrinking the window in which we have the second descriptor open. It's
easy to place this in such a way that no die() is possible. We could
still die due to a signal in the exact wrong moment, but we already
tolerate races there (e.g., a signal could come before we manage to put
the file on the cleanup list in the first place).

As a bonus, this shields create_bundle() itself from the duplicate-fd
trick, and we can simplify its error handling (note that the lock
rollback now happens unconditionally, but that's OK; it's a noop if we
didn't open the lock in the first place).

The included test uses an empty bundle to cause a failure at the right
spot in the code, because that's easy to trigger (the other likely
errors are write() problems like ENOSPC).  Note that it would already
pass on non-Windows systems (because they are happy to unlink an
already-open file).

Based-on-a-patch-by: Gaël Lhez <gael.lhez@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-16 16:43:09 +01:00
Johannes Schindelin
a057a1a63f fixup! bundle: refuse to create empty bundle
This reverts the patch we carried for a long time, as it did not survive
contact with the Git mailing list. In preparation for applying Jeff
King's better version, let's revert our version.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-16 16:41:57 +01:00
Johannes Schindelin
2d3b4935cc Merge remote-tracking branch 'benpeart/fscache-per-thread-gfw'
This brings substantial wins in performance because the FSCache is now
per-thread, being merged to the primary thread only at the end, so we do
not have to lock (except while merging).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-16 15:42:47 +01:00
Ben Peart
52fc4db0f5 fscache: teach fscache to use mempool
Now that the fscache is single threaded, take advantage of the mem_pool as
the allocator to significantly reduce the cost of allocations and frees.

With the reduced cost of free, in future patches, we can start freeing the
fscache at the end of commands instead of just leaking it.

Signed-off-by: Ben Peart <benpeart@microsoft.com>
2018-11-16 08:06:14 -05:00
Ben Peart
f3ea036ad3 fscache: update fscache to be thread specific instead of global
The threading model for fscache has been to have a single, global cache.
This puts requirements on it to be thread safe so that callers like
preload-index can call it from multiple threads.  This was implemented
with a single mutex and completion events which introduces contention
between the calling threads.

Simplify the threading model by making fscache thread specific.  This allows
us to remove the global mutex and synchronization events entirely and instead
associate a fscache with every thread that requests one. This works well with
the current multi-threading which divides the cache entries into blocks with
a separate thread processing each block.

At the end of each worker thread, if there is a fscache on the primary
thread, merge the cached results from the worker into the primary thread
cache. This enables us to reuse the cache later especially when scanning for
untracked files.

In testing, this reduced the time spent in preload_index() by about 25% and
also reduced the CPU utilization significantly.  On a repo with ~200K files,
it reduced overall status times by ~12%.

Signed-off-by: Ben Peart <benpeart@microsoft.com>
2018-11-16 08:06:14 -05:00
Ben Peart
5ed0dce1f5 fscache: fscache takes an initial size
Update enable_fscache() to take an optional initial size parameter which is
used to initialize the hashmap so that it can avoid having to rehash as
additional entries are added.

Add a separate disable_fscache() macro to make the code clearer and easier
to read.

Signed-off-by: Ben Peart <benpeart@microsoft.com>
2018-11-16 08:06:10 -05:00
Johannes Schindelin
8d05b911b1 Merge pull request #1910 from benpeart/fscache_statistics-gfw
fscache: add fscache hit statistics
2018-11-16 12:07:19 +01:00
Johannes Schindelin
ba4743c757 Merge pull request #1914 from benpeart/free-fscache-after-add-gfw
At the end of the add command, disable and free the fscache
2018-11-16 12:04:15 +01:00
Johannes Schindelin
6a07e39811 fixup! Add a build definition for Azure DevOps
Let's use the "Hosted" pool, it seems to be under less load, i.e. it
will make our builds faster.

While at it, also disable chain linting and bin-wrappers in the Windows
phase, to save even more time.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-16 10:35:07 +01:00
Johannes Schindelin
d2884a866f tests: optionally skip bin-wrappers/
This speeds up the tests by a bit on Windows, where running Unix shell
scripts (and spawning processes) is not exactly a cheap operation.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-16 10:33:15 +01:00
Johannes Schindelin
49d2a03e6d t0061: fix with --with-dashes and RUNTIME_PREFIX
When building Git with RUNTIME_PREFIX and starting a test helper from
t/helper/, it fails to detect the system prefix correctly.

This is the reason that the warning

	RUNTIME_PREFIX requested, but prefix computation failed. [...]

to be printed.

In t0061, we did not expect that to happen, and it actually did not
happen in the normal case, because bin-wrappers/test-tool specifically
sets GIT_TEXTDOMAINDIR (and as a consequence, nothing in test-tool wants
to know about the runtime prefix).

However, with --with-dashes, bin-wrappers/test-tool is no longer called,
but t/helper/test-tool is called directly.

So let's just ignore the RUNTIME_PREFIX warning.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-16 10:33:06 +01:00
Johannes Schindelin
d482f01fe1 squash! tests: explicitly use git.exe on Windows
For the same reason, we need to handle test-tool.exe the same way.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-16 10:32:49 +01:00
Johannes Schindelin
1cdbf4d059 tests: add t/helper/ to the PATH with --with-dashes
We really need to be able to find the test helpers... Really. This
change was forgotten when we moved the test helpers into t/helper/

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-16 10:32:27 +01:00
Johannes Schindelin
a2b43c9e73 fixup! stash/rebase: default to the non-builtin versions
In Git for Windows v2.20.0, we will want to default to the built-in
versions.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-16 10:15:44 +01:00
Johannes Schindelin
598c3704db Merge branch 'vcxproj' into prepare-for-2.20.0-rc0
These patches will supersede the corresponding patches that are already
in the branch thicket, during the next merging-rebase.

That merging-rebase is scheduled for v2.20.0-rc0, which is scheduled
later tonight. However, the plan is to hold off with pushing this to
`master` until the merging-rebase to v2.20.0-rc1.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-16 09:58:07 +01:00
Johannes Schindelin
98970ee830 Merge branch 'kblees/kb/symlinks' 2018-11-15 22:20:06 +01:00