Commit Graph

73814 Commits

Author SHA1 Message Date
Johannes Schindelin
94755db6f7 Merge pull request #994 from jeffhostetler/jeffhostetler/fscache_nfd
fscache: add not-found directory cache to fscache
2017-07-13 08:30:50 +02:00
Johannes Schindelin
7d78589ac7 Merge pull request #1032 from max630/gitgui_GIT_GIT_unset
git-gui: correctly restore GIT_DIR after invoking commands
2017-07-13 08:30:49 +02:00
Johannes Schindelin
724a0db439 Merge branch 'test-unc-alternates'
The fix we introduced in Git for Windows will be made obsolete by a more
general fix that has been already accepted into upstream Git's `next`
branch.

But we still can introduce a regression test that verifies that this bug
will be caught very quickly, if reintroduced.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:48 +02:00
Johannes Schindelin
a96116ee72 fscache: add a test for the dir-not-found optimization
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:35 +02:00
Jeff Hostetler
839f7ed363 fscache: remember not-found directories
Teach FSCACHE to remember "not found" directories.

This is a performance optimization.

FSCACHE is a performance optimization available for Windows.  It
intercepts Posix-style lstat() calls into an in-memory directory
using FindFirst/FindNext.  It improves performance on Windows by
catching the first lstat() call in a directory, using FindFirst/
FindNext to read the list of files (and attribute data) for the
entire directory into the cache, and short-cut subsequent lstat()
calls in the same directory.  This gives a major performance
boost on Windows.

However, it does not remember "not found" directories.  When STATUS
runs and there are missing directories, the lstat() interception
fails to find the parent directory and simply return ENOENT for the
file -- it does not remember that the FindFirst on the directory
failed. Thus subsequent lstat() calls in the same directory, each
re-attempt the FindFirst.  This completely defeats any performance
gains.

This can be seen by doing a sparse-checkout on a large repo and
then doing a read-tree to reset the skip-worktree bits and then
running status.

This change reduced status times for my very large repo by 60%.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:35 +02:00
Jeff Hostetler
1c74c5edd7 fscache: add key for GIT_TRACE_FSCACHE
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:35 +02:00
Johannes Schindelin
75462d0d35 Merge branch 'long-paths'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:35 +02:00
Max Kirillov
619c160ae1 git-gui: correctly restore GIT_DIR after invoking gitk
git-gui tries to temporary set GIT_DIR for starting gitk and restore
it back after they are started. But in case of GIT_DIR which was not set
prior to invocation it is not unset after it. This affects commands
which can be later started from that git gui, for example "Git Bash".

Fix it.

Signed-off-by: Max Kirillov <max@max630.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:32 +02:00
Johannes Schindelin
969161601b Merge branch 'git-gui'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:32 +02:00
Johannes Schindelin
1921c99496 t5580: verify that alternates can be UNC paths
On Windows, UNC paths are a very convenient way to share data, and
alternates are all about sharing data.

We fixed a bug where alternates specifying UNC paths were not handled
properly, and it is high time that we add a regression test to ensure
that this bug is not reintroduced.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:30 +02:00
Johannes Schindelin
aa229dbfff t9001: work around hard-to-debug hangs
Just like the workaround we added for t9116, t9001.83 hangs sometimes --
but not always! -- when being run in the Git for Windows SDK.

The issue seems to be related to redirection via a pipe, but it is really
hard to diagnose, what with git.exe (a non-MSYS2 program) calling a Perl
script (which is executed by an MSYS2 Perl), piping into another MSYS2
program.

As hunting time is scarce these days, simply work around this for now and
leave the real diagnosis and resolution for later.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:29 +02:00
Johannes Schindelin
b7011a9198 t9116: work around hard-to-debug hangs
As of a couple of weeks ago, t9116 hangs sometimes -- but not always! --
when being run in the Git for Windows SDK.

The issue seems to be related to redirection via a pipe, but it is really
hard to diagnose, what with git.exe (a non-MSYS2 program) calling a Perl
script (which is executed by an MSYS2 Perl), piping into another MSYS2
program.

As hunting time is scarce these days, simply work around this for now and
leave the real diagnosis and resolution for later.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:29 +02:00
Johannes Schindelin
a99470491e Merge branch 'interactive-rebase-current'
This series of branches introduces the git-rebase--helper, a builtin
helping to accelerate the interactive rebase dramatically.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:28 +02:00
Johannes Schindelin
cf4cba90e3 Merge pull request #996 from jeffhostetler/jeffhostetler/register_rename_src
diffcore-rename: speed up register_rename_src
2017-07-13 08:30:27 +02:00
Johannes Schindelin
48775b1721 Merge pull request #1004 from whoisj/nolock-env
Carry non-locking status value in the environment.
2017-07-13 08:30:26 +02:00
Johannes Schindelin
a20f37bddc Merge pull request #1003 from shoelzer/master
poll: Use GetTickCount64 to avoid wraparound issues
2017-07-13 08:30:25 +02:00
Johannes Schindelin
bfdbbddd60 Merge branch 'rebase-i-extra-v4'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:23 +02:00
Johannes Schindelin
125fe3c100 rebase -i: rearrange fixup/squash lines using the rebase--helper
This operation has quadratic complexity, which is especially painful
on Windows, where shell scripts are *already* slow (mainly due to the
overhead of the POSIX emulation layer).

Let's reimplement this with linear complexity (using a hash map to
match the commits' subject lines) for the common case; Sadly, the
fixup/squash feature's design neglected performance considerations,
allowing arbitrary prefixes (read: `fixup! hell` will match the
commit subject `hello world`), which means that we are stuck with
quadratic performance in the worst case.

The reimplemented logic also happens to fix a bug where commented-out
lines (representing empty patches) were dropped by the previous code.

While at it, clarify how the fixup/squash feature works in `git rebase
-i`'s man page.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:21 +02:00
Johannes Schindelin
eba008c2fa t3415: test fixup with wrapped oneline
The `git commit --fixup` command unwraps wrapped onelines when
constructing the commit message, without wrapping the result.

We need to make sure that `git rebase --autosquash` keeps handling such
cases correctly, in particular since we are about to move the autosquash
handling into the rebase--helper.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:21 +02:00
Johannes Schindelin
2ffaffbe76 rebase -i: skip unnecessary picks using the rebase--helper
In particular on Windows, where shell scripts are even more expensive
than on MacOSX or Linux, it makes sense to move a loop that forks
Git at least once for every line in the todo list into a builtin.

Note: The original code did not try to skip unnecessary picks of root
commits but punts instead (probably --root was not considered common
enough of a use case to bother optimizing). We do the same, for now.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:21 +02:00
Johannes Schindelin
5e68ea28c6 rebase -i: check for missing commits in the rebase--helper
In particular on Windows, where shell scripts are even more expensive
than on MacOSX or Linux, it makes sense to move a loop that forks
Git at least once for every line in the todo list into a builtin.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:21 +02:00
Johannes Schindelin
10404523ea t3404: relax rebase.missingCommitsCheck tests
These tests were a bit anal about the *exact* warning/error message
printed by git rebase. But those messages are intended for the *end
user*, therefore it does not make sense to test so rigidly for the
*exact* wording.

In the following, we will reimplement the missing commits check in
the sequencer, with slightly different words.

So let's just test for the parts in the warning/error message that
we *really* care about, nothing more, nothing less.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:20 +02:00
Johannes Schindelin
5da101194a rebase -i: also expand/collapse the SHA-1s via the rebase--helper
This is crucial to improve performance on Windows, as the speed is now
mostly dominated by the SHA-1 transformation (because it spawns a new
rev-parse process for *every* line, and spawning processes is pretty
slow from Git for Windows' MSYS2 Bash).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:20 +02:00
Johannes Schindelin
ba7178dcd3 rebase -i: do not invent onelines when expanding/collapsing SHA-1s
To avoid problems with short SHA-1s that become non-unique during the
rebase, we rewrite the todo script with short/long SHA-1s before and
after letting the user edit the script. Since SHA-1s are not intuitive
for humans, rebase -i also provides the onelines (commit message
subjects) in the script, purely for the user's convenience.

It is very possible to generate a todo script via different means than
rebase -i and then to let rebase -i run with it; In this case, these
onelines are not required.

And this is where the expand/collapse machinery has a bug: it *expects*
that oneline, and failing to find one reuses the previous SHA-1 as
"oneline".

It was most likely an oversight, and made implementation in the (quite
limiting) shell script language less convoluted. However, we are about
to reimplement performance-critical parts in C (and due to spawning a
git.exe process for every single line of the todo script, the
expansion/collapsing of the SHA-1s *is* performance-hampering on
Windows), therefore let's fix this bug to make cross-validation with the
C version of that functionality possible.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:20 +02:00
Johannes Schindelin
b53f131b58 rebase -i: remove useless indentation
The commands used to be indented, and it is nice to look at, but when we
transform the SHA-1s, the indentation is removed. So let's do away with it.

For the moment, at least: when we will use the upcoming rebase--helper
to transform the SHA-1s, we *will* keep the indentation and can
reintroduce it. Yet, to be able to validate the rebase--helper against
the output of the current shell script version, we need to remove the
extra indentation.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:20 +02:00
Johannes Schindelin
955613240a rebase -i: generate the script via rebase--helper
The first step of an interactive rebase is to generate the so-called "todo
script", to be stored in the state directory as "git-rebase-todo" and to
be edited by the user.

Originally, we adjusted the output of `git log <options>` using a simple
sed script. Over the course of the years, the code became more
complicated. We now use shell scripting to edit the output of `git log`
conditionally, depending whether to keep "empty" commits (i.e. commits
that do not change any files).

On platforms where shell scripting is not native, this can be a serious
drag. And it opens the door for incompatibilities between platforms when
it comes to shell scripting or to Unix-y commands.

Let's just re-implement the todo script generation in plain C, using the
revision machinery directly.

This is substantially faster, improving the speed relative to the
shell script version of the interactive rebase from 2x to 3x on Windows.

Note that the rearrange_squash() function in git-rebase--interactive
relied on the fact that we set the "format" variable to the config setting
rebase.instructionFormat. Relying on a side effect like this is no good,
hence we explicitly perform that assignment (possibly again) in
rearrange_squash().

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:20 +02:00
Johannes Schindelin
960171bfa1 t3415: verify that an empty instructionFormat is handled as before
An upcoming patch will move the todo list generation into the
rebase--helper. An early version of that patch regressed on an empty
rebase.instructionFormat value (the shell version could not discern
between an empty one and a non-existing one, but the C version used the
empty one as if that was intended to skip the oneline from the `pick
<hash>` lines).

Let's verify that this still works as before.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:20 +02:00
Jeff Hostetler
b3f2b90d02 diffcore-rename: speed up register_rename_src
Teach register_rename_src() to see if new file pair
can simply be appended to the rename_src[] array before
performing the binary search to find the proper insertion
point.

This is a performance optimization.  This routine is called
during run_diff_files in status and the caller is iterating
over the sorted index, so we should expect to be able to
append in the normal case.  The existing insert logic is
preserved so we don't have to assume that, but simply take
advantage of it if possible.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2017-07-13 08:30:17 +02:00
J Wyman
8204773637 Carry non-locking status value in the environment.
If the user has specified '--no-lock-index' when calling git-status, it only seems reasonable that the user intends that option to be carried through to any child forks/procs as well. Currently, the '--no-lock-status' call is lost when submodules are checked. This change places the desired option into the environment, which is in turn passed down to all subsequent children.

With cmd_status checking for '--no-lock--status' first from args then from environment, we're able to keep the option set in all children.

Signed-off-by: J Wyman <jeremy.wyman@microsoft.com>
2017-07-13 08:30:16 +02:00
Johannes Schindelin
0764cfac7e poll: lazy-load GetTickCount64()
This fixes the compilation, actually, as we still did not make the jump to
post-Windows XP completely: we still compile with _WIN32_WINNT set to
0x0502 (which corresponds to Windows Server 2003 and is technically
greater than Windows XP's 0x0501).

However, GetTickCount64() is only available starting with Windows
Vista/Windows Server 2008.

Let's just lazy-load the function, which should also help Git for Windows
contributors who want to reinstate Windows XP support.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:15 +02:00
Johannes Schindelin
a16b1eb219 Merge pull request #159 from dscho/vagrant
Add Vagrant support (easy Linux VM setup)
2017-07-13 08:30:14 +02:00
Steve Hoelzer
aba57f6ac2 poll: Use GetTickCount64 to avoid wraparound issues
From Visual Studio 2015 Code Analysis: Warning C28159 Consider using
'GetTickCount64' instead of 'GetTickCount'.

Reason: GetTickCount overflows roughly every 49 days. Code that does not
take that into account can loop indefinitely. GetTickCount64 operates on
64 bit values and does not have that problem.

Signed-off-by: Steve Hoelzer <shoelzer@gmail.com>
2017-07-13 08:30:14 +02:00
Johannes Schindelin
dca0854865 Merge pull request #978 from jeffhostetler/jeffhostetler/thread_verify_hdr
read-cache: run verify_hdr() in background thread
2017-07-13 08:30:13 +02:00
Johannes Schindelin
ab1f989aeb Merge 'misc-vs-fixes-extra' into HEAD
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:12 +02:00
Johannes Schindelin
4778dc9bb5 Merge branch 'visual-studio'
This topic branch teaches the project generator to generate a Visual
Studio solution, ready to be opened in Visual Studio 2010 or later.

The idea, of course, is to let some automatic build job generate and
commit the project files with

	make MSVC=1 vcxproj

and then (force-)push to a special-purpose branch.

The major part of this branch thicket concerns itself not only with
generating the Visual Studio project files, but making sure that the
user can then run the test suite from a regular Git Bash (i.e. *not*
requiring a Git for Windows SDK), e.g. by running

	cd t
	prove --timer --jobs 15 ./t[0-9]*.sh

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:11 +02:00
Johannes Schindelin
a30b3cd609 Merge pull request #971 from jeffhostetler/jeffhostetler/add_preload_fscache
add: use preload-index and fscache for performance
2017-07-13 08:30:10 +02:00
Johannes Schindelin
29a104ad67 mingw: make readlink() independent of core.symlinks
Regardless whether we think we are able to create symbolic links, we
should always read them.

This fixes https://github.com/git-for-windows/git/issues/958

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:09 +02:00
Johannes Schindelin
8058d484cf Merge pull request #938 from virtuald/patch-1
git-cvsexportcommit.perl: Force crlf translation
2017-07-13 08:30:09 +02:00
Johannes Schindelin
4ce93a3351 Merge branch 'reset-stdin'
This topic branch adds the (experimental) --stdin/-z options to `git
reset`. Those patches are still under review in the upstream Git project,
but are already merged in their experimental form into Git for Windows'
`master` branch, in preparation for a MinGit-only release.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:08 +02:00
Johannes Schindelin
42b3d2eb20 Merge branch 'mingw-strftime'
This topic branch works around an out-of-memory bug when the user
specified a format via --date=format:<format> that strftime() does
not like.

Reported by Stefan Naewe.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:07 +02:00
Johannes Schindelin
a460da1bd7 Unbreak interactive GPG prompt upon signing
With the recent update in efee955 (gpg-interface: check gpg signature
creation status, 2016-06-17), we ask GPG to send all status updates to
stderr, and then catch the stderr in an strbuf.

But GPG might fail, and send error messages to stderr. And we simply
do not show them to the user.

Even worse: this swallows any interactive prompt for a passphrase. And
detaches stderr from the tty so that the passphrase cannot be read.

So while the first problem could be fixed (by printing the captured
stderr upon error), the second problem cannot be easily fixed, and
presents a major regression.

So let's just revert commit efee9553a4.

This fixes https://github.com/git-for-windows/git/issues/871

Cc: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:06 +02:00
Johannes Schindelin
1f42687dc2 Merge pull request #866 from landstander668/add_platform
Add reporting of build platform
2017-07-13 08:30:05 +02:00
Johannes Schindelin
9e7e755d36 Merge branch 'unhidden-git'
It has been reported that core.hideDotFiles=false stopped working...
This topic branch fixes it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:04 +02:00
Johannes Schindelin
063c297541 Merge branch 'status-no-lock-index'
This branch allows third-party tools to call `git status
--no-lock-index` to avoid lock contention with the interactive Git usage
of the actual human user.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:03 +02:00
Johannes Schindelin
a80de09703 Merge pull request #797 from glhez/master
`git bundle create <bundle>` leaks handle the revlist is empty.
2017-07-13 08:30:02 +02:00
Johannes Schindelin
1792bdecbc Merge 'release-gc-repack' into HEAD 2017-07-13 08:30:01 +02:00
Johannes Schindelin
971c4105bb Merge branch 'spawn-with-spaces'
This change lets us spawn .bat scripts whose paths contain spaces.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:30:00 +02:00
Johannes Schindelin
fdb93fd965 Support Vagrant: quick & easy Linux virtual machine setup
When developing Git for Windows, we always have to ensure that we do not
break any non-Windows platforms, e.g. by introducing Windows-specific code
into the platform-independent source code.

At other times, it is necessary to test whether a bug is Windows-specific
or not, in order to send the bug report to the correct place. Having
access to a Linux-based Git comes in really handy in such a situation.

Vagrant offers a painless way to install and use a defined Linux
development environment on Windows (and other Operating Systems). We offer
a Vagrantfile to that end for two reasons:

1) To allow Windows users to gain the full power of Linux' Git

2) To offer users an easy path to verify that the issue they are about
   to report is really a Windows-specific issue; otherwise they would
   need to report it to git@vger.kernel.org instead.

Using it is easy: Download and install https://www.virtualbox.org/, then
download and install https://www.vagrantup.com/, then direct your
command-line window to the Git source directory containing the Vagrantfile
and run the commands:

	vagrant up
	vagrant ssh

See https://github.com/git-for-windows/git/wiki/Vagrant for details.

As part of switching Git for Windows' development environment from msysGit
to the MSys2-based Git SDK, this Vagrantfile was copy-edited from msysGit:

	https://github.com/msysgit/msysgit/blob/0be8f2208/Vagrantfile

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:29:59 +02:00
Jeff Hostetler
a9b6362ea0 read-cache: run verify_hdr() in background thread
This is a performance optimization.

Teach do_read_index() to call verify_hdr() using a thread
and allow SHA1 verification to run concurrently with the
parsing of index-entries and extensions.

For large index files, this cuts the startup time in half.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
2017-07-13 08:29:57 +02:00
Johannes Schindelin
27bbb60241 msvc: avoid debug assertion windows in Debug Mode
For regular debugging, it is pretty helpful when a debug assertion in a
running application triggers a window that offers to start the debugger.

However, when running the test suite, it is not so helpful, in
particular when the debug assertions are then suppressed anyway because
we disable the invalid parameter checking (via invalidcontinue.obj, see
the comment in config.mak.uname about that object for more information).

So let's simply disable that window in Debug Mode (it is already
disabled in Release Mode).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-07-13 08:29:56 +02:00