Commit Graph

61652 Commits

Author SHA1 Message Date
Johannes Schindelin
0772c2dfd0 Merge pull request #93 from nalla/asciidoctor-fixes
Asciidoctor fixes

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-13 07:33:13 +02:00
Johannes Schindelin
bfb165abaf Merge 'long-paths' into HEAD 2016-08-13 07:33:12 +02:00
Johannes Schindelin
939bd90b78 Merge 'fscache' into HEAD 2016-08-13 07:33:12 +02:00
nalla
1ac6e5a3c6 asciidoctor: Fix giteveryday.txt to be built with asciidoctor.
When building the `doc` with `asciidoctor`, `asciidoctor` complains about
a nested code block in a callout list. This is a really dirty solution to
restore the callout list to function properly. There is a minimal visual
sideeffect; the *immitated* codeblock has no overall greyish background.
Instead the individual lines have it.

Note: When building this patch with `asciidoc` the background is totally
gone but the font is still monospaced.

Signed-off-by: nalla <nalla@hamal.uberspace.de>
2016-08-13 07:33:07 +02:00
nalla
fd209999d5 asciidoctor: Fix user-manual to be built by asciidoctor
The `user-manual.txt` ist designed as a `book` but the `Makefile` wants to
build it as an `article`. This seems to be a problem when building the
documentation with `asciidoctor`. Furthermore the parts *Git Glossary*
and *Apendix B* had no subsections which is not allowed when building with
`asciidoctor`. So lets add a *dummy* section.

Signed-off-by: nalla <nalla@hamal.uberspace.de>
2016-08-13 07:33:06 +02:00
Karsten Blees
6f21d7090a Win32: fix 'lstat("dir/")' with long paths
Use a suffciently large buffer to strip the trailing slash.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-08-13 07:33:04 +02:00
Karsten Blees
4497b995fa Win32: support long paths
Windows paths are typically limited to MAX_PATH = 260 characters, even
though the underlying NTFS file system supports paths up to 32,767 chars.
This limitation is also evident in Windows Explorer, cmd.exe and many
other applications (including IDEs).

Particularly annoying is that most Windows APIs return bogus error codes
if a relative path only barely exceeds MAX_PATH in conjunction with the
current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the
infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG.

Many Windows wide char APIs support longer than MAX_PATH paths through the
file namespace prefix ('\\?\' or '\\?\UNC\') followed by an absolute path.
Notable exceptions include functions dealing with executables and the
current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as
well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...).

Introduce a handle_long_path function to check the length of a specified
path properly (and fail with ENAMETOOLONG), and to optionally expand long
paths using the '\\?\' file namespace prefix. Short paths will not be
modified, so we don't need to worry about device names (NUL, CON, AUX).

Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be
limited to MAX_PATH (at least not on Win7), so we can use it to do the
heavy lifting of the conversion (translate '/' to '\', eliminate '.' and
'..', and make an absolute path).

Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH
limit.

Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs
that support long paths.

While improved error checking is always active, long paths support must be
explicitly enabled via 'core.longpaths' option. This is to prevent end
users to shoot themselves in the foot by checking out files that Windows
Explorer, cmd/bash or their favorite IDE cannot handle.

Test suite:
Test the case is when the full pathname length of a dir is close
to 260 (MAX_PATH).
Bug report and an original reproducer by Andrey Rogozhnikov:
https://github.com/msysgit/git/pull/122#issuecomment-43604199

Note that the test cannot rely on the presence of short names, as they
are not enabled by default except on the system drive.

[jes: adjusted test number to avoid conflicts, reinstated && chain,
adjusted test to work without short names]

Thanks-to: Martin W. Kirst <maki@bitkings.de>
Thanks-to: Doug Kelly <dougk.ff7@gmail.com>
Signed-off-by: Karsten Blees <blees@dcon.de>
Original-test-by: Andrey Rogozhnikov <rogozhnikov.andrey@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-13 07:33:03 +02:00
Johannes Schindelin
3b60eda19d Win32: support long paths
Windows paths are typically limited to MAX_PATH = 260 characters, even
though the underlying NTFS file system supports paths up to 32,767 chars.
This limitation is also evident in Windows Explorer, cmd.exe and many
other applications (including IDEs).

Particularly annoying is that most Windows APIs return bogus error codes
if a relative path only barely exceeds MAX_PATH in conjunction with the
current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the
infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG.

Many Windows wide char APIs support longer than MAX_PATH paths through the
file namespace prefix ('\\?\' or '\\?\UNC\') followed by an absolute path.
Notable exceptions include functions dealing with executables and the
current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as
well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...).

Introduce a handle_long_path function to check the length of a specified
path properly (and fail with ENAMETOOLONG), and to optionally expand long
paths using the '\\?\' file namespace prefix. Short paths will not be
modified, so we don't need to worry about device names (NUL, CON, AUX).

Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be
limited to MAX_PATH (at least not on Win7), so we can use it to do the
heavy lifting of the conversion (translate '/' to '\', eliminate '.' and
'..', and make an absolute path).

Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH
limit.

Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs
that support long paths.

While improved error checking is always active, long paths support must be
explicitly enabled via 'core.longpaths' option. This is to prevent end
users to shoot themselves in the foot by checking out files that Windows
Explorer, cmd/bash or their favorite IDE cannot handle.

Test suite:
Test the case is when the full pathname length of a dir is close
to 260 (MAX_PATH).
Bug report and an original reproducer by Andrey Rogozhnikov:
https://github.com/msysgit/git/pull/122#issuecomment-43604199

[jes: adjusted test number to avoid conflicts]

Thanks-to: Martin W. Kirst <maki@bitkings.de>
Thanks-to: Doug Kelly <dougk.ff7@gmail.com>
Signed-off-by: Karsten Blees <blees@dcon.de>
Original-test-by: Andrey Rogozhnikov <rogozhnikov.andrey@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-13 07:33:03 +02:00
Doug Kelly
89549f4c6b Add a test demonstrating a problem with long submodule paths
[jes: adusted test number to avoid conflicts, fixed non-portable use of
the 'export' statement]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-13 07:33:02 +02:00
Karsten Blees
4e8464aaae fscache: load directories only once
If multiple threads access a directory that is not yet in the cache, the
directory will be loaded by each thread. Only one of the results is added
to the cache, all others are leaked. This wastes performance and memory.

On cache miss, add a future object to the cache to indicate that the
directory is currently being loaded. Subsequent threads register themselves
with the future object and wait. When the first thread has loaded the
directory, it replaces the future object with the result and notifies
waiting threads.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-08-13 07:33:00 +02:00
Karsten Blees
881dbe0ed0 Win32: add a cache below mingw's lstat and dirent implementations
Checking the work tree status is quite slow on Windows, due to slow lstat
emulation (git calls lstat once for each file in the index). Windows
operating system APIs seem to be much better at scanning the status
of entire directories than checking single files.

Add an lstat implementation that uses a cache for lstat data. Cache misses
read the entire parent directory and add it to the cache. Subsequent lstat
calls for the same directory are served directly from the cache.

Also implement opendir / readdir / closedir so that they create and use
directory listings in the cache.

The cache doesn't track file system changes and doesn't plug into any
modifying file APIs, so it has to be explicitly enabled for git functions
that don't modify the working copy.

Note: in an earlier version of this patch, the cache was always active and
tracked file system changes via ReadDirectoryChangesW. However, this was
much more complex and had negative impact on the performance of modifying
git commands such as 'git checkout'.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-08-13 07:32:59 +02:00
Karsten Blees
18a6818a89 add infrastructure for read-only file system level caches
Add a macro to mark code sections that only read from the file system,
along with a config option and documentation.

This facilitates implementation of relatively simple file system level
caches without the need to synchronize with the file system.

Enable read-only sections for 'git status' and preload_index.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-08-13 07:32:58 +02:00
Karsten Blees
85a754b68f Win32: make the lstat implementation pluggable
Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite
slow. Windows operating system APIs seem to be much better at scanning the
status of entire directories than checking single files. A caching
implementation may improve performance by bulk-reading entire directories
or reusing data obtained via opendir / readdir.

Make the lstat implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-08-13 07:32:57 +02:00
Karsten Blees
589cc61958 Win32: Make the dirent implementation pluggable
Emulating the POSIX dirent API on Windows via FindFirstFile/FindNextFile is
pretty staightforward, however, most of the information provided in the
WIN32_FIND_DATA structure is thrown away in the process. A more
sophisticated implementation may cache this data, e.g. for later reuse in
calls to lstat.

Make the dirent implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Define a base DIR structure with pointers to readdir/closedir that match
the opendir implementation (i.e. similar to vtable pointers in OOP).
Define readdir/closedir so that they call the function pointers in the DIR
structure. This allows to choose the opendir implementation on a
call-by-call basis.

Move the fixed sized dirent.d_name buffer to the dirent-specific DIR
structure, as d_name may be implementation specific (e.g. a caching
implementation may just set d_name to point into the cache instead of
copying the entire file name string).

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-08-13 07:32:57 +02:00
Karsten Blees
a3ddbbf9be Win32: dirent.c: Move opendir down
Move opendir down in preparation for the next patch.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-08-13 07:32:56 +02:00
Karsten Blees
836c5bcf5f Win32: make FILETIME conversion functions public
Signed-off-by: Karsten Blees <blees@dcon.de>
2016-08-13 07:32:55 +02:00
Johannes Schindelin
db1c6eaf89 Start the merging-rebase to f883596e99
This commit starts the rebase of 2075773 to f883596
2016-08-13 07:32:30 +02:00
Junio C Hamano
e0c1ceafc5 Git 2.9.3
Signed-off-by: Junio C Hamano <gitster@pobox.com>
v2.9.3
2016-08-12 09:17:51 -07:00
Junio C Hamano
9b601eafd1 Merge branch 'jk/difftool-in-subdir' into maint
"git difftool <paths>..." started in a subdirectory failed to
interpret the paths relative to that directory, which has been
fixed.

* jk/difftool-in-subdir:
  difftool: use Git::* functions instead of passing around state
  difftool: avoid $GIT_DIR and $GIT_WORK_TREE
  difftool: fix argument handling in subdirs
2016-08-12 09:16:57 -07:00
Junio C Hamano
f4fd627661 Merge branch 'jk/reset-ident-time-per-commit' into maint
Not-so-recent rewrite of "git am" that started making internal
calls into the commit machinery had an unintended regression, in
that no matter how many seconds it took to apply many patches, the
resulting committer timestamp for the resulting commits were all
the same.

* jk/reset-ident-time-per-commit:
  am: reset cached ident date for each patch
2016-08-12 09:16:56 -07:00
Johannes Schindelin
05ee656689 Merge branch 'unhidden-git'
It has been reported that core.hideDotFiles=false stopped working...
This topic branch fixes it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-12 17:23:12 +02:00
Johannes Schindelin
d21f1b07b4 Merge branch 'status-no-lock-index'
This branch allows third-party tools to call `git status
--no-lock-index` to avoid lock contention with the interactive Git usage
of the actual human user.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-12 16:16:24 +02:00
Johannes Schindelin
9557c5e88b Merge 'cat-file-smudge'
This topic branch adds support to cat-file to apply the smudge filter
(if any) when showing blobs.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-12 16:16:22 +02:00
Johannes Schindelin
0694cf5609 Merge 'kw/patch-ids-optim'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-12 16:16:21 +02:00
Johannes Schindelin
b9577139f2 Merge 'jh/status-v2-porcelain'
Backported onto `maint` by me...

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-12 16:16:21 +02:00
Johannes Schindelin
b98c987d5f Merge branch 'consolez'
This fixes an issue where the Git wrapper would terminate upon Ctrl+C,
even in the case when its child process would *not* terminate.

Note: while the original intention was to fix running Git Bash in
ConsoleZ, the bug fix applies also to running

	C:\Program Files\Git\bin\bash -l -i

in a cmd window.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-12 16:16:20 +02:00
Johannes Schindelin
5ddfc3bdd9 Merge branch 'git-wrapper-interpolate'
There was a bug in the wrapper where it would interpolate incorrectly if
the name of the environment variable to expand was longer than the value.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-12 16:16:19 +02:00
Johannes Schindelin
b3d83f9528 Merge branch 'home-bin' 2016-08-12 16:16:18 +02:00
Johannes Schindelin
4b4780a977 Merge branch 'conhost-git-bash' 2016-08-12 16:16:17 +02:00
Johannes Schindelin
d5a1976c2b Merge branch 'bash-redirector' 2016-08-12 16:16:16 +02:00
Johannes Schindelin
982a36a811 Merge branch 'pinnable'
Part 2/3 of fixing https://github.com/git-for-windows/git/issues/263

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-12 16:16:15 +02:00
Johannes Schindelin
a6bf9d933e Merge branch 'git-wrapper--command'
This topic branch adds the --command=<command> option that allows
starting the Git Bash (or Git CMD) with different terminal emulators
than the one encoded via embedded string resources.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-12 16:16:14 +02:00
Johannes Schindelin
e0f32ab251 Merge 'git-wrapper' into HEAD
Use msysGit's `git-wrapper` instead of the builtins. This works around
two issues:

- when the file system does not allow hard links, we would waste over
  800 megabyte by having 109 copies of a multi-megabyte executable

- even when the file system allows hard links, the Windows Explorer
  counts the disk usage as if it did not. Many users complained about
  Git for Windows using too much space (when it actually did not). We
  can easily avoid those user complaints by merging this branch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-12 16:16:14 +02:00
Johannes Schindelin
db5c81edff Merge pull request #797 from glhez/master
`git bundle create <bundle>` leaks handle the revlist is empty.
2016-08-12 16:16:12 +02:00
Johannes Schindelin
fd2862e68d Merge 'release-gc-repack' into HEAD 2016-08-12 16:16:11 +02:00
Johannes Schindelin
7df91b900b Merge pull request #755 from bwijen/master
Prevent child processes from inheriting a handle to index.lock
2016-08-12 16:16:10 +02:00
Johannes Schindelin
e760f6b26e Merge pull request #726 from orgads/git-gui-preserve-author-windows
git-gui: Do not reset author details on amend
2016-08-12 16:16:09 +02:00
Johannes Schindelin
5cb2e99f88 Merge branch 'spawn-with-spaces'
This change lets us spawn .bat scripts whose paths contain spaces.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-12 16:16:09 +02:00
Johannes Schindelin
213349abe7 cat-file --textconv/--smudge: allow specifying the path separately
There are circumstances when it is relatively easy to figure out the
object name for a given path, but not the revision. For example, when
looking at a diff generated by Git, the object names are recorded, but
not the revision. As a matter of fact, the revisions from which the diff
was generated may not even exist locally.

In such a case, the user would have to generate a fake revision just to
be able to use --textconv or --smudge.

Let's simplify this dramatically, because we do not really need that
revision at all: all we care about is that we know the path. In the
scenario described above, we do know the path, and we just want to
specify it separately from the object name.

Example usage:

	git cat-file --textconv --use-path=main.c 0f1937fd

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-12 16:16:06 +02:00
Johannes Schindelin
c6b63f7572 cat-file: introduce the --smudge option
As suggested by its name, the --smudge option applies the smudge filter
that is currently configured for the specified path.

This feature comes in handy when a 3rd-party tool wants to work with the
contents of files from past revisions as if they had been checked out,
but without detouring via temporary files.

Note that we ensure that symbolic links are unaffected (we know from
looking at the mode).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-12 16:16:06 +02:00
Johannes Schindelin
383705a9e8 cat-file: fix a grammo in the man page
"... has be ..." -> "... has to be ..."

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-08-12 16:16:05 +02:00
Kevin Willford
75dd2eccb5 rebase: avoid computing unnecessary patch IDs
The `rebase` family of Git commands avoid applying patches that were
already integrated upstream. They do that by using the revision walking
option that computes the patch IDs of the two sides of the rebase
(local-only patches vs upstream-only ones) and skipping those local
patches whose patch ID matches one of the upstream ones.

In many cases, this causes unnecessary churn, as already the set of
paths touched by a given commit would suffice to determine that an
upstream patch has no local equivalent.

This hurts performance in particular when there are a lot of upstream
patches, and/or large ones.

Therefore, let's introduce the concept of a "diff-header-only" patch ID,
compare those first, and only evaluate the "full" patch ID lazily.

Please note that in contrast to the "full" patch IDs, those
"diff-header-only" patch IDs are prone to collide with one another, as
adjacent commits frequently touch the very same files. Hence we now
have to be careful to allow multiple hash entries with the same hash.
We accomplish that by using the hashmap_add() function that does not even
test for hash collisions.  This also allows us to evaluate the full patch ID
lazily, i.e. only when we found commits with matching diff-header-only
patch IDs.

We add a performance test that demonstrates ~1-6% improvement.  In
practice this will depend on various factors such as how many upstream
changes and how big those changes are along with whether file system
caches are cold or warm.  As Git's test suite has no way of catching
performance regressions, we also add a regression test that verifies
that the full patch ID computation is skipped when the diff-header-only
computation suffices.

Signed-off-by: Kevin Willford <kcwillford@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-12 16:16:03 +02:00
Kevin Willford
b723806ce2 patch-ids: add flag to create the diff patch id using header only data
This will allow a diff patch id to be created using only the header data
so that the contents of the file will not have to be loaded.

Signed-off-by: Kevin Willford <kcwillford@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-12 16:16:02 +02:00
Kevin Willford
196f83d0c7 patch-ids: replace the seen indicator with a commit pointer
The cherry_pick_list was looping through the original side checking the
seen indicator and setting the cherry_flag on the commit.  If we save
off the commit in the patch_id we can set the cherry_flag on the correct
commit when running through the other side when a patch_id match is found.

Signed-off-by: Kevin Willford <kcwillford@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-12 16:16:01 +02:00
Kevin Willford
64778e9818 patch-ids: stop using a hand-rolled hashmap implementation
This change will use the hashmap from the hashmap.h to keep track of the
patch_ids that have been encountered instead of using an internal
implementation.  This simplifies the implementation of the patch ids.

Signed-off-by: Kevin Willford <kcwillford@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-12 16:16:01 +02:00
Jeff Hostetler
2bc61a0f3e status: unit tests for --porcelain=v2
Test porcelain v2 status format.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-12 16:15:58 +02:00
Jeff Hostetler
9338815097 test-lib-functions.sh: Add lf_to_nul
Add lf_to_nul() function to test-lib-functions.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-12 16:15:58 +02:00
Jeff Hostetler
c46dee45e4 git-status.txt: describe --porcelain=v2 format
Update status manpage to include information about
porcelain v2 format.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-12 16:15:57 +02:00
Jeff Hostetler
23d023db84 status: print branch info with --porcelain=v2 --branch
Expand porcelain v2 output to include branch and tracking
branch information. This includes the commit id, the branch,
the upstream branch, and the ahead and behind counts.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-12 16:15:56 +02:00
Jeff Hostetler
f1c2b65421 status: print per-file porcelain v2 status data
Print per-file information in porcelain v2 format.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-12 16:15:56 +02:00