Commit Graph

59218 Commits

Author SHA1 Message Date
Johannes Schindelin
d64c26780d Merge pull request #93 from nalla/asciidoctor-fixes
Asciidoctor fixes

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-05-20 07:04:01 +02:00
Johannes Schindelin
963c127a05 Merge 'long-paths' into HEAD 2016-05-20 07:04:00 +02:00
Johannes Schindelin
67a9fff176 Merge 'fscache' into HEAD 2016-05-20 07:04:00 +02:00
nalla
eac5d406e0 asciidoctor: Fix giteveryday.txt to be built with asciidoctor.
When building the `doc` with `asciidoctor`, `asciidoctor` complains about
a nested code block in a callout list. This is a really dirty solution to
restore the callout list to function properly. There is a minimal visual
sideeffect; the *immitated* codeblock has no overall greyish background.
Instead the individual lines have it.

Note: When building this patch with `asciidoc` the background is totally
gone but the font is still monospaced.

Signed-off-by: nalla <nalla@hamal.uberspace.de>
2016-05-20 07:03:59 +02:00
nalla
e65d06e4b4 asciidoctor: Fix user-manual to be built by asciidoctor
The `user-manual.txt` ist designed as a `book` but the `Makefile` wants to
build it as an `article`. This seems to be a problem when building the
documentation with `asciidoctor`. Furthermore the parts *Git Glossary*
and *Apendix B* had no subsections which is not allowed when building with
`asciidoctor`. So lets add a *dummy* section.

Signed-off-by: nalla <nalla@hamal.uberspace.de>
2016-05-20 07:03:59 +02:00
Karsten Blees
287a611eb1 Win32: fix 'lstat("dir/")' with long paths
Use a suffciently large buffer to strip the trailing slash.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-05-20 07:03:58 +02:00
Karsten Blees
6cf7c02688 Win32: support long paths
Windows paths are typically limited to MAX_PATH = 260 characters, even
though the underlying NTFS file system supports paths up to 32,767 chars.
This limitation is also evident in Windows Explorer, cmd.exe and many
other applications (including IDEs).

Particularly annoying is that most Windows APIs return bogus error codes
if a relative path only barely exceeds MAX_PATH in conjunction with the
current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the
infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG.

Many Windows wide char APIs support longer than MAX_PATH paths through the
file namespace prefix ('\\?\' or '\\?\UNC\') followed by an absolute path.
Notable exceptions include functions dealing with executables and the
current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as
well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...).

Introduce a handle_long_path function to check the length of a specified
path properly (and fail with ENAMETOOLONG), and to optionally expand long
paths using the '\\?\' file namespace prefix. Short paths will not be
modified, so we don't need to worry about device names (NUL, CON, AUX).

Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be
limited to MAX_PATH (at least not on Win7), so we can use it to do the
heavy lifting of the conversion (translate '/' to '\', eliminate '.' and
'..', and make an absolute path).

Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH
limit.

Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs
that support long paths.

While improved error checking is always active, long paths support must be
explicitly enabled via 'core.longpaths' option. This is to prevent end
users to shoot themselves in the foot by checking out files that Windows
Explorer, cmd/bash or their favorite IDE cannot handle.

Test suite:
Test the case is when the full pathname length of a dir is close
to 260 (MAX_PATH).
Bug report and an original reproducer by Andrey Rogozhnikov:
https://github.com/msysgit/git/pull/122#issuecomment-43604199

[jes: adjusted test number to avoid conflicts, reinstated && chain]

Thanks-to: Martin W. Kirst <maki@bitkings.de>
Thanks-to: Doug Kelly <dougk.ff7@gmail.com>
Signed-off-by: Karsten Blees <blees@dcon.de>
Original-test-by: Andrey Rogozhnikov <rogozhnikov.andrey@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-05-20 07:03:58 +02:00
Johannes Schindelin
4db189fedf Win32: support long paths
Windows paths are typically limited to MAX_PATH = 260 characters, even
though the underlying NTFS file system supports paths up to 32,767 chars.
This limitation is also evident in Windows Explorer, cmd.exe and many
other applications (including IDEs).

Particularly annoying is that most Windows APIs return bogus error codes
if a relative path only barely exceeds MAX_PATH in conjunction with the
current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the
infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG.

Many Windows wide char APIs support longer than MAX_PATH paths through the
file namespace prefix ('\\?\' or '\\?\UNC\') followed by an absolute path.
Notable exceptions include functions dealing with executables and the
current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as
well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...).

Introduce a handle_long_path function to check the length of a specified
path properly (and fail with ENAMETOOLONG), and to optionally expand long
paths using the '\\?\' file namespace prefix. Short paths will not be
modified, so we don't need to worry about device names (NUL, CON, AUX).

Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be
limited to MAX_PATH (at least not on Win7), so we can use it to do the
heavy lifting of the conversion (translate '/' to '\', eliminate '.' and
'..', and make an absolute path).

Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH
limit.

Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs
that support long paths.

While improved error checking is always active, long paths support must be
explicitly enabled via 'core.longpaths' option. This is to prevent end
users to shoot themselves in the foot by checking out files that Windows
Explorer, cmd/bash or their favorite IDE cannot handle.

Test suite:
Test the case is when the full pathname length of a dir is close
to 260 (MAX_PATH).
Bug report and an original reproducer by Andrey Rogozhnikov:
https://github.com/msysgit/git/pull/122#issuecomment-43604199

[jes: adjusted test number to avoid conflicts]

Thanks-to: Martin W. Kirst <maki@bitkings.de>
Thanks-to: Doug Kelly <dougk.ff7@gmail.com>
Signed-off-by: Karsten Blees <blees@dcon.de>
Original-test-by: Andrey Rogozhnikov <rogozhnikov.andrey@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-05-20 07:03:57 +02:00
Doug Kelly
25a5ddd8e1 Add a test demonstrating a problem with long submodule paths
[jes: adusted test number to avoid conflicts, fixed non-portable use of
the 'export' statement]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-05-20 07:03:57 +02:00
Karsten Blees
1777fcebcf fscache: load directories only once
If multiple threads access a directory that is not yet in the cache, the
directory will be loaded by each thread. Only one of the results is added
to the cache, all others are leaked. This wastes performance and memory.

On cache miss, add a future object to the cache to indicate that the
directory is currently being loaded. Subsequent threads register themselves
with the future object and wait. When the first thread has loaded the
directory, it replaces the future object with the result and notifies
waiting threads.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-05-20 07:03:56 +02:00
Karsten Blees
8e3c68e4a9 Win32: add a cache below mingw's lstat and dirent implementations
Checking the work tree status is quite slow on Windows, due to slow lstat
emulation (git calls lstat once for each file in the index). Windows
operating system APIs seem to be much better at scanning the status
of entire directories than checking single files.

Add an lstat implementation that uses a cache for lstat data. Cache misses
read the entire parent directory and add it to the cache. Subsequent lstat
calls for the same directory are served directly from the cache.

Also implement opendir / readdir / closedir so that they create and use
directory listings in the cache.

The cache doesn't track file system changes and doesn't plug into any
modifying file APIs, so it has to be explicitly enabled for git functions
that don't modify the working copy.

Note: in an earlier version of this patch, the cache was always active and
tracked file system changes via ReadDirectoryChangesW. However, this was
much more complex and had negative impact on the performance of modifying
git commands such as 'git checkout'.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-05-20 07:03:56 +02:00
Karsten Blees
8b78f4f3de add infrastructure for read-only file system level caches
Add a macro to mark code sections that only read from the file system,
along with a config option and documentation.

This facilitates implementation of relatively simple file system level
caches without the need to synchronize with the file system.

Enable read-only sections for 'git status' and preload_index.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-05-20 07:03:56 +02:00
Karsten Blees
5e140838c2 Win32: make the lstat implementation pluggable
Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite
slow. Windows operating system APIs seem to be much better at scanning the
status of entire directories than checking single files. A caching
implementation may improve performance by bulk-reading entire directories
or reusing data obtained via opendir / readdir.

Make the lstat implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-05-20 07:03:55 +02:00
Karsten Blees
4615d6ab3f Win32: Make the dirent implementation pluggable
Emulating the POSIX dirent API on Windows via FindFirstFile/FindNextFile is
pretty staightforward, however, most of the information provided in the
WIN32_FIND_DATA structure is thrown away in the process. A more
sophisticated implementation may cache this data, e.g. for later reuse in
calls to lstat.

Make the dirent implementation pluggable so that it can be switched at
runtime, e.g. based on a config option.

Define a base DIR structure with pointers to readdir/closedir that match
the opendir implementation (i.e. similar to vtable pointers in OOP).
Define readdir/closedir so that they call the function pointers in the DIR
structure. This allows to choose the opendir implementation on a
call-by-call basis.

Move the fixed sized dirent.d_name buffer to the dirent-specific DIR
structure, as d_name may be implementation specific (e.g. a caching
implementation may just set d_name to point into the cache instead of
copying the entire file name string).

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-05-20 07:03:55 +02:00
Karsten Blees
4c3d912dfe Win32: dirent.c: Move opendir down
Move opendir down in preparation for the next patch.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-05-20 07:03:55 +02:00
Karsten Blees
4a21aac3ed Win32: make FILETIME conversion functions public
Signed-off-by: Karsten Blees <blees@dcon.de>
2016-05-20 07:03:55 +02:00
Pat Thoyts
5084a56b88 mingw: add tests for the hidden attribute on the git directory
With msysGit the .git directory is supposed to be hidden, unless it is
a bare git repository. Test this.

Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>
2016-05-20 07:03:50 +02:00
Johannes Schindelin
a3a903c71e When initializing .git/, record the current setting of core.hideDotFiles
This is on Windows only, of course.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-05-20 07:03:50 +02:00
Erik Faye-Lund
d6ba720ddf core.hideDotFiles: hide '.git' dir by default
At least for cross-platform projects, it makes sense to hide the
files starting with a dot, as this is the behavior on Unix/MacOSX.

However, at least Eclipse has problems interpreting the hidden flag
correctly, so the default is to hide only the .git/ directory.

The config setting core.hideDotFiles therefore supports not only
'true' and 'false', but also 'dotGitOnly'.

[jes: clarified the commit message, made git init respect the setting
by marking the .git/ directory only after reading the config, and added
documentation, and rebased on top of current junio/next]

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-05-20 07:03:49 +02:00
Johannes Schindelin
e5ae6293dc Start the merging-rebase to v2.8.3
This commit starts the rebase of 5433917 to 90b4299
2016-05-20 07:03:18 +02:00
Junio C Hamano
0f8e831356 Git 2.8.3
Signed-off-by: Junio C Hamano <gitster@pobox.com>
v2.8.3
2016-05-18 15:32:41 -07:00
Junio C Hamano
66106691a1 Merge branch 'sb/misc-cleanups' into HEAD
* sb/misc-cleanups:
  submodule-config: don't shadow `cache`
  config.c: drop local variable
  credential-cache, send_request: close fd when done
  bundle: don't leak an fd in case of early return
  abbrev_sha1_in_line: don't leak memory
  notes: don't leak memory in git_config_get_notes_strategy
2016-05-18 14:40:15 -07:00
Junio C Hamano
989cbd4556 Merge branch 'ew/doc-split-pack-disables-bitmap' into HEAD
Doc update.

* ew/doc-split-pack-disables-bitmap:
  pack-objects: warn on split packs disabling bitmaps
2016-05-18 14:40:15 -07:00
Junio C Hamano
10b6646afc Merge branch 'sb/clean-test-fix' into HEAD
* sb/clean-test-fix:
  t7300: mark test with SANITY
2016-05-18 14:40:14 -07:00
Junio C Hamano
8d61f0f07d Merge branch 'rn/glossary-typofix' into HEAD
* rn/glossary-typofix:
  Documentation: fix typo 'In such these cases'
2016-05-18 14:40:14 -07:00
Junio C Hamano
977cb3e2c5 Merge branch 'ew/normal-to-e' into HEAD
* ew/normal-to-e:
  .mailmap: update to my shorter email address
2016-05-18 14:40:12 -07:00
Junio C Hamano
258b862edb Merge branch 'sb/config-exit-status-list' into HEAD
Doc update.

* sb/config-exit-status-list:
  config doc: improve exit code listing
2016-05-18 14:40:12 -07:00
Junio C Hamano
87c594471d Merge branch 'rt/string-list-lookup-cleanup' into HEAD
Code cleanup.

* rt/string-list-lookup-cleanup:
  string_list: use string-list API in unsorted_string_list_lookup()
2016-05-18 14:40:12 -07:00
Junio C Hamano
9ba3b14c64 Merge branch 'jk/fix-attribute-macro-in-2.5' into HEAD
Code fixup.

* jk/fix-attribute-macro-in-2.5:
  remote.c: spell __attribute__ correctly
2016-05-18 14:40:12 -07:00
Junio C Hamano
777dec64bd Merge branch 'sg/test-lib-simplify-expr-away' into HEAD
Code cleanup.

* sg/test-lib-simplify-expr-away:
  test-lib: simplify '--option=value' parsing
2016-05-18 14:40:11 -07:00
Junio C Hamano
14af79b93d Merge branch 'nd/remove-unused' into HEAD
Code cleanup.

* nd/remove-unused:
  wrapper.c: delete dead function git_mkstemps()
  dir.c: remove dead function fnmatch_icase()
2016-05-18 14:40:11 -07:00
Junio C Hamano
13af774e26 Merge branch 'sk/gitweb-highlight-encoding' into HEAD
Some multi-byte encoding can have a backslash byte as a later part
of one letter, which would confuse "highlight" filter used in
gitweb.

* sk/gitweb-highlight-encoding:
  gitweb: apply fallback encoding before highlight
2016-05-18 14:40:10 -07:00
Junio C Hamano
09687585d1 Merge branch 'ls/travis-submitting-patches' into HEAD
* ls/travis-submitting-patches:
  Documentation: add setup instructions for Travis CI
2016-05-18 14:40:09 -07:00
Junio C Hamano
1cfb225aba Merge branch 'js/close-packs-before-gc' into HEAD
* js/close-packs-before-gc:
  t5510: run auto-gc in the foreground
2016-05-18 14:40:09 -07:00
Junio C Hamano
803fd70cee Merge branch 'ls/p4-lfs' into HEAD
Recent update to Git LFS broke "git p4" by changing the output from
its "lfs pointer" subcommand.

* ls/p4-lfs:
  git-p4: fix Git LFS pointer parsing
  travis-ci: express Linux/OS X dependency versions more clearly
  travis-ci: update Git-LFS and P4 to the latest version
2016-05-18 14:40:08 -07:00
Junio C Hamano
7ab6da3c40 Merge branch 'ls/p4-lfs-test-fix-2.7.0' into HEAD
Fix a broken test.

* ls/p4-lfs-test-fix-2.7.0:
  t9824: fix wrong reference value
  t9824: fix broken &&-chain in a subshell
2016-05-18 14:40:08 -07:00
Junio C Hamano
f735a50ffd Merge branch 'nf/mergetool-prompt' into HEAD
UI consistency improvements.

* nf/mergetool-prompt:
  difftool/mergetool: make the form of yes/no questions consistent
2016-05-18 14:40:07 -07:00
Junio C Hamano
1f7b196e21 Merge branch 'jd/send-email-to-whom' into HEAD
A question by "git send-email" to ask the identity of the sender
has been updated.

* jd/send-email-to-whom:
  send-email: fix grammo in the prompt that asks e-mail recipients
2016-05-18 14:40:07 -07:00
Junio C Hamano
f12fffd347 Merge branch 'js/win32-mmap' into HEAD
mmap emulation on Windows has been optimized and work better without
consuming paging store when not needed.

* js/win32-mmap:
  mmap(win32): avoid expensive fstat() call
  mmap(win32): avoid copy-on-write when it is unnecessary
  win32mmap: set errno appropriately
2016-05-18 14:40:06 -07:00
Junio C Hamano
c555e529ac Merge branch 'jk/push-client-deadlock-fix' into HEAD
Some Windows SDK lacks pthread_sigmask() implementation and fails
to compile the recently updated "git push" codepath that uses it.

* jk/push-client-deadlock-fix:
  Windows: only add a no-op pthread_sigmask() when needed
  Windows: add pthread_sigmask() that does nothing
  t5504: drop sigpipe=ok from push tests
  fetch-pack: isolate sigpipe in demuxer thread
  send-pack: isolate sigpipe in demuxer thread
  run-command: teach async threads to ignore SIGPIPE
  send-pack: close demux pipe before finishing async process
2016-05-18 14:40:06 -07:00
Junio C Hamano
920f2ea33b Merge branch 'sb/mv-submodule-fix' into HEAD
"git mv old new" did not adjust the path for a submodule that lives
as a subdirectory inside old/ directory correctly.

* sb/mv-submodule-fix:
  mv: allow moving nested submodules
2016-05-18 14:40:05 -07:00
Junio C Hamano
e9ef83a299 Merge branch 'da/user-useconfigonly' into HEAD
The "user.useConfigOnly" configuration variable makes it an error
if users do not explicitly set user.name and user.email.  However,
its check was not done early enough and allowed another error to
trigger, reporting that the default value we guessed from the
system setting was unusable.  This was a suboptimal end-user
experience as we want the users to set user.name/user.email without
relying on the auto-detection at all.

* da/user-useconfigonly:
  ident: give "please tell me" message upon useConfigOnly error
  ident: check for useConfigOnly before auto-detection of name/email
2016-05-18 14:40:05 -07:00
Junio C Hamano
787a490cee Merge branch 'ld/p4-test-py3' into HEAD
The test scripts for "git p4" (but not "git p4" implementation
itself) has been updated so that they would work even on a system
where the installed version of Python is python 3.

* ld/p4-test-py3:
  git-p4 tests: time_in_seconds should use $PYTHON_PATH
  git-p4 tests: work with python3 as well as python2
  git-p4 tests: cd to / before running python
2016-05-18 14:40:04 -07:00
Johannes Schindelin
ed84387a6b Windows: only add a no-op pthread_sigmask() when needed
In f924b52 (Windows: add pthread_sigmask() that does nothing,
2016-05-01), we introduced a no-op for Windows. However, this breaks
building Git in Git for Windows' SDK because pthread_sigmask() is
already a no-op there, #define'd in the pthread_signal.h header in
/mingw64/x86_64-w64-mingw32/include/.

Let's wrap the definition of pthread_sigmask() in a guard that skips
it when compiling with MinGW-w64' headers.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-11 14:02:10 -07:00
Junio C Hamano
d92347f59f Almost ready for 2.8.3
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-06 14:53:36 -07:00
Junio C Hamano
cc601901a7 Merge branch 'sb/submodule-path-misc-bugs' into maint
"git submodule" reports the paths of submodules the command
recurses into, but this was incorrect when the command was not run
from the root level of the superproject.

* sb/submodule-path-misc-bugs:
  t7407: make expectation as clear as possible
  submodule update: test recursive path reporting from subdirectory
  submodule update: align reporting path for custom command execution
  submodule status: correct path handling in recursive submodules
  submodule update --init: correct path handling in recursive submodules
  submodule foreach: correct path display in recursive submodules
2016-05-06 14:53:25 -07:00
Junio C Hamano
a0c9cf51c0 Merge branch 'ky/imap-send-openssl-1.1.0' into maint
Upcoming OpenSSL 1.1.0 will break compilation b updating a few APIs
we use in imap-send, which has been adjusted for the change.

* ky/imap-send-openssl-1.1.0:
  configure: remove checking for HMAC_CTX_cleanup
  imap-send: avoid deprecated TLSv1_method()
  imap-send: check NULL return of SSL_CTX_new()
  imap-send: use HMAC() function provided by OpenSSL
2016-05-06 14:53:24 -07:00
Junio C Hamano
8854ded7af Merge branch 'js/replace-edit-use-editor-configuration' into maint
"git replace -e" did not honour "core.editor" configuration.

* js/replace-edit-use-editor-configuration:
  replace --edit: respect core.editor
2016-05-06 14:53:24 -07:00
Junio C Hamano
b450a39bea Merge branch 'cc/apply' into maint
Minor code clean-up.

* cc/apply:
  builtin/apply: free patch when parse_chunk() fails
  builtin/apply: handle parse_binary() failure
  apply: remove unused call to free() in gitdiff_{old,new}name()
  builtin/apply: get rid of useless 'name' variable
2016-05-06 14:53:23 -07:00
Junio C Hamano
c75fb77d9a Merge branch 'kn/for-each-tag-branch' into maint
A minor documentation update.

* kn/for-each-tag-branch:
  for-each-ref: fix description of '--contains' in manpage
2016-05-06 14:53:23 -07:00