Commit Graph

57546 Commits

Author SHA1 Message Date
Karsten Blees
e118e9ecdc Unicode file name support (gitk and git-gui)
Assumes file names in git tree objects are UTF-8 encoded.

On most unix systems, the system encoding (and thus the TCL system
encoding) will be UTF-8, so file names will be displayed correctly.

On Windows, it is impossible to set the system encoding to UTF-8. Changing
the TCL system encoding (via 'encoding system ...', e.g. in the startup
code) is explicitly discouraged by the TCL docs.

Change gitk and git-gui functions dealing with file names to always convert
from and to UTF-8.

Signed-off-by: Karsten Blees <blees@dcon.de>
2016-03-18 01:09:01 +01:00
Johannes Schindelin
5c7b59c6cd mingw: do not bother to test funny file names
MSYS2 actually allows to create files or directories whose names contain
tabs, newlines or colors, even if plain Win32 API cannot access them.
As we are using an MSYS2 bash to run the tests, such files or
directories are created successfully, but Git itself has no chance to
work with them because it is a regular Windows program, hence limited by
the Win32 API.

With this change, on Windows otherwise failing tests in
t3300-funny-names.sh, t3600-rm.sh, t3703-add-magic-pathspec.sh,
t3902-quoted.sh, t4016-diff-quote.sh, t4135-apply-weird-filenames.sh,
t9200-git-cvsexportcommit.sh, and t9903-bash-prompt.sh are skipped.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:09:00 +01:00
Johannes Schindelin
e1158dbdea mingw: skip a test in t9130 that cannot pass on Windows
On Windows, Git itself has no clue about POSIX paths, but its shell
scripts do. In this instance, we get mixed paths as a result, and when
comparing the path of the author file, we get a mismatch that is
entirely due to the POSIX path vs Windows path clash.

Let's just skip this test so that t9130-git-svn-authors-file.sh passes
in Git for Windows' SDK.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:59 +01:00
Johannes Schindelin
3735288639 mingw: handle the missing POSIXPERM prereq in t9124
On Windows, the permission system works completely differently than
expected by some of the tests. So let's make sure that we do not test
POSIX functionality on Windows.

This lets t9124-git-svn-dcommit-auto-props.sh pass in Git for Windows'
SDK.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:59 +01:00
Johannes Schindelin
2bca7a815b mingw: avoid illegal filename in t9118
On Windows' file systems, file names with trailing dots are forbidden.
The POSIX emulation layer used by Git for Windows' Subversion emulates
those file names, therefore the test adding the file would actually
succeed, but when we would ask git.exe (which does not leverage the
POSIX emulation layer) to check out the tree, it would fail.

Let's just guard the test using a filename that is illegal on Windows
by the MINGW prereq.

This lets t9118-git-svn-funky-branch-names.sh pass in Git for Windows'
SDK.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:58 +01:00
Johannes Schindelin
520cdea9b6 mingw: mark t9100's test cases with appropriate prereqs
Many a test requires either POSIXPERM (to change the executable bit) or
SYMLINKS, and neither are available on Windows.

This lets t9100-git-svn-basic.sh pass in Git for Windows' SDK.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:58 +01:00
Pat Thoyts
0321f5d4e7 Avoid absolute path in t0008
The colon is used by check-ignore to separate paths from other output
values. If we use an absolute path, however, on Windows it will be
converted into a Windows path that very much contains a colon.

It is actually not at all necessary to make the path of the global
excludes absolute, so let's just not even do that.

Based on suggestions by Karsten Blees and Junio Hamano.

Suggested-by: Karsten Blees <karsten.blees@gmail.com>
Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:57 +01:00
Johannes Schindelin
3a5007fb15 mingw: work around pwd issues in the tests
In Git for Windows' SDK, the tests are run using a Bash that relies on
the POSIX emulation layer MSYS2 (itself a friendly fork of Cygwin). As
such, paths in tests can be POSIX paths. As soon as those paths are
passed to git.exe (which does *not* use the POSIX emulation layer),
those paths are converted into Windows paths, though. This happens
for command-line parameters, but not when reading, say, config variables.

To help with that, the `pwd` command is overridden to return the Windows
path of the current working directory when testing Git on Windows.

However, when talking to anything using the POSIX emulation layer, it is
really much better to use POSIX paths because Windows paths contain a
colon after the drive letter that will easily be mistaken for the common
separator in path lists.

So let's just use the $PWD variable when the POSIX path is needed.

This lets t7800-difftool.sh, t9400-git-cvsserver-server.sh,
t9402-git-cvsserver-refs.sh and t9401-git-cvsserver-crlf.sh pass in Git
for Windows' SDK.

Note: the cvsserver tests require not only the `cvs` package (install
it into Git for Windows' SDK via `pacman -S cvs`) but also the Perl
SQLite bindings (install them into Git for Windows' SDK via
`cpan DBD::SQLite`).

This patch is based on earlier work by 마누엘 and Karsten Blees.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:57 +01:00
Johannes Schindelin
d94c9b894f mingw: fix t9700's assumption about directory separators
This test assumed that there is only one directory separator (the
forward slash), not two equivalent directory separators.
However, on Windows, the back slash and the forward slash *are*
equivalent.

Let's paper over this issue by converting the backward slashes to
forward ones in the test that fails with MSYS2 otherwise.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:56 +01:00
Johannes Schindelin
00c0c92302 mingw: skip test in t1508 that fails due to path conversion
In Git for Windows, the MSYS2 POSIX emulation layer used by the Bash
converts command-line arguments that looks like they refer to a POSIX
path containing a file list (i.e. @<absolute-path>) into a Windows path
equivalent when calling non-MSYS2 executables, such as git.exe.

Let's just skip the test that uses the parameter `@/at-test` that
confuses the MSYS2 runtime.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:56 +01:00
Johannes Schindelin
883fd4e0e8 tests: turn off git-daemon tests if FIFOs are not available
The Git daemon tests create a FIFO first thing and will hang if said
FIFO is not available.

This is a problem with Git for Windows, where `mkfifo` is an MSYS2
program that leverages MSYS2's POSIX emulation layer, but
`git-daemon.exe` is a MINGW program that has not the first clue about
that POSIX emulation layer and therefore blinks twice when it sees
MSYS2's emulated FIFOs and then just stares into space.

This lets t5570-git-daemon.sh and t5811-proto-disable-git.sh pass.

Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:55 +01:00
Johannes Schindelin
00780054ed mingw: disable mkfifo-based tests
MSYS2 (the POSIX emulation layer used by Git for Windows' Bash) actually
has a working mkfifo. The only problem is that it is only emulating
named pipes through the MSYS2 runtime; The Win32 API has no idea about
named pipes, hence the Git executable cannot access those pipes either.

The symptom is that Git fails with a '<name>: No such file or directory'
because MSYS2 emulates named pipes through special-crafted '.lnk' files.

The solution is to tell the test suite explicitly that we cannot use
named pipes when we want to test on Windows.

This lets t4056-diff-order.sh, t9010-svn-fe.sh and t9300-fast-import.sh
pass.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:55 +01:00
Johannes Schindelin
3e555dbe9e mingw: accomodate t0060-path-utils for MSYS2
On Windows, there are no POSIX paths, only Windows ones (an absolute
Windows path looks like "C:\Program Files\Git\ReleaseNotes.html", under
most circumstances, forward slashes are also allowed and synonymous to
backslashes).

So when a POSIX shell (such as MSYS2's Bash, which is used by Git for
Windows to execute all those shell scripts that are part of Git) passes
a POSIX path to test-path-utils.exe (which is not POSIX-aware), the path
is translated into a Windows path. For example, /etc/profile becomes
C:/Program Files/Git/etc/profile.

This path translation poses a problem when passing the root directory as
parameter to test-path-utils.exe, as it is not well defined whether the
translated root directory should end in a slash or not. MSys1 stripped
the trailing slash, but MSYS2 does not.

Originally, the Git for Windows project patched MSYS2's runtime to
accomodate Git's regression test, but we really should do it the other
way round.

To work with both of MSys1's and MSYS2's behaviors, we simply test what
the current system does in the beginning of t0060-path-utils.sh and then
adjust the expected longest ancestor length accordingly.

It looks quite a bit tricky what we actually do in this patch: first, we
adjust the expected length for the trailing slash we did not originally
expect (subtracting one). So far, so good.

But now comes the part where things work in a surprising way: when the
expected length was 0, the prefix to match is the root directory. If the
root directory is converted into a path with a trailing slash, however,
we know that the logic in longest_ancestor_length() cannot match: to
avoid partial matches of the last directory component, it verifies that
the character after the matching prefix is a slash (but because the
slash was part of the matching prefix, the next character cannot be a
slash). So the return value is -1. Alas, this is exactly what the
expected length is after subtracting the value of $rootslash! So we skip
adding the $rootoff value in that case (and only in that case).

Directories other than the root directory are handled fine (as they are
specified without a trailing slash, something not possible for the root
directory, and MSYS2 converts them into Windows paths that also lack
trailing slashes), therefore we do not need any more special handling.

Thanks to Ray Donnelly for his patient help with this issue.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:54 +01:00
Johannes Schindelin
3c39fdf96e mingw: fix t5601-clone.sh
Since baaf233 (connect: improve check for plink to reduce false
positives, 2015-04-26), t5601 writes out a `plink.exe` for testing that
is actually a shell script. So the assumption that the `.exe` extension
implies that the file is *not* a shell script is now wrong.

Since there was no love for the idea of allowing `.exe` files to be
shell scripts on Windows, let's go the other way round: *make*
`plink.exe` a real `.exe`.

This fixes t5601-clone.sh in Git for Windows' SDK.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:54 +01:00
Johannes Schindelin
ad1637f115 mingw: let lstat() fail with errno == ENOTDIR when appropriate
POSIX semantics requires lstat() to fail with ENOTDIR when "[a]
component of the path prefix names an existing file that is neither a
directory nor a symbolic link to a directory".

See http://pubs.opengroup.org/onlinepubs/9699919799/functions/lstat.html

This behavior is expected by t1404-update-ref-df-conflicts now.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:53 +01:00
마누엘
1b518ce351 mingw: try to delete target directory before renaming
When the rename() function tries to move a directory it fails if the
target directory exists. It should check if it can delete the (possibly
empty) target directory and then try again to move the directory.

This partially fixes t9100-git-svn-basic.sh.

Signed-off-by: 마누엘 <nalla@hamal.uberspace.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:52 +01:00
Johannes Schindelin
15f1e222c2 mingw: prepare the TMPDIR environment variable for shell scripts
When shell scripts access a $TMPDIR variable containing backslashes,
they will be mistaken for escape characters. Let's not let that happen
by converting them to forward slashes.

This partially fixes t7800 with MSYS2.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:52 +01:00
Karsten Blees
cf1cfe8c08 mingw: factor out Windows specific environment setup
We will add more environment-related code to that new function
in the next patch.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:51 +01:00
Johannes Schindelin
1d21978ab2 Git.pm: stop assuming that absolute paths start with a slash
On Windows, absolute paths never start with a slash, unless a POSIX
emulation layer is used. The latter is the case for MSYS2's Perl that
Git for Windows leverages. However, in the tests we also go through
plain `git.exe`, which does *not* leverage the POSIX emulation layer,
and therefore the paths we pass to Perl may actually be DOS-style paths
such as C:/Program Files/Git.

So let's just use Perl's own way to test whether a given path is
absolute or not instead of home-brewing our own.

This patch partially fixes t7800 and t9700 when running in Git for
Windows' SDK.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:51 +01:00
Johannes Schindelin
ddbd46a123 mingw: do not trust MSYS2's MinGW gettext.sh
It does not quite work because it produces DOS line endings which the
shell does not like at all.

This lets t0200-gettext-basic.sh, t0204-gettext-reencode-sanity.sh,
t3406-rebase-message.sh, t3903-stash.sh, t7400-submodule-basic.sh,
t7401-submodule-summary.sh, t7406-submodule-update.sh and
t7407-submodule-foreach.sh pass in Git for Windows' SDK.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:50 +01:00
Johannes Schindelin
b9c9f48332 mingw: let's use gettext with MSYS2
This solves two problems:

- we now have proper localisation even on Windows

- we sidestep the infamous "BUG: your vsnprintf is broken (returned -1)"
  message when running "git init" (which otherwise prevents the entire
  test suite from running) because libintl.h overrides vsnprintf() with
  libintl_vsnprintf() [*1*]

The latter issue is rather crucial, as *no* test passes in Git for
Windows without this fix.

Footnote *1*: gettext_git=http://git.savannah.gnu.org/cgit/gettext.git
$gettext_git/tree/gettext-runtime/intl/libgnuintl.in.h#n380

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2016-03-18 01:08:50 +01:00
Johannes Schindelin
d0ea0b66c5 Merge branch 'js/msys2' into next
Beginning of the upstreaming process of Git for Windows effort.

* js/msys2:
  mingw: uglify (a, 0) definitions to shut up warnings
  mingw: squash another warning about a cast
  mingw: avoid warnings when casting HANDLEs to int
  mingw: avoid redefining S_* constants
  compat/winansi: support compiling with MSys2
  compat/mingw: support MSys2-based MinGW build
  nedmalloc: allow compiling with MSys2's compiler
  config.mak.uname: supporting 64-bit MSys2
  config.mak.uname: support MSys2
2016-03-18 01:08:49 +01:00
Johannes Schindelin
3a2c0a164c mingw: uglify (a, 0) definitions to shut up warnings
When the result of a (a, 0) expression is not used, MSys2's GCC version
finds it necessary to complain with a warning:

	right-hand operand of comma expression has no effect

Let's just pretend to use the 0 value and have a peaceful and quiet life
again.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-18 01:08:48 +01:00
Johannes Schindelin
acf2b8c1db mingw: squash another warning about a cast
MSys2's compiler is correct that casting a "void *" to a "DWORD" loses
precision, but in the case of pthread_exit() we know that the value
fits into a DWORD.

Just like casting handles to DWORDs, let's work around this issue by
casting to "intrptr_t" first, and immediately cast to the final type.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-18 01:08:48 +01:00
Johannes Schindelin
adf296f4f2 mingw: avoid warnings when casting HANDLEs to int
HANDLE is defined internally as a void *, but in many cases it is
actually guaranteed to be a 32-bit integer. In these cases, GCC should
not warn about a cast of a pointer to an integer of a different type
because we know exactly what we are doing.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-18 01:08:47 +01:00
Johannes Schindelin
97e13129a6 mingw: avoid redefining S_* constants
When compiling with MSys2's compiler, these constants are already defined.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-18 01:08:47 +01:00
Johannes Schindelin
bb850def7b compat/winansi: support compiling with MSys2
MSys2 already defines the _CONSOLE_FONT_INFOEX structure.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-18 01:08:46 +01:00
Johannes Schindelin
1b2a3f1b9e compat/mingw: support MSys2-based MinGW build
The excellent MSys2 project brings a substantially updated MinGW
environment including newer GCC versions and new headers. To support
compiling Git, let's special-case the new MinGW (tell-tale: the
_MINGW64_VERSION_MAJOR constant is defined).

Note: this commit only addresses compile failures, not compile warnings
(that task is left for a future patch).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-18 01:08:45 +01:00
Johannes Schindelin
eef1295fb4 nedmalloc: allow compiling with MSys2's compiler
With MSys2's GCC, `ReadWriteBarrier` is already defined, and FORCEINLINE
unfortunately gets defined incorrectly.

Let's work around both problems, using the MSys2-specific
__MINGW64_VERSION_MAJOR constant to guard the FORCEINLINE definition so
as not to affect other platforms.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-18 01:08:45 +01:00
Johannes Schindelin
abe6ddcc3e config.mak.uname: supporting 64-bit MSys2
This just makes things compile, the test suite needs extra tender loving
care in addition to this change. We will address these issues in later
commits.

While at it, also allow building MSys2 Git (i.e. a Git that uses MSys2's
POSIX emulation layer).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-18 01:08:44 +01:00
Johannes Schindelin
b51f74f678 config.mak.uname: support MSys2
For a long time, Git for Windows lagged behind Git's 2.x releases because
the Git for Windows developers wanted to let that big jump coincide with
a well-needed jump away from MSys to MSys2.

To understand why this is such a big issue, it needs to be noted that
many parts of Git are not written in portable C, but instead Git relies
on a POSIX shell and Perl to be available.

To support the scripts, Git for Windows has to ship a minimal POSIX
emulation layer with Bash and Perl thrown in, and when the Git for
Windows effort started in August 2007, this developer settled on using
MSys, a stripped down version of Cygwin. Consequently, the original name
of the project was "msysGit" (which, sadly, caused a *lot* of confusion
because few Windows users know about MSys, and even less care).

To compile the C code of Git for Windows, MSys was used, too: it sports
two versions of the GNU C Compiler: one that links implicitly to the
POSIX emulation layer, and another one that targets the plain Win32 API
(with a few convenience functions thrown in).  Git for Windows'
executables are built using the latter, and therefore they are really
just Win32 programs. To discern executables requiring the POSIX
emulation layer from the ones that do not, the latter are called MinGW
(Minimal GNU for Windows) when the former are called MSys executables.

This reliance on MSys incurred challenges, too, though: some of our
changes to the MSys runtime -- necessary to support Git for Windows
better -- were not accepted upstream, so we had to maintain our own
fork. Also, the MSys runtime was not developed further to support e.g.
UTF-8 or 64-bit, and apart from lacking a package management system
until much later (when mingw-get was introduced), many packages provided
by the MSys/MinGW project lag behind the respective source code
versions, in particular Bash and OpenSSL. For a while, the Git for
Windows project tried to remedy the situation by trying to build newer
versions of those packages, but the situation quickly became untenable,
especially with problems like the Heartbleed bug requiring swift action
that has nothing to do with developing Git for Windows further.

Happily, in the meantime the MSys2 project (https://msys2.github.io/)
emerged, and was chosen to be the base of the Git for Windows 2.x. Just
like MSys, MSys2 is a stripped down version of Cygwin, but it is
actively kept up-to-date with Cygwin's source code.  Thereby, it already
supports Unicode internally, and it also offers the 64-bit support that
we yearned for since the beginning of the Git for Windows project.

MSys2 also ported the Pacman package management system from Arch Linux
and uses it heavily. This brings the same convenience to which Linux
users are used to from `yum` or `apt-get`, and to which MacOSX users are
used to from Homebrew or MacPorts, or BSD users from the Ports system,
to MSys2: a simple `pacman -Syu` will update all installed packages to
the newest versions currently available.

MSys2 is also *very* active, typically providing package updates
multiple times per week.

It still required a two-month effort to bring everything to a state
where Git's test suite passes, many more months until the first official
Git for Windows 2.x was released, and a couple of patches still await
their submission to the respective upstream projects. Yet without MSys2,
the modernization of Git for Windows would simply not have happened.

This commit lays the ground work to supporting MSys2-based Git builds.

Assisted-by: Waldek Maleska <weakcamel@users.github.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-18 01:08:44 +01:00
Johannes Schindelin
5463f646a3 Start the merging-rebase to v2.7.4
This commit starts the rebase of 945fda2 to 8a23e77
2016-03-18 01:08:42 +01:00
Junio C Hamano
937978e0f3 Git 2.7.4
Signed-off-by: Junio C Hamano <gitster@pobox.com>
v2.7.4
2016-03-17 11:32:13 -07:00
Junio C Hamano
8e9cc5f3e2 Sync with Git 2.6.6
* maint-2.6:
  Git 2.6.6
  Git 2.5.5
  Git 2.4.11
2016-03-17 11:28:52 -07:00
Junio C Hamano
e46579643d Git 2.6.6
Signed-off-by: Junio C Hamano <gitster@pobox.com>
v2.6.6
2016-03-17 11:26:41 -07:00
Junio C Hamano
ce4d4e763c Merge branch 'maint-2.5' into maint-2.6
* maint-2.5:
  Git 2.5.5
  Git 2.4.11
  list-objects: pass full pathname to callbacks
  list-objects: drop name_path entirely
  list-objects: convert name_path to a strbuf
  show_object_with_name: simplify by using path_name()
  http-push: stop using name_path
  tree-diff: catch integer overflow in combine_diff_path allocation
  add helpers for detecting size_t overflow
2016-03-17 11:26:18 -07:00
Junio C Hamano
e568e563ad Git 2.5.5
Signed-off-by: Junio C Hamano <gitster@pobox.com>
v2.5.5
2016-03-17 11:24:59 -07:00
Junio C Hamano
c638f3e4d5 Merge branch 'maint-2.4' into maint-2.5
* maint-2.4:
  Git 2.4.11
  list-objects: pass full pathname to callbacks
  list-objects: drop name_path entirely
  list-objects: convert name_path to a strbuf
  show_object_with_name: simplify by using path_name()
  http-push: stop using name_path
  tree-diff: catch integer overflow in combine_diff_path allocation
  add helpers for detecting size_t overflow
2016-03-17 11:24:14 -07:00
Junio C Hamano
765428699a Git 2.4.11
Signed-off-by: Junio C Hamano <gitster@pobox.com>
v2.4.11
2016-03-17 11:23:05 -07:00
Junio C Hamano
32c6dca8c4 Merge branch 'jk/path-name-safety-2.4' into maint-2.4
Bugfix patches were backported from the 'master' front to plug heap
corruption holes, to catch integer overflow in the computation of
pathname lengths, and to get rid of the name_path API.  Both of
these would have resulted in writing over an under-allocated buffer
when formulating pathnames while tree traversal.

* jk/path-name-safety-2.4:
  list-objects: pass full pathname to callbacks
  list-objects: drop name_path entirely
  list-objects: convert name_path to a strbuf
  show_object_with_name: simplify by using path_name()
  http-push: stop using name_path
  tree-diff: catch integer overflow in combine_diff_path allocation
  add helpers for detecting size_t overflow
2016-03-17 11:22:24 -07:00
Junio C Hamano
d79db92483 Merge branch 'jk/path-name-safety-2.7' into maint
* jk/path-name-safety-2.7:
  list-objects: pass full pathname to callbacks
  list-objects: drop name_path entirely
  list-objects: convert name_path to a strbuf
  show_object_with_name: simplify by using path_name()
  http-push: stop using name_path
  tree-diff: catch integer overflow in combine_diff_path allocation
  add helpers for detecting size_t overflow
2016-03-16 13:15:04 -07:00
Junio C Hamano
55c45a7325 Merge branch 'jk/path-name-safety-2.6' into jk/path-name-safety-2.7
* jk/path-name-safety-2.6:
  list-objects: pass full pathname to callbacks
  list-objects: drop name_path entirely
  list-objects: convert name_path to a strbuf
  show_object_with_name: simplify by using path_name()
  http-push: stop using name_path
  tree-diff: catch integer overflow in combine_diff_path allocation
  add helpers for detecting size_t overflow
2016-03-16 10:42:32 -07:00
Junio C Hamano
717e3551b9 Merge branch 'jk/path-name-safety-2.5' into jk/path-name-safety-2.6
* jk/path-name-safety-2.5:
  list-objects: pass full pathname to callbacks
  list-objects: drop name_path entirely
  list-objects: convert name_path to a strbuf
  show_object_with_name: simplify by using path_name()
  http-push: stop using name_path
  tree-diff: catch integer overflow in combine_diff_path allocation
  add helpers for detecting size_t overflow
2016-03-16 10:42:02 -07:00
Junio C Hamano
253ce7a15c Merge branch 'jk/path-name-safety-2.4' into jk/path-name-safety-2.5
* jk/path-name-safety-2.4:
  list-objects: pass full pathname to callbacks
  list-objects: drop name_path entirely
  list-objects: convert name_path to a strbuf
  show_object_with_name: simplify by using path_name()
  http-push: stop using name_path
  tree-diff: catch integer overflow in combine_diff_path allocation
  add helpers for detecting size_t overflow
2016-03-16 10:41:43 -07:00
Jeff King
2824e1841b list-objects: pass full pathname to callbacks
When we find a blob at "a/b/c", we currently pass this to
our show_object_fn callbacks as two components: "a/b/" and
"c". Callbacks which want the full value then call
path_name(), which concatenates the two. But this is an
inefficient interface; the path is a strbuf, and we could
simply append "c" to it temporarily, then roll back the
length, without creating a new copy.

So we could improve this by teaching the callsites of
path_name() this trick (and there are only 3). But we can
also notice that no callback actually cares about the
broken-down representation, and simply pass each callback
the full path "a/b/c" as a string. The callback code becomes
even simpler, then, as we do not have to worry about freeing
an allocated buffer, nor rolling back our modification to
the strbuf.

This is theoretically less efficient, as some callbacks
would not bother to format the final path component. But in
practice this is not measurable. Since we use the same
strbuf over and over, our work to grow it is amortized, and
we really only pay to memcpy a few bytes.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-16 10:41:04 -07:00
Jeff King
dc06dc8800 list-objects: drop name_path entirely
In the previous commit, we left name_path as a thin wrapper
around a strbuf. This patch drops it entirely. As a result,
every show_object_fn callback needs to be adjusted. However,
none of their code needs to be changed at all, because the
only use was to pass it to path_name(), which now handles
the bare strbuf.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-16 10:41:03 -07:00
Jeff King
f3badaed51 list-objects: convert name_path to a strbuf
The "struct name_path" data is examined in only two places:
we generate it in process_tree(), and we convert it to a
single string in path_name(). Everyone else just passes it
through to those functions.

We can further note that process_tree() already keeps a
single strbuf with the leading tree path, for use with
tree_entry_interesting().

Instead of building a separate name_path linked list, let's
just use the one we already build in "base". This reduces
the amount of code (especially tricky code in path_name()
which did not check for integer overflows caused by deep
or large pathnames).

It is also more efficient in some instances.  Any time we
were using tree_entry_interesting, we were building up the
strbuf anyway, so this is an immediate and obvious win
there. In cases where we were not, we trade off storing
"pathname/" in a strbuf on the heap for each level of the
path, instead of two pointers and an int on the stack (with
one pointer into the tree object). On a 64-bit system, the
latter is 20 bytes; so if path components are less than that
on average, this has lower peak memory usage.  In practice
it probably doesn't matter either way; we are already
holding in memory all of the tree objects leading up to each
pathname, and for normal-depth pathnames, we are only
talking about hundreds of bytes.

This patch leaves "struct name_path" as a thin wrapper
around the strbuf, to avoid disrupting callbacks. We should
fix them, but leaving it out makes this diff easier to view.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-16 10:41:03 -07:00
Jeff King
8eee9f9277 show_object_with_name: simplify by using path_name()
When "git rev-list" shows an object with its associated path
name, it does so by walking the name_path linked list and
printing each component (stopping at any embedded NULs or
newlines).

We'd like to eventually get rid of name_path entirely in
favor of a single buffer, and dropping this custom printing
code is part of that. As a first step, let's use path_name()
to format the list into a single buffer, and print that.
This is strictly less efficient than the original, but it's
a temporary step in the refactoring; our end game will be to
get the fully formatted name in the first place.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-16 10:41:03 -07:00
Jeff King
c6bd2a1dec http-push: stop using name_path
The graph traversal code here passes along a name_path to
build up the pathname at which we find each blob. But we
never actually do anything with the resulting names, making
it a waste of code and memory.

This usage came in aa1dbc9 (Update http-push functionality,
2006-03-07), and originally the result was passed to
"add_object" (which stored it, but didn't really use it,
either). But we stopped using that function in 1f1e895 (Add
"named object array" concept, 2006-06-19) in favor of
storing just the objects themselves.

Moreover, the generation of the name in process_tree() is
buggy. It sticks "name" onto the end of the name_path linked
list, and then passes it down again as it recurses (instead
of "entry.path"). So it's a good thing this was unused, as
the resulting path for "a/b/c/d" would end up as "a/a/a/a".

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-16 10:41:02 -07:00
Jeff King
d770187872 tree-diff: catch integer overflow in combine_diff_path allocation
A combine_diff_path struct has two "flex" members allocated
alongside the struct: a string to hold the pathname, and an
array of parent pointers. We use an "int" to compute this,
meaning we may easily overflow it if the pathname is
extremely long.

We can fix this by using size_t, and checking for overflow
with the st_add helper.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-16 10:41:02 -07:00