In my previous fixup, I forgot to close(fd)... D'oh. Squashed one bug,
introduced another...
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Direct Git for Windows enhancements to their contributions page.
Space out the text for easy reading.
Also clarify that the mailing list is plain text, no HTML.
Signed-off-by: Philip Oakley <philipoakley@iee.org>
It is unlikely that the fstat() call fails, but not impossible...
Coverity pointed out that we check for its return value elsewhere, so we
might just as well do it here, too.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The git/ directory should already exist when we write that .targets
file... and then we should also add it explicitly...
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It was discovered that the current make_environment_block()
implementation was fixed incorrectly in eaae98ef59 (fixup! msvc:
convert environment from/to UTF-16 on the fly, 2017-02-28): the total
number of inserts was not adjusted when deltaenv tried to set a variable
and then immediately delete it. This could result in an access
violation, as we later looked through all keys slated to be inserted.
This bug (and its age) is a pretty good indicator that the code may not
have been all that obvious. So let's rewrite make_environment_block()
from scratch.
Let's replace it by a new implementation that
1) accumulates all environment variables into an array,
2) appends the deltaenv,
3) applies a stable sort, and finally
4) copies the entries (but only one for any given key)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The buffer returned by _wenviron (which looks like a constant, but is
not) is not thread-safe.
This caused debug assertions when running t7814-grep-recurse-submodules
with MSVC-built code (but only something like 9 out of 10 times).
Let's use the thread-safe GetEnvironmentStringsW() function instead
(which requires an accompanying call to FreeEnvironmentStringsW()). It
is a little bit more expensive because it needs to lock and it needs to
copy the entire environment block. But spawning a new process is *far*
more expensive anyway.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Earlier in this codepath, we (ab)used "%<len>c" to read the hostname
recorded in the lockfile into locking_host[HOST_NAME_MAX + 1] while
substituting <len> with the actual value of HOST_NAME_MAX.
This turns out to be incorrect, as it is an instruction to read
exactly the specified number of bytes. Because we are trying to
read at most that many bytes, we should be using "%<len>s" instead.
[jes: this is a backport of afe2fab72c (gc: call fscanf() with %<len>s,
not %<len>c, when reading hostname, 2017-09-17) to the vs2017_vcpkg
branch, as it turns out that MSVC's fscanf() is more correct than GNU
libc's, and returns only 1 correctly scanned field instead of the
2 expected by Git, making t6500-gc.sh fail.]
Helped-by: A. Wilcox <awilfox@adelielinux.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The problem with not having, say, git-receive-pack.exe after a full
build is that the test suite will then happily use the *installed*
git-receive-pack.exe because it finds nothing else.
Absolutely not what we want. We want to have confidence that our test
covers the MSVC-built Git executables, and not some random stuff.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The idea of the `vcxproj` target is to generate .sln/.vcxproj files and
then commit them, to be used elsewhere. Typically, this is done in a
VSTS job whenever `master` changes. So there is little use in
initializing vcpkg and building all the dependencies: they are not
necessary here.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The `vcxproj` target does not, in fact, depend on MSVC being defined, so
let's just move it outside of that block.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We just introduced a way to build Git for Windows with MSVC on the
command line using vcpkg-generated, up-to-date dependencies. Let's bring
that convenience to the Visual Studio project, too.
(The previous method, fetching NuGet packages, is fraught with problems:
as C++ libraries have to be built for every architecture and for every
toolset, the NuGet packages which we would like to consume fell behind
and are not up-to-date with the current versions of the libraries, e.g.
cURL and OpenSSL. By using vcpkg we avoid that problem, always building
the newest dependency versions.)
The trick is to initialize the VCPKG system once, and then build Git's
dependencies using it. We do that by attaching a pre-build event to the
libgit project (which is now the base project on which all others
depend, therefore no other project is built in paralleli, side-stepping
issues with vcpkg being unprepared for being run in parallel).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Fix regression described in:
https://github.com/git-for-windows/git/issues/1392
which was introduced in:
b2353379bb
Problem Symptoms
================
When the user has a .gitignore file that is a symlink, the fscache
optimization introduced above caused the stat-data from the symlink,
rather that of the target file, to be returned. Later when the ignore
file was read, the buffer length did not match the stat.st_size field
and we called die("cannot use <path> as an exclude file")
Optimization Rationale
======================
The above optimization calls lstat() before open() primarily to ask
fscache if the file exists. It gets the current stat-data as a side
effect essentially for free (since we already have it in memory).
If the file does not exist, it does not need to call open(). And
since very few directories have .gitignore files, we can greatly
reduce time spent in the filesystem.
Discussion of Fix
=================
The above optimization calls lstat() rather than stat() because the
fscache only intercepts lstat() calls. Calls to stat() stay directed
to the mingw_stat() completly bypassing fscache. Furthermore, calls
to mingw_stat() always call {open, fstat, close} so that symlinks are
properly dereferenced, which adds *additional* open/close calls on top
of what the original code in dir.c is doing.
Since the problem only manifests for symlinks, we add code to overwrite
the stat-data when the path is a symlink. This preserves the effect of
the performance gains provided by the fscache in the normal case.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Make fscache_enabled() function public rather than static.
Remove unneeded fscache_is_enabled() function.
Change is_fscache_enabled() macro to call fscache_enabled().
is_fscache_enabled() now takes a pathname so that the answer
is more precise and mean "is fscache enabled for this pathname",
since fscache only stores repo-relative paths and not absolute
paths, we can avoid attempting lookups for absolute paths.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Since picking up a33fc72fe9 (read-cache: force_verify_index_checksum,
2017-04-14), we do not even check the SHA-1 of the index by default
anymore, therefore we no longer need this workaround.
Noticed by Ben Peart.
This reverts commit 480829f131.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The .vcxproj's text nodes do not actually need to URL-encode double
quotes. So let's not do that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It really does depend on libgit. It does not hurt to let it depend on
xdiff, and it makes the code simpler.
It is necessary to get this dependency chain right, because we will
introduce a change where the vcpkg system is initialized before building
libgit. The vcpkg system will then build the dependencies needed by Git
(and thereby make the include headers available):
As the vcpkg system cannot be run in parallel (it does not lock,
wreaking havoc with files being accessed and written at the same time,
letting the vcpkg processes stumble over each others' toes. We prevent
that by ensuring that only one project is built at first: libgit. And
this project's PreBuildEvent will be used to initialize vcpkg and build
all dependencies. Subsequently, the other projects can be built in
parallel.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
As we do not consume NuGet packages any longer, there is no sense to try
to point PATH to their unpacked .dll files, either.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Dependencies such as cURL and OpenSSL are necessary to build and run
Git. Previously, we obtained those dependencies by fetching NuGet
packages.
However, it is notoriously hard to keep NuGet packages of C/C++
libraries up-to-date, as the toolsets for different Visual Studio
versions are different, and the NuGet packages would have to ship them
all.
That is the reason why the NuGet packages we use are quite old, and even
insecure in the case of cURL and OpenSSL (the versions contain known
security flaws that have been addressed by later versions for which no
NuGet packages are available).
The better way to handle this situation is to use the vcpkg system:
https://github.com/Microsoft/vcpkg
The idea is that a single Git repository contains enough supporting
files to build up-to-date versions of a large number of Open Source
libraries on demand, including cURL and OpenSSL.
We integrate this system via four new .bat files to
1) initialize the vcpkg system,
2) build the packages,
4) set up Git's Makefile system to find the build artifacts, and
3) copy the artifacts into the top-level directory
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The script assumes that we're in the top-level directory of the
checkout. That does not need to be true.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
One time too many did this developer call the `generate` script passing
a `--make-out=<PATH>` option that was happily ignored (because there
should be a space, not an equal sign, between `--make-out` and the
path).
And one time too many, this script not only ignored it but did not even
complain. Let's fix that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The hashmap API is just complicated enough that even at least one
long-time Git contributor has to look up how to use it every time he
finds a new use case. When that happens, it is really useful if the
provided example code is correct...
While at it, "fix a memory leak", avoid statements before variable
declarations, fix a const -> no-const cast, several %l specifiers (which
want to be %ld), avoid using an undefined constant, call scanf()
correctly, use FLEX_ALLOC_STR() where appropriate, and adjust the style
here and there.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The "--format=..." option "git for-each-ref" takes learned to show
the name of the 'remote' repository and the ref at the remote side
that is affected for 'upstream' and 'push' via "%(push:remotename)"
and friends.
* js/for-each-ref-remote-name-and-ref:
for-each-ref: test :remotename and :remoteref
for-each-ref: let upstream/push report the remote ref name
for-each-ref: let upstream/push optionally report the remote name
This is the final iteration that made it into upstream Git's master
branch and that started out as 8148ee40ad (Merge branch
'mingw/ref-filter-remote-name', 2017-10-29).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When I run git fetch, git tries to find object for each local and remote
refs. Without specifying OBJECT_INFO_QUICK, has_object_file list up
entries in pack directory for each calls.
This patch makes git fetch fast for the repositories having large number
of refs, especially for windows because it's directory list up api is
much slower than linux.
Note: this patch was developed independently also by Jeff King:
https://public-inbox.org/git/20171120202920.7ppcwmzkxifywtoj@sigill.intra.peff.net/
Signed-off-by: Takuto Ikuta <tikuta@chromium.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This not only prevents regressions, but also serves as documentation
what this new feature is expected to do.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are times when scripts want to know not only the name of the
push branch on the remote, but also the name of the branch as known
by the remote repository.
An example of this is when a tool wants to push to the very same branch
from which it would pull automatically, i.e. the `<remote>` and the `<to>`
in `git push <remote> <from>:<to>` would be provided by
`%(upstream:remotename)` and `%(upstream:remoteref)`, respectively.
This patch offers the new suffix :remoteref for the `upstream` and `push`
atoms, allowing to show exactly that. Example:
$ cat .git/config
...
[remote "origin"]
url = https://where.do.we.come/from
fetch = refs/heads/*:refs/remote/origin/*
[branch "master"]
remote = origin
merge = refs/heads/master
[branch "develop/with/topics"]
remote = origin
merge = refs/heads/develop/with/topics
...
$ git for-each-ref \
--format='%(push) %(push:remoteref)' \
refs/heads
refs/remotes/origin/master refs/heads/master
refs/remotes/origin/develop/with/topics refs/heads/develop/with/topics
Signed-off-by: J Wyman <jwyman@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are times when e.g. scripts want to know not only the name of the
upstream branch on the remote repository, but also the name of the
remote.
This patch offers the new suffix :remotename for the upstream and for
the push atoms, allowing to show exactly that. Example:
$ cat .git/config
...
[remote "origin"]
url = https://where.do.we.come/from
fetch = refs/heads/*:refs/remote/origin/*
[remote "hello-world"]
url = https://hello.world/git
fetch = refs/heads/*:refs/remote/origin/*
pushURL = hello.world:git
push = refs/heads/*:refs/heads/*
[branch "master"]
remote = origin
pushRemote = hello-world
...
$ git for-each-ref \
--format='%(upstream) %(upstream:remotename) %(push:remotename)' \
refs/heads/master
refs/remotes/origin/master origin hello-world
The implementation chooses *not* to DWIM the push remote if no explicit
push remote was configured; The reason is that it is possible to DWIM this
by using
%(if)%(push:remotename)%(then)
%(push:remotename)
%(else)
%(upstream:remotename)
%(end)
while it would be impossible to "un-DWIM" the information in case the
caller is really only interested in explicit push remotes.
While `:remote` would be shorter, it would also be a bit more ambiguous,
and it would also shut the door e.g. for `:remoteref` (which would
obviously refer to the corresponding ref in the remote repository).
Note: the dashless, non-CamelCased form `:remotename` follows the
example of the `:trackshort` example.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The option is deprecated now, and we better make sure that keeps saying
so until we finally remove it.
Suggested by Kevin Willford.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Teach read_directory_recursive() and add_excludes() to
be aware of optional fscache and avoid trying to open()
and fstat() non-existant ".gitignore" files in every
directory in the worktree.
The current code in add_excludes() calls open() and then
fstat() for a ".gitignore" file in each directory present
in the worktree. Change that when fscache is enabled to
call lstat() first and if present, call open().
This seems backwards because both lstat needs to do more
work than fstat. But when fscache is enabled, fscache will
already know if the .gitignore file exists and can completely
avoid the IO calls. This works because of the lstat diversion
to mingw_lstat when fscache is enabled.
This reduced status times on a 350K file enlistment of the
Windows repo on a NVMe SSD by 0.25 seconds.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
It was a bad idea to just remove that option from Git for Windows
v2.15.0, as early users of that (still experimental) option would have
been puzzled what they are supposed to do now.
So let's reintroduce the flag, but make sure to show the user good
advice how to fix this going forward.
We'll remove this option in a more orderly fashion either in v2.16.0 or
in v2.17.0.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We used to install into $HOME/bin/, which wreaks havoc with installed
versions of Git for Windows (because $HOME/bin is *prepended* to the
PATH, hence would override `git.exe` in Git Bash).
Let's align the MSVC case with the non-MSVC case and install into
/mingw64/bin/ (or /mingw32/bin/ in 32-bit Git for Windows SDKs) instead.
Noticed by Derrick Stolee.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This is an early version of the latest iteration of what used to
be 08f088c809 (Merge branch 'show-ignored-directory', 2017-10-23),
i.e. support for the then-experimental, now-dropped
`--show-ignored-directory` option.
The set of paths output from "git status --ignored" was tied
closely with its "--untracked=<mode>" option, but now it can be
controlled more flexibly. Most notably, a directory that is
ignored because it is listed to be ignored in the ignore/exclude
mechanism can be handled differently from a directory that ends up
to be ignored only because all files in it are ignored.
* jm/status-ignored-files-list:
status: test ignored modes
status: document options to show matching ignored files
status: report matching ignored and normal untracked
status: add option to show ignored files differently
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Add tests around status reporting ignord files that match an exclude
pattern for both --untracked-files=normal and --untracked-files=all.
Signed-off-by: Jameson Miller <jamill@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach status command to handle `--ignored=matching` with
`--untracked-files=normal`
Signed-off-by: Jameson Miller <jamill@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach the status command more flexibility in how ignored files are
reported. Currently, the reporting of ignored files and untracked
files are linked. You cannot control how ignored files are reported
independently of how untracked files are reported (i.e. `all` vs
`normal`). This makes it impossible to show untracked files with the
`all` option, but show ignored files with the `normal` option.
This work 1) adds the ability to control the reporting of ignored
files independently of untracked files and 2) introduces the concept
of status reporting ignored paths that explicitly match an ignored
pattern. There are 2 benefits to these changes: 1) if a consumer needs
all untracked files but not all ignored files, there is a performance
benefit to not scanning all contents of an ignored directory and 2)
returning ignored files that explicitly match a path allow a consumer
to make more informed decisions about when a status result might be
stale.
This commit implements --ignored=matching with --untracked-files=all.
The following commit will implement --ignored=matching with
--untracked=files=normal.
As an example of where this flexibility could be useful is that our
application (Visual Studio) runs the status command and presents the
output. It shows all untracked files individually (e.g. using the
'--untracked-files==all' option), and would like to know about which
paths are ignored. It uses information about ignored paths to make
decisions about when the status result might have changed.
Additionally, many projects place build output into directories inside
a repository's working directory (e.g. in "bin/" and "obj/"
directories). Normal usage is to explicitly ignore these 2 directory
names in the .gitignore file (rather than or in addition to the *.obj
pattern).If an application could know that these directories are
explicitly ignored, it could infer that all contents are ignored as
well and make better informed decisions about files in these
directories. It could infer that any changes under these paths would
not affect the output of status. Additionally, there can be a
significant performance benefit by avoiding scanning through ignored
directories.
When status is set to report matching ignored files, it has the
following behavior. Ignored files and directories that explicitly
match an exclude pattern are reported. If an ignored directory matches
an exclude pattern, then the path of the directory is returned. If a
directory does not match an exclude pattern, but all of its contents
are ignored, then the contained files are reported instead of the
directory.
Signed-off-by: Jameson Miller <jamill@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A very common assumption in Git's source code base is that
offset_1st_component() returns either 0 for relative paths, or 1 for
absolute paths that start with a slash. In other words, the return value
is either 0 or points just after the dir separator.
This assumption is not fulfilled when calling offset_1st_component()
e.g. on UNC paths on Windows, e.g. "//my-server/my-share". In this case,
offset_1st_component() returns the length of the entire string (which is
correct, because stripping the last "component" would not result in a
valid directory), yet the return value still does not point just after a
dir separator.
This assumption is most prominently seen in the
setup_git_directory_gently_1() function, where we want to append a
".git" component and simply assume that there is already a dir
separator. In the UNC example given above, this assumption is incorrect.
As a consequence, Git will fail to handle a worktree at the top of a UNC
share correctly.
Let's fix this by adding a dir separator specifically for that case: we
found that there is no first component in the path and it does not end
in a dir separator? Then add it.
This fixes https://github.com/git-for-windows/git/issues/1320
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It was observed that the current implementation of of get_proc_addr()
fails to load the kernel32.dll with code ERROR_INVALID_PARAMETER.
Probably the reason is that kernel32.dll is already loaded. The
behavior was seen at Windows SP1, both 32bit and 64bit. Probably it
would behave same way in some or all other Windows versions.
This breaks all usages of "clone --local", including the automatic
tests where they call it.
The function CreateHardLink is available in all supported Windows
versions (since Windows XP), so there is no more need to resolve it
in runtime.
Signed-off-by: Max Kirillov <max@max630.net>
When running an external diff from, say, a diff tool, it is safe to
assume that we want to write the files in question. On Windows, that
means that there cannot be any other process holding an open handle to
said files.
So let's make sure that `git diff` itself is not holding any open handle
to the files in question.
This fixes https://github.com/git-for-windows/git/issues/1315
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We have this loop where we try to remove the read-only attribute when
rename() fails and try again. If it fails again, let's not try to remove
the read-only attribute and try *again*.
This fixes https://github.com/git-for-windows/git/issues/1299
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
