Some tests assumed "iconv" is available without honoring ICONV
prerequisite, which has been corrected.
* ps/tests-wo-iconv-fixes:
t6006: don't use iconv(1) without ICONV prereq
t5550: add ICONV prereq to tests that use "$HTTPD_URL/error"
t4205: improve handling of ICONV prerequisite
t40xx: don't use iconv(1) without ICONV prereq
t: don't set ICONV prereq when iconv(1) is missing
CI update.
* ps/ci-gitlab-msvc-updates:
gitlab-ci: handle failed tests on MSVC+Meson job
gitlab-ci: use "run-test-slice-meson.sh"
ci: make test slicing consistent across Meson/Make
github: fix Meson tests not executing at all
meson: fix MERGE_TOOL_DIR with "--no-bin-wrappers"
ci: don't skip smallest test slice in GitLab
ci: handle failures of test-slice helper
It does not make much sense to apply the "incomplete-line"
whitespace rule to symbolic links, whose contents almost always
lack the final newline. "git apply" and "git diff" are now taught
to exclude them for a change to symbolic links.
* jc/whitespace-incomplete-line:
whitespace: symbolic links usually lack LF at the end
"git switch <name>", in an attempt to create a local branch <name>
after a remote tracking branch of the same name gave an advise
message to disambiguate using "git checkout", which has been
updated to use "git switch".
* jc/checkout-switch-restore:
checkout: tell "parse_remote_branch" which command is calling it
checkout: pass program-readable token to unified "main"
UI improvements for "git history reword".
* ps/history-ergonomics-updates:
Documentation/git-history: document default for "--update-refs="
builtin/history: rename "--ref-action=" to "--update-refs="
builtin/history: replace "--ref-action=print" with "--dry-run"
builtin/history: check for merges before asking for user input
builtin/history: perform revwalk checks before asking for user input
A handful of places used refs_for_each_ref_in() API incorrectly,
which has been corrected.
* ps/for-each-ref-in-fixes:
bisect: simplify string_list memory handling
bisect: fix misuse of `refs_for_each_ref_in()`
pack-bitmap: fix bug with exact ref match in "pack.preferBitmapTips"
pack-bitmap: deduplicate logic to iterate over preferred bitmap tips
"git repo info" learns "--keys" action to list known keys.
* lo/repo-info-keys:
repo: add new flag --keys to git-repo-info
repo: rename the output format "keyvalue" to "lines"
Add process ancestry data to trace2 on macOS to match what we
already do on Linux and Windows. Also adjust the way Windows
implementation reports this information to match the other two.
* mc/tr2-process-ancestry-cleanup:
t0213: add trace2 cmd_ancestry tests
test-tool: extend trace2 helper with 400ancestry
trace2: emit cmd_ancestry data for Windows
trace2: refactor Windows process ancestry trace2 event
build: include procinfo.c impl for macOS
trace2: add macOS process ancestry tracing
"git pack-objects --stdin-packs" with "--exclude-promisor-objects"
fetched objects that are promised, which was not wanted. This has
been fixed.
* ps/pack-concat-wo-backfill:
builtin/pack-objects: don't fetch objects when merging packs
"git rev-list" and friends learn "--maximal-only" to show only the
commits that are not reachable by other commits.
* ds/revision-maximal-only:
revision: add --maximal-only option
"auto filter" logic for large-object promisor remote.
* cc/lop-filter-auto:
fetch-pack: wire up and enable auto filter logic
promisor-remote: change promisor_remote_reply()'s signature
promisor-remote: keep advertised filters in memory
list-objects-filter-options: support 'auto' mode for --filter
doc: fetch: document `--filter=<filter-spec>` option
fetch: make filter_options local to cmd_fetch()
clone: make filter_options local to cmd_clone()
promisor-remote: allow a client to store fields
promisor-remote: refactor initialising field lists
Update sample commit-msg hook to complain when a log message has
material mailinfo considers the end of log message in the middle.
* pw/commit-msg-sample-hook:
templates: detect commit messages containing diffs
templates: add .gitattributes entry for sample hooks
Small clean-up of xdiff library to remove unnecessary data
duplication.
* pw/xdiff-cleanups:
xdiff: remove unused data from xdlclass_t
xdiff: remove "line_hash" field from xrecord_t
When a strip option to the %(refname) placeholder is asked to leave N
path components, we first count up the path components to know how many
to remove. That happens with a loop like this:
/* Find total no of '/' separated path-components */
for (i = 0; p[i]; p[i] == '/' ? i++ : *p++)
;
which is a little hard to understand for two reasons.
First, the dereference in "*p++" is seemingly useless, since nobody
looks at the result. And static analyzers like Coverity will complain
about that. But removing the "*" will cause gcc to complain with
-Wint-conversion, since the two sides of the ternary do not match (one
is a pointer and the other an int).
Second, it is not clear what the meaning of "p" is at each iteration of
the loop, as its position with respect to our walk over the string
depends on how many slashes we've seen. The answer is that by itself, it
doesn't really mean anything: "p + i" represents the current state of
our walk, with "i" counting up slashes, and "p" by itself essentially
meaningless.
None of this behaves incorrectly, but ultimately the loop is just
counting the slashes in the refname. We can do that much more simply
with a for-loop iterating over the string and a separate slash counter.
We can also drop the comment, which is somewhat misleading. We are
counting slashes, not components (and a comment later in the function
makes it clear that we must add one to compensate). In the new code it
is obvious that we are counting slashes here.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Two tests in t6006 depend on the iconv(1) prerequisite to reencode a
commit message. This executable may not even exist though in case the
prereq is not set, which will cause the tests to fail.
Fix this by using UTF-8 instead when the prereq is not set.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We've got a bunch of tests in t5550 that connect to "$HTTPD_URL/error"
to ensure that error messages are properly forwarded. This URL executes
the "t/lib-httpd/error.sh" script, which in turn depends on the iconv(1)
executable to reencode the message.
This executable may not exist on platforms, which will make the tests
fail. Guard them with the ICONV prereq to fix such failures.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In t4205 we have a bunch of tests that depend on the iconv prereq. This
is for most of the part because we format commit messages that have been
encoded in an encoding different than UTF-8.
Those tests fall into two classes though:
- One class of tests outputs the data as-is without reencoding.
- One class of tests outputs the data with "i18n.logOutputEncoding" to
reencode it.
Curiously enough, both of these classes are marked with the ICONV
prereq, even though one might expect that the first class wouldn't need
the prereq. This is because we unconditionally use ISO-8859-1 encoding
for the initial commit message, and thus we depend on converting to
UTF-8 indeed.
This creates another problem though: when the iconv(1) executable does
not exist the test setup fails, even in the case where the ICONV prereq
has not been set.
Fix these issues by making the test encoding conditional on ICONV: if
it's available we use ISO-8859-1, otherwise we use UTF-8. This fixes the
test setup on platforms without iconv(1), and it allows us to drop the
ICONV prereq from a bunch of tests.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We've got a couple of tests related to diffs in t40xx that use the
iconv(1) executable to convert the encoding of a commit message. All of
these tests are prepared to handle a missing ICONV prereq, in which case
they will simply use UTF-8 encoding.
But even if the ICONV prerequisite has failed we try to use the iconv(1)
executable, even though it's not safe to assume that the executable
exists in that case. And besides that, it's also unnecessary to use
iconv(1) in the first place, as we would only use it to convert from
UTF-8 to UTF-8, which should be equivalent to a no-op.
Fix the issue and skip the call to iconv(1) in case the prerequisite is
not set. This makes tests work on systems that don't have iconv at all.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We've got a couple of tests that exercise Git with different encodings,
typically around commit messages. All of these tests depend on the ICONV
prerequisite, which is set when Git was built with support for iconv.
Many of those tests also end up using the iconv(1) executable to
reencode text. But while tests can rely on the fact that Git does have
support for iconv, they cannot assume that the iconv(1) executable
exists. The consequence is thus that tests will break in case Git is
built with iconv, but the executable doesn't exist. In fact, some of the
tests even use the iconv(1) executable unconditionally, regardless of
whether or not the ICONV prerequisite is set.
Git for Windows has recently (unintentionally) shipped a change where
the iconv(1) binary is not getting installed anymore [1]. And as we use
Git for Windows directly in MSVC+Meson jobs in GitLab CI this has caused
such tests to break. The missing iconv(1) binary is considered a bug
that will be fixed in Git for Windows. But regardless of that it makes
sense to not assume the binary to always exist so that our test suite
passes on platforms that don't have iconv at all.
Extend the ICONV prerequisite so that we know to skip tests in case the
iconv(1) binary doesn't exist. We'll adapt tests that are currently
broken in subsequent commits.
[1]: https://github.com/git-for-windows/git/issues/6083
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We declare the refs_for_removal string_list as NODUP, forcing us to
manually allocate strings we insert. And then when it comes time to
clean up, we set strdup_strings so that string_list_clear() will free
them for us.
This is a confusing pattern, and can be done much more simply by just
declaring the list with the DUP initializer in the first place.
It was written this way originally because one of the callsites
generated the item using xstrfmt(). But that spot switched to a plain
xstrdup() in the preceding commit. That means we can now just let the
string_list code handle allocation itself.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
All callers of `refs_for_each_ref_in()` pass in a string that is
terminated with a trailing slash to indicate that they only want to see
refs in that specific ref hierarchy. This is in fact a requirement if
one wants to use this function, as the function trims the prefix from
each yielded ref. So if there was a reference that was called
"refs/bisect" as in our example, the result after trimming would be the
empty string, and that's something we disallow.
Fix this by adding the trailing slash.
Furthermore, taking a closer look, we strip the prefix only to re-add it
in `mark_for_removal()`. This is somewhat roundabout, as we can instead
call `refs_for_each_fullref_in()` to not do any stripping at all. Do so
to simplify the code a bit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "pack.preferBitmapTips" configuration allows the user to specify
which references should be preferred when generating bitmaps. This
option is typically expected to be set to a reference prefix, like for
example "refs/heads/".
It's not unreasonable though for a user to configure one specific
reference as preferred. But if they do, they'll hit a `BUG()`:
$ git -c pack.preferBitmapTips=refs/heads/main repack -adb
BUG: ../refs/iterator.c:366: attempt to trim too many characters
error: pack-objects died of signal 6
The root cause for this bug is how we enumerate these references. We
call `refs_for_each_ref_in()`, which will:
- Yield all references that have a user-specified prefix.
- Trim each of these references so that the prefix is removed.
Typically, this function is called with a trailing slash, like
"refs/heads/", and in that case things work alright. But if the function
is called with the name of an existing reference then we'll try to trim
the full reference name, which would leave us with an empty name. And as
this would not really leave us with anything sensible, we call `BUG()`
instead of yielding this reference.
One could argue that this is a bug in `refs_for_each_ref_in()`. But the
question then becomes what the correct behaviour would be:
- Do we want to skip exact matches? In our case we certainly don't
want that, as the user has asked us to generate a bitmap for it.
- Do we want to yield the reference with the empty refname? That would
lead to a somewhat weird result.
Neither of these feel like viable options, so calling `BUG()` feels like
a sensible way out. The root cause ultimately is that we even try to
trim the whole refname in the first place. There are two possible ways
to fix this issue:
- We can fix the bug by using `refs_for_each_fullref_in()` instead,
which does not strip the prefix at all. Consequently, we would now
start to accept all references that start with the configured
prefix, including exact matches. So if we had "refs/heads/main", we
would both match "refs/heads/main" and "refs/heads/main-branch".
- Or we can fix the bug by appending a slash to the prefix if it
doesn't already have one. This would mean that we only match
ref hierarchies that start with this prefix.
While the first fix leaves the user with strictly _more_ configuration
options, we have already fixed a similar case in 10e8a9352b (refs.c:
stop matching non-directory prefixes in exclude patterns, 2025-03-06) by
using the second option. So for the sake of consistency, let's apply the
same fix here.
Clarify the documentation accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We have two locations that iterate over the preferred bitmap tips as
configured by the user via "pack.preferBitmapTips". Both of these
callsites are subtly wrong: when the preferred bitmap tips contain an
exact refname match, then we will hit a `BUG()`.
Prepare for the fix by unifying the two callsites into a new
`for_each_preferred_bitmap_tip()` function.
This removes the last callsite of `bitmap_preferred_tips()` outside of
"pack-bitmap.c". As such, convert the function to be local to that file
only. Note that the function is still used by a second caller, so we
cannot just inline it.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The MSVC+Meson job does not currently have any logic to print failing
tests, nor does it upload the failed test artifacts. Backfill this logic
to make help debugging efforts in case any of its jobs has failed.
GitHub already knows to do this, so we don't need an equivalent change
over there.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
While our GitHub workflow already uses "ci/run-test-slice-meson.sh",
GitLab CI open-codes the parameters. Adapt the latter to also use the
same script so that we always use the same Meson options across both CI
systems.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the preceding commit we have adjusted test slicing to be one-based
when using the "ci/run-test-slice.sh" script. But we also have an
equivalent script for Meson that is still zero-based, which is of course
inconsistent.
Adapt the script to be one-based, as well, and adapt the GitHub workflow
accordingly. Note that GitLab doesn't yet use the script, so it does not
need to be adapted. This will change in the next commit though.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
While the win+Meson test jobs run in GitHub workflows, the shell script
that is supposed to run the jobs is seemingly not running at all. All
that the CI job prints is the following:
Run ci/run-test-slice-meson.sh build 1 10
ci/run-test-slice-meson.sh build 1 10
shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
env:
DEVELOPER: 1
The step is currently defined to use PowerShell, and of course it
doesn't know how to execute POSIX shell scripts. What's surprising
though is that this step doesn't even lead to a CI failure.
Fix the issue by using Bash instead of PowerShell, as we do in other
steps that execute shell scripts.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On Windows, we execute tests with "--no-bin-wrappers". This has been
introduced via a87e427e35 (ci: speed up Windows phase, 2019-01-29) to
save some time: spawning processes is expensive on Windows, and shell
scripts tend to spawn a bunch of them. So overall, the bin-wrappers led
to a performance overhead of ~10-30%.
This causes test failures when using Meson on Windows:
failure: t7610.28 mergetool --tool-help shows recognized tools
++ git mergetool --tool-help
/d/a/git/git/build/git-mergetool--lib: line 45: cd: D:/a/git/git/build/mergetools: No such file or directory
The root cause here is that our bin-wrappers are usually responsible for
setting up the `MERGE_TOOL_DIR` environment variable so that we can
locate these scripts. But as we don't use the bin-wrappers, we'll
instead use the default location for merge tools, which is derived from
`GIT_EXEC_PATH`. And as `GIT_EXEC_PATH` points to our build directory,
which won't ever contain any of the merge tools, we will fail to locate
any of the merge tools.
This issue has went unnoticed for a long time given that we only skip
bin-wrappers on Windows, and because the CI jobs on Windows didn't
execute due to a bug.
Fix the issue by always setting the `MERGE_TOOL_DIR` environment
variable to the correct directory.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "ci/run-test-slice.sh" script can be used to slice up all of our
tests into N pieces and then run each of them on a separate CI job.
This is used by both GitLab and GitHub CI to speed up Windows tests,
which would otherwise be painfully slow.
The infra itself is fueled by `test-tool path-utils slice-tests`. This
tool receives as input an "offset" and a "stride" that can be combined
to slice up tests. This framing can be misleading though: you are
expected to pass a zero-based index as "offset", and the complete number
of slices to the "stride". The latter makes sense, but it is somewhat
surprising that the offset needs to be zero-based. And this is in fact
biting us: while GitHub passes zero-based indices, GitLab passes
`$CI_NODE_INDEX`, which is a one-based indice.
Ideally, we should have verification that the parameters make sense.
And naturally, one would for example expect that it's an error to call
the binary with an offset larger than the stride. But with the current
framing as "offset" it's not even wrong to do so, as it is of course
well-defined to start at a larger offset than the stride.
This means that we get this wrong on GitLab's CI, as we pass a one based
index there, and this causes us to skip one of the tests. Interestingly,
it's not the lexicographically first test that we skip. Instead, as we
sort tests by size before slicing them, we skip the _smallest_ test.
Reframe the problem to instead talk about "slice number" and "total
number of slices". For all of our use cases this is semantically
equivalent, but it allows us to perform some verifications:
- The total number of slices must be greater than 1.
- The selected slice must be between 1 <= nr <= slices_total.
As the indices are now one-based it means that GitLab's CI is fixed.
The GitHub workflow is updated accordingly.
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "run-test-slice.sh" script executes the test helper to slice up
tests passed to it. As the execution is part of a pipe though, we end up
ignoring any potential error code returned by the helper.
Make the code more robust by storing the tests in a variable first so
that we can split up the pipeline.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git merge-file" can be run outside a repository, but it ignored
all configuration, even the per-user ones. The command now uses
available configuration files to find its customization.
* yt/merge-file-outside-a-repository:
merge-file: honor merge.conflictStyle outside of a repository
A handful of documentation pages have been modernized to use the
"synopsis" style.
* ja/doc-synopsis-style-even-more:
doc: convert git-show to synopsis style
doc: fix some style issues in git-clone and for-each-ref-options
doc: finalize git-clone documentation conversion to synopsis style
doc: convert git-submodule to synopsis style
