Sometimes, failures in a test case are actually caused by issues in
earlier test cases.
To make it easier to see those issues, let's attach the output from
before the failing test case (i.e. stdout/stderr since the previous
failing test case, or the start of the test script). This will be
visible in the "Attachments" of the details of the failed test.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The JUnit XML format lends itself to be presented in a powerful UI,
where you can drill down to the information you are interested in very
quickly.
For test failures, this usually means that you want to see the detailed
trace of the failing tests.
With Travis CI, we passed the `--verbose-log` option to get those
traces. However, that seems excessive, as we do not need/use the logs in
almost all of those cases: only when a test fails do we have a way to
include the trace.
So let's do something different when using Azure DevOps: let's run all
the tests with `--quiet` first, and only if a failure is encountered,
try to trace the commands as they are executed.
Of course, we cannot turn on `--verbose-log` after the fact. So let's
just re-run the test with all the same options, adding `--verbose-log`.
And then munging the output file into the JUnit XML on the fly.
Note: there is an off chance that re-running the test in verbose mode
"fixes" the failures (and this does happen from time to time!). That is
a possibility we should be able to live with. Ideally, we would label
this as "Passed upon rerun", and Azure Pipelines even know about that
outcome, but it is not available when using the JUnit XML format for
now:
https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This should be more reliable than the current method, and prepares the
test suite for a consistent way to clean up before re-running the tests
with different options.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This makes use of the just-introduced consistent way to specify that a
long-running process needs to be terminated at the end of a test script
run.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When running the p4 daemon or `git daemon`, we want to kill it at the
end of the test script.
So far, we do this "manually".
However, in the next few commits we want to teach the test suite to
optionally re-run scripts with different options, therefore we will have
to have a consistent way to stop daemons.
Let's introduce `test_atexit`, which is loosely modeled after
`test_when_finished` (but has a broader scope: rather than running the
commands after the current test case, run them when the test script
finishes, and also run them when the `--immediate` option is in effect).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This commit adds an azure-pipelines.yml file which is Azure DevOps'
equivalent to Travis CI's .travis.yml.
To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).
Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This patch introduces a conditional arm that defines some environment
variables and a function that displays the URL given the job id (to
identify previous runs for known-good trees).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This will come in handy when publishing the results of Git's test suite
during an automated Azure DevOps run.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In the next commit, we want to teach Git's test suite to optionally
output test results in JUnit-style .xml files. These files contain
information about the time spent. So we need a way to measure time.
While we could use `date +%s` for that, this will give us only seconds,
i.e. very coarse-grained timings.
GNU `date` supports `date +%s.%N` (i.e. nanosecond-precision output),
but there is no equivalent in BSD `date` (read: on macOS, we would not
be able to obtain precise timings).
So let's introduce `test-tool date getnanos`, with an optional start
time, that outputs preciser values.
Granted, it is a bit pointless to try measuring times accurately in
shell scripts, certainly to nanosecond precision. But it is better than
second-granularity.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The upcoming patches will allow building git.git via Azure Pipelines
(i.e. Azure DevOps' Continuous Integration), where variable names and
URLs look a bit different than in Travis CI.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The name is hard-coded to reflect that we use Travis CI for continuous
testing.
In the next commits, we will extend this to be able use Azure DevOps,
too.
So let's adjust the name to make it more generic.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This speeds up the tests by a bit on Windows, where running Unix shell
scripts (and spawning processes) is not exactly a cheap operation.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When building Git with RUNTIME_PREFIX and starting a test helper from
t/helper/, it fails to detect the system prefix correctly.
This is the reason that the warning
RUNTIME_PREFIX requested, but prefix computation failed. [...]
to be printed.
In t0061, we did not expect that to happen, and it actually did not
happen in the normal case, because bin-wrappers/test-tool specifically
sets GIT_TEXTDOMAINDIR (and as a consequence, nothing in test-tool wants
to know about the runtime prefix).
However, with --with-dashes, bin-wrappers/test-tool is no longer called,
but t/helper/test-tool is called directly.
So let's just ignore the RUNTIME_PREFIX warning.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We really need to be able to find the test helpers... Really. This
change was forgotten when we moved the test helpers into t/helper/
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
For a short while, we had two tracks: the `master` one, based on
v2.19.2, and the `rebase-to-v2.20.0-rc<N>` one, following the -rc
versions of v2.20.0.
Since we forward-ported all changes from `master` to the -rc track (or
backported them in the other direction), at this point we have all of
`origin/master` already, and can continue the merging rebase to v2.20.0
based on the `rebase-to-v2.20.0-rc<N>` track (which has a lot of the
merge conflicts resolved already).
So let's just merge in the commit history of `master`, without changing
the tree (i.e. with `-s ours`, just like the fake merge starting the
merging rebase). That way, the new `master` will fast-forward from the
old `master`.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Usually we don't need to set libcurl to choose which version of the
HTTP protocol to use to communicate with a server.
But different versions of libcurl, the default value is not the same.
CURL >= 7.62.0: CURL_HTTP_VERSION_2TLS
CURL < 7.62: CURL_HTTP_VERSION_1_1
In order to give users the freedom to control the HTTP version,
we need to add a setting to choose which HTTP version to use.
Signed-off-by: Force Charlie <charlieio@outlook.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Usually we don't need to set libcurl to choose which version of the
HTTP protocol to use to communicate with a server.
But different versions of libcurl, the default value is not the same.
CURL >= 7.62.0: CURL_HTTP_VERSION_2TLS
CURL < 7.62: CURL_HTTP_VERSION_1_1
In order to give users the freedom to control the HTTP version,
we need to add a setting to choose which HTTP version to use.
Signed-off-by: Force Charlie <charlieio@outlook.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This fixes the reflog messages, as expected by t3406 in v2.20.0-rc2, as
well as fixing a compiler warning about "" being an invalid
printf()-style format. Also, while at it, ensure that
.git/rebased-patches is truncated if it already exists.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When switching a branch *and* updating said branch to a different
revision, let's avoid a double entry by first updating the branch and
then adjusting the symbolic ref HEAD.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The text body of section Behavioral Differences is typeset as code,
but should be regular text. Remove the indentation to achieve that.
While here, prettify the language:
- use "the x backend" instead of "x-based rebase";
- use present tense instead of future tense;
and use subsections instead of a list.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We have three double-quote characters, which is one too many or too few.
Dropping the last one seems to match the original intention best.
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
I had to read this sentence a few times to understand it. Let's try to
clarify it.
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some items that should be in "Performance, Internal Implementation,
Development Support etc." have ended up in "UI, Workflows & Features"
and "Fixes since v2.19". Move them, and do s/uses/use/ while at it.
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit d8981c3f88 ("format-patch: do not let its diff-options affect
--range-diff", 2018-11-30) taught `show_range_diff()` to accept a
NULL-pointer as an indication that it should use its own "reasonable
default". That fixed a regression from a5170794 ("Merge branch
'ab/range-diff-no-patch'", 2018-11-18), but unfortunately it introduced
a regression of its own.
In particular, it means we forget the `file` member of the diff options,
so rather than placing a range-diff in the cover-letter, we write it to
stdout. In order to fix this, rewrite the two callers adjusted by
d8981c3f88 to instead create a "dummy" set of diff options where they
only fill in the fields we absolutely require, such as output file and
color.
Modify and extend the existing tests to try and verify that the right
contents end up in the right place.
Don't revert `show_range_diff()`, i.e., let it keep accepting NULL.
Rather than removing what is dead code and figuring out it isn't
actually dead and we've broken 2.20, just leave it for now.
[es: retain diff coloring when going to stdout]
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For many Win32 functions, there actually exist two variants: one with
the `A` suffix that takes ANSI parameters (`char *` or `const char *`)
and one with the `W` suffix that takes Unicode parameters (`wchar_t *`
or `const wchar_t *`).
Let's be precise what we want to use.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
