When using colors, the shell needs to identify 0-width substrings
in PS1 - such as color escape sequences - when calculating the
on-screen width of the prompt.
Until now, we used the form %F{<color>} in zsh - which it knows is
0-width, or otherwise use standard SGR esc sequences wrapped between
byte values 1 and 2 (SOH, STX) as 0-width start/end markers, which
bash/readline identify as such.
But now that more shells are supported, the standard SGR sequences
typically work, but the SOH/STX markers might not be identified.
This commit adds support for vars GIT_PS1_COLOR_{PRE,POST} which
set custom 0-width markers or disable the markers.
Signed-off-by: Avi Halachmi (:avih) <avihpit@yahoo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With one big exception, git-prompt.sh should now be both almost posix
compliant, and also compatible with most (posix-ish) shells.
That exception is the use of "local" vars in functions, which happens
extensively in the current code, and is not simple to replace with
posix compliant code (but also not impossible).
Luckily, almost all shells support "local" as used by the current
code, with the notable exception of ksh93[u+m], but also the Schily
minimal posix sh (pbosh), and yash in posix mode.
See assessment below that "local" is likely the only blocker in those.
So except mainly ksh93, git-prompt.sh now works in most shells:
- bash, zsh, dash since at least 0.5.8, free/net bsd sh, busybox-ash,
mksh, openbsd sh, pdksh(!), Schily extended Bourne sh (bosh), yash.
which is quite nice.
As an anecdote, replacing the 1st line in __git_ps1() (local exit=$?)
with these 2 makes it work in all tested shells, even without "local":
# handles only 0/1 args for simplicity. needs +5 LOC for any $#
__git_e=$?; local exit="$__git_e" 2>/dev/null ||
{(eval 'local() { export "$@"; }'; __git_ps1 "$@"); return "$__git_e"; }
Explanation:
If the shell doesn't have the command "local", define our own
function "local" which instead does plain (global) assignents.
Then use __git_ps1 in a subshell to not clober the caller's vars.
This happens to work because currently there are no name conflicts
(shadow) at the code, initial value is not assumed (i.e. always
doing either 'local x=...' or 'local x;... x=...'), and assigned
initial values are quoted (local x="$y"), preventing word split and
glob expansion (i.e. assignment context is not assumed).
The last two (always init, quote values) seem to be enough to use
"local" portably if supported, and otherwise shells indeed differ.
Uses "eval", else shells with "local" may reject it during parsing.
We don't need "export", but it's smaller than writing our own loop.
While cute, this approach is not really sustainable because all the
vars become global, which is hard to maintain without conflicts
(but hey, it currently has no conflicts - without even trying...).
However, regardless of being an anecdote, it provides some support to
the assessment that "local" is the only blocker in those shells.
Signed-off-by: Avi Halachmi (:avih) <avihpit@yahoo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
$'...' is new in POSIX (2024), and some shells support it in recent
versions, while others have had it for decades (bash, zsh, ksh93).
However, there are still enough shells which don't support it, and
it's cheap to use an alternative form which works in all shells,
so let's do that instead of dismissing it as "it's compliant".
It was agreed to use one form rather than $'...' where supported and
fallback otherwise.
shells where $'...' works:
- bash, zsh, ksh93, mksh, busybox-ash, dash master, free/net bsd sh.
shells where it doesn't work, but the new fallback works:
- all dash releases (up to 0.5.12), older versions of free/net bsd sh,
openbsd sh, pdksh, all Schily Bourne sh variants, yash.
Signed-off-by: Avi Halachmi (:avih) <avihpit@yahoo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The issues which this commit fixes are unlikely to be broken
in real life, but the fixes improve correctness, and would prevent
bugs in some uncommon cases, such as weird IFS values.
Listing some portability guidelines here for future reference.
I'm leaving it to someone else to decide whether to include
it in the file itself, place it as a new file, or not.
---------
The command "local" is non standard, but is allowed in this file:
- Quote initialization if it can expand (local x="$y"). See below.
- Don't assume initial value after "local x". Either initialize it
(local x=..), or set before first use (local x;.. x=..; <use $x>).
(between shells, "local x" can unset x, or inherit it, or do x= )
Other non-standard features beyond "local" are to be avoided.
Use the standard "test" - [...] instead of non-standard [[...]] .
--------
Quotes (some portability things, but mainly general correctness):
Quotes prevent tilde-expansion of some unquoted literal tildes (~).
If the expansion is undesirable, quotes would ensure that.
Tilds expanded: a=~user:~/ ; echo ~user ~/dir
not expanded: t="~"; a=${t}user b=\~foo~; echo "~user" $t/dir
But the main reason for quoting is to prevent IFS field splitting
(which also coalesces IFS chars) and glob expansion in parts which
contain parameter/arithmetic expansion or command substitution.
"Simple command" (POSIX term) is assignment[s] and/or command [args].
Examples:
foo=bar # one assignment
foo=$bar x=y # two assignments
foo bar # command, no assignments
x=123 foo bar # one assignment and a command
The assignments part is not IFS-split or glob-expanded.
The command+args part does get IFS field split and glob expanded,
but only at unquoted expanded/substituted parts.
In the command+args part, expanded/substituted values must be quoted.
(the commands here are "[" and "local"):
Good: [ "$mode" = yes ]; local s="*" x="$y" e="$?" z="$(cmd ...)"
Bad: [ $mode = yes ]; local s=* x=$y e=$? z=$(cmd...)
The arguments to "local" do look like assignments, but they're not
the assignment part of a simple command; they're at the command part.
Still at the command part, no need to quote non-expandable values:
Good: local x= y=yes; echo OK
OK, but not required: local x="" y="yes"; echo "OK"
But completely empty (NULL) arguments must be quoted:
foo "" is not the same as: foo
Assignments in simple commands - with or without an actual command,
don't need quoting becase there's no IFS split or glob expansion:
Good: s=* a=$b c=$(cmd...)${x# foo }${y- } [cmd ...]
It's also OK to use double quotes, but not required.
This behavior (no IFS/glob) is called "assignment context", and
"local" does not behave with assignment context in some shells,
hence we require quotes when using "local" - for compatibility.
The value between 'case' and 'in' doesn't IFS-split/glob-expand:
Good: case * $foo $(cmd...) in ... ; esac
identical: case "* $foo $(cmd...)" in ... ; esac
Nested quotes in command substitution are fine, often necessary:
Good: echo "$(foo... "$x" "$(bar ...)")"
Nested quotes in substring ops are legal, and sometimes needed
to prevent interpretation as a pattern, but not the most readable:
Legal: foo "${x#*"$y" }"
Nested quotes in "maybe other value" subst are invalid, unnecessary:
Good: local x="${y- }"; foo "${z:+ $a }"
Bad: local x="${y-" "}"; foo "${z:+" $a "}"
Outer/inner quotes in "maybe other value" have different use cases:
"${x-$y}" always one quoted arg: "$x" if x is set, else "$y".
${x+"$x"} one quoted arg "$x" if x is set, else no arg at all.
Unquoted $x is similar to the second case, but it would get split
into few arguments if it includes any of the IFS chars.
Assignments don't need the outer quotes, and the braces delimit the
value, so nested quotes can be avoided, for readability:
a=$(foo "$x") a=${x#*"$y" } c=${y- }; bar "$a" "$b" "$c"
Signed-off-by: Avi Halachmi (:avih) <avihpit@yahoo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The existing [[...]] tests were either already valid as standard [...]
tests, or only required minimal retouch:
Notes:
- [[...]] doesn't do field splitting and glob expansion, so $var
or $(cmd...) don't need quoting, but [... does need quotes.
- [[ X == Y ]] when Y is a string is same as [ X = Y ], but if Y is
a pattern, then we need: case X in Y)... ; esac .
- [[ ... && ... ]] was replaced with [ ... ] && [ ... ] .
- [[ -o <zsh-option> ]] requires [[...]], so put it in "eval" and only
eval it in zsh, so other shells would not abort on syntax error
(posix says [[ has unspecified results, shells allowed to reject it)
- ((x++)) was changed into x=$((x+1)) (yeah, not [[...]] ...)
Shells which accepted the previous forms:
- bash, zsh, ksh93, mksh, openbsd sh, pdksh.
Shells which didn't, and now can process it:
- dash, free/net bsd sh, busybox-ash, Schily Bourne sh, yash.
Signed-off-by: Avi Halachmi (:avih) <avihpit@yahoo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Arrays only existed in the svn-upstream code, used to:
- Keep a list of svn remotes.
- Convert commit msg to array of words, extract the 2nd-to-last word.
Except bash/zsh, nearly all shells failed load on syntax errors here.
Now:
- The svn remotes are a list of newline-terminated values.
- The 2nd-to-last word is extracted using standard shell substrings.
- All shells can digest the svn-upstream code.
While using shell field splitting to extract the word is simple, and
doesn't even need non-standard code, e.g. set -- $(git log -1 ...),
it would have the same issues as the old array code: it depends on IFS
which we don't control, and it's subject to glob-expansion, e.g. if
the message happens to include * or **/* (as this commit message just
did), then the array could get huge. This was not great.
Now it uses standard shell substrings, and we know the exact delimiter
to expect, because it's the match from our grep just one line earlier.
The new word extraction code also fixes svn-upstream in zsh, because
previously it used arr[len-2], but because in zsh, unlike bash, array
subscripts are 1-based, it incorrectly extracted the 3rd-to-last word.
symptom: missing upstream status in a git-svn repo: u=, u+N-M, etc.
The breakage in zsh is surprising, because it was last touched by
commit d0583da838 (prompt: fix show upstream with svn and zsh),
claiming to fix exactly that. However, it only mentions syntax fixes.
It's unclear if behavior was fixed too. But it was broken, now fixed.
Note LF=$'\n' and then using $LF instead of $'\n' few times.
A future commit will add fallback for shells without $'...', so this
would be the only line to touch instead of replacing every $'\n' .
Shells which could run the previous array code:
- bash
Shells which have arrays but were broken anyway:
- zsh: 1-based subscript
- ksh93: no "local" (the new code can't fix this part...)
- mksh, openbsd sh, pdksh: failed load on syntax error: "for ((...))".
More shells which Failed to load due to syntax error:
- dash, free/net bsd sh, busybox-ash, Schily Bourne shell, yash.
Signed-off-by: Avi Halachmi (:avih) <avihpit@yahoo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
First use is in the form: local var; ...; var=$var$whatever...
If the variable was unset (as bash and others do after "local x"),
then it would error if set -u is in effect.
Also, many shells inherit the existing value after "local var"
without init, but in this case it's unlikely to have a prior value.
Now we initialize it.
(local var= is enough, but local var="" is the custom in this file)
Signed-off-by: Avi Halachmi (:avih) <avihpit@yahoo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Here-documend is standard, and works in all shells.
Both here-string and here-doc add final newline, which is important
in this case, because $output is without final newline, but we do
want "read" to succeed on the last line as well.
Shells which support here-string:
- bash, zsh, mksh, ksh93, yash (non-posix-mode).
shells which don't, and got fixed:
- ash-derivatives (dash, free/net bsd sh, busybox-ash).
- pdksh, openbsd sh.
- All Schily Bourne shell variants.
Signed-off-by: Avi Halachmi (:avih) <avihpit@yahoo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Work around asciidoctor's css that renders `monospace` material
in the SYNOPSIS section of manual pages as block elements.
* js/doc-markup-updates-fix:
Doc: fix Asciidoctor css workaround
asciidoctor: fix `synopsis` rendering
Repacking a repository with multi-pack index started making stupid
pack selections in Git 2.45, which has been corrected.
* ds/midx-write-repack-fix:
midx-write: revert use of --stdin-packs
t5319: add failing test case for repack/expire
The previous step introduced docinfo.html to be used to tweak the
CSS used by the asciidoctor, that by default renders <code> inside
<pre> as a block element, breaking the SYNOPSIS section of a few
pages that adopted a new convention we use since Git 2.45.
But in this project, HTML files are all generated. We do not force
any human to write HTML by hand, which is an unusual and cruel
punishment. "*.html" is in the .gitignore file, and "make clean"
removes them. Having a tracked .html file makes "make clean" make
the tree dirty by removing the tracked docinfo.html file.
Let's do an obvious, minimum and stupid workaround to generate that
file at runtime instead. The mark-up is being rethought in a major
way for the next development cycle, and the CSS workaround we added
in the previous step may have to adjusted, possibly in a large way,
anyway.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It was reported that t1460-refs-migrate.sh fails when using Cygwin with
errors like the following:
error: could not link file '.git/ref_migration.sr9pEF/reftable' to '.git/reftable': Permission denied
As some debugging surfaced, the root cause of this is that some files of
the newly-initialized ref store are still open when the target format is
the "reftable" format, and Cygwin refuses to rename open files.
Fix this issue by closing the new ref store before renaming its files
into place. This is a slight change in behaviour compared to before,
where we kept the new ref store open and then updated the repository's
ref store to point to it.
While we could re-open the new ref store after we have moved files
around, this is ultimately unnecessary. We know that the only user of
`repo_migrate_ref_storage_format()` is the git-refs(1) command, and it
won't access the ref store after it has been migrated anyway. So
reinitializing the ref store would be a waste of time. Regardless of
that it is still sensible to leave the repository in a consistent state.
But instead of reinitializing the ref store, we can simply unset the
repo's ref store altogether and let `get_main_ref_store()` lazily
initialize the new ref store as required.
Reported-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since 76880f0510 (doc: git-clone: apply new documentation formatting
guidelines, 2024-03-29), the synopsis of `git clone`'s manual page is
rendered differently than before; Its parent commit did the same for
`git init`.
The result looks quite nice. When rendered with AsciiDoc, that is. When
rendered using AsciiDoctor and displayed in a graphical web browser such
as Firefox, Chrome, Edge, etc, the result is quite unpleasant to my eye,
reading something like this:
SYNOPSIS
git clone
[
--template=
<template-directory>]
[
-l
] [
-s
] [
--no-hardlinks
] [
-q
] [
[... continuing like this ...]
The reason is that AsciiDoctor's default style sheet contains this (see
https://github.com/asciidoctor/asciidoctor/blob/854923b15533/src/stylesheets/asciidoctor.css#L519-L521
for context):
pre > code {
display: block;
}
It is this `display: block` that forces the parts that are enclosed in
`<code>` tags (such as the `git clone` or the `--template=` part) to be
rendered on their own line.
Side note: This seems not to affect console web browsers like `lynx` or
`w3m`, most likely because most style sheet directions cannot be
respected in text terminals and therefore they seem to punt on style
sheets altogether.
To fix this, let's apply the method recommended by AsciiDoctor in
https://docs.asciidoctor.org/asciidoctor/latest/html-backend/default-stylesheet/#customize-docinfo
to partially override AsciiDoctor's default style sheet so that the
`<code>` sections of the synopsis are no longer each rendered on their
own, individual lines.
This fixes https://github.com/git-for-windows/git/issues/5063.
Even on the Git home page, where AsciiDoctor's default stylesheet is
_not_ used, this change resulted in some unpleasant rendering where not
only the font is changed for the `<code>` sections of the synopsis, but
padding and a different background color make the visual impression
quite uneven. This has been addressed in the meantime, via
https://github.com/git/git-scm.com/commit/a492d0565512.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Asciidoc.py does not have the concept of generalized roles, whereas
asciidoctor interprets [foo]`blah` as blah with role foo in the
synopsis, making in effect foo disappear in the output. Note that
square brackets not directly followed by an inline markup do not
define a role, which is why we do not have the issue on other parts of
the documentation.
In order to get a consistant result across asciidoctor and
asciidoc.py, the hack is to use the {empty} entity
to split the bracket part from the inline format part.
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This reverts b7d6f23a17 (midx-write.c: use `--stdin-packs` when
repacking, 2024-04-01) and then marks the test created in the previous
change as passing.
The fundamental issue with the reverted change is that the focus on
pack-files separates the object selection from how the multi-pack-index
selects a single pack-file for an object ID with multiple copies among
the tracked pack-files.
The change was made with the intention of improving delta compression in
the resulting pack-file, but that can be resolved with the existing
object list mechanism. There are other potential pitfalls of doing an
object walk at this time if the repository is a blobless partial clone,
and that will require additional testing on top of the one that changes
here.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Acked-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git 2.45.0 included the change b7d6f23a17 (midx-write.c: use
`--stdin-packs` when repacking, 2024-04-01) which caused the 'git
multi-pack-index repack' command to use 'git pack-objects --stdin-packs'
instead of listing the objects to repack. While this change was
motivated by efficient cross-process communication and the ability to
improve delta compression, it breaks a fundamental function of the
'incremental-repack' task that is enabled by default in Scalar clones or
Git repositories that run 'git maintenance start'.
The 'incremental-repack' task performs a two-step process of the
'expire' and 'repack' subcommands of the 'git multi-pack-index' builtin.
The 'expire' command removes any pack-files listed in the
multi-pack-index but without any referenced objects. The 'repack' task
then finds a batch of pack-files to repack and sends their objects to
'git pack-objects'. Both the pack-files chosen for the batch and the
objects chosen to repack are based on the ones that the multi-pack-index
references. Objects that appear in a pack-file but have a duplicate copy
in a newer pack-file are not considered in this case. Since the
multi-pack-index references only the newest copy of an object, this
allows the next 'incremental-repack' task to remove the pack-files in
the next 'expire' task. This delay is intentional due to how Windows
handles may block deletion of files with open read handles.
However, the mentioned commit changed this behavior to divorce the set
of objects referenced by the multi-pack-index and instead use a set of
"included" and "excluded" pack-files in the 'git pack-objects' builtin.
When a pack-file is selected as "included", only the objects it contains
but are not in any "excluded" pack-files are considered for repacking.
This has led to client repositories failing to remove old pack-files as
they still have some referenced objects. This grows over time until the
point that Git is trying to repack the same pack-files over and over.
For now, create a test case that demonstrates the expected behavior, but
also fails in its final line. The setup here it attempting to recreate a
typical situation for a repository that uses a blobless partial clone.
There would be a large initial pack-file from the clone that is never
selected in the 'repack' batch. There are other pack-files that have a
combination of new objects from incremental fetches and possibly blobs
that are not connected to those incremental fetches; these blobs could
be filled in from commands like 'git checkout' or 'git blame'. The
pack-files also have some overlap on purpose so test-1 has some
duplicates in test-2 and test-2 has some duplicates in test-3.
At the end of the test, the test-2 pack-file still exists though it
should have been expired. This test will pass when reverting the
offending commit.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git var GIT_SHELL_PATH" should report the path to the shell used
to spawn external commands, but it didn't do so on Windows, which
has been corrected.
* js/var-git-shell-path:
var(win32): do report the GIT_SHELL_PATH that is actually used
run-command: declare the `git_shell_path()` function globally
run-command(win32): resolve the path to the Unix shell early
mingw(is_msys2_sh): handle forward slashes in the `sh.exe` path, too
win32: override `fspathcmp()` with a directory separator-aware version
strvec: declare the `strvec_push_nodup()` function globally
run-command: refactor getting the Unix shell path into its own function
What happens when http.cookieFile gets the special value "" has
been clarified in the documentation.
* ps/doc-http-empty-cookiefile:
doc: update http.cookieFile with in-memory cookie processing
"git push '' HEAD:there" used to hit a BUG(); it has been corrected
to die with "fatal: bad repository ''".
* kn/push-empty-fix:
builtin/push: call set_refspecs after validating remote
