196 Commits

Author SHA1 Message Date
Junio C Hamano
2c5c4d3b5f Merge branch 'pt/fsmonitor-linux' into seen
The fsmonitor daemon has been implemented for Linux.

* pt/fsmonitor-linux:
  fsmonitor: close inherited file descriptors and detach in daemon
  run-command: add close_fd_above_stderr option
  fsmonitor: add tests for Linux
  fsmonitor: implement filesystem change listener for Linux
  fsmonitor: deduplicate settings logic for Unix platforms
  fsmonitor: deduplicate IPC path logic for Unix platforms
  fsmonitor: use pthread_cond_timedwait for cookie wait
  compat/win32: add pthread_cond_timedwait
  fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon
  fsmonitor: fix khash memory leak in do_handle_client
2026-03-03 11:08:56 -08:00
Junio C Hamano
2a554ddd35 Merge branch 'ps/upload-pack-buffer-more-writes' into jch
Reduce system overhead "git upload-pack" spends relaying "git
pack-objects" output to the "git fetch" running on the other end of
the connection.

* ps/upload-pack-buffer-more-writes:
  builtin/pack-objects: reduce lock contention when writing packfile data
  csum-file: drop `hashfd_throughput()`
  csum-file: introduce `hashfd_ext()`
  sideband: use writev(3p) to send pktlines
  wrapper: introduce writev(3p) wrappers
  compat/posix: introduce writev(3p) wrapper
  git-compat-util: introduce `cast_size_t_to_ssize_t()`
  upload-pack: reduce lock contention when writing packfile data
  upload-pack: adapt keepalives based on buffering
  upload-pack: fix debug statement when flushing packfile data
2026-03-03 11:08:35 -08:00
Junio C Hamano
0281085819 Merge branch 'ps/odb-sources' into jch
The object source API is getting restructured to allow plugging new
backends.

Comments?

* ps/odb-sources:
  odb/source: make `begin_transaction()` function pluggable
  odb/source: make `write_alternate()` function pluggable
  odb/source: make `read_alternates()` function pluggable
  odb/source: make `write_object_stream()` function pluggable
  odb/source: make `write_object()` function pluggable
  odb/source: make `freshen_object()` function pluggable
  odb/source: make `for_each_object()` function pluggable
  odb/source: make `read_object_stream()` function pluggable
  odb/source: make `read_object_info()` function pluggable
  odb/source: make `close()` function pluggable
  odb/source: make `reprepare()` function pluggable
  odb/source: make `free()` function pluggable
  odb/source: introduce source type for robustness
  odb: move reparenting logic into respective subsystems
  odb: embed base source in the "files" backend
  odb: introduce "files" source
  odb: split `struct odb_source` into separate header
2026-03-03 11:08:32 -08:00
Junio C Hamano
5a9ff19394 Merge branch 'dk/meson-regen-config-list' into jch
Fix dependency screw-up in meson-based builds.

* dk/meson-regen-config-list:
  build: regenerate config-list.h when Documentation changes
2026-03-03 11:08:21 -08:00
Patrick Steinhardt
50869da7e6 compat/posix: introduce writev(3p) wrapper
In a subsequent commit we're going to add the first caller to
writev(3p). Introduce a compatibility wrapper for this syscall that we
can use on systems that don't have this syscall.

The syscall exists on modern Unixes like Linux and macOS, and seemingly
even for NonStop according to [1]. It doesn't seem to exist on Windows
though.

[1]: http://nonstoptools.com/manuals/OSS-SystemCalls.pdf
[2]: https://www.gnu.org/software/gnulib/manual/html_node/writev.html

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-03 09:52:34 -08:00
Junio C Hamano
427d39ca4f Merge branch 'ps/meson-gitk-git-gui'
Plumb gitk/git-gui build and install procedure in meson based
builds.

* ps/meson-gitk-git-gui:
  meson: wire up gitk and git-gui
2026-03-02 17:06:52 -08:00
Paul Tarjan
149c19ba1b fsmonitor: implement filesystem change listener for Linux
Implement the built-in fsmonitor daemon for Linux using the inotify
API, bringing it to feature parity with the existing Windows and macOS
implementations.

The implementation uses inotify rather than fanotify because fanotify
requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it
unsuitable for an unprivileged user-space daemon.  While inotify has
the limitation of requiring a separate watch on every directory (unlike
macOS's FSEvents, which can monitor an entire directory tree with a
single watch), it operates without elevated privileges and provides
the per-file event granularity needed for fsmonitor.

The listener uses inotify_init1(O_NONBLOCK) with a poll loop that
checks for events with a 50-millisecond timeout, keeping the inotify
queue well-drained to minimize the risk of overflows.  Bidirectional
hashmaps map between watch descriptors and directory paths for efficient
event resolution.  Directory renames are tracked using inotify's cookie
mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a
periodic check detects stale renames where the matching IN_MOVED_TO
never arrived, forcing a resync.

New directory creation triggers recursive watch registration to ensure
all subdirectories are monitored.  The IN_MASK_CREATE flag is used
where available to prevent modifying existing watches, with a fallback
for older kernels.  When IN_MASK_CREATE is available and
inotify_add_watch returns EEXIST, it means another thread or recursive
scan has already registered the watch, so it is safe to ignore.

Remote filesystem detection uses statfs() to identify network-mounted
filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers.
Mount point information is read from /proc/mounts and matched against
the statfs f_fsid to get accurate, human-readable filesystem type names
for logging.  When the .git directory is on a remote filesystem, the
IPC socket falls back to $HOME or a user-configured directory via the
fsmonitor.socketDir setting.

Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com>
Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com>
Signed-off-by: Paul Tarjan <github@paulisageek.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-26 07:29:46 -08:00
Paul Tarjan
0e210668c7 fsmonitor: deduplicate settings logic for Unix platforms
The macOS fsm-settings-darwin.c is applicable to other Unix variants
as well.  Rename it to fsm-settings-unix.c, using the safer
xstrdup()+dirname() approach and including the "vfat" filesystem check.

Now that both fsm-ipc and fsm-settings use the "unix" variant name,
set FSMONITOR_OS_SETTINGS to "unix" for macOS in config.mak.uname and
remove the if/else conditionals from the build files.

Signed-off-by: Paul Tarjan <github@paulisageek.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-26 07:29:45 -08:00
Paul Tarjan
72b1daeb13 fsmonitor: deduplicate IPC path logic for Unix platforms
The macOS fsm-ipc-darwin.c is applicable to other Unix variants as
well.  Rename it to fsm-ipc-unix.c and add a worktree NULL check
(BUG guard) that was missing from the macOS version.

To support this, introduce FSMONITOR_OS_SETTINGS which is set to
"unix" for both macOS and Linux, distinct from FSMONITOR_DAEMON_BACKEND
which remains platform-specific (darwin, linux, win32).  Move
fsm-path-utils from FSMONITOR_OS_SETTINGS to FSMONITOR_DAEMON_BACKEND
since the path-utils files are platform-specific.

Signed-off-by: Paul Tarjan <github@paulisageek.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-26 07:29:45 -08:00
Junio C Hamano
422cae6687 Merge branch 'mc/tr2-process-ancestry-cleanup'
Add process ancestry data to trace2 on macOS to match what we
already do on Linux and Windows.  Also adjust the way Windows
implementation reports this information to match the other two.

* mc/tr2-process-ancestry-cleanup:
  t0213: add trace2 cmd_ancestry tests
  test-tool: extend trace2 helper with 400ancestry
  trace2: emit cmd_ancestry data for Windows
  trace2: refactor Windows process ancestry trace2 event
  build: include procinfo.c impl for macOS
  trace2: add macOS process ancestry tracing
2026-02-25 11:54:18 -08:00
D. Ben Knoble
ebeea3c471 build: regenerate config-list.h when Documentation changes
The Meson-based build doesn't know when to rebuild config-list.h, so the
header is sometimes stale.

For example, an old build directory might have config-list.h from before
4173df5187 (submodule: introduce extensions.submodulePathConfig,
2026-01-12), which added submodule.<name>.gitdir to the list. Without
it, t9902-completion.sh fails. Regenerating the config-list.h artifact
from sources fixes the artifact and the test.

Since Meson does not have (or want) builtin support for globbing like
Make, teach generate-configlist.sh to also generate a list of
Documentation files its output depends on, and incorporate that into the
Meson build. We honor the undocumented GCC/Clang contract of outputting
empty targets for all the dependencies (like they do with -MP). That is,
generate lines like

    build/config-list.h: $SOURCE_DIR/Documentation/config.adoc
    $SOURCE_DIR/Documentation/config.adoc:

We assume that if a user adds a new file under
Documentation/config then they will also edit one of the existing files
to include that new file, and that will trigger a rebuild. Also mark the
generator script as a dependency.

While we're at it, teach the Makefile to use the same "the script knows
it's dependencies" logic.

For Meson, combining the following commands helps debug dependencies:

    ninja -C <builddir> -t deps config-list.h
    ninja -C <builddir> -t browse config-list.h

The former lists all the dependencies discovered from our output ".d"
file (the config documentation) and the latter shows the dependency on
the script itself, among other useful edges in the dependency graph.

Helped-by: Patrick Steinhardt <ps@pks.im>
Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: D. Ben Knoble <ben.knoble+github@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-24 10:51:54 -08:00
Patrick Steinhardt
60a546bf17 odb: introduce "files" source
Introduce a new "files" object database source. This source encapsulates
access to both loose object files and the packfile store, similar to how
the "files" backend for refs encapsulates access to loose refs and the
packed-refs file.

Note that for now the "files" source is still a direct member of a
`struct odb_source`. This architecture will be reversed in the next
commit so that the files source contains a `struct odb_source`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-23 13:50:00 -08:00
Patrick Steinhardt
cfcfdf6592 odb: split struct odb_source into separate header
Subsequent commits will expand the `struct odb_source` to become a
generic interface for accessing an object database source. As part of
these refactorings we'll add a set of function pointers that will
significantly expand the structure overall.

Prepare for this by splitting out the `struct odb_source` into a
separate header. This keeps the high-level object database interface
detached from the low-level object database sources.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-23 13:49:59 -08:00
Matthew John Cheetham
088aaf1d41 build: include procinfo.c impl for macOS
Include an implementation of trace2_collect_process_info for macOS.

Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-13 12:18:31 -08:00
Patrick Steinhardt
1e3962c6b7 meson: wire up gitk and git-gui
Wire up both gitk and git-gui in Meson as subprojects. These two
programs should be the last missing pieces for feature compatibility
with our Makefile for distributors.

Note that Meson expects subprojects to live in the "subprojects/"
directory. Create symlinks to fulfill this requirement.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-04 09:42:50 -08:00
Patrick Steinhardt
a675183d48 builtin: add new "history" command
When rewriting history via git-rebase(1) there are a few very common use
cases:

  - The ordering of two commits should be reversed.

  - A commit should be split up into two commits.

  - A commit should be dropped from the history completely.

  - Multiple commits should be squashed into one.

  - Editing an existing commit that is not the tip of the current
    branch.

While these operations are all doable, it often feels needlessly kludgey
to do so by doing an interactive rebase, using the editor to say what
one wants, and then perform the actions. Also, some operations like
splitting up a commit into two are way more involved than that and
require a whole series of commands.

Rebases also do not update dependent branches. The use of stacked
branches has grown quite common with competing version control systems
like Jujutsu though, so it clearly is a need that users have. While
rebases _can_ serve this use case if one always works on the latest
stacked branch, it is somewhat awkward and very easy to get wrong.

Add a new "history" command to plug these gaps. This command will have
several different subcommands to imperatively rewrite history for common
use cases like the above.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-01-13 05:41:17 -08:00
Patrick Steinhardt
6aeda3cf5b builtin/replay: move core logic into "libgit.a"
Move the core logic used to replay commits into "libgit.a" so that it
can be easily reused by other commands. It will be used in a subsequent
commit where we're about to introduce a new git-history(1) command.

Note that with this change we have no sign-comparison warnings anymore,
and neither do we depend on `the_repository`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-01-13 05:41:16 -08:00
Junio C Hamano
91bfbf49b6 Merge branch 'rs/ban-mktemp'
Rewrite the only use of "mktemp()" that is subject to TOCTOU race
and Stop using the insecure "mktemp()" function.

* rs/ban-mktemp:
  compat: remove gitmkdtemp()
  banned.h: ban mktemp(3)
  compat: remove mingw_mktemp()
  compat: use git_mkdtemp()
  wrapper: add git_mkdtemp()
2025-12-16 11:08:35 +09:00
Junio C Hamano
dbe54273a7 Merge branch 'ps/object-read-stream'
The "git_istream" abstraction has been revamped to make it easier
to interface with pluggable object database design.

* ps/object-read-stream:
  streaming: drop redundant type and size pointers
  streaming: move into object database subsystem
  streaming: refactor interface to be object-database-centric
  streaming: move logic to read packed objects streams into backend
  streaming: move logic to read loose objects streams into backend
  streaming: make the `odb_read_stream` definition public
  streaming: get rid of `the_repository`
  streaming: rely on object sources to create object stream
  packfile: introduce function to read object info from a store
  streaming: move zlib stream into backends
  streaming: create structure for filtered object streams
  streaming: create structure for packed object streams
  streaming: create structure for loose object streams
  streaming: create structure for in-core object streams
  streaming: allocate stream inside the backend-specific logic
  streaming: explicitly pass packfile info when streaming a packed object
  streaming: propagate final object type via the stream
  streaming: drop the `open()` callback function
  streaming: rename `git_istream` into `odb_read_stream`
2025-12-16 11:08:34 +09:00
René Scharfe
10bba537c4 compat: remove gitmkdtemp()
gitmkdtemp() has become a trivial wrapper around git_mkdtemp().  Remove
this now unnecessary layer of indirection.

Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-12-07 07:28:13 +09:00
Toon Claes
4061692ba4 meson: use is_cross_build() where possible
In previous commit the first use of meson.can_run_host_binaries() was
introduced. This is a guard around compiler.run() to ensure it's
actually possible to execute the provided.

In other places we've been having the same issue, but here `not
meson.is_cross_build()` is used as guard. This does the trick, but it
also prevents the code from running even when an exe_wrapper is
configured.

Switch to using meson.can_run_host_binaries() here as well.

There is another place left that still uses `not
meson.is_cross_build()`, but here it's a guard around fs.exists(). That
function will always run on the build machine, so checking for
cross-compilation is still in place here.

Signed-off-by: Toon Claes <toon@iotcl.com>
Acked-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-12-05 11:11:15 +09:00
Toon Claes
574ac61076 meson: only detect ICONV_OMITS_BOM if possible
In our Meson setup it automatically detects whether ICONV_OMITS_BOM
should be defined. To check this, a piece of code is compiled and ran.

When cross-compiling, it's not possible to run this piece of code. Guard
this test with a can_run_host_binaries() check to ensure it can run.

Signed-off-by: Toon Claes <toon@iotcl.com>
Acked-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-12-05 11:11:11 +09:00
Junio C Hamano
aea8cc3a10 Merge branch 'jk/asan-bonanza'
Various issues detected by Asan have been corrected.

* jk/asan-bonanza:
  t: enable ASan's strict_string_checks option
  fsck: avoid parse_timestamp() on buffer that isn't NUL-terminated
  fsck: remove redundant date timestamp check
  fsck: avoid strcspn() in fsck_ident()
  fsck: assert newline presence in fsck_ident()
  cache-tree: avoid strtol() on non-string buffer
  Makefile: turn on NO_MMAP when building with ASan
  pack-bitmap: handle name-hash lookups in incremental bitmaps
  compat/mmap: mark unused argument in git_munmap()
2025-11-30 18:31:41 -08:00
Patrick Steinhardt
1599b68d5e streaming: move into object database subsystem
The "streaming" terminology is somewhat generic, so it may not be
immediately obvious that "streaming.{c,h}" is specific to the object
database. Rectify this by moving it into the "odb/" directory so that it
can be immediately attributed to the object subsystem.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-23 12:56:46 -08:00
Jeff King
a9990f8ec0 Makefile: turn on NO_MMAP when building with ASan
Git often uses mmap() to access on-disk files. This leaves a blind spot
in our SANITIZE=address builds, since ASan does not seem to handle mmap
at all. Nor does the OS notice most out-of-bounds access, since it tends
to round up to the nearest page size (so depending on how big the map
is, you might have to overrun it by up to 4095 bytes to trigger a
segfault).

The previous commit demonstrates a memory bug that we missed. We could
have made a new test where the out-of-bounds access was much larger, or
where the mapped file ended closer to a page boundary. But the point of
running the test suite with sanitizers is to catch these problems
without having to construct specific tests.

Let's enable NO_MMAP for our ASan builds by default, which should give
us better coverage. This does increase the memory usage of Git, since
we're copying from the filesystem into heap. But the repositories in the
test suite tend to be small, so the overhead isn't really noticeable
(and ASan already has quite a performance penalty).

There are a few other known bugs that this patch will help flush out.
However, they aren't directly triggered in the test suite (yet). So
it's safe to turn this on now without breaking the test suite, which
will help us add new tests to demonstrate those other bugs as we fix
them.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-18 09:36:06 -08:00
D. Ben Knoble
d63417e3ad meson: make GIT_HTML_PATH configurable
Makefile-based builds can configure Git's internal HTML_PATH by defining
htmldir, which is useful for packagers that put documentation in
different locations. Gentoo, for example, uses version-suffixed
directories like ${prefix}/share/doc/git-2.51 and puts the HTML
documentation in an 'html' subdirectory of the same.

Propagate the same configuration knob to Meson-based builds so that
"git --html-path" on such systems can be configured to output the
correct directory.

Signed-off-by: D. Ben Knoble <ben.knoble+github@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-06 09:58:56 -08:00
Junio C Hamano
c1b23bd8aa Merge branch 'tb/incremental-midx-part-3.1'
Clean-up "git repack" machinery to prepare for incremental update
of midx files.

* tb/incremental-midx-part-3.1: (49 commits)
  builtin/repack.c: clean up unused `#include`s
  repack: move `write_cruft_pack()` out of the builtin
  repack: move `write_filtered_pack()` out of the builtin
  repack: move `pack_kept_objects` to `struct pack_objects_args`
  repack: move `finish_pack_objects_cmd()` out of the builtin
  builtin/repack.c: pass `write_pack_opts` to `finish_pack_objects_cmd()`
  repack: extract `write_pack_opts_is_local()`
  repack: move `find_pack_prefix()` out of the builtin
  builtin/repack.c: use `write_pack_opts` within `write_cruft_pack()`
  builtin/repack.c: introduce `struct write_pack_opts`
  repack: 'write_midx_included_packs' API from the builtin
  builtin/repack.c: inline packs within `write_midx_included_packs()`
  builtin/repack.c: pass `repack_write_midx_opts` to `midx_included_packs`
  builtin/repack.c: inline `remove_redundant_bitmaps()`
  builtin/repack.c: reorder `remove_redundant_bitmaps()`
  repack: keep track of MIDX pack names using existing_packs
  builtin/repack.c: use a string_list for 'midx_pack_names'
  builtin/repack.c: extract opts struct for 'write_midx_included_packs()'
  builtin/repack.c: remove ref snapshotting from builtin
  repack: remove pack_geometry API from the builtin
  ...
2025-10-29 12:38:24 -07:00
Junio C Hamano
fe95c55549 Merge branch 'ps/ci-rust'
CI improvements to handle the recent Rust integration better.

* ps/ci-rust:
  rust: support for Windows
  ci: verify minimum supported Rust version
  ci: check for common Rust mistakes via Clippy
  rust/varint: add safety comments
  ci: check formatting of our Rust code
  ci: deduplicate calls to `apt-get update`
2025-10-28 10:29:09 -07:00
Taylor Blau
09797bd966 repack: move write_cruft_pack() out of the builtin
In an identical fashion as the previous commit, move the function
`write_cruft_pack()` into its own compilation unit, and make the
function visible through the repack.h API.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-16 10:08:57 -07:00
Taylor Blau
7ac4231b42 repack: move write_filtered_pack() out of the builtin
In a similar fashion as in previous commits, move the function
`write_filtered_pack()` out of the builtin and into its own compilation
unit.

This function is now part of the repack.h API, but implemented in its
own "repack-filtered.c" unit as it is a separate component from other
kinds of repacking operations.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-16 10:08:57 -07:00
Taylor Blau
ccb7f822d5 builtin/repack.c: remove ref snapshotting from builtin
When writing a MIDX, 'git repack' takes a snapshot of the repository's
references and writes the result out to a file, which it then passes to
'git multi-pack-index write' via the '--refs-snapshot'.

This is done in order to make bitmap selections with respect to what we
are packing, thus avoiding a race where an incoming reference update
causes us to try and write a bitmap for a commit not present in the
MIDX.

Extract this functionality out into a new repack-midx.c compilation
unit, and expose the necessary functions via the repack.h API.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-16 10:08:55 -07:00
Taylor Blau
62d3fa09b3 repack: remove pack_geometry API from the builtin
Now that the pack_geometry API is fully factored and isolated from the
rest of the builtin, declare it within repack.h and move its
implementation to "repack-geometry.c" as a separate component.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-16 10:08:55 -07:00
Taylor Blau
29e935515d builtin/repack.c: remove "repack_promisor_objects()" from the builtin
Now that we have properly factored the portion of the builtin which is
responsible for repacking promisor objects, we can move that function
(and associated dependencies) out of the builtin entirely.

Similar to previous extractions, this function is declared in repack.h,
but implemented in a separate repack-promisor.c file. This is done to
separate promisor-specific repacking functionality from generic repack
utilities (like "existing_packs", and "generated_pack" APIs).

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-16 10:08:55 -07:00
Taylor Blau
c7a120722e repack: introduce new compilation unit
Over the years, builtin/repack.c has turned into a grab-bag of
functionality powering the 'git repack' builtin. Among its many
capabilities, it:

 - can build and spawn 'git pack-objects' commands, which in turn
   generate new packs
 - has infrastructure to manage the set of existing packs in a
   repository
 - has infrastructure to split a sequence of packs into a geometric
   progression based on object size
 - can manage both generating and combining cruft packs together
 - can write new MIDXs

to name a few.

As a result, this builtin has accumulated a lot of code, making adding
new functionality difficult. In the future, 'repack' will learn how to
manage a chain of incremental MIDXs, adding yet more functionality into
the builtin.

As a prerequisite step, let's first move some of the functionality in
the builtin into its own repack.[ch].

This will be done over the course of many steps, since there are many
individual components, some of which will end up in other, yet-to-exist
compilation units of their own. Some of the code movement here is also
non-trivial, so performing it in individual steps will make it easier to
verify.

Let's start by migrating 'struct pack_objects_args' (and the related
corresponding pack_objects_args_release() function) into repack.h, and
teach both the Makefile and Meson how to build the new compilation unit.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-16 10:08:53 -07:00
Patrick Steinhardt
e509b5b8be rust: support for Windows
The initial patch series that introduced Rust into the core of Git only
cared about macOS and Linux. This specifically leaves out Windows, which
indeed fails to build right now due to two issues:

  - The Rust runtime requires `GetUserProfileDirectoryW()`, but we don't
    link against "userenv.dll".

  - The path of the Rust library built on Windows is different than on
    most other systems systems.

Fix both of these issues to support Windows.

Note that this commit fixes the Meson-based job in GitHub's CI. Meson
auto-detects the availability of Rust, and as the Windows runner has
Rust installed by default it already enabled Rust support there. But due
to the above issues that job fails consistently.

Install Rust on GitLab CI, as well, to improve test coverage there.

Based-on-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Based-on-patch-by: Ezekiel Newren <ezekielnewren@gmail.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-15 08:10:17 -07:00
Junio C Hamano
f50f046794 Merge branch 'kn/reftable-consistency-checks'
The reftable backend learned to sanity check its on-disk data more
carefully.

* kn/reftable-consistency-checks:
  refs/reftable: add fsck check for checking the table name
  reftable: add code to facilitate consistency checks
  fsck: order 'fsck_msg_type' alphabetically
  Documentation/fsck-msgids: remove duplicate msg id
  reftable: check for trailing newline in 'tables.list'
  refs: move consistency check msg to generic layer
  refs: remove unused headers
2025-10-13 22:00:35 -07:00
Junio C Hamano
75f8dfabaa Merge branch 'ps/rust-balloon'
Dip our toes a bit to (optionally) use Rust implemented helper
called from our C code.

* ps/rust-balloon:
  ci: enable Rust for breaking-changes jobs
  ci: convert "pedantic" job into full build with breaking changes
  BreakingChanges: announce Rust becoming mandatory
  varint: reimplement as test balloon for Rust
  varint: use explicit width for integers
  help: report on whether or not Rust is enabled
  Makefile: introduce infrastructure to build internal Rust library
  Makefile: reorder sources after includes
  meson: add infrastructure to build internal Rust library
2025-10-08 12:17:55 -07:00
Junio C Hamano
5f91b2c43f Merge branch 'ps/rust-balloon' into ps/ci-rust
* ps/rust-balloon:
  ci: enable Rust for breaking-changes jobs
  ci: convert "pedantic" job into full build with breaking changes
  BreakingChanges: announce Rust becoming mandatory
  varint: reimplement as test balloon for Rust
  varint: use explicit width for integers
  help: report on whether or not Rust is enabled
  Makefile: introduce infrastructure to build internal Rust library
  Makefile: reorder sources after includes
  meson: add infrastructure to build internal Rust library
2025-10-07 10:55:39 -07:00
Karthik Nayak
9051638519 reftable: add code to facilitate consistency checks
The `git refs verify` command is used to run consistency checks on the
reference backends. This command is also invoked when users run 'git
fsck'. While the files-backend has some fsck checks added, the reftable
backend lacks such checks. Let's add the required infrastructure and a
check to test for the files present in the reftable directory.

Since the reftable library is treated as an independent library we
should ensure that the library code works independently without
knowledge about Git's internals. To do this, add both 'reftable/fsck.c'
and 'reftable/reftable-fsck.h'. Which provide an entry point
'reftable_fsck_check' for running fsck checks over a provided reftable
stack. The callee provides the function with callbacks to handle issue
and information reporting.

The added check, goes over all tables in the reftable stack validates
that they have a valid name. It not, it raises an error.

While here, move 'reftable/error.o' in the Makefile to retain
lexicographic ordering.

Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-07 09:22:58 -07:00
Junio C Hamano
2f49ec7991 Merge branch 'ps/meson-build-docs'
The build procedure based on meson learned a target to only build
documentation, similar to "make doc".

* ps/meson-build-docs:
  ci: don't compile whole project when testing docs with Meson
  meson: print docs backend as part of the summary
  meson: introduce a "docs" alias to compile documentation only
2025-10-02 12:26:12 -07:00
Junio C Hamano
db0babf9b2 Merge branch 'ms/refs-optimize'
"git refs optimize" is added for not very well explained reason
despite it does the same thing as "git pack-refs"...

* ms/refs-optimize:
  t: add test for git refs optimize subcommand
  t0601: refactor tests to be shareable
  builtin/refs: add optimize subcommand
  doc: pack-refs: factor out common options
  builtin/pack-refs: factor out core logic into a shared library
  builtin/pack-refs: convert to use the generic refs_optimize() API
  reftable-backend: implement 'optimize' action
  files-backend: implement 'optimize' action
  refs: add a generic 'optimize' API
2025-10-02 12:26:12 -07:00
Patrick Steinhardt
8832e728d3 varint: reimplement as test balloon for Rust
Implement a trivial test balloon for our Rust build infrastructure by
reimplementing the "varint.c" subsystem in Rust. This subsystem is
chosen because it is trivial to convert and because it doesn't have any
dependencies to other components of Git.

If support for Rust is enabled, we stop compiling "varint.c" and instead
compile and use "src/varint.rs".

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-02 09:32:32 -07:00
Patrick Steinhardt
c184795fc0 meson: add infrastructure to build internal Rust library
Add the infrastructure into Meson to build an internal Rust library.
Building the Rust parts of Git are for now entirely optional, as they
are mostly intended as a test balloon for both Git developers, but also
for distributors of Git. So for now, they may contain:

  - New features that are not mission critical to Git and that users can
    easily live without.

  - Alternative implementations of small subsystems.

If these test balloons are successful, we will eventually make Rust a
mandatory dependency for our build process in Git 3.0.

The availability of a Rust toolchain will be auto-detected by Meson at
setup time. This behaviour can be tweaked via the `-Drust=` feature
toggle.

Next to the linkable Rust library, also wire up tests that can be
executed via `meson test`. This allows us to use the native unit testing
capabilities of Rust.

Note that the Rust edition is currently set to 2018. This edition is
supported by Rust 1.49, which is the target for the upcoming gcc-rs
backend. For now we don't use any features of Rust that would require a
newer version, so settling on this old version makes sense so that
gcc-rs may become an alternative backend for compiling Git. If we _do_
want to introduce features that were added in more recent editions of
Rust though we should reevaluate that choice.

Inspired-by: Ezekiel Newren <ezekielnewren@gmail.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-10-02 09:32:31 -07:00
Meet Soni
0d4ec33922 builtin/pack-refs: factor out core logic into a shared library
The implementation of `git pack-refs` is monolithic within
`cmd_pack_refs()`, making it impossible to share its logic with other
commands. To enable code reuse for the upcoming `git refs optimize`
subcommand, refactor the core logic into a shared helper function.

Split the original `builtin/pack-refs.c` file into two parts:

- A new shared library file, `pack-refs.c`, which contains the
  core option parsing and packing logic in a new `pack_refs_core()`
  helper function.

- The original `builtin/pack-refs.c`, which is now a thin wrapper
  responsible only for defining the `git pack-refs` command and
  calling the shared helper.

A new `pack-refs.h` header is also introduced to define the public
interface for this shared logic.

Mentored-by: Patrick Steinhardt <ps@pks.im>
Mentored-by: shejialuo <shejialuo@gmail.com>
Signed-off-by: Meet Soni <meetsoni3017@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-09-19 10:02:55 -07:00
Justin Tobler
78839e9cde object-file: relocate ODB transaction code
The bulk-checkin subsystem provides various functions to manage ODB
transactions. Apart from {begin,end}_odb_transaction(), these functions
are only used by the object-file subsystem to manage aspects of a
transaction implementation specific to the files object source.

Relocate all the transaction code in bulk-checkin to object-file. This
simplifies the exposed transaction interface by reducing it to only
{begin,end}_odb_transaction(). Function and type names are adjusted in
the subsequent commit to better fit the new location.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-09-16 11:37:05 -07:00
Patrick Steinhardt
b64579dff9 meson: print docs backend as part of the summary
Our documentation can be built with either Asciidoc or Asciidoctor as
backend. When Meson is configured to build documentation, then it will
automatically detect which of these tools is available and use them.
It's not obvious to the user though which of these backends is used
unless the user explicitly asks for one backend via `-Ddocs_backend=`.

Improve the status quo by printing the docs backend as part of the
"backends" summary.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-09-11 09:57:24 -07:00
Patrick Steinhardt
197f0d0f39 meson: introduce a "docs" alias to compile documentation only
Meson does not currently provide a target to compile documentation,
only. Instead, users needs to compile the whole project, which may be
way more than they really intend to do.

Introduce a new "docs" alias to plug this gap. This alias can be invoked
e.g. with `meson compile docs`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-09-11 09:57:24 -07:00
Junio C Hamano
95a8428323 Merge branch 'tc/last-modified'
A new command "git last-modified" has been added to show the closest
ancestor commit that touched each path.

* tc/last-modified:
  last-modified: use Bloom filters when available
  t/perf: add last-modified perf script
  last-modified: new subcommand to show when files were last modified
2025-09-08 14:54:35 -07:00
Toon Claes
32f74582bc last-modified: new subcommand to show when files were last modified
Similar to git-blame(1), introduce a new subcommand
git-last-modified(1). This command shows the most recent modification to
paths in a tree. It does so by expanding the tree at a given commit,
taking note of the current state of each path, and then walking
backwards through history looking for commits where each path changed
into its final commit ID.

Based-on-patch-by: Jeff King <peff@peff.net>
Improved-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Toon Claes <toon@iotcl.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-08-28 16:44:58 -07:00
Junio C Hamano
ebb45da976 Merge branch 'lo/repo-info'
A new subcommand "git repo" gives users a way to grab various
repository characteristics.

* lo/repo-info:
  repo: add the --format flag
  repo: add the field layout.shallow
  repo: add the field layout.bare
  repo: add the field references.format
  repo: declare the repo command
2025-08-25 14:22:04 -07:00