From 96eaffebbf3d00a498c28bc689e70d9d94bc9911 Mon Sep 17 00:00:00 2001 From: Derrick Stolee Date: Tue, 19 Jan 2021 12:52:03 +0000 Subject: [PATCH 1/2] maintenance: set log.excludeDecoration durin prefetch The 'prefetch' task fetches refs from all remotes and places them in the refs/prefetch// refspace. As this task is intended to run in the background, this allows users to keep their local data very close to the remote servers' data while not updating the users' understanding of the remote refs in refs/remotes//. However, this can clutter 'git log' decorations with copies of the refs with the full name 'refs/prefetch//'. The log.excludeDecoration config option was added in a6be5e67 (log: add log.excludeDecoration config option, 2020-05-16) for exactly this purpose. Ensure we set this only for users that would benefit from it by assigning it at the beginning of the prefetch task. Other alternatives would be during 'git maintenance register' or 'git maintenance start', but those might assign the config even when the prefetch task is disabled by existing config. Further, users could run 'git maintenance run --task=prefetch' using their own scripting or scheduling. This provides the best coverage to automatically update the config when valuable. It is improbable, but possible, that users might want to run the prefetch task _and_ see these refs in their log decorations. This seems incredibly unlikely to me, but users can always opt-in on a command-by-command basis using --decorate-refs=refs/prefetch/. Test that this works in a few cases. In particular, ensure that our assignment of log.excludeDecoration=refs/prefetch/ is additive to other existing exclusions. Further, ensure we do not add multiple copies in multiple runs. Signed-off-by: Derrick Stolee Reviewed-by: Taylor Blau Signed-off-by: Junio C Hamano --- builtin/gc.c | 6 ++++++ t/t7900-maintenance.sh | 26 +++++++++++++++++++++++++- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/builtin/gc.c b/builtin/gc.c index b315b2ad58..54bae7f0c4 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -897,6 +897,12 @@ static int maintenance_task_prefetch(struct maintenance_run_opts *opts) struct string_list_item *item; struct string_list remotes = STRING_LIST_INIT_DUP; + git_config_set_multivar_gently("log.excludedecoration", + "refs/prefetch/", + "refs/prefetch/", + CONFIG_FLAGS_FIXED_VALUE | + CONFIG_FLAGS_MULTI_REPLACE); + if (for_each_remote(append_remote, &remotes)) { error(_("failed to fill remotes")); result = 1; diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh index 1074009cc0..f9031cbb44 100755 --- a/t/t7900-maintenance.sh +++ b/t/t7900-maintenance.sh @@ -149,7 +149,31 @@ test_expect_success 'prefetch multiple remotes' ' git log prefetch/remote2/two && git fetch --all && test_cmp_rev refs/remotes/remote1/one refs/prefetch/remote1/one && - test_cmp_rev refs/remotes/remote2/two refs/prefetch/remote2/two + test_cmp_rev refs/remotes/remote2/two refs/prefetch/remote2/two && + + test_cmp_config refs/prefetch/ log.excludedecoration && + git log --oneline --decorate --all >log && + ! grep "prefetch" log +' + +test_expect_success 'prefetch and existing log.excludeDecoration values' ' + git config --unset-all log.excludeDecoration && + git config log.excludeDecoration refs/remotes/remote1/ && + git maintenance run --task=prefetch && + + git config --get-all log.excludeDecoration >out && + grep refs/remotes/remote1/ out && + grep refs/prefetch/ out && + + git log --oneline --decorate --all >log && + ! grep "prefetch" log && + ! grep "remote1" log && + grep "remote2" log && + + # a second run does not change the config + git maintenance run --task=prefetch && + git log --oneline --decorate --all >log2 && + test_cmp log log2 ' test_expect_success 'loose-objects task' ' From 3cf5f221bec9cefcb472e71e2cf378aecbb6f33f Mon Sep 17 00:00:00 2001 From: Derrick Stolee Date: Tue, 19 Jan 2021 12:52:04 +0000 Subject: [PATCH 2/2] t7900: clean up some broken refs The tests for the 'prefetch' task create remotes and fetch refs into 'refs/prefetch//' and tags into 'refs/tags/'. These tests use the remotes to create objects not intended to be seen by the "local" repository. In that sense, the incrmental-repack tasks did not have these objects and refs in mind. That test replaces the object directory with a specific pack-file layout for testing the batch-size logic. However, this causes some operations to start showing warnings such as: error: refs/prefetch/remote1/one does not point to a valid object! error: refs/tags/one does not point to a valid object! This only shows up if you run the tests verbosely and watch the output. It caught my eye and I _thought_ that there was a bug where 'git gc' or 'git repack' wouldn't check 'refs/prefetch/' before pruning objects. That is incorrect. Those commands do handle 'refs/prefetch/' correctly. All that is left is to clean up the tests in t7900-maintenance.sh to remove these tags and refs that are not being repacked for the incremental-repack tests. Use update-ref to ensure this works with all ref backends. Helped-by: Taylor Blau Signed-off-by: Derrick Stolee Reviewed-by: Taylor Blau Signed-off-by: Junio C Hamano --- t/t7900-maintenance.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh index f9031cbb44..78ccf4b33f 100755 --- a/t/t7900-maintenance.sh +++ b/t/t7900-maintenance.sh @@ -256,6 +256,13 @@ test_expect_success 'incremental-repack task' ' HEAD ^HEAD~1 EOF + + # Delete refs that have not been repacked in these packs. + git for-each-ref --format="delete %(refname)" \ + refs/prefetch refs/tags >refs && + git update-ref --stdin packs-before &&