undef/git-commit-vandalism
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
e841a79a13
git-commit-vandalism/t/t1416-ref-transaction-hooks.sh

137 lines
3.2 KiB
Bash
Raw Normal View History

refs: implement reference transaction hook The low-level reference transactions used to update references are currently completely opaque to the user. While certainly desirable in most usecases, there are some which might want to hook into the transaction to observe all queued reference updates as well as observing the abortion or commit of a prepared transaction. One such usecase would be to have a set of replicas of a given Git repository, where we perform Git operations on all of the repositories at once and expect the outcome to be the same in all of them. While there exist hooks already for a certain subset of Git commands that could be used to implement a voting mechanism for this, many others currently don't have any mechanism for this. The above scenario is the motivation for the new "reference-transaction" hook that reaches directly into Git's reference transaction mechanism. The hook receives as parameter the current state the transaction was moved to ("prepared", "committed" or "aborted") and gets via its standard input all queued reference updates. While the exit code gets ignored in the "committed" and "aborted" states, a non-zero exit code in the "prepared" state will cause the transaction to be aborted prematurely. Given the usecase described above, a voting mechanism can now be implemented via this hook: as soon as it gets called, it will take all of stdin and use it to cast a vote to a central service. When all replicas of the repository agree, the hook will exit with zero, otherwise it will abort the transaction by returning non-zero. The most important upside is that this will catch _all_ commands writing references at once, allowing to implement strong consistency for reference updates via a single mechanism. In order to test the impact on the case where we don't have any "reference-transaction" hook installed in the repository, this commit introduce two new performance tests for git-update-refs(1). Run against an empty repository, it produces the following results: Test origin/master HEAD -------------------------------------------------------------------- 1400.2: update-ref 2.70(2.10+0.71) 2.71(2.10+0.73) +0.4% 1400.3: update-ref --stdin 0.21(0.09+0.11) 0.21(0.07+0.14) +0.0% The performance test p1400.2 creates, updates and deletes a branch a thousand times, thus averaging runtime of git-update-refs over 3000 invocations. p1400.3 instead calls `git-update-refs --stdin` three times and queues a thousand creations, updates and deletes respectively. As expected, p1400.3 consistently shows no noticeable impact, as for each batch of updates there's a single call to access(3P) for the negative hook lookup. On the other hand, for p1400.2, one can see an impact caused by this patchset. But doing five runs of the performance tests where each one was run with GIT_PERF_REPEAT_COUNT=10, the overhead ranged from -1.5% to +1.1%. These inconsistent performance numbers can be explained by the overhead of spawning 3000 processes. This shows that the overhead of assembling the hook path and executing access(3P) once to check if it's there is mostly outweighed by the operating system's overhead. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-19 08:56:14 +02:00
#!/bin/sh
test_description='reference transaction hooks'
. ./test-lib.sh
test_expect_success setup '
mkdir -p .git/hooks &&
test_commit PRE &&
t1416: avoid hard-coded sha1 ids The test added by e5256c82e5 (refs: fix interleaving hook calls with reference-transaction hook, 2020-08-07) uses hard-coded sha1 object ids in its expected output. This causes it to fail when run with GIT_TEST_DEFAULT_HASH=sha256. Let's make use of the oid variables we define earlier, as the rest of the nearby tests do. Signed-off-by: Jeff King <peff@peff.net> Reviewed-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-11 08:53:47 +02:00
PRE_OID=$(git rev-parse PRE) &&
refs: implement reference transaction hook The low-level reference transactions used to update references are currently completely opaque to the user. While certainly desirable in most usecases, there are some which might want to hook into the transaction to observe all queued reference updates as well as observing the abortion or commit of a prepared transaction. One such usecase would be to have a set of replicas of a given Git repository, where we perform Git operations on all of the repositories at once and expect the outcome to be the same in all of them. While there exist hooks already for a certain subset of Git commands that could be used to implement a voting mechanism for this, many others currently don't have any mechanism for this. The above scenario is the motivation for the new "reference-transaction" hook that reaches directly into Git's reference transaction mechanism. The hook receives as parameter the current state the transaction was moved to ("prepared", "committed" or "aborted") and gets via its standard input all queued reference updates. While the exit code gets ignored in the "committed" and "aborted" states, a non-zero exit code in the "prepared" state will cause the transaction to be aborted prematurely. Given the usecase described above, a voting mechanism can now be implemented via this hook: as soon as it gets called, it will take all of stdin and use it to cast a vote to a central service. When all replicas of the repository agree, the hook will exit with zero, otherwise it will abort the transaction by returning non-zero. The most important upside is that this will catch _all_ commands writing references at once, allowing to implement strong consistency for reference updates via a single mechanism. In order to test the impact on the case where we don't have any "reference-transaction" hook installed in the repository, this commit introduce two new performance tests for git-update-refs(1). Run against an empty repository, it produces the following results: Test origin/master HEAD -------------------------------------------------------------------- 1400.2: update-ref 2.70(2.10+0.71) 2.71(2.10+0.73) +0.4% 1400.3: update-ref --stdin 0.21(0.09+0.11) 0.21(0.07+0.14) +0.0% The performance test p1400.2 creates, updates and deletes a branch a thousand times, thus averaging runtime of git-update-refs over 3000 invocations. p1400.3 instead calls `git-update-refs --stdin` three times and queues a thousand creations, updates and deletes respectively. As expected, p1400.3 consistently shows no noticeable impact, as for each batch of updates there's a single call to access(3P) for the negative hook lookup. On the other hand, for p1400.2, one can see an impact caused by this patchset. But doing five runs of the performance tests where each one was run with GIT_PERF_REPEAT_COUNT=10, the overhead ranged from -1.5% to +1.1%. These inconsistent performance numbers can be explained by the overhead of spawning 3000 processes. This shows that the overhead of assembling the hook path and executing access(3P) once to check if it's there is mostly outweighed by the operating system's overhead. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-19 08:56:14 +02:00
test_commit POST &&
POST_OID=$(git rev-parse POST)
'
test_expect_success 'hook allows updating ref if successful' '
test_when_finished "rm .git/hooks/reference-transaction" &&
git reset --hard PRE &&
write_script .git/hooks/reference-transaction <<-\EOF &&
echo "$*" >>actual
EOF
cat >expect <<-EOF &&
prepared
committed
EOF
git update-ref HEAD POST &&
test_cmp expect actual
'
test_expect_success 'hook aborts updating ref in prepared state' '
test_when_finished "rm .git/hooks/reference-transaction" &&
git reset --hard PRE &&
write_script .git/hooks/reference-transaction <<-\EOF &&
if test "$1" = prepared
then
exit 1
fi
EOF
test_must_fail git update-ref HEAD POST 2>err &&
test_i18ngrep "ref updates aborted by hook" err
'
test_expect_success 'hook gets all queued updates in prepared state' '
test_when_finished "rm .git/hooks/reference-transaction actual" &&
git reset --hard PRE &&
write_script .git/hooks/reference-transaction <<-\EOF &&
if test "$1" = prepared
then
while read -r line
do
printf "%s\n" "$line"
done >actual
fi
EOF
cat >expect <<-EOF &&
$ZERO_OID $POST_OID HEAD
$ZERO_OID $POST_OID refs/heads/master
EOF
git update-ref HEAD POST <<-EOF &&
update HEAD $ZERO_OID $POST_OID
update refs/heads/master $ZERO_OID $POST_OID
EOF
test_cmp expect actual
'
test_expect_success 'hook gets all queued updates in committed state' '
test_when_finished "rm .git/hooks/reference-transaction actual" &&
git reset --hard PRE &&
write_script .git/hooks/reference-transaction <<-\EOF &&
if test "$1" = committed
then
while read -r line
do
printf "%s\n" "$line"
done >actual
fi
EOF
cat >expect <<-EOF &&
$ZERO_OID $POST_OID HEAD
$ZERO_OID $POST_OID refs/heads/master
EOF
git update-ref HEAD POST &&
test_cmp expect actual
'
test_expect_success 'hook gets all queued updates in aborted state' '
test_when_finished "rm .git/hooks/reference-transaction actual" &&
git reset --hard PRE &&
write_script .git/hooks/reference-transaction <<-\EOF &&
if test "$1" = aborted
then
while read -r line
do
printf "%s\n" "$line"
done >actual
fi
EOF
cat >expect <<-EOF &&
$ZERO_OID $POST_OID HEAD
$ZERO_OID $POST_OID refs/heads/master
EOF
git update-ref --stdin <<-EOF &&
start
update HEAD POST $ZERO_OID
update refs/heads/master POST $ZERO_OID
abort
EOF
test_cmp expect actual
'
refs: fix interleaving hook calls with reference-transaction hook In order to not repeatedly search for the reference-transaction hook in case it's getting called multiple times, we use a caching mechanism to only call `find_hook()` once. What was missed though is that the return value of `find_hook()` actually comes from a static strbuf, which means it will get overwritten when calling `find_hook()` again. As a result, we may call the wrong hook with parameters of the reference-transaction hook. This scenario was spotted in the wild when executing a git-push(1) with multiple references, where there are interleaving calls to both the update and the reference-transaction hook. While initial calls to the reference-transaction hook work as expected, it will stop working after the next invocation of the update hook. The result is that we now start calling the update hook with parameters and stdin of the reference-transaction hook. This commit fixes the issue by storing a copy of `find_hook()`'s return value in the cache. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-07 09:05:58 +02:00
test_expect_success 'interleaving hook calls succeed' '
test_when_finished "rm -r target-repo.git" &&
git init --bare target-repo.git &&
write_script target-repo.git/hooks/reference-transaction <<-\EOF &&
echo $0 "$@" >>actual
EOF
write_script target-repo.git/hooks/update <<-\EOF &&
echo $0 "$@" >>actual
EOF
cat >expect <<-EOF &&
t1416: avoid hard-coded sha1 ids The test added by e5256c82e5 (refs: fix interleaving hook calls with reference-transaction hook, 2020-08-07) uses hard-coded sha1 object ids in its expected output. This causes it to fail when run with GIT_TEST_DEFAULT_HASH=sha256. Let's make use of the oid variables we define earlier, as the rest of the nearby tests do. Signed-off-by: Jeff King <peff@peff.net> Reviewed-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-11 08:53:47 +02:00
hooks/update refs/tags/PRE $ZERO_OID $PRE_OID
refs: fix interleaving hook calls with reference-transaction hook In order to not repeatedly search for the reference-transaction hook in case it's getting called multiple times, we use a caching mechanism to only call `find_hook()` once. What was missed though is that the return value of `find_hook()` actually comes from a static strbuf, which means it will get overwritten when calling `find_hook()` again. As a result, we may call the wrong hook with parameters of the reference-transaction hook. This scenario was spotted in the wild when executing a git-push(1) with multiple references, where there are interleaving calls to both the update and the reference-transaction hook. While initial calls to the reference-transaction hook work as expected, it will stop working after the next invocation of the update hook. The result is that we now start calling the update hook with parameters and stdin of the reference-transaction hook. This commit fixes the issue by storing a copy of `find_hook()`'s return value in the cache. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-07 09:05:58 +02:00
hooks/reference-transaction prepared
hooks/reference-transaction committed
t1416: avoid hard-coded sha1 ids The test added by e5256c82e5 (refs: fix interleaving hook calls with reference-transaction hook, 2020-08-07) uses hard-coded sha1 object ids in its expected output. This causes it to fail when run with GIT_TEST_DEFAULT_HASH=sha256. Let's make use of the oid variables we define earlier, as the rest of the nearby tests do. Signed-off-by: Jeff King <peff@peff.net> Reviewed-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-11 08:53:47 +02:00
hooks/update refs/tags/POST $ZERO_OID $POST_OID
refs: fix interleaving hook calls with reference-transaction hook In order to not repeatedly search for the reference-transaction hook in case it's getting called multiple times, we use a caching mechanism to only call `find_hook()` once. What was missed though is that the return value of `find_hook()` actually comes from a static strbuf, which means it will get overwritten when calling `find_hook()` again. As a result, we may call the wrong hook with parameters of the reference-transaction hook. This scenario was spotted in the wild when executing a git-push(1) with multiple references, where there are interleaving calls to both the update and the reference-transaction hook. While initial calls to the reference-transaction hook work as expected, it will stop working after the next invocation of the update hook. The result is that we now start calling the update hook with parameters and stdin of the reference-transaction hook. This commit fixes the issue by storing a copy of `find_hook()`'s return value in the cache. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-07 09:05:58 +02:00
hooks/reference-transaction prepared
hooks/reference-transaction committed
EOF
git push ./target-repo.git PRE POST &&
test_cmp expect target-repo.git/actual
'
refs: implement reference transaction hook The low-level reference transactions used to update references are currently completely opaque to the user. While certainly desirable in most usecases, there are some which might want to hook into the transaction to observe all queued reference updates as well as observing the abortion or commit of a prepared transaction. One such usecase would be to have a set of replicas of a given Git repository, where we perform Git operations on all of the repositories at once and expect the outcome to be the same in all of them. While there exist hooks already for a certain subset of Git commands that could be used to implement a voting mechanism for this, many others currently don't have any mechanism for this. The above scenario is the motivation for the new "reference-transaction" hook that reaches directly into Git's reference transaction mechanism. The hook receives as parameter the current state the transaction was moved to ("prepared", "committed" or "aborted") and gets via its standard input all queued reference updates. While the exit code gets ignored in the "committed" and "aborted" states, a non-zero exit code in the "prepared" state will cause the transaction to be aborted prematurely. Given the usecase described above, a voting mechanism can now be implemented via this hook: as soon as it gets called, it will take all of stdin and use it to cast a vote to a central service. When all replicas of the repository agree, the hook will exit with zero, otherwise it will abort the transaction by returning non-zero. The most important upside is that this will catch _all_ commands writing references at once, allowing to implement strong consistency for reference updates via a single mechanism. In order to test the impact on the case where we don't have any "reference-transaction" hook installed in the repository, this commit introduce two new performance tests for git-update-refs(1). Run against an empty repository, it produces the following results: Test origin/master HEAD -------------------------------------------------------------------- 1400.2: update-ref 2.70(2.10+0.71) 2.71(2.10+0.73) +0.4% 1400.3: update-ref --stdin 0.21(0.09+0.11) 0.21(0.07+0.14) +0.0% The performance test p1400.2 creates, updates and deletes a branch a thousand times, thus averaging runtime of git-update-refs over 3000 invocations. p1400.3 instead calls `git-update-refs --stdin` three times and queues a thousand creations, updates and deletes respectively. As expected, p1400.3 consistently shows no noticeable impact, as for each batch of updates there's a single call to access(3P) for the negative hook lookup. On the other hand, for p1400.2, one can see an impact caused by this patchset. But doing five runs of the performance tests where each one was run with GIT_PERF_REPEAT_COUNT=10, the overhead ranged from -1.5% to +1.1%. These inconsistent performance numbers can be explained by the overhead of spawning 3000 processes. This shows that the overhead of assembling the hook path and executing access(3P) once to check if it's there is mostly outweighed by the operating system's overhead. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-19 08:56:14 +02:00
test_done
Reference in New Issue Copy Permalink