Skip to content

Commit e499896

Browse files
committed
Merge branch 'ar/parallel-hooks' into jch
* ar/parallel-hooks: t1800: test SIGPIPE with parallel hooks hook: allow hook.jobs=-1 to use all available CPU cores hook: add hook.<event>.enabled switch hook: move is_known_hook() to hook.c for wider use hook: warn when hook.<friendly-name>.jobs is set hook: add per-event jobs config hook: add -j/--jobs option to git hook run hook: mark non-parallelizable hooks hook: allow pre-push parallel execution hook: allow parallel hook execution hook: parse the hook.jobs config config: add a repo_config_get_uint() helper repository: fix repo_init() memleak due to missing _clear()
2 parents 65e3e4b + 75b7cb5 commit e499896

21 files changed

Lines changed: 1119 additions & 70 deletions

Documentation/config/hook.adoc

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,10 +15,86 @@ hook.<friendly-name>.event::
1515
events, specify the key more than once. An empty value resets
1616
the list of events, clearing any previously defined events for
1717
`hook.<friendly-name>`. See linkgit:git-hook[1].
18+
+
19+
The `<friendly-name>` must not be the same as a known hook event name
20+
(e.g. do not use `hook.pre-commit.event`). Using a known event name as
21+
a friendly-name is a fatal error because it creates an ambiguity with
22+
`hook.<event>.enabled` and `hook.<event>.jobs`. For unknown event names,
23+
a warning is issued when `<friendly-name>` matches the event value.
1824

1925
hook.<friendly-name>.enabled::
2026
Whether the hook `hook.<friendly-name>` is enabled. Defaults to `true`.
2127
Set to `false` to disable the hook without removing its
2228
configuration. This is particularly useful when a hook is defined
2329
in a system or global config file and needs to be disabled for a
2430
specific repository. See linkgit:git-hook[1].
31+
32+
hook.<friendly-name>.parallel::
33+
Whether the hook `hook.<friendly-name>` may run in parallel with other hooks
34+
for the same event. Defaults to `false`. Set to `true` only when the
35+
hook script is safe to run concurrently with other hooks for the same
36+
event. If any hook for an event does not have this set to `true`,
37+
all hooks for that event run sequentially regardless of `hook.jobs`.
38+
Only configured (named) hooks need to declare this. Traditional hooks
39+
found in the hooks directory do not need to, and run in parallel when
40+
the effective job count is greater than 1. See linkgit:git-hook[1].
41+
42+
hook.<event>.enabled::
43+
Switch to enable or disable all hooks for the `<event>` hook event.
44+
When set to `false`, no hooks fire for that event, regardless of any
45+
per-hook `hook.<friendly-name>.enabled` settings. Defaults to `true`.
46+
See linkgit:git-hook[1].
47+
+
48+
Note on naming: `<event>` must be the event name (e.g. `pre-commit`),
49+
not a hook friendly-name. Since using a known event name as a
50+
friendly-name is disallowed (see `hook.<friendly-name>.event` above),
51+
there is no ambiguity between event-level and per-hook `.enabled`
52+
settings for known events. For unknown events, if a friendly-name
53+
matches the event name despite the warning, `.enabled` is treated
54+
as per-hook only.
55+
56+
hook.<event>.jobs::
57+
Specifies how many hooks can be run simultaneously for the `<event>`
58+
hook event (e.g. `hook.post-receive.jobs = 4`). Overrides `hook.jobs`
59+
for this specific event. The same parallelism restrictions apply: this
60+
setting has no effect unless all configured hooks for the event have
61+
`hook.<friendly-name>.parallel` set to `true`. Set to `-1` to use the
62+
number of available CPU cores. Must be a positive integer or `-1`;
63+
zero is rejected with a warning. See linkgit:git-hook[1].
64+
+
65+
Note on naming: although this key resembles `hook.<friendly-name>.*`
66+
(a per-hook setting), `<event>` must be the event name, not a hook
67+
friendly name. The key component is stored literally and looked up by
68+
event name at runtime with no translation between the two namespaces.
69+
A key like `hook.my-hook.jobs` is stored under `"my-hook"` but the
70+
lookup at runtime uses the event name (e.g. `"post-receive"`), so
71+
`hook.my-hook.jobs` is silently ignored even when `my-hook` is
72+
registered for that event. Use `hook.post-receive.jobs` or any other
73+
valid event name when setting `hook.<event>.jobs`.
74+
75+
hook.jobs::
76+
Specifies how many hooks can be run simultaneously during parallelized
77+
hook execution. If unspecified, defaults to 1 (serial execution).
78+
Set to `-1` to use the number of available CPU cores.
79+
Can be overridden on a per-event basis with `hook.<event>.jobs`.
80+
Some hooks always run sequentially regardless of this setting because
81+
they operate on shared data and cannot safely be parallelized:
82+
+
83+
--
84+
`applypatch-msg`;;
85+
`prepare-commit-msg`;;
86+
`commit-msg`;;
87+
Receive a commit message file and may rewrite it in place.
88+
`pre-commit`;;
89+
`post-checkout`;;
90+
`push-to-checkout`;;
91+
`post-commit`;;
92+
Access the working tree, index, or repository state.
93+
--
94+
+
95+
This setting has no effect unless all configured hooks for the event have
96+
`hook.<friendly-name>.parallel` set to `true`.
97+
+
98+
For `pre-push` hooks, which normally keep stdout and stderr separate,
99+
setting this to a value greater than 1 (or passing `-j`) will merge stdout
100+
into stderr to allow correct de-interleaving of parallel output.

Documentation/git-hook.adoc

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,8 @@ git-hook - Run git hooks
88
SYNOPSIS
99
--------
1010
[verse]
11-
'git hook' run [--allow-unknown-hook-name] [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]
11+
'git hook' run [--allow-unknown-hook-name] [--ignore-missing] [--to-stdin=<path>] [(-j|--jobs) <n>]
12+
<hook-name> [-- <hook-args>]
1213
'git hook' list [--allow-unknown-hook-name] [-z] [--show-scope] <hook-name>
1314

1415
DESCRIPTION
@@ -147,6 +148,23 @@ OPTIONS
147148
mirroring the output style of `git config --show-scope`. Traditional
148149
hooks from the hookdir are unaffected.
149150

151+
-j::
152+
--jobs::
153+
Only valid for `run`.
154+
+
155+
Specify how many hooks to run simultaneously. If this flag is not specified,
156+
the value of the `hook.jobs` config is used, see linkgit:git-config[1]. If
157+
neither is specified, defaults to 1 (serial execution).
158+
+
159+
When greater than 1, it overrides the per-hook `hook.<friendly-name>.parallel`
160+
setting, allowing all hooks for the event to run concurrently, even if they
161+
are not individually marked as parallel.
162+
+
163+
Some hooks always run sequentially regardless of this flag or the
164+
`hook.jobs` config, because git knows they cannot safely run in parallel:
165+
`applypatch-msg`, `pre-commit`, `prepare-commit-msg`, `commit-msg`,
166+
`post-commit`, `post-checkout`, and `push-to-checkout`.
167+
150168
WRAPPERS
151169
--------
152170
@@ -169,7 +187,8 @@ running:
169187
git hook run --allow-unknown-hook-name mywrapper-start-tests \
170188
# providing something to stdin
171189
--stdin some-tempfile-123 \
172-
# execute hooks in serial
190+
# execute multiple hooks in parallel
191+
--jobs 3 \
173192
# plus some arguments of your own...
174193
-- \
175194
--testname bar \

Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2671,7 +2671,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
26712671

26722672
help.sp help.s help.o: command-list.h
26732673
builtin/bugreport.sp builtin/bugreport.s builtin/bugreport.o: hook-list.h
2674-
builtin/hook.sp builtin/hook.s builtin/hook.o: hook-list.h
2674+
hook.sp hook.s hook.o: hook-list.h
26752675

26762676
builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
26772677
builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \

builtin/am.c

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -490,9 +490,11 @@ static int run_applypatch_msg_hook(struct am_state *state)
490490

491491
assert(state->msg);
492492

493-
if (!state->no_verify)
494-
ret = run_hooks_l(the_repository, "applypatch-msg",
495-
am_path(state, "final-commit"), NULL);
493+
if (!state->no_verify) {
494+
struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL;
495+
strvec_push(&opt.args, am_path(state, "final-commit"));
496+
ret = run_hooks_opt(the_repository, "applypatch-msg", &opt);
497+
}
496498

497499
if (!ret) {
498500
FREE_AND_NULL(state->msg);

builtin/checkout.c

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@
3131
#include "resolve-undo.h"
3232
#include "revision.h"
3333
#include "setup.h"
34+
#include "strvec.h"
3435
#include "submodule.h"
3536
#include "symlinks.h"
3637
#include "trace2.h"
@@ -123,13 +124,19 @@ static void branch_info_release(struct branch_info *info)
123124
static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
124125
int changed)
125126
{
126-
return run_hooks_l(the_repository, "post-checkout",
127-
oid_to_hex(old_commit ? &old_commit->object.oid : null_oid(the_hash_algo)),
128-
oid_to_hex(new_commit ? &new_commit->object.oid : null_oid(the_hash_algo)),
129-
changed ? "1" : "0", NULL);
130-
/* "new_commit" can be NULL when checking out from the index before
131-
a commit exists. */
127+
struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL;
132128

129+
/*
130+
* "new_commit" can be NULL when checking out from the index before
131+
* a commit exists.
132+
*/
133+
strvec_pushl(&opt.args,
134+
oid_to_hex(old_commit ? &old_commit->object.oid : null_oid(the_hash_algo)),
135+
oid_to_hex(new_commit ? &new_commit->object.oid : null_oid(the_hash_algo)),
136+
changed ? "1" : "0",
137+
NULL);
138+
139+
return run_hooks_opt(the_repository, "post-checkout", &opt);
133140
}
134141

135142
static int update_some(const struct object_id *oid, struct strbuf *base,

builtin/clone.c

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -647,6 +647,7 @@ static int checkout(int submodule_progress,
647647
struct tree *tree;
648648
struct tree_desc t;
649649
int err = 0;
650+
struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL;
650651

651652
if (option_no_checkout)
652653
return 0;
@@ -697,8 +698,9 @@ static int checkout(int submodule_progress,
697698
if (write_locked_index(the_repository->index, &lock_file, COMMIT_LOCK))
698699
die(_("unable to write new index file"));
699700

700-
err |= run_hooks_l(the_repository, "post-checkout", oid_to_hex(null_oid(the_hash_algo)),
701-
oid_to_hex(&oid), "1", NULL);
701+
strvec_pushl(&hook_opt.args, oid_to_hex(null_oid(the_hash_algo)),
702+
oid_to_hex(&oid), "1", NULL);
703+
err |= run_hooks_opt(the_repository, "post-checkout", &hook_opt);
702704

703705
if (!err && (option_recurse_submodules.nr > 0)) {
704706
struct child_process cmd = CHILD_PROCESS_INIT;

builtin/hook.c

Lines changed: 29 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -4,23 +4,15 @@
44
#include "environment.h"
55
#include "gettext.h"
66
#include "hook.h"
7-
#include "hook-list.h"
87
#include "parse-options.h"
8+
#include "thread-utils.h"
99

1010
#define BUILTIN_HOOK_RUN_USAGE \
11-
N_("git hook run [--allow-unknown-hook-name] [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
11+
N_("git hook run [--allow-unknown-hook-name] [--ignore-missing] [--to-stdin=<path>] [(-j|--jobs) <n>]\n" \
12+
"<hook-name> [-- <hook-args>]")
1213
#define BUILTIN_HOOK_LIST_USAGE \
1314
N_("git hook list [--allow-unknown-hook-name] [-z] [--show-scope] <hook-name>")
1415

15-
static int is_known_hook(const char *name)
16-
{
17-
const char **p;
18-
for (p = hook_name_list; *p; p++)
19-
if (!strcmp(*p, name))
20-
return 1;
21-
return 0;
22-
}
23-
2416
static const char * const builtin_hook_usage[] = {
2517
BUILTIN_HOOK_RUN_USAGE,
2618
BUILTIN_HOOK_LIST_USAGE,
@@ -96,14 +88,22 @@ static int list(int argc, const char **argv, const char *prefix,
9688
const char *name = h->u.configured.friendly_name;
9789
const char *scope = show_scope ?
9890
config_scope_name(h->u.configured.scope) : NULL;
91+
/*
92+
* Show the most relevant disable reason. Event-level
93+
* takes precedence: if the whole event is off, that
94+
* is what the user needs to know. The per-hook
95+
* "disabled" surfaces once the event is re-enabled.
96+
*/
97+
const char *disability =
98+
h->u.configured.event_disabled ? "event-disabled\t" :
99+
h->u.configured.disabled ? "disabled\t" :
100+
"";
99101
if (scope)
100-
printf("%s\t%s%s%c", scope,
101-
h->u.configured.disabled ? "disabled\t" : "",
102-
name, line_terminator);
102+
printf("%s\t%s%s%c", scope, disability, name,
103+
line_terminator);
103104
else
104-
printf("%s%s%c",
105-
h->u.configured.disabled ? "disabled\t" : "",
106-
name, line_terminator);
105+
printf("%s%s%c", disability, name,
106+
line_terminator);
107107
break;
108108
}
109109
default:
@@ -124,6 +124,7 @@ static int run(int argc, const char **argv, const char *prefix,
124124
struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
125125
int ignore_missing = 0;
126126
int allow_unknown = 0;
127+
int jobs = 0;
127128
const char *hook_name;
128129
struct option run_options[] = {
129130
OPT_BOOL(0, "allow-unknown-hook-name", &allow_unknown,
@@ -132,6 +133,8 @@ static int run(int argc, const char **argv, const char *prefix,
132133
N_("silently ignore missing requested <hook-name>")),
133134
OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
134135
N_("file to read into hooks' stdin")),
136+
OPT_INTEGER('j', "jobs", &jobs,
137+
N_("run up to <n> hooks simultaneously (-1 for CPU count)")),
135138
OPT_END(),
136139
};
137140
int ret;
@@ -140,6 +143,15 @@ static int run(int argc, const char **argv, const char *prefix,
140143
builtin_hook_run_usage,
141144
PARSE_OPT_KEEP_DASHDASH);
142145

146+
if (jobs == -1)
147+
opt.jobs = online_cpus();
148+
else if (jobs < 0)
149+
die(_("invalid value for -j: %d"
150+
" (use -1 for CPU count or a"
151+
" positive integer)"), jobs);
152+
else
153+
opt.jobs = jobs;
154+
143155
if (!argc)
144156
goto usage;
145157

builtin/receive-pack.c

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1456,7 +1456,8 @@ static const char *push_to_checkout(unsigned char *hash,
14561456
struct strvec *env,
14571457
const char *work_tree)
14581458
{
1459-
struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
1459+
struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL;
1460+
14601461
opt.invoked_hook = invoked_hook;
14611462

14621463
strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));

builtin/worktree.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -609,7 +609,7 @@ static int add_worktree(const char *path, const char *refname,
609609
* is_junk is cleared, but do return appropriate code when hook fails.
610610
*/
611611
if (!ret && opts->checkout && !opts->orphan) {
612-
struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
612+
struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL;
613613

614614
strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
615615
strvec_pushl(&opt.args,

commit.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1970,7 +1970,7 @@ size_t ignored_log_message_bytes(const char *buf, size_t len)
19701970
int run_commit_hook(int editor_is_used, const char *index_file,
19711971
int *invoked_hook, const char *name, ...)
19721972
{
1973-
struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
1973+
struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL;
19741974
va_list args;
19751975
const char *arg;
19761976

0 commit comments

Comments
 (0)