bake: add execution mode - #4044
Conversation
b027761 to
05970de
Compare
crazy-max
changed the title
7a76c27 to
1311cec
Compare
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
| if bh.Execution.Parallel > 0 && bh.Execution.Parallel < len(opts) { | ||
| return nil, errors.Errorf("sync-output execution requires parallelism to be unlimited or at least the number of targets") | ||
| } |
There was a problem hiding this comment.
I kept sync-output compatible with parallel=N only when the limit is unlimited or at least the number of selected targets. A smaller limit can deadlock because sync-output waits for every target to reach the output boundary together.
Does it match what we want? Or should sync-output reject any explicit parallel limit for now?
| if bh.Execution.Parallel < len(opts) && linkedTargets.hasLinks() { | ||
| return nil, errors.Errorf("limited parallelism is not supported with linked targets") | ||
| } |
There was a problem hiding this comment.
I rejected limited target parallelism when linked targets are present because linked target evaluation can require parent and child targets to be alive together.
Does it make sense for this first version or should linked targets be allowed when the requested limit is high enough to cover each linked group?
1311cec to
c65a9ad
Compare
| flags.StringArrayVar(&options.vars, "var", nil, `Set a variable value (e.g., "name=value")`) | ||
| flags.StringVar(&options.callFunc, "call", "build", `Set method for evaluating build ("check", "outline", "targets")`) | ||
| flags.StringArrayVar(&options.allow, "allow", nil, "Allow build to access specified resources") | ||
| flags.StringVar(&options.execution, "execution", "fail-fast", `Set target execution behavior (format: "mode[,parallel=N]")`) |
There was a problem hiding this comment.
I used --execution as the flag for target failure behavior, output synchronization, and target parallelism, but I'm not fully convinced this is the best name. Any ideas?
cc @dvdksn
There was a problem hiding this comment.
--jobs (also -j N), --strategy (Ansible).
2 participants
This carries forward the syncable output work from #1197 and expands it into a general
--executionoption for Bake. The new option keeps the default behavior asfail-fast, addssync-outputso multi-target builds only write output after all selected targets reach the output boundary successfully, addsdefer-errorso independent targets can continue after another target fails, and supportsparallel=Nas a comma-separated execution option for limiting target concurrency.This addresses the same class of partial-output and multi-target coordination problems discussed in #1089, #1668, #3297, #3428, and #3989. The sync-output mode is the piece needed for atomic-ish local output across targets, including failures during ref evaluation such as missing files. The defer-error mode covers workflows that need all independent targets to finish before reporting failure. The parallel option provides explicit target-level concurrency control without changing BuildKit internal parallelism.