test(workflow-compiler): Extend WorkflowCompilerSpec to cover the Python code-generation error path

[eugenegujing]

What changes were proposed in this PR?

WorkflowCompiler scans every Python-based physical operator's generated code for the #EXCEPTION DURING CODE GENERATION: marker that PythonOperatorDescriptor embeds when an operator's generatePythonCode throws. On a hit it either appends a RuntimeException("Operator is not configured properly: ...") to the caller's error list (the editing-time, lenient path) or throws it immediately when no error list was given (the pre-execution, strict path). WorkflowCompilerSpec's 15 tests never exercised either branch; the only existing test of the marker is the producer-side PythonOperatorDescriptorSpec, which asserts the marker is written but not that the compiler reacts to it.

This adds 7 tests to WorkflowCompilerSpec (15 → 22). Test-only: no production file is touched, and none of the 15 existing tests is modified — the diff removes exactly three lines, all of them import lines being widened.

Lenient path:

1. should accumulate a per-operator error when a Python operator's code generation fails — the error is attributed to the right logical operator, carries the expected message, and compilation continues (the rest of the plan still lands in the physical plan and the storage set).
2. should attribute each Python code-generation failure to its own logical operator — two failing Python operators with two distinct messages, each keyed to its own id.
3. should trim the marker's message and report it as a plain RuntimeException
4. should report no code-generation error for a well-formed Python operator — asserts the operator really is still Python-based, so it cannot pass by quietly ceasing to be one.
5. should not subject non-Python operators to the code-generation check

Strict path:

6. in strict mode should throw immediately when a Python operator's code generation failed
7. in strict mode should not throw for a well-formed Python operator

Five of the seven drive the real shipped SortOpDesc, whose generatePythonCode opens with require(attributes.nonEmpty, ...) and a per-key require (SortOpDesc.scala:34-38). A Sort dropped on the canvas and left unconfigured is therefore a genuine, user-reachable route into the marker state, and it conveniently yields two distinct messages — which is what makes test 2's per-operator attribution meaningful. A configured Sort gives the negative control in tests 4 and 7: same operator, same code path, codegen simply succeeds.

Test 3 needs a small test-only fixture (PaddedFailurePyOp, ~15 lines, modelled on PythonOperatorDescriptorSpec's ThrowingPyOp): no shipped operator raises a whitespace-padded message, and without padding the .trim is unobservable. It is also the spec's only source-operator Python case.

Two notes:

• Four assertions hard-code SortOpDesc's exact require wording, so rewording those messages in workflow-operator will fail these workflow-compiler tests. That cross-module coupling is deliberate — it pins the end-to-end string a user actually sees — but it is worth knowing.
• Test 3 also asserts the reported message starts with java.lang.RuntimeException: . That prefix comes from err.toString at WorkflowCompiler.scala:66 and is part of what the UI renders today; the assertion carries a comment saying so, so if that pre-existing wart is ever fixed there is one self-explaining test to update.

Any related issues, documentation, discussions?

Closes #7647

Builds on the specs added by #5019 / #5022 and the module unification in #6143.

How was this PR tested?

sbt "WorkflowCompiler/testOnly *WorkflowCompilerSpec" — 22 tests, all passing (15 pre-existing + 7 new). Running the whole module (WorkflowCompiler/test) is green too: 3 suites / 57 tests / 0 failures. WorkflowCompiler/scalafmtCheck and WorkflowCompiler/Test/scalafmtCheck are clean.

Every new test was mutation-checked, twice and independently: the production check was temporarily broken, the suite re-run, and the file reverted (verified byte-identical afterwards). Highlights, with the tests that are the sole killer of a mutant:

Mutant in WorkflowCompiler.scala	Caught by
lenient arm appends and then throws (abort instead of continue)	1 only
record only the first codegen failure per compile	2 only
drop .trim on the captured message	3 only
skip the check for source operators (&& !isSourceOperator)	3 only
strict arm swallows instead of throwing (case None => ())	6 only
flip the errorList arms (Some → throw, None → skip)	1, 6
change the message prefix	1, 2, 3, 6
group(1) → group(0)	1, 2, 3, 6
marker regex changed so it never matches	1, 2, 3, 6
regex relaxed to (.*) (matches every operator)	1, 2, 3, 4, 6, 7
attribute the error to the wrong logical operator id	1, 2, 3
build the error but never record it	1, 2, 3, 6
RuntimeException → IllegalArgumentException	3

Two mutants survive, both semantically equivalent rather than gaps: reporting the last marker match instead of the first (a codegen-failure body is exactly the one-line marker, so first == last), and anchoring the regex with ^ — the latter is in fact the fix for a bug found along the way (below), and all 22 tests stay green under it, so the suite does not over-fit to the current unanchored check.

One more mutant is worth calling out honestly: removing the isPythonBased guard entirely does not fail test 5 — it aborts the whole suite, because PhysicalOp.getCode throws an IllegalAccessError (a LinkageError, which is not NonFatal) for non-code operators. Test 5 therefore documents the guard's intent rather than detecting its removal. It is kept for that reason, with a self-check that stops it silently becoming a no-op.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5)

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @Yicong-Huang
  You can notify them by mentioning @Yicong-Huang in a comment.

[github-actions]

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 9 worse · ⚪ 6 noise (<±5%) · 0 without baseline

Compared against main 4013ca7 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	375	0.229	24,763/38,079/38,079 us	🔴 +19.9% / 🔴 +132.9%
🔴	bs=100 sw=10 sl=64	785	0.479	124,787/160,364/160,364 us	🔴 +6.0% / 🔴 +49.0%
🔴	bs=1000 sw=10 sl=64	902	0.55	1,102,390/1,206,352/1,206,352 us	🔴 +7.6% / 🔴 +16.4%

Baseline details

Latest main 4013ca7 from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	375 tuples/sec	436 tuples/sec	774.89 tuples/sec	-14.0%	-51.6%
bs=10 sw=10 sl=64	MB/s	0.229 MB/s	0.266 MB/s	0.473 MB/s	-13.9%	-51.6%
bs=10 sw=10 sl=64	p50	24,763 us	22,729 us	12,738 us	+8.9%	+94.4%
bs=10 sw=10 sl=64	p95	38,079 us	31,748 us	16,348 us	+19.9%	+132.9%
bs=10 sw=10 sl=64	p99	38,079 us	31,748 us	18,848 us	+19.9%	+102.0%
bs=100 sw=10 sl=64	throughput	785 tuples/sec	815 tuples/sec	1,005 tuples/sec	-3.7%	-21.9%
bs=100 sw=10 sl=64	MB/s	0.479 MB/s	0.497 MB/s	0.613 MB/s	-3.6%	-21.9%
bs=100 sw=10 sl=64	p50	124,787 us	120,054 us	100,970 us	+3.9%	+23.6%
bs=100 sw=10 sl=64	p95	160,364 us	151,328 us	107,605 us	+6.0%	+49.0%
bs=100 sw=10 sl=64	p99	160,364 us	151,328 us	116,429 us	+6.0%	+37.7%
bs=1000 sw=10 sl=64	throughput	902 tuples/sec	927 tuples/sec	1,030 tuples/sec	-2.7%	-12.5%
bs=1000 sw=10 sl=64	MB/s	0.55 MB/s	0.566 MB/s	0.629 MB/s	-2.8%	-12.5%
bs=1000 sw=10 sl=64	p50	1,102,390 us	1,076,891 us	991,433 us	+2.4%	+11.2%
bs=1000 sw=10 sl=64	p95	1,206,352 us	1,121,647 us	1,036,668 us	+7.6%	+16.4%
bs=1000 sw=10 sl=64	p99	1,206,352 us	1,121,647 us	1,070,470 us	+7.6%	+12.7%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,533.79,200,128000,375,0.229,24763.19,38079.13,38079.13
1,100,10,64,20,2548.11,2000,1280000,785,0.479,124787.39,160363.75,160363.75
2,1000,10,64,20,22178.99,20000,12800000,902,0.550,1102390.03,1206351.73,1206351.73

[Copilot]

Pull request overview

Extends common/workflow-compiler’s unit coverage to exercise WorkflowCompiler’s handling of the #EXCEPTION DURING CODE GENERATION: marker emitted by PythonOperatorDescriptor when generatePythonCode() throws. This improves confidence that both the editing-time (lenient) and pre-execution (strict) compilation paths translate codegen failures into the correct user-facing diagnostics.

Changes:

• Adds new lenient-mode tests verifying per-operator attribution, continued compilation behavior, and message extraction/trim.
• Adds strict-mode tests verifying fail-fast behavior for the same marker-driven failures.
• Introduces a small test-only Python operator fixture to produce a padded error message that makes .trim behavior observable.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.66%. Comparing base (4013ca7) to head (eda8b3c).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #7649      +/-   ##
============================================
- Coverage     89.68%   89.66%   -0.03%     
+ Complexity     4399     4396       -3     
============================================
  Files          1175     1176       +1     
  Lines         46979    46992      +13     
  Branches       5265     5268       +3     
============================================
+ Hits          42133    42134       +1     
- Misses         3082     3091       +9     
- Partials       1764     1767       +3     

Flag	Coverage Δ		*Carryforward flag
access-control-service	70.00% <ø> (ø)
agent-service	98.62% <ø> (ø)		Carriedforward from 12169c2
amber	86.36% <ø> (-0.03%)	⬇️
computing-unit-managing-service	72.46% <ø> (ø)
config-service	77.31% <ø> (ø)
file-service	68.90% <ø> (ø)
frontend	90.67% <ø> (-0.04%)	⬇️	Carriedforward from 12169c2
notebook-migration-service	78.89% <ø> (ø)
pyamber	97.57% <ø> (ø)		Carriedforward from 12169c2
workflow-compiling-service	57.89% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.