Skip to content

feat(operator-demo-video): add the operator drag-and-connect controller - #7641

Open
xuang7 wants to merge 5 commits into
apache:mainfrom
xuang7:pr/operator-demo-c3-operator-controller
Open

feat(operator-demo-video): add the operator drag-and-connect controller#7641
xuang7 wants to merge 5 commits into
apache:mainfrom
xuang7:pr/operator-demo-c3-operator-controller

Conversation

@xuang7

@xuang7 xuang7 commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

This PR adds the controller that puts an operator on the canvas, the step every demo builds around.

Changes:

  • OperatorControllerBuilder.insertViaDrag: locates the operator in the left panel by its metadata group hierarchy (falling back to the search box), drags it onto the canvas, and wires it up. Wiring follows the operator's port topology: source operators connect nothing, single-input operators connect to the anchor, and two-input operators pull a second edge from a given upstream port into a given input port.
  • OperatorGroups: flattens the operator group tree into name -> path, shared metadata used to walk the nested group panels.
  • Adds the module's dependency on the operator-metadata modules, with the jackson overrides that keep Playwright's transitive jackson on the repo-wide version.
  • Extends Utils (canvas-geometry helpers and a shared name normalizer) and the timing constants the controller waits on.
  • Four data-testid hooks (left-panel Operators button, operator search input, per-operator list items operator-item-<type>, and the property-panel title the controller waits on after a drop), each covered by a guard test in the component's existing spec.

demo:

bar-chart_demo.webm

Any related issues, documentation, discussions?

Closes #7521
Depends on #7600.

How was this PR tested?

No behavior change to any existing module. sbt DemoVideoGenerator/compile, scalafmtCheck, and scalafix --check all pass; the three touched component specs (65 tests, including the new guard tests) pass. The drag-and-connect flow was exercised against a local deployment across all three port topologies (source, single-input, two-input join).

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Fable 5)

xuang7 and others added 4 commits August 11, 2026 23:27
…ht automation foundation

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…dules use

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions github-actions Bot added dependencies Pull requests that update a dependency file frontend Changes related to the frontend GUI ci changes related to CI docs Changes related to documentations common labels Aug 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @aglinxinyuan, @Neilk1021, @mengw15
    You can notify them by mentioning @aglinxinyuan, @Neilk1021, @mengw15 in a comment.

@codecov-commenter

codecov-commenter commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.66%. Comparing base (12169c2) to head (658512c).

Additional details and impacted files
@@            Coverage Diff            @@
##               main    #7641   +/-   ##
=========================================
  Coverage     89.65%   89.66%           
+ Complexity     4397     4396    -1     
=========================================
  Files          1177     1177           
  Lines         46996    46947   -49     
  Branches       5268     5260    -8     
=========================================
- Hits          42136    42096   -40     
  Misses         3094     3094           
+ Partials       1766     1757    -9     
Flag Coverage Δ *Carryforward flag
access-control-service 70.00% <ø> (ø)
agent-service 98.62% <ø> (ø)
amber 86.38% <ø> (+0.03%) ⬆️ Carriedforward from 4afe8d4
computing-unit-managing-service 72.46% <ø> (ø)
config-service 77.31% <ø> (ø)
file-service 68.90% <ø> (ø)
frontend 90.67% <ø> (-0.01%) ⬇️
notebook-migration-service 78.89% <ø> (ø)
pyamber 97.52% <ø> (-0.05%) ⬇️
workflow-compiling-service 57.89% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 7 worse · ⚪ 8 noise (<±5%) · 0 without baseline

Compared against main 12169c2 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 356 0.217 27,605/35,107/35,107 us 🔴 +17.0% / 🔴 +116.7%
🔴 bs=100 sw=10 sl=64 769 0.47 127,381/160,450/160,450 us 🔴 +16.8% / 🔴 +49.1%
bs=1000 sw=10 sl=64 905 0.552 1,108,489/1,144,482/1,144,482 us ⚪ within ±5% / 🔴 -12.2%
Baseline details

Latest main 12169c2 from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 356 tuples/sec 401 tuples/sec 774.89 tuples/sec -11.2% -54.1%
bs=10 sw=10 sl=64 MB/s 0.217 MB/s 0.245 MB/s 0.473 MB/s -11.4% -54.1%
bs=10 sw=10 sl=64 p50 27,605 us 23,596 us 12,738 us +17.0% +116.7%
bs=10 sw=10 sl=64 p95 35,107 us 36,603 us 16,348 us -4.1% +114.7%
bs=10 sw=10 sl=64 p99 35,107 us 36,603 us 18,848 us -4.1% +86.3%
bs=100 sw=10 sl=64 throughput 769 tuples/sec 812 tuples/sec 1,005 tuples/sec -5.3% -23.5%
bs=100 sw=10 sl=64 MB/s 0.47 MB/s 0.495 MB/s 0.613 MB/s -5.1% -23.4%
bs=100 sw=10 sl=64 p50 127,381 us 123,434 us 100,970 us +3.2% +26.2%
bs=100 sw=10 sl=64 p95 160,450 us 137,386 us 107,605 us +16.8% +49.1%
bs=100 sw=10 sl=64 p99 160,450 us 137,386 us 116,429 us +16.8% +37.8%
bs=1000 sw=10 sl=64 throughput 905 tuples/sec 907 tuples/sec 1,030 tuples/sec -0.2% -12.2%
bs=1000 sw=10 sl=64 MB/s 0.552 MB/s 0.554 MB/s 0.629 MB/s -0.4% -12.2%
bs=1000 sw=10 sl=64 p50 1,108,489 us 1,106,932 us 991,433 us +0.1% +11.8%
bs=1000 sw=10 sl=64 p95 1,144,482 us 1,157,207 us 1,036,668 us -1.1% +10.4%
bs=1000 sw=10 sl=64 p99 1,144,482 us 1,157,207 us 1,070,470 us -1.1% +6.9%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,561.80,200,128000,356,0.217,27604.88,35106.63,35106.63
1,100,10,64,20,2599.28,2000,1280000,769,0.470,127381.30,160450.49,160450.49
2,1000,10,64,20,22097.42,20000,12800000,905,0.552,1108489.17,1144482.34,1144482.34

@xuang7

xuang7 commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

Could you take a look when you're available? Thanks! @carloea2

@carloea2 carloea2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new node lookup can select an older node when the workflow already has the same operator type. Please use the node added after beforeCount. Also, alreadyConnected can never be true because expectedLinkCount is the current count plus one. Please check the target port instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci changes related to CI common dependencies Pull requests that update a dependency file docs Changes related to documentations frontend Changes related to the frontend GUI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add operator drag-and-connect controller

3 participants