ci: drive website docs from release branches instead of pushing - #6948
ci: drive website docs from release branches instead of pushing#6948Ma77Ball wants to merge 5 commits into
Conversation
The old sync-docs-to-site.yml pushed docs/ into apache/incubator-texera-site on every main change. It has failed on every run: its SITE_SYNC_TOKEN identity has no write access to the site repo, which is why the published docs drifted. Switch to a pull model owned by the website: it pulls docs/ from the release branches at build time (see the companion site PR), so nothing is committed cross-repo and the site cannot drift. This repo's only job is to tell the site to rebuild when a release candidate is cut, via a repository_dispatch appended to create-release-candidate.yml. The site's daily schedule is the safety net. - Remove sync-docs-to-site.yml (broken push sync). - Add a notify-website job to create-release-candidate.yml. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
👋 Thanks for opening this pull request, @Ma77Ball! It looks like the pull request description doesn't quite follow our template yet:
Filling out the template helps reviewers understand and triage your contribution faster. Please edit the description to complete it. This message will disappear automatically once the template is followed. You can find the template prompts by editing the description, or see CONTRIBUTING.md for the full contribution flow. |
Automated Reviewer SuggestionsBased on the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #6948 +/- ##
============================================
- Coverage 83.20% 83.17% -0.03%
+ Complexity 4117 4111 -6
============================================
Files 1166 1166
Lines 46456 46456
Branches 5177 5177
============================================
- Hits 38654 38642 -12
- Misses 6096 6102 +6
- Partials 1706 1712 +6
☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 373 | 0.228 | 26,358/34,660/34,660 us | 🔴 +18.5% / 🔴 +123.1% |
| ⚪ | bs=100 sw=10 sl=64 | 782 | 0.477 | 126,126/152,322/152,322 us | ⚪ within ±5% / 🔴 +39.7% |
| ⚪ | bs=1000 sw=10 sl=64 | 901 | 0.55 | 1,108,114/1,160,403/1,160,403 us | ⚪ within ±5% / 🔴 +11.7% |
Baseline details
Latest main 5a3da70 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 373 tuples/sec | 428 tuples/sec | 767.32 tuples/sec | -12.9% | -51.4% |
| bs=10 sw=10 sl=64 | MB/s | 0.228 MB/s | 0.261 MB/s | 0.468 MB/s | -12.6% | -51.3% |
| bs=10 sw=10 sl=64 | p50 | 26,358 us | 22,249 us | 12,772 us | +18.5% | +106.4% |
| bs=10 sw=10 sl=64 | p95 | 34,660 us | 31,723 us | 15,538 us | +9.3% | +123.1% |
| bs=10 sw=10 sl=64 | p99 | 34,660 us | 31,723 us | 18,948 us | +9.3% | +82.9% |
| bs=100 sw=10 sl=64 | throughput | 782 tuples/sec | 810 tuples/sec | 972.51 tuples/sec | -3.5% | -19.6% |
| bs=100 sw=10 sl=64 | MB/s | 0.477 MB/s | 0.494 MB/s | 0.594 MB/s | -3.4% | -19.6% |
| bs=100 sw=10 sl=64 | p50 | 126,126 us | 120,140 us | 103,020 us | +5.0% | +22.4% |
| bs=100 sw=10 sl=64 | p95 | 152,322 us | 146,669 us | 109,070 us | +3.9% | +39.7% |
| bs=100 sw=10 sl=64 | p99 | 152,322 us | 146,669 us | 118,964 us | +3.9% | +28.0% |
| bs=1000 sw=10 sl=64 | throughput | 901 tuples/sec | 928 tuples/sec | 1,005 tuples/sec | -2.9% | -10.3% |
| bs=1000 sw=10 sl=64 | MB/s | 0.55 MB/s | 0.567 MB/s | 0.613 MB/s | -3.0% | -10.3% |
| bs=1000 sw=10 sl=64 | p50 | 1,108,114 us | 1,078,748 us | 1,002,400 us | +2.7% | +10.5% |
| bs=1000 sw=10 sl=64 | p95 | 1,160,403 us | 1,121,206 us | 1,039,228 us | +3.5% | +11.7% |
| bs=1000 sw=10 sl=64 | p99 | 1,160,403 us | 1,121,206 us | 1,069,081 us | +3.5% | +8.5% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,535.66,200,128000,373,0.228,26358.26,34660.36,34660.36
1,100,10,64,20,2558.77,2000,1280000,782,0.477,126125.94,152322.17,152322.17
2,1000,10,64,20,22204.56,20000,12800000,901,0.550,1108114.41,1160402.83,1160402.83
| run: | | ||
| gh api repos/apache/incubator-texera-site/dispatches \ | ||
| -f event_type=docs-updated \ | ||
| -f "client_payload[tag]=${{ github.event.inputs.tag }}" |
There was a problem hiding this comment.
how dose it pass the exact RC SHA?
There was a problem hiding this comment.
It wasn't passing the RC SHA before, but now it will, so the website can be built with the precise RC commit.
|
@Yicong-Huang please review again |
Yicong-Huang
left a comment
Yicong-Huang
left a comment
There was a problem hiding this comment.
🔴 2 must-fix · 3 advisory · 0 polish — the design is right and the removal is clean; both must-fix items sit in the 25 new lines and are small edits.
Design & architecture (2)
create-release-candidate.yml:484— the dispatchedclient_payloadhas no consumer in the companion site PR (must-fix, see inline)create-release-candidate.yml:479— reuses a shared release credential where the deleted workflow had a dedicated one (must-fix, see inline)
Simplifications (1)
create-release-candidate.yml:477— a failed ping is invisible; an annotation would makecontinue-on-errorunnecessary (advisory, see inline)
Conventions (2)
- Description: state that docs now publish only from
release/vX.Ybranches — a change merged tomainno longer reaches the website until it lands on one (advisory) - Description: no proof run linked for a CI change; the
notify-websitestep is separately exercisable on a fork (advisory)
Verification trace
Checked the dispatch contract against companion PR apache/incubator-texera-site#68: a paginated sweep of every file and patch there finds zero occurrences of client_payload. Its build runs scripts/pull-release-docs.sh, which enumerates release/vX.Y branch tips rather than any dispatched sha. The docs-updated event type does match, so the "go" signal itself lands. Merge ordering was checked in both directions and is safe either way.
| # Pass the tag and its resolved commit sha. | ||
| gh api repos/apache/incubator-texera-site/dispatches \ | ||
| -f event_type=docs-updated \ | ||
| -f "client_payload[tag]=${{ github.event.inputs.tag }}" \ | ||
| -f "client_payload[sha]=${{ needs.create-rc.outputs.commit_hash }}" |
There was a problem hiding this comment.
@Yicong-Huang asked on line 483 how the exact RC SHA reaches the site. The other side says it does not: a paginated sweep of apache/incubator-texera-site#68 finds no client_payload, and its build runs pull-release-docs.sh against release/vX.Y branch tips.
Honoring it later would create a new problem. Dispatch builds would pin an RC sha while the daily cron builds the branch tip, so docs would flip between two sources. Suggest dropping the payload and keeping the pure "go" signal.
| # Pass the tag and its resolved commit sha. | |
| gh api repos/apache/incubator-texera-site/dispatches \ | |
| -f event_type=docs-updated \ | |
| -f "client_payload[tag]=${{ github.event.inputs.tag }}" \ | |
| -f "client_payload[sha]=${{ needs.create-rc.outputs.commit_hash }}" | |
| gh api repos/apache/incubator-texera-site/dispatches \ | |
| -f event_type=docs-updated |
| - name: Trigger website rebuild | ||
| continue-on-error: true # never fail the release over a website ping | ||
| env: | ||
| GH_TOKEN: ${{ secrets.AUTO_MERGE_TOKEN }} |
There was a problem hiding this comment.
This borrows AUTO_MERGE_TOKEN, shared with the backport workflows, and the job comment notes its scope must be broadened first. The deleted workflow did the same job through a purpose-scoped SITE_SYNC_TOKEN with a narrow permissions: block — a step back from what the repo already had.
This PR frees the SITE_SYNC_TOKEN name. Pointing the job at it is the same one-time INFRA request, without widening what the backport workflows carry. That request gates either option, so this is not a merge blocker. Worth adding a permissions: block too.
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Trigger website rebuild | ||
| continue-on-error: true # never fail the release over a website ping |
There was a problem hiding this comment.
continue-on-error: true with no annotation means a 403 — the exact failure mode #6949 exists to fix — produces a green release run. Advisory rather than a defect: the RC workflow is hand-launched and watched, and #68's daily cron caps the staleness at ~24h.
Wrapping the call as if ! gh api …; then echo "::warning::…"; fi surfaces the miss and keeps the release green, making continue-on-error unnecessary.
|
Closing as superseded. The website's build-time docs sync shipped in apache/incubator-texera-site#72, whose publish workflow uses workflow_dispatch rather than the repository_dispatch: docs-updated event this PR fires. The replacement is #7659, which removes sync-docs-to-site.yml and dispatches the site's publish-site.yml (staging) on release-branch doc changes. |
4 participants
What changes were proposed in this PR?
sync-docs-to-site.yml, the doc push-sync that 403s on every run because its token identity (aicam) has no write access toapache/incubator-texera-site.notify-websitejob tocreate-release-candidate.ymlthat fires arepository_dispatch(docs-updated) at the site repo after an RC uploads, so the site rebuilds from the release branches.AUTO_MERGE_TOKENand iscontinue-on-error; the site's daily schedule is the backstop if that PAT is not scoped for the site repo.Any related issues, documentation, discussions?
Closes: #6949
How was this PR tested?
python3 -c "import yaml; yaml.safe_load(open('.github/workflows/create-release-candidate.yml'))".create-release-candidate.yml, so it cannot be exercised in this PR; reviewers can confirm the wiring by reading thenotify-websitejob (event_type: docs-updatedmatches the site trigger in the companion PR).Was this PR authored or co-authored using generative AI tooling?
Co-authored with Claude Opus 4.8 in compliance with ASF