Skip to content

ci: consolidate release publishing flow #1752

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cpcloud merged 2 commits into from
Mar 12, 2026
Merged

ci: consolidate release publishing flow #1752

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d7a7749
ci: consolidate release publishing flow
cpcloud Mar 11, 2026
27dc68d
ci: preserve automatic release run detection
cpcloud Mar 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 7 additions & 27 deletions .github/RELEASE-core.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,41 +123,21 @@ Pushing the tag triggers a CI run automatically. Monitor it in the

## Upload wheels to PyPI

This is a two-stage process: first publish to TestPyPI, verify, then
publish to PyPI.

### Stage 1: TestPyPI
This is a single `CI: Release` workflow run with two sequential stages:
publish to TestPyPI, then publish the same wheel set to PyPI.

1. Go to **Actions > CI: Release** and run the workflow with:
- **Component**: `cuda-core`
- **The release git tag**: `cuda-core-v0.6.0`
- **The GHA run ID that generated validated artifacts**: This is the
run ID of the successful tag-triggered CI run from the previous step.
You can find it in the URL when viewing the run in the Actions tab
(e.g. `https://github.com/NVIDIA/cuda-python/actions/runs/123456789`
— the run ID is `123456789`).
- **Which wheel index to publish to**: `testpypi`

The workflow automatically looks up the successful tag-triggered CI run
for the selected release tag.

2. Wait for the workflow to complete.

3. Verify the TestPyPI upload by installing and running tests from a
checked-out copy of the repository:

```bash
pip install -i https://test.pypi.org/simple/ \
--extra-index-url https://pypi.org/simple/ \
cuda-core==0.6.0
cd cuda_core/tests && pytest
```

### Stage 2: PyPI

Once TestPyPI verification passes, rerun the same workflow with:
- **Which wheel index to publish to**: `pypi`
2. Wait for the workflow to complete. It will:
- publish the selected wheels to TestPyPI
- publish the same wheel set to PyPI

After completion, verify:
3. After completion, verify the final PyPI upload:

```bash
pip install cuda-core==0.6.0
Expand Down
66 changes: 39 additions & 27 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@

name: "CI: Release"

# Manually-triggered release workflow. Creates a release draft if one doesn't exist for the given tag, or uses existing draft.
# Manually-triggered release workflow. Creates a release draft if one doesn't exist
# for the given tag, or uses an existing draft, then publishes the selected wheels
# to TestPyPI followed by PyPI.

on:
workflow_dispatch:
Expand All @@ -28,13 +30,6 @@ on:
required: false
type: string
default: ""
wheel-dst:
description: "Which wheel index to publish to?"
required: true
type: choice
options:
- testpypi
- pypi

defaults:
run:
Expand All @@ -60,7 +55,7 @@ jobs:
echo "Auto-detecting successful tag-triggered run ID for tag: ${{ inputs.git-tag }}"
RUN_ID=$(./ci/tools/lookup-run-id "${{ inputs.git-tag }}" "${{ github.repository }}")
echo "Auto-detected run ID: $RUN_ID"
echo "run-id=$RUN_ID" >> $GITHUB_OUTPUT
echo "run-id=$RUN_ID" >> "$GITHUB_OUTPUT"

check-tag:
runs-on: ubuntu-latest
Expand All @@ -74,17 +69,11 @@ jobs:
env:
GH_TOKEN: ${{ github.token }}
run: |
tags=
for i in $(gh release list -R ${{ github.repository }} --json tagName --jq '.[]| .tagName'); do
tags+=( $i )
done
is_draft=
for i in $(gh release list -R ${{ github.repository }} --json isDraft --jq '.[]| .isDraft'); do
is_draft+=( $i )
done
mapfile -t tags < <(gh release list -R "${{ github.repository }}" --json tagName --jq '.[] | .tagName')
mapfile -t is_draft < <(gh release list -R "${{ github.repository }}" --json isDraft --jq '.[] | .isDraft')

found=0
for idx in ${!tags[@]}; do
for idx in "${!tags[@]}"; do
if [[ "${tags[$idx]}" == "${{ inputs.git-tag }}" ]]; then
echo "found existing release for ${{ inputs.git-tag }}"
found=1
Expand Down Expand Up @@ -134,16 +123,16 @@ jobs:
run-id: ${{ needs.determine-run-id.outputs.run-id }}
component: ${{ inputs.component }}

publish-wheels:
name: Publish wheels
publish-testpypi:
name: Publish wheels to TestPyPI
runs-on: ubuntu-latest
needs:
- check-tag
- determine-run-id
- doc
environment:
name: ${{ inputs.wheel-dst }}
url: https://${{ (inputs.wheel-dst == 'testpypi' && 'test.') || '' }}pypi.org/p/${{ inputs.component }}/
name: testpypi
url: https://test.pypi.org/${{ inputs.component != 'all' && format('p/{0}/', inputs.component) || '' }}
permissions:
id-token: write
steps:
Expand All @@ -160,14 +149,37 @@ jobs:
run: |
./ci/tools/validate-release-wheels "${{ inputs.git-tag }}" "${{ inputs.component }}" "dist"

- name: Publish package distributions to PyPI
if: ${{ inputs.wheel-dst == 'pypi' }}
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0

- name: Publish package distributions to TestPyPI
if: ${{ inputs.wheel-dst == 'testpypi' }}
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0
with:
repository-url: https://test.pypi.org/legacy/

publish-pypi:
name: Publish wheels to PyPI
runs-on: ubuntu-latest
needs:
- determine-run-id
- publish-testpypi
environment:
name: pypi
url: https://pypi.org/${{ inputs.component != 'all' && format('p/{0}/', inputs.component) || '' }}
permissions:
id-token: write
steps:
- name: Checkout Source
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
Copy link
Collaborator

@rparolin rparolin Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider: Hoisting all the explicit SHA to the top and give them a variable name. So all tunables can be changed in a single place.

Copy link
Contributor Author

@cpcloud cpcloud Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That'd be nice, except there's still repetition across files.

Also, I don't think it matters because dependabot is updating these automatically for us. It handles repetition with aplomb.


- name: Download component wheels
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
./ci/tools/download-wheels "${{ needs.determine-run-id.outputs.run-id }}" "${{ inputs.component }}" "${{ github.repository }}" "dist"

- name: Validate wheel versions for release tag
run: |
./ci/tools/validate-release-wheels "${{ inputs.git-tag }}" "${{ inputs.component }}" "dist"

- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0

# TODO: add another job to make the release leave the draft state?
Loading