ci(python-sdk): build, unit test and e2e workflow against the dev relayer - #661
Draft
HoangDucBach wants to merge 1 commit into
Draft
ci(python-sdk): build, unit test and e2e workflow against the dev relayer#661HoangDucBach wants to merge 1 commit into
HoangDucBach wants to merge 1 commit into
Conversation
HoangDucBach
force-pushed
the
test/e2e-python-sdk
branch
from
August 17, 2026 08:29
b92ee54 to
326187f
Compare
HoangDucBach
marked this pull request as draft
August 17, 2026 08:37
HoangDucBach
force-pushed
the
test/e2e-python-sdk
branch
2 times, most recently
from
August 17, 2026 09:40
da2a027 to
7aeb92d
Compare
HoangDucBach
force-pushed
the
test/e2e-python-sdk
branch
from
August 17, 2026 09:54
7aeb92d to
81801b4
Compare
…ayer The Python SDK had no CI of its own: test.yml only runs the relayer's own e2e script, and release-python-sdk.yml just publishes. Nothing built the package from source or ran either test suite on a push. Add test-python-sdk.yml, named to sit alongside release-python-sdk.yml, with two jobs: - `unit` — installs from source and runs ruff plus the mocked suite across Python 3.9-3.12, matching requires-python and the pyproject classifiers. Runs on every pull request touching the package. - `e2e` — runs tests/test_integration.py against the dev relayer on pushes to dev, manual runs, and a weekly cron. Staging and production are left out until their credentials exist. E2E reuses the `benchmark-dev` environment rather than introducing new secrets: BENCH_DELEGATE_KEY is the delegate private key the SDK signs with, which test_integration.py reads as MEMWAL_PRIVATE_KEY. Sharing the account is safe because the namespaces do not overlap — benchmark-live.yml writes to `benchmark`, these tests use `sdk-test`, `default` and a per-run `sdk-e2e-<uuid>`. E2E is skipped on pull requests: environment secrets are not exposed to fork PRs, and each authenticated run writes real memories, which costs embedding, Walrus storage and gas. The relayer URL falls back to the dev endpoint recorded in docs/relayer/benchmark-ci-setup.md so a missing BENCH_SERVER_URL cannot silently point a run at the wrong relayer. Missing credentials only warn — the authenticated tests self-skip via the `requires_key` marker, so the seven unauthenticated tests still run: two health checks and five auth-rejection cases covering unsigned requests, a mismatched signature, expired and future timestamps, and an unregistered key. The e2e job name is static on purpose. A `name:` holding an expression is left unevaluated when the job is skipped, so the check would surface under two different names and could never back a required status check. The three pre-existing ruff offences in tests/test_signing.py are fixed here because the new lint step covers tests/ and would otherwise fail on arrival.
HoangDucBach
force-pushed
the
test/e2e-python-sdk
branch
from
August 17, 2026 10:31
81801b4 to
b978b4a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket
CI for the Python SDK: build from source, unit test, e2e against the dev relayer.
What changed?
New
.github/workflows/test-python-sdk.yml, named to sit alongsiderelease-python-sdk.yml:unit— installs from source, runsruff, then the mocked suite across Python 3.9–3.12. Every pull request touching the package.e2e— runstests/test_integration.pyagainst the dev relayer on pushes todev, manual runs, and a weekly cron. Reuses thebenchmark-devenvironment, so no new secrets are needed.Also fixes the three pre-existing
ruffoffences intests/test_signing.py, which the new lint step would otherwise fail on.Why is this needed?
The Python SDK had no CI.
test.ymlonly runs the relayer's own e2e script andrelease-python-sdk.ymljust publishes, so nothing built the package or ran either suite on a push — 121 mocked and 23 integration tests sat unexecuted.What e2e covers
Unauthenticated, runs without any credential:
/healthreturns ok; exposes a version stringMemWalErrorAuthenticated, needs the environment secrets:
remember_and_waitreturns blob and owner; default namespace; custom namespacelimit; result field shapememories_usedThe auth-rejection group builds its requests with
httpxrather than through the SDK, so it checks the relayer's own contract: timestamp window and signature verification.Scope
The workflow. Both suites already exist and
tests/test_integration.pyalready readsMEMWAL_SERVER_URL/MEMWAL_PRIVATE_KEY/MEMWAL_ACCOUNT_IDand self-skips its authenticated cases — this wires that up rather than adding tests.Out of scope
Staging and production environments; tighter namespace isolation for e2e writes.
How was this tested?
Ran what the
unitjob runs:ruffclean, 116 passed. Workflow validated withactionlint. All four matrix versions pass on this PR, ande2ecorrectly skips on pull requests.How can the reviewer verify it?
Risks and dependencies
benchmark-devrather than adding secrets.BENCH_DELEGATE_KEYis the delegate private key the SDK signs with, whichtest_integration.pyreads asMEMWAL_PRIVATE_KEY;BENCH_ACCOUNT_IDmaps toMEMWAL_ACCOUNT_ID. Sharing the account is safe because the namespaces do not overlap.BENCH_SERVER_URLfalls back tohttps://relayer.dev.memwal.aiso a missing variable cannot point a run at the wrong relayer. If those secrets turn out not to be set, the unauthenticated group still runs and the rest warn.sdk-testanddefaultnamespaces, wherebenchmark-live.ymlusesbenchmark— no overlap, but full flow is the only group that isolates itself per run. Worth tightening before this is pointed at production.Author checklist