Skip to content

feat(ci): compile only shard-assigned packages in unit test shards - #9440

Merged
quirogas merged 2 commits into
ci/shard-compile/1-workspace-test-runnerfrom
ci/shard-compile/2-shard-aware-compilation
Sep 30, 2026
Merged

quirogas merged 2 commits into
ci/shard-compile/1-workspace-test-runnerfrom
ci/shard-compile/2-shard-aware-compilation

Conversation

@quirogas

@quirogas quirogas commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Right now, every unit test shard runs pnpm run compile (turbo run compile --filter='...[HEAD^1]') before starting ci/run_conditional_tests.sh. On large PRs, that means all 15 shards compile every changed package in the repo, even though each shard only tests a fraction of them. On the flip side, if a PR only touches ci/, run_conditional_tests.sh runs tests for all packages, but --filter='...[HEAD^1]' sees no changed packages and compiles nothing.

This PR moves the turbo run compile step into ci/run_conditional_tests.sh after the shard's package list is calculated, so each shard only compiles the packages it is actually going to test.

What changed

  • ci/run_conditional_tests.sh: Calculate shard_dirs first and run turbo run compile filtered only to the directories assigned to that shard (--filter={./<dir>}). Also replace ls -d in the directory loop with standard bash globbing.
  • .github/actions/run-unit-tests/action.yaml & .github/workflows/presubmit-bun.yml: Remove the top-level pnpm run compile step (since run_conditional_tests.sh now handles it) and include the shard index in the .turbo cache key.
  • .github/workflows/presubmit.yaml: Skip pnpm run compile in the lint job when no .ts files changed in the PR.
  • .github/actions/check-shard-status/action.yaml: Add per_page: 100 when listing workflow jobs so runs with more than 30 jobs don't miss shard results.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request optimizes the CI workflow by introducing shard-specific Turborepo caching and compiling only the packages assigned to each shard within ci/run_conditional_tests.sh. It also increases the page size for fetching workflow jobs and refines OS and path matching for Windows environments. Feedback highlights two key improvements: first, correcting the Turborepo filter syntax from ./${d%/}... to ..../${d%/} to ensure that local monorepo dependencies are compiled first; second, replacing the use of ls in a bash loop with direct shell globbing to make the script more robust and idiomatic.

Comment thread ci/run_conditional_tests.sh
Comment thread ci/run_conditional_tests.sh Outdated
@quirogas
quirogas marked this pull request as draft September 23, 2026 23:33
@quirogas
quirogas added this pull request to stack #9441 September 23, 2026 23:39
@quirogas
quirogas force-pushed the ci/shard-compile/2-shard-aware-compilation branch from df0e73d to 894910f Compare September 23, 2026 23:49
@quirogas
quirogas marked this pull request as ready for review September 24, 2026 01:03
@quirogas
quirogas force-pushed the ci/shard-compile/2-shard-aware-compilation branch from 894910f to dc1746b Compare September 24, 2026 01:06
Comment thread .github/workflows/presubmit.yaml
Comment thread ci/run_conditional_tests.sh Outdated
@quirogas
quirogas requested a review from a team as a code owner September 29, 2026 00:11
@quirogas
quirogas force-pushed the ci/shard-compile/2-shard-aware-compilation branch from dc1746b to 385965c Compare September 29, 2026 00:11
@quirogas
quirogas requested review from bshaffer and removed request for a team September 29, 2026 06:25
Comment thread ci/run_conditional_tests.sh Outdated
@quirogas
quirogas force-pushed the ci/shard-compile/2-shard-aware-compilation branch from 385965c to 8c17855 Compare September 29, 2026 22:51
@quirogas
quirogas requested a review from bshaffer September 29, 2026 22:57
@quirogas
quirogas force-pushed the ci/shard-compile/2-shard-aware-compilation branch from 8c17855 to b8e832d Compare September 30, 2026 00:12
@quirogas quirogas changed the title feat(ci): compile only shard-assigned packages with TURBO_DAEMON=false feat(ci): compile only shard-assigned packages in unit test shards Sep 30, 2026
@quirogas
quirogas force-pushed the ci/shard-compile/2-shard-aware-compilation branch 2 times, most recently from af80a50 to 02b8eba Compare September 30, 2026 17:36
@quirogas
quirogas force-pushed the ci/shard-compile/2-shard-aware-compilation branch from 02b8eba to 5383dcf Compare September 30, 2026 20:27
@quirogas
quirogas merged commit d087572 into main Sep 30, 2026
53 of 59 checks passed
@quirogas
quirogas deleted the ci/shard-compile/2-shard-aware-compilation branch September 30, 2026 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants