Add CircleCI job for maestro E2E tests#1637
Merged
Merged
Conversation
ajpallares
force-pushed
the
add-maestro-e2e-test-ci-job
branch
from
b005a36 to
4630a5a
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test
branch
from
0dc69d1 to
fa69d48
Compare
ajpallares
added
pr:other
ajpallares
force-pushed
the
add-maestro-e2e-test
branch
from
fa69d48 to
6b88b35
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test-ci-job
branch
2 times, most recently
from
88b1ea0 to
b6f68c4
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test
branch
from
6b88b35 to
39d6ca6
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test-ci-job
branch
from
b6f68c4 to
1b892fb
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test
branch
from
39d6ca6 to
cf08b12
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test
branch
from
87fecba to
1fcde2d
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test-ci-job
branch
from
bb49db0 to
d8f5df9
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test
branch
from
1fcde2d to
ef227dc
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test-ci-job
branch
from
d8f5df9 to
2420267
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test
branch
from
ef227dc to
599e3f6
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test-ci-job
branch
2 times, most recently
from
980e26e to
45e8f5b
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test
branch
from
7fb7ce4 to
e52229f
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test-ci-job
branch
3 times, most recently
from
de68dd0 to
c710f92
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test
branch
from
b451373 to
877003e
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test-ci-job
branch
from
c710f92 to
5b64b40
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test
branch
from
877003e to
94450ba
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test-ci-job
branch
from
5b64b40 to
6632446
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test
branch
from
94450ba to
9ad5d97
Compare
ajpallares
force-pushed
the
add-maestro-e2e-test-ci-job
branch
3 times, most recently
from
d0b20e7 to
8d1913a
Compare
Made-with: Cursor
- Prefix Dir.chdir and maestro test paths with ../ since Fastlane runs from the fastlane/ subdirectory - Wait for sys.boot_completed before sending keyevent to emulator Made-with: Cursor
Made-with: Cursor
Ensures workspace dependencies are properly installed from the project root before running Fastlane lanes, matching how other CI jobs work. Made-with: Cursor
The CI install-dependencies step runs yarn from the workspace root first, but the Fastlane lane's redundant yarn install from the MaestroTestApp directory triggers Yarn Berry's immutable check due to CI=true. Allow lockfile modifications for this second install. Made-with: Cursor
Made-with: Cursor
When yarn resolves to a newer react-native patch version, the hermes-engine version changes and conflicts with the committed Podfile.lock. Regenerating it avoids version mismatch errors. Made-with: Cursor
The Gradle build takes ~7 minutes, during which the emulator may become disconnected. Add adb start-server and wait-for-device before adb install to ensure the emulator is still available. Made-with: Cursor
Debug builds require a running Metro bundler server to serve JS. On CI there is no Metro server, so the app fails to load. Release builds pre-bundle the JS, so the app works standalone. Made-with: Cursor
…put dir - Revert iOS/Android back to Debug configuration (test store requires it) - Add FORCE_BUNDLING=1 to iOS xcodebuild so JS bundle is embedded without needing Metro running on CI - Add --test-output-dir to maestro commands so screenshots are stored as CI artifacts - Add takeScreenshot steps to Maestro flow for CI debugging Made-with: Cursor
The emulator may die (OOM) during the Gradle build step. Add timeout-based detection and automatic emulator restart if no device is found within 30 seconds after the build completes. Made-with: Cursor
Follow purchases-android pattern for emulator management: - Add circleci/android orb for proper AVD lifecycle management - Split Fastlane lane into build_maestro_app_android + run_maestro_e2e_tests_android - Build app first (memory-intensive), then create/start emulator - Eliminates OOM risk from concurrent build + emulator - Remove hacky emulator recovery logic Made-with: Cursor
Pre-launch the app and capture simulator logs to diagnose why the app shows the springboard instead of the Test Cases screen. Made-with: Cursor
Remove pre-launch, container check, and simulator log dump steps that were added during debugging. Maestro handles app launching on its own. Made-with: Cursor
Stop deleting Podfile.lock before pod install. The lockfile is committed (consistent with purchaseTesterTypescript) and ensures reproducible builds. Made-with: Cursor
- Upgrade maestro iOS job to Xcode 26.4 - Add grep check after sed to fail fast if the API key placeholder was not replaced (e.g. env var missing or placeholder renamed) Made-with: Cursor
Since Podfile.lock is no longer committed, use --repo-update to ensure the pod spec repo is fresh when generating the lockfile from scratch. Made-with: Cursor
Since yarn.lock is committed and Podfile.lock is now also committed in PR1, we can rely on Yarn's default immutable installs (CI=true) to enforce exact dependency resolution. This prevents version drift that was causing hermes-engine mismatches with the committed Podfile.lock. Also removed --repo-update from pod install since the committed Podfile.lock provides exact version pins. Made-with: Cursor
CircleCI's checkout doesn't clean untracked directories. If a previous build left a Pods/ directory (which is gitignored), stale Local Podspecs can conflict with the committed Podfile.lock when dependencies change. Cleaning Pods/ before pod install ensures a fresh resolution from the lockfile every time. Made-with: Cursor
Yarn Berry defaults to immutable installs when CI=true, but the MaestroTestApp workspace needs to resolve react-native freely (the root lockfile pins 0.78.0, CI may resolve ^0.78.0 to a newer patch). YARN_ENABLE_IMMUTABLE_INSTALLS=false allows this. Also cleans ios/Pods before pod install to prevent stale Local Podspecs from conflicting when the resolved react-native version changes between CI runs. Uses --repo-update since Podfile.lock is not committed. Made-with: Cursor
react-native 0.78.0's bundled fmt library has consteval C++20 compilation errors on Xcode 26.4's clang. Using 26.3 until the root react-native version is bumped. Made-with: Cursor
rickvdl
approved these changes
Apr 13, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds CircleCI jobs and Fastlane lanes to build and run Maestro E2E tests for the MaestroTestApp on both iOS and Android.
CircleCI jobs
run-maestro-e2e-tests-ios: macOS executor (Xcode 26.3, iPhone 17 simulator), builds the test app, runs Maestro testsrun-maestro-e2e-tests-android:android:2024.11.1machine image, builds a debug APK, creates an emulator viacircleci/androidorb, runs Maestro testsbuild-testworkflow) and on amaestro_e2e_testsscheduleFastlane lanes
run_maestro_e2e_tests_ios: replaces API key placeholder, installs JS + pod dependencies, builds viaxcodebuildwithFORCE_BUNDLING=1, installs on booted simulator, runs Maestrobuild_maestro_app_android: replaces API key placeholder, installs JS dependencies, builds via Gradlerun_maestro_e2e_tests_android: installs APK viaadb, runs MaestroImplementation notes
FORCE_BUNDLING=1forces the React Native build script to generatemain.jsbundlein Debug mode (normally skipped because Debug expects Metro). The AppDelegate in PR1 always loads the pre-bundled JS.YARN_ENABLE_IMMUTABLE_INSTALLS=falseis required because Yarn Berry defaults to immutable installs whenCI=true. The rootyarn.lockpinsreact-nativeto0.78.0, butMaestroTestApp/package.jsondeclares"react-native": "^0.78.0"(a range). When Yarn resolves this range to a newer patch (e.g.0.78.3), it needs to update the lockfile — which immutable mode forbids. We intentionally allow this drift because: (1) the MaestroTestApp is a CI-only test app rebuilt from scratch every run, (2) staying on the latest compatible patch avoids Xcode/toolchain incompatibilities, and (3)Podfile.lockis not committed for this app (gitignored in PR1) so CocoaPods resolves fresh each time too.react-native@0.78.0's bundledfmtlibrary hasconstevalC++20 compilation errors on Xcode 26.4's clang. This can be upgraded to 26.4 once the rootreact-nativeversion is bumped to a patch that includes the fix.grepcheck that fails immediately if the placeholder was not replaced.rm -rf ios/Podsis run beforepod installbecause CircleCI'scheckoutdoes not clean untracked directories. A stalePods/Local Podspecsfrom a previous build can conflict when the resolvedreact-nativeversion changes between runs.store_test_resultsandstore_artifacts.Contexts
e2e-tests: providesRC_E2E_TEST_API_KEY_PRODUCTION_TEST_STOREmaestro-e2e-tests: Maestro-specific configurationDepends on #1636