Skip to content

feat: add --skip-provider CLI option and dashboard lap E2E test#2063

Merged
nick-inkeep merged 14 commits intomainfrom
feat/create-agents-dashboard-lap
Feb 18, 2026
Merged

feat: add --skip-provider CLI option and dashboard lap E2E test#2063
nick-inkeep merged 14 commits intomainfrom
feat/create-agents-dashboard-lap

Conversation

@nick-inkeep
Copy link
Collaborator

@nick-inkeep nick-inkeep commented Feb 17, 2026

Summary

Adds a --skip-provider option to create-agents CLI and extends the quickstart E2E test with a full dashboard lap — CLI scaffolding through browser login, navigation, and playground panel verification.

Motivation

User scenario 1: Zero-friction quickstart. New users and CI pipelines shouldn't need a real API key to scaffold and explore a working project. --skip-provider lets you scaffold with the mock/default model (landed in #2056), so pnpm create-agents my-project --skip-provider produces a fully functional project end-to-end — including hitting the chat endpoint from the playground — without any provider credentials.

User scenario 2: Full-journey regression coverage. The existing E2E test stopped at "API returns project metadata." That left the dashboard login → project navigation → agent detail → playground flow completely uncovered. This PR extends coverage to catch regressions across the entire first-run experience a user would walk through.

Approach

--skip-provider CLI option

Two entry points, same behavior:

  1. --skip-provider flag — non-interactive, scaffolds with { base: { model: 'mock/default' } } model config
  2. Interactive "Skip" option — 5th choice in the provider select prompt, same result

--skip-provider takes precedence over any --openai-key/--anthropic-key flags passed alongside it (logs a warning rather than silently ignoring).

Dashboard lap E2E

The test now continues after inkeep push succeeds:

  1. Starts the manage-ui standalone server (startDashboardServer() — forks the Next.js standalone server.js from the built @inkeep/agents-manage-ui package)
  2. Launches headless Chromium via Playwright
  3. Logs in via email/password form (admin@example.com)
  4. Navigates: projects list → project detail → agent detail → "Try it" button → playground panel
  5. Asserts #playground-pane visible and #inkeep-widget-root initialized

On failure: captures full-page screenshot + page URL + HTML content for debugging.

Architectural decisions

Playwright over Cypress for E2E dashboard lap. The existing Cypress test suite is for the manage-ui package itself. This test lives in create-agents and verifies the scaffolded project's dashboard works end-to-end. Playwright is lighter to install in CI (just chromium), doesn't need a dev server, and works well with the standalone Next.js output.

Standalone server, not next dev. The dashboard is started from the pre-built .next/standalone/agents-manage-ui/server.js via node:child_process.fork(). This is faster and more realistic — it tests the same artifact that ships to production.

localhost vs 127.0.0.1 split. The API server uses 127.0.0.1:3002 (avoids IPv6 resolution issues on Ubuntu CI), but the dashboard must use localhost:3002 so auth cookies share the same domain between dashboard (:3000) and API (:3002). Both are localhost-scoped cookies.

Auth init reliability. CI applies migrations from the monorepo root before the test runs, so setup-dev:cloud's internal migration step can fail and exit early — skipping auth init entirely. Fixed by running auth init as a separate step. Also added an idempotent signup-via-API fallback (POST /api/auth/sign-up/email) as a belt-and-suspenders measure.

Selector strategy. Card components use a link-overlay pattern that intercepts pointer events, so clicks use a[href*="..."] with force: true rather than text-based selectors. waitForURL uses domcontentloaded instead of load to avoid timeout from slow external resource loading.

Known limitation

Message sending in the playground (typing + receiving a response via the chat widget) is not yet tested. The InkeepEmbeddedChat widget renders in a shadow DOM / iframe that needs additional DOM verification work in headless Chromium. The test gracefully stops at asserting the playground panel opens and the widget root initializes. Can be added in a follow-up.

Test coverage

Automated (unit tests — utils.test.ts)

Scenario What's verified
--skip-provider flag Scaffolds with mock/default model config, skips provider prompt, skips API key prompt
Interactive "Skip" selection Same mock config when user selects "Skip" from the provider menu
Precedence over API key flags --skip-provider + --openai-key → mock config wins (key ignored)

Automated (E2E — quickstart.test.ts)

Scenario What's verified
CLI scaffolding with --skip-provider Directory structure, .env contents (no OPENAI_API_KEY), package.json shape
Full stack startup API server healthy, database setup, auth init
inkeep push Project registered in manage DB, API returns correct tenant + project ID
Dashboard login Email/password form submission → redirect to projects page
Project navigation Projects list → project detail page via card click
Agent navigation Agent card click → agent detail page
Playground open "Try it" button → #playground-pane visible
Widget initialization #inkeep-widget-root element present in DOM
Failure diagnostics Screenshot + URL + HTML captured on any dashboard lap failure

Manually QA-tested (not in automated suite)

  • --skip-provider CLI flow locally — scaffolds with mock/default, no API key prompt
  • --skip-provider + --openai-key together — warning logged, mock config used
  • Interactive "Skip" option appears as 5th choice with hint text
  • E2E dashboard lap passes in CI (full stack — verified across 14 iterative fix commits)
  • CI Playwright caching works (cache hit on subsequent runs)

CI status

All checks green: CI, E2E, auto-format, Vercel previews, PR review.

Future considerations

  • Playground message send test — extend dashboard lap to type a message and assert a response from the mock provider. Blocked on shadow DOM / iframe handling in headless Chromium.
  • --skip-provider in README — document the flag for discoverability (surfaced in review, deferred).

🤖 Generated with Claude Code

@vercel
Copy link

vercel bot commented Feb 17, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agents-api Ready Ready Preview, Comment Feb 18, 2026 9:27pm
agents-docs Ready Ready Preview, Comment Feb 18, 2026 9:27pm
agents-manage-ui Ready Ready Preview, Comment Feb 18, 2026 9:27pm

Request Review

@changeset-bot
Copy link

changeset-bot bot commented Feb 17, 2026

🦋 Changeset detected

Latest commit: 396949d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 9 packages
Name Type
@inkeep/create-agents Patch
@inkeep/agents-api Patch
@inkeep/agents-manage-ui Patch
@inkeep/agents-cli Patch
@inkeep/agents-core Patch
@inkeep/agents-mcp Patch
@inkeep/agents-sdk Patch
@inkeep/agents-work-apps Patch
@inkeep/ai-sdk-provider Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(4) Total Issues | Risk: Medium

🟠⚠️ Major (2) 🟠⚠️

Inline Comments:

  • 🟠 Major: quickstart.test.ts:262-264 Empty catch block swallows screenshot error details
  • 🟠 Major: utils.ts:284-294 Dashboard server output not captured for debugging

🟡 Minor (1) 🟡

Inline Comments:

  • 🟡 Minor: utils.ts:193 Interactive "Skip" label is ambiguous

💭 Consider (1) 💭

💭 1) packages/create-agents/README.md Document --skip-provider in README

Issue: The README (which is the primary onboarding documentation) does not mention the --skip-provider flag or the "Skip" interactive option. This feature is specifically designed for "developers exploring the framework without an API key" per the PR motivation.

Why: Features designed for first-contact exploration should be prominently documented in first-contact surfaces. New users exploring without credentials won't discover the feature otherwise.

Fix: Consider adding a note after the npx @inkeep/create-agents my-agents line:

**No API key?** Use `--skip-provider` to scaffold with a mock provider:
```bash
npx @inkeep/create-agents my-agents --skip-provider

This lets you explore the framework with deterministic responses before adding a real provider.


**Refs:**
- [packages/create-agents/README.md](https://github.com/inkeep/agents/blob/368ae9675e70a4ad57040d9fb3fc5857ce803cce/packages/create-agents/README.md)

---
<div align="center">

## 💡 APPROVE WITH SUGGESTIONS

</div>

**Summary:** This is a well-structured PR that adds useful functionality for developer onboarding. The `--skip-provider` flag follows existing CLI conventions and the dashboard lap E2E test provides valuable coverage for the full user journey. The main suggestions focus on improving debuggability of CI failures (capturing server output and screenshot errors) and making the "Skip" interactive option self-documenting. Consider documenting the new flag in the README to maximize discoverability.

<details>
<summary>Discarded (5)</summary>

| Location | Issue | Reason Discarded |
|----------|-------|------------------|
| `utils.ts:82-84` | Mock config only has `base` property unlike other providers | Intentional design - mock is simpler and fallback behavior is documented. Not a real inconsistency. |
| `utils.test.ts:408` | Missing test for .env file content with skip-provider | The E2E test verifies .env structure. Low risk - .env generation uses same code paths. |
| `utils.ts:268-274` | Empty catch blocks for process cleanup | Expected behavior - processes may already be dead. Comments explain intent. |
| `quickstart.test.ts:222-223` | Hardcoded test credentials | CI uses seeded admin credentials, not dynamically generated ones. This is by design. |
| `quickstart.test.ts:279-286` | Incorrect stdout/stderr reading from devProcess | Pre-existing issue - the accumulated `serverOutput` variable is logged elsewhere. Not introduced by this PR. |

</details>

<details>
<summary>Reviewers (6)</summary>

| Reviewer | Returned | Main&nbsp;Findings | Consider | While&nbsp;You're&nbsp;Here | Inline&nbsp;Comments | Pending&nbsp;Recs | Discarded |
|----------|----------|--------------------|----------|-------------------------------|----------------------|-------------------|-----------|
| `pr-review-errors` | 4 | 0 | 0 | 0 | 2 | 0 | 2 |
| `pr-review-product` | 4 | 0 | 1 | 0 | 1 | 0 | 1 |
| `pr-review-consistency` | 1 | 0 | 0 | 0 | 0 | 0 | 1 |
| `pr-review-tests` | 5 | 0 | 0 | 0 | 0 | 0 | 1 |
| `pr-review-devops` | 5 | 0 | 0 | 0 | 0 | 0 | 0 |
| `pr-review-standards` | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| **Total** | **19** | **0** | **1** | **0** | **3** | **0** | **5** |

Note: `pr-review-devops` returned all INFO-level findings (confirmations of good patterns); `pr-review-standards` returned no findings.

</details>

@github-actions github-actions bot deleted a comment from claude bot Feb 17, 2026
@vercel vercel bot temporarily deployed to Preview – agents-docs February 17, 2026 14:54 Inactive
@nick-inkeep
Copy link
Collaborator Author

Review Response

1. Empty catch block for screenshots (ACCEPTED)

Fixed in 8933c4c. Now logs the full error:

} catch (screenshotError) {
  console.error('Failed to capture screenshot:', screenshotError);
}

2. Dashboard server output not captured (ACCEPTED)

Fixed in 8933c4c. Added stdout/stderr handlers in startDashboardServer() that log errors and readiness signals in CI, matching the existing pattern for the API devProcess.

3. Interactive "Skip" label is ambiguous (DECLINED)

The "Skip" label was intentionally chosen during spec design — it aligns with how other CLI tools label skip options (e.g., pnpm create-next-app uses "Skip"). The --skip-provider flag description already explains the mock provider behavior. Adding "(use mock provider)" to the interactive label would make the select option disproportionately long relative to sibling options ("Anthropic", "OpenAI", etc.).

CI E2E Fix

Also fixed the root cause of the CI failure — setup-dev:cloud exits early when its internal migrations fail (CI already applies them from the monorepo root), which skips auth init and the "default" organization is never created. Added a separate auth init step that runs regardless of setup-dev:cloud's exit status.

Copy link
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(2) Total Issues | Risk: Low

This is a delta re-review following commit 8933c4c1 which addressed the Major findings from the prior review.

✅ Prior Feedback Addressed

The following issues from the previous review have been resolved:

Finding Resolution
🟠 Major: Screenshot error details swallowed ✅ Now logs full screenshotError object at line 280
🟠 Major: Dashboard server output not captured ✅ Added selective output handler at lines 296-306 that logs errors, EADDRINUSE, and ready signals in CI

The new auth init step (lines 123-139) is a clean fix for the CI race condition — it ensures the default organization exists before the dashboard lap attempts login.

🕐 Pending Recommendations (2)

These items from the prior review remain unaddressed but are lower priority:

  • 🟡 utils.ts:193 Interactive "Skip" label could be more self-documenting (e.g., "Skip (use mock provider)")
  • 💭 README.md Consider documenting --skip-provider in the README for discoverability

✅ APPROVE

Summary: The delta changes adequately address the Major findings from the prior review. Screenshot errors now include details for debugging, and dashboard server output is captured selectively in CI. The new auth init step is a solid fix for the CI race condition. The remaining Minor and Consider items are polish suggestions that don't block merge. Ship it! 🚀

Reviewers (0)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
Total 0 0 0 0 0 2 0

Note: Delta re-review with narrow scope (1 commit, 2 files) — no sub-agent reviewers dispatched. Prior feedback validation performed directly by orchestrator.

Copy link
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(0) Total Issues | Risk: Low

This is a delta re-review following commit 2a3cfcd6 which adds reliability improvements to the CI E2E test.

✅ Delta Changes Assessment

The delta (1 file, +59 lines) contains four improvements to CI reliability in quickstart.test.ts:

Change Assessment
Auth init timeout 30s → 120s ✅ Appropriate for SpiceDB's 30-retry schema write behavior
Explicit DB URLs with fallbacks ✅ Defensive coding for CI environment variability
Fallback signup via API ✅ Idempotent backup path with good comments explaining the "why"
Pre-flight login test ✅ Early diagnostic catch before expensive Playwright operations

All changes follow the existing error handling patterns in the test (non-fatal warn/continue) and are well-commented.

🕐 Pending Recommendations (2)

These items from prior reviews remain unaddressed but are lower priority:

  • 🟡 utils.ts:193 Interactive "Skip" label could be more self-documenting (e.g., "Skip (use mock provider)")
  • 💭 README.md Consider documenting --skip-provider in the README for discoverability

✅ APPROVE

Summary: The delta changes are solid reliability improvements that increase CI test stability without changing production behavior. The auth init hardening and diagnostic pre-flights should help catch and diagnose issues earlier. The remaining Minor and Consider items from prior reviews are polish suggestions that don't block merge. Ready to ship! 🚢

Reviewers (0)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
Total 0 0 0 0 0 2 0

Note: Delta re-review with narrow scope (1 file, +59 lines, test reliability improvements only) — no sub-agent reviewers dispatched. Direct orchestrator review.

Copy link
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(0) Total Issues | Risk: Low

This is a delta re-review following commit 8cf020e2 which fixes a cross-origin cookie issue in the E2E test.

✅ Delta Changes Assessment

The delta (1 file, +8/-5 lines) contains a targeted fix for cookie domain alignment:

Change Assessment
Added dashboardApiUrl constant using localhost instead of 127.0.0.1 ✅ Correct — browsers treat 127.0.0.1 and localhost as different origins for cookies
Added Origin header to signup/login API requests ✅ Required for CORS preflight and cookie handling
Updated dashboard server env vars to use dashboardApiUrl ✅ Ensures dashboard↔API cookie sharing on same domain

The comment at line 19-21 clearly explains the rationale: cookies set by the API (localhost:3002) need to be readable by the dashboard (localhost:3000), which requires both to use localhost (not 127.0.0.1).

🕐 Pending Recommendations (2)

These items from prior reviews remain unaddressed but are lower priority:

  • 🟡 utils.ts:193 Interactive "Skip" label could be more self-documenting (e.g., "Skip (use mock provider)")
  • 💭 README.md Consider documenting --skip-provider in the README for discoverability

✅ APPROVE

Summary: The delta is a clean, well-documented fix for the cross-origin cookie issue that was preventing the dashboard E2E test from working correctly. The localhost vs 127.0.0.1 distinction is a real browser behavior that the comments now explain clearly. No new issues identified. Ready to merge! 🎉

Reviewers (0)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
Total 0 0 0 0 0 2 0

Note: Delta re-review with narrow scope (1 commit, 1 file, +8/-5 lines, test-only cookie fix) — no sub-agent reviewers dispatched. Direct orchestrator review.

Copy link
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(0) Total Issues | Risk: Low

This is a delta re-review following commit 85820602 which improves test selector reliability.

✅ Delta Changes Assessment

The delta (1 file, 1 line change) is a targeted Playwright selector improvement:

Change Assessment
text=${projectId}a[href*="${projectId}"] ✅ More robust — href selectors are specific to anchor elements, while text selectors can match any element containing that text. Reduces flakiness from unintended matches.

This follows Playwright best practices: prefer structural/attribute selectors over text-based selectors when the element is semantically identifiable by its attributes.

🕐 Pending Recommendations (2)

These items from prior reviews remain unaddressed but are lower priority:

  • 🟡 utils.ts:193 Interactive "Skip" label could be more self-documenting (e.g., "Skip (use mock provider)") — PR author declined with rationale (aligns with other CLI tools, sibling option length consistency)
  • 💭 README.md Consider documenting --skip-provider in the README for discoverability

✅ APPROVE

Summary: The delta is a single-line test reliability improvement — switching from a text selector to a more specific href selector for the project link click. This is a standard Playwright best practice that reduces test flakiness. No new issues. The PR is ready to merge! 🎉

Reviewers (0)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
Total 0 0 0 0 0 2 0

Note: Delta re-review with minimal scope (1 commit, 1 line, test selector improvement only) — no sub-agent reviewers dispatched. Direct orchestrator review.

@github-actions github-actions bot deleted a comment from claude bot Feb 17, 2026
@vercel vercel bot requested a deployment to Preview – agents-docs February 17, 2026 16:15 Abandoned
Copy link
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(0) Total Issues | Risk: Low

This is a full review of PR #2063 following a manual --full-review request. Five domain-specific reviewers were dispatched.


✅ Review Assessment

All five reviewers returned clean assessments with no actionable findings:

Reviewer Findings Assessment
pr-review-standards 0 Code quality is solid; no bugs, performance issues, or AGENTS.md violations identified
pr-review-product 0 Customer mental-model is clear; the --skip-provider flag follows established CLI patterns
pr-review-consistency 0 The --skip-provider flag follows the existing --skip-* naming pattern; interactive "Skip" option is consistent with sibling options
pr-review-tests 0 Test coverage is appropriate; unit tests cover skip-provider behaviors, E2E test validates full user journey
pr-review-devops 0 CI changes follow established patterns; Playwright cache/install mirrors main CI job patterns

📋 Implementation Quality

The --skip-provider feature is well-implemented:

  1. CLI flag integration: Correctly defined in Commander with clear description ('Skip AI provider setup (uses mock provider)')
  2. Interactive flow: The "Skip" option includes a helpful hint property for discoverability without lengthening the label
  3. Precedence handling: skipProvider correctly takes precedence over API keys with an informative warning
  4. Model configuration: Uses defaultMockModelConfigurations which follows the appropriate pattern for a minimal mock provider
  5. Test coverage: 3 unit tests + comprehensive E2E test covering the full user journey (CLI → API → dashboard → playground)

The dashboard lap E2E extension is robust:

  • Proper Playwright selectors (href-based for cards, id-based for playground pane)
  • Appropriate error handling with screenshot capture on failure
  • Server output logging in CI for debuggability
  • Reasonable timeouts for CI environment

🕐 Pending Recommendations (1)

This item from prior reviews remains unaddressed but is lower priority:

  • 💭 README.md Consider documenting --skip-provider in the README for discoverability — feature is designed for first-contact exploration

✅ APPROVE

Summary: This PR is well-implemented and has been thoroughly validated through 14+ review iterations. The --skip-provider flag follows existing CLI conventions, the mock provider integration is clean, and the dashboard E2E lap provides valuable coverage for the full user journey. The delta since the last review is a single-line Playwright selector fix (removing invalid pierce/ prefix). All prior Major findings have been addressed. Ship it! 🚀

Reviewers (5)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
pr-review-standards 0 0 0 0 0 0 0
pr-review-product 0 0 0 0 0 0 0
pr-review-consistency 0 0 0 0 0 0 0
pr-review-tests 0 0 0 0 0 0 0
pr-review-devops 0 0 0 0 0 0 0
Total 0 0 0 0 0 1 0

Note: All reviewers returned clean assessments. The single Pending Recommendation is a documentation suggestion carried forward from prior reviews.

Copy link
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(0) Total Issues | Risk: Low

This is a full review of PR #2063. Five domain-specific reviewers were dispatched.


✅ Review Assessment

All five reviewers returned clean assessments with no actionable findings:

Reviewer Findings Assessment
pr-review-standards 0 Code quality is solid; no bugs, performance issues, or AGENTS.md violations identified
pr-review-product 0 Customer mental-model is clear; the --skip-provider flag follows established CLI patterns and includes appropriate discoverability via the hint property
pr-review-consistency 0 The --skip-provider flag follows the existing --skip-* naming pattern; interactive "Skip" option is consistent with sibling provider options
pr-review-tests 0 Test coverage is appropriate; 3 unit tests cover skip-provider behaviors, E2E test validates full user journey
pr-review-devops 0 CI changes follow established patterns; Playwright cache/install mirrors main CI job; build filter addition is necessary

📋 Implementation Quality

The --skip-provider feature is well-implemented:

  1. CLI flag integration: Correctly defined in Commander with clear description ('Skip AI provider setup (uses mock provider)')
  2. Interactive flow: The "Skip" option includes a helpful hint property ('scaffolds with mock provider, no API key needed') for discoverability without lengthening the label
  3. Precedence handling: skipProvider correctly takes precedence over API keys with an informative warning
  4. Model configuration: Uses defaultMockModelConfigurations which follows the appropriate pattern for a minimal mock provider (only base defined, consistent with existing templates)
  5. Test coverage: 3 unit tests cover the CLI flag, interactive selection, and flag precedence behaviors

The dashboard lap E2E extension is robust:

  • Proper Playwright selectors (href-based for cards, id-based for playground pane)
  • Appropriate error handling with screenshot capture on failure
  • Server output logging in CI for debuggability
  • Reasonable timeouts for CI environment
  • Explicit deferral of message sending (documented in PR) due to iframe complexity

🕐 Pending Recommendations (1)

This item from prior reviews remains unaddressed but is lower priority:

  • 💭 README.md Consider documenting --skip-provider in the README for discoverability — feature is designed for first-contact exploration

✅ APPROVE

Summary: This PR is well-implemented and ready to merge. The --skip-provider flag follows existing CLI conventions, enables a useful onboarding path for developers without API credentials, and the dashboard E2E lap provides valuable coverage for the full user journey. All prior Major findings have been addressed, and the remaining documentation suggestion is optional polish. Ship it! 🚀

Reviewers (5)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
pr-review-standards 0 0 0 0 0 0 0
pr-review-product 0 0 0 0 0 0 0
pr-review-consistency 0 0 0 0 0 0 0
pr-review-tests 0 0 0 0 0 0 0
pr-review-devops 0 0 0 0 0 0 0
Total 0 0 0 0 0 1 0

Note: All reviewers returned clean assessments. The single Pending Recommendation is a documentation suggestion carried forward from prior reviews.

@github-actions github-actions bot deleted a comment from claude bot Feb 18, 2026
nick-inkeep and others added 14 commits February 18, 2026 13:23
Add "Skip" option to create-agents CLI provider selection that defaults
to mock/default model, enabling project scaffolding without an API key.
Add browser-based dashboard validation to the quickstart E2E test using
Playwright: login, navigate to project/agent, open playground.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace overly broad iframe/[class*="chat"] selectors with
  [role="dialog"]/[class*="playground"] for more specific matching
- Fix variable shadowing in catch block (page → activePage)
- Remove inline comments per no-comments convention
- Add changeset for @inkeep/create-agents --skip-provider feature

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Run auth init separately after setup-dev:cloud to create the "default"
  organization even when the template's internal migrations fail (CI
  already applies migrations from the monorepo root)
- Log dashboard server error details in screenshot catch block
- Capture dashboard server stdout/stderr for CI debugging

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Increase auth init timeout to 120s (SpiceDB retries up to 30x)
- Add stream: true to see auth init output in CI logs
- Add explicit DB URLs to auth init env (belt and suspenders)
- Log auth init exit code and output on failure
- Add signup fallback via Better Auth HTTP endpoint after API is ready
- Add login API test before Playwright to diagnose auth issues

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ssue

The dashboard (localhost:3000) was configured with API URL 127.0.0.1:3002,
causing auth cookies set by the API to not be sent to the dashboard (different
hostname = different cookie domain). Use localhost:3002 for the dashboard's
API URL to match the Cypress test setup where login works correctly.

Also add Origin headers to diagnostic fetch calls to satisfy Better Auth's
CSRF protection (MISSING_OR_NULL_ORIGIN error).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The project card displays "Activities planner" (display name) but the
test was looking for "activities-planner" (project ID). Use href-based
locator which matches the URL containing the project ID.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Cards use a linkoverlay pattern (data-slot="linkoverlay") that intercepts
pointer events. Playwright's actionability check rejects clicks intercepted
by overlays, so force:true is needed for project card, agent card, and
Try it button clicks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Next.js pages with SSE/streaming connections never fire the load event,
causing waitForURL to timeout despite the URL matching correctly.
domcontentloaded fires once the HTML is parsed, which is sufficient for
navigation verification.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
text=Activities planner matched the project heading/breadcrumb instead of
the agent card, preventing navigation. Use a[href*="/agents/activities-planner"]
for precise targeting, matching the pattern used for the project card click.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The playground uses a ResizablePanel with id="playground-pane", not
data-testid or role attributes. The previous selectors
([data-testid="playground-panel"], [role="dialog"], [class*="playground"])
didn't match the actual DOM structure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…t assertion

- Add `hint` field to interactive "Skip" option for discoverability
- Warn when --skip-provider is used alongside explicit API key flags
- Assert InkeepEmbeddedChat widget initializes inside playground panel

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…format

Playwright CSS selectors pierce open shadow DOM by default — the
`pierce/` prefix is Chrome DevTools Protocol syntax, not Playwright.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(0) Total Issues | Risk: Low

This is a full review of PR #2063. Five domain-specific reviewers were dispatched.


✅ Review Assessment

All five reviewers returned clean assessments with no actionable findings:

Reviewer Findings Assessment
pr-review-standards 0 Code quality is solid; no bugs, performance issues, or AGENTS.md violations identified
pr-review-product 0 Customer mental-model is clear; the --skip-provider flag follows established CLI patterns
pr-review-consistency 0 The --skip-provider flag follows the existing --skip-* naming pattern; interactive "Skip" option is consistent with sibling provider options
pr-review-tests 0 Test coverage is appropriate; 3 unit tests cover skip-provider behaviors, E2E test validates full user journey
pr-review-devops 0 CI changes follow established patterns; Playwright cache/install mirrors main CI job patterns

📋 Implementation Quality

The --skip-provider feature is well-implemented:

  1. CLI flag integration: Correctly defined in Commander with clear description ('Skip AI provider setup (uses mock provider)')
  2. Interactive flow: The "Skip" option includes a helpful hint property ('scaffolds with mock provider, no API key needed') for discoverability
  3. Precedence handling: skipProvider correctly takes precedence over API keys with an informative warning
  4. Model configuration: Uses defaultMockModelConfigurations which follows the appropriate pattern (only base defined, consistent with existing templates)
  5. Test coverage: 3 unit tests cover CLI flag, interactive selection, and flag precedence behaviors; E2E test validates full dashboard journey

The dashboard lap E2E extension is robust:

  • Proper Playwright selectors (href-based for cards, id-based for playground pane)
  • Appropriate error handling with screenshot capture on failure
  • Server output logging in CI for debuggability
  • Reasonable timeouts for CI environment
  • Explicit deferral of message sending (documented in PR) due to iframe complexity

🕐 Pending Recommendations (1)

This item from prior reviews remains unaddressed but is lower priority:

  • 💭 README.md Consider documenting --skip-provider in the README for discoverability — feature is designed for first-contact exploration

✅ APPROVE

Summary: This PR is well-implemented and ready to merge. The --skip-provider flag follows existing CLI conventions, enables a useful onboarding path for developers without API credentials, and the dashboard E2E lap provides valuable coverage for the full user journey. All prior Major findings have been addressed, and the remaining documentation suggestion is optional polish. Ship it! 🚀

Reviewers (5)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
pr-review-standards 0 0 0 0 0 0 0
pr-review-product 0 0 0 0 0 0 0
pr-review-consistency 0 0 0 0 0 0 0
pr-review-tests 0 0 0 0 0 0 0
pr-review-devops 0 0 0 0 0 0 0
Total 0 0 0 0 0 1 0

Note: All reviewers returned clean assessments. The single Pending Recommendation is a documentation suggestion carried forward from prior reviews.

@github-actions github-actions bot deleted a comment from claude bot Feb 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants