Read this first. This repo is frozen evidence, not a maintained product. It is the unedited output of an autonomous AI run (one packaging commit added on top, nothing else touched). The built app has a known, documented, unfixed SSRF vulnerability and an unproven product thesis. It must not be deployed publicly as-is. Full disclaimers at the bottom.
This is the full artifact of stress-testing the idea-to-ship autonomous tier (autopilot + build-loop) on the heaviest mandate I could write: find your own grounded niche, build a real frontend and backend, hold a serious design bar, and don't stop to ask. Claude (Fable 5) ran it unattended overnight: ideate, a 13-agent deep-dive, a 16-unit prompt pack it wrote for itself, and a 47-pass build loop. The 22-commit history in this repo, one unit landing every ~34 minutes from 4:37pm to 4:37am, is the receipt.
What it built: Redline, a design-crit engine for AI-built frontends. Paste a URL, a Playwright worker renders it deterministically at two viewports, takes a census of DOM, computed styles, pixel samples, and focus/hover probes, runs ~26 typed rules across 6 locked dimensions (typography, spacing, color, hierarchy, components, slop), and serves an editorial crit page: verdict, scores, numbered evidence overlays on annotated screenshots, copy-ready fixes, re-crit diffs, share cards, fix-pack exports. Next.js app + sibling worker + SQLite, one npm start, zero env vars. There are zero LLM calls in the product: the AI-synthesis seam exists but is human-gated, never imported, and a source-scan test fails the build if it ever is.
This is verbatim what was typed. Nothing else.
Run /autopilot Persona: a world-class, award-winning UI/UX designer — late 20s, brilliant product instincts, witty, deeply in tune with 2026 culture and tech. Build the persona out robustly and run in-character. Grounding (firewall): have them land on a product idea in a space with real demand and ground it in real 2026 signals — don't invent a market; if you genuinely can't ground it, stop and tell me. Ambition: intentionally heavy build to stress-test you — frontend + backend; design is load-bearing and should be genuinely beautiful/award-caliber (carry a substantive design direction; run the full multi-pass visual loop + the different-model critic). Real external APIs are human-gated — emit them, never fake them. Scope: lock a heavy-but-coherent scope in the brief; I expect a robust CONCEPT_BRIEF and larger, detailed prompt pack(s). Narrate phases, use files as memory, emit a handoff if you run long — take the iterations and time you need. At the end, be honest about what's validated vs. a bet, and the taste/finish/market tail that's mine.
There is no product idea in that prompt. Producing the scope is the first half of the pipeline: the run invented a founder persona ("Sasha Moreau", with a written firewall that her claims about demand never count as evidence), grounded a niche in a 94-item corpus of real 2026 signals, and locked a concept brief with a success metric and a kill criterion before any code existed.
Re-checked after the run from git, the on-disk session transcripts, and a cold re-run of the full test wall in an isolated copy. Self-reports are labeled as such in the debrief.
| What | Number |
|---|---|
| Wall clock | 11h 59m first to last commit; ~13.5h including grounding and research |
| Commits | 22, one unit every ~34 minutes, overnight, unattended |
| Prompt-pack units | 16 of 16 passed |
| Code | 27,154 lines of app + test TypeScript (158 files) |
| Tests | 387 unit + 92 e2e (26 desktop-only skips, 0 failures), reproduced green cold |
| Iterations | 42 build-loop passes + 5 blind critic rounds (562 separate-model critic calls, transcript-verified) |
| Research | 13-agent deep-dive in 42 minutes; 17 live-site spikes; a 52-URL render pilot |
| Evidence kept | 372 screenshots in docs/ledgers/screens/ |
| Tokens | ~24.1M fresh (1.5M uncached input + 19.9M cache writes + 2.6M output) over 5,523 API calls; ~784M total processed, 97% prompt-cache reads |
| Human gates | 7 emitted, all open at close, none faked (see docs/GATES.md) |
1. A real, unfixed SSRF hole. The worker's liveness probe pins DNS correctly; the actual page render does not. page.goto() hands Chromium a raw hostname, Chromium re-resolves it, so a host that answers public to the guard and private to the browser can reach internal addresses at render time. The run's own pre-build research named this exact failure mode and called egress filtering "the load-bearing control". The fix got encoded at one rung, lost at the seam between rungs, and then the hardening tests locked the safe half, making the wall greener while the hole stayed open. The compensating control is procedural: docs/DEPLOY_CHECKLIST.md makes a worker egress firewall a BLOCKER behind a human sign-off that has never been given. An external audit found this in hours; 12 hours of self-verification structurally could not.
2. The product thesis is unproven. Whether the engine separates good design from AI slop on sites it has never seen was never tested. Every threshold was tuned on, then "validated" against, the same 24-site corpus. No holdout set exists, and ~44% of even the training AI sites produce a thin crit (0 or 1 findings). The build knew this and carried it as an explicit bet. The independent audit's split: execution and honesty ~8/10, thesis ~4/10.
3. Smaller honesty cracks, recorded, none load-bearing. A calibration:"final" flag stamped while a human taste review is still owed; small count drift across docs (15 vs 16 vs 17 prompts); one critic-gate sequence the ledger never explains. A human editor catches these in the last mile, which is the point of the last mile.
The honesty was mechanized, not narrated: gates are tests, enums are CHECK constraints, forbidden patterns are source-scans, and a "strength" structurally cannot coexist with a finding on the same dimension. When the product failed its own engine's cursor rule, the run fixed the product, not the rule. When its spec's draft heuristics measured broken on real data, it re-derived them from the data instead of shipping them. The external audit it never saw confirmed its own pre-build risk flags on 4 of 5 load-bearing risks. The fifth is the SSRF above.
The run happened in a private repo. Before publishing, the history was rewritten once with git filter-repo to remove third-party assets the research phase had saved to disk while doing hands-on competitor verification:
- a competitor's production JavaScript bundle and saved page captures (their copyrighted code; analyzing it locally was normal competitive research, republishing it would not be)
- another competitor's npm package tarball and its unpacked source (Apache-2.0, so legal to redistribute, but unnecessary bulk here)
The run's own analysis scripts and extracted findings from that research are all still here (docs/research/redline-deep-dive/spikes/). Nothing else was changed: every commit's content, message, order, timestamp, and authorship is otherwise identical. Because the rewrite changed commit SHAs, and the ledgers and audit cite the originals, the full old-to-new mapping is in docs/COMMIT_MAP.md.
Also not published, by design: data/ (runtime DB and full render artifacts of third-party sites, gitignored from day one) and the external audit package (kept separate so "the build never saw the audit" stays a clean provenance claim; its conclusions are in the debrief).
- The debrief (the honest write-up of the whole experiment, verified numbers, failures):
docs/DEBRIEF.md - Per-unit build ledgers (iteration tables, defects found, "Signals NOT RUN" sections):
docs/ledgers/ - The human gates it refused to fake:
docs/GATES.md - The do-not-deploy wall:
docs/DEPLOY_CHECKLIST.md - The run's own close-out:
docs/AUTOPILOT_CLOSEOUT.md - The prompt pack it wrote for itself:
docs/CRITROOM_PROMPT_PACK.md - The 13-agent deep-dive + spikes:
docs/research/redline-deep-dive/ - 372 archived verification screenshots:
docs/ledgers/screens/
Running it locally is fine (npm install && npm start, then http://localhost:3100 and paste a URL; /specimen shows the design system). Local only. The deploy checklist explains why a public deployment is forbidden until a human clears the security gate.
- Experimental tier.
autopilotis an early, lightly proven experiment: one heavy run plus a few small proofs. This case study is one data point, not a benchmark. A different model or harness may behave very differently on the same prompt.- Token-heavy, with real numbers. This run consumed about 24.1 million fresh tokens across 5,523 API calls (~784M total processed, 97% prompt-cache reads, ~2.6M generated). It ran on a heavy subscription plan; on metered pricing this would be a real bill. Budget before you press go.
- The ~80% ceiling is real. The output is a first draft a human finishes. On this run the unfinished 20% included a security vulnerability, every market truth, and the taste sign-off.
- Do NOT deploy this publicly as-is. Redline ships with a known, documented, unfixed SSRF vulnerability (DNS rebinding at render time). This repo is published as evidence, not as software to run on the open internet.
- AI-generated content notice. The code, docs, and ledgers in this repo were generated by an autonomous Claude run. The debrief and this README were drafted with AI assistance from the run's transcripts and an independent audit, then human-reviewed. Numbers labeled "verified" were re-checked against git, disk, and a cold re-run of the test suites; anything labeled a self-report stays a self-report.
- Third-party content. Research docs include small screenshots/crops and frozen style censuses of public third-party sites, kept as calibration and critique evidence, plus a calibration corpus that lists publicly-showcased AI-built sites by URL as factual provenance (it is not a quality leaderboard). None of that is covered by this repo's license. Site owners who want something removed can open an issue on idea-to-ship-skills (this repo is archived/read-only).
Code and original documentation: MIT (see LICENSE). Third-party material is excluded from that license: the research screenshots/censuses noted above, and the self-hosted typefaces (Newsreader, Space Grotesk, JetBrains Mono), which are under the SIL Open Font License. The slop-detection research credits trope hypotheses from Impeccable (Apache-2.0), see NOTICE and src/engine/slop.ts.
The skills that flew this run: https://github.com/nelsonwerd/idea-to-ship-skills · The Reddit debrief thread has the story.