[Feature] Prompt Contest with a Language Model Compiler

[JacobLinCool]

Summary

Add a Prompt Contest mode where participants submit a prompt instead of handwritten source code. A contest-pinned Language Model Compiler turns the prompt into source code, then the generated source enters the existing Official Submission compiler and judge pipeline.

flowchart LR
    P["Prompt"] --> L["Pinned LLM Compiler"]
    C["Immutable public problem context"] --> L
    L --> G["Generated source"]
    G --> S["Existing Official Submission"]
    S --> J["Existing compiler and judge"]

Loading

This creates a competition around expressing algorithms, data structures, and performance intent clearly without requiring participants to implement every detail themselves.

Locked MVP rules

• One normal contest window with exactly five problems.
• Each participant receives three consumed attempts per problem (15 maximum across the contest).
• Submitting a prompt immediately starts generation, compilation, and judging.
• The Organizer pins one model, system prompt, decoding configuration, and version digest for the whole contest.
• The Organizer pins one output language, target, and optimization profile per problem.
• Participants cannot edit generated source before judging.
• A successful model response consumes an attempt even when the generated program has a compile error.
• Provider timeouts, provider 5xx responses, and platform infrastructure failures are retried and do not consume an attempt if they remain terminal infrastructure errors.
• Rejudge always reuses the original generated source and never invokes the model again.
• Participants immediately see their own prompt, generated source, compiler diagnostics, score, and resource summary.
• After the contest ends, each participant's best prompt and generated source for every problem are automatically public.

Ranking

For each participant, select the best submission for each of the five problems. Rank by:

1. highest total score;
2. lowest total deterministic cost across those five selected submissions;
3. earliest time at which that final best-result combination was achieved.

Architecture

Model Prompt Contest as a producer of normal Official Submissions rather than adding a fake programming language:

1. Admit a prompt_attempt and store the prompt as a submission-source object.
2. A deterministic LLM Compiler Workflow reads the immutable contest/problem context and pinned compiler configuration.
3. Strictly parse the model response into the existing multi-file source contract.
4. Store the generated source as a separate submission-source object.
5. Atomically create a normal Official Submission using that generated source.
6. Reuse the existing dispatcher, compiler, no-network Judge Container, result projection, rejudge, and leaderboard machinery.

Only the first validated model response may be committed. Duplicate or late provider responses must not create additional generated sources, submissions, or quota consumption.

Data model

• llm_compiler_configs: immutable provider/model identity, system-template digest, decoding parameters, token limits, and creation metadata. Provider credentials remain Worker secrets and are never persisted here.
• prompt_contest_configs: one-to-one with a prompt contest; pins compiler config, five-problem requirement, three-attempt limit, input limit, and output limit.
• prompt_problem_configs: fixed output profile and immutable public compiler-context digest for each contest problem.
• prompt_attempts: user/contest/problem identity, prompt source, compiler config digest, generation state, quota-consumption time, generated source, Official Submission, bounded failure code, and timestamps.
• Add content_kind = contestant-code | prompt | model-generated-code to submission_sources. Prompt and generated source use the existing erasure epoch, conditional PUT, tombstone, and retention behavior.
• Add prompt_generation_id as a fourth typed workflow_outbox target while retaining the exactly-one-target invariant.

Admission must atomically reject a new request when the same user/problem already has three consumed attempts or active reservations. Active generation temporarily reserves capacity; terminal infrastructure failure releases it. Idempotency keys must return the original attempt.

Immutable model context

Publishing a Prompt Contest materializes a content-addressed object at prompt-contexts/v1/{sha256} containing only:

• contest-public statement and examples;
• input/output contract;
• resource and output limits;
• the fixed output profile.

It must not contain hidden cases, expected answers, checker internals, GitHub credentials, provider credentials, or judge-package bytes. This becomes a third intentional R2 runtime byte class alongside judge packages and submission sources.

API and UI

Add:

• POST /api/contests/:id/prompt-attempts
  • body: { problemVersionId, prompt, idempotencyKey }
• GET /api/prompt-attempts/:id
  • generation state plus nested Official Submission summary/events when available
• GET /api/contests/:id/prompt-attempts
  • the current participant's attempt history and remaining quota
• GET /api/contests/:id/prompt-gallery
  • available only after the contest ends; returns each participant's best public prompt/generated source per problem

Extend contest create/update with submissionMode: "code" | "prompt" and prompt settings. Existing code contests and their HTTP behavior remain unchanged.

Organizer UI must support:

• selecting Prompt Contest mode;
• selecting exactly five problems;
• selecting one pinned LLM Compiler configuration;
• selecting one fixed allowed output profile per problem;
• reviewing the immutable context digest before publish.

Participant UI replaces the code editor with a prompt editor, remaining-attempt counter, generation progress, generated-source viewer, compiler/judge results, and attempt history.

Limits and security

• Prompt: one UTF-8 text value, maximum 16 KiB, no attachments.
• Model response: strict JSON files contract; no Markdown wrapper, unknown fields, path traversal, undeclared language/profile, or oversized source.
• Generated files retain the existing Official Submission limits.
• LLM requests have no tools, network browsing, filesystem access, secrets, hidden judge input, or conversation memory.
• Generated source still runs exclusively inside the existing no-network Judge Container.
• Account erasure tombstones both prompt and generated source; public gallery entries disappear when their source is erased.

Capacity and failure behavior

• D1 queued rows remain the only scheduling authority; do not add Queue, Redis, or another scheduler.
• Use deterministic Workflow IDs and status-before-create lost-ack handling.
• Initial limits: one active generation per user, ten globally active generations, and 100 globally queued generations.
• Retry retryable provider/platform failures with bounded exponential backoff.
• A terminal infrastructure failure stores no generated source, creates no Official Submission, and releases quota.
• Invalid model output or generated compile failure is contestant-visible and consumes quota after a model response was successfully received.

Acceptance criteria

• A Prompt Contest cannot publish unless it has exactly five problems, a pinned compiler config, and a valid fixed output profile for each problem.
• Four concurrent requests cannot exceed three active/consumed attempts for one user/problem.
• Same idempotency key returns exactly one attempt and one eventual Official Submission.
• Provider timeout/5xx/platform failure is retried and does not permanently consume quota.
• Generated compile error consumes quota and exposes the generated source and diagnostics.
• Model output cannot select another language/profile or access hidden data, credentials, tools, or network resources.
• Rejudge performs zero LLM calls and reuses the exact generated-source digest.
• Ranking uses best total score, then total deterministic cost, then achievement time.
• During the contest, prompt/generated source is owner-only; after the end, only each user's best work is public.
• Account erasure removes both private and public prompt/generated-source access without a late-writer race.
• Existing code-contest behavior and Official Submission APIs remain unchanged.

Non-goals

• Participant-selected models or bring-your-own provider keys.
• Editing generated source before judging.
• Agent/tool calls, browsing, retrieval, or multi-turn model conversations.
• Regenerating code during rejudge.
• Claiming byte-for-byte determinism from repeated model calls; fairness comes from pinned configuration and preserving the actual generated output.