Skip to content

feat(frontend): isolate public trust and add adjudication#445

Closed
ShivamB25 wants to merge 10 commits into
junhoyeo:mainfrom
ShivamB25:split/public-trust-adjudication
Closed

feat(frontend): isolate public trust and add adjudication#445
ShivamB25 wants to merge 10 commits into
junhoyeo:mainfrom
ShivamB25:split/public-trust-adjudication

Conversation

@ShivamB25

@ShivamB25 ShivamB25 commented Apr 18, 2026

Copy link
Copy Markdown

Summary

  • stack the adjudication layer on fix(submit): harden trusted submission ingestion #443 without mixing review operations into submit ingestion
  • add dedicated review list, detail, and adjudication APIs for quarantined submissions
  • replay accepted review payloads through the same current-main persistence path used by trusted submits
  • keep leaderboard, profile, embed, badge, landing, and list API surfaces synchronized after accepted writes

Rebase onto current main and #443

  • retained current main's submission model: competitive rows are trusted writes, while quarantined payloads remain in submission_reviews
  • extracted the current submit persistence transaction so direct trusted submits and accepted reviews share device-aware merges, first-write race recovery, savepoints, timestamp and active-time aggregation, regression guards, and MCP server persistence
  • added 0016_nostalgic_senator_kelly.sql plus generated snapshot/journal metadata for reviewed_at, reviewed_by_username, review_note, and the submission metadata receipt watermark
  • kept review cost storage at numeric(18,4) and extended migration checks through journal index 16
  • removed obsolete public status-filter/profile-metadata changes from the old branch because the current quarantine architecture no longer stores untrusted data in public submissions

Adjudication contract

  • review APIs fail closed unless SUBMISSION_REVIEW_API_TOKEN is configured; browser sessions and ordinary personal tokens are not accepted
  • SUBMISSION_REVIEW_OPERATOR_USERNAME records the external operator identity, with a bounded fallback when unset
  • PATCH accepts only trusted or rejected, validates UUIDs, bounds request bodies, rejects non-object JSON, and records audit metadata
  • accepted payloads are revalidated with current server validation before persistence
  • row locking makes adjudication single-use; competitive persistence and the review-state transition commit atomically
  • rejection records the decision without creating competitive data
  • a partial submission increments submit_count when its trusted partition is first written; later approval of its queued partition does not increment it again
  • a fully queued submission increments submit_count once when approval creates or updates competitive data

Metadata ordering

  • submissions.metadata_received_at is a dedicated receipt-order watermark; migration 0016 backfills it from each row's existing updated_at before setting the default and NOT NULL
  • delayed approvals compare the review's latest submission receipt time (submission_reviews.updated_at before adjudication) against that watermark
  • aggregate usage and device/day data still merge from an older approved review, but stale cliVersion, submission hash, session-shape metrics, schema metadata, and MCP server metadata cannot overwrite a newer direct submission
  • pending-review retries preserve competitive_write_applied = true monotonically with SQL OR

Cache and rank consistency

  • compare pre/post all-time RANK() snapshots using the same aggregate sources and numeric(18,4) cost semantics as current public rank queries
  • invalidate every affected user's rank-bearing caches and public paths after acceptance
  • normalize and deduplicate mixed-case usernames before cache-tag invalidation
  • include profile, API, embed, and badge paths for canonical and lowercase username variants
  • preserve async group-leaderboard invalidation for the adjudicated user

Verification

  • Frontend Vitest after latest-main restack: 74 files, 670 tests passed
  • Frontend typecheck: bun run typecheck
  • Rust CLI after main.rs conflict resolution: 1004 tests passed, 1 ignored
  • Frontend production build: bun run build
  • Frontend lint: 0 errors (6 pre-existing warnings outside this change)
  • Fresh PostgreSQL migration/runtime checker: contiguous journal 0..16, every migration applied, receipt watermark and competitive-write marker defaults/nullability verified, plus constraints/indexes/concurrency/rollback checks passed
  • Live app + PostgreSQL: listed queued reviews, loaded detail, accepted a review into submissions/daily/device/MCP data, rejected a review without competitive data, confirmed repeat adjudication returns 409, and accepted a mixed-case user after rank snapshot/cache changes
  • Submit-count runtime: first trusted write persisted 1; partial review approval kept 1 while adding its queued day
  • Metadata-order runtime: queued version 1.0.0, accepted newer direct version 9.0.0, then approved the older review; aggregate totals reached 200 while cli_version = 9.0.0, MCP metadata stayed newer-direct-mcp, and submit_count = 2

Stack

Depends on #443 commit 095a6975 and is intentionally kept as a separate stack layer. #444 remains closed and untouched.

Refs #441

@vercel

vercel Bot commented Apr 18, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
tokscale Ready Ready Preview, Comment Jul 14, 2026 9:02am

Request Review

@ShivamB25

Copy link
Copy Markdown
Author

This PR is intentionally stacked.

If it were marked as a standalone ready-to-merge change, reviewers would be looking at a diff that conceptually depends on code still under review in the earlier PR(s). Reviewing it in order keeps the trust-boundary changes, replay semantics, and public-surface changes easier to reason about.

@ShivamB25
ShivamB25 marked this pull request as ready for review April 18, 2026 21:36

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4 issues found across 45 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/frontend/src/lib/leaderboard/getLeaderboard.ts">

<violation number="1" location="packages/frontend/src/lib/leaderboard/getLeaderboard.ts:354">
P1: All-time leaderboard and user-rank now rank the full verified-user set in memory on each cache miss instead of using DB-side pagination/ranking.</violation>
</file>

<file name="packages/frontend/src/lib/submissionPersistence.ts">

<violation number="1" location="packages/frontend/src/lib/submissionPersistence.ts:329">
P3: Unsorted Set materialization makes persisted client/model arrays nondeterministic across identical submissions.</violation>
</file>

<file name="packages/frontend/src/app/u/[username]/ProfilePageClient.tsx">

<violation number="1" location="packages/frontend/src/app/u/[username]/ProfilePageClient.tsx:235">
P2: Locale/timezone-dependent `toLocaleString()` can cause server/client hydration mismatches and unstable displayed timestamps.</violation>
</file>

<file name="packages/frontend/__tests__/api/settingsSubmittedDataDelete.test.ts">

<violation number="1" location="packages/frontend/__tests__/api/settingsSubmittedDataDelete.test.ts:145">
P3: Use different mocked row counts for submissions vs. reviews so the test can catch a mix-up in which delete result is used to compute `deletedSubmissions`.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

@@ -1,7 +1,8 @@
import { unstable_cache } from "next/cache";

@cubic-dev-ai cubic-dev-ai Bot Apr 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: All-time leaderboard and user-rank now rank the full verified-user set in memory on each cache miss instead of using DB-side pagination/ranking.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/frontend/src/lib/leaderboard/getLeaderboard.ts, line 354:

<comment>All-time leaderboard and user-rank now rank the full verified-user set in memory on each cache miss instead of using DB-side pagination/ranking.</comment>

<file context>
@@ -397,59 +307,65 @@ async function fetchLeaderboardData(
 
-  const totalUsers = Number(globalStats[0]?.uniqueUsers) || 0;
-  const totalPages = Math.ceil(totalUsers / limit);
+  const rankedUsers = rankLeaderboardUsers(allTimeUsers, sortBy);
+  const filteredUsers = rankedUsers.filter((user) =>
+    matchesLeaderboardSearch(user, search)
</file context>
Fix with Cubic

Comment thread packages/frontend/src/lib/submissionPersistence.ts
Comment thread packages/frontend/src/lib/validation/submissionTrust.ts Outdated
Comment thread packages/frontend/src/lib/submissionReviews.ts Outdated
<ReviewMetaList>
<ReviewMetaItem>
<ReviewMetaLabel>Updated</ReviewMetaLabel>
<ReviewMetaValue>{new Date(data.nonCompetitive.updatedAt).toLocaleString()}</ReviewMetaValue>

@cubic-dev-ai cubic-dev-ai Bot Apr 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Locale/timezone-dependent toLocaleString() can cause server/client hydration mismatches and unstable displayed timestamps.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/frontend/src/app/u/[username]/ProfilePageClient.tsx, line 235:

<comment>Locale/timezone-dependent `toLocaleString()` can cause server/client hydration mismatches and unstable displayed timestamps.</comment>

<file context>
@@ -164,6 +190,80 @@ const EARLY_ADOPTERS = ["code-yeongyu", "gtg7784", "qodot"];
+              <ReviewMetaList>
+                <ReviewMetaItem>
+                  <ReviewMetaLabel>Updated</ReviewMetaLabel>
+                  <ReviewMetaValue>{new Date(data.nonCompetitive.updatedAt).toLocaleString()}</ReviewMetaValue>
+                </ReviewMetaItem>
+                {data.nonCompetitive.reasonCodes.length > 0 ? (
</file context>
Suggested change
<ReviewMetaValue>{new Date(data.nonCompetitive.updatedAt).toLocaleString()}</ReviewMetaValue>
<ReviewMetaValue>
{new Intl.DateTimeFormat("en-US", {
dateStyle: "medium",
timeStyle: "short",
timeZone: "UTC",
}).format(new Date(data.nonCompetitive.updatedAt))}
</ReviewMetaValue>
Fix with Cubic

@@ -0,0 +1,353 @@
import { eq, inArray, sql } from "drizzle-orm";

@cubic-dev-ai cubic-dev-ai Bot Apr 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: Unsorted Set materialization makes persisted client/model arrays nondeterministic across identical submissions.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/frontend/src/lib/submissionPersistence.ts, line 329:

<comment>Unsorted Set materialization makes persisted client/model arrays nondeterministic across identical submissions.</comment>

<file context>
@@ -0,0 +1,353 @@
+      reasoningTokens: totalReasoning,
+      dateStart: aggregates.dateStart,
+      dateEnd: aggregates.dateEnd,
+      sourcesUsed: Array.from(allClients),
+      modelsUsed: Array.from(allModels),
+      cliVersion: data.meta.version,
</file context>
Fix with Cubic

});
mockState.setDeletedRows([{ id: "submission-1" }]);
mockState.setDeletedSubmissionRows([{ id: "submission-1" }]);
mockState.setDeletedReviewRows([{ id: "review-1" }]);

@cubic-dev-ai cubic-dev-ai Bot Apr 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: Use different mocked row counts for submissions vs. reviews so the test can catch a mix-up in which delete result is used to compute deletedSubmissions.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/frontend/__tests__/api/settingsSubmittedDataDelete.test.ts, line 145:

<comment>Use different mocked row counts for submissions vs. reviews so the test can catch a mix-up in which delete result is used to compute `deletedSubmissions`.</comment>

<file context>
@@ -125,7 +141,8 @@ describe("DELETE /api/settings/submitted-data", () => {
     });
-    mockState.setDeletedRows([{ id: "submission-1" }]);
+    mockState.setDeletedSubmissionRows([{ id: "submission-1" }]);
+    mockState.setDeletedReviewRows([{ id: "review-1" }]);
 
     const response = await DELETE(createRequest());
</file context>
Suggested change
mockState.setDeletedReviewRows([{ id: "review-1" }]);
mockState.setDeletedReviewRows([{ id: "review-1-1" }, { id: "review-1-2" }]);
Fix with Cubic

@junhoyeo

junhoyeo commented Jul 6, 2026

Copy link
Copy Markdown
Owner

Hi @ShivamB25 — first, thank you for this substantial piece of work, and apologies that it sat unreviewed for so long. The public-trust isolation + adjudication design here is genuinely valuable and is not something main has implemented in the meantime, so we'd love to land it — but the branch can no longer be merged as-is.

Where things stand: the merge base (48aa05c, Apr 17) is now ~350 commits behind main, and a test merge produces 20 conflicted files. The conflicts sit exactly on the subsystems this PR rewrites, and main has since rewritten them too:

A few smaller items from the earlier automated review that are still valid and worth folding in:

  • listSubmissionReviewArtifacts is typed as returning summaries but maps full Detail objects, so the admin list endpoint ships every submission's full payload/hash — a summary-scoped mapper would fix both the type lie and the response bloat.
  • getUtcDateStringFromTimestamp can throw a RangeError for schema-valid timestampMs above 8.64e15 (max JS Date) — tighten the schema bound or guard Number.isFinite(date.getTime()).
  • toLocaleString() for review timestamps risks SSR/CSR hydration mismatches — prefer a fixed UTC formatter.

Also worth confirming before you invest: the adjudication routes gate on session.isAdmin, which doesn't exist on current main — that's fine as a net-new concept, but it's worth a quick alignment on whether admin-only is the intended auth model.

How this fits with the rest of your stack: this PR stacks on #443 and #444. I've left rebase notes on #443 (which we want landed close together with this one, so REVIEW_REQUIRED submissions have an adjudication path and nothing gets stranded), and I've closed #444 since main's regression-guarded merge superseded that branch structurally — see the note there for what remains salvageable. So the practical path is: rebase #443 first, then rework this one on top — or, given how much main has moved, possibly collapse the remaining work into a single fresh PR against current main. Happy to review promptly this time either way, and thanks again for the effort you put into this.

@ShivamB25

Copy link
Copy Markdown
Author

@junhoyeo #445 is now rebased as a separate layer on the final #443 head. I adapted adjudication to current main: accepted reviews use the shared current submit persistence transaction, audit fields are migration 0016, review APIs use a dedicated fail-closed operator token, stored payloads are revalidated, and row locking keeps replay plus state transition atomic. Rank/cache refresh now mirrors current aggregate rank semantics, handles ties and mixed-case cache keys, and covers profile/API/embed/badge/leaderboard surfaces. Final verification: 71 Vitest files / 646 tests, typecheck, production build, migration checker through 0..16, and live PostgreSQL accept/reject/idempotency/mixed-case-rank scenarios all passed. The PR description has the full deployment and verification contract. Ready for re-review after #443.

@ShivamB25

Copy link
Copy Markdown
Author

@junhoyeo #445 is restacked on final #443 head 3a4d529f; current head is 674524ad.

The follow-up fixes two delayed-adjudication invariants:

  • partial submissions increment submit_count only on their initial trusted write; approval of the queued partition leaves the count unchanged, while fully queued submissions count once on approval;
  • older queued payloads cannot overwrite newer direct-submit metadata. 0016 adds submissions.metadata_received_at, backfilled from existing updated_at, and approvals compare the review receipt watermark before updating CLI version, hash, schema/session-shape fields, or MCP metadata. Aggregate usage still merges.

Regression coverage now includes sticky review upserts, first-write count = 1, stale metadata omission, receipt timestamp forwarding, and delayed A/B ordering. Verification: 73 files / 651 tests passed, typecheck and production build passed, lint has 0 errors (6 pre-existing warnings), fresh PostgreSQL migrations/checker passed through 0..16, and live runtime checks confirmed partial approval stays at count 1 plus an older queued v1 approval preserves newer direct v9/MCP metadata while totals merge.

@ShivamB25

Copy link
Copy Markdown
Author

@junhoyeo Restacked #445’s five adjudication-only commits onto repaired #443 head 095a6975; new #445 head is 61c561d6. The resolved main.rs is inherited from #443, so the CLI conflict was resolved once rather than independently in both PRs.

GitHub now reports MERGEABLE. Final stacked verification: Rust CLI 1004 passed, 1 ignored; frontend 74 files / 670 tests passed; typecheck passed; production build passed; lint has 0 errors and the same 6 pre-existing warnings.

@ShivamB25 ShivamB25 closed this Jul 19, 2026
@ShivamB25

Copy link
Copy Markdown
Author

fk this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants