Skip to content

[DT-3629] Enable Postgres, Session, and Cookie plugins#3703

Draft
rushtong wants to merge 43 commits into
developfrom
gr-DT-3629-session-work
Draft

[DT-3629] Enable Postgres, Session, and Cookie plugins#3703
rushtong wants to merge 43 commits into
developfrom
gr-DT-3629-session-work

Conversation

@rushtong

@rushtong rushtong commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Addresses

https://broadworkbench.atlassian.net/browse/DT-3629

Summary

Phase 1 of the BFF migration: server-side, PostgreSQL-backed session infrastructure for the Fastify server. No user-visible changes — the infrastructure is inert until users are routed to it by the Phase 2 auth routes.

The DB/cookie/session plugins register only when the deployment provides the database configuration (DUOS_DB_HOST etc.), so the legacy client-side auth flow is completely unaffected until an environment is configured for the BFF — and every pod of a deployment behaves identically, with no network dependency at boot.

Changes

Session infrastructure (server/src/)

  • index.ts: registers @fastify/postgres, @fastify/cookie, and @fastify/session only when DUOS_DB_HOST is set. Validates DUOS_SESSION_SECRET (32+ chars) with an actionable error. Sets trustProxy: 1 so request.protocol honors the reverse proxy's X-Forwarded-Proto — without it, secure cookies silently never persist behind TLS-terminating proxies.
  • session/pgStore.ts: a thin typed SessionStore (get/set/destroy) against the user_sessions table. Expiry is computed in SQL from the same clock get filters with, so app/DB clock drift can't shorten or extend sessions. No rolling expiry — sessions get a fixed maxAge; Phase 2 adds throttled sliding expiry instead of a DB write per request.
  • types/session.ts: module augmentation typing the session payload (tokens live only server-side, never sent to the browser).
  • Postgres TLS is secure-by-default: DUOS_DB_SSL must be explicitly set to false for loopback transports (Cloud SQL Proxy sidecar, local compose), and doing so logs a warning.

Client config serving

  • config.ts: /config.json is now served by the server (via an onRequest hook, to avoid colliding with @fastify/vite's static route for the same file), with DUOS_API_URL overriding the static file's apiUrl. Deployed environments already carry the matching value via terra-helmfile; locally there's now one place to point at a different consent API.

Local development

  • docker-compose.yaml is split into a shared base plus two overlays: docker-compose.override.yaml (standalone — bundled Postgres stands in for the consent DB, auto-merged by Compose) and docker-compose.consent.yaml (shares a local consent compose stack's database, avoiding port collisions with its sqlproxy and app).
  • New .env.example template; both compose modes and pnpm start:server read .env.local.
  • DEVNOTES.md: full walkthrough of both modes, including the consent-DB sharing setup.

Deployment notes


Have you read Terra's Contributing Guide lately? If not, do that first.

  • Label PR with a Jira ticket number and include a link to the ticket
  • Label PR with a security risk modifier [no, low, medium, high]
  • PR describes scope of changes
  • Get a minimum of one thumbs worth of review, preferably two if enough team members are available
  • Get PO sign-off for all non-trivial UI or workflow changes
  • Verify all tests go green
  • Test this change deployed correctly and works on dev environment after deployment

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Coverage Report for DUOS Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 85.74% 9908 / 11556
🔵 Statements 85.18% 10507 / 12335
🔵 Functions 81.73% 2885 / 3530
🔵 Branches 77.13% 6000 / 7779
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
server/src/config.ts 100% 100% 100% 100%
server/src/index.ts 78.57% 67.3% 83.33% 78.43% 87, 171, 184-196
server/src/session/pgStore.ts 100% 100% 100% 100%
server/src/types/session.ts 100% 100% 100% 100%
Generated in workflow #6291 for commit 584eded by the Vitest Coverage Report Action

rushtong and others added 13 commits July 1, 2026 12:43
# Conflicts:
#	test/pages/BackgroundSignIn.spec.tsx
@fastify/vite's production static plugin (wildcard: false) registers its
own explicit route for every file under build/, including config.json,
which collided with our fastify.get('/config.json', ...) and crashed the
server at startup with FST_ERR_DUPLICATED_ROUTE. Intercepting the request
in onRequest instead avoids declaring a competing route.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The single-file profile-based setup had every mode's concerns
(bundled db vs. pointing at a local consent compose) interleaved via
profiles, conditional depends_on, and dual-purpose env var defaults,
making it hard to reason about either mode in isolation.

Splits into docker-compose.yaml (shared base), docker-compose.override.yaml
(standalone mode, auto-merged by Compose) and docker-compose.consent.yaml
(explicit opt-in for pointing at a local consent stack's Postgres). Moving
every mode-specific field out of the base file means the two overlays never
need to override each other, so no custom YAML merge tags are needed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds the Phase 1 “server foundation” pieces for the BFF migration by introducing PostgreSQL-backed server sessions in the Fastify server, and gating all DB/cookie/session startup behind the consent-served BFF_ENABLED feature flag so the existing client-side auth flow remains unaffected until rollout.

Changes:

  • Introduces a Postgres-backed @fastify/session store (createPgSessionStore) and session type augmentation for server-side auth state.
  • Gates Postgres + cookie + session plugin registration behind isBffEnabled() (fetched from {DUOS_API_URL}/feature/BFF_ENABLED), and adds /config.json interception to support DUOS_API_URL overrides.
  • Adds Vitest coverage for the session store, feature flag behavior, and client config behavior; updates local-dev Docker Compose files and DEVNOTES accordingly.

Reviewed changes

Copilot reviewed 17 out of 18 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
server/test/session.test.ts New integration-ish test exercising cookie/session behavior end-to-end with an in-memory PG stand-in.
server/test/pgStore.test.ts New unit tests validating SQL and expiry behavior for the Postgres session store.
server/test/index.test.ts Extends server bootstrap tests to cover /config.json override and BFF flag gating.
server/test/featureFlags.test.ts New tests for isBffEnabled() behavior and request shape.
server/test/clientConfig.test.ts New tests for config path resolution and DUOS_API_URL override behavior.
server/src/types/session.ts Adds Fastify Session module augmentation for typed session fields.
server/src/session/pgStore.ts Implements the Postgres-backed session store used by @fastify/session.
server/src/index.ts Gates DB/cookie/session startup behind BFF_ENABLED, enables trustProxy, and intercepts /config.json.
server/src/featureFlags.ts Adds consent-backed feature flag fetcher for BFF_ENABLED.
server/src/config.ts Removes legacy config loader/caching module (superseded by clientConfig flow).
server/src/clientConfig.ts Adds config.json path resolution + runtime read with optional DUOS_API_URL override.
docs/plans/BFF_Overview.md Marks Phase 1 as complete.
docker-compose.yaml Refactors base compose file and documents the two local dev modes and env var strategy.
docker-compose.override.yaml Adds “standalone” mode overlay with a bundled Postgres container.
docker-compose.consent.yaml Adds overlay for running against a local consent stack’s Postgres/proxy.
DEVNOTES.md Updates Docker run instructions and documents the two compose modes and DB host behavior.
.gitignore Ignores .env and the local consent DB dump used by the bundled DB container.
.env.example Adds a template env file for local BFF-related configuration.

Comment thread server/test/session.test.ts Outdated
Comment thread server/test/session.test.ts
Comment thread server/test/pgStore.test.ts Outdated
Comment thread server/test/pgStore.test.ts Outdated
Comment thread server/src/featureFlags.ts Outdated
Comment thread server/test/index.test.ts
Comment thread .env.example Outdated
@rushtong rushtong marked this pull request as ready for review July 7, 2026 12:04
@rushtong rushtong requested a review from a team as a code owner July 7, 2026 12:05
@rushtong rushtong requested review from fboulnois and otchet-broad and removed request for a team July 7, 2026 12:05
Comment thread server/src/session/pgStore.ts Outdated
Comment thread Dockerfile Outdated
@rushtong rushtong requested a review from kevinmarete July 8, 2026 13:52

@fboulnois fboulnois left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few suggestions. Claude has also flagged that the way we do feature flags isn't optimal. This is a preexisting issue, but worthwhile to think about:

await isBffEnabled(...) runs once in buildApp() and is never re-checked. A transient consent blip or >5s response during a rolling deploy makes a pod come up with no DB/session plugins for its entire lifetime, silently serving legacy auth even after consent recovers and flipping the flag on requires restarting every pod. During rollout this yields per-pod inconsistency visible only in logs.

Comment thread server/src/index.ts Outdated
Comment thread server/src/index.ts Outdated
Comment thread server/src/session/pgStore.ts Outdated
Comment thread server/src/featureFlags.ts Outdated
Comment thread server/src/session/pgStore.ts Outdated
@rushtong

rushtong commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

Great feedback @fboulnois - I'm going to revisit the plan and make some fundamental changes and hope to address all of the feedback in one go.

…me flag

- Register DB/cookie/session plugins when DUOS_DB_HOST is present instead of
  querying consent's BFF_ENABLED flag at startup. Pods now behave
  deterministically with no network dependency at boot, and BFF_ENABLED
  reverts to its planned role: the client-side rollout gate. Deletes
  server/src/featureFlags.ts. A half-configured deployment (DB host set,
  session secret missing) fails loud instead of silently booting legacy.
- Postgres TLS is secure-by-default: DUOS_DB_SSL now defaults to on and
  requires an explicit opt-out for loopback transports (Cloud SQL Proxy
  sidecar, local compose), parsed through an envBool coercion helper so a
  typo can't pick the insecure branch. Deployment envs must set
  DUOS_DB_SSL=false alongside the sqlproxy sidecar (terra-helmfile change).
- Drop rolling sessions: with the PG store wired, rolling re-saved the
  session (SELECT + UPSERT) on every session-bearing request just to bump
  expire. Sessions get a fixed maxAge; Phase 2 adds a throttled sliding
  expiry (re-save only near expiry).
- Compute session expiry in SQL (NOW() + maxAge) so writes use the same
  clock get filters with, and guard non-positive cookie.maxAge, not just
  null. Collapse the store methods through one run() helper for uniform
  error handling. This also makes the pgStore tests deterministic (no more
  Date.now() timing windows).
- Document the two-switch rollout model in docs/plans/BFF_Overview.md:
  session infra is deployment config; user-facing cutover is the boolean
  BFF_ENABLED flag, per environment for deterministic testing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@rushtong

Copy link
Copy Markdown
Contributor Author

@fboulnois thanks for the review — the feature-flag concern was the important one, and it exposed a deeper problem: the plan defines BFF_ENABLED as a client-side rollout gate, so the server checking it at boot was a divergence, with exactly the failure modes you described (a consent blip permanently downgrades a pod; flag flips need restarts).

1fcde3b removes the boot-time flag check entirely. The server now registers its DB/cookie/session plugins whenever the deployment provides the database config (DUOS_DB_HOST), so every pod of a deployment behaves identically with no network dependency at startup — a half-configured pod (DB host set, session secret missing) fails loud instead of silently booting legacy. The infrastructure is inert until users are routed to it (saveUninitialized: false, no auth routes yet). BFF_ENABLED goes back to its planned role: the client-side flag deciding which sign-in flow users get, flippable with no redeploys. The two-switch model is documented in docs/plans/BFF_Overview.md.

Also in that commit, per the inline threads: secure-by-default Postgres TLS with an explicit coerced opt-out, rolling dropped (throttled sliding expiry planned for Phase 2), session expiry computed on Postgres's clock with a non-positive-maxAge guard, and the store methods collapsed through a single run() helper.

… script

pnpm forwards args placed after the script name into the script itself, so
`pnpm run build --silent` appended --silent to the root build's trailing
`pnpm --filter duos-server build`, which re-forwarded it to the server's
`tsc` — an unknown compiler option that fails the build. Moving the flag
before the script name keeps it as pnpm's own output-suppression flag.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@rushtong rushtong marked this pull request as draft July 10, 2026 13:22
rushtong and others added 3 commits July 10, 2026 09:28
The integration-tests workflow waits on http://localhost:3000, but `vite
preview` inherited host local.dsde-dev.broadinstitute.org, which CI runners
can't resolve (ENOTFOUND). Extend the existing isCI gate to host and open,
matching how https is already handled.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
pnpm ci alone installs the npm package but not the Cypress binary (the
cypress-io action runs with install: false). Mirror component-tests.yml:
pnpm ci && pnpm cypress install.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts:
#	.github/workflows/integration-tests.yml
@rushtong rushtong changed the title [DT-3629] Enable Postgres, Session, and Cookie plugins gated behind feature flag [DT-3629] Enable Postgres, Session, and Cookie plugins Jul 10, 2026
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants