[DT-3629] Enable Postgres, Session, and Cookie plugins#3703
Conversation
# Conflicts: # server/src/index.ts
Coverage Report for DUOS Coverage Report
File Coverage
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||
# Conflicts: # test/pages/BackgroundSignIn.spec.tsx
@fastify/vite's production static plugin (wildcard: false) registers its
own explicit route for every file under build/, including config.json,
which collided with our fastify.get('/config.json', ...) and crashed the
server at startup with FST_ERR_DUPLICATED_ROUTE. Intercepting the request
in onRequest instead avoids declaring a competing route.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The single-file profile-based setup had every mode's concerns (bundled db vs. pointing at a local consent compose) interleaved via profiles, conditional depends_on, and dual-purpose env var defaults, making it hard to reason about either mode in isolation. Splits into docker-compose.yaml (shared base), docker-compose.override.yaml (standalone mode, auto-merged by Compose) and docker-compose.consent.yaml (explicit opt-in for pointing at a local consent stack's Postgres). Moving every mode-specific field out of the base file means the two overlays never need to override each other, so no custom YAML merge tags are needed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds the Phase 1 “server foundation” pieces for the BFF migration by introducing PostgreSQL-backed server sessions in the Fastify server, and gating all DB/cookie/session startup behind the consent-served BFF_ENABLED feature flag so the existing client-side auth flow remains unaffected until rollout.
Changes:
- Introduces a Postgres-backed
@fastify/sessionstore (createPgSessionStore) and session type augmentation for server-side auth state. - Gates Postgres + cookie + session plugin registration behind
isBffEnabled()(fetched from{DUOS_API_URL}/feature/BFF_ENABLED), and adds/config.jsoninterception to supportDUOS_API_URLoverrides. - Adds Vitest coverage for the session store, feature flag behavior, and client config behavior; updates local-dev Docker Compose files and DEVNOTES accordingly.
Reviewed changes
Copilot reviewed 17 out of 18 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| server/test/session.test.ts | New integration-ish test exercising cookie/session behavior end-to-end with an in-memory PG stand-in. |
| server/test/pgStore.test.ts | New unit tests validating SQL and expiry behavior for the Postgres session store. |
| server/test/index.test.ts | Extends server bootstrap tests to cover /config.json override and BFF flag gating. |
| server/test/featureFlags.test.ts | New tests for isBffEnabled() behavior and request shape. |
| server/test/clientConfig.test.ts | New tests for config path resolution and DUOS_API_URL override behavior. |
| server/src/types/session.ts | Adds Fastify Session module augmentation for typed session fields. |
| server/src/session/pgStore.ts | Implements the Postgres-backed session store used by @fastify/session. |
| server/src/index.ts | Gates DB/cookie/session startup behind BFF_ENABLED, enables trustProxy, and intercepts /config.json. |
| server/src/featureFlags.ts | Adds consent-backed feature flag fetcher for BFF_ENABLED. |
| server/src/config.ts | Removes legacy config loader/caching module (superseded by clientConfig flow). |
| server/src/clientConfig.ts | Adds config.json path resolution + runtime read with optional DUOS_API_URL override. |
| docs/plans/BFF_Overview.md | Marks Phase 1 as complete. |
| docker-compose.yaml | Refactors base compose file and documents the two local dev modes and env var strategy. |
| docker-compose.override.yaml | Adds “standalone” mode overlay with a bundled Postgres container. |
| docker-compose.consent.yaml | Adds overlay for running against a local consent stack’s Postgres/proxy. |
| DEVNOTES.md | Updates Docker run instructions and documents the two compose modes and DB host behavior. |
| .gitignore | Ignores .env and the local consent DB dump used by the bundled DB container. |
| .env.example | Adds a template env file for local BFF-related configuration. |
# Conflicts: # Dockerfile
fboulnois
left a comment
There was a problem hiding this comment.
A few suggestions. Claude has also flagged that the way we do feature flags isn't optimal. This is a preexisting issue, but worthwhile to think about:
await isBffEnabled(...) runs once in buildApp() and is never re-checked. A transient consent blip or >5s response during a rolling deploy makes a pod come up with no DB/session plugins for its entire lifetime, silently serving legacy auth even after consent recovers and flipping the flag on requires restarting every pod. During rollout this yields per-pod inconsistency visible only in logs.
|
Great feedback @fboulnois - I'm going to revisit the plan and make some fundamental changes and hope to address all of the feedback in one go. |
…me flag - Register DB/cookie/session plugins when DUOS_DB_HOST is present instead of querying consent's BFF_ENABLED flag at startup. Pods now behave deterministically with no network dependency at boot, and BFF_ENABLED reverts to its planned role: the client-side rollout gate. Deletes server/src/featureFlags.ts. A half-configured deployment (DB host set, session secret missing) fails loud instead of silently booting legacy. - Postgres TLS is secure-by-default: DUOS_DB_SSL now defaults to on and requires an explicit opt-out for loopback transports (Cloud SQL Proxy sidecar, local compose), parsed through an envBool coercion helper so a typo can't pick the insecure branch. Deployment envs must set DUOS_DB_SSL=false alongside the sqlproxy sidecar (terra-helmfile change). - Drop rolling sessions: with the PG store wired, rolling re-saved the session (SELECT + UPSERT) on every session-bearing request just to bump expire. Sessions get a fixed maxAge; Phase 2 adds a throttled sliding expiry (re-save only near expiry). - Compute session expiry in SQL (NOW() + maxAge) so writes use the same clock get filters with, and guard non-positive cookie.maxAge, not just null. Collapse the store methods through one run() helper for uniform error handling. This also makes the pgStore tests deterministic (no more Date.now() timing windows). - Document the two-switch rollout model in docs/plans/BFF_Overview.md: session infra is deployment config; user-facing cutover is the boolean BFF_ENABLED flag, per environment for deterministic testing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@fboulnois thanks for the review — the feature-flag concern was the important one, and it exposed a deeper problem: the plan defines 1fcde3b removes the boot-time flag check entirely. The server now registers its DB/cookie/session plugins whenever the deployment provides the database config ( Also in that commit, per the inline threads: secure-by-default Postgres TLS with an explicit coerced opt-out, |
… script pnpm forwards args placed after the script name into the script itself, so `pnpm run build --silent` appended --silent to the root build's trailing `pnpm --filter duos-server build`, which re-forwarded it to the server's `tsc` — an unknown compiler option that fails the build. Moving the flag before the script name keeps it as pnpm's own output-suppression flag. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The integration-tests workflow waits on http://localhost:3000, but `vite preview` inherited host local.dsde-dev.broadinstitute.org, which CI runners can't resolve (ENOTFOUND). Extend the existing isCI gate to host and open, matching how https is already handled. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
pnpm ci alone installs the npm package but not the Cypress binary (the cypress-io action runs with install: false). Mirror component-tests.yml: pnpm ci && pnpm cypress install. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts: # .github/workflows/integration-tests.yml
|



Addresses
https://broadworkbench.atlassian.net/browse/DT-3629
Summary
Phase 1 of the BFF migration: server-side, PostgreSQL-backed session infrastructure for the Fastify server. No user-visible changes — the infrastructure is inert until users are routed to it by the Phase 2 auth routes.
The DB/cookie/session plugins register only when the deployment provides the database configuration (
DUOS_DB_HOSTetc.), so the legacy client-side auth flow is completely unaffected until an environment is configured for the BFF — and every pod of a deployment behaves identically, with no network dependency at boot.Changes
Session infrastructure (
server/src/)index.ts: registers@fastify/postgres,@fastify/cookie, and@fastify/sessiononly whenDUOS_DB_HOSTis set. ValidatesDUOS_SESSION_SECRET(32+ chars) with an actionable error. SetstrustProxy: 1sorequest.protocolhonors the reverse proxy'sX-Forwarded-Proto— without it, secure cookies silently never persist behind TLS-terminating proxies.session/pgStore.ts: a thin typedSessionStore(get/set/destroy) against theuser_sessionstable. Expiry is computed in SQL from the same clockgetfilters with, so app/DB clock drift can't shorten or extend sessions. Norollingexpiry — sessions get a fixedmaxAge; Phase 2 adds throttled sliding expiry instead of a DB write per request.types/session.ts: module augmentation typing the session payload (tokens live only server-side, never sent to the browser).DUOS_DB_SSLmust be explicitly set tofalsefor loopback transports (Cloud SQL Proxy sidecar, local compose), and doing so logs a warning.Client config serving
config.ts:/config.jsonis now served by the server (via anonRequesthook, to avoid colliding with@fastify/vite's static route for the same file), withDUOS_API_URLoverriding the static file'sapiUrl. Deployed environments already carry the matching value via terra-helmfile; locally there's now one place to point at a different consent API.Local development
docker-compose.yamlis split into a shared base plus two overlays:docker-compose.override.yaml(standalone — bundled Postgres stands in for the consent DB, auto-merged by Compose) anddocker-compose.consent.yaml(shares a localconsentcompose stack's database, avoiding port collisions with itssqlproxyandapp)..env.exampletemplate; both compose modes andpnpm start:serverread.env.local.DEVNOTES.md: full walkthrough of both modes, including the consent-DB sharing setup.Deployment notes
DUOS_DB_SSL: "false"for the Cloud SQL Proxy sidecar). The X-Forwarded-Proto +duos_api_urlchanges already landed in https://github.com/broadinstitute/terra-helmfile/pull/6460.Have you read Terra's Contributing Guide lately? If not, do that first.