Clean up the persistence logic

AGENTS.md

@@ -14,15 +14,15 @@ This repository is a Go service that syncs calendar status and exposes availabil
 ## Architecture
 - `main.go` loads config, opens Pebble, builds the HTTP mux, and starts the sync loops after startup validation.
 - `internal/calendar` fetches iCal feeds, parses events, expands recurrences, and extracts feed timezone; `internal/calendar/handler.go` owns the status syncer.
-- `internal/availability` fetches and stores the raw availability feed, optionally caches GOV.UK bank holidays, computes free blocks, and serves `/api/availability`.
-- `internal/store` owns Pebble persistence for status, events, channels, sync tokens, and the availability and holiday snapshots.
+- `internal/availability` fetches and stores the raw availability feed, optionally caches GOV.UK bank holidays, computes free blocks, and serves precomputed `/api/availability` JSON.
+- `internal/store` owns Pebble persistence for fixed-key status, availability, holiday, and deployment projections.
 - `internal/target` defines the status target interface; `internal/github` implements the GitHub target.
 - `internal/server` wraps `http.Server` with graceful shutdown.
 - `internal/config` loads defaults, `config.yaml`, and environment variables.
 
 ## Sync Flow
-- Status sync is enabled only when `status.enabled` is true; it fetches `status.sources.ical.url`, stores events, computes the current active event, and syncs configured targets on `status.sources.ical.interval`.
-- Availability sync is enabled only when `availability.enabled` is true; it fetches `availability.sources.ical.url` on `availability.sources.ical.interval`, stores the raw ICS body plus timezone metadata in Pebble, exposes the API, and starts availability targets.
+- Status sync is enabled only when `status.enabled` is true; it fetches `status.sources.ical.url`, stores raw/current projections, computes the current active event in memory, and syncs configured targets on `status.sources.ical.interval`.
+- Availability sync is enabled only when `availability.enabled` is true; it fetches `availability.sources.ical.url` on `availability.sources.ical.interval`, stores the raw ICS body plus timezone metadata in Pebble, stores a precomputed API response projection, exposes the API, and starts availability targets.
 - If a feature-level `enabled` flag is false, both fetching and publishing for that feature must stay disabled regardless of nested configuration.
 - If `availability.suppressions.exclude_england_bank_holidays` is enabled, the app fetches GOV.UK bank holidays once at startup and stores the parsed holiday dates locally.
 - That startup seed is required for the availability endpoint when holiday exclusion is enabled, so startup should fail if the holiday feed cannot be read or parsed.
@@ -42,18 +42,20 @@ This repository is a Go service that syncs calendar status and exposes availabil
 - Status and availability fetch intervals are separate config values and default to `5m`.
 - Time blocks come from config and are checked in order.
 - `availability.suppressions.working_hours.start` defaults to `09:00` and `availability.suppressions.working_hours.end` defaults to `17:50`; together they are treated as weekday working time, not as an availability block.
-- Availability data is stored as the fetched raw ICS body plus metadata, not as live network state.
+- Status and availability raw data are stored as fetched ICS bodies plus metadata, not as live network state.
 - When enabled, bank holiday data is fetched from `https://www.gov.uk/bank-holidays.json` at startup and cached in Pebble.
 - The availability route is disabled when `availability.enabled` is false.
 - Empty env vars are treated as unset.
-- Pebble key design includes:
-  - `status` for the current status record
-  - `event:{eventID}` for stored calendar events
-  - `availability` for the latest raw availability snapshot
-  - `availability_dirty` for tracking if availability changed since last deploy (stores pending JSON)
-  - `availability_last_deployed` for tracking the availability entries JSON from the last successful deploy
-  - `availability_holidays` for the cached England bank holiday snapshot
-- The availability snapshot stores the raw ICS body, extracted timezone, and fetch timestamp so computation can happen locally without another network read.
+- Pebble key design is documented in `internal/store/README.md` and uses versioned fixed keys:
+  - `v1:status:current` for the current status record
+  - `v1:status:raw` for the latest raw status calendar snapshot
+  - `v1:availability:raw` for the latest raw availability snapshot
+  - `v1:availability:current` for the precomputed availability API response
+  - `v1:availability:deploy:dirty` for pending deploy JSON
+  - `v1:availability:deploy:last` for the last successfully deployed availability JSON
+  - `v1:availability:holidays:england` for the cached England bank holiday snapshot
+- Runtime Pebble reads should be O(1) direct key lookups; avoid iterator-backed reads and prefix scans on request or hot paths.
+- Raw calendar snapshots store the ICS body, extracted timezone, and fetch timestamp so computation can happen locally without another network read.
 - The holiday snapshot stores the raw GOV.UK JSON body, parsed dates, and fetch timestamp so availability can be computed offline.
 - Status is single-tenant.
 - Time zone handling should use the feed timezone when available, with UTC fallback.
@@ -69,7 +71,7 @@ This repository is a Go service that syncs calendar status and exposes availabil
 ## Important Implementation Notes
 - `internal/calendar/ical.go` is the central parser; keep recurrence expansion and date handling there.
 - Avoid reimplementing iCal line folding or date parsing by hand; keep tests around parser behavior instead.
-- Cancelled events are stored but do not count as active.
+- Cancelled status events do not count as active.
 - Availability computation checks today plus the next 9 days and returns the first free configured block per day.
 - On weekdays, blocks that overlap working hours are suppressed unless the day is a bank holiday and holiday exclusion is enabled.
 - If `availability.enabled` is true but availability config is incomplete, startup should fail fast.
@@ -78,7 +80,7 @@ This repository is a Go service that syncs calendar status and exposes availabil
 ## Cloudflare Pages auto-deploy
 
 - Feature: When `availability.enabled` is true, the application triggers a Cloudflare Pages deployment at a regular interval.
-- Change Tracking: To save build minutes, deployments are only triggered if the availability calendar has changed since the last successful deployment. This is managed by the `availability.Syncer`, which compares the current computed availability JSON with the `availability_last_deployed` JSON in Pebble. If a change is detected (including time-based changes), the new JSON is stored in `availability_dirty`, signaling the `Deployer` to trigger a build.
+- Change Tracking: To save build minutes, deployments are only triggered if the availability calendar has changed since the last successful deployment. This is managed by the `availability.Syncer`, which compares the current computed availability JSON with the `v1:availability:deploy:last` JSON in Pebble. If a change is detected (including time-based changes), the new JSON is stored in `v1:availability:deploy:dirty`, signaling the `Deployer` to trigger a build.
 - Config keys: `availability.enabled` (bool), `availability.targets.cloudflare_pages.interval` (Go duration string, e.g., "10m"), `availability.targets.cloudflare_pages.deploy_hook` (Pages Build Hook URL).
 - Scheduling: Deploys are scheduled to always fall offset by one minute after the hour. Example: with `availability.targets.cloudflare_pages.interval = 10m` deploys occur at HH:01, HH:11, HH:21, ... This reduces the chance of publishing stale calendar events that commonly start at round minutes (e.g., HH:20, HH:30).
 - Security: Do not commit `availability.targets.cloudflare_pages.deploy_hook` into source control; provide it via `config.yaml` or the `AVAILABILITY_TARGETS_CLOUDFLARE_PAGES_DEPLOY_HOOK` environment variable.

chart/Chart.yaml

@@ -4,5 +4,5 @@ description: A Helm chart for Kubernetes
 
 type: application
 
-version: 0.2.8
-appVersion: "v0.2.8"
+version: 0.2.9
+appVersion: "v0.2.9"

internal/availability/availability.go

@@ -88,14 +88,56 @@ func NewProvider(st *store.Store, blocks []Block, workingHours WorkingHours, exc
 
 // GetEntries returns current availability entries.
 func (p *Provider) GetEntries() ([]Entry, error) {
-	snap, ok, err := p.store.GetAvailabilitySnapshot()
+	data, err := p.GetEntriesJSON()
 	if err != nil {
-		return nil, fmt.Errorf("get availability snapshot: %w", err)
+		return nil, err
+	}
+	var entries []Entry
+	if err := json.Unmarshal(data, &entries); err != nil {
+		return nil, fmt.Errorf("decode current availability: %w", err)
+	}
+	return entries, nil
+}
+
+// GetEntriesJSON returns current availability entries serialized as JSON.
+func (p *Provider) GetEntriesJSON() ([]byte, error) {
+	data, ok, err := p.store.GetAvailabilityCurrent()
+	if err != nil {
+		return nil, fmt.Errorf("get current availability: %w", err)
 	}
 	if !ok {
 		return nil, ErrSnapshotNotFound
 	}
+	return data, nil
+}
 
+// RefreshCurrentFromStoredSnapshot recomputes and stores the current API
+// response from the latest persisted raw availability snapshot.
+func (p *Provider) RefreshCurrentFromStoredSnapshot() error {
+	snap, ok, err := p.store.GetAvailabilityRawSnapshot()
+	if err != nil {
+		return fmt.Errorf("get availability raw snapshot: %w", err)
+	}
+	if !ok {
+		if err := p.store.ClearAvailabilityCurrent(); err != nil {
+			return fmt.Errorf("clear current availability: %w", err)
+		}
+		return ErrSnapshotNotFound
+	}
+
+	data, err := p.ComputeEntriesJSONFromSnapshot(snap)
+	if err != nil {
+		return err
+	}
+	if err := p.store.SetAvailabilityCurrent(data); err != nil {
+		return fmt.Errorf("store current availability: %w", err)
+	}
+	return nil
+}
+
+// ComputeEntriesJSONFromSnapshot computes the current API response JSON from a
+// raw snapshot. This is intended for sync/startup paths, not request serving.
+func (p *Provider) ComputeEntriesJSONFromSnapshot(snap *store.CalendarSnapshot) ([]byte, error) {
 	opts := ComputeOptions{
 		WorkingHours: p.workingHours,
 		Now:          p.nowFunc(),
@@ -112,12 +154,7 @@ func (p *Provider) GetEntries() ([]Entry, error) {
 		opts.HolidayDates = holidaySnap.Dates
 	}
 
-	return Compute(snap.Body, snap.Timezone, p.blocks, opts)
-}
-
-// GetEntriesJSON returns current availability entries serialized as JSON.
-func (p *Provider) GetEntriesJSON() ([]byte, error) {
-	entries, err := p.GetEntries()
+	entries, err := Compute(snap.Body, snap.Timezone, p.blocks, opts)
 	if err != nil {
 		return nil, err
 	}

internal/availability/availability_test.go

@@ -3,6 +3,7 @@ package availability
 import (
 	"context"
 	"encoding/json"
+	"errors"
 	"net/http"
 	"net/http/httptest"
 	"testing"
@@ -241,15 +242,18 @@ END:VCALENDAR`
 func TestHandler_AuthorizationAndResponse(t *testing.T) {
 	st := newTestStore(t)
 	blocks := testBlocks(t)
-	if err := st.SetAvailabilitySnapshot(&store.AvailabilitySnapshot{
+	if err := st.SetAvailabilityRawSnapshot(&store.CalendarSnapshot{
 		Body:      baseCalendarBody(),
 		Timezone:  "Europe/London",
 		FetchedAt: time.Now().UTC(),
 	}); err != nil {
-		t.Fatalf("SetAvailabilitySnapshot: %v", err)
+		t.Fatalf("SetAvailabilityRawSnapshot: %v", err)
 	}
 
 	p := NewProvider(st, blocks, testWorkingHours(t), false)
+	if err := p.RefreshCurrentFromStoredSnapshot(); err != nil {
+		t.Fatalf("RefreshCurrentFromStoredSnapshot: %v", err)
+	}
 	h := NewHandler(p, "secret", zerolog.Nop())
 
 	t.Run("unauthorized", func(t *testing.T) {
@@ -287,43 +291,62 @@ func TestHandler_AuthorizationAndResponse(t *testing.T) {
 	})
 }
 
-func TestHandler_HolidaySnapshotRequiredWhenEnabled(t *testing.T) {
+func TestProvider_HolidaySnapshotRequiredWhenRefreshing(t *testing.T) {
 	st := newTestStore(t)
 	blocks := testBlocks(t)
-	if err := st.SetAvailabilitySnapshot(&store.AvailabilitySnapshot{
+	if err := st.SetAvailabilityRawSnapshot(&store.CalendarSnapshot{
 		Body:      baseCalendarBody(),
 		Timezone:  "Europe/London",
 		FetchedAt: time.Now().UTC(),
 	}); err != nil {
-		t.Fatalf("SetAvailabilitySnapshot: %v", err)
+		t.Fatalf("SetAvailabilityRawSnapshot: %v", err)
 	}
 
 	p := NewProvider(st, blocks, testWorkingHours(t), true)
-	h := NewHandler(p, "secret", zerolog.Nop())
 
+	if err := p.RefreshCurrentFromStoredSnapshot(); !errors.Is(err, ErrSnapshotNotFound) {
+		t.Fatalf("RefreshCurrentFromStoredSnapshot error: got %v, want %v", err, ErrSnapshotNotFound)
+	}
+}
+
+func TestHandler_ServesPrecomputedAvailability(t *testing.T) {
+	st := newTestStore(t)
+	current := []byte(`[{"day_of_week":"Monday","block":"Morning","date":"2026-04-06"}]`)
+	if err := st.SetAvailabilityCurrent(current); err != nil {
+		t.Fatalf("SetAvailabilityCurrent: %v", err)
+	}
+
+	p := NewProvider(st, testBlocks(t), testWorkingHours(t), false)
+	h := NewHandler(p, "secret", zerolog.Nop())
 	req := httptest.NewRequest(http.MethodGet, "/api/availability", nil)
 	req.Header.Set("Authorization", "secret")
 	rec := httptest.NewRecorder()
 
 	h.ServeHTTP(rec, req)
 
-	if rec.Code != http.StatusServiceUnavailable {
-		t.Fatalf("status: got %d, want %d", rec.Code, http.StatusServiceUnavailable)
+	if rec.Code != http.StatusOK {
+		t.Fatalf("status: got %d, want %d", rec.Code, http.StatusOK)
+	}
+	if got := rec.Body.String(); got != string(current) {
+		t.Fatalf("body: got %s, want %s", got, current)
 	}
 }
 
 func TestProvider_GetEntriesJSON(t *testing.T) {
 	st := newTestStore(t)
 	blocks := testBlocks(t)
-	if err := st.SetAvailabilitySnapshot(&store.AvailabilitySnapshot{
+	if err := st.SetAvailabilityRawSnapshot(&store.CalendarSnapshot{
 		Body:      baseCalendarBody(),
 		Timezone:  "Europe/London",
 		FetchedAt: time.Now().UTC(),
 	}); err != nil {
-		t.Fatalf("SetAvailabilitySnapshot: %v", err)
+		t.Fatalf("SetAvailabilityRawSnapshot: %v", err)
 	}
 
 	p := NewProvider(st, blocks, testWorkingHours(t), false)
+	if err := p.RefreshCurrentFromStoredSnapshot(); err != nil {
+		t.Fatalf("RefreshCurrentFromStoredSnapshot: %v", err)
+	}
 
 	data, err := p.GetEntriesJSON()
 	if err != nil {
@@ -339,7 +362,27 @@ func TestProvider_GetEntriesJSON(t *testing.T) {
 	}
 }
 
-func TestSyncer_StoresAvailabilitySnapshot(t *testing.T) {
+func TestProvider_RefreshCurrentFromStoredSnapshotClearsCurrentWhenMissing(t *testing.T) {
+	st := newTestStore(t)
+	if err := st.SetAvailabilityCurrent([]byte(`[{"date":"stale"}]`)); err != nil {
+		t.Fatalf("SetAvailabilityCurrent: %v", err)
+	}
+	p := NewProvider(st, testBlocks(t), testWorkingHours(t), false)
+
+	err := p.RefreshCurrentFromStoredSnapshot()
+	if !errors.Is(err, ErrSnapshotNotFound) {
+		t.Fatalf("RefreshCurrentFromStoredSnapshot error: got %v, want %v", err, ErrSnapshotNotFound)
+	}
+	_, ok, err := st.GetAvailabilityCurrent()
+	if err != nil {
+		t.Fatalf("GetAvailabilityCurrent: %v", err)
+	}
+	if ok {
+		t.Fatal("expected stale current availability to be cleared")
+	}
+}
+
+func TestSyncer_StoresAvailabilityRawSnapshot(t *testing.T) {
 	st := newTestStore(t)
 	blocks := testBlocks(t)
 	p := NewProvider(st, blocks, testWorkingHours(t), false)
@@ -352,9 +395,9 @@ func TestSyncer_StoresAvailabilitySnapshot(t *testing.T) {
 		t.Fatalf("syncOnce: %v", err)
 	}
 
-	snap, ok, err := st.GetAvailabilitySnapshot()
+	snap, ok, err := st.GetAvailabilityRawSnapshot()
 	if err != nil {
-		t.Fatalf("GetAvailabilitySnapshot: %v", err)
+		t.Fatalf("GetAvailabilityRawSnapshot: %v", err)
 	}
 	if !ok {
 		t.Fatal("expected stored availability snapshot")
@@ -364,17 +407,17 @@ func TestSyncer_StoresAvailabilitySnapshot(t *testing.T) {
 	}
 }
 
-func TestSyncer_PreservesAvailabilitySnapshotOnFetchError(t *testing.T) {
+func TestSyncer_PreservesAvailabilityRawSnapshotOnFetchError(t *testing.T) {
 	st := newTestStore(t)
 	blocks := testBlocks(t)
 	p := NewProvider(st, blocks, testWorkingHours(t), false)
-	seed := &store.AvailabilitySnapshot{
+	seed := &store.CalendarSnapshot{
 		Body:      "seed",
 		Timezone:  "UTC",
 		FetchedAt: time.Now().UTC(),
 	}
-	if err := st.SetAvailabilitySnapshot(seed); err != nil {
-		t.Fatalf("SetAvailabilitySnapshot: %v", err)
+	if err := st.SetAvailabilityRawSnapshot(seed); err != nil {
+		t.Fatalf("SetAvailabilityRawSnapshot: %v", err)
 	}
 
 	s := NewSyncer(st, p, &mockFetchClient{err: context.Canceled}, zerolog.Nop())
@@ -383,9 +426,9 @@ func TestSyncer_PreservesAvailabilitySnapshotOnFetchError(t *testing.T) {
 		t.Fatal("expected syncOnce to fail")
 	}
 
-	snap, ok, err := st.GetAvailabilitySnapshot()
+	snap, ok, err := st.GetAvailabilityRawSnapshot()
 	if err != nil {
-		t.Fatalf("GetAvailabilitySnapshot: %v", err)
+		t.Fatalf("GetAvailabilityRawSnapshot: %v", err)
 	}
 	if !ok {
 		t.Fatal("expected snapshot to remain after failure")

internal/availability/handler.go

@@ -1,14 +1,13 @@
 package availability
 
 import (
-	"encoding/json"
 	"errors"
 	"net/http"
 
 	"github.com/rs/zerolog"
 )
 
-// Handler serves availability entries from the stored snapshot.
+// Handler serves precomputed availability entries from the store.
 type Handler struct {
 	provider *Provider
 	apiKey   string
@@ -31,7 +30,7 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
-	entries, err := h.provider.GetEntries()
+	data, err := h.provider.GetEntriesJSON()
 	if err != nil {
 		if errors.Is(err, ErrSnapshotNotFound) {
 			http.Error(w, http.StatusText(http.StatusServiceUnavailable), http.StatusServiceUnavailable)
@@ -43,7 +42,7 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 	}
 
 	w.Header().Set("Content-Type", "application/json")
-	if err := json.NewEncoder(w).Encode(entries); err != nil {
-		h.logger.Error().Err(err).Msg("encode availability response")
+	if _, err := w.Write(data); err != nil {
+		h.logger.Error().Err(err).Msg("write availability response")
 	}
 }