Skip to content

feat(soup): forward-looking due-date buckets for grouped queries - #2

Closed
kopertop wants to merge 1 commit into
mainfrom
feature/gtd-workflow
Closed

feat(soup): forward-looking due-date buckets for grouped queries#2
kopertop wants to merge 1 commit into
mainfrom
feature/gtd-workflow

Conversation

@kopertop

Copy link
Copy Markdown
Member

First of five PRs toward an Asana-style My Tasks view. Backend only — nothing requests the new grouping field yet, so this is inert until PR 5. Full plan and PR sequence.

What it adds

GroupByField::DueDateBucket — group items into Today / Upcoming / Later / Backlog by reading a Date-typed property forward from the viewer's current day. Generic over the property rather than pinned to the system due date, so a custom date property groups the same way.

Why neither existing grouping mode can do this

  • GroupByField::Date buckets sort_ts backwards (Today / Yesterday / Last week) to answer "what did I touch recently".
  • GroupByField::Property extracts values by expanding values->'value' with jsonb_array_elements, but PropertyValue::Date is a JSON scalar — that path returns NULL for every row, so grouping tasks by due date today files all of them under "Not Set".

Hence a separate bucketer (models_grouping::gtd_buckets) and a separate scalar-reading lateral join.

Two decisions worth review

Boundaries are computed in Rust, not SQL. CURRENT_DATE is the database server's date. The test boundaries_follow_the_viewer_not_the_server shows the consequence: at 22:00 EDT the server is already on tomorrow's date, and a UTC-derived "end of today" files tomorrow-morning's work under Today. The field carries an IANA time_zone; an unrecognized value degrades to UTC rather than failing the request, since it originates from a client. horizon_days is a parameter too, so changing the Upcoming window later is a request change rather than a migration.

The comparison is textual, not cast. (values->>'value')::timestamptz cannot be indexed — text→temporal casts are only stable, not immutable — so PR 2's partial B-tree needs the raw text. Z-suffixed RFC 3339 compares lexicographically in chronological order, which makes that work.

The boundary literal deliberately omits the trailing Z, because '.' (0x2E) < 'Z' (0x5A):

"2026-08-13T00:00:00.000Z" < "2026-08-13T00:00:00Z"   -- true, and WRONG
"2026-08-13T00:00:00.000Z" < "2026-08-13T00:00:00"    -- false, correct

Against a bare prefix, any value inside that second sorts after it, so a midnight-exact due date falls on the later side — which is what "due tomorrow" means. text_comparison_agrees_with_instant_comparison asserts text and instant comparison agree across those cases, and a_z_suffixed_boundary_would_misbucket_fractional_values pins the trap itself so nobody "tidies" the format back.

Also here

  • group_select_expr_at / group_order_expr_at take an explicit now. The key expression is emitted three times per query (select, partition, filter); generating them from one pinned moment stops a request landing on midnight from partitioning against one set of boundaries and counting against another. The old signatures remain, delegating to Utc::now().
  • Both transports: REST ApiGroupByField::DueDateBucket and GraphQL DUE_DATE_BUCKET. static_assets/schema.graphql regenerated with cargo run -p complete_graph --bin graphql_schema; without_property_options now rejects timeZone/horizonDays on modes that read no property.
  • DST safety: zones that shift at midnight (Cuba) have no 00:00 on that date, so boundary resolution walks forward to the first resolvable hour instead of unwrapping.

Verification

Passing locally: models_grouping 19 tests + 2 doctests, complete_graph 27 SDL tests, cargo fmt, clippy clean on all three crates. Rebased onto current main (26e94d8) with no conflicts, and regenerating the schema against that base produces no diff.

Not run locally: the 8 unit tests and 1 Postgres test added to crates/soup/.../grouping/test.rs. crates/soup gates pub mod outbound behind a cargo feature, so cargo check --tests never compiles that module; with the feature on, the test binary needs a live database because pre-existing sqlx::query! calls in test code aren't in the offline .sqlx cache. No Docker runtime on my machine. CI is the first real build of those tests. My one new insert uses the unchecked sqlx::query() form on purpose, so this adds no .sqlx cache requirement.

The Postgres test is the one that matters: it executes the generated CASE against seeded rows — overdue, last instant of today, midnight with and without fractional seconds, past the horizon — and asserts Postgres and the Rust implementation reach the same bucket for each.

Deliberately out of scope

  • The entity_properties due-date index lands in PR 2, alongside the filter that needs it.
  • openapi.json and the generated TS client aren't regenerated: they're fetched from a running service (localhost:8086/api-doc/openapi.json, see apps/web/scripts/services.ts), and nothing consumes the new variant until PR 5.
  • Sorting within a bucket is still sort_ts DESC; due-date sort is PR 2.

First step toward an Asana-style "My Tasks" view: group tasks into Today /
Upcoming / Later / Backlog by reading a Date-typed property forward from the
viewer's current day. Backend only — no caller requests the new field yet.

Neither existing grouping mode can do this:

- `GroupByField::Date` buckets `sort_ts` *backwards* (Today / Yesterday /
  Last week) to answer "what did I touch recently".
- `GroupByField::Property` extracts values by expanding `values->'value'` with
  `jsonb_array_elements`, but `PropertyValue::Date` is a JSON *scalar*, so that
  path yields NULL and files every task under "Not Set".

So `DueDateBucket` gets its own scalar-reading lateral join, and its own
bucketer in `models_grouping::gtd_buckets`. Generic over the property rather
than pinned to the system due date, so a custom date property groups the same
way.

Two decisions worth review:

- **Boundaries are computed in Rust, not SQL.** `CURRENT_DATE` is the database
  server's date (UTC), which runs a day ahead of any viewer in the Americas
  during their evening and would file tomorrow morning's work under Today. The
  field carries an IANA `time_zone`; an unrecognized value degrades to UTC
  rather than failing the request. `horizon_days` is likewise a parameter, so
  changing the Upcoming window later is a request change, not a migration.

- **The comparison is textual, not cast.** `(values->>'value')::timestamptz` is
  only *stable*, not immutable, so Postgres rejects an index on it. Z-suffixed
  RFC 3339 compares lexicographically in chronological order, keeping a plain
  B-tree usable. The boundary literal deliberately omits the trailing `Z`:
  `'.' < 'Z'`, so a value with fractional seconds would otherwise sort before a
  Z-suffixed boundary it should sort after. Both properties are covered by
  tests, including one that executes the bucketing in Postgres and asserts it
  agrees with the Rust implementation.

The group-by expression is also now generated from a single pinned `now`
(`group_select_expr_at`), since it is emitted three times per query and a
request landing on midnight would otherwise partition against one set of
boundaries and count against another.

Both transports get the variant (REST `ApiGroupByField`, GraphQL
`GraphqlGroupByField`); `static_assets/schema.graphql` regenerated.
@kopertop

Copy link
Copy Markdown
Member Author

Superseded by macro-inc#5633 — this is intended as an upstream proposal, not a fork-only change.

@kopertop kopertop closed this Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant