Skip to content

feat(api): support number-tile color in the external dashboards API#2428

Open
alex-fedotyev wants to merge 3 commits into
mainfrom
alex/HDX-1360-ext-api-number-color
Open

feat(api): support number-tile color in the external dashboards API#2428
alex-fedotyev wants to merge 3 commits into
mainfrom
alex/HDX-1360-ext-api-number-color

Conversation

@alex-fedotyev

Copy link
Copy Markdown
Contributor

Number tiles let you set a static color and ordered conditional color rules in the editor, but the v2 external dashboards API carried neither field, so dashboards authored through POST /api/v2/dashboards could not set them. A raw SQL number tile that set a color in the UI also lost it on a GET then PUT round-trip, because the external conversion never emitted color.

This adds number-tile color to the external API so the authoring surfaces match.

Closes #2359 (that issue predates the conditional rules and only mentioned the static color; this covers both).
Builds on #2265 (static color) and #2386 (conditional rules).

What

  • Builder number chart config gains color (a palette token) and colorRules (ordered conditional rules, last match wins, max 10).
  • Raw SQL number chart config gains color. colorRules is intentionally left off the raw SQL variant: the editor shows the rules section for raw SQL number tiles, but its save path drops them, so persisted raw SQL configs never carry rules and the API should not invent capability the UI lacks.
  • Both directions of the internal/external conversion round-trip the new fields.
  • The OpenAPI spec adds a ChartPaletteToken enum and a ColorCondition schema, referenced from the number config. openapi.json is regenerated.

Why the palette-token handling looks the way it does

Colors persist as palette tokens, not hex, so they reflow across light and dark themes. The canonical tokens were renamed to hue names (chart-blue, chart-red, and so on) recently, and the older numeric tokens (chart-1 through chart-10) can still sit in stored dashboards. So:

  • On input, the API accepts hue-named tokens only (the same enum the schema validates). A legacy numeric token in a hand-written payload is rejected with a clear enum error.
  • On output, a stored legacy token is normalized to its hue name via the shared resolveChartPaletteToken helper, matching the app's fetch-time normalizer. Older dashboards keep returning a valid token and keep rendering.

I reused the existing ChartPaletteTokenSchema and ColorConditionSchema from common-utils rather than re-declaring them in the external schema (the file already imports leaf enums and the OnClick schemas the same way). That keeps the external surface from drifting from what the editor persists.

Test plan

  • yarn workspace @hyperdx/api tsc --noEmit
  • yarn workspace @hyperdx/api lint
  • yarn workspace @hyperdx/api lint:openapi
  • make dev-int FILE=external-api/__tests__/dashboards.test.ts (111 passed)
    • Round-trip: builder color plus colorRules covering every operator family, and raw SQL color.
    • Rejections: non-palette token, legacy numeric token on input, more than 10 rules, between without a two-number tuple, a numeric operator with a string value, an invalid regex, and an over-length label.
    • Backward compat: a tile with neither field; a stored legacy token normalized to its hue name on read, for both builder and raw SQL number tiles.
  • The two "round-trip all supported fields" tests (POST and PUT) now include color and colorRules on the number tile.

What is not in this PR

  • MCP hyperdx_save_dashboard parity for the same fields. That is the next PR in this series and reuses this conversion.
  • Customer docs for the number-tile color options (tracked in ClickHouse/clickhouse-docs#6298).

Number tiles can already set a static color and ordered conditional color
rules in the editor, but the v2 external API carried neither, so
dashboards-as-code could not author them. A raw SQL number tile that set a
color in the UI also lost it on a GET then PUT round-trip because the
external conversion never emitted it.

Add `color` and `colorRules` to the builder number chart config and `color`
to the raw SQL number chart config (the editor shows the rules section for
raw SQL number tiles but its save path drops them, so rules stay
builder-only). Round-trip both through the internal/external conversion and
regenerate the OpenAPI spec. Colors are validated as hue-named palette
tokens on input; tiles saved before the palette rename are normalized to
hue names on read so older dashboards keep working.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Jun 9, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 9e37785

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
@hyperdx/api Minor
@hyperdx/app Minor
@hyperdx/otel-collector Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions github-actions Bot added the review/tier-4 Critical — deep review + domain expert sign-off label Jun 9, 2026
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

🔴 Tier 4 — Critical

Touches auth, data models, config, tasks, OTel pipeline, ClickHouse, or CI/CD.

Why this tier:

  • Critical-path files (2):
    • packages/api/src/routers/external-api/v2/dashboards.ts
    • packages/api/src/routers/external-api/v2/utils/dashboards.ts
  • Cross-layer change: touches backend (packages/api) + shared utils (packages/common-utils)

Review process: Deep review from a domain expert. Synchronous walkthrough may be required.
SLA: Schedule synchronous review within 2 business days.

Stats
  • Production files changed: 5
  • Production lines changed: 530 (+ 376 in test files, excluded from tier calculation)
  • Branch: alex/HDX-1360-ext-api-number-color
  • Author: alex-fedotyev

To override this classification, remove the review/tier-4 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@vercel

vercel Bot commented Jun 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hyperdx-oss Ready Ready Preview, Comment Jun 9, 2026 3:29pm
hyperdx-storybook Ready Ready Preview, Comment Jun 9, 2026 3:29pm

Request Review

@greptile-apps

greptile-apps Bot commented Jun 9, 2026

Copy link
Copy Markdown

Greptile Summary

This PR extends the v2 external dashboards API to carry number-tile color fields (color palette token and colorRules ordered conditional rules) for builder tiles, and color only for raw SQL tiles, closing a round-trip data-loss bug where a tile's color was silently dropped on GET → PUT cycles.

  • NumberTileColorConditionSchema (numeric + equality operators only) is extracted in common-utils and reused by the API's Zod validation layer, so the external surface is structurally bound to what the editor persists and cannot accept contains/regex operators the number-tile UI never emits.
  • toExternalColorRules() normalizes legacy chart-1..10 tokens on read and drops any rule whose color cannot be resolved, returning undefined rather than [] when all rules are filtered out.
  • OpenAPI spec is regenerated with discriminated-union schemas for all three condition variants; integration tests cover all operator families, rejection cases, and legacy-token backward compatibility.

Confidence Score: 5/5

Safe to merge — additive fields only, validated input, normalized output, and full backward compatibility for stored legacy tokens.

The change is purely additive: new optional fields on existing request/response schemas, strict Zod validation on write, and defensive normalization on read. The conversion helpers handle all known edge cases (legacy tokens, unresolvable tokens, string-match/regex rules written via direct DB access), and the previously flagged asymmetric-fallback issue was resolved before this review. Integration tests cover round-trips for both tile variants, every operator family, all rejection conditions, and three backward-compat scenarios. No existing behavior is altered.

No files require special attention.

Important Files Changed

Filename Overview
packages/common-utils/src/types.ts Refactors ColorConditionSchema sub-schemas into named variables and adds the narrower NumberTileColorConditionSchema (numeric + equality operators only); both schemas share the same concrete sub-schema objects so they cannot drift from one another.
packages/api/src/utils/zod.ts Adds color and colorRules to the builder number chart schema, and color only to the raw SQL number schema; both import from common-utils so the external validation layer cannot diverge from the internal type.
packages/api/src/routers/external-api/v2/utils/dashboards.ts Adds toExternalColorRules() which resolves palette tokens, drops unresolvable ones and string-match/regex rules via re-validation, and returns undefined instead of [] when all rules are dropped; updates both conversion paths correctly.
packages/api/src/routers/external-api/tests/dashboards.test.ts Adds 376 lines of integration tests covering round-trips, all operator families, raw SQL static color, colorRules stripping for raw SQL, all rejection cases, and backward-compat legacy token normalization.
packages/api/openapi.json Regenerated spec adds ChartPaletteToken enum and condition schemas with discriminator; shapes match the Zod validation.

Reviews (3): Last reviewed commit: "fix(api): restrict number-tile colorRule..." | Re-trigger Greptile

Comment thread packages/api/src/routers/external-api/v2/utils/dashboards.ts Outdated
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

E2E Test Results

All tests passed • 198 passed • 3 skipped • 1257s

Status Count
✅ Passed 198
❌ Failed 0
⚠️ Flaky 4
⏭️ Skipped 3

Tests ran across 4 shards in parallel.

View full report →

…n read

The colorRules normalizer fell back to the raw stored color when a rule's
token could not be resolved, so a color written directly to Mongo that is
neither a hue token nor a legacy numeric token would pass through verbatim
in the GET response, outside the documented ChartPaletteToken enum. The
static color field already omits an unresolvable token. Drop a rule whose
color cannot be resolved instead, so the response stays within the enum.
Only reachable via a direct DB write, since the input schema validates rule
colors.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Deep Review

✅ No critical issues found.

This is a tightly scoped, well-tested additive change to the v2 external dashboards API. The OpenAPI enum and ColorCondition oneOf branches match the reused common-utils Zod schemas, and the read/write conversion round-trips correctly for valid hue tokens. No P0/P1 issues survived re-grading. The items below are hardening and coverage recommendations.

🟡 P2 -- recommended

  • packages/api/src/utils/zod.ts:335 -- The external number-tile colorRules schema reuses the full ColorConditionSchema, so it accepts regex and string-match operators that the number-tile editor never emits; a stored regex pattern (up to 500 chars, validated only by new RegExp() compiling) is later evaluated unbounded on the main thread at packages/app/src/utils.ts:661, letting a malicious authenticated payload freeze a same-team viewer's browser when the tile renders a string value.
    • Fix: Narrow the external number-tile colorRules schema to the numeric and equality operators the editor actually persists, or strip regex/string-match rules on the write path, mirroring the PR's own choice to not expose colorRules capability the UI lacks for raw SQL tiles.
    • adversarial, security
  • packages/api/src/routers/external-api/__tests__/dashboards.test.ts:4655 -- No test verifies what happens when colorRules is posted to a raw SQL number tile whose schema omits the field; the Zod object silently strips the unknown key, so the contract (drop vs reject) is unpinned and could regress.
    • Fix: Add a test posting colorRules on a raw SQL number tile asserting the field is absent from the 200 response (or returns 400 if rejection is intended).
    • testing
  • packages/api/src/routers/external-api/__tests__/dashboards.test.ts:4672 -- An invalid palette token inside a colorRules item (the per-rule color) is never exercised at the HTTP layer; rule-color validation is only covered by common-utils unit tests, not the external API path this PR adds.
    • Fix: Add negative tests posting a colorRules entry with color: 'red' and color: 'chart-1', each expecting 400.
    • testing
🔵 P3 nitpicks (6)
  • packages/api/src/routers/external-api/v2/dashboards.ts:823 -- The OpenAPI value schemas use type: number, which permits Infinity/NaN, but the Zod schema enforces .finite(), so a client generating payloads from the spec can send values the runtime rejects with a 400.
    • Fix: Note in the numeric/between/equality value descriptions that only finite numbers are accepted.
    • api-contract
  • packages/api/src/routers/external-api/v2/dashboards.ts:200 -- The ColorCondition oneOf has no discriminator, so code generators emit an untyped union even though the Zod runtime discriminates on operator.
    • Fix: Add discriminator: { propertyName: operator, mapping: {...} } to the ColorCondition schema.
    • api-contract
  • packages/api/src/routers/external-api/v2/utils/dashboards.ts:164 -- toExternalColorRules returns [] (not undefined) when a stored colorRules array is empty, emitting colorRules: [] in the response instead of omitting the field, inconsistent with how color is dropped when absent.
    • Fix: Normalize an empty result to undefined before returning.
    • maintainability, correctness
  • packages/api/src/routers/external-api/__tests__/dashboards.test.ts:4636 -- The operator-family round-trip test omits lt, lte, neq, startsWith, and endsWith.
    • Fix: Add one representative rule from each untested discriminated-union branch.
    • testing
  • packages/api/src/routers/external-api/__tests__/dashboards.test.ts:4674 -- The rejection tests assert only HTTP 400, not which field caused it, so an unrelated validation failure would let them pass falsely.
    • Fix: For the most distinct cases (bad token, bad regex, >10 rules), assert the error path/issue scope.
    • testing
  • packages/common-utils/src/types.ts:1083 -- The drift-prevention comment points editors only at packages/api/src/utils/zod.ts, not the hand-written OpenAPI JSDoc in dashboards.ts that also duplicates the schema and must be kept in sync.
    • Fix: Extend the comment to also name the @openapi JSDoc in packages/api/src/routers/external-api/v2/dashboards.ts.
    • maintainability

Reviewers (8): correctness, api-contract, testing, maintainability, kieran-typescript, adversarial, security, project-standards.

Testing gaps:

  • No HTTP-layer test that a raw SQL number tile's colorRules is stripped/rejected.
  • No negative test for an invalid color token nested in a colorRules item.
  • Operators lt/lte/neq/startsWith/endsWith are not exercised in the round-trip test.
  • No test for the all-rules-unresolvable read path (emits colorRules: []).

The external number-tile colorRules schema reused the full
ColorConditionSchema, which accepts the contains, startsWith, endsWith, and
regex operators that the number-tile editor never emits. A stored regex
pattern is compiled and evaluated at render time, so accepting one over the
API let an authenticated payload run an arbitrary pattern in a same-team
viewer's browser. Add NumberTileColorConditionSchema (numeric and equality
operators only) in common-utils and validate the external schema against
it, so the API surface matches what the editor can author. The MCP
dashboard tool can reuse the same schema.

Normalize an empty colorRules result to undefined on read so the field is
omitted rather than returned as an empty array, matching the static color
field. Regenerate the OpenAPI spec: NumberTileColorCondition gains an
operator discriminator and the unused string-match and regex schemas are
dropped.

Extend the integration coverage: all seven allowed operators round-trip,
unsupported operators and invalid per-rule tokens are rejected, colorRules
on a raw SQL number tile is stripped, and color plus colorRules round-trip
through PUT.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@alex-fedotyev

Copy link
Copy Markdown
Contributor Author

Pushed 9e37785 addressing the review feedback:

  • Restricted number-tile colorRules to the operators the editor actually emits (gt, gte, lt, lte, between, eq, neq) via a shared NumberTileColorConditionSchema in common-utils. The API no longer accepts the string-match or regex rules the UI can't produce, and the MCP dashboard tool can reuse the same schema so the restriction stays in one place.
  • colorRules posted on a raw SQL number tile is now covered by a test asserting it is stripped from the response (the config omits the field).
  • Added an HTTP-layer negative test for an invalid per-rule color token, and the rejection tests now assert the offending field rather than just the 400 status.
  • Empty colorRules now omits the field instead of returning [], matching the static color field.
  • OpenAPI: NumberTileColorCondition gains an operator discriminator and the now-unused string-match and regex schemas are dropped. Spec regenerated.

Integration suite: 116 passing, including the new operator-coverage, rejection, raw SQL strip, and PUT round-trip tests.

Hardening the render-time regex evaluation itself (so a future table-tile editor that exposes those operators is safe) is a separate follow-up; no current authoring path reaches it now that the API matches the editor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/tier-4 Critical — deep review + domain expert sign-off

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[HDX-1360] External API parity for number-tile color

1 participant