feat(downsampling): persist decimated blocks and interpolate rate-like functions over them

[u-veles-a]

Why

Prom++ keeps raw samples only for --storage.tsdb.retention.time; everything older is deleted. Long-range graphs (weeks, months) are therefore either empty or force an oversized raw retention, which costs memory and disk.

This branch introduces downsampling of persisted blocks: when a head is persisted, a decimated block is written next to the raw one, and wide-range queries are served from it instead of from raw data.

Decimation has an unavoidable side effect: when the decimation interval is comparable to the window of a range function, rate(m[2m]) may not catch two samples in the window and returns NaN — the data is there, just sparser than the window needs. The read path is therefore extended with an upsampler: for a narrow allow-list of functions the missing samples are restored by linear interpolation.

The feature is off by default: without downsampling in PROMPP_FEATURES none of the paths below activate, and neither behavior nor the on-disk block format changes.

What

Configuration

• New key PROMPP_FEATURES=downsampling=<duration> (pp-pkg/featuresflags). On startup the value is validated to be larger than scrape_interval, otherwise the process exits with code 2.
• The value is plumbed into pp_storage.Options.Downsampling, manager.Options.DownsamplingMS and the new pp_pkg_storage.AdapterOptions{RetentionMS, DownsamplingMS}.

Write path

• With downsampling enabled, pp/go/storage/block.Writer writes a second block for the same interval with Thanos.Downsample.Resolution = downsamplingMs (the decimation itself is done by the existing DownsamplingDecodeIterator in the C++ core).

Read path — source selection

• Manager.Querier/ChunkQuerier pick blocks by query width (skipBlock): with needDownsampling(maxt-mint) only downsampled blocks are selected, otherwise only raw ones. In the downsampling case the left border is extended back by one resolution, so that the first sample of the window has a predecessor to interpolate from.
• Head querier (pp/go/storage/querier): new activeHead field and WouldDownsample(); for allow-list functions the active head stops applying its own query-time downsampling and returns raw samples. ChunkQuerier received retentionMS/downsamplingMS for symmetry.

Read path — interpolation

• New package pp-pkg/blocks/upsampler: Querier/SeriesSet/Series/Iterator. Synthetic samples are injected only for {rate, increase, delta, deriv, irate, idelta}, with a step of Range/2, and only into gaps between Range/2 and resolutionMS*2 — a wider gap is a real outage and is left alone. Histogram samples pass through untouched.
• pp/go/storage/upsampler.NeedsUpsampling — the shared allow-list predicate, placed in pp/go because pp/go cannot import pp-pkg.
• upsampler.ResolutionQuerier — a transparent wrapper a storage uses to declare its nominal resolution upwards.
• pp-pkg/blocks/fanout — a fork of the vanilla storage/fanout.go: it collects the maximum resolution across all sources of the query and wraps the merged querier in upsampler.Querier once. Otherwise interpolation breaks at the storage boundary: the anchor pair "last sample of a block → first sample of the head" lives in two separate wrappers and no synthetic sample appears there. The fork needs its own noop querier, so storage/remote/read.go now returns fanout.NoopQuerier()/NoopSeriesSet().
• web/api/v1: queries reaching beyond retention get 2*downsampling added to LookbackDelta — without it an instant selection finds no sample on sparse data.

Observability and tooling

• prometheus_tsdb_blocks_loaded_by_duration now counts raw blocks only; decimated ones go to the new prometheus_tsdb_downsampled_blocks_loaded_by_duration with the same duration_minutes label. The name of the existing metric is unchanged.
• The loaded-blocks log line now includes resolution.
• prompptool blocks — new command listing the blocks on disk (ulid, interval, size, resolution), with -r/--human-readable.

Tests

Go

• pp/go/storage/upsampler — the predicate: nil hints, Range <= 0, every allow-list function and functions outside it.
• pp-pkg/blocks/upsampler — Iterator: no gap, gap, gap at the end of the slice, both bounds (step, resolution*2) and their independence from Range, histogram passthrough, Reset; Seek: before the first sample, exactly on a real one, into the middle of a gap, past the last sample, a repeated seek inside a partially yielded window, the seekAdvanceBase path. Plus Querier.shouldWrap, SeriesSet/Series, and benchmarks for the seek-heavy and the non-triggering paths.
• pp-pkg/blocks/fanout — a port of the upstream fanout tests (merge, primary/secondary error propagation) and the fork specifics: max resolution across sources, no wrapper when no resolution is declared, noop secondary skipped, queriers opened before a failure being closed. Separately the regression TestInterpolationAcrossStorageBoundary: a synthetic sample inside the gap between the blocks and the head (with per-storage wrapping there is none).
• pp-pkg/blocks/manager — block selection by query width, ResolutionQuerier returned on a downsampling range, split raw/downsampled metrics.
• pp-pkg/storage — adapter_upsampler_test (the WouldDownsample gate), adapter_promql_test (rate over a gap yields a value instead of NaN, plus the regression that behavior is unchanged when downsampling is off), adapter_bench_test.
• pp/go/storage/querier — downsamplingMS is zeroed only for allow-list functions, the result comes from the raw path, WouldDownsample(), updated optimize tests.

Manual check: run with PROMPP_FEATURES=downsampling=5m and a short --storage.tsdb.retention.time, wait for blocks to be persisted, and evaluate rate(metric[2m]) over a range beyond raw retention — previously empty/NaN, now a series with interpolated samples.