diff --git a/tickets/057-rate-line-chart-x-axis-collapses-to-sparse-occurrence-window.md b/tickets/057-rate-line-chart-x-axis-collapses-to-sparse-occurrence-window.md new file mode 100644 index 0000000..323f458 --- /dev/null +++ b/tickets/057-rate-line-chart-x-axis-collapses-to-sparse-occurrence-window.md @@ -0,0 +1,97 @@ +# 057 — Line chart for a rate() over sparse/bursty occurrences collapses its x-axis to the data's own tiny time span + +## Summary + +Reported symptom: the dashboard line chart for `sum by (code) (rate(KeyPressed[5m]))` +looks bad/unreadable against a real capture +(`ottyl --file .../sim-world.log`, a ~10-hour log where `KeyPressed` is a +user-declared metric matching `msg="key_event"` lines, labeled by +`code`/`kind`/`msg` — see the profile's metric declaration in +`~/Library/Application Support/ottyl/settings.json`). In the sample log +all 20 `key_event` lines (16 `Left`, 3 `Up`, 1 `Down`, all `kind="Press"`) +occur within the first ~10 seconds of ingestion; there are zero further +key presses across the remaining ~9h55m of the log. + +Two compounding causes, both reproducible from this shape of data: + +1. **`rate()` never emits a trailing zero once occurrences stop.** + `crate::rate::windowed_rate_series` (`src/rate.rs:40-45`) evaluates + `rate_at` only *at each existing point's own timestamp* — there is no + synthetic point emitted at the query's `end_ms` (or at "now") once the + trailing window has aged out. `evaluate_promql`'s `Sum`/`Avg` branch + (`src/engine.rs:1621-1648`) calls this per group then merges with + `merge_series_locf` (`src/engine.rs:1508-1529`), which sweeps only the + *union of each sub-series' own timestamps* — again nothing beyond the + last real occurrence. So a series for `code="Left"` literally ends at + the timestamp of the last `Left` press; there is no data point saying + "rate is now 0" for the following ~9h55m. +2. **The frontend's line chart x-axis auto-scales to whatever timestamps + happen to be in the returned series, not to the queried time range.** + `PanelChart.tsx`'s `toLineData` (`webapp/src/components/PanelChart.tsx:60-77`) + builds `rows` only from timestamps present in `series[].points`; the + `` (`PanelChart.tsx:105-110`) has no + explicit `domain`, so Recharts defaults to `['auto','auto']` computed + from `rows` alone. Combined with (1), a panel queried over "last 6 + hours" (or any range spanning the whole log) that only has data in the + first 10 seconds silently renders an x-axis spanning just those 10 + seconds — not the selected range — with no visual indication the axis + has been narrowed or that activity stopped. The result is a cramped, + jagged little chart (peak rate ≈ 16 presses / 300s ≈ 0.053/s, so also + visually flat/near-zero) that looks broken rather than showing "a + burst, then nothing." + +The backend range filter (`start_ms`/`end_ms` in `evaluate_promql`, +`src/engine.rs:1565-1584`) is applied correctly to which raw occurrences +are considered — this is not a filtering bug, it's a rendering/shape +problem in what gets returned and how it's charted. + +## Scope + +- Decide and implement a way for the charted line to reflect that the + rate has decayed to (and stayed at) zero, rather than simply stopping. + Options to weigh (pick one, document the choice in the PR): + - Emit a synthetic trailing point at `end_ms` (or "now" for a live + range) per group whenever the last real point's window has fully + aged out and its rate would read 0 — cheapest fix, keeps all + resampling logic where it already lives in `evaluate_promql`. + - Resample `rate()`/`sum(rate())`/`avg(rate())` series onto a regular + step grid across `[start_ms, end_ms]` (à la Prometheus's `step`) + instead of only at occurrence timestamps — bigger change, but fixes + the same problem for gauges/counters with irregular sampling more + generally; note `merge_series_windowed` (`src/engine.rs:1484-1503`) + already buckets on a fixed `MERGE_WINDOW_MS` grid for the + non-rate `sum()`/`avg()` path, which is closer to this shape already. +- Whichever approach is chosen, make `PanelChart.tsx`'s line chart honor + the panel's actual queried range for the x-axis domain (pass + `range.startMs`/`range.endMs` down from `Dashboards.tsx`'s + `PanelPreview` into `PanelChart` and set ``, or equivalent) so a chart with sparse + data doesn't silently narrow the visible window — this is worth doing + regardless of which backend option above is chosen, since it's the + more directly misleading half of the bug. +- Add a fixture reproducing this shape (a metric with occurrences + clustered in a short burst near the start of a much longer ingested + window) and a test on `evaluate_promql`/`windowed_rate_series` (or + wherever the chosen fix lives) asserting the returned series covers + the full requested range, not just the burst. +- Out of scope: changing how `KeyPressed`-style occurrence metrics are + declared, or adding a dedicated "gauge decay" concept — this is about + the query range/x-axis mismatch specifically. + +## Acceptance criteria + +- Querying `sum by (code) (rate(KeyPressed[5m]))` (or an equivalent + fixture) over a time range much wider than the span of actual + occurrences returns a series (or is charted) that visibly reflects + rate-zero for the remainder of the range, rather than a line that + simply stops. +- `PanelChart`'s line chart x-axis reflects the panel's selected/queried + time range, not just the min/max timestamp actually present in the + returned series — verified by a case where returned data covers only + a fraction of the selected range. +- Existing `evaluate_promql`/`rate.rs`/`PanelChart` tests for + regularly-sampled or continuously-occurring metrics are unaffected. + +## Depends on + +None.