53 lines
2.5 KiB
Markdown
53 lines
2.5 KiB
Markdown
# 081 — PromQL dashboard queries rebuild full label→points maps from scratch every poll
|
|
|
|
## Summary
|
|
|
|
A dashboard panel backed by a PromQL-style query re-evaluates its
|
|
entire series (all labels, all points) from scratch on every polling
|
|
interval, with no reuse of the previous evaluation's work and no
|
|
incremental narrowing by the time range the panel actually needs —
|
|
same "narrow the range, don't rescan everything" principle as the
|
|
other read-path tickets, applied to the dashboard/metrics query path
|
|
this time.
|
|
|
|
## Where
|
|
|
|
- `evaluate_promql` and its helpers
|
|
(`src/engine/snapshot/query.rs:412-548`) build label→points maps for
|
|
the query's underlying metric(s) fresh on every call — there is no
|
|
reuse across repeated calls with the same query string and an
|
|
overlapping time window, which is exactly the pattern dashboard
|
|
polling produces (`webapp/src/lib/polling.ts` re-requests the same
|
|
panel query on an interval).
|
|
- This sits on top of `bucket_values` (ticket 078), so once that's
|
|
narrowed to a metric's own key range, `evaluate_promql` still
|
|
recomputes the full aggregation/label-grouping logic over whatever
|
|
`bucket_values` returns, every single call, even for the portion of
|
|
the time range it already computed on the previous poll.
|
|
|
|
## Scope
|
|
|
|
- Start by checking whether panels pass a time range/step down to
|
|
`evaluate_promql` at all — if they do, apply the same key-range
|
|
narrowing as ticket 077/078 first (bound the underlying
|
|
`bucket_values` range by the panel's requested window) before
|
|
considering caching, since that alone may remove most of the
|
|
redundant work.
|
|
- If polling the *same* query with an advancing time window is common
|
|
(sliding dashboard windows), investigate caching prior evaluation
|
|
results keyed by `(query string, label set)` and only computing the
|
|
delta for the newly-advanced portion of the window — this is more
|
|
involved (needs a cache invalidation story tied to new bucket
|
|
ingestion, similar to `sort_helper_indexes`'
|
|
`purge_stale_sort_helper_indexes`,
|
|
`src/engine/snapshot/sort.rs`) — do not build this speculatively
|
|
without first confirming via profiling that repeated PromQL
|
|
evaluation (not `bucket_values` scanning, addressed by ticket 078)
|
|
is the actual bottleneck for dashboards with multiple panels.
|
|
|
|
## Depends on
|
|
|
|
Should land after ticket 078 (metric bucket range narrowing), since
|
|
that changes what `evaluate_promql` is scanning underneath and may
|
|
reduce the size of this problem before any caching work is designed.
|