38 lines
1.8 KiB
Markdown
38 lines
1.8 KiB
Markdown
# 003 — Metric bucket derivation
|
|
|
|
## Summary
|
|
|
|
Apply metric declarations to log lines and populate `IndexKey::Bucket` /
|
|
`DataValue::BucketValue` entries, both retroactively (backfill already-ingested logs
|
|
when a metric is declared) and going forward (index each new log against all current
|
|
metric declarations as it's ingested).
|
|
|
|
## Scope
|
|
|
|
- On `MetricDeclaration` add: scan the existing `IngestedLog` range, run the metric's
|
|
regex against each log's text, and for each match compute the labeled `Bucket` key
|
|
(`metric_name` + `labels: BTreeMap<String, MetricPrimitive>`) and append
|
|
`(sequence, timestamp, value)` to that bucket's `BucketValue`.
|
|
- On new log ingest: run all currently-declared metrics' regexes against the line and
|
|
update buckets incrementally, same as backfill.
|
|
- Decide bucket cardinality behavior: a single log line can match multiple metrics: it
|
|
should update all matching buckets. Document how differing label sets from the same
|
|
metric become distinct `Bucket` keys (labels are part of the key, so this falls out
|
|
naturally from `BTreeMap` — just needs the insert-or-append logic).
|
|
- This is the piece that makes `MetricPrimitive`, `DataValue::BucketValue`, and the
|
|
`Bucket` index range (currently defined but never populated) actually load-bearing.
|
|
|
|
## Acceptance criteria
|
|
|
|
- Declaring a metric against a log stream that already has matching lines populates
|
|
buckets for those lines without needing new input.
|
|
- Ingesting a new log line after metrics are declared updates the relevant buckets
|
|
live.
|
|
- A log line matching zero metrics doesn't touch the bucket range.
|
|
- Bucket values preserve insertion order / are queryable by sequence and timestamp
|
|
range (needed by [004](004-metrics-api.md) and dashboards).
|
|
|
|
## Depends on
|
|
|
|
- [002 — Metric declarations](002-metric-declarations.md)
|