Metrics rethink: counters as the primitive, histogram()/gauge() derived (ticket 022)
Every metric declaration is now explicitly documented and modeled as
a counter: a match is an occurrence/increment, optionally labeled.
The bucket representation (BucketValue { values: Vec<(Sequence,
Timestamp, MetricPrimitive)> }) already was this; what it lacked was
any way to capture a *value* distinct from the occurrence-count-of-1
marker. Adds RegexToMetricPrimitiveMatcher::role (ParamRole::Label,
the default, or ParamRole::Value — at most one per metric, enforced
at declare time) so a metric can designate one param's captured
number as its per-event value instead of a label.
promql.rs/engine.rs add two new PromQL-evaluable forms built on top
of that value, both consumed as regular PromqlSeriesDto series like
sum()/avg()/rate():
- histogram(metric_name[width]): buckets matching events' values into
fixed-width buckets, one series per bucket labeled `le`.
- gauge(metric_name): the most-recent value per label combination in
the queried range.
Existing raw counter queries (metrics with only Label-role params,
sum/avg/rate) are unchanged. Profile files saved before this ticket
still load (role defaults to Label via #[serde(default)]). Metrics.tsx
gains a label/value role picker per param, and the Docs panel explains
the model and the two derived query forms.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
1d0f324c43
commit
0c57ed2953
@@ -9,6 +9,23 @@ pub struct RegexToMetricPrimitiveMatcher {
|
||||
pub param_name: String,
|
||||
pub capture_group: String,
|
||||
pub kind: MetricPrimitiveKind,
|
||||
/// Ticket 022: whether this param contributes to the bucket's `labels`
|
||||
/// (the default) or is the bucket's per-event captured `value` (at most
|
||||
/// one `Value`-role param per metric — `declare_metric` rejects more
|
||||
/// than one). `#[serde(default)]` so profile files saved before this
|
||||
/// field existed still load, with every param defaulting to `Label` —
|
||||
/// i.e. unchanged behavior (the fixed occurrence-count-of-1 value).
|
||||
#[serde(default)]
|
||||
pub role: ParamRole,
|
||||
}
|
||||
|
||||
/// See [`RegexToMetricPrimitiveMatcher::role`].
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)]
|
||||
#[serde(rename_all = "lowercase")]
|
||||
pub enum ParamRole {
|
||||
#[default]
|
||||
Label,
|
||||
Value,
|
||||
}
|
||||
|
||||
/// `rename_all = "lowercase"` matches every other `kind`-style enum exposed
|
||||
|
||||
+229
-15
@@ -1,4 +1,4 @@
|
||||
use crate::data_value::{DataValue, MetricPrimitiveKind, RegexToMetricPrimitiveMatcher};
|
||||
use crate::data_value::{DataValue, MetricPrimitiveKind, ParamRole, RegexToMetricPrimitiveMatcher};
|
||||
use crate::dto::{
|
||||
BucketValueDto, LogEntryDto, LogsParams, MetricDeclarationDto, ParamDeclarationDto,
|
||||
PromqlSeriesDto, StatsDto, TimelineBucketDto, TimelineParams,
|
||||
@@ -231,6 +231,13 @@ fn declare_metric(
|
||||
return Err(errors.join("; "));
|
||||
}
|
||||
|
||||
let value_params = request.params.iter().filter(|matcher| matcher.role == ParamRole::Value).count();
|
||||
if value_params > 1 {
|
||||
return Err(format!(
|
||||
"a metric may declare at most one Value-role param, found {value_params}"
|
||||
));
|
||||
}
|
||||
|
||||
backfill_metric(index, &request.name, &re, &request.params);
|
||||
index.insert(
|
||||
IndexKey::MetricDeclaration { name: request.name },
|
||||
@@ -263,17 +270,22 @@ fn parse_metric_primitive(raw: &str, kind: MetricPrimitiveKind) -> Option<Metric
|
||||
}
|
||||
|
||||
/// Runs one metric's regex against a single log line and, on a match, appends a
|
||||
/// bucket value entry.
|
||||
/// bucket value entry: this is the "counter increment" (ticket 022) — a match is
|
||||
/// an occurrence at a point in time, optionally labeled.
|
||||
///
|
||||
/// Design decision: `Bucket.labels` is the only structured place a `Bucket` key has
|
||||
/// for a metric's declared params, so v1 folds ALL of a metric's params into
|
||||
/// `labels` rather than splitting them into "label" vs "value" params. The stored
|
||||
/// value is therefore a fixed presence marker (`MetricPrimitive::Number(Number::Int(1))`)
|
||||
/// — a bucket's `values` is the timeseries of *when that label combination occurred*
|
||||
/// (a count/occurrence series), not a record of the captured values themselves. If a
|
||||
/// param's capture group doesn't match, or its captured text fails to parse as its
|
||||
/// declared `kind`, the whole match is skipped for this line rather than storing a
|
||||
/// partially-labeled bucket.
|
||||
/// `Bucket.labels` holds every `Label`-role param (the default role — see
|
||||
/// [`ParamRole`]); at most one param may instead be `Value`-role (enforced at
|
||||
/// declare time), and its parsed primitive becomes the pushed event's value.
|
||||
/// With no `Value`-role param, the pushed value is a fixed presence marker
|
||||
/// (`MetricPrimitive::Number(Number::Int(1))`), matching the original v1
|
||||
/// behavior where a bucket's `values` was purely the timeseries of *when that
|
||||
/// label combination occurred* — histogram()/gauge() derived queries
|
||||
/// (`crate::promql::AggOp`) read this same `value`, so a `Value`-role param is
|
||||
/// what makes those meaningful (bucketing/reading an occurrence-count-of-1
|
||||
/// series degenerates to "everything in one bucket"/"always 1"). If a param's
|
||||
/// capture group doesn't match, or its captured text fails to parse as its
|
||||
/// declared `kind`, the whole match is skipped for this line rather than
|
||||
/// storing a partially-labeled bucket.
|
||||
fn apply_metric_to_log(
|
||||
index: &mut BTreeMap<IndexKey, DataValue>,
|
||||
metric_name: &str,
|
||||
@@ -288,14 +300,22 @@ fn apply_metric_to_log(
|
||||
};
|
||||
|
||||
let mut labels = BTreeMap::new();
|
||||
let mut value = None;
|
||||
for matcher in params {
|
||||
let Some(raw) = captures.name(&matcher.capture_group) else {
|
||||
return;
|
||||
};
|
||||
let Some(value) = parse_metric_primitive(raw.as_str(), matcher.kind) else {
|
||||
let Some(parsed) = parse_metric_primitive(raw.as_str(), matcher.kind) else {
|
||||
return;
|
||||
};
|
||||
labels.insert(matcher.param_name.clone(), value);
|
||||
match matcher.role {
|
||||
ParamRole::Label => {
|
||||
labels.insert(matcher.param_name.clone(), parsed);
|
||||
}
|
||||
ParamRole::Value => {
|
||||
value = Some(parsed);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let key = IndexKey::Bucket { metric_name: metric_name.to_string(), labels };
|
||||
@@ -303,7 +323,7 @@ fn apply_metric_to_log(
|
||||
.entry(key)
|
||||
.or_insert_with(|| DataValue::BucketValue { values: Vec::new() });
|
||||
if let DataValue::BucketValue { values } = entry {
|
||||
values.push((sequence, timestamp, MetricPrimitive::Number(Number::Int(1))));
|
||||
values.push((sequence, timestamp, value.unwrap_or(MetricPrimitive::Number(Number::Int(1)))));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -980,6 +1000,51 @@ fn evaluate_promql(
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
Some(AggOp::Gauge) => {
|
||||
// Ticket 022: collapse each label combination's counter events
|
||||
// down to their most-recent value in the queried time range —
|
||||
// "value at time T" derived from the last matching event, rather
|
||||
// than a distinct stored gauge kind.
|
||||
series_points
|
||||
.into_iter()
|
||||
.filter_map(|(labels, points)| {
|
||||
let last = *points.last()?;
|
||||
Some(PromqlSeriesDto { labels, points: vec![last] })
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
Some(AggOp::Histogram) => {
|
||||
// Validated by `promql::parse_query`: `histogram()` never parses
|
||||
// without one.
|
||||
let bucket_width =
|
||||
parsed.bucket_width.expect("histogram() always carries a bucket width");
|
||||
let last_ts = series_points
|
||||
.values()
|
||||
.flat_map(|points| points.iter().map(|(ts, _)| *ts))
|
||||
.max()
|
||||
.unwrap_or(0);
|
||||
// Ticket 022: bucket every matching counter event's value
|
||||
// (across all label combinations — histogram collapses the
|
||||
// label dimension the same way sum()/avg() do) into fixed-width
|
||||
// buckets, one output series per bucket labeled `le` (Prometheus's
|
||||
// "less-than-or-equal" convention) holding a single
|
||||
// `(last_ts, count)` point.
|
||||
let mut counts: BTreeMap<i64, u64> = BTreeMap::new();
|
||||
for points in series_points.values() {
|
||||
for (_, value) in points {
|
||||
let bucket_index = (value / bucket_width).floor() as i64;
|
||||
*counts.entry(bucket_index).or_insert(0) += 1;
|
||||
}
|
||||
}
|
||||
counts
|
||||
.into_iter()
|
||||
.map(|(bucket_index, count)| {
|
||||
let upper_bound = (bucket_index + 1) as f64 * bucket_width;
|
||||
let labels = BTreeMap::from([("le".to_string(), upper_bound.to_string())]);
|
||||
PromqlSeriesDto { labels, points: vec![(last_ts, count as f64)] }
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1103,16 +1168,32 @@ mod tests {
|
||||
name: &str,
|
||||
pattern: &str,
|
||||
params: Vec<(&str, &str, MetricPrimitiveKind)>,
|
||||
) -> DeclareMetricRequest {
|
||||
metric_request_with_roles(
|
||||
name,
|
||||
pattern,
|
||||
params
|
||||
.into_iter()
|
||||
.map(|(param_name, capture_group, kind)| (param_name, capture_group, kind, ParamRole::Label))
|
||||
.collect(),
|
||||
)
|
||||
}
|
||||
|
||||
fn metric_request_with_roles(
|
||||
name: &str,
|
||||
pattern: &str,
|
||||
params: Vec<(&str, &str, MetricPrimitiveKind, ParamRole)>,
|
||||
) -> DeclareMetricRequest {
|
||||
DeclareMetricRequest {
|
||||
name: name.to_string(),
|
||||
message_match_regex: pattern.to_string(),
|
||||
params: params
|
||||
.into_iter()
|
||||
.map(|(param_name, capture_group, kind)| RegexToMetricPrimitiveMatcher {
|
||||
.map(|(param_name, capture_group, kind, role)| RegexToMetricPrimitiveMatcher {
|
||||
param_name: param_name.to_string(),
|
||||
capture_group: capture_group.to_string(),
|
||||
kind,
|
||||
role,
|
||||
})
|
||||
.collect(),
|
||||
}
|
||||
@@ -1439,6 +1520,139 @@ mod tests {
|
||||
assert_eq!(series[0].points, vec![(5, 1.0)]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn declares_metric_with_value_role_param() {
|
||||
let mut index = BTreeMap::new();
|
||||
let result = declare_metric(
|
||||
&mut index,
|
||||
metric_request_with_roles(
|
||||
"durations",
|
||||
r"request path=(?P<path>\S+) duration=(?P<duration>\d+)",
|
||||
vec![
|
||||
("path", "path", MetricPrimitiveKind::String, ParamRole::Label),
|
||||
("duration", "duration", MetricPrimitiveKind::Number, ParamRole::Value),
|
||||
],
|
||||
),
|
||||
);
|
||||
assert!(result.is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_metric_with_multiple_value_role_params() {
|
||||
let mut index = BTreeMap::new();
|
||||
let result = declare_metric(
|
||||
&mut index,
|
||||
metric_request_with_roles(
|
||||
"bad",
|
||||
r"a=(?P<a>\d+) b=(?P<b>\d+)",
|
||||
vec![
|
||||
("a", "a", MetricPrimitiveKind::Number, ParamRole::Value),
|
||||
("b", "b", MetricPrimitiveKind::Number, ParamRole::Value),
|
||||
],
|
||||
),
|
||||
);
|
||||
assert!(result.is_err());
|
||||
assert!(result.unwrap_err().contains("at most one Value-role param"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn value_role_param_becomes_the_bucket_value_not_a_label() {
|
||||
let mut index = BTreeMap::new();
|
||||
declare_metric(
|
||||
&mut index,
|
||||
metric_request_with_roles(
|
||||
"durations",
|
||||
r"request path=(?P<path>\S+) duration=(?P<duration>\d+)",
|
||||
vec![
|
||||
("path", "path", MetricPrimitiveKind::String, ParamRole::Label),
|
||||
("duration", "duration", MetricPrimitiveKind::Number, ParamRole::Value),
|
||||
],
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
ingest_log(&mut index, 0, "request path=/a duration=5");
|
||||
|
||||
let values: Vec<_> = bucket_values(&index, "durations", None).collect();
|
||||
assert_eq!(values.len(), 1);
|
||||
let (labels, (_, _, value)) = &values[0];
|
||||
assert!(!labels.contains_key("duration"));
|
||||
assert_eq!(labels.get("path").map(|v| v.to_string()), Some("/a".to_string()));
|
||||
assert_eq!(*value, MetricPrimitive::Number(Number::Int(5)));
|
||||
}
|
||||
|
||||
fn declare_durations_metric(index: &mut BTreeMap<IndexKey, DataValue>) {
|
||||
declare_metric(
|
||||
index,
|
||||
metric_request_with_roles(
|
||||
"durations",
|
||||
r"request path=(?P<path>\S+) duration=(?P<duration>\d+)",
|
||||
vec![
|
||||
("path", "path", MetricPrimitiveKind::String, ParamRole::Label),
|
||||
("duration", "duration", MetricPrimitiveKind::Number, ParamRole::Value),
|
||||
],
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn gauge_returns_most_recent_value_per_label_combination() {
|
||||
let mut index = BTreeMap::new();
|
||||
declare_durations_metric(&mut index);
|
||||
ingest_log(&mut index, 0, "request path=/a duration=5");
|
||||
ingest_log(&mut index, 1, "request path=/a duration=9");
|
||||
ingest_log(&mut index, 2, "request path=/b duration=42");
|
||||
|
||||
let parsed = promql::parse_query("gauge(durations)").unwrap();
|
||||
let mut series = evaluate_promql(&index, &parsed, None, None);
|
||||
series.sort_by(|a, b| a.labels.get("path").cmp(&b.labels.get("path")));
|
||||
|
||||
assert_eq!(series.len(), 2);
|
||||
assert_eq!(series[0].labels.get("path").map(String::as_str), Some("/a"));
|
||||
assert_eq!(series[0].points, vec![(1, 9.0)]);
|
||||
assert_eq!(series[1].labels.get("path").map(String::as_str), Some("/b"));
|
||||
assert_eq!(series[1].points, vec![(2, 42.0)]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn gauge_respects_time_range_bounds() {
|
||||
let mut index = BTreeMap::new();
|
||||
declare_durations_metric(&mut index);
|
||||
ingest_log(&mut index, 0, "request path=/a duration=5");
|
||||
ingest_log(&mut index, 5, "request path=/a duration=9");
|
||||
ingest_log(&mut index, 10, "request path=/a duration=42");
|
||||
|
||||
let parsed = promql::parse_query("gauge(durations)").unwrap();
|
||||
let series = evaluate_promql(&index, &parsed, None, Some(9));
|
||||
|
||||
assert_eq!(series.len(), 1);
|
||||
assert_eq!(series[0].points, vec![(5, 9.0)]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn histogram_buckets_captured_values_by_fixed_width() {
|
||||
let mut index = BTreeMap::new();
|
||||
declare_durations_metric(&mut index);
|
||||
for (seq, duration) in [(0, 2), (1, 4), (2, 12), (3, 15), (4, 21)] {
|
||||
ingest_log(&mut index, seq, &format!("request path=/a duration={duration}"));
|
||||
}
|
||||
|
||||
let parsed = promql::parse_query("histogram(durations[10])").unwrap();
|
||||
let mut series = evaluate_promql(&index, &parsed, None, None);
|
||||
series.sort_by(|a, b| a.labels.get("le").cmp(&b.labels.get("le")));
|
||||
|
||||
// durations 2,4 -> bucket [0,10) le=10 (count 2); 12,15 -> [10,20) le=20 (count 2);
|
||||
// 21 -> [20,30) le=30 (count 1).
|
||||
assert_eq!(series.len(), 3);
|
||||
assert_eq!(series[0].labels.get("le").map(String::as_str), Some("10"));
|
||||
assert_eq!(series[0].points[0].1, 2.0);
|
||||
assert_eq!(series[1].labels.get("le").map(String::as_str), Some("20"));
|
||||
assert_eq!(series[1].points[0].1, 2.0);
|
||||
assert_eq!(series[2].labels.get("le").map(String::as_str), Some("30"));
|
||||
assert_eq!(series[2].points[0].1, 1.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn handle_query_promql_rejects_malformed_query() {
|
||||
let index = BTreeMap::new();
|
||||
|
||||
+8
-1
@@ -40,7 +40,7 @@ use std::time::{SystemTime, UNIX_EPOCH};
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::data_value::{MetricPrimitiveKind, RegexToMetricPrimitiveMatcher};
|
||||
use crate::data_value::{MetricPrimitiveKind, ParamRole, RegexToMetricPrimitiveMatcher};
|
||||
use crate::engine::{DeclareMetricRequest, DeclareParamRequest, EngineHandle, EngineMessage};
|
||||
use crate::settings::Settings;
|
||||
use tokio::sync::oneshot;
|
||||
@@ -119,6 +119,10 @@ pub struct MetricParamRecord {
|
||||
pub param_name: String,
|
||||
pub capture_group: String,
|
||||
pub kind: ProfileMetricPrimitiveKind,
|
||||
/// See `ParamRole`. `#[serde(default)]` so profiles saved before ticket
|
||||
/// 022 still load, with every param defaulting to `Label`.
|
||||
#[serde(default)]
|
||||
pub role: ParamRole,
|
||||
}
|
||||
|
||||
/// A stored metric declaration, matching `DeclareMetricRequest`'s shape (ticket 002).
|
||||
@@ -141,6 +145,7 @@ impl From<&DeclareMetricRequest> for MetricDeclarationRecord {
|
||||
param_name: matcher.param_name.clone(),
|
||||
capture_group: matcher.capture_group.clone(),
|
||||
kind: matcher.kind.into(),
|
||||
role: matcher.role,
|
||||
})
|
||||
.collect(),
|
||||
}
|
||||
@@ -159,6 +164,7 @@ impl From<&MetricDeclarationRecord> for DeclareMetricRequest {
|
||||
param_name: param.param_name.clone(),
|
||||
capture_group: param.capture_group.clone(),
|
||||
kind: param.kind.into(),
|
||||
role: param.role,
|
||||
})
|
||||
.collect(),
|
||||
}
|
||||
@@ -708,6 +714,7 @@ mod tests {
|
||||
param_name: param_name.to_string(),
|
||||
capture_group: capture_group.to_string(),
|
||||
kind: MetricPrimitiveKind::Number,
|
||||
role: ParamRole::Label,
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
+91
-13
@@ -14,21 +14,34 @@
|
||||
//! ```text
|
||||
//! query := aggregation | vector_selector
|
||||
//! aggregation := agg_op "(" vector_selector ")"
|
||||
//! agg_op := "sum" | "avg" | "rate"
|
||||
//! agg_op := "sum" | "avg" | "rate" | "histogram" | "gauge"
|
||||
//! vector_selector := metric_name label_matchers? range_selector?
|
||||
//! metric_name := [a-zA-Z_][a-zA-Z0-9_]*
|
||||
//! label_matchers := "{" label_matcher ("," label_matcher)* "}"
|
||||
//! label_matcher := label_name "=" quoted_string
|
||||
//! label_name := [a-zA-Z_][a-zA-Z0-9_]*
|
||||
//! quoted_string := '"' (any char except '"' or '\')* '"'
|
||||
//! range_selector := "[" digits duration_unit "]"
|
||||
//! range_selector := "[" digits duration_unit? "]"
|
||||
//! duration_unit := "s" | "m" | "h" | "d"
|
||||
//! ```
|
||||
//!
|
||||
//! Only exact-match label matchers (`=`) are supported — PromQL's `!=`, `=~`, `!~`
|
||||
//! are out of scope for this ticket's required subset. `rate()` requires its inner
|
||||
//! vector selector to carry a range selector (matching real PromQL), and is rejected
|
||||
//! at parse time otherwise.
|
||||
//! vector selector to carry a range selector with a duration unit (matching real
|
||||
//! PromQL), and is rejected at parse time otherwise.
|
||||
//!
|
||||
//! Ticket 022 (metrics-as-counters rethink) adds two derived forms on top of the
|
||||
//! same grammar, both consumed as regular `PromqlSeriesDto` series like every
|
||||
//! other query shape:
|
||||
//! - `histogram(metric_name[width])`: buckets every matching counter event's
|
||||
//! (coerced-to-f64) value into fixed-width buckets of size `width` (a bare
|
||||
//! number, no duration unit — reusing the `[...]` range-selector slot for a
|
||||
//! different purpose than `rate()`'s duration), and emits one series per
|
||||
//! bucket, labeled `le` (Prometheus's "less-than-or-equal" convention) with
|
||||
//! that bucket's upper bound, holding a single `(timestamp_ms, count)` point.
|
||||
//! - `gauge(metric_name)`: takes no range selector; emits the most-recent
|
||||
//! matching counter event's value per label combination as a single point,
|
||||
//! i.e. the counter's series collapsed down to its last value.
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
@@ -37,6 +50,11 @@ pub enum AggOp {
|
||||
Sum,
|
||||
Avg,
|
||||
Rate,
|
||||
/// Ticket 022: buckets counter values into fixed-width buckets. Requires
|
||||
/// `bucket_width`.
|
||||
Histogram,
|
||||
/// Ticket 022: the most-recent counter value per label combination.
|
||||
Gauge,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
@@ -50,7 +68,18 @@ pub struct PromqlQuery {
|
||||
pub agg: Option<AggOp>,
|
||||
pub metric_name: String,
|
||||
pub matchers: Vec<LabelMatcher>,
|
||||
/// Duration range selector, e.g. `[5m]`. Only meaningful for `rate()`.
|
||||
pub range: Option<Duration>,
|
||||
/// Bare-number range selector, e.g. `[10]`. Only meaningful for
|
||||
/// `histogram()`, where it's the bucket width.
|
||||
pub bucket_width: Option<f64>,
|
||||
}
|
||||
|
||||
/// What a `[...]` range selector turned out to hold, disambiguated by whether
|
||||
/// a duration unit followed the digits.
|
||||
enum RangeSelector {
|
||||
Duration(Duration),
|
||||
BareWidth(f64),
|
||||
}
|
||||
|
||||
/// Parses a PromQL query string per the grammar documented above. Returns a
|
||||
@@ -66,6 +95,11 @@ pub fn parse_query(input: &str) -> Result<PromqlQuery, String> {
|
||||
if query.agg == Some(AggOp::Rate) && query.range.is_none() {
|
||||
return Err("rate() requires a range selector, e.g. rate(my_metric[5m])".to_string());
|
||||
}
|
||||
if query.agg == Some(AggOp::Histogram) && query.bucket_width.is_none() {
|
||||
return Err(
|
||||
"histogram() requires a bucket width, e.g. histogram(my_metric[10])".to_string(),
|
||||
);
|
||||
}
|
||||
Ok(query)
|
||||
}
|
||||
|
||||
@@ -180,8 +214,10 @@ impl<'a> Parser<'a> {
|
||||
Ok(matchers)
|
||||
}
|
||||
|
||||
/// `[<digits><unit>]`, e.g. `[5m]`.
|
||||
fn parse_range_selector(&mut self) -> Result<Option<Duration>, String> {
|
||||
/// `[<digits><unit>]` (e.g. `[5m]`, a duration — used by `rate()`) or a
|
||||
/// bare `[<digits>]` with no unit (a plain number — used by `histogram()`
|
||||
/// as a bucket width).
|
||||
fn parse_range_selector(&mut self) -> Result<Option<RangeSelector>, String> {
|
||||
self.skip_whitespace();
|
||||
if self.peek() != Some('[') {
|
||||
return Ok(None);
|
||||
@@ -190,16 +226,21 @@ impl<'a> Parser<'a> {
|
||||
let rest = self.remaining();
|
||||
let digits_end = rest
|
||||
.find(|c: char| !c.is_ascii_digit())
|
||||
.ok_or("unterminated range selector (missing unit and ']')")?;
|
||||
.ok_or("unterminated range selector (missing ']')")?;
|
||||
if digits_end == 0 {
|
||||
return Err("range selector must start with a number, e.g. [5m]".to_string());
|
||||
return Err("range selector must start with a number, e.g. [5m] or [10]".to_string());
|
||||
}
|
||||
let digits: i64 = rest[..digits_end]
|
||||
.parse()
|
||||
.map_err(|_| "range selector duration is not a valid number".to_string())?;
|
||||
.map_err(|_| "range selector number is not a valid number".to_string())?;
|
||||
self.pos += digits_end;
|
||||
|
||||
let unit = self.peek().ok_or("range selector is missing a duration unit")?;
|
||||
if self.peek() == Some(']') {
|
||||
self.expect_char(']')?;
|
||||
return Ok(Some(RangeSelector::BareWidth(digits as f64)));
|
||||
}
|
||||
|
||||
let unit = self.peek().ok_or("range selector is missing a duration unit or ']'")?;
|
||||
let seconds_per_unit: i64 = match unit {
|
||||
's' => 1,
|
||||
'm' => 60,
|
||||
@@ -213,14 +254,20 @@ impl<'a> Parser<'a> {
|
||||
};
|
||||
self.pos += unit.len_utf8();
|
||||
self.expect_char(']')?;
|
||||
Ok(Some(Duration::from_secs((digits * seconds_per_unit).max(0) as u64)))
|
||||
Ok(Some(RangeSelector::Duration(Duration::from_secs(
|
||||
(digits * seconds_per_unit).max(0) as u64,
|
||||
))))
|
||||
}
|
||||
|
||||
fn parse_vector_selector(&mut self) -> Result<PromqlQuery, String> {
|
||||
let metric_name = self.parse_identifier()?;
|
||||
let matchers = self.parse_label_matchers()?;
|
||||
let range = self.parse_range_selector()?;
|
||||
Ok(PromqlQuery { agg: None, metric_name, matchers, range })
|
||||
let (range, bucket_width) = match self.parse_range_selector()? {
|
||||
None => (None, None),
|
||||
Some(RangeSelector::Duration(d)) => (Some(d), None),
|
||||
Some(RangeSelector::BareWidth(w)) => (None, Some(w)),
|
||||
};
|
||||
Ok(PromqlQuery { agg: None, metric_name, matchers, range, bucket_width })
|
||||
}
|
||||
|
||||
fn parse_query(&mut self) -> Result<PromqlQuery, String> {
|
||||
@@ -231,6 +278,8 @@ impl<'a> Parser<'a> {
|
||||
"sum" => Some(AggOp::Sum),
|
||||
"avg" => Some(AggOp::Avg),
|
||||
"rate" => Some(AggOp::Rate),
|
||||
"histogram" => Some(AggOp::Histogram),
|
||||
"gauge" => Some(AggOp::Gauge),
|
||||
_ => None,
|
||||
};
|
||||
self.skip_whitespace();
|
||||
@@ -366,4 +415,33 @@ mod tests {
|
||||
let err = parse_query("requests extra").unwrap_err();
|
||||
assert!(err.contains("unexpected trailing input"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parses_histogram_with_bare_width() {
|
||||
let query = parse_query("histogram(latency_ms[10])").unwrap();
|
||||
assert_eq!(query.agg, Some(AggOp::Histogram));
|
||||
assert_eq!(query.metric_name, "latency_ms");
|
||||
assert_eq!(query.bucket_width, Some(10.0));
|
||||
assert_eq!(query.range, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_histogram_without_width() {
|
||||
let err = parse_query("histogram(latency_ms)").unwrap_err();
|
||||
assert!(err.contains("bucket width"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parses_gauge_with_no_range_selector() {
|
||||
let query = parse_query(r#"gauge(connections{host="a"})"#).unwrap();
|
||||
assert_eq!(query.agg, Some(AggOp::Gauge));
|
||||
assert_eq!(query.matchers, vec![LabelMatcher { name: "host".to_string(), value: "a".to_string() }]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rate_range_selector_is_still_a_duration_not_a_bare_width() {
|
||||
let query = parse_query("rate(requests[5m])").unwrap();
|
||||
assert_eq!(query.range, Some(Duration::from_secs(300)));
|
||||
assert_eq!(query.bucket_width, None);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -128,9 +128,17 @@ const SECTIONS: DocSection[] = [
|
||||
body: (
|
||||
<div className="flex flex-col gap-2">
|
||||
<p>
|
||||
A metric declaration bundles a message-match regex with one or more named param matchers (each with its own
|
||||
capture group and type). Every log line matching the regex produces a <em>bucket value</em>: a timestamped
|
||||
occurrence, tagged with the labels captured from that line.
|
||||
Every metric is fundamentally a <em>counter</em>: a message-match regex paired with one or more named param
|
||||
matchers (each with its own capture group and type). Every log line matching the regex is one counter
|
||||
increment — a timestamped occurrence, tagged with the labels captured from that line.
|
||||
</p>
|
||||
<p>
|
||||
Each param is either a <strong>label</strong> (the default) or the metric's single <strong>value</strong>{' '}
|
||||
param. Label params identify which bucket an occurrence belongs to; a value param's captured number becomes
|
||||
that occurrence's stored value instead of a label — e.g. a <Code>duration</Code> param marked{' '}
|
||||
<em>value</em> lets you query the distribution or latest reading of durations, not just how often they
|
||||
occurred. A metric with only label params still works exactly as before: every occurrence's value defaults
|
||||
to a fixed <Code>1</Code> (a pure occurrence count).
|
||||
</p>
|
||||
<p>
|
||||
Declaring a metric backfills it against everything already ingested, then continues indexing new lines as
|
||||
@@ -139,8 +147,25 @@ const SECTIONS: DocSection[] = [
|
||||
<p>
|
||||
The Metrics panel lets you declare a metric, pick one from the declared list, and see its bucket values
|
||||
grouped by label combination over the active time range — occurrence count per group, plus a small trend
|
||||
sparkline. That same data is what dashboard panels query via PromQL (see below).
|
||||
sparkline. That same data is what dashboard panels query via PromQL (see below), including two forms{' '}
|
||||
<em>derived</em> from a counter's value param rather than stored separately:
|
||||
</p>
|
||||
<ul className="list-disc pl-5">
|
||||
<li>
|
||||
<span className="font-semibold text-text">
|
||||
<Code>histogram(metric_name[width])</Code>
|
||||
</span>{' '}
|
||||
— buckets every matching event's value into fixed-width buckets of size <Code>width</Code>, e.g.{' '}
|
||||
<Code>histogram(durations[10])</Code>.
|
||||
</li>
|
||||
<li>
|
||||
<span className="font-semibold text-text">
|
||||
<Code>gauge(metric_name)</Code>
|
||||
</span>{' '}
|
||||
— the most-recent value per label combination in the active range, i.e. the counter collapsed to its
|
||||
latest reading.
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
),
|
||||
},
|
||||
|
||||
@@ -214,6 +214,15 @@ export function Metrics({ range }: { range: TimeRange | null }) {
|
||||
<option value="number">number</option>
|
||||
<option value="timestamp">timestamp</option>
|
||||
</select>
|
||||
<select
|
||||
className="rounded border border-border-soft bg-surface-1 px-1 py-1 text-xs text-text focus:border-border-strong focus:outline-none"
|
||||
value={matcher.role ?? 'label'}
|
||||
onChange={(e) => updateMatcher(i, { role: e.target.value as MetricParamMatcher['role'] })}
|
||||
title="Label params identify a bucket; at most one Value param becomes the counter's per-event captured value (used by histogram()/gauge())"
|
||||
>
|
||||
<option value="label">label</option>
|
||||
<option value="value">value</option>
|
||||
</select>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => removeMatcher(i)}
|
||||
|
||||
@@ -43,10 +43,15 @@ export interface Stats {
|
||||
last_timestamp_ms: number | null
|
||||
}
|
||||
|
||||
/** Whether a param contributes to a bucket's labels or is its captured value (ticket 022). */
|
||||
export type ParamRole = 'label' | 'value'
|
||||
|
||||
export interface MetricParamMatcher {
|
||||
param_name: string
|
||||
capture_group: string
|
||||
kind: 'string' | 'number' | 'timestamp'
|
||||
/** Defaults to 'label' server-side when omitted. */
|
||||
role?: ParamRole
|
||||
}
|
||||
|
||||
export interface MetricDeclaration {
|
||||
|
||||
Reference in New Issue
Block a user