- 059: log stream header/column detection is now generic
whitespace-columnar splitting (2+ spaces, single tab) instead of one
fixed six-column regex. Declared params come from whatever columns
the header actually names, plus a trailing unlabelled message field
that's never swallowed by the last named column.
- 060: timeline sort-by-property now walks an incrementally-maintained
per-sort-key helper index (BTreeMap keyed so "has the param, ordered
by value" sorts before "missing it, ordered by ottyl_seq") instead of
materializing and re-sorting the full filtered range on every query.
- 061: helper indexes are swept for staleness on ingest and dropped
after 5 minutes unused; a purged (or never-built) index transparently
rebuilds via the same lazy one-time scan on the next sorted query.
- Also invalidate a param's helper index on manual re-declaration,
removal, or ClearParams, so a redefined regex can't leave a sort
helper index serving values derived under the old declaration.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1ZJ3SrayzLzVndhVq9BmS
A rate() series over a metric whose occurrences are confined to a short
burst within a much wider queried range (e.g. sum by (code)
(rate(KeyPressed[5m])) against a ~10h log where all key presses happen
in the first 10s) rendered as broken: the line simply stopped at the
last real occurrence, and the chart's x-axis auto-scaled to the tiny
span of actual data points rather than the queried range, silently
narrowing a 10-hour view down to ~10 seconds with no indication anything
was off.
Backend (src/engine.rs): windowed_rate_series now appends a synthetic
boundary point at start_ms/end_ms (computed via the existing
rate::rate_at against the original occurrence points, so a boundary
inside a still-warm window gets its true partial value rather than a
hardcoded 0) whenever the real series doesn't already reach that edge.
Wired into both the bare rate() and sum()/avg(rate()) evaluate_promql
branches, which already had start_ms/end_ms in scope.
Frontend (PanelChart.tsx/Dashboards.tsx): the line chart's XAxis now
uses an explicit numeric domain pinned to the panel's queried range
(passed down from Dashboards.tsx's PanelPreview) instead of recharts'
default auto-scale-to-data-extent, so sparse data can no longer make
the axis look narrower than what was actually queried.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Resolved conflicts with ticket 051's unknown-metric guard in evaluate_promql
(reordered so ottyl_fmt's categorical rejection and the reserved-name
skip both run before 051's declaration/bucket-data check, since
ottyl_seq/ottyl_ts are synthesized from the log index and never have a
MetricDeclaration or bucket entries) and merged both tickets' additions
to the reserved-params Docs.tsx paragraph.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
auto_declare_params (src/engine.rs) previously never overwrote or pruned a
declaration, so a burst of unusual key=value-shaped lines could permanently
inflate the params UI forever. A real macOS `log stream` dump
(huig_dump.txt, copied into src/fixtures/log_stream_sample.txt) discovered
220 distinct auto-declared params from under 700 lines, mostly one-off
WiFi/radio stat counters (20+ pairs on a single line), plus base64-blob
false positives from key()'s greedy alnum matching landing right before
`==` padding.
Fix, combining two bounds per the ticket's options:
- src/logfmt.rs: extract_pairs/pair now rejects a candidate pair whose
bareword value starts with a bare '=' (looks_like_base64_padding_artifact)
— the exact shape produced when a base64/hash blob's padding sits right
after a run of alnum chars that key() consumed as if it were a key.
- src/engine.rs: auto_declare_params now enforces a per-line cap
(MAX_NEW_AUTO_PARAMS_PER_LINE = 8) on new params from one line, and a
total ceiling (MAX_AUTO_DECLARED_PARAMS = 64) on auto-derived params
overall — the per-line cap alone doesn't bound unbounded key *variety*
across many lines, so both are needed.
Also adds structured extraction for log stream's fixed-column shape
(Timestamp Thread Type Activity PID TTL, then `process: message`):
detect_format gains a new "log_stream" ottyl_fmt value (checked first,
since it's the most specific of the four), and
auto_declare_log_stream_params declares log_type/process/pid as standalone
queryable params the first time such a line is seen — derived positionally
from the column layout rather than =-scanning, so there's no ambiguity or
false-positive risk. This is a separate always-attempted extraction
alongside the existing key=value pass, not a replacement for it: a log
stream line's message tail can still contain real key=value pairs (e.g.
the WiFi stat lines) that auto_declare_params keeps discovering
independently. This also fully addresses ticket 054, confirming huig_dump.txt
is space-padded fixed-width columns, not TSV, and that no separate TSV
format is needed.
Throughput (src/fixtures/log_stream_sample.txt x50, release build):
before (caps disabled) ~369k lines/sec, 206 auto-declared params;
after (caps enabled) ~1.20M lines/sec, 64 auto-declared params — about
3.2x faster, confirming the ever-growing declaration set was the
bottleneck rather than per-line regex/nom scanning cost.
Removes tickets/050 and tickets/054 now that both are implemented.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
evaluate_promql previously resolved every metric name through bucket_values,
which only returns data for a real MetricDeclaration — so a dashboard panel
querying ottyl_seq/ottyl_ts/ottyl_fmt (already supported in the filter bar
and alert conditions) silently returned an empty series. ottyl_seq/ottyl_ts
are now synthesized directly from the ingested-log index instead of requiring
a declaration; ottyl_fmt is categorical (json/logfmt/none) with no sensible
numeric reading, so it's rejected up front with a clear error pointing back
at the filter bar rather than returning nothing.
GET /api/params now also lists the three reserved names (marked `reserved:
true`, distinct from `auto_derived`) purely for discoverability — merged in
at the API layer only, not the engine's internal ParamDeclarations query, so
profile-switch removal and metric-param-promotion logic are unaffected.
declare_param also now rejects declaring a param under one of these names.
Docs.tsx and the Params panel are updated to reflect the new dashboard
support and the reserved/non-editable listing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Wires the engine's existing RemoveMetricDeclaration message (previously
only used internally by delete_profile's cleanup) up to a new DELETE
route, persists the removal into the active profile so it survives a
restart, and adds a delete affordance in the Metrics UI.
Decisions:
- Bucket-value history under the deleted metric's name is purged, not
left orphaned (engine.rs's new remove_metric_declaration reuses
update_metric's clear_metric_data helper). This is what lets a
dashboard panel or PromQL query still referencing the metric by name
fail with a readable "unknown metric" error instead of silently
returning stale data.
- evaluate_promql now errors when a metric has neither a declaration nor
any bucket data, since a metric ingested via --metrics-url scraping
never gets a MetricDeclaration but must stay queryable. Verified with
a seeded fixture (promql_query_against_deleted_metric_fails_with_readable_error
in src/api.rs).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
DELETE /api/params clears the entire standalone-param set — manual and
auto-derived alike — then immediately re-runs auto_declare_params against
every IndexKey::IngestedLog entry still held in the engine's index (oldest
to newest), so params come back fresh from currently-retained log text
without requiring new lines to be ingested. This is a deliberate full
reset, unlike reload_engine_declarations's profile-switch cleanup, which
preserves auto-derived params since those are engine-discovered facts
rather than profile state; here the immediate re-extraction step makes
clearing them safe.
Settings::clear_param_declarations empties the active profile's persisted
params so they don't reappear on restart. The webapp's Params section gets
a single "Clear all" button (with a confirm dialog and a status line
reporting removed/re-derived/logs_considered) rather than a per-param
delete loop; logs_considered surfaces that lines already evicted by
max_logs can't regain their params.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
/api/logs now accepts sort_by (an already-declared param name) and
sort_missing_fallback: lines carrying that param are ordered by its value,
while lines missing it are either appended in ottyl_seq order (fallback,
the default) or dropped from the response. Default ottyl_ts-tracking
order is unchanged when sort_by is omitted.
The log viewer (both the inline panel and its popout) exposes this as a
"Sort by" control next to the column picker, persisted to localStorage
under one shared key (ottyl.logPreview.sort) — the same global-viewer-
setting convention already used for visible param columns and the log
viewer's height, since there's no per-dashboard scope for this to divide
by.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Ticket 044: `sequence` and `timestamp` collided with ordinary user-declared
param names and didn't signal they're ottyl-owned, so they're renamed
(breaking change, no back-compat alias) to `ottyl_seq`/`ottyl_ts` in the
filter grammar (src/filter_parser.pest, src/filter_parser.rs), alert query
restriction/evaluation (src/alerts.rs), and the filter-bar docs/placeholder
in the webapp.
Also adds a new reserved param `ottyl_fmt` exposing which format a line was
auto-detected as ("json"/"logfmt"/"none") via a new
`crate::logfmt::detect_format`, wired into the same filter grammar and alert
restriction list as `ottyl_seq`/`ottyl_ts`. A mixed-format fixture test in
engine.rs verifies `ottyl_fmt` reflects the actual detected format per line.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Previously a Value-role param with kind: Duration silently meant "elapsed
time since the previous same-labeled event" instead of parsed text — the
same MetricPrimitiveKind::Duration variant meant two unrelated things
depending on which list (label_params vs value_param) referenced it.
Add ParamDeclaration::elapsed_since_last, a third extraction source
alongside capture_group/json_field_path: when set, the param is computed
(via a new resolve_param_value, used uniformly for label_params and
value_param) rather than text-extracted, and declare_param validates it's
paired with kind: Duration. apply_metric_to_log no longer branches on
(role, kind) — it dispatches purely on what each resolved param declares.
Migrate existing profiles: migrate_duration_value_params converts any
Duration-kind param referenced as a metric's value_param into the new
extractor, and resolve_or_create_legacy_param (ticket 037's pre-existing
migration) now carries the same meaning forward for pre-037 inline
Value-role Duration matchers.
Params.tsx offers "Elapsed since previous event" as its own Extraction
option (hides the now-irrelevant message-match-regex/capture-group
fields and forces kind to duration); Metrics.tsx labels such params
accordingly wherever they're listed as label/value param choices.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
compile_metric_comparison used to look up `param` by reading whatever key
happened to be present in a bucket's `labels` map at query time, so a typo
or a param never declared as a label on that metric silently matched
nothing instead of being rejected. Now it looks up the target metric's
MetricDeclaration (ticket 037's unified label_params/value_param shape) and
rejects the filter up front with an error naming the metric and its actual
declared label params when `param` isn't among them.
Metric names with no declaration at all are left unvalidated here, matching
prior behavior (bucket_values simply yields nothing for them).
Added tests: metric[nonexistent_param] is rejected with an error naming the
metric and its real label params; existing metric[real_param] filters still
match as before.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Fix one test call site to unwrap the Result now returned by
evaluate_promql (introduced by ticket 041). Full suite (194 tests),
clippy, and webapp build all pass after merging all of 037, 040, 041,
042, and 043.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
The 041 branch was built against the pre-037 ParamRole/inline-params shape;
rebase the string-kind lookup and test helper onto the unified
ParamDeclaration/label_params/value_param model from ticket 037.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Collapse the Params-panel ParamDeclaration and the metric-embedded
RegexToMetricPrimitiveMatcher into a single ParamDeclaration type
(name, regex/capture-group-or-json-field-path, kind). MetricDeclaration
now references params by name via label_params: Vec<String> and
value_param: Option<String> instead of embedding its own extractor per
param, so a param declared once in the Params panel can be reused as a
label or value by any number of metrics. Declaring a metric that
references an undeclared param name is now rejected at declare-time.
Backend:
- src/data_value.rs: unified ParamDeclaration struct; MetricDeclaration
data value now carries label_params/value_param instead of a Vec of
inline matchers.
- src/engine.rs: DeclareParamRequest is now an alias for
ParamDeclaration; DeclareMetricRequest carries label_params/
value_param; validate_metric_request checks referenced param names
against the declared-param store; apply_metric_to_log resolves
label_params/value_param against a per-ingest resolved-param map
instead of reading an inline matcher list.
- src/profile.rs: MetricDeclarationRecord carries label_params/
value_param plus a private, serialize-skipped legacy_params field
that only ever gets populated by deserializing a pre-037 profile
file's old "params" key. ProfileStore::load migrates any such legacy
metric into the unified shape (reusing an existing same-shaped bare
param by name, or creating one implicitly named
"<metric_name>.<param_name>"), then persists the migrated store so
the on-disk file only carries the legacy shape once. Migration is
idempotent (deterministic naming) and covered by dedicated tests.
- src/api.rs / src/dto.rs: MetricDeclarationDto and the declare/update
metric response bodies updated to the new shape.
Frontend:
- webapp/src/lib/api.ts: MetricDeclaration type now has label_params/
value_param; MetricParamMatcher/ParamRole types removed.
- webapp/src/components/Metrics.tsx: the declaration form now fetches
declared params and lets you check off label params / pick a value
param from that list, instead of re-entering a regex/capture-group/
kind per metric param.
Leaves ticket 038 (duration as a derived param kind) and 039 (param
query resolving declared params) untouched, as scoped.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Extract the trailing-window rate/count implementation into a new src/rate.rs,
shared by PromQL's rate() (engine::evaluate_promql, via a thin i64<->Timestamp
adapter) and alert Threshold/Ratio conditions (alerts::evaluate_condition, via
the new incremental RateWindow), replacing alerts' bespoke VecDeque+prune loop.
Both now agree on what "windowed rate" means for the same data, verified by a
new regression test (engine::tests::dashboard_rate_and_alert_threshold_agree_on_same_windowed_data).
Behavior change: alert Threshold/Ratio windows are now backfilled once, on
first evaluation, from already-ingested logs within their own trailing window
(bounded cost, not a full history rescan) - a rule created mid-session, or a
process that replays existing log content (e.g. --file tailing, which rereads
the whole file from the start on every process start), no longer starts with
a silently empty window. A bare restart with no replayed history (e.g. stdin
ingestion) still starts empty, since there is genuinely no log history to
backfill from in that case - alert state itself remains in-memory only.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
merge_series_exact required an exact-millisecond timestamp match to combine
sub-series, so sum(requests) by (path)-style queries over raw occurrence
counts produced one point per underlying (label, timestamp) pair instead of
a meaningful combined total, since real occurrence timestamps across
different label groups essentially never coincide. Added a regression test
confirming the sparse behavior, then replaced the merge with
merge_series_windowed: a single-sub-series group is passed through
unchanged (preserving existing single-label-group resolution exactly), and
groups with multiple sub-series are bucketed into fixed 1s windows before
summing/averaging, mirroring the fixed-width bucketing rate() already uses.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
`coerce_to_f64` (src/engine.rs) mapped every MetricPrimitive::String to a
flat 1.0, so sum()/avg() over a string-kind metric silently became an
occurrence count with no signal that the aggregate didn't mean what it
looked like it meant.
evaluate_promql now looks up the target metric's declared value kind (the
kind of its Value-role param, defaulting to Number when it has none) and
rejects sum()/avg() with a clear, actionable error when that kind is
String. sum(rate(...))/avg(rate(...)) are exempt since rate() only counts
occurrences per window and never reads the coerced value. Also add a real
count() PromQL aggregation (parser + evaluator) so occurrence-counting is
something a user opts into by name, matching the ticket's preferred
"count() or numeric metric" error message and the existing
sum()/avg()/rate()/histogram()/gauge() aggregation design.
Audited existing fixtures/tests for sum()/avg() over string-kind metrics
that this would newly reject: none found — the repo's only sum()/avg()
usage in tests and docs is over the "requests" occurrence-counter metric,
which has no Value-role param and so is Number-kind, unaffected by this
change.
EngineResponse::Promql already carried Result<Vec<PromqlSeriesDto>,
String> for parser errors, so evaluate_promql now returns that same
Result and the new error flows through the existing 400 Bad Request path
in run_promql_query with no API changes needed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
sum/avg/rate already grouped series by the query's by_labels, but
histogram() collapsed every label combination into one histogram
regardless of by(), silently discarding the grouping. evaluate_promql's
Histogram branch now groups series by by_labels first (reusing the same
project_labels mechanism as sum/avg), then buckets each group's values
independently, so histogram(metric) by (label) yields one histogram per
label group carrying that group's labels alongside `le`. An empty
by_labels still collapses to a single histogram, matching prior
behavior exactly, so sum/avg/rate and unqualified histogram() queries
are unaffected.
The dashboard panel UI needed no change: PromQL is entered as free
text and PanelChart's seriesLabel()/legend rendering already renders
arbitrary label maps generically, so grouped histogram series display
correctly out of the box.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
RegexHelpIcon previously used a single `open` boolean driven by both hover
and click, so clicking to pin the popover open was undone by the next
mouseleave. Split into separate `pinned` (click) and `hovering` (mouse-over)
states so a click-toggle actually persists until clicked again.
The seeded `log_category` dashboard's line panel plotted the bare metric
name, since `sum()`/`rate()` always collapsed all label combinations into
one series and PromQL syntax had no way to preserve a label dimension
through an aggregation. Added `by (label, ...)` grouping and nested
`sum(rate(metric[range]))`/`avg(rate(...))` parsing, and made `rate()`
compute a real windowed time series instead of a single trailing point, so
a query like `sum(rate(log_category[1m])) by (category)` now renders one
line per category.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Resolves conflicts with ticket 018 (boolean param kind) in
src/engine.rs, webapp/src/lib/api.ts, and Params.tsx: boolean-kind
params and JSON-field-path extraction now coexist as independent
extraction modes alongside regex capture groups.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
RegexToMetricPrimitiveMatcher and the standalone param declaration
(DeclareParamRequest/DataValue::ParamDeclaration) gain an optional
json_field_path, an alternative to capture_group that reads a
dot-separated field path (e.g. "foo.bar") out of lines that parse as
JSON objects. It's additive and serde-defaulted, so existing
regex-based params/metrics are unaffected. Params.tsx now offers a
"JSON field" extraction option when the message match regex looks
like a JSON object.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Clicking a declared metric badge in Metrics.tsx now loads its regex/params
into the declaration form for editing (in addition to selecting it for the
bucket-values view), and shows those properties inline once selected rather
than only on hover. The backend gains PUT /api/metrics/{name}, backed by a
new engine::update_metric that reuses declare_metric's validation, requires
the metric to already exist, supports renaming, and re-backfills bucket data
from the new regex/params instead of leaving stale buckets from the old one.
Submitting the form while editing calls the update endpoint in place rather
than creating a second declaration; declaring a brand-new metric is unchanged.
Note: tickets/022-edit-existing-metric-declarations.md does not exist on this
worktree's branch (it was added to tickets/ on main after this branch was
created), so there is nothing to delete here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
MetricPrimitiveKind gains Boolean, serialized as "boolean". A param
declared with kind Boolean skips capture-group validation and, instead
of being omitted when message_match_regex doesn't match, is present on
every log line as true/false based purely on whether the regex matches
(case-insensitively or per the regex's own flags). Params.tsx lets you
pick boolean as a kind and disables the now-unused capture group input.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Extends the FilterBar query language with a json[field] <op> value
comparison, parsed by filter_parser.pest/rs alongside metric[param],
and evaluated in engine.rs by parsing each line as JSON at query time
(lines that aren't valid JSON, or lack the field, simply don't match).
Also wired into alerts.rs since json[field], like regex/sequence/
timestamp, only needs the line's own text.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Tracks a running total of ingested log bytes (approximated as each
log's raw UTF-8 text length), incremented on ingest and decremented
in evict_excess_logs when a log is dropped. Exposed via StatsDto's
new approx_bytes_total field on GET /api/stats, and displayed in
NavMenu's stats panel formatted with toLocaleString() (comma
separators).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
--settings-path's help text now names the actual resolved default path
(computed once via LazyLock) instead of only describing where to look for
one. Also adds an optional --metrics-url flag that polls a Prometheus/
OpenMetrics-compatible endpoint on an interval, pushing samples straight
into a metric's bucket (bypassing log-line regex matching) so they're
queryable via PromQL/dashboards alongside log-derived metrics.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Alert rules no longer carry their own ntfy topic/server; Settings now
holds one process-wide ntfy sink (with its own "send test notification"),
and the Alerts panel is left to just define rule conditions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Pass over src/*.rs, webapp/src/**, and the in-app "?" docs prose:
condensed multi-paragraph doc comments to single lines that state the
non-obvious why, deleted comments that only restated adjacent code, and
tightened the Docs.tsx help-panel wording. No logic, structure, or code
formatting changed — verified via cargo test (143 passed) and
tsc -b && vite build.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Adds AlertRule/AlertCondition/AlertSink models with CRUD persisted on
Settings (mirroring CssEntry's shape), per-line trigger evaluation hooked
into the engine's ingest path (Immediate/Threshold/NewValue/Ratio
conditions), and best-effort ntfy delivery spawned off the ingest hot path.
Design notes:
- Alert queries reuse FilterBar syntax but only support the
regex()/sequence/timestamp subset (no metric[param] comparisons), since
resolving those would require a full bucket rescan per alert per ingested
line rather than cost proportional to new logs.
- Ratio's zero-denominator case skips evaluation for that line rather than
treating the ratio as infinite.
- NewValue's seen-value set is in-memory only and does not survive a
restart, matching the engine's other non-persisted runtime state.
engine::run now takes a SettingsHandle (moved from api.rs to settings.rs so
engine.rs can depend on it without a circular import) to read the current
alert rules on each ingest.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
ensure_default now seeds a brand-new Default profile (not one that
already existed) with a "log_category" counter metric and an
"Overview" dashboard containing a line and a pie panel, both querying
that metric so each renders both series from a single PromQL query
like any user-created panel.
The counter classifies every line into exactly "category=error" or
"category=other" via two new fixed-label fields on metric
declarations: DeclareMetricRequest::matched_labels/unmatched_labels.
These apply literal label values instead of parsing captured text,
which is what keeps a case-insensitive match (e.g. "ERROR" vs "Error")
collapsed into one clean series instead of fragmenting by source
casing, and is also the only way to express the "line does NOT match"
side at all: plain regex alternation can't do it without lookaround,
which the `regex` crate doesn't support (a catch-all alternation
branch always wins at the search's leftmost position). Both fields
default to None and are backward compatible with existing profiles.
Seeded panels are ordinary editable/deletable dashboard state,
persisted through the normal profile save path; ensure_default never
re-seeds or duplicates them on a later call once the profile exists.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
MetricPrimitiveKind::Duration/MetricPrimitive::Duration(i64 ms) is a
new primitive kind. On a metric's Value-role param (ticket 022), kind
Duration is special-cased in apply_metric_to_log: instead of parsing
the captured text, the value is derived as elapsed time since the
previous counter event with the same label set, using the ingestion
timestamp. That "previous event" lookup is tracked via a new
IndexKey::LastMetricEvent/DataValue::LastEventTimestamp bookkeeping
entry rather than Bucket.values itself, because a label set's first
occurrence has nothing to diff against and pushes no Bucket entry at
all (per spec: omit, don't emit a bogus zero) — but it still needs to
be recorded so the *next* occurrence has something to diff against.
Duration values compose with the existing bucket/PromQL surface
unchanged (coerce_to_f64, filter comparisons, histogram()/gauge()).
On any other param (Label-role metric param, or a standalone
ParamDeclaration), Duration kind instead parses captured text as a
literal i64 millisecond count, same as Number. Profile round-trip
(ProfileMetricPrimitiveKind) and the Metrics.tsx declaration form are
updated to match.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Every metric declaration is now explicitly documented and modeled as
a counter: a match is an occurrence/increment, optionally labeled.
The bucket representation (BucketValue { values: Vec<(Sequence,
Timestamp, MetricPrimitive)> }) already was this; what it lacked was
any way to capture a *value* distinct from the occurrence-count-of-1
marker. Adds RegexToMetricPrimitiveMatcher::role (ParamRole::Label,
the default, or ParamRole::Value — at most one per metric, enforced
at declare time) so a metric can designate one param's captured
number as its per-event value instead of a label.
promql.rs/engine.rs add two new PromQL-evaluable forms built on top
of that value, both consumed as regular PromqlSeriesDto series like
sum()/avg()/rate():
- histogram(metric_name[width]): buckets matching events' values into
fixed-width buckets, one series per bucket labeled `le`.
- gauge(metric_name): the most-recent value per label combination in
the queried range.
Existing raw counter queries (metrics with only Label-role params,
sum/avg/rate) are unchanged. Profile files saved before this ticket
still load (role defaults to Label via #[serde(default)]). Metrics.tsx
gains a label/value role picker per param, and the Docs panel explains
the model and the two derived query forms.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Settings.max_logs (default None = unlimited) is pushed to the engine
via a new EngineMessage::SetMaxLogs, enforced on every ingest and
whenever the setting changes at runtime (no restart required).
Eviction drops the oldest (lowest-sequence) logs; sequence numbering
isn't renumbered and buckets already derived from an evicted log are
left in place. Exposed via PATCH /api/settings (max_logs: number |
null) and a NavMenu numeric field (blank = unlimited).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Adds a filter evaluator in engine.rs that compiles a parsed FilterExpr
against the current index once per query (regex compiled once,
sequence/timestamp compared directly against IngestedLog fields,
other metric comparisons resolved to a HashSet<u64> of matching
sequences via a bucket scan) and evaluates it per log line with no
further index access. Wires this into EngineQuery::Logs/Timeline via
a new LogsParams::filter/TimelineParams::filter field, and adds
EngineResponse::Error so a malformed filter string surfaces as a
query-level error (mapped to 400 in api.rs).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Hand-rolls a minimal PromQL parser (src/promql.rs) for metric-name +
label-matcher selection, optional [range] selectors, and top-level
sum()/avg()/rate() wrapping, rather than pulling in a full PromQL parser
crate — the required subset is small and a hand-rolled parser stays fully
engine-independent (mirrors filter_parser.rs) with no unused-AST overhead.
Adds EngineQuery::Promql/EngineResponse::Promql (engine parses + evaluates
in place against the index, no copying) and GET /api/query, returning
Vec<PromqlSeriesDto> (labels + (timestamp_ms, f64) points) for charting,
with malformed queries surfaced as 400 + ErrorDto.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Extend GET /api/logs to derive each declared param's value per log line
(regex + capture group + kind, matching that line's text) and include it as
a params map on LogEntryDto, omitting entries that don't match rather than
sending nulls. Add GET /api/params (list_params, mirroring list_metrics) so
the frontend can discover currently-declared param names. In LogPreview, add
a column-picker popover backed by that list, with the selection persisted in
localStorage under `ottyl.logPreview.visibleParamColumns`.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Exposes param/metric declarations and derived bucket data over HTTP:
POST /api/params and POST /api/metrics validate and declare against the
engine, persisting into the active profile (if any) so declarations survive
a restart; GET /api/metrics lists declarations; GET /api/metrics/:name
queries bucket values with label, time-range, sequence, and limit filters.
Adds a new EngineQuery::BucketValues variant that layers this filtering on
top of ticket 003's bucket_values helper.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
Runs each declared metric's regex against log text both retroactively
(backfill on DeclareMetric) and going forward (on each Ingest), appending
(sequence, timestamp, value) entries to IndexKey::Bucket/DataValue::BucketValue.
All of a metric's declared params become the Bucket key's labels (the only
structured place for them), and the stored value is a fixed presence marker
(Number::Int(1)) — a bucket's values are an occurrence timeseries for that
label combination, not the captured values themselves. A param whose capture
doesn't match or fails to parse under its declared kind skips the whole match
rather than storing a partially-labeled bucket.
Exposes an internal bucket_values(index, metric_name, labels_filter) helper
for the upcoming metrics-API ticket to build on.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
