Merge branch 'worktree-agent-a63474e92b05425a5'

# Conflicts:
#	Cargo.lock
#	Cargo.toml
This commit is contained in:
Hugh Kaznowski
2026-09-05 12:46:45 +01:00
28 changed files with 958 additions and 3 deletions
+35
View File
@@ -0,0 +1,35 @@
# 001 — Param declarations
## Summary
Implement the `ParamDeclaration` slice of the primary index: users can declare how a
named parameter is derived from a log line (e.g. deriving a timestamp different from
the ingestion timestamp, or a log level), and the engine stores/serves those
declarations.
## Scope
- `IndexKey::ParamDeclaration { name }` and a matching `DataValue` variant for the
declaration body (a regex against the raw log text, plus how the capture maps to a
`MetricPrimitive` — reuse `RegexToMetricPrimitiveMatcher` / `MetricPrimitiveKind`
from `data_value.rs`, which already exist but are unused).
- Engine message(s) to add/list/remove a param declaration.
- Validate declared regexes at declaration time (reject invalid regex / unknown
primitive kind up front rather than failing per-log later).
- No derivation/backfill against existing logs yet — that's
[002](002-metric-declarations.md) / [003](003-metric-bucket-derivation.md), since
metrics are what actually get indexed into buckets. This ticket is just the
declaration storage + validation.
## Acceptance criteria
- Declaring a param with a valid regex + capture group succeeds and is retrievable.
- Declaring a param with an invalid regex or referencing a non-existent capture group
is rejected with a clear error, not silently stored.
- Declarations sit in their own contiguous range of the BTreeMap
(`ParamDeclaration` between `LogParamSplit` and `ParamMetricSplit`), per the
`IndexKey` ordering in the README.
## Depends on
None — this is foundational.
+35
View File
@@ -0,0 +1,35 @@
# 002 — Metric declarations
## Summary
Implement the `MetricDeclaration` slice of the index: a language/API for declaring a
metric — a regex that matches a log message plus params extracted from it — so that
matching logs can later be bucketed ([003](003-metric-bucket-derivation.md)).
## Scope
- `IndexKey::MetricDeclaration { name }` paired with `DataValue::MetricDeclaration`
(already sketched in `data_value.rs`: `message_match_regex` +
`Vec<RegexToMetricPrimitiveMatcher>`).
- Engine message(s) to add/list/remove a metric declaration.
- A metric declaration's params may reference params already declared via
[001](001-param-declarations.md) (e.g. label a bucket by an existing param) in
addition to inline capture groups on the metric's own regex — decide and document
which is supported for v1 (README implies inline capture-group derivation is the
primary mechanism: "buckets and parameters can be derived" from the metric
declaration itself).
- Validate the regex and capture-group → primitive-kind mapping at declaration time,
same as param declarations.
## Acceptance criteria
- Declaring a metric with a valid regex + param matchers succeeds and is retrievable.
- Invalid regex, or a matcher referencing a capture group the regex doesn't have, is
rejected at declaration time.
- Declarations sit in their own contiguous range of the BTreeMap
(`MetricDeclaration` between `ParamMetricSplit` and `MetricBucketSplit`).
## Depends on
- [001 — Param declarations](001-param-declarations.md) (shares matcher/primitive
validation code and the "derive a value from a regex capture" concept).
+37
View File
@@ -0,0 +1,37 @@
# 003 — Metric bucket derivation
## Summary
Apply metric declarations to log lines and populate `IndexKey::Bucket` /
`DataValue::BucketValue` entries, both retroactively (backfill already-ingested logs
when a metric is declared) and going forward (index each new log against all current
metric declarations as it's ingested).
## Scope
- On `MetricDeclaration` add: scan the existing `IngestedLog` range, run the metric's
regex against each log's text, and for each match compute the labeled `Bucket` key
(`metric_name` + `labels: BTreeMap<String, MetricPrimitive>`) and append
`(sequence, timestamp, value)` to that bucket's `BucketValue`.
- On new log ingest: run all currently-declared metrics' regexes against the line and
update buckets incrementally, same as backfill.
- Decide bucket cardinality behavior: a single log line can match multiple metrics: it
should update all matching buckets. Document how differing label sets from the same
metric become distinct `Bucket` keys (labels are part of the key, so this falls out
naturally from `BTreeMap` — just needs the insert-or-append logic).
- This is the piece that makes `MetricPrimitive`, `DataValue::BucketValue`, and the
`Bucket` index range (currently defined but never populated) actually load-bearing.
## Acceptance criteria
- Declaring a metric against a log stream that already has matching lines populates
buckets for those lines without needing new input.
- Ingesting a new log line after metrics are declared updates the relevant buckets
live.
- A log line matching zero metrics doesn't touch the bucket range.
- Bucket values preserve insertion order / are queryable by sequence and timestamp
range (needed by [004](004-metrics-api.md) and dashboards).
## Depends on
- [002 — Metric declarations](002-metric-declarations.md)
+31
View File
@@ -0,0 +1,31 @@
# 004 — Metrics API
## Summary
Expose param/metric declarations and bucket data over HTTP so the frontend (and the
filter/dashboard query engines) can declare metrics and read their values.
## Scope
- `POST /api/params` — declare a param ([001](001-param-declarations.md)).
- `POST /api/metrics` — declare a metric ([002](002-metric-declarations.md)).
- `GET /api/metrics` — list declared metrics (+ their declared params/labels).
- `GET /api/metrics/:name` — query bucket values for a metric, filtered by label
selector and time/sequence range, mirroring the existing `LogsParams`/
`TimelineParams` pattern in `dto.rs`.
- Follow the existing engine architecture: new `EngineQuery`/`EngineMessage` variants,
handled in `engine.rs`'s single-owner loop, DTOs added to `dto.rs`, routes added to
`api.rs`'s router — no new locking or shared state.
## Acceptance criteria
- Declaring a param/metric via the API round-trips through to engine state and shows
up in `GET /api/metrics`.
- Querying a metric's bucket values supports the same start/end and limit-style
params as the existing logs/timeline endpoints.
- Bad declarations (invalid regex etc.) return a 4xx with a useful error body, not a
500 or silent no-op.
## Depends on
- [003 — Metric bucket derivation](003-metric-bucket-derivation.md)
+37
View File
@@ -0,0 +1,37 @@
# 005 — FilterBar query language (pest grammar)
## Summary
Build the syntax-aware search language described in the README's FilterBar section,
using `pest`, supporting regex searches and metric comparisons combined with
boolean operators.
## Scope
- Add `pest` + `pest_derive` dependencies.
- Grammar covering:
- `regex("...")` — raw-text regex search over log lines.
- `metric_name[param] <op> value` comparisons (`<`, `>`, `<=`, `>=`, `==`, `!=`) —
`metric_name[param]<5` per the README example.
- `sequence` and `timestamp` as reserved metric names usable the same way
(`sequence<5`, `timestamp>12:00`).
- Boolean combinators `and` / `or` with standard precedence, e.g.
`regex("error") and sequence<5 or timestamp>12:00`.
- Parse into an AST (not directly into engine queries — that's
[007](007-filterbar-integration.md)).
- Surface parse errors with position info suitable for showing under the FilterBar
input.
## Acceptance criteria
- Grammar test cases cover: bare regex, single metric comparison, reserved-metric
comparison, `and`/`or` combinations with correct precedence, and at least one
malformed-input case producing a located error rather than a panic.
- Parser is a standalone module with no dependency on the engine/index types, so it
can be unit tested in isolation.
## Depends on
None directly, but the metric-comparison side of the grammar is only meaningful once
[002](002-metric-declarations.md) exists — build/test this ticket against metric
names as opaque strings.
+30
View File
@@ -0,0 +1,30 @@
# 006 — FilterBar datetime parsing
## Summary
Support the several datetime formats the README requires for timestamp comparisons
in the filter language: relative humantime, absolute time, and full dates.
## Scope
- Relative humantime: `5h`, `-3m`, etc. — resolved against "now" (or against the
query's reference time, if one is introduced later) at parse/eval time.
- Absolute time: e.g. `12:00` (time-of-day, as in the README's
`timestamp>12:00` example) — needs a defined rule for which day it resolves to.
- Full date: complete date(+time) literals.
- Produce a single normalized value (`Timestamp`/millis) regardless of input form, for
the filter evaluator ([007](007-filterbar-integration.md)) to compare against.
- Pick and document a crate (e.g. `humantime` for the relative case) rather than
hand-rolling a datetime parser.
## Acceptance criteria
- Each of the three formats parses to the correct absolute millis value in unit
tests, including edge cases (negative relative offsets, time-of-day near midnight).
- Ambiguous/invalid input produces a clear error rather than a silently wrong
timestamp.
## Depends on
None — standalone parsing module, consumed by
[007 — FilterBar integration](007-filterbar-integration.md).
+36
View File
@@ -0,0 +1,36 @@
# 007 — FilterBar query integration
## Summary
Wire the parsed filter AST into actual log/timeline queries against the engine, so
FilterBar's syntax (not just a naive substring match) drives what's shown in
LogPreview and LogTimeline.
## Scope
- Evaluator that walks the AST from [005](005-filterbar-pest-parser.md), resolving
`regex(...)` against log text and `metric_name[param] <op> value` against bucket
data ([004](004-metrics-api.md)), using datetime literals resolved via
[006](006-filterbar-datetime-parsing.md).
- New/extended `EngineQuery::Logs` (and `Timeline`) filter parameter that accepts the
compiled filter, evaluated inside the engine's single-owner loop (per the README's
"engine is sole owner of the btreemap" architecture — don't evaluate against a
copy).
- `sequence`/`timestamp` reserved-metric comparisons map directly to the
`IngestedLog` key fields rather than a declared metric lookup.
- API layer: accept the raw filter string, parse it (return 4xx on parse error),
compile to whatever the engine query needs, send to engine.
## Acceptance criteria
- `GET /api/logs?filter=...` and the timeline equivalent apply the full filter
language, not just substring matching.
- A filter combining a regex clause and a metric comparison with `and`/`or` returns
correct results against a seeded log+metric fixture.
- Malformed filter strings return a 4xx with the parser's error message.
## Depends on
- [004 — Metrics API](004-metrics-api.md)
- [005 — FilterBar pest parser](005-filterbar-pest-parser.md)
- [006 — FilterBar datetime parsing](006-filterbar-datetime-parsing.md)
+33
View File
@@ -0,0 +1,33 @@
# 008 — Settings model
## Summary
Implement the single, process-wide settings model described in the README's NavMenu
section: colour scheme, CSS path config, and the settings file location — settable
via CLI args, persisted at a sensible default OS location.
## Scope
- A `Settings` struct: colour scheme choice, list of imported/linked CSS entries
(see [010](010-css-management.md)), and whatever else profiles need to hang off of.
- Default settings file location using an OS-conventions crate (e.g. `directories`)
rather than a hardcoded path.
- `clap` arg(s) to override the settings file path at runtime (extend the existing
`Args` in `main.rs`, which today only has `--port`/`--static-dir`).
- Load-on-start / save-on-change semantics; "at runtime, there is only 1 settings" —
i.e. a single in-memory settings instance, not per-request state, analogous to how
the engine owns the index.
- Settings "owns" the profiles per the README — this ticket only needs to define
where profiles live under settings, not implement profile CRUD itself (that's
[009](009-presets-profiles.md)).
## Acceptance criteria
- Running with no args creates/reads a settings file at the documented default
location.
- `--settings-path` (or similar) overrides it.
- Settings persist across restarts (edit, restart, re-read matches).
## Depends on
None.
+34
View File
@@ -0,0 +1,34 @@
# 009 — Presets / profiles
## Summary
Implement presets ("profiles") that track parsing/indexing rules (param + metric
declarations) and stored dashboards, so restarting ottyl against a given CLI app
doesn't lose that configuration.
## Scope
- Profile = a named bundle of: param declarations ([001](001-param-declarations.md)),
metric declarations ([002](002-metric-declarations.md)), and saved dashboards
([014](014-dashboards-ui.md), once it exists — can stub the dashboard field until
then).
- Persisted under the settings storage location from
[008](008-settings-model.md); settings reference which profile(s) exist / which is
active.
- On startup, load the active profile's param/metric declarations into the engine
before ingestion starts (or apply-and-backfill if logs arrive before load
completes — define the ordering explicitly).
- CRUD for profiles, exposed later via NavMenu UI ([011](011-navmenu-ui.md)).
## Acceptance criteria
- Declaring params/metrics through a profile, restarting the process, and re-running
the same CLI command reproduces the same bucket derivation without re-declaring
anything.
- Multiple profiles can exist; switching the active profile changes which
declarations are loaded on next start.
## Depends on
- [008 — Settings model](008-settings-model.md)
- [002 — Metric declarations](002-metric-declarations.md)
+30
View File
@@ -0,0 +1,30 @@
# 010 — CSS import/management
## Summary
Let users import custom CSS (stored alongside settings/profiles, deletable) or link
to an external CSS path/URL (not deletable, just referenced).
## Scope
- `POST /api/settings/css` (import): accept raw CSS content, store it in the same
directory as settings/profiles ([008](008-settings-model.md)), return an id/handle.
- `DELETE /api/settings/css/:id`: remove an imported CSS file; reject deletion for
linked (non-imported) entries.
- `POST /api/settings/css/link`: register a path/URL reference without copying
content; these are never deletable via the delete endpoint (only "unlink"-able,
if that distinction matters — clarify in implementation).
- Settings model tracks the list of CSS entries (imported + linked) with enough
metadata to distinguish the two kinds.
- Serve imported CSS content back so the frontend can `<link>`/inline it.
## Acceptance criteria
- Importing CSS persists content under the settings directory and survives restart.
- Deleting an imported CSS entry removes the file and the settings reference.
- Attempting to delete a linked (non-imported) entry is rejected.
- Linked entries resolve to their original path/URL, not a copy.
## Depends on
- [008 — Settings model](008-settings-model.md)
+30
View File
@@ -0,0 +1,30 @@
# 011 — NavMenu UI
## Summary
Build out the NavMenu frontend component for managing settings, presets, and CSS,
per the README's block diagram (NavMenu occupies its own column alongside the rest
of the dashboard).
## Scope
- Settings panel: colour scheme picker, CSS import (file/text input) + list of
imported/linked CSS with delete where applicable.
- Presets panel: list/create/switch/delete profiles ([009](009-presets-profiles.md)),
showing each profile's param/metric declaration counts and saved dashboards.
- Wire to the settings/profile/CSS APIs from
[008](008-settings-model.md)/[009](009-presets-profiles.md)/[010](010-css-management.md).
- Apply the active colour scheme / imported CSS to the app shell at runtime.
## Acceptance criteria
- Switching active profile in the UI changes engine state (declared params/metrics)
without a manual restart, or clearly communicates that a restart is needed if
that's the chosen design from [009](009-presets-profiles.md).
- Importing/deleting CSS in the UI is reflected immediately in the rendered app.
## Depends on
- [008 — Settings model](008-settings-model.md)
- [009 — Presets / profiles](009-presets-profiles.md)
- [010 — CSS import/management](010-css-management.md)
+27
View File
@@ -0,0 +1,27 @@
# 012 — Metrics UI
## Summary
Replace the current "Coming Soon" Metrics placeholder with a real component: declare
metrics via the metric language, and view their derived values.
## Scope
- A form/editor for declaring a metric (regex + param matchers) that calls
`POST /api/metrics` ([004](004-metrics-api.md)), with inline validation errors
surfaced from the API's 4xx responses.
- List of currently-declared metrics with their labels.
- A view of a selected metric's bucket values over the active time range (reuse the
time-range plumbing already in `App.tsx`/`LogTimeline`).
## Acceptance criteria
- Declaring a metric in the UI immediately reflects newly-matching logs (existing
and new) in the metric's value view, per the backfill behavior from
[003](003-metric-bucket-derivation.md).
- Invalid metric definitions show the backend's validation error instead of failing
silently.
## Depends on
- [004 — Metrics API](004-metrics-api.md)
+30
View File
@@ -0,0 +1,30 @@
# 013 — Dashboards: PromQL query support
## Summary
Support PromQL-style queries over declared metrics to derive dashboard
visualizations, per the README's Dashboards section ("There should be support for
promql queries to derive the visualisation").
## Scope
- Evaluate a useful PromQL subset (instant/range vector selection by metric name +
label matchers, at minimum; consider `rate()`/`sum()`/`avg()` aggregations as
stretch) against `Bucket` data ([003](003-metric-bucket-derivation.md)).
- Decide whether to hand-roll the subset or bring in an existing PromQL parser
crate — document the choice.
- New `EngineQuery` variant for PromQL evaluation, following the existing
single-owner-evaluates-in-place pattern.
- API endpoint (e.g. `GET /api/query?promql=...`) returning a chart-ready series
format (labels + time-value pairs), shaped for bar/line/pie consumption by
[014](014-dashboards-ui.md).
## Acceptance criteria
- A basic instant/range query against a declared metric with a label matcher returns
correct series data against a seeded fixture.
- Malformed PromQL returns a 4xx with a useful error.
## Depends on
- [004 — Metrics API](004-metrics-api.md)
+30
View File
@@ -0,0 +1,30 @@
# 014 — Dashboards UI
## Summary
Replace the current "Coming Soon" Dashboards placeholder with a Grafana-style
dashboard builder: bar, line, and pie charts backed by PromQL queries, saved as part
of a profile.
## Scope
- Chart components for bar/line/pie (pick a charting lib — none is currently in
`webapp/package.json`).
- Dashboard = a named set of panels, each with a PromQL query
([013](013-dashboards-promql.md)) and a chart type.
- Save/load dashboards as part of the active profile
([009](009-presets-profiles.md)), so they persist across restarts.
- Dashboard panels respect the app's active time range, same as LogTimeline/
LogPreview.
## Acceptance criteria
- Creating a panel with a PromQL query against a real declared metric renders the
correct chart type with live data.
- Dashboards persist across restart via the active profile.
- Panels re-query and update when the global time range changes.
## Depends on
- [013 — Dashboards: PromQL query support](013-dashboards-promql.md)
- [009 — Presets / profiles](009-presets-profiles.md)
+28
View File
@@ -0,0 +1,28 @@
# 015 — LogPreview: selectable parameter columns
## Summary
The README's LogPreview spec says it should be possible to pick which parameters are
visible as columns, alongside the always-visible sequence/timestamp. Currently
LogPreview only shows sequence, timestamp, and raw text.
## Scope
- Extend the logs API/DTO ([004](004-metrics-api.md) territory, but specifically the
existing `GET /api/logs`) to include derived param values per log line, sourced
from [001](001-param-declarations.md)'s declarations applied to that line.
- Frontend: column picker UI in/near LogPreview; selected columns persist (localStorage
is fine for a per-viewer preference, or fold into settings if it should be shared —
pick localStorage unless product direction says otherwise).
## Acceptance criteria
- Declared params with a match on a given log line appear as selectable, populated
columns in LogPreview.
- Column selection persists across a page reload.
## Depends on
- [001 — Param declarations](001-param-declarations.md)
- [004 — Metrics API](004-metrics-api.md) (for the pattern of extending log query
responses)
+26
View File
@@ -0,0 +1,26 @@
# 016 — FilterBar UI integration
## Summary
Wire the FilterBar frontend component to the real filter query language instead of
the current naive client-side substring match in `App.tsx`
(`logs.filter((l) => l.text.includes(filter))`).
## Scope
- Send the raw filter string to `GET /api/logs`/`GET /api/timeline` via the new
`filter` param from [007](007-filterbar-integration.md), instead of filtering
client-side.
- Surface parse errors from the API (4xx + message) inline under the FilterBar input.
- Remove the now-dead client-side `visibleLogs` substring filter in `App.tsx`.
## Acceptance criteria
- Typing a valid filter expression (regex + metric comparisons combined with
and/or) narrows both LogPreview and LogTimeline via server-side evaluation.
- An invalid expression shows an inline error and does not clear/break the
currently-displayed logs.
## Depends on
- [007 — FilterBar query integration](007-filterbar-integration.md)
+24
View File
@@ -0,0 +1,24 @@
# 017 — Ntfy.sh alerts (future idea)
> The README lists this under "Ideas... that can be implemented in the future, but
> not now." Not scoped for current implementation — tracked here so it isn't lost.
## Summary
Let users configure matchers of interest and an ntfy.sh channel, and send a
notification to that channel when a matching event occurs.
## Scope (sketch, not final)
- Matcher definition, likely reusing the FilterBar query language
([005](005-filterbar-pest-parser.md)) or the metric declaration language
([002](002-metric-declarations.md)) rather than inventing a third syntax.
- Ntfy channel config, persisted via settings ([008](008-settings-model.md)).
- Trigger evaluation on log ingest (engine-side, to avoid re-scanning) or on bucket
update, sending an HTTP request to the configured ntfy.sh channel.
## Depends on
- [008 — Settings model](008-settings-model.md)
- [005 — FilterBar pest parser](005-filterbar-pest-parser.md) (likely, for matcher
syntax reuse)
+28
View File
@@ -0,0 +1,28 @@
# 018 — Sequence diagram of events (future idea)
> The README lists this under "Ideas... that can be implemented in the future, but
> not now." Not scoped for current implementation — tracked here so it isn't lost.
## Summary
Let users declare sequence-diagram participants from params or matchers (e.g.
`node_id`), then render a sequence diagram of captured events of interest. No
relation between nodes is modeled — only event sequence/ordering determines the
diagram.
## Scope (sketch, not final)
- Participant declaration referencing an existing param
([001](001-param-declarations.md)), e.g. "one participant per distinct value of
`node_id`".
- Event-of-interest selection, likely reusing FilterBar syntax
([005](005-filterbar-pest-parser.md)) to define which log lines become diagram
events.
- Rendering: a sequence-diagram view (e.g. mermaid `sequenceDiagram`) driven by
ordered matching events per participant.
## Depends on
- [001 — Param declarations](001-param-declarations.md)
- [005 — FilterBar pest parser](005-filterbar-pest-parser.md) (likely, for
event-matcher syntax reuse)