Compare commits

...
48 Commits
Author SHA1 Message Date
Hugh Kaznowski 738e90c45c Tickets for FE and actor split 2026-09-07 14:54:21 +01:00
Hugh KaznowskiandClaude Sonnet 5 920629dbad Add a deterministic profile-actor wait primitive for tests (ticket 108)
Ticket 108 anticipated a large test-harness migration (primary-only/
profile-only helpers, migrating ~150 tests in engine/tests.rs), but
that turned out unnecessary: the profile actor reuses EngineSnapshot
as its own snapshot type (see ticket 104), so every existing
EngineSnapshot unit test already exercises primary-shaped and
profile-shaped behavior unchanged -- zero migration needed, confirmed
by the full suite passing untouched through tickets 100-107.

What was still a real gap: tests asserting on profile-actor-owned data
(declared metrics, alerts) after ingesting had no deterministic way to
wait for the profile actor to catch up -- EngineHandle::barrier() only
confirms the primary's queue drained, not that the profile actor has
processed what it published. They were passing on scheduler luck on
the single-threaded test runtime, not a real guarantee.

Added ProfileHandle::processed_sequence, a permanent (not test-only)
watch channel tracking the profile actor's watermark, mirrored onto
EngineHandle as profile_processed_sequence, with a
wait_for_profile_sequence(sequence) test helper built on it. This
replaces the ad hoc on_advance/spawn_observed mechanism ticket 103
added just to prove the wake loop worked -- that test now uses the
same real primitive. Retrofitted it into
promql_query_returns_series_for_seeded_fixture to eliminate its
latent race.

Not done: a dedicated test for the accepted eviction/lag tradeoff
(primary evicts on schedule regardless of profile-actor progress, per
idea 0001's resolved decision) -- deterministically forcing that race
without new test-injection hooks proved more involved than the
remaining scope justified; left as a known gap rather than shipping a
flaky or synthetic test for it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 14:04:26 +01:00
Hugh KaznowskiandClaude Sonnet 5 9322aac7e1 Route bare ottyl_seq/ottyl_ts Promql queries to the primary (ticket 107)
Audited every EngineQuery variant for the cross-snapshot join ticket
107 anticipated: Logs/Timeline/Stats/ParamDeclarations only ever read
primary data (raw text + param declarations, which never left the
primary), and MetricDeclarations/BucketValues/Promql only ever read
profile data -- no variant actually needs both, so no join/fan-out
layer is needed after all.

One real gap surfaced by that audit: a bare ottyl_seq/ottyl_ts Promql
query is evaluated by iterating self.logs_range() (evaluate_promql's
reserved-metric branch), but such queries were routed to the profile
actor's snapshot alongside real declared metrics -- and the profile
actor stores no raw log rows, so these queries silently returned empty
series instead of real data. EngineHandle::ask now parses the query
just far enough to check its metric name and routes ottyl_seq/ottyl_ts
to the primary instead, where evaluate_promql already works correctly
against real log data. Added an actor-level integration test
(promql_bare_ottyl_seq_works_through_the_real_actor_pipeline) that
fails without this fix -- verified by temporarily disabling the
routing check and confirming the test catches it.

Also deduplicated config/profile.rs's local `ask` test helper, which
had already once diverged from api::EngineHandle::ask's routing logic
and caused ticket 104 test failures (fixed then, but the duplication
itself was the root cause) -- it now delegates to the real one.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 14:01:29 +01:00
Hugh KaznowskiandClaude Sonnet 5 3ee6dddb0e Document EngineHandle's already-realized primary/profile routing (ticket 106)
Ticket 106 asked for a real way for API handlers to address the
primary vs. profile actor. That routing was already built out of
necessity while landing tickets 104/105 (metric queries had to reach
the profile actor's snapshot for those tickets' own tests to pass) --
EngineHandle ended up as one combined handle (option (b) from the
ticket's own scope note) that internally proxies metric/alert writes
to the profile actor and routes MetricDeclarations/BucketValues/Promql
reads to its snapshot, with no changes needed to any API handler,
reload_engine_declarations/activate_profile, or Profile::load_into_engine
-- they already just send EngineMessages the same way they did before
the split. mcp_server.rs talks to the HTTP API rather than EngineHandle
directly, so it needed no changes either. This commit documents that
resolution in handle.rs so a future reader doesn't go looking for
routing logic that isn't there.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 13:56:59 +01:00
Hugh KaznowskiandClaude Sonnet 5 3751508de8 Move alert evaluation to the profile actor (ticket 105)
Alert rules are profile-scoped and read metric buckets, which now live
on the profile actor (ticket 104) rather than the primary. alert_states
and the per-line evaluate_alerts/deliver_for_rule calls move from the
primary's Ingest arm to the profile actor's per-line loop, evaluated
against its own snapshot right after that line's apply_all_metrics
publishes -- matching the primary's original per-message publish-then-
evaluate ordering. The primary's run() no longer needs a SettingsHandle
at all now that alerts are gone from its Ingest arm; settings (for
alert rules and ntfy config, read fresh per line) moves to
engine::profile::spawn instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 13:55:51 +01:00
Hugh KaznowskiandClaude Sonnet 5 ce3e741cb0 Move metric declarations and bucket computation to the profile actor (ticket 104)
Metric declare/update/remove, backfill, and per-line apply_all_metrics
now run on the profile actor (engine::profile) instead of inline in the
primary's ingest closure, per idea 0001's resolved decision that the
primary handles params only. The profile actor reuses EngineSnapshot as
its own snapshot type -- declare_metric/update_metric/apply_all_metrics
work unchanged -- and mirrors the primary's param declarations into its
own snapshot on every wake (and synchronously before handling a
DeclareMetric/UpdateMetric message, since there's no ordering guarantee
between the watch-wake and message branches in tokio::select!) so
metric label/value resolution keeps working without storing a second
copy of raw log text; backfill reads lines through its cloned primary
IndexReader instead.

EngineHandle gained a profile_reader field so MetricDeclarations/
BucketValues/Promql queries route to the profile actor's snapshot
instead of the primary's -- without it, metrics could be declared but
never queried, since the primary no longer computes or stores them.
DeclareMetric/UpdateMetric/RemoveMetricDeclaration messages are still
sent to the primary's mpsc channel and proxied to the profile actor by
handle.rs's run() loop; EngineHandle's write-side API is otherwise
unchanged (ticket 106 gives API handlers a direct route). Alert
evaluation still reads the primary's (now metric-less) snapshot
directly -- an accepted gap until ticket 105 moves it, per idea 0001's
resolved decision that alerts move with metric computation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 13:53:06 +01:00
Hugh KaznowskiandClaude Sonnet 5 033e8bbab5 Stand up the profile actor skeleton and primary->profile watch signaling (ticket 103)
Adds a second actor task with its own left_right-backed ProfileSnapshot
and a tokio::sync::watch the primary's IndexWriter bumps after every
publish, so the profile actor wakes and re-reads newly-visible primary
lines through its own cloned IndexReader instead of polling. No metric
computation, alert evaluation, or query routing moves yet -- this is
pure plumbing, proven with a test that ingests a line on the primary
and observes the profile actor's watermark advance in response.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 13:39:19 +01:00
Hugh KaznowskiandClaude Sonnet 5 2c1aaa6e9c Break ticket 102's actor split into reviewable sub-tickets (103-108)
Ticket 102 (splitting the engine into reader/primary/profile actors)
was a single multi-thousand-line ticket covering a new actor
topology, an EngineHandle API redesign, a query fan-out layer, moving
alerts, and a full test-harness rewrite, with some design decisions
deliberately left open. Splitting it into six sequenced tickets
(profile actor skeleton + signaling, move metric computation, move
alert evaluation, split the handle API, query fan-out, test harness
rework) lets each land as its own reviewable commit instead of one
large speculative rewrite.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 13:33:01 +01:00
Hugh KaznowskiandClaude Sonnet 5 756e054527 Extract param extraction into its own EngineSnapshot::ingest_params stage (ticket 101)
Raw storage, log-stream header derivation, and auto-param detection
were inlined directly in the ingest actor's write closure alongside
metric computation. Pulling them into their own method gives the
ingest arm two clearly separated, independently-callable stages —
param extraction (ingest_params) and metric computation
(apply_all_metrics) — with no behavior change, in preparation for
splitting these onto separate actors (ticket 102).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 13:27:49 +01:00
Hugh KaznowskiandClaude Sonnet 5 f43445b280 Arc-ify stored log text to remove per-line double clones (ticket 100)
DataValue::LogLiteral and IngestedLine now carry an Arc<str> instead
of an owned String, so the two clones the ingest path paid per line
(once to move text into the write closure, once to store it) become
cheap refcount bumps instead of full string copies. Also cuts the
per-string-content cost of left_right's full-EngineSnapshot deep clone
on each reconciliation sync.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 13:26:03 +01:00
Hugh Kaznowski 5fee6909f2 Update actor split idea file 2026-09-07 13:14:19 +01:00
Hugh Kaznowski c7173cf9dc Update agents from repowise 2026-09-07 12:37:07 +01:00
Hugh KaznowskiandClaude Sonnet 5 dbe2ea9961 Add histogram_quantile() PromQL query for percentiles
The PromQL layer only had a fixed-width histogram() (per-bucket
counts), with no way to ask for p50/p95/p99. Adds
histogram_quantile(p, metric[range]), interpolating over each
series's retained raw sample values (linear interpolation between
closest ranks) rather than building a separate pre-aggregated
bucketed-histogram representation — exact for the data actually kept.
An optional bracketed range narrows the computation to a trailing
window, mirroring rate()'s range selector.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 12:34:00 +01:00
Hugh KaznowskiandClaude Sonnet 5 4a0fe92ce8 Add gauge/counter semantics to metric declarations (ticket 097)
Metrics previously only declared a data type (String/Number/...), so
rate() could be applied to a point-in-time gauge reading and sum()/avg()
could be applied directly to a cumulative counter reading with no guard
against either. Adds a MetricSemantics enum (Gauge, default; Counter)
threaded through DeclareMetricRequest, the engine's MetricDeclaration
data value, profile persistence, the API/MCP declaration routes, and the
Metrics panel's declaration form, then uses it in evaluate_promql to
reject rate() on a declared gauge and bare sum()/avg() on a declared
counter (suggesting rate() first). The guard only applies to metrics
with a captured value_param — bare occurrence-count metrics are always
safe under both operations, so existing dashboards are unaffected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 12:30:06 +01:00
Hugh KaznowskiandClaude Sonnet 5 c3108677ce Adopt rustfmt and clean up clippy warnings
Formats the whole backend to rustfmt defaults and fixes all 23
outstanding clippy warnings (collapsible ifs, clone-to-slice-ref,
type-complexity, single-arm match, unnecessary is_none check), then
wires cargo fmt --check and -D warnings into make lint so both stay
enforced going forward.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 12:21:38 +01:00
Hugh KaznowskiandClaude Sonnet 5 2cc8ac8138 Default JSON log-line viewer nodes to collapsed
A log line whose text parses as JSON was rendering every nested node
fully expanded, dumping large payloads into the timeline instead of a
collapsed summary the user opts into expanding.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 12:19:47 +01:00
Hugh Kaznowski 49bfd6b020 Add tickets and ideas 2026-09-07 12:16:36 +01:00
Hugh KaznowskiandClaude Sonnet 5 0c41048e0f Split lib/api.ts by REST domain and add buildSearchParams helper
api.ts had grown to 35 exported symbols in one 432-line file covering
every backend endpoint. Split into webapp/src/lib/api/{shared,logs,
params,metrics,dashboards,alerts,settings}.ts, re-exported from
lib/api/index.ts as a barrel so none of the ~10 importing files needed
their import paths changed (TS/bundler module resolution falls back to
a directory's index.ts when the sibling .ts file is gone).

Also added buildSearchParams to replace fetchLogs/fetchTimeline's
repeated hand-written `if (x !== undefined) search.set(...)` blocks.
Left fetchMetricBucketValues/fetchPromqlSeries's query-building
untouched (byte-for-byte) since the ticket scoped the helper to
fetchLogs/fetchTimeline only.

Reimplemented directly against current main (ticket 089) after the
worktree agent's branch was based on a stale pre-theme-removal state
and reintroduced CSS-upload endpoints main had deliberately deleted.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 10:54:47 +01:00
Hugh Kaznowski f323847748 Merge ticket branch t092 2026-09-07 10:47:01 +01:00
Hugh Kaznowski fc2f511d38 Merge ticket branch t090 2026-09-07 10:47:01 +01:00
Hugh KaznowskiandClaude Sonnet 5 8b2d512c94 Extract shared useRangeSelection hook from App.tsx's three page variants
StandaloneRanged, StandaloneLog, and FullApp each reimplemented the same
range/preset state machine (range/activePreset state, a rangeRef synced
via effect, selectPreset/handleManualRangeSelect, and rolling-preset
re-resolution at the top of the poll loop). Move that bookkeeping into
webapp/src/lib/useRangeSelection.ts; each component keeps only its own
polling logic.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 10:46:30 +01:00
Hugh KaznowskiandClaude Sonnet 5 28689b1b8d Extract paramBadgeSuffix helper from nested ternary chain
Replace the 4-deep nested ternary building each declared-param badge's
suffix text with a small if/else-if helper (paramBadgeSuffix), called
from the badge JSX. No behavior change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 10:44:24 +01:00
Hugh Kaznowski 1014aeaf68 Merge ticket branch t087 2026-09-07 10:41:34 +01:00
Hugh KaznowskiandClaude Sonnet 5 8008e5611e Extract shared SectionHeader and FormFooter components (ticket 087)
Params/Metrics/Alerts/Dashboards each hand-assembled the same collapsible-
section header wiring (role/tabIndex/aria-expanded/onClick/onKeyDown around
CollapseToggle+HelpIcon+PopOutButton), and Dashboards duplicated it a second
time for its no-active-profile branch. Params/Metrics/Alerts also duplicated
the declare/edit form footer (editing banner, submit label toggle, Cancel
button). Added SectionHeader and FormFooter to CollapsibleSection.tsx and
adopted them at all five call sites; pure extraction, no behavior change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 10:41:17 +01:00
Hugh Kaznowski 000efa5166 Merge ticket branch t088 2026-09-07 10:36:17 +01:00
Hugh Kaznowski 73627090a1 Merge ticket branch t091 2026-09-07 10:36:17 +01:00
Hugh KaznowskiandClaude Sonnet 5 c0174963d8 Extract useClickOutside hook, dedupe popover close-on-outside-click logic
RegexHelpIcon and LogPreview's ColumnPicker each reimplemented the same
mousedown-outside-ref-closes-popover effect; share it via a small hook.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 10:35:41 +01:00
Hugh KaznowskiandClaude Sonnet 5 4cc9205abf Move TimeRange types to lib/timeRange to break import cycle
CollapsibleSection.tsx and LogTimeline.tsx imported from each other
(TimeRange/TimeRangePresetId one way, sectionHeaderClassName the
other). Move the range types/helpers into a dependency-free
webapp/src/lib/timeRange.ts and repoint all importers there.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 10:35:05 +01:00
Hugh KaznowskiandClaude Sonnet 5 a47f771087 Extract shared declare/update-metric backfill and auto-param insertion helpers
declare_metric and update_metric duplicated the resolve-params +
backfill_metric + insert-MetricDeclaration sequence; extracted into
resolve_backfill_and_store_metric so the declare-vs-rename difference
is expressed only in caller-side logic. Also extracted
auto_declare_params's per-candidate insertion body into
try_insert_auto_param so the candidate-building and insertion steps
read as separate concerns.

Reimplemented directly against current main (ticket 085) after the
original worktree agent's branch was based on a stale pre-consolidation
snapshot module layout that no longer matches main.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 10:32:27 +01:00
Hugh Kaznowski cdd781c904 Merge ticket branch t083 2026-09-07 10:31:23 +01:00
Hugh Kaznowski d987663aaa Merge ticket branch t084 2026-09-07 10:31:23 +01:00
Hugh Kaznowski 157b4e7b1d Merge ticket branch t086 2026-09-07 10:31:23 +01:00
Hugh Kaznowski 94c52f8a17 Merge ticket branch t082 2026-09-07 10:31:23 +01:00
Hugh KaznowskiandClaude Sonnet 5 f67cfc5d94 Extract filter-compile and sort-by-param helpers out of handle_query_inner
The Logs and Timeline arms of handle_query_inner each duplicated the same
"compile filter, bail out on error" block; the Logs arm also inlined its
sort-helper-index walk. Pull both into named helpers (compile_optional_filter,
logs_sorted_by_param) so the Logs arm reads as "compile filter, then dispatch
to sorted or unsorted." Pure decomposition, no behavior change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 10:30:37 +01:00
Hugh KaznowskiandClaude Sonnet 5 564a0e8b2a Extract filter_value_as_f64 helper in compare_json_value
Flattens the Number arm's nested target-coercion match into a
one-line comparison, matching the shape of the String arm. No
behavior change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 10:29:28 +01:00
Hugh KaznowskiandClaude Sonnet 5 1b6c50075a Split cli::run() into named startup-phase functions
run() bundled settings loading, engine spawn, active-profile resolve/load,
ingest spawn, and session registration inline. Extract each into a named
function (spawn_engine, load_active_profile, spawn_ingest,
register_session_and_log) so run() reads as a sequence of steps. No
behavior change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 10:28:51 +01:00
Hugh KaznowskiandClaude Sonnet 5 4398acfa47 Flatten run_metrics_poll's nested match pyramid
Extract fetch_metrics_body(client, url) -> Result<String, String> to
handle the request/error_for_status/text chain with `?`, centralizing
the three warning messages. run_metrics_poll now matches once on the
result instead of nesting three match arms deep on the happy path.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 10:28:13 +01:00
Hugh Kaznowski f96de8f38d Compress impls 2026-09-07 02:04:36 +01:00
Hugh Kaznowski 55820982b1 Simplify test helper 2026-09-07 01:53:16 +01:00
Hugh KaznowskiandClaude Sonnet 5 d0ce524e95 Move approx_log_bytes into a flavour::LogLiteralHelper trait
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 01:24:06 +01:00
Hugh Kaznowski 82fa0dfeb4 Remove uploading css functionality. Themes are hard-coded. Change the code 2026-09-07 01:02:41 +01:00
Hugh Kaznowski 901dcf0a3f Makefile cleans profiling 2026-09-07 00:43:52 +01:00
Hugh KaznowskiandClaude Sonnet 5 1b2f690278 Consolidate EngineSnapshot's impl blocks into snapshot/mod.rs
They were scattered across filter/metrics/params/query/sort.rs by an
earlier file-splitting pass. Files can be split as much as needed, but
a single type's impl should never be split across files — so all 15
impl EngineSnapshot blocks now live in mod.rs, while each submodule
keeps its own types, free helper functions, and tests.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-07 00:43:38 +01:00
Hugh KaznowskiandClaude Sonnet 5 7df6026a21 Remove tickets 079/080 (no profiling evidence to justify the fix)
Both tickets explicitly scope themselves to "confirm via profiling
before implementing" (079: a trigram/token pre-filter for free-text
regex search; 080: caching parsed JSON per log line for JSON-field
filters). Neither has profiling data showing it's the dominant cost
once ticket 077's range-narrowing lands, and building either
speculatively risks unbounded memory (079) or penalizing the more
common non-JSON case (080) for no measured benefit. Revisit if
`samply` profiling against a realistic log volume shows these are
actually hot.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-06 20:51:29 +01:00
Hugh KaznowskiandClaude Sonnet 5 946fd50d53 Seek log/metric-bucket queries by key range instead of scanning
Log reads (logs_since, the unsorted Logs query path) now seek the
IngestedLog range directly from since_sequence/before_sequence instead
of scanning from IndexKey::Start and filtering every prior entry out
in a closure. Metric-comparison filters and bucket_values now seek to
a single metric's Bucket key range instead of scanning every declared
metric's buckets. evaluate_promql sits on top of bucket_values, so its
per-poll dashboard evaluation is narrowed by the same fix without
further changes; caching prior evaluations across polls is left for a
follow-up if profiling ever shows it's needed (per ticket 081's own
scope, not built speculatively).

Resolves tickets 077, 078, 081.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
2026-09-06 20:51:24 +01:00
Hugh Kaznowski 65330d37e0 Add tickets, ideas and samply ignore 2026-09-06 20:45:33 +01:00
Hugh Kaznowski d44c14a83b Improve ingestion performance 2026-09-06 19:02:44 +01:00
Hugh KaznowskiandClaude Sonnet 5 f19fa50a25 Split src/engine/mod.rs into handle/message/snapshot submodules
Pure code-motion refactor, no logic changes. Follows the repo convention
of one struct/enum + its impl per module file:

- engine/handle.rs: the left-right write/read plumbing (EngineOp,
  IndexWriter, IndexReadHandle, IndexReader, EngineHandle) and the actor
  loop (spawn/run) that owns EngineSnapshot.
- engine/message.rs: EngineQuery/EngineMessage/EngineResponse.
- engine/snapshot/: EngineSnapshot itself, split by concern into
  params.rs (declaration + auto-detect), metrics.rs (declare/apply/
  backfill), filter.rs (search-bar filter compilation/eval), sort.rs
  (sort-helper indexes), and query.rs (handle_query_inner + PromQL
  evaluation), with mod.rs holding the struct and the accessors shared
  across all of them.
- engine/tests.rs: the inline test module, kept as one file (per the
  task's own escape hatch) since most tests exercise ingest -> declare
  -> query across several of the new submodules at once rather than one
  in isolation.

Visibility was narrowed per call site: pub(super) where only sibling
snapshot submodules need an item, pub(crate) where engine::handle or
engine::tests (outside the snapshot subtree) also need it.

cargo build --all-targets and cargo test are green (291 passed, 1
ignored, same as before the split).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1ZJ3SrayzLzVndhVq9BmS
2026-09-06 18:36:53 +01:00
85 changed files with 10861 additions and 7365 deletions
+3
View File
@@ -1,3 +1,6 @@
/target /target
target/ target/
.claude/worktrees/ .claude/worktrees/
profile.json.gz
.codex
+88
View File
@@ -0,0 +1,88 @@
# AGENTS.md
<!-- Add your project instructions above or below the Repowise section. Repowise only updates the managed section between markers. -->
<!-- REPOWISE_AGENTS:START — Do not edit below this line. Auto-generated by Repowise. -->
## Codebase Intelligence for ottyl (Repowise)
Indexed by [Repowise](https://repowise.dev). Last indexed: 2026-09-07 (commit 920629d). Confidence: 100%.
### How to work in this repo
- **Trust the index.** `verified: true` and `_meta.complete` mean the bytes were checked against the live tree, so never re-read them. Re-read only what `bounds: "approximate"` or `_meta.stale_warning` names. `confidence` rates the prose, not the evidence: on `low` read the `fallback_targets` or `best_guesses` the reply names, and run `repowise update` and ask again if `_meta.hint` says the index is behind HEAD. `index_behind: true` alone is informational.
- **A zero carries its basis.** An empty `callers`/`callees`/`used_by` comes with a `*_basis` saying how much of that language's calls the graph resolved, so read it before concluding nothing calls a symbol. `_meta.scope_hint` names the areas the answer did not touch.
- **Pre-edit, not instead-of-edit.** These tools decide *which* files to read and edit. Reading a file before you edit it is correct and expected.
- **Noisy commands** (tests, builds, `git log`/`diff`, searches, listings): prefer `repowise distill <cmd>`, the same command with its exit code preserved and errors-first output. A `[repowise#<ref>: N lines omitted]` marker is recoverable via `repowise expand <ref>` (add `-q <regex>` to filter); never re-run the command to see omitted output.
- **Recording a decision** you had to reason out: `repowise decision add --title T --decision D` records it without prompting and prints the id (`--format json` to parse it back). It lands `proposed`, for a person to confirm.
### Tools
| Tool | When and why |
|------|--------------|
| `get_answer(question)` | First call for any how/where/why question. Cite `confidence: "high"` or `grounding: "extracted"` directly; `degraded` means judge by `retrieval_quality`. `symbol_bodies` has live bodies. |
| `get_context(targets=[...])` | Triage card for files/modules/symbols: docs, signatures, hotspot, fix history. No source bytes — `include=["skeleton"]` for the whole file verified, `["callers"|"decisions"]` for depth. Batch targets. |
| `get_symbol(id, depth?)` | **Follow-up, not an entry point** — one verified body for an id a prior response named (`path.py::Name`, `path.py:140-180`, `repowise#<hex>`). Never walk a file symbol by symbol; Read it. |
| `search_codebase(query)` | Hybrid search, auto-routed by query shape; force with `mode=symbol|path|concept|hybrid`. A hit whose `sources` are `[fts]` only has no semantic agreement, so verify it. |
| `get_why(query, targets?)` | Why the code is shaped this way: decision records, git archaeology, rationale comments. Call before a refactor or a pattern divergence. |
| `get_risk(targets, changed_files?, include?)` | File history and structural reach. PR mode leads with `directive`; its 0-10 structural heuristic is uncalibrated, not a probability. Read typed test recommendations and coverage state first. |
| `get_change_risk(revspec?, extensions?, exclude_patterns?)` | Deterministic live-diff review signal for a commit or range. Lead with benchmarked percentile/classification; the 0-10 diff-shape score is supporting, not a probability. `get_risk` scores paths. |
| `get_health(targets?, include?)` | Defect / maintainability / performance scores and findings. Self-check the files you touched before finishing. |
| `get_dead_code(tier?, min_confidence?, safe_only?)` | Confidence-tiered unreachable files / unused exports / zombie packages. For cleanup sweeps, not targeted fixes. |
| `get_overview()` | Architecture map. Call once, first, in an unfamiliar repo; skip it after that. |
### Architecture
**Files:** 81 | **Lines:** 18237 | **Import cycles:** 1
ottyl is a rust codebase of 81 files. Execution starts at src/main.rs, webapp/src/App.tsx, webapp/src/main.tsx. ---
*Built from the code's structure. It states what is there, not why it is that
way.
### Key modules
- `webapp/src` — webapp/src · webapp/src/components · webapp/src/lib · webapp/src/lib/api
**Language:** typescript | **Files:** 30 | **Public symbols:** 97…
- `webapp/src/lib` — webapp/src/lib · webapp/src/lib/api
**Language:** typescript | **Files:** 14 | **Public symbols:** 67 / 74
Covers the 14 source files in 2…
- `src/engine` — src/engine
**Language:** rust | **Files:** 14 | **Public symbols:** 122 / 531
Covers the 14 source files in src/engine
- `webapp/src/components` — webapp/src/components
**Language:** typescript | **Files:** 14 | **Public symbols:** 29 / 94
Covers the 14 source files in…
- `src` — src · src/api · src/cli · src/config · src/flavour
**Language:** rust | **Files:** 10 | **Public symbols:** 300 / 598
Covers the 10 source…
- `src/engine/snapshot` — src/engine/snapshot
**Language:** rust | **Files:** 6 | **Public symbols:** 0 / 139
Covers the 6 source files in src/engine/snapshot
- `webapp` — webapp · webapp/src
**Language:** typescript | **Files:** 3 | **Public symbols:** 1 / 19
Covers the 3 source files in 2 directories under…
### Entry points
- `src/main.rs`
- `webapp/src/main.tsx`
- `webapp/src/App.tsx`
### Files that need care (bug-fix history first, then churn — check `get_risk` before editing)
- `webapp/src/App.tsx` — 3 bug fixes, last fix today; 22 commits/90d
- `webapp/src/components/Dashboards.tsx` — 3 bug fixes, last fix yesterday; 17 commits/90d
- `webapp/src/components/Params.tsx` — 2 bug fixes, last fix yesterday; 15 commits/90d
- `webapp/src/components/NavMenu.tsx` — 1 bug fix, last fix today; 22 commits/90d
- `webapp/src/components/Docs.tsx` — 1 bug fix, last fix today; 17 commits/90d
### Code health
Three co-equal signals: defect risk 5.83/10 avg, hotspot health 4.96/10 (stable), worst `src/engine/snapshot/mod.rs` at 2.0/10 · maintainability 7.4/10 · performance risk 16 open static I/O-in-loop / N+1 findings. Detail: `get_health()`.
Critical files:
- `webapp/src/components/LogPreview.tsx` — untested hotspot — impact −2.0
- `webapp/src/lib/useSettingsPanel.ts` — untested hotspot — impact −2.0
- `webapp/src/components/NavMenu.tsx` — change entropy — impact −1.3
- `src/main.rs` — change entropy — impact −1.3
- `webapp/src/App.tsx` — change entropy — impact −1.3
### Commands
- Build: `make build`
- Test: `make test`
- Lint: `make lint`
- Dev: `make dev`
<!-- REPOWISE_AGENTS:END -->
+3 -1
View File
@@ -27,7 +27,7 @@ Backend: Rust (axum + tokio), single binary, in `src/`. Frontend: React 19 + Typ
**Data flow:** `main.rs` wires everything together at startup — loads `Settings`, spawns the engine actor task, replays the active profile's param/metric declarations into it (before ingest starts, since there's no way to reprocess earlier lines later), then starts ingest (stdin/file tailing and/or Prometheus polling) and the axum HTTP server. **Data flow:** `main.rs` wires everything together at startup — loads `Settings`, spawns the engine actor task, replays the active profile's param/metric declarations into it (before ingest starts, since there's no way to reprocess earlier lines later), then starts ingest (stdin/file tailing and/or Prometheus polling) and the axum HTTP server.
- **`engine.rs`** (largest file, ~3400 lines) — the actor owning all live state: ingested log lines, declared params/metrics, computed metric buckets, alert evaluation. Runs as a single tokio task; all access is through `EngineHandle` sending `EngineMessage` over an mpsc channel and awaiting a response via oneshot — there is no shared mutable engine state accessed directly from request handlers. This is the file to read first to understand how declarations, filtering, and bucketing actually work. - **`engine/`** — owns all live state, split across two actor tasks under `engine/actor/`: `actor/primary.rs` (ingested log lines, param declarations) and `actor/profile.rs` (metric declarations, computed metric buckets, alert evaluation — moved off the primary per `ideas/0001-actor-split.md`). Both are plain `tokio::spawn`ed loops; `actor/mod.rs` holds the `left_right`-based `IndexReader`/`IndexReadHandle` read plumbing they share. All access is through `EngineHandle` (defined in `actor/primary.rs`, re-exported from `engine/mod.rs`) sending `EngineMessage` over an mpsc channel and awaiting a response via oneshot — there is no shared mutable engine state accessed directly from request handlers. `engine/snapshot/` is the state (`EngineSnapshot`) both actors mutate; this is the place to start reading to understand how declarations, filtering, and bucketing actually work.
- **`api.rs`** — axum routes and HTTP-facing `ApiState` (holds `EngineHandle` + `SettingsHandle`). Translates HTTP requests into `EngineMessage`s and DTOs back into JSON. Also serves the embedded webapp. - **`api.rs`** — axum routes and HTTP-facing `ApiState` (holds `EngineHandle` + `SettingsHandle`). Translates HTTP requests into `EngineMessage`s and DTOs back into JSON. Also serves the embedded webapp.
- **`settings.rs` / `profile.rs`** — persisted configuration. A **profile** is per-app: parsing rules (param/metric declarations) and dashboards, saved so they're there again next time that app runs through ottyl (see README's "Profiles"). Declarations are persisted here so the engine can replay them on restart; `api.rs` handlers that mutate engine state (e.g. `declare_metric`) call a corresponding persist method here so the two stay in sync — this dual-write is a recurring pattern (see `ApiState::persist_param_declaration` / `persist_metric_declaration` in `api.rs`), and adding a new declaration-mutating endpoint means updating both sides plus the load-time replay path. - **`settings.rs` / `profile.rs`** — persisted configuration. A **profile** is per-app: parsing rules (param/metric declarations) and dashboards, saved so they're there again next time that app runs through ottyl (see README's "Profiles"). Declarations are persisted here so the engine can replay them on restart; `api.rs` handlers that mutate engine state (e.g. `declare_metric`) call a corresponding persist method here so the two stay in sync — this dual-write is a recurring pattern (see `ApiState::persist_param_declaration` / `persist_metric_declaration` in `api.rs`), and adding a new declaration-mutating endpoint means updating both sides plus the load-time replay path.
- **`filter_parser.rs`** (grammar in `filter_parser.pest`, via `pest`) + **`filter_datetime.rs`** — parse the search/filter bar syntax (regex matches, metric comparisons like `latency_ms[value]<200`, relative/absolute datetimes). - **`filter_parser.rs`** (grammar in `filter_parser.pest`, via `pest`) + **`filter_datetime.rs`** — parse the search/filter bar syntax (regex matches, metric comparisons like `latency_ms[value]<200`, relative/absolute datetimes).
@@ -43,3 +43,5 @@ Frontend structure: `webapp/src/App.tsx` is the shell; `webapp/src/components/`
- **`tickets/`** holds scoped, written-up work items (problem statement, pointers to exact call sites with `file.rs:line` references, and an explicit scope section) — check here for planned/in-flight work before starting something that might overlap. Tickets are removed from this directory once implemented (see recent commit history — e.g. "Remove ticket 047 (implemented: ...)"). - **`tickets/`** holds scoped, written-up work items (problem statement, pointers to exact call sites with `file.rs:line` references, and an explicit scope section) — check here for planned/in-flight work before starting something that might overlap. Tickets are removed from this directory once implemented (see recent commit history — e.g. "Remove ticket 047 (implemented: ...)").
- Declaration-related features (params, metrics) consistently need three coordinated changes: an `EngineMessage` variant + handler in `engine.rs`, a persistence method in `settings.rs`/`profile.rs`, and an `api.rs` route wiring them together (plus a webapp UI affordance). Grep for an existing similar declaration (e.g. how metric declaration is wired) as the template before adding a new one. - Declaration-related features (params, metrics) consistently need three coordinated changes: an `EngineMessage` variant + handler in `engine.rs`, a persistence method in `settings.rs`/`profile.rs`, and an `api.rs` route wiring them together (plus a webapp UI affordance). Grep for an existing similar declaration (e.g. how metric declaration is wired) as the template before adding a new one.
- Doc comments in this codebase tend to explain *why* a non-obvious choice was made (e.g. ordering constraints, a workaround, a past ticket's rationale) rather than restating what the code does — match that style rather than adding descriptive comments. - Doc comments in this codebase tend to explain *why* a non-obvious choice was made (e.g. ordering constraints, a workaround, a past ticket's rationale) rather than restating what the code does — match that style rather than adding descriptive comments.
- All markdown files (`*.md`) in this repo should follow one-sentence-per-line: each sentence starts on its own line, even within the same paragraph.
+4 -1
View File
@@ -23,7 +23,8 @@ test-fe:
lint: lint-be lint-fe lint: lint-be lint-fe
lint-be: lint-be:
cargo clippy --all-targets cargo fmt -- --check
cargo clippy --all-targets -- -D warnings
lint-fe: lint-fe:
cd webapp && npm run lint cd webapp && npm run lint
@@ -40,3 +41,5 @@ dev:
clean: clean:
cargo clean cargo clean
rm -rf webapp/dist webapp/node_modules rm -rf webapp/dist webapp/node_modules
rm profile.json.gz
+4 -6
View File
@@ -124,12 +124,10 @@ instance's active profile.
### Settings & theming ### Settings & theming
- **`get_settings`** / **`update_settings`** — read/update color scheme, active profile, - **`get_settings`** / **`update_settings`** — read/update the selected theme, active profile,
`max_logs`, custom theme, ntfy topic/server. `max_logs`, ntfy topic/server.
- **`list_css`** — list imported/linked custom CSS theme entries. - **`list_themes`** — list the names of the themes embedded in this build.
- **`import_css`** — import a CSS theme by file content. - **`get_theme_content`** — read a theme's raw CSS content by name.
- **`link_css`** — link an external CSS file/URL as a theme instead of importing content.
- **`remove_css`** / **`get_css_content`** — remove a CSS entry / read its raw content by id.
### Alerts ### Alerts
+317
View File
@@ -0,0 +1,317 @@
# Idea: split the engine's single actor into a log-ingestion worker + per-profile worker(s)
## Original note
> This needs to be fleshed out with ai.
>
> The general idea is that the primary engine ingesting worker is
> doing too much. It should only parse logs to extract params.
>
> Since profiles carry additional (non-derived) params, metrics,
> dashboards, and panels - those should be stored separately and can
> be handled by a different worker. The worker would have read access
> to the primary log sequence and derived params index, but would
> construct it's own profile perspective. It has write access and
> then
>
> The profiles need to be informed of new messages to index.
>
> What if message needs to be different format? Thats per-profile.
>
> System param suppressions?
>
> Regex indexing? We probably want text indexing on the primary while
> also
>
> Values in the primary index should be arcs, to prevent cloning
> allocations.
## Resolved decisions (2026-09-07)
Clarified interactively; supersedes the open questions raised below where
they conflict.
- **No profiling gate.** Proceeding on the code-reading evidence already in
this doc (double/triple string clones per line, full-`EngineSnapshot`
deep-clone on every `left_right` reconciliation) rather than waiting on a
`samply` profile first.
- **Three actors, not two.** The split is: **reader actor** (tails
stdin/file, splits into lines, timestamps, wraps each line in an `Arc`,
sends downstream) → **primary actor** (ingests timestamped `Arc` lines,
extracts declared + auto params only, maintains the `param=value` index —
no metrics, no profile logic) → **profile actor** (holds its own
`left_right` reader over the primary's index; builds metrics, custom
params, timeseries, and PromQL evaluation for the one active profile).
This resolves the "per-profile message format" question below: there is
no per-profile parsing fork on the primary — format/shape parsing of an
already-classified line is entirely the profile actor's concern, on top
of the shared raw text.
- **Still exactly one active profile at a time.** No concurrent
multi-profile indexing; the split changes *how* the one active profile's
data is computed relative to the primary, not the user-facing
single-active-profile model.
- **Alerts move to the profile actor.** Alert rules are profile-scoped and
read metric buckets, so they move with metric computation rather than
staying on the primary.
- **Primary→profile signaling:** the primary actor notifies subscribers
(e.g. a `tokio::sync::watch` carrying the latest processed
position/index) right after each `left_right` sync completes. The
profile actor wakes on that signal and re-reads whatever's newly visible
through its own `left_right` reader — no busy-polling, no second copy of
the data in transit.
- **Eviction wins over profile-actor lag.** If the profile actor is
behind (e.g. mid-backfill on a newly declared metric) when the primary
evicts old lines (`SetMaxLogs`), the primary evicts on schedule
regardless — the profile actor silently misses data for the evicted
range. Simplest option, matches today's single-actor retention model.
- **System param suppressions: dropped from scope.** Not needed right now;
revisit as a separate idea if it becomes a real requirement.
- **Migration path: incremental**, in this order:
1. Arc-ify stored values (`Arc<str>`/`Arc<String>` for log text and
metric-captured strings) — independent win, and the load-bearing
precondition for cheap cross-actor reads in steps 2–3.
2. Extract metric computation into a separately-callable stage, still
within the single actor.
3. Split into the reader/primary/profile actors described above.
## Current architecture (confirmed in code)
- **One process, one actor, today.** `spawn_engine`
(`src/cli/mod.rs:95`) calls `engine::spawn(...)` exactly once per
server run. `load_active_profile` (`src/cli/mod.rs:105-124`) loads
*one* active profile's param/metric declarations into that single
engine at startup, before ingest starts, with an explicit comment
explaining why: "there's no way to reprocess earlier lines later, so
this must happen before `spawn_ingest`" (`src/cli/mod.rs:104-106`).
All other `engine::spawn` call sites (`src/api/mod.rs:853,1459`,
several in `src/config/profile.rs`) are test harnesses, not a second
production engine instance.
- The actor loop is `run()`, `src/engine/handle.rs:130-365` — a single
`while let Some(message) = rx.recv().await` over one
`mpsc::Receiver<EngineMessage>` (`:141`). Every mutation (`Ingest`,
`IngestMetricSample`, `DeclareParam`/`RemoveParamDeclaration`,
`ClearParams`, `DeclareMetric`/`UpdateMetric`/
`RemoveMetricDeclaration`, `SetMaxLogs`, `EnsureSortHelperIndex`,
`Barrier` — full list `src/engine/message.rs:30-63`) funnels through
this one channel and one task.
- **Queries already bypass this actor.** `EngineQuery`
(`src/engine/message.rs:12-27`) doesn't go through the mpsc channel
— `EngineHandle::ask` reads directly off a `left_right`
`IndexReader`/`ReadHandle` (`src/engine/handle.rs:49-73`, entry point
`src/api/mod.rs:218-219`, evaluated by `handle_query_ro`,
`src/engine/snapshot/mod.rs:856-858`). So read-side scaling is
already decoupled from the actor loop today — this idea is entirely
about the **write/mutation** path.
- Per-line ingest (`EngineMessage::Ingest` arm,
`src/engine/handle.rs:143-219`) does, synchronously, in one
`left_right` write op, per line: ingest-lag/watermark bookkeeping,
log-stream header detection, storing the line as
`DataValue::LogLiteral`, auto-param detection
(`auto_declare_params`), running **every declared metric's regex**
against the line (`apply_all_metrics` →
`apply_metric_to_log`/`resolve_param_value`,
`src/engine/snapshot/mod.rs:409-460,821-851`), sort-helper-index
maintenance, log eviction, and then — after publish —
`alerts::evaluate_alerts` against the resulting snapshot
(`handle.rs:199-219`), which reads metric buckets that were just
computed in the same message.
- Module size: `src/engine/` is ~5.5k lines excluding tests (`snapshot/
mod.rs` alone is 1100 lines — the whole `EngineSnapshot` impl per its
own header comment).
- **No `Arc`, everywhere owned data.** The index is one flat
`BTreeMap<IndexKey, DataValue>` (`snapshot/mod.rs:43`) of owned
`String`s (`DataValue::LogLiteral { text: String }`,
`src/engine/data_value.rs:81`). Per ingested line, the raw text is
cloned at least twice for storage alone (`handle.rs:175,184-186`),
and every metric that matches the line clones its own copy of the
label/value strings it captures (`apply_metric_to_log`,
`mod.rs:426,439,444`) — with N metrics referencing overlapping
params, one line produces N independent owned-String allocations of
essentially the same substrings.
- **`left_right`'s reconciliation deep-clones everything.** After a
batch of writes, `left_right::Absorb::sync_with` clones the *entire*
`EngineSnapshot` (`*self = first.clone()`, `handle.rs:29-31`) to
reconcile the read-side copy — i.e. the "values should be Arc" note
isn't just about avoiding per-line clones, it's also about avoiding
a full-`BTreeMap`-of-Strings deep clone on every reconciliation.
- **No text/regex index exists.** `CompiledFilter::Regex(re) =>
re.is_match(text)` (`snapshot/filter.rs:159`) runs per line at query
time; `IndexKey` (`src/engine/index_key.rs:8-32`) has no
inverted/full-text structure. Every filter-bar query re-scans and
re-matches the whole retained range.
- **Backfill is a full synchronous history replay today, in the same
actor.** `declare_metric`/`update_metric`
(`snapshot/mod.rs:330-391`) → `backfill_metric` (`:481-509`) collects
**all** retained lines into a `Vec` and replays
`apply_metric_to_log` over every one, inside the single actor's
message handling — blocking ingest/alerts/other declarations for the
duration. `load_into_engine` (`src/config/profile.rs:332-384`) does
this once per metric, awaiting each response, so loading a profile
with N metrics against L retained lines costs O(N·L) synchronous
scan time. This is also the mechanism [[0004-profile-switch-reindexing-cost]]
describes for *runtime* profile switching via the `activate_profile`
API handler (`src/api/mod.rs:712-768`, which removes then
re-declares via the same messages) — worth reconciling that this
research pass separately characterized profile activation at
startup as one-shot ("no way to reprocess earlier lines later"),
while the earlier research for ticket 099/idea 0004 confirmed a
*runtime* reload path exists via `activate_profile`. Both are true:
startup load is one-shot by necessity (ingest hasn't started yet),
and the runtime reload path reuses the same declare/backfill
machinery after the fact — but this is worth double-checking before
design work starts, since the two research passes framed it
differently.
- **Alerts are coupled to the single actor's local state.** Alert
evaluation runs directly in the ingest arm against the just-mutated
snapshot (`handle.rs:199-211`, `alerts::evaluate_alerts`,
`src/engine/alerts.rs:211`), reading `alert_states:
HashMap<String, AlertRuntimeState>` that lives on the single actor
task's own stack frame (`handle.rs:137`) — there's no mechanism today
for one actor to read another actor's in-memory state.
- **Existing multi-task precedent is producer-side only.**
`src/engine/ingest.rs`'s stdin/file-tailing and Prometheus-polling
tasks each hold a cloned `EngineHandle` and push into the *same*
actor's mpsc channel (spawned at `src/cli/mod.rs:136-145`) — a
plausible pattern for "many producers, one consumer," but there's no
existing precedent for multiple *consuming* actors each owning their
own `left_right` write side that need to stay coherent with each
other or with a shared upstream.
## Implications and things to consider
### Performance
- The stated goal — "the primary worker should only parse logs to
extract params" — would remove per-line metric-regex evaluation
(`apply_all_metrics`) from the primary's critical path, which is
real work today (N metrics × regex match per line). That's a
genuine win *if* profile workers can consume the primary's output
without re-paying an equivalent cost themselves — but since a
profile worker still has to run its own metrics' regexes against
each line to build its own buckets, the total CPU work done across
the process doesn't shrink; what changes is that it's no longer all
serialized on one task, so it can run concurrently with the primary
(a real win on a multi-core box) and a slow/backfilling profile
worker no longer blocks raw ingestion.
- The "values in the primary index should be Arcs" change is
independently valuable regardless of whether the actor split
happens — it directly addresses the confirmed double/triple string
cloning per line (storage clone + per-metric label/value clones) and
the full-snapshot deep-clone on every `left_right` reconciliation.
Worth sequencing as a prerequisite: switching to `Arc<str>` (or
`Arc<String>`) for stored text means a profile worker reading "the
primary log sequence" can cheaply hold references to the same
underlying bytes instead of needing its own copies — this is likely
a load-bearing precondition for the split being worth it at all, not
a nice-to-have alongside it.
- A profile worker needing "read access to the primary log sequence
and derived params index" implies either (a) a second `left_right`
reader over the primary's structures (cheap, if values are Arc'd —
see above) or (b) a message-replay feed (the primary notifies
profile workers of new lines, per the note's "profiles need to be
informed of new messages to index"). Option (b) reintroduces a
second data copy per profile worker unless the message itself only
carries an `Arc`/index reference rather than owned data — otherwise
this recreates the same N-copies-per-line problem this idea is
trying to remove, just distributed across processes/tasks instead of
within one.
- Backfill cost (O(N metrics × L retained lines), see [[0004-profile-switch-reindexing-cost]])
doesn't disappear with a split — it moves to whichever worker owns
the profile. The upside is it no longer blocks the *primary's*
ingest/query-serving while it happens, which is the main perf
argument for doing this at all, per that idea's "Performance"
section.
- Full-text/regex indexing on the primary (per the note's "Regex
indexing?") is a materially different performance investment from
the Arc/split work — an inverted or trigram index trades ingest-time
CPU and memory for query-time speed on filter-bar searches, and is
worth scoping as its own effort (see [[0005-additional-read-indexes]])
rather than folded into this split, since it doesn't depend on the
actor topology.
### Code structure
- This is the single largest structural change on the idea list —
`src/engine/` (~5.5k lines) currently assumes one writer owning one
`left_right` pair. Splitting introduces: a defined "primary
perspective" (raw logs + derived/auto params only, no
profile-specific metrics) as its own bounded module, a
"profile perspective" abstraction for the one active profile, a
notification/subscription mechanism from primary to the profile actor
(the note's "profiles need to be informed of new messages to index" —
doesn't exist in any form today; the closest analogue, `ingest.rs`'s
producer tasks feeding one consumer, runs the opposite direction).
**Resolved** (see "Resolved decisions"): the primary signals via e.g. a
`tokio::sync::watch` after each `left_right` sync completes, and the
profile actor re-reads through its own `left_right` reader rather than
the primary pushing data. A query-fan-out/merge layer is still needed
for any request
that needs both raw logs and profile-derived data in one response
(today `Logs`/`Timeline` queries read both from a single
`EngineSnapshot` in one pass, e.g. `query.rs:154`'s
`log_entry_params` call against `compiled_params` from the same
snapshot — splitting the data sources means this becomes a real join
across two snapshots/workers instead of one borrow).
- Alert evaluation (`handle.rs:199-211`) currently depends on running
immediately after metric bucketing in the same message, against the
same snapshot. **Resolved** (see "Resolved decisions"): alerts move
with metric computation into the profile actor, since alert rules are
profile-scoped and read the metric buckets computed there.
- `EngineHandle`'s current API (`handle.rs:49-73` for reads,
the mpsc `EngineMessage` send for writes) would need to become two
APIs, or one handle type parameterized by "primary" vs. "profile" —
worth deciding whether `api.rs`'s handlers (which currently hold one
`EngineHandle` in `ApiState`) address a single combined handle that
internally fans out, or explicitly juggle a primary handle plus an
active-profile handle.
- Given `CLAUDE.md`'s existing note that `engine.rs` is "the file to
read first" and already the largest/highest-churn file in the repo
(13 commits/90d per the repowise summary), this split is also a
legibility opportunity, not just a performance one — today a reader
has to understand ingest, metrics, alerts, and backfill as one
intertwined concern to understand any one of them.
### Functionality
- **Per-profile message format** — **resolved**, see "Resolved decisions"
above: there's no per-profile parsing fork on the primary. Log-stream
header detection and auto-param extraction stay a one-time, primary-side,
profile-agnostic concern (`derive_log_stream_params_from_row`,
`auto_declare_params`); any further message-shape interpretation the
profile actor needs happens independently, on top of the shared raw
`Arc` text, in the profile actor itself.
- **System param suppressions** — **dropped from scope** (see "Resolved
decisions"). Revisit as a separate idea if it becomes a real need.
- **Concurrency** — **resolved**: still strictly one active profile at a
time. The split changes *how* that one profile's data is computed
relative to the primary, not the user-facing model — no multi-profile
concurrent indexing.
### Not yet considered
- **Failure/restart semantics** — partially resolved: the profile actor
falling behind the primary during a backfill is expected and accepted
(see "Resolved decisions" — eviction wins over lag, silent gaps for the
evicted range). Still open: what happens if the profile actor *crashes*
outright (not just lags) — does it restart and replay from whatever the
primary currently retains, or is there a persisted checkpoint? Not yet
decided; likely fine to punt until the split is actually implemented,
since "restart and rebuild from currently-retained history" is a
reasonable default consistent with the accepted-gaps decision above.
- **Retention/eviction coordination** — **resolved**, see "Resolved
decisions": no low-water-mark. The primary evicts on its own schedule
(`evict_excess_logs`, `handle.rs:195`, `SetMaxLogs`) regardless of the
profile actor's progress; a lagging profile actor silently misses data
for the evicted range rather than blocking primary eviction.
- **Testing story**: `src/engine/tests.rs` (2348 lines) and the
`test_handle` pattern (see ticket 099) are built entirely around one
actor with one channel — a split roughly doubles the kinds of
integration tests needed (primary alone, profile worker alone,
primary+profile interaction) and the existing test harness would
need real rework, not just extension. Still open — no resolution yet.
- **Migration path** — **resolved**, see "Resolved decisions": incremental,
in three steps (Arc-ify stored values → extract metric computation into
a still-single-actor stage → split into reader/primary/profile actors).
- **Whether this is worth it at all without profiling evidence** —
**resolved**, see "Resolved decisions": proceeding without a `samply`
profile, on the code-reading evidence already documented in this file.
@@ -0,0 +1,158 @@
# Idea: dependent/nested param extraction and jq-style JSON extraction
## Original note
1. A param currently auto-detects or is declared via a regex against
the raw log line — that's the baseline, already true today.
2. Semantics for "has a value": if the message has a non-empty or
missing capture for the target group, treat that as the param
having a value (as opposed to today's behavior — needs
confirmation, see Open questions).
3. Extract a param from *another param's already-captured string*,
applying a second regex on top of it, rather than always
re-matching the raw line.
4. When a string param (or the whole message) is JSON, extract fields
via `jq` syntax rather than the current plain dot-path.
## Current state (confirmed in code)
- `ParamDeclaration` (`src/engine/data_value.rs:6-20`) supports exactly
one regex + one named capture group per param, or one JSON
dot-path, or the computed `elapsed_since_last` — these three modes
are mutually exclusive (validated in
`src/engine/snapshot/mod.rs:618-633`).
- `extract_param_raw` (`src/engine/snapshot/params.rs:85-99`) *always*
operates on the raw ingested line's text — there is no "source
param" concept anywhere in `ParamDeclaration`/
`ParamDeclarationRecord`/`ParamDeclarationDto`. Every param
independently re-parses the same raw line; none can chain off
another param's captured value.
- JSON extraction (`extract_json_field`, `src/engine/data_value.rs:22-39`)
requires the whole line to parse as a JSON object, and walks a plain
dot-separated path through nested objects only — no array indexing,
no jq filters/pipes/functions. Frontend surface:
`webapp/src/components/Params.tsx:24-33` (gates the "JSON field"
option on `looksLikeJsonObject`), free-text path input, no
path-builder/autocomplete.
- No `jq` crate or jq-like dependency exists in the codebase today.
## Why this is an idea, not a ticket yet
This is a real feature addition with several open design questions
that change its shape and scope significantly — it's not a
well-bounded bug fix or mechanical change.
## Open questions
- **Chaining model**: should a param declaration gain an optional
"source param" field (extract from another param's value instead of
the raw line)? If so, declarations form a small dependency graph —
what happens on a cycle, or when the source param has no value for a
given line (does the dependent param also have no value, silently)?
Does extraction order need to become topological instead of the
current independent-per-declaration model?
- **Missing/empty capture semantics**: what should currently happen
when a regex matches but the named group didn't participate (e.g. an
optional group in an alternation)? Worth checking
`extract_param_raw`'s exact behavior before assuming this needs a
change — it may already do the right thing.
- **jq scope**: is full jq syntax the actual goal, or would extending
the existing dot-path extractor with array indexing (`items[0].id`)
cover most real cases far more cheaply than pulling in a jq
implementation (e.g. the `jaq` crate) as a new dependency? jq's
filter language (pipes, `select()`, `map()`) is a much bigger surface
to expose safely in a declaration UI than a path string.
- **UI shape**: `Params.tsx`'s declaration form is built around one
regex/one capture group or one JSON path; a chained/jq model needs a
materially different UI (e.g. "extract from: [raw message ▾] or
[param: session_id ▾]" plus a jq expression box with live preview
against a sample line).
## Suggested next step
Worth a short design pass (e.g. `ce-brainstorm` or a quick
conversation) to settle the chaining model and jq-vs-extended-dot-path
question before this gets broken into tickets — the two features (2/3
and 4) could also ship independently rather than as one big change.
## Implications and things to consider
### Performance
- Chained extraction means a line can now do N regex/JSON passes
instead of 1 per param, and a dependent param's cost is paid *after*
its source param already ran — worst case, a profile with a long
dependency chain multiplies per-line ingest cost by chain depth.
This lands directly in the single actor's per-line ingest path (see
[[0001-actor-split]] — if that split happens first, param extraction
is exactly the workload it isolates, so it's worth sequencing this
after that decision rather than before).
- A `jq`-style engine (even a Rust one like `jaq`) is meaningfully
more expensive per invocation than a compiled `regex::Regex` capture
— evaluating a jq filter against a parsed `serde_json::Value` for
*every* ingested line for *every* JSON-extraction param is a
different cost profile than today's simple dot-path walk. Worth
benchmarking before committing to full jq rather than assuming it's
free.
- Topological ordering of dependent params (if declarations form a
DAG) adds a per-declaration-change cost (recompute order) but not a
per-line cost, as long as the order is cached after each
add/remove/edit rather than recomputed per line.
### Code structure
- `extract_param_raw` (`src/engine/snapshot/params.rs:85-99`) currently
has a flat signature `(text: &str, declaration) -> Option<String>`.
Chaining requires either passing in the full set of already-extracted
param values for the current line (so a declaration can look up its
source param's value) or restructuring extraction as a fold over
declarations in dependency order — a real shape change to this
function and its caller, not a param-by-param tweak.
- The "three mutually exclusive modes" validation in
`src/engine/snapshot/mod.rs:618-633` needs a fourth mode (or an
orthogonal "source" field crossed with the existing regex/JSON
modes) — worth deciding whether source-param and JSON-field are
independent axes (e.g. "JSON-extract from param X" as well as from
the raw line) since that combination is implied by note #3.
- This is exactly the shape of change flagged in this repo's
`CLAUDE.md` as needing three coordinated updates (engine message +
handler, settings/profile persistence, api.rs route) plus a frontend
affordance — touches more surface area than the "Current state"
section alone suggests.
### Functionality
- Cycle detection isn't optional once chaining exists — a user could
declare param B from param A and (accidentally, via UI) param A from
param B. Needs either a hard validation error at declare-time or a
documented "first N declarations in insertion order" fallback.
- Auto-derived params (JSON auto-flattening,
`flatten_json_object`/`flatten_json_object_into`,
`src/engine/data_value.rs:41-60+`) aren't user declarations — decide
whether a chained/jq param can source from an *auto-derived* param,
which would tie this feature to auto-detection internals that
weren't designed to be a stable "source" API.
- Backfill: declaring a new dependent param triggers the same
`backfill_metric`-style history replay as any param declaration
(see [[0004-profile-switch-reindexing-cost]] for the general
backfill-cost concern) — a chain of N dependent params means N
passes over retained history on declare, not just N passes per new
line going forward.
### Not yet considered
- Error/partial-failure UX: what does the UI show for a param whose
jq expression is syntactically valid but fails on a particular
line's JSON shape (e.g. missing key, type mismatch) — silent
no-value, or a visible per-line extraction error surfaced somewhere?
- Whether jq's more powerful operations (`map`, `select`, arithmetic)
should be allowed at all in a param *value* context, where the
result is expected to be a single scalar — most of jq's power is
about reshaping/filtering collections, which doesn't map cleanly
onto "this param has one value per log line."
- No investigation yet into how this interacts with `sort_by`
(`SortHelperKey`) or PromQL — do chained/jq-derived params get the
same downstream treatment (sortable, usable in metric declarations,
usable in dashboard queries) as directly-extracted params, or are
there gaps once the extraction pipeline is no longer "one regex
against the raw line"?
+117
View File
@@ -0,0 +1,117 @@
# Idea: timeline ordering — confirm/harden the sequence-is-primary invariant
## Original note
"The timeline is time ordered. That is always derived from the
sequence ordering. That means the primary sequence is simply sequence
ordered." — reads as a statement of intent/invariant rather than a bug
report, so it's unclear whether this needs a code change or is just
confirming existing behavior.
## Current state (confirmed in code)
- `IndexKey::IngestedLog { sequence, timestamp }`
(`src/engine/index_key.rs:10-13`) derives `Ord` with `sequence` as
the primary field and `timestamp` only as a (practically unused)
tie-breaker, since `sequence` is already unique.
- `sequence` is a strictly monotonic counter incremented per ingested
line (`src/engine/ingest.rs:13,52`); `timestamp` is a separate
wall-clock `Timestamp::now()` read at the same point in the ingest
loop (`src/engine/ingest.rs:18,93`) — not derived from `sequence`.
- The default log listing path (`logs_in_sequence_range`,
`src/engine/snapshot/mod.rs:180-208`, used by the `Logs` query in
`src/engine/snapshot/query.rs:118-172`) is explicitly sequence-order,
with an existing comment documenting this
(`src/engine/snapshot/mod.rs:176-179`).
- The only other ordering mode is `sort_by` (sort by a declared param's
value, sequence as tie-break, via `SortHelperKey`,
`src/engine/snapshot/sort.rs`) — there's no "sort by timestamp" mode
distinct from sequence order.
- **The one place ordering is genuinely timestamp-driven**: the
`Timeline` query (`src/engine/snapshot/query.rs:173-198`) buckets
logs by `timestamp.as_millis() / bucket_ms`
(`:191`), not by sequence.
## Why this is an idea, not a ticket
The invariant the note describes already appears to hold for the
default log listing — sequence is the primary and effectively only
ordering key there. The one place `timestamp` (not `sequence`) drives
grouping is the `Timeline` bucketing, and `sequence`/`timestamp` are
captured independently rather than one being derived from the other.
In the current single-threaded ingest loop they're expected to
co-vary, but nothing guards against a backward wall-clock step (NTP
correction, clock skew) between two `Timestamp::now()` calls — if that
ever happened, sequence-ordered views and timestamp-bucketed views
could disagree on relative order for the affected lines. This has not
been observed as an actual bug; it's a latent gap worth a decision.
## Open question
Is the ask just "confirm this holds" (in which case: it does, for the
default listing — no code change needed), or do you want the
`Timeline` bucketing hardened so it can never diverge from sequence
order even under clock skew (e.g. clamp each line's timestamp to be
`>= previous line's timestamp`, or derive bucket assignment from
sequence position rather than raw wall-clock read)? The fix is small
either way, but scoping it as a ticket needs that decision first.
## Implications and things to consider
### Performance
- Clamping `timestamp` to be monotonic (`max(this_line_ts,
previous_line_ts)`) is O(1) per line — negligible cost, just one
extra comparison in the existing ingest loop
(`src/engine/ingest.rs:13-18,52,93`). Deriving bucket assignment from
sequence instead of wall-clock would be a larger change (see below)
but isn't inherently more expensive at ingest time.
- No performance concern either way — this is a correctness/consistency
question, not a hot-path cost question.
### Code structure
- If bucket assignment moves from wall-clock `timestamp` to something
sequence-derived, `Timeline`'s bucketing
(`src/engine/snapshot/query.rs:173-198`) still fundamentally needs a
*time value* per bucket (users read timeline buckets as "logs between
10:32:00 and 10:32:05", not "logs 100-150") — sequence can't replace
timestamp as the bucketing axis, only guard it. The realistic fix is
the monotonic-clamp approach, not a switch to sequence-based
bucketing.
- Clamping at ingest time (`ingest.rs`) is the natural single place to
fix this, rather than patching every read path that consumes
`timestamp` — confirm there's exactly one ingest entry point today
(there appear to be two: stdin/file tailing and Prometheus metrics
polling, per `src/engine/ingest.rs` — a clock-skew guard needs to
apply to both, and they'd need to share one "last observed timestamp"
reference point to clamp against consistently).
### Functionality
- A clamped timestamp is no longer "true" wall-clock time for that
line — if any UI surface displays a log's literal ingest timestamp
(as opposed to just using it for bucketing), clamping would make a
displayed timestamp lie slightly during a clock step-back event. Low
probability, but worth deciding whether clamping is silent or
surfaced (e.g. a flag on the log entry noting its timestamp was
adjusted).
- This only matters at all during an actual backward clock step, which
in most deployments (a piped CLI process, not a long-running server
through NTP corrections) may never occur in practice — worth weighing
whether this is worth any code change versus documenting it as an
accepted limitation.
### Not yet considered
- No investigation into whether the same two independently-captured
timestamp/sequence pattern exists anywhere else besides ingest (e.g.
alert evaluation timestamps, or the Prometheus-polling ingest path
specifically) — the research so far only confirmed the stdin/file
ingest loop.
- No check on how a forward clock jump (not just backward) would
interact with `Timeline` bucketing — a large forward jump would
create a sparse/empty bucket range, which is a different (probably
more common) failure mode than the backward-step case this idea
focuses on, and might be worth addressing in the same pass if one
happens.
@@ -0,0 +1,142 @@
# Idea: profile switching triggers a full synchronous re-index per metric
## Original note
"Metrics and dashboards are tied with the profile. However, they are
indexed on the engine. Switching profiles means re-indexing." — a
description of a real coupling; unclear whether it's being flagged as
a performance problem to fix or just an architectural note to be aware
of.
## Current state (confirmed in code)
- `Profile` (`src/config/profile.rs:309-318`) is pure config — params,
metrics, dashboards — with no live engine state of its own.
`Settings::set_active_profile` (`src/config/profile.rs:489-502`)
only flips which profile is active and persists to disk; it doesn't
touch the engine.
- The actual switch side effect lives in the API layer:
`activate_profile` (`src/api/mod.rs:746-768`) calls
`reload_engine_declarations` (`src/api/mod.rs:712-744`), which:
1. Removes every current param/metric declaration from the engine
(`EngineMessage::RemoveParamDeclaration`/`RemoveMetricDeclaration`,
skipping auto-derived params).
2. Calls `profile.load_into_engine(engine)`
(`src/config/profile.rs:332-384`), re-declaring every param/metric
in the newly-activated profile.
- There is no dedicated `EngineMessage` for "switch profile" — it's
entirely remove-everything-then-redeclare-everything via the
existing declare/remove messages
(`src/engine/message.rs:30-63` has no such variant).
- Each metric redeclare triggers a full rebuild: `declare_metric`
(`src/engine/snapshot/mod.rs:330-335`) →
`resolve_backfill_and_store_metric` → `backfill_metric`
(`:481-509`), which replays **all retained log history**
(`self.logs_range()`) through `apply_metric_to_log` for that metric.
Raw ingested lines themselves are never cleared on switch — only
metric/param declarations and their derived buckets are dropped and
rebuilt.
- This all happens synchronously, one metric at a time, inside the
engine's single-writer actor loop
(`src/engine/handle.rs:303-338`) — so switching profiles blocks that
loop for a duration that scales with retained log volume × number of
metrics in the newly-activated profile.
## Why this is an idea, not a ticket
Whether this is worth fixing depends on facts not available from
reading code alone: typical retained log volume in real usage, typical
metric count per profile, and how often users actually switch profiles
mid-session. Without knowing the current switch latency is actually
noticeable, a fix would be speculative optimization.
## Options if this turns out to matter
- Diff-based reload: only remove/re-add metrics whose *definition*
actually changed between the old and new profile, instead of
removing everything and redeclaring everything (a metric present
unchanged in both profiles currently still gets a full remove +
backfill for no reason).
- Make the removal/backfill sequence async relative to the HTTP
response, with the UI showing a "reindexing…" state rather than
blocking the request (and the engine loop) until complete.
- Lazily backfill only the time range currently being viewed, deferring
the rest — a bigger change, and only worth it if the switch cost is
dominated by backfill work the user doesn't immediately look at
anyway.
## Suggested next step
Before scoping a ticket: confirm with a profile that has a realistic
number of metrics and log volume whether switch latency is actually
perceptible. If it is, the diff-based reload (first option) is the
cheapest fix and a reasonable first ticket.
## Implications and things to consider
### Performance
- Cost today is O(retained lines × metrics in new profile), done
synchronously in the single-writer engine actor
(`src/engine/handle.rs:303-338`) — every other engine message
(ingest, queries, alert evaluation) queues up behind a profile switch
until it finishes, so the blast radius isn't just "the switch feels
slow", it's "the whole app appears frozen" for that duration,
including live ingestion of new lines arriving mid-switch.
- The diff-based reload option only helps when profiles share metrics
— if switching to a profile with a mostly-disjoint metric set (the
common case for genuinely different apps/profiles, which is the
whole point of profiles), it doesn't reduce the backfill cost much.
It's a bigger win for "tweak one metric in the currently active
profile and reactivate" workflows than for switching between
unrelated profiles.
- If [[0001-actor-split]] happens, this problem's shape changes
significantly — a per-profile worker rebuilding its own perspective
from a shared primary index could do that rebuild without blocking
ingestion or query-serving on the primary at all, which addresses the
"whole app freezes" part of this problem as a side effect, even if
the total backfill work stays the same. Worth reading that idea
alongside this one.
### Code structure
- The remove-then-readd flow lives entirely in the API layer
(`src/api/mod.rs:712-768`), not the engine — a diff-based reload
would need `reload_engine_declarations` to compute a diff (metrics
present in old profile but not new → remove; new but not old →
declare; present in both — compare full declaration equality, not
just name, to decide if a changed regex/threshold needs a rebuild)
before sending any engine messages. That diff logic is new code, not
a variant of existing code.
- Async/non-blocking reload would need a new `EngineMessage` variant
(today there is none for "switch profile" — it's just existing
declare/remove messages) if the goal is to let the engine
acknowledge the switch request immediately and do backfill work
in the background while still serving other queries — this is a
bigger structural change than diff-based reload, since the engine's
single-actor model currently has no notion of "background work
interleaved with message handling."
### Functionality
- If reload becomes async/background, the UI needs a defined
"reindexing in progress" state — what does a dashboard panel show
for a metric that's declared but not yet backfilled? An empty chart
is misleading (looks like "no data") versus a genuinely no-data
metric.
- Partial-failure semantics: if backfill for metric 3 of 5 fails or is
interrupted (e.g. app restart mid-switch), what state is left behind?
Today's synchronous approach has no partial-state problem because it
either completes or the whole process is still running; async reload
introduces a new failure mode that doesn't exist today.
### Not yet considered
- No data gathered yet on real profile sizes/log volumes in actual
usage (flagged already in "Why this is an idea") — this entire idea
is speculative until that's known.
- No consideration of whether profile switching happens automatically
(e.g. auto-detected based on the piped command) or is always a
manual user action — if it's always manual and rare, the UX cost of
a blocking-but-rare operation may be perfectly acceptable and not
worth any of the above complexity.
+128
View File
@@ -0,0 +1,128 @@
# Idea: additional indexes to speed up reads
## Original note
"Ask AI to come up with additional indexes that will help speed up
reads." — an open-ended research request rather than a scoped
change; this note captures what's already known so a future deep-dive
doesn't start from zero.
## Current index surface (confirmed in code)
- `IndexKey` (`src/engine/index_key.rs`) — the engine's indexing key
space includes at least `IngestedLog { sequence, timestamp }`,
`Bucket` (metric bucket data), and `LastMetricEvent`. Ordering is
primary-`sequence` for logs (see
[[0003-timeline-ordering-invariant]] for detail).
- `SortHelperKey` (`src/engine/snapshot/sort.rs`) — a secondary index
keyed by a declared param's value (plus sequence as tie-break),
built to support `sort_by` queries; has explicit staleness handling
(`purge_stale_sort_helper_indexes`, referenced from ticket 081's
history) tied to new bucket ingestion.
- `.claude/CLAUDE.md`'s repowise health summary already flags "19 open
static I/O-in-loop / N+1 findings" and calls out `src/engine.rs`'s
`handle_query` as a function/brain-method hotspot — likely overlaps
with whatever a fresh performance pass on read paths would surface,
worth checking those findings first (`get_health()` /
`get_risk(['src/engine.rs'])` via the repowise MCP tools, once the
repo's index is initialized — it wasn't available this session, see
note below) rather than starting from a blank slate.
- Related, already-ticketed read-path work referenced in prior ticket
history: ticket 077/078 (metric bucket range narrowing — "narrow the
range, don't rescan everything") and ticket 081 (PromQL dashboard
queries rebuild full series from scratch per poll) — both were
framed around the same "avoid full rescans" principle this idea is
chasing generally.
## Why this is an idea, not a ticket
This is explicitly a "come up with ideas" ask, not a pre-scoped
change — it needs a dedicated investigation (likely with actual
profiling under a representative workload, per this project's
`samply`-based profiling convention) rather than guesses from reading
code structure alone. Ticket 081's history is a useful precedent here:
079/080 (also performance tickets) were removed for lacking "profiling
evidence to justify the fix" — so a fix in this space should follow
`samply record` evidence, not speculation.
## Note
The repowise MCP tools reported no index for this repo this session
("This repository has no repowise index yet") despite
`.claude/CLAUDE.md` describing one as current — worth running `repowise
init --yes` (or checking why the existing index isn't being picked up)
before the next performance-focused session, since `get_risk`/
`get_health`/`get_change_risk` would materially speed up this kind of
investigation.
## Suggested next step
Profile a representative workload with `samply` against the read paths
(`Logs`, `Timeline`, dashboard PromQL panel queries) to find actual
bottlenecks, then propose specific new index structures (or narrowings
of existing ones) grounded in what the flamegraph shows — rather than
guessing at indexes with no evidence they'd matter.
## Implications and things to consider
### Performance
- Any new index is a write-time cost paid on every ingested line in
exchange for a read-time win — worth stating explicitly per proposed
index, since the engine's ingest path is already doing param
extraction + metric bucketing + alert evaluation per line
(see [[0001-actor-split]]'s findings on how much work already happens
synchronously per line); an expensive new index (e.g. a full-text
inverted index for regex/text search) could turn ingest into the new
bottleneck instead of read.
- A text/regex index in particular is a different kind of structure
than the existing `IndexKey`/`SortHelperKey` model — those are
ordered key→value indexes; full-text search typically wants an
inverted index (token → line positions) or a trigram index, which is
a new subsystem, not a new `IndexKey` variant. Worth scoping as its
own investigation rather than bundling into "additional indexes"
generally.
- Memory cost matters as much as CPU cost here — this is a
for-the-life-of-the-process in-memory tool (per `CLAUDE.md`'s
description), so any new index's memory footprint competes directly
with how much log history a user can retain before running out of
RAM. A `samply` CPU profile alone won't surface this — a memory
profile of retained-line-count vs. RSS would be needed too.
### Code structure
- New indexes need to hook into the same lifecycle the existing ones
do: built incrementally at ingest, invalidated/pruned when retention
drops old lines, and rebuilt on backfill (declare/profile-switch)
the same way `SortHelperKey`'s `purge_stale_sort_helper_indexes`
does. A proposal that only covers "how to build it" without covering
eviction/backfill parity with existing indexes is incomplete.
- If [[0001-actor-split]] happens first, new indexes would likely be
scoped to whichever worker owns the data they index (e.g. a
profile-specific index lives with that profile's worker, not the
primary log worker) — worth sequencing this investigation after that
architectural decision rather than before, so new indexes aren't
designed against a data-ownership model that's about to change.
### Functionality
- Read-path speedups are invisible to users unless something is
currently slow enough to notice — without a concrete complaint or
profiling evidence of an actual slow query, this risks becoming
effort spent on indexes nobody was blocked on. The `samply`-first
approach in "Suggested next step" is meant to guard against exactly
this.
### Not yet considered
- No investigation yet into what "reads" actually means for the
intended scope — dashboard panel polling (already covered by ticket
081's history), the log timeline's default view, ad-hoc filter-bar
queries, and the MCP server's query surface (`src/api/mcp_server.rs`)
are all "reads" with potentially different bottlenecks; a single
"additional indexes" effort may need to be split once profiling
shows which of these is actually slow.
- No check on whether retention limits already bound the problem —
if there's a hard cap on retained log lines, some read-path costs may
already be bounded in practice regardless of indexing, which changes
how much this is worth investing in.
+8 -113
View File
@@ -1,8 +1,8 @@
use crate::config::Settings;
use crate::config::profile::{Panel, Profile};
use crate::engine::alerts::AlertCondition; use crate::engine::alerts::AlertCondition;
use crate::engine::data_value::MetricPrimitiveKind; use crate::engine::data_value::{MetricPrimitiveKind, MetricSemantics};
use crate::engine::metric_primitive::MetricPrimitive; use crate::engine::metric_primitive::MetricPrimitive;
use crate::config::profile::{ChartType, Dashboard, Panel, Profile};
use crate::config::{ColorScheme, CssEntry, Settings};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::collections::BTreeMap; use std::collections::BTreeMap;
@@ -35,6 +35,7 @@ pub struct MetricDeclarationDto {
pub value_param: Option<String>, pub value_param: Option<String>,
pub matched_labels: Option<BTreeMap<String, String>>, pub matched_labels: Option<BTreeMap<String, String>>,
pub unmatched_labels: Option<BTreeMap<String, String>>, pub unmatched_labels: Option<BTreeMap<String, String>>,
pub semantics: MetricSemantics,
} }
#[derive(Debug, Clone, Serialize)] #[derive(Debug, Clone, Serialize)]
@@ -108,57 +109,11 @@ pub struct ErrorDto {
pub error: String, pub error: String,
} }
#[derive(Debug, Deserialize)]
pub struct ImportCssRequest {
pub filename: String,
pub content: String,
}
#[derive(Debug, Deserialize)]
pub struct LinkCssRequest {
pub path_or_url: String,
}
#[derive(Debug, Clone, Serialize)]
pub struct CssEntryDto {
pub id: String,
pub kind: CssEntryKind,
pub filename: Option<String>,
pub path_or_url: Option<String>,
}
#[derive(Debug, Clone, Copy, Serialize)]
#[serde(rename_all = "lowercase")]
pub enum CssEntryKind {
Imported,
Linked,
}
impl From<&CssEntry> for CssEntryDto {
fn from(entry: &CssEntry) -> Self {
match entry {
CssEntry::Imported { id, filename } => CssEntryDto {
id: id.clone(),
kind: CssEntryKind::Imported,
filename: Some(filename.clone()),
path_or_url: None,
},
CssEntry::Linked { path_or_url } => CssEntryDto {
id: crate::config::css_entry_id(entry),
kind: CssEntryKind::Linked,
filename: None,
path_or_url: Some(path_or_url.clone()),
},
}
}
}
#[derive(Debug, Clone, Serialize)] #[derive(Debug, Clone, Serialize)]
pub struct SettingsDto { pub struct SettingsDto {
pub color_scheme: ColorScheme, pub theme: String,
pub active_profile: Option<String>, pub active_profile: Option<String>,
pub max_logs: Option<usize>, pub max_logs: Option<usize>,
pub custom_theme_id: Option<String>,
pub ntfy_topic: Option<String>, pub ntfy_topic: Option<String>,
pub ntfy_server: Option<String>, pub ntfy_server: Option<String>,
} }
@@ -166,10 +121,9 @@ pub struct SettingsDto {
impl From<&Settings> for SettingsDto { impl From<&Settings> for SettingsDto {
fn from(settings: &Settings) -> Self { fn from(settings: &Settings) -> Self {
SettingsDto { SettingsDto {
color_scheme: settings.color_scheme, theme: settings.theme.clone(),
active_profile: settings.active_profile.clone(), active_profile: settings.active_profile.clone(),
max_logs: settings.max_logs, max_logs: settings.max_logs,
custom_theme_id: settings.custom_theme_id.clone(),
ntfy_topic: settings.ntfy_topic.clone(), ntfy_topic: settings.ntfy_topic.clone(),
ntfy_server: settings.ntfy_server.clone(), ntfy_server: settings.ntfy_server.clone(),
} }
@@ -178,12 +132,10 @@ impl From<&Settings> for SettingsDto {
#[derive(Debug, Deserialize)] #[derive(Debug, Deserialize)]
pub struct UpdateSettingsRequest { pub struct UpdateSettingsRequest {
pub color_scheme: Option<ColorScheme>, pub theme: Option<String>,
#[serde(default, deserialize_with = "deserialize_double_option")] #[serde(default, deserialize_with = "deserialize_double_option")]
pub max_logs: Option<Option<usize>>, pub max_logs: Option<Option<usize>>,
#[serde(default, deserialize_with = "deserialize_double_option")] #[serde(default, deserialize_with = "deserialize_double_option")]
pub custom_theme_id: Option<Option<String>>,
#[serde(default, deserialize_with = "deserialize_double_option")]
pub ntfy_topic: Option<Option<String>>, pub ntfy_topic: Option<Option<String>>,
#[serde(default, deserialize_with = "deserialize_double_option")] #[serde(default, deserialize_with = "deserialize_double_option")]
pub ntfy_server: Option<Option<String>>, pub ntfy_server: Option<Option<String>>,
@@ -241,63 +193,6 @@ pub struct CreateProfileRequest {
pub name: String, pub name: String,
} }
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PanelDto {
pub id: String,
pub title: String,
pub promql: String,
pub chart_type: ChartType,
}
impl From<&Panel> for PanelDto {
fn from(panel: &Panel) -> Self {
PanelDto {
id: panel.id.clone(),
title: panel.title.clone(),
promql: panel.promql.clone(),
chart_type: panel.chart_type,
}
}
}
impl From<&PanelDto> for Panel {
fn from(dto: &PanelDto) -> Self {
Panel {
id: dto.id.clone(),
title: dto.title.clone(),
promql: dto.promql.clone(),
chart_type: dto.chart_type,
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DashboardDto {
pub id: String,
pub name: String,
pub panels: Vec<PanelDto>,
}
impl From<&Dashboard> for DashboardDto {
fn from(dashboard: &Dashboard) -> Self {
DashboardDto {
id: dashboard.id.clone(),
name: dashboard.name.clone(),
panels: dashboard.panels.iter().map(PanelDto::from).collect(),
}
}
}
impl From<&DashboardDto> for Dashboard {
fn from(dto: &DashboardDto) -> Self {
Dashboard {
id: dto.id.clone(),
name: dto.name.clone(),
panels: dto.panels.iter().map(Panel::from).collect(),
}
}
}
#[derive(Debug, Deserialize)] #[derive(Debug, Deserialize)]
pub struct CreateDashboardRequest { pub struct CreateDashboardRequest {
pub name: String, pub name: String,
@@ -307,5 +202,5 @@ pub struct CreateDashboardRequest {
pub struct SaveDashboardRequest { pub struct SaveDashboardRequest {
pub name: String, pub name: String,
#[serde(default)] #[serde(default)]
pub panels: Vec<PanelDto>, pub panels: Vec<Panel>,
} }
+156 -100
View File
@@ -69,6 +69,10 @@ pub struct DeclareMetricParams {
pub matched_labels: Option<BTreeMap<String, String>>, pub matched_labels: Option<BTreeMap<String, String>>,
#[serde(default)] #[serde(default)]
pub unmatched_labels: Option<BTreeMap<String, String>>, pub unmatched_labels: Option<BTreeMap<String, String>>,
/// Whether this metric's value is a point-in-time reading ("gauge", e.g. queue depth) or a
/// monotonically-increasing total ("counter", e.g. requests served). Defaults to "gauge".
#[serde(default, skip_serializing_if = "Option::is_none")]
pub semantics: Option<String>,
} }
#[derive(Debug, Deserialize, schemars::JsonSchema)] #[derive(Debug, Deserialize, schemars::JsonSchema)]
@@ -186,36 +190,23 @@ pub struct DeleteDashboardParams {
pub dashboard_id: String, pub dashboard_id: String,
} }
// ---- settings / css ---- // ---- settings / theming ----
#[derive(Debug, Serialize, Deserialize, schemars::JsonSchema)] #[derive(Debug, Serialize, Deserialize, schemars::JsonSchema)]
pub struct UpdateSettingsParams { pub struct UpdateSettingsParams {
#[serde(default)] #[serde(default)]
pub color_scheme: Option<String>, pub theme: Option<String>,
#[serde(default)] #[serde(default)]
pub max_logs: Option<Option<usize>>, pub max_logs: Option<Option<usize>>,
#[serde(default)] #[serde(default)]
pub custom_theme_id: Option<Option<String>>,
#[serde(default)]
pub ntfy_topic: Option<Option<String>>, pub ntfy_topic: Option<Option<String>>,
#[serde(default)] #[serde(default)]
pub ntfy_server: Option<Option<String>>, pub ntfy_server: Option<Option<String>>,
} }
#[derive(Debug, Serialize, Deserialize, schemars::JsonSchema)]
pub struct ImportCssParams {
pub filename: String,
pub content: String,
}
#[derive(Debug, Serialize, Deserialize, schemars::JsonSchema)]
pub struct LinkCssParams {
pub path_or_url: String,
}
#[derive(Debug, Deserialize, schemars::JsonSchema)] #[derive(Debug, Deserialize, schemars::JsonSchema)]
pub struct CssIdParams { pub struct GetThemeContentParams {
pub id: String, pub name: String,
} }
// ---- alerts ---- // ---- alerts ----
@@ -394,7 +385,9 @@ impl OttylMcp {
// ---- logs / timeline / stats ---- // ---- logs / timeline / stats ----
#[tool(description = "Count or read log lines matching a FilterBar query against the running ottyl instance.")] #[tool(
description = "Count or read log lines matching a FilterBar query against the running ottyl instance."
)]
async fn query_logs( async fn query_logs(
&self, &self,
Parameters(params): Parameters<QueryLogsParams>, Parameters(params): Parameters<QueryLogsParams>,
@@ -427,7 +420,9 @@ impl OttylMcp {
} }
} }
#[tool(description = "Read the log-count timeline (bucketed histogram) matching an optional FilterBar query — the same data backing the Log Timeline chart.")] #[tool(
description = "Read the log-count timeline (bucketed histogram) matching an optional FilterBar query — the same data backing the Log Timeline chart."
)]
async fn get_timeline( async fn get_timeline(
&self, &self,
Parameters(params): Parameters<GetTimelineParams>, Parameters(params): Parameters<GetTimelineParams>,
@@ -448,14 +443,18 @@ impl OttylMcp {
json_result(self.get_json("/api/timeline", &query).await?) json_result(self.get_json("/api/timeline", &query).await?)
} }
#[tool(description = "Read overall ingest stats: log count, sequence/timestamp bounds, ingest lag.")] #[tool(
description = "Read overall ingest stats: log count, sequence/timestamp bounds, ingest lag."
)]
async fn get_stats(&self) -> Result<CallToolResult, McpError> { async fn get_stats(&self) -> Result<CallToolResult, McpError> {
json_result(self.get_json("/api/stats", &[]).await?) json_result(self.get_json("/api/stats", &[]).await?)
} }
// ---- params ---- // ---- params ----
#[tool(description = "List all currently declared params (name, regex, kind, etc.), including auto-derived ones.")] #[tool(
description = "List all currently declared params (name, regex, kind, etc.), including auto-derived ones."
)]
async fn list_params(&self) -> Result<CallToolResult, McpError> { async fn list_params(&self) -> Result<CallToolResult, McpError> {
json_result(self.get_json("/api/params", &[]).await?) json_result(self.get_json("/api/params", &[]).await?)
} }
@@ -465,13 +464,22 @@ impl OttylMcp {
&self, &self,
Parameters(params): Parameters<DeclareParamParams>, Parameters(params): Parameters<DeclareParamParams>,
) -> Result<CallToolResult, McpError> { ) -> Result<CallToolResult, McpError> {
let body = serde_json::to_value(&params).map_err(|err| mcp_error("failed to encode params", err))?; let body = serde_json::to_value(&params)
json_result(self.send_json(reqwest::Method::POST, "/api/params", Some(&body)).await?) .map_err(|err| mcp_error("failed to encode params", err))?;
json_result(
self.send_json(reqwest::Method::POST, "/api/params", Some(&body))
.await?,
)
} }
#[tool(description = "Clear all declared params and re-derive auto params from retained log lines.")] #[tool(
description = "Clear all declared params and re-derive auto params from retained log lines."
)]
async fn clear_params(&self) -> Result<CallToolResult, McpError> { async fn clear_params(&self) -> Result<CallToolResult, McpError> {
json_result(self.send_json(reqwest::Method::DELETE, "/api/params", None).await?) json_result(
self.send_json(reqwest::Method::DELETE, "/api/params", None)
.await?,
)
} }
// ---- metrics ---- // ---- metrics ----
@@ -481,13 +489,19 @@ impl OttylMcp {
json_result(self.get_json("/api/metrics", &[]).await?) json_result(self.get_json("/api/metrics", &[]).await?)
} }
#[tool(description = "Declare a metric: named event counts/values bucketed by label, derived from matching log lines.")] #[tool(
description = "Declare a metric: named event counts/values bucketed by label, derived from matching log lines."
)]
async fn declare_metric( async fn declare_metric(
&self, &self,
Parameters(params): Parameters<DeclareMetricParams>, Parameters(params): Parameters<DeclareMetricParams>,
) -> Result<CallToolResult, McpError> { ) -> Result<CallToolResult, McpError> {
let body = serde_json::to_value(&params).map_err(|err| mcp_error("failed to encode params", err))?; let body = serde_json::to_value(&params)
json_result(self.send_json(reqwest::Method::POST, "/api/metrics", Some(&body)).await?) .map_err(|err| mcp_error("failed to encode params", err))?;
json_result(
self.send_json(reqwest::Method::POST, "/api/metrics", Some(&body))
.await?,
)
} }
#[tool(description = "Update an existing metric declaration in place by name.")] #[tool(description = "Update an existing metric declaration in place by name.")]
@@ -522,7 +536,9 @@ impl OttylMcp {
) )
} }
#[tool(description = "Read a metric's raw bucket values (labels, sequence, timestamp, value), optionally filtered by label and time range.")] #[tool(
description = "Read a metric's raw bucket values (labels, sequence, timestamp, value), optionally filtered by label and time range."
)]
async fn get_metric_bucket_values( async fn get_metric_bucket_values(
&self, &self,
Parameters(params): Parameters<GetMetricBucketValuesParams>, Parameters(params): Parameters<GetMetricBucketValuesParams>,
@@ -544,19 +560,27 @@ impl OttylMcp {
query.push(("limit", limit.to_string())); query.push(("limit", limit.to_string()));
} }
json_result( json_result(
self.get_json(&format!("/api/metrics/{}", urlencoding_path(&params.name)), &query) self.get_json(
&format!("/api/metrics/{}", urlencoding_path(&params.name)),
&query,
)
.await?, .await?,
) )
} }
// ---- promql ---- // ---- promql ----
#[tool(description = "Evaluate a PromQL-style query directly and get back raw series data (labels + timestamped points) — the same evaluation dashboards use to render charts.")] #[tool(
description = "Evaluate a PromQL-style query directly and get back raw series data (labels + timestamped points) — the same evaluation dashboards use to render charts."
)]
async fn run_promql_query( async fn run_promql_query(
&self, &self,
Parameters(params): Parameters<RunPromqlQueryParams>, Parameters(params): Parameters<RunPromqlQueryParams>,
) -> Result<CallToolResult, McpError> { ) -> Result<CallToolResult, McpError> {
json_result(self.run_promql(&params.promql, params.start_ms, params.end_ms).await?) json_result(
self.run_promql(&params.promql, params.start_ms, params.end_ms)
.await?,
)
} }
// ---- dashboards / panels ---- // ---- dashboards / panels ----
@@ -569,7 +593,10 @@ impl OttylMcp {
let profile = self.resolve_profile(params.profile).await?; let profile = self.resolve_profile(params.profile).await?;
json_result( json_result(
self.get_json( self.get_json(
&format!("/api/settings/profiles/{}/dashboards", urlencoding_path(&profile)), &format!(
"/api/settings/profiles/{}/dashboards",
urlencoding_path(&profile)
),
&[], &[],
) )
.await?, .await?,
@@ -585,14 +612,19 @@ impl OttylMcp {
json_result( json_result(
self.send_json( self.send_json(
reqwest::Method::POST, reqwest::Method::POST,
&format!("/api/settings/profiles/{}/dashboards", urlencoding_path(&profile)), &format!(
"/api/settings/profiles/{}/dashboards",
urlencoding_path(&profile)
),
Some(&serde_json::json!({ "name": params.name })), Some(&serde_json::json!({ "name": params.name })),
) )
.await?, .await?,
) )
} }
#[tool(description = "Read a dashboard's current panels (definitions only — use render_panel or render_dashboard for the data a panel would chart).")] #[tool(
description = "Read a dashboard's current panels (definitions only — use render_panel or render_dashboard for the data a panel would chart)."
)]
async fn get_dashboard( async fn get_dashboard(
&self, &self,
Parameters(params): Parameters<GetDashboardParams>, Parameters(params): Parameters<GetDashboardParams>,
@@ -644,12 +676,18 @@ impl OttylMcp {
.get_mut("panels") .get_mut("panels")
.and_then(Value::as_array_mut) .and_then(Value::as_array_mut)
.ok_or_else(|| mcp_error("unexpected dashboard shape", "missing panels array"))?; .ok_or_else(|| mcp_error("unexpected dashboard shape", "missing panels array"))?;
match panels.iter().position(|panel| panel.get("id") == new_panel.get("id")) { match panels
.iter()
.position(|panel| panel.get("id") == new_panel.get("id"))
{
Some(index) => panels[index] = new_panel, Some(index) => panels[index] = new_panel,
None => panels.push(new_panel), None => panels.push(new_panel),
} }
json_result(self.save_dashboard(&profile, &params.dashboard_id, dashboard).await?) json_result(
self.save_dashboard(&profile, &params.dashboard_id, dashboard)
.await?,
)
} }
#[tool(description = "Remove a panel from a dashboard by panel_id.")] #[tool(description = "Remove a panel from a dashboard by panel_id.")]
@@ -663,8 +701,13 @@ impl OttylMcp {
.get_mut("panels") .get_mut("panels")
.and_then(Value::as_array_mut) .and_then(Value::as_array_mut)
.ok_or_else(|| mcp_error("unexpected dashboard shape", "missing panels array"))?; .ok_or_else(|| mcp_error("unexpected dashboard shape", "missing panels array"))?;
panels.retain(|panel| panel.get("id").and_then(Value::as_str) != Some(params.panel_id.as_str())); panels.retain(|panel| {
json_result(self.save_dashboard(&profile, &params.dashboard_id, dashboard).await?) panel.get("id").and_then(Value::as_str) != Some(params.panel_id.as_str())
});
json_result(
self.save_dashboard(&profile, &params.dashboard_id, dashboard)
.await?,
)
} }
async fn save_dashboard( async fn save_dashboard(
@@ -685,7 +728,9 @@ impl OttylMcp {
.await .await
} }
#[tool(description = "Render one panel: resolve its promql from the dashboard definition, evaluate it, and return the chart's underlying series data (labels + timestamped points) as JSON — this is how an agent should 'see' a chart without a browser.")] #[tool(
description = "Render one panel: resolve its promql from the dashboard definition, evaluate it, and return the chart's underlying series data (labels + timestamped points) as JSON — this is how an agent should 'see' a chart without a browser."
)]
async fn render_panel( async fn render_panel(
&self, &self,
Parameters(params): Parameters<RenderPanelParams>, Parameters(params): Parameters<RenderPanelParams>,
@@ -695,20 +740,30 @@ impl OttylMcp {
let panel = dashboard let panel = dashboard
.get("panels") .get("panels")
.and_then(Value::as_array) .and_then(Value::as_array)
.and_then(|panels| panels.iter().find(|panel| panel.get("id").and_then(Value::as_str) == Some(params.panel_id.as_str()))) .and_then(|panels| {
.ok_or_else(|| McpError::invalid_params("no panel with that panel_id in the dashboard", None))?; panels.iter().find(|panel| {
panel.get("id").and_then(Value::as_str) == Some(params.panel_id.as_str())
})
})
.ok_or_else(|| {
McpError::invalid_params("no panel with that panel_id in the dashboard", None)
})?;
let promql = panel let promql = panel
.get("promql") .get("promql")
.and_then(Value::as_str) .and_then(Value::as_str)
.ok_or_else(|| mcp_error("unexpected panel shape", "missing promql"))?; .ok_or_else(|| mcp_error("unexpected panel shape", "missing promql"))?;
let series = self.run_promql(promql, params.start_ms, params.end_ms).await?; let series = self
.run_promql(promql, params.start_ms, params.end_ms)
.await?;
json_result(serde_json::json!({ json_result(serde_json::json!({
"panel": panel, "panel": panel,
"series": series, "series": series,
})) }))
} }
#[tool(description = "Render every panel in a dashboard: evaluate each panel's promql and return all their chart data as JSON, so an agent can reason about the whole dashboard without a browser.")] #[tool(
description = "Render every panel in a dashboard: evaluate each panel's promql and return all their chart data as JSON, so an agent can reason about the whole dashboard without a browser."
)]
async fn render_dashboard( async fn render_dashboard(
&self, &self,
Parameters(params): Parameters<RenderDashboardParams>, Parameters(params): Parameters<RenderDashboardParams>,
@@ -722,8 +777,13 @@ impl OttylMcp {
.unwrap_or_default(); .unwrap_or_default();
let mut rendered = Vec::with_capacity(panels.len()); let mut rendered = Vec::with_capacity(panels.len());
for panel in panels { for panel in panels {
let promql = panel.get("promql").and_then(Value::as_str).unwrap_or_default(); let promql = panel
let series = self.run_promql(promql, params.start_ms, params.end_ms).await?; .get("promql")
.and_then(Value::as_str)
.unwrap_or_default();
let series = self
.run_promql(promql, params.start_ms, params.end_ms)
.await?;
rendered.push(serde_json::json!({ "panel": panel, "series": series })); rendered.push(serde_json::json!({ "panel": panel, "series": series }));
} }
json_result(serde_json::json!({ json_result(serde_json::json!({
@@ -734,66 +794,41 @@ impl OttylMcp {
// ---- settings / theming ---- // ---- settings / theming ----
#[tool(description = "Read the running instance's global settings (color scheme, active profile, max_logs, ntfy config).")] #[tool(
description = "Read the running instance's global settings (theme, active profile, max_logs, ntfy config)."
)]
async fn get_settings(&self) -> Result<CallToolResult, McpError> { async fn get_settings(&self) -> Result<CallToolResult, McpError> {
json_result(self.get_json("/api/settings", &[]).await?) json_result(self.get_json("/api/settings", &[]).await?)
} }
#[tool(description = "Update global settings. Only fields provided are changed; pass null for an Option field to clear it.")] #[tool(
description = "Update global settings. Only fields provided are changed; pass null for an Option field to clear it."
)]
async fn update_settings( async fn update_settings(
&self, &self,
Parameters(params): Parameters<UpdateSettingsParams>, Parameters(params): Parameters<UpdateSettingsParams>,
) -> Result<CallToolResult, McpError> { ) -> Result<CallToolResult, McpError> {
let body = serde_json::to_value(&params).map_err(|err| mcp_error("failed to encode params", err))?; let body = serde_json::to_value(&params)
json_result(self.send_json(reqwest::Method::PATCH, "/api/settings", Some(&body)).await?) .map_err(|err| mcp_error("failed to encode params", err))?;
}
#[tool(description = "List imported/linked custom CSS theme entries.")]
async fn list_css(&self) -> Result<CallToolResult, McpError> {
json_result(self.get_json("/api/settings/css", &[]).await?)
}
#[tool(description = "Import a custom CSS theme file by content.")]
async fn import_css(
&self,
Parameters(params): Parameters<ImportCssParams>,
) -> Result<CallToolResult, McpError> {
let body = serde_json::to_value(&params).map_err(|err| mcp_error("failed to encode params", err))?;
json_result(self.send_json(reqwest::Method::POST, "/api/settings/css", Some(&body)).await?)
}
#[tool(description = "Link an external CSS file or URL as a custom theme, instead of importing its content.")]
async fn link_css(
&self,
Parameters(params): Parameters<LinkCssParams>,
) -> Result<CallToolResult, McpError> {
let body = serde_json::to_value(&params).map_err(|err| mcp_error("failed to encode params", err))?;
json_result(self.send_json(reqwest::Method::POST, "/api/settings/css/link", Some(&body)).await?)
}
#[tool(description = "Remove a custom CSS entry by id.")]
async fn remove_css(
&self,
Parameters(params): Parameters<CssIdParams>,
) -> Result<CallToolResult, McpError> {
json_result( json_result(
self.send_json( self.send_json(reqwest::Method::PATCH, "/api/settings", Some(&body))
reqwest::Method::DELETE,
&format!("/api/settings/css/{}", urlencoding_path(&params.id)),
None,
)
.await?, .await?,
) )
} }
#[tool(description = "Read the raw content of an imported CSS entry by id.")] #[tool(description = "List the names of the themes embedded in this build (see /api/themes).")]
async fn get_css_content( async fn list_themes(&self) -> Result<CallToolResult, McpError> {
json_result(self.get_json("/api/themes", &[]).await?)
}
#[tool(description = "Read the raw CSS content of a theme by name.")]
async fn get_theme_content(
&self, &self,
Parameters(params): Parameters<CssIdParams>, Parameters(params): Parameters<GetThemeContentParams>,
) -> Result<CallToolResult, McpError> { ) -> Result<CallToolResult, McpError> {
let response = self let response = self
.client .client
.get(self.url(&format!("/api/settings/css/{}/content", urlencoding_path(&params.id)))) .get(self.url(&format!("/api/themes/{}", urlencoding_path(&params.name))))
.send() .send()
.await .await
.map_err(|err| mcp_error("failed to reach ottyl", err))?; .map_err(|err| mcp_error("failed to reach ottyl", err))?;
@@ -801,7 +836,7 @@ impl OttylMcp {
let text = response let text = response
.text() .text()
.await .await
.map_err(|err| mcp_error("failed to read css content response", err))?; .map_err(|err| mcp_error("failed to read theme content response", err))?;
if !status.is_success() { if !status.is_success() {
return Err(mcp_error("ottyl rejected the request", text)); return Err(mcp_error("ottyl rejected the request", text));
} }
@@ -825,7 +860,10 @@ impl OttylMcp {
"enabled": params.enabled, "enabled": params.enabled,
"condition": params.condition, "condition": params.condition,
}); });
json_result(self.send_json(reqwest::Method::POST, "/api/settings/alerts", Some(&body)).await?) json_result(
self.send_json(reqwest::Method::POST, "/api/settings/alerts", Some(&body))
.await?,
)
} }
#[tool(description = "Update an existing alert rule by id.")] #[tool(description = "Update an existing alert rule by id.")]
@@ -863,21 +901,30 @@ impl OttylMcp {
) )
} }
#[tool(description = "Send a test ntfy notification to verify alert delivery is configured correctly.")] #[tool(
description = "Send a test ntfy notification to verify alert delivery is configured correctly."
)]
async fn test_alert_notification( async fn test_alert_notification(
&self, &self,
Parameters(params): Parameters<TestAlertNotificationParams>, Parameters(params): Parameters<TestAlertNotificationParams>,
) -> Result<CallToolResult, McpError> { ) -> Result<CallToolResult, McpError> {
let body = serde_json::to_value(&params).map_err(|err| mcp_error("failed to encode params", err))?; let body = serde_json::to_value(&params)
.map_err(|err| mcp_error("failed to encode params", err))?;
json_result( json_result(
self.send_json(reqwest::Method::POST, "/api/settings/alerts/test", Some(&body)) self.send_json(
reqwest::Method::POST,
"/api/settings/alerts/test",
Some(&body),
)
.await?, .await?,
) )
} }
// ---- profiles ---- // ---- profiles ----
#[tool(description = "List all saved profiles with summary counts (params, metrics, dashboards).")] #[tool(
description = "List all saved profiles with summary counts (params, metrics, dashboards)."
)]
async fn list_profiles(&self) -> Result<CallToolResult, McpError> { async fn list_profiles(&self) -> Result<CallToolResult, McpError> {
json_result(self.get_json("/api/settings/profiles", &[]).await?) json_result(self.get_json("/api/settings/profiles", &[]).await?)
} }
@@ -887,8 +934,12 @@ impl OttylMcp {
&self, &self,
Parameters(params): Parameters<CreateProfileParams>, Parameters(params): Parameters<CreateProfileParams>,
) -> Result<CallToolResult, McpError> { ) -> Result<CallToolResult, McpError> {
let body = serde_json::to_value(&params).map_err(|err| mcp_error("failed to encode params", err))?; let body = serde_json::to_value(&params)
json_result(self.send_json(reqwest::Method::POST, "/api/settings/profiles", Some(&body)).await?) .map_err(|err| mcp_error("failed to encode params", err))?;
json_result(
self.send_json(reqwest::Method::POST, "/api/settings/profiles", Some(&body))
.await?,
)
} }
#[tool(description = "Delete a profile by name.")] #[tool(description = "Delete a profile by name.")]
@@ -906,7 +957,9 @@ impl OttylMcp {
) )
} }
#[tool(description = "Activate a profile: swaps in its param/metric declarations and dashboards as the running instance's live state.")] #[tool(
description = "Activate a profile: swaps in its param/metric declarations and dashboards as the running instance's live state."
)]
async fn activate_profile( async fn activate_profile(
&self, &self,
Parameters(params): Parameters<ProfileNameParams>, Parameters(params): Parameters<ProfileNameParams>,
@@ -914,7 +967,10 @@ impl OttylMcp {
json_result( json_result(
self.send_json( self.send_json(
reqwest::Method::POST, reqwest::Method::POST,
&format!("/api/settings/profiles/{}/activate", urlencoding_path(&params.name)), &format!(
"/api/settings/profiles/{}/activate",
urlencoding_path(&params.name)
),
None, None,
) )
.await?, .await?,
@@ -962,7 +1018,7 @@ impl ServerHandler for OttylMcp {
"Proxies to a running ottyl instance's HTTP API. Covers everything the web UI can do: \ "Proxies to a running ottyl instance's HTTP API. Covers everything the web UI can do: \
logs/timeline/stats, param & metric declarations, PromQL queries, dashboards & panels \ logs/timeline/stats, param & metric declarations, PromQL queries, dashboards & panels \
(including render_panel/render_dashboard, which return a chart's underlying series data \ (including render_panel/render_dashboard, which return a chart's underlying series data \
as JSON so an agent can reason about it without a browser), settings & custom CSS themes, \ as JSON so an agent can reason about it without a browser), settings & themes, \
alert rules, and profile management. See the embedded SKILL.md (GET /SKILL.md on the \ alert rules, and profile management. See the embedded SKILL.md (GET /SKILL.md on the \
target instance) for examples." target instance) for examples."
.to_string(), .to_string(),
+439 -184
View File
File diff suppressed because it is too large Load Diff
+82 -57
View File
@@ -22,28 +22,28 @@ struct Args {
#[command(subcommand)] #[command(subcommand)]
command: Option<Command>, command: Option<Command>,
#[arg(long, default_value_t = 0)] #[arg(short, long, default_value_t = 0)]
port: u16, port: u16,
#[arg(long, help = SETTINGS_PATH_HELP.as_str())] #[arg(short, long, help = SETTINGS_PATH_HELP.as_str())]
settings_path: Option<PathBuf>, settings_path: Option<PathBuf>,
#[arg(long)] #[arg(short, long)]
file: Option<PathBuf>, file: Option<PathBuf>,
#[arg(long)] #[arg(short, long)]
metrics_url: Option<String>, metrics_url: Option<String>,
#[arg(long)] #[arg(short, long)]
name: Option<String>, name: Option<String>,
} }
#[derive(Subcommand, Debug)] #[derive(Subcommand, Debug)]
enum Command { enum Command {
Mcp { Mcp {
#[arg(long)] #[arg(short, long)]
session: Option<String>, session: Option<String>,
#[arg(long)] #[arg(short, long)]
port: Option<u16>, port: Option<u16>,
}, },
Sessions, Sessions,
@@ -71,50 +71,9 @@ pub async fn run() {
let settings = Arc::new(Mutex::new(Settings::load(&settings_path))); let settings = Arc::new(Mutex::new(Settings::load(&settings_path)));
let engine_tx = crate::engine::spawn(settings.clone(), crate::consts::ENGINE_CHANNEL_CAPACITY); let engine_tx = spawn_engine(&settings).await;
{ load_active_profile(&settings, &settings_path, &engine_tx).await;
let max_logs = settings.lock().unwrap().max_logs; spawn_ingest(&args, &engine_tx);
let _ = engine_tx.tx.send(crate::engine::EngineMessage::SetMaxLogs(max_logs)).await;
}
{
let (active_profile_name, active_profile) = {
let mut settings_guard = settings.lock().unwrap();
settings_guard
.ensure_default_profile(&settings_path)
.expect("failed to ensure the Default profile exists");
let active_profile_name = settings_guard.active_profile.clone();
let active_profile = active_profile_name
.as_deref()
.and_then(|name| settings_guard.profile(name).cloned());
(active_profile_name, active_profile)
};
match (active_profile_name, active_profile) {
(Some(_), Some(active_profile)) => {
active_profile.load_into_engine(&engine_tx).await;
}
(Some(profile_name), None) => {
eprintln!(
"warning: active profile '{profile_name}' not found in {}; skipping",
settings_path.display()
);
}
(None, _) => {}
}
}
match args.file.clone() {
Some(path) => {
tokio::spawn(crate::engine::ingest::run_file(engine_tx.clone(), path));
}
None => {
tokio::spawn(crate::engine::ingest::run(engine_tx.clone()));
}
}
if let Some(metrics_url) = args.metrics_url.clone() {
tokio::spawn(crate::engine::ingest::run_metrics_poll(engine_tx.clone(), metrics_url));
}
let app = crate::api::router(engine_tx, settings, settings_path.clone()); let app = crate::api::router(engine_tx, settings, settings_path.clone());
let listener = TcpListener::bind((crate::consts::DEFAULT_BIND_HOST, args.port)) let listener = TcpListener::bind((crate::consts::DEFAULT_BIND_HOST, args.port))
@@ -125,12 +84,79 @@ pub async fn run() {
.expect("failed to read back bound address") .expect("failed to read back bound address")
.port(); .port();
register_session_and_log(&settings_path, &args, bound_port);
axum::serve(listener, app).await.expect("server error");
}
async fn spawn_engine(settings: &Arc<Mutex<Settings>>) -> crate::engine::EngineHandle {
let engine_tx = crate::engine::spawn(settings.clone(), crate::consts::ENGINE_CHANNEL_CAPACITY);
let max_logs = settings.lock().unwrap().max_logs;
let _ = engine_tx
.tx
.send(crate::engine::EngineMessage::SetMaxLogs(max_logs))
.await;
engine_tx
}
/// Resolves the active profile (creating the Default profile if none exists yet) and, if
/// found, replays its declarations into the engine before ingest starts — there's no way to
/// reprocess earlier lines later, so this must happen before `spawn_ingest`.
async fn load_active_profile(
settings: &Arc<Mutex<Settings>>,
settings_path: &std::path::Path,
engine_tx: &crate::engine::EngineHandle,
) {
let (active_profile_name, active_profile) = {
let mut settings_guard = settings.lock().unwrap();
settings_guard
.ensure_default_profile(settings_path)
.expect("failed to ensure the Default profile exists");
let active_profile_name = settings_guard.active_profile.clone();
let active_profile = active_profile_name
.as_deref()
.and_then(|name| settings_guard.profile(name).cloned());
(active_profile_name, active_profile)
};
match (active_profile_name, active_profile) {
(Some(_), Some(active_profile)) => {
active_profile.load_into_engine(engine_tx).await;
}
(Some(profile_name), None) => {
eprintln!(
"warning: active profile '{profile_name}' not found in {}; skipping",
settings_path.display()
);
}
(None, _) => {}
}
}
fn spawn_ingest(args: &Args, engine_tx: &crate::engine::EngineHandle) {
match args.file.clone() {
Some(path) => {
tokio::spawn(crate::engine::ingest::run_file(engine_tx.clone(), path));
}
None => {
tokio::spawn(crate::engine::ingest::run(engine_tx.clone()));
}
}
if let Some(metrics_url) = args.metrics_url.clone() {
tokio::spawn(crate::engine::ingest::run_metrics_poll(
engine_tx.clone(),
metrics_url,
));
}
}
fn register_session_and_log(settings_path: &std::path::Path, args: &Args, bound_port: u16) {
let source = match &args.file { let source = match &args.file {
Some(path) => format!("tailing {}", path.display()), Some(path) => format!("tailing {}", path.display()),
None => "reading logs from stdin".to_string(), None => "reading logs from stdin".to_string(),
}; };
let session_name = args.name.clone().unwrap_or_else(sessions::generate_name); let session_name = args.name.clone().unwrap_or_else(sessions::generate_name);
match sessions::register_session(&settings_path, &session_name, bound_port) { match sessions::register_session(settings_path, &session_name, bound_port) {
Ok(_) => eprintln!( Ok(_) => eprintln!(
"ottyl serving on http://localhost:{} ({}), session name: {}", "ottyl serving on http://localhost:{} ({}), session name: {}",
bound_port, source, session_name bound_port, source, session_name
@@ -140,10 +166,6 @@ pub async fn run() {
bound_port, source, session_name, err bound_port, source, session_name, err
), ),
} }
axum::serve(listener, app)
.await
.expect("server error");
} }
async fn run_mcp(settings_path: &std::path::Path, session: Option<String>, port: Option<u16>) { async fn run_mcp(settings_path: &std::path::Path, session: Option<String>, port: Option<u16>) {
@@ -180,7 +202,10 @@ fn print_sessions(settings_path: &std::path::Path) {
return; return;
} }
for session in sessions { for session in sessions {
println!("{}\tport={}\tpid={}", session.name, session.port, session.pid); println!(
"{}\tport={}\tpid={}",
session.name, session.port, session.pid
);
} }
} }
+5 -1
View File
@@ -131,7 +131,11 @@ mod tests {
let sessions_dir_path = sessions_dir(&settings_path); let sessions_dir_path = sessions_dir(&settings_path);
fs::create_dir_all(&sessions_dir_path).unwrap(); fs::create_dir_all(&sessions_dir_path).unwrap();
let stale = SessionInfo { name: "foo".to_string(), port: 1111, pid: 999_999 }; let stale = SessionInfo {
name: "foo".to_string(),
port: 1111,
pid: 999_999,
};
fs::write( fs::write(
session_file_path(&sessions_dir_path, "foo"), session_file_path(&sessions_dir_path, "foo"),
serde_json::to_string(&stale).unwrap(), serde_json::to_string(&stale).unwrap(),
+46 -291
View File
@@ -1,55 +1,22 @@
pub mod profile; pub mod profile;
use std::collections::hash_map::DefaultHasher;
use std::fmt;
use std::fs; use std::fs;
use std::hash::{Hash, Hasher};
use std::io; use std::io;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex}; use std::sync::{Arc, Mutex};
use std::time::{SystemTime, UNIX_EPOCH};
use directories::ProjectDirs; use directories::ProjectDirs;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use crate::engine::alerts::AlertRule;
use crate::config::profile::Profile; use crate::config::profile::Profile;
use crate::engine::alerts::AlertRule;
pub type SettingsHandle = Arc<Mutex<Settings>>; pub type SettingsHandle = Arc<Mutex<Settings>>;
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)] /// Name of the selected theme file in `themes/` (see `THEMES` in `src/api/mod.rs`), or
#[serde(rename_all = "lowercase")] /// `"system"` to follow the OS light/dark preference between the `dark`/`light` themes.
pub enum ColorScheme { fn default_theme() -> String {
Light, "system".to_string()
#[serde(alias = "grey")]
Dark,
Red,
Green,
Blue,
Yellow,
Pink,
Orange,
#[serde(rename = "light_green")]
LightGreen,
#[serde(rename = "light_red")]
LightRed,
#[serde(rename = "light_blue")]
LightBlue,
#[serde(rename = "light_yellow")]
LightYellow,
#[serde(rename = "light_pink")]
LightPink,
#[serde(rename = "light_orange")]
LightOrange,
#[default]
System,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "lowercase")]
pub enum CssEntry {
Imported { id: String, filename: String },
Linked { path_or_url: String },
} }
#[derive(Debug, Clone, Deserialize, Default)] #[derive(Debug, Clone, Deserialize, Default)]
@@ -76,11 +43,13 @@ impl Default for ProfilesOnDisk {
#[derive(Debug, Clone, Deserialize)] #[derive(Debug, Clone, Deserialize)]
struct SettingsOnDisk { struct SettingsOnDisk {
color_scheme: ColorScheme,
#[serde(default)] #[serde(default)]
css_entries: Vec<CssEntry>, theme: Option<String>,
// Pre-theme-directory settings stored the selection as `color_scheme` (a fixed enum); a
// separate `custom_theme_id` pointed at an uploaded/linked CSS entry (both removed) and can't
// be mapped onto a theme file, so it's left out here and just ignored by serde on load.
#[serde(default)] #[serde(default)]
custom_theme_id: Option<String>, color_scheme: Option<String>,
#[serde(default)] #[serde(default)]
profiles: ProfilesOnDisk, profiles: ProfilesOnDisk,
#[serde(default)] #[serde(default)]
@@ -101,10 +70,15 @@ impl From<SettingsOnDisk> for Settings {
ProfilesOnDisk::Inline(profiles) => (profiles, None), ProfilesOnDisk::Inline(profiles) => (profiles, None),
ProfilesOnDisk::Legacy(pointer) => (Vec::new(), pointer.active_profile), ProfilesOnDisk::Legacy(pointer) => (Vec::new(), pointer.active_profile),
}; };
let theme = on_disk
.theme
.unwrap_or_else(|| match on_disk.color_scheme.as_deref() {
Some("grey") => "dark".to_string(),
Some(scheme) => scheme.to_string(),
None => default_theme(),
});
Settings { Settings {
color_scheme: on_disk.color_scheme, theme,
css_entries: on_disk.css_entries,
custom_theme_id: on_disk.custom_theme_id,
profiles, profiles,
active_profile: on_disk.active_profile.or(legacy_active_profile), active_profile: on_disk.active_profile.or(legacy_active_profile),
max_logs: on_disk.max_logs, max_logs: on_disk.max_logs,
@@ -115,13 +89,11 @@ impl From<SettingsOnDisk> for Settings {
} }
} }
#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(from = "SettingsOnDisk")] #[serde(from = "SettingsOnDisk")]
pub struct Settings { pub struct Settings {
pub color_scheme: ColorScheme, #[serde(default = "default_theme")]
pub css_entries: Vec<CssEntry>, pub theme: String,
#[serde(default)]
pub custom_theme_id: Option<String>,
pub profiles: Vec<Profile>, pub profiles: Vec<Profile>,
pub active_profile: Option<String>, pub active_profile: Option<String>,
#[serde(default)] #[serde(default)]
@@ -134,6 +106,20 @@ pub struct Settings {
pub ntfy_server: Option<String>, pub ntfy_server: Option<String>,
} }
impl Default for Settings {
fn default() -> Self {
Settings {
theme: default_theme(),
profiles: Vec::new(),
active_profile: None,
max_logs: None,
alert_rules: Vec::new(),
ntfy_topic: None,
ntfy_server: None,
}
}
}
pub fn default_settings_path() -> Option<PathBuf> { pub fn default_settings_path() -> Option<PathBuf> {
ProjectDirs::from( ProjectDirs::from(
crate::consts::APP_QUALIFIER, crate::consts::APP_QUALIFIER,
@@ -199,7 +185,8 @@ impl Settings {
.map(|parent| parent.join("profiles")) .map(|parent| parent.join("profiles"))
.unwrap_or_else(|| PathBuf::from("profiles")) .unwrap_or_else(|| PathBuf::from("profiles"))
.join("profiles.json"); .join("profiles.json");
let Some(profiles) = crate::config::profile::Profile::load_legacy_store(&legacy_path) else { let Some(profiles) = crate::config::profile::Profile::load_legacy_store(&legacy_path)
else {
return; return;
}; };
self.profiles = profiles; self.profiles = profiles;
@@ -231,119 +218,6 @@ impl Settings {
} }
} }
#[derive(Debug)]
pub enum CssError {
NotFound,
NotImported,
Io(io::Error),
}
impl fmt::Display for CssError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
CssError::NotFound => write!(f, "no CSS entry with that id"),
CssError::NotImported => write!(f, "CSS entry is linked, not imported"),
CssError::Io(err) => write!(f, "I/O error: {err}"),
}
}
}
impl std::error::Error for CssError {}
impl From<io::Error> for CssError {
fn from(err: io::Error) -> Self {
CssError::Io(err)
}
}
pub fn css_entry_id(entry: &CssEntry) -> String {
match entry {
CssEntry::Imported { id, .. } => id.clone(),
CssEntry::Linked { path_or_url, .. } => linked_entry_id(path_or_url),
}
}
fn linked_entry_id(path_or_url: &str) -> String {
let mut hasher = DefaultHasher::new();
path_or_url.hash(&mut hasher);
format!("linked-{:016x}", hasher.finish())
}
fn generate_imported_css_id(filename: &str, content: &str) -> String {
let mut hasher = DefaultHasher::new();
filename.hash(&mut hasher);
content.hash(&mut hasher);
let nanos = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0);
nanos.hash(&mut hasher);
format!("{:016x}", hasher.finish())
}
fn imported_css_file_path(css_dir: &Path, id: &str) -> PathBuf {
css_dir.join(format!("{id}.css"))
}
impl Settings {
pub fn import_css(
&mut self,
settings_path: &Path,
css_dir: &Path,
filename: String,
content: &str,
) -> Result<String, CssError> {
fs::create_dir_all(css_dir)?;
let id = generate_imported_css_id(&filename, content);
fs::write(imported_css_file_path(css_dir, &id), content)?;
self.css_entries.push(CssEntry::Imported {
id: id.clone(),
filename,
});
self.save(settings_path)?;
Ok(id)
}
pub fn link_css(&mut self, settings_path: &Path, path_or_url: String) -> Result<(), CssError> {
self.css_entries.push(CssEntry::Linked { path_or_url });
self.save(settings_path)?;
Ok(())
}
pub fn remove_css(
&mut self,
settings_path: &Path,
css_dir: &Path,
id: &str,
) -> Result<(), CssError> {
let index = self
.css_entries
.iter()
.position(|entry| css_entry_id(entry) == id);
match index.map(|i| &self.css_entries[i]) {
None => Err(CssError::NotFound),
Some(CssEntry::Linked { .. }) => Err(CssError::NotImported),
Some(CssEntry::Imported { .. }) => {
let idx = index.unwrap();
self.css_entries.remove(idx);
let _ = fs::remove_file(imported_css_file_path(css_dir, id));
self.save(settings_path)?;
Ok(())
}
}
}
pub fn css_content(&self, css_dir: &Path, id: &str) -> Result<String, CssError> {
match self.css_entries.iter().find(|entry| css_entry_id(entry) == id) {
None => Err(CssError::NotFound),
Some(CssEntry::Linked { .. }) => Err(CssError::NotImported),
Some(CssEntry::Imported { .. }) => {
Ok(fs::read_to_string(imported_css_file_path(css_dir, id))?)
}
}
}
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -371,29 +245,12 @@ mod tests {
} }
#[test] #[test]
fn light_variant_color_schemes_round_trip_through_serde() { fn legacy_grey_color_scheme_migrates_to_dark_theme() {
for (scheme, rename) in [
(ColorScheme::LightGreen, "light_green"),
(ColorScheme::LightRed, "light_red"),
(ColorScheme::LightBlue, "light_blue"),
(ColorScheme::LightYellow, "light_yellow"),
(ColorScheme::LightPink, "light_pink"),
(ColorScheme::LightOrange, "light_orange"),
] {
let json = serde_json::to_string(&scheme).unwrap();
assert_eq!(json, format!("\"{rename}\""));
let deserialized: ColorScheme = serde_json::from_str(&json).unwrap();
assert_eq!(deserialized, scheme);
}
}
#[test]
fn legacy_grey_color_scheme_deserializes_as_dark() {
let settings: Settings = serde_json::from_str( let settings: Settings = serde_json::from_str(
r#"{"color_scheme":"grey","css_entries":[],"custom_theme_id":null,"profiles":{"names":[],"active_profile":null,"profiles_dir":"profiles"},"max_logs":null,"alert_rules":[],"ntfy_topic":null,"ntfy_server":null}"#, r#"{"color_scheme":"grey","css_entries":[],"custom_theme_id":null,"profiles":{"names":[],"active_profile":null,"profiles_dir":"profiles"},"max_logs":null,"alert_rules":[],"ntfy_topic":null,"ntfy_server":null}"#,
) )
.unwrap(); .unwrap();
assert_eq!(settings.color_scheme, ColorScheme::Dark); assert_eq!(settings.theme, "dark");
} }
#[test] #[test]
@@ -411,16 +268,13 @@ mod tests {
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("settings.json"); let path = dir.path().join("settings.json");
let mut settings = Settings::default(); let mut settings = Settings {
settings.color_scheme = ColorScheme::Dark; theme: "dark".to_string(),
settings.css_entries.push(CssEntry::Imported { ..Settings::default()
id: "abc123".to_string(), };
filename: "theme.css".to_string(), settings
}); .profiles
settings.css_entries.push(CssEntry::Linked { .push(crate::config::profile::Profile::new("default".to_string()));
path_or_url: "https://example.com/style.css".to_string(),
});
settings.profiles.push(crate::config::profile::Profile::new("default".to_string()));
settings.active_profile = Some("default".to_string()); settings.active_profile = Some("default".to_string());
settings.alert_rules.push(crate::engine::alerts::AlertRule { settings.alert_rules.push(crate::engine::alerts::AlertRule {
id: "rule1".to_string(), id: "rule1".to_string(),
@@ -449,105 +303,6 @@ mod tests {
assert!(path.exists()); assert!(path.exists());
} }
#[test]
fn import_css_persists_file_and_entry_across_reload() {
let dir = tempfile::tempdir().unwrap();
let settings_path = dir.path().join("settings.json");
let css_dir = dir.path().to_path_buf();
let mut settings = Settings::load(&settings_path);
let id = settings
.import_css(
&settings_path,
&css_dir,
"theme.css".to_string(),
"body { color: red; }",
)
.unwrap();
let css_file = css_dir.join(format!("{id}.css"));
assert!(css_file.exists());
assert_eq!(fs::read_to_string(&css_file).unwrap(), "body { color: red; }");
drop(settings);
let reloaded = Settings::load(&settings_path);
assert!(reloaded.css_entries.iter().any(|entry| matches!(
entry,
CssEntry::Imported { id: entry_id, filename, .. } if entry_id == &id && filename == "theme.css"
)));
assert!(css_file.exists());
assert_eq!(
reloaded.css_content(&css_dir, &id).unwrap(),
"body { color: red; }"
);
}
#[test]
fn remove_css_deletes_file_and_entry() {
let dir = tempfile::tempdir().unwrap();
let settings_path = dir.path().join("settings.json");
let css_dir = dir.path().to_path_buf();
let mut settings = Settings::load(&settings_path);
let id = settings
.import_css(&settings_path, &css_dir, "a.css".to_string(), "a{}")
.unwrap();
let css_file = css_dir.join(format!("{id}.css"));
assert!(css_file.exists());
settings.remove_css(&settings_path, &css_dir, &id).unwrap();
assert!(!css_file.exists());
assert!(!settings.css_entries.iter().any(|e| css_entry_id(e) == id));
let reloaded = Settings::load(&settings_path);
assert!(!reloaded.css_entries.iter().any(|e| css_entry_id(e) == id));
}
#[test]
fn remove_css_rejects_linked_entry() {
let dir = tempfile::tempdir().unwrap();
let settings_path = dir.path().join("settings.json");
let css_dir = dir.path().to_path_buf();
let mut settings = Settings::load(&settings_path);
settings
.link_css(&settings_path, "https://example.com/style.css".to_string())
.unwrap();
let linked_id = css_entry_id(&settings.css_entries[0]);
let result = settings.remove_css(&settings_path, &css_dir, &linked_id);
assert!(matches!(result, Err(CssError::NotImported)));
assert_eq!(settings.css_entries.len(), 1);
}
#[test]
fn remove_css_rejects_unknown_id() {
let dir = tempfile::tempdir().unwrap();
let settings_path = dir.path().join("settings.json");
let css_dir = dir.path().to_path_buf();
let mut settings = Settings::load(&settings_path);
let result = settings.remove_css(&settings_path, &css_dir, "does-not-exist");
assert!(matches!(result, Err(CssError::NotFound)));
}
#[test]
fn css_content_rejects_linked_entry() {
let dir = tempfile::tempdir().unwrap();
let settings_path = dir.path().join("settings.json");
let css_dir = dir.path().to_path_buf();
let mut settings = Settings::load(&settings_path);
settings
.link_css(&settings_path, "https://example.com/style.css".to_string())
.unwrap();
let linked_id = css_entry_id(&settings.css_entries[0]);
let result = settings.css_content(&css_dir, &linked_id);
assert!(matches!(result, Err(CssError::NotImported)));
}
#[test] #[test]
fn load_migrates_legacy_profiles_file_into_inline_settings() { fn load_migrates_legacy_profiles_file_into_inline_settings() {
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();
+304 -93
View File
@@ -1,7 +1,7 @@
#![allow(dead_code)] #![allow(dead_code)]
use std::collections::hash_map::DefaultHasher;
use std::collections::BTreeMap; use std::collections::BTreeMap;
use std::collections::hash_map::DefaultHasher;
use std::fmt; use std::fmt;
use std::fs; use std::fs;
use std::hash::{Hash, Hasher}; use std::hash::{Hash, Hasher};
@@ -11,10 +11,10 @@ use std::time::{SystemTime, UNIX_EPOCH};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use crate::consts::{DEFAULT_PROFILE_NAME, SEEDED_METRIC_LABEL, SEEDED_METRIC_NAME};
use crate::engine::data_value::MetricPrimitiveKind;
use crate::engine::{DeclareMetricRequest, DeclareParamRequest, EngineHandle, EngineMessage};
use crate::config::Settings; use crate::config::Settings;
use crate::consts::{DEFAULT_PROFILE_NAME, SEEDED_METRIC_LABEL, SEEDED_METRIC_NAME};
use crate::engine::data_value::{MetricPrimitiveKind, MetricSemantics};
use crate::engine::{DeclareMetricRequest, DeclareParamRequest, EngineHandle, EngineMessage};
use tokio::sync::oneshot; use tokio::sync::oneshot;
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
@@ -122,6 +122,8 @@ pub struct MetricDeclarationRecord {
pub matched_labels: Option<std::collections::BTreeMap<String, String>>, pub matched_labels: Option<std::collections::BTreeMap<String, String>>,
#[serde(default)] #[serde(default)]
pub unmatched_labels: Option<std::collections::BTreeMap<String, String>>, pub unmatched_labels: Option<std::collections::BTreeMap<String, String>>,
#[serde(default)]
pub semantics: MetricSemantics,
#[serde(default, rename = "params", skip_serializing_if = "Vec::is_empty")] #[serde(default, rename = "params", skip_serializing_if = "Vec::is_empty")]
legacy_params: Vec<LegacyMetricParamRecord>, legacy_params: Vec<LegacyMetricParamRecord>,
} }
@@ -135,6 +137,7 @@ impl From<&DeclareMetricRequest> for MetricDeclarationRecord {
value_param: request.value_param.clone(), value_param: request.value_param.clone(),
matched_labels: request.matched_labels.clone(), matched_labels: request.matched_labels.clone(),
unmatched_labels: request.unmatched_labels.clone(), unmatched_labels: request.unmatched_labels.clone(),
semantics: request.semantics,
legacy_params: Vec::new(), legacy_params: Vec::new(),
} }
} }
@@ -149,6 +152,7 @@ impl From<&MetricDeclarationRecord> for DeclareMetricRequest {
value_param: record.value_param.clone(), value_param: record.value_param.clone(),
matched_labels: record.matched_labels.clone(), matched_labels: record.matched_labels.clone(),
unmatched_labels: record.unmatched_labels.clone(), unmatched_labels: record.unmatched_labels.clone(),
semantics: record.semantics,
} }
} }
} }
@@ -167,8 +171,11 @@ impl Profile {
let mut label_params = Vec::new(); let mut label_params = Vec::new();
let mut value_param = None; let mut value_param = None;
for matcher in &legacy { for matcher in &legacy {
let param_name = let param_name = self.resolve_or_create_legacy_param(
self.resolve_or_create_legacy_param(&metric_name, &message_match_regex, matcher); &metric_name,
&message_match_regex,
matcher,
);
match matcher.role { match matcher.role {
LegacyParamRole::Label => label_params.push(param_name), LegacyParamRole::Label => label_params.push(param_name),
LegacyParamRole::Value => value_param = Some(param_name), LegacyParamRole::Value => value_param = Some(param_name),
@@ -187,15 +194,14 @@ impl Profile {
message_match_regex: &str, message_match_regex: &str,
matcher: &LegacyMetricParamRecord, matcher: &LegacyMetricParamRecord,
) -> String { ) -> String {
if let Some(existing) = self.params.iter().find(|p| p.name == matcher.param_name) { if let Some(existing) = self.params.iter().find(|p| p.name == matcher.param_name)
if existing.message_match_regex == message_match_regex && existing.message_match_regex == message_match_regex
&& existing.capture_group == matcher.capture_group && existing.capture_group == matcher.capture_group
&& existing.json_field_path == matcher.json_field_path && existing.json_field_path == matcher.json_field_path
&& existing.kind == matcher.kind && existing.kind == matcher.kind
{ {
return existing.name.clone(); return existing.name.clone();
} }
}
let implicit_name = format!("{metric_name}.{}", matcher.param_name); let implicit_name = format!("{metric_name}.{}", matcher.param_name);
if !self.params.iter().any(|p| p.name == implicit_name) { if !self.params.iter().any(|p| p.name == implicit_name) {
@@ -213,8 +219,11 @@ impl Profile {
} }
fn migrate_duration_value_params(&mut self) -> bool { fn migrate_duration_value_params(&mut self) -> bool {
let value_param_names: std::collections::HashSet<&str> = let value_param_names: std::collections::HashSet<&str> = self
self.metrics.iter().filter_map(|metric| metric.value_param.as_deref()).collect(); .metrics
.iter()
.filter_map(|metric| metric.value_param.as_deref())
.collect();
let mut migrated = false; let mut migrated = false;
for param in &mut self.params { for param in &mut self.params {
if !param.elapsed_since_last if !param.elapsed_since_last
@@ -267,7 +276,11 @@ fn generate_id(seed: &str) -> String {
impl Dashboard { impl Dashboard {
pub fn new(name: String) -> Dashboard { pub fn new(name: String) -> Dashboard {
let id = generate_id(&name); let id = generate_id(&name);
Dashboard { id, name, panels: Vec::new() } Dashboard {
id,
name,
panels: Vec::new(),
}
} }
} }
@@ -278,8 +291,15 @@ impl MetricDeclarationRecord {
message_match_regex: r"(?i)error".to_string(), message_match_regex: r"(?i)error".to_string(),
label_params: Vec::new(), label_params: Vec::new(),
value_param: None, value_param: None,
matched_labels: Some(BTreeMap::from([(SEEDED_METRIC_LABEL.to_string(), "error".to_string())])), matched_labels: Some(BTreeMap::from([(
unmatched_labels: Some(BTreeMap::from([(SEEDED_METRIC_LABEL.to_string(), "other".to_string())])), SEEDED_METRIC_LABEL.to_string(),
"error".to_string(),
)])),
unmatched_labels: Some(BTreeMap::from([(
SEEDED_METRIC_LABEL.to_string(),
"other".to_string(),
)])),
semantics: MetricSemantics::default(),
legacy_params: Vec::new(), legacy_params: Vec::new(),
} }
} }
@@ -336,11 +356,17 @@ impl Profile {
let name = request.name.clone(); let name = request.name.clone();
if engine if engine
.tx .tx
.send(EngineMessage::DeclareParam { request, respond_to }) .send(EngineMessage::DeclareParam {
request,
respond_to,
})
.await .await
.is_err() .is_err()
{ {
eprintln!("warning: engine unavailable while loading profile '{}'", self.name); eprintln!(
"warning: engine unavailable while loading profile '{}'",
self.name
);
return; return;
} }
match response.await { match response.await {
@@ -362,11 +388,17 @@ impl Profile {
let name = request.name.clone(); let name = request.name.clone();
if engine if engine
.tx .tx
.send(EngineMessage::DeclareMetric { request, respond_to }) .send(EngineMessage::DeclareMetric {
request,
respond_to,
})
.await .await
.is_err() .is_err()
{ {
eprintln!("warning: engine unavailable while loading profile '{}'", self.name); eprintln!(
"warning: engine unavailable while loading profile '{}'",
self.name
);
return; return;
} }
match response.await { match response.await {
@@ -444,7 +476,11 @@ impl Settings {
self.profiles.iter().find(|p| p.name == name) self.profiles.iter().find(|p| p.name == name)
} }
pub fn create_profile(&mut self, settings_path: &Path, name: String) -> Result<(), ProfileError> { pub fn create_profile(
&mut self,
settings_path: &Path,
name: String,
) -> Result<(), ProfileError> {
if self.profile(&name).is_some() { if self.profile(&name).is_some() {
return Err(ProfileError::DuplicateName); return Err(ProfileError::DuplicateName);
} }
@@ -463,7 +499,11 @@ impl Settings {
if name == DEFAULT_PROFILE_NAME { if name == DEFAULT_PROFILE_NAME {
return Err(ProfileError::CannotDeleteDefault); return Err(ProfileError::CannotDeleteDefault);
} }
let index = self.profiles.iter().position(|p| p.name == name).ok_or(ProfileError::NotFound)?; let index = self
.profiles
.iter()
.position(|p| p.name == name)
.ok_or(ProfileError::NotFound)?;
self.profiles.remove(index); self.profiles.remove(index);
if self.active_profile.as_deref() == Some(name) { if self.active_profile.as_deref() == Some(name) {
self.active_profile = None; self.active_profile = None;
@@ -475,7 +515,9 @@ impl Settings {
pub fn ensure_default_profile(&mut self, settings_path: &Path) -> Result<(), ProfileError> { pub fn ensure_default_profile(&mut self, settings_path: &Path) -> Result<(), ProfileError> {
if self.profile(DEFAULT_PROFILE_NAME).is_none() { if self.profile(DEFAULT_PROFILE_NAME).is_none() {
let mut profile = Profile::new(DEFAULT_PROFILE_NAME.to_string()); let mut profile = Profile::new(DEFAULT_PROFILE_NAME.to_string());
profile.metrics.push(MetricDeclarationRecord::seeded_error_split()); profile
.metrics
.push(MetricDeclarationRecord::seeded_error_split());
profile.dashboards.push(Dashboard::seeded_default()); profile.dashboards.push(Dashboard::seeded_default());
self.profiles.push(profile); self.profiles.push(profile);
} }
@@ -491,11 +533,11 @@ impl Settings {
settings_path: &Path, settings_path: &Path,
name: Option<String>, name: Option<String>,
) -> Result<(), ProfileError> { ) -> Result<(), ProfileError> {
if let Some(name) = &name { if let Some(name) = &name
if self.profile(name).is_none() { && self.profile(name).is_none()
{
return Err(ProfileError::NotFound); return Err(ProfileError::NotFound);
} }
}
self.active_profile = name; self.active_profile = name;
self.save(settings_path)?; self.save(settings_path)?;
Ok(()) Ok(())
@@ -678,10 +720,10 @@ impl Settings {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use crate::engine::data_value::MetricPrimitiveKind;
use crate::api::dto::LogsParams; use crate::api::dto::LogsParams;
use crate::engine::{EngineMessage, EngineQuery, EngineResponse, IngestedLine}; use crate::engine::data_value::MetricPrimitiveKind;
use crate::engine::timestamp::Timestamp; use crate::engine::timestamp::Timestamp;
use crate::engine::{EngineMessage, EngineQuery, EngineResponse, IngestedLine};
fn param_request(name: &str, pattern: &str, capture_group: &str) -> DeclareParamRequest { fn param_request(name: &str, pattern: &str, capture_group: &str) -> DeclareParamRequest {
DeclareParamRequest { DeclareParamRequest {
@@ -704,13 +746,14 @@ mod tests {
unmatched_labels: None, unmatched_labels: None,
label_params: vec![param_name.to_string()], label_params: vec![param_name.to_string()],
value_param: None, value_param: None,
semantics: MetricSemantics::default(),
} }
} }
async fn ask(engine: &EngineHandle, query: EngineQuery) -> EngineResponse { async fn ask(engine: &EngineHandle, query: EngineQuery) -> EngineResponse {
let read_handle = engine.reader.handle(); // Delegates to the real routing in `api::EngineHandle::ask` rather than reimplementing
let guard = read_handle.enter().expect("engine unavailable"); // it -- an earlier duplicate here diverged from it and silently read stale/empty data.
guard.handle_query_ro(query) engine.ask(query).await.expect("engine unavailable")
} }
#[test] #[test]
@@ -719,8 +762,12 @@ mod tests {
let settings_path = dir.path().join("settings.json"); let settings_path = dir.path().join("settings.json");
let mut settings = Settings::default(); let mut settings = Settings::default();
settings.create_profile(&settings_path, "web".to_string()).unwrap(); settings
settings.create_profile(&settings_path, "worker".to_string()).unwrap(); .create_profile(&settings_path, "web".to_string())
.unwrap();
settings
.create_profile(&settings_path, "worker".to_string())
.unwrap();
assert_eq!(settings.profiles.len(), 2); assert_eq!(settings.profiles.len(), 2);
assert!(settings.profile("web").is_some()); assert!(settings.profile("web").is_some());
@@ -750,13 +797,27 @@ mod tests {
settings.ensure_default_profile(&settings_path).unwrap(); settings.ensure_default_profile(&settings_path).unwrap();
assert!(settings.profile(DEFAULT_PROFILE_NAME).is_some()); assert!(settings.profile(DEFAULT_PROFILE_NAME).is_some());
assert_eq!(settings.active_profile, Some(DEFAULT_PROFILE_NAME.to_string())); assert_eq!(
settings.active_profile,
Some(DEFAULT_PROFILE_NAME.to_string())
);
settings.create_profile(&settings_path, "web".to_string()).unwrap(); settings
settings.set_active_profile(&settings_path, Some("web".to_string())).unwrap(); .create_profile(&settings_path, "web".to_string())
.unwrap();
settings
.set_active_profile(&settings_path, Some("web".to_string()))
.unwrap();
settings.ensure_default_profile(&settings_path).unwrap(); settings.ensure_default_profile(&settings_path).unwrap();
assert_eq!(settings.profiles.iter().filter(|p| p.name == DEFAULT_PROFILE_NAME).count(), 1); assert_eq!(
settings
.profiles
.iter()
.filter(|p| p.name == DEFAULT_PROFILE_NAME)
.count(),
1
);
assert_eq!(settings.active_profile, Some("web".to_string())); assert_eq!(settings.active_profile, Some("web".to_string()));
} }
@@ -773,20 +834,33 @@ mod tests {
assert_eq!(default_profile.metrics[0].name, SEEDED_METRIC_NAME); assert_eq!(default_profile.metrics[0].name, SEEDED_METRIC_NAME);
assert_eq!( assert_eq!(
default_profile.metrics[0].matched_labels, default_profile.metrics[0].matched_labels,
Some(BTreeMap::from([(SEEDED_METRIC_LABEL.to_string(), "error".to_string())])) Some(BTreeMap::from([(
SEEDED_METRIC_LABEL.to_string(),
"error".to_string()
)]))
); );
assert_eq!( assert_eq!(
default_profile.metrics[0].unmatched_labels, default_profile.metrics[0].unmatched_labels,
Some(BTreeMap::from([(SEEDED_METRIC_LABEL.to_string(), "other".to_string())])) Some(BTreeMap::from([(
SEEDED_METRIC_LABEL.to_string(),
"other".to_string()
)]))
); );
assert_eq!(default_profile.dashboards.len(), 1); assert_eq!(default_profile.dashboards.len(), 1);
let dashboard = &default_profile.dashboards[0]; let dashboard = &default_profile.dashboards[0];
assert_eq!(dashboard.panels.len(), 2); assert_eq!(dashboard.panels.len(), 2);
assert!(dashboard assert!(
dashboard
.panels .panels
.iter() .iter()
.any(|p| p.chart_type == ChartType::Line && p.promql.contains(SEEDED_METRIC_NAME))); .any(|p| p.chart_type == ChartType::Line && p.promql.contains(SEEDED_METRIC_NAME))
assert!(dashboard.panels.iter().any(|p| p.chart_type == ChartType::Pie && p.promql == SEEDED_METRIC_NAME)); );
assert!(
dashboard
.panels
.iter()
.any(|p| p.chart_type == ChartType::Pie && p.promql == SEEDED_METRIC_NAME)
);
let mut reloaded = Settings::load(&settings_path); let mut reloaded = Settings::load(&settings_path);
reloaded.ensure_default_profile(&settings_path).unwrap(); reloaded.ensure_default_profile(&settings_path).unwrap();
@@ -797,8 +871,8 @@ mod tests {
#[tokio::test] #[tokio::test]
async fn seeded_metric_renders_real_error_vs_other_data() { async fn seeded_metric_renders_real_error_vs_other_data() {
use crate::engine::{self, EngineMessage, EngineQuery, EngineResponse, IngestedLine};
use crate::engine::timestamp::Timestamp; use crate::engine::timestamp::Timestamp;
use crate::engine::{self, EngineMessage, EngineQuery, EngineResponse, IngestedLine};
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();
let settings_path = dir.path().join("settings.json"); let settings_path = dir.path().join("settings.json");
@@ -810,13 +884,17 @@ mod tests {
let engine_tx = engine::spawn(settings_handle, 128); let engine_tx = engine::spawn(settings_handle, 128);
default_profile.load_into_engine(&engine_tx).await; default_profile.load_into_engine(&engine_tx).await;
for (sequence, text) in [(0u64, "all good"), (1, "an ERROR happened"), (2, "fine again")] { for (sequence, text) in [
(0u64, "all good"),
(1, "an ERROR happened"),
(2, "fine again"),
] {
engine_tx engine_tx
.tx .tx
.send(EngineMessage::Ingest(IngestedLine { .send(EngineMessage::Ingest(IngestedLine {
sequence, sequence,
timestamp: Timestamp::from_millis(sequence as i64), timestamp: Timestamp::from_millis(sequence as i64),
text: text.to_string(), text: text.into(),
})) }))
.await .await
.unwrap(); .unwrap();
@@ -825,18 +903,38 @@ mod tests {
let response = ask( let response = ask(
&engine_tx, &engine_tx,
EngineQuery::Promql { query: SEEDED_METRIC_NAME.to_string(), start_ms: None, end_ms: None }, EngineQuery::Promql {
query: SEEDED_METRIC_NAME.to_string(),
start_ms: None,
end_ms: None,
},
) )
.await; .await;
let EngineResponse::Promql(Ok(mut series)) = response else { let EngineResponse::Promql(Ok(mut series)) = response else {
panic!("expected a successful Promql response"); panic!("expected a successful Promql response");
}; };
series.sort_by(|a, b| a.labels.get(SEEDED_METRIC_LABEL).cmp(&b.labels.get(SEEDED_METRIC_LABEL))); series.sort_by(|a, b| {
a.labels
.get(SEEDED_METRIC_LABEL)
.cmp(&b.labels.get(SEEDED_METRIC_LABEL))
});
assert_eq!(series.len(), 2); assert_eq!(series.len(), 2);
assert_eq!(series[0].labels.get(SEEDED_METRIC_LABEL).map(String::as_str), Some("error")); assert_eq!(
series[0]
.labels
.get(SEEDED_METRIC_LABEL)
.map(String::as_str),
Some("error")
);
assert_eq!(series[0].points.len(), 1); assert_eq!(series[0].points.len(), 1);
assert_eq!(series[1].labels.get(SEEDED_METRIC_LABEL).map(String::as_str), Some("other")); assert_eq!(
series[1]
.labels
.get(SEEDED_METRIC_LABEL)
.map(String::as_str),
Some("other")
);
assert_eq!(series[1].points.len(), 2); assert_eq!(series[1].points.len(), 2);
} }
@@ -859,19 +957,31 @@ mod tests {
let settings_path = dir.path().join("settings.json"); let settings_path = dir.path().join("settings.json");
let mut settings = Settings::default(); let mut settings = Settings::default();
settings.create_profile(&settings_path, "web".to_string()).unwrap(); settings
.create_profile(&settings_path, "web".to_string())
.unwrap();
let param_req = param_request("level", r"level=(?P<level>\w+)", "level"); let param_req = param_request("level", r"level=(?P<level>\w+)", "level");
settings.record_param_declaration(&settings_path, "web", &param_req).unwrap(); settings
settings.create_dashboard(&settings_path, "web", "Overview".to_string()).unwrap(); .record_param_declaration(&settings_path, "web", &param_req)
settings.set_active_profile(&settings_path, Some("web".to_string())).unwrap(); .unwrap();
settings
.create_dashboard(&settings_path, "web", "Overview".to_string())
.unwrap();
settings
.set_active_profile(&settings_path, Some("web".to_string()))
.unwrap();
settings.create_profile(&settings_path, "web-copy".to_string()).unwrap(); settings
.create_profile(&settings_path, "web-copy".to_string())
.unwrap();
let copy = settings.profile("web-copy").unwrap().clone(); let copy = settings.profile("web-copy").unwrap().clone();
assert_eq!(copy.params, settings.profile("web").unwrap().params); assert_eq!(copy.params, settings.profile("web").unwrap().params);
assert_eq!(copy.dashboards, settings.profile("web").unwrap().dashboards); assert_eq!(copy.dashboards, settings.profile("web").unwrap().dashboards);
settings.set_active_profile(&settings_path, None).unwrap(); settings.set_active_profile(&settings_path, None).unwrap();
settings.create_profile(&settings_path, "blank".to_string()).unwrap(); settings
.create_profile(&settings_path, "blank".to_string())
.unwrap();
let blank = settings.profile("blank").unwrap(); let blank = settings.profile("blank").unwrap();
assert!(blank.params.is_empty()); assert!(blank.params.is_empty());
assert!(blank.dashboards.is_empty()); assert!(blank.dashboards.is_empty());
@@ -883,13 +993,21 @@ mod tests {
let settings_path = dir.path().join("settings.json"); let settings_path = dir.path().join("settings.json");
let mut settings = Settings::default(); let mut settings = Settings::default();
settings.create_profile(&settings_path, "web".to_string()).unwrap(); settings
settings.create_profile(&settings_path, "worker".to_string()).unwrap(); .create_profile(&settings_path, "web".to_string())
.unwrap();
settings
.create_profile(&settings_path, "worker".to_string())
.unwrap();
settings.set_active_profile(&settings_path, Some("web".to_string())).unwrap(); settings
.set_active_profile(&settings_path, Some("web".to_string()))
.unwrap();
assert_eq!(settings.active_profile, Some("web".to_string())); assert_eq!(settings.active_profile, Some("web".to_string()));
settings.set_active_profile(&settings_path, Some("worker".to_string())).unwrap(); settings
.set_active_profile(&settings_path, Some("worker".to_string()))
.unwrap();
let reloaded = Settings::load(&settings_path); let reloaded = Settings::load(&settings_path);
assert_eq!(reloaded.active_profile, Some("worker".to_string())); assert_eq!(reloaded.active_profile, Some("worker".to_string()));
@@ -907,13 +1025,19 @@ mod tests {
let settings_path = dir.path().join("settings.json"); let settings_path = dir.path().join("settings.json");
let mut settings = Settings::default(); let mut settings = Settings::default();
settings.create_profile(&settings_path, "web".to_string()).unwrap(); settings
.create_profile(&settings_path, "web".to_string())
.unwrap();
let param_req = param_request("level", r"level=(?P<level>\w+)", "level"); let param_req = param_request("level", r"level=(?P<level>\w+)", "level");
settings.record_param_declaration(&settings_path, "web", &param_req).unwrap(); settings
.record_param_declaration(&settings_path, "web", &param_req)
.unwrap();
let metric_req = metric_request("requests", r"duration=(?P<duration>\d+)", "duration"); let metric_req = metric_request("requests", r"duration=(?P<duration>\d+)", "duration");
settings.record_metric_declaration(&settings_path, "web", &metric_req, &[]).unwrap(); settings
.record_metric_declaration(&settings_path, "web", &metric_req, &[])
.unwrap();
let profile = settings.profile("web").unwrap(); let profile = settings.profile("web").unwrap();
assert_eq!(profile.params.len(), 1); assert_eq!(profile.params.len(), 1);
@@ -922,7 +1046,9 @@ mod tests {
assert_eq!(profile.metrics[0].name, "requests"); assert_eq!(profile.metrics[0].name, "requests");
let param_req_v2 = param_request("level", r"lvl=(?P<level>\w+)", "level"); let param_req_v2 = param_request("level", r"lvl=(?P<level>\w+)", "level");
settings.record_param_declaration(&settings_path, "web", &param_req_v2).unwrap(); settings
.record_param_declaration(&settings_path, "web", &param_req_v2)
.unwrap();
let profile = settings.profile("web").unwrap(); let profile = settings.profile("web").unwrap();
assert_eq!(profile.params.len(), 1); assert_eq!(profile.params.len(), 1);
assert_eq!(profile.params[0].message_match_regex, r"lvl=(?P<level>\w+)"); assert_eq!(profile.params[0].message_match_regex, r"lvl=(?P<level>\w+)");
@@ -939,15 +1065,23 @@ mod tests {
let settings_path = dir.path().join("settings.json"); let settings_path = dir.path().join("settings.json");
let mut settings = Settings::default(); let mut settings = Settings::default();
settings.create_profile(&settings_path, "web".to_string()).unwrap(); settings
.create_profile(&settings_path, "web".to_string())
.unwrap();
let metric_req = metric_request("requests", r"duration=(?P<duration>\d+)", "duration"); let metric_req = metric_request("requests", r"duration=(?P<duration>\d+)", "duration");
settings.record_metric_declaration(&settings_path, "web", &metric_req, &[]).unwrap(); settings
.record_metric_declaration(&settings_path, "web", &metric_req, &[])
.unwrap();
assert_eq!(settings.profile("web").unwrap().metrics.len(), 1); assert_eq!(settings.profile("web").unwrap().metrics.len(), 1);
settings.remove_metric_declaration(&settings_path, "web", "requests").unwrap(); settings
.remove_metric_declaration(&settings_path, "web", "requests")
.unwrap();
assert!(settings.profile("web").unwrap().metrics.is_empty()); assert!(settings.profile("web").unwrap().metrics.is_empty());
settings.remove_metric_declaration(&settings_path, "web", "never-declared").unwrap(); settings
.remove_metric_declaration(&settings_path, "web", "never-declared")
.unwrap();
let reloaded = Settings::load(&settings_path); let reloaded = Settings::load(&settings_path);
assert!(reloaded.profile("web").unwrap().metrics.is_empty()); assert!(reloaded.profile("web").unwrap().metrics.is_empty());
@@ -959,7 +1093,9 @@ mod tests {
let settings_path = dir.path().join("settings.json"); let settings_path = dir.path().join("settings.json");
let mut settings = Settings::default(); let mut settings = Settings::default();
settings.create_profile(&settings_path, "web".to_string()).unwrap(); settings
.create_profile(&settings_path, "web".to_string())
.unwrap();
settings settings
.record_param_declaration( .record_param_declaration(
&settings_path, &settings_path,
@@ -971,18 +1107,28 @@ mod tests {
.record_param_declaration( .record_param_declaration(
&settings_path, &settings_path,
"web", "web",
&param_request("duration", r"request duration=(?P<duration>\d+)", "duration"), &param_request(
"duration",
r"request duration=(?P<duration>\d+)",
"duration",
),
) )
.unwrap(); .unwrap();
settings settings
.record_metric_declaration( .record_metric_declaration(
&settings_path, &settings_path,
"web", "web",
&metric_request("requests", r"request duration=(?P<duration>\d+)", "duration"), &metric_request(
"requests",
r"request duration=(?P<duration>\d+)",
"duration",
),
&[], &[],
) )
.unwrap(); .unwrap();
settings.set_active_profile(&settings_path, Some("web".to_string())).unwrap(); settings
.set_active_profile(&settings_path, Some("web".to_string()))
.unwrap();
drop(settings); drop(settings);
let settings = Settings::load(&settings_path); let settings = Settings::load(&settings_path);
@@ -998,7 +1144,7 @@ mod tests {
.send(EngineMessage::Ingest(IngestedLine { .send(EngineMessage::Ingest(IngestedLine {
sequence: 0, sequence: 0,
timestamp: Timestamp::from_millis(0), timestamp: Timestamp::from_millis(0),
text: "level=info request duration=42".to_string(), text: "level=info request duration=42".into(),
})) }))
.await .await
.unwrap(); .unwrap();
@@ -1020,7 +1166,10 @@ mod tests {
}; };
assert_eq!(metrics.len(), 1); assert_eq!(metrics.len(), 1);
assert_eq!(metrics[0].name, "requests"); assert_eq!(metrics[0].name, "requests");
assert_eq!(metrics[0].message_match_regex, r"request duration=(?P<duration>\d+)"); assert_eq!(
metrics[0].message_match_regex,
r"request duration=(?P<duration>\d+)"
);
assert_eq!(metrics[0].label_params, vec!["duration".to_string()]); assert_eq!(metrics[0].label_params, vec!["duration".to_string()]);
let logs = match ask( let logs = match ask(
@@ -1050,9 +1199,13 @@ mod tests {
let settings_path = dir.path().join("settings.json"); let settings_path = dir.path().join("settings.json");
let mut settings = Settings::default(); let mut settings = Settings::default();
settings.create_profile(&settings_path, "web".to_string()).unwrap(); settings
.create_profile(&settings_path, "web".to_string())
.unwrap();
let dashboard = settings.create_dashboard(&settings_path, "web", "Overview".to_string()).unwrap(); let dashboard = settings
.create_dashboard(&settings_path, "web", "Overview".to_string())
.unwrap();
assert_eq!(dashboard.name, "Overview"); assert_eq!(dashboard.name, "Overview");
assert!(dashboard.panels.is_empty()); assert!(dashboard.panels.is_empty());
@@ -1077,7 +1230,9 @@ mod tests {
assert_eq!(dashboards[0].panels.len(), 1); assert_eq!(dashboards[0].panels.len(), 1);
assert_eq!(dashboards[0].panels[0].promql, r#"requests{path="/a"}"#); assert_eq!(dashboards[0].panels[0].promql, r#"requests{path="/a"}"#);
reloaded.delete_dashboard(&settings_path, "web", &dashboard.id).unwrap(); reloaded
.delete_dashboard(&settings_path, "web", &dashboard.id)
.unwrap();
assert!(reloaded.list_dashboards("web").unwrap().is_empty()); assert!(reloaded.list_dashboards("web").unwrap().is_empty());
let reloaded_again = Settings::load(&settings_path); let reloaded_again = Settings::load(&settings_path);
assert!(reloaded_again.list_dashboards("web").unwrap().is_empty()); assert!(reloaded_again.list_dashboards("web").unwrap().is_empty());
@@ -1125,16 +1280,39 @@ mod tests {
let profile = settings.profile("legacy").unwrap(); let profile = settings.profile("legacy").unwrap();
assert_eq!(profile.params.len(), 2); assert_eq!(profile.params.len(), 2);
let path_param = profile.params.iter().find(|p| p.name == "requests.path").unwrap(); let path_param = profile
assert_eq!(path_param.message_match_regex, profile.metrics[0].message_match_regex); .params
.iter()
.find(|p| p.name == "requests.path")
.unwrap();
assert_eq!(
path_param.message_match_regex,
profile.metrics[0].message_match_regex
);
assert_eq!(path_param.capture_group, "path"); assert_eq!(path_param.capture_group, "path");
assert!(matches!(path_param.kind, ProfileMetricPrimitiveKind::String)); assert!(matches!(
let duration_param = profile.params.iter().find(|p| p.name == "requests.duration").unwrap(); path_param.kind,
assert!(matches!(duration_param.kind, ProfileMetricPrimitiveKind::Number)); ProfileMetricPrimitiveKind::String
));
let duration_param = profile
.params
.iter()
.find(|p| p.name == "requests.duration")
.unwrap();
assert!(matches!(
duration_param.kind,
ProfileMetricPrimitiveKind::Number
));
assert_eq!(profile.metrics.len(), 1); assert_eq!(profile.metrics.len(), 1);
assert_eq!(profile.metrics[0].label_params, vec!["requests.path".to_string()]); assert_eq!(
assert_eq!(profile.metrics[0].value_param, Some("requests.duration".to_string())); profile.metrics[0].label_params,
vec!["requests.path".to_string()]
);
assert_eq!(
profile.metrics[0].value_param,
Some("requests.duration".to_string())
);
let reloaded_raw = fs::read_to_string(&settings_path).unwrap(); let reloaded_raw = fs::read_to_string(&settings_path).unwrap();
assert!(!reloaded_raw.contains("param_name")); assert!(!reloaded_raw.contains("param_name"));
@@ -1149,7 +1327,7 @@ mod tests {
.send(EngineMessage::Ingest(IngestedLine { .send(EngineMessage::Ingest(IngestedLine {
sequence: 0, sequence: 0,
timestamp: Timestamp::from_millis(0), timestamp: Timestamp::from_millis(0),
text: "request path=/a duration=42".to_string(), text: "request path=/a duration=42".into(),
})) }))
.await .await
.unwrap(); .unwrap();
@@ -1192,10 +1370,20 @@ mod tests {
let settings = Settings::load(&settings_path); let settings = Settings::load(&settings_path);
let profile = settings.profile("legacy").unwrap(); let profile = settings.profile("legacy").unwrap();
let gap_param = profile.params.iter().find(|p| p.name == "login_gap.gap").unwrap(); let gap_param = profile
assert!(matches!(gap_param.kind, ProfileMetricPrimitiveKind::Duration)); .params
.iter()
.find(|p| p.name == "login_gap.gap")
.unwrap();
assert!(matches!(
gap_param.kind,
ProfileMetricPrimitiveKind::Duration
));
assert!(gap_param.elapsed_since_last); assert!(gap_param.elapsed_since_last);
assert_eq!(profile.metrics[0].value_param, Some("login_gap.gap".to_string())); assert_eq!(
profile.metrics[0].value_param,
Some("login_gap.gap".to_string())
);
} }
#[test] #[test]
@@ -1229,7 +1417,14 @@ mod tests {
let gap_param = profile.params.iter().find(|p| p.name == "gap").unwrap(); let gap_param = profile.params.iter().find(|p| p.name == "gap").unwrap();
assert!(gap_param.elapsed_since_last); assert!(gap_param.elapsed_since_last);
assert!(!profile.params.iter().find(|p| p.name == "user").unwrap().elapsed_since_last); assert!(
!profile
.params
.iter()
.find(|p| p.name == "user")
.unwrap()
.elapsed_since_last
);
} }
#[test] #[test]
@@ -1271,7 +1466,9 @@ mod tests {
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();
let settings_path = dir.path().join("settings.json"); let settings_path = dir.path().join("settings.json");
let mut settings = Settings::default(); let mut settings = Settings::default();
settings.create_profile(&settings_path, "web".to_string()).unwrap(); settings
.create_profile(&settings_path, "web".to_string())
.unwrap();
settings settings
.record_param_declaration( .record_param_declaration(
&settings_path, &settings_path,
@@ -1280,7 +1477,12 @@ mod tests {
) )
.unwrap(); .unwrap();
settings settings
.record_metric_declaration(&settings_path, "web", &metric_request("requests", r"path=(?P<path>\S+)", "path"), &[]) .record_metric_declaration(
&settings_path,
"web",
&metric_request("requests", r"path=(?P<path>\S+)", "path"),
&[],
)
.unwrap(); .unwrap();
let before = settings.profile("web").unwrap().clone(); let before = settings.profile("web").unwrap().clone();
@@ -1295,8 +1497,12 @@ mod tests {
let settings_path = dir.path().join("settings.json"); let settings_path = dir.path().join("settings.json");
let mut settings = Settings::default(); let mut settings = Settings::default();
settings.create_profile(&settings_path, "web".to_string()).unwrap(); settings
settings.set_active_profile(&settings_path, Some("web".to_string())).unwrap(); .create_profile(&settings_path, "web".to_string())
.unwrap();
settings
.set_active_profile(&settings_path, Some("web".to_string()))
.unwrap();
let settings_handle = std::sync::Arc::new(std::sync::Mutex::new(settings.clone())); let settings_handle = std::sync::Arc::new(std::sync::Mutex::new(settings.clone()));
let engine_tx = crate::engine::spawn(settings_handle, 16); let engine_tx = crate::engine::spawn(settings_handle, 16);
@@ -1305,7 +1511,7 @@ mod tests {
.send(EngineMessage::Ingest(IngestedLine { .send(EngineMessage::Ingest(IngestedLine {
sequence: 0, sequence: 0,
timestamp: Timestamp::from_millis(0), timestamp: Timestamp::from_millis(0),
text: "dx=1 moved".to_string(), text: "dx=1 moved".into(),
})) }))
.await .await
.unwrap(); .unwrap();
@@ -1314,7 +1520,10 @@ mod tests {
let (respond_to, response) = oneshot::channel(); let (respond_to, response) = oneshot::channel();
engine_tx engine_tx
.tx .tx
.send(EngineMessage::DeclareMetric { request: metric_req.clone(), respond_to }) .send(EngineMessage::DeclareMetric {
request: metric_req.clone(),
respond_to,
})
.await .await
.unwrap(); .unwrap();
response.await.unwrap().unwrap(); response.await.unwrap().unwrap();
@@ -1334,7 +1543,9 @@ mod tests {
kind: dx.kind.into(), kind: dx.kind.into(),
elapsed_since_last: dx.elapsed_since_last, elapsed_since_last: dx.elapsed_since_last,
}; };
settings.record_metric_declaration(&settings_path, "web", &metric_req, &[promoted]).unwrap(); settings
.record_metric_declaration(&settings_path, "web", &metric_req, &[promoted])
.unwrap();
drop(settings); drop(settings);
let reloaded_settings = Settings::load(&settings_path); let reloaded_settings = Settings::load(&settings_path);
+46
View File
@@ -0,0 +1,46 @@
//! The two actor tasks that own all live engine state, plus the `left_right` read plumbing they
//! share.
//!
//! [`primary`] owns ingested log lines and param declarations; [`profile`] owns metric
//! declarations, bucket data, and alert evaluation (moved off the primary per tickets 103-105,
//! see `profile`'s module doc). Each is a `tokio::spawn`ed loop reading from its own `mpsc`
//! channel -- there is no shared mutable state accessed directly by callers, only the handles
//! and read-only [`IndexReader`]s below.
use crate::engine::snapshot::EngineSnapshot;
pub(crate) mod primary;
pub(crate) mod profile;
/// A read-only view over one actor's [`EngineSnapshot`], obtained via [`IndexReader::handle`].
/// Cheap to acquire per-query; the actual zero-copy read guard comes from `enter()`.
pub struct IndexReadHandle(left_right::ReadHandle<EngineSnapshot>);
impl IndexReadHandle {
pub(crate) fn enter(&self) -> Option<left_right::ReadGuard<'_, EngineSnapshot>> {
self.0.enter()
}
}
/// A cloneable factory for [`IndexReadHandle`]s over one actor's `EngineSnapshot` -- every
/// actor (primary and profile) publishes through its own `left_right::WriteHandle` but hands
/// out reads through one of these. Not generic over the `left_right::Absorb` operation type,
/// since `left_right::ReadHandle` itself isn't -- only the write side needs to know how a
/// snapshot gets mutated.
pub struct IndexReader {
pub(crate) factory: left_right::ReadHandleFactory<EngineSnapshot>,
}
impl Clone for IndexReader {
fn clone(&self) -> Self {
IndexReader {
factory: self.factory.clone(),
}
}
}
impl IndexReader {
pub(crate) fn handle(&self) -> IndexReadHandle {
IndexReadHandle(self.factory.handle())
}
}
+426
View File
@@ -0,0 +1,426 @@
//! The primary actor: owns ingested log lines and param declarations, and the `EngineHandle`
//! other tasks use to talk to it (and, by proxy, to `super::profile`).
//!
//! Ticket 106 (giving API handlers a route to both the primary and profile actors) resolved to
//! option (b) from its own scope note: one combined `EngineHandle`, not two separate handle
//! types. Callers never pick a target explicitly -- `EngineHandle::tx` accepts every
//! `EngineMessage` variant, and `run()` below internally proxies the metric/alert-related ones
//! to the profile actor (`super::profile::ProfileMessage`) rather than mutating the primary's
//! own snapshot; `EngineHandle::profile_reader` similarly gives read access to the profile
//! actor's snapshot for the query variants it owns (routed in `api::EngineHandle::ask`). This
//! means `reload_engine_declarations`/`activate_profile`/`Profile::load_into_engine` needed no
//! changes at all to correctly split their remove-then-redeclare sequence across both actors --
//! they already just send `EngineMessage`s the same way they did before the split.
use super::IndexReader;
use crate::config::SettingsHandle;
use crate::consts::{INGEST_LAG_WATERMARK_MAX_ENTRIES, INGEST_LAG_WATERMARK_WINDOW_MS};
use crate::engine::index_key::IndexKey;
use crate::engine::message::EngineMessage;
use crate::engine::metric_primitive::{MetricPrimitive, Number};
use crate::engine::snapshot::EngineSnapshot;
use crate::engine::snapshot::params::log_stream_header_transition;
use crate::engine::timestamp::Timestamp;
use crate::engine::{ClearParamsResult, IngestedLine};
use left_right::Absorb;
use std::collections::BTreeMap;
use std::sync::Arc;
use tokio::sync::{mpsc, oneshot, watch};
struct EngineOp(Box<dyn FnMut(&mut EngineSnapshot) + Send + 'static>);
impl Absorb<EngineOp> for EngineSnapshot {
fn absorb_first(&mut self, operation: &mut EngineOp, _other: &Self) {
(operation.0)(self);
}
fn drop_first(self: Box<Self>) {}
fn sync_with(&mut self, first: &Self) {
*self = first.clone();
}
}
struct IndexWriter {
handle: left_right::WriteHandle<EngineSnapshot, EngineOp>,
/// Bumped and re-sent on every publish so the profile actor (`super::profile`) can wake on
/// change rather than polling; the value itself carries no meaning beyond "something new is
/// visible; go re-read what you need through your own `IndexReader`."
version: u64,
version_tx: watch::Sender<u64>,
}
impl IndexWriter {
fn mutate(&mut self, op: impl FnMut(&mut EngineSnapshot) + Send + 'static) {
self.handle.append(EngineOp(Box::new(op)));
self.handle.publish();
self.version += 1;
let _ = self.version_tx.send(self.version);
}
fn peek(&mut self) -> Option<left_right::ReadGuard<'_, EngineSnapshot>> {
self.handle.enter()
}
}
pub struct EngineHandle {
pub tx: mpsc::Sender<EngineMessage>,
pub reader: IndexReader,
/// Read-only view of the profile actor's snapshot (`super::profile`), which owns metric
/// declarations and bucket data (ticket 104) — `MetricDeclarations`/`BucketValues`/`Promql`
/// queries read through this instead of `reader`. Not part of the write path; the profile
/// actor's own `mpsc` channel (proxied through `EngineMessage`'s metric variants) is that.
pub(crate) profile_reader: IndexReader,
/// The profile actor's own processed-sequence watermark (`ProfileHandle::processed_sequence`)
/// — see `wait_for_profile_sequence` below.
pub(crate) profile_processed_sequence: watch::Receiver<Option<u64>>,
}
impl Clone for EngineHandle {
fn clone(&self) -> Self {
EngineHandle {
tx: self.tx.clone(),
reader: self.reader.clone(),
profile_reader: self.profile_reader.clone(),
profile_processed_sequence: self.profile_processed_sequence.clone(),
}
}
}
impl EngineHandle {
pub async fn barrier(&self) {
let (respond_to, response) = oneshot::channel();
if self
.tx
.send(EngineMessage::Barrier(respond_to))
.await
.is_ok()
{
let _ = response.await;
}
}
/// Waits until the profile actor has finished processing primary sequence `sequence` (metrics
/// computed, alerts evaluated) — ticket 108's deterministic alternative to a sleep for tests
/// that assert on profile-actor-owned data after ingesting; `barrier()` alone only confirms
/// the *primary*'s queue drained, not that the profile actor has caught up to what it published.
#[cfg(test)]
pub(crate) async fn wait_for_profile_sequence(&self, sequence: u64) {
let mut rx = self.profile_processed_sequence.clone();
while *rx.borrow() < Some(sequence) {
if rx.changed().await.is_err() {
return; // profile actor gone
}
}
}
}
/// Builds the primary actor's sender/receiver/reader/writer/version-receiver, shared by `spawn`
/// and, in tests, by callers that want the raw receiver without an actor loop draining it.
/// Stops short of building `EngineHandle` itself, since that also needs a profile-actor reader
/// that doesn't exist until the caller spawns (or fakes, see `test_handle`) the profile actor.
fn new_channel_and_handle(
channel_capacity: usize,
) -> (
mpsc::Sender<EngineMessage>,
mpsc::Receiver<EngineMessage>,
IndexReader,
IndexWriter,
watch::Receiver<u64>,
) {
let (tx, rx) = mpsc::channel(channel_capacity);
let (write, read) = left_right::new::<EngineSnapshot, EngineOp>();
let reader = IndexReader {
factory: read.factory(),
};
let (version_tx, version_rx) = watch::channel(0u64);
(
tx,
rx,
reader,
IndexWriter {
handle: write,
version: 0,
version_tx,
},
version_rx,
)
}
pub fn spawn(settings: SettingsHandle, channel_capacity: usize) -> EngineHandle {
let (tx, rx, reader, write, version_rx) = new_channel_and_handle(channel_capacity);
let profile = super::profile::spawn(reader.clone(), version_rx, settings);
let handle = EngineHandle {
tx,
reader,
profile_reader: profile.reader.clone(),
profile_processed_sequence: profile.processed_sequence.clone(),
};
tokio::spawn(run(rx, write, profile));
handle
}
#[cfg(test)]
pub(crate) fn test_handle(
channel_capacity: usize,
) -> (EngineHandle, mpsc::Receiver<EngineMessage>) {
let (tx, rx, reader, _write, _version_rx) = new_channel_and_handle(channel_capacity);
// No actor loop is spawned here at all, so nothing ever publishes -- an inert reader over
// an always-empty snapshot is all a query through this handle could see anyway.
let (_processed_tx, processed_rx) = watch::channel(None);
let handle = EngineHandle {
tx,
reader,
profile_reader: super::profile::inert_reader(),
profile_processed_sequence: processed_rx,
};
(handle, rx)
}
/// The primary actor's loop: extracts declared + auto params only from each ingested line (per
/// `ideas/0001-actor-split.md`'s resolved decision) -- no metrics, no alert evaluation, both of
/// which live on `super::profile` and read this actor's publishes via `IndexReader`/`watch`.
async fn run(
mut rx: mpsc::Receiver<EngineMessage>,
mut write: IndexWriter,
profile: crate::engine::actor::profile::ProfileHandle,
) {
let mut max_logs: Option<usize> = None;
let mut metric_sample_sequence: u64 = 0;
let mut log_stream_header: Option<crate::engine::logfmt::LogStreamHeader> = None;
let mut ingest_lag_samples: Vec<(Timestamp, i64)> = Vec::new();
while let Some(message) = rx.recv().await {
match message {
EngineMessage::Ingest(line) => {
let IngestedLine {
sequence,
timestamp,
text,
} = line;
let processed_at = Timestamp::now();
record_ingest_lag(
&mut ingest_lag_samples,
processed_at,
(processed_at.as_millis() - timestamp.as_millis()).max(0),
);
let ingest_lag_high_watermark_ms =
ingest_lag_high_watermark_ms(&ingest_lag_samples, processed_at);
let is_new_header = crate::engine::logfmt::LogStreamHeader::detect(&text).is_some();
log_stream_header = log_stream_header_transition(&log_stream_header, &text);
let row_header = if is_new_header {
None
} else {
log_stream_header.clone()
};
let text_for_op = Arc::clone(&text);
write.mutate(move |snap: &mut EngineSnapshot| {
// Primary/param extraction only (ticket 101) -- metric computation
// (ticket 104) and alert evaluation (ticket 105) now happen on the
// profile actor, which wakes on this publish and reads the
// newly-visible line itself.
snap.ingest_params(sequence, timestamp, &text_for_op, row_header.as_ref());
snap.update_sort_helper_indexes_on_ingest(sequence, timestamp, &text_for_op);
snap.purge_stale_sort_helper_indexes();
snap.evict_excess_logs(max_logs);
snap.ingest_lag_high_watermark_ms = ingest_lag_high_watermark_ms;
});
}
EngineMessage::IngestMetricSample(sample) => {
let labels: BTreeMap<String, MetricPrimitive> = sample
.labels
.into_iter()
.map(|(name, value)| (name, MetricPrimitive::String(value)))
.collect();
let metric_name = sample.metric_name;
let sequence = metric_sample_sequence;
let timestamp = sample.timestamp;
let value = MetricPrimitive::Number(Number::Float(sample.value));
write.mutate(move |snap: &mut EngineSnapshot| {
snap.push_bucket_value(
&metric_name,
labels.clone(),
sequence,
timestamp,
value.clone(),
);
});
metric_sample_sequence += 1;
}
EngineMessage::SetMaxLogs(new_max) => {
max_logs = new_max;
write.mutate(move |snap: &mut EngineSnapshot| {
snap.evict_excess_logs(new_max);
});
}
EngineMessage::DeclareParam {
request,
respond_to,
} => {
let result = write
.peek()
.map(|guard| guard.validate_param_declaration(&request))
.unwrap_or_else(|| Err("engine is shutting down".to_string()));
let succeeded = result.is_ok();
let name = request.name.clone();
let request_for_op = request.clone();
write.mutate(move |snap: &mut EngineSnapshot| {
snap.sort_helper_indexes.remove(&name);
if succeeded {
snap.insert_param_declaration(request_for_op.clone());
}
});
let _ = respond_to.send(result);
}
EngineMessage::RemoveParamDeclaration { name, respond_to } => {
let existed = write.peek().is_some_and(|guard| {
guard
.index
.contains_key(&IndexKey::ParamDeclaration { name: name.clone() })
});
let name_for_op = name.clone();
write.mutate(move |snap: &mut EngineSnapshot| {
snap.index.remove(&IndexKey::ParamDeclaration {
name: name_for_op.clone(),
});
snap.sort_helper_indexes.remove(&name_for_op);
snap.param_regex_cache.remove(&name_for_op);
});
let _ = respond_to.send(existed);
}
EngineMessage::ClearParams { respond_to } => {
let mut scratch = EngineSnapshot::from_index(
write
.peek()
.map(|guard| guard.index.clone())
.unwrap_or_default(),
);
let removed = scratch.clear_all_param_declarations();
let (reextracted, logs_considered) = scratch.reextract_params_from_retained_logs();
write.mutate(move |snap: &mut EngineSnapshot| {
snap.clear_all_param_declarations();
snap.reextract_params_from_retained_logs();
snap.sort_helper_indexes.clear();
});
let _ = respond_to.send(ClearParamsResult {
removed,
reextracted,
logs_considered,
});
}
EngineMessage::DeclareMetric {
request,
respond_to,
} => {
// Metric declarations live on the profile actor (ticket 104), not here --
// proxy the message rather than mutating the primary's snapshot.
let _ = profile
.tx
.send(crate::engine::actor::profile::ProfileMessage::DeclareMetric {
request,
respond_to,
})
.await;
}
EngineMessage::UpdateMetric {
name,
request,
respond_to,
} => {
let _ = profile
.tx
.send(crate::engine::actor::profile::ProfileMessage::UpdateMetric {
name,
request,
respond_to,
})
.await;
}
EngineMessage::RemoveMetricDeclaration { name, respond_to } => {
let _ = profile
.tx
.send(
crate::engine::actor::profile::ProfileMessage::RemoveMetricDeclaration {
name,
respond_to,
},
)
.await;
}
EngineMessage::EnsureSortHelperIndex {
param_name,
respond_to,
} => {
write.mutate(move |snap: &mut EngineSnapshot| {
snap.ensure_sort_helper_index_built(&param_name);
});
let _ = respond_to.send(());
}
EngineMessage::Barrier(respond_to) => {
let _ = respond_to.send(());
}
}
}
}
pub(crate) fn record_ingest_lag(samples: &mut Vec<(Timestamp, i64)>, now: Timestamp, lag_ms: i64) {
samples.retain(|(recorded_at, _)| {
now.as_millis() - recorded_at.as_millis() < INGEST_LAG_WATERMARK_WINDOW_MS
});
if samples.len() < INGEST_LAG_WATERMARK_MAX_ENTRIES {
samples.push((now, lag_ms));
return;
}
let Some((min_index, &(_, min_lag))) =
samples.iter().enumerate().min_by_key(|(_, (_, lag))| *lag)
else {
return;
};
if lag_ms > min_lag {
samples[min_index] = (now, lag_ms);
}
}
pub(crate) fn ingest_lag_high_watermark_ms(samples: &[(Timestamp, i64)], now: Timestamp) -> i64 {
samples
.iter()
.filter(|(recorded_at, _)| {
now.as_millis() - recorded_at.as_millis() < INGEST_LAG_WATERMARK_WINDOW_MS
})
.map(|(_, lag)| *lag)
.max()
.unwrap_or(0)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::config::Settings;
use std::sync::Mutex;
use std::time::Duration;
/// Proves ticket 103's wake/read loop end-to-end: the profile actor wakes on the primary's
/// publish signal and reads the newly-visible line through its own primary `IndexReader`,
/// without the primary ever being told the profile actor exists.
#[tokio::test]
async fn profile_actor_observes_newly_ingested_lines_via_the_primary_watch_signal() {
let settings: SettingsHandle = Arc::new(Mutex::new(Settings::default()));
let engine = spawn(settings, 8);
engine
.tx
.send(EngineMessage::Ingest(IngestedLine {
sequence: 0,
timestamp: Timestamp::from_millis(0),
text: "hello".into(),
}))
.await
.unwrap();
tokio::time::timeout(Duration::from_secs(2), engine.wait_for_profile_sequence(0))
.await
.expect("timed out waiting for the profile actor to observe the new line");
}
}
+348
View File
@@ -0,0 +1,348 @@
//! The profile actor (tickets 103-105). Owns metric declarations, bucket computation, and alert
//! evaluation, moved off the primary per `ideas/0001-actor-split.md`'s resolved decision that
//! the primary "extracts declared + auto params only... no metrics, no profile logic" and that
//! alerts move with metric computation since they read the buckets it produces. Reuses
//! `EngineSnapshot` itself as this actor's own snapshot type — `declare_metric`/`update_metric`/
//! `remove_metric_declaration`/`apply_all_metrics` all work unchanged — rather than inventing a
//! parallel type, since metric evaluation needs param declarations present in the same snapshot
//! it reads (`resolved_param_declarations`); this actor mirrors the primary's param declarations
//! into its own snapshot on every wake so that stays true, but stores no raw log text of its
//! own, reading it through `primary_reader` (a clone of the primary's `IndexReader`) instead.
//! `EngineHandle`'s public API is unchanged by this ticket — `DeclareMetric`/`UpdateMetric`/
//! `RemoveMetricDeclaration` are still sent to the primary's `mpsc` channel and proxied here by
//! `super::primary`'s `run()` loop; ticket 106 gives API handlers a direct route.
use super::IndexReader;
use crate::config::SettingsHandle;
use crate::engine::DeclareMetricRequest;
use crate::engine::alerts::{self, AlertRuntimeState};
use crate::engine::data_value::{DataValue, ParamDeclaration};
use crate::engine::index_key::IndexKey;
use crate::engine::snapshot::EngineSnapshot;
use crate::engine::timestamp::Timestamp;
use left_right::Absorb;
use regex::Regex;
use std::collections::HashMap;
use tokio::sync::{mpsc, oneshot, watch};
struct ProfileOp(Box<dyn FnMut(&mut EngineSnapshot) + Send + 'static>);
impl Absorb<ProfileOp> for EngineSnapshot {
fn absorb_first(&mut self, operation: &mut ProfileOp, _other: &Self) {
(operation.0)(self);
}
fn drop_first(self: Box<Self>) {}
fn sync_with(&mut self, first: &Self) {
*self = first.clone();
}
}
/// Metric-declaration messages proxied to the profile actor by the primary's `run()` loop.
pub(crate) enum ProfileMessage {
DeclareMetric {
request: DeclareMetricRequest,
respond_to: oneshot::Sender<Result<(), String>>,
},
UpdateMetric {
name: String,
request: DeclareMetricRequest,
respond_to: oneshot::Sender<Result<(), String>>,
},
RemoveMetricDeclaration {
name: String,
respond_to: oneshot::Sender<bool>,
},
}
pub(crate) struct ProfileHandle {
pub(crate) tx: mpsc::Sender<ProfileMessage>,
/// Read-only view of this actor's own snapshot (metric declarations, bucket data) --
/// `MetricDeclarations`/`BucketValues`/`Promql` queries read through this (see
/// `EngineHandle::profile_reader` in `super::primary`) instead of the primary's reader.
pub(crate) reader: IndexReader,
/// The highest primary log sequence this actor has finished processing (metrics computed,
/// alerts evaluated), or `None` before it's processed anything -- ticket 108's deterministic
/// alternative to a sleep for tests that need to wait until the profile actor has caught up
/// to a specific point, mirroring what `EngineHandle::barrier` gives the primary.
pub(crate) processed_sequence: watch::Receiver<Option<u64>>,
}
/// A read-only view over a fresh, never-written `EngineSnapshot` -- used by `primary::test_handle`,
/// which spawns no actor loop at all (so nothing would ever publish to a real profile reader
/// anyway), just to give `EngineHandle::profile_reader` a valid value.
#[cfg(test)]
pub(crate) fn inert_reader() -> IndexReader {
let (_write, read) = left_right::new::<EngineSnapshot, ProfileOp>();
IndexReader {
factory: read.factory(),
}
}
/// Spawns the profile actor. `primary_reader` is a clone of the primary's `IndexReader`, giving
/// this actor its own read-only view of the primary's index (raw log text, param declarations);
/// `primary_version` is bumped by the primary's `IndexWriter::mutate` after every `left_right`
/// publish, so this actor blocks on `changed()` between primary writes rather than polling.
/// `settings` is used to read alert rules and ntfy config fresh for each evaluated line (ticket
/// 105 -- alert evaluation moved here from the primary alongside the metric buckets it reads).
pub(crate) fn spawn(
primary_reader: IndexReader,
primary_version: watch::Receiver<u64>,
settings: SettingsHandle,
) -> ProfileHandle {
let (tx, rx) = mpsc::channel(crate::consts::ENGINE_CHANNEL_CAPACITY);
let (write, read) = left_right::new::<EngineSnapshot, ProfileOp>();
let reader = IndexReader {
factory: read.factory(),
};
let (processed_tx, processed_rx) = watch::channel(None);
tokio::spawn(run(
primary_reader,
primary_version,
rx,
write,
settings,
processed_tx,
));
ProfileHandle {
tx,
reader,
processed_sequence: processed_rx,
}
}
async fn run(
primary_reader: IndexReader,
mut primary_version: watch::Receiver<u64>,
mut rx: mpsc::Receiver<ProfileMessage>,
mut write: left_right::WriteHandle<EngineSnapshot, ProfileOp>,
settings: SettingsHandle,
processed_tx: watch::Sender<Option<u64>>,
) {
let mut last_seen_sequence: Option<u64> = None;
let mut alert_states: HashMap<String, AlertRuntimeState> = HashMap::new();
loop {
tokio::select! {
changed = primary_version.changed() => {
if changed.is_err() {
break; // primary actor gone, shut down
}
let primary_handle = primary_reader.handle();
let Some(primary) = primary_handle.enter() else {
continue;
};
mirror_param_declarations(&primary, &mut write);
let range_start = last_seen_sequence.map_or(0, |sequence| sequence.saturating_add(1));
let new_lines: Vec<(u64, Timestamp, String)> = primary
.logs_in_sequence_range(range_start..)
.map(|(sequence, timestamp, text)| (sequence, timestamp, text.to_string()))
.collect();
drop(primary);
for (sequence, timestamp, text) in new_lines {
let text_for_op = text.clone();
write.append(ProfileOp(Box::new(move |snap: &mut EngineSnapshot| {
snap.apply_all_metrics(sequence, timestamp, &text_for_op);
})));
write.publish();
last_seen_sequence = Some(sequence);
let (rules, ntfy_topic, ntfy_server) = {
let settings = settings.lock().unwrap();
(
settings.alert_rules.clone(),
settings.ntfy_topic.clone(),
settings.ntfy_server.clone(),
)
};
let fired = write
.enter()
.map(|guard| {
alerts::evaluate_alerts(
&rules,
&mut alert_states,
&guard,
sequence,
timestamp,
&text,
)
})
.unwrap_or_default();
for (rule, message) in fired {
tokio::spawn(alerts::deliver_for_rule(
rule.name,
message,
ntfy_topic.clone(),
ntfy_server.clone(),
));
}
}
if last_seen_sequence.is_some() {
processed_tx.send_replace(last_seen_sequence);
}
}
message = rx.recv() => {
let Some(message) = message else {
break; // primary actor dropped its ProfileHandle, shut down
};
handle_message(message, &primary_reader, &mut write);
}
}
}
}
fn handle_message(
message: ProfileMessage,
primary_reader: &IndexReader,
write: &mut left_right::WriteHandle<EngineSnapshot, ProfileOp>,
) {
match message {
ProfileMessage::DeclareMetric {
request,
respond_to,
} => {
// Freshen the param mirror synchronously rather than relying on the watch-wake
// loop having already caught up -- a param declared just before this metric (e.g.
// `Profile::load_into_engine` declaring params then metrics in sequence) may not
// have been mirrored yet when this message is handled, since there's no ordering
// guarantee between the wake branch and this one in `tokio::select!`.
if let Some(primary) = primary_reader.handle().enter() {
mirror_param_declarations(&primary, write);
}
let result = write
.enter()
.map(|guard| guard.validate_metric_request(&request).map(|_| ()))
.unwrap_or_else(|| Err("engine is shutting down".to_string()));
let succeeded = result.is_ok();
let lines = primary_owned_lines(primary_reader);
let request_for_op = request.clone();
write.append(ProfileOp(Box::new(move |snap: &mut EngineSnapshot| {
if succeeded {
let _ = snap.declare_metric_with_lines(request_for_op.clone(), &lines);
}
})));
write.publish();
let _ = respond_to.send(result);
}
ProfileMessage::UpdateMetric {
name,
request,
respond_to,
} => {
if let Some(primary) = primary_reader.handle().enter() {
mirror_param_declarations(&primary, write);
}
let result = write
.enter()
.map(|guard| guard.validate_update_metric(&name, &request).map(|_| ()))
.unwrap_or_else(|| Err("engine is shutting down".to_string()));
let succeeded = result.is_ok();
let lines = primary_owned_lines(primary_reader);
let existing_name = name.clone();
let request_for_op = request.clone();
write.append(ProfileOp(Box::new(move |snap: &mut EngineSnapshot| {
if succeeded {
let _ = snap.update_metric_with_lines(
&existing_name,
request_for_op.clone(),
&lines,
);
}
})));
write.publish();
let _ = respond_to.send(result);
}
ProfileMessage::RemoveMetricDeclaration { name, respond_to } => {
let existed = write.enter().is_some_and(|guard| {
guard
.index
.contains_key(&IndexKey::MetricDeclaration { name: name.clone() })
});
let name_for_op = name.clone();
write.append(ProfileOp(Box::new(move |snap: &mut EngineSnapshot| {
snap.remove_metric_declaration(&name_for_op);
})));
write.publish();
let _ = respond_to.send(existed);
}
}
}
/// Reads all currently-retained primary lines through `primary_reader`, for backfilling a
/// newly declared/updated metric — the profile actor's own snapshot holds no raw log text, so
/// this is its only source of backfill history (see `EngineSnapshot::declare_metric_with_lines`).
fn primary_owned_lines(primary_reader: &IndexReader) -> Vec<(u64, Timestamp, String)> {
let primary_handle = primary_reader.handle();
primary_handle
.enter()
.map(|guard| {
guard
.logs_range()
.map(|(sequence, timestamp, text)| (sequence, timestamp, text.to_string()))
.collect()
})
.unwrap_or_default()
}
/// Replaces the profile snapshot's mirrored `ParamDeclaration` entries and `param_regex_cache`
/// with the primary's current ones, so metric label/value resolution
/// (`EngineSnapshot::resolved_param_declarations`) sees the same params the primary has —
/// cheap enough to do unconditionally on every wake since declared params are capped
/// (`MAX_AUTO_DECLARED_PARAMS`).
fn mirror_param_declarations(
primary: &EngineSnapshot,
write: &mut left_right::WriteHandle<EngineSnapshot, ProfileOp>,
) {
let declarations: Vec<ParamDeclaration> = primary
.param_declarations_range()
.map(
|(
name,
message_match_regex,
capture_group,
json_field_path,
kind,
elapsed_since_last,
auto_derived,
log_stream_gated,
)| ParamDeclaration {
name: name.to_string(),
message_match_regex: message_match_regex.to_string(),
capture_group: capture_group.to_string(),
json_field_path: json_field_path.map(str::to_string),
kind,
elapsed_since_last,
auto_derived,
log_stream_gated,
},
)
.collect();
write.append(ProfileOp(Box::new(move |snap: &mut EngineSnapshot| {
let stale: Vec<IndexKey> = snap
.index
.range(IndexKey::LogParamSplit..IndexKey::ParamMetricSplit)
.map(|(key, _)| key.clone())
.collect();
for key in stale {
snap.index.remove(&key);
}
snap.param_regex_cache.clear();
for declaration in &declarations {
if let Ok(re) = Regex::new(&declaration.message_match_regex) {
snap.param_regex_cache.insert(declaration.name.clone(), re);
}
snap.index.insert(
IndexKey::ParamDeclaration {
name: declaration.name.clone(),
},
DataValue::ParamDeclaration(declaration.clone()),
);
}
})));
write.publish();
}
+244 -60
View File
@@ -9,12 +9,12 @@ use std::time::{SystemTime, UNIX_EPOCH};
use regex::Regex; use regex::Regex;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use crate::engine::{self, eval_json_comparison}; use crate::config::Settings;
use crate::engine::filter_datetime; use crate::engine::filter_datetime;
use crate::engine::filter_parser::{self, CompareOp, FilterExpr, FilterValue}; use crate::engine::filter_parser::{self, CompareOp, FilterExpr, FilterValue};
use crate::engine::rate::RateWindow; use crate::engine::rate::RateWindow;
use crate::config::Settings;
use crate::engine::timestamp::Timestamp; use crate::engine::timestamp::Timestamp;
use crate::engine::{self, eval_json_comparison};
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct AlertRule { pub struct AlertRule {
@@ -27,7 +27,9 @@ pub struct AlertRule {
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "lowercase")] #[serde(tag = "kind", rename_all = "lowercase")]
pub enum AlertCondition { pub enum AlertCondition {
Immediate { query: String }, Immediate {
query: String,
},
Threshold { Threshold {
query: String, query: String,
op: CompareOp, op: CompareOp,
@@ -89,7 +91,12 @@ impl Settings {
enabled: bool, enabled: bool,
condition: AlertCondition, condition: AlertCondition,
) -> Result<AlertRule, AlertError> { ) -> Result<AlertRule, AlertError> {
let rule = AlertRule { id: generate_alert_rule_id(&name), name, enabled, condition }; let rule = AlertRule {
id: generate_alert_rule_id(&name),
name,
enabled,
condition,
};
self.alert_rules.push(rule.clone()); self.alert_rules.push(rule.clone());
self.save(settings_path)?; self.save(settings_path)?;
Ok(rule) Ok(rule)
@@ -147,36 +154,56 @@ fn backfill_window(
window_ms: i64, window_ms: i64,
) -> RateWindow { ) -> RateWindow {
let since_ms = now.as_millis() - window_ms; let since_ms = now.as_millis() - window_ms;
let matching_timestamps = let matching_timestamps = snapshot
snapshot.logs_since(before_sequence, since_ms, now.as_millis()).filter_map( .logs_since(before_sequence, since_ms, now.as_millis())
|(sequence, timestamp, text)| { .filter_map(|(sequence, timestamp, text)| {
(eval_alert_query(query, sequence, timestamp, text) == Ok(true)).then_some(timestamp) (eval_alert_query(query, sequence, timestamp, text) == Ok(true)).then_some(timestamp)
}, });
);
RateWindow::backfill(matching_timestamps, now, window_ms) RateWindow::backfill(matching_timestamps, now, window_ms)
} }
fn eval_alert_query(query: &str, sequence: u64, timestamp: Timestamp, text: &str) -> Result<bool, String> { fn eval_alert_query(
query: &str,
sequence: u64,
timestamp: Timestamp,
text: &str,
) -> Result<bool, String> {
let expr = filter_parser::parse(query).map_err(|err| err.to_string())?; let expr = filter_parser::parse(query).map_err(|err| err.to_string())?;
eval_alert_expr(&expr, sequence, timestamp, text) eval_alert_expr(&expr, sequence, timestamp, text)
} }
fn eval_alert_expr(expr: &FilterExpr, sequence: u64, timestamp: Timestamp, text: &str) -> Result<bool, String> { fn eval_alert_expr(
expr: &FilterExpr,
sequence: u64,
timestamp: Timestamp,
text: &str,
) -> Result<bool, String> {
match expr { match expr {
FilterExpr::Regex(pattern) => { FilterExpr::Regex(pattern) => {
let re = Regex::new(pattern).map_err(|err| format!("invalid regex in alert query: {err}"))?; let re = Regex::new(pattern)
.map_err(|err| format!("invalid regex in alert query: {err}"))?;
Ok(re.is_match(text)) Ok(re.is_match(text))
} }
FilterExpr::Comparison { metric, param: None, op, value } if metric == "ottyl_seq" => { FilterExpr::Comparison {
metric,
param: None,
op,
value,
} if metric == "ottyl_seq" => {
let target = match value { let target = match value {
FilterValue::Number(n) => *n, FilterValue::Number(n) => *n,
FilterValue::Raw(raw) => raw FilterValue::Raw(raw) => raw.parse::<f64>().map_err(|_| {
.parse::<f64>() format!("'ottyl_seq' comparisons require a numeric value, got {raw:?}")
.map_err(|_| format!("'ottyl_seq' comparisons require a numeric value, got {raw:?}"))?, })?,
}; };
Ok(op.matches((sequence as f64).total_cmp(&target))) Ok(op.matches((sequence as f64).total_cmp(&target)))
} }
FilterExpr::Comparison { metric, param: None, op, value } if metric == "ottyl_ts" => { FilterExpr::Comparison {
metric,
param: None,
op,
value,
} if metric == "ottyl_ts" => {
let target_ms = match value { let target_ms = match value {
FilterValue::Number(n) => *n as i64, FilterValue::Number(n) => *n as i64,
FilterValue::Raw(raw) => filter_datetime::parse_datetime(raw, Timestamp::now()) FilterValue::Raw(raw) => filter_datetime::parse_datetime(raw, Timestamp::now())
@@ -185,7 +212,12 @@ fn eval_alert_expr(expr: &FilterExpr, sequence: u64, timestamp: Timestamp, text:
}; };
Ok(op.matches(timestamp.as_millis().cmp(&target_ms))) Ok(op.matches(timestamp.as_millis().cmp(&target_ms)))
} }
FilterExpr::Comparison { metric, param: None, op, value } if metric == "ottyl_fmt" => { FilterExpr::Comparison {
metric,
param: None,
op,
value,
} if metric == "ottyl_fmt" => {
let target = match value { let target = match value {
FilterValue::Number(n) => n.to_string(), FilterValue::Number(n) => n.to_string(),
FilterValue::Raw(raw) => raw.clone(), FilterValue::Raw(raw) => raw.clone(),
@@ -199,12 +231,10 @@ fn eval_alert_expr(expr: &FilterExpr, sequence: u64, timestamp: Timestamp, text:
FilterExpr::JsonComparison { field, op, value } => { FilterExpr::JsonComparison { field, op, value } => {
Ok(eval_json_comparison(text, field, *op, value)) Ok(eval_json_comparison(text, field, *op, value))
} }
FilterExpr::And(lhs, rhs) => { FilterExpr::And(lhs, rhs) => Ok(eval_alert_expr(lhs, sequence, timestamp, text)?
Ok(eval_alert_expr(lhs, sequence, timestamp, text)? && eval_alert_expr(rhs, sequence, timestamp, text)?) && eval_alert_expr(rhs, sequence, timestamp, text)?),
} FilterExpr::Or(lhs, rhs) => Ok(eval_alert_expr(lhs, sequence, timestamp, text)?
FilterExpr::Or(lhs, rhs) => { || eval_alert_expr(rhs, sequence, timestamp, text)?),
Ok(eval_alert_expr(lhs, sequence, timestamp, text)? || eval_alert_expr(rhs, sequence, timestamp, text)?)
}
} }
} }
@@ -222,9 +252,15 @@ pub fn evaluate_alerts(
continue; continue;
} }
let state = states.entry(rule.id.clone()).or_default(); let state = states.entry(rule.id.clone()).or_default();
if let Some(message) = if let Some(message) = evaluate_condition(
evaluate_condition(&rule.condition, state, snapshot, sequence, timestamp, text, &rule.name) &rule.condition,
{ state,
snapshot,
sequence,
timestamp,
text,
&rule.name,
) {
fired.push((rule.clone(), message)); fired.push((rule.clone(), message));
} }
} }
@@ -241,17 +277,25 @@ fn evaluate_condition(
rule_name: &str, rule_name: &str,
) -> Option<String> { ) -> Option<String> {
match condition { match condition {
AlertCondition::Immediate { query } => match eval_alert_query(query, sequence, timestamp, text) { AlertCondition::Immediate { query } => {
match eval_alert_query(query, sequence, timestamp, text) {
Ok(true) => Some(format!("{rule_name}: matched line: {text}")), Ok(true) => Some(format!("{rule_name}: matched line: {text}")),
Ok(false) => None, Ok(false) => None,
Err(err) => { Err(err) => {
eprintln!("warning: alert '{rule_name}' query error: {err}"); eprintln!("warning: alert '{rule_name}' query error: {err}");
None None
} }
}, }
AlertCondition::Threshold { query, op, threshold, window_ms } => { }
AlertCondition::Threshold {
query,
op,
threshold,
window_ms,
} => {
if !state.threshold_backfilled { if !state.threshold_backfilled {
state.threshold_window = backfill_window(snapshot, query, sequence, timestamp, *window_ms); state.threshold_window =
backfill_window(snapshot, query, sequence, timestamp, *window_ms);
state.threshold_backfilled = true; state.threshold_backfilled = true;
} }
match eval_alert_query(query, sequence, timestamp, text) { match eval_alert_query(query, sequence, timestamp, text) {
@@ -269,7 +313,10 @@ fn evaluate_condition(
} }
} }
} }
AlertCondition::NewValue { field_regex, capture_group } => { AlertCondition::NewValue {
field_regex,
capture_group,
} => {
let re = match Regex::new(field_regex) { let re = match Regex::new(field_regex) {
Ok(re) => re, Ok(re) => re,
Err(err) => { Err(err) => {
@@ -284,7 +331,13 @@ fn evaluate_condition(
.insert(value.clone()) .insert(value.clone())
.then(|| format!("{rule_name}: new value seen: {value} (line: {text})")) .then(|| format!("{rule_name}: new value seen: {value} (line: {text})"))
} }
AlertCondition::Ratio { query_a, query_b, op, ratio, window_ms } => { AlertCondition::Ratio {
query_a,
query_b,
op,
ratio,
window_ms,
} => {
let matched_a = eval_alert_query(query_a, sequence, timestamp, text); let matched_a = eval_alert_query(query_a, sequence, timestamp, text);
let matched_b = eval_alert_query(query_b, sequence, timestamp, text); let matched_b = eval_alert_query(query_b, sequence, timestamp, text);
let (matched_a, matched_b) = match (matched_a, matched_b) { let (matched_a, matched_b) = match (matched_a, matched_b) {
@@ -298,11 +351,13 @@ fn evaluate_condition(
return None; return None;
} }
if !state.ratio_a_backfilled { if !state.ratio_a_backfilled {
state.ratio_a_window = backfill_window(snapshot, query_a, sequence, timestamp, *window_ms); state.ratio_a_window =
backfill_window(snapshot, query_a, sequence, timestamp, *window_ms);
state.ratio_a_backfilled = true; state.ratio_a_backfilled = true;
} }
if !state.ratio_b_backfilled { if !state.ratio_b_backfilled {
state.ratio_b_window = backfill_window(snapshot, query_b, sequence, timestamp, *window_ms); state.ratio_b_window =
backfill_window(snapshot, query_b, sequence, timestamp, *window_ms);
state.ratio_b_backfilled = true; state.ratio_b_backfilled = true;
} }
if matched_a { if matched_a {
@@ -326,7 +381,12 @@ fn evaluate_condition(
} }
} }
pub async fn deliver_for_rule(rule_name: String, message: String, topic: Option<String>, server: Option<String>) { pub async fn deliver_for_rule(
rule_name: String,
message: String,
topic: Option<String>,
server: Option<String>,
) {
let Some(topic) = topic else { let Some(topic) = topic else {
return; return;
}; };
@@ -364,7 +424,9 @@ mod tests {
id: "r1".to_string(), id: "r1".to_string(),
name: "test rule".to_string(), name: "test rule".to_string(),
enabled: true, enabled: true,
condition: AlertCondition::Immediate { query: query.to_string() }, condition: AlertCondition::Immediate {
query: query.to_string(),
},
} }
} }
@@ -379,7 +441,9 @@ mod tests {
&settings_path, &settings_path,
"spike".to_string(), "spike".to_string(),
true, true,
AlertCondition::Immediate { query: r#"regex("error")"#.to_string() }, AlertCondition::Immediate {
query: r#"regex("error")"#.to_string(),
},
) )
.unwrap(); .unwrap();
assert_eq!(settings.alert_rules.len(), 1); assert_eq!(settings.alert_rules.len(), 1);
@@ -393,14 +457,18 @@ mod tests {
&created.id, &created.id,
"renamed".to_string(), "renamed".to_string(),
false, false,
AlertCondition::Immediate { query: r#"regex("warn")"#.to_string() }, AlertCondition::Immediate {
query: r#"regex("warn")"#.to_string(),
},
) )
.unwrap(); .unwrap();
assert_eq!(updated.id, created.id); assert_eq!(updated.id, created.id);
assert_eq!(updated.name, "renamed"); assert_eq!(updated.name, "renamed");
assert!(!updated.enabled); assert!(!updated.enabled);
settings.remove_alert_rule(&settings_path, &created.id).unwrap(); settings
.remove_alert_rule(&settings_path, &created.id)
.unwrap();
assert!(settings.alert_rules.is_empty()); assert!(settings.alert_rules.is_empty());
let reloaded = Settings::load(&settings_path); let reloaded = Settings::load(&settings_path);
assert!(reloaded.alert_rules.is_empty()); assert!(reloaded.alert_rules.is_empty());
@@ -417,7 +485,9 @@ mod tests {
"does-not-exist", "does-not-exist",
"x".to_string(), "x".to_string(),
true, true,
AlertCondition::Immediate { query: r#"regex("x")"#.to_string() }, AlertCondition::Immediate {
query: r#"regex("x")"#.to_string(),
},
); );
assert!(matches!(result, Err(AlertError::NotFound))); assert!(matches!(result, Err(AlertError::NotFound)));
@@ -466,12 +536,36 @@ mod tests {
let mut states = HashMap::new(); let mut states = HashMap::new();
let index = engine::EngineSnapshot::default(); let index = engine::EngineSnapshot::default();
for (seq, ms) in [(0u64, 0i64), (1, 1000)] { for (seq, ms) in [(0u64, 0i64), (1, 1000)] {
let fired = evaluate_alerts(&[rule.clone()], &mut states, &index, seq, Timestamp::from_millis(ms), "error"); let fired = evaluate_alerts(
assert!(fired.is_empty(), "should not fire before threshold is crossed"); std::slice::from_ref(&rule),
&mut states,
&index,
seq,
Timestamp::from_millis(ms),
"error",
);
assert!(
fired.is_empty(),
"should not fire before threshold is crossed"
);
} }
let fired = evaluate_alerts(&[rule.clone()], &mut states, &index, 2, Timestamp::from_millis(2000), "error"); let fired = evaluate_alerts(
std::slice::from_ref(&rule),
&mut states,
&index,
2,
Timestamp::from_millis(2000),
"error",
);
assert_eq!(fired.len(), 1); assert_eq!(fired.len(), 1);
let fired = evaluate_alerts(&[rule], &mut states, &index, 3, Timestamp::from_millis(2100), "fine"); let fired = evaluate_alerts(
&[rule],
&mut states,
&index,
3,
Timestamp::from_millis(2100),
"fine",
);
assert!(fired.is_empty()); assert!(fired.is_empty());
} }
@@ -490,8 +584,28 @@ mod tests {
}; };
let mut states = HashMap::new(); let mut states = HashMap::new();
let index = engine::EngineSnapshot::default(); let index = engine::EngineSnapshot::default();
assert!(evaluate_alerts(&[rule.clone()], &mut states, &index, 0, Timestamp::from_millis(0), "error").is_empty()); assert!(
assert!(evaluate_alerts(&[rule], &mut states, &index, 1, Timestamp::from_millis(5_000), "error").is_empty()); evaluate_alerts(
std::slice::from_ref(&rule),
&mut states,
&index,
0,
Timestamp::from_millis(0),
"error"
)
.is_empty()
);
assert!(
evaluate_alerts(
&[rule],
&mut states,
&index,
1,
Timestamp::from_millis(5_000),
"error"
)
.is_empty()
);
} }
#[test] #[test]
@@ -507,11 +621,32 @@ mod tests {
}; };
let mut states = HashMap::new(); let mut states = HashMap::new();
let index = engine::EngineSnapshot::default(); let index = engine::EngineSnapshot::default();
let fired = evaluate_alerts(&[rule.clone()], &mut states, &index, 0, Timestamp::from_millis(0), "agent=curl"); let fired = evaluate_alerts(
std::slice::from_ref(&rule),
&mut states,
&index,
0,
Timestamp::from_millis(0),
"agent=curl",
);
assert_eq!(fired.len(), 1); assert_eq!(fired.len(), 1);
let fired = evaluate_alerts(&[rule.clone()], &mut states, &index, 1, Timestamp::from_millis(1), "agent=curl"); let fired = evaluate_alerts(
std::slice::from_ref(&rule),
&mut states,
&index,
1,
Timestamp::from_millis(1),
"agent=curl",
);
assert!(fired.is_empty()); assert!(fired.is_empty());
let fired = evaluate_alerts(&[rule], &mut states, &index, 2, Timestamp::from_millis(2), "agent=wget"); let fired = evaluate_alerts(
&[rule],
&mut states,
&index,
2,
Timestamp::from_millis(2),
"agent=wget",
);
assert_eq!(fired.len(), 1); assert_eq!(fired.len(), 1);
} }
@@ -531,12 +666,43 @@ mod tests {
}; };
let mut states = HashMap::new(); let mut states = HashMap::new();
let index = engine::EngineSnapshot::default(); let index = engine::EngineSnapshot::default();
assert!(evaluate_alerts(&[rule.clone()], &mut states, &index, 0, Timestamp::from_millis(0), "error").is_empty()); assert!(
let fired = evaluate_alerts(&[rule.clone()], &mut states, &index, 1, Timestamp::from_millis(1), "request"); evaluate_alerts(
std::slice::from_ref(&rule),
&mut states,
&index,
0,
Timestamp::from_millis(0),
"error"
)
.is_empty()
);
let fired = evaluate_alerts(
std::slice::from_ref(&rule),
&mut states,
&index,
1,
Timestamp::from_millis(1),
"request",
);
assert_eq!(fired.len(), 1); assert_eq!(fired.len(), 1);
let fired = evaluate_alerts(&[rule.clone()], &mut states, &index, 2, Timestamp::from_millis(2), "request"); let fired = evaluate_alerts(
std::slice::from_ref(&rule),
&mut states,
&index,
2,
Timestamp::from_millis(2),
"request",
);
assert_eq!(fired.len(), 1); assert_eq!(fired.len(), 1);
let fired = evaluate_alerts(&[rule], &mut states, &index, 3, Timestamp::from_millis(3), "request"); let fired = evaluate_alerts(
&[rule],
&mut states,
&index,
3,
Timestamp::from_millis(3),
"request",
);
assert!(fired.is_empty()); assert!(fired.is_empty());
} }
@@ -547,7 +713,7 @@ mod tests {
let index = engine::EngineSnapshot::default(); let index = engine::EngineSnapshot::default();
let fired = evaluate_alerts( let fired = evaluate_alerts(
&[rule.clone()], std::slice::from_ref(&rule),
&mut states, &mut states,
&index, &index,
0, 0,
@@ -556,7 +722,14 @@ mod tests {
); );
assert_eq!(fired.len(), 1); assert_eq!(fired.len(), 1);
let fired = evaluate_alerts(&[rule], &mut states, &index, 1, Timestamp::from_millis(1), "plain text"); let fired = evaluate_alerts(
&[rule],
&mut states,
&index,
1,
Timestamp::from_millis(1),
"plain text",
);
assert!(fired.is_empty()); assert!(fired.is_empty());
} }
@@ -565,15 +738,22 @@ mod tests {
let rule = immediate_rule("some_metric[param]==5"); let rule = immediate_rule("some_metric[param]==5");
let mut states = HashMap::new(); let mut states = HashMap::new();
let index = engine::EngineSnapshot::default(); let index = engine::EngineSnapshot::default();
let fired = evaluate_alerts(&[rule], &mut states, &index, 0, Timestamp::from_millis(0), "anything"); let fired = evaluate_alerts(
&[rule],
&mut states,
&index,
0,
Timestamp::from_millis(0),
"anything",
);
assert!(fired.is_empty()); assert!(fired.is_empty());
} }
#[tokio::test] #[tokio::test]
async fn deliver_ntfy_posts_message_to_server_topic_path() { async fn deliver_ntfy_posts_message_to_server_topic_path() {
use axum::Router;
use axum::extract::State; use axum::extract::State;
use axum::routing::post; use axum::routing::post;
use axum::Router;
use std::sync::{Arc, Mutex}; use std::sync::{Arc, Mutex};
use tokio::net::TcpListener; use tokio::net::TcpListener;
@@ -590,7 +770,9 @@ mod tests {
} }
let captured = Captured::default(); let captured = Captured::default();
let app = Router::new().route("/{topic}", post(capture)).with_state(captured.clone()); let app = Router::new()
.route("/{topic}", post(capture))
.with_state(captured.clone());
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap(); let addr = listener.local_addr().unwrap();
tokio::spawn(async move { tokio::spawn(async move {
@@ -598,7 +780,9 @@ mod tests {
}); });
let server = format!("http://{addr}"); let server = format!("http://{addr}");
deliver_ntfy(&server, "alerts", "hello world").await.unwrap(); deliver_ntfy(&server, "alerts", "hello world")
.await
.unwrap();
let (topic, body) = captured.0.lock().unwrap().clone().unwrap(); let (topic, body) = captured.0.lock().unwrap().clone().unwrap();
assert_eq!(topic, "alerts"); assert_eq!(topic, "alerts");
+26 -3
View File
@@ -2,6 +2,7 @@ use crate::engine::metric_primitive::MetricPrimitive;
use crate::engine::timestamp::Timestamp; use crate::engine::timestamp::Timestamp;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::collections::BTreeMap; use std::collections::BTreeMap;
use std::sync::Arc;
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct ParamDeclaration { pub struct ParamDeclaration {
@@ -39,7 +40,8 @@ pub fn extract_json_field(text: &str, field_path: &str) -> Option<String> {
} }
pub fn flatten_json_object(text: &str) -> Vec<(String, String)> { pub fn flatten_json_object(text: &str) -> Vec<(String, String)> {
let Ok(serde_json::Value::Object(root)) = serde_json::from_str::<serde_json::Value>(text) else { let Ok(serde_json::Value::Object(root)) = serde_json::from_str::<serde_json::Value>(text)
else {
return Vec::new(); return Vec::new();
}; };
let mut results = Vec::new(); let mut results = Vec::new();
@@ -53,7 +55,11 @@ fn flatten_json_object_into(
results: &mut Vec<(String, String)>, results: &mut Vec<(String, String)>,
) { ) {
for (key, value) in object { for (key, value) in object {
let path = if prefix.is_empty() { key.clone() } else { format!("{prefix}.{key}") }; let path = if prefix.is_empty() {
key.clone()
} else {
format!("{prefix}.{key}")
};
match value { match value {
serde_json::Value::String(s) => results.push((path, s.clone())), serde_json::Value::String(s) => results.push((path, s.clone())),
serde_json::Value::Number(n) => results.push((path, n.to_string())), serde_json::Value::Number(n) => results.push((path, n.to_string())),
@@ -75,10 +81,26 @@ pub enum MetricPrimitiveKind {
Boolean, Boolean,
} }
/// Whether a declared metric's value is a point-in-time reading (`Gauge`, e.g. queue depth) or a
/// monotonically-increasing total (`Counter`, e.g. requests served) — orthogonal to
/// `MetricPrimitiveKind`, which describes the value's data type rather than how it accumulates.
/// `Gauge` is the default so existing declarations (persisted before this field existed) keep
/// behaving the way bare selectors and the `gauge()` PromQL wrapper already did.
#[allow(dead_code)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)]
#[serde(rename_all = "lowercase")]
pub enum MetricSemantics {
#[default]
Gauge,
Counter,
}
#[allow(dead_code)] #[allow(dead_code)]
#[derive(Debug, Clone, Serialize)] #[derive(Debug, Clone, Serialize)]
pub enum DataValue { pub enum DataValue {
LogLiteral { text: String }, LogLiteral {
text: Arc<str>,
},
ParamDeclaration(ParamDeclaration), ParamDeclaration(ParamDeclaration),
MetricDeclaration { MetricDeclaration {
message_match_regex: String, message_match_regex: String,
@@ -86,6 +108,7 @@ pub enum DataValue {
value_param: Option<String>, value_param: Option<String>,
matched_labels: Option<BTreeMap<String, String>>, matched_labels: Option<BTreeMap<String, String>>,
unmatched_labels: Option<BTreeMap<String, String>>, unmatched_labels: Option<BTreeMap<String, String>>,
semantics: MetricSemantics,
}, },
BucketValue { BucketValue {
values: Vec<(u64, Timestamp, MetricPrimitive)>, values: Vec<(u64, Timestamp, MetricPrimitive)>,
+9 -10
View File
@@ -58,10 +58,7 @@ fn try_relative(input: &str, now: Timestamp) -> Option<Result<Timestamp, DateTim
}, },
}; };
let looks_like_duration = rest let looks_like_duration = rest.chars().next().is_some_and(|c| c.is_ascii_digit())
.chars()
.next()
.is_some_and(|c| c.is_ascii_digit())
&& rest.chars().any(|c| c.is_ascii_alphabetic()) && rest.chars().any(|c| c.is_ascii_alphabetic())
&& !rest.contains('-') && !rest.contains('-')
&& !rest.contains(':'); && !rest.contains(':');
@@ -107,10 +104,7 @@ fn try_full_datetime(input: &str) -> Option<Result<Timestamp, DateTimeParseError
} }
} }
fn try_time_of_day( fn try_time_of_day(input: &str, now: Timestamp) -> Option<Result<Timestamp, DateTimeParseError>> {
input: &str,
now: Timestamp,
) -> Option<Result<Timestamp, DateTimeParseError>> {
if !input.contains(':') { if !input.contains(':') {
return None; return None;
} }
@@ -129,8 +123,13 @@ fn try_time_of_day(
}; };
let now_millis = now.as_millis(); let now_millis = now.as_millis();
let now_dt = Utc.timestamp_millis_opt(now_millis).single().unwrap_or_else(|| { let now_dt = Utc
Utc.timestamp_millis_opt(0).single().expect("epoch is valid") .timestamp_millis_opt(now_millis)
.single()
.unwrap_or_else(|| {
Utc.timestamp_millis_opt(0)
.single()
.expect("epoch is valid")
}); });
let today = now_dt.date_naive(); let today = now_dt.date_naive();
let dt = Utc.from_utc_datetime(&today.and_time(time)); let dt = Utc.from_utc_datetime(&today.and_time(time));
+9 -13
View File
@@ -88,9 +88,7 @@ pub fn parse(input: &str) -> Result<FilterExpr, FilterParseError> {
fn build_or_expr(pair: Pair<Rule>) -> Result<FilterExpr, FilterParseError> { fn build_or_expr(pair: Pair<Rule>) -> Result<FilterExpr, FilterParseError> {
debug_assert_eq!(pair.as_rule(), Rule::or_expr); debug_assert_eq!(pair.as_rule(), Rule::or_expr);
let mut and_exprs = pair let mut and_exprs = pair.into_inner().filter(|p| p.as_rule() != Rule::or_kw);
.into_inner()
.filter(|p| p.as_rule() != Rule::or_kw);
let first = and_exprs let first = and_exprs
.next() .next()
.ok_or_else(|| FilterParseError::new("malformed 'or' expression"))?; .ok_or_else(|| FilterParseError::new("malformed 'or' expression"))?;
@@ -104,9 +102,7 @@ fn build_or_expr(pair: Pair<Rule>) -> Result<FilterExpr, FilterParseError> {
fn build_and_expr(pair: Pair<Rule>) -> Result<FilterExpr, FilterParseError> { fn build_and_expr(pair: Pair<Rule>) -> Result<FilterExpr, FilterParseError> {
debug_assert_eq!(pair.as_rule(), Rule::and_expr); debug_assert_eq!(pair.as_rule(), Rule::and_expr);
let mut terms = pair let mut terms = pair.into_inner().filter(|p| p.as_rule() != Rule::and_kw);
.into_inner()
.filter(|p| p.as_rule() != Rule::and_kw);
let first = terms let first = terms
.next() .next()
.ok_or_else(|| FilterParseError::new("malformed 'and' expression"))?; .ok_or_else(|| FilterParseError::new("malformed 'and' expression"))?;
@@ -182,9 +178,9 @@ fn build_comparison(pair: Pair<Rule>) -> Result<FilterExpr, FilterParseError> {
fn build_json_comparison(pair: Pair<Rule>) -> Result<FilterExpr, FilterParseError> { fn build_json_comparison(pair: Pair<Rule>) -> Result<FilterExpr, FilterParseError> {
let mut inner = pair.into_inner(); let mut inner = pair.into_inner();
let field_pair = inner let field_pair = inner.next().ok_or_else(|| {
.next() FilterParseError::new("malformed json[...] comparison: missing field name")
.ok_or_else(|| FilterParseError::new("malformed json[...] comparison: missing field name"))?; })?;
let field = field_pair.as_str().to_string(); let field = field_pair.as_str().to_string();
let op_pair = inner let op_pair = inner
@@ -355,8 +351,8 @@ mod tests {
#[test] #[test]
fn parses_and_or_with_and_binding_tighter() { fn parses_and_or_with_and_binding_tighter() {
let expr = parse(r#"regex("error") and ottyl_seq<5 or ottyl_ts>12:00"#) let expr =
.expect("should parse"); parse(r#"regex("error") and ottyl_seq<5 or ottyl_ts>12:00"#).expect("should parse");
let expected = FilterExpr::Or( let expected = FilterExpr::Or(
Box::new(FilterExpr::And( Box::new(FilterExpr::And(
@@ -381,8 +377,8 @@ mod tests {
#[test] #[test]
fn parenthesized_expression_overrides_precedence() { fn parenthesized_expression_overrides_precedence() {
let expr = parse(r#"regex("error") and (ottyl_seq<5 or ottyl_ts>12:00)"#) let expr =
.expect("should parse"); parse(r#"regex("error") and (ottyl_seq<5 or ottyl_ts>12:00)"#).expect("should parse");
let expected = FilterExpr::And( let expected = FilterExpr::And(
Box::new(FilterExpr::Regex("error".to_string())), Box::new(FilterExpr::Regex("error".to_string())),
+56 -31
View File
@@ -2,8 +2,8 @@ use std::collections::BTreeMap;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use crate::consts::{METRICS_POLL_INTERVAL, STDIN_POLL_INTERVAL as POLL_INTERVAL}; use crate::consts::{METRICS_POLL_INTERVAL, STDIN_POLL_INTERVAL as POLL_INTERVAL};
use crate::engine::{EngineHandle, EngineMessage, IngestedLine, MetricSample};
use crate::engine::timestamp::Timestamp; use crate::engine::timestamp::Timestamp;
use crate::engine::{EngineHandle, EngineMessage, IngestedLine, MetricSample};
use tokio::fs::File; use tokio::fs::File;
use tokio::io::{AsyncBufReadExt, BufReader}; use tokio::io::{AsyncBufReadExt, BufReader};
@@ -16,7 +16,7 @@ pub async fn run(engine: EngineHandle) {
let line = IngestedLine { let line = IngestedLine {
sequence, sequence,
timestamp: Timestamp::now(), timestamp: Timestamp::now(),
text, text: text.into(),
}; };
sequence += 1; sequence += 1;
@@ -67,31 +67,26 @@ pub async fn run_file(engine: EngineHandle, path: PathBuf) {
let mut chunk = String::new(); let mut chunk = String::new();
match reader.read_line(&mut chunk).await { match reader.read_line(&mut chunk).await {
Ok(0) => { Ok(0) => {
match tokio::fs::metadata(&path).await { if let Ok(meta) = tokio::fs::metadata(&path).await {
Ok(meta) => {
let new_id = file_id(&meta); let new_id = file_id(&meta);
if new_id != id || meta.len() < offset { if new_id != id || meta.len() < offset {
continue 'reopen; continue 'reopen;
} }
id = new_id; id = new_id;
} }
Err(_) => {}
}
tokio::time::sleep(POLL_INTERVAL).await; tokio::time::sleep(POLL_INTERVAL).await;
} }
Ok(n) => { Ok(n) => {
offset += n as u64; offset += n as u64;
pending.push_str(&chunk); pending.push_str(&chunk);
if pending.ends_with('\n') { if pending.ends_with('\n') {
let text = pending let text = pending.trim_end_matches(['\n', '\r']).to_string();
.trim_end_matches(['\n', '\r'])
.to_string();
pending.clear(); pending.clear();
let line = IngestedLine { let line = IngestedLine {
sequence, sequence,
timestamp: Timestamp::now(), timestamp: Timestamp::now(),
text, text: text.into(),
}; };
sequence += 1; sequence += 1;
@@ -106,32 +101,48 @@ pub async fn run_file(engine: EngineHandle, path: PathBuf) {
} }
} }
async fn fetch_metrics_body(client: &reqwest::Client, url: &str) -> Result<String, String> {
let response = client
.get(url)
.send()
.await
.map_err(|err| format!("warning: failed to poll metrics endpoint {url}: {err}"))?;
let response = response
.error_for_status()
.map_err(|err| format!("warning: metrics endpoint {url} returned an error: {err}"))?;
response
.text()
.await
.map_err(|err| format!("warning: failed to read metrics response from {url}: {err}"))
}
pub async fn run_metrics_poll(engine: EngineHandle, url: String) { pub async fn run_metrics_poll(engine: EngineHandle, url: String) {
let client = reqwest::Client::new(); let client = reqwest::Client::new();
loop { loop {
match client.get(&url).send().await { match fetch_metrics_body(&client, &url).await {
Ok(response) => match response.error_for_status() {
Ok(response) => match response.text().await {
Ok(body) => { Ok(body) => {
let now = Timestamp::now(); let now = Timestamp::now();
for sample in parse_openmetrics(&body, now) { for sample in parse_openmetrics(&body, now) {
if engine.tx.send(EngineMessage::IngestMetricSample(sample)).await.is_err() { if engine
.tx
.send(EngineMessage::IngestMetricSample(sample))
.await
.is_err()
{
return; return;
} }
} }
} }
Err(err) => eprintln!("warning: failed to read metrics response from {url}: {err}"), Err(warning) => eprintln!("{warning}"),
},
Err(err) => eprintln!("warning: metrics endpoint {url} returned an error: {err}"),
},
Err(err) => eprintln!("warning: failed to poll metrics endpoint {url}: {err}"),
} }
tokio::time::sleep(METRICS_POLL_INTERVAL).await; tokio::time::sleep(METRICS_POLL_INTERVAL).await;
} }
} }
fn parse_openmetrics(body: &str, now: Timestamp) -> Vec<MetricSample> { fn parse_openmetrics(body: &str, now: Timestamp) -> Vec<MetricSample> {
body.lines().filter_map(|line| parse_openmetrics_line(line, now)).collect() body.lines()
.filter_map(|line| parse_openmetrics_line(line, now))
.collect()
} }
fn parse_openmetrics_line(line: &str, now: Timestamp) -> Option<MetricSample> { fn parse_openmetrics_line(line: &str, now: Timestamp) -> Option<MetricSample> {
@@ -152,7 +163,10 @@ fn parse_openmetrics_line(line: &str, now: Timestamp) -> Option<MetricSample> {
let (metric_name, labels) = match name_and_labels.find('{') { let (metric_name, labels) = match name_and_labels.find('{') {
Some(open) => { Some(open) => {
let close = name_and_labels.rfind('}')?; let close = name_and_labels.rfind('}')?;
(name_and_labels[..open].to_string(), parse_openmetrics_labels(&name_and_labels[open + 1..close])) (
name_and_labels[..open].to_string(),
parse_openmetrics_labels(&name_and_labels[open + 1..close]),
)
} }
None => (name_and_labels.to_string(), BTreeMap::new()), None => (name_and_labels.to_string(), BTreeMap::new()),
}; };
@@ -160,13 +174,23 @@ fn parse_openmetrics_line(line: &str, now: Timestamp) -> Option<MetricSample> {
return None; return None;
} }
Some(MetricSample { metric_name, labels, value, timestamp }) Some(MetricSample {
metric_name,
labels,
value,
timestamp,
})
} }
fn parse_openmetrics_labels(raw: &str) -> BTreeMap<String, String> { fn parse_openmetrics_labels(raw: &str) -> BTreeMap<String, String> {
raw.split(',') raw.split(',')
.filter_map(|pair| pair.split_once('=')) .filter_map(|pair| pair.split_once('='))
.map(|(name, value)| (name.trim().to_string(), value.trim().trim_matches('"').to_string())) .map(|(name, value)| {
(
name.trim().to_string(),
value.trim().trim_matches('"').to_string(),
)
})
.collect() .collect()
} }
@@ -188,10 +212,8 @@ mod openmetrics_tests {
#[test] #[test]
fn parses_labels_and_explicit_timestamp() { fn parses_labels_and_explicit_timestamp() {
let now = Timestamp::from_millis(1000); let now = Timestamp::from_millis(1000);
let samples = parse_openmetrics( let samples =
r#"http_requests_total{method="GET",path="/a"} 7 5000"#, parse_openmetrics(r#"http_requests_total{method="GET",path="/a"} 7 5000"#, now);
now,
);
assert_eq!(samples.len(), 1); assert_eq!(samples.len(), 1);
let sample = &samples[0]; let sample = &samples[0];
assert_eq!(sample.metric_name, "http_requests_total"); assert_eq!(sample.metric_name, "http_requests_total");
@@ -214,9 +236,9 @@ mod openmetrics_tests {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use std::time::Duration;
use std::io::Write;
use crate::engine::test_handle; use crate::engine::test_handle;
use std::io::Write;
use std::time::Duration;
use tokio::sync::mpsc; use tokio::sync::mpsc;
async fn recv_text(rx: &mut mpsc::Receiver<EngineMessage>) -> String { async fn recv_text(rx: &mut mpsc::Receiver<EngineMessage>) -> String {
@@ -225,7 +247,7 @@ mod tests {
.expect("timed out waiting for ingested line") .expect("timed out waiting for ingested line")
.expect("engine channel closed") .expect("engine channel closed")
{ {
EngineMessage::Ingest(line) => line.text, EngineMessage::Ingest(line) => line.text.to_string(),
_ => panic!("expected Ingest"), _ => panic!("expected Ingest"),
} }
} }
@@ -241,7 +263,10 @@ mod tests {
assert_eq!(recv_text(&mut rx).await, "first"); assert_eq!(recv_text(&mut rx).await, "first");
let mut f = std::fs::OpenOptions::new().append(true).open(&path).unwrap(); let mut f = std::fs::OpenOptions::new()
.append(true)
.open(&path)
.unwrap();
writeln!(f, "second").unwrap(); writeln!(f, "second").unwrap();
assert_eq!(recv_text(&mut rx).await, "second"); assert_eq!(recv_text(&mut rx).await, "second");
} }
+81 -21
View File
@@ -1,9 +1,9 @@
use nom::IResult;
use nom::branch::alt; use nom::branch::alt;
use nom::bytes::complete::{escaped, tag, take_until, take_while1}; use nom::bytes::complete::{escaped, tag, take_until, take_while1};
use nom::character::complete::{char, none_of, one_of}; use nom::character::complete::{char, none_of, one_of};
use nom::error::{Error, ErrorKind}; use nom::error::{Error, ErrorKind};
use nom::sequence::delimited; use nom::sequence::delimited;
use nom::IResult;
use regex::Regex; use regex::Regex;
use std::sync::LazyLock; use std::sync::LazyLock;
@@ -16,7 +16,10 @@ fn key(input: &str) -> IResult<&str, &str> {
fn quoted_value(input: &str) -> IResult<&str, &str> { fn quoted_value(input: &str) -> IResult<&str, &str> {
delimited( delimited(
char('"'), char('"'),
alt((escaped(none_of("\\\""), '\\', one_of("\"\\nt")), nom::combinator::success(""))), alt((
escaped(none_of("\\\""), '\\', one_of("\"\\nt")),
nom::combinator::success(""),
)),
char('"'), char('"'),
)(input) )(input)
} }
@@ -85,7 +88,8 @@ fn is_separator_run(run: &str) -> bool {
run.len() >= 2 || run.contains('\t') run.len() >= 2 || run.contains('\t')
} }
static WHITESPACE_RUN: LazyLock<Regex> = LazyLock::new(|| Regex::new(r"[ \t]+").expect("static regex must compile")); static WHITESPACE_RUN: LazyLock<Regex> =
LazyLock::new(|| Regex::new(r"[ \t]+").expect("static regex must compile"));
fn split_columns_with_offsets(line: &str, max_columns: Option<usize>) -> Vec<(usize, &str)> { fn split_columns_with_offsets(line: &str, max_columns: Option<usize>) -> Vec<(usize, &str)> {
let mut columns = Vec::new(); let mut columns = Vec::new();
@@ -107,7 +111,10 @@ fn split_columns_with_offsets(line: &str, max_columns: Option<usize>) -> Vec<(us
} }
fn split_columns(line: &str, max_columns: Option<usize>) -> Vec<&str> { fn split_columns(line: &str, max_columns: Option<usize>) -> Vec<&str> {
split_columns_with_offsets(line, max_columns).into_iter().map(|(_, s)| s).collect() split_columns_with_offsets(line, max_columns)
.into_iter()
.map(|(_, s)| s)
.collect()
} }
static COLUMN_NAME: LazyLock<Regex> = static COLUMN_NAME: LazyLock<Regex> =
@@ -151,13 +158,21 @@ impl LogStreamHeader {
} }
let leading_ws = slice.len() - slice.trim_start().len(); let leading_ws = slice.len() - slice.trim_start().len();
let start = line[..byte_start + leading_ws].chars().count(); let start = line[..byte_start + leading_ws].chars().count();
columns.push(LogStreamColumn { name: trimmed.to_string(), start }); columns.push(LogStreamColumn {
name: trimmed.to_string(),
start,
});
} }
if columns.len() < 2 || !columns.iter().all(|c| COLUMN_NAME.is_match(&c.name)) { if columns.len() < 2 || !columns.iter().all(|c| COLUMN_NAME.is_match(&c.name)) {
return None; return None;
} }
let tab_delimited = WHITESPACE_RUN.find_iter(line).any(|m| m.as_str().contains('\t')); let tab_delimited = WHITESPACE_RUN
Some(LogStreamHeader { columns, tab_delimited }) .find_iter(line)
.any(|m| m.as_str().contains('\t'));
Some(LogStreamHeader {
columns,
tab_delimited,
})
} }
pub fn split_row<'a>(&self, line: &'a str) -> Vec<&'a str> { pub fn split_row<'a>(&self, line: &'a str) -> Vec<&'a str> {
@@ -210,7 +225,9 @@ impl LogStreamHeader {
return pattern; return pattern;
} }
let start = self.columns.last().map(|c| c.start).unwrap_or(0); let start = self.columns.last().map(|c| c.start).unwrap_or(0);
format!("^.{{{start}}}{COLUMN_VALUE_PATTERN}{COLUMN_SEPARATOR_PATTERN}(?P<{capture_group}>.*)$") format!(
"^.{{{start}}}{COLUMN_VALUE_PATTERN}{COLUMN_SEPARATOR_PATTERN}(?P<{capture_group}>.*)$"
)
} }
} }
@@ -240,7 +257,9 @@ fn py_log_prefix(input: &str) -> IResult<&str, (&str, &str, &str, &str)> {
let (input, level) = take_while1(|c: char| c.is_ascii_uppercase())(input)?; let (input, level) = take_while1(|c: char| c.is_ascii_uppercase())(input)?;
let (input, _) = char(' ')(input)?; let (input, _) = char(' ')(input)?;
let (input, logger) = let (input, logger) =
take_while1(|c: char| c.is_ascii_alphanumeric() || c == '_' || c == '-' || c == '.')(input)?; take_while1(|c: char| c.is_ascii_alphanumeric() || c == '_' || c == '-' || c == '.')(
input,
)?;
let (input, _) = char(' ')(input)?; let (input, _) = char(' ')(input)?;
Ok((input, (date, time, level, logger))) Ok((input, (date, time, level, logger)))
} }
@@ -323,7 +342,10 @@ mod tests {
#[test] #[test]
fn detects_key_value_lines_as_logfmt() { fn detects_key_value_lines_as_logfmt() {
assert_eq!(detect_format(r#"ts=1788648105773 msg="entity_moved" x1=-4"#), "logfmt"); assert_eq!(
detect_format(r#"ts=1788648105773 msg="entity_moved" x1=-4"#),
"logfmt"
);
} }
#[test] #[test]
@@ -371,7 +393,10 @@ mod tests {
fn detects_log_stream_header_columns_generically() { fn detects_log_stream_header_columns_generically() {
let header_line = LOG_STREAM_FIXTURE.lines().next().unwrap(); let header_line = LOG_STREAM_FIXTURE.lines().next().unwrap();
let header = LogStreamHeader::detect(header_line).expect("fixture header must be detected"); let header = LogStreamHeader::detect(header_line).expect("fixture header must be detected");
assert_eq!(column_names(&header), vec!["Timestamp", "Thread", "Type", "Activity", "PID", "TTL"]); assert_eq!(
column_names(&header),
vec!["Timestamp", "Thread", "Type", "Activity", "PID", "TTL"]
);
assert!(!header.tab_delimited); assert!(!header.tab_delimited);
} }
@@ -400,7 +425,10 @@ mod tests {
assert_eq!(row[3], "0x0"); assert_eq!(row[3], "0x0");
assert_eq!(row[4], "606"); assert_eq!(row[4], "606");
assert_eq!(row[5], "0"); assert_eq!(row[5], "0");
assert_eq!(row[6], "WindowServer: (HSTouchHIDService) something happened"); assert_eq!(
row[6],
"WindowServer: (HSTouchHIDService) something happened"
);
} }
#[test] #[test]
@@ -426,32 +454,57 @@ mod tests {
let line = "2026-09-06 01:48:54.518117+0100 0x4dfed2 Default 0x0 606 0 WindowServer: (HSTouchHIDService) something happened"; let line = "2026-09-06 01:48:54.518117+0100 0x4dfed2 Default 0x0 606 0 WindowServer: (HSTouchHIDService) something happened";
let type_re = Regex::new(&header.column_pattern(2, "Type")).unwrap(); let type_re = Regex::new(&header.column_pattern(2, "Type")).unwrap();
assert_eq!(type_re.captures(line).unwrap().name("Type").unwrap().as_str(), "Default"); assert_eq!(
type_re
.captures(line)
.unwrap()
.name("Type")
.unwrap()
.as_str(),
"Default"
);
let pid_re = Regex::new(&header.column_pattern(4, "PID")).unwrap(); let pid_re = Regex::new(&header.column_pattern(4, "PID")).unwrap();
assert_eq!(pid_re.captures(line).unwrap().name("PID").unwrap().as_str(), "606"); assert_eq!(
pid_re.captures(line).unwrap().name("PID").unwrap().as_str(),
"606"
);
let message_re = Regex::new(&header.message_pattern("message")).unwrap(); let message_re = Regex::new(&header.message_pattern("message")).unwrap();
assert_eq!( assert_eq!(
message_re.captures(line).unwrap().name("message").unwrap().as_str(), message_re
.captures(line)
.unwrap()
.name("message")
.unwrap()
.as_str(),
"WindowServer: (HSTouchHIDService) something happened" "WindowServer: (HSTouchHIDService) something happened"
); );
let timestamp_re = Regex::new(&header.column_pattern(0, "Timestamp")).unwrap(); let timestamp_re = Regex::new(&header.column_pattern(0, "Timestamp")).unwrap();
assert_eq!( assert_eq!(
timestamp_re.captures(line).unwrap().name("Timestamp").unwrap().as_str(), timestamp_re
.captures(line)
.unwrap()
.name("Timestamp")
.unwrap()
.as_str(),
"2026-09-06 01:48:54.518117+0100" "2026-09-06 01:48:54.518117+0100"
); );
} }
#[test] #[test]
fn is_log_stream_line_rejects_non_columnar_lines_that_a_fixed_offset_column_regex_would_still_match() { fn is_log_stream_line_rejects_non_columnar_lines_that_a_fixed_offset_column_regex_would_still_match()
{
let header_line = LOG_STREAM_FIXTURE.lines().next().unwrap(); let header_line = LOG_STREAM_FIXTURE.lines().next().unwrap();
let header = LogStreamHeader::detect(header_line).unwrap(); let header = LogStreamHeader::detect(header_line).unwrap();
let timestamp_re = Regex::new(&header.column_pattern(0, "Timestamp")).unwrap(); let timestamp_re = Regex::new(&header.column_pattern(0, "Timestamp")).unwrap();
let continuation_line = "so_gencnt: 4042707 so_state: 0x0"; let continuation_line = "so_gencnt: 4042707 so_state: 0x0";
assert!(timestamp_re.captures(continuation_line).is_some(), "sanity: regex alone still matches garbage"); assert!(
timestamp_re.captures(continuation_line).is_some(),
"sanity: regex alone still matches garbage"
);
assert!(!is_log_stream_line(continuation_line)); assert!(!is_log_stream_line(continuation_line));
assert!(!is_log_stream_line(header_line)); assert!(!is_log_stream_line(header_line));
assert!(!is_log_stream_line("}")); assert!(!is_log_stream_line("}"));
@@ -473,7 +526,10 @@ mod tests {
#[test] #[test]
fn detects_go_log_shaped_lines() { fn detects_go_log_shaped_lines() {
for line in GO_LOG_FIXTURE.lines() { for line in GO_LOG_FIXTURE.lines() {
assert!(is_go_log_line(line), "expected go-log line to match: {line}"); assert!(
is_go_log_line(line),
"expected go-log line to match: {line}"
);
assert_eq!(detect_format(line), "go-log"); assert_eq!(detect_format(line), "go-log");
} }
} }
@@ -481,7 +537,10 @@ mod tests {
#[test] #[test]
fn detects_py_log_shaped_lines() { fn detects_py_log_shaped_lines() {
for line in PY_LOG_FIXTURE.lines() { for line in PY_LOG_FIXTURE.lines() {
assert!(is_py_log_line(line), "expected py-log line to match: {line}"); assert!(
is_py_log_line(line),
"expected py-log line to match: {line}"
);
assert_eq!(detect_format(line), "py-log"); assert_eq!(detect_format(line), "py-log");
} }
} }
@@ -514,7 +573,8 @@ mod tests {
#[test] #[test]
fn py_log_prefix_extracts_date_time_level_and_logger() { fn py_log_prefix_extracts_date_time_level_and_logger() {
let (rest, (date, time, level, logger)) = let (rest, (date, time, level, logger)) =
py_log_prefix("2026-09-06 10:35:55,217 INFO delete-py-log event=eventC labelC=1").unwrap(); py_log_prefix("2026-09-06 10:35:55,217 INFO delete-py-log event=eventC labelC=1")
.unwrap();
assert_eq!(date, "2026-09-06"); assert_eq!(date, "2026-09-06");
assert_eq!(time, "10:35:55,217"); assert_eq!(time, "10:35:55,217");
assert_eq!(level, "INFO"); assert_eq!(level, "INFO");
+90
View File
@@ -0,0 +1,90 @@
//! The engine actor's inbound message/query types and its outbound response type.
use crate::api::dto::{
BucketValueDto, LogEntryDto, LogsParams, MetricDeclarationDto, ParamDeclarationDto,
PromqlSeriesDto, StatsDto, TimelineBucketDto, TimelineParams,
};
use crate::engine::{
ClearParamsResult, DeclareMetricRequest, DeclareParamRequest, IngestedLine, MetricSample,
};
use std::collections::BTreeMap;
use tokio::sync::oneshot;
#[allow(dead_code)]
pub enum EngineQuery {
Logs(LogsParams),
Timeline(TimelineParams),
Stats,
ParamDeclarations,
MetricDeclarations,
BucketValues {
metric_name: String,
labels_filter: Option<BTreeMap<String, String>>,
start_ms: Option<i64>,
end_ms: Option<i64>,
since_sequence: Option<u64>,
limit: Option<usize>,
},
Promql {
query: String,
start_ms: Option<i64>,
end_ms: Option<i64>,
},
}
#[allow(dead_code)]
pub enum EngineMessage {
Ingest(IngestedLine),
IngestMetricSample(MetricSample),
DeclareParam {
request: DeclareParamRequest,
respond_to: oneshot::Sender<Result<(), String>>,
},
RemoveParamDeclaration {
name: String,
respond_to: oneshot::Sender<bool>,
},
ClearParams {
respond_to: oneshot::Sender<ClearParamsResult>,
},
DeclareMetric {
request: DeclareMetricRequest,
respond_to: oneshot::Sender<Result<(), String>>,
},
UpdateMetric {
name: String,
request: DeclareMetricRequest,
respond_to: oneshot::Sender<Result<(), String>>,
},
RemoveMetricDeclaration {
name: String,
respond_to: oneshot::Sender<bool>,
},
SetMaxLogs(Option<usize>),
EnsureSortHelperIndex {
param_name: String,
respond_to: oneshot::Sender<()>,
},
Barrier(oneshot::Sender<()>),
}
#[allow(dead_code)]
pub enum EngineResponse {
Logs(Vec<LogEntryDto>),
Timeline(Vec<TimelineBucketDto>),
Stats(StatsDto),
ParamDeclarations(Vec<ParamDeclarationDto>),
MetricDeclarations(Vec<MetricDeclarationDto>),
BucketValues(Vec<BucketValueDto>),
Promql(Result<Vec<PromqlSeriesDto>, String>),
Error(String),
}
impl EngineQuery {
pub(crate) fn sort_by(&self) -> Option<&str> {
match self {
EngineQuery::Logs(params) => params.sort_by.as_deref(),
_ => None,
}
}
}
+18 -4384
View File
File diff suppressed because it is too large Load Diff
+149 -12
View File
@@ -8,6 +8,7 @@ pub enum AggOp {
Rate, Rate,
Histogram, Histogram,
Gauge, Gauge,
Quantile,
} }
#[derive(Debug, Clone, PartialEq, Eq)] #[derive(Debug, Clone, PartialEq, Eq)]
@@ -23,6 +24,9 @@ pub struct PromqlQuery {
pub matchers: Vec<LabelMatcher>, pub matchers: Vec<LabelMatcher>,
pub range: Option<Duration>, pub range: Option<Duration>,
pub bucket_width: Option<f64>, pub bucket_width: Option<f64>,
/// The `p` in `histogram_quantile(p, metric[range])`, e.g. `0.95` for p95. Only set when
/// `agg` is `Some(AggOp::Quantile)`.
pub quantile: Option<f64>,
pub inner_rate: bool, pub inner_rate: bool,
pub by_labels: Vec<String>, pub by_labels: Vec<String>,
} }
@@ -38,7 +42,10 @@ impl PromqlQuery {
let query = parser.parse_query()?; let query = parser.parse_query()?;
parser.skip_whitespace(); parser.skip_whitespace();
if !parser.is_at_end() { if !parser.is_at_end() {
return Err(format!("unexpected trailing input: '{}'", parser.remaining())); return Err(format!(
"unexpected trailing input: '{}'",
parser.remaining()
));
} }
if query.agg == Some(AggOp::Rate) && query.range.is_none() { if query.agg == Some(AggOp::Rate) && query.range.is_none() {
return Err("rate() requires a range selector, e.g. rate(my_metric[5m])".to_string()); return Err("rate() requires a range selector, e.g. rate(my_metric[5m])".to_string());
@@ -48,6 +55,13 @@ impl PromqlQuery {
"histogram() requires a bucket width, e.g. histogram(my_metric[10])".to_string(), "histogram() requires a bucket width, e.g. histogram(my_metric[10])".to_string(),
); );
} }
if query.agg == Some(AggOp::Quantile) && query.quantile.is_none() {
return Err(
"histogram_quantile() requires a quantile between 0 and 1 and a metric, e.g. \
histogram_quantile(0.95, latency_ms)"
.to_string(),
);
}
Ok(query) Ok(query)
} }
} }
@@ -113,7 +127,9 @@ impl<'a> Parser<'a> {
fn parse_quoted_string(&mut self) -> Result<String, String> { fn parse_quoted_string(&mut self) -> Result<String, String> {
self.expect_char('"')?; self.expect_char('"')?;
let rest = self.remaining(); let rest = self.remaining();
let end = rest.find('"').ok_or("unterminated string literal (missing closing '\"')")?; let end = rest
.find('"')
.ok_or("unterminated string literal (missing closing '\"')")?;
let value = &rest[..end]; let value = &rest[..end];
if value.contains('\\') { if value.contains('\\') {
return Err("escape sequences in label values are not supported".to_string()); return Err("escape sequences in label values are not supported".to_string());
@@ -185,7 +201,9 @@ impl<'a> Parser<'a> {
return Ok(Some(RangeSelector::BareWidth(digits as f64))); return Ok(Some(RangeSelector::BareWidth(digits as f64)));
} }
let unit = self.peek().ok_or("range selector is missing a duration unit or ']'")?; let unit = self
.peek()
.ok_or("range selector is missing a duration unit or ']'")?;
let seconds_per_unit: i64 = match unit { let seconds_per_unit: i64 = match unit {
's' => 1, 's' => 1,
'm' => 60, 'm' => 60,
@@ -218,11 +236,42 @@ impl<'a> Parser<'a> {
matchers, matchers,
range, range,
bucket_width, bucket_width,
quantile: None,
inner_rate: false, inner_rate: false,
by_labels: Vec::new(), by_labels: Vec::new(),
}) })
} }
/// Parses a decimal number (e.g. `0`, `1`, `0.95`) — used for `histogram_quantile(p, ...)`'s
/// `p` argument, which isn't an integer like the range-selector widths elsewhere in this grammar.
fn parse_float(&mut self) -> Result<f64, String> {
self.skip_whitespace();
let rest = self.remaining();
let mut end = 0;
let mut seen_digit = false;
let mut seen_dot = false;
for (idx, c) in rest.char_indices() {
if c.is_ascii_digit() {
seen_digit = true;
end = idx + c.len_utf8();
} else if c == '.' && !seen_dot {
seen_dot = true;
end = idx + c.len_utf8();
} else {
break;
}
}
if !seen_digit {
return Err("expected a number".to_string());
}
let text = &rest[..end];
let value: f64 = text
.parse()
.map_err(|_| format!("'{text}' is not a valid number"))?;
self.pos += end;
Ok(value)
}
fn parse_by_clause(&mut self) -> Result<Vec<String>, String> { fn parse_by_clause(&mut self) -> Result<Vec<String>, String> {
self.skip_whitespace(); self.skip_whitespace();
let checkpoint = self.pos; let checkpoint = self.pos;
@@ -269,15 +318,33 @@ impl<'a> Parser<'a> {
"rate" => Some(AggOp::Rate), "rate" => Some(AggOp::Rate),
"histogram" => Some(AggOp::Histogram), "histogram" => Some(AggOp::Histogram),
"gauge" => Some(AggOp::Gauge), "gauge" => Some(AggOp::Gauge),
"histogram_quantile" => Some(AggOp::Quantile),
_ => None, _ => None,
}; };
self.skip_whitespace(); self.skip_whitespace();
let prefix_by_labels = if agg.is_some() { self.parse_by_clause()? } else { Vec::new() }; let prefix_by_labels = if agg.is_some() {
self.parse_by_clause()?
} else {
Vec::new()
};
self.skip_whitespace(); self.skip_whitespace();
if agg.is_some() && self.peek() == Some('(') { if agg.is_some() && self.peek() == Some('(') {
self.expect_char('(')?; self.expect_char('(')?;
self.skip_whitespace(); self.skip_whitespace();
let mut inner = if matches!(agg, Some(AggOp::Sum) | Some(AggOp::Avg)) { let mut inner = if agg == Some(AggOp::Quantile) {
let quantile = self.parse_float()?;
if !(0.0..=1.0).contains(&quantile) {
return Err(format!(
"histogram_quantile() quantile must be between 0 and 1, got {quantile}"
));
}
self.skip_whitespace();
self.expect_char(',')?;
self.skip_whitespace();
let mut vector = self.parse_vector_selector()?;
vector.quantile = Some(quantile);
vector
} else if matches!(agg, Some(AggOp::Sum) | Some(AggOp::Avg)) {
let inner_checkpoint = self.pos; let inner_checkpoint = self.pos;
let took_inner_rate = self.parse_identifier().is_ok_and(|id| id == "rate"); let took_inner_rate = self.parse_identifier().is_ok_and(|id| id == "rate");
if took_inner_rate { if took_inner_rate {
@@ -314,8 +381,11 @@ impl<'a> Parser<'a> {
.to_string(), .to_string(),
); );
} }
inner.by_labels = inner.by_labels = if prefix_by_labels.is_empty() {
if prefix_by_labels.is_empty() { trailing_by_labels } else { prefix_by_labels }; trailing_by_labels
} else {
prefix_by_labels
};
Ok(inner) Ok(inner)
} else { } else {
self.pos = checkpoint; self.pos = checkpoint;
@@ -343,7 +413,10 @@ mod tests {
assert_eq!(query.metric_name, "requests"); assert_eq!(query.metric_name, "requests");
assert_eq!( assert_eq!(
query.matchers, query.matchers,
vec![LabelMatcher { name: "path".to_string(), value: "/a".to_string() }] vec![LabelMatcher {
name: "path".to_string(),
value: "/a".to_string()
}]
); );
} }
@@ -472,7 +545,13 @@ mod tests {
fn parses_gauge_with_no_range_selector() { fn parses_gauge_with_no_range_selector() {
let query = PromqlQuery::parse(r#"gauge(connections{host="a"})"#).unwrap(); let query = PromqlQuery::parse(r#"gauge(connections{host="a"})"#).unwrap();
assert_eq!(query.agg, Some(AggOp::Gauge)); assert_eq!(query.agg, Some(AggOp::Gauge));
assert_eq!(query.matchers, vec![LabelMatcher { name: "host".to_string(), value: "a".to_string() }]); assert_eq!(
query.matchers,
vec![LabelMatcher {
name: "host".to_string(),
value: "a".to_string()
}]
);
} }
#[test] #[test]
@@ -497,7 +576,13 @@ mod tests {
let query = PromqlQuery::parse(r#"avg(rate(requests{path="/a"}[5m]))"#).unwrap(); let query = PromqlQuery::parse(r#"avg(rate(requests{path="/a"}[5m]))"#).unwrap();
assert_eq!(query.agg, Some(AggOp::Avg)); assert_eq!(query.agg, Some(AggOp::Avg));
assert!(query.inner_rate); assert!(query.inner_rate);
assert_eq!(query.matchers, vec![LabelMatcher { name: "path".to_string(), value: "/a".to_string() }]); assert_eq!(
query.matchers,
vec![LabelMatcher {
name: "path".to_string(),
value: "/a".to_string()
}]
);
} }
#[test] #[test]
@@ -518,7 +603,10 @@ mod tests {
fn parses_sum_of_rate_with_by_clause_multiple_labels() { fn parses_sum_of_rate_with_by_clause_multiple_labels() {
let query = PromqlQuery::parse("sum(rate(requests[1m])) by (path, method)").unwrap(); let query = PromqlQuery::parse("sum(rate(requests[1m])) by (path, method)").unwrap();
assert!(query.inner_rate); assert!(query.inner_rate);
assert_eq!(query.by_labels, vec!["path".to_string(), "method".to_string()]); assert_eq!(
query.by_labels,
vec!["path".to_string(), "method".to_string()]
);
} }
#[test] #[test]
@@ -548,7 +636,10 @@ mod tests {
#[test] #[test]
fn parses_sum_with_prefix_by_clause_multiple_labels() { fn parses_sum_with_prefix_by_clause_multiple_labels() {
let query = PromqlQuery::parse("sum by (path, method) (requests)").unwrap(); let query = PromqlQuery::parse("sum by (path, method) (requests)").unwrap();
assert_eq!(query.by_labels, vec!["path".to_string(), "method".to_string()]); assert_eq!(
query.by_labels,
vec!["path".to_string(), "method".to_string()]
);
} }
#[test] #[test]
@@ -556,4 +647,50 @@ mod tests {
let err = PromqlQuery::parse("sum by (path) (requests) by (method)").unwrap_err(); let err = PromqlQuery::parse("sum by (path) (requests) by (method)").unwrap_err();
assert!(err.contains("only appear once")); assert!(err.contains("only appear once"));
} }
#[test]
fn parses_histogram_quantile() {
let query = PromqlQuery::parse("histogram_quantile(0.95, latency_ms)").unwrap();
assert_eq!(query.agg, Some(AggOp::Quantile));
assert_eq!(query.metric_name, "latency_ms");
assert_eq!(query.quantile, Some(0.95));
assert_eq!(query.range, None);
}
#[test]
fn parses_histogram_quantile_with_range_and_matchers() {
let query =
PromqlQuery::parse(r#"histogram_quantile(0.5, latency_ms{path="/a"}[5m])"#).unwrap();
assert_eq!(query.quantile, Some(0.5));
assert_eq!(query.range, Some(Duration::from_secs(300)));
assert_eq!(
query.matchers,
vec![LabelMatcher {
name: "path".to_string(),
value: "/a".to_string()
}]
);
}
#[test]
fn parses_histogram_quantile_with_by_clause() {
let query = PromqlQuery::parse("histogram_quantile(0.99, latency_ms) by (path)").unwrap();
assert_eq!(query.by_labels, vec!["path".to_string()]);
}
#[test]
fn rejects_histogram_quantile_missing_comma() {
assert!(PromqlQuery::parse("histogram_quantile(0.95 latency_ms)").is_err());
}
#[test]
fn rejects_histogram_quantile_out_of_range() {
let err = PromqlQuery::parse("histogram_quantile(1.5, latency_ms)").unwrap_err();
assert!(err.contains("between 0 and 1"));
}
#[test]
fn rejects_histogram_quantile_without_number() {
assert!(PromqlQuery::parse("histogram_quantile(latency_ms)").is_err());
}
} }
+17 -5
View File
@@ -20,7 +20,10 @@ pub fn rate_at(points: &[(Timestamp, f64)], now: Timestamp, window_ms: i64) -> f
} }
pub fn windowed_rate_series(points: &[(Timestamp, f64)], window_ms: i64) -> Vec<(Timestamp, f64)> { pub fn windowed_rate_series(points: &[(Timestamp, f64)], window_ms: i64) -> Vec<(Timestamp, f64)> {
points.iter().map(|(ts, _)| (*ts, rate_at(points, *ts, window_ms))).collect() points
.iter()
.map(|(ts, _)| (*ts, rate_at(points, *ts, window_ms)))
.collect()
} }
#[derive(Debug, Default, Clone)] #[derive(Debug, Default, Clone)]
@@ -29,7 +32,11 @@ pub struct RateWindow {
} }
impl RateWindow { impl RateWindow {
pub fn backfill(timestamps: impl IntoIterator<Item = Timestamp>, now: Timestamp, window_ms: i64) -> Self { pub fn backfill(
timestamps: impl IntoIterator<Item = Timestamp>,
now: Timestamp,
window_ms: i64,
) -> Self {
let mut window = Self::default(); let mut window = Self::default();
for ts in timestamps { for ts in timestamps {
window.push(ts, window_ms); window.push(ts, window_ms);
@@ -94,9 +101,14 @@ mod tests {
for ms in millis { for ms in millis {
window.push(Timestamp::from_millis(ms), 1000); window.push(Timestamp::from_millis(ms), 1000);
} }
let points: Vec<(Timestamp, f64)> = let points: Vec<(Timestamp, f64)> = millis
millis.iter().map(|ms| (Timestamp::from_millis(*ms), 1.0)).collect(); .iter()
assert_eq!(window.count(), count_in_window(&points, Timestamp::from_millis(1200), 1000)); .map(|ms| (Timestamp::from_millis(*ms), 1.0))
.collect();
assert_eq!(
window.count(),
count_in_window(&points, Timestamp::from_millis(1200), 1000)
);
} }
#[test] #[test]
+213
View File
@@ -0,0 +1,213 @@
//! Compiling and evaluating the search-bar filter syntax (`filter_parser`) against log lines
//! and buckets: regex matches, the `ottyl_seq`/`ottyl_ts`/`ottyl_fmt` pseudo-metrics, per-param
//! metric-bucket comparisons, and ad-hoc JSON field comparisons.
use crate::engine::filter_datetime;
use crate::engine::filter_parser::{CompareOp, FilterValue};
use crate::engine::metric_primitive::MetricPrimitive;
use crate::engine::timestamp::Timestamp;
use regex::Regex;
use std::cmp::Ordering;
use std::collections::HashSet;
pub(super) enum CompiledFilter {
Regex(Regex),
Sequence {
op: CompareOp,
target: f64,
},
Timestamp {
op: CompareOp,
target_ms: i64,
},
Fmt {
op: CompareOp,
target: String,
},
MetricMatch(HashSet<u64>),
Json {
field: String,
op: CompareOp,
value: FilterValue,
},
And(Box<CompiledFilter>, Box<CompiledFilter>),
Or(Box<CompiledFilter>, Box<CompiledFilter>),
}
impl CompiledFilter {
pub(super) fn from_seq_comparison(op: CompareOp, value: &FilterValue) -> Result<Self, String> {
let target = match value {
FilterValue::Number(n) => *n,
FilterValue::Raw(raw) => raw.parse::<f64>().map_err(|_| {
format!("'ottyl_seq' comparisons require a numeric value, got {raw:?}")
})?,
};
Ok(CompiledFilter::Sequence { op, target })
}
pub(super) fn from_ts_comparison(
op: CompareOp,
value: &FilterValue,
now: Timestamp,
) -> Result<Self, String> {
let target_ms = match value {
FilterValue::Number(n) => *n as i64,
FilterValue::Raw(raw) => filter_datetime::parse_datetime(raw, now)
.map_err(|err| err.to_string())?
.as_millis(),
};
Ok(CompiledFilter::Timestamp { op, target_ms })
}
pub(super) fn from_fmt_comparison(op: CompareOp, value: &FilterValue) -> Self {
let target = match value {
FilterValue::Number(n) => n.to_string(),
FilterValue::Raw(raw) => raw.clone(),
};
CompiledFilter::Fmt { op, target }
}
}
impl MetricPrimitive {
pub(super) fn compare(
&self,
op: CompareOp,
value: &FilterValue,
now: Timestamp,
) -> Result<bool, String> {
let ordering = match (self, value) {
(MetricPrimitive::Number(n), FilterValue::Number(v)) => n.as_f64().partial_cmp(v),
(MetricPrimitive::Number(n), FilterValue::Raw(raw)) => {
let v: f64 = raw
.parse()
.map_err(|_| format!("expected a numeric value, got {raw:?}"))?;
n.as_f64().partial_cmp(&v)
}
(MetricPrimitive::Timestamp(ts), FilterValue::Number(v)) => {
ts.as_millis().partial_cmp(&(*v as i64))
}
(MetricPrimitive::Timestamp(ts), FilterValue::Raw(raw)) => {
let target =
filter_datetime::parse_datetime(raw, now).map_err(|err| err.to_string())?;
ts.as_millis().partial_cmp(&target.as_millis())
}
(MetricPrimitive::String(s), FilterValue::Raw(raw)) => {
s.as_str().partial_cmp(raw.as_str())
}
(MetricPrimitive::String(s), FilterValue::Number(n)) => {
s.as_str().partial_cmp(n.to_string().as_str())
}
(MetricPrimitive::Duration(ms), FilterValue::Number(v)) => (*ms as f64).partial_cmp(v),
(MetricPrimitive::Duration(ms), FilterValue::Raw(raw)) => {
let v: f64 = raw
.parse()
.map_err(|_| format!("expected a numeric value, got {raw:?}"))?;
(*ms as f64).partial_cmp(&v)
}
(MetricPrimitive::Boolean(b), FilterValue::Raw(raw)) => {
let v: bool = raw
.parse()
.map_err(|_| format!("expected true or false, got {raw:?}"))?;
b.partial_cmp(&v)
}
(MetricPrimitive::Boolean(_), FilterValue::Number(_)) => None,
};
Ok(ordering.is_some_and(|ordering| op.matches(ordering)))
}
}
pub(crate) fn eval_json_comparison(
text: &str,
field: &str,
op: CompareOp,
value: &FilterValue,
) -> bool {
let Ok(parsed) = serde_json::from_str::<serde_json::Value>(text) else {
return false;
};
let Some(field_value) = parsed.as_object().and_then(|obj| obj.get(field)) else {
return false;
};
compare_json_value(field_value, op, value)
}
fn filter_value_as_f64(value: &FilterValue) -> Option<f64> {
match value {
FilterValue::Number(v) => Some(*v),
FilterValue::Raw(raw) => raw.parse::<f64>().ok(),
}
}
fn compare_json_value(field_value: &serde_json::Value, op: CompareOp, value: &FilterValue) -> bool {
match field_value {
serde_json::Value::Number(n) => matches!(
(n.as_f64(), filter_value_as_f64(value)),
(Some(f), Some(v)) if f.partial_cmp(&v).is_some_and(|ordering| op.matches(ordering))
),
serde_json::Value::String(s) => {
let target = match value {
FilterValue::Raw(raw) => raw.clone(),
FilterValue::Number(n) => n.to_string(),
};
s.as_str()
.partial_cmp(target.as_str())
.is_some_and(|ordering| op.matches(ordering))
}
serde_json::Value::Bool(b) => {
let FilterValue::Raw(raw) = value else {
return false;
};
match raw.parse::<bool>() {
Ok(v) => match op {
CompareOp::Eq => *b == v,
CompareOp::Ne => *b != v,
_ => false,
},
Err(_) => false,
}
}
serde_json::Value::Null | serde_json::Value::Array(_) | serde_json::Value::Object(_) => {
false
}
}
}
impl CompareOp {
pub(crate) fn matches(&self, ordering: Ordering) -> bool {
match self {
CompareOp::Lt => ordering == Ordering::Less,
CompareOp::Gt => ordering == Ordering::Greater,
CompareOp::Le => ordering != Ordering::Greater,
CompareOp::Ge => ordering != Ordering::Less,
CompareOp::Eq => ordering == Ordering::Equal,
CompareOp::Ne => ordering != Ordering::Equal,
}
}
}
impl CompiledFilter {
pub(super) fn eval(&self, sequence: u64, timestamp: Timestamp, text: &str) -> bool {
match self {
CompiledFilter::Regex(re) => re.is_match(text),
CompiledFilter::Sequence { op, target } => {
op.matches((sequence as f64).total_cmp(target))
}
CompiledFilter::Timestamp { op, target_ms } => {
op.matches(timestamp.as_millis().cmp(target_ms))
}
CompiledFilter::Fmt { op, target } => {
op.matches(crate::engine::logfmt::detect_format(text).cmp(target.as_str()))
}
CompiledFilter::MetricMatch(sequences) => sequences.contains(&sequence),
CompiledFilter::Json { field, op, value } => {
eval_json_comparison(text, field, *op, value)
}
CompiledFilter::And(lhs, rhs) => {
lhs.eval(sequence, timestamp, text) && rhs.eval(sequence, timestamp, text)
}
CompiledFilter::Or(lhs, rhs) => {
lhs.eval(sequence, timestamp, text) || rhs.eval(sequence, timestamp, text)
}
}
}
}
+29
View File
@@ -0,0 +1,29 @@
//! Metric declaration and application: validating/storing declarations, applying them to
//! ingested lines to populate metric buckets, and backfilling a newly (re)declared metric
//! against already-ingested logs.
use crate::engine::data_value::MetricPrimitiveKind;
use crate::engine::metric_primitive::{MetricPrimitive, Number};
use crate::engine::timestamp::Timestamp;
pub(super) fn parse_metric_primitive(
raw: &str,
kind: MetricPrimitiveKind,
) -> Option<MetricPrimitive> {
match kind {
MetricPrimitiveKind::String => Some(MetricPrimitive::String(raw.to_string())),
MetricPrimitiveKind::Number => raw
.parse::<i64>()
.map(Number::Int)
.or_else(|_| raw.parse::<f64>().map(Number::Float))
.ok()
.map(MetricPrimitive::Number),
MetricPrimitiveKind::Timestamp => {
crate::engine::filter_datetime::parse_datetime(raw, Timestamp::now())
.ok()
.map(MetricPrimitive::Timestamp)
}
MetricPrimitiveKind::Duration => raw.parse::<i64>().ok().map(MetricPrimitive::Duration),
MetricPrimitiveKind::Boolean => raw.parse::<bool>().ok().map(MetricPrimitive::Boolean),
}
}
File diff suppressed because it is too large Load Diff
+112
View File
@@ -0,0 +1,112 @@
//! Param declaration: validating and storing explicit declarations, and the auto-detection
//! logic that derives declarations from logfmt/json/log-stream-shaped lines as they're ingested.
#[cfg(test)]
use crate::engine::DeclareParamRequest;
use crate::engine::data_value::MetricPrimitiveKind;
#[cfg(test)]
use crate::engine::snapshot::EngineSnapshot;
use regex::Regex;
#[cfg(test)]
pub(crate) fn declare_param(
snapshot: &mut EngineSnapshot,
request: DeclareParamRequest,
) -> Result<(), String> {
snapshot.validate_param_declaration(&request)?;
snapshot.insert_param_declaration(request);
Ok(())
}
pub(crate) fn sanitize_group_name(key: &str) -> String {
let mut group: String = key
.chars()
.map(|c| {
if c.is_ascii_alphanumeric() || c == '_' {
c
} else {
'_'
}
})
.collect();
if group.chars().next().is_none_or(|c| c.is_ascii_digit()) {
group.insert(0, '_');
}
group
}
pub(crate) fn infer_kind(sample: &str) -> MetricPrimitiveKind {
if sample.parse::<i64>().is_ok() || sample.parse::<f64>().is_ok() {
MetricPrimitiveKind::Number
} else {
MetricPrimitiveKind::String
}
}
pub(crate) fn build_auto_param_pattern(key: &str, group: &str, quoted: bool) -> String {
let escaped_key = regex::escape(key);
if quoted {
format!(r#"{escaped_key}="(?P<{group}>[^"]*)""#)
} else {
format!(r"{escaped_key}=(?P<{group}>\S+)")
}
}
pub(crate) struct AutoParamCandidate {
pub(crate) name: String,
pub(crate) value: String,
pub(crate) message_match_regex: String,
pub(crate) capture_group: String,
pub(crate) json_field_path: Option<String>,
}
pub(crate) fn log_stream_header_transition(
current: &Option<crate::engine::logfmt::LogStreamHeader>,
text: &str,
) -> Option<crate::engine::logfmt::LogStreamHeader> {
crate::engine::logfmt::LogStreamHeader::detect(text).or_else(|| current.clone())
}
#[cfg(test)]
pub(crate) fn auto_declare_log_stream_params(
snapshot: &mut EngineSnapshot,
header: &mut Option<crate::engine::logfmt::LogStreamHeader>,
text: &str,
) {
let is_new_header = crate::engine::logfmt::LogStreamHeader::detect(text).is_some();
*header = log_stream_header_transition(header, text);
if is_new_header {
return;
}
if let Some(header) = header.clone() {
snapshot.derive_log_stream_params_from_row(&header, text);
}
}
pub(super) struct ResolvedParam {
pub(super) regex: Regex,
pub(super) capture_group: String,
pub(super) json_field_path: Option<String>,
pub(super) kind: MetricPrimitiveKind,
pub(super) elapsed_since_last: bool,
pub(super) log_stream_gated: bool,
}
pub(crate) fn extract_param_raw(
text: &str,
regex: &Regex,
capture_group: &str,
json_field_path: Option<&str>,
log_stream_gated: bool,
) -> Option<String> {
if log_stream_gated && !crate::engine::logfmt::is_log_stream_line(text) {
return None;
}
match json_field_path.filter(|path| !path.is_empty()) {
Some(field_path) => crate::engine::data_value::extract_json_field(text, field_path),
None => regex
.captures(text)?
.name(capture_group)
.map(|m| m.as_str().to_string()),
}
}
+477
View File
@@ -0,0 +1,477 @@
//! Answering `EngineQuery`s: log/timeline listing (including the sort-helper-index path),
//! stats, declaration listing, bucket-value listing, and PromQL evaluation.
use crate::api::dto::{
BucketValueDto, LogEntryDto, MetricDeclarationDto, ParamDeclarationDto, StatsDto,
TimelineBucketDto,
};
use crate::consts::{DEFAULT_TIMELINE_BUCKET_MS, MERGE_WINDOW_MS};
use crate::engine::data_value::DataValue;
use crate::engine::index_key::IndexKey;
use crate::engine::message::{EngineQuery, EngineResponse};
use crate::engine::metric_primitive::{MetricPrimitive, Number};
use crate::engine::promql::{LabelMatcher, PromqlQuery};
use crate::engine::rate;
use crate::engine::snapshot::EngineSnapshot;
use crate::engine::snapshot::filter::CompiledFilter;
#[cfg(test)]
use crate::engine::snapshot::sort::SortHelperIndexEntry;
use crate::engine::snapshot::sort::{
CompiledParamDeclaration, SortHelperAccess, SortHelperKey, log_entry_params,
};
use crate::engine::timestamp::Timestamp;
use std::collections::BTreeMap;
#[cfg(test)]
use std::collections::HashMap;
#[cfg(test)]
pub(crate) fn handle_query(
snapshot: &EngineSnapshot,
query: EngineQuery,
sort_helper_indexes: &mut HashMap<String, SortHelperIndexEntry>,
) -> EngineResponse {
handle_query_inner(
snapshot,
query,
SortHelperAccess::Mutable(sort_helper_indexes),
)
}
/// Compiles `filter` against `snapshot`, or returns `Ok(None)` when there's no filter string
/// to compile. Shared by the `Logs` and `Timeline` query arms, which both bail out early with
/// `EngineResponse::Error` on a compile failure.
fn compile_optional_filter(
snapshot: &EngineSnapshot,
filter: Option<&str>,
now: Timestamp,
) -> Result<Option<CompiledFilter>, String> {
filter
.map(|filter| snapshot.compile_filter_query(filter, now))
.transpose()
}
/// Walks the sort-helper index for `sort_by` — rather than `snapshot.logs_range()`, which is
/// ordered by sequence, not by the param's value — collecting up to `limit` matching entries
/// in sort-by-param order.
///
/// Not range-bounded by since_sequence/start_ms the way the unsorted path is: `SortHelperKey::
/// WithParam` sorts by the param's value first (that's the point — it's what gives sort-by-param
/// order), so sequence/time bounds can't narrow a BTreeMap range seek here.
#[allow(clippy::too_many_arguments)]
fn logs_sorted_by_param(
snapshot: &EngineSnapshot,
sort_by: &str,
sort_helper_access: SortHelperAccess<'_>,
compiled_filter: Option<&CompiledFilter>,
compiled_params: &[CompiledParamDeclaration],
since_sequence: Option<u64>,
start_ms: Option<i64>,
end_ms: Option<i64>,
sort_missing_fallback: bool,
limit: usize,
) -> Vec<LogEntryDto> {
let index = &snapshot.index;
static EMPTY_HELPER: std::sync::LazyLock<BTreeMap<SortHelperKey, Timestamp>> =
std::sync::LazyLock::new(BTreeMap::new);
let helper_index = sort_helper_access
.resolve(index, sort_by)
.unwrap_or(&EMPTY_HELPER);
let mut entries = Vec::new();
for (helper_key, timestamp) in helper_index.iter() {
let sequence = match helper_key {
SortHelperKey::WithParam(_, sequence) => *sequence,
SortHelperKey::WithoutParam(sequence) => {
if !sort_missing_fallback {
continue;
}
*sequence
}
};
if !(since_sequence.is_none_or(|since| sequence > since)
&& start_ms.is_none_or(|start| timestamp.as_millis() >= start)
&& end_ms.is_none_or(|end| timestamp.as_millis() <= end))
{
continue;
}
let Some(DataValue::LogLiteral { text }) = index.get(&IndexKey::IngestedLog {
sequence,
timestamp: *timestamp,
}) else {
continue;
};
if !compiled_filter.is_none_or(|filter| filter.eval(sequence, *timestamp, text)) {
continue;
}
entries.push(LogEntryDto {
sequence,
timestamp_ms: timestamp.as_millis(),
text: text.to_string(),
params: log_entry_params(text, compiled_params),
});
if entries.len() >= limit {
break;
}
}
entries
}
pub(super) fn handle_query_inner(
snapshot: &EngineSnapshot,
query: EngineQuery,
sort_helper_access: SortHelperAccess<'_>,
) -> EngineResponse {
let approx_bytes_total = snapshot.approx_bytes_total;
let ingest_lag_high_watermark_ms = snapshot.ingest_lag_high_watermark_ms;
match query {
EngineQuery::Logs(params) => {
let compiled_filter =
match compile_optional_filter(snapshot, params.filter.as_deref(), Timestamp::now())
{
Ok(compiled) => compiled,
Err(err) => return EngineResponse::Error(err),
};
if let Some(sort_by) = params.sort_by.as_deref()
&& !snapshot
.param_declarations_range()
.any(|(name, ..)| name == sort_by)
{
return EngineResponse::Error(format!("unknown param '{sort_by}' for sort_by"));
}
let limit = params.limit.unwrap_or(500).min(10_000);
let compiled_params = snapshot.compiled_param_declarations();
let since_bound = match params.since_sequence {
Some(since) => std::ops::Bound::Excluded(since),
None => std::ops::Bound::Unbounded,
};
let filtered = snapshot
.logs_in_sequence_range((since_bound, std::ops::Bound::Unbounded))
.filter(|(_, timestamp, _)| {
params
.start_ms
.is_none_or(|start| timestamp.as_millis() >= start)
&& params.end_ms.is_none_or(|end| timestamp.as_millis() <= end)
})
.filter(|(sequence, timestamp, text)| {
compiled_filter
.as_ref()
.is_none_or(|filter| filter.eval(*sequence, *timestamp, text))
});
let entries: Vec<LogEntryDto> = match params.sort_by.as_deref() {
None => filtered
.take(limit)
.map(|(sequence, timestamp, text)| LogEntryDto {
sequence,
timestamp_ms: timestamp.as_millis(),
text: text.to_string(),
params: log_entry_params(text, &compiled_params),
})
.collect(),
Some(sort_by) => logs_sorted_by_param(
snapshot,
sort_by,
sort_helper_access,
compiled_filter.as_ref(),
&compiled_params,
params.since_sequence,
params.start_ms,
params.end_ms,
params.sort_missing_fallback.unwrap_or(true),
limit,
),
};
let entries: Vec<LogEntryDto> = entries.into_iter().take(limit).collect();
EngineResponse::Logs(entries)
}
EngineQuery::Timeline(params) => {
let compiled_filter =
match compile_optional_filter(snapshot, params.filter.as_deref(), Timestamp::now())
{
Ok(compiled) => compiled,
Err(err) => return EngineResponse::Error(err),
};
let bucket_ms = params
.bucket_ms
.unwrap_or(DEFAULT_TIMELINE_BUCKET_MS)
.max(1);
let mut buckets: BTreeMap<i64, u64> = BTreeMap::new();
for (sequence, timestamp, text) in snapshot.logs_range().filter(|(_, timestamp, _)| {
params
.start_ms
.is_none_or(|start| timestamp.as_millis() >= start)
&& params.end_ms.is_none_or(|end| timestamp.as_millis() <= end)
}) {
if compiled_filter
.as_ref()
.is_some_and(|filter| !filter.eval(sequence, timestamp, text))
{
continue;
}
let bucket_start = (timestamp.as_millis() / bucket_ms) * bucket_ms;
*buckets.entry(bucket_start).or_insert(0) += 1;
}
let out = buckets
.into_iter()
.map(|(start_ms, count)| TimelineBucketDto { start_ms, count })
.collect();
EngineResponse::Timeline(out)
}
EngineQuery::Stats => {
let mut log_count = 0u64;
let mut first_sequence = None;
let mut last_sequence = None;
let mut first_timestamp_ms = None;
let mut last_timestamp_ms = None;
for (sequence, timestamp, _) in snapshot.logs_range() {
log_count += 1;
first_sequence.get_or_insert(sequence);
last_sequence = Some(sequence);
first_timestamp_ms.get_or_insert(timestamp.as_millis());
last_timestamp_ms = Some(timestamp.as_millis());
}
EngineResponse::Stats(StatsDto {
log_count,
first_sequence,
last_sequence,
first_timestamp_ms,
last_timestamp_ms,
approx_bytes_total,
ingest_lag_high_watermark_ms,
})
}
EngineQuery::ParamDeclarations => {
let declarations = snapshot
.param_declarations_range()
.map(
|(
name,
message_match_regex,
capture_group,
json_field_path,
kind,
elapsed_since_last,
auto_derived,
_,
)| {
ParamDeclarationDto {
name: name.to_string(),
message_match_regex: message_match_regex.to_string(),
capture_group: capture_group.to_string(),
json_field_path: json_field_path.map(|path| path.to_string()),
kind,
elapsed_since_last,
auto_derived,
reserved: false,
}
},
)
.collect();
EngineResponse::ParamDeclarations(declarations)
}
EngineQuery::MetricDeclarations => {
let declarations = snapshot
.metric_declarations_range()
.map(
|(
name,
message_match_regex,
label_params,
value_param,
matched_labels,
unmatched_labels,
semantics,
)| {
MetricDeclarationDto {
name: name.to_string(),
message_match_regex: message_match_regex.to_string(),
label_params: label_params.to_vec(),
value_param: value_param.map(|s| s.to_string()),
matched_labels: matched_labels.cloned(),
unmatched_labels: unmatched_labels.cloned(),
semantics,
}
},
)
.collect();
EngineResponse::MetricDeclarations(declarations)
}
EngineQuery::BucketValues {
metric_name,
labels_filter,
start_ms,
end_ms,
since_sequence,
limit,
} => {
let limit = limit.unwrap_or(500).min(10_000);
let values: Vec<BucketValueDto> = snapshot
.bucket_values(&metric_name, None)
.filter(|(labels, _)| {
labels_filter.as_ref().is_none_or(|filter| {
filter.iter().all(|(name, expected)| {
labels
.get(name)
.is_some_and(|value| &value.to_string() == expected)
})
})
})
.filter(|(_, (sequence, timestamp, _))| {
since_sequence.is_none_or(|since| *sequence > since)
&& start_ms.is_none_or(|start| timestamp.as_millis() >= start)
&& end_ms.is_none_or(|end| timestamp.as_millis() <= end)
})
.take(limit)
.map(|(labels, (sequence, timestamp, value))| BucketValueDto {
labels: labels.clone(),
sequence: *sequence,
timestamp_ms: timestamp.as_millis(),
value: value.clone(),
})
.collect();
EngineResponse::BucketValues(values)
}
EngineQuery::Promql {
query,
start_ms,
end_ms,
} => {
let response = PromqlQuery::parse(&query)
.and_then(|parsed| snapshot.evaluate_promql(&parsed, start_ms, end_ms));
EngineResponse::Promql(response)
}
}
}
pub(crate) fn coerce_to_f64(value: &MetricPrimitive) -> f64 {
match value {
MetricPrimitive::Number(n) => match n {
Number::Int(v) => *v as f64,
Number::Float(v) => *v,
},
MetricPrimitive::Timestamp(ts) => ts.as_millis() as f64,
MetricPrimitive::Duration(ms) => *ms as f64,
MetricPrimitive::String(_) => 1.0,
MetricPrimitive::Boolean(b) => {
if *b {
1.0
} else {
0.0
}
}
}
}
pub(crate) fn matches_label_matchers(
labels: &BTreeMap<String, MetricPrimitive>,
matchers: &[LabelMatcher],
) -> bool {
matchers.iter().all(|matcher| {
labels
.get(&matcher.name)
.is_some_and(|value| value.to_string() == matcher.value)
})
}
pub(crate) fn project_labels(
labels: &BTreeMap<String, String>,
by_labels: &[String],
) -> BTreeMap<String, String> {
by_labels
.iter()
.filter_map(|name| labels.get(name).map(|value| (name.clone(), value.clone())))
.collect()
}
pub(crate) fn windowed_rate_series(
points: &[(i64, f64)],
range_ms: i64,
start_ms: Option<i64>,
end_ms: Option<i64>,
) -> Vec<(i64, f64)> {
let ts_points: Vec<(Timestamp, f64)> = points
.iter()
.map(|(ms, value)| (Timestamp::from_millis(*ms), *value))
.collect();
let mut series: Vec<(i64, f64)> = rate::windowed_rate_series(&ts_points, range_ms)
.into_iter()
.map(|(ts, value)| (ts.as_millis(), value))
.collect();
if let Some(end) = end_ms
&& series.last().is_none_or(|(ts, _)| *ts < end)
{
let value = rate::rate_at(&ts_points, Timestamp::from_millis(end), range_ms);
series.push((end, value));
}
if let Some(start) = start_ms
&& series.first().is_none_or(|(ts, _)| *ts > start)
{
let value = rate::rate_at(&ts_points, Timestamp::from_millis(start), range_ms);
series.insert(0, (start, value));
}
series
}
pub(crate) fn merge_series_windowed(
sub_series: &[Vec<(i64, f64)>],
is_avg: bool,
) -> Vec<(i64, f64)> {
if sub_series.len() <= 1 {
return sub_series.first().cloned().unwrap_or_default();
}
let mut grouped: BTreeMap<i64, Vec<f64>> = BTreeMap::new();
for points in sub_series {
for (timestamp_ms, value) in points {
let window_start = timestamp_ms.div_euclid(MERGE_WINDOW_MS) * MERGE_WINDOW_MS;
grouped.entry(window_start).or_default().push(*value);
}
}
grouped
.into_iter()
.map(|(timestamp_ms, values)| {
let sum: f64 = values.iter().sum();
let value = if is_avg {
sum / values.len() as f64
} else {
sum
};
(timestamp_ms, value)
})
.collect()
}
pub(crate) fn merge_series_locf(sub_series: &[Vec<(i64, f64)>], is_avg: bool) -> Vec<(i64, f64)> {
let mut all_timestamps: std::collections::BTreeSet<i64> = std::collections::BTreeSet::new();
for points in sub_series {
all_timestamps.extend(points.iter().map(|(ts, _)| *ts));
}
let mut cursors = vec![0usize; sub_series.len()];
let mut last_values = vec![0.0f64; sub_series.len()];
all_timestamps
.into_iter()
.map(|ts| {
for (i, points) in sub_series.iter().enumerate() {
while cursors[i] < points.len() && points[cursors[i]].0 <= ts {
last_values[i] = points[cursors[i]].1;
cursors[i] += 1;
}
}
let sum: f64 = last_values.iter().sum();
let value = if is_avg {
sum / last_values.len() as f64
} else {
sum
};
(ts, value)
})
.collect()
}
pub(crate) fn ottyl_reserved_numeric_value_fn(metric_name: &str) -> Option<fn(u64, i64) -> f64> {
match metric_name {
"ottyl_seq" => Some(|sequence, _timestamp_ms| sequence as f64),
"ottyl_ts" => Some(|_sequence, timestamp_ms| timestamp_ms as f64),
_ => None,
}
}
+143
View File
@@ -0,0 +1,143 @@
//! Sort-helper indexes: a per-param `sort_by` lets the logs view return results ordered by a
//! declared param's value instead of ingest order. Building that ordering from scratch is
//! O(n log n) over every log, so it's memoized in `EngineSnapshot::sort_helper_indexes` and
//! kept incrementally up to date on ingest, rather than rebuilt per query.
use crate::engine::data_value::DataValue;
use crate::engine::data_value::MetricPrimitiveKind;
use crate::engine::index_key::IndexKey;
use crate::engine::metric_primitive::MetricPrimitive;
#[cfg(test)]
use crate::engine::snapshot::EngineSnapshot;
use crate::engine::snapshot::metrics::parse_metric_primitive;
use crate::engine::timestamp::Timestamp;
use regex::Regex;
use std::collections::{BTreeMap, HashMap};
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)]
pub(crate) enum SortHelperKey {
WithParam(MetricPrimitive, u64),
WithoutParam(u64),
}
#[derive(Debug, Clone)]
pub(crate) struct SortHelperIndexEntry {
pub(crate) index: BTreeMap<SortHelperKey, Timestamp>,
pub(crate) last_accessed: std::time::Instant,
}
pub(super) type CompiledParamDeclaration = (
String,
Regex,
String,
Option<String>,
MetricPrimitiveKind,
bool,
);
pub(crate) fn derive_single_param(
text: &str,
declarations: &[CompiledParamDeclaration],
param_name: &str,
) -> Option<MetricPrimitive> {
let (_, regex, capture_group, json_field_path, kind, log_stream_gated) =
declarations.iter().find(|(name, ..)| name == param_name)?;
if *log_stream_gated && !crate::engine::logfmt::is_log_stream_line(text) {
return None;
}
if *kind == MetricPrimitiveKind::Boolean {
return Some(MetricPrimitive::Boolean(regex.is_match(text)));
}
let captures = regex.captures(text)?;
let raw: String = match json_field_path.as_deref().filter(|path| !path.is_empty()) {
Some(field_path) => crate::engine::data_value::extract_json_field(text, field_path)?,
None => captures.name(capture_group)?.as_str().to_string(),
};
parse_metric_primitive(&raw, *kind)
}
pub(crate) fn derive_log_params(
text: &str,
declarations: &[CompiledParamDeclaration],
) -> BTreeMap<String, MetricPrimitive> {
let mut params = BTreeMap::new();
for (name, regex, capture_group, json_field_path, kind, log_stream_gated) in declarations {
if *log_stream_gated && !crate::engine::logfmt::is_log_stream_line(text) {
continue;
}
if *kind == MetricPrimitiveKind::Boolean {
params.insert(name.clone(), MetricPrimitive::Boolean(regex.is_match(text)));
continue;
}
let Some(captures) = regex.captures(text) else {
continue;
};
let raw: String = match json_field_path.as_deref().filter(|path| !path.is_empty()) {
Some(field_path) => {
match crate::engine::data_value::extract_json_field(text, field_path) {
Some(value) => value,
None => continue,
}
}
None => match captures.name(capture_group) {
Some(m) => m.as_str().to_string(),
None => continue,
},
};
let Some(value) = parse_metric_primitive(&raw, *kind) else {
continue;
};
params.insert(name.clone(), value);
}
params
}
pub(super) fn log_entry_params(
text: &str,
declarations: &[CompiledParamDeclaration],
) -> BTreeMap<String, MetricPrimitive> {
let mut params = derive_log_params(text, declarations);
params.insert(
"ottyl_fmt".to_string(),
MetricPrimitive::String(crate::engine::logfmt::detect_format(text).to_string()),
);
params
}
pub(super) enum SortHelperAccess<'a> {
#[cfg(test)]
Mutable(&'a mut HashMap<String, SortHelperIndexEntry>),
Immutable(&'a HashMap<String, SortHelperIndexEntry>),
}
impl<'a> SortHelperAccess<'a> {
#[cfg_attr(not(test), allow(unused_variables))]
pub(super) fn resolve(
self,
index: &BTreeMap<IndexKey, DataValue>,
sort_by: &str,
) -> Option<&'a BTreeMap<SortHelperKey, Timestamp>> {
match self {
#[cfg(test)]
SortHelperAccess::Mutable(map) => {
if !map.contains_key(sort_by) {
let built =
EngineSnapshot::from_index(index.clone()).build_sort_helper_index(sort_by);
map.insert(
sort_by.to_string(),
SortHelperIndexEntry {
index: built,
last_accessed: std::time::Instant::now(),
},
);
}
let entry = map
.get_mut(sort_by)
.expect("just inserted above if missing");
entry.last_accessed = std::time::Instant::now();
map.get(sort_by).map(|entry| &entry.index)
}
SortHelperAccess::Immutable(map) => map.get(sort_by).map(|entry| &entry.index),
}
}
}
+3274
View File
File diff suppressed because it is too large Load Diff
+5
View File
@@ -5,6 +5,11 @@ use std::time::{SystemTime, UNIX_EPOCH};
pub struct Timestamp(i64); pub struct Timestamp(i64);
impl Timestamp { impl Timestamp {
/// The smallest representable timestamp — used as a synthetic sub-key when seeking
/// `IndexKey::IngestedLog { sequence, timestamp }` by `sequence` alone, since it sorts
/// below any real timestamp recorded for that sequence.
pub const MIN: Timestamp = Timestamp(i64::MIN);
pub fn now() -> Self { pub fn now() -> Self {
let nanos = SystemTime::now() let nanos = SystemTime::now()
.duration_since(UNIX_EPOCH) .duration_since(UNIX_EPOCH)
+12
View File
@@ -0,0 +1,12 @@
pub trait LogLiteralHelper {
fn approx_log_bytes(&self) -> u64;
}
impl<T> LogLiteralHelper for T
where
T: AsRef<str>,
{
fn approx_log_bytes(&self) -> u64 {
self.as_ref().len() as u64
}
}
+1
View File
@@ -3,6 +3,7 @@ mod cli;
mod config; mod config;
mod consts; mod consts;
mod engine; mod engine;
pub(crate) mod flavour;
#[tokio::main] #[tokio::main]
async fn main() { async fn main() {
+28
View File
@@ -0,0 +1,28 @@
:root {
--c-surface-0: hsl(210 32% 10%);
--c-surface-1: hsl(210 28% 16%);
--c-surface-2: hsl(210 26% 22%);
--c-border: hsl(210 24% 24%);
--c-border-soft: hsl(210 22% 33%);
--c-border-faint: hsl(210 28% 16%);
--c-border-strong: hsl(210 15% 62%);
--c-text: hsl(210 15% 94%);
--c-text-secondary: hsl(210 12% 82%);
--c-text-muted: hsl(210 10% 65%);
--c-text-subtle: hsl(210 8% 50%);
--c-text-faint: hsl(210 8% 38%);
--c-text-faintest: hsl(210 10% 28%);
--c-danger-bg: #450a0a;
--c-danger-border: #7f1d1d;
--c-danger-text: #f87171;
--c-danger-text-strong: #fca5a5;
--c-accent: hsl(30 70% 62%);
--c-accent-soft: hsl(0 55% 34%);
--c-accent-text: hsl(30 75% 80%);
--c-accent-2: hsl(300 65% 68%);
--c-accent-3: hsl(120 65% 55%);
}
+28
View File
@@ -0,0 +1,28 @@
:root {
--c-surface-0: #0a0a0a;
--c-surface-1: #171717;
--c-surface-2: #262626;
--c-border: #262626;
--c-border-soft: #404040;
--c-border-faint: #171717;
--c-border-strong: #a3a3a3;
--c-text: #f5f5f5;
--c-text-secondary: #d4d4d4;
--c-text-muted: #a3a3a3;
--c-text-subtle: #737373;
--c-text-faint: #525252;
--c-text-faintest: #404040;
--c-danger-bg: #450a0a;
--c-danger-border: #7f1d1d;
--c-danger-text: #f87171;
--c-danger-text-strong: #fca5a5;
--c-accent: #0ea5e9;
--c-accent-soft: #0369a1;
--c-accent-text: #7dd3fc;
--c-accent-2: #a78bfa;
--c-accent-3: #4ade80;
}
+28
View File
@@ -0,0 +1,28 @@
:root {
--c-surface-0: hsl(140 32% 10%);
--c-surface-1: hsl(140 28% 16%);
--c-surface-2: hsl(140 26% 22%);
--c-border: hsl(140 24% 24%);
--c-border-soft: hsl(140 22% 33%);
--c-border-faint: hsl(140 28% 16%);
--c-border-strong: hsl(140 15% 62%);
--c-text: hsl(140 15% 94%);
--c-text-secondary: hsl(140 12% 82%);
--c-text-muted: hsl(140 10% 65%);
--c-text-subtle: hsl(140 8% 50%);
--c-text-faint: hsl(140 8% 38%);
--c-text-faintest: hsl(140 10% 28%);
--c-danger-bg: #450a0a;
--c-danger-border: #7f1d1d;
--c-danger-text: #f87171;
--c-danger-text-strong: #fca5a5;
--c-accent: hsl(320 70% 62%);
--c-accent-soft: hsl(290 55% 34%);
--c-accent-text: hsl(320 75% 80%);
--c-accent-2: hsl(230 65% 62%);
--c-accent-3: hsl(50 65% 58%);
}
+28
View File
@@ -0,0 +1,28 @@
:root {
--c-surface-0: #fafafa;
--c-surface-1: #f5f5f5;
--c-surface-2: #e5e5e5;
--c-border: #e5e5e5;
--c-border-soft: #d4d4d4;
--c-border-faint: #f5f5f5;
--c-border-strong: #525252;
--c-text: #171717;
--c-text-secondary: #404040;
--c-text-muted: #525252;
--c-text-subtle: #737373;
--c-text-faint: #a3a3a3;
--c-text-faintest: #d4d4d4;
--c-danger-bg: #fee2e2;
--c-danger-border: #fca5a5;
--c-danger-text: #b91c1c;
--c-danger-text-strong: #991b1b;
--c-accent: #0284c7;
--c-accent-soft: #7dd3fc;
--c-accent-text: #0369a1;
--c-accent-2: #7c3aed;
--c-accent-3: #16a34a;
}
+28
View File
@@ -0,0 +1,28 @@
:root {
--c-surface-0: hsl(210 40% 96%);
--c-surface-1: hsl(210 35% 92%);
--c-surface-2: hsl(210 30% 85%);
--c-border: hsl(210 25% 80%);
--c-border-soft: hsl(210 22% 70%);
--c-border-faint: hsl(210 35% 92%);
--c-border-strong: hsl(210 20% 40%);
--c-text: hsl(210 30% 12%);
--c-text-secondary: hsl(210 20% 25%);
--c-text-muted: hsl(210 15% 40%);
--c-text-subtle: hsl(210 10% 50%);
--c-text-faint: hsl(210 10% 65%);
--c-text-faintest: hsl(210 12% 80%);
--c-danger-bg: #fee2e2;
--c-danger-border: #fca5a5;
--c-danger-text: #b91c1c;
--c-danger-text-strong: #991b1b;
--c-accent: hsl(30 70% 40%);
--c-accent-soft: hsl(0 60% 85%);
--c-accent-text: hsl(30 75% 28%);
--c-accent-2: hsl(300 70% 45%);
--c-accent-3: hsl(120 70% 38%);
}
+28
View File
@@ -0,0 +1,28 @@
:root {
--c-surface-0: hsl(140 40% 96%);
--c-surface-1: hsl(140 35% 92%);
--c-surface-2: hsl(140 30% 85%);
--c-border: hsl(140 25% 80%);
--c-border-soft: hsl(140 22% 70%);
--c-border-faint: hsl(140 35% 92%);
--c-border-strong: hsl(140 20% 40%);
--c-text: hsl(140 30% 12%);
--c-text-secondary: hsl(140 20% 25%);
--c-text-muted: hsl(140 15% 40%);
--c-text-subtle: hsl(140 10% 50%);
--c-text-faint: hsl(140 10% 65%);
--c-text-faintest: hsl(140 12% 80%);
--c-danger-bg: #fee2e2;
--c-danger-border: #fca5a5;
--c-danger-text: #b91c1c;
--c-danger-text-strong: #991b1b;
--c-accent: hsl(320 70% 40%);
--c-accent-soft: hsl(290 60% 85%);
--c-accent-text: hsl(320 75% 28%);
--c-accent-2: hsl(230 70% 45%);
--c-accent-3: hsl(50 70% 38%);
}
+28
View File
@@ -0,0 +1,28 @@
:root {
--c-surface-0: hsl(30 40% 96%);
--c-surface-1: hsl(30 35% 92%);
--c-surface-2: hsl(30 30% 85%);
--c-border: hsl(30 25% 80%);
--c-border-soft: hsl(30 22% 70%);
--c-border-faint: hsl(30 35% 92%);
--c-border-strong: hsl(30 20% 40%);
--c-text: hsl(30 30% 12%);
--c-text-secondary: hsl(30 20% 25%);
--c-text-muted: hsl(30 15% 40%);
--c-text-subtle: hsl(30 10% 50%);
--c-text-faint: hsl(30 10% 65%);
--c-text-faintest: hsl(30 12% 80%);
--c-danger-bg: #fee2e2;
--c-danger-border: #fca5a5;
--c-danger-text: #b91c1c;
--c-danger-text-strong: #991b1b;
--c-accent: hsl(210 70% 40%);
--c-accent-soft: hsl(180 60% 85%);
--c-accent-text: hsl(210 75% 28%);
--c-accent-2: hsl(120 70% 45%);
--c-accent-3: hsl(300 70% 38%);
}
+28
View File
@@ -0,0 +1,28 @@
:root {
--c-surface-0: hsl(330 40% 96%);
--c-surface-1: hsl(330 35% 92%);
--c-surface-2: hsl(330 30% 85%);
--c-border: hsl(330 25% 80%);
--c-border-soft: hsl(330 22% 70%);
--c-border-faint: hsl(330 35% 92%);
--c-border-strong: hsl(330 20% 40%);
--c-text: hsl(330 30% 12%);
--c-text-secondary: hsl(330 20% 25%);
--c-text-muted: hsl(330 15% 40%);
--c-text-subtle: hsl(330 10% 50%);
--c-text-faint: hsl(330 10% 65%);
--c-text-faintest: hsl(330 12% 80%);
--c-danger-bg: #fee2e2;
--c-danger-border: #fca5a5;
--c-danger-text: #b91c1c;
--c-danger-text-strong: #991b1b;
--c-accent: hsl(150 70% 40%);
--c-accent-soft: hsl(120 60% 85%);
--c-accent-text: hsl(150 75% 28%);
--c-accent-2: hsl(60 70% 45%);
--c-accent-3: hsl(240 70% 38%);
}
+28
View File
@@ -0,0 +1,28 @@
:root {
--c-surface-0: hsl(0 40% 96%);
--c-surface-1: hsl(0 35% 92%);
--c-surface-2: hsl(0 30% 85%);
--c-border: hsl(0 25% 80%);
--c-border-soft: hsl(0 22% 70%);
--c-border-faint: hsl(0 35% 92%);
--c-border-strong: hsl(0 20% 40%);
--c-text: hsl(0 30% 12%);
--c-text-secondary: hsl(0 20% 25%);
--c-text-muted: hsl(0 15% 40%);
--c-text-subtle: hsl(0 10% 50%);
--c-text-faint: hsl(0 10% 65%);
--c-text-faintest: hsl(0 12% 80%);
--c-danger-bg: #fee2e2;
--c-danger-border: #fca5a5;
--c-danger-text: #b91c1c;
--c-danger-text-strong: #991b1b;
--c-accent: hsl(180 70% 40%);
--c-accent-soft: hsl(150 60% 85%);
--c-accent-text: hsl(180 75% 28%);
--c-accent-2: hsl(90 70% 45%);
--c-accent-3: hsl(270 70% 38%);
}
+28
View File
@@ -0,0 +1,28 @@
:root {
--c-surface-0: hsl(50 40% 96%);
--c-surface-1: hsl(50 35% 92%);
--c-surface-2: hsl(50 30% 85%);
--c-border: hsl(50 25% 80%);
--c-border-soft: hsl(50 22% 70%);
--c-border-faint: hsl(50 35% 92%);
--c-border-strong: hsl(50 20% 40%);
--c-text: hsl(50 30% 12%);
--c-text-secondary: hsl(50 20% 25%);
--c-text-muted: hsl(50 15% 40%);
--c-text-subtle: hsl(50 10% 50%);
--c-text-faint: hsl(50 10% 65%);
--c-text-faintest: hsl(50 12% 80%);
--c-danger-bg: #fee2e2;
--c-danger-border: #fca5a5;
--c-danger-text: #b91c1c;
--c-danger-text-strong: #991b1b;
--c-accent: hsl(230 70% 40%);
--c-accent-soft: hsl(200 60% 85%);
--c-accent-text: hsl(230 75% 28%);
--c-accent-2: hsl(140 70% 45%);
--c-accent-3: hsl(320 70% 38%);
}
+28
View File
@@ -0,0 +1,28 @@
:root {
--c-surface-0: hsl(30 32% 10%);
--c-surface-1: hsl(30 28% 16%);
--c-surface-2: hsl(30 26% 22%);
--c-border: hsl(30 24% 24%);
--c-border-soft: hsl(30 22% 33%);
--c-border-faint: hsl(30 28% 16%);
--c-border-strong: hsl(30 15% 62%);
--c-text: hsl(30 15% 94%);
--c-text-secondary: hsl(30 12% 82%);
--c-text-muted: hsl(30 10% 65%);
--c-text-subtle: hsl(30 8% 50%);
--c-text-faint: hsl(30 8% 38%);
--c-text-faintest: hsl(30 10% 28%);
--c-danger-bg: #450a0a;
--c-danger-border: #7f1d1d;
--c-danger-text: #f87171;
--c-danger-text-strong: #fca5a5;
--c-accent: hsl(210 70% 62%);
--c-accent-soft: hsl(180 55% 34%);
--c-accent-text: hsl(210 75% 80%);
--c-accent-2: hsl(120 65% 55%);
--c-accent-3: hsl(300 65% 68%);
}
+28
View File
@@ -0,0 +1,28 @@
:root {
--c-surface-0: hsl(330 32% 10%);
--c-surface-1: hsl(330 28% 16%);
--c-surface-2: hsl(330 26% 22%);
--c-border: hsl(330 24% 24%);
--c-border-soft: hsl(330 22% 33%);
--c-border-faint: hsl(330 28% 16%);
--c-border-strong: hsl(330 15% 62%);
--c-text: hsl(330 15% 94%);
--c-text-secondary: hsl(330 12% 82%);
--c-text-muted: hsl(330 10% 65%);
--c-text-subtle: hsl(330 8% 50%);
--c-text-faint: hsl(330 8% 38%);
--c-text-faintest: hsl(330 10% 28%);
--c-danger-bg: #450a0a;
--c-danger-border: #7f1d1d;
--c-danger-text: #f87171;
--c-danger-text-strong: #fca5a5;
--c-accent: hsl(150 70% 62%);
--c-accent-soft: hsl(120 55% 34%);
--c-accent-text: hsl(150 75% 80%);
--c-accent-2: hsl(60 65% 58%);
--c-accent-3: hsl(240 65% 68%);
}
+28
View File
@@ -0,0 +1,28 @@
:root {
--c-surface-0: hsl(0 32% 10%);
--c-surface-1: hsl(0 28% 16%);
--c-surface-2: hsl(0 26% 22%);
--c-border: hsl(0 24% 24%);
--c-border-soft: hsl(0 22% 33%);
--c-border-faint: hsl(0 28% 16%);
--c-border-strong: hsl(0 15% 62%);
--c-text: hsl(0 15% 94%);
--c-text-secondary: hsl(0 12% 82%);
--c-text-muted: hsl(0 10% 65%);
--c-text-subtle: hsl(0 8% 50%);
--c-text-faint: hsl(0 8% 38%);
--c-text-faintest: hsl(0 10% 28%);
--c-danger-bg: #450a0a;
--c-danger-border: #7f1d1d;
--c-danger-text: #f87171;
--c-danger-text-strong: #fca5a5;
--c-accent: hsl(180 70% 62%);
--c-accent-soft: hsl(150 55% 34%);
--c-accent-text: hsl(180 75% 80%);
--c-accent-2: hsl(90 65% 58%);
--c-accent-3: hsl(270 65% 68%);
}
+28
View File
@@ -0,0 +1,28 @@
:root {
--c-surface-0: hsl(50 32% 10%);
--c-surface-1: hsl(50 28% 16%);
--c-surface-2: hsl(50 26% 22%);
--c-border: hsl(50 24% 24%);
--c-border-soft: hsl(50 22% 33%);
--c-border-faint: hsl(50 28% 16%);
--c-border-strong: hsl(50 15% 62%);
--c-text: hsl(50 15% 94%);
--c-text-secondary: hsl(50 12% 82%);
--c-text-muted: hsl(50 10% 65%);
--c-text-subtle: hsl(50 8% 50%);
--c-text-faint: hsl(50 8% 38%);
--c-text-faintest: hsl(50 10% 28%);
--c-danger-bg: #450a0a;
--c-danger-border: #7f1d1d;
--c-danger-text: #f87171;
--c-danger-text-strong: #fca5a5;
--c-accent: hsl(230 70% 62%);
--c-accent-soft: hsl(200 55% 34%);
--c-accent-text: hsl(230 75% 80%);
--c-accent-2: hsl(140 65% 55%);
--c-accent-3: hsl(320 65% 68%);
}
@@ -0,0 +1,72 @@
# 109 — Stop binding `ottyl mcp` to one session for its whole lifetime
## Summary
`ottyl mcp --session <name>` resolves the session name to a port
once, at process startup, and then serves stdio JSON-RPC for the
life of the process against that fixed URL. In practice this means
every ottyl session an agent wants to talk to needs its own
registered MCP server entry (e.g. `ottyl-hygienic-gyrfalcon`), and
switching which running `ottyl` instance a client is looking at
requires killing that entry and re-registering a new one — which,
for MCP clients that only read server config at startup, means the
whole host process needs restarting before the new tools are even
callable. This got hit directly: an agent needed to switch from an
old dead session (`enormous-warthog`) to a new one
(`hygienic-gyrfalcon`) mid-conversation and had no way to do it
without editing MCP config and waiting for a restart.
## Where
- `src/cli/mod.rs:43-49` — `Command::Mcp { session: Option<String>,
port: Option<u16> }`, the CLI surface.
- `src/cli/mod.rs:171-195` (`run_mcp`) — resolves `session` to a port
via `sessions::resolve_session` exactly once, builds one
`OttylMcp::new(base_url)`, then blocks forever on
`service.waiting()`. There is no re-resolution after startup.
- `src/api/mcp_server.rs:254-282` — `OttylMcp` stores `base_url:
String` as a plain field set once in `new()`; every `#[tool]`
method builds request URLs via `self.url(path)`
(`mcp_server.rs:282-284`), so the fixed base URL is threaded
through every single tool call.
- `src/cli/sessions.rs` — `SessionInfo { name, port, pid }` and
`resolve_session`/`list_sessions`, the registry this would need to
re-query if the bound session can change after startup.
## Direction
Two ways to remove the "pick a session forever at startup" behavior;
pick one during implementation rather than building both:
1. **Request/response CLI, no long-running process.** Add a CLI
subcommand (e.g. `ottyl call <session> <tool> <json-args>`) that
resolves the session, makes the one HTTP call, prints the result,
and exits — no `rmcp`/stdio JSON-RPC server at all. Simplest fix
for the "MCP config never changes, session does" mismatch, but
loses whatever the MCP client gives you over a bare CLI call
(tool discovery/schema, a persistent connection) — check whether
that matters to how this is actually invoked before going this
route.
2. **Long-running MCP server, session switchable at runtime.** Keep
`ottyl mcp` as a stdio server, but stop fixing `base_url` at
construction: either (a) accept a `session` (or `port`) argument
on every tool call, resolved against `sessions::resolve_session`
per-call instead of once in `run_mcp`, with the constructor's
`--session`/`--port` becoming just a default; or (b) add an
explicit `switch_session` tool that mutates the stored target
(`base_url` would need to move from a plain field to something
mutable, e.g. `Mutex<String>` or resolved fresh from the registry
on every call from a stored session name instead of a
pre-resolved URL).
## Scope
- Land one of the two directions above for `ottyl mcp` /
`src/api/mcp_server.rs`; out of scope to build both.
- If going with direction 2, update `SKILL.md`'s "Connect an MCP
client" section, since it currently documents the
resolve-once-at-startup model.
- Out of scope: changing anything about the underlying `/api/...`
HTTP routes themselves, or the session registry format in
`src/cli/sessions.rs` — this is purely about how the MCP-facing
process picks and holds (or doesn't hold) its target session.
@@ -0,0 +1,72 @@
# 110 — Clicking a legend entry should focus that series, fading the rest
## Summary
`PanelChart`'s line and pie charts render a recharts `<Legend>`
(`webapp/src/components/PanelChart.tsx:134`, `:190`) but it's purely
decorative — clicking an entry does nothing. Bar charts have no
`<Legend>` at all (`:151-168`), since each bar is its own labeled
category on the x-axis rather than a separate series.
Add click-to-focus: clicking a legend entry marks that series
"focused"; any focused series stay at full opacity/stroke while every
other series fades. Clicking the same entry again (or clicking empty
space in the legend, per whatever recharts exposes) clears focus and
returns everyone to equal opacity — i.e. "no focused series" means
"all at the same opacity", not "all faded".
## Where
- `webapp/src/components/PanelChart.tsx:116-148` — line chart. Series
are recharts `<Line>` elements keyed by `keys` (from `toLineData`,
`:60-78`), colored via `COLORS[i % COLORS.length]` (`:140`). Fading
a line means varying `strokeOpacity` (or swapping `stroke` for a
`color-mix`'d faded variant) per key based on focus state.
- `webapp/src/components/PanelChart.tsx:170-193` — pie chart. Slices
are `<Cell>` elements (`:179-181`) keyed by row index into `rows`
(from `squashRows`, `:50-57`); fading a slice means varying its
`fillOpacity`.
- `webapp/src/components/PanelChart.tsx:134`, `:190` — the two
`<Legend>` usages. recharts' `<Legend>` takes an `onClick` handler
on its payload entries (`(entry) => void`, where `entry.dataKey` /
`entry.value` identifies which series was clicked) — this is the
hook point rather than building a custom legend from scratch.
- Bar charts (`:151-168`) have no legend to click, since bars aren't
named series here — out of scope for this ticket unless the
direction chosen below turns out to need bar-chart parity (see
Scope).
## Direction
- Add focus state as a `Set<string>` (or single nullable string, if
only one series can be focused at a time — decide during
implementation) of series keys, local to `PanelChart` via
`useState`. Reset it when `series` changes identity to avoid stale
focus surviving a query change onto a chart with different series.
- Wire the `<Legend>`'s `onClick` to toggle membership in that set.
- Compute per-series opacity from the set: full opacity if the set is
empty or the series is a member, faded (e.g. `0.25`) otherwise.
Thread that into `<Line stroke=...>` / `<Cell fill=...>` via
`strokeOpacity`/`fillOpacity` rather than swapping the base color,
so the existing `COLORS` palette and its `color-mix` theming stay
untouched.
- The legend entries themselves should visually reflect focus too
(e.g. dim the faded entries' swatch/text) so it's clear what's
focused without hunting across the plot — recharts' legend
`formatter` prop can restyle each entry's label.
## Scope
- Line and pie charts only, matching where `<Legend>` already
renders. Bar charts stay as-is — no legend, no per-bar fade — unless
investigation shows recharts' bar `<Legend>` support is trivial to
add alongside this; if so, note it as a follow-up rather than
folding scope creep into this ticket.
- Out of scope: persisting focus state across re-renders driven by
polling (`Dashboards.tsx`'s `PanelPreview`, `:46-92`, re-fetches on
every poll tick) — focus should survive a data refresh as long as
the series identity (legend key) is unchanged, but there's no need
to persist it to `localStorage` or across a page reload.
- Out of scope: multi-select "compare exactly these two series" UX
beyond simple toggle-membership — keep it to click-to-add/click-to-
remove from the focused set.
+51
View File
@@ -0,0 +1,51 @@
# 111 — Legend must scroll instead of overflowing the panel
## Summary
`PanelChart`'s recharts `<Legend>` (`webapp/src/components/
PanelChart.tsx:134`, `:190`) has no height constraint or scroll
container. A panel with many series (e.g. a high-cardinality PromQL
query, or the post-`squashRows` `MAX_KEPT_SERIES` = 7 kept rows plus
"Other") wraps the legend across as many lines as it needs, growing
the panel's rendered height and pushing surrounding dashboard layout
around rather than staying within the chart's fixed footprint.
## Where
- `webapp/src/components/PanelChart.tsx:117`, `:154`, `:176` — each
chart type wraps its `<ResponsiveContainer>` at a fixed `height={220}`,
but the `<Legend>` inside it (`:134`, `:190`) isn't given its own
bounded, scrollable region — recharts lays it out in the space left
after the plot area, which can push the container's effective
rendered height past 220 when the legend wraps to multiple rows.
- `webapp/src/components/Dashboards.tsx:46-92` (`PanelPreview`) is the
only caller of `PanelChart` today — worth checking that whatever
fixed-height/scroll wrapper gets added here doesn't fight with
`PanelPreview`'s own loading/error states (`:85-90`), which render
outside `PanelChart` at a different fixed height (`h-48`).
## Direction
- recharts' `<Legend>` supports a `wrapperStyle` prop (already used
for `fontSize`, `:134`/`:190`) — extending it with a fixed
`maxHeight` and `overflowY: 'auto'` is the natural fix and avoids
building a custom legend renderer just for this.
- Verify this doesn't conflict with the click-to-focus work in
[[110-legend-click-to-focus-series]] — that ticket also touches the
`<Legend>` props (`onClick`, `formatter`) on these same two call
sites, so land whichever lands second by rebasing onto the other's
`<Legend>` props rather than both independently restructuring it.
- Decide a concrete max height (e.g. two or three legend rows' worth)
that keeps the panel's total footprint predictable in a dashboard
grid; check `Dashboards.tsx`'s panel grid/card layout for whatever
existing panel height convention should inform the number.
## Scope
- Line and pie charts only, matching where `<Legend>` already
renders — same boundary as [[110-legend-click-to-focus-series]].
- Out of scope: virtualizing the legend list for very large series
counts — `squashRows` (`PanelChart.tsx:50-57`) already caps bar/pie
series at `MAX_KEPT_SERIES` (7) + "Other", and line charts have no
equivalent cap today, but adding one is a separate concern from
making the legend scrollable.
+80
View File
@@ -0,0 +1,80 @@
# 112 — Let a single dashboard panel pop out into its own window
## Summary
There's already a pop-out mechanism for whole primary sections (log,
params, metrics, dashboards) — `PopOutButton` opens a new tab at
`?section=<id>` and `App.tsx` reads that query param at startup to
render a standalone, full-page version of just that section (see
`webapp/src/components/CollapsibleSection.tsx:193-232` and
`webapp/src/App.tsx:129-140`, `:372-376`). That grain is a whole
section, not a single panel — the Dashboards section can only be
popped out in its entirety (all panels of the active dashboard, or
all dashboards stacked via `AllDashboardsView`,
`Dashboards.tsx:107-135`). This ticket adds the finer-grained version:
popping out one panel from one dashboard into its own window,
independent of the rest of that dashboard.
## Where
- `webapp/src/components/Dashboards.tsx:46-92` (`PanelPreview`) is
the component that would need to render standalone, full-page, the
same way `StandaloneLog`/`StandaloneParams`/`StandaloneRanged` do
today for whole sections (referenced in `App.tsx:373-376`, exact
component locations elsewhere in `Dashboards.tsx`/`Metrics.tsx` per
the skeleton read during this ticket's writeup — re-read
`Dashboards.tsx` in full before implementing, since it was read via
repowise's skeleton view here and several ranges (`:1-45`, `:136-
434`) were elided).
- `webapp/src/components/CollapsibleSection.tsx:193-206`
(`PopOutSectionId`, `popOutUrl`) — the existing section-grain
pop-out routing. A panel-grain pop-out needs to identify a specific
`(dashboardId, panelId)` pair in the URL, not just a section name —
either extend this query-param scheme with `dashboardId`/`panelId`
params alongside `section=dashboards`, or give panels their own
distinct query shape if reusing `PopOutSectionId` doesn't fit
cleanly.
- `webapp/src/App.tsx:129-140` (`parsePopOutQuery`), `:372-376` (the
dispatch on parsed `section`) — where the new panel-grain case needs
a branch, reading whatever new params are added above.
- Each `Panel` (`webapp/src/lib/api.ts`'s `Panel` type, imported at
`Dashboards.tsx:19`) needs a stable identifier already, or one to
add, so a popped-out URL can point back at "this exact panel of
this exact dashboard" — check whether `Panel` already carries an
`id` field before inventing a new addressing scheme.
## Direction
- Mirror the existing section pop-out exactly: a small pop-out button
per panel (using or extending `PopOutButton`,
`CollapsibleSection.tsx:209-232`) that opens `?section=dashboards&
dashboardId=<id>&panelId=<id>`, plus range params the same way
`popOutUrl` already carries over a rolling preset or fixed range
(`CollapsibleSection.tsx:196-206`).
- The standalone page for this case renders just that one panel's
`PanelPreview` (or a thin wrapper around it) full-page — no
dashboard chrome, no other panels — analogous to how
`StandaloneRanged` (`App.tsx:179-...`) drives `LogTimeline`'s
picker UI on its own poll cadence for the metrics/dashboards
standalone views.
- The popped-out panel window should keep polling independently
(same `POLL_MS` pattern `PanelPreview` already uses,
`Dashboards.tsx:52-55`) rather than freezing at open time.
## Scope
- One panel popping into a new browser tab/window, matching the
existing section pop-out's mechanism (new tab via `window.open`,
not an in-app floating/detached window widget) — stay consistent
with the pattern already established for log/params/metrics/
dashboards rather than introducing a second, different pop-out
mechanism.
- Out of scope: editing a panel's query/chart type from the popped-out
view — read-only preview only, matching what pop-out gives you for
the other sections today.
- Out of scope: [[110-legend-click-to-focus-series]] and
[[111-scrollable-panel-legend]] — this ticket is purely about the
window/routing mechanism; the popped-out panel should get those
legend behaviors for free once they land in `PanelChart`, since
it's the same component, but implementing them isn't this ticket's
job.
+17 -100
View File
@@ -3,15 +3,12 @@ import { NavMenu } from './components/NavMenu'
import { FilterBar } from './components/FilterBar' import { FilterBar } from './components/FilterBar'
import { import {
LogTimeline, LogTimeline,
TIME_RANGE_PRESETS,
computeBucketMs, computeBucketMs,
computePresetRange,
loadLogSortSettings, loadLogSortSettings,
saveLogSortSettings, saveLogSortSettings,
type LogSortSettings, type LogSortSettings,
type TimeRange,
type TimeRangePresetId,
} from './components/LogTimeline' } from './components/LogTimeline'
import { TIME_RANGE_PRESETS, type TimeRange, type TimeRangePresetId } from './lib/timeRange'
import { LogPreview } from './components/LogPreview' import { LogPreview } from './components/LogPreview'
import { Params } from './components/Params' import { Params } from './components/Params'
import { Metrics } from './components/Metrics' import { Metrics } from './components/Metrics'
@@ -29,6 +26,7 @@ import {
type TimelineBucket, type TimelineBucket,
} from './lib/api' } from './lib/api'
import { DocsNavProvider } from './lib/docsNav' import { DocsNavProvider } from './lib/docsNav'
import { useRangeSelection } from './lib/useRangeSelection'
import { useSettingsPanel } from './lib/useSettingsPanel' import { useSettingsPanel } from './lib/useSettingsPanel'
import { POLL_MS } from './lib/polling' import { POLL_MS } from './lib/polling'
@@ -189,47 +187,18 @@ function StandaloneRanged({
initialFixedRange: TimeRange | null initialFixedRange: TimeRange | null
}) { }) {
const [buckets, setBuckets] = useState<TimelineBucket[]>([]) const [buckets, setBuckets] = useState<TimelineBucket[]>([])
const [range, setRange] = useState<TimeRange | null>(
initialPresetId ? computePresetRange(initialPresetId) : initialFixedRange,
)
// No preset and no fixed range means the popout was opened with nothing carried over (e.g. a bare URL) — default to 'all', matching FullApp's own default. // No preset and no fixed range means the popout was opened with nothing carried over (e.g. a bare URL) — default to 'all', matching FullApp's own default.
const [activePreset, setActivePreset] = useState<TimeRangePresetId | null>( const { range, activePreset, selectPreset, handleManualRangeSelect, resolveEffectiveRange } = useRangeSelection({
initialPresetId ?? (initialFixedRange ? null : 'all'), preset: initialPresetId ?? (initialFixedRange ? null : 'all'),
) range: initialFixedRange,
})
const rangeRef = useRef(range)
useEffect(() => {
rangeRef.current = range
}, [range])
function selectPreset(id: TimeRangePresetId) {
setActivePreset(id)
setRange(computePresetRange(id))
}
function handleManualRangeSelect(next: TimeRange | null) {
if (next === null) {
selectPreset('all')
} else {
setActivePreset(null)
setRange(next)
}
}
useEffect(() => { useEffect(() => {
let cancelled = false let cancelled = false
async function poll() { async function poll() {
// Rolling presets re-resolve on this same polling cadence instead of freezing the window at selection time — this is what makes a carried-over preset independently slide rather than track the source tab. // Rolling presets re-resolve on this same polling cadence instead of freezing the window at selection time — this is what makes a carried-over preset independently slide rather than track the source tab.
const effectiveRange = const effectiveRange = resolveEffectiveRange()
activePreset && activePreset !== 'all' ? computePresetRange(activePreset) : rangeRef.current
if (activePreset && activePreset !== 'all') {
setRange((current) =>
current && effectiveRange && current.startMs === effectiveRange.startMs && current.endMs === effectiveRange.endMs
? current
: effectiveRange,
)
}
try { try {
const nextBuckets = await fetchTimeline({ const nextBuckets = await fetchTimeline({
startMs: effectiveRange?.startMs, startMs: effectiveRange?.startMs,
@@ -298,8 +267,10 @@ function StandaloneLog() {
const [logs, setLogs] = useState<LogEntry[]>([]) const [logs, setLogs] = useState<LogEntry[]>([])
const [params, setParams] = useState<ParamDeclaration[]>([]) const [params, setParams] = useState<ParamDeclaration[]>([])
const [buckets, setBuckets] = useState<TimelineBucket[]>([]) const [buckets, setBuckets] = useState<TimelineBucket[]>([])
const [range, setRange] = useState<TimeRange | null>(null) const { range, activePreset, selectPreset, handleManualRangeSelect, resolveEffectiveRange } = useRangeSelection({
const [activePreset, setActivePreset] = useState<TimeRangePresetId | null>('all') preset: 'all',
range: null,
})
const [filter, setFilter] = useState('') const [filter, setFilter] = useState('')
const [filterError, setFilterError] = useState<string | null>(null) const [filterError, setFilterError] = useState<string | null>(null)
const [sortSettings, setSortSettings] = useState<LogSortSettings>(loadLogSortSettings) const [sortSettings, setSortSettings] = useState<LogSortSettings>(loadLogSortSettings)
@@ -310,11 +281,6 @@ function StandaloneLog() {
saveLogSortSettings(next) saveLogSortSettings(next)
} }
const rangeRef = useRef(range)
useEffect(() => {
rangeRef.current = range
}, [range])
useEffect(() => { useEffect(() => {
fetchParamDeclarations() fetchParamDeclarations()
.then(setParams) .then(setParams)
@@ -323,33 +289,11 @@ function StandaloneLog() {
}) })
}, []) }, [])
function selectPreset(id: TimeRangePresetId) {
setActivePreset(id)
setRange(computePresetRange(id))
}
function handleManualRangeSelect(next: TimeRange | null) {
if (next === null) {
selectPreset('all')
} else {
setActivePreset(null)
setRange(next)
}
}
useEffect(() => { useEffect(() => {
let cancelled = false let cancelled = false
async function poll() { async function poll() {
const effectiveRange = const effectiveRange = resolveEffectiveRange()
activePreset && activePreset !== 'all' ? computePresetRange(activePreset) : rangeRef.current
if (activePreset && activePreset !== 'all') {
setRange((current) =>
current && effectiveRange && current.startMs === effectiveRange.startMs && current.endMs === effectiveRange.endMs
? current
: effectiveRange,
)
}
try { try {
const [nextBuckets, nextLogs, nextParams] = await Promise.all([ const [nextBuckets, nextLogs, nextParams] = await Promise.all([
fetchTimeline({ fetchTimeline({
@@ -439,8 +383,10 @@ function FullApp() {
const [logs, setLogs] = useState<LogEntry[]>([]) const [logs, setLogs] = useState<LogEntry[]>([])
const [params, setParams] = useState<ParamDeclaration[]>([]) const [params, setParams] = useState<ParamDeclaration[]>([])
const [buckets, setBuckets] = useState<TimelineBucket[]>([]) const [buckets, setBuckets] = useState<TimelineBucket[]>([])
const [range, setRange] = useState<TimeRange | null>(null) const { range, activePreset, selectPreset, handleManualRangeSelect, resolveEffectiveRange } = useRangeSelection({
const [activePreset, setActivePreset] = useState<TimeRangePresetId | null>('all') preset: 'all',
range: null,
})
const [filter, setFilter] = useState('') const [filter, setFilter] = useState('')
const [filterError, setFilterError] = useState<string | null>(null) const [filterError, setFilterError] = useState<string | null>(null)
const [logViewerCollapsed, setLogViewerCollapsed] = useSectionCollapsed('log-viewer') const [logViewerCollapsed, setLogViewerCollapsed] = useSectionCollapsed('log-viewer')
@@ -464,12 +410,6 @@ function FullApp() {
}) })
} }
// Read via ref inside the poll loop instead of depending on `range` directly, which would restart the interval every rolling-preset tick.
const rangeRef = useRef(range)
useEffect(() => {
rangeRef.current = range
}, [range])
async function loadParams() { async function loadParams() {
try { try {
setParams(await fetchParamDeclarations()) setParams(await fetchParamDeclarations())
@@ -482,35 +422,12 @@ function FullApp() {
loadParams() loadParams()
}, []) }, [])
function selectPreset(id: TimeRangePresetId) {
setActivePreset(id)
setRange(computePresetRange(id))
}
// A drag always yields a fixed (non-rolling) range, overriding any active preset.
function handleManualRangeSelect(next: TimeRange | null) {
if (next === null) {
selectPreset('all')
} else {
setActivePreset(null)
setRange(next)
}
}
useEffect(() => { useEffect(() => {
let cancelled = false let cancelled = false
async function poll() { async function poll() {
// Rolling presets re-resolve on this same polling cadence instead of freezing the window at selection time. // Rolling presets re-resolve on this same polling cadence instead of freezing the window at selection time.
const effectiveRange = const effectiveRange = resolveEffectiveRange()
activePreset && activePreset !== 'all' ? computePresetRange(activePreset) : rangeRef.current
if (activePreset && activePreset !== 'all') {
setRange((current) =>
current && effectiveRange && current.startMs === effectiveRange.startMs && current.endMs === effectiveRange.endMs
? current
: effectiveRange,
)
}
try { try {
const [nextStats, nextBuckets, nextLogs, nextParams] = await Promise.all([ const [nextStats, nextBuckets, nextLogs, nextParams] = await Promise.all([
fetchStats(), fetchStats(),
+9 -37
View File
@@ -9,13 +9,7 @@ import {
type AlertRule, type AlertRule,
type CompareOp, type CompareOp,
} from '../lib/api' } from '../lib/api'
import { import { FormFooter, SectionHeader, useSectionCollapsed } from './CollapsibleSection'
CollapseToggle,
handleSectionHeaderKeyDown,
sectionHeaderInteractiveClassName,
sectionHeaderLeftClassName,
useSectionCollapsed,
} from './CollapsibleSection'
import { HelpIcon } from './HelpIcon' import { HelpIcon } from './HelpIcon'
import { RegexHelpIcon } from './RegexHelpIcon' import { RegexHelpIcon } from './RegexHelpIcon'
import { queryInputClassName } from './FilterBar' import { queryInputClassName } from './FilterBar'
@@ -238,22 +232,7 @@ export function Alerts() {
return ( return (
<div className="border-b border-border text-sm"> <div className="border-b border-border text-sm">
<div <SectionHeader title="Alerts" helpSectionId="alerts" collapsed={collapsed} onToggle={() => setCollapsed(!collapsed)} />
className={sectionHeaderInteractiveClassName}
role="button"
tabIndex={0}
aria-expanded={!collapsed}
onClick={() => setCollapsed(!collapsed)}
onKeyDown={handleSectionHeaderKeyDown(() => setCollapsed(!collapsed))}
>
<div className={sectionHeaderLeftClassName}>
Alerts
<HelpIcon sectionId="alerts" label="Help: Alerts" />
</div>
<div className="flex items-center gap-1.5">
<CollapseToggle collapsed={collapsed} />
</div>
</div>
<div hidden={collapsed}> <div hidden={collapsed}>
<p className="px-3 pt-3 text-xs text-text-faint"> <p className="px-3 pt-3 text-xs text-text-faint">
Every rule delivers to the ntfy sink configured in Settings. Every rule delivers to the ntfy sink configured in Settings.
@@ -420,20 +399,13 @@ export function Alerts() {
{submitError && <div className="rounded border border-danger-border bg-danger-bg/50 px-2 py-1 text-xs text-danger-text">{submitError}</div>} {submitError && <div className="rounded border border-danger-border bg-danger-bg/50 px-2 py-1 text-xs text-danger-text">{submitError}</div>}
<div className="flex gap-2"> <FormFooter
<button editing={editingId !== null}
type="submit" submitting={submitting}
disabled={submitting} idleLabel="Create rule"
className="rounded border border-border-soft px-3 py-1 text-sm text-text-secondary hover:bg-surface-2 disabled:opacity-50" editingLabel="Save rule"
> onCancel={startCreate}
{submitting ? 'Saving…' : editingId ? 'Save rule' : 'Create rule'} />
</button>
{editingId && (
<button type="button" onClick={startCreate} className="rounded border border-border-soft px-3 py-1 text-sm text-text-secondary hover:bg-surface-2">
Cancel
</button>
)}
</div>
</form> </form>
<div className="min-w-0 flex-1"> <div className="min-w-0 flex-1">
+96 -1
View File
@@ -1,5 +1,6 @@
import { useState } from 'react' import { useState } from 'react'
import type { TimeRange, TimeRangePresetId } from './LogTimeline' import type { TimeRange, TimeRangePresetId } from '../lib/timeRange'
import { HelpIcon } from './HelpIcon'
/// Shared look for a primary section's title bar: shaded so it reads as a /// Shared look for a primary section's title bar: shaded so it reads as a
/// distinct band rather than blending into the section body below it. /// distinct band rather than blending into the section body below it.
@@ -95,6 +96,100 @@ export function handleSectionHeaderKeyDown(onToggle: () => void) {
} }
} }
/// The header row every settings panel (Params/Metrics/Alerts/Dashboards) opens with: a click/keyboard-toggleable
/// title bar pairing a title + `HelpIcon` on the left with `CollapseToggle` (and, for panels with a standalone
/// popped-out view, a `PopOutButton`) on the right. Assembles `sectionHeaderInteractiveClassName` /
/// `handleSectionHeaderKeyDown` / `sectionHeaderLeftClassName` so callers don't each re-wire the same
/// `role="button"`/`tabIndex`/`aria-expanded` boilerplate by hand.
export function SectionHeader({
title,
helpSectionId,
helpLabel,
collapsed,
onToggle,
popOut,
}: {
title: string
helpSectionId: string
helpLabel?: string
collapsed: boolean
onToggle: () => void
popOut?: React.ReactNode
}) {
return (
<div
className={sectionHeaderInteractiveClassName}
role="button"
tabIndex={0}
aria-expanded={!collapsed}
onClick={onToggle}
onKeyDown={handleSectionHeaderKeyDown(onToggle)}
>
<div className={sectionHeaderLeftClassName}>
{title}
<HelpIcon sectionId={helpSectionId} label={helpLabel ?? `Help: ${title}`} />
</div>
<div className="flex items-center gap-1.5">
<CollapseToggle collapsed={collapsed} />
{popOut}
</div>
</div>
)
}
/// The declare/edit form footer shared by `Params.tsx`/`Metrics.tsx`/`Alerts.tsx`: a submit button whose label
/// swaps between create/edit and idle/in-flight variants, plus a Cancel button that only appears while editing.
/// Doesn't render the "Editing X" banner some callers show above this — that banner's placement relative to
/// `submitError` differs per caller, so it stays theirs to render; this covers only the part that's identical
/// everywhere. Label text stays a per-caller prop (e.g. "Declare param" vs "Declare metric") rather than
/// hardcoded here, since that's the one part of this footer each panel's copy actually differs on.
export function FormFooter({
editing,
submitting,
idleLabel,
editingLabel,
submittingLabel = 'Saving…',
editingSubmittingLabel,
onCancel,
}: {
editing: boolean
submitting: boolean
idleLabel: string
editingLabel: string
submittingLabel?: string
editingSubmittingLabel?: string
onCancel: () => void
}) {
const label = submitting
? editing
? (editingSubmittingLabel ?? submittingLabel)
: submittingLabel
: editing
? editingLabel
: idleLabel
return (
<div className="flex items-center gap-2">
<button
type="submit"
disabled={submitting}
className="rounded border border-border-soft px-3 py-1 text-sm text-text-secondary hover:bg-surface-2 disabled:opacity-50"
>
{label}
</button>
{editing && (
<button
type="button"
onClick={onCancel}
disabled={submitting}
className="rounded border border-border-soft px-3 py-1 text-sm text-text-secondary hover:bg-surface-2 disabled:opacity-50"
>
Cancel
</button>
)}
</div>
)
}
/// The four primary sections poppable into their own tab; matches the `?section=` query param a standalone page reads. /// The four primary sections poppable into their own tab; matches the `?section=` query param a standalone page reads.
export type PopOutSectionId = 'log' | 'params' | 'metrics' | 'dashboards' export type PopOutSectionId = 'log' | 'params' | 'metrics' | 'dashboards'
+16 -44
View File
@@ -1,5 +1,5 @@
import { useEffect, useRef, useState } from 'react' import { useEffect, useRef, useState } from 'react'
import type { TimeRange, TimeRangePresetId } from './LogTimeline' import type { TimeRange, TimeRangePresetId } from '../lib/timeRange'
import { PanelChart } from './PanelChart' import { PanelChart } from './PanelChart'
import { import {
ApiError, ApiError,
@@ -15,15 +15,7 @@ import {
type PromqlSeries, type PromqlSeries,
} from '../lib/api' } from '../lib/api'
import { POLL_MS } from '../lib/polling' import { POLL_MS } from '../lib/polling'
import { import { PopOutButton, SectionHeader, useSectionCollapsed } from './CollapsibleSection'
CollapseToggle,
handleSectionHeaderKeyDown,
PopOutButton,
sectionHeaderInteractiveClassName,
sectionHeaderLeftClassName,
useSectionCollapsed,
} from './CollapsibleSection'
import { HelpIcon } from './HelpIcon'
const CHART_TYPES: ChartType[] = ['line', 'bar', 'pie'] const CHART_TYPES: ChartType[] = ['line', 'bar', 'pie']
@@ -282,23 +274,13 @@ export function Dashboards({ range, activePreset }: { range: TimeRange | null; a
if (!activeProfile) { if (!activeProfile) {
return ( return (
<div className="border-b border-border text-sm"> <div className="border-b border-border text-sm">
<div <SectionHeader
className={sectionHeaderInteractiveClassName} title="Dashboards"
role="button" helpSectionId="dashboards"
tabIndex={0} collapsed={collapsed}
aria-expanded={!collapsed} onToggle={() => setCollapsed(!collapsed)}
onClick={() => setCollapsed(!collapsed)} popOut={<PopOutButton section="dashboards" range={range} activePreset={activePreset} />}
onKeyDown={handleSectionHeaderKeyDown(() => setCollapsed(!collapsed))} />
>
<div className={sectionHeaderLeftClassName}>
Dashboards
<HelpIcon sectionId="dashboards" label="Help: Dashboards" />
</div>
<div className="flex items-center gap-1.5">
<CollapseToggle collapsed={collapsed} />
<PopOutButton section="dashboards" range={range} activePreset={activePreset} />
</div>
</div>
<div className="p-3 text-xs text-text-faint"> <div className="p-3 text-xs text-text-faint">
Activate a profile (see the settings menu) to create and save dashboards. Activate a profile (see the settings menu) to create and save dashboards.
</div> </div>
@@ -308,23 +290,13 @@ export function Dashboards({ range, activePreset }: { range: TimeRange | null; a
return ( return (
<div className="border-b border-border text-sm"> <div className="border-b border-border text-sm">
<div <SectionHeader
className={sectionHeaderInteractiveClassName} title="Dashboards"
role="button" helpSectionId="dashboards"
tabIndex={0} collapsed={collapsed}
aria-expanded={!collapsed} onToggle={() => setCollapsed(!collapsed)}
onClick={() => setCollapsed(!collapsed)} popOut={<PopOutButton section="dashboards" range={range} activePreset={activePreset} />}
onKeyDown={handleSectionHeaderKeyDown(() => setCollapsed(!collapsed))} />
>
<div className={sectionHeaderLeftClassName}>
Dashboards
<HelpIcon sectionId="dashboards" label="Help: Dashboards" />
</div>
<div className="flex items-center gap-1.5">
<CollapseToggle collapsed={collapsed} />
<PopOutButton section="dashboards" range={range} activePreset={activePreset} />
</div>
</div>
{error && !collapsed && ( {error && !collapsed && (
<div className="mx-3 mt-3 rounded border border-danger-border bg-danger-bg/50 px-2 py-1 text-xs text-danger-text">{error}</div> <div className="mx-3 mt-3 rounded border border-danger-border bg-danger-bg/50 px-2 py-1 text-xs text-danger-text">{error}</div>
)} )}
+9
View File
@@ -381,6 +381,15 @@ const SECTIONS: DocSection[] = [
— the most-recent value per label combination in the active range, i.e. the counter collapsed to its — the most-recent value per label combination in the active range, i.e. the counter collapsed to its
latest reading. latest reading.
</li> </li>
<li>
<span className="font-semibold text-text">
<Code>histogram_quantile(p, metric_name)</Code>
</span>{' '}
— the <Code>p</Code>th quantile (<Code>0</Code>–<Code>1</Code>, e.g. <Code>0.95</Code> for p95) of each
label combination's retained values, interpolated over the raw samples rather than pre-aggregated
buckets. Add a bracketed range to narrow to a trailing window instead of the whole active range, e.g.{' '}
<Code>histogram_quantile(0.95, durations[5m])</Code>.
</li>
</ul> </ul>
<p className="font-semibold text-text">Worked example</p> <p className="font-semibold text-text">Worked example</p>
<p> <p>
+3 -9
View File
@@ -1,5 +1,6 @@
import { useEffect, useRef, useState } from 'react' import { useEffect, useRef, useState } from 'react'
import { metricPrimitiveToDisplayString, type LogEntry, type ParamDeclaration } from '../lib/api' import { metricPrimitiveToDisplayString, type LogEntry, type ParamDeclaration } from '../lib/api'
import { useClickOutside } from '../lib/useClickOutside'
import { sectionHeaderClassName, sectionHeaderLeftClassName } from './CollapsibleSection' import { sectionHeaderClassName, sectionHeaderLeftClassName } from './CollapsibleSection'
import { HelpIcon } from './HelpIcon' import { HelpIcon } from './HelpIcon'
@@ -37,14 +38,7 @@ function ColumnPicker({
const [open, setOpen] = useState(false) const [open, setOpen] = useState(false)
const ref = useRef<HTMLDivElement>(null) const ref = useRef<HTMLDivElement>(null)
useEffect(() => { useClickOutside(ref, () => setOpen(false), open)
if (!open) return
function onClickOutside(e: MouseEvent) {
if (ref.current && !ref.current.contains(e.target as Node)) setOpen(false)
}
document.addEventListener('mousedown', onClickOutside)
return () => document.removeEventListener('mousedown', onClickOutside)
}, [open])
return ( return (
<div ref={ref} className="relative"> <div ref={ref} className="relative">
@@ -119,7 +113,7 @@ function JsonNode({
entries: ReadonlyArray<readonly [string, unknown]> entries: ReadonlyArray<readonly [string, unknown]>
bracket: [string, string] bracket: [string, string]
}) { }) {
const [expanded, setExpanded] = useState(true) const [expanded, setExpanded] = useState(false)
if (entries.length === 0) { if (entries.length === 0) {
return ( return (
+1 -26
View File
@@ -1,34 +1,9 @@
import { useRef, useState } from 'react' import { useRef, useState } from 'react'
import type { TimelineBucket } from '../lib/api' import type { TimelineBucket } from '../lib/api'
import { TIME_RANGE_PRESETS, type TimeRange, type TimeRangePresetId } from '../lib/timeRange'
import { sectionHeaderClassName, sectionHeaderLeftClassName } from './CollapsibleSection' import { sectionHeaderClassName, sectionHeaderLeftClassName } from './CollapsibleSection'
import { HelpIcon } from './HelpIcon' import { HelpIcon } from './HelpIcon'
export interface TimeRange {
startMs: number
endMs: number
}
// `'all'` clears the range (the implicit default); the rest resolve to a window ending "now" at the moment they're (re-)computed — see `computePresetRange`.
export type TimeRangePresetId = 'all' | '1m' | '2m' | '5m' | '15m' | '1h' | '24h'
export const TIME_RANGE_PRESETS: { id: TimeRangePresetId; label: string; ms: number | null }[] = [
{ id: 'all', label: 'All', ms: null },
{ id: '1m', label: '1m', ms: 60 * 1000 },
{ id: '2m', label: '2m', ms: 2 * 60 * 1000 },
{ id: '5m', label: '5m', ms: 5 * 60 * 1000 },
{ id: '15m', label: '15m', ms: 15 * 60 * 1000 },
{ id: '1h', label: '1h', ms: 60 * 60 * 1000 },
{ id: '24h', label: '24h', ms: 24 * 60 * 60 * 1000 },
]
// Callers re-invoke this on their existing poll cadence (see `App.tsx`) to keep a rolling preset's window moving with time, rather than freezing it at selection time.
export function computePresetRange(id: TimeRangePresetId): TimeRange | null {
const preset = TIME_RANGE_PRESETS.find((p) => p.id === id)
if (!preset || preset.ms == null) return null
const now = Date.now()
return { startMs: now - preset.ms, endMs: now }
}
/// Buckets a fixed range into this many slots, so e.g. a rolling `5m` preset always requests a resolution fine enough to visibly slide instead of a handful of coarse, jumpy bars. /// Buckets a fixed range into this many slots, so e.g. a rolling `5m` preset always requests a resolution fine enough to visibly slide instead of a handful of coarse, jumpy bars.
const TIMELINE_TARGET_BUCKETS = 60 const TIMELINE_TARGET_BUCKETS = 60
+43 -46
View File
@@ -1,5 +1,5 @@
import { useEffect, useState } from 'react' import { useEffect, useState } from 'react'
import type { TimeRange, TimeRangePresetId } from './LogTimeline' import type { TimeRange, TimeRangePresetId } from '../lib/timeRange'
import { import {
ApiError, ApiError,
declareMetric, declareMetric,
@@ -11,17 +11,10 @@ import {
updateMetric, updateMetric,
type BucketValue, type BucketValue,
type MetricDeclaration, type MetricDeclaration,
type MetricSemantics,
type ParamDeclaration, type ParamDeclaration,
} from '../lib/api' } from '../lib/api'
import { import { FormFooter, PopOutButton, SectionHeader, useSectionCollapsed } from './CollapsibleSection'
CollapseToggle,
handleSectionHeaderKeyDown,
PopOutButton,
sectionHeaderInteractiveClassName,
sectionHeaderLeftClassName,
useSectionCollapsed,
} from './CollapsibleSection'
import { HelpIcon } from './HelpIcon'
import { RegexHelpIcon } from './RegexHelpIcon' import { RegexHelpIcon } from './RegexHelpIcon'
// One row in the labels -> occurrence count table; `values` keeps the raw points for the sparkline rendering. // One row in the labels -> occurrence count table; `values` keeps the raw points for the sparkline rendering.
@@ -90,6 +83,7 @@ export function Metrics({ range, activePreset }: { range: TimeRange | null; acti
// (ticket 037) rather than re-entering a regex/capture-group/kind per param. // (ticket 037) rather than re-entering a regex/capture-group/kind per param.
const [labelParams, setLabelParams] = useState<string[]>([]) const [labelParams, setLabelParams] = useState<string[]>([])
const [valueParam, setValueParam] = useState<string>('') const [valueParam, setValueParam] = useState<string>('')
const [semantics, setSemantics] = useState<MetricSemantics>('gauge')
const [submitError, setSubmitError] = useState<string | null>(null) const [submitError, setSubmitError] = useState<string | null>(null)
const [submitting, setSubmitting] = useState(false) const [submitting, setSubmitting] = useState(false)
const [collapsed, setCollapsed] = useSectionCollapsed('metrics') const [collapsed, setCollapsed] = useSectionCollapsed('metrics')
@@ -149,6 +143,7 @@ export function Metrics({ range, activePreset }: { range: TimeRange | null; acti
setRegex('') setRegex('')
setLabelParams([]) setLabelParams([])
setValueParam('') setValueParam('')
setSemantics('gauge')
setSubmitError(null) setSubmitError(null)
} }
@@ -159,6 +154,7 @@ export function Metrics({ range, activePreset }: { range: TimeRange | null; acti
setRegex(metric.message_match_regex) setRegex(metric.message_match_regex)
setLabelParams(metric.label_params) setLabelParams(metric.label_params)
setValueParam(metric.value_param ?? '') setValueParam(metric.value_param ?? '')
setSemantics(metric.semantics ?? 'gauge')
setSubmitError(null) setSubmitError(null)
} }
@@ -190,6 +186,7 @@ export function Metrics({ range, activePreset }: { range: TimeRange | null; acti
message_match_regex: regex, message_match_regex: regex,
label_params: labelParams, label_params: labelParams,
value_param: valueParam || null, value_param: valueParam || null,
semantics,
} }
const saved = editingName ? await updateMetric(editingName, request) : await declareMetric(request) const saved = editingName ? await updateMetric(editingName, request) : await declareMetric(request)
resetForm() resetForm()
@@ -207,23 +204,13 @@ export function Metrics({ range, activePreset }: { range: TimeRange | null; acti
return ( return (
<div className="border-b border-border text-sm"> <div className="border-b border-border text-sm">
<div <SectionHeader
className={sectionHeaderInteractiveClassName} title="Metrics"
role="button" helpSectionId="metrics"
tabIndex={0} collapsed={collapsed}
aria-expanded={!collapsed} onToggle={() => setCollapsed(!collapsed)}
onClick={() => setCollapsed(!collapsed)} popOut={<PopOutButton section="metrics" range={range} activePreset={activePreset} />}
onKeyDown={handleSectionHeaderKeyDown(() => setCollapsed(!collapsed))} />
>
<div className={sectionHeaderLeftClassName}>
Metrics
<HelpIcon sectionId="metrics" label="Help: Metrics" />
</div>
<div className="flex items-center gap-1.5">
<CollapseToggle collapsed={collapsed} />
<PopOutButton section="metrics" range={range} activePreset={activePreset} />
</div>
</div>
<div className="flex gap-4 p-3" hidden={collapsed}> <div className="flex gap-4 p-3" hidden={collapsed}>
<form onSubmit={handleSubmit} className="w-80 shrink-0 space-y-2"> <form onSubmit={handleSubmit} className="w-80 shrink-0 space-y-2">
<div> <div>
@@ -294,31 +281,37 @@ export function Metrics({ range, activePreset }: { range: TimeRange | null; acti
))} ))}
</select> </select>
</div> </div>
<div>
<label
className="mb-1 block text-xs text-text-subtle"
title="Gauge: a point-in-time reading (e.g. queue depth) — rate() over it is rejected. Counter: a monotonically-increasing total (e.g. requests served) — sum()/avg() directly on it is rejected; wrap it in rate() first."
>
Semantics
</label>
<select
className="w-full rounded border border-border-soft bg-surface-1 px-2 py-1 text-xs text-text focus:border-border-strong focus:outline-none"
value={semantics}
onChange={(e) => setSemantics(e.target.value as MetricSemantics)}
>
<option value="gauge">Gauge (point-in-time reading)</option>
<option value="counter">Counter (monotonically increasing)</option>
</select>
</div>
{editingName && ( {editingName && (
<div className="rounded border border-accent bg-accent/5 px-2 py-1 text-xs text-accent-text"> <div className="rounded border border-accent bg-accent/5 px-2 py-1 text-xs text-accent-text">
Editing "{editingName}" Editing "{editingName}"
</div> </div>
)} )}
{submitError && <div className="rounded border border-danger-border bg-danger-bg/50 px-2 py-1 text-xs text-danger-text">{submitError}</div>} {submitError && <div className="rounded border border-danger-border bg-danger-bg/50 px-2 py-1 text-xs text-danger-text">{submitError}</div>}
<div className="flex items-center gap-2"> <FormFooter
<button editing={editingName !== null}
type="submit" submitting={submitting}
disabled={submitting} idleLabel="Declare metric"
className="rounded border border-border-soft px-3 py-1 text-sm text-text-secondary hover:bg-surface-2 disabled:opacity-50" editingLabel="Save changes"
> submittingLabel="Declaring…"
{submitting ? (editingName ? 'Saving…' : 'Declaring…') : editingName ? 'Save changes' : 'Declare metric'} editingSubmittingLabel="Saving…"
</button> onCancel={resetForm}
{editingName && ( />
<button
type="button"
onClick={resetForm}
disabled={submitting}
className="rounded border border-border-soft px-3 py-1 text-sm text-text-secondary hover:bg-surface-2 disabled:opacity-50"
>
Cancel
</button>
)}
</div>
</form> </form>
<div className="min-w-0 flex-1"> <div className="min-w-0 flex-1">
@@ -373,6 +366,10 @@ export function Metrics({ range, activePreset }: { range: TimeRange | null; acti
<span className="text-text-subtle">Value param: </span> <span className="text-text-subtle">Value param: </span>
<span className="font-mono">{selectedMetric.value_param ?? <span className="text-text-faint">none</span>}</span> <span className="font-mono">{selectedMetric.value_param ?? <span className="text-text-faint">none</span>}</span>
</div> </div>
<div>
<span className="text-text-subtle">Semantics: </span>
<span className="font-mono">{selectedMetric.semantics ?? 'gauge'}</span>
</div>
</div> </div>
)} )}
{groups.length === 0 ? ( {groups.length === 0 ? (
+20 -198
View File
@@ -1,11 +1,10 @@
import { useEffect, useState } from 'react' import { useEffect, useState } from 'react'
import type { Stats } from '../lib/api' import type { Stats } from '../lib/api'
import { cssContentUrl, sendTestAlertNotification, type ColorScheme, type CssEntry } from '../lib/api' import { sendTestAlertNotification } from '../lib/api'
import { useSettingsPanel } from '../lib/useSettingsPanel' import { useSettingsPanel } from '../lib/useSettingsPanel'
import { useDocsNav, type DocsTarget } from '../lib/docsNav' import { useDocsNav, type DocsTarget } from '../lib/docsNav'
import { DocsPanel } from './Docs' import { DocsPanel } from './Docs'
import { HelpIcon } from './HelpIcon' import { HelpIcon } from './HelpIcon'
import { BUILTIN_THEMES, downloadThemeFile, generateThemeCss, parseThemeCss } from '../lib/themes'
import ottylIcon from '../assets/ottyl-icon.svg' import ottylIcon from '../assets/ottyl-icon.svg'
type PanelName = 'settings' | 'docs' | null type PanelName = 'settings' | 'docs' | null
@@ -18,27 +17,9 @@ const PANEL_TITLES: Record<Exclude<PanelName, null>, string> = {
/** Mirrors the backend's `profile::DEFAULT_PROFILE_NAME` — always exists, can't be deleted. */ /** Mirrors the backend's `profile::DEFAULT_PROFILE_NAME` — always exists, can't be deleted. */
const DEFAULT_PROFILE_NAME = 'Default' const DEFAULT_PROFILE_NAME = 'Default'
const COLOR_SCHEMES: ColorScheme[] = [ /** Display label for a theme name: underscores become spaces so `capitalize` (which titles each word) renders e.g. "light_green" as "Light Green". */
'system', function themeLabel(theme: string): string {
'light', return theme.replace(/_/g, ' ')
'dark',
'red',
'green',
'blue',
'yellow',
'pink',
'orange',
'light_green',
'light_red',
'light_blue',
'light_yellow',
'light_pink',
'light_orange',
]
/** Display label for a `ColorScheme`: underscores become spaces so `capitalize` (which titles each word) renders e.g. "light_green" as "Light Green". */
function schemeLabel(scheme: ColorScheme): string {
return scheme.replace(/_/g, ' ')
} }
/** Ticket 073: plain `Xms` under a second (this stat is normally near-zero, so that's the common case), a humantime-style `Xs`/`Xm` above it — mirrors the backend's own humantime-duration inputs (see `Docs.tsx`'s window-parsing note) without pulling in a full duration-formatting library for one stat. */ /** Ticket 073: plain `Xms` under a second (this stat is normally near-zero, so that's the common case), a humantime-style `Xs`/`Xm` above it — mirrors the backend's own humantime-duration inputs (see `Docs.tsx`'s window-parsing note) without pulling in a full duration-formatting library for one stat. */
@@ -213,10 +194,6 @@ function PresetsDropdown({ panel }: { panel: ReturnType<typeof useSettingsPanel>
function SettingsPanel({ panel }: { panel: ReturnType<typeof useSettingsPanel> }) { function SettingsPanel({ panel }: { panel: ReturnType<typeof useSettingsPanel> }) {
const [maxLogsInput, setMaxLogsInput] = useState('') const [maxLogsInput, setMaxLogsInput] = useState('')
const [cssFilename, setCssFilename] = useState('')
const [cssContent, setCssContent] = useState('')
const [cssUploadError, setCssUploadError] = useState<string | null>(null)
const [cssLinkUrl, setCssLinkUrl] = useState('')
const [ntfyTopicInput, setNtfyTopicInput] = useState('') const [ntfyTopicInput, setNtfyTopicInput] = useState('')
const [ntfyServerInput, setNtfyServerInput] = useState('') const [ntfyServerInput, setNtfyServerInput] = useState('')
const [ntfyTestError, setNtfyTestError] = useState<string | null>(null) const [ntfyTestError, setNtfyTestError] = useState<string | null>(null)
@@ -272,198 +249,43 @@ function SettingsPanel({ panel }: { panel: ReturnType<typeof useSettingsPanel> }
panel.setMaxLogs(Math.floor(parsed)) panel.setMaxLogs(Math.floor(parsed))
} }
async function handleCssFileChange(e: React.ChangeEvent<HTMLInputElement>) {
const file = e.target.files?.[0]
if (!file) return
setCssFilename(file.name)
setCssContent(await file.text())
setCssUploadError(null)
}
// Every imported/linked stylesheet is always a selectable style, listed alongside the built-in
// colour schemes (ticket 020/047), so it's always validated against the canonical `--c-*`
// variable set before it's sent — that's a floor, not a ceiling: the stylesheet is free to also
// carry rules for any other selector, up to and including a full layout redesign.
async function handleCssImport(e: React.FormEvent) {
e.preventDefault()
setCssUploadError(null)
if (!cssFilename.trim() || !cssContent.trim()) return
const { missing } = parseThemeCss(cssContent)
if (missing.length > 0) {
setCssUploadError(`Missing variable${missing.length > 1 ? 's' : ''}: ${missing.join(', ')}`)
return
}
await panel.importCssEntry(cssFilename.trim(), cssContent)
setCssFilename('')
setCssContent('')
}
async function handleCssLink(e: React.FormEvent) {
e.preventDefault()
if (!cssLinkUrl.trim()) return
await panel.linkCssEntry(cssLinkUrl.trim())
setCssLinkUrl('')
}
function downloadBuiltinTheme(name: string) {
const vars = BUILTIN_THEMES[name]
if (!vars) return
downloadThemeFile(`${name}.css`, generateThemeCss(vars))
}
async function downloadCssEntry(entry: CssEntry) {
if (entry.kind === 'imported') {
const res = await fetch(cssContentUrl(entry.id))
const text = await res.text()
downloadThemeFile(entry.filename ?? `${entry.id}.css`, text)
} else if (entry.path_or_url) {
window.open(entry.path_or_url, '_blank', 'noopener')
}
}
return ( return (
<div className="flex flex-col gap-5 text-sm text-text-secondary"> <div className="flex flex-col gap-5 text-sm text-text-secondary">
<section> <section>
<div className="mb-2 flex items-center gap-1.5"> <div className="mb-2 flex items-center gap-1.5">
<h3 className="text-xs font-semibold uppercase tracking-wide text-text-subtle">Style</h3> <h3 className="text-xs font-semibold uppercase tracking-wide text-text-subtle">Theme</h3>
<span className="text-[11px] text-text-faint"> <span className="text-[11px] text-text-faint">
one list of interchangeable styles — a built-in palette or your own imported/linked stylesheet. Any of each theme is a complete stylesheet (colours, spacing, fonts, sizes, ...); only one is active at a time.
them can be a simple colour swap or a full layout redesign; only one is active at a time. To add your own, drop a `.css` file in `themes/` and rebuild.
</span> </span>
</div> </div>
<div className="mb-3 flex flex-wrap gap-1"> <div className="flex flex-wrap gap-1">
{COLOR_SCHEMES.map((scheme) => (
<div key={scheme} className="flex items-center">
<button <button
type="button" type="button"
onClick={() => panel.setColorScheme(scheme)} onClick={() => panel.setTheme('system')}
className={`rounded-l border px-2 py-1 text-xs capitalize ${ className={`rounded border px-2 py-1 text-xs capitalize ${
!panel.activeCustomTheme && panel.settings?.color_scheme === scheme panel.settings?.theme === 'system'
? 'border-border-strong bg-surface-2 text-text' ? 'border-border-strong bg-surface-2 text-text'
: 'border-border-soft text-text-muted hover:bg-surface-2' : 'border-border-soft text-text-muted hover:bg-surface-2'
}`} }`}
> >
{schemeLabel(scheme)} System
</button> </button>
{scheme !== 'system' && ( {panel.themes.map((theme) => (
<button <button
key={theme}
type="button" type="button"
onClick={() => downloadBuiltinTheme(scheme)} onClick={() => panel.setTheme(theme)}
className="rounded-r border border-l-0 border-border-soft px-1.5 py-1 text-xs text-text-faint hover:bg-surface-2 hover:text-text" className={`rounded border px-2 py-1 text-xs capitalize ${
aria-label={`Download ${schemeLabel(scheme)} theme`} panel.settings?.theme === theme
title="Download as .css" ? 'border-border-strong bg-surface-2 text-text'
> : 'border-border-soft text-text-muted hover:bg-surface-2'
↓
</button>
)}
</div>
))}
</div>
{panel.cssEntries.length === 0 ? (
<div className="mb-2 text-xs text-text-faint">No styles imported or linked yet.</div>
) : (
<ul className="mb-2 flex flex-col gap-1">
{panel.cssEntries.map((entry) => (
<li
key={entry.id}
className="flex items-center justify-between rounded border border-border px-2 py-1 text-xs"
>
<div className="min-w-0 flex-1">
<button
type="button"
onClick={() => panel.setCustomTheme(entry.id)}
className={`block w-full truncate text-left capitalize ${
panel.activeCustomTheme?.id === entry.id ? 'text-text' : 'text-text-muted hover:text-text'
}`} }`}
> >
{entry.filename ?? entry.path_or_url} {themeLabel(theme)}
</button> </button>
<div className="text-[10px] uppercase text-text-faint">{entry.kind}</div>
</div>
<div className="ml-2 flex shrink-0 items-center gap-1">
<button
type="button"
onClick={() => downloadCssEntry(entry)}
className="rounded px-1.5 text-text-subtle hover:bg-surface-2 hover:text-text"
aria-label={`Download ${entry.filename ?? entry.path_or_url}`}
title="Download as .css"
>
↓
</button>
{entry.kind === 'imported' ? (
<button
type="button"
onClick={() => panel.removeCssEntry(entry.id)}
className="rounded px-1.5 text-text-subtle hover:bg-surface-2 hover:text-danger-text"
aria-label={`Delete ${entry.filename}`}
>
delete
</button>
) : (
<span className="text-text-faintest" title="Linked CSS isn't deletable here">
—
</span>
)}
</div>
</li>
))} ))}
</ul>
)}
<p className="mb-2 text-[11px] text-text-faint">
Add your own style below — a stylesheet, uploaded or linked by URL/path. It must at least define every{' '}
<code>--c-*</code> variable the built-ins do, but nothing stops it from also targeting any other selector
in the app (layout, spacing, fonts, anything) — it isn't limited to palette overrides.
</p>
<form onSubmit={handleCssImport} className="flex flex-col gap-2">
<input
type="file"
accept=".css,text/css"
onChange={handleCssFileChange}
className="text-xs text-text-muted file:mr-2 file:rounded file:border file:border-border-soft file:bg-surface-2 file:px-2 file:py-1 file:text-text"
/>
<input
type="text"
placeholder="enter name"
value={cssFilename}
onChange={(e) => setCssFilename(e.target.value)}
className="rounded border border-border-soft bg-surface-1 px-2 py-1 text-xs text-text placeholder:text-text-subtle focus:border-border-strong focus:outline-none"
/>
<textarea
placeholder="Paste CSS content here"
value={cssContent}
onChange={(e) => setCssContent(e.target.value)}
rows={4}
className="resize-y rounded border border-border-soft bg-surface-1 px-2 py-1 font-mono text-xs text-text placeholder:text-text-subtle focus:border-border-strong focus:outline-none"
/>
{cssUploadError && <div className="text-xs text-danger-text-strong">{cssUploadError}</div>}
<button
type="submit"
className="rounded border border-border-soft px-2 py-1 text-xs text-text hover:bg-surface-2"
>
Import
</button>
</form>
<form onSubmit={handleCssLink} className="mt-2 flex flex-col gap-2">
<input
type="text"
placeholder="https://example.com/style.css"
value={cssLinkUrl}
onChange={(e) => setCssLinkUrl(e.target.value)}
className="min-w-0 flex-1 rounded border border-border-soft bg-surface-1 px-2 py-1 text-xs text-text placeholder:text-text-subtle focus:border-border-strong focus:outline-none"
/>
<div className="flex items-center gap-2">
<button
type="submit"
className="shrink-0 rounded border border-border-soft px-2 py-1 text-xs text-text hover:bg-surface-2"
>
Link
</button>
</div> </div>
</form>
</section> </section>
<section> <section>
+1 -1
View File
@@ -14,7 +14,7 @@ import {
YAxis, YAxis,
} from 'recharts' } from 'recharts'
import type { ChartType, PromqlSeries } from '../lib/api' import type { ChartType, PromqlSeries } from '../lib/api'
import type { TimeRange } from './LogTimeline' import type { TimeRange } from '../lib/timeRange'
/// Renders a label combination as a series key/legend string, e.g. `{"path":"/a"}` -> `path=/a`; no labels renders as `value`. /// Renders a label combination as a series key/legend string, e.g. `{"path":"/a"}` -> `path=/a`; no labels renders as `value`.
export function seriesLabel(labels: Record<string, string>): string { export function seriesLabel(labels: Record<string, string>): string {
+28 -54
View File
@@ -1,14 +1,6 @@
import { useMemo, useState } from 'react' import { useMemo, useState } from 'react'
import { ApiError, clearParams, declareParam, type ParamDeclaration } from '../lib/api' import { ApiError, clearParams, declareParam, type ParamDeclaration } from '../lib/api'
import { import { FormFooter, PopOutButton, SectionHeader, useSectionCollapsed } from './CollapsibleSection'
CollapseToggle,
handleSectionHeaderKeyDown,
PopOutButton,
sectionHeaderInteractiveClassName,
sectionHeaderLeftClassName,
useSectionCollapsed,
} from './CollapsibleSection'
import { HelpIcon } from './HelpIcon'
import { RegexHelpIcon } from './RegexHelpIcon' import { RegexHelpIcon } from './RegexHelpIcon'
const EMPTY_KIND: ParamDeclaration['kind'] = 'string' const EMPTY_KIND: ParamDeclaration['kind'] = 'string'
@@ -19,6 +11,16 @@ const EMPTY_KIND: ParamDeclaration['kind'] = 'string'
// sharing the same label set (ticket 038). // sharing the same label set (ticket 038).
type ExtractionSource = 'regex' | 'json_field' | 'elapsed_since_last' type ExtractionSource = 'regex' | 'json_field' | 'elapsed_since_last'
// The parenthesized suffix shown after a declared-param badge's kind, e.g. "(string, group path)" —
// one of five possible forms depending on how the param's value is produced.
function paramBadgeSuffix(p: ParamDeclaration): string {
if (p.elapsed_since_last) return ', elapsed since previous'
if (p.kind === 'boolean') return ''
if (p.json_field_path) return `, json field ${p.json_field_path}`
if (p.capture_group !== p.name) return `, group ${p.capture_group}`
return ''
}
/// A JSON *object* (not an array/scalar) parses cleanly and pretty-prints with braces — the same /// A JSON *object* (not an array/scalar) parses cleanly and pretty-prints with braces — the same
/// shape a JSON field path can be read from, so this is the "line looks like JSON" gate that /// shape a JSON field path can be read from, so this is the "line looks like JSON" gate that
/// unlocks the "JSON field" extraction option. /// unlocks the "JSON field" extraction option.
@@ -155,23 +157,13 @@ export function Params({ params, onDeclared }: { params: ParamDeclaration[]; onD
return ( return (
<div className="border-b border-border text-sm"> <div className="border-b border-border text-sm">
<div <SectionHeader
className={sectionHeaderInteractiveClassName} title="Params"
role="button" helpSectionId="parameters"
tabIndex={0} collapsed={collapsed}
aria-expanded={!collapsed} onToggle={() => setCollapsed(!collapsed)}
onClick={() => setCollapsed(!collapsed)} popOut={<PopOutButton section="params" />}
onKeyDown={handleSectionHeaderKeyDown(() => setCollapsed(!collapsed))} />
>
<div className={sectionHeaderLeftClassName}>
Params
<HelpIcon sectionId="parameters" label="Help: Params" />
</div>
<div className="flex items-center gap-1.5">
<CollapseToggle collapsed={collapsed} />
<PopOutButton section="params" />
</div>
</div>
<div className="flex gap-4 p-3" hidden={collapsed}> <div className="flex gap-4 p-3" hidden={collapsed}>
<form onSubmit={handleSubmit} className="w-80 shrink-0 space-y-2"> <form onSubmit={handleSubmit} className="w-80 shrink-0 space-y-2">
<div> <div>
@@ -276,25 +268,15 @@ export function Params({ params, onDeclared }: { params: ParamDeclaration[]; onD
</div> </div>
)} )}
{submitError && <div className="rounded border border-danger-border bg-danger-bg/50 px-2 py-1 text-xs text-danger-text">{submitError}</div>} {submitError && <div className="rounded border border-danger-border bg-danger-bg/50 px-2 py-1 text-xs text-danger-text">{submitError}</div>}
<div className="flex items-center gap-2"> <FormFooter
<button editing={editingName !== null}
type="submit" submitting={submitting}
disabled={submitting} idleLabel="Declare param"
className="rounded border border-border-soft px-3 py-1 text-sm text-text-secondary hover:bg-surface-2 disabled:opacity-50" editingLabel="Save changes"
> submittingLabel="Declaring…"
{submitting ? (editingName ? 'Saving…' : 'Declaring…') : editingName ? 'Save changes' : 'Declare param'} editingSubmittingLabel="Saving…"
</button> onCancel={resetForm}
{editingName && ( />
<button
type="button"
onClick={resetForm}
disabled={submitting}
className="rounded border border-border-soft px-3 py-1 text-sm text-text-secondary hover:bg-surface-2 disabled:opacity-50"
>
Cancel
</button>
)}
</div>
</form> </form>
<div className="min-w-0 flex-1"> <div className="min-w-0 flex-1">
@@ -329,15 +311,7 @@ export function Params({ params, onDeclared }: { params: ParamDeclaration[]; onD
title={p.elapsed_since_last ? undefined : p.message_match_regex} title={p.elapsed_since_last ? undefined : p.message_match_regex}
> >
{p.name} ({p.kind} {p.name} ({p.kind}
{p.elapsed_since_last {paramBadgeSuffix(p)}
? ', elapsed since previous'
: p.kind === 'boolean'
? ''
: p.json_field_path
? `, json field ${p.json_field_path}`
: p.capture_group !== p.name
? `, group ${p.capture_group}`
: ''}
) )
</button> </button>
))} ))}
+3 -9
View File
@@ -1,4 +1,5 @@
import { useEffect, useRef, useState } from 'react' import { useRef, useState } from 'react'
import { useClickOutside } from '../lib/useClickOutside'
// Regex syntax cheat sheet, scoped to what the Rust `regex` crate actually supports (used server-side // Regex syntax cheat sheet, scoped to what the Rust `regex` crate actually supports (used server-side
// for message-match/field-match regexes and the filter bar's `regex(...)`) — no PCRE-only lookaround // for message-match/field-match regexes and the filter bar's `regex(...)`) — no PCRE-only lookaround
@@ -30,14 +31,7 @@ export function RegexHelpIcon({ label }: { label?: string }) {
const open = pinned || hovering const open = pinned || hovering
const ref = useRef<HTMLDivElement>(null) const ref = useRef<HTMLDivElement>(null)
useEffect(() => { useClickOutside(ref, () => setPinned(false), pinned)
if (!pinned) return
function onClickOutside(e: MouseEvent) {
if (ref.current && !ref.current.contains(e.target as Node)) setPinned(false)
}
document.addEventListener('mousedown', onClickOutside)
return () => document.removeEventListener('mousedown', onClickOutside)
}, [pinned])
return ( return (
<div <div
+9 -411
View File
@@ -12,36 +12,16 @@
* danger banners, accent highlights) is a plain CSS custom property here, * danger banners, accent highlights) is a plain CSS custom property here,
* bridged into real Tailwind utilities (bg-surface-0, text-text-muted, * bridged into real Tailwind utilities (bg-surface-0, text-text-muted,
* border-accent, ...) via `@theme inline`. Components use those utilities * border-accent, ...) via `@theme inline`. Components use those utilities
* instead of hardcoded palette classes (bg-neutral-950, text-sky-400, ...), * instead of hardcoded palette classes (bg-neutral-950, text-sky-400, ...).
* so re-declaring the `--c-*` variables under a `html.<theme>` selector
* re-skins the whole app with no per-component changes.
* *
* Derivation: `dark` uses one true black-based neutral (S=0) scale, `light` * The actual theme files live in `themes/` at the repo root (embedded into
* is its exact lightness mirror — these two stay grey throughout. (There * the binary at build time — see `THEMES` in `src/api/mod.rs`), one CSS file
* used to be a separate `grey` theme sharing `dark`'s neutral scale with * per theme, each free to redeclare `--c-*` under `:root` and/or add any
* only its accent trio differing; it was a near-duplicate of `dark` and was * other rules (spacing, fonts, layout, ...) it wants. The `:root` block below
* removed in favour of a single `dark` theme — ticket 004. Settings saved * is only the pre-theme-load fallback (identical to `themes/dark.css`) so
* with `color_scheme: "grey"` still load fine: the backend aliases it to * there's no flash of unstyled content before the selected theme's
* `dark`, see `ColorScheme` in `src/settings.rs`.) * stylesheet is injected — see `useSettingsPanel.ts`. To add a new theme,
* Each named theme (red/green/blue/yellow/pink/orange) instead tints its * drop another file in `themes/` and rebuild; no code changes needed.
* whole surface/border/text scale in its own hue at low saturation (so the
* chrome itself reads as "a shade of red", not grey-with-a-red-light), and
* pulls its accent trio from two *other* hues on the wheel: --c-accent and
* --c-accent-text share the base hue's complement (base+180°), while
* --c-accent-soft sits at a split-complement (~base+150°/-150°) — a
* "composite" accent distinct from the straight complement. To add a new
* themed variant: pick a base hue H, a complement C = H+180, and a
* composite K = H+150 (mod 360), then fill --c-surface-, --c-border-,
* --c-text- as hsl(H, ...) at the saturation/lightness steps below, accent
* + accent-text as hsl(C, ...), and accent-soft as hsl(K, ...).
*
* --c-accent (base look, unchanged above) is joined by two more accent
* colours (ticket 048) so charts/highlights can draw from a 3-colour set
* instead of one hue: --c-accent-2 sits at H+90 and --c-accent-3 at H+270 —
* a quadrature pair straddling the base hue, distinct from both the
* complement (accent) and the split-complement (accent-soft). These are
* purely additive: nothing that already reads --c-accent/-soft/-text
* changes look.
*/ */
:root { :root {
color-scheme: light dark; color-scheme: light dark;
@@ -74,388 +54,6 @@
--c-accent-3: #4ade80; --c-accent-3: #4ade80;
} }
/* Exact lightness mirror of :root's neutral-950..50 scale (Tailwind neutral,
steps flipped: 950<->50, 900<->100, 800<->200, 700<->300, 600<->400). */
html.light {
--c-surface-0: #fafafa;
--c-surface-1: #f5f5f5;
--c-surface-2: #e5e5e5;
--c-border: #e5e5e5;
--c-border-soft: #d4d4d4;
--c-border-faint: #f5f5f5;
--c-border-strong: #525252;
--c-text: #171717;
--c-text-secondary: #404040;
--c-text-muted: #525252;
--c-text-subtle: #737373;
--c-text-faint: #a3a3a3;
--c-text-faintest: #d4d4d4;
--c-danger-bg: #fee2e2;
--c-danger-border: #fca5a5;
--c-danger-text: #b91c1c;
--c-danger-text-strong: #991b1b;
--c-accent: #0284c7;
--c-accent-soft: #7dd3fc;
--c-accent-text: #0369a1;
--c-accent-2: #7c3aed;
--c-accent-3: #16a34a;
}
/* base hue 0 (red): surfaces/borders/text tinted red; accent complement 180
(cyan), accent-soft composite 150 (spring green) */
html.red {
--c-surface-0: hsl(0 32% 10%);
--c-surface-1: hsl(0 28% 16%);
--c-surface-2: hsl(0 26% 22%);
--c-border: hsl(0 24% 24%);
--c-border-soft: hsl(0 22% 33%);
--c-border-faint: hsl(0 28% 16%);
--c-border-strong: hsl(0 15% 62%);
--c-text: hsl(0 15% 94%);
--c-text-secondary: hsl(0 12% 82%);
--c-text-muted: hsl(0 10% 65%);
--c-text-subtle: hsl(0 8% 50%);
--c-text-faint: hsl(0 8% 38%);
--c-text-faintest: hsl(0 10% 28%);
--c-accent: hsl(180 70% 62%);
--c-accent-soft: hsl(150 55% 34%);
--c-accent-text: hsl(180 75% 80%);
--c-accent-2: hsl(90 65% 58%);
--c-accent-3: hsl(270 65% 68%);
}
/* base hue 140 (green): surfaces/borders/text tinted green; accent complement
320 (magenta), accent-soft composite 290 (purple) */
html.green {
--c-surface-0: hsl(140 32% 10%);
--c-surface-1: hsl(140 28% 16%);
--c-surface-2: hsl(140 26% 22%);
--c-border: hsl(140 24% 24%);
--c-border-soft: hsl(140 22% 33%);
--c-border-faint: hsl(140 28% 16%);
--c-border-strong: hsl(140 15% 62%);
--c-text: hsl(140 15% 94%);
--c-text-secondary: hsl(140 12% 82%);
--c-text-muted: hsl(140 10% 65%);
--c-text-subtle: hsl(140 8% 50%);
--c-text-faint: hsl(140 8% 38%);
--c-text-faintest: hsl(140 10% 28%);
--c-accent: hsl(320 70% 62%);
--c-accent-soft: hsl(290 55% 34%);
--c-accent-text: hsl(320 75% 80%);
--c-accent-2: hsl(230 65% 62%);
--c-accent-3: hsl(50 65% 58%);
}
/* base hue 210 (blue): surfaces/borders/text tinted blue; accent complement
30 (orange), accent-soft composite 0 (red) */
html.blue {
--c-surface-0: hsl(210 32% 10%);
--c-surface-1: hsl(210 28% 16%);
--c-surface-2: hsl(210 26% 22%);
--c-border: hsl(210 24% 24%);
--c-border-soft: hsl(210 22% 33%);
--c-border-faint: hsl(210 28% 16%);
--c-border-strong: hsl(210 15% 62%);
--c-text: hsl(210 15% 94%);
--c-text-secondary: hsl(210 12% 82%);
--c-text-muted: hsl(210 10% 65%);
--c-text-subtle: hsl(210 8% 50%);
--c-text-faint: hsl(210 8% 38%);
--c-text-faintest: hsl(210 10% 28%);
--c-accent: hsl(30 70% 62%);
--c-accent-soft: hsl(0 55% 34%);
--c-accent-text: hsl(30 75% 80%);
--c-accent-2: hsl(300 65% 68%);
--c-accent-3: hsl(120 65% 55%);
}
/* base hue 50 (yellow): surfaces/borders/text tinted yellow; accent
complement 230 (blue), accent-soft composite 200 (azure) */
html.yellow {
--c-surface-0: hsl(50 32% 10%);
--c-surface-1: hsl(50 28% 16%);
--c-surface-2: hsl(50 26% 22%);
--c-border: hsl(50 24% 24%);
--c-border-soft: hsl(50 22% 33%);
--c-border-faint: hsl(50 28% 16%);
--c-border-strong: hsl(50 15% 62%);
--c-text: hsl(50 15% 94%);
--c-text-secondary: hsl(50 12% 82%);
--c-text-muted: hsl(50 10% 65%);
--c-text-subtle: hsl(50 8% 50%);
--c-text-faint: hsl(50 8% 38%);
--c-text-faintest: hsl(50 10% 28%);
--c-accent: hsl(230 70% 62%);
--c-accent-soft: hsl(200 55% 34%);
--c-accent-text: hsl(230 75% 80%);
--c-accent-2: hsl(140 65% 55%);
--c-accent-3: hsl(320 65% 68%);
}
/* base hue 330 (pink): surfaces/borders/text tinted pink; accent complement
150 (spring green), accent-soft composite 120 (green) */
html.pink {
--c-surface-0: hsl(330 32% 10%);
--c-surface-1: hsl(330 28% 16%);
--c-surface-2: hsl(330 26% 22%);
--c-border: hsl(330 24% 24%);
--c-border-soft: hsl(330 22% 33%);
--c-border-faint: hsl(330 28% 16%);
--c-border-strong: hsl(330 15% 62%);
--c-text: hsl(330 15% 94%);
--c-text-secondary: hsl(330 12% 82%);
--c-text-muted: hsl(330 10% 65%);
--c-text-subtle: hsl(330 8% 50%);
--c-text-faint: hsl(330 8% 38%);
--c-text-faintest: hsl(330 10% 28%);
--c-accent: hsl(150 70% 62%);
--c-accent-soft: hsl(120 55% 34%);
--c-accent-text: hsl(150 75% 80%);
--c-accent-2: hsl(60 65% 58%);
--c-accent-3: hsl(240 65% 68%);
}
/* base hue 30 (orange): surfaces/borders/text tinted orange; accent
complement 210 (blue), accent-soft composite 180 (cyan) */
html.orange {
--c-surface-0: hsl(30 32% 10%);
--c-surface-1: hsl(30 28% 16%);
--c-surface-2: hsl(30 26% 22%);
--c-border: hsl(30 24% 24%);
--c-border-soft: hsl(30 22% 33%);
--c-border-faint: hsl(30 28% 16%);
--c-border-strong: hsl(30 15% 62%);
--c-text: hsl(30 15% 94%);
--c-text-secondary: hsl(30 12% 82%);
--c-text-muted: hsl(30 10% 65%);
--c-text-subtle: hsl(30 8% 50%);
--c-text-faint: hsl(30 8% 38%);
--c-text-faintest: hsl(30 10% 28%);
--c-accent: hsl(210 70% 62%);
--c-accent-soft: hsl(180 55% 34%);
--c-accent-text: hsl(210 75% 80%);
--c-accent-2: hsl(120 65% 55%);
--c-accent-3: hsl(300 65% 68%);
}
/* Brighter, light-surfaced counterpart to `green` (ticket 048): same base
hue (140) and accent hues as `green`, but with a `light`-style inverted
surface/border/text scale (high lightness instead of low) so it reads as
a distinctly brighter theme, not a re-tint of the dark `green`. Danger
colours are `light`'s (hue-neutral), matching how `light` itself doesn't
tint danger either. */
html.light_green {
--c-surface-0: hsl(140 40% 96%);
--c-surface-1: hsl(140 35% 92%);
--c-surface-2: hsl(140 30% 85%);
--c-border: hsl(140 25% 80%);
--c-border-soft: hsl(140 22% 70%);
--c-border-faint: hsl(140 35% 92%);
--c-border-strong: hsl(140 20% 40%);
--c-text: hsl(140 30% 12%);
--c-text-secondary: hsl(140 20% 25%);
--c-text-muted: hsl(140 15% 40%);
--c-text-subtle: hsl(140 10% 50%);
--c-text-faint: hsl(140 10% 65%);
--c-text-faintest: hsl(140 12% 80%);
--c-danger-bg: #fee2e2;
--c-danger-border: #fca5a5;
--c-danger-text: #b91c1c;
--c-danger-text-strong: #991b1b;
--c-accent: hsl(320 70% 40%);
--c-accent-soft: hsl(290 60% 85%);
--c-accent-text: hsl(320 75% 28%);
--c-accent-2: hsl(230 70% 45%);
--c-accent-3: hsl(50 70% 38%);
}
/* Brighter, light-surfaced counterpart to `red` (ticket 056): same base hue
(0) and accent hues as `red`, with `light`'s inverted surface/border/text
scale and `light`'s hue-neutral danger colours. */
html.light_red {
--c-surface-0: hsl(0 40% 96%);
--c-surface-1: hsl(0 35% 92%);
--c-surface-2: hsl(0 30% 85%);
--c-border: hsl(0 25% 80%);
--c-border-soft: hsl(0 22% 70%);
--c-border-faint: hsl(0 35% 92%);
--c-border-strong: hsl(0 20% 40%);
--c-text: hsl(0 30% 12%);
--c-text-secondary: hsl(0 20% 25%);
--c-text-muted: hsl(0 15% 40%);
--c-text-subtle: hsl(0 10% 50%);
--c-text-faint: hsl(0 10% 65%);
--c-text-faintest: hsl(0 12% 80%);
--c-danger-bg: #fee2e2;
--c-danger-border: #fca5a5;
--c-danger-text: #b91c1c;
--c-danger-text-strong: #991b1b;
--c-accent: hsl(180 70% 40%);
--c-accent-soft: hsl(150 60% 85%);
--c-accent-text: hsl(180 75% 28%);
--c-accent-2: hsl(90 70% 45%);
--c-accent-3: hsl(270 70% 38%);
}
/* Brighter, light-surfaced counterpart to `blue` (ticket 056): same base hue
(210) and accent hues as `blue`, with `light`'s inverted surface/border/text
scale and `light`'s hue-neutral danger colours. */
html.light_blue {
--c-surface-0: hsl(210 40% 96%);
--c-surface-1: hsl(210 35% 92%);
--c-surface-2: hsl(210 30% 85%);
--c-border: hsl(210 25% 80%);
--c-border-soft: hsl(210 22% 70%);
--c-border-faint: hsl(210 35% 92%);
--c-border-strong: hsl(210 20% 40%);
--c-text: hsl(210 30% 12%);
--c-text-secondary: hsl(210 20% 25%);
--c-text-muted: hsl(210 15% 40%);
--c-text-subtle: hsl(210 10% 50%);
--c-text-faint: hsl(210 10% 65%);
--c-text-faintest: hsl(210 12% 80%);
--c-danger-bg: #fee2e2;
--c-danger-border: #fca5a5;
--c-danger-text: #b91c1c;
--c-danger-text-strong: #991b1b;
--c-accent: hsl(30 70% 40%);
--c-accent-soft: hsl(0 60% 85%);
--c-accent-text: hsl(30 75% 28%);
--c-accent-2: hsl(300 70% 45%);
--c-accent-3: hsl(120 70% 38%);
}
/* Brighter, light-surfaced counterpart to `yellow` (ticket 056): same base
hue (50) and accent hues as `yellow`, with `light`'s inverted
surface/border/text scale and `light`'s hue-neutral danger colours. */
html.light_yellow {
--c-surface-0: hsl(50 40% 96%);
--c-surface-1: hsl(50 35% 92%);
--c-surface-2: hsl(50 30% 85%);
--c-border: hsl(50 25% 80%);
--c-border-soft: hsl(50 22% 70%);
--c-border-faint: hsl(50 35% 92%);
--c-border-strong: hsl(50 20% 40%);
--c-text: hsl(50 30% 12%);
--c-text-secondary: hsl(50 20% 25%);
--c-text-muted: hsl(50 15% 40%);
--c-text-subtle: hsl(50 10% 50%);
--c-text-faint: hsl(50 10% 65%);
--c-text-faintest: hsl(50 12% 80%);
--c-danger-bg: #fee2e2;
--c-danger-border: #fca5a5;
--c-danger-text: #b91c1c;
--c-danger-text-strong: #991b1b;
--c-accent: hsl(230 70% 40%);
--c-accent-soft: hsl(200 60% 85%);
--c-accent-text: hsl(230 75% 28%);
--c-accent-2: hsl(140 70% 45%);
--c-accent-3: hsl(320 70% 38%);
}
/* Brighter, light-surfaced counterpart to `pink` (ticket 056): same base hue
(330) and accent hues as `pink`, with `light`'s inverted surface/border/text
scale and `light`'s hue-neutral danger colours. */
html.light_pink {
--c-surface-0: hsl(330 40% 96%);
--c-surface-1: hsl(330 35% 92%);
--c-surface-2: hsl(330 30% 85%);
--c-border: hsl(330 25% 80%);
--c-border-soft: hsl(330 22% 70%);
--c-border-faint: hsl(330 35% 92%);
--c-border-strong: hsl(330 20% 40%);
--c-text: hsl(330 30% 12%);
--c-text-secondary: hsl(330 20% 25%);
--c-text-muted: hsl(330 15% 40%);
--c-text-subtle: hsl(330 10% 50%);
--c-text-faint: hsl(330 10% 65%);
--c-text-faintest: hsl(330 12% 80%);
--c-danger-bg: #fee2e2;
--c-danger-border: #fca5a5;
--c-danger-text: #b91c1c;
--c-danger-text-strong: #991b1b;
--c-accent: hsl(150 70% 40%);
--c-accent-soft: hsl(120 60% 85%);
--c-accent-text: hsl(150 75% 28%);
--c-accent-2: hsl(60 70% 45%);
--c-accent-3: hsl(240 70% 38%);
}
/* Brighter, light-surfaced counterpart to `orange` (ticket 056): same base
hue (30) and accent hues as `orange`, with `light`'s inverted
surface/border/text scale and `light`'s hue-neutral danger colours. */
html.light_orange {
--c-surface-0: hsl(30 40% 96%);
--c-surface-1: hsl(30 35% 92%);
--c-surface-2: hsl(30 30% 85%);
--c-border: hsl(30 25% 80%);
--c-border-soft: hsl(30 22% 70%);
--c-border-faint: hsl(30 35% 92%);
--c-border-strong: hsl(30 20% 40%);
--c-text: hsl(30 30% 12%);
--c-text-secondary: hsl(30 20% 25%);
--c-text-muted: hsl(30 15% 40%);
--c-text-subtle: hsl(30 10% 50%);
--c-text-faint: hsl(30 10% 65%);
--c-text-faintest: hsl(30 12% 80%);
--c-danger-bg: #fee2e2;
--c-danger-border: #fca5a5;
--c-danger-text: #b91c1c;
--c-danger-text-strong: #991b1b;
--c-accent: hsl(210 70% 40%);
--c-accent-soft: hsl(180 60% 85%);
--c-accent-text: hsl(210 75% 28%);
--c-accent-2: hsl(120 70% 45%);
--c-accent-3: hsl(300 70% 38%);
}
@theme inline { @theme inline {
--color-surface-0: var(--c-surface-0); --color-surface-0: var(--c-surface-0);
--color-surface-1: var(--c-surface-1); --color-surface-1: var(--c-surface-1);
-471
View File
@@ -1,471 +0,0 @@
// Externally-tagged JSON matching the Rust `MetricPrimitive` enum's derived `Serialize`, e.g. `{ "Duration": 1500 }` (milliseconds).
export type MetricPrimitive =
| { String: string }
| { Number: { Int: number } | { Float: number } }
| { Timestamp: number }
| { Duration: number }
| { Boolean: boolean }
export function metricPrimitiveToDisplayString(value: MetricPrimitive): string {
if ('String' in value) return value.String
if ('Timestamp' in value) return String(value.Timestamp)
if ('Duration' in value) return `${value.Duration}ms`
if ('Boolean' in value) return String(value.Boolean)
if ('Int' in value.Number) return String(value.Number.Int)
return String(value.Number.Float)
}
export interface LogEntry {
sequence: number
timestamp_ms: number
text: string
// A param with no match on this line is simply absent from the map.
params: Record<string, MetricPrimitive>
}
export interface ParamDeclaration {
name: string
message_match_regex: string
capture_group: string
/** Alternative to `capture_group`: a dot-separated JSON field path (e.g. "foo.bar"), read from lines that parse as a JSON object. When set (non-empty), takes precedence over `capture_group`. */
json_field_path?: string | null
kind: 'string' | 'number' | 'timestamp' | 'duration' | 'boolean'
/** A third extraction source, distinct from `capture_group`/`json_field_path`: when true, this param's value is never parsed from text — it's computed as elapsed time since the previous event sharing the same label set. Only meaningful paired with `kind: 'duration'`. */
elapsed_since_last?: boolean
/** Set only for the built-in `ottyl_seq`/`ottyl_ts`/`ottyl_fmt` names, synthesized into `GET /api/params` for discoverability — they're usable in the filter bar (and `ottyl_seq`/`ottyl_ts` in dashboard PromQL queries) without ever being declared, and can't be edited or removed. */
reserved?: boolean
}
export interface TimelineBucket {
start_ms: number
count: number
}
export interface Stats {
log_count: number
first_sequence: number | null
last_sequence: number | null
first_timestamp_ms: number | null
last_timestamp_ms: number | null
/** Ticket 011: running approximate byte size of all currently-ingested logs. */
approx_bytes_total: number
/** Ticket 073: highest ingestion-to-engine-processing lag (ms) seen within the trailing 5s. */
ingest_lag_high_watermark_ms: number
}
export interface MetricDeclaration {
name: string
message_match_regex: string
/** Names of already-declared `ParamDeclaration`s (see the Params panel) whose values become this metric's bucket labels. */
label_params: string[]
/** Name of an already-declared param (if any) whose value becomes this metric's per-event bucket value instead of a label. */
value_param?: string | null
/** Fixed labels applied on a match/non-match, on top of any captured params. */
matched_labels?: Record<string, string> | null
unmatched_labels?: Record<string, string> | null
}
export interface BucketValue {
labels: Record<string, MetricPrimitive>
sequence: number
timestamp_ms: number
value: MetricPrimitive
}
// Error shape returned by API handlers on a 4xx/5xx response (see `ErrorDto` on the backend).
export interface ApiErrorBody {
error: string
}
export class ApiError extends Error {
constructor(message: string) {
super(message)
this.name = 'ApiError'
}
}
export type ColorScheme =
| 'light'
| 'dark'
| 'system'
| 'red'
| 'green'
| 'blue'
| 'yellow'
| 'pink'
| 'orange'
| 'light_green'
| 'light_red'
| 'light_blue'
| 'light_yellow'
| 'light_pink'
| 'light_orange'
export interface Settings {
color_scheme: ColorScheme
active_profile: string | null
/** Maximum ingested logs kept in memory; `null` means unlimited. */
max_logs: number | null
/** Id of the selected custom theme entry, or `null` if `color_scheme` alone applies. */
custom_theme_id: string | null
/** Ntfy topic every alert rule delivers to; `null` until configured. */
ntfy_topic: string | null
/** Ntfy server override; `null` means `https://ntfy.sh`. */
ntfy_server: string | null
}
export type CssEntryKind = 'imported' | 'linked'
export interface CssEntry {
id: string
kind: CssEntryKind
filename: string | null
path_or_url: string | null
}
export interface ProfileSummary {
name: string
param_count: number
metric_count: number
dashboard_count: number
}
// One chart-ready time series from `GET /api/query`: a label combination plus its `(timestamp_ms, value)` points, sorted by timestamp.
export interface PromqlSeries {
labels: Record<string, string>
points: [number, number][]
}
export type ChartType = 'bar' | 'line' | 'pie'
export interface Panel {
id: string
title: string
promql: string
chart_type: ChartType
}
export interface Dashboard {
id: string
name: string
panels: Panel[]
}
export type CompareOp = 'lt' | 'gt' | 'le' | 'ge' | 'eq' | 'ne'
// Externally tagged on `kind`, matching `AlertCondition`'s `#[serde(tag = "kind")]` on the backend.
export type AlertCondition =
| { kind: 'immediate'; query: string }
| { kind: 'threshold'; query: string; op: CompareOp; threshold: number; window_ms: number }
| { kind: 'newvalue'; field_regex: string; capture_group: string }
| { kind: 'ratio'; query_a: string; query_b: string; op: CompareOp; ratio: number; window_ms: number }
export interface AlertRule {
id: string
name: string
enabled: boolean
condition: AlertCondition
}
async function getJson<T>(path: string): Promise<T> {
const res = await fetch(path)
if (!res.ok) {
let message = `${path} failed: ${res.status}`
try {
const body: unknown = await res.json()
if (body && typeof body === 'object' && 'error' in body && typeof (body as { error: unknown }).error === 'string') {
message = (body as { error: string }).error
}
} catch {
// body wasn't JSON (or had no `error` field) — fall back to the generic message
}
throw new Error(message)
}
return res.json() as Promise<T>
}
// Throws `ApiError` (not `Error`) with the parsed `{ "error": "..." }` message so callers can surface it inline.
async function postJson<T>(path: string, body: unknown): Promise<T> {
const res = await fetch(path, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
})
if (!res.ok) {
let message = `${path} failed: ${res.status}`
try {
const errorBody = (await res.json()) as Partial<ApiErrorBody>
if (errorBody && typeof errorBody.error === 'string') {
message = errorBody.error
}
} catch {
// response body wasn't JSON; fall back to the generic message
}
throw new ApiError(message)
}
return res.json() as Promise<T>
}
// Same error handling as `postJson` (throws `ApiError` with the parsed message) but for a PUT — used where updating in place, rather than creating, is the operation.
async function putJson<T>(path: string, body: unknown): Promise<T> {
const res = await fetch(path, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
})
if (!res.ok) {
let message = `${path} failed: ${res.status}`
try {
const errorBody = (await res.json()) as Partial<ApiErrorBody>
if (errorBody && typeof errorBody.error === 'string') {
message = errorBody.error
}
} catch {
// response body wasn't JSON; fall back to the generic message
}
throw new ApiError(message)
}
return res.json() as Promise<T>
}
async function sendJson<T>(path: string, method: string, body?: unknown): Promise<T> {
const res = await fetch(path, {
method,
headers: body !== undefined ? { 'Content-Type': 'application/json' } : undefined,
body: body !== undefined ? JSON.stringify(body) : undefined,
})
if (!res.ok) {
let message = `${path} failed: ${res.status}`
try {
const data = await res.json()
if (data?.error) message = data.error
} catch {
// response body wasn't JSON (or was empty) — fall back to the generic message
}
throw new Error(message)
}
if (res.status === 204) return undefined as T
return res.json() as Promise<T>
}
export function fetchLogs(params: {
sinceSequence?: number
startMs?: number
endMs?: number
limit?: number
filter?: string
/** Ticket 045: an already-declared param name to order entries by, instead of the default (`ottyl_ts`-tracking) order. */
sortBy?: string
/** Ticket 045: only meaningful alongside `sortBy`. `true` (default) orders lines missing that param by `ottyl_seq`, appended after the sorted lines; `false` drops them. */
sortMissingFallback?: boolean
}): Promise<LogEntry[]> {
const search = new URLSearchParams()
if (params.sinceSequence !== undefined) search.set('since_sequence', String(params.sinceSequence))
if (params.startMs !== undefined) search.set('start_ms', String(params.startMs))
if (params.endMs !== undefined) search.set('end_ms', String(params.endMs))
if (params.limit !== undefined) search.set('limit', String(params.limit))
if (params.filter) search.set('filter', params.filter)
if (params.sortBy) search.set('sort_by', params.sortBy)
if (params.sortMissingFallback !== undefined) search.set('sort_missing_fallback', String(params.sortMissingFallback))
return getJson(`/api/logs?${search.toString()}`)
}
export function fetchTimeline(params: {
startMs?: number
endMs?: number
bucketMs?: number
filter?: string
}): Promise<TimelineBucket[]> {
const search = new URLSearchParams()
if (params.startMs !== undefined) search.set('start_ms', String(params.startMs))
if (params.endMs !== undefined) search.set('end_ms', String(params.endMs))
if (params.bucketMs !== undefined) search.set('bucket_ms', String(params.bucketMs))
if (params.filter) search.set('filter', params.filter)
return getJson(`/api/timeline?${search.toString()}`)
}
export function fetchStats(): Promise<Stats> {
return getJson('/api/stats')
}
export function fetchParamDeclarations(): Promise<ParamDeclaration[]> {
return getJson('/api/params')
}
export function declareParam(request: ParamDeclaration): Promise<ParamDeclaration> {
return postJson('/api/params', request)
}
export interface ClearParamsResult {
/** How many param declarations (manual + auto-derived) were removed. */
removed: number
/** How many params were immediately re-derived from log text still retained by the engine. */
reextracted: number
/** How many retained log lines were re-scanned — lines already evicted by `max_logs` before the clear can't regain their params, so this can be smaller than the total lines ever ingested. */
logs_considered: number
}
// Clears every declared param (ticket 052) and re-derives whatever the engine's still-retained log
// text supports — see `ClearParamsResult`'s fields for what came back.
export function clearParams(): Promise<ClearParamsResult> {
return sendJson('/api/params', 'DELETE')
}
export function fetchMetricDeclarations(): Promise<MetricDeclaration[]> {
return getJson('/api/metrics')
}
export function declareMetric(request: MetricDeclaration): Promise<MetricDeclaration> {
return postJson('/api/metrics', request)
}
// Updates the metric declaration named `existingName` in place (`existingName` may differ from `request.name` to rename it), rather than declaring a second metric or failing on a duplicate-name conflict.
export function updateMetric(existingName: string, request: MetricDeclaration): Promise<MetricDeclaration> {
return putJson(`/api/metrics/${encodeURIComponent(existingName)}`, request)
}
export function deleteMetric(name: string): Promise<void> {
return sendJson(`/api/metrics/${encodeURIComponent(name)}`, 'DELETE')
}
export function fetchMetricBucketValues(
name: string,
params: {
labels?: string
sinceSequence?: number
startMs?: number
endMs?: number
limit?: number
} = {},
): Promise<BucketValue[]> {
const search = new URLSearchParams()
if (params.labels !== undefined) search.set('labels', params.labels)
if (params.sinceSequence !== undefined) search.set('since_sequence', String(params.sinceSequence))
if (params.startMs !== undefined) search.set('start_ms', String(params.startMs))
if (params.endMs !== undefined) search.set('end_ms', String(params.endMs))
if (params.limit !== undefined) search.set('limit', String(params.limit))
return getJson(`/api/metrics/${encodeURIComponent(name)}?${search.toString()}`)
}
export function fetchSettings(): Promise<Settings> {
return getJson('/api/settings')
}
export function updateSettings(update: {
colorScheme?: ColorScheme
/** Omit to leave unchanged; `null` clears the cap back to unlimited. */
maxLogs?: number | null
/** Omit to leave unchanged; `null` clears the selection (falls back to `colorScheme`). */
customThemeId?: string | null
/** Omit to leave unchanged; `null` clears the configured topic. */
ntfyTopic?: string | null
/** Omit to leave unchanged; `null` falls back to `https://ntfy.sh`. */
ntfyServer?: string | null
}): Promise<Settings> {
const body: Record<string, unknown> = { color_scheme: update.colorScheme }
if ('maxLogs' in update) body.max_logs = update.maxLogs
if ('customThemeId' in update) body.custom_theme_id = update.customThemeId
if ('ntfyTopic' in update) body.ntfy_topic = update.ntfyTopic
if ('ntfyServer' in update) body.ntfy_server = update.ntfyServer
return sendJson('/api/settings', 'PATCH', body)
}
export function fetchCssEntries(): Promise<CssEntry[]> {
return getJson('/api/settings/css')
}
export function importCss(filename: string, content: string): Promise<CssEntry> {
return sendJson('/api/settings/css', 'POST', { filename, content })
}
export function linkCss(pathOrUrl: string): Promise<CssEntry> {
return sendJson('/api/settings/css/link', 'POST', { path_or_url: pathOrUrl })
}
export function deleteCssEntry(id: string): Promise<void> {
return sendJson(`/api/settings/css/${encodeURIComponent(id)}`, 'DELETE')
}
export function cssContentUrl(id: string): string {
return `/api/settings/css/${encodeURIComponent(id)}/content`
}
export function fetchProfiles(): Promise<ProfileSummary[]> {
return getJson('/api/settings/profiles')
}
export function createProfile(name: string): Promise<ProfileSummary> {
return sendJson('/api/settings/profiles', 'POST', { name })
}
export function deleteProfile(name: string): Promise<void> {
return sendJson(`/api/settings/profiles/${encodeURIComponent(name)}`, 'DELETE')
}
export function activateProfile(name: string): Promise<Settings> {
return sendJson(`/api/settings/profiles/${encodeURIComponent(name)}/activate`, 'POST')
}
// Throws on a malformed query (400); callers should catch and surface the message inline rather than letting it crash the page.
export function fetchPromqlSeries(params: {
promql: string
startMs?: number
endMs?: number
}): Promise<PromqlSeries[]> {
const search = new URLSearchParams()
search.set('promql', params.promql)
if (params.startMs !== undefined) search.set('start_ms', String(params.startMs))
if (params.endMs !== undefined) search.set('end_ms', String(params.endMs))
return getJson(`/api/query?${search.toString()}`)
}
export function fetchDashboards(profileName: string): Promise<Dashboard[]> {
return getJson(`/api/settings/profiles/${encodeURIComponent(profileName)}/dashboards`)
}
export function createDashboard(profileName: string, name: string): Promise<Dashboard> {
return sendJson(`/api/settings/profiles/${encodeURIComponent(profileName)}/dashboards`, 'POST', { name })
}
export function saveDashboard(
profileName: string,
dashboardId: string,
update: { name: string; panels: Panel[] },
): Promise<Dashboard> {
return sendJson(
`/api/settings/profiles/${encodeURIComponent(profileName)}/dashboards/${encodeURIComponent(dashboardId)}`,
'PUT',
update,
)
}
export function deleteDashboard(profileName: string, dashboardId: string): Promise<void> {
return sendJson(
`/api/settings/profiles/${encodeURIComponent(profileName)}/dashboards/${encodeURIComponent(dashboardId)}`,
'DELETE',
)
}
export function fetchAlertRules(): Promise<AlertRule[]> {
return getJson('/api/settings/alerts')
}
export function createAlertRule(rule: { name: string; enabled: boolean; condition: AlertCondition }): Promise<AlertRule> {
return postJson('/api/settings/alerts', rule)
}
export function updateAlertRule(
id: string,
rule: { name: string; enabled: boolean; condition: AlertCondition },
): Promise<AlertRule> {
return sendJson(`/api/settings/alerts/${encodeURIComponent(id)}`, 'PUT', rule)
}
export function deleteAlertRule(id: string): Promise<void> {
return sendJson(`/api/settings/alerts/${encodeURIComponent(id)}`, 'DELETE')
}
// Delivers a test notification without requiring the rule to be saved first; throws `ApiError` on failure so callers can surface it inline.
export function sendTestAlertNotification(topic: string, server: string | null): Promise<void> {
return sendJson('/api/settings/alerts/test', 'POST', { topic, server })
}
+41
View File
@@ -0,0 +1,41 @@
import { getJson, postJson, sendJson } from './shared'
export type CompareOp = 'lt' | 'gt' | 'le' | 'ge' | 'eq' | 'ne'
// Externally tagged on `kind`, matching `AlertCondition`'s `#[serde(tag = "kind")]` on the backend.
export type AlertCondition =
| { kind: 'immediate'; query: string }
| { kind: 'threshold'; query: string; op: CompareOp; threshold: number; window_ms: number }
| { kind: 'newvalue'; field_regex: string; capture_group: string }
| { kind: 'ratio'; query_a: string; query_b: string; op: CompareOp; ratio: number; window_ms: number }
export interface AlertRule {
id: string
name: string
enabled: boolean
condition: AlertCondition
}
export function fetchAlertRules(): Promise<AlertRule[]> {
return getJson('/api/settings/alerts')
}
export function createAlertRule(rule: { name: string; enabled: boolean; condition: AlertCondition }): Promise<AlertRule> {
return postJson('/api/settings/alerts', rule)
}
export function updateAlertRule(
id: string,
rule: { name: string; enabled: boolean; condition: AlertCondition },
): Promise<AlertRule> {
return sendJson(`/api/settings/alerts/${encodeURIComponent(id)}`, 'PUT', rule)
}
export function deleteAlertRule(id: string): Promise<void> {
return sendJson(`/api/settings/alerts/${encodeURIComponent(id)}`, 'DELETE')
}
// Delivers a test notification without requiring the rule to be saved first; throws `ApiError` on failure so callers can surface it inline.
export function sendTestAlertNotification(topic: string, server: string | null): Promise<void> {
return sendJson('/api/settings/alerts/test', 'POST', { topic, server })
}
+62
View File
@@ -0,0 +1,62 @@
import { getJson, sendJson } from './shared'
// One chart-ready time series from `GET /api/query`: a label combination plus its `(timestamp_ms, value)` points, sorted by timestamp.
export interface PromqlSeries {
labels: Record<string, string>
points: [number, number][]
}
export type ChartType = 'bar' | 'line' | 'pie'
export interface Panel {
id: string
title: string
promql: string
chart_type: ChartType
}
export interface Dashboard {
id: string
name: string
panels: Panel[]
}
// Throws on a malformed query (400); callers should catch and surface the message inline rather than letting it crash the page.
export function fetchPromqlSeries(params: {
promql: string
startMs?: number
endMs?: number
}): Promise<PromqlSeries[]> {
const search = new URLSearchParams()
search.set('promql', params.promql)
if (params.startMs !== undefined) search.set('start_ms', String(params.startMs))
if (params.endMs !== undefined) search.set('end_ms', String(params.endMs))
return getJson(`/api/query?${search.toString()}`)
}
export function fetchDashboards(profileName: string): Promise<Dashboard[]> {
return getJson(`/api/settings/profiles/${encodeURIComponent(profileName)}/dashboards`)
}
export function createDashboard(profileName: string, name: string): Promise<Dashboard> {
return sendJson(`/api/settings/profiles/${encodeURIComponent(profileName)}/dashboards`, 'POST', { name })
}
export function saveDashboard(
profileName: string,
dashboardId: string,
update: { name: string; panels: Panel[] },
): Promise<Dashboard> {
return sendJson(
`/api/settings/profiles/${encodeURIComponent(profileName)}/dashboards/${encodeURIComponent(dashboardId)}`,
'PUT',
update,
)
}
export function deleteDashboard(profileName: string, dashboardId: string): Promise<void> {
return sendJson(
`/api/settings/profiles/${encodeURIComponent(profileName)}/dashboards/${encodeURIComponent(dashboardId)}`,
'DELETE',
)
}
+7
View File
@@ -0,0 +1,7 @@
export * from './shared'
export * from './logs'
export * from './params'
export * from './metrics'
export * from './dashboards'
export * from './alerts'
export * from './settings'
+85
View File
@@ -0,0 +1,85 @@
import { buildSearchParams, getJson } from './shared'
// Externally-tagged JSON matching the Rust `MetricPrimitive` enum's derived `Serialize`, e.g. `{ "Duration": 1500 }` (milliseconds).
export type MetricPrimitive =
| { String: string }
| { Number: { Int: number } | { Float: number } }
| { Timestamp: number }
| { Duration: number }
| { Boolean: boolean }
export function metricPrimitiveToDisplayString(value: MetricPrimitive): string {
if ('String' in value) return value.String
if ('Timestamp' in value) return String(value.Timestamp)
if ('Duration' in value) return `${value.Duration}ms`
if ('Boolean' in value) return String(value.Boolean)
if ('Int' in value.Number) return String(value.Number.Int)
return String(value.Number.Float)
}
export interface LogEntry {
sequence: number
timestamp_ms: number
text: string
// A param with no match on this line is simply absent from the map.
params: Record<string, MetricPrimitive>
}
export interface TimelineBucket {
start_ms: number
count: number
}
export interface Stats {
log_count: number
first_sequence: number | null
last_sequence: number | null
first_timestamp_ms: number | null
last_timestamp_ms: number | null
/** Ticket 011: running approximate byte size of all currently-ingested logs. */
approx_bytes_total: number
/** Ticket 073: highest ingestion-to-engine-processing lag (ms) seen within the trailing 5s. */
ingest_lag_high_watermark_ms: number
}
export function fetchLogs(params: {
sinceSequence?: number
startMs?: number
endMs?: number
limit?: number
filter?: string
/** Ticket 045: an already-declared param name to order entries by, instead of the default (`ottyl_ts`-tracking) order. */
sortBy?: string
/** Ticket 045: only meaningful alongside `sortBy`. `true` (default) orders lines missing that param by `ottyl_seq`, appended after the sorted lines; `false` drops them. */
sortMissingFallback?: boolean
}): Promise<LogEntry[]> {
const search = buildSearchParams({
since_sequence: params.sinceSequence,
start_ms: params.startMs,
end_ms: params.endMs,
limit: params.limit,
filter: params.filter,
sort_by: params.sortBy,
sort_missing_fallback: params.sortMissingFallback,
})
return getJson(`/api/logs?${search.toString()}`)
}
export function fetchTimeline(params: {
startMs?: number
endMs?: number
bucketMs?: number
filter?: string
}): Promise<TimelineBucket[]> {
const search = buildSearchParams({
start_ms: params.startMs,
end_ms: params.endMs,
bucket_ms: params.bucketMs,
filter: params.filter,
})
return getJson(`/api/timeline?${search.toString()}`)
}
export function fetchStats(): Promise<Stats> {
return getJson('/api/stats')
}
+61
View File
@@ -0,0 +1,61 @@
import { getJson, postJson, putJson, sendJson } from './shared'
import type { MetricPrimitive } from './logs'
export type MetricSemantics = 'gauge' | 'counter'
export interface MetricDeclaration {
name: string
message_match_regex: string
/** Names of already-declared `ParamDeclaration`s (see the Params panel) whose values become this metric's bucket labels. */
label_params: string[]
/** Name of an already-declared param (if any) whose value becomes this metric's per-event bucket value instead of a label. */
value_param?: string | null
/** Fixed labels applied on a match/non-match, on top of any captured params. */
matched_labels?: Record<string, string> | null
unmatched_labels?: Record<string, string> | null
/** Point-in-time reading ('gauge', e.g. queue depth) vs. monotonically-increasing total ('counter', e.g. requests served); guards which PromQL aggregations are valid. Defaults to 'gauge'. */
semantics?: MetricSemantics
}
export interface BucketValue {
labels: Record<string, MetricPrimitive>
sequence: number
timestamp_ms: number
value: MetricPrimitive
}
export function fetchMetricDeclarations(): Promise<MetricDeclaration[]> {
return getJson('/api/metrics')
}
export function declareMetric(request: MetricDeclaration): Promise<MetricDeclaration> {
return postJson('/api/metrics', request)
}
// Updates the metric declaration named `existingName` in place (`existingName` may differ from `request.name` to rename it), rather than declaring a second metric or failing on a duplicate-name conflict.
export function updateMetric(existingName: string, request: MetricDeclaration): Promise<MetricDeclaration> {
return putJson(`/api/metrics/${encodeURIComponent(existingName)}`, request)
}
export function deleteMetric(name: string): Promise<void> {
return sendJson(`/api/metrics/${encodeURIComponent(name)}`, 'DELETE')
}
export function fetchMetricBucketValues(
name: string,
params: {
labels?: string
sinceSequence?: number
startMs?: number
endMs?: number
limit?: number
} = {},
): Promise<BucketValue[]> {
const search = new URLSearchParams()
if (params.labels !== undefined) search.set('labels', params.labels)
if (params.sinceSequence !== undefined) search.set('since_sequence', String(params.sinceSequence))
if (params.startMs !== undefined) search.set('start_ms', String(params.startMs))
if (params.endMs !== undefined) search.set('end_ms', String(params.endMs))
if (params.limit !== undefined) search.set('limit', String(params.limit))
return getJson(`/api/metrics/${encodeURIComponent(name)}?${search.toString()}`)
}
+37
View File
@@ -0,0 +1,37 @@
import { getJson, postJson, sendJson } from './shared'
export interface ParamDeclaration {
name: string
message_match_regex: string
capture_group: string
/** Alternative to `capture_group`: a dot-separated JSON field path (e.g. "foo.bar"), read from lines that parse as a JSON object. When set (non-empty), takes precedence over `capture_group`. */
json_field_path?: string | null
kind: 'string' | 'number' | 'timestamp' | 'duration' | 'boolean'
/** A third extraction source, distinct from `capture_group`/`json_field_path`: when true, this param's value is never parsed from text — it's computed as elapsed time since the previous event sharing the same label set. Only meaningful paired with `kind: 'duration'`. */
elapsed_since_last?: boolean
/** Set only for the built-in `ottyl_seq`/`ottyl_ts`/`ottyl_fmt` names, synthesized into `GET /api/params` for discoverability — they're usable in the filter bar (and `ottyl_seq`/`ottyl_ts` in dashboard PromQL queries) without ever being declared, and can't be edited or removed. */
reserved?: boolean
}
export function fetchParamDeclarations(): Promise<ParamDeclaration[]> {
return getJson('/api/params')
}
export function declareParam(request: ParamDeclaration): Promise<ParamDeclaration> {
return postJson('/api/params', request)
}
export interface ClearParamsResult {
/** How many param declarations (manual + auto-derived) were removed. */
removed: number
/** How many params were immediately re-derived from log text still retained by the engine. */
reextracted: number
/** How many retained log lines were re-scanned — lines already evicted by `max_logs` before the clear can't regain their params, so this can be smaller than the total lines ever ingested. */
logs_considered: number
}
// Clears every declared param (ticket 052) and re-derives whatever the engine's still-retained log
// text supports — see `ClearParamsResult`'s fields for what came back.
export function clearParams(): Promise<ClearParamsResult> {
return sendJson('/api/params', 'DELETE')
}
+67
View File
@@ -0,0 +1,67 @@
import { getJson, sendJson } from './shared'
/** Name of a theme embedded in this build (see `GET /api/themes`), or `"system"` to follow the OS light/dark preference. */
export type Theme = string
export interface Settings {
theme: Theme
active_profile: string | null
/** Maximum ingested logs kept in memory; `null` means unlimited. */
max_logs: number | null
/** Ntfy topic every alert rule delivers to; `null` until configured. */
ntfy_topic: string | null
/** Ntfy server override; `null` means `https://ntfy.sh`. */
ntfy_server: string | null
}
export interface ProfileSummary {
name: string
param_count: number
metric_count: number
dashboard_count: number
}
export function fetchSettings(): Promise<Settings> {
return getJson('/api/settings')
}
export function updateSettings(update: {
theme?: Theme
/** Omit to leave unchanged; `null` clears the cap back to unlimited. */
maxLogs?: number | null
/** Omit to leave unchanged; `null` clears the configured topic. */
ntfyTopic?: string | null
/** Omit to leave unchanged; `null` falls back to `https://ntfy.sh`. */
ntfyServer?: string | null
}): Promise<Settings> {
const body: Record<string, unknown> = { theme: update.theme }
if ('maxLogs' in update) body.max_logs = update.maxLogs
if ('ntfyTopic' in update) body.ntfy_topic = update.ntfyTopic
if ('ntfyServer' in update) body.ntfy_server = update.ntfyServer
return sendJson('/api/settings', 'PATCH', body)
}
/** Names of the themes embedded in this build (see `themes/` at the repo root). */
export function fetchThemes(): Promise<string[]> {
return getJson('/api/themes')
}
export function themeCssUrl(name: string): string {
return `/api/themes/${encodeURIComponent(name)}`
}
export function fetchProfiles(): Promise<ProfileSummary[]> {
return getJson('/api/settings/profiles')
}
export function createProfile(name: string): Promise<ProfileSummary> {
return sendJson('/api/settings/profiles', 'POST', { name })
}
export function deleteProfile(name: string): Promise<void> {
return sendJson(`/api/settings/profiles/${encodeURIComponent(name)}`, 'DELETE')
}
export function activateProfile(name: string): Promise<Settings> {
return sendJson(`/api/settings/profiles/${encodeURIComponent(name)}/activate`, 'POST')
}
+103
View File
@@ -0,0 +1,103 @@
// Error shape returned by API handlers on a 4xx/5xx response (see `ErrorDto` on the backend).
export interface ApiErrorBody {
error: string
}
export class ApiError extends Error {
constructor(message: string) {
super(message)
this.name = 'ApiError'
}
}
// Builds a `URLSearchParams` from an object of optional query fields, skipping any that are
// `undefined` (falsy strings like `''` are also skipped, matching the hand-written `if` checks
// this replaces).
export function buildSearchParams(fields: Record<string, string | number | boolean | undefined>): URLSearchParams {
const search = new URLSearchParams()
for (const [key, value] of Object.entries(fields)) {
if (value !== undefined && value !== '') search.set(key, String(value))
}
return search
}
export async function getJson<T>(path: string): Promise<T> {
const res = await fetch(path)
if (!res.ok) {
let message = `${path} failed: ${res.status}`
try {
const body: unknown = await res.json()
if (body && typeof body === 'object' && 'error' in body && typeof (body as { error: unknown }).error === 'string') {
message = (body as { error: string }).error
}
} catch {
// body wasn't JSON (or had no `error` field) — fall back to the generic message
}
throw new Error(message)
}
return res.json() as Promise<T>
}
// Throws `ApiError` (not `Error`) with the parsed `{ "error": "..." }` message so callers can surface it inline.
export async function postJson<T>(path: string, body: unknown): Promise<T> {
const res = await fetch(path, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
})
if (!res.ok) {
let message = `${path} failed: ${res.status}`
try {
const errorBody = (await res.json()) as Partial<ApiErrorBody>
if (errorBody && typeof errorBody.error === 'string') {
message = errorBody.error
}
} catch {
// response body wasn't JSON; fall back to the generic message
}
throw new ApiError(message)
}
return res.json() as Promise<T>
}
// Same error handling as `postJson` (throws `ApiError` with the parsed message) but for a PUT — used where updating in place, rather than creating, is the operation.
export async function putJson<T>(path: string, body: unknown): Promise<T> {
const res = await fetch(path, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
})
if (!res.ok) {
let message = `${path} failed: ${res.status}`
try {
const errorBody = (await res.json()) as Partial<ApiErrorBody>
if (errorBody && typeof errorBody.error === 'string') {
message = errorBody.error
}
} catch {
// response body wasn't JSON; fall back to the generic message
}
throw new ApiError(message)
}
return res.json() as Promise<T>
}
export async function sendJson<T>(path: string, method: string, body?: unknown): Promise<T> {
const res = await fetch(path, {
method,
headers: body !== undefined ? { 'Content-Type': 'application/json' } : undefined,
body: body !== undefined ? JSON.stringify(body) : undefined,
})
if (!res.ok) {
let message = `${path} failed: ${res.status}`
try {
const data = await res.json()
if (data?.error) message = data.error
} catch {
// response body wasn't JSON (or was empty) — fall back to the generic message
}
throw new Error(message)
}
if (res.status === 204) return undefined as T
return res.json() as Promise<T>
}
-412
View File
@@ -1,412 +0,0 @@
// A user-supplied style is any CSS file that defines every variable in `THEME_CSS_VARS` somewhere in
// a `:root { --c-*: ...; }` block (ticket 020/047: styles are selectable on equal footing with the
// built-ins, and there's no requirement that this block be the *only* thing the file contains — a
// style is free to also target any other selector, e.g. to restyle layout, spacing, or fonts, not
// just the palette). `useSettingsPanel` applies the selected one as an unscoped stylesheet only
// while no built-in `html.<scheme>` class is present, so it wins the cascade without rewriting the
// selector.
/** Canonical list of every `--c-*` variable every theme in `index.css` defines. */
export const THEME_CSS_VARS = [
'--c-surface-0',
'--c-surface-1',
'--c-surface-2',
'--c-border',
'--c-border-soft',
'--c-border-faint',
'--c-border-strong',
'--c-text',
'--c-text-secondary',
'--c-text-muted',
'--c-text-subtle',
'--c-text-faint',
'--c-text-faintest',
'--c-danger-bg',
'--c-danger-border',
'--c-danger-text',
'--c-danger-text-strong',
'--c-accent',
'--c-accent-soft',
'--c-accent-text',
'--c-accent-2',
'--c-accent-3',
] as const
export type ThemeVars = Record<(typeof THEME_CSS_VARS)[number], string>
// Fully-resolved variable values for each built-in theme, including inherited fallbacks, so a downloaded file is always complete; keep in sync with `index.css` by hand, there's no build step linking the two.
export const BUILTIN_THEMES: Record<string, ThemeVars> = {
dark: {
'--c-surface-0': '#0a0a0a',
'--c-surface-1': '#171717',
'--c-surface-2': '#262626',
'--c-border': '#262626',
'--c-border-soft': '#404040',
'--c-border-faint': '#171717',
'--c-border-strong': '#a3a3a3',
'--c-text': '#f5f5f5',
'--c-text-secondary': '#d4d4d4',
'--c-text-muted': '#a3a3a3',
'--c-text-subtle': '#737373',
'--c-text-faint': '#525252',
'--c-text-faintest': '#404040',
'--c-danger-bg': '#450a0a',
'--c-danger-border': '#7f1d1d',
'--c-danger-text': '#f87171',
'--c-danger-text-strong': '#fca5a5',
'--c-accent': '#0ea5e9',
'--c-accent-soft': '#0369a1',
'--c-accent-text': '#7dd3fc',
'--c-accent-2': '#a78bfa',
'--c-accent-3': '#4ade80',
},
light: {
'--c-surface-0': '#fafafa',
'--c-surface-1': '#f5f5f5',
'--c-surface-2': '#e5e5e5',
'--c-border': '#e5e5e5',
'--c-border-soft': '#d4d4d4',
'--c-border-faint': '#f5f5f5',
'--c-border-strong': '#525252',
'--c-text': '#171717',
'--c-text-secondary': '#404040',
'--c-text-muted': '#525252',
'--c-text-subtle': '#737373',
'--c-text-faint': '#a3a3a3',
'--c-text-faintest': '#d4d4d4',
'--c-danger-bg': '#fee2e2',
'--c-danger-border': '#fca5a5',
'--c-danger-text': '#b91c1c',
'--c-danger-text-strong': '#991b1b',
'--c-accent': '#0284c7',
'--c-accent-soft': '#7dd3fc',
'--c-accent-text': '#0369a1',
'--c-accent-2': '#7c3aed',
'--c-accent-3': '#16a34a',
},
red: {
'--c-surface-0': 'hsl(0 32% 10%)',
'--c-surface-1': 'hsl(0 28% 16%)',
'--c-surface-2': 'hsl(0 26% 22%)',
'--c-border': 'hsl(0 24% 24%)',
'--c-border-soft': 'hsl(0 22% 33%)',
'--c-border-faint': 'hsl(0 28% 16%)',
'--c-border-strong': 'hsl(0 15% 62%)',
'--c-text': 'hsl(0 15% 94%)',
'--c-text-secondary': 'hsl(0 12% 82%)',
'--c-text-muted': 'hsl(0 10% 65%)',
'--c-text-subtle': 'hsl(0 8% 50%)',
'--c-text-faint': 'hsl(0 8% 38%)',
'--c-text-faintest': 'hsl(0 10% 28%)',
'--c-danger-bg': '#450a0a',
'--c-danger-border': '#7f1d1d',
'--c-danger-text': '#f87171',
'--c-danger-text-strong': '#fca5a5',
'--c-accent': 'hsl(180 70% 62%)',
'--c-accent-soft': 'hsl(150 55% 34%)',
'--c-accent-text': 'hsl(180 75% 80%)',
'--c-accent-2': 'hsl(90 65% 58%)',
'--c-accent-3': 'hsl(270 65% 68%)',
},
green: {
'--c-surface-0': 'hsl(140 32% 10%)',
'--c-surface-1': 'hsl(140 28% 16%)',
'--c-surface-2': 'hsl(140 26% 22%)',
'--c-border': 'hsl(140 24% 24%)',
'--c-border-soft': 'hsl(140 22% 33%)',
'--c-border-faint': 'hsl(140 28% 16%)',
'--c-border-strong': 'hsl(140 15% 62%)',
'--c-text': 'hsl(140 15% 94%)',
'--c-text-secondary': 'hsl(140 12% 82%)',
'--c-text-muted': 'hsl(140 10% 65%)',
'--c-text-subtle': 'hsl(140 8% 50%)',
'--c-text-faint': 'hsl(140 8% 38%)',
'--c-text-faintest': 'hsl(140 10% 28%)',
'--c-danger-bg': '#450a0a',
'--c-danger-border': '#7f1d1d',
'--c-danger-text': '#f87171',
'--c-danger-text-strong': '#fca5a5',
'--c-accent': 'hsl(320 70% 62%)',
'--c-accent-soft': 'hsl(290 55% 34%)',
'--c-accent-text': 'hsl(320 75% 80%)',
'--c-accent-2': 'hsl(230 65% 62%)',
'--c-accent-3': 'hsl(50 65% 58%)',
},
blue: {
'--c-surface-0': 'hsl(210 32% 10%)',
'--c-surface-1': 'hsl(210 28% 16%)',
'--c-surface-2': 'hsl(210 26% 22%)',
'--c-border': 'hsl(210 24% 24%)',
'--c-border-soft': 'hsl(210 22% 33%)',
'--c-border-faint': 'hsl(210 28% 16%)',
'--c-border-strong': 'hsl(210 15% 62%)',
'--c-text': 'hsl(210 15% 94%)',
'--c-text-secondary': 'hsl(210 12% 82%)',
'--c-text-muted': 'hsl(210 10% 65%)',
'--c-text-subtle': 'hsl(210 8% 50%)',
'--c-text-faint': 'hsl(210 8% 38%)',
'--c-text-faintest': 'hsl(210 10% 28%)',
'--c-danger-bg': '#450a0a',
'--c-danger-border': '#7f1d1d',
'--c-danger-text': '#f87171',
'--c-danger-text-strong': '#fca5a5',
'--c-accent': 'hsl(30 70% 62%)',
'--c-accent-soft': 'hsl(0 55% 34%)',
'--c-accent-text': 'hsl(30 75% 80%)',
'--c-accent-2': 'hsl(300 65% 68%)',
'--c-accent-3': 'hsl(120 65% 55%)',
},
yellow: {
'--c-surface-0': 'hsl(50 32% 10%)',
'--c-surface-1': 'hsl(50 28% 16%)',
'--c-surface-2': 'hsl(50 26% 22%)',
'--c-border': 'hsl(50 24% 24%)',
'--c-border-soft': 'hsl(50 22% 33%)',
'--c-border-faint': 'hsl(50 28% 16%)',
'--c-border-strong': 'hsl(50 15% 62%)',
'--c-text': 'hsl(50 15% 94%)',
'--c-text-secondary': 'hsl(50 12% 82%)',
'--c-text-muted': 'hsl(50 10% 65%)',
'--c-text-subtle': 'hsl(50 8% 50%)',
'--c-text-faint': 'hsl(50 8% 38%)',
'--c-text-faintest': 'hsl(50 10% 28%)',
'--c-danger-bg': '#450a0a',
'--c-danger-border': '#7f1d1d',
'--c-danger-text': '#f87171',
'--c-danger-text-strong': '#fca5a5',
'--c-accent': 'hsl(230 70% 62%)',
'--c-accent-soft': 'hsl(200 55% 34%)',
'--c-accent-text': 'hsl(230 75% 80%)',
'--c-accent-2': 'hsl(140 65% 55%)',
'--c-accent-3': 'hsl(320 65% 68%)',
},
pink: {
'--c-surface-0': 'hsl(330 32% 10%)',
'--c-surface-1': 'hsl(330 28% 16%)',
'--c-surface-2': 'hsl(330 26% 22%)',
'--c-border': 'hsl(330 24% 24%)',
'--c-border-soft': 'hsl(330 22% 33%)',
'--c-border-faint': 'hsl(330 28% 16%)',
'--c-border-strong': 'hsl(330 15% 62%)',
'--c-text': 'hsl(330 15% 94%)',
'--c-text-secondary': 'hsl(330 12% 82%)',
'--c-text-muted': 'hsl(330 10% 65%)',
'--c-text-subtle': 'hsl(330 8% 50%)',
'--c-text-faint': 'hsl(330 8% 38%)',
'--c-text-faintest': 'hsl(330 10% 28%)',
'--c-danger-bg': '#450a0a',
'--c-danger-border': '#7f1d1d',
'--c-danger-text': '#f87171',
'--c-danger-text-strong': '#fca5a5',
'--c-accent': 'hsl(150 70% 62%)',
'--c-accent-soft': 'hsl(120 55% 34%)',
'--c-accent-text': 'hsl(150 75% 80%)',
'--c-accent-2': 'hsl(60 65% 58%)',
'--c-accent-3': 'hsl(240 65% 68%)',
},
orange: {
'--c-surface-0': 'hsl(30 32% 10%)',
'--c-surface-1': 'hsl(30 28% 16%)',
'--c-surface-2': 'hsl(30 26% 22%)',
'--c-border': 'hsl(30 24% 24%)',
'--c-border-soft': 'hsl(30 22% 33%)',
'--c-border-faint': 'hsl(30 28% 16%)',
'--c-border-strong': 'hsl(30 15% 62%)',
'--c-text': 'hsl(30 15% 94%)',
'--c-text-secondary': 'hsl(30 12% 82%)',
'--c-text-muted': 'hsl(30 10% 65%)',
'--c-text-subtle': 'hsl(30 8% 50%)',
'--c-text-faint': 'hsl(30 8% 38%)',
'--c-text-faintest': 'hsl(30 10% 28%)',
'--c-danger-bg': '#450a0a',
'--c-danger-border': '#7f1d1d',
'--c-danger-text': '#f87171',
'--c-danger-text-strong': '#fca5a5',
'--c-accent': 'hsl(210 70% 62%)',
'--c-accent-soft': 'hsl(180 55% 34%)',
'--c-accent-text': 'hsl(210 75% 80%)',
'--c-accent-2': 'hsl(120 65% 55%)',
'--c-accent-3': 'hsl(300 65% 68%)',
},
light_green: {
'--c-surface-0': 'hsl(140 40% 96%)',
'--c-surface-1': 'hsl(140 35% 92%)',
'--c-surface-2': 'hsl(140 30% 85%)',
'--c-border': 'hsl(140 25% 80%)',
'--c-border-soft': 'hsl(140 22% 70%)',
'--c-border-faint': 'hsl(140 35% 92%)',
'--c-border-strong': 'hsl(140 20% 40%)',
'--c-text': 'hsl(140 30% 12%)',
'--c-text-secondary': 'hsl(140 20% 25%)',
'--c-text-muted': 'hsl(140 15% 40%)',
'--c-text-subtle': 'hsl(140 10% 50%)',
'--c-text-faint': 'hsl(140 10% 65%)',
'--c-text-faintest': 'hsl(140 12% 80%)',
'--c-danger-bg': '#fee2e2',
'--c-danger-border': '#fca5a5',
'--c-danger-text': '#b91c1c',
'--c-danger-text-strong': '#991b1b',
'--c-accent': 'hsl(320 70% 40%)',
'--c-accent-soft': 'hsl(290 60% 85%)',
'--c-accent-text': 'hsl(320 75% 28%)',
'--c-accent-2': 'hsl(230 70% 45%)',
'--c-accent-3': 'hsl(50 70% 38%)',
},
light_red: {
'--c-surface-0': 'hsl(0 40% 96%)',
'--c-surface-1': 'hsl(0 35% 92%)',
'--c-surface-2': 'hsl(0 30% 85%)',
'--c-border': 'hsl(0 25% 80%)',
'--c-border-soft': 'hsl(0 22% 70%)',
'--c-border-faint': 'hsl(0 35% 92%)',
'--c-border-strong': 'hsl(0 20% 40%)',
'--c-text': 'hsl(0 30% 12%)',
'--c-text-secondary': 'hsl(0 20% 25%)',
'--c-text-muted': 'hsl(0 15% 40%)',
'--c-text-subtle': 'hsl(0 10% 50%)',
'--c-text-faint': 'hsl(0 10% 65%)',
'--c-text-faintest': 'hsl(0 12% 80%)',
'--c-danger-bg': '#fee2e2',
'--c-danger-border': '#fca5a5',
'--c-danger-text': '#b91c1c',
'--c-danger-text-strong': '#991b1b',
'--c-accent': 'hsl(180 70% 40%)',
'--c-accent-soft': 'hsl(150 60% 85%)',
'--c-accent-text': 'hsl(180 75% 28%)',
'--c-accent-2': 'hsl(90 70% 45%)',
'--c-accent-3': 'hsl(270 70% 38%)',
},
light_blue: {
'--c-surface-0': 'hsl(210 40% 96%)',
'--c-surface-1': 'hsl(210 35% 92%)',
'--c-surface-2': 'hsl(210 30% 85%)',
'--c-border': 'hsl(210 25% 80%)',
'--c-border-soft': 'hsl(210 22% 70%)',
'--c-border-faint': 'hsl(210 35% 92%)',
'--c-border-strong': 'hsl(210 20% 40%)',
'--c-text': 'hsl(210 30% 12%)',
'--c-text-secondary': 'hsl(210 20% 25%)',
'--c-text-muted': 'hsl(210 15% 40%)',
'--c-text-subtle': 'hsl(210 10% 50%)',
'--c-text-faint': 'hsl(210 10% 65%)',
'--c-text-faintest': 'hsl(210 12% 80%)',
'--c-danger-bg': '#fee2e2',
'--c-danger-border': '#fca5a5',
'--c-danger-text': '#b91c1c',
'--c-danger-text-strong': '#991b1b',
'--c-accent': 'hsl(30 70% 40%)',
'--c-accent-soft': 'hsl(0 60% 85%)',
'--c-accent-text': 'hsl(30 75% 28%)',
'--c-accent-2': 'hsl(300 70% 45%)',
'--c-accent-3': 'hsl(120 70% 38%)',
},
light_yellow: {
'--c-surface-0': 'hsl(50 40% 96%)',
'--c-surface-1': 'hsl(50 35% 92%)',
'--c-surface-2': 'hsl(50 30% 85%)',
'--c-border': 'hsl(50 25% 80%)',
'--c-border-soft': 'hsl(50 22% 70%)',
'--c-border-faint': 'hsl(50 35% 92%)',
'--c-border-strong': 'hsl(50 20% 40%)',
'--c-text': 'hsl(50 30% 12%)',
'--c-text-secondary': 'hsl(50 20% 25%)',
'--c-text-muted': 'hsl(50 15% 40%)',
'--c-text-subtle': 'hsl(50 10% 50%)',
'--c-text-faint': 'hsl(50 10% 65%)',
'--c-text-faintest': 'hsl(50 12% 80%)',
'--c-danger-bg': '#fee2e2',
'--c-danger-border': '#fca5a5',
'--c-danger-text': '#b91c1c',
'--c-danger-text-strong': '#991b1b',
'--c-accent': 'hsl(230 70% 40%)',
'--c-accent-soft': 'hsl(200 60% 85%)',
'--c-accent-text': 'hsl(230 75% 28%)',
'--c-accent-2': 'hsl(140 70% 45%)',
'--c-accent-3': 'hsl(320 70% 38%)',
},
light_pink: {
'--c-surface-0': 'hsl(330 40% 96%)',
'--c-surface-1': 'hsl(330 35% 92%)',
'--c-surface-2': 'hsl(330 30% 85%)',
'--c-border': 'hsl(330 25% 80%)',
'--c-border-soft': 'hsl(330 22% 70%)',
'--c-border-faint': 'hsl(330 35% 92%)',
'--c-border-strong': 'hsl(330 20% 40%)',
'--c-text': 'hsl(330 30% 12%)',
'--c-text-secondary': 'hsl(330 20% 25%)',
'--c-text-muted': 'hsl(330 15% 40%)',
'--c-text-subtle': 'hsl(330 10% 50%)',
'--c-text-faint': 'hsl(330 10% 65%)',
'--c-text-faintest': 'hsl(330 12% 80%)',
'--c-danger-bg': '#fee2e2',
'--c-danger-border': '#fca5a5',
'--c-danger-text': '#b91c1c',
'--c-danger-text-strong': '#991b1b',
'--c-accent': 'hsl(150 70% 40%)',
'--c-accent-soft': 'hsl(120 60% 85%)',
'--c-accent-text': 'hsl(150 75% 28%)',
'--c-accent-2': 'hsl(60 70% 45%)',
'--c-accent-3': 'hsl(240 70% 38%)',
},
light_orange: {
'--c-surface-0': 'hsl(30 40% 96%)',
'--c-surface-1': 'hsl(30 35% 92%)',
'--c-surface-2': 'hsl(30 30% 85%)',
'--c-border': 'hsl(30 25% 80%)',
'--c-border-soft': 'hsl(30 22% 70%)',
'--c-border-faint': 'hsl(30 35% 92%)',
'--c-border-strong': 'hsl(30 20% 40%)',
'--c-text': 'hsl(30 30% 12%)',
'--c-text-secondary': 'hsl(30 20% 25%)',
'--c-text-muted': 'hsl(30 15% 40%)',
'--c-text-subtle': 'hsl(30 10% 50%)',
'--c-text-faint': 'hsl(30 10% 65%)',
'--c-text-faintest': 'hsl(30 12% 80%)',
'--c-danger-bg': '#fee2e2',
'--c-danger-border': '#fca5a5',
'--c-danger-text': '#b91c1c',
'--c-danger-text-strong': '#991b1b',
'--c-accent': 'hsl(210 70% 40%)',
'--c-accent-soft': 'hsl(180 60% 85%)',
'--c-accent-text': 'hsl(210 75% 28%)',
'--c-accent-2': 'hsl(120 70% 45%)',
'--c-accent-3': 'hsl(300 70% 38%)',
},
}
export function generateThemeCss(vars: ThemeVars): string {
const lines = THEME_CSS_VARS.map((name) => ` ${name}: ${vars[name]};`)
return `:root {\n${lines.join('\n')}\n}\n`
}
export interface ParsedTheme {
vars: ThemeVars
missing: string[]
}
// Parses `--c-*: value;` declarations from the first `:root { ... }` block (or the whole file, tolerant of a dropped selector); reports missing canonical variables so callers can reject an incomplete upload.
export function parseThemeCss(content: string): ParsedTheme {
const rootMatch = content.match(/:root\s*\{([^}]*)\}/)
const body = rootMatch ? rootMatch[1] : content
const found: Record<string, string> = {}
const declRegex = /(--c-[a-z0-9-]+)\s*:\s*([^;]+);/gi
let match: RegExpExecArray | null
while ((match = declRegex.exec(body))) {
found[match[1]] = match[2].trim()
}
const missing = THEME_CSS_VARS.filter((name) => !(name in found))
const vars = Object.fromEntries(THEME_CSS_VARS.map((name) => [name, found[name] ?? ''])) as ThemeVars
return { vars, missing }
}
export function downloadThemeFile(filename: string, content: string) {
const blob = new Blob([content], { type: 'text/css' })
const url = URL.createObjectURL(blob)
const link = document.createElement('a')
link.href = url
link.download = filename
document.body.appendChild(link)
link.click()
link.remove()
URL.revokeObjectURL(url)
}
+25
View File
@@ -0,0 +1,25 @@
export interface TimeRange {
startMs: number
endMs: number
}
// `'all'` clears the range (the implicit default); the rest resolve to a window ending "now" at the moment they're (re-)computed — see `computePresetRange`.
export type TimeRangePresetId = 'all' | '1m' | '2m' | '5m' | '15m' | '1h' | '24h'
export const TIME_RANGE_PRESETS: { id: TimeRangePresetId; label: string; ms: number | null }[] = [
{ id: 'all', label: 'All', ms: null },
{ id: '1m', label: '1m', ms: 60 * 1000 },
{ id: '2m', label: '2m', ms: 2 * 60 * 1000 },
{ id: '5m', label: '5m', ms: 5 * 60 * 1000 },
{ id: '15m', label: '15m', ms: 15 * 60 * 1000 },
{ id: '1h', label: '1h', ms: 60 * 60 * 1000 },
{ id: '24h', label: '24h', ms: 24 * 60 * 60 * 1000 },
]
// Callers re-invoke this on their existing poll cadence (see `App.tsx`) to keep a rolling preset's window moving with time, rather than freezing it at selection time.
export function computePresetRange(id: TimeRangePresetId): TimeRange | null {
const preset = TIME_RANGE_PRESETS.find((p) => p.id === id)
if (!preset || preset.ms == null) return null
const now = Date.now()
return { startMs: now - preset.ms, endMs: now }
}
+14
View File
@@ -0,0 +1,14 @@
import { useEffect, type RefObject } from 'react'
// Shared by popover-style UI (RegexHelpIcon, LogPreview's ColumnPicker) that closes itself when a
// mousedown lands outside its container ref, while `active`.
export function useClickOutside(ref: RefObject<HTMLElement | null>, onOutside: () => void, active: boolean) {
useEffect(() => {
if (!active) return
function onClickOutside(e: MouseEvent) {
if (ref.current && !ref.current.contains(e.target as Node)) onOutside()
}
document.addEventListener('mousedown', onClickOutside)
return () => document.removeEventListener('mousedown', onClickOutside)
}, [active, ref, onOutside])
}
+48
View File
@@ -0,0 +1,48 @@
import { useEffect, useRef, useState } from 'react'
import { computePresetRange, type TimeRange, type TimeRangePresetId } from './timeRange'
// `resolveEffectiveRange` is meant to be called once at the top of each variant's own poll
// callback: it re-resolves a rolling preset to "now" on every tick (instead of freezing the
// window at selection time) while reading a manually-selected fixed range via a ref, so the poll
// loop's own `useEffect` doesn't need `range` itself as a dependency and restart its interval on
// every tick.
export function useRangeSelection(initial: { preset: TimeRangePresetId | null; range: TimeRange | null }) {
const [range, setRange] = useState<TimeRange | null>(
initial.preset ? computePresetRange(initial.preset) : initial.range,
)
const [activePreset, setActivePreset] = useState<TimeRangePresetId | null>(initial.preset)
const rangeRef = useRef(range)
useEffect(() => {
rangeRef.current = range
}, [range])
function selectPreset(id: TimeRangePresetId) {
setActivePreset(id)
setRange(computePresetRange(id))
}
// A drag always yields a fixed (non-rolling) range, overriding any active preset.
function handleManualRangeSelect(next: TimeRange | null) {
if (next === null) {
selectPreset('all')
} else {
setActivePreset(null)
setRange(next)
}
}
function resolveEffectiveRange(): TimeRange | null {
const effectiveRange = activePreset && activePreset !== 'all' ? computePresetRange(activePreset) : rangeRef.current
if (activePreset && activePreset !== 'all') {
setRange((current) =>
current && effectiveRange && current.startMs === effectiveRange.startMs && current.endMs === effectiveRange.endMs
? current
: effectiveRange,
)
}
return effectiveRange
}
return { range, activePreset, selectPreset, handleManualRangeSelect, resolveEffectiveRange }
}
+41 -117
View File
@@ -2,75 +2,49 @@ import { useCallback, useEffect, useState } from 'react'
import { import {
activateProfile, activateProfile,
createProfile, createProfile,
cssContentUrl,
deleteCssEntry,
deleteProfile, deleteProfile,
fetchCssEntries,
fetchProfiles, fetchProfiles,
fetchSettings, fetchSettings,
importCss, fetchThemes,
linkCss, themeCssUrl,
updateSettings, updateSettings,
type ColorScheme,
type CssEntry,
type ProfileSummary, type ProfileSummary,
type Settings, type Settings,
type Theme,
} from './api' } from './api'
// `data-*` marker for the single injected `<link>` for the active custom theme, if any. // `data-*` marker for the single injected `<link>` for the active theme's stylesheet.
const INJECTED_THEME_ATTR = 'data-ottyl-theme-css-id' const INJECTED_THEME_ATTR = 'data-ottyl-theme-css'
// Themes selectable independently of `system`, each backed by a `html.<name>` block in `index.css`. // `"system"` isn't an embedded theme file — it resolves to one of these two at apply time, based
const THEME_CLASSES = [ // on the OS light/dark preference.
'light', const SYSTEM_LIGHT_THEME = 'light'
'dark', const SYSTEM_DARK_THEME = 'dark'
'red',
'green',
'blue',
'yellow',
'pink',
'orange',
'light_green',
'light_red',
'light_blue',
'light_yellow',
'light_pink',
'light_orange',
] as const
// Reflects `colorScheme` onto `<html>` as a class + `data-color-scheme`; when a custom theme is active, no built-in class is added at all, since a class selector would out-specificity its injected `:root` override (see `applyCustomThemeCss`) and mask it. function resolveTheme(theme: Theme): string {
function applyColorScheme(colorScheme: ColorScheme, customThemeActive: boolean) { if (theme !== 'system') return theme
const root = document.documentElement
root.classList.remove(...THEME_CLASSES)
if (customThemeActive) {
root.dataset.colorScheme = 'custom'
return
}
const prefersDark = window.matchMedia?.('(prefers-color-scheme: dark)').matches ?? true const prefersDark = window.matchMedia?.('(prefers-color-scheme: dark)').matches ?? true
const effective = colorScheme === 'system' ? (prefersDark ? 'dark' : 'light') : colorScheme return prefersDark ? SYSTEM_DARK_THEME : SYSTEM_LIGHT_THEME
root.classList.add(effective)
root.dataset.colorScheme = colorScheme
} }
// Injects a `<link>` for the active custom theme entry, if any (ticket 020: every CSS entry is // Injects a `<link>` for the resolved theme's stylesheet, replacing any previously-injected one.
// always a selectable theme now, so this is the only injection path — there's no more // `webapp/src/index.css`'s `:root` block is the only styling until this loads, so there's no
// always-on-overlay mode). `entry` null just removes any previously-injected tag. // flash of unstyled content on first paint.
function applyCustomThemeCss(entry: CssEntry | null) { function applyTheme(theme: Theme) {
const resolved = resolveTheme(theme)
document.querySelectorAll(`link[${INJECTED_THEME_ATTR}]`).forEach((el) => el.remove()) document.querySelectorAll(`link[${INJECTED_THEME_ATTR}]`).forEach((el) => el.remove())
if (!entry) return
const href = entry.kind === 'imported' ? cssContentUrl(entry.id) : entry.path_or_url
if (!href) return
const link = document.createElement('link') const link = document.createElement('link')
link.rel = 'stylesheet' link.rel = 'stylesheet'
link.href = href link.href = themeCssUrl(resolved)
link.setAttribute(INJECTED_THEME_ATTR, entry.id) link.setAttribute(INJECTED_THEME_ATTR, resolved)
document.head.appendChild(link) document.head.appendChild(link)
} }
// Owns fetching + mutating settings/CSS/profile state for the NavMenu panels, and applies colour scheme / imported CSS to the document as a side effect so changes are visible immediately, with no reload. // Owns fetching + mutating settings/theme/profile state for the NavMenu panels, and applies the
// selected theme to the document as a side effect so changes are visible immediately, with no reload.
export function useSettingsPanel() { export function useSettingsPanel() {
const [settings, setSettings] = useState<Settings | null>(null) const [settings, setSettings] = useState<Settings | null>(null)
const [cssEntries, setCssEntries] = useState<CssEntry[]>([]) const [themes, setThemes] = useState<string[]>([])
const [profiles, setProfiles] = useState<ProfileSummary[]>([]) const [profiles, setProfiles] = useState<ProfileSummary[]>([])
const [error, setError] = useState<string | null>(null) const [error, setError] = useState<string | null>(null)
@@ -80,9 +54,9 @@ export function useSettingsPanel() {
return next return next
}, []) }, [])
const refreshCss = useCallback(async () => { const refreshThemes = useCallback(async () => {
const next = await fetchCssEntries() const next = await fetchThemes()
setCssEntries(next) setThemes(next)
return next return next
}, []) }, [])
@@ -94,21 +68,20 @@ export function useSettingsPanel() {
useEffect(() => { useEffect(() => {
refreshSettings().catch(() => {}) refreshSettings().catch(() => {})
refreshCss().catch(() => {}) refreshThemes().catch(() => {})
refreshProfiles().catch(() => {}) refreshProfiles().catch(() => {})
}, [refreshSettings, refreshCss, refreshProfiles]) }, [refreshSettings, refreshThemes, refreshProfiles])
const activeCustomTheme = settings?.custom_theme_id
? cssEntries.find((entry) => entry.id === settings.custom_theme_id) ?? null
: null
useEffect(() => { useEffect(() => {
if (settings) applyColorScheme(settings.color_scheme, activeCustomTheme !== null) if (!settings) return
}, [settings, activeCustomTheme]) applyTheme(settings.theme)
if (settings.theme !== 'system') return
useEffect(() => { // Re-resolve if the OS preference changes while "system" is selected.
applyCustomThemeCss(activeCustomTheme) const media = window.matchMedia?.('(prefers-color-scheme: dark)')
}, [activeCustomTheme]) const onChange = () => applyTheme(settings.theme)
media?.addEventListener('change', onChange)
return () => media?.removeEventListener('change', onChange)
}, [settings])
const guarded = useCallback((action: () => Promise<void>) => { const guarded = useCallback((action: () => Promise<void>) => {
return async () => { return async () => {
@@ -121,20 +94,10 @@ export function useSettingsPanel() {
} }
}, []) }, [])
const setColorScheme = useCallback( const setTheme = useCallback(
(colorScheme: ColorScheme) => (theme: Theme) =>
guarded(async () => { guarded(async () => {
// Selecting a built-in scheme always deselects any custom theme — the two are mutually exclusive (see `applyColorScheme`). const next = await updateSettings({ theme })
const next = await updateSettings({ colorScheme, customThemeId: null })
setSettings(next)
})(),
[guarded],
)
const setCustomTheme = useCallback(
(id: string) =>
guarded(async () => {
const next = await updateSettings({ customThemeId: id })
setSettings(next) setSettings(next)
})(), })(),
[guarded], [guarded],
@@ -158,40 +121,6 @@ export function useSettingsPanel() {
[guarded], [guarded],
) )
// Every entry is always a selectable style, on equal footing with the built-in colour schemes
// (ticket 020 unified the data model; ticket 047 unified the UI), so there's a single import path.
const importCssEntry = useCallback(
(filename: string, content: string) =>
guarded(async () => {
await importCss(filename, content)
await refreshCss()
})(),
[guarded, refreshCss],
)
const linkCssEntry = useCallback(
(pathOrUrl: string) =>
guarded(async () => {
await linkCss(pathOrUrl)
await refreshCss()
})(),
[guarded, refreshCss],
)
const removeCssEntry = useCallback(
(id: string) =>
guarded(async () => {
await deleteCssEntry(id)
await refreshCss()
// The removed entry may have been the active custom theme — fall back to `color_scheme` rather than leaving a dangling selection.
if (settings?.custom_theme_id === id) {
const next = await updateSettings({ customThemeId: null })
setSettings(next)
}
})(),
[guarded, refreshCss, settings],
)
const addProfile = useCallback( const addProfile = useCallback(
(name: string) => (name: string) =>
guarded(async () => { guarded(async () => {
@@ -221,17 +150,12 @@ export function useSettingsPanel() {
return { return {
settings, settings,
cssEntries, themes,
activeCustomTheme,
profiles, profiles,
error, error,
setColorScheme, setTheme,
setCustomTheme,
setMaxLogs, setMaxLogs,
setNtfySink, setNtfySink,
importCssEntry,
linkCssEntry,
removeCssEntry,
addProfile, addProfile,
removeProfile, removeProfile,
switchProfile, switchProfile,