Ticket 108 anticipated a large test-harness migration (primary-only/
profile-only helpers, migrating ~150 tests in engine/tests.rs), but
that turned out unnecessary: the profile actor reuses EngineSnapshot
as its own snapshot type (see ticket 104), so every existing
EngineSnapshot unit test already exercises primary-shaped and
profile-shaped behavior unchanged -- zero migration needed, confirmed
by the full suite passing untouched through tickets 100-107.
What was still a real gap: tests asserting on profile-actor-owned data
(declared metrics, alerts) after ingesting had no deterministic way to
wait for the profile actor to catch up -- EngineHandle::barrier() only
confirms the primary's queue drained, not that the profile actor has
processed what it published. They were passing on scheduler luck on
the single-threaded test runtime, not a real guarantee.
Added ProfileHandle::processed_sequence, a permanent (not test-only)
watch channel tracking the profile actor's watermark, mirrored onto
EngineHandle as profile_processed_sequence, with a
wait_for_profile_sequence(sequence) test helper built on it. This
replaces the ad hoc on_advance/spawn_observed mechanism ticket 103
added just to prove the wake loop worked -- that test now uses the
same real primitive. Retrofitted it into
promql_query_returns_series_for_seeded_fixture to eliminate its
latent race.
Not done: a dedicated test for the accepted eviction/lag tradeoff
(primary evicts on schedule regardless of profile-actor progress, per
idea 0001's resolved decision) -- deterministically forcing that race
without new test-injection hooks proved more involved than the
remaining scope justified; left as a known gap rather than shipping a
flaky or synthetic test for it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
Audited every EngineQuery variant for the cross-snapshot join ticket
107 anticipated: Logs/Timeline/Stats/ParamDeclarations only ever read
primary data (raw text + param declarations, which never left the
primary), and MetricDeclarations/BucketValues/Promql only ever read
profile data -- no variant actually needs both, so no join/fan-out
layer is needed after all.
One real gap surfaced by that audit: a bare ottyl_seq/ottyl_ts Promql
query is evaluated by iterating self.logs_range() (evaluate_promql's
reserved-metric branch), but such queries were routed to the profile
actor's snapshot alongside real declared metrics -- and the profile
actor stores no raw log rows, so these queries silently returned empty
series instead of real data. EngineHandle::ask now parses the query
just far enough to check its metric name and routes ottyl_seq/ottyl_ts
to the primary instead, where evaluate_promql already works correctly
against real log data. Added an actor-level integration test
(promql_bare_ottyl_seq_works_through_the_real_actor_pipeline) that
fails without this fix -- verified by temporarily disabling the
routing check and confirming the test catches it.
Also deduplicated config/profile.rs's local `ask` test helper, which
had already once diverged from api::EngineHandle::ask's routing logic
and caused ticket 104 test failures (fixed then, but the duplication
itself was the root cause) -- it now delegates to the real one.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
Ticket 106 asked for a real way for API handlers to address the
primary vs. profile actor. That routing was already built out of
necessity while landing tickets 104/105 (metric queries had to reach
the profile actor's snapshot for those tickets' own tests to pass) --
EngineHandle ended up as one combined handle (option (b) from the
ticket's own scope note) that internally proxies metric/alert writes
to the profile actor and routes MetricDeclarations/BucketValues/Promql
reads to its snapshot, with no changes needed to any API handler,
reload_engine_declarations/activate_profile, or Profile::load_into_engine
-- they already just send EngineMessages the same way they did before
the split. mcp_server.rs talks to the HTTP API rather than EngineHandle
directly, so it needed no changes either. This commit documents that
resolution in handle.rs so a future reader doesn't go looking for
routing logic that isn't there.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
Alert rules are profile-scoped and read metric buckets, which now live
on the profile actor (ticket 104) rather than the primary. alert_states
and the per-line evaluate_alerts/deliver_for_rule calls move from the
primary's Ingest arm to the profile actor's per-line loop, evaluated
against its own snapshot right after that line's apply_all_metrics
publishes -- matching the primary's original per-message publish-then-
evaluate ordering. The primary's run() no longer needs a SettingsHandle
at all now that alerts are gone from its Ingest arm; settings (for
alert rules and ntfy config, read fresh per line) moves to
engine::profile::spawn instead.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
Metric declare/update/remove, backfill, and per-line apply_all_metrics
now run on the profile actor (engine::profile) instead of inline in the
primary's ingest closure, per idea 0001's resolved decision that the
primary handles params only. The profile actor reuses EngineSnapshot as
its own snapshot type -- declare_metric/update_metric/apply_all_metrics
work unchanged -- and mirrors the primary's param declarations into its
own snapshot on every wake (and synchronously before handling a
DeclareMetric/UpdateMetric message, since there's no ordering guarantee
between the watch-wake and message branches in tokio::select!) so
metric label/value resolution keeps working without storing a second
copy of raw log text; backfill reads lines through its cloned primary
IndexReader instead.
EngineHandle gained a profile_reader field so MetricDeclarations/
BucketValues/Promql queries route to the profile actor's snapshot
instead of the primary's -- without it, metrics could be declared but
never queried, since the primary no longer computes or stores them.
DeclareMetric/UpdateMetric/RemoveMetricDeclaration messages are still
sent to the primary's mpsc channel and proxied to the profile actor by
handle.rs's run() loop; EngineHandle's write-side API is otherwise
unchanged (ticket 106 gives API handlers a direct route). Alert
evaluation still reads the primary's (now metric-less) snapshot
directly -- an accepted gap until ticket 105 moves it, per idea 0001's
resolved decision that alerts move with metric computation.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
Adds a second actor task with its own left_right-backed ProfileSnapshot
and a tokio::sync::watch the primary's IndexWriter bumps after every
publish, so the profile actor wakes and re-reads newly-visible primary
lines through its own cloned IndexReader instead of polling. No metric
computation, alert evaluation, or query routing moves yet -- this is
pure plumbing, proven with a test that ingests a line on the primary
and observes the profile actor's watermark advance in response.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
Ticket 102 (splitting the engine into reader/primary/profile actors)
was a single multi-thousand-line ticket covering a new actor
topology, an EngineHandle API redesign, a query fan-out layer, moving
alerts, and a full test-harness rewrite, with some design decisions
deliberately left open. Splitting it into six sequenced tickets
(profile actor skeleton + signaling, move metric computation, move
alert evaluation, split the handle API, query fan-out, test harness
rework) lets each land as its own reviewable commit instead of one
large speculative rewrite.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
Raw storage, log-stream header derivation, and auto-param detection
were inlined directly in the ingest actor's write closure alongside
metric computation. Pulling them into their own method gives the
ingest arm two clearly separated, independently-callable stages —
param extraction (ingest_params) and metric computation
(apply_all_metrics) — with no behavior change, in preparation for
splitting these onto separate actors (ticket 102).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
DataValue::LogLiteral and IngestedLine now carry an Arc<str> instead
of an owned String, so the two clones the ingest path paid per line
(once to move text into the write closure, once to store it) become
cheap refcount bumps instead of full string copies. Also cuts the
per-string-content cost of left_right's full-EngineSnapshot deep clone
on each reconciliation sync.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
The PromQL layer only had a fixed-width histogram() (per-bucket
counts), with no way to ask for p50/p95/p99. Adds
histogram_quantile(p, metric[range]), interpolating over each
series's retained raw sample values (linear interpolation between
closest ranks) rather than building a separate pre-aggregated
bucketed-histogram representation — exact for the data actually kept.
An optional bracketed range narrows the computation to a trailing
window, mirroring rate()'s range selector.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
Metrics previously only declared a data type (String/Number/...), so
rate() could be applied to a point-in-time gauge reading and sum()/avg()
could be applied directly to a cumulative counter reading with no guard
against either. Adds a MetricSemantics enum (Gauge, default; Counter)
threaded through DeclareMetricRequest, the engine's MetricDeclaration
data value, profile persistence, the API/MCP declaration routes, and the
Metrics panel's declaration form, then uses it in evaluate_promql to
reject rate() on a declared gauge and bare sum()/avg() on a declared
counter (suggesting rate() first). The guard only applies to metrics
with a captured value_param — bare occurrence-count metrics are always
safe under both operations, so existing dashboards are unaffected.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
Formats the whole backend to rustfmt defaults and fixes all 23
outstanding clippy warnings (collapsible ifs, clone-to-slice-ref,
type-complexity, single-arm match, unnecessary is_none check), then
wires cargo fmt --check and -D warnings into make lint so both stay
enforced going forward.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
A log line whose text parses as JSON was rendering every nested node
fully expanded, dumping large payloads into the timeline instead of a
collapsed summary the user opts into expanding.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
api.ts had grown to 35 exported symbols in one 432-line file covering
every backend endpoint. Split into webapp/src/lib/api/{shared,logs,
params,metrics,dashboards,alerts,settings}.ts, re-exported from
lib/api/index.ts as a barrel so none of the ~10 importing files needed
their import paths changed (TS/bundler module resolution falls back to
a directory's index.ts when the sibling .ts file is gone).
Also added buildSearchParams to replace fetchLogs/fetchTimeline's
repeated hand-written `if (x !== undefined) search.set(...)` blocks.
Left fetchMetricBucketValues/fetchPromqlSeries's query-building
untouched (byte-for-byte) since the ticket scoped the helper to
fetchLogs/fetchTimeline only.
Reimplemented directly against current main (ticket 089) after the
worktree agent's branch was based on a stale pre-theme-removal state
and reintroduced CSS-upload endpoints main had deliberately deleted.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
StandaloneRanged, StandaloneLog, and FullApp each reimplemented the same
range/preset state machine (range/activePreset state, a rangeRef synced
via effect, selectPreset/handleManualRangeSelect, and rolling-preset
re-resolution at the top of the poll loop). Move that bookkeeping into
webapp/src/lib/useRangeSelection.ts; each component keeps only its own
polling logic.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
Replace the 4-deep nested ternary building each declared-param badge's
suffix text with a small if/else-if helper (paramBadgeSuffix), called
from the badge JSX. No behavior change.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
Params/Metrics/Alerts/Dashboards each hand-assembled the same collapsible-
section header wiring (role/tabIndex/aria-expanded/onClick/onKeyDown around
CollapseToggle+HelpIcon+PopOutButton), and Dashboards duplicated it a second
time for its no-active-profile branch. Params/Metrics/Alerts also duplicated
the declare/edit form footer (editing banner, submit label toggle, Cancel
button). Added SectionHeader and FormFooter to CollapsibleSection.tsx and
adopted them at all five call sites; pure extraction, no behavior change.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
CollapsibleSection.tsx and LogTimeline.tsx imported from each other
(TimeRange/TimeRangePresetId one way, sectionHeaderClassName the
other). Move the range types/helpers into a dependency-free
webapp/src/lib/timeRange.ts and repoint all importers there.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
declare_metric and update_metric duplicated the resolve-params +
backfill_metric + insert-MetricDeclaration sequence; extracted into
resolve_backfill_and_store_metric so the declare-vs-rename difference
is expressed only in caller-side logic. Also extracted
auto_declare_params's per-candidate insertion body into
try_insert_auto_param so the candidate-building and insertion steps
read as separate concerns.
Reimplemented directly against current main (ticket 085) after the
original worktree agent's branch was based on a stale pre-consolidation
snapshot module layout that no longer matches main.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
The Logs and Timeline arms of handle_query_inner each duplicated the same
"compile filter, bail out on error" block; the Logs arm also inlined its
sort-helper-index walk. Pull both into named helpers (compile_optional_filter,
logs_sorted_by_param) so the Logs arm reads as "compile filter, then dispatch
to sorted or unsorted." Pure decomposition, no behavior change.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
run() bundled settings loading, engine spawn, active-profile resolve/load,
ingest spawn, and session registration inline. Extract each into a named
function (spawn_engine, load_active_profile, spawn_ingest,
register_session_and_log) so run() reads as a sequence of steps. No
behavior change.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
Extract fetch_metrics_body(client, url) -> Result<String, String> to
handle the request/error_for_status/text chain with `?`, centralizing
the three warning messages. run_metrics_poll now matches once on the
result instead of nesting three match arms deep on the happy path.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
They were scattered across filter/metrics/params/query/sort.rs by an
earlier file-splitting pass. Files can be split as much as needed, but
a single type's impl should never be split across files — so all 15
impl EngineSnapshot blocks now live in mod.rs, while each submodule
keeps its own types, free helper functions, and tests.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
Both tickets explicitly scope themselves to "confirm via profiling
before implementing" (079: a trigram/token pre-filter for free-text
regex search; 080: caching parsed JSON per log line for JSON-field
filters). Neither has profiling data showing it's the dominant cost
once ticket 077's range-narrowing lands, and building either
speculatively risks unbounded memory (079) or penalizing the more
common non-JSON case (080) for no measured benefit. Revisit if
`samply` profiling against a realistic log volume shows these are
actually hot.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
Log reads (logs_since, the unsorted Logs query path) now seek the
IngestedLog range directly from since_sequence/before_sequence instead
of scanning from IndexKey::Start and filtering every prior entry out
in a closure. Metric-comparison filters and bucket_values now seek to
a single metric's Bucket key range instead of scanning every declared
metric's buckets. evaluate_promql sits on top of bucket_values, so its
per-poll dashboard evaluation is narrowed by the same fix without
further changes; caching prior evaluations across polls is left for a
follow-up if profiling ever shows it's needed (per ticket 081's own
scope, not built speculatively).
Resolves tickets 077, 078, 081.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TeyhRfMYjMditek6geDRoQ
Pure code-motion refactor, no logic changes. Follows the repo convention
of one struct/enum + its impl per module file:
- engine/handle.rs: the left-right write/read plumbing (EngineOp,
IndexWriter, IndexReadHandle, IndexReader, EngineHandle) and the actor
loop (spawn/run) that owns EngineSnapshot.
- engine/message.rs: EngineQuery/EngineMessage/EngineResponse.
- engine/snapshot/: EngineSnapshot itself, split by concern into
params.rs (declaration + auto-detect), metrics.rs (declare/apply/
backfill), filter.rs (search-bar filter compilation/eval), sort.rs
(sort-helper indexes), and query.rs (handle_query_inner + PromQL
evaluation), with mod.rs holding the struct and the accessors shared
across all of them.
- engine/tests.rs: the inline test module, kept as one file (per the
task's own escape hatch) since most tests exercise ingest -> declare
-> query across several of the new submodules at once rather than one
in isolation.
Visibility was narrowed per call site: pub(super) where only sibling
snapshot submodules need an item, pub(crate) where engine::handle or
engine::tests (outside the snapshot subtree) also need it.
cargo build --all-targets and cargo test are green (291 passed, 1
ignored, same as before the split).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1ZJ3SrayzLzVndhVq9BmS
- src/api/: HTTP router + DTOs + the MCP server (the HTTP/MCP-facing
surface): mod.rs (formerly api.rs), dto.rs, mcp_server.rs.
- src/cli/: argument parsing and process entry (formerly inline in
main.rs) plus session registry: mod.rs, sessions.rs. main.rs is now
a thin entry point that just declares modules and calls cli::run().
- src/engine/: the actor + everything it depends on to parse/index/
query log lines: mod.rs (formerly engine.rs), alerts.rs, promql.rs,
rate.rs, ingest.rs, logfmt.rs, filter_parser.rs/.pest,
filter_datetime.rs, index_key.rs, data_value.rs, metric_primitive.rs,
timestamp.rs.
- src/config/: persisted configuration: mod.rs (formerly settings.rs),
profile.rs.
- src/consts.rs stays at the crate root, used from every module above.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1ZJ3SrayzLzVndhVq9BmS
