80 lines
4.1 KiB
Markdown
80 lines
4.1 KiB
Markdown
# 052 — Allow clearing all params, then re-extract them from already-ingested logs
|
|
|
|
## Summary
|
|
|
|
There is no bulk "clear params" action. The engine already supports
|
|
removing a single param declaration internally —
|
|
`EngineMessage::RemoveParamDeclaration { name, respond_to }`
|
|
(`src/engine.rs:85-92`), handled at `src/engine.rs:190-197` by removing
|
|
the `IndexKey::ParamDeclaration` entry — but it is only ever invoked from
|
|
`reload_engine_declarations` (`src/api.rs:664-682`), a profile-switch
|
|
cleanup path that filters out auto-derived params (comment at
|
|
`src/api.rs:665-668`) and removes the rest one at a time. There is no
|
|
route exposing this to the user, no persistence-side removal method
|
|
(`src/profile.rs` has `record_param_declaration` at `profile.rs:606` but
|
|
nothing to un-record it), and no webapp affordance
|
|
(`webapp/src/lib/api.ts:283-287` only exposes `getParams`/`declareParam`).
|
|
|
|
The harder half of this request — "logs get re-iterated to extract
|
|
params from them" — is not just a rewording of `auto_declare_params`.
|
|
That function runs once, at ingest time, against each incoming line
|
|
(`src/engine.rs:154`, `auto_declare_params` defined at
|
|
`src/engine.rs:307-325`, backed by `extract_pairs` in `src/logfmt.rs`) and
|
|
is never replayed. Re-extraction after a clear means running that same
|
|
logic again, after the fact, against lines already sitting in the index.
|
|
This is structurally possible: raw ingested text is retained as
|
|
`IndexKey::IngestedLog { sequence, timestamp } -> DataValue::LogLiteral`
|
|
in the same `BTreeMap` (`src/engine.rs:148-152`), occupying the index
|
|
range between `IndexKey::Start` and `IndexKey::LogParamSplit`
|
|
(iterated at `src/engine.rs:1166`, `1181`) — but only for lines that
|
|
haven't already been evicted by `max_logs`/`evict_excess_logs`
|
|
(`src/engine.rs:151-156`). Lines evicted before a clear+re-extract cannot
|
|
regain their params; this bound should be surfaced to the user, not
|
|
silently ignored.
|
|
|
|
Note `main.rs:75-96`'s `profile::load_into_engine` replay path is not
|
|
directly reusable here — it replays *declarations* into a fresh engine
|
|
before ingest starts, whereas this feature must re-run *extraction*
|
|
against logs the engine already holds.
|
|
|
|
## Scope
|
|
|
|
- Add a bulk "clear params" `EngineMessage` (or reuse
|
|
`RemoveParamDeclaration` in a loop, mirroring
|
|
`reload_engine_declarations`'s existing pattern at
|
|
`src/api.rs:664-682`) that removes all declared params — deciding
|
|
whether auto-derived and manually-declared params are both cleared or
|
|
only one category, and stating that choice in the PR (the existing
|
|
`reload_engine_declarations` precedent excludes auto-derived ones for a
|
|
reason worth re-reading before copying it here).
|
|
- Add a persisted-removal counterpart in `src/profile.rs`/`src/settings.rs`
|
|
so cleared params don't reappear from `settings.json` on next load.
|
|
- After clearing, re-run `auto_declare_params` against every retained
|
|
`IndexKey::IngestedLog` entry still in the index (oldest to newest, to
|
|
preserve declaration-order semantics), producing a fresh param set from
|
|
currently-available log text.
|
|
- Expose this as a single user-facing action (`DELETE /api/params` or
|
|
similar in `src/api.rs`, plus a button in whatever webapp component
|
|
hosts the params list) — not a per-param loop the user has to drive
|
|
from the UI.
|
|
- Surface to the user (via response body or UI copy) that re-extraction
|
|
only covers logs still retained in memory, not lines already evicted by
|
|
`max_logs`.
|
|
|
|
## Acceptance criteria
|
|
|
|
- Triggering "clear params" removes all param declarations from
|
|
`GET /api/params` and they do not reappear after a restart.
|
|
- Immediately after clearing, params re-appear derived from the log text
|
|
still held in the engine's index, without requiring new lines to be
|
|
ingested, verified with a seeded fixture where params are cleared and
|
|
re-extracted from a fixed set of already-ingested lines.
|
|
- If some lines have been evicted before the clear, the re-extracted
|
|
param set reflects only the retained subset, and this is not reported
|
|
as an error.
|
|
|
|
## Depends on
|
|
|
|
None (touches the same `RemoveParamDeclaration`/`reload_engine_declarations`
|
|
seam used by profile switching).
|