diff --git a/tickets/058-declared-params-not-refreshed-live-in-ui.md b/tickets/058-declared-params-not-refreshed-live-in-ui.md new file mode 100644 index 0000000..6a417a5 --- /dev/null +++ b/tickets/058-declared-params-not-refreshed-live-in-ui.md @@ -0,0 +1,42 @@ +# 058 — Newly auto-declared params don't appear in the UI until a manual refresh + +## Summary + +`auto_declare_params` / `auto_declare_log_stream_params` (`src/engine.rs:383`, +`:451`) can add new param declarations on any ingested line, but the +frontend only ever fetches the param list once, on mount +(`webapp/src/App.tsx:278-284`, `fetchParamDeclarations().then(setParams)`). +The main poll loop (`App.tsx:300-350`, interval from +`webapp/src/lib/polling.ts`) refetches timeline entries and logs every +tick but never re-fetches param declarations — the only other place +`setParams` is called is the `onDeclared` callback fired by a user's own +manual edit in `Params.tsx`. So a param that gets auto-declared mid-run +from freshly ingested lines is invisible in the Params panel until the +page is reloaded. + +There is no push infrastructure (SSE/websocket) anywhere in the +codebase; all live-refresh today is plain polling. + +## Scope + +- Add a periodic (or piggybacked on the existing poll loop) refetch of + param declarations in `App.tsx`, e.g. call `fetchParamDeclarations()` + alongside the existing timeline/log poll in the loop at + `App.tsx:300-350`, or on its own lighter-weight interval if refetching + every tick is wasteful. +- Only update state when the declaration list actually changed, to avoid + needlessly re-rendering `Params.tsx` on every poll tick. +- Out of scope: introducing SSE/websocket push infrastructure — polling + is consistent with how the rest of the app already refreshes. + +## Acceptance criteria + +- Ingesting new lines that trigger auto-declaration of a previously + unseen param causes that param to appear in the Params UI within one + poll interval, without a page reload. +- Existing manual param declaration/edit flows (`Params.tsx`'s + `onDeclared`) are unaffected. + +## Depends on + +None. diff --git a/tickets/059-log-stream-parsing-should-be-generic-whitespace-columnar.md b/tickets/059-log-stream-parsing-should-be-generic-whitespace-columnar.md new file mode 100644 index 0000000..9ec591e --- /dev/null +++ b/tickets/059-log-stream-parsing-should-be-generic-whitespace-columnar.md @@ -0,0 +1,62 @@ +# 059 — `log stream` parsing is one hardcoded regex, not generic whitespace-columnar detection + +## Summary + +Current `log stream` support is a single hardcoded regex, +`LOG_STREAM_RE` (`src/logfmt.rs:93-113`), matching exactly six fixed +columns (`Timestamp Thread Type Activity PID TTL`) by content shape +(dates/hex/decimals) rather than by parsing the header row generically. +`auto_declare_log_stream_params` (`src/engine.rs:451-472`) then extracts +only three hardcoded fields (`log_type`, `process`, `pid`) via three more +bespoke regexes (`LOG_STREAM_TYPE_PATTERN` etc., `logfmt.rs:108-113`). + +The fixture `src/fixtures/log_stream_sample.txt` has a real header row +(`Timestamp Thread Type Activity PID TTL `) +whose columns are separated by runs of 2+ spaces — the general CSV-like +shape the wishlist asks for — plus a trailing unlabelled `process: +message` tail that must not get merged into the preceding column. Today +nothing derives column names or positions from that header; every field +is matched by a fixed, hand-tuned regex, so any deviation from this +exact `log stream` output shape silently fails to parse. + +## Scope + +- Replace `LOG_STREAM_RE`/`LOG_STREAM_*_PATTERN`'s fixed-shape matching + with a generic whitespace-columnar detector: given a header line, + split on runs of duplicated whitespace (2+ spaces, or a single tab, or + " \t " — any whitespace repeated) to determine column names and + positions, then apply the same column boundaries to subsequent data + lines. + - The header's last named column's value must still capture through + to end-of-line without swallowing the trailing unlabelled message + column that `log stream` appends after the last header-named field + (i.e. that trailing text becomes its own unlabelled/message param, + not merged into the last header column). + - Single tabs are acceptable separators even where surrounding spaces + aren't doubled. +- Wire the new column-driven extraction into + `auto_declare_log_stream_params` in place of the three bespoke + per-field regexes, so declared params are derived from whatever + columns the header actually names, not a fixed list of three. +- Use `src/fixtures/log_stream_sample.txt` as the reference fixture and + add/extend tests in `src/logfmt.rs`/`src/engine.rs` asserting header + columns are correctly detected and the trailing message field is kept + separate. +- Out of scope: changing param-cardinality bounding introduced in ticket + 050/054 (commit f48d3c6f) — this is purely about how columns are + detected/split, not about pruning noisy auto-declarations. + +## Acceptance criteria + +- Piping real `log stream` output (as in `log_stream_sample.txt`) + through ottyl declares params matching the header's actual column + names, not the previous fixed three. +- The trailing unlabelled message text on each `log stream` line is + captured as its own field and never merged into the last named + header column. +- A header/data line whose columns are separated by a single tab is + parsed identically to one separated by 2+ spaces. + +## Depends on + +None. diff --git a/tickets/060-timeline-custom-sort-needs-helper-btreemap-index.md b/tickets/060-timeline-custom-sort-needs-helper-btreemap-index.md new file mode 100644 index 0000000..e57a5b8 --- /dev/null +++ b/tickets/060-timeline-custom-sort-needs-helper-btreemap-index.md @@ -0,0 +1,57 @@ +# 060 — Timeline sort-by-property does a full materialize+sort per query instead of using a helper index + +## Summary + +Sorting the timeline by an arbitrary property does work today (from +ticket 045): in the `EngineQuery::Logs` branch (`src/engine.rs:1408-1487`), +when `sort_by` is set, every filtered line is materialized into a +`Vec` (deriving params for all of them), then sorted +in-memory with `.sort_by(...)`, with `sort_missing_fallback` (default +true) partitioning entries that have the sort param from those that +don't and ordering the latter by `ottyl_seq` (`:1468-1482`). + +The problem is architectural: there is no incremental secondary index +keyed by the sort property, only the primary `BTreeMap` keyed by +`IndexKey::IngestedLog { sequence, timestamp }` (`src/index_key.rs`, +inserted at `engine.rs:156-181`). Every sorted query re-derives params +for and re-sorts the *entire* filtered range from scratch, and range +selection (e.g. "give me the next page in this order") has no way to +seek directly into a sorted position — it can only sort everything then +slice. + +## Scope + +- Introduce a "helper index": a secondary `BTreeMap` keyed by + `(sort-param-value, ottyl_seq)` (or equivalent composite key so + `ottyl_seq` is the tie-break/fallback ordering baked into the key + itself, not a post-hoc partition step) per distinct param name that's + been sorted by. +- Populate a helper index incrementally as lines are ingested (near + `engine.rs:156-181`, alongside the primary index insert), for any + param name that currently has a live helper index, rather than + rebuilding from scratch per query. +- Lines missing the sort param must still be captured in the helper + index's ordering, falling back to `ottyl_seq` ordering — matching + current `sort_missing_fallback` semantics but expressed as part of the + index's key/ordering rather than a separate partition-and-append step. +- Adapt the `EngineQuery::Logs` sorted branch (`engine.rs:1459-1483`) to + do a range walk over the relevant helper index instead of + materializing and sorting the full filtered set every time. +- Out of scope: the eviction/purge policy for helper indexes that + haven't been used recently — that's ticket 061. + +## Acceptance criteria + +- Querying the timeline sorted by a declared param returns results + ordered correctly, including lines missing that param falling back to + `ottyl_seq` order, matching current behavior. +- A sorted query no longer re-sorts the full filtered range from scratch + on every call once a helper index for that sort key exists — verified + by a test/benchmark showing incremental maintenance rather than + full materialize-then-sort. +- Existing sort-related tests (from ticket 045) continue to pass + unmodified in their assertions. + +## Depends on + +None. diff --git a/tickets/061-purge-unused-sort-helper-indexes.md b/tickets/061-purge-unused-sort-helper-indexes.md new file mode 100644 index 0000000..199507a --- /dev/null +++ b/tickets/061-purge-unused-sort-helper-indexes.md @@ -0,0 +1,43 @@ +# 061 — Sort helper indexes need to be purged after a period of no access + +## Summary + +Ticket 060 introduces per-sort-key helper `BTreeMap` indexes. Left +unbounded, a user (or dashboard) that sorts by many distinct params over +a session accumulates a helper index per param forever, each duplicating +the full timeline's worth of keys. There's no existing time-since-access +eviction pattern in `engine.rs` to model this on: the only eviction today +is `evict_excess_logs` (`src/engine.rs:1356-1363`, called from ingest at +`:181` and from `SetMaxLogs` at `:212`), which is a hard count-cap +("drop oldest once over `max_logs`"), not an LRU/last-accessed scheme. + +## Scope + +- Track a last-accessed timestamp per helper index (e.g. + `HashMap, Instant)>` or equivalent), updated + whenever a query actually reads from that index. +- Add a sweep (triggered periodically, or opportunistically on + ingest/query — match whichever fits the existing engine actor's + message-loop structure) that drops any helper index whose + last-accessed time exceeds a fixed duration. +- A dropped helper index must be transparently rebuildable: the next + query sorted by that param should lazily rebuild the index (a one-time + full scan) exactly as if it had never existed, rather than erroring or + falling back permanently to unsorted behavior. +- Out of scope: making the purge duration user-configurable via + `Settings` — pick a reasonable fixed constant unless the ticket + implementer finds a strong reason to expose it. + +## Acceptance criteria + +- A helper index untouched for longer than the purge duration is + removed from engine memory. +- Querying a sort key whose helper index was purged transparently + rebuilds it and returns correct results, with no user-visible error. +- Actively-used helper indexes (queried more recently than the purge + duration) are never evicted. + +## Depends on + +Ticket 060 (timeline custom sort needs helper BTreeMap index) — this +ticket assumes helper indexes already exist. diff --git a/tickets/062-app-title-should-be-ottyl.md b/tickets/062-app-title-should-be-ottyl.md new file mode 100644 index 0000000..d777e27 --- /dev/null +++ b/tickets/062-app-title-should-be-ottyl.md @@ -0,0 +1,24 @@ +# 062 — Browser tab title says "webapp" instead of "ottyl" + +## Summary + +`webapp/index.html:7` has `webapp`. Nothing else sets a +user-visible title — no `document.title` assignment exists in +`App.tsx`, and `webapp/package.json`'s `"name": "webapp"` is a separate +dev-tooling field not surfaced to users. The only fix needed is the +`index.html` title tag. + +## Scope + +- Change `webapp` to `ottyl` in + `webapp/index.html`. +- Out of scope: renaming the `webapp` directory or `package.json` name + field — those are internal and not user-facing. + +## Acceptance criteria + +- The browser tab shows "ottyl" when the dashboard is open. + +## Depends on + +None. diff --git a/tickets/063-params-list-drop-redundant-group-suffix.md b/tickets/063-params-list-drop-redundant-group-suffix.md new file mode 100644 index 0000000..63eb5d5 --- /dev/null +++ b/tickets/063-params-list-drop-redundant-group-suffix.md @@ -0,0 +1,38 @@ +# 063 — Declared params list shows a redundant "group name" alongside type + +## Summary + +`webapp/src/components/Params.tsx:331-337` renders each param as +`{p.name} ({p.kind}, ...)`, where the third segment is one of `elapsed +since previous`, nothing (for booleans), `json field {path}`, or `group +{p.capture_group}`. There is no distinct "group" concept in the domain +model — `src/dto.rs`, `src/profile.rs`, and `src/engine.rs` have no +`group` field at all. What the UI calls "group" is actually +`capture_group` (`dto.rs:22`, `profile.rs:69`), the regex named-capture +group used to extract the param's value. By convention, params are +declared with `(?P...)`, so `capture_group` is almost always +identical to `p.name` — displaying it is redundant noise in the common +case (`name (type, name)`). + +## Scope + +- Change the default rendering to `name (type)`. +- Only append the extra suffix when it's actually informative: + - Keep `json field {path}` and `elapsed since previous` suffixes — + these carry information not implied by `name`/`type`. + - For the capture-group case, only show it when + `p.capture_group !== p.name` (i.e. the regex group name genuinely + differs from the declared param name); otherwise omit it. +- Out of scope: changing the underlying `capture_group` field or param + declaration model — this is a display-only fix. + +## Acceptance criteria + +- A param whose capture group matches its name renders as `name (type)`. +- A param whose capture group differs from its name still shows that + distinction (e.g. `name (type, group other_name)`). +- `json field` and `elapsed since previous` suffixes are unchanged. + +## Depends on + +None. diff --git a/tickets/064-promql-panel-input-clobbered-by-autosave.md b/tickets/064-promql-panel-input-clobbered-by-autosave.md new file mode 100644 index 0000000..29a4ff2 --- /dev/null +++ b/tickets/064-promql-panel-input-clobbered-by-autosave.md @@ -0,0 +1,48 @@ +# 064 — Typing a panel's PromQL query gets clobbered by in-flight autosave responses + +## Summary + +In `webapp/src/components/Dashboards.tsx`, the panel query `` +(`:408-411`) is bound directly to `draft.panels[...].promql` / +`updatePanel`, and `draft` is autosaved via a 600ms-debounced effect +(`AUTOSAVE_DEBOUNCE_MS = 600`, line 34; effect at `:190-214`). On save +success, `setDraft(saved)` (`:203`) unconditionally replaces the entire +local draft with the server's response — including whatever `promql` +string was in flight at request time. If the user types more characters +after the debounce fires but before the save response returns, those +keystrokes exist only in local `draft` state and get overwritten the +instant the stale `saved` response resolves, producing the reported +"characters inserted then removed" behavior. A second effect +(`Dashboards.tsx:184-188`) also resets `draft` whenever `dashboards` +changes — which the same save's `setDashboards` call triggers — +compounding the race. + +## Scope + +- Stop unconditionally overwriting in-flight-edited fields on autosave + response. Options (pick one, document the choice): + - Track which panel/field is actively focused/being edited and skip + overwriting that specific field when reconciling `saved` into + `draft`. + - Only merge server-assigned fields (ids, timestamps) from `saved` + into `draft` rather than replacing `draft` wholesale. + - Guard the reconciliation with a "pending edits since request was + sent" check, applying `saved` only when no newer local edit + occurred after the request was issued. +- Also fix the related staleness effect at `Dashboards.tsx:184-188` so + it doesn't clobber in-progress edits when `dashboards` changes as a + side effect of the same save. +- This does not require adopting Redux or another state library — the + fix is local to this component's save/edit reconciliation. + +## Acceptance criteria + +- Typing continuously in a panel's PromQL input while an autosave + round-trip is in flight never drops or reorders characters. +- Autosave still persists the query to the backend and other + server-assigned fields (ids, etc.) still get reconciled into local + state correctly. + +## Depends on + +None. diff --git a/tickets/065-filter-bar-help-should-describe-filter-syntax-not-regex.md b/tickets/065-filter-bar-help-should-describe-filter-syntax-not-regex.md new file mode 100644 index 0000000..5b2dd52 --- /dev/null +++ b/tickets/065-filter-bar-help-should-describe-filter-syntax-not-regex.md @@ -0,0 +1,41 @@ +# 065 — Filter bar's help/prompt foregrounds regex syntax instead of the filter bar's own syntax + +## Summary + +A comprehensive filter-bar syntax reference already exists — operators, +`[param]` comparisons, datetime formats, worked examples — behind the +`` in `webapp/src/components/Docs.tsx:76-140`, +wired up at `FilterBar.tsx:32-34`. The problem is emphasis, not missing +docs: the input's `placeholder` (`FilterBar.tsx:38`) leads with +`regex("error") and ottyl_seq<5 or ottyl_ts>12:00`, making regex look +like the primary syntax, and a second `` (a pure regex +cheat-sheet, `webapp/src/components/RegexHelpIcon.tsx`) sits more +prominently next to the input than the actual filter-syntax help icon. +A user skimming the bar sees "regex" twice before noticing the filter +bar has its own broader query language. + +## Scope + +- Change the input `placeholder` at `FilterBar.tsx:38` to lead with a + non-regex filter-bar example (e.g. + `ottyl_ts>-15m and requests[duration]>200`), keeping a `regex(...)` + clause further in the example rather than first if desired. +- Reorder/re-emphasize the two help icons so the general filter-syntax + `HelpIcon` (→ `Docs.tsx`'s filtering section) reads as the primary + affordance, with the regex-specific cheat sheet secondary/clearly + labeled as being about regex specifically, not the filter bar as a + whole. +- Out of scope: rewriting the content of `Docs.tsx`'s filtering section + — it's already accurate and thorough; this is about surfacing it + correctly, not authoring new documentation. + +## Acceptance criteria + +- The filter bar's placeholder text no longer leads with a bare + `regex(...)` example as the first thing a user sees. +- The general filter-syntax help is at least as discoverable/prominent + as the regex-only cheat sheet. + +## Depends on + +None. diff --git a/tickets/066-filter-bar-value-doesnt-reflect-active-preset-or-sort.md b/tickets/066-filter-bar-value-doesnt-reflect-active-preset-or-sort.md new file mode 100644 index 0000000..cf7540e --- /dev/null +++ b/tickets/066-filter-bar-value-doesnt-reflect-active-preset-or-sort.md @@ -0,0 +1,53 @@ +# 066 — Filter bar text doesn't reflect the active time-range preset or sort order + +## Summary + +In `webapp/src/App.tsx`, `filter` (the raw typed filter text, `:264`), +`activePreset`/`range` (`:262-263`, set via `selectPreset` / +`handleManualRangeSelect`, `:286-298`), and `sortSettings` (`:266`) are +three entirely separate pieces of state. +`` (`:355`) only +ever reflects the manually-typed `filter` string — selecting a +quick-pick time-range preset or changing the sort order never touches +`filter`, so the filter bar can show empty/stale text even though a +preset or sort order is actively constraining or ordering what's +displayed. Separately, `FilterBar.tsx:22` +(`const [draft, setDraft] = useState(value)`) initializes its internal +`draft` from `value` only once on mount and never resyncs if `value` +changes externally afterward (e.g. a programmatic clear) — a related +staleness bug worth fixing alongside this. + +## Scope + +- Decide and implement one of: + - Synthesize a read-only textual summary of the active preset/sort + state and display it in or alongside the filter bar when the user + hasn't typed a manual filter (e.g. showing the equivalent + `ottyl_ts>-Xm`-style clause for the active preset). + - Make preset-selection and sort-order actions actually write into + the `filter` query string itself (e.g. `selectPreset` appends/ + updates an `ottyl_ts` clause), so `filter` is always the single + source of truth for what's displayed in the bar. + Document the choice in the PR — this is a real design decision, not + just a bug fix. +- Fix `FilterBar.tsx:22` so the internal `draft` resyncs when the + `value` prop changes externally (not just on mount), so programmatic + filter changes (from whichever approach above, or an external clear) + are reflected in the input. +- Out of scope: changing how presets/sort settings are computed or + stored server-side — this is about surfacing existing state in the + filter bar's displayed value. + +## Acceptance criteria + +- Selecting a quick-pick time-range preset causes the filter bar to + visibly reflect that a time constraint is active (either via a + synthesized display or an actual written filter clause). +- Changing sort order is likewise visible via the filter bar's + displayed state. +- The filter bar's input value stays in sync with externally-driven + changes to the active filter, not just the user's own typing. + +## Depends on + +None.