Create tickets

tickets/059-log-stream-parsing-should-be-generic-whitespace-columnar.md

@@ -0,0 +1,62 @@
+# 059 — `log stream` parsing is one hardcoded regex, not generic whitespace-columnar detection
+
+## Summary
+
+Current `log stream` support is a single hardcoded regex,
+`LOG_STREAM_RE` (`src/logfmt.rs:93-113`), matching exactly six fixed
+columns (`Timestamp Thread Type Activity PID TTL`) by content shape
+(dates/hex/decimals) rather than by parsing the header row generically.
+`auto_declare_log_stream_params` (`src/engine.rs:451-472`) then extracts
+only three hardcoded fields (`log_type`, `process`, `pid`) via three more
+bespoke regexes (`LOG_STREAM_TYPE_PATTERN` etc., `logfmt.rs:108-113`).
+
+The fixture `src/fixtures/log_stream_sample.txt` has a real header row
+(`Timestamp                       Thread     Type        Activity             PID    TTL  `)
+whose columns are separated by runs of 2+ spaces — the general CSV-like
+shape the wishlist asks for — plus a trailing unlabelled `process:
+message` tail that must not get merged into the preceding column. Today
+nothing derives column names or positions from that header; every field
+is matched by a fixed, hand-tuned regex, so any deviation from this
+exact `log stream` output shape silently fails to parse.
+
+## Scope
+
+- Replace `LOG_STREAM_RE`/`LOG_STREAM_*_PATTERN`'s fixed-shape matching
+  with a generic whitespace-columnar detector: given a header line,
+  split on runs of duplicated whitespace (2+ spaces, or a single tab, or
+  " \t " — any whitespace repeated) to determine column names and
+  positions, then apply the same column boundaries to subsequent data
+  lines.
+  - The header's last named column's value must still capture through
+    to end-of-line without swallowing the trailing unlabelled message
+    column that `log stream` appends after the last header-named field
+    (i.e. that trailing text becomes its own unlabelled/message param,
+    not merged into the last header column).
+  - Single tabs are acceptable separators even where surrounding spaces
+    aren't doubled.
+- Wire the new column-driven extraction into
+  `auto_declare_log_stream_params` in place of the three bespoke
+  per-field regexes, so declared params are derived from whatever
+  columns the header actually names, not a fixed list of three.
+- Use `src/fixtures/log_stream_sample.txt` as the reference fixture and
+  add/extend tests in `src/logfmt.rs`/`src/engine.rs` asserting header
+  columns are correctly detected and the trailing message field is kept
+  separate.
+- Out of scope: changing param-cardinality bounding introduced in ticket
+  050/054 (commit f48d3c6f) — this is purely about how columns are
+  detected/split, not about pruning noisy auto-declarations.
+
+## Acceptance criteria
+
+- Piping real `log stream` output (as in `log_stream_sample.txt`)
+  through ottyl declares params matching the header's actual column
+  names, not the previous fixed three.
+- The trailing unlabelled message text on each `log stream` line is
+  captured as its own field and never merged into the last named
+  header column.
+- A header/data line whose columns are separated by a single tab is
+  parsed identically to one separated by 2+ spaces.
+
+## Depends on
+
+None.