Files
ottyl/tickets/054-log-stream-format-detection.md
T

3.2 KiB

054 — log stream output isn't recognized as a known format (folds into ticket 050)

Summary

The report was "log stream |& ottyl should capture the format properly... tab separated format is being ignored, add it as a known type." Investigating huig_dump.txt (a real log stream capture already in the repo root) shows the premise needs correcting: the file is not tab-separated. grep -P "\t" finds a literal tab character on only 3 of 697 lines, and none of those are the header or a representative data row — they're incidental (two libinfo check path: continuation lines, one WiFi stat line with an embedded tab mid-message). od -c on the header and data rows confirms the real structure is space-padded fixed-width columns (Timestamp, Thread, Type, Activity, PID, TTL, followed by a free-text process: message tail) — visually aligned columns that read as tab-separated but aren't.

There is no TSV handling anywhere in the codebase today: src/logfmt.rs, src/engine.rs, and src/ingest.rs contain zero references to tabs or TSV. Format detection is entirely detect_format (src/logfmt.rs:70-78): try JSON, else fall back to extract_pairs (logfmt.rs:45-59) for logfmt, else "none" — no third branch exists, and ottyl_fmt (src/engine.rs:1005, filter compilation at engine.rs:775, 824) only ever reports one of those three values.

This is a duplicate of, not distinct from, ticket 050 (tickets/050-structured-header-extraction-and-bounded-logfmt-overfit.md), which already identifies the exact same root cause — log stream output "is not logfmt at all — it's a fixed-column header... followed by a free-text process: message tail" — and already scopes a structured- header extractor plus the question of whether it becomes a new ottyl_fmt variant (e.g. "log_stream"). The reporter most likely visually misread the aligned whitespace columns as tab-separated.

Scope

  • Do not implement a separate "TSV ottyl_fmt type" — there is no evidence of genuinely tab-delimited log stream output in this codebase's fixtures, and building detection for a format that doesn't occur would be speculative.
  • Close/merge this wishlist item into ticket 050's existing scope: its structured-header extractor for log stream-shaped input already covers "capturing the format properly" for this input shape.
  • If the user can supply a genuinely tab-delimited sample (e.g. from a different log stream --style flag, or a shell pipeline that reformats the output with real tabs), re-open this as a distinct ticket scoped to that concrete input — file:line references above should make it easy to confirm one way or the other before implementing anything.

Acceptance criteria

  • No code changes result directly from this ticket; instead, ticket 050 is updated (or its implementation, when picked up, is confirmed) to cover recognizing log stream's fixed-column shape as a known format.
  • If a true TSV sample is later provided, a new ticket is filed against it specifically, rather than reopening this one with an unconfirmed premise.

Depends on

050 (this ticket's concern is already inside 050's scope; treat as duplicate/fold-in rather than independent work).