Bound auto-declared param cardinality and recognize log stream format (ticket 050/054)
auto_declare_params (src/engine.rs) previously never overwrote or pruned a declaration, so a burst of unusual key=value-shaped lines could permanently inflate the params UI forever. A real macOS `log stream` dump (huig_dump.txt, copied into src/fixtures/log_stream_sample.txt) discovered 220 distinct auto-declared params from under 700 lines, mostly one-off WiFi/radio stat counters (20+ pairs on a single line), plus base64-blob false positives from key()'s greedy alnum matching landing right before `==` padding. Fix, combining two bounds per the ticket's options: - src/logfmt.rs: extract_pairs/pair now rejects a candidate pair whose bareword value starts with a bare '=' (looks_like_base64_padding_artifact) — the exact shape produced when a base64/hash blob's padding sits right after a run of alnum chars that key() consumed as if it were a key. - src/engine.rs: auto_declare_params now enforces a per-line cap (MAX_NEW_AUTO_PARAMS_PER_LINE = 8) on new params from one line, and a total ceiling (MAX_AUTO_DECLARED_PARAMS = 64) on auto-derived params overall — the per-line cap alone doesn't bound unbounded key *variety* across many lines, so both are needed. Also adds structured extraction for log stream's fixed-column shape (Timestamp Thread Type Activity PID TTL, then `process: message`): detect_format gains a new "log_stream" ottyl_fmt value (checked first, since it's the most specific of the four), and auto_declare_log_stream_params declares log_type/process/pid as standalone queryable params the first time such a line is seen — derived positionally from the column layout rather than =-scanning, so there's no ambiguity or false-positive risk. This is a separate always-attempted extraction alongside the existing key=value pass, not a replacement for it: a log stream line's message tail can still contain real key=value pairs (e.g. the WiFi stat lines) that auto_declare_params keeps discovering independently. This also fully addresses ticket 054, confirming huig_dump.txt is space-padded fixed-width columns, not TSV, and that no separate TSV format is needed. Throughput (src/fixtures/log_stream_sample.txt x50, release build): before (caps disabled) ~369k lines/sec, 206 auto-declared params; after (caps enabled) ~1.20M lines/sec, 64 auto-declared params — about 3.2x faster, confirming the ever-growing declaration set was the bottleneck rather than per-line regex/nom scanning cost. Removes tickets/050 and tickets/054 now that both are implemented. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XFUNZwCvz3kcod3352ZR2o
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
d17cfe9b83
commit
f48d3c6ffb
@@ -104,7 +104,10 @@ const SECTIONS: DocSection[] = [
|
||||
<p>
|
||||
<Code>ottyl_seq</Code>, <Code>ottyl_ts</Code>, and <Code>ottyl_fmt</Code> are reserved metric names present
|
||||
on every log line, so they're always filterable without declaring anything. <Code>ottyl_fmt</Code> holds the
|
||||
auto-detected format of the line's text — <Code>json</Code>, <Code>logfmt</Code>, or <Code>none</Code>.
|
||||
auto-detected format of the line's text — <Code>log_stream</Code> (macOS <Code>log stream</Code> output),{' '}
|
||||
<Code>json</Code>, <Code>logfmt</Code>, or <Code>none</Code>. A <Code>log_stream</Code> line also gets{' '}
|
||||
<Code>log_type</Code>, <Code>process</Code>, and <Code>pid</Code> params derived automatically from its
|
||||
fixed-column header.
|
||||
</p>
|
||||
<p className="font-semibold text-text">Date/time formats</p>
|
||||
<p>Timestamp comparisons accept three forms:</p>
|
||||
|
||||
Reference in New Issue
Block a user