# 080 — JSON field filters re-parse every candidate line's JSON on every query ## Summary A filter comparing a JSON field (e.g. a log line shaped as JSON, filtered by a field inside it) re-parses the full JSON body of every candidate log line from scratch on every query that uses such a filter — there's no memoization of the parse, even though the same log line's JSON gets re-parsed identically on every subsequent poll until the line ages out. ## Where - `eval_json_comparison` (`src/engine/snapshot/filter.rs:177-185`) calls `serde_json::from_str::(text)` on the raw log text every time it's invoked, and it's invoked once per candidate line per query (called from the per-line filter evaluation path used in `query.rs:59-70`/`111-116` via `CompiledFilter::eval`). - Contrast with `log_entry_params` (used a few lines below in `query.rs:79`/`121`), which at least only runs against lines that survive the filter, not before — the JSON parse here runs as part of filtering itself, so it can't be skipped even for lines that get discarded downstream for other reasons if this filter is combined with others. ## Scope - Lowest-effort option: nothing to cache at the snapshot level today since `DataValue::LogLiteral` only stores raw text (`src/engine/snapshot/mod.rs` / `data_value.rs`) — re-parsing per query is currently unavoidable without changing storage. Confirm this is worth doing before storing pre-parsed JSON: check whether most ingested lines are actually JSON-shaped in typical usage, since adding a parsed-JSON cache to every ingested line penalizes the (likely more common) non-JSON case. - If justified, consider parsing once at ingest time only for lines that look JSON-shaped (reuse the cheap detection already implied by `eval_json_comparison`'s `Ok(...) else return false` short-circuit) and storing the parsed `serde_json::Value` (or just the needed field extractions) alongside `LogLiteral`, invalidated/dropped the same way the line itself is evicted (`evict_excess_logs`, `src/engine/handle.rs:153`). - Alternative, cheaper fix: memoize the parse per-query-invocation rather than per-line-per-filter-eval, i.e. if a single query applies the JSON filter against a line more than once (it currently doesn't appear to, but double-check `CompiledFilter` composition — AND/OR combinators) avoid re-parsing within that single pass. ## Depends on None.