# 079 — Free-text regex filters have no index; full scan + regex eval per query ## Summary A free-text search-bar filter (a bare regex, not a `param[value]op`/`ottyl_seq`/`ottyl_ts` comparison) is evaluated line-by-line against every retained log on every query — there's no narrower key range to seek into here since the match isn't tied to a declared param or timestamp, so this is the one case in the read path where "narrow the key range" doesn't directly apply and an actual secondary index would be needed instead. ## Where - `CompiledFilter::Regex` (compiled in `src/engine/snapshot/filter.rs`, evaluated via `.eval(...)` per log line) is checked against the full text of every candidate log line returned by `logs_range()`/the sort-helper iteration in `handle_query_inner` (`src/engine/snapshot/query.rs:59-70` and `111-116`) — i.e. once ticket 077 narrows the *sequence*/time range, a regex-text filter still forces a full per-line regex match over everything left in that range, and if there's no `since_sequence` bound at all (e.g. a fresh search over the whole retained log window), this is a full linear scan with a regex eval per line. - There is no cached/precomputed text index (trigram, token, or otherwise) — every query recompiles nothing (the regex itself is cached fine, see param/metric regex caches in `src/engine/snapshot/mod.rs:26-34`), but re-*evaluates* the regex against the full text of every line in range, every time. ## Scope - This is the most invasive of the read-path tickets — flag as such. Do not implement speculatively; first confirm via profiling (`samply record` against a realistic log volume with an active free-text filter) that this is actually the dominant cost before building an index, since regex eval over a bounded/paginated result set may already be fast enough once ticket 077's range-narrowing lands. - If profiling confirms it's worth it: investigate a coarse pre-filter (e.g. a trigram index or per-line token set) built incrementally at ingest time (alongside `update_sort_helper_indexes_on_ingest`, `src/engine/handle.rs:151`) that can cheaply reject lines that can't possibly match a given regex's required literal substrings, before paying for the actual regex match — not a replacement for the regex eval, a way to skip most of it. - Keep the index bounded in size / evictable in step with `evict_excess_logs` (`src/engine/handle.rs:153`) so it doesn't grow unbounded relative to `max logs in memory`. ## Depends on Should land after ticket 077 (range narrowing) so profiling reflects the scan cost that's actually left, not the one about to be fixed anyway.