# 077 — Log queries scan the full index instead of seeking by key range ## Summary Every log read — including polling reads that only want logs newer than `since_sequence` — walks the *entire* retained log range and filters with a closure, rather than seeking the `BTreeMap` directly to the relevant key range. This is the read-path optimization the `left-right`/sort-helper-index work never got around to: iterators should narrow the underlying key range as far as the caller's bounds allow, and accept range bounds as a parameter instead of scanning from `Start`/`MetricBucketSplit` and filtering after the fact. ## Where - `EngineSnapshot::logs_range` (`src/engine/snapshot/mod.rs:159-168`) always ranges `IndexKey::Start..IndexKey::LogParamSplit` — i.e. every ingested log ever retained — with no way for a caller to pass a narrower bound. - `EngineSnapshot::logs_since` (`src/engine/snapshot/mod.rs:170-181`) calls `logs_range()` and then filters by `sequence < before_sequence` and a timestamp window in a closure — it has all the information needed to seek instead (`before_sequence` bounds `sequence` directly), but throws that away and scans linearly regardless of how small the actual window is. - The unsorted log-query path in `handle_query_inner` (`src/engine/snapshot/query.rs:59-65`) does the same thing: `snapshot.logs_range().filter(|(sequence, timestamp, _)| { since ... start_ms ... end_ms ... })`. On every poll (frontend passes `since_sequence` — see `webapp/src/lib/api.ts:263,343`), this walks every previously-seen log line again just to throw it away in the filter, before even getting to the new ones. - `IndexKey::IngestedLog { sequence, timestamp }` (`src/engine/index_key.rs:10-13`) derives `Ord` field-order-first on `sequence`, so a lower bound of `IndexKey::IngestedLog { sequence: since + 1, timestamp: Timestamp::MIN }` (or equivalent) is a valid, correct seek — sequence is assigned monotonically at ingest (`src/engine/handle.rs:118-119`), so this isn't a heuristic, it's exact. ## Scope - Change `logs_range` to accept a key range (e.g. `RangeBounds` or a `(Bound, Bound)` over sequence) instead of hardcoding `Start..LogParamSplit`, so callers can seek. - Rework `logs_since` and the `EngineQuery::Logs` handler in `query.rs:59-65` to build a lower bound from `since_sequence` (and, where possible, from `start_ms`/note below) and pass it through, instead of filtering post-hoc from the very start of the index. - Note: because `IngestedLog` keys sort by `(sequence, timestamp)` and not `(timestamp, sequence)`, a `start_ms`/`end_ms`-only query (no `since_sequence`) can't be turned into an exact key-range seek from this index alone even though ingestion is roughly time-ordered by sequence — call this out explicitly rather than seeking incorrectly; it may be acceptable to leave that case as a scan, or to binary-search for the first sequence at/after `start_ms` since timestamps are non-decreasing with sequence in practice. - The sorted branch (`query.rs:82-128`) iterates `sort_helper_index` directly rather than `logs_range`, so it's less affected, but still does a per-candidate `index.get()` (`query.rs:106-107`) with no early bound — check whether the helper index itself can be range-bounded once `since_sequence`/`start_ms` are known. - Add/extend a test asserting that a `since_sequence` poll on a large log store does not touch (or does not compare against) log entries at or before `since_sequence` — a range-based seek should make this observably true, not just faster. ## Depends on None. Related to ticket 060/061 (sort helper indexes) — same underlying principle (avoid full scans by indexing/seeking).