ottyl/tickets/061-purge-unused-sort-helper-indexes.md

061 — Sort helper indexes need to be purged after a period of no access

Summary

Ticket 060 introduces per-sort-key helper BTreeMap indexes. Left unbounded, a user (or dashboard) that sorts by many distinct params over a session accumulates a helper index per param forever, each duplicating the full timeline's worth of keys. There's no existing time-since-access eviction pattern in engine.rs to model this on: the only eviction today is evict_excess_logs (src/engine.rs:1356-1363, called from ingest at :181 and from SetMaxLogs at :212), which is a hard count-cap ("drop oldest once over max_logs"), not an LRU/last-accessed scheme.

Scope

• Track a last-accessed timestamp per helper index (e.g. HashMap<String, (BTreeMap<...>, Instant)> or equivalent), updated whenever a query actually reads from that index.
• Add a sweep (triggered periodically, or opportunistically on ingest/query — match whichever fits the existing engine actor's message-loop structure) that drops any helper index whose last-accessed time exceeds a fixed duration.
• A dropped helper index must be transparently rebuildable: the next query sorted by that param should lazily rebuild the index (a one-time full scan) exactly as if it had never existed, rather than erroring or falling back permanently to unsorted behavior.
• Out of scope: making the purge duration user-configurable via Settings — pick a reasonable fixed constant unless the ticket implementer finds a strong reason to expose it.

Acceptance criteria

• A helper index untouched for longer than the purge duration is removed from engine memory.
• Querying a sort key whose helper index was purged transparently rebuilds it and returns correct results, with no user-visible error.
• Actively-used helper indexes (queried more recently than the purge duration) are never evicted.

Depends on

Ticket 060 (timeline custom sort needs helper BTreeMap index) — this ticket assumes helper indexes already exist.