44 lines
2.0 KiB
Markdown
44 lines
2.0 KiB
Markdown
# 061 — Sort helper indexes need to be purged after a period of no access
|
|
|
|
## Summary
|
|
|
|
Ticket 060 introduces per-sort-key helper `BTreeMap` indexes. Left
|
|
unbounded, a user (or dashboard) that sorts by many distinct params over
|
|
a session accumulates a helper index per param forever, each duplicating
|
|
the full timeline's worth of keys. There's no existing time-since-access
|
|
eviction pattern in `engine.rs` to model this on: the only eviction today
|
|
is `evict_excess_logs` (`src/engine.rs:1356-1363`, called from ingest at
|
|
`:181` and from `SetMaxLogs` at `:212`), which is a hard count-cap
|
|
("drop oldest once over `max_logs`"), not an LRU/last-accessed scheme.
|
|
|
|
## Scope
|
|
|
|
- Track a last-accessed timestamp per helper index (e.g.
|
|
`HashMap<String, (BTreeMap<...>, Instant)>` or equivalent), updated
|
|
whenever a query actually reads from that index.
|
|
- Add a sweep (triggered periodically, or opportunistically on
|
|
ingest/query — match whichever fits the existing engine actor's
|
|
message-loop structure) that drops any helper index whose
|
|
last-accessed time exceeds a fixed duration.
|
|
- A dropped helper index must be transparently rebuildable: the next
|
|
query sorted by that param should lazily rebuild the index (a one-time
|
|
full scan) exactly as if it had never existed, rather than erroring or
|
|
falling back permanently to unsorted behavior.
|
|
- Out of scope: making the purge duration user-configurable via
|
|
`Settings` — pick a reasonable fixed constant unless the ticket
|
|
implementer finds a strong reason to expose it.
|
|
|
|
## Acceptance criteria
|
|
|
|
- A helper index untouched for longer than the purge duration is
|
|
removed from engine memory.
|
|
- Querying a sort key whose helper index was purged transparently
|
|
rebuilds it and returns correct results, with no user-visible error.
|
|
- Actively-used helper indexes (queried more recently than the purge
|
|
duration) are never evicted.
|
|
|
|
## Depends on
|
|
|
|
Ticket 060 (timeline custom sort needs helper BTreeMap index) — this
|
|
ticket assumes helper indexes already exist.
|