PERF/MEM: bound the in-memory Log journal and stop retaining one update entry per stream chunk forever #169

New issue

Open

opened 2026-09-09 15:05:21 +00:00 by nsaspy · 0 comments

nsaspy commented

2026-09-09 15:05:21 +00:00

Owner

Finding

helpers/log.py::Log keeps two unbounded in-memory arrays for the lifetime of a live context:

logs: list[LogItem]
updates: list[int]

Every _update_item() appends item.no to updates, including repeated streaming updates of the same log item. A long model/tool stream can therefore add thousands of duplicate integers for one logical message. Neither list is compacted until Log.reset().

Persistence already keeps only the last LOG_SIZE = 1000 log items, but the live process retains the complete log and complete update journal. With many eagerly-loaded/long-lived chats this compounds the memory problem tracked in #21.

Log.output(start=...) uses the update-list index as its client cursor, so simple list truncation would break current cursor semantics. The journal needs a base/version model or another bounded change-tracking design.

Impact

resident memory grows with all historical log items while a context stays live
updates grows with stream chunks, not just messages
long output streams can create large duplicate journals
state versions become tied to ever-growing Python lists
lazy-chat work (#21) helps old contexts but running/long-lived contexts still need bounded logs

Direction

Use a monotonic log version independent of raw list indexes and keep a bounded update/delta structure.

Possible shape:

monotonic version
bounded ring/deque of changed log IDs/versions
coalesce repeated updates to the same item within a state-push generation
explicit reset/full snapshot required signal when a client cursor predates retained deltas
separately define the in-memory retention policy for completed LogItems

Do not silently drop information the active UI is expected to show. If full durable history is desired, store it durably instead of pinning it forever in every live Python context.

Acceptance

Memory used by the update journal is bounded independently of stream duration.
Repeated chunks for one streaming item do not append one permanent journal entry per chunk.
Client log cursors remain monotonic and reconnect-safe across compaction.
Stale clients receive an explicit resync/full-window path.
Live log-item retention has a documented bound/durable-history contract.
Persist/reload semantics remain compatible with the existing 1,000-entry persisted window or an explicitly migrated replacement.
Benchmark a 250k-character streamed response, 10k tool updates, and multiple long-lived chats for RSS/journal size.

Mirrored from lost-rob0t/a0-symbolics#31 via tracker sync.

## Finding `helpers/log.py::Log` keeps two unbounded in-memory arrays for the lifetime of a live context: - `logs: list[LogItem]` - `updates: list[int]` Every `_update_item()` appends `item.no` to `updates`, including repeated streaming updates of the **same** log item. A long model/tool stream can therefore add thousands of duplicate integers for one logical message. Neither list is compacted until `Log.reset()`. Persistence already keeps only the last `LOG_SIZE = 1000` log items, but the live process retains the complete log and complete update journal. With many eagerly-loaded/long-lived chats this compounds the memory problem tracked in #21. `Log.output(start=...)` uses the update-list index as its client cursor, so simple list truncation would break current cursor semantics. The journal needs a base/version model or another bounded change-tracking design. ## Impact - resident memory grows with all historical log items while a context stays live - `updates` grows with *stream chunks*, not just messages - long output streams can create large duplicate journals - state versions become tied to ever-growing Python lists - lazy-chat work (#21) helps old contexts but running/long-lived contexts still need bounded logs ## Direction Use a monotonic log version independent of raw list indexes and keep a bounded update/delta structure. Possible shape: - monotonic `version` - bounded ring/deque of changed log IDs/versions - coalesce repeated updates to the same item within a state-push generation - explicit `reset/full snapshot required` signal when a client cursor predates retained deltas - separately define the in-memory retention policy for completed `LogItem`s Do not silently drop information the active UI is expected to show. If full durable history is desired, store it durably instead of pinning it forever in every live Python context. ## Acceptance - [ ] Memory used by the update journal is bounded independently of stream duration. - [ ] Repeated chunks for one streaming item do not append one permanent journal entry per chunk. - [ ] Client log cursors remain monotonic and reconnect-safe across compaction. - [ ] Stale clients receive an explicit resync/full-window path. - [ ] Live log-item retention has a documented bound/durable-history contract. - [ ] Persist/reload semantics remain compatible with the existing 1,000-entry persisted window or an explicitly migrated replacement. - [ ] Benchmark a 250k-character streamed response, 10k tool updates, and multiple long-lived chats for RSS/journal size. --- *Mirrored from [`lost-rob0t/a0-symbolics#31`](https://github.com/lost-rob0t/a0-symbolics/issues/31)* via tracker sync.