PERF/ARCH: lazy-load persisted chats instead of hydrating every conversation at startup

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

nsaspy/a0-symbolics#21

lost-rob0t commented

2026-08-24 00:46:27 +00:00

(Migrated from github.com)

Finding

Agent Zero currently loads every persisted chat into a full live AgentContext during startup:

initialize.py::initialize_chats() starts persist_chat.load_tmp_chats().
load_tmp_chats() enumerates every usr/chats/<id>/chat.json, reads the complete JSON file, parses it, and calls _deserialize_context().
_deserialize_context() initializes agent configuration, reconstructs the log, reconstructs every serialized agent/history chain, and creates a live AgentContext.
persisted logs can contain up to LOG_SIZE = 1000 entries per chat.

This couples startup latency and steady-state RAM to the entire historical chat corpus. It also makes every persisted chat part of AgentContext.all(), which then expands global state snapshots and frontend roster work.

The UI needs metadata for old chats; it does not need every old chat's full history/log/agent state resident in memory.

Direction

Separate a lightweight persistent chat catalog from hydrated execution contexts.

Catalog metadata should be sufficient for navigation/search/grouping without loading the chat body, e.g.:

id / name
created / last message timestamps
profile
project metadata
parent/branch relationship
running/task state where applicable
pin/archive/tag metadata when introduced
compact preview/search fields

Hydrate the full context only when selected, resumed, running, or otherwise required. Define safe eviction/dehydration for inactive contexts and preserve provider Responses state/history semantics.

Potential implementation models include a sidecar metadata index, versioned manifest, SQLite catalog, or compact metadata file per chat; choose via ADADR rather than baking the storage choice into this issue.

Acceptance

Startup does not parse/deserialise every historical chat.json into a live AgentContext.
Historical chats remain immediately discoverable through the chat catalog.
Selecting an unloaded chat hydrates it on demand with behavior equivalent to current loading.
Running/scheduled/task contexts that require residency remain safe.
Inactive hydrated contexts can be bounded/evicted without losing unsaved state.
Parent/child chat relationships and project/profile metadata work without full hydration.
Corrupt/unreadable chat bodies do not prevent the rest of the catalog from loading; error state is visible per entry.
Existing chat files migrate/read compatibly and export/import remains lossless.
Benchmark startup time and steady-state RSS for 100, 1,000, and 10,000 persisted chats.
Benchmark first-open latency for cold and warm chats.
Coordinate with the state-roster and modern chat-library issues so only lightweight catalog metadata crosses the WebUI boundary.

## Finding Agent Zero currently loads every persisted chat into a full live `AgentContext` during startup: - `initialize.py::initialize_chats()` starts `persist_chat.load_tmp_chats()`. - `load_tmp_chats()` enumerates every `usr/chats/<id>/chat.json`, reads the complete JSON file, parses it, and calls `_deserialize_context()`. - `_deserialize_context()` initializes agent configuration, reconstructs the log, reconstructs every serialized agent/history chain, and creates a live `AgentContext`. - persisted logs can contain up to `LOG_SIZE = 1000` entries per chat. This couples startup latency and steady-state RAM to the **entire historical chat corpus**. It also makes every persisted chat part of `AgentContext.all()`, which then expands global state snapshots and frontend roster work. The UI needs metadata for old chats; it does not need every old chat's full history/log/agent state resident in memory. ## Direction Separate a lightweight persistent **chat catalog** from hydrated execution contexts. Catalog metadata should be sufficient for navigation/search/grouping without loading the chat body, e.g.: - id / name - created / last message timestamps - profile - project metadata - parent/branch relationship - running/task state where applicable - pin/archive/tag metadata when introduced - compact preview/search fields Hydrate the full context only when selected, resumed, running, or otherwise required. Define safe eviction/dehydration for inactive contexts and preserve provider Responses state/history semantics. Potential implementation models include a sidecar metadata index, versioned manifest, SQLite catalog, or compact metadata file per chat; choose via ADADR rather than baking the storage choice into this issue. ## Acceptance - [ ] Startup does not parse/deserialise every historical `chat.json` into a live AgentContext. - [ ] Historical chats remain immediately discoverable through the chat catalog. - [ ] Selecting an unloaded chat hydrates it on demand with behavior equivalent to current loading. - [ ] Running/scheduled/task contexts that require residency remain safe. - [ ] Inactive hydrated contexts can be bounded/evicted without losing unsaved state. - [ ] Parent/child chat relationships and project/profile metadata work without full hydration. - [ ] Corrupt/unreadable chat bodies do not prevent the rest of the catalog from loading; error state is visible per entry. - [ ] Existing chat files migrate/read compatibly and export/import remains lossless. - [ ] Benchmark startup time and steady-state RSS for 100, 1,000, and 10,000 persisted chats. - [ ] Benchmark first-open latency for cold and warm chats. - [ ] Coordinate with the state-roster and modern chat-library issues so only lightweight catalog metadata crosses the WebUI boundary.

nsaspy referenced this issue

2026-09-09 15:05:17 +00:00

FEATURE: add a scalable Chat Library with search, grouping, filters and keyboard navigation #162

2026-09-09 15:05:21 +00:00

PERF/MEM: bound the in-memory Log journal and stop retaining one update entry per stream chunk forever #169

Rows
Columns

PERF/ARCH: lazy-load persisted chats instead of hydrating every conversation at startup #21

Finding

Direction

Acceptance