[P2] ZARA-029 — Optimize warm startup and remove avoidable work from the voice critical path #30

New issue

Open

opened 2026-07-18 19:25:48 +00:00 by lost-rob0t · 0 comments

lost-rob0t commented

2026-07-18 19:25:48 +00:00

(Migrated from github.com)

Objective

After streaming behavior is correct and measured, remove avoidable cold-start and per-turn overhead from the critical path.

Required implementation

Preload and warm the configured Whisper model, Prolog program, agent graph, LLM client/session, acknowledgement cache, TTS client, and persistent player during startup.
Publish readiness/health per actor and degrade explicitly when an optional service is unavailable.
Size CPU thread pools deliberately; do not pass os.cpu_count() as both Whisper workers and a separate executor without a measured policy.
Replace repeated np.concatenate growth, per-sample deque popping, forced gc.collect(), repeated player discovery, repeated graph compilation, and repeated client construction with bounded reusable structures.
Move optional memory retrieval, plugin discovery, verbose logging, summaries, and nonessential notifications off the first-response path where semantics permit.
Add warm/cold cache handling and configurable idle unload only if measurement justifies it.
Preserve deterministic shutdown and resource cleanup.

Required tests

Cold/warm startup and turn benchmarks with stage-level metrics.
CPU oversubscription, bounded-memory, idle/wake, actor restart, and repeated-turn resource tests.
Verify no hot-path optimization changes command results or test fixtures.
Add scripts/benchmark-startup.py and scripts/test-hot-path.sh.

Acceptance

Warm turns reuse all expensive clients/models/graphs/player resources.
The benchmark report demonstrates each retained optimization with before/after stage data.
No unbounded memory growth or thread/process leak occurs across repeated turns.

Branch

perf/zara-029-warm-hot-path

Dependencies

ZARA-022 through ZARA-028.

## Objective After streaming behavior is correct and measured, remove avoidable cold-start and per-turn overhead from the critical path. ## Required implementation - Preload and warm the configured Whisper model, Prolog program, agent graph, LLM client/session, acknowledgement cache, TTS client, and persistent player during startup. - Publish readiness/health per actor and degrade explicitly when an optional service is unavailable. - Size CPU thread pools deliberately; do not pass `os.cpu_count()` as both Whisper workers and a separate executor without a measured policy. - Replace repeated `np.concatenate` growth, per-sample deque popping, forced `gc.collect()`, repeated player discovery, repeated graph compilation, and repeated client construction with bounded reusable structures. - Move optional memory retrieval, plugin discovery, verbose logging, summaries, and nonessential notifications off the first-response path where semantics permit. - Add warm/cold cache handling and configurable idle unload only if measurement justifies it. - Preserve deterministic shutdown and resource cleanup. ## Required tests - Cold/warm startup and turn benchmarks with stage-level metrics. - CPU oversubscription, bounded-memory, idle/wake, actor restart, and repeated-turn resource tests. - Verify no hot-path optimization changes command results or test fixtures. - Add `scripts/benchmark-startup.py` and `scripts/test-hot-path.sh`. ## Acceptance - Warm turns reuse all expensive clients/models/graphs/player resources. - The benchmark report demonstrates each retained optimization with before/after stage data. - No unbounded memory growth or thread/process leak occurs across repeated turns. ## Branch `perf/zara-029-warm-hot-path` ## Dependencies ZARA-022 through ZARA-028.