[P2] ZARA-030 — Add the full-duplex soak, interruption, and latency release gate #31

New issue

Open

opened 2026-07-18 19:26:03 +00:00 by lost-rob0t · 0 comments

lost-rob0t commented

2026-07-18 19:26:03 +00:00

(Migrated from github.com)

Objective

Prove the repaired streaming voice assistant remains correct, responsive, bounded, and interruptible over realistic repeated use.

Required implementation

Build an end-to-end deterministic duplex harness connecting synthetic microphone input, wake/VAD/STT, coordinator, Prolog/agent, fake or local fixture LLM, TTS, playback sink, and telemetry.
Cover command-only turns, conversational turns, tools, failures, timeouts, rapid follow-ups, barge-in, stop phrases, no speech, noise, long speech, provider restarts, and shutdown.
Add a configurable long-running soak mode, with a one-hour local target and a shorter deterministic CI profile.
Track process/thread/task/file-descriptor counts, queue depths, resident memory, cancelled/stale events, and audio underruns.
Generate p50/p95/p99 reports for wake-to-ack, endpoint-to-final-STT, request-to-first-token, first-phrase-to-first-audio, and speech-onset-to-playback-stop.
Fail the release gate when deterministic budgets regress, correctness assertions fail, stale work is observed, or resources trend upward beyond tolerance.
Document the remaining hardware/provider-dependent measurements separately from deterministic CI gates.

Required tests/scripts

Add scripts/test-full-duplex.sh for the short CI profile.
Add scripts/soak-voice.py for configurable local soak runs.
Add scripts/report-latency.py to compare a run against a checked-in baseline/tolerance.
Integrate the short gate into scripts/test-all.sh and nix flake check.

Acceptance

The CI duplex profile passes all correctness and deterministic latency gates.
The local one-hour soak shows no unbounded queue, memory, task, thread, process, or descriptor growth.
Interrupted/cancelled turns never emit stale audio, notifications, tool side effects, or history entries.
The final report clearly identifies any remaining provider/hardware bottleneck rather than hiding it.

Branch

test/zara-030-duplex-release-gate

Dependencies

ZARA-021 through ZARA-029.

## Objective Prove the repaired streaming voice assistant remains correct, responsive, bounded, and interruptible over realistic repeated use. ## Required implementation - Build an end-to-end deterministic duplex harness connecting synthetic microphone input, wake/VAD/STT, coordinator, Prolog/agent, fake or local fixture LLM, TTS, playback sink, and telemetry. - Cover command-only turns, conversational turns, tools, failures, timeouts, rapid follow-ups, barge-in, stop phrases, no speech, noise, long speech, provider restarts, and shutdown. - Add a configurable long-running soak mode, with a one-hour local target and a shorter deterministic CI profile. - Track process/thread/task/file-descriptor counts, queue depths, resident memory, cancelled/stale events, and audio underruns. - Generate p50/p95/p99 reports for wake-to-ack, endpoint-to-final-STT, request-to-first-token, first-phrase-to-first-audio, and speech-onset-to-playback-stop. - Fail the release gate when deterministic budgets regress, correctness assertions fail, stale work is observed, or resources trend upward beyond tolerance. - Document the remaining hardware/provider-dependent measurements separately from deterministic CI gates. ## Required tests/scripts - Add `scripts/test-full-duplex.sh` for the short CI profile. - Add `scripts/soak-voice.py` for configurable local soak runs. - Add `scripts/report-latency.py` to compare a run against a checked-in baseline/tolerance. - Integrate the short gate into `scripts/test-all.sh` and `nix flake check`. ## Acceptance - The CI duplex profile passes all correctness and deterministic latency gates. - The local one-hour soak shows no unbounded queue, memory, task, thread, process, or descriptor growth. - Interrupted/cancelled turns never emit stale audio, notifications, tool side effects, or history entries. - The final report clearly identifies any remaining provider/hardware bottleneck rather than hiding it. ## Branch `test/zara-030-duplex-release-gate` ## Dependencies ZARA-021 through ZARA-029.