[P1] ZARA-021 — Establish the full deterministic regression and CI script gate #22

New issue

Closed

opened 2026-07-18 19:22:33 +00:00 by lost-rob0t · 0 comments

lost-rob0t commented

2026-07-18 19:22:33 +00:00

(Migrated from github.com)

Objective

Turn the per-issue tests added during ZARA-001 through ZARA-020 into one stable gate that later agent runs must execute before opening a PR.

Required implementation

Create a top-level scripts/test-all.sh that runs, in fail-fast order:
1. Python compile/import checks.
2. Prolog module load and resolver corpus.
3. Focused pytest suite.
4. Config/process/file-tool security scripts.
5. Packaging/Nix checks.
Add deterministic audio fixtures for silence, wake word, command speech, trailing silence, interruption, noise, and long speech. Fixtures must be generated or redistributable and small.
Use temporary HOME/XDG/data/database/PID/log directories for every test.
Add GitHub Actions or flake checks that invoke the same scripts rather than duplicating logic.
Produce machine-readable JUnit and a concise latency-neutral summary.
No live microphone, desktop, provider credential, model download, or external network requirement in the default gate.

Required tests

The issue is the test gate. Add self-tests proving:

a failing sub-script makes the gate fail;
cleanup occurs after failure;
scripts work from a clean checkout and Nix store wrapper context;
test order is deterministic.

Acceptance

One command validates the repaired repository: nix develop -c bash scripts/test-all.sh.
nix flake check runs the same core gate.
A later agent can reproduce failures locally from logs/artifacts.

Branch

test/zara-021-regression-gate

Dependencies

ZARA-001 through ZARA-020. Latency work must not begin until this closes.

## Objective Turn the per-issue tests added during ZARA-001 through ZARA-020 into one stable gate that later agent runs must execute before opening a PR. ## Required implementation - Create a top-level `scripts/test-all.sh` that runs, in fail-fast order: 1. Python compile/import checks. 2. Prolog module load and resolver corpus. 3. Focused pytest suite. 4. Config/process/file-tool security scripts. 5. Packaging/Nix checks. - Add deterministic audio fixtures for silence, wake word, command speech, trailing silence, interruption, noise, and long speech. Fixtures must be generated or redistributable and small. - Use temporary HOME/XDG/data/database/PID/log directories for every test. - Add GitHub Actions or flake checks that invoke the same scripts rather than duplicating logic. - Produce machine-readable JUnit and a concise latency-neutral summary. - No live microphone, desktop, provider credential, model download, or external network requirement in the default gate. ## Required tests The issue is the test gate. Add self-tests proving: - a failing sub-script makes the gate fail; - cleanup occurs after failure; - scripts work from a clean checkout and Nix store wrapper context; - test order is deterministic. ## Acceptance - One command validates the repaired repository: `nix develop -c bash scripts/test-all.sh`. - `nix flake check` runs the same core gate. - A later agent can reproduce failures locally from logs/artifacts. ## Branch `test/zara-021-regression-gate` ## Dependencies ZARA-001 through ZARA-020. Latency work must not begin until this closes.