[EPIC] Selectable post-STT transcript normalization backends, including local S1-mini #220

New issue

Open

opened 2026-08-28 03:23:53 +00:00 by nsaspy · 1 comment

nsaspy commented

2026-08-28 03:23:53 +00:00

Owner

Goal

Add a first-class post-STT transcript normalization stage to Zara so a user can select off, s1-mini, or future compatible normalizers without replacing the configured speech-to-text backend.

This stage turns the raw final ASR transcript into clean written text before semantic routing, dictation output, or agent input.

Target pipeline:

audio
  -> VAD/STT (#25 / #132)
  -> raw final transcript
  -> transcript normalizer (off | s1-mini | future backend)
  -> normalized transcript
  -> optional bounded semantic rewrite (#122)
  -> Prolog semantic routing / conversation runtime

Why this is a distinct subsystem

S1-mini is a narrow ASR text-normalization model, not an STT engine and not a general chat/agent model. It is intended to remove fillers, resolve false starts/self-corrections, apply punctuation/capitalization, and normalize spoken forms such as numbers, dates, times, currency, and email addresses.

Do not implement S1-mini as a Whisper/transcriber provider and do not fold this into #122. The responsibilities are different:

STT: audio -> raw text;
transcript normalization: raw speech-like text -> clean written text;
#122 bounded rewrite: optional semantic preparation before Prolog intent resolution.

Core architecture

Introduce one provider-neutral transcript-normalization boundary owned by the runtime/voice pipeline, for example conceptually:

TranscriptNormalizer.normalize(raw_text, context) -> NormalizationResult

Exact API is research/design-owned, but the contract must support:

off / identity backend;
local S1-mini backend;
future normalizers without editing the coordinator with backend-specific branches;
explicit raw and normalized text fields/events;
cancellation and deadline propagation;
bounded input/output sizes;
deterministic fallback policy;
language capability reporting;
backend/model/version metadata for diagnostics and benchmarks;
no backend-specific objects leaking into runtime events.

Placement and semantics

Normalization runs only on a final accepted transcript. Partial transcript events remain raw/display-oriented unless a later issue explicitly designs bounded partial normalization.

For a successful voice turn:

STT emits raw final transcript.
Selected normalizer receives that exact text.
Runtime validates/bounds the normalizer result.
Normalized text becomes the user utterance used by dictation/semantic routing/agent input according to the owning surface.
#122, if enabled, runs after normalization.
Raw text remains available for diagnostics/test evidence according to privacy policy, but is not silently substituted back into semantic history after normalization succeeds.

Normalization must never execute a command or tool itself.

Configuration direction

Use Zara's existing TOML configuration authority. Research exact keys, but support a shape equivalent to:

[voice.transcript_normalization]
backend = "off"          # off | s1-mini
failure_policy = "raw"   # raw | fail-turn
language_policy = "auto"

[voice.transcript_normalization.s1_mini]
runtime = "openai-compatible"
base_url = "http://127.0.0.1:11434/v1"
model = "s1-mini"
style = "balanced"
structure = "prose"
context = "general"
timeout_ms = 1000

Do not hard-code Ollama as the only runtime. Prefer one local OpenAI-compatible inference adapter usable with llama.cpp/Ollama/LM Studio-style endpoints where compatible; direct-process support may be added only if research justifies ownership/lifecycle.

S1-mini requirements

The first concrete backend must:

support the official S1-mini control format;
disable Qwen thinking/reasoning behavior as required by the selected serving path;
default to deterministic generation appropriate for normalization;
recognize that current S1-mini release is English-only;
expose explicit unavailable/unsupported-language/degraded state;
never send transcripts to a cloud endpoint unless the user explicitly configured a non-local endpoint;
keep model/license attribution requirements documented if weights are redistributed or bundled;
not auto-download mutable model weights in the default deterministic CI gate.

Safety / correctness invariants

A transcript normalizer is a text transformation, not a new authority boundary.

Treat output as untrusted user text.
No normalizer output can grant tool/device/admin capability.
Preserve turn/principal/session ownership.
Cancellation/stale-turn fencing applies across normalization.
Bound output growth relative to input.
Empty/malformed/timeout/backend failure follows explicit configured policy.
Never log transcript bodies in ordinary metrics/security logs.
Preserve trace IDs and stage timings without raw text.
A backend may improve formatting but must not silently summarize, answer, or expand the user's request.

Integration points

#25 streaming VAD/STT and stable transcriber boundary;
#23 latency instrumentation;
#30 warm startup/hot path;
#31 voice release gate;
#132 daemon voice transport;
#122 bounded semantic rewrite, which runs after normalization;
#90/#92 desktop voice/settings surfaces where selection/status is exposed;
#168 real-speech STT -> semantic regression corpus.

Ordered slices

Provider-neutral transcript normalizer contract + raw/normalized runtime semantics.
Local S1-mini provider adapter with deterministic fixtures and failure handling.
Config/CLI/desktop diagnostics and selectable backend settings.
Voice/dictation/runtime integration with strict placement before #122 and semantic routing.
Adversarial correctness, privacy, latency, warm-start and real-speech release gates.

Acceptance

A user can configure Zara to use backend = "off" or backend = "s1-mini" without changing the STT provider. With S1-mini enabled, a final Whisper/other ASR transcript is normalized locally before command/conversation handling, while raw/normalized provenance, cancellation, privacy, failure fallback, latency, and English-only limitations are explicit and regression-tested.

No Prolog-RLM dependency.

## Goal Add a first-class **post-STT transcript normalization stage** to Zara so a user can select `off`, `s1-mini`, or future compatible normalizers without replacing the configured speech-to-text backend. This stage turns the raw final ASR transcript into clean written text before semantic routing, dictation output, or agent input. Target pipeline: ```text audio -> VAD/STT (#25 / #132) -> raw final transcript -> transcript normalizer (off | s1-mini | future backend) -> normalized transcript -> optional bounded semantic rewrite (#122) -> Prolog semantic routing / conversation runtime ``` ## Why this is a distinct subsystem S1-mini is a narrow ASR text-normalization model, not an STT engine and not a general chat/agent model. It is intended to remove fillers, resolve false starts/self-corrections, apply punctuation/capitalization, and normalize spoken forms such as numbers, dates, times, currency, and email addresses. Do **not** implement S1-mini as a Whisper/transcriber provider and do **not** fold this into #122. The responsibilities are different: - STT: audio -> raw text; - transcript normalization: raw speech-like text -> clean written text; - #122 bounded rewrite: optional semantic preparation before Prolog intent resolution. ## Core architecture Introduce one provider-neutral transcript-normalization boundary owned by the runtime/voice pipeline, for example conceptually: ```text TranscriptNormalizer.normalize(raw_text, context) -> NormalizationResult ``` Exact API is research/design-owned, but the contract must support: - `off` / identity backend; - local S1-mini backend; - future normalizers without editing the coordinator with backend-specific branches; - explicit raw and normalized text fields/events; - cancellation and deadline propagation; - bounded input/output sizes; - deterministic fallback policy; - language capability reporting; - backend/model/version metadata for diagnostics and benchmarks; - no backend-specific objects leaking into runtime events. ## Placement and semantics Normalization runs only on a **final accepted transcript**. Partial transcript events remain raw/display-oriented unless a later issue explicitly designs bounded partial normalization. For a successful voice turn: 1. STT emits raw final transcript. 2. Selected normalizer receives that exact text. 3. Runtime validates/bounds the normalizer result. 4. Normalized text becomes the user utterance used by dictation/semantic routing/agent input according to the owning surface. 5. #122, if enabled, runs after normalization. 6. Raw text remains available for diagnostics/test evidence according to privacy policy, but is not silently substituted back into semantic history after normalization succeeds. Normalization must never execute a command or tool itself. ## Configuration direction Use Zara's existing TOML configuration authority. Research exact keys, but support a shape equivalent to: ```toml [voice.transcript_normalization] backend = "off" # off | s1-mini failure_policy = "raw" # raw | fail-turn language_policy = "auto" [voice.transcript_normalization.s1_mini] runtime = "openai-compatible" base_url = "http://127.0.0.1:11434/v1" model = "s1-mini" style = "balanced" structure = "prose" context = "general" timeout_ms = 1000 ``` Do not hard-code Ollama as the only runtime. Prefer one local OpenAI-compatible inference adapter usable with llama.cpp/Ollama/LM Studio-style endpoints where compatible; direct-process support may be added only if research justifies ownership/lifecycle. ## S1-mini requirements The first concrete backend must: - support the official S1-mini control format; - disable Qwen thinking/reasoning behavior as required by the selected serving path; - default to deterministic generation appropriate for normalization; - recognize that current S1-mini release is English-only; - expose explicit unavailable/unsupported-language/degraded state; - never send transcripts to a cloud endpoint unless the user explicitly configured a non-local endpoint; - keep model/license attribution requirements documented if weights are redistributed or bundled; - not auto-download mutable model weights in the default deterministic CI gate. ## Safety / correctness invariants A transcript normalizer is a **text transformation**, not a new authority boundary. - Treat output as untrusted user text. - No normalizer output can grant tool/device/admin capability. - Preserve turn/principal/session ownership. - Cancellation/stale-turn fencing applies across normalization. - Bound output growth relative to input. - Empty/malformed/timeout/backend failure follows explicit configured policy. - Never log transcript bodies in ordinary metrics/security logs. - Preserve trace IDs and stage timings without raw text. - A backend may improve formatting but must not silently summarize, answer, or expand the user's request. ## Integration points - #25 streaming VAD/STT and stable transcriber boundary; - #23 latency instrumentation; - #30 warm startup/hot path; - #31 voice release gate; - #132 daemon voice transport; - #122 bounded semantic rewrite, which runs **after** normalization; - #90/#92 desktop voice/settings surfaces where selection/status is exposed; - #168 real-speech STT -> semantic regression corpus. ## Ordered slices 1. Provider-neutral transcript normalizer contract + raw/normalized runtime semantics. 2. Local S1-mini provider adapter with deterministic fixtures and failure handling. 3. Config/CLI/desktop diagnostics and selectable backend settings. 4. Voice/dictation/runtime integration with strict placement before #122 and semantic routing. 5. Adversarial correctness, privacy, latency, warm-start and real-speech release gates. ## Acceptance A user can configure Zara to use `backend = "off"` or `backend = "s1-mini"` without changing the STT provider. With S1-mini enabled, a final Whisper/other ASR transcript is normalized locally before command/conversation handling, while raw/normalized provenance, cancellation, privacy, failure fallback, latency, and English-only limitations are explicit and regression-tested. No Prolog-RLM dependency.

nsaspy commented

2026-08-28 03:24:19 +00:00

Author

Owner

Created dependency-ordered implementation slices:

#216 — provider-neutral transcript normalizer contract + raw/normalized runtime semantics
#217 — local S1-mini backend
#218 — configuration, selection, status and diagnostics
#219 — voice/dictation/runtime integration before #122
#220 — adversarial correctness/privacy/latency/real-speech release gate

Execution order: #216 -> #217 -> #218 -> #219 -> #220.

Policy: keep normalization opt-in (off by default) until #220 produces evidence for any broader default change.

Created dependency-ordered implementation slices: 1. #216 — provider-neutral transcript normalizer contract + raw/normalized runtime semantics 2. #217 — local S1-mini backend 3. #218 — configuration, selection, status and diagnostics 4. #219 — voice/dictation/runtime integration before #122 5. #220 — adversarial correctness/privacy/latency/real-speech release gate Execution order: #216 -> #217 -> #218 -> #219 -> #220. Policy: keep normalization opt-in (`off` by default) until #220 produces evidence for any broader default change.