P1 — make voice conversation mode create durable chats and auto-finalize title/summary #358

New issue

Open

opened 2026-09-08 02:03:39 +00:00 by nsaspy · 0 comments

nsaspy commented

2026-09-08 02:03:39 +00:00

Owner

Parent: #323
Depends on: #133 session/client continuity and the runtime voice event contracts
Research: docs/research/002-unified-copilot-voice-visual-ci.md

Goal

Make Zara voice conversation mode a first-class producer/consumer of the same durable conversation history used by text chat, then finalize useful metadata when a voice session ends.

Do not create a separate voice-history database, desktop-only transcript model, or second assistant runtime.

Current leverage

Runtime events already expose turn_id + conversation_id correlation and voice events including VoiceTranscriptPartial and VoiceTranscriptFinal, alongside assistant/tool/audio events.

The missing work is explicit voice-session conversation ownership, exact-once commit semantics for final transcripts, and durable metadata finalization.

Conversation lifecycle

Required conceptual flow:

conversation mode enter
  -> create or explicitly resume conversation C
  -> bind voice session/generation to C

partial transcript
  -> ephemeral display only

final transcript
  -> append one USER message to C exactly once
  -> run normal semantic/agent turn under C

assistant/tool events
  -> mutate/render C

conversation mode exit
  -> close/finalize voice session
  -> enqueue title/summary finalization if eligible

Selection rules

Voice invoked without an explicitly selected chat creates a new durable voice conversation by default.
Voice explicitly started as continuation of a selected chat may reuse that conversation.
Background/system voice must not silently attach itself to whichever desktop conversation happened to be last visible/focused.
UI visibility does not own the session. A complete voice session while the Copilot is hidden still appears in normal history later.

Transcript requirements

partial transcript is never persisted as a user message;
accepted final transcript is persisted exactly once;
duplicate final events are idempotent;
raw vs normalized transcript stages from #219 must not create duplicate user turns;
final normalized text, according to the canonical normalization policy, is what routes onward and becomes the durable user utterance;
turn/session/trace correlation survives daemon/client transport;
stale prior-session events cannot append into the active conversation.

Assistant/tool/audio correlation

Every downstream visible event for a voice turn must retain the same durable conversation identity.

Reject/ignore stale generation/session output including:

assistant delta/complete;
tool state/result;
TTS/audio chunks;
late transcript final.

Barge-in/cancellation semantics remain canonical; do not fake a resumed turn after reconnect unless the daemon confirms it.

Automatic title and summary

Extend durable conversation metadata with fields equivalent to:

summary
title_source = auto | user | fallback
summary_updated_at
metadata revision/generation

Exact schema is design-owned.

Runtime/conversation service owns generation so desktop and Android cannot race with independent metadata writers.

Rules:

manual user rename always wins unless explicitly reset to auto;
summary is metadata, not an assistant chat message;
model input is bounded and sanitized to a conversation projection;
voice shutdown never waits for title/summary generation;
empty/trivial sessions may skip model generation;
provider failure gets deterministic fallback title from early user content;
retries are idempotent;
stale metadata generation cannot overwrite a newer manual title or newer revision;
restart during pending finalization is safe and does not corrupt conversation history.

UI contract

The unified Copilot from #324 should be able to render:

live ephemeral partial transcript;
atomic transition from partial -> final normal user message;
assistant response in the same chat;
compact voice state near the composer/header;
completed voice conversation in history after reopening.

Qt is not the state authority for this behavior.

Tests

TDD at minimum:

enter voice mode creates C;
explicit continue reuses selected C;
background invocation does not guess unrelated chat;
multiple partials commit zero messages;
one final commits one message;
duplicate final commits one message;
normalized final from #219 replaces raw display/route without second user turn;
UI hidden for entire session;
daemon reconnect mid-session;
stale old-session transcript/assistant/tool/audio rejected;
two conversations concurrently remain isolated;
two principals/devices remain isolated;
manual rename beats pending auto-title;
generation timeout/failure fallback;
restart during finalization;
additional messages make old summary result stale according to chosen revision policy.

Add screenshot fixture coverage for listening/partial/finalized chat through #324.

TDD / RAGE

Follow repository Auto-RAGE: immutable start SHA, research/design/adversarial review, RED first, exact-head verification and full repo/Nix gates.

Acceptance

A real voice conversation produces ordinary durable user/assistant chat history under one conversation ID even when the UI is hidden, stale/replayed events cannot corrupt another chat, and session exit safely produces shared automatic title/summary metadata without overwriting user choices.

Parent: #323 Depends on: #133 session/client continuity and the runtime voice event contracts Research: `docs/research/002-unified-copilot-voice-visual-ci.md` ## Goal Make Zara voice conversation mode a first-class producer/consumer of the same durable conversation history used by text chat, then finalize useful metadata when a voice session ends. Do not create a separate voice-history database, desktop-only transcript model, or second assistant runtime. ## Current leverage Runtime events already expose `turn_id` + `conversation_id` correlation and voice events including `VoiceTranscriptPartial` and `VoiceTranscriptFinal`, alongside assistant/tool/audio events. The missing work is explicit voice-session conversation ownership, exact-once commit semantics for final transcripts, and durable metadata finalization. ## Conversation lifecycle Required conceptual flow: ```text conversation mode enter -> create or explicitly resume conversation C -> bind voice session/generation to C partial transcript -> ephemeral display only final transcript -> append one USER message to C exactly once -> run normal semantic/agent turn under C assistant/tool events -> mutate/render C conversation mode exit -> close/finalize voice session -> enqueue title/summary finalization if eligible ``` ## Selection rules - Voice invoked without an explicitly selected chat creates a new durable voice conversation by default. - Voice explicitly started as continuation of a selected chat may reuse that conversation. - Background/system voice must not silently attach itself to whichever desktop conversation happened to be last visible/focused. - UI visibility does not own the session. A complete voice session while the Copilot is hidden still appears in normal history later. ## Transcript requirements - partial transcript is never persisted as a user message; - accepted final transcript is persisted exactly once; - duplicate final events are idempotent; - raw vs normalized transcript stages from #219 must not create duplicate user turns; - final normalized text, according to the canonical normalization policy, is what routes onward and becomes the durable user utterance; - turn/session/trace correlation survives daemon/client transport; - stale prior-session events cannot append into the active conversation. ## Assistant/tool/audio correlation Every downstream visible event for a voice turn must retain the same durable conversation identity. Reject/ignore stale generation/session output including: - assistant delta/complete; - tool state/result; - TTS/audio chunks; - late transcript final. Barge-in/cancellation semantics remain canonical; do not fake a resumed turn after reconnect unless the daemon confirms it. ## Automatic title and summary Extend durable conversation metadata with fields equivalent to: ```text summary title_source = auto | user | fallback summary_updated_at metadata revision/generation ``` Exact schema is design-owned. Runtime/conversation service owns generation so desktop and Android cannot race with independent metadata writers. Rules: - manual user rename always wins unless explicitly reset to auto; - summary is metadata, not an assistant chat message; - model input is bounded and sanitized to a conversation projection; - voice shutdown never waits for title/summary generation; - empty/trivial sessions may skip model generation; - provider failure gets deterministic fallback title from early user content; - retries are idempotent; - stale metadata generation cannot overwrite a newer manual title or newer revision; - restart during pending finalization is safe and does not corrupt conversation history. ## UI contract The unified Copilot from #324 should be able to render: - live ephemeral partial transcript; - atomic transition from partial -> final normal user message; - assistant response in the same chat; - compact voice state near the composer/header; - completed voice conversation in history after reopening. Qt is not the state authority for this behavior. ## Tests TDD at minimum: - enter voice mode creates C; - explicit continue reuses selected C; - background invocation does not guess unrelated chat; - multiple partials commit zero messages; - one final commits one message; - duplicate final commits one message; - normalized final from #219 replaces raw display/route without second user turn; - UI hidden for entire session; - daemon reconnect mid-session; - stale old-session transcript/assistant/tool/audio rejected; - two conversations concurrently remain isolated; - two principals/devices remain isolated; - manual rename beats pending auto-title; - generation timeout/failure fallback; - restart during finalization; - additional messages make old summary result stale according to chosen revision policy. Add screenshot fixture coverage for listening/partial/finalized chat through #324. ## TDD / RAGE Follow repository Auto-RAGE: immutable start SHA, research/design/adversarial review, RED first, exact-head verification and full repo/Nix gates. ## Acceptance A real voice conversation produces ordinary durable user/assistant chat history under one conversation ID even when the UI is hidden, stale/replayed events cannot corrupt another chat, and session exit safely produces shared automatic title/summary metadata without overwriting user choices.