nsaspy/zara
1
0
Fork 0
Code Issues 95 Pull requests 18 Projects Releases Packages Wiki Activity Actions

Add personality-aware agent hooks and true TTS barge-in #271

Closed
nsaspy wants to merge 0 commits from feature/agent-mode-core into master
pull from: feature/agent-mode-core
merge into: nsaspy:master
Conversation 0 Commits - Files changed -
nsaspy commented 2026-09-04 22:44:05 +00:00
Owner
Copy link

Follow-up to #257

#257 shipped the toggleable core agent-mode and is already merged. This PR adds the missing autonomous-action context hooks and wires tool-driven TTS into Zara's existing live voice/barge-in path.

Personality/context hooks

  • centralize Zara's effective default/custom agent system prompt in zara.agent.prompting
  • make normal AgentManager turns and autonomous lifecycle hooks read the same personality source
  • add ordered, thread-safe lifecycle stages:
    • before_task_create
    • before_task_run
    • after_task_result
    • before_proactive_question
    • after_proactive_question
    • before_speak
    • after_speak
  • each AgentModeActionContext exposes the live personality prompt plus a short fingerprint before the action occurs
  • allow hooks to rewrite task prompts, proactive prompts, or speech text
  • persist only the personality fingerprint with recurring task state; never persist the raw personality/system prompt
  • recurring executions use the current personality while retaining the creation fingerprint for audit/debugging

Barge-in / shared mic

  • add a process-wide speech playback activity registry without acquiring a microphone
  • mark agent-mode mpv playback active while TTS is speaking
  • make daemon RuntimeVoiceIngress playback-aware using existing BargeInConfig thresholds
  • while Zara speaks, raise VAD from the ordinary default 0.5 / 4 frames to playback defaults 0.7 / 6 frames (never lower stricter user settings)
  • keep the existing microphone stream active throughout playback
  • on canonical VoiceSpeechStarted, terminate active agent-mode playback when barge_in=true
  • let the same user utterance continue through normal STT and become the next turn
  • no second sounddevice stream and no pause/close/reopen of wake capture

Configuration

[plugins.agent-mode]
enabled = true
barge_in = true

Regression coverage

  • personality is visible before task creation
  • hooks can rewrite task prompts
  • raw personality is not persisted to state; fingerprint is
  • speech playback activity is marked and cleared
  • confirmed voice speech interrupts active TTS
  • playback-aware VAD raises/restores thresholds
  • stricter existing VAD settings are never weakened
## Follow-up to #257 #257 shipped the toggleable core `agent-mode` and is already merged. This PR adds the missing autonomous-action context hooks and wires tool-driven TTS into Zara's existing live voice/barge-in path. ## Personality/context hooks - centralize Zara's effective default/custom agent system prompt in `zara.agent.prompting` - make normal `AgentManager` turns and autonomous lifecycle hooks read the same personality source - add ordered, thread-safe lifecycle stages: - `before_task_create` - `before_task_run` - `after_task_result` - `before_proactive_question` - `after_proactive_question` - `before_speak` - `after_speak` - each `AgentModeActionContext` exposes the live personality prompt plus a short fingerprint before the action occurs - allow hooks to rewrite task prompts, proactive prompts, or speech text - persist only the personality fingerprint with recurring task state; never persist the raw personality/system prompt - recurring executions use the *current* personality while retaining the creation fingerprint for audit/debugging ## Barge-in / shared mic - add a process-wide speech playback activity registry without acquiring a microphone - mark agent-mode `mpv` playback active while TTS is speaking - make daemon `RuntimeVoiceIngress` playback-aware using existing `BargeInConfig` thresholds - while Zara speaks, raise VAD from the ordinary default 0.5 / 4 frames to playback defaults 0.7 / 6 frames (never lower stricter user settings) - keep the existing microphone stream active throughout playback - on canonical `VoiceSpeechStarted`, terminate active agent-mode playback when `barge_in=true` - let the same user utterance continue through normal STT and become the next turn - no second `sounddevice` stream and no pause/close/reopen of wake capture ## Configuration ```toml [plugins.agent-mode] enabled = true barge_in = true ``` ## Regression coverage - personality is visible before task creation - hooks can rewrite task prompts - raw personality is not persisted to state; fingerprint is - speech playback activity is marked and cleared - confirmed voice speech interrupts active TTS - playback-aware VAD raises/restores thresholds - stricter existing VAD settings are never weakened
nsaspy closed this pull request 2026-09-04 23:09:27 +00:00
Some checks failed
CI / test (pull_request) Failing after 2s
Details
CI / shared mic / Arch Linux (pull_request) Failing after 1m18s
Details
CI / android skeleton gate (pull_request) Failing after 2s
Details
CI / shared mic / Ubuntu 24.04 (pull_request) Has been cancelled
Details

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug
Something isn't working

  
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
invalid
This doesn't seem right

  
question
Further information is requested

  
wontfix
This will not be worked on

No labels
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
nsaspy/zara!271
No description provided.