[EPIC] Portable semantic intents, multi-turn slot dialogue, and capability/service routing #150

Open
opened 2026-08-22 21:49:52 +00:00 by lost-rob0t · 2 comments
lost-rob0t commented 2026-08-22 21:49:52 +00:00 (Migrated from github.com)

Goal

Refactor Zara's command path after the daemon/client split so intent meaning, missing-argument dialogue, capability selection, and execution location are separate first-class contracts.

The target is one portable semantic core that can serve Linux desktop/CLI, the long-lived zara-server, future Android clients, and future offline/device-local execution without duplicating intent vocabularies or turning the wire protocol into a remote shell.

This epic is future implementation work. Do not begin implementation before #133 has completed the supported client split. Voice-specific integration additionally depends on #132. The daemon release decision remains owned by #134.

Code-review baseline

This epic is designed against PR #148 as though its intended principal-isolation architecture will merge, but does not pretend the current PR is merge-ready. Review on #148 identified pre-merge #131 integration blockers that must be closed first:

  • RuntimeSupervisor._build_default_host() currently discards its PrincipalContext, so default AgentManager construction still creates memory without an explicit authenticated principal;
  • the owner-bound ConversationStore introduced by #148 is not yet the default daemon agent conversation path; AgentManager still owns an in-memory ConversationManager history;
  • exact-head #148 CI currently fails because a new isolation fake publishes a RuntimeCommand into a RuntimeEventBus that correctly accepts only RuntimeEvent;
  • principal typing/transient-principal policy and concurrent SQLite migration behavior need hardening/proof.

The future architecture may assume the design outcome of #131 (principal-bound private state), but must not depend on those gaps remaining.

Current problems this epic owns

Parsing is conflated with execution

Current Prolog intent results are essentially kind + name + args, and several execution paths still jump from a resolved verb directly toward command handling.

Missing arguments are only partially modeled

modules/intent_resolver.pl already has a useful seed: pending(Intent) plus named missing slots for cases such as bare open and text. It is not general. A bare timer goes through the special timer parser and fails instead of yielding a typed request for duration.

Provider location is implicit

Current kb/config.pl is heavily Linux/device shaped (xdg-open, GNOME utilities, desktop apps, shutdown commands). Those mappings must not become server policy just because Prolog moves behind zara-server.

The LLM agent currently duplicates routing policy in prompt text

AgentManager contains a command-verb list and tells the model to invoke query_prolog. Semantic command routing should become a runtime service contract rather than prompt-only policy duplicated beside Prolog.

ZARA/1 has no typed device capability/action plane yet

Protocol v1 is closed and currently supports the daemon text/runtime vocabulary plus reserved voice messages. Future device capabilities must extend that closed schema explicitly rather than smuggling executable names or arbitrary shell through body.

Required architecture

1. Portable semantic IntentFrame

Introduce one typed, transport-neutral semantic representation. Exact fields are research-owned, but it must represent at least:

  • stable intent identifier;
  • typed slots and provenance;
  • required vs optional slots;
  • missing/invalid/ambiguous slots;
  • conversation/dialogue correlation;
  • safe resolver evidence/source metadata;
  • completion state;
  • requested abstract capability, once selected;
  • no executable shell/program string as semantic authority.

Parsing an intent must not select an OS command or execute a tool.

2. First-class dialogue/slot completion

Generalize existing pending(Intent) behavior into a state machine capable of:

  • asking one focused question for missing required information;
  • filling only the pending slot from a follow-up utterance;
  • correction (actually make that five minutes);
  • cancellation (never mind);
  • expiration/timeouts;
  • invalid-value retry;
  • multiple missing slots with deterministic ordering;
  • concurrent pending intents without a process-global pending_command;
  • principal + conversation/session + intent ownership;
  • stale-turn rejection after cancellation/replacement/reconnect.

Example contract:

"set a timer"
  -> intent(timer), missing(duration)
  -> ask "How long?"
"twenty minutes"
  -> fill duration=1200 on the original intent
  -> continue capability resolution

The second utterance must not be treated as an unrelated top-level command when an unambiguous active slot request owns it.

3. Capability/provider reasoning

After a complete semantic intent, resolve an abstract capability/provider independently from parsing.

Conceptual model:

IntentFrame
    -> required capability
    -> candidate providers
    -> policy/authorization/availability constraints
    -> ExecutionPlan(provider, location, typed arguments)

Provider location must be explicit, for example:

  • server: API/service/tool/database/memory/search/server administration;
  • device: desktop/Android actions such as open URI/app, clipboard, notification, screenshot, volume;
  • future local/offline provider where negotiated.

Reuse and critically re-evaluate the ideas in draft PR #119 as prior art. Do not merge/revive it blindly; its provider reasoning must be reconciled with the authenticated daemon, typed protocol, and no-Prolog-RLM rule.

4. Server api_service boundary

Create a closed server-side service/capability layer behind RuntimeHost for actions that correctly belong to the daemon. Examples include search, memory, server timer/service operations, databases, MCP/tool-backed work, and explicitly authorized daemon administration.

The ZeroMQ gateway must never call services, Prolog, tools, or AgentManager directly. RuntimeHost remains the application-service boundary.

5. Separate semantic and platform configuration

Do not copy current Linux kb/config.pl command mappings onto the server.

Research and implement a clear split such as:

shared semantic intents/slots/capabilities
server provider/service configuration
Linux device provider configuration
future Android provider configuration

Exact filenames/predicates are research-owned. Prefer a normalized provider fact model internally when it makes reasoning easier, while preserving understandable user configuration.

6. Closed typed device actions over ZARA/1

Clients may advertise an allow-listed capability set such as open_uri, open_app, clipboard, notification, speaker, microphone, or future Android actions.

Server-to-client execution must use typed requests/results with:

  • stable action/request id;
  • initiating/target session/device identity;
  • abstract capability name;
  • closed typed argument schema;
  • deadline/cancellation;
  • authorization decision;
  • result/error type;
  • replay/idempotency policy where side effects matter.

Never send arbitrary shell, Python import/class names, executable paths, or eval-able code as a generic device action.

7. Existing subsystem integration

  • #51 remains the context-assembly owner; dialogue state must not become an alternate context manager.
  • #122 may canonicalize an utterance before semantic resolution; it may not invent capabilities/slot values or bypass this contract.
  • #124 remains the embedded Prolog LLM client where model-backed Prolog behavior is actually required.
  • #82/#83 runtime events/commands remain the runtime boundary.
  • #132 remains voice transport/device-edge owner.
  • #133 remains client migration owner.
  • #134 remains daemon Voice release gate.
  • no Prolog-RLM dependency/path.

Required example-command corpus

Tests for every slice must use realistic commands, not only synthetic predicate calls. Minimum corpus includes:

  • set a timer for twenty minutes -> complete timer duration 1200s;
  • set a timer -> missing duration -> How long?;
  • follow-up twenty minutes fills the pending timer;
  • actually make that five minutes corrects the pending/completed draft before execution where policy permits;
  • never mind cancels pending work;
  • open Firefox -> semantic open-app capability, device provider;
  • open -> ask what to open;
  • text Sarah -> missing message;
  • text Sarah tell her I'm running late -> complete message frame;
  • search for ZeroMQ CURVE authentication -> server search/service capability;
  • take a screenshot -> initiating-device capability;
  • what do you remember about X? -> server memory capability;
  • unavailable device capability -> typed unavailable result, no arbitrary fallback;
  • same semantic command from Linux and Android fixtures -> same intent, provider may differ.

Every new intent/capability added later must extend the corpus.

Adversarial requirements

Prove:

  • payload-supplied owner/device identities cannot redirect an action;
  • one principal/session cannot complete another principal/session's pending slots;
  • stale reconnect data cannot complete an old intent;
  • duplicate/replayed action requests do not duplicate side effects according to declared policy;
  • malformed/unknown capability names fail closed;
  • client capability advertisement is bounded and schema-validated;
  • a compromised/untrusted client cannot advertise itself into server/admin authority;
  • provider fallback never converts a typed capability into arbitrary shell;
  • clarification cannot be prompt-injected into bypassing capability policy;
  • resource limits bound pending dialogue state and action queues.

RAGE/TDD requirements

Each child issue must be consumed individually through the repository RAGE protocol. Before implementation its work log records exact immutable start SHA + issue. Research must be capable of changing this proposed design. Tests first, prove expected red, minimum production change, focused + full repo/Nix gates, changed-code coverage review, exact-head GitHub Actions, merge only exact green/mergeable head.

Ordered implementation slices

Create/consume child issues in this order unless RAGE research changes dependencies:

  1. architecture research + code audit + portable IntentFrame specification;
  2. typed slot schema and principal/conversation-scoped dialogue state machine;
  3. semantic resolver adaptation and comprehensive example-command corpus;
  4. capability/provider graph + ExecutionPlan and reconciliation of PR #119 prior art;
  5. server api_service provider boundary + server/device config split;
  6. ZARA/1 capability advertisement and typed device action request/result plane;
  7. migrate command routing through RuntimeHost and remove duplicated prompt-only command authority;
  8. adversarial/fuzz/concurrency/reconnect integration gate.

Completion rule

This epic is complete only when Zara can accept a realistic command, construct one portable semantic intent, clarify missing slots across turns, choose a policy-valid provider based on authenticated server/device capabilities, execute through the correct service/device boundary, and prove the same semantics across at least the Linux client and deterministic future-client fixtures without arbitrary remote execution.

## Goal Refactor Zara's command path after the daemon/client split so **intent meaning, missing-argument dialogue, capability selection, and execution location are separate first-class contracts**. The target is one portable semantic core that can serve Linux desktop/CLI, the long-lived `zara-server`, future Android clients, and future offline/device-local execution without duplicating intent vocabularies or turning the wire protocol into a remote shell. This epic is future implementation work. **Do not begin implementation before #133 has completed the supported client split.** Voice-specific integration additionally depends on #132. The daemon release decision remains owned by #134. ## Code-review baseline This epic is designed against PR #148 **as though its intended principal-isolation architecture will merge**, but does not pretend the current PR is merge-ready. Review on #148 identified pre-merge #131 integration blockers that must be closed first: - `RuntimeSupervisor._build_default_host()` currently discards its `PrincipalContext`, so default `AgentManager` construction still creates memory without an explicit authenticated principal; - the owner-bound `ConversationStore` introduced by #148 is not yet the default daemon agent conversation path; `AgentManager` still owns an in-memory `ConversationManager` history; - exact-head #148 CI currently fails because a new isolation fake publishes a `RuntimeCommand` into a `RuntimeEventBus` that correctly accepts only `RuntimeEvent`; - principal typing/transient-principal policy and concurrent SQLite migration behavior need hardening/proof. The future architecture may assume the **design outcome** of #131 (principal-bound private state), but must not depend on those gaps remaining. ## Current problems this epic owns ### Parsing is conflated with execution Current Prolog intent results are essentially `kind + name + args`, and several execution paths still jump from a resolved verb directly toward command handling. ### Missing arguments are only partially modeled `modules/intent_resolver.pl` already has a useful seed: `pending(Intent)` plus named missing slots for cases such as bare `open` and `text`. It is not general. A bare timer goes through the special timer parser and fails instead of yielding a typed request for `duration`. ### Provider location is implicit Current `kb/config.pl` is heavily Linux/device shaped (`xdg-open`, GNOME utilities, desktop apps, shutdown commands). Those mappings must not become server policy just because Prolog moves behind `zara-server`. ### The LLM agent currently duplicates routing policy in prompt text `AgentManager` contains a command-verb list and tells the model to invoke `query_prolog`. Semantic command routing should become a runtime service contract rather than prompt-only policy duplicated beside Prolog. ### `ZARA/1` has no typed device capability/action plane yet Protocol v1 is closed and currently supports the daemon text/runtime vocabulary plus reserved voice messages. Future device capabilities must extend that closed schema explicitly rather than smuggling executable names or arbitrary shell through `body`. ## Required architecture ### 1. Portable semantic `IntentFrame` Introduce one typed, transport-neutral semantic representation. Exact fields are research-owned, but it must represent at least: - stable intent identifier; - typed slots and provenance; - required vs optional slots; - missing/invalid/ambiguous slots; - conversation/dialogue correlation; - safe resolver evidence/source metadata; - completion state; - requested abstract capability, once selected; - no executable shell/program string as semantic authority. Parsing an intent must not select an OS command or execute a tool. ### 2. First-class dialogue/slot completion Generalize existing `pending(Intent)` behavior into a state machine capable of: - asking one focused question for missing required information; - filling only the pending slot from a follow-up utterance; - correction (`actually make that five minutes`); - cancellation (`never mind`); - expiration/timeouts; - invalid-value retry; - multiple missing slots with deterministic ordering; - concurrent pending intents without a process-global `pending_command`; - principal + conversation/session + intent ownership; - stale-turn rejection after cancellation/replacement/reconnect. Example contract: ```text "set a timer" -> intent(timer), missing(duration) -> ask "How long?" "twenty minutes" -> fill duration=1200 on the original intent -> continue capability resolution ``` The second utterance must not be treated as an unrelated top-level command when an unambiguous active slot request owns it. ### 3. Capability/provider reasoning After a complete semantic intent, resolve an abstract capability/provider independently from parsing. Conceptual model: ```text IntentFrame -> required capability -> candidate providers -> policy/authorization/availability constraints -> ExecutionPlan(provider, location, typed arguments) ``` Provider location must be explicit, for example: - `server`: API/service/tool/database/memory/search/server administration; - `device`: desktop/Android actions such as open URI/app, clipboard, notification, screenshot, volume; - future local/offline provider where negotiated. Reuse and critically re-evaluate the ideas in draft PR #119 as prior art. Do **not** merge/revive it blindly; its provider reasoning must be reconciled with the authenticated daemon, typed protocol, and no-Prolog-RLM rule. ### 4. Server `api_service` boundary Create a closed server-side service/capability layer behind `RuntimeHost` for actions that correctly belong to the daemon. Examples include search, memory, server timer/service operations, databases, MCP/tool-backed work, and explicitly authorized daemon administration. The ZeroMQ gateway must never call services, Prolog, tools, or AgentManager directly. `RuntimeHost` remains the application-service boundary. ### 5. Separate semantic and platform configuration Do not copy current Linux `kb/config.pl` command mappings onto the server. Research and implement a clear split such as: ```text shared semantic intents/slots/capabilities server provider/service configuration Linux device provider configuration future Android provider configuration ``` Exact filenames/predicates are research-owned. Prefer a normalized provider fact model internally when it makes reasoning easier, while preserving understandable user configuration. ### 6. Closed typed device actions over `ZARA/1` Clients may advertise an allow-listed capability set such as `open_uri`, `open_app`, `clipboard`, `notification`, `speaker`, `microphone`, or future Android actions. Server-to-client execution must use typed requests/results with: - stable action/request id; - initiating/target session/device identity; - abstract capability name; - closed typed argument schema; - deadline/cancellation; - authorization decision; - result/error type; - replay/idempotency policy where side effects matter. **Never send arbitrary shell, Python import/class names, executable paths, or eval-able code as a generic device action.** ### 7. Existing subsystem integration - #51 remains the context-assembly owner; dialogue state must not become an alternate context manager. - #122 may canonicalize an utterance before semantic resolution; it may not invent capabilities/slot values or bypass this contract. - #124 remains the embedded Prolog LLM client where model-backed Prolog behavior is actually required. - #82/#83 runtime events/commands remain the runtime boundary. - #132 remains voice transport/device-edge owner. - #133 remains client migration owner. - #134 remains daemon Voice release gate. - no Prolog-RLM dependency/path. ## Required example-command corpus Tests for every slice must use realistic commands, not only synthetic predicate calls. Minimum corpus includes: - `set a timer for twenty minutes` -> complete timer duration 1200s; - `set a timer` -> missing duration -> `How long?`; - follow-up `twenty minutes` fills the pending timer; - `actually make that five minutes` corrects the pending/completed draft before execution where policy permits; - `never mind` cancels pending work; - `open Firefox` -> semantic open-app capability, device provider; - `open` -> ask what to open; - `text Sarah` -> missing message; - `text Sarah tell her I'm running late` -> complete message frame; - `search for ZeroMQ CURVE authentication` -> server search/service capability; - `take a screenshot` -> initiating-device capability; - `what do you remember about X?` -> server memory capability; - unavailable device capability -> typed unavailable result, no arbitrary fallback; - same semantic command from Linux and Android fixtures -> same intent, provider may differ. Every new intent/capability added later must extend the corpus. ## Adversarial requirements Prove: - payload-supplied owner/device identities cannot redirect an action; - one principal/session cannot complete another principal/session's pending slots; - stale reconnect data cannot complete an old intent; - duplicate/replayed action requests do not duplicate side effects according to declared policy; - malformed/unknown capability names fail closed; - client capability advertisement is bounded and schema-validated; - a compromised/untrusted client cannot advertise itself into server/admin authority; - provider fallback never converts a typed capability into arbitrary shell; - clarification cannot be prompt-injected into bypassing capability policy; - resource limits bound pending dialogue state and action queues. ## RAGE/TDD requirements Each child issue must be consumed individually through the repository RAGE protocol. Before implementation its work log records exact immutable start SHA + issue. Research must be capable of changing this proposed design. Tests first, prove expected red, minimum production change, focused + full repo/Nix gates, changed-code coverage review, exact-head GitHub Actions, merge only exact green/mergeable head. ## Ordered implementation slices Create/consume child issues in this order unless RAGE research changes dependencies: 1. architecture research + code audit + portable `IntentFrame` specification; 2. typed slot schema and principal/conversation-scoped dialogue state machine; 3. semantic resolver adaptation and comprehensive example-command corpus; 4. capability/provider graph + `ExecutionPlan` and reconciliation of PR #119 prior art; 5. server `api_service` provider boundary + server/device config split; 6. `ZARA/1` capability advertisement and typed device action request/result plane; 7. migrate command routing through RuntimeHost and remove duplicated prompt-only command authority; 8. adversarial/fuzz/concurrency/reconnect integration gate. ## Completion rule This epic is complete only when Zara can accept a realistic command, construct one portable semantic intent, clarify missing slots across turns, choose a policy-valid provider based on authenticated server/device capabilities, execute through the correct service/device boundary, and prove the same semantics across at least the Linux client and deterministic future-client fixtures without arbitrary remote execution.
lost-rob0t commented 2026-08-22 21:59:44 +00:00 (Migrated from github.com)

Canonical child issue map

This epic is now fully issue-owned. Consume these children in order unless a RAGE research iteration changes a dependency:

  • #154 — adversarial architecture/code audit + frozen IntentFrame contract (blocked by #133)
  • #155 — typed slots + principal/conversation-scoped clarification dialogue
  • #156 — Prolog semantic adaptation + canonical realistic command corpus
  • #157 — capability/provider reasoning + typed ExecutionPlan; PR #119 is prior art only
  • #158 — server api_service providers + semantic/server/device config split
  • #159 — bounded ZARA/1 client capability advertisement + typed device action lifecycle
  • #160 — RuntimeHost-owned semantic routing; remove duplicated LLM prompt routing authority
  • #161 — adversarial reconnect/concurrency/security/capability-placement release gate

Do not invent implementation directly from the parent prose when one of these issues owns it.

## Canonical child issue map This epic is now fully issue-owned. Consume these children in order unless a RAGE research iteration changes a dependency: - [ ] #154 — adversarial architecture/code audit + frozen `IntentFrame` contract (blocked by #133) - [ ] #155 — typed slots + principal/conversation-scoped clarification dialogue - [ ] #156 — Prolog semantic adaptation + canonical realistic command corpus - [ ] #157 — capability/provider reasoning + typed `ExecutionPlan`; PR #119 is prior art only - [ ] #158 — server `api_service` providers + semantic/server/device config split - [ ] #159 — bounded ZARA/1 client capability advertisement + typed device action lifecycle - [ ] #160 — RuntimeHost-owned semantic routing; remove duplicated LLM prompt routing authority - [ ] #161 — adversarial reconnect/concurrency/security/capability-placement release gate Do not invent implementation directly from the parent prose when one of these issues owns it.
Owner

Architecture reconciliation — 2026-09-07

Current master has advanced beyond two assumptions in this epic body:

  • #133 is now closed/completed, so the old do not begin before #133 dependency gate is satisfied. Child-order/dependency gates still apply normally.
  • #124 is closed/not planned. Merged PR #233 intentionally replaced that proposed embedded-client design with the current pinned, bounded Prolog-RLM direct-mode rewrite path.

This does not change this epic's authority model. #150 remains the semantic/typed-slot/capability/ExecutionPlan authority. #122 has been evolved in place to harden the current #233 rewrite adapter, and model/RLM output remains proposal-only: it cannot invent capabilities, bypass typed slot validation, or directly authorize/execute side effects.

Interpret the old #124 remains ... and blanket no Prolog-RLM path prose in this issue as superseded implementation-mechanism text, not as a reason to reopen #124 or fork a second model stack. The safety intent remains: no unbounded model/RLM agent path and no model/RLM side-effect authority.

Do not close #150 as superseded by #624/#625/#628; those later cognitive-kernel epics build on this unfinished semantic contract.

## Architecture reconciliation — 2026-09-07 Current `master` has advanced beyond two assumptions in this epic body: - #133 is now closed/completed, so the old `do not begin before #133` dependency gate is satisfied. Child-order/dependency gates still apply normally. - #124 is closed/not planned. Merged PR #233 intentionally replaced that proposed embedded-client design with the current pinned, bounded Prolog-RLM direct-mode rewrite path. This **does not change this epic's authority model**. #150 remains the semantic/typed-slot/capability/ExecutionPlan authority. #122 has been evolved in place to harden the current #233 rewrite adapter, and model/RLM output remains proposal-only: it cannot invent capabilities, bypass typed slot validation, or directly authorize/execute side effects. Interpret the old `#124 remains ...` and blanket `no Prolog-RLM path` prose in this issue as superseded implementation-mechanism text, not as a reason to reopen #124 or fork a second model stack. The safety intent remains: no unbounded model/RLM agent path and no model/RLM side-effect authority. Do not close #150 as superseded by #624/#625/#628; those later cognitive-kernel epics build on this unfinished semantic contract.
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
nsaspy/zara#150
No description provided.