P0 — harden the bounded utterance rewriter on the current Prolog-RLM direct-mode path #122

Open
opened 2026-08-16 02:41:43 +00:00 by lost-rob0t · 0 comments
lost-rob0t commented 2026-08-16 02:41:43 +00:00 (Migrated from github.com)

Status reconciliation — 2026-09-07

The feature is still needed, but its original backend dependency is obsolete.

  • #124 was correctly closed not planned: merged PR #233 deliberately replaced the proposed Zara-owned full embedded Prolog LLM client with a pinned, optional Prolog-RLM direct-mode rewrite path over the current provider architecture.
  • Current master already contains modules/rlm_rewrite.pl and deterministic tests for the basic direct-mode adapter.
  • Do not close this issue as implemented. The current adapter does not yet prove the stronger bounded rewrite contract below. In particular, malformed/unparseable model output currently falls back to Intent = ask, Args = [Resp]; prompt-time enumeration of canonical intents is not the same as structural validation of returned intent/args; and literal-slot preservation/ambiguity behavior remains part of this issue's required contract.

This issue is therefore evolved in place rather than closed or duplicated.

Goal

Provide one narrow, typed utterance-rewrite stage that improves phrasing/misspelling tolerance without letting model/RLM output become command authority.

Target ordering for voice paths follows #219:

final STT transcript
  -> transcript normalization (#215/#219, when enabled)
  -> deterministic utterance normalization
  -> optional bounded Prolog-RLM direct-mode rewrite
  -> typed rewrite validation
  -> authoritative semantic/Prolog resolution (#150)
  -> capability/policy execution

CLI/text/plugin callers must converge on the same rewrite+resolve boundary rather than creating surface-specific rewriters.

Rewrite result contract

Freeze a typed result equivalent to:

  • rewrite(CanonicalIntentOrText, TypedSlots, Evidence);
  • no_rewrite;
  • ambiguous(Candidates);
  • rewrite_error(Error).

Exact term/schema is design-owned, but raw model text is never itself a successful command.

Required behavior

Deterministic first

Handle cheap safe cases without a model where possible:

  • casing/punctuation;
  • known aliases/paraphrases;
  • reviewed spelling variants such as forcast -> forecast;
  • exact existing commands remain semantically unchanged.

Bounded current model path

When configured, reuse the pinned Prolog-RLM direct-mode path introduced by #233 rather than resurrecting #124.

Maintain/strengthen its current bounds:

  • no arbitrary tools or context operations;
  • explicit model/provider configuration;
  • deadline/cancellation;
  • bounded model calls/tokens/output bytes/cost;
  • deterministic/low-temperature structured rewrite behavior;
  • typed failures for missing runtime/credential/contract/provider errors.

Structural validation after the model

Do not trust the prompt to enforce the contract. Validate returned data before semantic resolution:

  • returned intent/canonical command belongs to the closed registered vocabulary;
  • args/slots have the expected shape, count, type and bounds for that intent;
  • unknown fields/intents fail closed;
  • malformed JSON/schema mismatch returns typed rewrite failure, not generic ask success;
  • output growth is bounded;
  • no shell/Python/Prolog goal/executable/capability injection through generic string fields;
  • canonical result is still revalidated by #150/Prolog semantic resolution before execution.

Literal slot preservation

Rewriting may normalize the command frame but must not silently invent or mutate user-provided filenames, contacts, URLs, message bodies, durations, locations or other literal/typed arguments. Where preservation cannot be proven, return ambiguity/no-rewrite and let the dialogue layer clarify.

Honest ambiguity/failure

Model failure, timeout, invalid output or multiple materially different interpretations must remain explicit. Falling back to conversational ask is allowed only as the downstream product response to a failed/ambiguous rewrite, not as a fake successful rewrite result containing arbitrary model output.

Diagnostics

Emit bounded structured evidence/status suitable for runtime diagnostics and #93/#634-style reasoning inspection without exposing hidden model chain-of-thought or transcript bodies in ordinary metrics.

Required tests

Table-driven deterministic corpus must cover at least:

  • what is the weather?;
  • tell me the forecast;
  • tell me the forcast;
  • could you please tell me the weather;
  • existing exact deterministic commands unchanged;
  • ambiguous phrases not guessed;
  • quoted/literal slot values preserved byte/semantic-equivalently as appropriate;
  • model returns unknown intent;
  • model returns malformed JSON;
  • model returns wrong arg count/type;
  • model attempts shell/tool/Prolog/capability injection;
  • oversized output;
  • timeout/cancel/provider/RLM failure;
  • normalized voice text from #219 reaches this stage exactly once;
  • stale/cancelled rewrite cannot execute or append history;
  • same contract through text/voice shared runtime path.

No default gate may require network/live provider credentials. Use scripted RLM/model handlers and local fixtures.

Acceptance

  • Natural variants of a registered capability can be canonicalized before authoritative intent resolution.
  • Exact existing commands are not semantically changed.
  • The rewriter cannot execute, grant authority, invent a capability, or bypass semantic/capability policy.
  • Model/RLM output is structurally validated rather than trusted because the prompt asked for JSON.
  • Literal user slot values are preserved or the rewrite fails/clarifies honestly.
  • Ambiguity and failures are typed and never masquerade as successful resolution.
  • One shared rewrite boundary serves voice/text/client surfaces.
  • Current pinned Prolog-RLM direct mode is reused when model-backed rewriting is enabled; #124 remains obsolete/closed.

Dependencies / integration

  • Current model implementation baseline: merged PR #233.
  • Voice ordering: #215/#219 run before this stage.
  • Semantic authority and typed slots/capabilities: #150.
  • This issue does not grant Prolog-RLM direct side-effect authority.
## Status reconciliation — 2026-09-07 The feature is still needed, but its original backend dependency is obsolete. - #124 was correctly closed `not planned`: merged PR #233 deliberately replaced the proposed Zara-owned full embedded Prolog LLM client with a pinned, optional Prolog-RLM direct-mode rewrite path over the current provider architecture. - Current `master` already contains `modules/rlm_rewrite.pl` and deterministic tests for the basic direct-mode adapter. - **Do not close this issue as implemented.** The current adapter does not yet prove the stronger bounded rewrite contract below. In particular, malformed/unparseable model output currently falls back to `Intent = ask, Args = [Resp]`; prompt-time enumeration of canonical intents is not the same as structural validation of returned intent/args; and literal-slot preservation/ambiguity behavior remains part of this issue's required contract. This issue is therefore evolved in place rather than closed or duplicated. ## Goal Provide one narrow, typed utterance-rewrite stage that improves phrasing/misspelling tolerance **without letting model/RLM output become command authority**. Target ordering for voice paths follows #219: ```text final STT transcript -> transcript normalization (#215/#219, when enabled) -> deterministic utterance normalization -> optional bounded Prolog-RLM direct-mode rewrite -> typed rewrite validation -> authoritative semantic/Prolog resolution (#150) -> capability/policy execution ``` CLI/text/plugin callers must converge on the same rewrite+resolve boundary rather than creating surface-specific rewriters. ## Rewrite result contract Freeze a typed result equivalent to: - `rewrite(CanonicalIntentOrText, TypedSlots, Evidence)`; - `no_rewrite`; - `ambiguous(Candidates)`; - `rewrite_error(Error)`. Exact term/schema is design-owned, but **raw model text is never itself a successful command**. ## Required behavior ### Deterministic first Handle cheap safe cases without a model where possible: - casing/punctuation; - known aliases/paraphrases; - reviewed spelling variants such as `forcast` -> `forecast`; - exact existing commands remain semantically unchanged. ### Bounded current model path When configured, reuse the pinned Prolog-RLM direct-mode path introduced by #233 rather than resurrecting #124. Maintain/strengthen its current bounds: - no arbitrary tools or context operations; - explicit model/provider configuration; - deadline/cancellation; - bounded model calls/tokens/output bytes/cost; - deterministic/low-temperature structured rewrite behavior; - typed failures for missing runtime/credential/contract/provider errors. ### Structural validation after the model Do not trust the prompt to enforce the contract. Validate returned data before semantic resolution: - returned intent/canonical command belongs to the closed registered vocabulary; - args/slots have the expected shape, count, type and bounds for that intent; - unknown fields/intents fail closed; - malformed JSON/schema mismatch returns typed rewrite failure, **not generic `ask` success**; - output growth is bounded; - no shell/Python/Prolog goal/executable/capability injection through generic string fields; - canonical result is still revalidated by #150/Prolog semantic resolution before execution. ### Literal slot preservation Rewriting may normalize the command frame but must not silently invent or mutate user-provided filenames, contacts, URLs, message bodies, durations, locations or other literal/typed arguments. Where preservation cannot be proven, return ambiguity/no-rewrite and let the dialogue layer clarify. ### Honest ambiguity/failure Model failure, timeout, invalid output or multiple materially different interpretations must remain explicit. Falling back to conversational `ask` is allowed only as the **downstream product response to a failed/ambiguous rewrite**, not as a fake successful rewrite result containing arbitrary model output. ### Diagnostics Emit bounded structured evidence/status suitable for runtime diagnostics and #93/#634-style reasoning inspection without exposing hidden model chain-of-thought or transcript bodies in ordinary metrics. ## Required tests Table-driven deterministic corpus must cover at least: - `what is the weather?`; - `tell me the forecast`; - `tell me the forcast`; - `could you please tell me the weather`; - existing exact deterministic commands unchanged; - ambiguous phrases not guessed; - quoted/literal slot values preserved byte/semantic-equivalently as appropriate; - model returns unknown intent; - model returns malformed JSON; - model returns wrong arg count/type; - model attempts shell/tool/Prolog/capability injection; - oversized output; - timeout/cancel/provider/RLM failure; - normalized voice text from #219 reaches this stage exactly once; - stale/cancelled rewrite cannot execute or append history; - same contract through text/voice shared runtime path. No default gate may require network/live provider credentials. Use scripted RLM/model handlers and local fixtures. ## Acceptance - Natural variants of a registered capability can be canonicalized before authoritative intent resolution. - Exact existing commands are not semantically changed. - The rewriter cannot execute, grant authority, invent a capability, or bypass semantic/capability policy. - Model/RLM output is structurally validated rather than trusted because the prompt asked for JSON. - Literal user slot values are preserved or the rewrite fails/clarifies honestly. - Ambiguity and failures are typed and never masquerade as successful resolution. - One shared rewrite boundary serves voice/text/client surfaces. - Current pinned Prolog-RLM direct mode is reused when model-backed rewriting is enabled; #124 remains obsolete/closed. ## Dependencies / integration - Current model implementation baseline: merged PR #233. - Voice ordering: #215/#219 run before this stage. - Semantic authority and typed slots/capabilities: #150. - This issue does not grant Prolog-RLM direct side-effect authority.
nsaspy changed title from P0 — add a bounded utterance rewriter before Prolog intent resolution to P0 — harden the bounded utterance rewriter on the current Prolog-RLM direct-mode path 2026-09-08 02:03:28 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
nsaspy/zara#122
No description provided.