Add structured outcomes, inspection, and bounded repair (#10) #29

Merged

lost-rob0t merged 26 commits from feature/issue-10-structured-outcomes-repair into main

2026-08-12 17:09:14 +00:00

lost-rob0t commented

2026-08-12 15:40:49 +00:00

(Migrated from github.com)

Closes #10.

Canonical structured outcomes

Add rlm_outcome with one explicit execution vocabulary:

success
logical_failure
timeout
depth_exhausted
resource_exhausted
capability_denied
validation_failure
exception

plan_outcome/5 preserves final value, bindings, checkpoints, remaining plan budget, structured plan errors, and bounded transition traces. goal_outcome/3 bounds trusted-host goals by wall time/depth and preserves bound goals plus residual constraints via copy_term/3.

Inspection

plan_inspect/4 parses/validates without executing and reports normalized plans, required/provided capabilities, estimates, and structured failures.
predicate_inspect/2 exposes bounded SWI predicate metadata only; it does not dump predicate source or create an arbitrary execution path.
outcome_trace/3 enforces explicit node and serialized-byte ceilings. Oversized traces collapse to a bounded summary instead of becoming an observability escape hatch.

Scoped repair

plan_repair/6 gives a trusted repair adapter a structured observation containing status, phase, structured error, bounded trace, and remaining budget. It never decides repairs by scraping stderr or exception text.

Repair semantics preserve the original run envelope:

execution failures carry remaining step/model/tool/context/output counters into the repaired attempt;
validation failures may repair without spending counters that were never executed;
consumed counters are never reset;
repair proposal count is bounded;
each repair callback is bounded;
callback time is additionally capped by the remaining original plan wall-time, so repair itself cannot escape the run deadline;
repaired plans are parsed/validated/executed through the existing closed plan runtime and original capability set.

Control-signal integrity

Harden the trusted-tool boundary in rlm_plan so time_limit_exceeded and rlm_cancelled(...) remain control signals rather than being converted into ordinary tool exceptions. This is required for canonical timeout/cancellation semantics above running tools.

Deterministic acceptance

The branch-head deterministic suite covers:

all canonical plan/goal outcome classes;
logical failure versus exception/resource exhaustion;
CLP(FD) residual constraints;
plan and predicate inspection;
node- and byte-bounded traces;
plan timeout propagation through a running trusted tool;
execute-fail -> structured observation -> repair -> succeed;
consumed step/tool counters remaining consumed after repair;
exhausted tool budget not being reset by repair;
validation failure repair without execution-budget loss;
a slow repair callback being interrupted by the original 50ms wall-time budget before its completion marker can fire.

No temporary write-capable patch workflow remains in the branch.

REAL OpenRouter repair gate

Same-repository PR CI performs a production-provider repair path with no fake provider fallback:

invalid typed plan -> structured validation_failure observation -> trusted repair adapter -> real OpenRouter strategy selection -> Prolog materializes the allow-listed typed repair plan -> revalidation -> successful execution (REPAIR_OK)

The live adapter exposes only a closed strategy vocabulary (REPAIR_LITERAL_FINAL or ABORT). OpenRouter must choose the repair strategy from the structured diagnostic; model output is never executed as Prolog code. Prolog owns the strategy-to-plan mapping and the existing typed validator/interpreter owns execution. The adapter may make at most two real-provider attempts to tolerate free-route output variance.

Live evidence is limited to non-secret status fields: provider/requested/selected model, HTTP 200, observation status, selected strategy, plan-materialized flag, response channel, provider attempt, repair count, budget-preserved flag, and final success. Planner/repair text, API keys, Authorization headers, and environment dumps are not intentionally logged.

Do not merge until deterministic CI and the REAL OpenRouter PR job are both green on the exact head and the final diff/credential scan is clean.

Closes #10. ## Canonical structured outcomes Add `rlm_outcome` with one explicit execution vocabulary: - `success` - `logical_failure` - `timeout` - `depth_exhausted` - `resource_exhausted` - `capability_denied` - `validation_failure` - `exception` `plan_outcome/5` preserves final value, bindings, checkpoints, remaining plan budget, structured plan errors, and bounded transition traces. `goal_outcome/3` bounds trusted-host goals by wall time/depth and preserves bound goals plus residual constraints via `copy_term/3`. ## Inspection - `plan_inspect/4` parses/validates without executing and reports normalized plans, required/provided capabilities, estimates, and structured failures. - `predicate_inspect/2` exposes bounded SWI predicate metadata only; it does not dump predicate source or create an arbitrary execution path. - `outcome_trace/3` enforces explicit node and serialized-byte ceilings. Oversized traces collapse to a bounded summary instead of becoming an observability escape hatch. ## Scoped repair `plan_repair/6` gives a trusted repair adapter a structured observation containing status, phase, structured error, bounded trace, and remaining budget. It never decides repairs by scraping stderr or exception text. Repair semantics preserve the original run envelope: - execution failures carry remaining step/model/tool/context/output counters into the repaired attempt; - validation failures may repair without spending counters that were never executed; - consumed counters are never reset; - repair proposal count is bounded; - each repair callback is bounded; - callback time is additionally capped by the remaining **original** plan wall-time, so repair itself cannot escape the run deadline; - repaired plans are parsed/validated/executed through the existing closed plan runtime and original capability set. ## Control-signal integrity Harden the trusted-tool boundary in `rlm_plan` so `time_limit_exceeded` and `rlm_cancelled(...)` remain control signals rather than being converted into ordinary tool exceptions. This is required for canonical timeout/cancellation semantics above running tools. ## Deterministic acceptance The branch-head deterministic suite covers: - all canonical plan/goal outcome classes; - logical failure versus exception/resource exhaustion; - CLP(FD) residual constraints; - plan and predicate inspection; - node- and byte-bounded traces; - plan timeout propagation through a running trusted tool; - execute-fail -> structured observation -> repair -> succeed; - consumed step/tool counters remaining consumed after repair; - exhausted tool budget not being reset by repair; - validation failure repair without execution-budget loss; - a slow repair callback being interrupted by the original 50ms wall-time budget before its completion marker can fire. No temporary write-capable patch workflow remains in the branch. ## REAL OpenRouter repair gate Same-repository PR CI performs a production-provider repair path with **no fake provider fallback**: `invalid typed plan -> structured validation_failure observation -> trusted repair adapter -> real OpenRouter strategy selection -> Prolog materializes the allow-listed typed repair plan -> revalidation -> successful execution (REPAIR_OK)` The live adapter exposes only a closed strategy vocabulary (`REPAIR_LITERAL_FINAL` or `ABORT`). OpenRouter must choose the repair strategy from the structured diagnostic; model output is never executed as Prolog code. Prolog owns the strategy-to-plan mapping and the existing typed validator/interpreter owns execution. The adapter may make at most two real-provider attempts to tolerate free-route output variance. Live evidence is limited to non-secret status fields: provider/requested/selected model, HTTP 200, observation status, selected strategy, plan-materialized flag, response channel, provider attempt, repair count, budget-preserved flag, and final success. Planner/repair text, API keys, Authorization headers, and environment dumps are not intentionally logged. Do not merge until deterministic CI and the REAL OpenRouter PR job are both green on the exact head and the final diff/credential scan is clean.