nsaspy/prolog-rlm

Fork 0

[P0] Converge skills into prompt compiler and add permanent RLM operator context #183

New issue

Open

opened 2026-08-25 00:14:05 +00:00 by lost-rob0t · 20 comments

lost-rob0t commented

2026-08-25 00:14:05 +00:00

(Migrated from github.com)

Parent/related: #117, #173, #174, #176, #172

Goal

Make skills a first-class part of the one existing provider-context compiler, then use that machinery to ship a small set of RLM operating skills that are enabled by default and remain present throughout an RLM/agent run unless trusted host configuration disables them.

This is the ADARD decision for the current skills/RLM-context work. No code is part of this issue creation.

Research findings

Agent Skills

The open Agent Skills specification defines SKILL.md as portable instructions plus metadata/resources and recommends progressive disclosure: metadata first, full instructions after activation, resources on demand. It provides metadata as a string->string extension point; allowed-tools is experimental and must not be treated as authority.

The client implementation guide also explicitly recommends protecting activated skill instructions from later context compaction because silently losing skill instructions degrades agent behavior. It allows harness-side/programmatic injection of skill content; model-driven activation is only one possible client strategy.

References:

Recursive Language Models

The RLM paper/runtime treats context interaction and recursive sub-calls as part of the model's operating environment. The authors' reference implementation uses a default system prompt that explicitly teaches the model how to use context, llm_query/rlm_query, and finalization; it also exposes a custom system-prompt override. Operational RLM instructions therefore must be guaranteed provider-visible runtime context, not merely implicit knowledge.

References:

Typed/symbolic RLM research also supports keeping recursion/control flow explicit rather than relying on unconstrained free-form execution:

https://arxiv.org/abs/2603.20105

Prolog constraints

SWI-Prolog CLP(FD) is explicitly intended for declarative integer reasoning and combinatorial problems such as planning, scheduling, and allocation. The agent should recognize those problem classes and prefer an available constraint capability over model guessing.

Reference:

https://www.swi-prolog.org/man/clpfd.html

Current repository findings

#173 already freezes the correct architecture

#173 says:

SKILL.md
 -> normalized_skill
 -> existing prompt_unit{unit:skill(...)} IR
 -> existing rlm_prompt_compiler

It explicitly rejects a second prompt compiler/skill IR and makes provider visibility and mandatory-context policy host-owned.

PR #122 currently diverges from that decision

PR #122 (feature/117-prolog-skill-activation) has useful loader/confinement/resource/default-corpus work, but it also implements a separate skill_compile/4 selector with its own lexical scoring, thresholds, count/token limits, dependency resolution and fingerprinting, then merges selected skill text into planner_instruction through rlm_skill_completion.

That selection path overlaps the already-landed rlm_prompt_compiler, which already supports kind:skill, dependencies, conflicts, capabilities, packing, reasons and fingerprints.

PR #122 should therefore be reconciled as loader/normalizer + compatibility adapter + package/resource layer, not preserved as a second independent provider-context selector.

At the time of this decision, current main is 982bad23c59dba0dd9157c1904e7723950889717; PR #122 head is 1a34d01dad9020a72a17b9bbae2a698451f575aa and is substantially diverged from current main.

#174 has the right generic permanent-visibility primitive

activation:always should apply to ordinary prompt units, not only tools. mandatory_context:true remains the post-selection packing invariant. Natural-language negation must not unpin a host-pinned unit. Trusted host disable/narrowing may.

#176 is the provider-surface gate

The design is incomplete until the actual provider-bound request is compiled from the prompt compiler. Internal selected/active_units state is not sufficient evidence that the model received the instructions.

Decision

1. One compiler

There is one canonical provider-context selector/packer:

SKILL.md loader/normalizer
        |
        v
prompt_unit{unit:skill(...)}
        |
        v
rlm_prompt_compiler
        |
        +--> permanent active units
        +--> relevant active units
        +--> active tool schemas
        +--> provider-visible instructions/resources
        v
provider renderer/request

rlm_skill may own filesystem discovery, standard/vendor parsing, normalization, lazy resources and package security. It must not own a competing routing/scoring/packing engine.

2. Permanent context is host policy

An ordinary skill/tool/instruction can be host-pinned:

activation:always
mandatory_context:true
provider_visible:true

Meaning:

no lexical/relevance evidence required;
ordinary candidate limits cannot evict it;
user/model prose cannot disable it;
context packing cannot silently omit it;
insufficient budget fails structurally;
only trusted host configuration can disable/narrow it.

A SKILL.md package MUST NOT be able to mark itself permanent/mandatory/trusted. #173 already establishes mandatory/provider policy as host-owned. Standard/vendor activation metadata may disable or influence ordinary automatic activation, but cannot promote itself to always.

3. Default RLM operating skill bundle

Ship a small bundled set of trusted skills, enabled and pinned by the default RLM agent profile:

`rlm-operate`

Teach the actual typed runtime contract: available plan/context/model/tool/RLM operations, bindings/references, finalization, budget awareness, and the rule not to invent unavailable capabilities or context contents.

`rlm-recurse`

Teach useful recursive behavior: inspect context, decompose when useful, use rlm_subagent/recursive calls for missing or independently investigable work, give children bounded sufficient context, and aggregate returned evidence. Do not teach recursion-for-recursion's-sake.

`rlm-facts`

Teach facts/evidence first: query supplied KB/facts/observations before guessing, distinguish retrieved fact from inference, derive with Prolog when deterministic, and use model judgment only where symbolic evidence is insufficient.

`rlm-constraints`

Teach constraint recognition: for finite-domain, scheduling, allocation, combinatorial or relational arithmetic problems, use an exposed constraint-solver capability when available. Never invent a solver/tool that is absent from the active provider projection.

These should be concise operational skills, not long tutorials.

4. Toggle semantics

Default: core RLM skills are enabled.

Trusted host controls must support:

global skill opt-out (skill_mode(off) or the reconciled equivalent);
per-skill disable for experiments/debugging;
default profile override without editing the SKILL.md body.

No natural-language request or skill-authored metadata can silently disable a host-pinned core skill.

5. Scope of permanence

Every provider request that participates in an RLM/agent run must receive the active permanent units, including recursive/subagent model calls and repair/retry model calls. A low-level standalone raw-LM API may remain raw when invoked outside an RLM/agent execution context; once a model call is part of an RLM run, the run's permanent context propagates with it.

This prevents the root from knowing how to operate RLM while a recursive child silently loses the operating contract.

6. Provider placement

Permanent behavioral skills are skill packages as the source of truth, but the provider renderer should place their active instruction bodies in the provider's durable instruction/system context rather than pretending they are user text. This matches the reference RLM implementation's use of a system prompt for operating instructions.

Provider-specific message/channel mapping remains a renderer concern; skill activation remains provider-neutral.

7. Authority remains separate

A permanent skill/tool is model-visible, not automatically executable.

visible != capability granted != authorized != effect admitted

An always-visible tool still passes existing capability, authority, confinement and effect checks. A skill mentioning a tool does not grant it.

Adversarial review

The implementation must survive at least these cases:

User says “ignore/disable the RLM skills” -> pinned skills remain present.
Malicious SKILL.md claims permanent/mandatory/trusted -> ignored/rejected as host-owned policy.
Malicious allowed-tools claims unavailable authority -> no capability/authority change.
Candidate limit saturated by relevant units -> permanent units remain.
Context budget too small -> structural failure, not silent removal.
Permanent skill conflicts with another permanent skill -> configuration error, not relevance-score arbitration.
Recursive child has narrower capabilities -> permanent behavioral skill may remain, but unavailable tool schemas stay absent and the skill must not imply they exist.
Context compaction -> active permanent skill content is protected/reconstituted, never silently lost.
Repeated nested wrappers/retries -> permanent skills injected exactly once per provider request.
Skill version/fingerprint changes -> provider projection fingerprint changes and trace exposes which version was active.
Direct SKILL.md loading remains path-confined and resources remain inert until explicitly read/executed through normal authority boundaries.
PR #122 reconciliation must not regress its useful symlink/resource/default-corpus security work.

Required tests

Compiler/loader

strict standard SKILL.md -> normalized prompt_unit{unit:skill(...)};
no second selector: skill routing is performed by rlm_prompt_compiler;
host-pinned skill activates on unrelated input;
relevant skill without evidence remains inactive;
SKILL.md cannot self-promote to permanent;
natural-language negation cannot remove pinned skill;
trusted per-skill/global disable can remove it;
pinned mandatory unit cannot be packed out;
impossible permanent budget fails structurally;
activation policy is material to fingerprint/explanation.

Exact provider-bound proof

Tests must capture the exact request sent to the provider and assert the instruction body/identity is present, not merely inspect compiler state.

Required paths:

root RLM planner request contains all default core RLM skills;
recursive/subagent provider request contains them;
RLM-internal leaf/model request contains them when part of the run;
retry/repair request contains them;
unrelated task still contains them;
user text attempting to disable them does not remove them;
trusted opt-out removes them;
ordinary relevant skill is absent on unrelated task and present on matching task;
no duplicate injection.

Behavioral model tests — no spoon-fed plan

Do not use planner_instruction to provide the exact plan JSON/steps under test.

At least:

decomposable fixture: model independently chooses recursive/subagent work and combines results;
trivial fixture: model does not recurse unnecessarily;
facts fixture: model uses authoritative supplied facts rather than inventing an answer;
finite constraint fixture: with solver capability exposed, model chooses the solver path;
same constraint fixture without solver exposure: model does not hallucinate the unavailable solver;
unknown-information fixture: model uses an available subagent/context/tool path instead of guessing.

Reconciliation requirements

#117 / PR #122

Keep and reconcile the useful package loader, standard/vendor compatibility, resource indexing/confinement, pinned corpus and provenance work. Replace/delegate the independent skill scoring/selection/packing path with normalized prompt-unit registration into rlm_prompt_compiler, per #173.

#174

Treat activation:always as generic to all prompt units, including skill(...) and instruction(...), not a tool-only feature. Host policy owns permanent activation.

#176

Extend production provider-surface acceptance to prove permanent skill bodies and instructions reach the exact model request across the RLM run, not just tool schemas at the root planner.

#172

Skill-aware subagent delegation should inherit the parent's permanent skill set by default while still narrowing child capabilities/authority and dynamically selecting additional child-specific relevant skills.

Non-goals

no second skill compiler/router;
no special core_skill registry;
no hardcoded giant planner prompt;
no model-controlled permanence;
no skill-controlled authority;
no requirement that all ordinary skills be permanent;
no requirement that a standalone raw LM call outside an RLM/agent run become an RLM agent;
no implementation in this ADARD issue.

Acceptance decision

APPROVED architecture: standard SKILL.md loader -> normalized skill/prompt unit -> single prompt compiler -> host-pinned permanent + dynamically relevant context -> provider-bound request, with a default toggleable RLM operating skill bundle and exact-request tests proving the model actually receives it.

Parent/related: #117, #173, #174, #176, #172 ## Goal Make skills a first-class part of the **one existing provider-context compiler**, then use that machinery to ship a small set of RLM operating skills that are enabled by default and remain present throughout an RLM/agent run unless trusted host configuration disables them. This is the ADARD decision for the current skills/RLM-context work. No code is part of this issue creation. ## Research findings ### Agent Skills The open Agent Skills specification defines `SKILL.md` as portable instructions plus metadata/resources and recommends progressive disclosure: metadata first, full instructions after activation, resources on demand. It provides `metadata` as a string->string extension point; `allowed-tools` is experimental and must not be treated as authority. The client implementation guide also explicitly recommends protecting activated skill instructions from later context compaction because silently losing skill instructions degrades agent behavior. It allows harness-side/programmatic injection of skill content; model-driven activation is only one possible client strategy. References: - https://agentskills.io/specification - https://agentskills.io/client-implementation/adding-skills-support - https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview ### Recursive Language Models The RLM paper/runtime treats context interaction and recursive sub-calls as part of the model's operating environment. The authors' reference implementation uses a default **system prompt** that explicitly teaches the model how to use context, `llm_query`/`rlm_query`, and finalization; it also exposes a custom system-prompt override. Operational RLM instructions therefore must be guaranteed provider-visible runtime context, not merely implicit knowledge. References: - https://arxiv.org/abs/2512.24601 - https://github.com/alexzhang13/rlm - https://github.com/alexzhang13/rlm-minimal/blob/main/rlm/utils/prompts.py Typed/symbolic RLM research also supports keeping recursion/control flow explicit rather than relying on unconstrained free-form execution: - https://arxiv.org/abs/2603.20105 ### Prolog constraints SWI-Prolog CLP(FD) is explicitly intended for declarative integer reasoning and combinatorial problems such as planning, scheduling, and allocation. The agent should recognize those problem classes and prefer an available constraint capability over model guessing. Reference: - https://www.swi-prolog.org/man/clpfd.html ## Current repository findings ### #173 already freezes the correct architecture #173 says: ```text SKILL.md -> normalized_skill -> existing prompt_unit{unit:skill(...)} IR -> existing rlm_prompt_compiler ``` It explicitly rejects a second prompt compiler/skill IR and makes provider visibility and mandatory-context policy host-owned. ### PR #122 currently diverges from that decision PR #122 (`feature/117-prolog-skill-activation`) has useful loader/confinement/resource/default-corpus work, but it also implements a separate `skill_compile/4` selector with its own lexical scoring, thresholds, count/token limits, dependency resolution and fingerprinting, then merges selected skill text into `planner_instruction` through `rlm_skill_completion`. That selection path overlaps the already-landed `rlm_prompt_compiler`, which already supports `kind:skill`, dependencies, conflicts, capabilities, packing, reasons and fingerprints. PR #122 should therefore be reconciled as **loader/normalizer + compatibility adapter + package/resource layer**, not preserved as a second independent provider-context selector. At the time of this decision, current `main` is `982bad23c59dba0dd9157c1904e7723950889717`; PR #122 head is `1a34d01dad9020a72a17b9bbae2a698451f575aa` and is substantially diverged from current main. ### #174 has the right generic permanent-visibility primitive `activation:always` should apply to ordinary prompt units, not only tools. `mandatory_context:true` remains the post-selection packing invariant. Natural-language negation must not unpin a host-pinned unit. Trusted host disable/narrowing may. ### #176 is the provider-surface gate The design is incomplete until the actual provider-bound request is compiled from the prompt compiler. Internal `selected`/`active_units` state is not sufficient evidence that the model received the instructions. ## Decision ### 1. One compiler There is one canonical provider-context selector/packer: ```text SKILL.md loader/normalizer | v prompt_unit{unit:skill(...)} | v rlm_prompt_compiler | +--> permanent active units +--> relevant active units +--> active tool schemas +--> provider-visible instructions/resources v provider renderer/request ``` `rlm_skill` may own filesystem discovery, standard/vendor parsing, normalization, lazy resources and package security. It must not own a competing routing/scoring/packing engine. ### 2. Permanent context is host policy An ordinary skill/tool/instruction can be host-pinned: ```text activation:always mandatory_context:true provider_visible:true ``` Meaning: - no lexical/relevance evidence required; - ordinary candidate limits cannot evict it; - user/model prose cannot disable it; - context packing cannot silently omit it; - insufficient budget fails structurally; - only trusted host configuration can disable/narrow it. A `SKILL.md` package MUST NOT be able to mark itself permanent/mandatory/trusted. #173 already establishes mandatory/provider policy as host-owned. Standard/vendor activation metadata may disable or influence ordinary automatic activation, but cannot promote itself to `always`. ### 3. Default RLM operating skill bundle Ship a small bundled set of trusted skills, enabled and pinned by the default RLM agent profile: #### `rlm-operate` Teach the actual typed runtime contract: available plan/context/model/tool/RLM operations, bindings/references, finalization, budget awareness, and the rule not to invent unavailable capabilities or context contents. #### `rlm-recurse` Teach useful recursive behavior: inspect context, decompose when useful, use `rlm_subagent`/recursive calls for missing or independently investigable work, give children bounded sufficient context, and aggregate returned evidence. Do not teach recursion-for-recursion's-sake. #### `rlm-facts` Teach facts/evidence first: query supplied KB/facts/observations before guessing, distinguish retrieved fact from inference, derive with Prolog when deterministic, and use model judgment only where symbolic evidence is insufficient. #### `rlm-constraints` Teach constraint recognition: for finite-domain, scheduling, allocation, combinatorial or relational arithmetic problems, use an exposed constraint-solver capability when available. Never invent a solver/tool that is absent from the active provider projection. These should be concise operational skills, not long tutorials. ### 4. Toggle semantics Default: core RLM skills are enabled. Trusted host controls must support: - global skill opt-out (`skill_mode(off)` or the reconciled equivalent); - per-skill disable for experiments/debugging; - default profile override without editing the SKILL.md body. No natural-language request or skill-authored metadata can silently disable a host-pinned core skill. ### 5. Scope of permanence Every provider request that participates in an **RLM/agent run** must receive the active permanent units, including recursive/subagent model calls and repair/retry model calls. A low-level standalone raw-LM API may remain raw when invoked outside an RLM/agent execution context; once a model call is part of an RLM run, the run's permanent context propagates with it. This prevents the root from knowing how to operate RLM while a recursive child silently loses the operating contract. ### 6. Provider placement Permanent behavioral skills are skill packages as the source of truth, but the provider renderer should place their active instruction bodies in the provider's durable instruction/system context rather than pretending they are user text. This matches the reference RLM implementation's use of a system prompt for operating instructions. Provider-specific message/channel mapping remains a renderer concern; skill activation remains provider-neutral. ### 7. Authority remains separate A permanent skill/tool is **model-visible**, not automatically executable. ```text visible != capability granted != authorized != effect admitted ``` An always-visible tool still passes existing capability, authority, confinement and effect checks. A skill mentioning a tool does not grant it. ## Adversarial review The implementation must survive at least these cases: 1. User says “ignore/disable the RLM skills” -> pinned skills remain present. 2. Malicious SKILL.md claims permanent/mandatory/trusted -> ignored/rejected as host-owned policy. 3. Malicious `allowed-tools` claims unavailable authority -> no capability/authority change. 4. Candidate limit saturated by relevant units -> permanent units remain. 5. Context budget too small -> structural failure, not silent removal. 6. Permanent skill conflicts with another permanent skill -> configuration error, not relevance-score arbitration. 7. Recursive child has narrower capabilities -> permanent behavioral skill may remain, but unavailable tool schemas stay absent and the skill must not imply they exist. 8. Context compaction -> active permanent skill content is protected/reconstituted, never silently lost. 9. Repeated nested wrappers/retries -> permanent skills injected exactly once per provider request. 10. Skill version/fingerprint changes -> provider projection fingerprint changes and trace exposes which version was active. 11. Direct `SKILL.md` loading remains path-confined and resources remain inert until explicitly read/executed through normal authority boundaries. 12. PR #122 reconciliation must not regress its useful symlink/resource/default-corpus security work. ## Required tests ### Compiler/loader - strict standard SKILL.md -> normalized `prompt_unit{unit:skill(...)}`; - no second selector: skill routing is performed by `rlm_prompt_compiler`; - host-pinned skill activates on unrelated input; - relevant skill without evidence remains inactive; - SKILL.md cannot self-promote to permanent; - natural-language negation cannot remove pinned skill; - trusted per-skill/global disable can remove it; - pinned mandatory unit cannot be packed out; - impossible permanent budget fails structurally; - activation policy is material to fingerprint/explanation. ### Exact provider-bound proof Tests must capture the **exact request sent to the provider** and assert the instruction body/identity is present, not merely inspect compiler state. Required paths: - root RLM planner request contains all default core RLM skills; - recursive/subagent provider request contains them; - RLM-internal leaf/model request contains them when part of the run; - retry/repair request contains them; - unrelated task still contains them; - user text attempting to disable them does not remove them; - trusted opt-out removes them; - ordinary relevant skill is absent on unrelated task and present on matching task; - no duplicate injection. ### Behavioral model tests — no spoon-fed plan Do not use `planner_instruction` to provide the exact plan JSON/steps under test. At least: 1. decomposable fixture: model independently chooses recursive/subagent work and combines results; 2. trivial fixture: model does not recurse unnecessarily; 3. facts fixture: model uses authoritative supplied facts rather than inventing an answer; 4. finite constraint fixture: with solver capability exposed, model chooses the solver path; 5. same constraint fixture without solver exposure: model does not hallucinate the unavailable solver; 6. unknown-information fixture: model uses an available subagent/context/tool path instead of guessing. ## Reconciliation requirements ### #117 / PR #122 Keep and reconcile the useful package loader, standard/vendor compatibility, resource indexing/confinement, pinned corpus and provenance work. Replace/delegate the independent skill scoring/selection/packing path with normalized prompt-unit registration into `rlm_prompt_compiler`, per #173. ### #174 Treat `activation:always` as generic to **all prompt units**, including `skill(...)` and `instruction(...)`, not a tool-only feature. Host policy owns permanent activation. ### #176 Extend production provider-surface acceptance to prove permanent skill bodies and instructions reach the exact model request across the RLM run, not just tool schemas at the root planner. ### #172 Skill-aware subagent delegation should inherit the parent's permanent skill set by default while still narrowing child capabilities/authority and dynamically selecting additional child-specific relevant skills. ## Non-goals - no second skill compiler/router; - no special `core_skill` registry; - no hardcoded giant planner prompt; - no model-controlled permanence; - no skill-controlled authority; - no requirement that all ordinary skills be permanent; - no requirement that a standalone raw LM call outside an RLM/agent run become an RLM agent; - no implementation in this ADARD issue. ## Acceptance decision **APPROVED architecture:** standard SKILL.md loader -> normalized skill/prompt unit -> single prompt compiler -> host-pinned permanent + dynamically relevant context -> provider-bound request, with a default toggleable RLM operating skill bundle and exact-request tests proving the model actually receives it.

lost-rob0t commented

2026-08-25 00:18:41 +00:00

(Migrated from github.com)

Workflow policy update

Effective for this design and follow-on implementation work:

Normal ARARD flow is the default.
Auto-RAGE is not the default workflow.
RAGE is gated to explicitly approved designs only.
A RAGE loop may implement/verify/refine within an approved design boundary, but it must not silently make new architectural decisions.
If implementation exposes a design conflict, missing architecture, or materially different direction, stop the RAGE expansion and return to ARARD/design review for approval first.

For the current skills/permanent-context work, #183 is the approved design gate. PRs #122, #125, and #132 have been linked back here with their specific reconciliation/ownership constraints.

## Workflow policy update Effective for this design and follow-on implementation work: - **Normal ARARD flow is the default.** - **Auto-RAGE is not the default workflow.** - **RAGE is gated to explicitly approved designs only.** - A RAGE loop may implement/verify/refine within an approved design boundary, but it must not silently make new architectural decisions. - If implementation exposes a design conflict, missing architecture, or materially different direction, stop the RAGE expansion and return to ARARD/design review for approval first. For the current skills/permanent-context work, **#183 is the approved design gate**. PRs #122, #125, and #132 have been linked back here with their specific reconciliation/ownership constraints.

lost-rob0t commented

2026-08-26 11:11:18 +00:00

(Migrated from github.com)

Live RAGE reconciliation against canonical main 267697bef10a3fffff7c093e1435ece770e7444b:

#197 has landed the approved one-compiler skill path, generic activation:always, trusted default core bundle (rlm-operate, rlm-recurse, rlm-facts, rlm-constraints), and exact root-planner provider projection.
#201 has landed permanent RLM provider-context propagation into internal/nested/retry model calls exactly once, without changing raw standalone llm_query/3.
#216 has landed compiler-owned root-planner local-tool schema visibility while keeping trusted executable bindings separate.
#225 already owns the real finite-constraint behavioral benchmark under #223, including the authoritative core_minimal lane with no benchmark-specific exact-plan instruction. Do not duplicate that work here.

The remaining #183 acceptance is therefore primarily behavioral provider evidence, not another compiler/skill implementation. Current test/live_completion_openrouter_test.pl is still a plumbing acceptance test: it supplies an exact plan through planner_instruction, so it cannot prove the model independently learned the RLM operating contract from the default skills.

Remaining non-overlapping behavioral matrix inside the approved #183 design:

decomposable task: model independently chooses useful recursion/delegation and combines evidence;
trivial task: model avoids unnecessary recursion;
facts task: model uses authoritative supplied facts/KB evidence rather than inventing an answer;
unknown-information task: model chooses an available context/tool/subagent path instead of guessing;
constraint behavior not already covered by #225: with the solver absent from the active projection, the model must not hallucinate/invoke it.

No new architecture is implied by this reconciliation. These are provider-behavior acceptance tests over the already-approved #183 compiler/permanent-context design. Keep them out of #225's owned CSP implementation and out of downstream A0/AgentProlog product glue.

Live RAGE reconciliation against canonical `main` `267697bef10a3fffff7c093e1435ece770e7444b`: - #197 has landed the approved one-compiler skill path, generic `activation:always`, trusted default core bundle (`rlm-operate`, `rlm-recurse`, `rlm-facts`, `rlm-constraints`), and exact root-planner provider projection. - #201 has landed permanent RLM provider-context propagation into internal/nested/retry model calls exactly once, without changing raw standalone `llm_query/3`. - #216 has landed compiler-owned root-planner local-tool schema visibility while keeping trusted executable bindings separate. - #225 already owns the real finite-constraint behavioral benchmark under #223, including the authoritative `core_minimal` lane with no benchmark-specific exact-plan instruction. Do not duplicate that work here. The remaining #183 acceptance is therefore primarily **behavioral provider evidence**, not another compiler/skill implementation. Current `test/live_completion_openrouter_test.pl` is still a plumbing acceptance test: it supplies an exact plan through `planner_instruction`, so it cannot prove the model independently learned the RLM operating contract from the default skills. Remaining non-overlapping behavioral matrix inside the approved #183 design: 1. decomposable task: model independently chooses useful recursion/delegation and combines evidence; 2. trivial task: model avoids unnecessary recursion; 3. facts task: model uses authoritative supplied facts/KB evidence rather than inventing an answer; 4. unknown-information task: model chooses an available context/tool/subagent path instead of guessing; 5. constraint behavior not already covered by #225: with the solver absent from the active projection, the model must not hallucinate/invoke it. No new architecture is implied by this reconciliation. These are provider-behavior acceptance tests over the already-approved #183 compiler/permanent-context design. Keep them out of #225's owned CSP implementation and out of downstream A0/AgentProlog product glue.

lost-rob0t commented

2026-08-26 12:30:29 +00:00

(Migrated from github.com)

RAGE continuation gate on exact canonical main 267697bef10a3fffff7c093e1435ece770e7444b.

Analyze / research

The approved #183 architecture is already substantially realized: the one compiler/default skill path, permanent nested provider-context propagation, and compiler-owned root tool projection are merged. Current test/live_completion_openrouter_test.pl still spoon-feeds the exact plan through planner_instruction/1, so it proves plumbing rather than independent model operation. #225 already owns the solver-exposed finite-CSP benchmark; no open PR or branch owns the remaining non-overlapping #183 behavioral-provider acceptance.

Downstream live backlog confirms the ownership boundary: a0-symbolics expects a thin pinned Prolog-RLM consumer and explicitly rejects duplicated compiler/authority policy; agentProlog likewise assigns generic RLM/compiler/subagent semantics upstream.

Design — first behavioral vertical slice

Add a credential-backed live acceptance suite that uses the ordinary rlm_completion/4 path with the default core skills and no planner_instruction exact-plan injection. Start with three falsifiable behaviors:

trivial — a task satisfiable directly must complete with recursion.recursive_calls = 0;
unknown information with an available typed tool — the model must discover/use the visible tool and return evidence from its result rather than guess;
decomposable opaque-context task — with rlm + bounded child model capability available, the model must choose at least one recursive call and combine child evidence.

Keep this as provider-behavior acceptance, not a second planner policy. The live gate may choose a pinned/configured OpenRouter model, but the test must inspect runtime transitions/results rather than accept prose claims about what the model did.

Adversarial review

No exact plan JSON or step sequence in model instructions.
No benchmark-specific hidden answer in the system skill bodies.
Tool visibility remains separate from capability/authority; the test uses the canonical registry path.
A malformed/failed provider plan is a hard test failure, not an empty success or skip.
Trivial-case recursion is a failure, preventing rlm-recurse from degenerating into recursion-for-recursion's-sake.
Tool case requires observable tool transition/result evidence, not merely the correct final token.
Recursive case requires observable canonical rlm transition/depth evidence, not self-reported decomposition.
Keep #225's finite constraint lane untouched.

Decision gate: GO. This stays entirely inside the already-approved #183 behavioral-test boundary; if live evidence shows the default skills are insufficient, preserve that failure and return to the existing #183 design constraints rather than spoon-feeding a plan or weakening the test.

RAGE continuation gate on exact canonical `main` `267697bef10a3fffff7c093e1435ece770e7444b`. ## Analyze / research The approved #183 architecture is already substantially realized: the one compiler/default skill path, permanent nested provider-context propagation, and compiler-owned root tool projection are merged. Current `test/live_completion_openrouter_test.pl` still spoon-feeds the exact plan through `planner_instruction/1`, so it proves plumbing rather than independent model operation. #225 already owns the solver-exposed finite-CSP benchmark; no open PR or branch owns the remaining non-overlapping #183 behavioral-provider acceptance. Downstream live backlog confirms the ownership boundary: `a0-symbolics` expects a thin pinned Prolog-RLM consumer and explicitly rejects duplicated compiler/authority policy; `agentProlog` likewise assigns generic RLM/compiler/subagent semantics upstream. ## Design — first behavioral vertical slice Add a credential-backed live acceptance suite that uses the ordinary `rlm_completion/4` path with the default core skills and **no `planner_instruction` exact-plan injection**. Start with three falsifiable behaviors: 1. **trivial** — a task satisfiable directly must complete with `recursion.recursive_calls = 0`; 2. **unknown information with an available typed tool** — the model must discover/use the visible tool and return evidence from its result rather than guess; 3. **decomposable opaque-context task** — with `rlm` + bounded child model capability available, the model must choose at least one recursive call and combine child evidence. Keep this as provider-behavior acceptance, not a second planner policy. The live gate may choose a pinned/configured OpenRouter model, but the test must inspect runtime transitions/results rather than accept prose claims about what the model did. ## Adversarial review - No exact plan JSON or step sequence in model instructions. - No benchmark-specific hidden answer in the system skill bodies. - Tool visibility remains separate from capability/authority; the test uses the canonical registry path. - A malformed/failed provider plan is a hard test failure, not an empty success or skip. - Trivial-case recursion is a failure, preventing `rlm-recurse` from degenerating into recursion-for-recursion's-sake. - Tool case requires observable tool transition/result evidence, not merely the correct final token. - Recursive case requires observable canonical `rlm` transition/depth evidence, not self-reported decomposition. - Keep #225's finite constraint lane untouched. Decision gate: **GO**. This stays entirely inside the already-approved #183 behavioral-test boundary; if live evidence shows the default skills are insufficient, preserve that failure and return to the existing #183 design constraints rather than spoon-feeding a plan or weakening the test.

lost-rob0t commented

2026-08-26 12:42:56 +00:00

(Migrated from github.com)

RAGE TDD evidence / design refinement on current branch head 00384f592cd245645860cd3462dc7e53e0dd8f11, still based on canonical main 267697bef10a3fffff7c093e1435ece770e7444b.

The initial harness-placement failure is resolved without weakening deterministic corpus ownership: the credential-only suite now lives under the existing test/support/ exclusion and is invoked explicitly by test/run_live_openrouter.pl.

The real provider regression is now isolated. On exact head 00384f..., deterministic CI, Nix, clean-pack, and Tree-sitter pass their relevant jobs, while both REAL OpenRouter and Paid OpenRouter fail the new unscripted behavior suite. Existing scripted live OpenRouter cases pass first. All three new cases then fail after two planner attempts with the same structured cause:

plan_parse_failed -> invalid_plan -> missing_field(steps)

This is strong evidence that transport/provider plumbing works but the default operator contract does not currently teach the unscripted root planner enough of the typed-plan output ABI. The failure is before the trivial/tool/recursion policy assertions, so those remain valid downstream behavioral gates.

Refined design

First realization change is deliberately skill-owned and generic: strengthen only the bundled rlm-operate instruction so it states the minimal planner ABI already required by rlm_plan:

when acting as a planner, emit exactly one JSON object with top-level steps;
each step is one supported typed runtime operation;
use only current capabilities, active tool schemas, and earlier bindings/references;
inspect opaque context through typed context operations rather than inventing contents;
recurse only through typed rlm when useful;
terminate through typed final;
respect budgets and never invent authority/tools/context/bindings.

This does not provide any fixture-specific plan, operation sequence, hidden answer, or benchmark coaching. It directly implements #183's approved rlm-operate requirement to teach the actual typed runtime contract while leaving selection/authority/execution in their canonical modules.

Downstream ownership remains clean: a0-symbolics explicitly consumes the upstream compiler/runtime through a thin bridge, and agentProlog explicitly keeps generic skill loading/selection/runtime/subagent semantics upstream.

Adversarial decision: GO for this concise rlm-operate realization. If the next exact-head live gate still cannot form a typed envelope, preserve that evidence and return to design before changing the root planner prompt; do not weaken the behavioral tests or spoon-feed exact plans.

RAGE TDD evidence / design refinement on current branch head `00384f592cd245645860cd3462dc7e53e0dd8f11`, still based on canonical `main` `267697bef10a3fffff7c093e1435ece770e7444b`. The initial harness-placement failure is resolved without weakening deterministic corpus ownership: the credential-only suite now lives under the existing `test/support/` exclusion and is invoked explicitly by `test/run_live_openrouter.pl`. The real provider regression is now isolated. On exact head `00384f...`, deterministic CI, Nix, clean-pack, and Tree-sitter pass their relevant jobs, while both REAL OpenRouter and Paid OpenRouter fail the new unscripted behavior suite. Existing scripted live OpenRouter cases pass first. All three new cases then fail after two planner attempts with the same structured cause: ```text plan_parse_failed -> invalid_plan -> missing_field(steps) ``` This is strong evidence that transport/provider plumbing works but the default operator contract does not currently teach the unscripted root planner enough of the typed-plan output ABI. The failure is before the trivial/tool/recursion policy assertions, so those remain valid downstream behavioral gates. ## Refined design First realization change is deliberately skill-owned and generic: strengthen only the bundled `rlm-operate` instruction so it states the minimal planner ABI already required by `rlm_plan`: - when acting as a planner, emit exactly one JSON object with top-level `steps`; - each step is one supported typed runtime operation; - use only current capabilities, active tool schemas, and earlier bindings/references; - inspect opaque context through typed context operations rather than inventing contents; - recurse only through typed `rlm` when useful; - terminate through typed `final`; - respect budgets and never invent authority/tools/context/bindings. This does **not** provide any fixture-specific plan, operation sequence, hidden answer, or benchmark coaching. It directly implements #183's approved `rlm-operate` requirement to teach the actual typed runtime contract while leaving selection/authority/execution in their canonical modules. Downstream ownership remains clean: `a0-symbolics` explicitly consumes the upstream compiler/runtime through a thin bridge, and `agentProlog` explicitly keeps generic skill loading/selection/runtime/subagent semantics upstream. Adversarial decision: **GO** for this concise `rlm-operate` realization. If the next exact-head live gate still cannot form a typed envelope, preserve that evidence and return to design before changing the root planner prompt; do not weaken the behavioral tests or spoon-feed exact plans.

lost-rob0t commented

2026-08-26 13:01:01 +00:00

(Migrated from github.com)

RAGE failure-driven refinement — exact head `a04ad2f37e0985dc25e52abb55e6b9c3974d2835`

Regression evidence

The new #242 behavioral acceptance is doing its job. Canonical deterministic CI remains green, but both credential-backed REAL OpenRouter and pinned Paid OpenRouter fail all three no-spoon-fed-plan cases on the exact candidate head.

Observed failures are structural plan-contract failures, not provider transport failures:

trivial case: after 2 planner attempts, plan_parse_failed with missing_field(op);
typed-tool case: after 2 attempts, plan_parse_failed with missing_field(op);
decomposable recursion case: plan_parse_failed; observed causes include missing_field(op) and unknown_operator(slice) across the two live runs.

The surrounding live suite still proves OpenRouter HTTP 200, native tool selection/execution, ordinary typed-plan parsing, and recursive completion plumbing. So the failure is specifically the model independently emitting the canonical typed plan without a benchmark-specific exact-plan instruction.

Analyze / research

This exposes an incompleteness in the already-approved #183 rlm-operate contract, not a need for new architecture. #183 explicitly requires rlm-operate to teach the actual typed runtime contract. Current branch text says every step must be a supported typed operation, but it still does not teach the JSON encoding that rlm_plan actually accepts.

rlm_plan requires top-level {"steps":[...]} and each step dict requires an op. Context selectors such as slice are context action types, not top-level operations. That directly explains the observed missing_field(op) / unknown_operator(slice) evidence.

The existing planner retry currently repeats the same provider messages after a parse failure. I am not changing that retry architecture in this slice: the first fix should be the approved permanent operator skill teaching the real closed plan grammar concisely. If live evidence still fails after that, preserve it and return to design before adding repair semantics.

Downstream ownership remains clean: A0 Symbolics #8 explicitly consumes Prolog-RLM through a thin bridge without duplicating policy, and AgentProlog #6 keeps generic skill/prompt/subagent semantics upstream.

Design

Refine only bundled rlm-operate so its provider-visible body contains a compact canonical JSON grammar:

exactly one top-level object with steps;
step op values are the closed runtime operations;
context operations use op:"context" plus action.type (peek|slice|search|partition|map|reduce) — never slice/search as a step op;
tool uses op:"tool", name, args, bind;
recursion uses op:"rlm", nested plan:{"steps":[...]}, bind;
model uses op:"model", provider, prompt expression, options, bind;
references use {"ref":"input"|"var"|"field",...}; other JSON values are literals;
final appears exactly once and last.

No task-specific plan, answer, hidden benchmark hint, second compiler, planner selector, authority rule, or product glue.

Adversarial review

This is generic syntax knowledge already required by #183, not a benchmark-specific instruction.
The grammar must not imply a capability exists; the existing root/child capability lists and active schemas remain authoritative.
action.type vs op must be explicit to prevent the exact slice failure seen live.
Keep the body concise; do not paste the full typed-plans manual or a giant exemplar.
Do not weaken #242 assertions. A malformed plan stays a hard failure.
Do not add parse-repair prompting unless this grammar refinement is disproven by fresh live evidence.

Decision

GO within the already-approved #183 design boundary. Realize the smallest rlm-operate contract refinement, then rerun the exact same no-spoon-fed-plan live gate on a fresh immutable head.

## RAGE failure-driven refinement — exact head `a04ad2f37e0985dc25e52abb55e6b9c3974d2835` ### Regression evidence The new #242 behavioral acceptance is doing its job. Canonical deterministic CI remains green, but both credential-backed REAL OpenRouter and pinned Paid OpenRouter fail all three no-spoon-fed-plan cases on the exact candidate head. Observed failures are structural plan-contract failures, not provider transport failures: - trivial case: after 2 planner attempts, `plan_parse_failed` with `missing_field(op)`; - typed-tool case: after 2 attempts, `plan_parse_failed` with `missing_field(op)`; - decomposable recursion case: `plan_parse_failed`; observed causes include `missing_field(op)` and `unknown_operator(slice)` across the two live runs. The surrounding live suite still proves OpenRouter HTTP 200, native tool selection/execution, ordinary typed-plan parsing, and recursive completion plumbing. So the failure is specifically the model independently emitting the canonical typed plan without a benchmark-specific exact-plan instruction. ### Analyze / research This exposes an incompleteness in the already-approved #183 `rlm-operate` contract, not a need for new architecture. #183 explicitly requires `rlm-operate` to teach the **actual typed runtime contract**. Current branch text says every step must be a supported typed operation, but it still does not teach the JSON encoding that `rlm_plan` actually accepts. `rlm_plan` requires top-level `{"steps":[...]}` and each step dict requires an `op`. Context selectors such as `slice` are **context action types**, not top-level operations. That directly explains the observed `missing_field(op)` / `unknown_operator(slice)` evidence. The existing planner retry currently repeats the same provider messages after a parse failure. I am **not** changing that retry architecture in this slice: the first fix should be the approved permanent operator skill teaching the real closed plan grammar concisely. If live evidence still fails after that, preserve it and return to design before adding repair semantics. Downstream ownership remains clean: A0 Symbolics #8 explicitly consumes Prolog-RLM through a thin bridge without duplicating policy, and AgentProlog #6 keeps generic skill/prompt/subagent semantics upstream. ### Design Refine only bundled `rlm-operate` so its provider-visible body contains a compact canonical JSON grammar: - exactly one top-level object with `steps`; - step `op` values are the closed runtime operations; - context operations use `op:"context"` plus `action.type` (`peek|slice|search|partition|map|reduce`) — never `slice`/`search` as a step `op`; - tool uses `op:"tool"`, `name`, `args`, `bind`; - recursion uses `op:"rlm"`, nested `plan:{"steps":[...]}`, `bind`; - model uses `op:"model"`, provider, prompt expression, options, bind; - references use `{"ref":"input"|"var"|"field",...}`; other JSON values are literals; - `final` appears exactly once and last. No task-specific plan, answer, hidden benchmark hint, second compiler, planner selector, authority rule, or product glue. ### Adversarial review - This is generic syntax knowledge already required by #183, not a benchmark-specific instruction. - The grammar must not imply a capability exists; the existing root/child capability lists and active schemas remain authoritative. - `action.type` vs `op` must be explicit to prevent the exact `slice` failure seen live. - Keep the body concise; do not paste the full typed-plans manual or a giant exemplar. - Do not weaken #242 assertions. A malformed plan stays a hard failure. - Do not add parse-repair prompting unless this grammar refinement is disproven by fresh live evidence. ### Decision **GO within the already-approved #183 design boundary.** Realize the smallest `rlm-operate` contract refinement, then rerun the exact same no-spoon-fed-plan live gate on a fresh immutable head.

lost-rob0t commented

2026-08-26 13:15:48 +00:00

(Migrated from github.com)

RAGE live-provider refinement — `ed95b1a49789b6d9366c528abb54fe774c36a4a0`

Fresh pinned Paid OpenRouter evidence materially narrows #183:

trivial no-recursion case now passes;
unknown-information typed-tool case now passes with a successful tool(behavior_lookup) transition and authoritative wrapped tool result;
ordinary nested model execution now returns task prose rather than another typed plan after distinguishing planner vs model-step behavior;
decomposable case still fails because the root planner chooses direct context(slice) -> model -> final instead of rlm recursion.

The same run exposed two contract details rather than transport/runtime breakage:

the initial recursion fixture gave the child only model(openrouter), making evidence-grounded recursion impossible; this is corrected so the child may use the parent's context(slice) + model capabilities;
the root planner invented {{text}} interpolation in a model prompt. rlm_plan has no string-template expression: model prompts resolve from literal/reference/object/list expressions, and a model prompt must resolve directly to text. Therefore rlm-operate must explicitly teach that refs are runtime expressions, not interpolation syntax.

Design refinement inside approved #183 scope

rlm-operate: state that there is no implicit {{...}} interpolation; to pass inspected text to a model step, use the bound/input reference directly as the prompt expression. Do not invent a templating feature.
rlm-recurse: make the existing heuristic operational: when a task has multiple independently investigable evidence streams/subproblems and recursion is available within budget, bounded rlm/parallel decomposition is preferred over collapsing all work into one direct model call; trivial/single-step work remains direct.
keep capability/authority semantics unchanged and do not force recursion when child capabilities cannot perform the required evidence work.
keep the live acceptance hard: recursion must be observable and final evidence must contain both authoritative codes.

No planner-specific benchmark plan, no exact-plan hint, no second selector, no retry repair layer, and no new prompt interpolation feature.

Decision: GO for this narrower operator-skill refinement, then rerun the same credential-backed behavior suite unchanged except for correcting the structurally invalid child capability/test-result assumptions.

## RAGE live-provider refinement — `ed95b1a49789b6d9366c528abb54fe774c36a4a0` Fresh pinned Paid OpenRouter evidence materially narrows #183: - trivial no-recursion case now passes; - unknown-information typed-tool case now passes with a successful `tool(behavior_lookup)` transition and authoritative wrapped tool result; - ordinary nested `model` execution now returns task prose rather than another typed plan after distinguishing planner vs model-step behavior; - decomposable case still fails because the root planner chooses direct `context(slice) -> model -> final` instead of `rlm` recursion. The same run exposed two contract details rather than transport/runtime breakage: 1. the initial recursion fixture gave the child only `model(openrouter)`, making evidence-grounded recursion impossible; this is corrected so the child may use the parent's `context(slice)` + model capabilities; 2. the root planner invented `{{text}}` interpolation in a model prompt. `rlm_plan` has no string-template expression: model prompts resolve from literal/reference/object/list expressions, and a model prompt must resolve directly to text. Therefore `rlm-operate` must explicitly teach that refs are runtime expressions, not interpolation syntax. ### Design refinement inside approved #183 scope - `rlm-operate`: state that there is no implicit `{{...}}` interpolation; to pass inspected text to a model step, use the bound/input reference directly as the prompt expression. Do not invent a templating feature. - `rlm-recurse`: make the existing heuristic operational: when a task has multiple independently investigable evidence streams/subproblems and recursion is available within budget, bounded `rlm`/parallel decomposition is preferred over collapsing all work into one direct model call; trivial/single-step work remains direct. - keep capability/authority semantics unchanged and do not force recursion when child capabilities cannot perform the required evidence work. - keep the live acceptance hard: recursion must be observable and final evidence must contain both authoritative codes. No planner-specific benchmark plan, no exact-plan hint, no second selector, no retry repair layer, and no new prompt interpolation feature. **Decision: GO** for this narrower operator-skill refinement, then rerun the same credential-backed behavior suite unchanged except for correcting the structurally invalid child capability/test-result assumptions.

lost-rob0t commented

2026-08-26 13:31:43 +00:00

(Migrated from github.com)

RAGE live-provider failure analysis — PR #242 head `635bcf2810aeb29846391cd3bdc155fcb03b1f34`

Credential-backed REAL OpenRouter and pinned paid OpenRouter both preserve the same useful falsifiable result:

trivial behavior passes with zero recursion;
unknown-information behavior passes through the typed behavior_lookup tool;
decomposable behavior independently chooses recursion, but canonical recursive validation rejects the generated child plan with child_capability_denied(parallel).

This is not a provider outage or a reason to weaken/skip the gate. The fixture exposes root rlm, context(slice), model(openrouter) and child context(slice), model(openrouter) only. parallel is intentionally absent. The approved #183 invariant is that children operate only within exposed child capabilities; adding parallel merely because the model requested it would weaken the behavioral acceptance.

Analyze / research

The current rlm-recurse body already says parallel branches are appropriate only when the parallel capability/budget is available and that child plans stay within exposed child capabilities. Live evidence shows this is still too implicit for the tested provider: the model recognized a useful decomposition but invented one unavailable child operation.

Downstream ownership remains clean: A0 Symbolics expects a thin pinned Prolog-RLM consumer without duplicated compiler/authority policy, and AgentProlog assigns generic RLM/compiler/subagent semantics upstream.

Design

Refine the permanent generic rlm-recurse operating contract, not the fixture: make capability lists explicit hard allowlists for recursive planning and name the observed failure mode (parallel absent => no nested parallel op). Keep the instruction concise and provider-neutral. Do not inject a plan, step sequence, benchmark answer, or new selector/policy layer.

Adversarial review

Do not grant parallel just to green the test.
Do not change canonical recursive capability validation.
Do not add exact-plan hints to the user fixture.
Do not turn an expected live behavior failure into xfail/skip/soft success.
Preserve the successful trivial/tool cases as regression evidence.
Any new candidate head must rerun the credentialed live gate; previous exact-head evidence is invalid after the skill change.

Decision: GO for this narrow skill-contract refinement inside the already-approved #183 architecture; current #242 head remains HOLD until fresh live evidence passes.

## RAGE live-provider failure analysis — PR #242 head `635bcf2810aeb29846391cd3bdc155fcb03b1f34` Credential-backed REAL OpenRouter and pinned paid OpenRouter both preserve the same useful falsifiable result: - trivial behavior passes with zero recursion; - unknown-information behavior passes through the typed `behavior_lookup` tool; - decomposable behavior independently chooses recursion, but canonical recursive validation rejects the generated child plan with `child_capability_denied(parallel)`. This is not a provider outage or a reason to weaken/skip the gate. The fixture exposes root `rlm, context(slice), model(openrouter)` and child `context(slice), model(openrouter)` only. `parallel` is intentionally absent. The approved #183 invariant is that children operate only within exposed child capabilities; adding `parallel` merely because the model requested it would weaken the behavioral acceptance. ### Analyze / research The current `rlm-recurse` body already says parallel branches are appropriate only when the parallel capability/budget is available and that child plans stay within exposed child capabilities. Live evidence shows this is still too implicit for the tested provider: the model recognized a useful decomposition but invented one unavailable child operation. Downstream ownership remains clean: A0 Symbolics expects a thin pinned Prolog-RLM consumer without duplicated compiler/authority policy, and AgentProlog assigns generic RLM/compiler/subagent semantics upstream. ### Design Refine the permanent generic `rlm-recurse` operating contract, not the fixture: make capability lists explicit hard allowlists for recursive planning and name the observed failure mode (`parallel` absent => no nested `parallel` op). Keep the instruction concise and provider-neutral. Do not inject a plan, step sequence, benchmark answer, or new selector/policy layer. ### Adversarial review - Do not grant `parallel` just to green the test. - Do not change canonical recursive capability validation. - Do not add exact-plan hints to the user fixture. - Do not turn an expected live behavior failure into xfail/skip/soft success. - Preserve the successful trivial/tool cases as regression evidence. - Any new candidate head must rerun the credentialed live gate; previous exact-head evidence is invalid after the skill change. Decision: **GO** for this narrow skill-contract refinement inside the already-approved #183 architecture; current #242 head remains **HOLD** until fresh live evidence passes.

lost-rob0t commented

2026-08-26 13:36:44 +00:00

(Migrated from github.com)

RAGE iteration 2 — failure moved past capability validation

Fresh paid OpenRouter evidence on candidate d4c7d82f2c0c634f8c0c0401d25532004a57f2e3 shows the previous child_capability_denied(parallel) failure is gone. The same decomposable case now reaches nested execution and fails explicitly as:

plan_error{phase:execute,kind:invalid_prompt,message:"model prompt did not resolve to text"}

Trivial and typed-tool behavior still pass.

Analyze

rlm_plan resolves a model step's prompt expression and requires the resulting value itself to be text. A model step bind stores the full model_response{...} object, whose textual answer is the text field. Therefore reusing a bound model result directly as a later model.prompt is invalid; it must use a field reference to the response's text. The current rlm-operate body explains generic field references but does not explicitly state that a model bind is a response object rather than a text binding.

Design / adversarial review

Refine only the generic typed operating contract: state that model binds a full response object and that downstream model prompts/finals wanting assistant text must reference its text field. Do not auto-coerce arbitrary dicts to strings in the executor, because that would weaken the typed boundary and could silently turn structured provider state into prompt text. Do not change the fixture or inject an exact plan.

Decision: GO for a narrow rlm-operate clarification inside approved #183. Current head remains HOLD until a new credential-backed live run passes.

## RAGE iteration 2 — failure moved past capability validation Fresh paid OpenRouter evidence on candidate `d4c7d82f2c0c634f8c0c0401d25532004a57f2e3` shows the previous `child_capability_denied(parallel)` failure is gone. The same decomposable case now reaches nested execution and fails explicitly as: `plan_error{phase:execute,kind:invalid_prompt,message:"model prompt did not resolve to text"}` Trivial and typed-tool behavior still pass. ### Analyze `rlm_plan` resolves a `model` step's `prompt` expression and requires the resulting value itself to be text. A `model` step bind stores the full `model_response{...}` object, whose textual answer is the `text` field. Therefore reusing a bound model result directly as a later `model.prompt` is invalid; it must use a field reference to the response's text. The current `rlm-operate` body explains generic field references but does not explicitly state that a model bind is a response object rather than a text binding. ### Design / adversarial review Refine only the generic typed operating contract: state that `model` binds a full response object and that downstream model prompts/finals wanting assistant text must reference its `text` field. Do not auto-coerce arbitrary dicts to strings in the executor, because that would weaken the typed boundary and could silently turn structured provider state into prompt text. Do not change the fixture or inject an exact plan. Decision: **GO** for a narrow `rlm-operate` clarification inside approved #183. Current head remains HOLD until a new credential-backed live run passes.

lost-rob0t commented

2026-08-26 13:42:10 +00:00

(Migrated from github.com)

RAGE iteration 3 — strengthen the behavioral premise, not the assertion

Exact 90e926142a4320d6545a38a6d2d1b18491aa6871 proves the two typed-operating fixes worked: the decomposable case now emits a valid context -> model -> final(field(...,text)) plan and returns both ALPHA-17 and BETA-42. It fails only because recursion remains zero.

The live trajectory shows why: the fixture grants the root context(slice) and model(openrouter), and both evidence briefs fit inside one 256-byte slice. A direct context/model plan is therefore legal, bounded, and sufficient. Forcing the permanent skill to recurse anyway would contradict its own invariant: recurse only when decomposition adds value.

Design

Strengthen the acceptance setup so it actually tests the intended runtime boundary: root execution receives only rlm; the bounded child receives context(slice) and model(openrouter). The root planner still sees no exact plan and receives no hidden answer. The only legal way to inspect opaque context and perform model synthesis is through the canonical bounded rlm child path.

This is a stricter test, not a weakened one: direct root context/model execution becomes impossible, child capability narrowing remains authoritative, and the same assertions still require an actual successful rlm transition, depth/call evidence, and both opaque codes in the final result.

Adversarial review

no planner_instruction or explicit JSON plan;
no new capability is granted;
no runtime validation is relaxed;
no assertion is removed;
no provider failure is tolerated;
the task still distinguishes trivial/direct behavior from genuinely delegated behavior.

Decision: GO for this fixture correction. Any new head must rerun deterministic + credentialed live gates from scratch.

## RAGE iteration 3 — strengthen the behavioral premise, not the assertion Exact `90e926142a4320d6545a38a6d2d1b18491aa6871` proves the two typed-operating fixes worked: the decomposable case now emits a valid `context -> model -> final(field(...,text))` plan and returns both `ALPHA-17` and `BETA-42`. It fails only because recursion remains zero. The live trajectory shows why: the fixture grants the root `context(slice)` and `model(openrouter)`, and both evidence briefs fit inside one 256-byte slice. A direct context/model plan is therefore legal, bounded, and sufficient. Forcing the permanent skill to recurse anyway would contradict its own invariant: recurse only when decomposition adds value. ### Design Strengthen the acceptance setup so it actually tests the intended runtime boundary: root execution receives only `rlm`; the bounded child receives `context(slice)` and `model(openrouter)`. The root planner still sees no exact plan and receives no hidden answer. The only legal way to inspect opaque context and perform model synthesis is through the canonical bounded `rlm` child path. This is a stricter test, not a weakened one: direct root context/model execution becomes impossible, child capability narrowing remains authoritative, and the same assertions still require an actual successful `rlm` transition, depth/call evidence, and both opaque codes in the final result. ### Adversarial review - no `planner_instruction` or explicit JSON plan; - no new capability is granted; - no runtime validation is relaxed; - no assertion is removed; - no provider failure is tolerated; - the task still distinguishes trivial/direct behavior from genuinely delegated behavior. Decision: **GO** for this fixture correction. Any new head must rerun deterministic + credentialed live gates from scratch.

lost-rob0t commented

2026-08-26 13:45:51 +00:00

(Migrated from github.com)

RAGE iteration 4 — reject invalid child-widening fixture; strengthen the generic recursion policy

Candidate 2d37b095d52f62e5e12ce3fcabafc6db49fb58e9 failed before planning with the correct canonical error: child [context(slice),model(openrouter)] would widen parent [rlm]. This disproves the attempted fixture design. Preserve the narrowing invariant; do not special-case the acceptance.

The valid architecture requires parent capabilities to be a superset of child capabilities, which necessarily means a direct root route can remain technically possible. Therefore the acceptance is fundamentally testing the permanent operator policy: when the request explicitly requires independent evidence investigation and rlm is available within budget, the planner should choose the bounded recursive scope rather than flattening the work merely because a direct model step is possible.

Design

Restore the valid parent capability set [rlm,context(slice),model(openrouter)] with child [context(slice),model(openrouter)].
Keep all existing recursion/depth/transition/evidence assertions.
Strengthen the generic rlm-recurse skill from a soft preference to an explicit semantic rule: an explicit requirement to investigate multiple independent evidence streams separately is a strong recursion signal; use an rlm step when capability/budget permits, while trivial/single-step tasks remain direct.

This remains provider-neutral and benchmark-neutral. It does not give an exact plan or step sequence; it teaches the model how the runtime's decomposition policy maps to user intent.

Adversarial review

no capability widening;
no relaxation of recursive validation;
no new hidden tool/context/provider;
no removal of the trivial no-recursion control;
no exact JSON plan hint;
no executor coercion;
live provider failure remains a hard gate.

Decision: GO for the valid-fixture restoration plus concise generic recursion-policy clarification.

## RAGE iteration 4 — reject invalid child-widening fixture; strengthen the generic recursion policy Candidate `2d37b095d52f62e5e12ce3fcabafc6db49fb58e9` failed before planning with the correct canonical error: child `[context(slice),model(openrouter)]` would widen parent `[rlm]`. This disproves the attempted fixture design. Preserve the narrowing invariant; do not special-case the acceptance. The valid architecture requires parent capabilities to be a superset of child capabilities, which necessarily means a direct root route can remain technically possible. Therefore the acceptance is fundamentally testing the permanent operator policy: when the request explicitly requires independent evidence investigation and `rlm` is available within budget, the planner should choose the bounded recursive scope rather than flattening the work merely because a direct model step is possible. ### Design 1. Restore the valid parent capability set `[rlm,context(slice),model(openrouter)]` with child `[context(slice),model(openrouter)]`. 2. Keep all existing recursion/depth/transition/evidence assertions. 3. Strengthen the generic `rlm-recurse` skill from a soft preference to an explicit semantic rule: an explicit requirement to investigate multiple independent evidence streams separately is a strong recursion signal; use an `rlm` step when capability/budget permits, while trivial/single-step tasks remain direct. This remains provider-neutral and benchmark-neutral. It does not give an exact plan or step sequence; it teaches the model how the runtime's decomposition policy maps to user intent. ### Adversarial review - no capability widening; - no relaxation of recursive validation; - no new hidden tool/context/provider; - no removal of the trivial no-recursion control; - no exact JSON plan hint; - no executor coercion; - live provider failure remains a hard gate. Decision: **GO** for the valid-fixture restoration plus concise generic recursion-policy clarification.

lost-rob0t commented

2026-08-26 13:52:32 +00:00

(Migrated from github.com)

RAGE iteration 5 — planner token ceiling is the current live blocker

At exact f7d029bbde044c3fca39201147b7b081d0ee3adf, deterministic CI, Nix, clean pack, and Tree-sitter are green, but both credentialed OpenRouter lanes fail in the decomposable case during root planning. The failure is plan_parse_failed after two attempts; usage reports exactly completion_tokens:2800 for planner_attempts(2) with planner_max_tokens(1400), i.e. both attempts consume the full 1400-token ceiling before yielding a valid {steps:[...]} plan.

The runtime already has the closed planner_reasoning_effort control from #185, so this is not missing provider-option plumbing. This acceptance fixture itself chose a 1400 planner ceiling. The earlier behavior runs produced valid plans under shorter reasoning, but the strengthened decomposition policy causes the pinned reasoning model to need more planner output budget.

Design / adversarial review

Raise only the live behavior fixture's planner ceiling to 2600 while keeping:

the same total 12k token budget and $0.25 cost ceiling;
the same two-attempt hard failure behavior;
the same no-spoon-fed-plan contract;
every trivial/tool/recursion/depth/evidence assertion;
unchanged reasoning semantics and provider selection.

Do not suppress reasoning, skip parse failures, increase global runtime defaults, or accept malformed plans. A planner that still cannot produce a typed plan under the larger bounded allowance must continue to fail.

Decision: GO for the bounded acceptance-budget correction; fresh exact-head live verification remains mandatory.

## RAGE iteration 5 — planner token ceiling is the current live blocker At exact `f7d029bbde044c3fca39201147b7b081d0ee3adf`, deterministic CI, Nix, clean pack, and Tree-sitter are green, but both credentialed OpenRouter lanes fail in the decomposable case during root planning. The failure is `plan_parse_failed` after two attempts; usage reports exactly `completion_tokens:2800` for `planner_attempts(2)` with `planner_max_tokens(1400)`, i.e. both attempts consume the full 1400-token ceiling before yielding a valid `{steps:[...]}` plan. The runtime already has the closed `planner_reasoning_effort` control from #185, so this is not missing provider-option plumbing. This acceptance fixture itself chose a 1400 planner ceiling. The earlier behavior runs produced valid plans under shorter reasoning, but the strengthened decomposition policy causes the pinned reasoning model to need more planner output budget. ### Design / adversarial review Raise only the live behavior fixture's planner ceiling to 2600 while keeping: - the same total 12k token budget and $0.25 cost ceiling; - the same two-attempt hard failure behavior; - the same no-spoon-fed-plan contract; - every trivial/tool/recursion/depth/evidence assertion; - unchanged reasoning semantics and provider selection. Do not suppress reasoning, skip parse failures, increase global runtime defaults, or accept malformed plans. A planner that still cannot produce a typed plan under the larger bounded allowance must continue to fail. Decision: **GO** for the bounded acceptance-budget correction; fresh exact-head live verification remains mandatory.

lost-rob0t commented

2026-08-27 04:03:35 +00:00

(Migrated from github.com)

RAGE live-provider adversarial evidence after #245 landed

Canonical main is now 7fa4c2547b387acdbe5fc269e8ebcb4919249b97; #245/#246 is merged, so the structural-plan retry blocker is gone. PR #242 has already been refreshed onto that exact base at head c971c287e6d3980a31ebf8fb078d0bbe927b1679.

Fresh exact-head CI gives a useful behavioral split:

deterministic unit/load/benchmark/deep-recursion/CLI/restart checks: PASS;
Nix, clean SWI pack, Tree-sitter: PASS;
REAL OpenRouter ordinary provider/stream/typed-plan/tool/completion plumbing: PASS;
#183 trivial behavior: PASS (zero recursion);
#183 unknown-information behavior: PASS (typed behavior_lookup tool actually invoked);
#183 decomposable behavior: FAIL.

The failing provider plan is not malformed and does not ignore decomposition. It chose:

context slice -> context slice -> model -> model -> final

with three total model calls, but zero rlm transitions / recursion depth 0. This is stronger evidence than the earlier #245 failure: the model now understands enough of the runtime to build and execute a valid multi-step plan, but it still treats two independent evidence streams as direct sibling model calls instead of using the approved bounded recursive RLM path.

There is also a fixture-quality problem visible in the same run: fixed byte slicing split the 289-byte context at byte 256, so brief1 contained almost all of both ALPHA and BETA while brief2 was merely "rough runtime context operations.". The first child therefore saw both answers and the second saw junk. That means this particular fixture is not a clean proof that useful recursive decomposition was available/necessary, even though the explicit recursion assertion correctly caught zero recursion.

A second assertion artifact is visible too: provider prose rendered ALPHA‑17/BETA‑42 with a Unicode non-breaking hyphen, so exact ASCII substring checks failed despite the semantic values appearing in the returned evidence. That should not be used to weaken the recursion requirement; it means the evidence fixture should use machine-stable structured tokens if exact lexical equality is part of acceptance.

Adversarial decision

HOLD #242 promotion. Do not weaken recursive_calls >= 1 into model_calls >= 2, and do not accept direct sibling model calls as proof of the #183 recursion/delegation requirement. Also do not add an exact plan or hidden rlm step hint.

The next TDD refinement should first make the decomposable fixture itself unambiguous and machine-stable: two independently addressable evidence regions whose boundaries are known to the harness/runtime, with distinct ASCII-safe opaque tokens, such that each child can receive only its own bounded evidence and the parent must synthesize both. Then rerun the same no-spoon-fed-plan live contract. If the model still chooses direct root model calls despite the corrected fixture and active rlm-recurse skill, that is clean evidence that the default operating skill/runtime projection needs further refinement inside the already-approved #183 architecture.

No downstream A0/AgentProlog workaround is justified; this remains generic RLM operator behavior.

## RAGE live-provider adversarial evidence after #245 landed Canonical `main` is now `7fa4c2547b387acdbe5fc269e8ebcb4919249b97`; #245/#246 is merged, so the structural-plan retry blocker is gone. PR #242 has already been refreshed onto that exact base at head `c971c287e6d3980a31ebf8fb078d0bbe927b1679`. Fresh exact-head CI gives a useful behavioral split: - deterministic unit/load/benchmark/deep-recursion/CLI/restart checks: **PASS**; - Nix, clean SWI pack, Tree-sitter: **PASS**; - REAL OpenRouter ordinary provider/stream/typed-plan/tool/completion plumbing: **PASS**; - #183 trivial behavior: **PASS** (zero recursion); - #183 unknown-information behavior: **PASS** (typed `behavior_lookup` tool actually invoked); - #183 decomposable behavior: **FAIL**. The failing provider plan is not malformed and does not ignore decomposition. It chose: ```text context slice -> context slice -> model -> model -> final ``` with three total model calls, but **zero `rlm` transitions / recursion depth 0**. This is stronger evidence than the earlier #245 failure: the model now understands enough of the runtime to build and execute a valid multi-step plan, but it still treats two independent evidence streams as direct sibling model calls instead of using the approved bounded recursive RLM path. There is also a fixture-quality problem visible in the same run: fixed byte slicing split the 289-byte context at byte 256, so `brief1` contained almost all of both ALPHA and BETA while `brief2` was merely `"rough runtime context operations."`. The first child therefore saw both answers and the second saw junk. That means this particular fixture is not a clean proof that useful recursive decomposition was available/necessary, even though the explicit recursion assertion correctly caught zero recursion. A second assertion artifact is visible too: provider prose rendered `ALPHA‑17`/`BETA‑42` with a Unicode non-breaking hyphen, so exact ASCII substring checks failed despite the semantic values appearing in the returned evidence. That should not be used to weaken the recursion requirement; it means the evidence fixture should use machine-stable structured tokens if exact lexical equality is part of acceptance. ### Adversarial decision **HOLD #242 promotion. Do not weaken `recursive_calls >= 1` into `model_calls >= 2`, and do not accept direct sibling model calls as proof of the #183 recursion/delegation requirement.** Also do not add an exact plan or hidden `rlm` step hint. The next TDD refinement should first make the decomposable fixture itself unambiguous and machine-stable: two independently addressable evidence regions whose boundaries are known to the harness/runtime, with distinct ASCII-safe opaque tokens, such that each child can receive only its own bounded evidence and the parent must synthesize both. Then rerun the same no-spoon-fed-plan live contract. If the model still chooses direct root model calls despite the corrected fixture and active `rlm-recurse` skill, that is clean evidence that the default operating skill/runtime projection needs further refinement inside the already-approved #183 architecture. No downstream A0/AgentProlog workaround is justified; this remains generic RLM operator behavior.

lost-rob0t commented

2026-08-27 05:04:57 +00:00

(Migrated from github.com)

RAGE exact-head provider evidence — #242 `f0e09b9ddfe59d91772ae4b98a2c3b8a5ba08ab4`

Fresh inspection of both credential-backed runs changes the failure boundary again; preserve this evidence before changing the skill contract.

What is proved

Deterministic unit/load, Nix, clean SWI pack, and Tree-sitter are green on the current #242 merge candidate. The trivial case and typed-tool case also pass live.

The decomposable case is failing for two different reasons across the two real OpenRouter runs:

CI / REAL OpenRouter: the planner did attempt a plan that crossed the model-call budget and the runtime correctly rejected it with model_call_budget_exceeded(3,2). rlm_completion subtracts planner-attempt model calls from max_model_calls before validating model steps in the selected plan. With planner_attempts(2) and fixture max_model_calls:4, a repaired planner attempt can leave only two calls for a decomposition whose selected plan contains three model calls. This is budget enforcement, not evidence that recursion itself is broken.
Pinned Paid OpenRouter: the planner still chose two root context(slice) + two root model(openrouter) calls, recursive_calls:0, so the explicit recursion acceptance correctly failed. The emitted slices were slice(0,144) and slice(144,145), splitting the fixture inside the evidence text: brief1 ends in ...BETA: the verified recovery c and brief2 starts ode is BETA-42.... The child model outputs then became garbage for this acceptance (one asks for clarification; the other emits another plan).

Adversarial decision

HOLD #242. Do not weaken the recursion assertion and do not count sibling root model calls as recursive decomposition. Also do not interpret the first run's budget rejection as permission to raise production budgets globally.

The next test-only repair inside the already-approved #183 behavioral design should make the fixture falsifiable before further skill tuning:

represent the two evidence streams as independently addressable bounded regions so no legal slice boundary bisects either brief;
use machine-stable opaque evidence tokens rather than typography-sensitive prose tokens;
provision the fixture's model-call budget so the allowed planner_attempts(2) plus the minimum intended recursive investigation+synthesis path can fit, while keeping the budget finite and asserted;
keep the same no-planner_instruction / no exact-plan-hint rule and the hard requirement for an observed successful rlm transition and nonzero recursive depth.

After that fixture repair, rerun the same real provider gate. If the clean fixture still selects sibling root model calls, that is valid evidence to refine the default rlm-recurse operating skill within #183. If it selects recursion but violates a correctly-sized budget, treat that separately as a runtime/accounting defect rather than changing the behavioral acceptance.

Current canonical main inspected immediately before this write: 2e1264d80d02fecfb9f946e1328caaf1053e7a3b. #242 remains draft and mergeable, head unchanged at f0e09b9ddfe59d91772ae4b98a2c3b8a5ba08ab4; no reviews or unresolved review threads.

## RAGE exact-head provider evidence — #242 `f0e09b9ddfe59d91772ae4b98a2c3b8a5ba08ab4` Fresh inspection of both credential-backed runs changes the failure boundary again; preserve this evidence before changing the skill contract. ### What is proved Deterministic unit/load, Nix, clean SWI pack, and Tree-sitter are green on the current #242 merge candidate. The trivial case and typed-tool case also pass live. The decomposable case is failing for **two different reasons across the two real OpenRouter runs**: 1. **CI / REAL OpenRouter:** the planner did attempt a plan that crossed the model-call budget and the runtime correctly rejected it with `model_call_budget_exceeded(3,2)`. `rlm_completion` subtracts planner-attempt model calls from `max_model_calls` before validating model steps in the selected plan. With `planner_attempts(2)` and fixture `max_model_calls:4`, a repaired planner attempt can leave only two calls for a decomposition whose selected plan contains three model calls. This is budget enforcement, not evidence that recursion itself is broken. 2. **Pinned Paid OpenRouter:** the planner still chose two root `context(slice)` + two root `model(openrouter)` calls, `recursive_calls:0`, so the explicit recursion acceptance correctly failed. The emitted slices were `slice(0,144)` and `slice(144,145)`, splitting the fixture **inside** the evidence text: brief1 ends in `...BETA: the verified recovery c` and brief2 starts `ode is BETA-42...`. The child model outputs then became garbage for this acceptance (one asks for clarification; the other emits another plan). ### Adversarial decision **HOLD #242. Do not weaken the recursion assertion and do not count sibling root model calls as recursive decomposition.** Also do not interpret the first run's budget rejection as permission to raise production budgets globally. The next test-only repair inside the already-approved #183 behavioral design should make the fixture falsifiable before further skill tuning: - represent the two evidence streams as independently addressable bounded regions so no legal slice boundary bisects either brief; - use machine-stable opaque evidence tokens rather than typography-sensitive prose tokens; - provision the fixture's model-call budget so the allowed `planner_attempts(2)` plus the *minimum intended recursive investigation+synthesis path* can fit, while keeping the budget finite and asserted; - keep the same no-`planner_instruction` / no exact-plan-hint rule and the hard requirement for an observed successful `rlm` transition and nonzero recursive depth. After that fixture repair, rerun the same real provider gate. If the clean fixture still selects sibling root model calls, that is valid evidence to refine the default `rlm-recurse` operating skill within #183. If it selects recursion but violates a correctly-sized budget, treat that separately as a runtime/accounting defect rather than changing the behavioral acceptance. Current canonical `main` inspected immediately before this write: `2e1264d80d02fecfb9f946e1328caaf1053e7a3b`. #242 remains draft and mergeable, head unchanged at `f0e09b9ddfe59d91772ae4b98a2c3b8a5ba08ab4`; no reviews or unresolved review threads.

lost-rob0t commented

2026-08-27 05:23:32 +00:00

(Migrated from github.com)

RAGE exact-head re-analysis — #242 `36e5b83123a645cab22f3bd5e773d134e54f80ca`

The cleaned fixture and two skill refinements have now separated fixture noise from actual operator behavior.

Evidence that is now proved

The independently addressable terms(...) fixture removed the old byte-split / Unicode-token contamination and the finite max_model_calls:6 budget removed the earlier artificial planner-attempt squeeze.
With only context(peek) active, the first clean live run initially emitted forbidden context(slice) and canonical validation rejected it. After rlm-operate gained the exact capability/action + peek(item) contract, the next live provider run selected peek(item(0)) and peek(item(1)) correctly. The permanent operator context changed real provider behavior without weakening capability enforcement.
That provider then still returned the two records directly with recursive_calls:0, proving the remaining problem was genuinely recursion behavior rather than context-shape syntax.
After rlm-recurse was tightened so an explicit request for separate investigation is a recursive-work contract, current REAL OpenRouter now passes the hard recursive_calls >= 1, max_depth >= 1, and successful rlm-transition assertions. Its only #183 assertion failure is Result.usage.model_calls >= 2, with actual value 1.
The same REAL job's existing canonical recursive-completion test reports completion_recursive_calls:1 and completion_total_model_calls:2, so parent usage accounting does include a child provider call when one actually occurs. The #183 model_calls:1 result is therefore evidence that this generated child plan recursed without doing model investigation, not evidence that usage aggregation dropped a child call.
On the same exact #242 head, the pinned Paid OpenRouter lane passes the entire #183 core behavior suite (trivial, typed tool, and decomposable recursion) and only fails later in the separately-owned deep-integration benchmark. Useful positive evidence, but it does not erase the failing REAL lane.

Adversarial decision: restart fixture design, keep acceptance hard

HOLD #242. Preserve both the real rlm-transition requirement and the model_calls >= 2 requirement. Do not redefine a child that only peeks a record and returns it as successful model investigation, and do not weaken exact-head REAL provider coverage because another lane passed.

The current record contents are themselves too answer-like: each already contains the exact code and confidence=high, so a child can satisfy the wording mechanically with context retrieval only. The next TDD step is to keep two independently addressable records but make each require a small child-local interpretation/judgment before parent synthesis. Add stable expected per-stream verdicts as well as the opaque evidence tokens. Keep all of these boundaries:

no planner_instruction or exact JSON-plan hint;
planner remains free to choose the typed plan;
context access remains context(peek) and capability checked;
recursive child remains bounded by the existing child allowlist;
successful acceptance still requires observed rlm, nonzero depth, and at least one provider model call beyond the root planner;
parent final must contain both independently derived verdicts and both source tokens.

If that falsifiable interpretation fixture still creates an rlm child that performs retrieval-only work, refine the generic rlm-recurse instruction to state that child-worthy interpretation/judgment must actually use an exposed child reasoning/model capability; merely copying context through a nested plan is retrieval, not investigation. Do not encode a fixed step sequence.

Current canonical main re-read immediately before this write: 2e1264d80d02fecfb9f946e1328caaf1053e7a3b. #242 remains draft/mergeable and exact head is unchanged at 36e5b83123a645cab22f3bd5e773d134e54f80ca.

## RAGE exact-head re-analysis — #242 `36e5b83123a645cab22f3bd5e773d134e54f80ca` The cleaned fixture and two skill refinements have now separated fixture noise from actual operator behavior. ### Evidence that is now proved 1. The independently addressable `terms(...)` fixture removed the old byte-split / Unicode-token contamination and the finite `max_model_calls:6` budget removed the earlier artificial planner-attempt squeeze. 2. With only `context(peek)` active, the first clean live run initially emitted forbidden `context(slice)` and canonical validation rejected it. After `rlm-operate` gained the exact capability/action + `peek(item)` contract, the next live provider run selected `peek(item(0))` and `peek(item(1))` correctly. **The permanent operator context changed real provider behavior without weakening capability enforcement.** 3. That provider then still returned the two records directly with `recursive_calls:0`, proving the remaining problem was genuinely recursion behavior rather than context-shape syntax. 4. After `rlm-recurse` was tightened so an explicit request for separate investigation is a recursive-work contract, current REAL OpenRouter now passes the hard `recursive_calls >= 1`, `max_depth >= 1`, and successful `rlm`-transition assertions. Its only #183 assertion failure is `Result.usage.model_calls >= 2`, with actual value 1. 5. The same REAL job's existing canonical recursive-completion test reports `completion_recursive_calls:1` and `completion_total_model_calls:2`, so parent usage accounting does include a child provider call when one actually occurs. The #183 `model_calls:1` result is therefore evidence that this generated child plan recursed without doing model investigation, not evidence that usage aggregation dropped a child call. 6. On the same exact #242 head, the pinned Paid OpenRouter lane passes the entire #183 core behavior suite (`trivial`, typed tool, and decomposable recursion) and only fails later in the separately-owned deep-integration benchmark. Useful positive evidence, but it does not erase the failing REAL lane. ### Adversarial decision: restart fixture design, keep acceptance hard **HOLD #242. Preserve both the real `rlm`-transition requirement and the `model_calls >= 2` requirement.** Do not redefine a child that only peeks a record and returns it as successful model investigation, and do not weaken exact-head REAL provider coverage because another lane passed. The current record contents are themselves too answer-like: each already contains the exact code and `confidence=high`, so a child can satisfy the wording mechanically with context retrieval only. The next TDD step is to keep two independently addressable records but make each require a small child-local interpretation/judgment before parent synthesis. Add stable expected per-stream verdicts as well as the opaque evidence tokens. Keep all of these boundaries: - no `planner_instruction` or exact JSON-plan hint; - planner remains free to choose the typed plan; - context access remains `context(peek)` and capability checked; - recursive child remains bounded by the existing child allowlist; - successful acceptance still requires observed `rlm`, nonzero depth, and at least one provider model call beyond the root planner; - parent final must contain both independently derived verdicts and both source tokens. If that falsifiable interpretation fixture still creates an `rlm` child that performs retrieval-only work, refine the generic `rlm-recurse` instruction to state that child-worthy interpretation/judgment must actually use an exposed child reasoning/model capability; merely copying context through a nested plan is retrieval, not investigation. Do not encode a fixed step sequence. Current canonical `main` re-read immediately before this write: `2e1264d80d02fecfb9f946e1328caaf1053e7a3b`. #242 remains draft/mergeable and exact head is unchanged at `36e5b83123a645cab22f3bd5e773d134e54f80ca`.

lost-rob0t commented

2026-08-27 07:08:28 +00:00

(Migrated from github.com)

RAGE realization update for #242.

Exact prior failing head: 3e7f04b4787be408158f97f92376236514bad1ca.

Regression boundary was the existing live depth 0/1/2 benchmark, not the new operator-behavior acceptance: depth 2 reached recursion depth 2 but exceeded the benchmark's fixed max_total_tokens:3000 after three live provider calls (~3795 tokens observed). Depth 0/1 were green. The permanent RLM operating context therefore made the depth-2 fixture's fixed ceiling mathematically insufficient without implying a production budget defect.

TDD-first repair is now on #242:

5a103d08d43028bb9f95f29718f7e4d5a177ef6d adds deterministic assertions to the existing rlm_deep_experiment suite requiring the live benchmark budget to scale with expected provider-call count, requiring depth 2 >= 6000 tokens, and retaining a finite <=12000 ceiling.
8a377b9d2b17a1b9996fe0539910869f4d85ad9d implements only the benchmark-local contract: max_total_tokens = (Depth+1)*2000 for depths 0/1/2.

Adversarial decision: GO for this benchmark-local repair. Production budgeting, recursion policy, provider limits, the depth assertion, and live credentialed execution are unchanged. This does not weaken or skip the depth gate; it makes the fixed benchmark budget sufficient for the fixed number of required provider calls while remaining bounded.

Exact-head verification is now required on 8a377b9d2b17a1b9996fe0539910869f4d85ad9d. Keep #242 draft/HOLD until deterministic CI, REAL OpenRouter, Paid OpenRouter, Nix, clean-pack and Tree-sitter all return green on that exact SHA. No earlier evidence is reusable.

RAGE realization update for #242. Exact prior failing head: `3e7f04b4787be408158f97f92376236514bad1ca`. Regression boundary was the existing live depth 0/1/2 benchmark, not the new operator-behavior acceptance: depth 2 reached recursion depth 2 but exceeded the benchmark's fixed `max_total_tokens:3000` after three live provider calls (~3795 tokens observed). Depth 0/1 were green. The permanent RLM operating context therefore made the depth-2 fixture's fixed ceiling mathematically insufficient without implying a production budget defect. TDD-first repair is now on #242: - `5a103d08d43028bb9f95f29718f7e4d5a177ef6d` adds deterministic assertions to the existing `rlm_deep_experiment` suite requiring the live benchmark budget to scale with expected provider-call count, requiring depth 2 >= 6000 tokens, and retaining a finite <=12000 ceiling. - `8a377b9d2b17a1b9996fe0539910869f4d85ad9d` implements only the benchmark-local contract: `max_total_tokens = (Depth+1)*2000` for depths 0/1/2. Adversarial decision: GO for this benchmark-local repair. Production budgeting, recursion policy, provider limits, the depth assertion, and live credentialed execution are unchanged. This does not weaken or skip the depth gate; it makes the fixed benchmark budget sufficient for the fixed number of required provider calls while remaining bounded. Exact-head verification is now required on `8a377b9d2b17a1b9996fe0539910869f4d85ad9d`. Keep #242 draft/HOLD until deterministic CI, REAL OpenRouter, Paid OpenRouter, Nix, clean-pack and Tree-sitter all return green on that exact SHA. No earlier evidence is reusable.

lost-rob0t commented

2026-08-27 08:11:16 +00:00

(Migrated from github.com)

RAGE exact-head adversarial refinement — #242 `8a377b9d2b17a1b9996fe0539910869f4d85ad9d`

Fresh inspection of the credential-backed pinned Paid OpenRouter job changes the current failure boundary again.

Evidence

The provider now executes the intended generic operator behavior rather than merely approximating it:

trivial case passes with no recursion;
unknown-information case passes through the typed tool path;
decomposable case selects two independently addressable context(peek(item(...))) records;
the root plan contains two real rlm child calls;
both children execute a model call at depth 1;
runtime reports recursive_calls:2, max_depth:1, and model_calls:4 overall;
the child-derived verdicts are correct: alpha -> RLM_VERDICT_REVIEW, beta -> RLM_VERDICT_CLEAR.

The only #183 failures in that paid-provider core case are the two final provenance assertions. Result.value is:

json{record0_verdict:"RLM_VERDICT_REVIEW", record1_verdict:"RLM_VERDICT_CLEAR"}

while the source records (and Result.vars) still contain RLM_EVID_ALPHA_7Q9X / RLM_EVID_BETA_4M2K. The task explicitly asks the parent synthesis to report each stream's exact token and derived verdict, and the prior approved fixture refinement explicitly retained both source-token assertions. This is therefore not permission to weaken the test.

Analysis / design

The remaining behavior defect is now parent synthesis provenance loss. Current rlm-recurse says to aggregate returned child evidence and distinguish evidence from inference, but it does not make the narrower operational obligation explicit: when the user asks for source/provenance identifiers alongside derived results, the parent must preserve those requested identifiers in the final synthesis rather than collapsing child evidence down to verdict-only values.

The smallest generic refinement remains inside approved #183 architecture: strengthen the default recursion/operator contract so parent synthesis preserves requested evidence/provenance identifiers from the investigated records/child results while still deriving judgments in the children. Do not introduce a fixed plan, benchmark-specific token names, a new compiler path, or any authority change.

Adversarial decision

HOLD #242. GO for a TDD-first generic provenance-preserving synthesis refinement.

Required boundary before implementation:

deterministic contract must prove the bundled recursion operating context instructs the parent to preserve explicitly requested source/provenance identifiers during synthesis;
keep the live token assertions, verdict assertions, real rlm transition, nonzero depth, and model-call requirements unchanged;
no exact plan/JSON hint and no benchmark token strings in the skill;
provider failures remain hard failures;
after realization, recover the existing #242 branch onto current main ad655ca51f57148833f1294f511c0b5d1f24cdde and require a completely fresh exact-head deterministic + REAL/Paid OpenRouter + Nix + clean-pack + Tree-sitter gate.

Current coordination check: #242 remains draft/mergeable at the exact head above with no submitted reviews or unresolved review threads. Its five pre-existing owned files are byte-identical between its merge-base and current main; the sixth file is PR-new, so the eight intervening main commits do not overlap this transaction. Downstream a0-symbolics/AgentProlog evidence still assigns generic RLM skill/recursion semantics upstream.

## RAGE exact-head adversarial refinement — #242 `8a377b9d2b17a1b9996fe0539910869f4d85ad9d` Fresh inspection of the **credential-backed pinned Paid OpenRouter** job changes the current failure boundary again. ### Evidence The provider now executes the intended generic operator behavior rather than merely approximating it: - trivial case passes with no recursion; - unknown-information case passes through the typed tool path; - decomposable case selects two independently addressable `context(peek(item(...)))` records; - the root plan contains **two real `rlm` child calls**; - both children execute a model call at depth 1; - runtime reports `recursive_calls:2`, `max_depth:1`, and `model_calls:4` overall; - the child-derived verdicts are correct: alpha -> `RLM_VERDICT_REVIEW`, beta -> `RLM_VERDICT_CLEAR`. The only #183 failures in that paid-provider core case are the two final provenance assertions. `Result.value` is: `json{record0_verdict:"RLM_VERDICT_REVIEW", record1_verdict:"RLM_VERDICT_CLEAR"}` while the source records (and `Result.vars`) still contain `RLM_EVID_ALPHA_7Q9X` / `RLM_EVID_BETA_4M2K`. The task explicitly asks the parent synthesis to report each stream's exact token **and** derived verdict, and the prior approved fixture refinement explicitly retained both source-token assertions. This is therefore not permission to weaken the test. ### Analysis / design The remaining behavior defect is now **parent synthesis provenance loss**. Current `rlm-recurse` says to aggregate returned child evidence and distinguish evidence from inference, but it does not make the narrower operational obligation explicit: when the user asks for source/provenance identifiers alongside derived results, the parent must preserve those requested identifiers in the final synthesis rather than collapsing child evidence down to verdict-only values. The smallest generic refinement remains inside approved #183 architecture: strengthen the default recursion/operator contract so parent synthesis preserves requested evidence/provenance identifiers from the investigated records/child results while still deriving judgments in the children. Do not introduce a fixed plan, benchmark-specific token names, a new compiler path, or any authority change. ### Adversarial decision **HOLD #242. GO for a TDD-first generic provenance-preserving synthesis refinement.** Required boundary before implementation: 1. deterministic contract must prove the bundled recursion operating context instructs the parent to preserve explicitly requested source/provenance identifiers during synthesis; 2. keep the live token assertions, verdict assertions, real `rlm` transition, nonzero depth, and model-call requirements unchanged; 3. no exact plan/JSON hint and no benchmark token strings in the skill; 4. provider failures remain hard failures; 5. after realization, recover the existing #242 branch onto current `main` `ad655ca51f57148833f1294f511c0b5d1f24cdde` and require a completely fresh exact-head deterministic + REAL/Paid OpenRouter + Nix + clean-pack + Tree-sitter gate. Current coordination check: #242 remains draft/mergeable at the exact head above with no submitted reviews or unresolved review threads. Its five pre-existing owned files are byte-identical between its merge-base and current main; the sixth file is PR-new, so the eight intervening main commits do not overlap this transaction. Downstream a0-symbolics/AgentProlog evidence still assigns generic RLM skill/recursion semantics upstream.

lost-rob0t commented

2026-08-27 08:13:44 +00:00

(Migrated from github.com)

RAGE TDD + realization — provenance-preserving recursive synthesis

Current canonical main immediately before this write: ad655ca51f57148833f1294f511c0b5d1f24cdde. Existing #242 transaction remained draft/mergeable and ownership-clean.

TDD-first commits on rage/183-live-operator-behavior:

6d88a2739aefb03bcea82801d3b4a192bc306c87 — adds deterministic provider-context contract requiring the bundled rlm-recurse operating instruction to preserve explicitly requested source/provenance identifiers in final synthesis.
9180794f7469622b50e89c2b01f01a1d13fae875 — minimal generic realization: parent recursive synthesis must keep explicitly requested source/provenance identifiers alongside derived results rather than collapsing them away.

No live acceptance was weakened: the decomposable case still requires both opaque source tokens, both derived verdicts, real rlm execution, nonzero recursive depth, and bounded model-call evidence. No benchmark token names, fixed plan, JSON step sequence, capability change, or authority change was added to the skill.

Fresh exact-head Actions have started for 9180794f... (CI/Nix queued; Paid OpenRouter, clean-pack and Tree-sitter started). HOLD promotion until every exact-head gate completes. The branch also remains behind current main; after behavioral verification, recover the existing transaction onto ad655ca5... and require another fresh exact-head gate rather than reusing this evidence.

## RAGE TDD + realization — provenance-preserving recursive synthesis Current canonical main immediately before this write: `ad655ca51f57148833f1294f511c0b5d1f24cdde`. Existing #242 transaction remained draft/mergeable and ownership-clean. TDD-first commits on `rage/183-live-operator-behavior`: - `6d88a2739aefb03bcea82801d3b4a192bc306c87` — adds deterministic provider-context contract requiring the bundled `rlm-recurse` operating instruction to preserve explicitly requested source/provenance identifiers in final synthesis. - `9180794f7469622b50e89c2b01f01a1d13fae875` — minimal generic realization: parent recursive synthesis must keep explicitly requested source/provenance identifiers alongside derived results rather than collapsing them away. No live acceptance was weakened: the decomposable case still requires both opaque source tokens, both derived verdicts, real `rlm` execution, nonzero recursive depth, and bounded model-call evidence. No benchmark token names, fixed plan, JSON step sequence, capability change, or authority change was added to the skill. Fresh exact-head Actions have started for `9180794f...` (CI/Nix queued; Paid OpenRouter, clean-pack and Tree-sitter started). **HOLD promotion until every exact-head gate completes.** The branch also remains behind current main; after behavioral verification, recover the existing transaction onto `ad655ca5...` and require another fresh exact-head gate rather than reusing this evidence.

lost-rob0t commented

2026-08-27 09:14:14 +00:00

(Migrated from github.com)

RAGE update — exact-head #242 budget regressions

Current canonical main: ad655ca51f57148833f1294f511c0b5d1f24cdde.

Regression 1 — CLI response cap collapsed mandatory runtime context

On exact #242 head 9180794f7469622b50e89c2b01f01a1d13fae875, both normal REAL OpenRouter and Paid OpenRouter passed the substantive provider/runtime suites (including the #183 operator-behavior acceptance and depth 0/1/2 experiment) and then failed identically in the one-command RLM CLI smoke before provider execution:

prompt_compiler_fault(context_budget_failed(no_feasible_context_pack(512)))

The CLI documents --max-tokens as the direct/child response limit, but completion_budget_from_options/2 also derived the whole-run token budget as max(512, max_tokens*4). The smoke's --max-tokens 96 therefore reduced the total runtime budget to 512, below the now-mandatory permanent skill context.

TDD-first evidence:

37576c176f01b4791b065fbba156da31b2f3bc7f — deterministic contract: a 96-token output cap must still retain finite >=2048 runtime headroom for mandatory context.
da69c15bf7c76eb08a6b46d4f1de5ba73669f0e8 — CLI-local realization: floor 512 -> 2048; response cap, cost ceiling, authority, capabilities and production prompt budgeting otherwise unchanged.

Adversarial note: the first realization commit accidentally changed an unrelated effect-store output binding during full-file replacement. Immediate diff review caught it before acceptance; da69c15... restores that binding. Net realization relative to the TDD commit is exactly the one-line budget-floor change.

Exact da69c15... results: deterministic CI job, Nix, clean SWI pack, Tree-sitter and Paid OpenRouter all pass. Normal REAL OpenRouter moved past the old CLI failure but exposed a separate live-fixture output-byte exhaustion.

Regression 2 — #183 behavior fixture output-byte headroom

Normal REAL OpenRouter at da69c15... successfully executed the decomposable behavior through both bounded child rlm calls and produced correct child verdicts (RLM_VERDICT_REVIEW, RLM_VERDICT_CLEAR). It then failed before parent synthesis with:

budget_exhausted / budget:output_bytes

Evidence: 32 KiB live-fixture ceiling, 6,645 bytes remaining, 8,968 requested. The provider emitted substantially more reasoning in this lane; Paid OpenRouter passed the same finite fixture. This is not evidence that recursion/authority/typed execution failed, and the behavioral assertions remain unchanged.

TDD-first follow-up:

c8d07f050c3c93cd9b5ccc5886df44540087a5b3 — contract requires the behavior fixture to keep finite provider-reasoning headroom: max_output_bytes >= 65536 and =< 131072.
2b30bc29ca5226e877ee7a21d185b7d2bf024678 — realization raises only this test fixture output-byte ceiling 32768 -> 65536. Production output budgeting and all #183 semantic acceptance assertions are unchanged.

Adversarial decision: GO for this bounded fixture-only headroom repair; HOLD PR #242 promotion until 2b30bc29... earns a completely fresh exact-head gate. Do not reuse the paid-green evidence from da69c15... for the new SHA.

## RAGE update — exact-head #242 budget regressions Current canonical `main`: `ad655ca51f57148833f1294f511c0b5d1f24cdde`. ### Regression 1 — CLI response cap collapsed mandatory runtime context On exact #242 head `9180794f7469622b50e89c2b01f01a1d13fae875`, both normal REAL OpenRouter and Paid OpenRouter passed the substantive provider/runtime suites (including the #183 operator-behavior acceptance and depth 0/1/2 experiment) and then failed identically in the one-command RLM CLI smoke before provider execution: `prompt_compiler_fault(context_budget_failed(no_feasible_context_pack(512)))` The CLI documents `--max-tokens` as the direct/child response limit, but `completion_budget_from_options/2` also derived the whole-run token budget as `max(512, max_tokens*4)`. The smoke's `--max-tokens 96` therefore reduced the total runtime budget to 512, below the now-mandatory permanent skill context. TDD-first evidence: - `37576c176f01b4791b065fbba156da31b2f3bc7f` — deterministic contract: a 96-token output cap must still retain finite >=2048 runtime headroom for mandatory context. - `da69c15bf7c76eb08a6b46d4f1de5ba73669f0e8` — CLI-local realization: floor `512 -> 2048`; response cap, cost ceiling, authority, capabilities and production prompt budgeting otherwise unchanged. Adversarial note: the first realization commit accidentally changed an unrelated effect-store output binding during full-file replacement. Immediate diff review caught it before acceptance; `da69c15...` restores that binding. Net realization relative to the TDD commit is exactly the one-line budget-floor change. Exact `da69c15...` results: deterministic CI job, Nix, clean SWI pack, Tree-sitter and **Paid OpenRouter all pass**. Normal REAL OpenRouter moved past the old CLI failure but exposed a separate live-fixture output-byte exhaustion. ### Regression 2 — #183 behavior fixture output-byte headroom Normal REAL OpenRouter at `da69c15...` successfully executed the decomposable behavior through both bounded child `rlm` calls and produced correct child verdicts (`RLM_VERDICT_REVIEW`, `RLM_VERDICT_CLEAR`). It then failed before parent synthesis with: `budget_exhausted / budget:output_bytes` Evidence: 32 KiB live-fixture ceiling, 6,645 bytes remaining, 8,968 requested. The provider emitted substantially more reasoning in this lane; Paid OpenRouter passed the same finite fixture. This is not evidence that recursion/authority/typed execution failed, and the behavioral assertions remain unchanged. TDD-first follow-up: - `c8d07f050c3c93cd9b5ccc5886df44540087a5b3` — contract requires the behavior fixture to keep finite provider-reasoning headroom: `max_output_bytes >= 65536` and `=< 131072`. - `2b30bc29ca5226e877ee7a21d185b7d2bf024678` — realization raises only this **test fixture** output-byte ceiling `32768 -> 65536`. Production output budgeting and all #183 semantic acceptance assertions are unchanged. Adversarial decision: **GO** for this bounded fixture-only headroom repair; **HOLD PR #242 promotion** until `2b30bc29...` earns a completely fresh exact-head gate. Do not reuse the paid-green evidence from `da69c15...` for the new SHA.

lost-rob0t commented

2026-08-27 10:05:53 +00:00

(Migrated from github.com)

RAGE exact-head correction for PR #242 after inspecting the completed credential-backed runs on 2b30bc29ca5226e877ee7a21d185b7d2bf024678 (PR merge ref 4d42b5a...).

The prior working hypothesis that the remaining failure was only output-byte headroom is disproven by the fresh exact-head gate. Deterministic CI, Nix, clean-pack, and Tree-sitter are green, but BOTH live core lanes fail inside live_rlm_operator_behavior_openrouter before later live suites run.

REAL OpenRouter failure is semantic and falsifiable: trivial and typed-tool cases pass, but the decomposable case chooses context(peek) -> context(peek) -> model -> model -> final, with recursive_calls:0, max_depth:0, and no successful rlm transition. It computes the correct REVIEW/CLEAR verdicts, but violates the approved #183 requirement that explicitly separate independent investigation use bounded recursion when rlm is available. This is not a fixture-byte/token failure.

Pinned Paid OpenRouter fails the same decomposable case differently: both planner attempts end in structural validation failure final_must_be_unique_and_last, surfaced correctly as plan_validation_failed. That proves the structural retry path is active and fail-closed; it does NOT satisfy the behavioral acceptance.

Adversarial decision: HOLD #242. Do not weaken recursive_calls >= 1, do not accept sibling root model calls as recursion, do not add an exact plan, and do not suppress the paid structural failure. The current rlm-recurse prose is already very explicit, so further prompt inflation is not automatically justified. Next analysis should determine whether the behavioral contract is too provider-stochastic for a single-shot gate versus whether the runtime needs a generic typed delegation/planning affordance inside the already-approved #183/#172 architecture. Any material new architecture must return to design approval rather than being patched into this PR.

Also note current canonical main has advanced to b891c27c3d488ab336ebc125e9e04821351b0f22; #242 is now non-mergeable and must be recovered/rebased only after the behavioral failure has a justified design-preserving repair. No old exact-head evidence may be reused.

RAGE exact-head correction for PR #242 after inspecting the completed credential-backed runs on `2b30bc29ca5226e877ee7a21d185b7d2bf024678` (PR merge ref `4d42b5a...`). The prior working hypothesis that the remaining failure was only output-byte headroom is disproven by the fresh exact-head gate. Deterministic CI, Nix, clean-pack, and Tree-sitter are green, but BOTH live core lanes fail inside `live_rlm_operator_behavior_openrouter` before later live suites run. REAL OpenRouter failure is semantic and falsifiable: trivial and typed-tool cases pass, but the decomposable case chooses `context(peek) -> context(peek) -> model -> model -> final`, with `recursive_calls:0`, `max_depth:0`, and no successful `rlm` transition. It computes the correct REVIEW/CLEAR verdicts, but violates the approved #183 requirement that explicitly separate independent investigation use bounded recursion when `rlm` is available. This is not a fixture-byte/token failure. Pinned Paid OpenRouter fails the same decomposable case differently: both planner attempts end in structural validation failure `final_must_be_unique_and_last`, surfaced correctly as `plan_validation_failed`. That proves the structural retry path is active and fail-closed; it does NOT satisfy the behavioral acceptance. Adversarial decision: HOLD #242. Do not weaken `recursive_calls >= 1`, do not accept sibling root model calls as recursion, do not add an exact plan, and do not suppress the paid structural failure. The current `rlm-recurse` prose is already very explicit, so further prompt inflation is not automatically justified. Next analysis should determine whether the behavioral contract is too provider-stochastic for a single-shot gate versus whether the runtime needs a generic typed delegation/planning affordance inside the already-approved #183/#172 architecture. Any material new architecture must return to design approval rather than being patched into this PR. Also note current canonical `main` has advanced to `b891c27c3d488ab336ebc125e9e04821351b0f22`; #242 is now non-mergeable and must be recovered/rebased only after the behavioral failure has a justified design-preserving repair. No old exact-head evidence may be reused.

lost-rob0t commented

2026-08-27 11:08:19 +00:00

(Migrated from github.com)

RAGE same-head repeatability experiment — #242 `2b30bc29ca5226e877ee7a21d185b7d2bf024678`

Fresh evidence changes the design diagnosis again. I reran only the failed credential-backed REAL OpenRouter job on the same immutable PR head / same PR merge ref; no source, fixture, prompt, model selection, or budget changed.

Evidence

Original exact-head REAL run on this SHA:

trivial case: pass;
typed-tool case: pass;
decomposable case selected sibling root work (context(peek) -> context(peek) -> model -> model -> final);
recursive_calls:0, max_depth:0, no successful rlm transition;
REVIEW/CLEAR verdicts were correct.

Same-SHA rerun (REAL OpenRouter job 98495025375, model openai/gpt-oss-120b):

trivial case: pass;
typed-tool case: pass;
decomposable case selected two real nested rlm children;
each child performed context(peek(item(...))) -> model(openrouter) -> final;
runtime reported recursive_calls:2, max_depth:1, model_calls:3 with two depth-1 provider calls;
REVIEW/CLEAR verdicts were again correct;
the rerun failed only because the final parent object dropped both explicitly requested source/provenance tokens.

So the exact same code + provider/model + fixture has now produced both sides of the hard recursion-choice assertion. This is direct evidence that the current single-shot behavioral gate is sampling provider policy behavior, not a deterministic runtime contract. It also shows the existing provenance instruction is not sufficient to make the final-shape behavior stable on every sample.

Additional runtime fact

Current rlm_completion already sends planner temperature:0 by default (planner_temperature defaults to 0). Therefore the observed plan-policy flip is not explained by this fixture accidentally using a nonzero planner temperature. The generic OpenAI-compatible transport supports seed, but rlm_completion currently does not expose a planner-seed control.

Adversarial decision

HOLD #242. Do not retry-until-green, weaken recursive_calls >= 1, drop provenance assertions, accept sibling root model calls as recursion, or add an exact plan. A passing rerun would not make a stochastic single-shot assertion a valid deterministic merge contract.

The next step is design analysis, not more prompt inflation. We need to decide inside the approved #183 design whether live provider behavior is intended as a repeated/evaluation signal while deterministic runtime contracts remain merge-blocking, or whether a trusted reproducibility control (for example a provider-supported planner seed) belongs in the generic runtime/test surface. Adding such a control or changing the semantics of the acceptance gate is a material design choice and must return to the #183 design gate before realization.

Current canonical main immediately before this write: 7f0050f524db6de5293163241176ac4660f04349. PR #242 remains draft/non-mergeable on 2b30bc29...; no submitted reviews or unresolved review threads exist.

## RAGE same-head repeatability experiment — #242 `2b30bc29ca5226e877ee7a21d185b7d2bf024678` Fresh evidence changes the design diagnosis again. I reran only the failed credential-backed REAL OpenRouter job on the **same immutable PR head / same PR merge ref**; no source, fixture, prompt, model selection, or budget changed. ### Evidence Original exact-head REAL run on this SHA: - trivial case: pass; - typed-tool case: pass; - decomposable case selected sibling root work (`context(peek) -> context(peek) -> model -> model -> final`); - `recursive_calls:0`, `max_depth:0`, no successful `rlm` transition; - REVIEW/CLEAR verdicts were correct. Same-SHA rerun (REAL OpenRouter job `98495025375`, model `openai/gpt-oss-120b`): - trivial case: pass; - typed-tool case: pass; - decomposable case selected **two real nested `rlm` children**; - each child performed `context(peek(item(...))) -> model(openrouter) -> final`; - runtime reported `recursive_calls:2`, `max_depth:1`, `model_calls:3` with two depth-1 provider calls; - REVIEW/CLEAR verdicts were again correct; - the rerun failed only because the final parent object dropped both explicitly requested source/provenance tokens. So the exact same code + provider/model + fixture has now produced both sides of the hard recursion-choice assertion. This is direct evidence that the current single-shot behavioral gate is sampling provider policy behavior, not a deterministic runtime contract. It also shows the existing provenance instruction is not sufficient to make the final-shape behavior stable on every sample. ### Additional runtime fact Current `rlm_completion` already sends planner `temperature:0` by default (`planner_temperature` defaults to `0`). Therefore the observed plan-policy flip is **not** explained by this fixture accidentally using a nonzero planner temperature. The generic OpenAI-compatible transport supports `seed`, but `rlm_completion` currently does not expose a planner-seed control. ### Adversarial decision **HOLD #242. Do not retry-until-green, weaken `recursive_calls >= 1`, drop provenance assertions, accept sibling root model calls as recursion, or add an exact plan.** A passing rerun would not make a stochastic single-shot assertion a valid deterministic merge contract. The next step is design analysis, not more prompt inflation. We need to decide inside the approved #183 design whether live provider behavior is intended as a repeated/evaluation signal while deterministic runtime contracts remain merge-blocking, or whether a trusted reproducibility control (for example a provider-supported planner seed) belongs in the generic runtime/test surface. Adding such a control or changing the semantics of the acceptance gate is a material design choice and must return to the #183 design gate before realization. Current canonical `main` immediately before this write: `7f0050f524db6de5293163241176ac4660f04349`. PR #242 remains draft/non-mergeable on `2b30bc29...`; no submitted reviews or unresolved review threads exist.