nsaspy/prolog-rlm

Fork 0

[EPIC] Real constraint-solving RLM benchmark + durable context mounts #223

New issue

Open

opened 2026-08-26 08:14:25 +00:00 by lost-rob0t · 11 comments

lost-rob0t commented

2026-08-26 08:14:25 +00:00

(Migrated from github.com)

Goal

Turn the current live deep experiment into a real end-to-end reasoning benchmark that exercises planning, recursive RLM execution, constraint solving, and trusted verification, then immediately add a first-class durable context-mount abstraction without bloating the core model prompt.

This epic is intentionally ordered:

Real benchmark first — establish an honest measurement surface before changing context semantics.
Durable/persistent context mounts second — add reusable context lifetime/visibility semantics after the benchmark can measure regressions/improvements.

The architectural rule is:

core RLM prompt = minimal runtime contract
CLI/downstream harness prompts = allowed to be richer/task-specific
verification = trusted host logic, never model self-report
permanent context != permanently injected prompt text

Slice 1 — Replace token-echo depth test with a real constraint-solving benchmark

Problem

benchmark/rlm_live_deep_experiment.pl currently injects a fixed typed plan and asks nested model calls to return LIVE_DEEP_OK.

That is useful for provider/recursion plumbing, but it does not prove:

the root planner can choose a useful plan;
the model can decompose a hard problem;
RLM recursion improves or preserves solution quality;
the produced answer satisfies the task rather than merely containing an expected token;
Plan -> Execute -> Observe -> Verify semantics are correct for a real reasoning task.

The benchmark must stop treating a magic token as correctness.

Benchmark contract

Create a deterministic, hard, uniquely-solvable finite-domain constraint problem tailored to the runtime.

Target properties:

enough variables/domains that blind guessing is effectively impossible;
all-different constraints;
arithmetic constraints;
ordering constraints;
implications / biconditionals;
cardinality constraints;
cross-group dependencies;
locally plausible but globally invalid partial assignments;
exactly one valid global solution, proven independently by trusted Prolog.

Prefer a structured logic-grid/scheduling/resource-assignment CSP whose entire instance can be generated/represented declaratively and checked with clpfd or equivalent trusted Prolog constraints.

Required execution lanes

A. `core-minimal`

The authoritative architecture test.

The benchmark supplies only the actual task and normal runtime metadata/capabilities. It must not inject an exact plan or a giant benchmark-specific planner prompt.

Core should continue to provide only the minimal typed-plan runtime contract already owned by rlm_completion.

This lane tells us whether core prompting/runtime semantics are sufficient.

B. `harness-guided`

A comparison lane owned by benchmark/CLI/downstream code.

It may add task-specific planning guidance, decomposition advice, output-shape reminders, or other scaffolding.

This lane must not change trusted verification.

The delta between A and B is diagnostic:

minimal weak + guided strong -> improve core interface/prompt semantics carefully
both weak                  -> likely planning/model/runtime limitation
both solve + verify rejects -> output/handoff/verification mismatch
verify catches confident bad answer -> verifier working correctly

Do not silently move harness guidance into the core just to make the benchmark green.

Depth / RLM comparison

Run meaningful depth variants where supported, at minimum depth 0/1/2.

Depth must not simply add identical echo calls. The selected/generated plan must actually use recursion/decomposition for the deeper lanes when RLM is selected.

Record at least:

requested/actual recursion depth;
recursive call count;
model call count;
planner attempt count if observable;
prompt/completion/total tokens;
cost;
latency;
selected provider/model;
plan parsed/validated status;
verification status;
solution completeness;
violated constraint IDs on failure;
whether the solution is the unique satisfying assignment;
lane (core-minimal vs harness-guided).

Trusted verification

Correctness must be decided by Prolog, not by substring matching and not by the LLM saying "verified".

Implement a verifier that:

parses/normalizes the model's final structured assignment;
requires every variable exactly once;
rejects unknown/missing/duplicate values;
checks every CSP constraint independently;
checks the assignment equals a satisfying solution;
proves the fixture itself has exactly one solution (deterministic test/startup validation);
reports structured failure details.

Prefer reusing the existing Spec/Verify semantic split where practical:

Frozen Spec = desired solution/acceptance requirements
Plan        = proposed strategy
Execution   = produced assignment
Observation = normalized observed assignment/constraint evidence
Verify      = trusted acceptance decision

Do not create executable authority from model-generated Prolog.

TDD / deterministic tests

Before relying on real provider runs, add deterministic tests proving:

fixture has exactly one solution;
valid known solution passes;
single-field mutation fails;
incomplete assignment fails;
duplicate domain value fails;
malformed output fails safely;
verifier reports violated constraint IDs;
benchmark status is based on verification, not token presence;
core-minimal does not install benchmark-specific exact-plan instructions;
harness-guided guidance remains downstream-only;
depth metrics are internally consistent.

Existing provider/plumbing benchmark

Do not lose the current useful provider/recursion smoke coverage. Either:

retain the old echo-depth experiment under an explicitly named smoke/plumbing mode; or
extract its assertions into a smaller provider-recursion smoke benchmark/test.

The main deep-integration reasoning benchmark should become the real CSP test.

CLI / benchmark runner

Update benchmark/run.pl as needed so users can explicitly run the real benchmark and compare lanes. Keep existing deterministic/integration behavior backwards-compatible where reasonable.

Suggested modes may include:

deep-integration
constraint-integration
constraint-guided

Exact naming can follow existing conventions; avoid redundant aliases.

Slice 2 — Durable context mounts / lifetime + visibility

Begin immediately after the benchmark slice is landed/working.

Current behavior

rlm_context currently supports process-local context records. Caller-owned context_ref / context_handle values can survive multiple rlm_completion/4 calls because completion does not delete caller-owned handles.

However the built-in memory backend is explicitly non-persistent, so handles do not survive a process restart.

Design principle

Do not add only permanent(true).

Separate two independent semantic axes:

Lifetime

ephemeral
session
persistent

Model visibility

At minimum:

opaque
prompt

Default persistent context visibility must be opaque.

Persistent does not mean inject into every prompt forever.

The planner should receive bounded metadata / mount identity and use normal context operations when content is required.

Proposed host-facing abstraction

Conceptually:

context_mount(Name,
              Source,
              [ lifetime(persistent),
                scope(project(prolog_rlm)),
                visibility(opaque)
              ],
              Outcome).

Exact names are open to implementation review, but preserve the separation of lifetime, scope, visibility and source identity.

Persistence model

Persist the mount/source identity and policy, not a magical serialized live handle.

Conceptually:

persistent mount record
  name
  adapter/backend
  closed source_ref
  lifetime/scope/visibility policy
  provenance/fingerprint/version
        |
        v
process restart
        |
        v
rehydrate a fresh context_handle

Adapters remain trusted host boundaries. Model-generated data must not register executable adapter callbacks.

Scope

Support explicit scope so durable context does not become accidental global ambient state.

Initial scope vocabulary can be small, e.g.:

runtime
session(SessionId)
project(ProjectId)

Avoid user-global implicit injection as a default.

Required tests

caller-owned existing context remains reusable across completions;
ephemeral context is deleted after owned completion cleanup;
session context survives repeated calls in the process;
persistent mount metadata survives backend reopen/restart simulation;
persistent mount rehydrates to a fresh valid handle;
persistent opaque content is not copied into root prompt text automatically;
model sees only allowed bounded mount metadata;
explicit visibility(prompt) is opt-in and bounded;
deleting/unmounting invalidates future resolution safely;
scope mismatch denies resolution;
adapter executable callbacks remain trusted registry state and are never serialized as model-controlled data;
no capability/authority escalation occurs from persistence.

Prompt ownership / benchmark interpretation

Keep the library core intentionally minimal.

rlm_completion may describe:

typed plan format/runtime contract;
goal;
bounded context metadata;
root capabilities;
child capabilities;
active tool schemas;
concise recursive semantics.

It should not grow benchmark-specific chain-of-thought coaching, exact plans, or domain-specific CSP tutorials just to improve benchmark results.

CLI, benchmark harnesses, applications and downstream agents may add richer guidance deliberately and observably.

The real benchmark must report both minimal and guided behavior so prompt changes can be evaluated instead of guessed.

Implementation order

Re-read exact current main and existing live benchmark/test APIs.
Add deterministic CSP fixture + trusted verifier tests.
Replace/add real live benchmark lane with minimal core prompting.
Add harness-guided comparison lane.
Preserve provider/recursion smoke coverage separately.
Run focused deterministic tests and aggregate test gate.
Run live provider benchmark where credential is available; publish metrics/artifacts without pretending an unavailable credential is a correctness pass.
Only then begin durable context-mount implementation.
Add persistence/lifetime/visibility tests first.
Implement smallest durable mount backend/interface consistent with rlm_context adapters and existing artifact/storage abstractions.
Re-run benchmark to detect prompt/context regressions.

Acceptance gate

The epic is complete only when:

a real provider can be asked to solve a genuinely difficult, uniquely-solvable CSP through rlm_completion;
correctness is independently machine-verified by trusted Prolog;
depth 0/1/2 produce comparable structured metrics;
the benchmark distinguishes core-minimal vs downstream-guided prompting;
no exact injected plan is required for the authoritative benchmark lane;
existing provider/recursion plumbing coverage is preserved;
failures identify verifier/constraint details rather than merely missing a magic token;
persistent context has explicit lifetime, scope and visibility semantics;
persistent opaque context is reusable without automatic prompt injection;
persistent mounts survive backend reopen/restart through rehydration;
core prompt remains minimal and domain-neutral;
focused tests + aggregate deterministic gates pass.

Start implementation with Slice 1 immediately.

## Goal Turn the current live deep experiment into a real end-to-end reasoning benchmark that exercises **planning, recursive RLM execution, constraint solving, and trusted verification**, then immediately add a first-class durable context-mount abstraction without bloating the core model prompt. This epic is intentionally ordered: 1. **Real benchmark first** — establish an honest measurement surface before changing context semantics. 2. **Durable/persistent context mounts second** — add reusable context lifetime/visibility semantics after the benchmark can measure regressions/improvements. The architectural rule is: ```text core RLM prompt = minimal runtime contract CLI/downstream harness prompts = allowed to be richer/task-specific verification = trusted host logic, never model self-report permanent context != permanently injected prompt text ``` --- # Slice 1 — Replace token-echo depth test with a real constraint-solving benchmark ## Problem `benchmark/rlm_live_deep_experiment.pl` currently injects a fixed typed plan and asks nested model calls to return `LIVE_DEEP_OK`. That is useful for provider/recursion plumbing, but it does **not** prove: - the root planner can choose a useful plan; - the model can decompose a hard problem; - RLM recursion improves or preserves solution quality; - the produced answer satisfies the task rather than merely containing an expected token; - Plan -> Execute -> Observe -> Verify semantics are correct for a real reasoning task. The benchmark must stop treating a magic token as correctness. ## Benchmark contract Create a deterministic, hard, uniquely-solvable finite-domain constraint problem tailored to the runtime. Target properties: - enough variables/domains that blind guessing is effectively impossible; - all-different constraints; - arithmetic constraints; - ordering constraints; - implications / biconditionals; - cardinality constraints; - cross-group dependencies; - locally plausible but globally invalid partial assignments; - exactly one valid global solution, proven independently by trusted Prolog. Prefer a structured logic-grid/scheduling/resource-assignment CSP whose entire instance can be generated/represented declaratively and checked with `clpfd` or equivalent trusted Prolog constraints. ## Required execution lanes ### A. `core-minimal` The authoritative architecture test. The benchmark supplies only the actual task and normal runtime metadata/capabilities. It must **not** inject an exact plan or a giant benchmark-specific planner prompt. Core should continue to provide only the minimal typed-plan runtime contract already owned by `rlm_completion`. This lane tells us whether core prompting/runtime semantics are sufficient. ### B. `harness-guided` A comparison lane owned by benchmark/CLI/downstream code. It may add task-specific planning guidance, decomposition advice, output-shape reminders, or other scaffolding. This lane must not change trusted verification. The delta between A and B is diagnostic: ```text minimal weak + guided strong -> improve core interface/prompt semantics carefully both weak -> likely planning/model/runtime limitation both solve + verify rejects -> output/handoff/verification mismatch verify catches confident bad answer -> verifier working correctly ``` Do not silently move harness guidance into the core just to make the benchmark green. ## Depth / RLM comparison Run meaningful depth variants where supported, at minimum depth 0/1/2. Depth must not simply add identical echo calls. The selected/generated plan must actually use recursion/decomposition for the deeper lanes when RLM is selected. Record at least: - requested/actual recursion depth; - recursive call count; - model call count; - planner attempt count if observable; - prompt/completion/total tokens; - cost; - latency; - selected provider/model; - plan parsed/validated status; - verification status; - solution completeness; - violated constraint IDs on failure; - whether the solution is the unique satisfying assignment; - lane (`core-minimal` vs `harness-guided`). ## Trusted verification Correctness must be decided by Prolog, not by substring matching and not by the LLM saying "verified". Implement a verifier that: 1. parses/normalizes the model's final structured assignment; 2. requires every variable exactly once; 3. rejects unknown/missing/duplicate values; 4. checks every CSP constraint independently; 5. checks the assignment equals a satisfying solution; 6. proves the fixture itself has exactly one solution (deterministic test/startup validation); 7. reports structured failure details. Prefer reusing the existing Spec/Verify semantic split where practical: ```text Frozen Spec = desired solution/acceptance requirements Plan = proposed strategy Execution = produced assignment Observation = normalized observed assignment/constraint evidence Verify = trusted acceptance decision ``` Do not create executable authority from model-generated Prolog. ## TDD / deterministic tests Before relying on real provider runs, add deterministic tests proving: - fixture has exactly one solution; - valid known solution passes; - single-field mutation fails; - incomplete assignment fails; - duplicate domain value fails; - malformed output fails safely; - verifier reports violated constraint IDs; - benchmark status is based on verification, not token presence; - `core-minimal` does not install benchmark-specific exact-plan instructions; - `harness-guided` guidance remains downstream-only; - depth metrics are internally consistent. ## Existing provider/plumbing benchmark Do not lose the current useful provider/recursion smoke coverage. Either: - retain the old echo-depth experiment under an explicitly named smoke/plumbing mode; or - extract its assertions into a smaller provider-recursion smoke benchmark/test. The main `deep-integration` reasoning benchmark should become the real CSP test. ## CLI / benchmark runner Update `benchmark/run.pl` as needed so users can explicitly run the real benchmark and compare lanes. Keep existing deterministic/integration behavior backwards-compatible where reasonable. Suggested modes may include: ```text deep-integration constraint-integration constraint-guided ``` Exact naming can follow existing conventions; avoid redundant aliases. --- # Slice 2 — Durable context mounts / lifetime + visibility Begin immediately after the benchmark slice is landed/working. ## Current behavior `rlm_context` currently supports process-local context records. Caller-owned `context_ref` / `context_handle` values can survive multiple `rlm_completion/4` calls because completion does not delete caller-owned handles. However the built-in memory backend is explicitly non-persistent, so handles do not survive a process restart. ## Design principle Do **not** add only `permanent(true)`. Separate two independent semantic axes: ### Lifetime ```prolog ephemeral session persistent ``` ### Model visibility At minimum: ```prolog opaque prompt ``` Default persistent context visibility must be `opaque`. **Persistent does not mean inject into every prompt forever.** The planner should receive bounded metadata / mount identity and use normal context operations when content is required. ## Proposed host-facing abstraction Conceptually: ```prolog context_mount(Name, Source, [ lifetime(persistent), scope(project(prolog_rlm)), visibility(opaque) ], Outcome). ``` Exact names are open to implementation review, but preserve the separation of lifetime, scope, visibility and source identity. ## Persistence model Persist the mount/source identity and policy, **not a magical serialized live handle**. Conceptually: ```text persistent mount record name adapter/backend closed source_ref lifetime/scope/visibility policy provenance/fingerprint/version | v process restart | v rehydrate a fresh context_handle ``` Adapters remain trusted host boundaries. Model-generated data must not register executable adapter callbacks. ## Scope Support explicit scope so durable context does not become accidental global ambient state. Initial scope vocabulary can be small, e.g.: ```prolog runtime session(SessionId) project(ProjectId) ``` Avoid user-global implicit injection as a default. ## Required tests - caller-owned existing context remains reusable across completions; - ephemeral context is deleted after owned completion cleanup; - session context survives repeated calls in the process; - persistent mount metadata survives backend reopen/restart simulation; - persistent mount rehydrates to a fresh valid handle; - persistent opaque content is **not** copied into root prompt text automatically; - model sees only allowed bounded mount metadata; - explicit `visibility(prompt)` is opt-in and bounded; - deleting/unmounting invalidates future resolution safely; - scope mismatch denies resolution; - adapter executable callbacks remain trusted registry state and are never serialized as model-controlled data; - no capability/authority escalation occurs from persistence. --- # Prompt ownership / benchmark interpretation Keep the library core intentionally minimal. `rlm_completion` may describe: - typed plan format/runtime contract; - goal; - bounded context metadata; - root capabilities; - child capabilities; - active tool schemas; - concise recursive semantics. It should **not** grow benchmark-specific chain-of-thought coaching, exact plans, or domain-specific CSP tutorials just to improve benchmark results. CLI, benchmark harnesses, applications and downstream agents may add richer guidance deliberately and observably. The real benchmark must report both minimal and guided behavior so prompt changes can be evaluated instead of guessed. --- # Implementation order 1. Re-read exact current `main` and existing live benchmark/test APIs. 2. Add deterministic CSP fixture + trusted verifier tests. 3. Replace/add real live benchmark lane with minimal core prompting. 4. Add harness-guided comparison lane. 5. Preserve provider/recursion smoke coverage separately. 6. Run focused deterministic tests and aggregate test gate. 7. Run live provider benchmark where credential is available; publish metrics/artifacts without pretending an unavailable credential is a correctness pass. 8. Only then begin durable context-mount implementation. 9. Add persistence/lifetime/visibility tests first. 10. Implement smallest durable mount backend/interface consistent with `rlm_context` adapters and existing artifact/storage abstractions. 11. Re-run benchmark to detect prompt/context regressions. --- # Acceptance gate The epic is complete only when: - a real provider can be asked to solve a genuinely difficult, uniquely-solvable CSP through `rlm_completion`; - correctness is independently machine-verified by trusted Prolog; - depth 0/1/2 produce comparable structured metrics; - the benchmark distinguishes core-minimal vs downstream-guided prompting; - no exact injected plan is required for the authoritative benchmark lane; - existing provider/recursion plumbing coverage is preserved; - failures identify verifier/constraint details rather than merely missing a magic token; - persistent context has explicit lifetime, scope and visibility semantics; - persistent opaque context is reusable without automatic prompt injection; - persistent mounts survive backend reopen/restart through rehydration; - core prompt remains minimal and domain-neutral; - focused tests + aggregate deterministic gates pass. Start implementation with Slice 1 immediately.

lost-rob0t commented

2026-08-26 08:27:09 +00:00

(Migrated from github.com)

Implementation started on draft PR #225.

Current Slice 1 state:

real 10-task / 30-value uniquely-solvable CSP fixture;
CLP(FD) oracle with exact violated constraint IDs;
final acceptance routed through Frozen Spec + production rlm_verify;
core_minimal vs harness_guided, depth ceilings 0/1/2;
old fixed-plan token echo preserved as deep-smoke;
focused deterministic runner added.

CI and the credentialed live OpenRouter depth gate are running now. Slice 2 (durable context mounts) stays next in this epic once this benchmark vertical slice is working.

Implementation started on draft PR #225. Current Slice 1 state: - real 10-task / 30-value uniquely-solvable CSP fixture; - CLP(FD) oracle with exact violated constraint IDs; - final acceptance routed through Frozen Spec + production `rlm_verify`; - `core_minimal` vs `harness_guided`, depth ceilings 0/1/2; - old fixed-plan token echo preserved as `deep-smoke`; - focused deterministic runner added. CI and the credentialed live OpenRouter depth gate are running now. Slice 2 (durable context mounts) stays next in this epic once this benchmark vertical slice is working.

lost-rob0t commented

2026-08-26 08:36:20 +00:00

(Migrated from github.com)

Slice 2 has now started as stacked draft PR #228 (feature/223-durable-context-mounts on top of #225). It reuses rlm_artifact for persistent mount descriptors, defaults visibility to opaque, rehydrates fresh context handles after restart, and keeps adapter callbacks process-local/trusted. Static production-module loading is already green on the stacked CI run; focused mount tests are in test/run_context_mount.pl.

Slice 2 has now started as stacked draft PR #228 (`feature/223-durable-context-mounts` on top of #225). It reuses `rlm_artifact` for persistent mount descriptors, defaults visibility to opaque, rehydrates fresh context handles after restart, and keeps adapter callbacks process-local/trusted. Static production-module loading is already green on the stacked CI run; focused mount tests are in `test/run_context_mount.pl`.

lost-rob0t commented

2026-08-26 19:05:27 +00:00

(Migrated from github.com)

RAGE live-provider diagnosis for Slice 1 / PR #225, exact candidate 75b04abc96f8520bdf762e43e8e40d0c179edc36 (synthetic merge 611eedc9bf70fb4b12d2825273719d2104466bea into unchanged main 267697bef10a3fffff7c093e1435ece770e7444b).

Deterministic/runtime packaging evidence is healthy: deterministic CI job, Nix, clean SWI pack and Tree-sitter are green. The failing surface is specifically the new credential-backed deep-integration CSP benchmark, and it reproduces in both normal REAL OpenRouter (openai/gpt-oss-120b) and pinned Paid OpenRouter lanes.

Observed exact failure matrix: all six core_minimal|harness_guided × depth 0|1|2 cases fail before execution/verification with completion_error{phase:planner,kind:plan_parse_failed,...cause:plan_error{phase:parse,kind:invalid_plan,detail:no_json_object,...}}. Each case consumed the configured two planner attempts; no candidate plan parsed, no CSP verification was reached, and no plan operation executed. The ordinary live core, structured-repair, tool, recursive-completion and direct integration benchmark tests immediately before this step all passed on the same workflow, so this is not evidence of general OpenRouter/runtime outage.

Adversarial interpretation / decision: HOLD Slice 1 as a live behavioral failure, not a deterministic-runtime failure and not a verifier failure. Do not weaken the benchmark, inject a fixed plan, or move task-specific CSP coaching into core to make it green. The next repair should preserve the core_minimal contract and first inspect the actual provider response/output-channel behavior that produced no_json_object; harness_guided may add downstream-only guidance, but both lanes currently failing identically means we do not yet have evidence that recursion depth or the trusted verifier is the limiting factor.

This also means stacked Slice 2 PR #228 must not be treated as epic-ready merely because its own focused/static checks advance: #223 explicitly orders the real benchmark first. No branch mutation performed here because #225/#228 are already active owned transactions.

RAGE live-provider diagnosis for Slice 1 / PR #225, exact candidate `75b04abc96f8520bdf762e43e8e40d0c179edc36` (synthetic merge `611eedc9bf70fb4b12d2825273719d2104466bea` into unchanged `main` `267697bef10a3fffff7c093e1435ece770e7444b`). Deterministic/runtime packaging evidence is healthy: deterministic CI job, Nix, clean SWI pack and Tree-sitter are green. The failing surface is specifically the new credential-backed `deep-integration` CSP benchmark, and it reproduces in both normal REAL OpenRouter (`openai/gpt-oss-120b`) and pinned Paid OpenRouter lanes. Observed exact failure matrix: all six `core_minimal|harness_guided × depth 0|1|2` cases fail before execution/verification with `completion_error{phase:planner,kind:plan_parse_failed,...cause:plan_error{phase:parse,kind:invalid_plan,detail:no_json_object,...}}`. Each case consumed the configured two planner attempts; no candidate plan parsed, no CSP verification was reached, and no plan operation executed. The ordinary live core, structured-repair, tool, recursive-completion and direct integration benchmark tests immediately before this step all passed on the same workflow, so this is not evidence of general OpenRouter/runtime outage. Adversarial interpretation / decision: HOLD Slice 1 as a live behavioral failure, not a deterministic-runtime failure and not a verifier failure. Do not weaken the benchmark, inject a fixed plan, or move task-specific CSP coaching into core to make it green. The next repair should preserve the `core_minimal` contract and first inspect the actual provider response/output-channel behavior that produced `no_json_object`; `harness_guided` may add downstream-only guidance, but both lanes currently failing identically means we do not yet have evidence that recursion depth or the trusted verifier is the limiting factor. This also means stacked Slice 2 PR #228 must not be treated as epic-ready merely because its own focused/static checks advance: #223 explicitly orders the real benchmark first. No branch mutation performed here because #225/#228 are already active owned transactions.

lost-rob0t commented

2026-08-26 21:05:22 +00:00

(Migrated from github.com)

RAGE follow-up on Slice 1 / PR #225 after inspecting the exact REAL OpenRouter job and current planner/provider code.

The earlier output-channel hypothesis is now partially disproven. rlm_completion:real_response_plan_input/3 already tries nonempty text first, then nonempty reasoning, then structured response fallback. On the same live workflow, ordinary plan generation succeeds from text, while the structured-repair suite succeeds from reasoning. So this is not simply “planner ignores reasoning output.”

The remaining live evidence points at planner termination/budget observability as the next falsifiable boundary:

the CSP benchmark explicitly sets planner_attempts(2) and planner_max_tokens(1800);
five of six failed cases report exactly completion_tokens:3600 across the two failed attempts; the sixth reports 4013;
every attempt still reaches no_json_object before any plan operation executes;
normalized provider responses already preserve finish_reason, but planner_parse_result/… drops provider-summary/finish information from the final parse error, so the benchmark cannot currently distinguish stop from length/truncation or otherwise prove why the JSON object never appeared.

Decision remains HOLD, but the next experiment should not change CSP semantics or inject a plan. Before changing prompting/runtime policy, capture the planner attempt's output channel + finish_reason (and only bounded/safe diagnostics, never raw private reasoning) in trusted trace/error evidence. Then test the specific hypothesis that the planner response is exhausting generation/reasoning budget before emitting the typed JSON plan. If that hypothesis is confirmed, evaluate the existing host-owned planner_reasoning_effort/1 / planner token-budget policy through normal provider capability semantics rather than adding CSP-specific core coaching.

Security/non-goals unchanged: do not log raw reasoning, do not weaken plan_parse, do not move harness guidance into core, and do not treat Slice 2 #228 as epic-ready while Slice 1 is still red.

RAGE follow-up on Slice 1 / PR #225 after inspecting the exact REAL OpenRouter job and current planner/provider code. The earlier output-channel hypothesis is now **partially disproven**. `rlm_completion:real_response_plan_input/3` already tries nonempty `text` first, then nonempty `reasoning`, then structured response fallback. On the same live workflow, ordinary plan generation succeeds from `text`, while the structured-repair suite succeeds from `reasoning`. So this is not simply “planner ignores reasoning output.” The remaining live evidence points at **planner termination/budget observability** as the next falsifiable boundary: - the CSP benchmark explicitly sets `planner_attempts(2)` and `planner_max_tokens(1800)`; - five of six failed cases report exactly `completion_tokens:3600` across the two failed attempts; the sixth reports 4013; - every attempt still reaches `no_json_object` before any plan operation executes; - normalized provider responses already preserve `finish_reason`, but `planner_parse_result/…` drops provider-summary/finish information from the final parse error, so the benchmark cannot currently distinguish `stop` from `length`/truncation or otherwise prove why the JSON object never appeared. Decision remains **HOLD**, but the next experiment should not change CSP semantics or inject a plan. Before changing prompting/runtime policy, capture the planner attempt's output channel + `finish_reason` (and only bounded/safe diagnostics, never raw private reasoning) in trusted trace/error evidence. Then test the specific hypothesis that the planner response is exhausting generation/reasoning budget before emitting the typed JSON plan. If that hypothesis is confirmed, evaluate the existing host-owned `planner_reasoning_effort/1` / planner token-budget policy through normal provider capability semantics rather than adding CSP-specific core coaching. Security/non-goals unchanged: do not log raw reasoning, do not weaken `plan_parse`, do not move harness guidance into core, and do not treat Slice 2 #228 as epic-ready while Slice 1 is still red.

lost-rob0t commented

2026-08-27 05:33:44 +00:00

(Migrated from github.com)

RAGE exact-head recheck for recovery PR #264 at 5b7891f575a5d7b181d597344478fc7058f96e90 on canonical main 2e1264d80d02fecfb9f946e1328caaf1053e7a3b.

Current evidence is still a genuine live behavioral HOLD, not an infra/deterministic failure: deterministic CI is green, and the same credential-backed jobs pass the ordinary OpenRouter core/repair/integration suites before failing only at Run REAL depth 0/1/2 recursion experiment. The recovery benchmark still runs the six core_minimal|harness_guided × depth 0|1|2 cases with planner_attempts(2) and planner_max_tokens(1800), no fixed plan, and trusted verification; the current live failures still occur before verification.

Adversarial check of the recovery sequence: the recent commits after the recovery primarily expose/validate the six-lane report (suite, lane presence, requested/model-selected depth assertions). That is good anti-false-green instrumentation, but it does not itself repair the planner failure. Do not interpret stricter report assertions as progress on the underlying live behavior.

Decision remains HOLD for Slice 1. The next useful generic-runtime experiment is still bounded planner-attempt diagnostics at the provider boundary: preserve output channel + normalized finish_reason (and safe token/attempt metadata) in trusted error/trace evidence, without logging raw reasoning. That makes the current no_json_object failure falsifiable as truncation/budget termination vs normal-stop malformed output before changing prompt semantics or token/reasoning policy. Do not weaken the CSP, parser, verifier, depth contract, or inject a plan. Stacked #228 remains non-promotable until Slice 1 has real live evidence.

RAGE exact-head recheck for recovery PR #264 at `5b7891f575a5d7b181d597344478fc7058f96e90` on canonical `main` `2e1264d80d02fecfb9f946e1328caaf1053e7a3b`. Current evidence is still a genuine live behavioral HOLD, not an infra/deterministic failure: deterministic CI is green, and the same credential-backed jobs pass the ordinary OpenRouter core/repair/integration suites before failing only at `Run REAL depth 0/1/2 recursion experiment`. The recovery benchmark still runs the six `core_minimal|harness_guided × depth 0|1|2` cases with `planner_attempts(2)` and `planner_max_tokens(1800)`, no fixed plan, and trusted verification; the current live failures still occur before verification. Adversarial check of the recovery sequence: the recent commits after the recovery primarily expose/validate the six-lane report (`suite`, lane presence, requested/model-selected depth assertions). That is good anti-false-green instrumentation, but it does not itself repair the planner failure. Do not interpret stricter report assertions as progress on the underlying live behavior. Decision remains HOLD for Slice 1. The next useful generic-runtime experiment is still bounded planner-attempt diagnostics at the provider boundary: preserve output channel + normalized `finish_reason` (and safe token/attempt metadata) in trusted error/trace evidence, without logging raw reasoning. That makes the current `no_json_object` failure falsifiable as truncation/budget termination vs normal-stop malformed output before changing prompt semantics or token/reasoning policy. Do not weaken the CSP, parser, verifier, depth contract, or inject a plan. Stacked #228 remains non-promotable until Slice 1 has real live evidence.

lost-rob0t commented

2026-08-27 11:11:18 +00:00

(Migrated from github.com)

RAGE Slice 2 recovery design — PR #228 after Slice 1 landed

Slice 1 is now on canonical main; Slice 2 can be evaluated on its own merits, but PR #228 is not safe to recover/merge as-is. Re-reading exact #228 head f87d98afe3849e50ed047a6e5bbbf8071c2cacda confirms the previously recorded adversarial findings are real code-level defects, not theoretical concerns.

Deterministic defects to pin TDD-first

Cross-store cache leakage. persistent_mount_cache/3 is keyed only by mount key + artifact version. The artifact store identity is absent. Two simultaneously open stores can therefore each have (scope,name,version=1) with different source descriptors and the second store may reuse the first store's live context handle. The artifact API already gives a closed host-owned store identity term (artifact_store(memory, Id) or artifact_store(persist, File)), so the mount cache can be partitioned by that identity without exposing content or credentials.
Resolve race can invalidate a returned winner. ensure_persistent_context/4 checks the cache outside the mutex, creates a fresh context, then replace_persistent_cache/3 retracts/deletes all old entries under the mutex. Two concurrent misses for the same store/key/version can each create a handle; the later installer can delete the earlier caller's already-returned handle.
Concurrent identical mount is not actually idempotent. persistent_publish_or_reuse/4 performs artifact_latest and conditional artifact_put as separate mount-level operations. Two callers can observe the same pre-state and both append identical immutable versions.

Smallest recovery design

TDD first on the recovered Slice 2 transaction, using deterministic synchronization/barriers rather than sleeps:

two-store regression: same scope/name/version, different source bytes, both stores resolve their own bytes and never share a live handle;
concurrent-resolve regression: two same-key/version resolves converge on one live cached winner and every returned handle remains usable;
concurrent-identical-mount regression: two simultaneous identical persistent mounts both return the same artifact ref/version and history contains one publication for that source/policy state.

Realization boundary:

key the process-local persistent cache by artifact-store identity + mount key + artifact version;
install rehydrated contexts with first-live-winner double-check semantics: create candidate, lock, re-check for a live winner, keep winner or install candidate; only the losing candidate is deleted;
serialize the mount-level latest -> reusable? -> conditional put transaction for a given store/mount identity so idempotency is true under concurrency. Do not mutate rlm_artifact semantics merely to make #228 pass;
keep visibility/authority unchanged: persistence remains opaque by default and cache identity never grants execution or prompt visibility.

Adversarial decision

GO for TDD-first recovery; HOLD realization/promotion until those three regressions fail before implementation and pass on a fresh changed head. Do not solve this by global reset, sleeps, accepting duplicate versions, or broadening prompt visibility.

PR #228 is still a draft stacked on obsolete feature/223-real-constraint-benchmark, exact head f87d98af...; recover the existing transaction onto current main rather than opening a replacement, preserving only the owned Slice 2 diff plus the new regressions.

## RAGE Slice 2 recovery design — PR #228 after Slice 1 landed Slice 1 is now on canonical `main`; Slice 2 can be evaluated on its own merits, but **PR #228 is not safe to recover/merge as-is**. Re-reading exact #228 head `f87d98afe3849e50ed047a6e5bbbf8071c2cacda` confirms the previously recorded adversarial findings are real code-level defects, not theoretical concerns. ### Deterministic defects to pin TDD-first 1. **Cross-store cache leakage.** `persistent_mount_cache/3` is keyed only by mount key + artifact version. The artifact store identity is absent. Two simultaneously open stores can therefore each have `(scope,name,version=1)` with different source descriptors and the second store may reuse the first store's live context handle. The artifact API already gives a closed host-owned store identity term (`artifact_store(memory, Id)` or `artifact_store(persist, File)`), so the mount cache can be partitioned by that identity without exposing content or credentials. 2. **Resolve race can invalidate a returned winner.** `ensure_persistent_context/4` checks the cache outside the mutex, creates a fresh context, then `replace_persistent_cache/3` retracts/deletes all old entries under the mutex. Two concurrent misses for the same store/key/version can each create a handle; the later installer can delete the earlier caller's already-returned handle. 3. **Concurrent identical mount is not actually idempotent.** `persistent_publish_or_reuse/4` performs `artifact_latest` and conditional `artifact_put` as separate mount-level operations. Two callers can observe the same pre-state and both append identical immutable versions. ### Smallest recovery design TDD first on the recovered Slice 2 transaction, using deterministic synchronization/barriers rather than sleeps: - two-store regression: same scope/name/version, different source bytes, both stores resolve their own bytes and never share a live handle; - concurrent-resolve regression: two same-key/version resolves converge on one live cached winner and every returned handle remains usable; - concurrent-identical-mount regression: two simultaneous identical persistent mounts both return the same artifact ref/version and history contains one publication for that source/policy state. Realization boundary: - key the process-local persistent cache by **artifact-store identity + mount key + artifact version**; - install rehydrated contexts with **first-live-winner double-check semantics**: create candidate, lock, re-check for a live winner, keep winner or install candidate; only the losing candidate is deleted; - serialize the mount-level `latest -> reusable? -> conditional put` transaction for a given store/mount identity so idempotency is true under concurrency. Do not mutate `rlm_artifact` semantics merely to make #228 pass; - keep visibility/authority unchanged: persistence remains opaque by default and cache identity never grants execution or prompt visibility. ### Adversarial decision **GO for TDD-first recovery; HOLD realization/promotion until those three regressions fail before implementation and pass on a fresh changed head.** Do not solve this by global reset, sleeps, accepting duplicate versions, or broadening prompt visibility. PR #228 is still a draft stacked on obsolete `feature/223-real-constraint-benchmark`, exact head `f87d98af...`; recover the existing transaction onto current `main` rather than opening a replacement, preserving only the owned Slice 2 diff plus the new regressions.

lost-rob0t commented

2026-08-27 12:29:40 +00:00

(Migrated from github.com)

RAGE Slice 2 recovery — corrected bug boundary

Current canonical main remains 7f0050f524db6de5293163241176ac4660f04349. Recovery PR #270 is draft on top of that immutable base.

Deterministic evidence

5b252c735e6ae0bcc7c20e5ad57cccdb879102e8: canonical PlUnit discovered 991 tests, passed 990, with exactly one failure: persistent_mount_cache_is_partitioned_by_artifact_store.
ab942104d71bc857c2f36eb4e6da4ee4dbcb59a3: first realization partitions persistent_mount_cache by artifact store, but the exact same 991/990/1 failure remains. Therefore store-qualified cache identity is necessary design work but not sufficient, and that realization is not accepted as the root fix.
Historical exact head f87d98afe3849e50ed047a6e5bbbf8071c2cacda from stale #228 already failed three basic persistent-mount tests (defaults_opaque, prompt_visible, idempotent_versions_changes_and_tombstones) before the concurrency hardening was added.

Root cause found by source reconciliation

rlm_artifact:canonical_value/2 intentionally canonicalizes persisted dicts by rebuilding them with tag artifact_data. A normalized mount source such as context_source{kind:text,value:...} is therefore stored/retrieved as artifact_data{kind:text,value:...}.

rlm_context_mount:register_descriptor/3 and prompt_source_text/2 currently exact-match the transient context_source{...} tag. Persistent mount realization calls register_descriptor(Artifact.value.source, ...), so a valid canonical persisted descriptor has no matching clause and the private goal fails. mount_outcome/3 catches exceptions but not ordinary goal failure, so the public mount call can fail as a predicate instead of returning a structured error(...).

Adversarial decision

RESTART BUG/TDD/ANALYZE before concurrency realization. Do not change artifact canonicalization, weaken persistence, or paper over the failing mount path. First pin a smallest single-store public regression proving a canonical persisted source rehydrates into a valid live context. Then make mount-source decoding tag-agnostic but shape/kind validated at the trusted host boundary. Only after that base contract is green should the store-qualified cache fix be re-evaluated and the two deterministic concurrency races be realized.

Authority/visibility invariants remain unchanged: persisted data stays closed; adapters stay host-registered/trusted; persistent defaults to opaque; no model-controlled executable callback is introduced.

## RAGE Slice 2 recovery — corrected bug boundary Current canonical `main` remains `7f0050f524db6de5293163241176ac4660f04349`. Recovery PR #270 is draft on top of that immutable base. ### Deterministic evidence - `5b252c735e6ae0bcc7c20e5ad57cccdb879102e8`: canonical PlUnit discovered 991 tests, passed 990, with exactly one failure: `persistent_mount_cache_is_partitioned_by_artifact_store`. - `ab942104d71bc857c2f36eb4e6da4ee4dbcb59a3`: first realization partitions `persistent_mount_cache` by artifact store, but the exact same 991/990/1 failure remains. Therefore store-qualified cache identity is necessary design work but **not sufficient**, and that realization is not accepted as the root fix. - Historical exact head `f87d98afe3849e50ed047a6e5bbbf8071c2cacda` from stale #228 already failed three *basic* persistent-mount tests (`defaults_opaque`, `prompt_visible`, `idempotent_versions_changes_and_tombstones`) before the concurrency hardening was added. ### Root cause found by source reconciliation `rlm_artifact:canonical_value/2` intentionally canonicalizes persisted dicts by rebuilding them with tag `artifact_data`. A normalized mount source such as `context_source{kind:text,value:...}` is therefore stored/retrieved as `artifact_data{kind:text,value:...}`. `rlm_context_mount:register_descriptor/3` and `prompt_source_text/2` currently exact-match the transient `context_source{...}` tag. Persistent mount realization calls `register_descriptor(Artifact.value.source, ...)`, so a valid canonical persisted descriptor has no matching clause and the private goal fails. `mount_outcome/3` catches exceptions but not ordinary goal failure, so the public mount call can fail as a predicate instead of returning a structured `error(...)`. ### Adversarial decision **RESTART BUG/TDD/ANALYZE before concurrency realization.** Do not change artifact canonicalization, weaken persistence, or paper over the failing mount path. First pin a smallest single-store public regression proving a canonical persisted source rehydrates into a valid live context. Then make mount-source decoding tag-agnostic but shape/kind validated at the trusted host boundary. Only after that base contract is green should the store-qualified cache fix be re-evaluated and the two deterministic concurrency races be realized. Authority/visibility invariants remain unchanged: persisted data stays closed; adapters stay host-registered/trusted; persistent defaults to opaque; no model-controlled executable callback is introduced.

lost-rob0t commented

2026-08-27 12:49:02 +00:00

(Migrated from github.com)

RAGE Slice 2 — restarted TDD boundary on canonical persisted source rehydration

Current canonical main is still 7f0050f524db6de5293163241176ac4660f04349. Recovery PR #270 was re-read immediately before the write and is still the active draft transaction.

The previous store-qualified cache realization is preserved as necessary but unaccepted: exact head f9e9d959f46666d070abf77521f931b3ea1b4211 ran the canonical deterministic gate with 991 discovered / 990 passed / exactly one failure, persistent_mount_cache_is_partitioned_by_artifact_store; REAL OpenRouter, Paid OpenRouter, Nix, clean pack and Tree-sitter were green on that same candidate. Source reconciliation shows the remaining base failure occurs earlier than cache reuse: rlm_artifact canonicalizes persisted dict tags to artifact_data, while mount rehydration still exact-matches transient context_source{...} tags.

TDD restart commit: d5a9f0a521612eb50da14bf8d7932a0e54bcd66e (test: pin canonical persisted mount rehydration). It adds the smallest single-store public contract: a persistent text mount must successfully rehydrate its canonical persisted descriptor into a live handle whose slice returns the exact bytes. No production code changed in this commit; the existing cross-store test remains intact.

Decision: HOLD realization until canonical CI observes this falsifiable contract on d5a9f0a.... Once red is confirmed, the approved smallest repair is tag-agnostic but exact-shape/kind-validated mount-source decoding at the trusted host boundary; do not change artifact canonicalization, visibility, authority, or serialize executable adapter callbacks. Only after the base rehydration contract is green should the two deterministic concurrency races be TDD-pinned/realized.

## RAGE Slice 2 — restarted TDD boundary on canonical persisted source rehydration Current canonical `main` is still `7f0050f524db6de5293163241176ac4660f04349`. Recovery PR #270 was re-read immediately before the write and is still the active draft transaction. The previous store-qualified cache realization is preserved as necessary but unaccepted: exact head `f9e9d959f46666d070abf77521f931b3ea1b4211` ran the canonical deterministic gate with 991 discovered / 990 passed / exactly one failure, `persistent_mount_cache_is_partitioned_by_artifact_store`; REAL OpenRouter, Paid OpenRouter, Nix, clean pack and Tree-sitter were green on that same candidate. Source reconciliation shows the remaining base failure occurs earlier than cache reuse: `rlm_artifact` canonicalizes persisted dict tags to `artifact_data`, while mount rehydration still exact-matches transient `context_source{...}` tags. TDD restart commit: `d5a9f0a521612eb50da14bf8d7932a0e54bcd66e` (`test: pin canonical persisted mount rehydration`). It adds the smallest single-store public contract: a persistent text mount must successfully rehydrate its canonical persisted descriptor into a live handle whose slice returns the exact bytes. No production code changed in this commit; the existing cross-store test remains intact. Decision: HOLD realization until canonical CI observes this falsifiable contract on `d5a9f0a...`. Once red is confirmed, the approved smallest repair is tag-agnostic but exact-shape/kind-validated mount-source decoding at the trusted host boundary; do not change artifact canonicalization, visibility, authority, or serialize executable adapter callbacks. Only after the base rehydration contract is green should the two deterministic concurrency races be TDD-pinned/realized.

lost-rob0t commented

2026-08-27 12:51:51 +00:00

(Migrated from github.com)

TDD red confirmed on immutable `d5a9f0a521612eb50da14bf8d7932a0e54bcd66e`

Canonical CI discovered 992 tests: 990 passed, exactly 2 failed, zero timeout/blocked/fixme. The only failures are:

persistent_mount_rehydrates_canonical_persisted_text_source — the new smallest single-store public contract;
persistent_mount_cache_is_partitioned_by_artifact_store — the existing two-store contract.

This is the required falsification before production repair. It confirms the cross-store failure is currently dominated by the earlier persisted-descriptor rehydration bug; cache partitioning alone cannot make the public mount path work.

Decision gate: GO for the already-approved minimal production repair: decode persisted mount-source dicts independent of transient dict tag while requiring the exact allowed source shape/kind at the trusted host boundary. Keep artifact canonicalization, opaque visibility, adapter trust, and authority semantics unchanged. After that changed head proves both base tests green, resume the separately approved concurrency TDD (first-live-winner resolve + serialized identical publish/reuse).

### TDD red confirmed on immutable `d5a9f0a521612eb50da14bf8d7932a0e54bcd66e` Canonical CI discovered **992 tests: 990 passed, exactly 2 failed, zero timeout/blocked/fixme**. The only failures are: 1. `persistent_mount_rehydrates_canonical_persisted_text_source` — the new smallest single-store public contract; 2. `persistent_mount_cache_is_partitioned_by_artifact_store` — the existing two-store contract. This is the required falsification before production repair. It confirms the cross-store failure is currently dominated by the earlier persisted-descriptor rehydration bug; cache partitioning alone cannot make the public mount path work. Decision gate: **GO** for the already-approved minimal production repair: decode persisted mount-source dicts independent of transient dict tag while requiring the exact allowed source shape/kind at the trusted host boundary. Keep artifact canonicalization, opaque visibility, adapter trust, and authority semantics unchanged. After that changed head proves both base tests green, resume the separately approved concurrency TDD (first-live-winner resolve + serialized identical publish/reuse).

lost-rob0t commented

2026-08-27 13:12:30 +00:00

(Migrated from github.com)

RAGE Slice 2 durable-context recovery update.

Exact red TDD head: d5a9f0a521612eb50da14bf8d7932a0e54bcd66e on canonical base 7f0050f524db6de5293163241176ac4660f04349.

Deterministic evidence is cleanly falsifiable: 992 tests discovered/completed, 990 passed, exactly 2 failed, zero timeout/blocked/fixme. The only failures are persistent_mount_rehydrates_canonical_persisted_text_source and persistent_mount_cache_is_partitioned_by_artifact_store. On that same head REAL OpenRouter, Paid OpenRouter, Nix, clean SWI pack, and Tree-sitter all pass.

Root cause confirmed: rlm_artifact canonicalizes nested dict tags during persistence, so a transient context_source{...} returns with a canonical artifact-data tag. The mount layer was incorrectly treating that transient dict tag as persistent wire semantics and passed Artifact.value.source directly to exact-tag register_descriptor/3.

Design/adversarial decision: GO for a mount-local decoder only. Persisted source decoding is tag-agnostic but exact-key-shape/kind validated; it reconstructs the canonical transient context_source descriptor before context registration or prompt projection. Unknown/extra shapes fail closed with invalid_persisted_source_shape. No artifact canonicalization change, authority/capability widening, prompt visibility change, serialized callback, or second persistence plane.

Realization: 8e144cef0ce45e125b1c4c8177fe9ee729d81930 (fix: decode canonical persisted mount sources). Adversarial diff audit against the TDD head is one production file, +38/-6, limited to persisted-source decode + its two consumers; the adapter prompt pattern is also corrected to retain its required source_ref field.

Decision: HOLD PR #270 at this changed exact head until fresh repository-native verification proves the two red contracts green. Do not reuse d5a9f0a... evidence. The two separately identified concurrency races remain next and must still be pinned with deterministic synchronization before their production repairs.

RAGE Slice 2 durable-context recovery update. Exact red TDD head: `d5a9f0a521612eb50da14bf8d7932a0e54bcd66e` on canonical base `7f0050f524db6de5293163241176ac4660f04349`. Deterministic evidence is cleanly falsifiable: 992 tests discovered/completed, 990 passed, exactly 2 failed, zero timeout/blocked/fixme. The only failures are `persistent_mount_rehydrates_canonical_persisted_text_source` and `persistent_mount_cache_is_partitioned_by_artifact_store`. On that same head REAL OpenRouter, Paid OpenRouter, Nix, clean SWI pack, and Tree-sitter all pass. Root cause confirmed: `rlm_artifact` canonicalizes nested dict tags during persistence, so a transient `context_source{...}` returns with a canonical artifact-data tag. The mount layer was incorrectly treating that transient dict tag as persistent wire semantics and passed `Artifact.value.source` directly to exact-tag `register_descriptor/3`. Design/adversarial decision: GO for a mount-local decoder only. Persisted source decoding is tag-agnostic but exact-key-shape/kind validated; it reconstructs the canonical transient `context_source` descriptor before context registration or prompt projection. Unknown/extra shapes fail closed with `invalid_persisted_source_shape`. No artifact canonicalization change, authority/capability widening, prompt visibility change, serialized callback, or second persistence plane. Realization: `8e144cef0ce45e125b1c4c8177fe9ee729d81930` (`fix: decode canonical persisted mount sources`). Adversarial diff audit against the TDD head is one production file, +38/-6, limited to persisted-source decode + its two consumers; the adapter prompt pattern is also corrected to retain its required `source_ref` field. Decision: HOLD PR #270 at this changed exact head until fresh repository-native verification proves the two red contracts green. Do not reuse `d5a9f0a...` evidence. The two separately identified concurrency races remain next and must still be pinned with deterministic synchronization before their production repairs.

lost-rob0t commented

2026-08-27 13:35:24 +00:00

(Migrated from github.com)

RAGE Slice 2 recovery update on PR #270.

BUG/TDD evidence is now exact on immutable pre-fix head 73007eb9b4ba8f5d1b688e654a6aaf28cb13ac6e: canonical deterministic CI discovered 993 tests, passed 990, failed exactly 3, with zero timeout/blocked/fixme. The first failure is the new contract public_projection_binds_artifact_value_before_field_access, raising an instantiation error. The two persisted-mount contracts (persistent_mount_rehydrates_canonical_persisted_text_source and cross-store cache partitioning) then fail downstream through that same projection path, so they are not valid evidence against the persisted-source decoder/cache-key repair yet.

Analysis/design: public_from_artifact/2 placed Value.name/etc. in the output dict in the predicate head, while Value = Artifact.value occurred only in the body. SWI therefore evaluates dict field access before Value is bound. The smallest repair is ordering-only: bind Value = Artifact.value first, then construct the same public context_mount{...} dict. No persistence representation, visibility, scope, authority, source content, or artifact semantics change.

Adversarial decision: GO for that exact one-predicate ordering repair; do not broaden the change or treat the two dominated mount failures as separate implementation defects yet.

Realization: commit a4c8be71cee38bbd6a48ef38933775af3d227461 (fix: bind artifact value before mount projection). Adversarial diff review shows exactly one file and one predicate changed, preserving every projected field.

Verification gate: HOLD until this changed exact head earns fresh deterministic evidence. If the three failures clear, resume the already-approved bug-first concurrency sequence: deterministic concurrent same-key/version resolve winner/liveness, then concurrent identical mount idempotence; no sleep-based races and no artifact-layer semantic rewrite.

RAGE Slice 2 recovery update on PR #270. BUG/TDD evidence is now exact on immutable pre-fix head `73007eb9b4ba8f5d1b688e654a6aaf28cb13ac6e`: canonical deterministic CI discovered 993 tests, passed 990, failed exactly 3, with zero timeout/blocked/fixme. The first failure is the new contract `public_projection_binds_artifact_value_before_field_access`, raising an instantiation error. The two persisted-mount contracts (`persistent_mount_rehydrates_canonical_persisted_text_source` and cross-store cache partitioning) then fail downstream through that same projection path, so they are not valid evidence against the persisted-source decoder/cache-key repair yet. Analysis/design: `public_from_artifact/2` placed `Value.name`/etc. in the output dict in the predicate head, while `Value = Artifact.value` occurred only in the body. SWI therefore evaluates dict field access before `Value` is bound. The smallest repair is ordering-only: bind `Value = Artifact.value` first, then construct the same public `context_mount{...}` dict. No persistence representation, visibility, scope, authority, source content, or artifact semantics change. Adversarial decision: GO for that exact one-predicate ordering repair; do not broaden the change or treat the two dominated mount failures as separate implementation defects yet. Realization: commit `a4c8be71cee38bbd6a48ef38933775af3d227461` (`fix: bind artifact value before mount projection`). Adversarial diff review shows exactly one file and one predicate changed, preserving every projected field. Verification gate: HOLD until this changed exact head earns fresh deterministic evidence. If the three failures clear, resume the already-approved bug-first concurrency sequence: deterministic concurrent same-key/version resolve winner/liveness, then concurrent identical mount idempotence; no sleep-based races and no artifact-layer semantic rewrite.