nsaspy/prolog-rlm

Fork 0

[P1] Route canonical effectful execution through the #57 effect boundary #79

New issue

Open

opened 2026-08-17 19:01:33 +00:00 by lost-rob0t · 3 comments

lost-rob0t commented

2026-08-17 19:01:33 +00:00

(Migrated from github.com)

Parent / blocker: #57

Goal

Adopt the durable effect-identity / observation substrate from #78 across the canonical effectful execution paths in prolog-rlm, without creating another ledger and without breaking the existing #53 authority or #54 async architecture.

This issue remains open because the generic substrate and the canonical tool-path slice are merged, but provider, MCP, lifecycle/process, and external integration adoption are not complete.

Current state — 2026-08-19

Canonical main now includes:

#78 — generic durable effect identity / observation substrate;
#83 — identity, adapter, retention, crash, and recovery hardening;
#85 — explicit pre-v2 migration;
#86 — merged tool-path Slice 1, routing canonical effectful rlm_tool execution through the #57 prepared-ticket boundary before handler mutation.

The #86 slice preserves exact Ticket-A authority/admission semantics, stable trusted executor identity, fresh-read behavior, structured preparation errors, conservative replay/uncertainty behavior, and crash/adversarial coverage. Its reconstructed exact head passed the normal CI and Tree-sitter workflows before merge.

Remaining #79 work:

model/provider requests;
effectful MCP requests;
MCP install/run/start/stop/restart and process-backed lifecycle effects where applicable;
the documented external process/network/repository-mutation adapter contract for #49/#50 and downstream tool packs.

Required invariant

For every canonical externally effectful operation in scope for durable identity:

normalize executable operation
-> capability / schema / hard policy
-> exact #53 authority decision
-> #57 durable attempt admission
-> #57 durable dispatch intent
-> external boundary
-> authoritative observation OR conservative uncertainty

A sync wrapper, async wrapper, Prolog backtracking, retry wrapper, graph/agent wrapper, or fresh-process restart must not create another implicit submission for the same admitted attempt.

Pure/read-only operations do not need indiscriminate memoization. Fresh-read policy remains explicit.

Architecture constraints

Preserve #54

Keep one canonical execution direction. Do not introduce a second sync implementation. Internal code already running inside a bounded async worker must call the trusted execute ABI directly rather than nesting Future waits.

Preserve #53

Capability/schema/hard-policy validation remains before authority. Authority remains host-controlled. Changed executable payload, edit, retry, or resample must receive identity and authority appropriate to the new normalized operation.

allow_once must not leak to retry/resample. dangerous must not bypass effect identity, confinement, accounting, or persistence. Do not build authority_v2.

Do not create another effect ledger

Provider/tool/MCP-specific code should implement static code-owned adapters or an equivalent narrow integration over the #57 executor. Domain job IDs and provider/MCP request IDs are provenance/reconciliation inputs, not a second generic once-only system.

Conservative remote uncertainty

Transport failure after durable dispatch is not proof the effect did not happen. If a provider/protocol has no reconciliation API, retain the admitted attempt as indeterminate rather than automatically resubmitting it.

Implementation slices

Canonical tool path — DONE in #86
- effectful rlm_tool execution crosses #57 after normalization/preflight and before mutation;
- exact prepared tickets cross authority and admission unchanged;
- edited pending proposals prepare a new ticket;
- trusted executor identity is stable across ephemeral registry allocation;
- fresh reads remain fresh;
- crash/replay/adversarial tests prove no duplicate external mutation for the covered tool path.
Model/provider path
- route paid/stateful provider calls through a code-owned effect adapter where appropriate;
- define stable request identity and provider idempotency/reconciliation behavior explicitly;
- preserve chain retry semantics as explicit retry/resample rather than accidental replay;
- do not claim provider-level exactly-once when the provider cannot supply it.
MCP request + lifecycle path
- classify effectful MCP commands/lifecycle actions versus pure/read-only operations;
- route effectful transport exchanges, install, run/start, stop/restart, and process-backed lifecycle mutations through #57;
- retain safe MCP request/operation IDs and endpoint/profile identity as reconciliation/provenance inputs.
External process/network/repository-mutation integration contract
- document and enforce one code-owned #57 adapter path for external tool libraries;
- concrete standard tool-pack implementation remains owned by #49/#50 as appropriate.

Required tests for remaining slices

Tests must use externally meaningful counters/state, not only ledger internals.

canonical fake provider crash after externally observable acceptance and before local observation does not submit again after fresh-process restart;
provider transport exception after dispatch becomes reconciliation-required/indeterminate rather than an automatic second request;
explicit provider retry/resample has stable linked new-attempt identity and correct fresh-authority behavior;
canonical MCP mutating request does not duplicate under wrapper/backtracking/restart;
MCP install/run/process fixture records at most one external mutation for one admitted attempt;
cancellation after dispatch never proves the remote effect did not occur;
pure/read-only paths remain usable without accidental permanent memoization;
#44/#45 accounting/trajectory work can consume authoritative attempt/observation lineage without reconstructing effect history from lexical variables.

Concurrency tests must use deterministic synchronization, not sleeps.

Acceptance criteria

rlm_tool effectful execution no longer jumps directly from authority to perform_tool_effect without #57 admission. (#86)
Canonical model/provider requests that may incur paid/stateful external work are routed through #57 where applicable.
Canonical effectful MCP requests are routed through #57.
MCP install/run/stop/restart and process-backed lifecycle mutations use the same effect-identity boundary where applicable.
External process/network/repository-mutation tool integrations have a documented code-owned #57 adapter path.
#53 exact authority semantics remain intact for changed payload, retry, resample, approve/deny/edit, allow_once, allow_session, and dangerous across all adopted paths.
#54 canonical execute -> Future -> sync-await direction remains intact; no nested-wait regression.
Backtracking, repeated waits, callbacks, and wrapper reconstruction do not resubmit the same admitted attempt.
Crash-after-remote-acceptance fixtures prove fresh-process recovery without automatic resubmission at canonical integration boundaries.
Non-reconcilable uncertain effects remain indeterminate until trusted policy resolves them.
No documentation claims generic exactly-once execution.
Deterministic CI is green for the completed adoption set.
Configured REAL provider gates are green without deliberately duplicating paid requests for destructive crash testing.

Non-goals

No competing generic effect ledger.
No rewrite of #53 authority.
No duplicate async scheduler.
No requirement to memoize pure reads forever.
No requirement for #57 itself to finish all #44/#45 accounting/trajectory presentation work.

References

#57 effect identity / once-only observation semantics
#78 generic substrate
#83 hardening
#85 migration
#86 merged canonical tool-path slice
#44 provider usage on failure
#45 trajectory fidelity
#53 authority boundary
#54 canonical async contract
#49/#50 external concrete tool work

Parent / blocker: #57 ## Goal Adopt the durable effect-identity / observation substrate from #78 across the **canonical effectful execution paths** in `prolog-rlm`, without creating another ledger and without breaking the existing #53 authority or #54 async architecture. This issue remains open because the generic substrate and the canonical tool-path slice are merged, but provider, MCP, lifecycle/process, and external integration adoption are not complete. ## Current state — 2026-08-19 Canonical `main` now includes: - #78 — generic durable effect identity / observation substrate; - #83 — identity, adapter, retention, crash, and recovery hardening; - #85 — explicit pre-v2 migration; - #86 — **merged tool-path Slice 1**, routing canonical effectful `rlm_tool` execution through the #57 prepared-ticket boundary before handler mutation. The #86 slice preserves exact Ticket-A authority/admission semantics, stable trusted executor identity, fresh-read behavior, structured preparation errors, conservative replay/uncertainty behavior, and crash/adversarial coverage. Its reconstructed exact head passed the normal CI and Tree-sitter workflows before merge. Remaining #79 work: 1. model/provider requests; 2. effectful MCP requests; 3. MCP install/run/start/stop/restart and process-backed lifecycle effects where applicable; 4. the documented external process/network/repository-mutation adapter contract for #49/#50 and downstream tool packs. ## Required invariant For every canonical externally effectful operation in scope for durable identity: ```text normalize executable operation -> capability / schema / hard policy -> exact #53 authority decision -> #57 durable attempt admission -> #57 durable dispatch intent -> external boundary -> authoritative observation OR conservative uncertainty ``` A sync wrapper, async wrapper, Prolog backtracking, retry wrapper, graph/agent wrapper, or fresh-process restart must not create another implicit submission for the same admitted attempt. Pure/read-only operations do not need indiscriminate memoization. Fresh-read policy remains explicit. ## Architecture constraints ### Preserve #54 Keep one canonical execution direction. Do not introduce a second sync implementation. Internal code already running inside a bounded async worker must call the trusted execute ABI directly rather than nesting Future waits. ### Preserve #53 Capability/schema/hard-policy validation remains before authority. Authority remains host-controlled. Changed executable payload, edit, retry, or resample must receive identity and authority appropriate to the new normalized operation. `allow_once` must not leak to retry/resample. `dangerous` must not bypass effect identity, confinement, accounting, or persistence. Do not build `authority_v2`. ### Do not create another effect ledger Provider/tool/MCP-specific code should implement static code-owned adapters or an equivalent narrow integration over the #57 executor. Domain job IDs and provider/MCP request IDs are provenance/reconciliation inputs, not a second generic once-only system. ### Conservative remote uncertainty Transport failure after durable dispatch is not proof the effect did not happen. If a provider/protocol has no reconciliation API, retain the admitted attempt as indeterminate rather than automatically resubmitting it. ## Implementation slices 1. **Canonical tool path — DONE in #86** - effectful `rlm_tool` execution crosses #57 after normalization/preflight and before mutation; - exact prepared tickets cross authority and admission unchanged; - edited pending proposals prepare a new ticket; - trusted executor identity is stable across ephemeral registry allocation; - fresh reads remain fresh; - crash/replay/adversarial tests prove no duplicate external mutation for the covered tool path. 2. **Model/provider path** - route paid/stateful provider calls through a code-owned effect adapter where appropriate; - define stable request identity and provider idempotency/reconciliation behavior explicitly; - preserve chain retry semantics as explicit retry/resample rather than accidental replay; - do not claim provider-level exactly-once when the provider cannot supply it. 3. **MCP request + lifecycle path** - classify effectful MCP commands/lifecycle actions versus pure/read-only operations; - route effectful transport exchanges, install, run/start, stop/restart, and process-backed lifecycle mutations through #57; - retain safe MCP request/operation IDs and endpoint/profile identity as reconciliation/provenance inputs. 4. **External process/network/repository-mutation integration contract** - document and enforce one code-owned #57 adapter path for external tool libraries; - concrete standard tool-pack implementation remains owned by #49/#50 as appropriate. ## Required tests for remaining slices Tests must use externally meaningful counters/state, not only ledger internals. - canonical fake provider crash after externally observable acceptance and before local observation does not submit again after fresh-process restart; - provider transport exception after dispatch becomes reconciliation-required/indeterminate rather than an automatic second request; - explicit provider retry/resample has stable linked new-attempt identity and correct fresh-authority behavior; - canonical MCP mutating request does not duplicate under wrapper/backtracking/restart; - MCP install/run/process fixture records at most one external mutation for one admitted attempt; - cancellation after dispatch never proves the remote effect did not occur; - pure/read-only paths remain usable without accidental permanent memoization; - #44/#45 accounting/trajectory work can consume authoritative attempt/observation lineage without reconstructing effect history from lexical variables. Concurrency tests must use deterministic synchronization, not sleeps. ## Acceptance criteria - [x] `rlm_tool` effectful execution no longer jumps directly from authority to `perform_tool_effect` without #57 admission. (#86) - [ ] Canonical model/provider requests that may incur paid/stateful external work are routed through #57 where applicable. - [ ] Canonical effectful MCP requests are routed through #57. - [ ] MCP install/run/stop/restart and process-backed lifecycle mutations use the same effect-identity boundary where applicable. - [ ] External process/network/repository-mutation tool integrations have a documented code-owned #57 adapter path. - [ ] #53 exact authority semantics remain intact for changed payload, retry, resample, approve/deny/edit, `allow_once`, `allow_session`, and `dangerous` across all adopted paths. - [ ] #54 canonical execute -> Future -> sync-await direction remains intact; no nested-wait regression. - [ ] Backtracking, repeated waits, callbacks, and wrapper reconstruction do not resubmit the same admitted attempt. - [ ] Crash-after-remote-acceptance fixtures prove fresh-process recovery without automatic resubmission at canonical integration boundaries. - [ ] Non-reconcilable uncertain effects remain indeterminate until trusted policy resolves them. - [ ] No documentation claims generic exactly-once execution. - [ ] Deterministic CI is green for the completed adoption set. - [ ] Configured REAL provider gates are green without deliberately duplicating paid requests for destructive crash testing. ## Non-goals - No competing generic effect ledger. - No rewrite of #53 authority. - No duplicate async scheduler. - No requirement to memoize pure reads forever. - No requirement for #57 itself to finish all #44/#45 accounting/trajectory presentation work. ## References - #57 effect identity / once-only observation semantics - #78 generic substrate - #83 hardening - #85 migration - #86 merged canonical tool-path slice - #44 provider usage on failure - #45 trajectory fidelity - #53 authority boundary - #54 canonical async contract - #49/#50 external concrete tool work

lost-rob0t commented

2026-08-17 19:55:49 +00:00

(Migrated from github.com)

Post-merge adversarial review of #78 found substrate blockers that should land before this canonical-adoption issue starts wiring real effectful paths:

#80 — prune/admission linearizability, adapter identity binding, abandoned retry semantics, observed reconciliation
#81 — durable store namespace/idempotency scope and in-flight store lifecycle binding

#82 is a P2 accounting/history crash-consistency follow-up and does not necessarily block the first #79 implementation slice if #44/#45 do not consume the event stream as authoritative yet.

Treat #80 and #81 as pre-adoption blockers for #79. #57 remains open.

Post-merge adversarial review of #78 found substrate blockers that should land before this canonical-adoption issue starts wiring real effectful paths: - #80 — prune/admission linearizability, adapter identity binding, abandoned retry semantics, observed reconciliation - #81 — durable store namespace/idempotency scope and in-flight store lifecycle binding #82 is a P2 accounting/history crash-consistency follow-up and does not necessarily block the first #79 implementation slice if #44/#45 do not consume the event stream as authoritative yet. Treat #80 and #81 as pre-adoption blockers for #79. #57 remains open.

lost-rob0t commented

2026-08-17 21:28:30 +00:00

(Migrated from github.com)

Predecessor hardening is complete: PR #83 merged to main as 6c60924cef7a03b9d483e4691646d4d56a13c0f3, closing #80/#81/#82.

#79 remains open and is now expected to adopt these hardened contracts:

construct adapter-bound tickets with rlm_effect_executor:effect_prepare/5 before #53 authority composition;
preserve the attached store namespace and execution epoch through dispatch/reconciliation;
treat store close/switch conflicts as fail-closed while adapter work is in flight;
rely on authoritative observations/attempts and idempotently repairable events;
never treat abandoned as retry authorization;
account for the explicit fail-closed migration requirement for non-empty pre-v2 effect ledgers.

No canonical provider/tool/MCP path was changed by #83. The full deterministic and REAL OpenRouter gates passed on its exact final head.

Predecessor hardening is complete: PR #83 merged to `main` as `6c60924cef7a03b9d483e4691646d4d56a13c0f3`, closing #80/#81/#82. #79 remains open and is now expected to adopt these hardened contracts: - construct adapter-bound tickets with `rlm_effect_executor:effect_prepare/5` before #53 authority composition; - preserve the attached store namespace and execution epoch through dispatch/reconciliation; - treat store close/switch conflicts as fail-closed while adapter work is in flight; - rely on authoritative observations/attempts and idempotently repairable events; - never treat `abandoned` as retry authorization; - account for the explicit fail-closed migration requirement for non-empty pre-v2 effect ledgers. No canonical provider/tool/MCP path was changed by #83. The full deterministic and REAL OpenRouter gates passed on its exact final head.

lost-rob0t commented

2026-08-17 23:39:22 +00:00

(Migrated from github.com)

Migration prerequisite #84 is complete via PR #85 (merge commit 88433863056cd39a1d152db2c0a3cbf1a7f43fc0).

Before #79 adopts the hardened substrate, operators with any non-empty PR #78 journal must run the explicit offline workflow documented in docs/effect-migration.md:

swipl -q -s bin/prolog-rlm.pl -- effect-store migrate \
  --source effects.db --output effects.v2.db --json

Unresolved legacy attempts require an operator-reviewed, digest-bound adapter manifest before trusted reconciliation; migration never guesses the adapter and never invokes a provider. In-place replacement requires a verified byte-exact backup. Migrated legacy admitted attempts cannot be freshly dispatched, and all post-migration external work must receive normal v2 namespace/epoch identity.

#79 remains open and is still the separate canonical provider/tool/MCP/process adoption slice. #57 remains open.

Migration prerequisite #84 is complete via PR #85 (merge commit `88433863056cd39a1d152db2c0a3cbf1a7f43fc0`). Before #79 adopts the hardened substrate, operators with any non-empty PR #78 journal must run the explicit offline workflow documented in `docs/effect-migration.md`: ```sh swipl -q -s bin/prolog-rlm.pl -- effect-store migrate \ --source effects.db --output effects.v2.db --json ``` Unresolved legacy attempts require an operator-reviewed, digest-bound adapter manifest before trusted reconciliation; migration never guesses the adapter and never invokes a provider. In-place replacement requires a verified byte-exact backup. Migrated legacy admitted attempts cannot be freshly dispatched, and all post-migration external work must receive normal v2 namespace/epoch identity. #79 remains open and is still the separate canonical provider/tool/MCP/process adoption slice. #57 remains open.