[P1] Route canonical effectful execution through the #57 effect boundary #436

New issue

Closed

opened 2026-09-10 21:18:10 +00:00 by nsaspy · 1 comment

nsaspy commented

2026-09-10 21:18:10 +00:00

Owner

Parent / blocker: #57

Goal

Adopt the durable effect-identity / observation substrate from #78 across the canonical effectful execution paths in prolog-rlm, without creating another ledger and without breaking the existing #53 authority or #54 async architecture.

This issue remains open because the generic substrate and the canonical tool-path slice are merged, but provider, MCP, lifecycle/process, and external integration adoption are not complete.

Current state — 2026-08-19

Canonical main now includes:

#78 — generic durable effect identity / observation substrate;
#83 — identity, adapter, retention, crash, and recovery hardening;
#85 — explicit pre-v2 migration;
#86 — merged tool-path Slice 1, routing canonical effectful rlm_tool execution through the #57 prepared-ticket boundary before handler mutation.

The #86 slice preserves exact Ticket-A authority/admission semantics, stable trusted executor identity, fresh-read behavior, structured preparation errors, conservative replay/uncertainty behavior, and crash/adversarial coverage. Its reconstructed exact head passed the normal CI and Tree-sitter workflows before merge.

Remaining #79 work:

model/provider requests;
effectful MCP requests;
MCP install/run/start/stop/restart and process-backed lifecycle effects where applicable;
the documented external process/network/repository-mutation adapter contract for #49/#50 and downstream tool packs.

Required invariant

For every canonical externally effectful operation in scope for durable identity:

normalize executable operation
-> capability / schema / hard policy
-> exact #53 authority decision
-> #57 durable attempt admission
-> #57 durable dispatch intent
-> external boundary
-> authoritative observation OR conservative uncertainty

A sync wrapper, async wrapper, Prolog backtracking, retry wrapper, graph/agent wrapper, or fresh-process restart must not create another implicit submission for the same admitted attempt.

Pure/read-only operations do not need indiscriminate memoization. Fresh-read policy remains explicit.

Architecture constraints

Preserve #54

Keep one canonical execution direction. Do not introduce a second sync implementation. Internal code already running inside a bounded async worker must call the trusted execute ABI directly rather than nesting Future waits.

Preserve #53

Capability/schema/hard-policy validation remains before authority. Authority remains host-controlled. Changed executable payload, edit, retry, or resample must receive identity and authority appropriate to the new normalized operation.

allow_once must not leak to retry/resample. dangerous must not bypass effect identity, confinement, accounting, or persistence. Do not build authority_v2.

Do not create another effect ledger

Provider/tool/MCP-specific code should implement static code-owned adapters or an equivalent narrow integration over the #57 executor. Domain job IDs and provider/MCP request IDs are provenance/reconciliation inputs, not a second generic once-only system.

Conservative remote uncertainty

Transport failure after durable dispatch is not proof the effect did not happen. If a provider/protocol has no reconciliation API, retain the admitted attempt as indeterminate rather than automatically resubmitting it.

Implementation slices

Canonical tool path — DONE in #86
- effectful rlm_tool execution crosses #57 after normalization/preflight and before mutation;
- exact prepared tickets cross authority and admission unchanged;
- edited pending proposals prepare a new ticket;
- trusted executor identity is stable across ephemeral registry allocation;
- fresh reads remain fresh;
- crash/replay/adversarial tests prove no duplicate external mutation for the covered tool path.
Model/provider path
- route paid/stateful provider calls through a code-owned effect adapter where appropriate;
- define stable request identity and provider idempotency/reconciliation behavior explicitly;
- preserve chain retry semantics as explicit retry/resample rather than accidental replay;
- do not claim provider-level exactly-once when the provider cannot supply it.
MCP request + lifecycle path
- classify effectful MCP commands/lifecycle actions versus pure/read-only operations;
- route effectful transport exchanges, install, run/start, stop/restart, and process-backed lifecycle mutations through #57;
- retain safe MCP request/operation IDs and endpoint/profile identity as reconciliation/provenance inputs.
External process/network/repository-mutation integration contract
- document and enforce one code-owned #57 adapter path for external tool libraries;
- concrete standard tool-pack implementation remains owned by #49/#50 as appropriate.

Required tests for remaining slices

Tests must use externally meaningful counters/state, not only ledger internals.

canonical fake provider crash after externally observable acceptance and before local observation does not submit again after fresh-process restart;
provider transport exception after dispatch becomes reconciliation-required/indeterminate rather than an automatic second request;
explicit provider retry/resample has stable linked new-attempt identity and correct fresh-authority behavior;
canonical MCP mutating request does not duplicate under wrapper/backtracking/restart;
MCP install/run/process fixture records at most one external mutation for one admitted attempt;
cancellation after dispatch never proves the remote effect did not occur;
pure/read-only paths remain usable without accidental permanent memoization;
#44/#45 accounting/trajectory work can consume authoritative attempt/observation lineage without reconstructing effect history from lexical variables.

Concurrency tests must use deterministic synchronization, not sleeps.

Acceptance criteria

rlm_tool effectful execution no longer jumps directly from authority to perform_tool_effect without #57 admission. (#86)
Canonical model/provider requests that may incur paid/stateful external work are routed through #57 where applicable.
Canonical effectful MCP requests are routed through #57.
MCP install/run/stop/restart and process-backed lifecycle mutations use the same effect-identity boundary where applicable.
External process/network/repository-mutation tool integrations have a documented code-owned #57 adapter path.
#53 exact authority semantics remain intact for changed payload, retry, resample, approve/deny/edit, allow_once, allow_session, and dangerous across all adopted paths.
#54 canonical execute -> Future -> sync-await direction remains intact; no nested-wait regression.
Backtracking, repeated waits, callbacks, and wrapper reconstruction do not resubmit the same admitted attempt.
Crash-after-remote-acceptance fixtures prove fresh-process recovery without automatic resubmission at canonical integration boundaries.
Non-reconcilable uncertain effects remain indeterminate until trusted policy resolves them.
No documentation claims generic exactly-once execution.
Deterministic CI is green for the completed adoption set.
Configured REAL provider gates are green without deliberately duplicating paid requests for destructive crash testing.

Non-goals

No competing generic effect ledger.
No rewrite of #53 authority.
No duplicate async scheduler.
No requirement to memoize pure reads forever.
No requirement for #57 itself to finish all #44/#45 accounting/trajectory presentation work.

References

#57 effect identity / once-only observation semantics
#78 generic substrate
#83 hardening
#85 migration
#86 merged canonical tool-path slice
#44 provider usage on failure
#45 trajectory fidelity
#53 authority boundary
#54 canonical async contract
#49/#50 external concrete tool work

Parent / blocker: #57 ## Goal Adopt the durable effect-identity / observation substrate from #78 across the **canonical effectful execution paths** in `prolog-rlm`, without creating another ledger and without breaking the existing #53 authority or #54 async architecture. This issue remains open because the generic substrate and the canonical tool-path slice are merged, but provider, MCP, lifecycle/process, and external integration adoption are not complete. ## Current state — 2026-08-19 Canonical `main` now includes: - #78 — generic durable effect identity / observation substrate; - #83 — identity, adapter, retention, crash, and recovery hardening; - #85 — explicit pre-v2 migration; - #86 — **merged tool-path Slice 1**, routing canonical effectful `rlm_tool` execution through the #57 prepared-ticket boundary before handler mutation. The #86 slice preserves exact Ticket-A authority/admission semantics, stable trusted executor identity, fresh-read behavior, structured preparation errors, conservative replay/uncertainty behavior, and crash/adversarial coverage. Its reconstructed exact head passed the normal CI and Tree-sitter workflows before merge. Remaining #79 work: 1. model/provider requests; 2. effectful MCP requests; 3. MCP install/run/start/stop/restart and process-backed lifecycle effects where applicable; 4. the documented external process/network/repository-mutation adapter contract for #49/#50 and downstream tool packs. ## Required invariant For every canonical externally effectful operation in scope for durable identity: ```text normalize executable operation -> capability / schema / hard policy -> exact #53 authority decision -> #57 durable attempt admission -> #57 durable dispatch intent -> external boundary -> authoritative observation OR conservative uncertainty ``` A sync wrapper, async wrapper, Prolog backtracking, retry wrapper, graph/agent wrapper, or fresh-process restart must not create another implicit submission for the same admitted attempt. Pure/read-only operations do not need indiscriminate memoization. Fresh-read policy remains explicit. ## Architecture constraints ### Preserve #54 Keep one canonical execution direction. Do not introduce a second sync implementation. Internal code already running inside a bounded async worker must call the trusted execute ABI directly rather than nesting Future waits. ### Preserve #53 Capability/schema/hard-policy validation remains before authority. Authority remains host-controlled. Changed executable payload, edit, retry, or resample must receive identity and authority appropriate to the new normalized operation. `allow_once` must not leak to retry/resample. `dangerous` must not bypass effect identity, confinement, accounting, or persistence. Do not build `authority_v2`. ### Do not create another effect ledger Provider/tool/MCP-specific code should implement static code-owned adapters or an equivalent narrow integration over the #57 executor. Domain job IDs and provider/MCP request IDs are provenance/reconciliation inputs, not a second generic once-only system. ### Conservative remote uncertainty Transport failure after durable dispatch is not proof the effect did not happen. If a provider/protocol has no reconciliation API, retain the admitted attempt as indeterminate rather than automatically resubmitting it. ## Implementation slices 1. **Canonical tool path — DONE in #86** - effectful `rlm_tool` execution crosses #57 after normalization/preflight and before mutation; - exact prepared tickets cross authority and admission unchanged; - edited pending proposals prepare a new ticket; - trusted executor identity is stable across ephemeral registry allocation; - fresh reads remain fresh; - crash/replay/adversarial tests prove no duplicate external mutation for the covered tool path. 2. **Model/provider path** - route paid/stateful provider calls through a code-owned effect adapter where appropriate; - define stable request identity and provider idempotency/reconciliation behavior explicitly; - preserve chain retry semantics as explicit retry/resample rather than accidental replay; - do not claim provider-level exactly-once when the provider cannot supply it. 3. **MCP request + lifecycle path** - classify effectful MCP commands/lifecycle actions versus pure/read-only operations; - route effectful transport exchanges, install, run/start, stop/restart, and process-backed lifecycle mutations through #57; - retain safe MCP request/operation IDs and endpoint/profile identity as reconciliation/provenance inputs. 4. **External process/network/repository-mutation integration contract** - document and enforce one code-owned #57 adapter path for external tool libraries; - concrete standard tool-pack implementation remains owned by #49/#50 as appropriate. ## Required tests for remaining slices Tests must use externally meaningful counters/state, not only ledger internals. - canonical fake provider crash after externally observable acceptance and before local observation does not submit again after fresh-process restart; - provider transport exception after dispatch becomes reconciliation-required/indeterminate rather than an automatic second request; - explicit provider retry/resample has stable linked new-attempt identity and correct fresh-authority behavior; - canonical MCP mutating request does not duplicate under wrapper/backtracking/restart; - MCP install/run/process fixture records at most one external mutation for one admitted attempt; - cancellation after dispatch never proves the remote effect did not occur; - pure/read-only paths remain usable without accidental permanent memoization; - #44/#45 accounting/trajectory work can consume authoritative attempt/observation lineage without reconstructing effect history from lexical variables. Concurrency tests must use deterministic synchronization, not sleeps. ## Acceptance criteria - [x] `rlm_tool` effectful execution no longer jumps directly from authority to `perform_tool_effect` without #57 admission. (#86) - [ ] Canonical model/provider requests that may incur paid/stateful external work are routed through #57 where applicable. - [ ] Canonical effectful MCP requests are routed through #57. - [ ] MCP install/run/stop/restart and process-backed lifecycle mutations use the same effect-identity boundary where applicable. - [ ] External process/network/repository-mutation tool integrations have a documented code-owned #57 adapter path. - [ ] #53 exact authority semantics remain intact for changed payload, retry, resample, approve/deny/edit, `allow_once`, `allow_session`, and `dangerous` across all adopted paths. - [ ] #54 canonical execute -> Future -> sync-await direction remains intact; no nested-wait regression. - [ ] Backtracking, repeated waits, callbacks, and wrapper reconstruction do not resubmit the same admitted attempt. - [ ] Crash-after-remote-acceptance fixtures prove fresh-process recovery without automatic resubmission at canonical integration boundaries. - [ ] Non-reconcilable uncertain effects remain indeterminate until trusted policy resolves them. - [ ] No documentation claims generic exactly-once execution. - [ ] Deterministic CI is green for the completed adoption set. - [ ] Configured REAL provider gates are green without deliberately duplicating paid requests for destructive crash testing. ## Non-goals - No competing generic effect ledger. - No rewrite of #53 authority. - No duplicate async scheduler. - No requirement to memoize pure reads forever. - No requirement for #57 itself to finish all #44/#45 accounting/trajectory presentation work. ## References - #57 effect identity / once-only observation semantics - #78 generic substrate - #83 hardening - #85 migration - #86 merged canonical tool-path slice - #44 provider usage on failure - #45 trajectory fidelity - #53 authority boundary - #54 canonical async contract - #49/#50 external concrete tool work

nsaspy commented

2026-09-10 21:23:34 +00:00

Author

Owner

Duplicate of #79 (pre-existing Forgejo mirror with GitHub number parity). Closing this accidental duplicate created by today's open-state sync; #79 stays canonical on Forgejo.

nsaspy closed this issue

2026-09-10 21:23:34 +00:00