Harden durable effect substrate across identity, retention, and recovery #83

Merged

lost-rob0t merged 5 commits from fix/issue-80-effect-substrate-adversarial-hardening into main

2026-08-17 21:27:51 +00:00

lost-rob0t commented

2026-08-17 21:08:47 +00:00

(Migrated from github.com)

Closes #80
Closes #81
Closes #82

Refs #57 #79 #53 #54 #44 #45.

Summary

This hardens the generic effect substrate merged in #78 without expanding into #79 canonical provider/tool/MCP adoption.

serializes preparation, ticket validation, admission, cancellation, dispatch, observation, resolution, and prune against one effect-state critical section;
advances a durable logical-call epoch before pruning, invalidating pre-prune tickets and deriving fresh post-prune call/attempt/provider identities;
creates a durable per-store UUID namespace, stable across restart/path aliases and distinct across independent ledgers;
binds executor adapter identity into executable identity and persisted attempt metadata through effect_prepare/5;
rejects wrong-adapter reconciliation before any callback and replays authoritative local observations without remote downgrade;
holds a store execution lease across the complete adapter lifecycle, so close/switch fails closed while work is in flight;
makes abandoned terminal rather than retry authorization;
treats attempts/observations as authoritative and repairs deterministic, idempotent lifecycle/accounting event projections after either local crash window;
fails closed on non-empty pre-v2 ledgers with an explicit migration error rather than silently creating a namespace that could reuse a remote identity.

The implementation follows the logic-native durable control-plane direction in research/RLM-RESEARCH-010-logic-native-control-plane.org: durable state-machine facts remain authoritative, host-controlled authority stays separate, and uncertain external outcomes are never converted into implicit permission to execute again. The four authority tiers remain unchanged and #54 retains execute → Future → sync-await.

Adversarial verification

New deterministic tests cover:

adapter A/B identity, authority, replay, reconciliation, and provider-key separation;
caller attempts to override reserved adapter identity;
wrong-adapter reconciliation with a callback counter proving no invocation;
local observed reconciliation with no remote callback;
abandoned retry/resample rejection;
stale-ticket and fresh-identity behavior after prune;
barrier races for prune vs retry admission and prune vs preparation, repeated locally;
independent-store namespace isolation plus same-store restart stability;
explicit legacy-ledger migration failure;
in-flight adapter barriers proving close/switch rejection and origin-store completion;
two fresh-process SIGKILL fixtures for observation-before-status and status-before-event recovery, with exactly one accounting event and no remote resubmission.

Local verification used SWI-Prolog 9.2.9 for module loading, runtime checks, focused restart/crash tests, and 30 repeated adversarial race-suite runs. Existing tests that use SWI 10 dict-function expansion inside assertion/1 are not portable to that fallback runtime, so this PR's canonical SWI 10 GitHub Actions suite is the merge gate.

Compatibility and scope

New empty ledgers initialize automatically at schema version 2.
Existing non-empty #78 ledgers fail closed with legacy_effect_store_requires_migration; no unsafe silent migration is attempted.
No distributed consensus or generic provider exactly-once claim is added.
#57 remains open.
#79 remains the separate canonical adoption slice.

Closes #80 Closes #81 Closes #82 Refs #57 #79 #53 #54 #44 #45. ## Summary This hardens the generic effect substrate merged in #78 without expanding into #79 canonical provider/tool/MCP adoption. - serializes preparation, ticket validation, admission, cancellation, dispatch, observation, resolution, and prune against one effect-state critical section; - advances a durable logical-call epoch before pruning, invalidating pre-prune tickets and deriving fresh post-prune call/attempt/provider identities; - creates a durable per-store UUID namespace, stable across restart/path aliases and distinct across independent ledgers; - binds executor adapter identity into executable identity and persisted attempt metadata through `effect_prepare/5`; - rejects wrong-adapter reconciliation before any callback and replays authoritative local observations without remote downgrade; - holds a store execution lease across the complete adapter lifecycle, so close/switch fails closed while work is in flight; - makes `abandoned` terminal rather than retry authorization; - treats attempts/observations as authoritative and repairs deterministic, idempotent lifecycle/accounting event projections after either local crash window; - fails closed on non-empty pre-v2 ledgers with an explicit migration error rather than silently creating a namespace that could reuse a remote identity. The implementation follows the logic-native durable control-plane direction in `research/RLM-RESEARCH-010-logic-native-control-plane.org`: durable state-machine facts remain authoritative, host-controlled authority stays separate, and uncertain external outcomes are never converted into implicit permission to execute again. The four authority tiers remain unchanged and #54 retains execute → Future → sync-await. ## Adversarial verification New deterministic tests cover: - adapter A/B identity, authority, replay, reconciliation, and provider-key separation; - caller attempts to override reserved adapter identity; - wrong-adapter reconciliation with a callback counter proving no invocation; - local observed reconciliation with no remote callback; - abandoned retry/resample rejection; - stale-ticket and fresh-identity behavior after prune; - barrier races for prune vs retry admission and prune vs preparation, repeated locally; - independent-store namespace isolation plus same-store restart stability; - explicit legacy-ledger migration failure; - in-flight adapter barriers proving close/switch rejection and origin-store completion; - two fresh-process SIGKILL fixtures for observation-before-status and status-before-event recovery, with exactly one accounting event and no remote resubmission. Local verification used SWI-Prolog 9.2.9 for module loading, runtime checks, focused restart/crash tests, and 30 repeated adversarial race-suite runs. Existing tests that use SWI 10 dict-function expansion inside `assertion/1` are not portable to that fallback runtime, so this PR's canonical SWI 10 GitHub Actions suite is the merge gate. ## Compatibility and scope - New empty ledgers initialize automatically at schema version 2. - Existing non-empty #78 ledgers fail closed with `legacy_effect_store_requires_migration`; no unsafe silent migration is attempted. - No distributed consensus or generic provider exactly-once claim is added. - #57 remains open. - #79 remains the separate canonical adoption slice.

lost-rob0t commented

2026-08-17 21:14:56 +00:00

(Migrated from github.com)

Second adversarial review — final head `ca6fa05b920c28bc7eff2fc90bc9ea41a7a9d60a`

I re-reviewed the final implementation against #80/#81/#82 after the initial canonical CI pass.

The review specifically rechecked lock ordering and atomicity around prepare/admit/cancel/dispatch/observe/resolve/prune; epoch-advance-before-delete crash behavior; store namespace restart and alias behavior; lease cleanup and store-switch rejection; adapter identity trust/override behavior; local-observation reconciliation; event projection idempotence; and legacy-store failure mode.

That pass found coverage omissions against the literal acceptance matrix. Commit ca6fa05 adds deterministic barrier cases for prune racing dispatch, observation, pre-dispatch cancellation, and indeterminate resolution; prune racing preparation; namespace persistence through relative/absolute reopen; and cross-store reconciliation refusal. The 18-case adversarial suite passed 20 consecutive local runs after the addition.

No unresolved correctness, authority-boundary, retention, or scope findings remain in this review. #79 is still intentionally untouched and #57 must remain open. Merge remains gated on both jobs for the exact final head, including REAL OpenRouter.

## Second adversarial review — final head `ca6fa05b920c28bc7eff2fc90bc9ea41a7a9d60a` I re-reviewed the final implementation against #80/#81/#82 after the initial canonical CI pass. The review specifically rechecked lock ordering and atomicity around prepare/admit/cancel/dispatch/observe/resolve/prune; epoch-advance-before-delete crash behavior; store namespace restart and alias behavior; lease cleanup and store-switch rejection; adapter identity trust/override behavior; local-observation reconciliation; event projection idempotence; and legacy-store failure mode. That pass found coverage omissions against the literal acceptance matrix. Commit `ca6fa05` adds deterministic barrier cases for prune racing dispatch, observation, pre-dispatch cancellation, and indeterminate resolution; prune racing preparation; namespace persistence through relative/absolute reopen; and cross-store reconciliation refusal. The 18-case adversarial suite passed 20 consecutive local runs after the addition. No unresolved correctness, authority-boundary, retention, or scope findings remain in this review. #79 is still intentionally untouched and #57 must remain open. Merge remains gated on both jobs for the exact final head, including REAL OpenRouter.