Add explicit pre-v2 effect-store migration #85

Merged

lost-rob0t merged 4 commits from feature/issue-84-effect-store-migration into main

2026-08-17 23:39:09 +00:00

lost-rob0t commented

2026-08-17 23:29:13 +00:00

(Migrated from github.com)

Closes #84.

Why

PR #83 correctly rejects non-empty PR #78 journals instead of silently assigning a v2 namespace. Silent assignment can change remote identity, detach observations/lineage, or make accepted work look new.

Architecture

adds an explicit offline effect-store migrate CLI;
reads the actual PR #78 persistency predicates under the canonical runtime sidecar lock;
validates calls, complete attempt revision chains, lineage, observations, and events before writing;
preserves legacy call IDs, attempt IDs, provider idempotency keys, revisions, observations, usage/provenance, and events exactly;
adds a deterministic v2 namespace/migration record without pretending legacy IDs were namespace-derived;
persists operator-reviewed adapter bindings separately from immutable legacy facts;
keeps unresolved unbound attempts fail-closed and never guesses an adapter;
creates new v2 identity for all post-migration execution;
path-binds migrated journals so a copy cannot silently become an independent writable store with the same namespace;
stages, compacts, reopens, validates, syncs, and atomically publishes in the destination filesystem;
requires and verifies a byte-exact backup for in-place replacement.

Migration never calls submit, cancel, or reconcile adapters.

Operator commands

Out-of-place:

swipl -q -s bin/prolog-rlm.pl -- effect-store migrate \
  --source effects.db --output effects.v2.db --json

In-place:

swipl -q -s bin/prolog-rlm.pl -- effect-store migrate \
  --source effects.db --in-place --backup effects.pre-v2.backup --json

The strict manifest/report formats, rollback process, uncertainty behavior, and copy semantics are documented in docs/effect-migration.md.

Threat model / invariants

Legacy observations remain authoritative and replay locally. Unresolved attempts retain the original provider key and require trusted adapter identity before reconciliation. Abandoned attempts remain terminal. Legacy tickets cannot admit work. Caller metadata cannot override migrated trusted identity.

This is local single-writer safety using the existing advisory-lock contract; it is not distributed consensus or generic exactly-once execution.

Scope

#79 canonical provider/tool/MCP/process adoption is not implemented.
#57 remains open.
#53 authority semantics are not rewritten; migrated adapter identity only composes with the existing exact reconciliation match.
#54 execute-to-Future architecture is unchanged.
References RLM-RESEARCH-010-logic-native-control-plane.org.

Validation

Adds a journal fixture emitted through the exact PR #78 persistent predicate schema and tests for preservation/replay, uncertainty, strict manifests, wrong-adapter fail-closed behavior, new v2 identity, byte-exact backup, idempotent reporting, and copied-store rejection.

CI is the execution oracle for this first pushed head because the work runner does not include SWI-Prolog. I will inspect and fix every failing job on this same branch before merge.

Closes #84. ## Why PR #83 correctly rejects non-empty PR #78 journals instead of silently assigning a v2 namespace. Silent assignment can change remote identity, detach observations/lineage, or make accepted work look new. ## Architecture - adds an explicit offline `effect-store migrate` CLI; - reads the actual PR #78 persistency predicates under the canonical runtime sidecar lock; - validates calls, complete attempt revision chains, lineage, observations, and events before writing; - preserves legacy call IDs, attempt IDs, provider idempotency keys, revisions, observations, usage/provenance, and events exactly; - adds a deterministic v2 namespace/migration record without pretending legacy IDs were namespace-derived; - persists operator-reviewed adapter bindings separately from immutable legacy facts; - keeps unresolved unbound attempts fail-closed and never guesses an adapter; - creates new v2 identity for all post-migration execution; - path-binds migrated journals so a copy cannot silently become an independent writable store with the same namespace; - stages, compacts, reopens, validates, syncs, and atomically publishes in the destination filesystem; - requires and verifies a byte-exact backup for in-place replacement. Migration never calls submit, cancel, or reconcile adapters. ## Operator commands Out-of-place: ```sh swipl -q -s bin/prolog-rlm.pl -- effect-store migrate \ --source effects.db --output effects.v2.db --json ``` In-place: ```sh swipl -q -s bin/prolog-rlm.pl -- effect-store migrate \ --source effects.db --in-place --backup effects.pre-v2.backup --json ``` The strict manifest/report formats, rollback process, uncertainty behavior, and copy semantics are documented in `docs/effect-migration.md`. ## Threat model / invariants Legacy observations remain authoritative and replay locally. Unresolved attempts retain the original provider key and require trusted adapter identity before reconciliation. Abandoned attempts remain terminal. Legacy tickets cannot admit work. Caller metadata cannot override migrated trusted identity. This is local single-writer safety using the existing advisory-lock contract; it is not distributed consensus or generic exactly-once execution. ## Scope - #79 canonical provider/tool/MCP/process adoption is **not implemented**. - #57 remains open. - #53 authority semantics are not rewritten; migrated adapter identity only composes with the existing exact reconciliation match. - #54 execute-to-Future architecture is unchanged. - References `RLM-RESEARCH-010-logic-native-control-plane.org`. ## Validation Adds a journal fixture emitted through the exact PR #78 persistent predicate schema and tests for preservation/replay, uncertainty, strict manifests, wrong-adapter fail-closed behavior, new v2 identity, byte-exact backup, idempotent reporting, and copied-store rejection. CI is the execution oracle for this first pushed head because the work runner does not include SWI-Prolog. I will inspect and fix every failing job on this same branch before merge.

lost-rob0t commented

2026-08-17 23:38:59 +00:00

(Migrated from github.com)

Second adversarial review — complete

Reviewed the complete final diff and migration state machine after opening this PR.

Actionable findings fixed on this branch:

Byte-identical independently migrated copies could share a namespace. Namespace and migration identity are now deterministically scoped by both the verified source digest and canonical destination path. A copied published journal is separately path-fenced at runtime.
Legacy caller metadata could masquerade as trusted adapter identity. Migrated legacy attempts now ignore old metadata.executor_identity; only an immutable manifest binding is trusted. Missing bindings remain unknown and no callback is invoked.
Revision validation checked contiguity but not lifecycle legality. Migration now validates allowed status transitions, observation-admissible states, globally unique event IDs, and unique provider idempotency keys.
A preserved legacy admitted attempt could reach the public ID-only dispatch predicate. Migrated legacy admitted attempts are now explicitly non-dispatchable. New external work must use a normal namespace-bound v2 ticket.
Crash/race evidence was too narrow. Added fresh-process tests for all eight durable phases, a real SIGKILL at the validated boundary, canonical source-lock conflict, CLI routing, rerun convergence, exact observation/provider-key preservation, and no half-published runtime store.

No submit, cancel, or reconcile adapter is reachable from the migration module.

Final head: 655ad60d2aaf5cb537f36b578c62420f134aae02.

CI run 795 on that exact head:

deterministic unit/load/PlUnit/benchmark/conformance/deep-recursion/CLI/restart/whitespace gates: green;
REAL OpenRouter integration: first attempt hit known #46 free-router sentinel instability despite HTTP 200 and a healthy completed stream; exact failed job was rerun, and the complete REAL lane is green.

#79 is not implemented. #57 remains open.

## Second adversarial review — complete Reviewed the complete final diff and migration state machine after opening this PR. Actionable findings fixed on this branch: 1. **Byte-identical independently migrated copies could share a namespace.** Namespace and migration identity are now deterministically scoped by both the verified source digest and canonical destination path. A copied published journal is separately path-fenced at runtime. 2. **Legacy caller metadata could masquerade as trusted adapter identity.** Migrated legacy attempts now ignore old `metadata.executor_identity`; only an immutable manifest binding is trusted. Missing bindings remain `unknown` and no callback is invoked. 3. **Revision validation checked contiguity but not lifecycle legality.** Migration now validates allowed status transitions, observation-admissible states, globally unique event IDs, and unique provider idempotency keys. 4. **A preserved legacy `admitted` attempt could reach the public ID-only dispatch predicate.** Migrated legacy admitted attempts are now explicitly non-dispatchable. New external work must use a normal namespace-bound v2 ticket. 5. **Crash/race evidence was too narrow.** Added fresh-process tests for all eight durable phases, a real SIGKILL at the validated boundary, canonical source-lock conflict, CLI routing, rerun convergence, exact observation/provider-key preservation, and no half-published runtime store. No submit, cancel, or reconcile adapter is reachable from the migration module. Final head: `655ad60d2aaf5cb537f36b578c62420f134aae02`. CI run 795 on that exact head: - deterministic unit/load/PlUnit/benchmark/conformance/deep-recursion/CLI/restart/whitespace gates: green; - REAL OpenRouter integration: first attempt hit known #46 free-router sentinel instability despite HTTP 200 and a healthy completed stream; exact failed job was rerun, and the complete REAL lane is green. #79 is not implemented. #57 remains open.