[P0] Explicitly migrate pre-v2 durable-effect ledgers #84

New issue

Closed

opened 2026-08-17 23:18:20 +00:00 by lost-rob0t · 0 comments

lost-rob0t commented

2026-08-17 23:18:20 +00:00

(Migrated from github.com)

Goal

Add an explicit, offline, exclusive migration for non-empty PR #78 effect journals that PR #83 correctly rejects with legacy_effect_store_requires_migration.

This issue is a prerequisite for #79. It does not implement canonical provider/tool/MCP/process adoption and does not close #57.

Threat model

Blindly assigning a v2 namespace to legacy calls can change call/attempt/provider identity, detach observations and lineage, choose the wrong adapter for unresolved work, or make accepted remote work look new. Migration must never cross an external boundary or treat absence of a local observation as proof that nothing happened.

Chosen compatibility model

Read the actual PR #78 persistency predicates under the same canonical sidecar lock used by runtime.
Validate the complete legacy revision graph and immutable observations before writing anything.
Preserve legacy call IDs, attempt IDs, provider idempotency keys, revisions, lineage, observations, events, usage, and provenance as authoritative history.
Add a v2 store namespace exactly once plus a versioned migration record/digest.
Keep operator adapter decisions in separate immutable audited binding records; never rewrite legacy metadata or infer an adapter.
Legacy records are replay/status/reconciliation inputs only. Legacy tickets are never admitted.
New post-migration work is prepared through the normal v2 namespace/epoch constructor and therefore receives a distinct v2 call/attempt/provider identity.
Bind migrated stores to their canonical destination path so a copied journal does not silently become an independent writable store with the same namespace. Backups are restoration material, not clones.
Migration writes a same-filesystem temporary journal, synchronizes and fully reopens/validates it, then atomically publishes. In-place mode first creates and verifies an independently restorable byte-for-byte backup.

Operator interface

bin/prolog-rlm.pl -- effect-store migrate --source LEDGER --output LEDGER.v2 [--manifest MANIFEST] [--backup BACKUP] [--in-place] [--json]

The manifest is strict, versioned JSON bound to the source digest. Unknown keys, duplicate/conflicting/nonexistent attempt bindings, invalid adapter atoms, observation mutation, identity changes, and incompatible digests are rejected.

Outcomes

Versioned machine-readable reports distinguish: migrated, already_migrated, incompatible, corrupt, ambiguous_adapter, lock_conflict, interrupted, and validation_failed.

Required proof

TDD coverage includes a representative journal emitted by PR #78; observed replay; uncertain/abandoned/retry lineage; exact provider-key preservation; manifest rejection; alias/collision/locking races; crash points with fresh processes and markers (no sleeps); backup restoration; copy/path binding; #53 migrated reconciliation; #54 regression; full effect/CLI/restart regression suites.

Non-goals

no #79 canonical adoption;
no #53 rewrite;
no second scheduler or generic ledger;
no retry authorization during migration;
no provider submit/cancel/reconcile calls;
no generic exactly-once or consensus claim.

References #57, #79, PR #78, PR #83, RLM-RESEARCH-010, #53, and #54.

## Goal Add an explicit, offline, exclusive migration for non-empty PR #78 effect journals that PR #83 correctly rejects with `legacy_effect_store_requires_migration`. This issue is a prerequisite for #79. It does not implement canonical provider/tool/MCP/process adoption and does not close #57. ## Threat model Blindly assigning a v2 namespace to legacy calls can change call/attempt/provider identity, detach observations and lineage, choose the wrong adapter for unresolved work, or make accepted remote work look new. Migration must never cross an external boundary or treat absence of a local observation as proof that nothing happened. ## Chosen compatibility model - Read the actual PR #78 persistency predicates under the same canonical sidecar lock used by runtime. - Validate the complete legacy revision graph and immutable observations before writing anything. - Preserve legacy call IDs, attempt IDs, provider idempotency keys, revisions, lineage, observations, events, usage, and provenance as authoritative history. - Add a v2 store namespace exactly once plus a versioned migration record/digest. - Keep operator adapter decisions in separate immutable audited binding records; never rewrite legacy metadata or infer an adapter. - Legacy records are replay/status/reconciliation inputs only. Legacy tickets are never admitted. - New post-migration work is prepared through the normal v2 namespace/epoch constructor and therefore receives a distinct v2 call/attempt/provider identity. - Bind migrated stores to their canonical destination path so a copied journal does not silently become an independent writable store with the same namespace. Backups are restoration material, not clones. - Migration writes a same-filesystem temporary journal, synchronizes and fully reopens/validates it, then atomically publishes. In-place mode first creates and verifies an independently restorable byte-for-byte backup. ## Operator interface `bin/prolog-rlm.pl -- effect-store migrate --source LEDGER --output LEDGER.v2 [--manifest MANIFEST] [--backup BACKUP] [--in-place] [--json]` The manifest is strict, versioned JSON bound to the source digest. Unknown keys, duplicate/conflicting/nonexistent attempt bindings, invalid adapter atoms, observation mutation, identity changes, and incompatible digests are rejected. ## Outcomes Versioned machine-readable reports distinguish: migrated, already_migrated, incompatible, corrupt, ambiguous_adapter, lock_conflict, interrupted, and validation_failed. ## Required proof TDD coverage includes a representative journal emitted by PR #78; observed replay; uncertain/abandoned/retry lineage; exact provider-key preservation; manifest rejection; alias/collision/locking races; crash points with fresh processes and markers (no sleeps); backup restoration; copy/path binding; #53 migrated reconciliation; #54 regression; full effect/CLI/restart regression suites. ## Non-goals - no #79 canonical adoption; - no #53 rewrite; - no second scheduler or generic ledger; - no retry authorization during migration; - no provider submit/cancel/reconcile calls; - no generic exactly-once or consensus claim. References #57, #79, PR #78, PR #83, RLM-RESEARCH-010, #53, and #54.