[SLICE 19][CORE-B] Implement SAFIRE segmented durable backend, monotonic ledger sequence, sparse frontiers, manifests, and atomic checkpoints #12

New issue

Open

opened 2026-09-08 01:57:00 +00:00 by nsaspy · 0 comments

nsaspy commented

2026-09-08 01:57:00 +00:00

Owner

Parent epic: #13
Depends on: #23 #24 #26
Research: prolog-rlm#402D

Goal

Replace whole-snapshot persistence as the scalable semantic-history path while keeping the storage SPI usable for small/bootstrap backends.

Implement one production backend profile with:

append-oriented immutable segments/object records;
dense monotonic sequence within one logical ledger;
exact ledger_frontier identity;
sparse/hierarchical frontier manifests where useful;
persistent range/direct indexes and cursors;
atomic durable commit publication before derived projection checkpoint;
projection checkpoints keyed by exact frontier + semantic/profile/identity/admission/redaction/integrity generations;
checksum/integrity metadata and explicit corrupted-segment status;
no requirement to materialize full history in SWI heap.

Backend choice must be evidence-driven and documented; SQLite/Postgres/segmented-log/KV variants may be compared, but one supported profile must land end-to-end.

Acceptance

Ordinary append does not rewrite O(total_history) state; restart resolves exact frontier; crash before/after commit/checkpoint yields deterministic ledger-wins behavior; stale/corrupt checkpoint is discarded/rebuilt; million-record benchmark fixture is runnable without claiming unmeasured billion-scale performance.

Parent epic: #13 Depends on: #23 #24 #26 Research: prolog-rlm#402D ## Goal Replace whole-snapshot persistence as the scalable semantic-history path while keeping the storage SPI usable for small/bootstrap backends. Implement one production backend profile with: - append-oriented immutable segments/object records; - dense monotonic sequence within one logical ledger; - exact `ledger_frontier` identity; - sparse/hierarchical frontier manifests where useful; - persistent range/direct indexes and cursors; - atomic durable commit publication before derived projection checkpoint; - projection checkpoints keyed by exact frontier + semantic/profile/identity/admission/redaction/integrity generations; - checksum/integrity metadata and explicit corrupted-segment status; - no requirement to materialize full history in SWI heap. Backend choice must be evidence-driven and documented; SQLite/Postgres/segmented-log/KV variants may be compared, but one supported profile must land end-to-end. ## Acceptance Ordinary append does not rewrite O(total_history) state; restart resolves exact frontier; crash before/after commit/checkpoint yields deterministic ledger-wins behavior; stale/corrupt checkpoint is discarded/rebuilt; million-record benchmark fixture is runnable without claiming unmeasured billion-scale performance.

nsaspy referenced this issue

2026-09-08 01:57:01 +00:00

[SLICE 03][CORE-A] Complete namespace/project identity, principal/capability policy, trust classes, and append-only audit #28

nsaspy referenced this issue

2026-09-08 01:57:02 +00:00

[SLICE 02][CORE-A] Stabilize library facade, SWI pack layout, store lifecycle, and backend SPI #29

nsaspy referenced this issue

2026-09-08 01:57:06 +00:00

[RAGE 001] Durable memory_remember + memory_get vertical slice #1

nsaspy referenced this issue

2026-09-08 01:57:39 +00:00

[EPIC][CORE] Library-first Symbolic Memory v1: durable semantic world model, learning history, federation, and zero-model recall #38