views: add canonical ingest-by-state materialization #59

Closed

nsaspy wants to merge 7 commits from rage/15-ingest-by-state into master

nsaspy commented

2026-09-04 20:36:52 +00:00

Owner

Advances #15 with the remaining first-useful investigation view, ingest/by-state, over Hackmode's canonical durable outbox database.

RED-first evidence: exact test-only head 0614e3b0bbc894af1cae46fa22f0355ca121a748 passed monorepo and agent-framework-boundary but the real Common Lisp Load Hackmode and run ASDF tests step failed after setup completed. That head had only the acceptance fixture and was still pinned to pre-named-source Tek9.

Upstream dependency: lost-rob0t/tek9#17 merged as 1cb978084da8b4f1d184b3ef2a1d30057f525608 after both Ubuntu test jobs, package smoke, benchmark regression, and merge gate passed. This PR pins Hackmode core CI to that immutable Tek9 merge.

Implementation:

adds Hackmode-owned ingest/by-state view backed by the existing named outbox Tek9 database;
preserves outbox as the sole canonical durable state (no copy into the primary operation DB);
creates only the empty canonical outbox keyspace when investigation views are initialized;
exposes deterministic state-prefixed rows and investigation-view-outbox-ids through Tek9's public view APIs;
rebuilds from the declared outbox source alongside existing assets/by-type and assets/by-tag views;
retains the existing asset incremental path. Outbox state-key replacement is not falsely treated as generic incremental deletion: Tek9 does not yet expose prior mapper emissions for removing a stale old-state key, so the helper uses a full source rebuild when wired to state-changing writes rather than risking duplicate/stale state rows.

Acceptance test persists queued/failed canonical outbox entries, selects each state, and proves full rebuild equivalence. No raw LMDB access and no shadow outbox/DB/graph/KB authority.

Current candidate head: ced9de2e9941b30ba71e57fc70e98d81bdf07f4f. Merge only if all fresh exact-head Hackmode gates are green.

Advances #15 with the remaining first-useful investigation view, `ingest/by-state`, over Hackmode's canonical durable outbox database. RED-first evidence: exact test-only head `0614e3b0bbc894af1cae46fa22f0355ca121a748` passed monorepo and agent-framework-boundary but the real Common Lisp `Load Hackmode and run ASDF tests` step failed after setup completed. That head had only the acceptance fixture and was still pinned to pre-named-source Tek9. Upstream dependency: lost-rob0t/tek9#17 merged as `1cb978084da8b4f1d184b3ef2a1d30057f525608` after both Ubuntu test jobs, package smoke, benchmark regression, and merge gate passed. This PR pins Hackmode core CI to that immutable Tek9 merge. Implementation: - adds Hackmode-owned `ingest/by-state` view backed by the existing named `outbox` Tek9 database; - preserves outbox as the sole canonical durable state (no copy into the primary operation DB); - creates only the empty canonical outbox keyspace when investigation views are initialized; - exposes deterministic state-prefixed rows and `investigation-view-outbox-ids` through Tek9's public view APIs; - rebuilds from the declared outbox source alongside existing `assets/by-type` and `assets/by-tag` views; - retains the existing asset incremental path. Outbox state-key replacement is not falsely treated as generic incremental deletion: Tek9 does not yet expose prior mapper emissions for removing a stale old-state key, so the helper uses a full source rebuild when wired to state-changing writes rather than risking duplicate/stale state rows. Acceptance test persists queued/failed canonical outbox entries, selects each state, and proves full rebuild equivalence. No raw LMDB access and no shadow outbox/DB/graph/KB authority. Current candidate head: `ced9de2e9941b30ba71e57fc70e98d81bdf07f4f`. Merge only if all fresh exact-head Hackmode gates are green.

nsaspy added 7 commits

2026-09-04 20:36:53 +00:00