Add crash-safe CouchDB document outbox #65

Merged

lost-rob0t merged 10 commits from agent/issue-14-couchdb-outbox into agent/schema-org-v0.9

2026-07-26 15:10:20 +00:00

lost-rob0t commented

2026-07-26 14:59:49 +00:00

(Migrated from github.com)

Summary

store mutation idempotency records and pending downstream events inside each CouchDB document revision
use stable mutation hashes and event IDs for retry-safe new and updated document flows
distinguish duplicate-success deliveries from conflicting idempotency-key reuse
separate ingress routes (documents.ingest.#, documents.update.#) from downstream routes (documents.new.*, documents.updated.*)
ACK Rabbit deliveries only after recoverable state is durable and publication succeeds
add a recovery scan over a dedicated CouchDB pending-outbox view
publish pending events in stable per-document sequence order
retain at-least-once physical delivery while providing stable logical event IDs for downstream deduplication

Design

This uses a transactional outbox embedded in the document revision:

extensions._server_mutations records mutation ID → content hash
extensions._server_outbox records ordered publication state
document mutation and pending event are persisted in one CouchDB revision
publish failure leaves a durable pending entry
retry or restart republishes the same logical event ID
successful publication is marked with a follow-up optimistic-concurrency revision

Regression coverage

forced publish failure after new-document persistence
forced publish failure after update persistence
restart recovery between persistence and publication
duplicate ingest and update deliveries
conflicting idempotency key with different content
missing update target rejection
per-document recovery ordering
pending-view and route-separation contract checks

Stack

base: agent/schema-org-v0.9
includes completed issues #11–#13

This remains draft until the required fixture and CLOS/outbox conformance gates are green.

Closes #14

## Summary - store mutation idempotency records and pending downstream events inside each CouchDB document revision - use stable mutation hashes and event IDs for retry-safe new and updated document flows - distinguish duplicate-success deliveries from conflicting idempotency-key reuse - separate ingress routes (`documents.ingest.#`, `documents.update.#`) from downstream routes (`documents.new.*`, `documents.updated.*`) - ACK Rabbit deliveries only after recoverable state is durable and publication succeeds - add a recovery scan over a dedicated CouchDB pending-outbox view - publish pending events in stable per-document sequence order - retain at-least-once physical delivery while providing stable logical event IDs for downstream deduplication ## Design This uses a **transactional outbox embedded in the document revision**: - `extensions._server_mutations` records mutation ID → content hash - `extensions._server_outbox` records ordered publication state - document mutation and pending event are persisted in one CouchDB revision - publish failure leaves a durable `pending` entry - retry or restart republishes the same logical event ID - successful publication is marked with a follow-up optimistic-concurrency revision ## Regression coverage - forced publish failure after new-document persistence - forced publish failure after update persistence - restart recovery between persistence and publication - duplicate ingest and update deliveries - conflicting idempotency key with different content - missing update target rejection - per-document recovery ordering - pending-view and route-separation contract checks ## Stack - base: `agent/schema-org-v0.9` - includes completed issues #11–#13 This remains draft until the required fixture and CLOS/outbox conformance gates are green. Closes #14