[P1 data integrity] “Atomic” document batches can leave partially committed corpus state #136

New issue

Open

opened 2026-07-28 23:04:19 +00:00 by lost-rob0t · 0 comments

lost-rob0t commented

2026-07-28 23:04:19 +00:00

(Migrated from github.com)

Finding

commitDocumentBatch() labels operations as atomic, writes all documents through bulkDocs(), and attempts a compensating rollback when some individual writes fail.

results = await database.bulkDocs(writes.map(({ document }) => document));

if (atomic && writeErrors.length && successful.length) {
  const rollback = await rollbackSuccessfulWrites(database, successful, existing);
  return {
    saved: rollback.surviving,
    // ...
  };
}

PouchDB/CouchDB bulk writes are not a cross-document transaction. Each document can succeed or fail independently. The rollback is another independent revision-sensitive bulk write and can itself conflict or fail. The function explicitly returns surviving writes when this happens.

That means an operation requested as atomic: true can still leave a partially modified corpus. This affects imports, actor output, queue ingestion, and any other path using the shared batch layer. It also contradicts the architecture claim that batches are atomic and undone as one transaction.

Impact

Relations may be committed without their intended endpoint documents or vice versa.
Actor/import batches can leave a corpus in a state the caller was told was rejected.
Concurrent replication or edits increase rollback-conflict probability.
Retrying the operation can produce version/conflict behavior that differs per document.
Undo history may not accurately represent the surviving partial state.

Required fix

Do not claim storage-level atomicity where the backend cannot provide it. Implement logical atomicity with an explicit batch journal and visibility boundary.

Transaction envelope

const batchId = `batch:${crypto.randomUUID()}`;

await putState({
  _id: batchId,
  type: "quasar.document-batch",
  status: "preparing",
  documentIds: writes.map(({ document }) => document._id),
  inverseDocuments,
  createdAt: new Date().toISOString()
});

Write candidate document revisions tagged with the batch ID:

const staged = writes.map(({ document }) => ({
  ...document,
  _quasar_batch: {
    id: batchId,
    state: "staged"
  }
}));

Application projections must ignore staged revisions until one small commit record becomes visible:

await putState({
  _id: batchId,
  _rev: batchRev,
  type: "quasar.document-batch",
  status: "committed",
  committedAt: new Date().toISOString()
});

The document source/view layer should expose a staged document only when its referenced batch journal is committed. Failed or interrupted batches can then be recovered deterministically without pretending rollback was atomic.

Alternative designs are acceptable, including a backend transaction service, but the invariant must be observable logical all-or-nothing behavior.

Immediate correction

Until logical transactions exist:

Rename atomic to rollbackOnError or bestEffortAtomic.
Surface partial survivors as a top-level failure requiring repair.
Block queue acknowledgement when survivors remain.
Do not record a normal single-step undo entry for a partially rolled-back batch.
Update architecture and UI copy.

if (rollback.surviving.length) {
  throw new PartialBatchCommitError({
    batchId,
    surviving: rollback.surviving,
    errors: [...writeErrors, ...rollback.errors]
  });
}

Acceptance criteria

An atomic batch is never visible partially to normal corpus queries.
Browser/process interruption between staging and commit is recoverable.
Replication conflicts cannot expose half of a committed logical batch.
Queue deliveries are acknowledged only after logical commit.
Undo/redo operates against the exact committed batch state.
UI and documentation distinguish true logical atomicity from best-effort rollback.
Tests inject per-document write failures, rollback conflicts, reloads, and concurrent replication.

## Finding `commitDocumentBatch()` labels operations as atomic, writes all documents through `bulkDocs()`, and attempts a compensating rollback when some individual writes fail. ```js results = await database.bulkDocs(writes.map(({ document }) => document)); if (atomic && writeErrors.length && successful.length) { const rollback = await rollbackSuccessfulWrites(database, successful, existing); return { saved: rollback.surviving, // ... }; } ``` PouchDB/CouchDB bulk writes are not a cross-document transaction. Each document can succeed or fail independently. The rollback is another independent revision-sensitive bulk write and can itself conflict or fail. The function explicitly returns surviving writes when this happens. That means an operation requested as `atomic: true` can still leave a partially modified corpus. This affects imports, actor output, queue ingestion, and any other path using the shared batch layer. It also contradicts the architecture claim that batches are atomic and undone as one transaction. ## Impact - Relations may be committed without their intended endpoint documents or vice versa. - Actor/import batches can leave a corpus in a state the caller was told was rejected. - Concurrent replication or edits increase rollback-conflict probability. - Retrying the operation can produce version/conflict behavior that differs per document. - Undo history may not accurately represent the surviving partial state. ## Required fix Do not claim storage-level atomicity where the backend cannot provide it. Implement logical atomicity with an explicit batch journal and visibility boundary. ### Transaction envelope ```js const batchId = `batch:${crypto.randomUUID()}`; await putState({ _id: batchId, type: "quasar.document-batch", status: "preparing", documentIds: writes.map(({ document }) => document._id), inverseDocuments, createdAt: new Date().toISOString() }); ``` Write candidate document revisions tagged with the batch ID: ```js const staged = writes.map(({ document }) => ({ ...document, _quasar_batch: { id: batchId, state: "staged" } })); ``` Application projections must ignore staged revisions until one small commit record becomes visible: ```js await putState({ _id: batchId, _rev: batchRev, type: "quasar.document-batch", status: "committed", committedAt: new Date().toISOString() }); ``` The document source/view layer should expose a staged document only when its referenced batch journal is committed. Failed or interrupted batches can then be recovered deterministically without pretending rollback was atomic. Alternative designs are acceptable, including a backend transaction service, but the invariant must be observable logical all-or-nothing behavior. ### Immediate correction Until logical transactions exist: - Rename `atomic` to `rollbackOnError` or `bestEffortAtomic`. - Surface partial survivors as a top-level failure requiring repair. - Block queue acknowledgement when survivors remain. - Do not record a normal single-step undo entry for a partially rolled-back batch. - Update architecture and UI copy. ```js if (rollback.surviving.length) { throw new PartialBatchCommitError({ batchId, surviving: rollback.surviving, errors: [...writeErrors, ...rollback.errors] }); } ``` ## Acceptance criteria - An `atomic` batch is never visible partially to normal corpus queries. - Browser/process interruption between staging and commit is recoverable. - Replication conflicts cannot expose half of a committed logical batch. - Queue deliveries are acknowledged only after logical commit. - Undo/redo operates against the exact committed batch state. - UI and documentation distinguish true logical atomicity from best-effort rollback. - Tests inject per-document write failures, rollback conflicts, reloads, and concurrent replication.