Repair duplicate relation documents in CouchDB #87

Merged

lost-rob0t merged 3 commits from fix/relation-db-dedup into master

2026-07-31 15:29:17 +00:00

lost-rob0t commented

2026-07-31 02:50:40 +00:00

(Migrated from github.com)

What this does

adds a dry-run-by-default CouchDB migration for relation documents
canonicalizes directed relation identity as (dataset, source, predicate, target)
merges sources and notes instead of discarding duplicate evidence
creates canonical relation:<sha256> documents before tombstoning legacy IDs
supports both flat 0.8 and nested 0.9 relation documents
writes a JSONL backup automatically before --apply
leaves malformed relations untouched and reports them

This is database repair only. No graph or UI behavior is changed.

Validation

python3 -m unittest -v scripts/test_dedupe_relations.py
Ran 5 tests ... OK
python3 -m py_compile scripts/dedupe_relations.py

Run order

pause relation-producing actors / RabbitMQ ingest
run python3 scripts/dedupe_relations.py and inspect the dry-run summary
run python3 scripts/dedupe_relations.py --apply
run the dry-run again and verify zero remaining canonical upserts/deletions

## What this does - adds a dry-run-by-default CouchDB migration for relation documents - canonicalizes directed relation identity as `(dataset, source, predicate, target)` - merges `sources` and notes instead of discarding duplicate evidence - creates canonical `relation:<sha256>` documents before tombstoning legacy IDs - supports both flat 0.8 and nested 0.9 relation documents - writes a JSONL backup automatically before `--apply` - leaves malformed relations untouched and reports them This is database repair only. No graph or UI behavior is changed. ## Validation ```text python3 -m unittest -v scripts/test_dedupe_relations.py Ran 5 tests ... OK python3 -m py_compile scripts/dedupe_relations.py ``` ## Run order 1. pause relation-producing actors / RabbitMQ ingest 2. run `python3 scripts/dedupe_relations.py` and inspect the dry-run summary 3. run `python3 scripts/dedupe_relations.py --apply` 4. run the dry-run again and verify zero remaining canonical upserts/deletions