Implement authenticated encryption and key providers #55

Closed

lost-rob0t wants to merge 34 commits from agent/issue-10-encryption into main

lost-rob0t commented

2026-07-18 18:23:02 +00:00

(Migrated from github.com)

What changed

Added envelope schema version 2 using per-record 256-bit DEKs and XChaCha20-Poly1305 authenticated encryption through PyNaCl/libsodium.
Added canonical associated data binding record ID, capture generation, immutable configuration revision, creation time, algorithm, payload format, schema version, and exact frame sizes.
Added a strict six-frame bounded codec for header, associated data, wrapped DEK, nonce, ciphertext, and digest.
Added a bounded duplicate (key, nonce) guard with deterministic random/clock injection for known-answer and retry testing.
Added key-provider strategies for:
- operating-system keyring;
- Argon2id-protected encrypted local key store;
- explicitly configured GPG wrapping fallback;
- synthetic in-memory contract testing.
Added explicit provider routing. GPG fallback is attempted only when the configured primary is unavailable; locked, invalid, revoked, or authentication-failing providers remain fail-closed.
Added lifecycle encryption preflight mapping provider failures to encryption_unavailable.
Added EncryptionStageProcessor, which accepts RedactedStageItem only and returns strict encrypted envelope frames.
Added DEK-only rewrapping for key rotation; record ciphertext, nonce, and associated data remain unchanged.
Added key revocation/destruction behavior and encrypted local-store recovery tests.
Added secret-safe inspection for primary and fallback key references.
Added PyNaCl and keyring recommended backends to the locked Python/Nix environment.
Added encryption documentation and ADR-0008.

Security behavior

Record plaintext is encrypted directly from bounded redacted in-memory frames.
Storage remains structurally restricted to EncryptedRecordEnvelope.
Ciphertext, nonce, associated-data, record identity, or wrapped-key tampering fails authentication or strict format validation without partial plaintext.
Key material is excluded from configuration, logs, exceptions, object representations, and envelope headers.
The local key store rejects symlinks, malformed/oversized documents, wrong passphrases, and revoked entries; writes are atomic mode-0600 replacements containing encrypted key material only.
GPG uses fixed argv, no shell, stdin/stdout, finite timeout, bounded output, and no plaintext temporary files.
Missing, locked, invalid, or revoked keys prevent encryption/decryption and therefore prevent persistence.
Rotation rewraps only the DEK in memory; no record plaintext file is created.

TDD evidence

Observed red results:

Initial focused tests failed because the crypto implementation package did not exist.
Wrong-key, ciphertext/AAD/nonce/wrapped-key tampering, revoked-key, locked-provider, wrong-passphrase, and malformed-codec fixtures were added before their respective implementations.
ADR-0004 requirements added deterministic known-answer and duplicate-nonce tests before the injectable clock/random source and nonce-reuse guard.

Focused local green before final publication:

19 encryption/provider tests passed.
Ruff formatting and linting passed.
Strict Pyright passed with zero errors.
Bandit passed.

The final deterministic vector, nonce-reuse, and security-boundary tests are included in this branch. GitHub Actions under the repository's required CPython 3.14 environment is the authoritative complete gate.

Acceptance criteria

Storage APIs accept encrypted envelopes only.
Tampering and wrong keys fail authentication cleanly.
GPG fallback is explicit and health checked, never silently selected.
OS keyring and encrypted local key-store strategies are available.
Rotation can rewrap DEKs without changing record ciphertext or creating plaintext files.
Key IDs and versions remain non-secret metadata; key material is never logged or configured.

Closes #10

## What changed - Added envelope schema version 2 using per-record 256-bit DEKs and XChaCha20-Poly1305 authenticated encryption through PyNaCl/libsodium. - Added canonical associated data binding record ID, capture generation, immutable configuration revision, creation time, algorithm, payload format, schema version, and exact frame sizes. - Added a strict six-frame bounded codec for header, associated data, wrapped DEK, nonce, ciphertext, and digest. - Added a bounded duplicate `(key, nonce)` guard with deterministic random/clock injection for known-answer and retry testing. - Added key-provider strategies for: - operating-system keyring; - Argon2id-protected encrypted local key store; - explicitly configured GPG wrapping fallback; - synthetic in-memory contract testing. - Added explicit provider routing. GPG fallback is attempted only when the configured primary is unavailable; locked, invalid, revoked, or authentication-failing providers remain fail-closed. - Added lifecycle encryption preflight mapping provider failures to `encryption_unavailable`. - Added `EncryptionStageProcessor`, which accepts `RedactedStageItem` only and returns strict encrypted envelope frames. - Added DEK-only rewrapping for key rotation; record ciphertext, nonce, and associated data remain unchanged. - Added key revocation/destruction behavior and encrypted local-store recovery tests. - Added secret-safe inspection for primary and fallback key references. - Added PyNaCl and keyring recommended backends to the locked Python/Nix environment. - Added encryption documentation and ADR-0008. ## Security behavior - Record plaintext is encrypted directly from bounded redacted in-memory frames. - Storage remains structurally restricted to `EncryptedRecordEnvelope`. - Ciphertext, nonce, associated-data, record identity, or wrapped-key tampering fails authentication or strict format validation without partial plaintext. - Key material is excluded from configuration, logs, exceptions, object representations, and envelope headers. - The local key store rejects symlinks, malformed/oversized documents, wrong passphrases, and revoked entries; writes are atomic mode-0600 replacements containing encrypted key material only. - GPG uses fixed argv, no shell, stdin/stdout, finite timeout, bounded output, and no plaintext temporary files. - Missing, locked, invalid, or revoked keys prevent encryption/decryption and therefore prevent persistence. - Rotation rewraps only the DEK in memory; no record plaintext file is created. ## TDD evidence **Observed red results:** - Initial focused tests failed because the crypto implementation package did not exist. - Wrong-key, ciphertext/AAD/nonce/wrapped-key tampering, revoked-key, locked-provider, wrong-passphrase, and malformed-codec fixtures were added before their respective implementations. - ADR-0004 requirements added deterministic known-answer and duplicate-nonce tests before the injectable clock/random source and nonce-reuse guard. **Focused local green before final publication:** - 19 encryption/provider tests passed. - Ruff formatting and linting passed. - Strict Pyright passed with zero errors. - Bandit passed. The final deterministic vector, nonce-reuse, and security-boundary tests are included in this branch. GitHub Actions under the repository's required CPython 3.14 environment is the authoritative complete gate. ## Acceptance criteria - Storage APIs accept encrypted envelopes only. - Tampering and wrong keys fail authentication cleanly. - GPG fallback is explicit and health checked, never silently selected. - OS keyring and encrypted local key-store strategies are available. - Rotation can rewrap DEKs without changing record ciphertext or creating plaintext files. - Key IDs and versions remain non-secret metadata; key material is never logged or configured. Closes #10

lost-rob0t commented

2026-07-18 18:36:39 +00:00

(Migrated from github.com)

Closing this duplicate implementation in favor of PR #54, which already passes formatting, lint, strict typing, all test layers, and failure propagation for issue #10. Remaining work is isolated to the security scanner on #54.

nsaspy changed title from ~~WIP: Implement authenticated encryption and key providers~~ to Implement authenticated encryption and key providers

2026-09-04 14:35:55 +00:00