nsaspy/local-recall

Fork 0

Implement capture policy engine and sensitive-context exclusions #69

Merged

lost-rob0t merged 39 commits from agent/issue-17-capture-policy into main

2026-08-12 18:12:24 +00:00

lost-rob0t commented

2026-08-12 17:09:24 +00:00

(Migrated from github.com)

Closes #17.

Problem and scope

Implements the authoritative Local Recall capture policy security boundary. Policy now makes separate, typed decisions for screenshot capture, metadata capture, OCR, indexing, summarization, and remote-provider eligibility using normalized Local Recall metadata and lifecycle generation state.

The policy engine remains side-effect free. Enforcement is provided through typed adapters/guards that prevent denied or stale work from invoking capture, persistence, indexing, summarization, or provider callbacks.

Architecture

local_recall.domain.policy: project-owned policy operations, phases, sanitized decisions/authorizations/status, reason codes, and temporary-sensitive scopes.
local_recall.policy.PolicyEngine: deterministic evaluator over validated configuration plus runtime privacy/lock/generation state.
CapturePolicyAdapter: bridges the engine to the existing CapturePolicy capture port rather than creating a parallel capture model.
PolicyGuard: revalidates policy revision/generation and capture generation immediately before side effects.
PolicyEnforcementBoundary: phase-specific callback boundary for pre-capture, post-capture/pre-persistence, and downstream operations.
Policy modules do not import storage, redaction, screenshot, HTTP-client, or provider implementations.

Policy phases

Pre-capture — screenshot denial/uncertainty prevents pixel acquisition from being invoked.
Post-capture / pre-persistence — policy is evaluated again, and stale pre-capture authorization is rechecked before persistence.
Downstream — OCR, indexing, summarization, and remote-provider eligibility are independently authorized.

Operations

Independent operations are:

screenshot
metadata
ocr
indexing
summarization
remote-provider

Remote-provider policy is eligibility only; it cannot perform network/provider calls.

Deterministic precedence

Security precedence is explicit:

unhealthy policy state
manual privacy mode
locked session
explicit configured deny
temporary sensitive window/workspace
built-in/configured sensitive-context deny
explicit configured allow
profile/default decision

Configured deny always beats configured allow regardless of numeric priority. Within matching rules of the same effect, higher priority wins and equal priority is resolved by stable rule ID, so configuration ordering cannot change the result.

Default-deny sensitive contexts

Narrow built-in safety rules cover password-manager classes, common lock-screen classes, and common authentication prompts/title indicators. Configured sensitive applications/workspaces support additional environments such as dedicated pentesting workspaces without treating every terminal as sensitive.

privacy-strict continues to require default deny.

Manual privacy mode

Manual privacy mode is engine state, not a UI label. Toggling it increments policy generation and invalidates prior authorization. Capture-sensitive operations and remote eligibility fail closed while active. Lock-state transitions use the same invalidation model.

Temporary sensitive context

Typed control methods support temporary sensitivity for:

current normalized window.id
current normalized workspace

Markers have explicit TTLs (1 second through 24 hours), are bound to capture generation, take effect immediately, can be explicitly cleared, expire deterministically, and cannot silently survive into a later capture generation.

Pattern security

Title patterns are validated and compiled before hot-path evaluation. Policy config bounds rules to 256, title patterns to 256 characters, and evaluated metadata strings to 4096 characters. Malformed expressions, unsupported complex inline constructs, alternation, backreferences, nested repetition, excessive unbounded repetition, and oversized patterns are rejected. Security tests cover catastrophic-pattern candidates, malformed/oversized patterns, Unicode, and oversized metadata.

No dependency was added for regex handling.

Domain, time, and metadata-source matching

Domain policy consumes canonical domain-only url.domain metadata and never reconstructs full URLs.
Domains are case-normalized; a single trailing root dot is accepted; malformed domains are rejected.
example.com cannot match notexample.com; subdomain semantics require an explicit include_subdomains flag and a label boundary.
IPv4/IPv6 literals are exact-match capable.
Time windows use configured IANA timezones, fixed aware timestamps, midnight-crossing semantics, and deterministic DST conversion.
Metadata-source rules use normalized provenance IDs such as xorg-generic, qtile, and activitywatch; adapter-specific raw payloads are not consumed.
Missing, stale, malformed, or rule-required unavailable policy context fails closed for capture-sensitive operations.

Policy revision and stale work

Each loaded policy has an immutable revision plus an in-memory generation. Privacy transitions, lock transitions, temporary-sensitive changes/expiry, and policy replacement invalidate older authorization. A later policy reload therefore cannot silently make queued work authorized under an older revision current again.

The existing validated configuration manager remains the outer atomic configuration boundary and fails malformed reloads to the repository's established safe configuration semantics.

Sanitized audit/status behavior

Policy decisions contain only control information: operation, phase, allow/deny, certainty, stable reason code, optional rule ID, revision, and generation. They cannot carry screenshots, OCR text, titles, domains, or matched substrings.

Status exposes revision/generation, enabled-rule count, privacy/lock state, health, and sanitized error code only. Effective-config inspection masks policy application/title/workspace/domain/source matcher values. Existing audit models reject raw policy-match attributes outside their closed safe field set.

Synthetic security fixtures seed recognizable credential/title/domain/command-like values and assert they do not appear in expected sanitized outputs. The repository's detect-secrets baseline remains unchanged; two intentionally synthetic test literals use the scanner-supported inline allowlist pragma only on those fixture lines.

Configuration/schema impact

The existing immutable Pydantic configuration is extended with typed policy selectors, closed enums, stable IDs, timezone/time-window validation, bounds, and extra='forbid'. Changes are additive with safe defaults, so schema version 1 remains valid and no persistent migration is required.

TDD evidence

RED

Commit f1c6cbda6d56de8dd3d19fa60251ed175705663d, Actions run 31621658847, job 94197728739.

Formatting, Ruff, and ShellCheck passed. Strict Pyright then failed specifically because the #17 production policy module/types/config fields did not exist yet. Earlier formatting/lint-only failures were explicitly rejected as invalid red evidence and corrected before recording this red state.

GREEN

Final exact PR head: c4d850900a6980487a45d7e4eae59c3f8ebc6703.

Actions run 31625941909 is fully green on the exact head:

Ubuntu 22.04 checks: green
Ubuntu 24.04 checks: green
failure-propagation: green
canonical repository gate: green

Functional coverage on the final path includes:

unit: 417 passed
contract: 6 passed
integration: 23 passed
security: 49 passed
repository policy: passed
Bandit: passed
detect-secrets: passed
failure-propagation: passed

Full canonical gate

Job 94212233905 ran the repository's canonical command exactly:

./scripts/check

and passed on exact head c4d850900a6980487a45d7e4eae59c3f8ebc6703.

The execution environment available to this agent could not materialize the private repository checkout locally, so local filesystem execution was not falsely claimed; the exact canonical command was added as a required PR CI job and executed successfully on the tested head.

Acceptance-criteria mapping

typed authoritative engine: domain.policy + PolicyEngine
separate screenshot/metadata/OCR/index/summarization/remote gates: operation enum + matrix tests
pre-capture deny prevents pixels: enforcement integration tests
post-capture deny/stale auth prevents persistence: enforcement integration tests
privacy/lock high precedence: unit + integration + security tests
temporary current window/workspace sensitivity: typed scopes, TTL, clear, generation tests
password manager/auth/lock/configured sensitive workspace representation: built-ins/config tests
deterministic deny-first precedence: order/priority/conflict tests
regex/pattern abuse resistance: config bounds + pathological-pattern security tests
domain suffix safety: exact/subdomain/suffix/IP tests
timezone/DST/midnight handling: fixed-time tests
normalized provenance participation: Xorg/Qtile/ActivityWatch integration tests
missing/stale/malformed context fail closed: unit/security tests
atomic safe configuration behavior: malformed reload security test
stale authorization invalidation: revision/privacy/lock tests
sanitized decisions/status/effective config/audit: security tests
no direct storage/provider/capture bypass: structural security test
performance bound: maximum 256-rule deterministic evaluation test
documentation: docs/policy.md

Known limitations / boundaries

Desktop lock/idle event acquisition itself belongs to the lifecycle/idle work tracked separately; this PR provides and tests the lock policy state/gate but does not implement #18.
Metadata source composition/conflict resolution remains owned by the normalized metadata composition layer from #13; policy intentionally consumes only its normalized result and fails closed when trustworthy required context is unavailable.
The richer CLI/desktop UI for temporary sensitivity is intentionally out of scope; this PR provides the typed internal/control boundary it can call.
Policy allow is never sufficient by itself: redaction, encryption, lifecycle validation, retention/storage policy, and provider routing remain mandatory independent controls.

Closes #17. ## Problem and scope Implements the authoritative Local Recall capture policy security boundary. Policy now makes separate, typed decisions for screenshot capture, metadata capture, OCR, indexing, summarization, and remote-provider eligibility using normalized Local Recall metadata and lifecycle generation state. The policy engine remains side-effect free. Enforcement is provided through typed adapters/guards that prevent denied or stale work from invoking capture, persistence, indexing, summarization, or provider callbacks. ## Architecture - `local_recall.domain.policy`: project-owned policy operations, phases, sanitized decisions/authorizations/status, reason codes, and temporary-sensitive scopes. - `local_recall.policy.PolicyEngine`: deterministic evaluator over validated configuration plus runtime privacy/lock/generation state. - `CapturePolicyAdapter`: bridges the engine to the existing `CapturePolicy` capture port rather than creating a parallel capture model. - `PolicyGuard`: revalidates policy revision/generation and capture generation immediately before side effects. - `PolicyEnforcementBoundary`: phase-specific callback boundary for pre-capture, post-capture/pre-persistence, and downstream operations. - Policy modules do not import storage, redaction, screenshot, HTTP-client, or provider implementations. ## Policy phases 1. **Pre-capture** — screenshot denial/uncertainty prevents pixel acquisition from being invoked. 2. **Post-capture / pre-persistence** — policy is evaluated again, and stale pre-capture authorization is rechecked before persistence. 3. **Downstream** — OCR, indexing, summarization, and remote-provider eligibility are independently authorized. ## Operations Independent operations are: - `screenshot` - `metadata` - `ocr` - `indexing` - `summarization` - `remote-provider` Remote-provider policy is eligibility only; it cannot perform network/provider calls. ## Deterministic precedence Security precedence is explicit: 1. unhealthy policy state 2. manual privacy mode 3. locked session 4. explicit configured deny 5. temporary sensitive window/workspace 6. built-in/configured sensitive-context deny 7. explicit configured allow 8. profile/default decision Configured deny always beats configured allow regardless of numeric priority. Within matching rules of the same effect, higher priority wins and equal priority is resolved by stable rule ID, so configuration ordering cannot change the result. ## Default-deny sensitive contexts Narrow built-in safety rules cover password-manager classes, common lock-screen classes, and common authentication prompts/title indicators. Configured sensitive applications/workspaces support additional environments such as dedicated pentesting workspaces without treating every terminal as sensitive. `privacy-strict` continues to require default deny. ## Manual privacy mode Manual privacy mode is engine state, not a UI label. Toggling it increments policy generation and invalidates prior authorization. Capture-sensitive operations and remote eligibility fail closed while active. Lock-state transitions use the same invalidation model. ## Temporary sensitive context Typed control methods support temporary sensitivity for: - current normalized `window.id` - current normalized `workspace` Markers have explicit TTLs (1 second through 24 hours), are bound to capture generation, take effect immediately, can be explicitly cleared, expire deterministically, and cannot silently survive into a later capture generation. ## Pattern security Title patterns are validated and compiled before hot-path evaluation. Policy config bounds rules to 256, title patterns to 256 characters, and evaluated metadata strings to 4096 characters. Malformed expressions, unsupported complex inline constructs, alternation, backreferences, nested repetition, excessive unbounded repetition, and oversized patterns are rejected. Security tests cover catastrophic-pattern candidates, malformed/oversized patterns, Unicode, and oversized metadata. No dependency was added for regex handling. ## Domain, time, and metadata-source matching - Domain policy consumes canonical domain-only `url.domain` metadata and never reconstructs full URLs. - Domains are case-normalized; a single trailing root dot is accepted; malformed domains are rejected. - `example.com` cannot match `notexample.com`; subdomain semantics require an explicit `include_subdomains` flag and a label boundary. - IPv4/IPv6 literals are exact-match capable. - Time windows use configured IANA timezones, fixed aware timestamps, midnight-crossing semantics, and deterministic DST conversion. - Metadata-source rules use normalized provenance IDs such as `xorg-generic`, `qtile`, and `activitywatch`; adapter-specific raw payloads are not consumed. - Missing, stale, malformed, or rule-required unavailable policy context fails closed for capture-sensitive operations. ## Policy revision and stale work Each loaded policy has an immutable revision plus an in-memory generation. Privacy transitions, lock transitions, temporary-sensitive changes/expiry, and policy replacement invalidate older authorization. A later policy reload therefore cannot silently make queued work authorized under an older revision current again. The existing validated configuration manager remains the outer atomic configuration boundary and fails malformed reloads to the repository's established safe configuration semantics. ## Sanitized audit/status behavior Policy decisions contain only control information: operation, phase, allow/deny, certainty, stable reason code, optional rule ID, revision, and generation. They cannot carry screenshots, OCR text, titles, domains, or matched substrings. Status exposes revision/generation, enabled-rule count, privacy/lock state, health, and sanitized error code only. Effective-config inspection masks policy application/title/workspace/domain/source matcher values. Existing audit models reject raw policy-match attributes outside their closed safe field set. Synthetic security fixtures seed recognizable credential/title/domain/command-like values and assert they do not appear in expected sanitized outputs. The repository's detect-secrets baseline remains unchanged; two intentionally synthetic test literals use the scanner-supported inline allowlist pragma only on those fixture lines. ## Configuration/schema impact The existing immutable Pydantic configuration is extended with typed policy selectors, closed enums, stable IDs, timezone/time-window validation, bounds, and `extra='forbid'`. Changes are additive with safe defaults, so schema version 1 remains valid and no persistent migration is required. ## TDD evidence ### RED Commit `f1c6cbda6d56de8dd3d19fa60251ed175705663d`, Actions run `31621658847`, job `94197728739`. Formatting, Ruff, and ShellCheck passed. Strict Pyright then failed specifically because the #17 production policy module/types/config fields did not exist yet. Earlier formatting/lint-only failures were explicitly rejected as invalid red evidence and corrected before recording this red state. ### GREEN Final exact PR head: `c4d850900a6980487a45d7e4eae59c3f8ebc6703`. Actions run `31625941909` is fully green on the exact head: - Ubuntu 22.04 checks: green - Ubuntu 24.04 checks: green - failure-propagation: green - canonical repository gate: green Functional coverage on the final path includes: - unit: 417 passed - contract: 6 passed - integration: 23 passed - security: 49 passed - repository policy: passed - Bandit: passed - detect-secrets: passed - failure-propagation: passed ### Full canonical gate Job `94212233905` ran the repository's canonical command exactly: `./scripts/check` and passed on exact head `c4d850900a6980487a45d7e4eae59c3f8ebc6703`. The execution environment available to this agent could not materialize the private repository checkout locally, so local filesystem execution was not falsely claimed; the exact canonical command was added as a required PR CI job and executed successfully on the tested head. ## Acceptance-criteria mapping - typed authoritative engine: `domain.policy` + `PolicyEngine` - separate screenshot/metadata/OCR/index/summarization/remote gates: operation enum + matrix tests - pre-capture deny prevents pixels: enforcement integration tests - post-capture deny/stale auth prevents persistence: enforcement integration tests - privacy/lock high precedence: unit + integration + security tests - temporary current window/workspace sensitivity: typed scopes, TTL, clear, generation tests - password manager/auth/lock/configured sensitive workspace representation: built-ins/config tests - deterministic deny-first precedence: order/priority/conflict tests - regex/pattern abuse resistance: config bounds + pathological-pattern security tests - domain suffix safety: exact/subdomain/suffix/IP tests - timezone/DST/midnight handling: fixed-time tests - normalized provenance participation: Xorg/Qtile/ActivityWatch integration tests - missing/stale/malformed context fail closed: unit/security tests - atomic safe configuration behavior: malformed reload security test - stale authorization invalidation: revision/privacy/lock tests - sanitized decisions/status/effective config/audit: security tests - no direct storage/provider/capture bypass: structural security test - performance bound: maximum 256-rule deterministic evaluation test - documentation: `docs/policy.md` ## Known limitations / boundaries - Desktop lock/idle event acquisition itself belongs to the lifecycle/idle work tracked separately; this PR provides and tests the lock policy state/gate but does not implement #18. - Metadata source composition/conflict resolution remains owned by the normalized metadata composition layer from #13; policy intentionally consumes only its normalized result and fails closed when trustworthy required context is unavailable. - The richer CLI/desktop UI for temporary sensitivity is intentionally out of scope; this PR provides the typed internal/control boundary it can call. - Policy allow is never sufficient by itself: redaction, encryption, lifecycle validation, retention/storage policy, and provider routing remain mandatory independent controls.