Add local vision-model analysis for redacted screenshots #92

Merged

lost-rob0t merged 2 commits from agent/issue-33-vision-analysis into main

2026-08-30 04:05:19 +00:00

lost-rob0t commented

2026-08-30 04:03:11 +00:00

(Migrated from github.com)

Closes #33.

RAGE transaction for vision analysis of redacted screenshots. Starting main: the #32 squash merge.

Completed:

single vision provider abstraction over the existing ModelCapability.VISION/supports_vision contract, supporting local (default) and remote providers through one architecture;
pipeline invariant enforced at the type boundary: only RedactedFrame can reach any provider; raw frames are rejected at construction and through tampered request objects (tests assert empty provider request logs after tampering);
remote vision runs only through the existing EgressGate with a per-query EgressAuthorization bound to the exact provider and granting REDACTED_IMAGE; no grant, wrong provider, or missing gate is refused with sanitized reasons; no silent local-to-remote fallback;
closed typed output schema (record link, provider ID, model version, visible application state, document type, broad task, uncertainty) with no identity/biometric/hidden-content fields and boundary validation of malformed values;
bounded image input budget; enrich_optional keeps provider unavailability non-blocking so OCR-based capture continues;
documentation: docs/vision.md.

Acceptance evidence (exact head of this PR):

local VLM enriches synthetic redacted records without network access;
provider unavailability does not block capture (optional path returns no analysis);
outputs schema-validated and linked to the exact record and model version;
unredacted frames cannot be passed to the vision provider (construction + tampered-request tests).

Canonical ./scripts/check on exact head: 923 tests passed, 8 failure modes verified, Bandit clean, detect-secrets clean, ruff clean, strict Pyright clean. RAGE log: rage/issue-33-vision-analysis.org.

Closes #33. RAGE transaction for vision analysis of redacted screenshots. Starting main: the #32 squash merge. Completed: - single vision provider abstraction over the existing `ModelCapability.VISION`/`supports_vision` contract, supporting local (default) and remote providers through one architecture; - pipeline invariant enforced at the type boundary: only `RedactedFrame` can reach any provider; raw frames are rejected at construction and through tampered request objects (tests assert empty provider request logs after tampering); - remote vision runs only through the existing `EgressGate` with a per-query `EgressAuthorization` bound to the exact provider and granting `REDACTED_IMAGE`; no grant, wrong provider, or missing gate is refused with sanitized reasons; no silent local-to-remote fallback; - closed typed output schema (record link, provider ID, model version, visible application state, document type, broad task, uncertainty) with no identity/biometric/hidden-content fields and boundary validation of malformed values; - bounded image input budget; `enrich_optional` keeps provider unavailability non-blocking so OCR-based capture continues; - documentation: `docs/vision.md`. Acceptance evidence (exact head of this PR): - local VLM enriches synthetic redacted records without network access; - provider unavailability does not block capture (optional path returns no analysis); - outputs schema-validated and linked to the exact record and model version; - unredacted frames cannot be passed to the vision provider (construction + tampered-request tests). Canonical `./scripts/check` on exact head: 923 tests passed, 8 failure modes verified, Bandit clean, detect-secrets clean, ruff clean, strict Pyright clean. RAGE log: `rage/issue-33-vision-analysis.org`.