[P2] Add local vision-model analysis for redacted screenshots #33

New issue

Closed

opened 2026-07-18 00:03:19 +00:00 by lost-rob0t · 0 comments

lost-rob0t commented

2026-07-18 00:03:19 +00:00

(Migrated from github.com)

Goal

Extract higher-level visual context from already-redacted screenshots using an optional local vision model.

Requirements

Vision-capable generation-provider contract or capability extension.
Only redacted screenshots may enter the vision stage.
Local execution by default, with strict CPU/GPU memory and timeout budgets.
Structured output for visible application state, document type, broad task, and uncertainty.
Never infer or persist biometric identity, sensitive personal attributes, or hidden content.
Remote image analysis remains separately denied unless explicitly enabled by routing policy.

Acceptance criteria

A supported local VLM can enrich synthetic records without network access.
Provider unavailability does not block normal OCR-based capture.
Outputs are schema validated and linked to the exact source record and model version.
Tests prove unredacted frame types cannot be passed to the vision provider.

Depends on

## Goal Extract higher-level visual context from already-redacted screenshots using an optional local vision model. ## Requirements - Vision-capable generation-provider contract or capability extension. - Only redacted screenshots may enter the vision stage. - Local execution by default, with strict CPU/GPU memory and timeout budgets. - Structured output for visible application state, document type, broad task, and uncertainty. - Never infer or persist biometric identity, sensitive personal attributes, or hidden content. - Remote image analysis remains separately denied unless explicitly enabled by routing policy. ## Acceptance criteria - A supported local VLM can enrich synthetic records without network access. - Provider unavailability does not block normal OCR-based capture. - Outputs are schema validated and linked to the exact source record and model version. - Tests prove unredacted frame types cannot be passed to the vision provider. ## Depends on - #9 - #21 - #23