[P2] Expose policy-gated visual context for Zara #60

New issue

Closed

opened 2026-07-19 06:38:26 +00:00 by lost-rob0t · 2 comments

lost-rob0t commented

2026-07-19 06:38:26 +00:00

(Migrated from github.com)

Goal

Let Zara answer commands such as “Zara, explain what you see” by querying Local Recall through a narrow owner-only interface.

Local Recall remains the authority for capture selection, lifecycle state, redaction, retrieval, provider routing, and audit. Zara is a client; this is not a codebase merger and Zara must not read screenshot files or storage directly.

User flow

The owner asks Zara to explain the current or recent desktop context.
Zara sends a typed request over the owner-only Local Recall IPC client.
Local Recall verifies lifecycle/query policy and selects the minimum approved context.
Local Recall performs local analysis by default, or uses an explicitly authorized remote provider through the existing egress gate.
Local Recall returns a bounded textual answer plus sanitized provenance.
Zara speaks or displays the answer without retaining image data.

Local Recall requirements

Define a versioned typed request/response contract for visual-context explanation.
Support at least current, recent, and explicit bounded time-window selectors.
Use the existing owner-only ipc:// query boundary; no TCP listener.
Authenticate the client and enforce request deadlines, size limits, and cancellation.
Refuse the request when capture/query policy, lock state, privacy mode, missing context, or provider policy does not permit it.
Select and decrypt only the minimum working set required for the request.
Keep raw pixels, raw OCR, and decrypted records memory-only.
Run deterministic redaction before any provider request.
Default to the configured local provider; never silently fall back to remote.
Route remote analysis only through EgressGate with explicit per-query authorization.
Return text plus sanitized fields such as opaque request ID, time range, record count, provider class, and confidence/provenance indicators.
Never return screenshot bytes, raw OCR, window titles, command lines, usernames, secrets, provider prompts, or storage paths to Zara.
Emit sanitized audit events for request acceptance/rejection, context selection, provider decision, completion, cancellation, and failure.
Do not reactivate or alter capture state while answering a query.

Suggested contract

ExplainVisualContextRequest
- protocol_version
- request_id
- selector: current | recent | bounded_window
- start/end when bounded
- maximum_records
- deadline
- remote_authorization: absent | explicit

ExplainVisualContextResponse
- request_id
- outcome
- explanation
- selected_time_range
- record_count
- provider_class: local | authorized_remote
- provenance/confidence summary
- sanitized reason code on rejection/failure

The concrete transport schema must use project-owned validated types rather than arbitrary dictionaries.

Security invariants

Zara receives explanation text, not captured artifacts.
Zara cannot bypass lifecycle, retrieval, redaction, provider-routing, or audit boundaries.
Request text and generated answers must not enter sanitized audit logs.
Debug mode cannot expose visual content or provider payloads.
Timeout, disconnect, malformed request, provider failure, and cancellation fail closed and destroy decrypted working data.

Acceptance criteria

An owner-authenticated client can request “current desktop explanation” and receive a bounded textual response from a mocked local provider.
Locked, private, unauthorized-remote, missing-context, malformed, oversized, expired, and cancelled requests return stable sanitized outcomes.
Tests prove query work cannot start or resume capture.
Tests seed screenshot text, OCR, titles, usernames, command lines, secrets, and prompts and verify none appear in IPC metadata or audit logs.
Remote provider tests prove explicit authorization and EgressGate are mandatory.
Zara can implement its client against documented fixtures without importing Local Recall internals.
Integration documentation identifies the Zara companion issue and protocol versioning rules.

Depends on

#12
Query/retrieval, provider routing, and IPC implementation issues required by the existing roadmap.

Ordering

Backlog integration work. Do not implement before the existing numbered Local Recall issue sequence reaches it.

## Goal Let Zara answer commands such as **“Zara, explain what you see”** by querying Local Recall through a narrow owner-only interface. Local Recall remains the authority for capture selection, lifecycle state, redaction, retrieval, provider routing, and audit. Zara is a client; this is not a codebase merger and Zara must not read screenshot files or storage directly. ## User flow 1. The owner asks Zara to explain the current or recent desktop context. 2. Zara sends a typed request over the owner-only Local Recall IPC client. 3. Local Recall verifies lifecycle/query policy and selects the minimum approved context. 4. Local Recall performs local analysis by default, or uses an explicitly authorized remote provider through the existing egress gate. 5. Local Recall returns a bounded textual answer plus sanitized provenance. 6. Zara speaks or displays the answer without retaining image data. ## Local Recall requirements - Define a versioned typed request/response contract for visual-context explanation. - Support at least `current`, `recent`, and explicit bounded time-window selectors. - Use the existing owner-only `ipc://` query boundary; no TCP listener. - Authenticate the client and enforce request deadlines, size limits, and cancellation. - Refuse the request when capture/query policy, lock state, privacy mode, missing context, or provider policy does not permit it. - Select and decrypt only the minimum working set required for the request. - Keep raw pixels, raw OCR, and decrypted records memory-only. - Run deterministic redaction before any provider request. - Default to the configured local provider; never silently fall back to remote. - Route remote analysis only through `EgressGate` with explicit per-query authorization. - Return text plus sanitized fields such as opaque request ID, time range, record count, provider class, and confidence/provenance indicators. - Never return screenshot bytes, raw OCR, window titles, command lines, usernames, secrets, provider prompts, or storage paths to Zara. - Emit sanitized audit events for request acceptance/rejection, context selection, provider decision, completion, cancellation, and failure. - Do not reactivate or alter capture state while answering a query. ## Suggested contract ```text ExplainVisualContextRequest - protocol_version - request_id - selector: current | recent | bounded_window - start/end when bounded - maximum_records - deadline - remote_authorization: absent | explicit ExplainVisualContextResponse - request_id - outcome - explanation - selected_time_range - record_count - provider_class: local | authorized_remote - provenance/confidence summary - sanitized reason code on rejection/failure ``` The concrete transport schema must use project-owned validated types rather than arbitrary dictionaries. ## Security invariants - Zara receives explanation text, not captured artifacts. - Zara cannot bypass lifecycle, retrieval, redaction, provider-routing, or audit boundaries. - Request text and generated answers must not enter sanitized audit logs. - Debug mode cannot expose visual content or provider payloads. - Timeout, disconnect, malformed request, provider failure, and cancellation fail closed and destroy decrypted working data. ## Acceptance criteria - An owner-authenticated client can request “current desktop explanation” and receive a bounded textual response from a mocked local provider. - Locked, private, unauthorized-remote, missing-context, malformed, oversized, expired, and cancelled requests return stable sanitized outcomes. - Tests prove query work cannot start or resume capture. - Tests seed screenshot text, OCR, titles, usernames, command lines, secrets, and prompts and verify none appear in IPC metadata or audit logs. - Remote provider tests prove explicit authorization and `EgressGate` are mandatory. - Zara can implement its client against documented fixtures without importing Local Recall internals. - Integration documentation identifies the Zara companion issue and protocol versioning rules. ## Depends on - #12 - Query/retrieval, provider routing, and IPC implementation issues required by the existing roadmap. ## Ordering Backlog integration work. Do **not** implement before the existing numbered Local Recall issue sequence reaches it.

lost-rob0t commented

2026-07-19 06:39:06 +00:00

(Migrated from github.com)

Zara-side companion issue created: lost-rob0t/zara#56. Local Recall remains the protocol and policy authority; Zara is a blocked client integration and should not be implemented ahead of the Local Recall roadmap.

lost-rob0t commented

2026-08-30 11:56:37 +00:00

(Migrated from github.com)

Implemented and merged via PR #100. Versioned typed contract (zara-visual-context-v1), owner-only IPC transport over the existing session-token boundary, local-first vision with EgressGate-mandatory remote, minimum working set held memory-only, sanitized visual_context_request audit action, and stable sanitized denial outcomes for locked/private/policy/missing-context/expired/cancelled/malformed/oversized requests. Seeded canary probes prove titles/OCR/secrets/commands/usernames/prompts never reach responses or audit; explain cannot start or resume capture. Zara client fixtures + protocol versioning rules documented in docs/zara-visual-context.md. Full suite 1155 passing; canonical gate green. RAGE: rage/issue-60-zara-visual-context.org.