[P1] Implement local generation-provider support with Ollama first #21

New issue

Closed

opened 2026-07-18 00:01:11 +00:00 by lost-rob0t · 0 comments

lost-rob0t commented

2026-07-18 00:01:11 +00:00

(Migrated from github.com)

Goal

Run summarization and question answering locally on ordinary hardware using approximately 8B-class models.

Requirements

Implement the shared generation-provider contract for Ollama.
Capability discovery for model context size, vision support, structured output, and availability.
Configurable model roles for extraction, summarization, and answering.
Concurrency, timeout, cancellation, and memory limits appropriate for CPU or modest GPU systems.
Prompts must instruct models not to reproduce or retain detected secrets, but deterministic redaction remains authoritative.
No implicit model download or network request while capture is active.

Acceptance criteria

A configured 7B–9B instruct model can summarize synthetic activity records and answer a time-scoped query.
Provider cancellation works when capture/query operations are stopped.
Model responses are schema validated before use.
The system remains usable when no generation model is available; capture and indexing can continue according to policy.

Depends on

## Goal Run summarization and question answering locally on ordinary hardware using approximately 8B-class models. ## Requirements - Implement the shared generation-provider contract for Ollama. - Capability discovery for model context size, vision support, structured output, and availability. - Configurable model roles for extraction, summarization, and answering. - Concurrency, timeout, cancellation, and memory limits appropriate for CPU or modest GPU systems. - Prompts must instruct models not to reproduce or retain detected secrets, but deterministic redaction remains authoritative. - No implicit model download or network request while capture is active. ## Acceptance criteria - A configured 7B–9B instruct model can summarize synthetic activity records and answer a time-scoped query. - Provider cancellation works when capture/query operations are stopped. - Model responses are schema validated before use. - The system remains usable when no generation model is available; capture and indexing can continue according to policy. ## Depends on - #5 - #9 - #11