Implement local embeddings and encrypted semantic index #63

Merged

lost-rob0t merged 1 commit from agent/issue-22-local-embeddings into main

2026-08-12 00:14:42 +00:00

lost-rob0t commented

2026-07-22 09:08:47 +00:00

(Migrated from github.com)

Hey GPT-5.6 here.

Closes #22.

Scope

add a loopback-only Ollama embedding provider with bounded batching, concurrency, request sizes, deadlines, cancellation, and strict response validation
accept only deterministically redacted content and approved metadata
add an owner-only XChaCha20-Poly1305 encrypted semantic-index snapshot containing vectors, record references, timestamps, model identity, and dimensions
add time-bounded cosine search with model/dimension compatibility checks
add encrypted resumable rebuild checkpoints so cancellation and model migration never replace the active index early
document the privacy boundary and rebuild behavior

TDD evidence

Red phase:

PYTHONPATH=src /tmp/local-recall-venv/bin/python -m pytest -q \
  tests/unit/providers/test_ollama_embeddings.py \
  tests/unit/index/test_semantic_index.py
# non-zero: OllamaEmbeddingProvider and local_recall.index did not exist

Green phase:

PYTHONPATH=src /tmp/local-recall-venv/bin/python -m pytest -q \
  tests/unit/providers/test_ollama_embeddings.py \
  tests/unit/index/test_semantic_index.py
# 10 passed in 0.27s

Full repository gate on the exact published file content:

UV_CACHE_DIR=/tmp/uv-cache \
UV_PYTHON_INSTALL_DIR=/tmp/uv-python \
UV_PROJECT_ENVIRONMENT=/tmp/local-recall-venv \
PATH=/tmp/local-recall-venv/bin:$PATH \
./scripts/check
# Ruff format/lint passed
# ShellCheck passed
# strict Pyright: 0 errors
# 231 passed in 1.67s
# verified 8 failure modes
# repository policy, Bandit, and secret scanning passed

Risk

The index is an exact encrypted snapshot rather than an approximate-nearest-neighbor structure, so search is linear in record count. This keeps the v0.1 persistence boundary small and auditable; a later implementation can replace the search structure without exposing vectors or document references. Rebuilds require the same ordered source-record sequence to resume an encrypted checkpoint safely.

CI

Current-head GitHub Actions is pending and remains the final validation gate.

Hey GPT-5.6 here. Closes #22. ## Scope - add a loopback-only Ollama embedding provider with bounded batching, concurrency, request sizes, deadlines, cancellation, and strict response validation - accept only deterministically redacted content and approved metadata - add an owner-only XChaCha20-Poly1305 encrypted semantic-index snapshot containing vectors, record references, timestamps, model identity, and dimensions - add time-bounded cosine search with model/dimension compatibility checks - add encrypted resumable rebuild checkpoints so cancellation and model migration never replace the active index early - document the privacy boundary and rebuild behavior ## TDD evidence Red phase: ``` PYTHONPATH=src /tmp/local-recall-venv/bin/python -m pytest -q \ tests/unit/providers/test_ollama_embeddings.py \ tests/unit/index/test_semantic_index.py # non-zero: OllamaEmbeddingProvider and local_recall.index did not exist ``` Green phase: ``` PYTHONPATH=src /tmp/local-recall-venv/bin/python -m pytest -q \ tests/unit/providers/test_ollama_embeddings.py \ tests/unit/index/test_semantic_index.py # 10 passed in 0.27s ``` Full repository gate on the exact published file content: ``` UV_CACHE_DIR=/tmp/uv-cache \ UV_PYTHON_INSTALL_DIR=/tmp/uv-python \ UV_PROJECT_ENVIRONMENT=/tmp/local-recall-venv \ PATH=/tmp/local-recall-venv/bin:$PATH \ ./scripts/check # Ruff format/lint passed # ShellCheck passed # strict Pyright: 0 errors # 231 passed in 1.67s # verified 8 failure modes # repository policy, Bandit, and secret scanning passed ``` ## Risk The index is an exact encrypted snapshot rather than an approximate-nearest-neighbor structure, so search is linear in record count. This keeps the v0.1 persistence boundary small and auditable; a later implementation can replace the search structure without exposing vectors or document references. Rebuilds require the same ordered source-record sequence to resume an encrypted checkpoint safely. ## CI Current-head GitHub Actions is pending and remains the final validation gate.