Implement ActivityWatch metadata adapter #68

Merged

lost-rob0t merged 21 commits from agent/issue-16-activitywatch-metadata into main

2026-08-12 15:56:57 +00:00

lost-rob0t commented

2026-08-12 14:37:52 +00:00

(Migrated from github.com)

Problem and scope

Implements #16 as an optional ActivityWatch metadata enrichment source. It correlates a tiny local ActivityWatch window with the current Local Recall capture; it is not an ActivityWatch importer, history synchronizer, persistence layer, or provider connector.

Closes #16.

Architecture

Adds ActivityWatchMetadataSource behind the existing MetadataSource / ContextMetadata boundary.
Adds typed ActivityWatch server, bucket, event, transport, and failure models; external JSON is parsed at the boundary rather than propagated as arbitrary mappings.
Preserves existing canonical fields, provenance, per-field confidence, adapter revisions, metadata composition, and redaction-before-persistence semantics.
Adds truthful ActivityWatch capability reporting and a dedicated DOMAIN metadata capability without changing Xorg/Qtile behavior.

Transport and security model

Default origin: loopback HTTP 127.0.0.1:5600.
Configuration accepts only bare loopback HTTP origins for 127.0.0.1, localhost, or ::1; remote hosts, HTTPS, credentials, base paths, queries, and fragments are rejected.
Direct bounded asyncio sockets are used. HTTP proxy environment variables are not consumed and redirects are never followed.
Fixed request targets only; no export-all or unbounded Query API path.
Explicit bounds cover connect/request deadlines, 16 KiB headers, response bodies, 32 inspected buckets, 8 candidates/type, 16 events/bucket, string lengths, event duration, <=10 s query span, and cancellation/resource closure.
Duplicate JSON keys, malformed JSON/types, redirects, transfer encoding, duplicate/invalid Content-Length, oversized/truncated responses, and unexpected HTTP statuses fail closed with fixed sanitized codes.
Security regression verifies ActivityWatch adapter modules have no persistence, provider, audit, or crypto import path.

Bucket discovery

Bucket IDs are treated as opaque. Discovery uses validated bucket metadata/event type and host metadata rather than username/hostname-derived IDs. Compatible types include current-window, AFK status, and optionally current web tab. Local-host candidates are preferred. Cross-host ambiguity fails closed; same-host duplicates are bounded and resolved deterministically from correlated events.

Correlation semantics

Default correlation tolerance is 2 seconds and configurable up to 5 seconds.
Only the tiny interval around the capture instant is queried.
Timestamps must be timezone-aware; durations must be finite, non-negative, and bounded.
Exact/zero-duration, overlapping, immediately-before, and immediately-after events are handled deterministically.
Stale, far-future, malformed, duplicate, and out-of-order results are covered.
An old long-running event cannot become current solely because its duration spans the capture instant; the event start must also satisfy the correlation tolerance.
Overlap conflicts prefer the closest event and then newest start deterministically.

URL/domain privacy

URL collection defaults to disabled.
The only enabled URL mode is domain-only; there is no full-URL mode.
Domain-only parsing discards path, query, fragment, credentials, and raw URL, and rejects IP-literal or malformed hosts.
Window-title collection remains independently disabled by default.

AFK and normalized metadata

app -> canonical application.
title -> canonical window.title when title collection is enabled.
Only afk / not-afk are accepted and normalized to bounded boolean idle.
Browser host -> url.domain only in domain-only mode.
Every retained value carries activitywatch provenance and the stable adapter revision.

Fallback and privacy boundary

ActivityWatch is optional. Unavailability/degradation produces sanitized failure state and does not disable a valid configured fallback such as generic Xorg. Synthetic integration coverage proves ActivityWatch metadata still crosses Local Recall's deterministic redaction boundary before downstream persistence eligibility. Disabled title/domain fields never appear downstream. ActivityWatch has no direct persistence or model-provider path.

TDD evidence

A test-only red phase was pushed before production implementation. CI run #274 passed formatting/lint/shell checks and then failed strict Pyright specifically because the ActivityWatch production config/types/source did not exist. Production code was added only after that observed red state.

Regression discoveries during implementation received tests for strict typing, hostile transport behavior, stale/duplicate correlation, old long-running events, future events, AFK conflicts, capability gating, redaction, fallback, and architecture boundaries.

Green evidence

The one-shot finalizer applied the last correlation/privacy regressions, removed its own temporary workflow/helper files, formatted the clean tree, and ran the canonical ./scripts/check successfully in Actions run 31614570141 before committing the final tree.

Final PR head 6e88ab24688147cad557f5c4062568f55ccd7d1a is an empty commit over that exact tested tree and passed normal CI run 31614770100 on Ubuntu 22.04 and 24.04, including:

Ruff formatting and lint;
ShellCheck;
strict Pyright;
unit tests;
contract tests;
integration tests;
security tests;
repository security checks (Bandit/secret scanning/policy as defined by the repo);
failure-propagation verification.

Acceptance criteria mapping

Loopback-only bounded local API transport: implemented and security-tested.
Dynamic bucket discovery with deterministic host/candidate selection: implemented and tested.
Tiny capture-correlation window only: implemented and tested.
Application/title/AFK/domain normalization: implemented with canonical fields and typed AFK status.
URLs disabled by default and domain-only when enabled: implemented and tested.
Malformed/stale/duplicate/future/overlap events: covered.
ActivityWatch failure with generic fallback: covered.
Local Recall redaction remains authoritative: integration-tested.
Synthetic unit/contract/integration/security coverage: added; no real ActivityWatch data required.
Sanitized errors/status and no raw payload persistence path: tested structurally and behaviorally.
Documentation: docs/activitywatch.md plus configuration cross-reference.

Known limitations

ActivityWatch's API remains version-unstable, so unsupported future schemas fail closed or degrade the optional source. This implementation intentionally does not copy the ActivityWatch database, synchronize history, collect full URLs, or contact a real ActivityWatch instance in CI. No real local smoke test was required or performed.

## Problem and scope Implements #16 as an optional ActivityWatch metadata enrichment source. It correlates a tiny local ActivityWatch window with the current Local Recall capture; it is not an ActivityWatch importer, history synchronizer, persistence layer, or provider connector. Closes #16. ## Architecture - Adds `ActivityWatchMetadataSource` behind the existing `MetadataSource` / `ContextMetadata` boundary. - Adds typed ActivityWatch server, bucket, event, transport, and failure models; external JSON is parsed at the boundary rather than propagated as arbitrary mappings. - Preserves existing canonical fields, provenance, per-field confidence, adapter revisions, metadata composition, and redaction-before-persistence semantics. - Adds truthful ActivityWatch capability reporting and a dedicated `DOMAIN` metadata capability without changing Xorg/Qtile behavior. ## Transport and security model - Default origin: loopback HTTP `127.0.0.1:5600`. - Configuration accepts only bare loopback HTTP origins for `127.0.0.1`, `localhost`, or `::1`; remote hosts, HTTPS, credentials, base paths, queries, and fragments are rejected. - Direct bounded asyncio sockets are used. HTTP proxy environment variables are not consumed and redirects are never followed. - Fixed request targets only; no export-all or unbounded Query API path. - Explicit bounds cover connect/request deadlines, 16 KiB headers, response bodies, 32 inspected buckets, 8 candidates/type, 16 events/bucket, string lengths, event duration, <=10 s query span, and cancellation/resource closure. - Duplicate JSON keys, malformed JSON/types, redirects, transfer encoding, duplicate/invalid Content-Length, oversized/truncated responses, and unexpected HTTP statuses fail closed with fixed sanitized codes. - Security regression verifies ActivityWatch adapter modules have no persistence, provider, audit, or crypto import path. ## Bucket discovery Bucket IDs are treated as opaque. Discovery uses validated bucket metadata/event type and host metadata rather than username/hostname-derived IDs. Compatible types include current-window, AFK status, and optionally current web tab. Local-host candidates are preferred. Cross-host ambiguity fails closed; same-host duplicates are bounded and resolved deterministically from correlated events. ## Correlation semantics - Default correlation tolerance is 2 seconds and configurable up to 5 seconds. - Only the tiny interval around the capture instant is queried. - Timestamps must be timezone-aware; durations must be finite, non-negative, and bounded. - Exact/zero-duration, overlapping, immediately-before, and immediately-after events are handled deterministically. - Stale, far-future, malformed, duplicate, and out-of-order results are covered. - An old long-running event cannot become current solely because its duration spans the capture instant; the event start must also satisfy the correlation tolerance. - Overlap conflicts prefer the closest event and then newest start deterministically. ## URL/domain privacy - URL collection defaults to `disabled`. - The only enabled URL mode is `domain-only`; there is no full-URL mode. - Domain-only parsing discards path, query, fragment, credentials, and raw URL, and rejects IP-literal or malformed hosts. - Window-title collection remains independently disabled by default. ## AFK and normalized metadata - `app` -> canonical `application`. - `title` -> canonical `window.title` when title collection is enabled. - Only `afk` / `not-afk` are accepted and normalized to bounded boolean `idle`. - Browser host -> `url.domain` only in domain-only mode. - Every retained value carries `activitywatch` provenance and the stable adapter revision. ## Fallback and privacy boundary ActivityWatch is optional. Unavailability/degradation produces sanitized failure state and does not disable a valid configured fallback such as generic Xorg. Synthetic integration coverage proves ActivityWatch metadata still crosses Local Recall's deterministic redaction boundary before downstream persistence eligibility. Disabled title/domain fields never appear downstream. ActivityWatch has no direct persistence or model-provider path. ## TDD evidence A test-only red phase was pushed before production implementation. CI run #274 passed formatting/lint/shell checks and then failed strict Pyright specifically because the ActivityWatch production config/types/source did not exist. Production code was added only after that observed red state. Regression discoveries during implementation received tests for strict typing, hostile transport behavior, stale/duplicate correlation, old long-running events, future events, AFK conflicts, capability gating, redaction, fallback, and architecture boundaries. ## Green evidence The one-shot finalizer applied the last correlation/privacy regressions, removed its own temporary workflow/helper files, formatted the clean tree, and ran the canonical `./scripts/check` successfully in Actions run `31614570141` before committing the final tree. Final PR head `6e88ab24688147cad557f5c4062568f55ccd7d1a` is an empty commit over that exact tested tree and passed normal CI run `31614770100` on Ubuntu 22.04 and 24.04, including: - Ruff formatting and lint; - ShellCheck; - strict Pyright; - unit tests; - contract tests; - integration tests; - security tests; - repository security checks (Bandit/secret scanning/policy as defined by the repo); - failure-propagation verification. ## Acceptance criteria mapping - Loopback-only bounded local API transport: implemented and security-tested. - Dynamic bucket discovery with deterministic host/candidate selection: implemented and tested. - Tiny capture-correlation window only: implemented and tested. - Application/title/AFK/domain normalization: implemented with canonical fields and typed AFK status. - URLs disabled by default and domain-only when enabled: implemented and tested. - Malformed/stale/duplicate/future/overlap events: covered. - ActivityWatch failure with generic fallback: covered. - Local Recall redaction remains authoritative: integration-tested. - Synthetic unit/contract/integration/security coverage: added; no real ActivityWatch data required. - Sanitized errors/status and no raw payload persistence path: tested structurally and behaviorally. - Documentation: `docs/activitywatch.md` plus configuration cross-reference. ## Known limitations ActivityWatch's API remains version-unstable, so unsupported future schemas fail closed or degrade the optional source. This implementation intentionally does not copy the ActivityWatch database, synchronize history, collect full URLs, or contact a real ActivityWatch instance in CI. No real local smoke test was required or performed.