API v1: public search and watch-safe stats #110

Merged

lost-rob0t merged 20 commits from agent/issue-109-public-api-stats into master

2026-08-24 16:25:12 +00:00

lost-rob0t commented

2026-08-24 06:50:17 +00:00

(Migrated from github.com)

Closes #109

RAGE slice

Implements the public/private authority split requested for the API overhaul while preserving the existing authenticated legacy/control plane.

Deployment mode

*public-mode* is an init-file setting and defaults to T
STAR_PUBLIC_MODE=false can also provide the pre-init default
example_configs/init.lisp sets *public-mode* t explicitly
when public mode is T, /api/v1/search and /api/v1/stats are anonymous read endpoints
when public mode is NIL, those same versioned routes remain mounted but require authentication
capabilities report public_mode and the effective search/stats authority

Public read plane

GET /api/v1/search
- no bearer credential required when public mode is enabled
- executes through a server-owned synthetic public_reader authorization context
- reuses authorized-search-query so configured dataset scope is applied before backend retrieval
- STAR_PUBLIC_SEARCH_DATASETS controls server-owned visible datasets; defaults to * for the public-server deployment model
- rejects caller dataset / tenant scope overrides
- query length <= 512, result limit <= 50
GET /api/v1/stats
- no bearer credential required when public mode is enabled
- compact aggregate-only response from fixed data/total + data/count_by_dtype views
- document totals, dtype counts, target-like total, service/version/generated timestamp
- 15s public cache policy

Protected plane remains protected

No target, document-create, bulk-ingest, deletion, admin, credential, or lease path becomes anonymous. Existing route-level capability checks remain authoritative.

Contract/discovery

adds public.search.get and stats.get to the canonical HTTP contract/OpenAPI/client manifest
capabilities expose endpoint authority/scopes plus the effective public-mode setting
capabilities advertise stats and the existing OpenAPI surface

TDD / review trail

initial red contract/security tests first: bdf763b
public-mode regression tests added before implementation changes
implementation follows without weakening mutation/auth assertions
service-backed integration coverage proves default anonymous stats/search, private-mode 401s, and 401 for anonymous ingest/target dispatch

Current exact head: 118fc8ce725d84e067d423384719920cda021740

Do not merge unless exact-head unit/integration/contract/CI gates are green.

Closes #109 ## RAGE slice Implements the public/private authority split requested for the API overhaul while preserving the existing authenticated legacy/control plane. ### Deployment mode - `*public-mode*` is an init-file setting and defaults to `T` - `STAR_PUBLIC_MODE=false` can also provide the pre-init default - `example_configs/init.lisp` sets `*public-mode* t` explicitly - when public mode is `T`, `/api/v1/search` and `/api/v1/stats` are anonymous read endpoints - when public mode is `NIL`, those same versioned routes remain mounted but require authentication - capabilities report `public_mode` and the effective search/stats authority ### Public read plane - `GET /api/v1/search` - no bearer credential required when public mode is enabled - executes through a server-owned synthetic `public_reader` authorization context - reuses `authorized-search-query` so configured dataset scope is applied before backend retrieval - `STAR_PUBLIC_SEARCH_DATASETS` controls server-owned visible datasets; defaults to `*` for the public-server deployment model - rejects caller `dataset` / `tenant` scope overrides - query length <= 512, result limit <= 50 - `GET /api/v1/stats` - no bearer credential required when public mode is enabled - compact aggregate-only response from fixed `data/total` + `data/count_by_dtype` views - document totals, dtype counts, target-like total, service/version/generated timestamp - 15s public cache policy ### Protected plane remains protected No target, document-create, bulk-ingest, deletion, admin, credential, or lease path becomes anonymous. Existing route-level capability checks remain authoritative. ### Contract/discovery - adds `public.search.get` and `stats.get` to the canonical HTTP contract/OpenAPI/client manifest - capabilities expose endpoint authority/scopes plus the effective public-mode setting - capabilities advertise stats and the existing OpenAPI surface ### TDD / review trail - initial red contract/security tests first: `bdf763b` - public-mode regression tests added before implementation changes - implementation follows without weakening mutation/auth assertions - service-backed integration coverage proves default anonymous stats/search, private-mode 401s, and 401 for anonymous ingest/target dispatch Current exact head: `118fc8ce725d84e067d423384719920cda021740` Do not merge unless exact-head unit/integration/contract/CI gates are green.

lost-rob0t commented

2026-08-24 15:27:16 +00:00

(Migrated from github.com)

RAGE/TDD diagnostic update for #109:

8e209ca24ae469e815a668dc45305ab1262fee00 changed CI only to retain the service-backed integration log as an artifact; it did not alter production behavior.
That retained RED isolated two independent failures. First, test-public-mode-defaults-enabled used invalid FiveAM syntax ((is star::*public-mode*)), so the test itself failed at compile time. Second, the public search request successfully passed the synthetic public_reader authorization boundary but the stock couchdb:3.5.2 CI service returned 503 Search is not available from _design/search/_search/fts.
Public stats returned 200 with the expected aggregate-only schema. Private-mode search/stats returned 401. Caller dataset/tenant widening attempts returned 400 public_scope_is_server_owned. Anonymous document ingest and target dispatch remained 401.
The FiveAM defect is repaired separately at c5e1e2f08ebeb3ef1c4c07d663cda8ceecba4a6a without weakening the assertion.
Current Apache CouchDB Search documentation confirms _search requires an external Clouseau service. Therefore the remaining RED is a service-backed CI topology deficiency, not evidence to weaken the public-search contract or bypass the authorization test. The next realization slice must provision compatible Clouseau for the integration gate while preserving the existing Dreyfus/Clouseau StarIntel search interface.

Do not merge until the exact current head proves real public-search success against a search-capable backend plus all existing authority-isolation gates.

RAGE/TDD diagnostic update for #109: - `8e209ca24ae469e815a668dc45305ab1262fee00` changed CI only to retain the service-backed integration log as an artifact; it did not alter production behavior. - That retained RED isolated two independent failures. First, `test-public-mode-defaults-enabled` used invalid FiveAM syntax (`(is star::*public-mode*)`), so the test itself failed at compile time. Second, the public search request successfully passed the synthetic `public_reader` authorization boundary but the stock `couchdb:3.5.2` CI service returned 503 `Search is not available` from `_design/search/_search/fts`. - Public stats returned 200 with the expected aggregate-only schema. Private-mode search/stats returned 401. Caller dataset/tenant widening attempts returned 400 `public_scope_is_server_owned`. Anonymous document ingest and target dispatch remained 401. - The FiveAM defect is repaired separately at `c5e1e2f08ebeb3ef1c4c07d663cda8ceecba4a6a` without weakening the assertion. - Current Apache CouchDB Search documentation confirms `_search` requires an external Clouseau service. Therefore the remaining RED is a service-backed CI topology deficiency, not evidence to weaken the public-search contract or bypass the authorization test. The next realization slice must provision compatible Clouseau for the integration gate while preserving the existing Dreyfus/Clouseau StarIntel search interface. Do not merge until the exact current head proves real public-search success against a search-capable backend plus all existing authority-isolation gates.