nsaspy/starintel-server

Fork 0

#32 authenticated target lease API and authoritative fencing #101

Open

lost-rob0t wants to merge 58 commits from feature/issue-32-target-lease-api into master

lost-rob0t commented

2026-08-16 15:12:34 +00:00

(Migrated from github.com)

Work in progress for #32. The branch targets canonical master; dev is not used.

Implemented

shared backend-neutral target lease application service used by embedded callers and HTTP;
trusted request principal + tenant/dataset/program/namespace/target/actor authorization;
server-stored dataset binding used for post-read authorization and hidden-scope filtering;
logical idempotency request IDs for acquire/renew/release;
process-owned Valkey lease runtime lifecycle;
canonical /api/v1/target-leases acquire/renew/release/get/list/revoke routes;
typed service-to-HTTP result translation;
atomic Valkey fenced-intent commit with exact lease/owner/service-instance/token/expiry checks;
immutable fenced-intent readback through the lease-store API, with real-Valkey coverage proving stale/forged receipt IDs do not resolve and committed values round-trip exactly;
target acceptance helpers that resolve caller lease locators through current-target-lease-authority, carry only the server-owned lease record into dispatch envelopes, and fence before persistence/scheduling;
delayed target side effects re-establish fencing immediately before local/Rabbit dispatch;
legacy/unfenced target acceptance now fails closed instead of fabricating a lease id or fencing token;
deterministic hermetic and real-Valkey stale-N/current-N fencing regressions;
standalone/container runtime refreshes the live Valkey configuration instead of retaining dumped build-time environment values;
Valkey password-file staging is restart-safe without weakening the root-owned Compose secret.

Remaining blocker: trusted target-ingress provenance

#32 is not complete.

The production target path is still POST /new/target/:actor -> documents.ingest.target -> CouchDB durable outbox -> documents.new.target.* -> target consumer. The authenticated HTTP publish boundary creates server-owned Rabbit service-context/authorization headers, but couchdb-process-outbox-mutation persists the document/event payload and publish-outbox-event later republishes the target event with only content type, delivery mode, and message id. By the time handle-target / accept-target-delivery runs, the trusted principal/service context needed by current-target-lease-authority is gone.

The fenced-intent readback substrate now gives the target consumer a way to verify a server-created immutable receipt, but the receipt is not yet wired across the durable outbox path and receipt existence alone is never authority. The consumer must still resolve the current lease and require an exact receipt value match before persistence, scheduling, or publication.

Do not fix this by trusting lease/principal fields copied from the target document or arbitrary Rabbit headers. Lease id/fencing token are locators, not authority. The next coherent slice must preserve or re-establish a server-owned authenticated provenance receipt across the durable outbox boundary, or move authoritative lease resolution to an earlier authenticated acceptance boundary while preserving crash-safe target persistence/recovery semantics.

The old recovery path also still carries legacy lease_owner / lease_expires_at fields and calls compatibility submit-target; it now fails closed rather than inventing authority, but #33 still needs startup reconciliation against the real lease store.

A separate Valkey preflight followed by an unconditional CouchDB/Rabbit write remains a TOCTOU race and is not an acceptable solution.

Work in progress for #32. The branch targets canonical `master`; `dev` is not used. ## Implemented - shared backend-neutral target lease application service used by embedded callers and HTTP; - trusted request principal + tenant/dataset/program/namespace/target/actor authorization; - server-stored dataset binding used for post-read authorization and hidden-scope filtering; - logical idempotency request IDs for acquire/renew/release; - process-owned Valkey lease runtime lifecycle; - canonical `/api/v1/target-leases` acquire/renew/release/get/list/revoke routes; - typed service-to-HTTP result translation; - atomic Valkey fenced-intent commit with exact lease/owner/service-instance/token/expiry checks; - immutable fenced-intent readback through the lease-store API, with real-Valkey coverage proving stale/forged receipt IDs do not resolve and committed values round-trip exactly; - target acceptance helpers that resolve caller lease locators through `current-target-lease-authority`, carry only the server-owned lease record into dispatch envelopes, and fence before persistence/scheduling; - delayed target side effects re-establish fencing immediately before local/Rabbit dispatch; - legacy/unfenced target acceptance now fails closed instead of fabricating a lease id or fencing token; - deterministic hermetic and real-Valkey stale-N/current-N fencing regressions; - standalone/container runtime refreshes the live Valkey configuration instead of retaining dumped build-time environment values; - Valkey password-file staging is restart-safe without weakening the root-owned Compose secret. ## Remaining blocker: trusted target-ingress provenance #32 is **not complete**. The production target path is still `POST /new/target/:actor` -> `documents.ingest.target` -> CouchDB durable outbox -> `documents.new.target.*` -> target consumer. The authenticated HTTP publish boundary creates server-owned Rabbit service-context/authorization headers, but `couchdb-process-outbox-mutation` persists the document/event payload and `publish-outbox-event` later republishes the target event with only content type, delivery mode, and message id. By the time `handle-target` / `accept-target-delivery` runs, the trusted principal/service context needed by `current-target-lease-authority` is gone. The fenced-intent readback substrate now gives the target consumer a way to verify a server-created immutable receipt, but the receipt is **not yet wired across the durable outbox path** and receipt existence alone is never authority. The consumer must still resolve the current lease and require an exact receipt value match before persistence, scheduling, or publication. Do **not** fix this by trusting lease/principal fields copied from the target document or arbitrary Rabbit headers. Lease id/fencing token are locators, not authority. The next coherent slice must preserve or re-establish a server-owned authenticated provenance receipt across the durable outbox boundary, or move authoritative lease resolution to an earlier authenticated acceptance boundary while preserving crash-safe target persistence/recovery semantics. The old recovery path also still carries legacy `lease_owner` / `lease_expires_at` fields and calls compatibility `submit-target`; it now fails closed rather than inventing authority, but #33 still needs startup reconciliation against the real lease store. A separate Valkey preflight followed by an unconditional CouchDB/Rabbit write remains a TOCTOU race and is not an acceptable solution.

lost-rob0t commented

2026-08-25 12:18:05 +00:00

(Migrated from github.com)

RAGE/ADADR governance reconciliation (2026-08-25): this PR is still draft and currently non-mergeable at head 9dd127ca71bc1be6cc52a95cd8223011221c4e90. The canonical Auto-Research design linked to issue #32 is roam/design/star-server/STAR-SERVER-032-authenticated-target-lease-http-execution.org. Under the current repository-wide AGENTS.md, every dedicated design requires the canonical Approval Table and implementation may proceed only from explicit recorded design approval. STAR-SERVER-032 currently has no Approval Table, so its historical existence and this branch's previously green exact-head CI do not establish implementation authority for the unresolved trusted-provenance/durable-outbox slice.

Do not treat this as a request to weaken or discard the existing fencing regressions. The implemented RED/green evidence remains useful. Before realizing the remaining provenance/outbox architecture, reconcile STAR-SERVER-032 through the authoritative Auto-Research workflow, preserve any real historical approval evidence without inventing it, obtain explicit operator design approval for the exact remaining scope, then implement from current master with TDD-first verification. A blind rebase-and-merge of this stale branch is not authorized.

RAGE/ADADR governance reconciliation (2026-08-25): this PR is still draft and currently non-mergeable at head `9dd127ca71bc1be6cc52a95cd8223011221c4e90`. The canonical Auto-Research design linked to issue #32 is `roam/design/star-server/STAR-SERVER-032-authenticated-target-lease-http-execution.org`. Under the current repository-wide `AGENTS.md`, every dedicated design requires the canonical Approval Table and implementation may proceed only from explicit recorded design approval. `STAR-SERVER-032` currently has no Approval Table, so its historical existence and this branch's previously green exact-head CI do not establish implementation authority for the unresolved trusted-provenance/durable-outbox slice. Do not treat this as a request to weaken or discard the existing fencing regressions. The implemented RED/green evidence remains useful. Before realizing the remaining provenance/outbox architecture, reconcile `STAR-SERVER-032` through the authoritative Auto-Research workflow, preserve any real historical approval evidence without inventing it, obtain explicit operator design approval for the exact remaining scope, then implement from current `master` with TDD-first verification. A blind rebase-and-merge of this stale branch is not authorized.

nsaspy changed title from ~~WIP: #32 authenticated target lease API and authoritative fencing~~ to #32 authenticated target lease API and authoritative fencing

2026-08-28 03:29:46 +00:00

nsaspy referenced this pull request

2026-08-28 03:29:46 +00:00

Add v1 capability discovery endpoint #102

nsaspy referenced this pull request

2026-08-28 03:29:46 +00:00

Own server lifecycle and expose truthful readiness #103

This pull request has changes conflicting with the target branch.

source/starintel-gserver.asd
source/target-dispatch-fixes.lisp

View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Merge commit title

Merge pull request '#32 authenticated target lease API and authoritative fencing' (#101) from feature/issue-32-target-lease-api into master

Merge commit body

Reviewed-on: https://git.starintel.actor/nsaspy/starintel-server/pulls/101

Merge pull request '#32 authenticated target lease API and authoritative fencing' (#101) from feature/issue-32-target-lease-api into master

Reviewed-on: https://git.starintel.actor/nsaspy/starintel-server/pulls/101

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin feature/issue-32-target-lease-api:feature/issue-32-target-lease-api

git switch feature/issue-32-target-lease-api

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch master

git merge --no-ff feature/issue-32-target-lease-api

git switch feature/issue-32-target-lease-api

git rebase master

git switch master

git merge --ff-only feature/issue-32-target-lease-api

git switch feature/issue-32-target-lease-api

git rebase master

git switch master

git merge --no-ff feature/issue-32-target-lease-api

git switch master

git merge --squash feature/issue-32-target-lease-api