starintel-labs/starintel-server
3
0
Fork 0
Code Issues 5 Pull requests 3 Projects Releases Packages Wiki Activity Actions

feat: canonical STAR URI actor identity for target dispatch (STAR-SERVER-041) #2

Open
nsaspy wants to merge 1 commit from agent/star-uri-identity-compat into master
pull from: agent/star-uri-identity-compat
merge into: starintel-labs:master
Conversation 0 Commits 1 Files changed 16 +1387 -37
nsaspy commented 2026-09-06 18:54:20 +00:00
Owner
Copy link

Summary

gserver did not previously understand canonical STAR identities at all: target dispatch resolved actors by bare actor-name strings only (target["actor"] → resolve-target-destination(actor-string) → get-dest-actor → local Sento actor, or actors.<actor>.new.target / documents.target.dispatch.<actor> when no local actor exists). There was no star:// parsing, canonicalization, authority ownership, resource-kind validation, or STAR URI actor resolution anywhere in source/ at base commit 8fb297d (verified by inspection and by STAR-RESEARCH-060's source audit).

This PR bridges that gap with a small, explicitly temporary Common Lisp STAR URI compatibility layer so BBPD can migrate safely, per the implementation slot recorded in STAR-SERVER-041 (starintel-labs/starintel-server issue #1).

Research / design documents followed (normative)

From lost-rob0t/starintel-auto-research:

  • roam/research/star-server/STAR-RESEARCH-041-unified-star-uri-authority-routing-and-fediwatch-rabbitmq.org (APPROVED) — RFC 3986 direction, canonicalization rules, legacy-input-only rule, gserver-owned authority, RabbitMQ durable dataflow.
  • roam/design/star-server/STAR-SERVER-041-star-uri-authority-routing-protocol.org (APPROVED) — exact v1 URI grammar, 12 canonicalization rules, resource-kind registry, authority semantics, legacy migration contract, conformance requirements, Phase A/D TDD order.
  • roam/design/star-lang/STAR-LANG-004-star-service-uri-discovery.org (APPROVED, amended) — shared URI value model (authority, path-segments, resource-kind, resource-path), no domain/address/actor-name fields, explicit migrate-legacy-star-uri ... :to migration.
  • roam/indexes/star-server/STAR-PROTOCOL-INDEX-000-authority-routing.org — protocol graph and implementation ordering.
  • roam/design/star-server/STAR-SERVER-040-starrouter-zeromq-transport-api-overhaul.org (APPROVED) — routing/registration contract that 041 extends; transport-neutrality of identity.
  • roam/research/star-server/STAR-RESEARCH-060-star-bbpd-runtime-modernization.org — BBPD wire-contract evidence: canonical dispatch key documents.target.dispatch.<actor>, compatibility key actors.<actor>.new.target, RabbitMQ stays BBPD's transport.

Current source inspected (base 8fb297d): source/actors.lisp (actor index, get-dest-actor, legacy target actor), source/target-dispatch.lisp + source/target-dispatch-fixes.lisp (durable coordinator, routing keys, validation), source/target-recovery.lisp/source/target-repository.lisp (target records), source/frontends/http-boundary-routes.lisp, source/frontends/http-authorization-routes.lisp (the live /new/target/:actor handler), source/frontends/http-target-v1.lisp/http-target-v1-route.lisp (canonical target create v1), source/frontends/http-bulk-jobs.lisp (AMQP property/header patterns), source/rabbit.lisp (exchange/key constants), source/package.lisp, source/starintel-gserver.asd, t/target-routing-test.lisp, t/http-target-v1-test.lisp, t/v09-runtime-test.lisp, tests/test_v09_runtime_contract.py.

Temporary compatibility-layer architecture

source/star-uri.lisp + source/star-uri-legacy.lisp (package :star.star-uri) implement the approved STAR v1 model — immutable authority + path-segments value with resource-kind/resource-path views and the narrow API: parse-star-uri, canonicalize-star-uri, serialize-star-uri, canonical-star-uri-p, valid-star-uri-p, actor-star-uri-p, star-uri-owned-p, *gserver-star-authority*, plus the gated legacy API (parse-legacy-star-uri, migrate-legacy-star-uri).

Canonicalization implements the STAR-SERVER-041 rules exactly: lowercase scheme and reg-name-profile authority, userinfo/port forbidden, pchar path validation, percent-encoded unreserved normalization, uppercase remaining escapes, dot segments rejected before and after percent normalization (%2E, %2E%2E, %2e%2e cannot sneak through), query/fragment/malformed-escape rejection, the seven reviewed resource kinds, and resource-path case preservation.

The package header carries a prominent replacement marker: it must be deleted in full once the shared StarLang/STAR URI library is consumable by gserver. The seam is structural: target routing only depends on the injectable *target-actor-identity-resolver* and the :star.star-uri package API — never on parser internals. A dedicated test (actor-routing-consumes-the-identity-interface-not-the-parser) injects a stub resolver and proves routing works without the temporary parser, and uri-parsing-stays-outside-the-actor-namespace asserts star.actors exports no URI parsing.

Backward compatibility retained (exact)

  • Bare actor names ("actor": "domain-enricher") resolve exactly as before: local registry first, then the unchanged external-actor route family documents.target.dispatch.<actor> with compatibility key actors.<actor>.new.target (asserted by tests).
  • POST /new/target/:actor remains a bounded bare-name compatibility adapter (the live authorized handler now explicitly rejects STAR URIs with 400 canonical_actor_identity_unsupported instead of leaking a URI into a routing key).
  • Canonical callers supply STAR identity through the migration interface selected from the current schema: POST /api/v1/targets (targets.create), whose actor field now accepts either a bare name or a canonical STAR actor URI (documented in the machine-readable contract).
  • No competing target schema was introduced; dispatch documents gain extensions.target_actor_uri only when canonical identity exists (bare-name dispatch metadata is byte-identical, asserted by test).

BBPD interoperability demonstrated

star://bbpd.starintel.actor/actor/subfinder is proven to be:

  1. accepted as a canonical actor identity (valid-target-actor-identity-p);
  2. parsed into authority bbpd.starintel.actor, resource-kind actor, resource-path ("subfinder");
  3. not interpreted as the literal local registry key — the local registry is never consulted for foreign authorities (resolver-call spy asserted);
  4. projected onto the current external-actor transport: documents.target.dispatch.subfinder + compatibility actors.subfinder.new.target (STAR-RESEARCH-041 keeps RabbitMQ as the durable transport; STAR-RESEARCH-060 records BBPD's dual-key binding);
  5. carried with canonical identity preserved: extensions.target_actor_uri in the document and x-star-destination-uri AMQP header — bbpd.starintel.actor/subfinder never becomes indistinguishable from another authority's subfinder;
  6. never emitted as the legacy star://domain:address:actor-name format.

A local-authority case is included too (star://gserver.starintel.actor/actor/user-hunt → local registry via the adapter; multi-segment actor/quasar/user-hunt resolves through slash-joined registry keys; fail-closed when the flat transport cannot express it).

Legacy handling

Legacy tuples are input-only through the explicit compat API. migrate-legacy-star-uri requires an explicit already-canonical :to target or an explicit :map entry, validates the target through the same canonical validator, and there is deliberately no automatic domain -> authority mapping and no legacy.<domain> invention (both are asserted as negative tests, per the BBPD review note).

What is not claimed

  • No StarRouter distributed authority routing, authority registration/lease, or discovery is implemented — star-uri-owned-p is a configured-namespace comparison only (STAR_GSERVER_AUTHORITY), and resolution boundaries that require more fail closed. Authority is never derived from HTTP origin, broker host, ZMQ endpoint, hostname, or process identity.
  • The temporary :star.star-uri layer is not the shared library and is not exported for ecosystem-wide reuse.

Replacement path

When the shared StarLang STAR URI library is consumable: rebind *target-actor-identity-resolver* (one defparameter) to a resolver backed by the shared library, delete source/star-uri.lisp / source/star-uri-legacy.lisp and the :star.star-uri package, and keep everything else. No actor, HTTP, or RabbitMQ code rewrites.

Test evidence (local)

  • nix run .#star-unit-tests (hermetic, exit 0): 23 required suites green — including new STAR-URI-TESTS discovered=33 executed=33 passed=33 failed=0 skipped=0 and STAR-ACTOR-IDENTITY-TESTS discovered=19 executed=19 passed=19 failed=0 skipped=0; all 21 pre-existing suites unchanged and green (no existing test weakened).
  • nix run .#star-integration-tests (exit 0) against live CouchDB 3.5.2 + Clouseau + RabbitMQ 4.3.4 + Valkey: VALKEY-LEASE-INTEGRATION-TESTS 24/24, COUCHDB-VIEW-INTEGRATION-TESTS 9/9, HTTP-API-TESTS 36/36 (+ public-api integration checks).
  • Worktree verification gate (prolog-verify check) passed with machine-recorded observations of both runs.
## Summary gserver did not previously understand canonical STAR identities at all: target dispatch resolved actors by bare actor-name strings only (`target["actor"]` → `resolve-target-destination(actor-string)` → `get-dest-actor` → local Sento actor, or `actors.<actor>.new.target` / `documents.target.dispatch.<actor>` when no local actor exists). There was no `star://` parsing, canonicalization, authority ownership, resource-kind validation, or STAR URI actor resolution anywhere in `source/` at base commit `8fb297d` (verified by inspection and by STAR-RESEARCH-060's source audit). This PR bridges that gap with a small, explicitly temporary Common Lisp STAR URI compatibility layer so BBPD can migrate safely, per the implementation slot recorded in STAR-SERVER-041 (starintel-labs/starintel-server issue #1). ## Research / design documents followed (normative) From `lost-rob0t/starintel-auto-research`: - `roam/research/star-server/STAR-RESEARCH-041-unified-star-uri-authority-routing-and-fediwatch-rabbitmq.org` (APPROVED) — RFC 3986 direction, canonicalization rules, legacy-input-only rule, gserver-owned authority, RabbitMQ durable dataflow. - `roam/design/star-server/STAR-SERVER-041-star-uri-authority-routing-protocol.org` (APPROVED) — exact v1 URI grammar, 12 canonicalization rules, resource-kind registry, authority semantics, legacy migration contract, conformance requirements, Phase A/D TDD order. - `roam/design/star-lang/STAR-LANG-004-star-service-uri-discovery.org` (APPROVED, amended) — shared URI value model (`authority`, `path-segments`, `resource-kind`, `resource-path`), no `domain/address/actor-name` fields, explicit `migrate-legacy-star-uri ... :to` migration. - `roam/indexes/star-server/STAR-PROTOCOL-INDEX-000-authority-routing.org` — protocol graph and implementation ordering. - `roam/design/star-server/STAR-SERVER-040-starrouter-zeromq-transport-api-overhaul.org` (APPROVED) — routing/registration contract that 041 extends; transport-neutrality of identity. - `roam/research/star-server/STAR-RESEARCH-060-star-bbpd-runtime-modernization.org` — BBPD wire-contract evidence: canonical dispatch key `documents.target.dispatch.<actor>`, compatibility key `actors.<actor>.new.target`, RabbitMQ stays BBPD's transport. Current source inspected (base `8fb297d`): `source/actors.lisp` (actor index, `get-dest-actor`, legacy target actor), `source/target-dispatch.lisp` + `source/target-dispatch-fixes.lisp` (durable coordinator, routing keys, validation), `source/target-recovery.lisp`/`source/target-repository.lisp` (target records), `source/frontends/http-boundary-routes.lisp`, `source/frontends/http-authorization-routes.lisp` (the live `/new/target/:actor` handler), `source/frontends/http-target-v1.lisp`/`http-target-v1-route.lisp` (canonical target create v1), `source/frontends/http-bulk-jobs.lisp` (AMQP property/header patterns), `source/rabbit.lisp` (exchange/key constants), `source/package.lisp`, `source/starintel-gserver.asd`, `t/target-routing-test.lisp`, `t/http-target-v1-test.lisp`, `t/v09-runtime-test.lisp`, `tests/test_v09_runtime_contract.py`. ## Temporary compatibility-layer architecture `source/star-uri.lisp` + `source/star-uri-legacy.lisp` (package `:star.star-uri`) implement the approved STAR v1 model — immutable `authority` + `path-segments` value with `resource-kind`/`resource-path` views and the narrow API: `parse-star-uri`, `canonicalize-star-uri`, `serialize-star-uri`, `canonical-star-uri-p`, `valid-star-uri-p`, `actor-star-uri-p`, `star-uri-owned-p`, `*gserver-star-authority*`, plus the gated legacy API (`parse-legacy-star-uri`, `migrate-legacy-star-uri`). Canonicalization implements the STAR-SERVER-041 rules exactly: lowercase scheme and reg-name-profile authority, userinfo/port forbidden, pchar path validation, percent-encoded unreserved normalization, uppercase remaining escapes, dot segments rejected before **and** after percent normalization (`%2E`, `%2E%2E`, `%2e%2e` cannot sneak through), query/fragment/malformed-escape rejection, the seven reviewed resource kinds, and resource-path case preservation. The package header carries a prominent **replacement marker**: it must be deleted in full once the shared StarLang/STAR URI library is consumable by gserver. The seam is structural: target routing only depends on the injectable `*target-actor-identity-resolver*` and the `:star.star-uri` package API — never on parser internals. A dedicated test (`actor-routing-consumes-the-identity-interface-not-the-parser`) injects a stub resolver and proves routing works without the temporary parser, and `uri-parsing-stays-outside-the-actor-namespace` asserts `star.actors` exports no URI parsing. ## Backward compatibility retained (exact) - Bare actor names (`"actor": "domain-enricher"`) resolve exactly as before: local registry first, then the unchanged external-actor route family `documents.target.dispatch.<actor>` with compatibility key `actors.<actor>.new.target` (asserted by tests). - `POST /new/target/:actor` remains a bounded bare-name compatibility adapter (the live authorized handler now explicitly rejects STAR URIs with `400 canonical_actor_identity_unsupported` instead of leaking a URI into a routing key). - Canonical callers supply STAR identity through the migration interface selected from the current schema: `POST /api/v1/targets` (`targets.create`), whose `actor` field now accepts either a bare name or a canonical STAR actor URI (documented in the machine-readable contract). - No competing target schema was introduced; dispatch documents gain `extensions.target_actor_uri` only when canonical identity exists (bare-name dispatch metadata is byte-identical, asserted by test). ## BBPD interoperability demonstrated `star://bbpd.starintel.actor/actor/subfinder` is proven to be: 1. accepted as a canonical actor identity (`valid-target-actor-identity-p`); 2. parsed into authority `bbpd.starintel.actor`, resource-kind `actor`, resource-path `("subfinder")`; 3. **not** interpreted as the literal local registry key — the local registry is never consulted for foreign authorities (resolver-call spy asserted); 4. projected onto the current external-actor transport: `documents.target.dispatch.subfinder` + compatibility `actors.subfinder.new.target` (STAR-RESEARCH-041 keeps RabbitMQ as the durable transport; STAR-RESEARCH-060 records BBPD's dual-key binding); 5. carried with canonical identity preserved: `extensions.target_actor_uri` in the document and `x-star-destination-uri` AMQP header — `bbpd.starintel.actor/subfinder` never becomes indistinguishable from another authority's `subfinder`; 6. never emitted as the legacy `star://domain:address:actor-name` format. A local-authority case is included too (`star://gserver.starintel.actor/actor/user-hunt` → local registry via the adapter; multi-segment `actor/quasar/user-hunt` resolves through slash-joined registry keys; fail-closed when the flat transport cannot express it). ## Legacy handling Legacy tuples are input-only through the explicit compat API. `migrate-legacy-star-uri` requires an explicit already-canonical `:to` target or an explicit `:map` entry, validates the target through the same canonical validator, and there is deliberately no automatic `domain -> authority` mapping and no `legacy.<domain>` invention (both are asserted as negative tests, per the BBPD review note). ## What is *not* claimed - No StarRouter distributed authority routing, authority registration/lease, or discovery is implemented — `star-uri-owned-p` is a configured-namespace comparison only (`STAR_GSERVER_AUTHORITY`), and resolution boundaries that require more fail closed. Authority is never derived from HTTP origin, broker host, ZMQ endpoint, hostname, or process identity. - The temporary `:star.star-uri` layer is not the shared library and is not exported for ecosystem-wide reuse. ## Replacement path When the shared StarLang STAR URI library is consumable: rebind `*target-actor-identity-resolver*` (one defparameter) to a resolver backed by the shared library, delete `source/star-uri.lisp` / `source/star-uri-legacy.lisp` and the `:star.star-uri` package, and keep everything else. No actor, HTTP, or RabbitMQ code rewrites. ## Test evidence (local) - `nix run .#star-unit-tests` (hermetic, exit 0): 23 required suites green — including new `STAR-URI-TESTS discovered=33 executed=33 passed=33 failed=0 skipped=0` and `STAR-ACTOR-IDENTITY-TESTS discovered=19 executed=19 passed=19 failed=0 skipped=0`; all 21 pre-existing suites unchanged and green (no existing test weakened). - `nix run .#star-integration-tests` (exit 0) against live CouchDB 3.5.2 + Clouseau + RabbitMQ 4.3.4 + Valkey: `VALKEY-LEASE-INTEGRATION-TESTS 24/24`, `COUCHDB-VIEW-INTEGRATION-TESTS 9/9`, `HTTP-API-TESTS 36/36` (+ public-api integration checks). - Worktree verification gate (`prolog-verify check`) passed with machine-recorded observations of both runs.
feat: canonical STAR URI actor identity for target dispatch (STAR-SERVER-041)
Some checks failed
Smoke Tests / unit-tests (pull_request) Failing after 4s
Details
Container Stack / stack (pull_request) Failing after 5s
Details
Smoke Tests / integration-tests (pull_request) Failing after 27s
Details
Operational salvage / source-contract (pull_request) Successful in 1m13s
Details
Canonical StarIntel schema lock / research-node-schema (pull_request) Successful in 1m44s
Details
1f7af28ea3 
Adopt the approved RFC 3986 star://<authority>/<resource-kind>/<resource-path>
identity model so gserver target dispatch speaks canonical STAR identity for
the BBPD migration (starintel-labs/starintel-server#1).

Normative sources (lost-rob0t/starintel-auto-research):
- STAR-RESEARCH-041 unified STAR URI authority routing and FediWatch RabbitMQ
- STAR-SERVER-041 RFC 3986 STAR URI authority routing extension (approved)
- STAR-LANG-004 STAR service URI discovery and dispatch (amended)
- STAR-PROTOCOL-INDEX-000 STAR authority routing protocol
- STAR-RESEARCH-060 star-bbpd runtime modernization (wire-contract evidence)

Temporary compatibility layer (:star.star-uri):
- source/star-uri.lisp implements the approved v1 canonicalization: lowercase
  scheme/authority (RFC 3986 reg-name profile), no userinfo or port, pchar
  path segments, percent-encoded unreserved normalization, uppercase remaining
  escapes, dot-segment rejection before and after percent normalization,
  query/fragment rejection, reviewed resource-kind registry, immutable
  authority + path-segments value with resource-kind/resource-path views.
- source/star-uri-legacy.lisp keeps star://domain:address:actor-name as
  compatibility input only: parsing is API-gated, migration requires an
  explicit canonical :to target or :map entry, and no domain->authority or
  legacy.<domain> rewrite exists.
- The package carries a prominent replacement marker: delete it in full once
  the shared StarLang STAR URI library is consumable by gserver.

Target dispatch (source/target-dispatch.lisp):
- canonical identity, local registry key, and RabbitMQ transport token stay
  distinct; local-authority actor URIs resolve through a dedicated registry
  adapter; foreign-authority URIs (star://bbpd.starintel.actor/actor/subfinder)
  never touch the local registry.
- the flat external-actor route family is preserved
  (documents.target.dispatch.<token>, compatibility actors.<token>.new.target);
  the canonical URI travels in extensions.target_actor_uri and the
  x-star-destination-uri AMQP header so authorities projecting the same token
  stay distinguishable.
- unsupported boundaries fail closed: malformed URIs, non-actor kinds, empty
  actor paths, multi-segment flat transport projections, legacy tuples.
- bare actor names keep resolving exactly as before.

HTTP surfaces: POST /api/v1/targets accepts canonical STAR actor identity in
its actor field (machine contract documented); the historical
/new/target/:actor adapter stays a bounded bare-name compatibility adapter and
rejects STAR URIs (canonical_actor_identity_unsupported).

Gserver authority is configured through STAR_GSERVER_AUTHORITY as a stable
logical namespace, never derived from HTTP origin, broker host, ZMQ endpoint,
hostname, or process identity. Routing depends only on the
*target-actor-identity-resolver* seam and the :star.star-uri package API, so
the shared StarLang URI library replaces the parser without touching actor or
target routing.

Tests: 52 new FiveAM fixtures (star-uri-tests, star-actor-identity-tests)
covering parse/serialize/canonicalization, adversarial encodings, legacy
compatibility and explicit migration, BBPD conformance, local-authority
resolution, bare-name compatibility, HTTP surfaces, and fail-closed behavior.
Some checks failed
Smoke Tests / unit-tests (pull_request) Failing after 4s
Details
Container Stack / stack (pull_request) Failing after 5s
Details
Smoke Tests / integration-tests (pull_request) Failing after 27s
Details
Operational salvage / source-contract (pull_request) Successful in 1m13s
Details
Canonical StarIntel schema lock / research-node-schema (pull_request) Successful in 1m44s
Details
This pull request has changes conflicting with the target branch.
  • source/actors.lisp
  • source/frontends/http-authorization-routes.lisp
  • source/frontends/http-boundary-routes.lisp
  • source/frontends/http-target-v1.lisp
  • source/starintel-gserver.asd
  • source/target-dispatch.lisp
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin agent/star-uri-identity-compat:agent/star-uri-identity-compat
git switch agent/star-uri-identity-compat

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch master
git merge --no-ff agent/star-uri-identity-compat
git switch agent/star-uri-identity-compat
git rebase master
git switch master
git merge --ff-only agent/star-uri-identity-compat
git switch agent/star-uri-identity-compat
git rebase master
git switch master
git merge --no-ff agent/star-uri-identity-compat
git switch master
git merge --squash agent/star-uri-identity-compat
git switch master
git merge --ff-only agent/star-uri-identity-compat
git switch master
git merge agent/star-uri-identity-compat
git push origin master
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starintel-labs/starintel-server!2
No description provided.