Formalize external-service integration registry and capability discovery #124

New issue

Open

opened 2026-08-25 09:29:56 +00:00 by lost-rob0t · 0 comments

lost-rob0t commented

2026-08-25 09:29:56 +00:00

(Migrated from github.com)

Goal

Give starintel-server one typed, testable integration surface for the new StarIntel services instead of scattering private endpoints and feature probes through callers.

Existing work to preserve

Reuse/extend the versioned capability-discovery contract from #109/#120 rather than creating a parallel discovery API.
Search-plane adapter is tracked in #123.
Structured logging/syslog are tracked in #121/#122.

Integrations to model

At minimum define configurable server-side integrations for:

federated search (SearXNG/YaCy boundary from #123)
webhook/event delivery where StarIntel emits outbound events or consumes internal normalized events
Node-RED/MCP automation only through an explicit narrow contract if the architecture actually requires server participation
future agent/Space integrations through registered adapters rather than hard-coded private URLs

Requirements

Add a typed integration registry/config layer with stable integration IDs, enabled state, base endpoint/service reference, auth credential reference, capability flags, timeouts, limits, and health state.
Resolve endpoints from runtime/deployment configuration; do not compile private .star.intel names or credentials into client-visible assets/contracts.
Separate server-side service discovery from public client capability discovery. /api/v1/capabilities may advertise that a feature exists, but must not leak private origin hostnames, tokens, internal ports, or admin URLs.
Every adapter gets bounded timeouts, response/request size limits, retry rules only where safe/idempotent, circuit-breaker/health semantics, and explicit error mapping.
Service credentials come from runtime credential mechanisms and are scoped per integration.
Emit integration health/failure/audit events through #121.
Add machine-readable admin/operator status for configured integrations without exposing secrets.
No direct CouchDB/RabbitMQ access should be granted to external services just because they integrate with StarIntel.

Capability contract

Extend the existing capability/openapi/client-manifest work only where necessary so clients can discover supported features (for example federated search) and authority requirements without learning private topology.

Tests

enabled/disabled integration discovery
no private hostname/secret leakage in public capabilities/OpenAPI/client manifest
timeout/unavailable/malformed upstream behavior
credential redaction
per-integration auth isolation
circuit-breaker/recovery behavior
exact-head contract consistency with existing #109/#120 surfaces

Execution mode — AUTO-RAGE

Run the full ADARD/ADADR loop autonomously: inspect current config/capability/client-contract architecture plus #109/#120/#123; design and record the integration-registry boundary; adversarial review + SSRF/credential/topology-leak/confused-deputy threat model; autonomous decision gate; TDD-first implementation; exact-head unit/contract/service-backed verification; push a focused branch and open/update a PR targeting master. Stop only for a genuinely unresolved requirement/security blocker, inaccessible dependency, or failed verification gate. Do not ask for approval between phases.

## Goal Give `starintel-server` one typed, testable integration surface for the new StarIntel services instead of scattering private endpoints and feature probes through callers. ## Existing work to preserve - Reuse/extend the versioned capability-discovery contract from #109/#120 rather than creating a parallel discovery API. - Search-plane adapter is tracked in #123. - Structured logging/syslog are tracked in #121/#122. ## Integrations to model At minimum define configurable server-side integrations for: - federated search (SearXNG/YaCy boundary from #123) - webhook/event delivery where StarIntel emits outbound events or consumes internal normalized events - Node-RED/MCP automation only through an explicit narrow contract if the architecture actually requires server participation - future agent/Space integrations through registered adapters rather than hard-coded private URLs ## Requirements - Add a typed integration registry/config layer with stable integration IDs, enabled state, base endpoint/service reference, auth credential reference, capability flags, timeouts, limits, and health state. - Resolve endpoints from runtime/deployment configuration; do not compile private `.star.intel` names or credentials into client-visible assets/contracts. - Separate **server-side service discovery** from **public client capability discovery**. `/api/v1/capabilities` may advertise that a feature exists, but must not leak private origin hostnames, tokens, internal ports, or admin URLs. - Every adapter gets bounded timeouts, response/request size limits, retry rules only where safe/idempotent, circuit-breaker/health semantics, and explicit error mapping. - Service credentials come from runtime credential mechanisms and are scoped per integration. - Emit integration health/failure/audit events through #121. - Add machine-readable admin/operator status for configured integrations without exposing secrets. - No direct CouchDB/RabbitMQ access should be granted to external services just because they integrate with StarIntel. ## Capability contract Extend the existing capability/openapi/client-manifest work only where necessary so clients can discover supported **features** (for example federated search) and authority requirements without learning private topology. ## Tests - enabled/disabled integration discovery - no private hostname/secret leakage in public capabilities/OpenAPI/client manifest - timeout/unavailable/malformed upstream behavior - credential redaction - per-integration auth isolation - circuit-breaker/recovery behavior - exact-head contract consistency with existing #109/#120 surfaces ## Execution mode — AUTO-RAGE Run the full ADARD/ADADR loop autonomously: inspect current config/capability/client-contract architecture plus #109/#120/#123; design and record the integration-registry boundary; adversarial review + SSRF/credential/topology-leak/confused-deputy threat model; autonomous decision gate; TDD-first implementation; exact-head unit/contract/service-backed verification; push a focused branch and open/update a PR targeting `master`. Stop only for a genuinely unresolved requirement/security blocker, inaccessible dependency, or failed verification gate. Do not ask for approval between phases.