nsaspy/starintel-auto-research

Fork 0

DESIGN_READY_FOR_OPERATOR_REVIEW: Identity Domain Server account enumeration #167

New issue

Open

opened 2026-08-29 10:28:05 +00:00 by nsaspy · 0 comments

nsaspy commented

2026-08-29 10:28:05 +00:00

Owner

Authority / state

Source research: #160
Shared orchestration design: #172
Source Acquisition design: #171
Owning implementation problem: lost-rob0t/starintel-server#45
ARDR policy: #170
Research state: READY_FOR_DESIGN
Design state: DESIGN_READY_FOR_OPERATOR_REVIEW
Implementation approval: PENDING — operator only

This design does not authorize implementation and must not enter the executable RAGE queue until the operator explicitly approves an implementation slice.

StarIntel problem solved

The current dormant user-hunt subsystem is architecturally broken and operationally unsafe. StarIntel needs one bounded Identity Domain Server capability for account enumeration that can consume versioned platform registries such as WhatsMyName and richer Sherlock/Maigret-derived definitions without copying those Python runtimes or inventing a second fetch/concurrency stack.

Ownership

StarLang / shared runtime

Owns only reusable semantics already required elsewhere:

typed bounded fan-out / partial aggregation from #172;
deadlines, cancellation, correlation and local/remote request/reply;
source-acquisition HTTP/browser capability calls from #171;
closed typed outcome envelopes;
immutable run-scoped registry/version identity.

Identity Domain Server

Owns:

identity.enumerateAccounts capability;
normalized Platform Probe Registry;
probe selection/filtering by identifier type, tags, policy, registry version and health;
conversion of acquisition results into typed account observations;
provider/site health observations and bounded demotion policy;
canonical ingest handoff for resulting observations.

Source Acquisition Domain Server

Owns HTTP/browser mechanics, redirects, response/body ceilings, rate limits, challenge continuation, credential/session refs and SSRF/network policy. Identity does not duplicate any of that.

Expert/entity-resolution layer

Owns corroboration, alias/entity merging, confidence reasoning and relationship inference. A matching username on two sites is not automatically the same person.

Platform Probe Registry

Use one normalized registry schema capable of importing multiple upstream registries while retaining lineage.

Required entry fields include:

stable probeId;
source registry + source entry identity;
source registry version/digest/license metadata;
platform canonical name/host;
identifier type;
profile URL template;
request template reference or closed fields: method/query/body/headers;
closed detection strategy type;
positive/negative indicators;
username/identifier validation constraint;
tags/categories/country/rank metadata where available;
enabled/disabled/degraded state;
claimed/unclaimed validation fixtures when available;
capability requirements such as http or browser;
provenance and last validation timestamp.

Registry data may never name arbitrary Lisp functions, shell commands or executable callbacks.

Probe strategy model

Initial closed strategy family:

status — classify allowed status sets;
body-marker — bounded required/forbidden marker checks;
final-url — classify bounded redirect/final URL patterns;
json-field — bounded typed field/path predicate for explicit APIs;
composite — bounded AND/OR composition of closed primitive strategies.

A positive or negative classification must preserve the exact strategy/reason used.

Unknown/malformed strategy is a registry validation error, not executable fallback behavior.

Capability contract

Conceptual request:

identity.enumerateAccounts {
  identifier,
  identifierType,
  registryRef,
  platformFilter?,
  tagFilter?,
  maxPlatforms,
  concurrency,
  deadline,
  perHostBudget,
  principalRef,
  runId
}

Per-probe result:

accountProbeResult {
  probeId,
  platform,
  outcome: found|notFound|unknown|blocked|unavailable|failed,
  reason,
  profileUrl?,
  evidenceRef?,
  registryVersion,
  observedAt,
  provenance
}

Terminal aggregate:

success|partial|cancelled|failed

with counts and refs to per-probe results. partial remains visible.

Execution semantics

choose only authorized, enabled probes matching identifier type and caller policy;
freeze registry version/digest for the run;
deduplicate imported registry entries by canonical platform + equivalent probe semantics while preserving all source lineages;
execute through #172 bounded fan-out and #171 acquisition capabilities;
enforce total max platforms, total deadline, per-host concurrency/rate and global concurrency;
cancellation fences late results;
each probe produces one typed outcome even when a source is blocked/unavailable;
no recursive username pivots unless a separate bounded playbook explicitly requests them;
no registry refresh mutates an in-flight run.

Result semantics

`found`

Requires a strategy-specific positive classification with enough evidence to distinguish it from known negative/ambiguous behavior.

`notFound`

Requires an explicit negative condition from the probe definition. Never derive this from timeout, network failure, CAPTCHA, 403, rate limiting, parse error or unknown response.

`unknown`

Response arrived but cannot be classified safely: drift, conflicting markers, unexpected status/redirect/schema, or insufficient evidence.

`blocked`

Challenge, policy restriction, denied transport or other explicit blocking condition.

`unavailable`

Provider/site cannot currently be contacted within policy, including bounded repeated transport failures.

`failed`

Internal validated-probe execution failure after policy/adapters accepted the operation.

Registry health and refresh

Registry refresh is an explicit job, never load-time network I/O.

Refresh flow:

fetch candidate registry
  -> validate schema
  -> normalize
  -> run claimed/unclaimed/self-check fixtures where permitted
  -> calculate drift report
  -> persist candidate version
  -> activate only after validation policy passes

Health state is scoped by probe + registry version and, where materially relevant, principal/credential/egress context. Do not use one global working boolean.

Demotion is reversible and preserves history. A failing health check does not erase prior account observations.

Canonical ingest/provenance

A found result yields a typed account observation, not a direct speculative identity merge.

Minimum provenance:

queried identifier;
platform/probe ID;
registry source/version/digest;
observed profile URL;
observation timestamp;
classification reason/strategy;
bounded evidence/artifact ref or digest;
originating research/target/run ID where applicable.

The Identity Domain Server hands observations to canonical ingest. It must never publish directly to documents.new.* as the old user-hunt actor does.

Adversarial review

Rejected designs:

Rebuild the old forever-loop user-hunt actor. It duplicates scheduling/concurrency and has no bounded lifecycle.
Make Maigret the service. Useful source semantics, wrong runtime/control-plane boundary.
Treat WMN as executable code. WMN is registry data; StarIntel owns typed execution.
Run every registry independently. This double-counts copied platform knowledge as corroboration.
Use nil as not-found/error. Destroys partial/blocked/drift evidence.
Global provider health. Credentials, egress and rate state can differ by context.
Hot registry replacement mid-run. Makes results irreproducible.
Direct Rabbit output. Bypasses canonical ingest and provenance validation.
Unlimited scan-all mode. Enumeration must always have explicit total ceilings.
Automatic same-person inference. Account existence is observation; identity correlation belongs to the expert/entity layer.

Dependency-ordered proposed implementation slices

All are AWAITING_OPERATOR_IMPLEMENTATION_APPROVAL.

Identity probe registry schema + deterministic normalizer — import WMN first; support lineage/version/digest and closed strategies.
Identity Domain Server enumerateAccounts contract — typed request/per-probe/aggregate outcomes, no production network yet.
Bounded execution integration — use #172 fan-out and #171 HTTP acquisition; deterministic fake-provider fixtures.
Registry refresh + drift/health lifecycle — explicit job and candidate-version activation, never load-time network.
Canonical account-observation ingest — provenance-preserving outputs, no direct Rabbit publication.
Optional richer registry adapters — Sherlock/Maigret normalization where their extra fields improve coverage; no Python runtime dependency unless a source-specific capability proves unavoidable.
Profile/pivot follow-up — separate capability only after existence enumeration is stable.

Mandatory RED-first targets

Slice 1

On untouched server/Identity code, a deterministic normalized WMN fixture must fail because there is no Platform Probe Registry schema/normalizer with source version/digest and closed strategy validation.

Negative RED fixtures:

executable callback/function field rejected;
unknown strategy rejected;
missing source lineage rejected;
duplicate canonical probe with conflicting semantics rejected or surfaced explicitly;
raw credential literal rejected.

Slice 2

A contract fixture must fail because current user-hunt cannot represent unknown, blocked, unavailable, registry version identity, per-probe provenance, or terminal partial.

Slice 3

A fake registry with more sites than configured maxPlatforms must prove untouched/current execution lacks the required bounded Identity contract. The GREEN implementation must never exceed configured total/concurrency ceilings.

Slice 4

A registry refresh fixture must prove loading the library performs zero network I/O and an invalid/drifting candidate registry never replaces the active version.

Slice 5

A fake found result must prove the old direct-Rabbit path is not acceptable: the new result must pass through canonical ingest and preserve probe/registry provenance.

Acceptance

loading StarIntel Server performs zero username-registry network I/O;
one finite enumeration request terminates with typed aggregate outcome;
notFound is only emitted from explicit negative evidence;
timeout/challenge/rate/drift remains non-negative typed evidence;
maximum total and concurrent probes are bounded and tested;
registry version is frozen for each run;
multiple imported registries do not create fake multi-source corroboration;
provider drift can demote a probe without erasing historical evidence;
cancellation/restart does not duplicate accepted observations;
all account observations enter through canonical ingest with provenance;
StarLang/Source Acquisition shared mechanisms are reused rather than duplicated;
implementation remains blocked until explicit operator approval.

Implementation approval

PENDING / AWAITING_OPERATOR_IMPLEMENTATION_APPROVAL

ARDR has no authority to change this state.

## Authority / state - Source research: #160 - Shared orchestration design: #172 - Source Acquisition design: #171 - Owning implementation problem: `lost-rob0t/starintel-server#45` - ARDR policy: #170 - Research state: `READY_FOR_DESIGN` - Design state: `DESIGN_READY_FOR_OPERATOR_REVIEW` - **Implementation approval: PENDING — operator only** This design does not authorize implementation and must not enter the executable RAGE queue until the operator explicitly approves an implementation slice. ## StarIntel problem solved The current dormant `user-hunt` subsystem is architecturally broken and operationally unsafe. StarIntel needs one bounded Identity Domain Server capability for account enumeration that can consume versioned platform registries such as WhatsMyName and richer Sherlock/Maigret-derived definitions without copying those Python runtimes or inventing a second fetch/concurrency stack. ## Ownership ### StarLang / shared runtime Owns only reusable semantics already required elsewhere: - typed bounded fan-out / partial aggregation from #172; - deadlines, cancellation, correlation and local/remote request/reply; - source-acquisition HTTP/browser capability calls from #171; - closed typed outcome envelopes; - immutable run-scoped registry/version identity. ### Identity Domain Server Owns: - `identity.enumerateAccounts` capability; - normalized Platform Probe Registry; - probe selection/filtering by identifier type, tags, policy, registry version and health; - conversion of acquisition results into typed account observations; - provider/site health observations and bounded demotion policy; - canonical ingest handoff for resulting observations. ### Source Acquisition Domain Server Owns HTTP/browser mechanics, redirects, response/body ceilings, rate limits, challenge continuation, credential/session refs and SSRF/network policy. Identity does not duplicate any of that. ### Expert/entity-resolution layer Owns corroboration, alias/entity merging, confidence reasoning and relationship inference. A matching username on two sites is not automatically the same person. ## Platform Probe Registry Use one normalized registry schema capable of importing multiple upstream registries while retaining lineage. Required entry fields include: - stable `probeId`; - source registry + source entry identity; - source registry version/digest/license metadata; - platform canonical name/host; - identifier type; - profile URL template; - request template reference or closed fields: method/query/body/headers; - closed detection strategy type; - positive/negative indicators; - username/identifier validation constraint; - tags/categories/country/rank metadata where available; - enabled/disabled/degraded state; - claimed/unclaimed validation fixtures when available; - capability requirements such as `http` or `browser`; - provenance and last validation timestamp. Registry data may never name arbitrary Lisp functions, shell commands or executable callbacks. ## Probe strategy model Initial closed strategy family: 1. `status` — classify allowed status sets; 2. `body-marker` — bounded required/forbidden marker checks; 3. `final-url` — classify bounded redirect/final URL patterns; 4. `json-field` — bounded typed field/path predicate for explicit APIs; 5. `composite` — bounded AND/OR composition of closed primitive strategies. A positive or negative classification must preserve the exact strategy/reason used. Unknown/malformed strategy is a registry validation error, not executable fallback behavior. ## Capability contract Conceptual request: ```text identity.enumerateAccounts { identifier, identifierType, registryRef, platformFilter?, tagFilter?, maxPlatforms, concurrency, deadline, perHostBudget, principalRef, runId } ``` Per-probe result: ```text accountProbeResult { probeId, platform, outcome: found|notFound|unknown|blocked|unavailable|failed, reason, profileUrl?, evidenceRef?, registryVersion, observedAt, provenance } ``` Terminal aggregate: ```text success|partial|cancelled|failed ``` with counts and refs to per-probe results. `partial` remains visible. ## Execution semantics - choose only authorized, enabled probes matching identifier type and caller policy; - freeze registry version/digest for the run; - deduplicate imported registry entries by canonical platform + equivalent probe semantics while preserving all source lineages; - execute through #172 bounded fan-out and #171 acquisition capabilities; - enforce total max platforms, total deadline, per-host concurrency/rate and global concurrency; - cancellation fences late results; - each probe produces one typed outcome even when a source is blocked/unavailable; - no recursive username pivots unless a separate bounded playbook explicitly requests them; - no registry refresh mutates an in-flight run. ## Result semantics ### `found` Requires a strategy-specific positive classification with enough evidence to distinguish it from known negative/ambiguous behavior. ### `notFound` Requires an explicit negative condition from the probe definition. Never derive this from timeout, network failure, CAPTCHA, 403, rate limiting, parse error or unknown response. ### `unknown` Response arrived but cannot be classified safely: drift, conflicting markers, unexpected status/redirect/schema, or insufficient evidence. ### `blocked` Challenge, policy restriction, denied transport or other explicit blocking condition. ### `unavailable` Provider/site cannot currently be contacted within policy, including bounded repeated transport failures. ### `failed` Internal validated-probe execution failure after policy/adapters accepted the operation. ## Registry health and refresh Registry refresh is an explicit job, never load-time network I/O. Refresh flow: ```text fetch candidate registry -> validate schema -> normalize -> run claimed/unclaimed/self-check fixtures where permitted -> calculate drift report -> persist candidate version -> activate only after validation policy passes ``` Health state is scoped by probe + registry version and, where materially relevant, principal/credential/egress context. Do not use one global `working` boolean. Demotion is reversible and preserves history. A failing health check does not erase prior account observations. ## Canonical ingest/provenance A `found` result yields a typed account observation, not a direct speculative identity merge. Minimum provenance: - queried identifier; - platform/probe ID; - registry source/version/digest; - observed profile URL; - observation timestamp; - classification reason/strategy; - bounded evidence/artifact ref or digest; - originating research/target/run ID where applicable. The Identity Domain Server hands observations to canonical ingest. It must never publish directly to `documents.new.*` as the old `user-hunt` actor does. ## Adversarial review Rejected designs: 1. **Rebuild the old forever-loop `user-hunt` actor.** It duplicates scheduling/concurrency and has no bounded lifecycle. 2. **Make Maigret the service.** Useful source semantics, wrong runtime/control-plane boundary. 3. **Treat WMN as executable code.** WMN is registry data; StarIntel owns typed execution. 4. **Run every registry independently.** This double-counts copied platform knowledge as corroboration. 5. **Use `nil` as not-found/error.** Destroys partial/blocked/drift evidence. 6. **Global provider health.** Credentials, egress and rate state can differ by context. 7. **Hot registry replacement mid-run.** Makes results irreproducible. 8. **Direct Rabbit output.** Bypasses canonical ingest and provenance validation. 9. **Unlimited scan-all mode.** Enumeration must always have explicit total ceilings. 10. **Automatic same-person inference.** Account existence is observation; identity correlation belongs to the expert/entity layer. ## Dependency-ordered proposed implementation slices All are **AWAITING_OPERATOR_IMPLEMENTATION_APPROVAL**. 1. **Identity probe registry schema + deterministic normalizer** — import WMN first; support lineage/version/digest and closed strategies. 2. **Identity Domain Server `enumerateAccounts` contract** — typed request/per-probe/aggregate outcomes, no production network yet. 3. **Bounded execution integration** — use #172 fan-out and #171 HTTP acquisition; deterministic fake-provider fixtures. 4. **Registry refresh + drift/health lifecycle** — explicit job and candidate-version activation, never load-time network. 5. **Canonical account-observation ingest** — provenance-preserving outputs, no direct Rabbit publication. 6. **Optional richer registry adapters** — Sherlock/Maigret normalization where their extra fields improve coverage; no Python runtime dependency unless a source-specific capability proves unavoidable. 7. **Profile/pivot follow-up** — separate capability only after existence enumeration is stable. ## Mandatory RED-first targets ### Slice 1 On untouched server/Identity code, a deterministic normalized WMN fixture must fail because there is no Platform Probe Registry schema/normalizer with source version/digest and closed strategy validation. Negative RED fixtures: - executable callback/function field rejected; - unknown strategy rejected; - missing source lineage rejected; - duplicate canonical probe with conflicting semantics rejected or surfaced explicitly; - raw credential literal rejected. ### Slice 2 A contract fixture must fail because current `user-hunt` cannot represent `unknown`, `blocked`, `unavailable`, registry version identity, per-probe provenance, or terminal `partial`. ### Slice 3 A fake registry with more sites than configured `maxPlatforms` must prove untouched/current execution lacks the required bounded Identity contract. The GREEN implementation must never exceed configured total/concurrency ceilings. ### Slice 4 A registry refresh fixture must prove loading the library performs zero network I/O and an invalid/drifting candidate registry never replaces the active version. ### Slice 5 A fake `found` result must prove the old direct-Rabbit path is not acceptable: the new result must pass through canonical ingest and preserve probe/registry provenance. ## Acceptance - loading StarIntel Server performs zero username-registry network I/O; - one finite enumeration request terminates with typed aggregate outcome; - `notFound` is only emitted from explicit negative evidence; - timeout/challenge/rate/drift remains non-negative typed evidence; - maximum total and concurrent probes are bounded and tested; - registry version is frozen for each run; - multiple imported registries do not create fake multi-source corroboration; - provider drift can demote a probe without erasing historical evidence; - cancellation/restart does not duplicate accepted observations; - all account observations enter through canonical ingest with provenance; - StarLang/Source Acquisition shared mechanisms are reused rather than duplicated; - implementation remains blocked until explicit operator approval. ## Implementation approval `PENDING / AWAITING_OPERATOR_IMPLEMENTATION_APPROVAL` ARDR has no authority to change this state.