nsaspy/starintel-auto-research

Fork 0

DESIGN_READY_FOR_OPERATOR_REVIEW: Identity email enrichment + Google public-profile pivots #159

New issue

Open

opened 2026-08-29 10:28:04 +00:00 by nsaspy · 0 comments

nsaspy commented

2026-08-29 10:28:04 +00:00

Owner

Authority / state

Source seed: #161
Canonical ARDR research: #180 — READY_FOR_DESIGN
Related Identity design: #173
Shared Source Acquisition design: #171
Shared bounded orchestration design: #172
Owning implementation repository: lost-rob0t/starintel-server
Current server master inspected: e99aadd3f44505bc692708aad5e34ab0c4ab0036
Design state: DESIGN_READY_FOR_OPERATOR_REVIEW
Implementation approval: PENDING / AWAITING_OPERATOR_APPROVAL — operator only

This design is not executable by RAGE until explicit operator implementation approval is recorded.

Goal

Add email-centered identity enrichment without creating a Holehe service, a GHunt service, a second provider registry, or an email-specific orchestration stack.

The Identity Domain Server owns the capability. StarLang/source-acquisition owns bounded collection mechanics. The expert/entity layer owns identity correlation.

Domain surface

Use one capability family:

identity.enrichEmail

Conceptual request:

{
  emailRef,
  registryRef,
  providerFilter?,
  includeGooglePublicProfile?,
  maxProviders,
  concurrency,
  deadline,
  perHostBudget,
  principalRef,
  runId
}

Do not put raw email credentials, Google cookies or session tokens into the request. emailRef may resolve to the normalized input under caller authorization; credential/session needs use opaque refs from the Source Acquisition design.

Result contract

Each provider/sub-source returns:

emailEnrichmentObservation {
  providerId,
  observationType,
  outcome: found|notFound|unknown|blocked|unavailable|failed,
  reason,
  subjectRef,
  publicProfileRef?,
  evidenceRef?,
  registryOrSourceVersion,
  observedAt,
  provenance
}

Terminal aggregate:

success | partial | cancelled | failed

partial remains visible and preserves every provider outcome.

Platform Probe Registry extension

Do not create an email-only registry. Extend the #173 Platform Probe Registry model to support identifierType=email plus one required safety field:

sideEffectClass:
  readOnly
  mayNotifyTarget
  mayMutateRemoteState
  unknown

Default execution policy permits only readOnly probes. All other classes fail closed unless a separately authorized policy explicitly permits the exact effect.

Simple Holehe-like checks lower into closed request + classification entries. Multi-step checks needing CSRF/preflight/session extraction reference a versioned StarLang source definition from #171 rather than arbitrary callbacks.

Classification rules

`found`

Requires an explicit current positive condition from the provider definition.

`notFound`

Requires an explicit current negative condition. It cannot be inferred from network failure, timeout, parse failure, CAPTCHA, 403, 429, anti-bot response, unknown status, schema drift or missing optional fields.

`unknown`

A response exists but cannot safely classify account existence or profile state.

`blocked`

Challenge, policy denial, anti-enumeration gate or side-effect policy blocks execution/classification.

`unavailable`

The provider/sub-source cannot be contacted within policy/budget.

`failed`

Internal execution failure after a valid definition and adapter were accepted.

Google enrichment boundary

Google enrichment is not a boolean email-registration probe. It is a richer source under the same Identity capability.

Potential typed observations are emitted independently for only evidence-supported/publicly exposed surfaces, including:

profile/person identity and stable IDs when returned;
public profile names/photos;
Play Games public profile;
Maps/public contribution observations;
public Calendar metadata/events.

Every sub-source is independently typed. Missing Google fields or changed response structures yield unknown or aggregate partial, not a crash and not account absence.

Credential/session boundary

no raw Google cookie, OAuth token, master token or password in StarLang source, registry data, normal actor messages, logs or provenance;
any authenticated collection uses a principal-scoped opaque credential/session ref from #171;
session refs have owner, expiry and fencing semantics;
no session sharing across principals by default;
cancellation/expiry fences late results and closes owned acquisition resources.

StarLang-first ownership

StarLang / existing planned extensions

Reuse #171/#172 for:

source definitions and closed request/extraction steps;
bounded fan-out;
deadlines/cancellation;
typed partial outcomes;
opaque credential/session/artifact refs;
provenance propagation;
local/remote request lifecycle.

No new email-specific StarLang syntax is required.

Common Lisp Identity implementation

Owns:

email normalization/validation;
Platform Probe Registry email-entry normalization;
side-effect policy;
provider selection;
classification into typed observations;
Google provider/sub-source normalization;
canonical ingest handoff;
run-scoped registry/source version freezing.

External process

No required Python runtime in the foundational implementation.

A GHunt-compatible process adapter is allowed only as a later separately reviewed provider adapter if a valuable Google surface cannot reasonably be implemented through the approved HTTP/browser/source ports. It remains replaceable, isolated and non-authoritative.

Execution

normalize/authorize email input;
freeze Platform Probe Registry/source-definition versions for the run;
select only enabled, authorized, policy-safe email probes;
enforce max provider count, total deadline, global/per-host concurrency, bytes, redirects and retries;
execute through #172 fan-out + #171 Source Acquisition;
optionally execute the Google public-profile source if requested/authorized;
normalize each outcome without collapsing uncertainty;
emit observations with provenance;
hand accepted observations to canonical ingest;
return terminal aggregate.

Persistence / health / drift

Reuse #173 registry health semantics. Health belongs to probe + version and, where relevant, credential/egress/principal context.

A provider drift event may demote a probe but cannot rewrite historical observations or mutate an in-flight run's frozen registry/source version.

Google schema drift must be observable by typed diagnostics and fixture validation rather than surfacing as an uncaught provider exception.

Entity/expert boundary

A found account/profile is an observation, not a person merge.

The expert/entity-resolution layer may later correlate observations using corroborating attributes and provenance. Identity enrichment itself does not claim two accounts belong to the same human merely because they share an email input or a provider-derived name.

Adversarial review

Rejected:

Holehe subprocess as the core implementation — duplicates orchestration, registry and semantics.
GHunt subprocess as the core implementation — Google-specific, brittle and credential-sensitive.
A second email registry — duplicates #173.
exists=false on transport/rate-limit failure — false-negative semantics.
Google parser exception = notFound — schema drift is uncertainty.
Recovery/reset probes by default — may notify the target or mutate remote state.
Registry-defined arbitrary Python/Lisp callbacks — turns data into executable authority.
One shared Google session — cross-principal leakage.
Automatic identity merge — wrong ownership.
Unlimited provider sweep — resource/abuse risk.
Provider-specific remote API — local/remote must use the ordinary StarLang domain lifecycle.
Persist raw provider responses by default — unnecessary private-data retention.

Dependency-ordered proposed implementation slices

All remain AWAITING_OPERATOR_IMPLEMENTATION_APPROVAL.

Platform Probe Registry email identifier + side-effect policy extension — pure schema/normalizer fixtures; no production network.
identity.enrichEmail typed request/result contract — mixed outcomes, partial aggregate, provenance; fake providers only.
Bounded email-probe execution — depends on the relevant #171/#172 implementation slices and reuses #173 registry mechanics.
Google public-profile provider contract + defensive normalizer — fake responses first; missing-field/schema-drift handling.
Canonical ingest integration — account/profile observations with provider/source provenance; no auto-merge.
Optional real Google acquisition adapter — Common Lisp/Source Acquisition first; isolated GHunt-compatible adapter only if separately justified.

Mandatory RED-first targets

Slice 1

On untouched server/Identity code, add a valid email probe-registry fixture requiring:

identifierType=email;
source/version/digest/lineage;
explicit positive + negative classification;
closed acquisition strategy/source-definition ref;
sideEffectClass.

It must fail because current code has no normalized email Platform Probe Registry implementation.

Negative fixtures must reject unknown side-effect class, executable callback/function field, raw credential literal and missing lineage.

Slice 2

A fake enrichment run with one found, one unknown and one blocked provider must require terminal partial and preserve all provider identities/provenance. Untouched code must fail because no identity.enrichEmail typed contract exists.

Slice 3

A provider set larger than maxProviders must prove the implementation never starts excess probes and respects per-host/global concurrency/deadline ceilings.

Slice 4

A fake Google People/profile response missing nested metadata that a previous provider version expected must produce unknown/partial without throwing. Raw Google credentials must be rejected in request/source fixtures.

Slice 5

A fake found observation must be accepted only through canonical ingest with provider/source provenance; no direct Rabbit publication and no implicit same-person merge.

Acceptance

email account existence and Google profile enrichment live under one Identity Domain Server;
no new email-specific registry or workflow engine exists;
notFound requires explicit negative evidence;
rate-limit/transport/challenge/drift never silently becomes absence;
side-effecting probes fail closed by default;
Google missing fields degrade to typed partial/unknown;
credentials/sessions use opaque scoped refs;
provider count/concurrency/deadline/resources are bounded;
registry/source versions are frozen per run;
observations enter canonical ingest with provenance;
entity correlation remains outside acquisition/enrichment;
no mandatory Python dependency is introduced;
implementation remains blocked until explicit operator approval.

Implementation approval

PENDING / AWAITING_OPERATOR_APPROVAL

Only the operator may authorize implementation.

## Authority / state - Source seed: #161 - Canonical ARDR research: #180 — `READY_FOR_DESIGN` - Related Identity design: #173 - Shared Source Acquisition design: #171 - Shared bounded orchestration design: #172 - Owning implementation repository: `lost-rob0t/starintel-server` - Current server master inspected: `e99aadd3f44505bc692708aad5e34ab0c4ab0036` - Design state: `DESIGN_READY_FOR_OPERATOR_REVIEW` - **Implementation approval: PENDING / AWAITING_OPERATOR_APPROVAL — operator only** This design is not executable by RAGE until explicit operator implementation approval is recorded. ## Goal Add email-centered identity enrichment without creating a Holehe service, a GHunt service, a second provider registry, or an email-specific orchestration stack. The Identity Domain Server owns the capability. StarLang/source-acquisition owns bounded collection mechanics. The expert/entity layer owns identity correlation. ## Domain surface Use one capability family: ```text identity.enrichEmail ``` Conceptual request: ```text { emailRef, registryRef, providerFilter?, includeGooglePublicProfile?, maxProviders, concurrency, deadline, perHostBudget, principalRef, runId } ``` Do not put raw email credentials, Google cookies or session tokens into the request. `emailRef` may resolve to the normalized input under caller authorization; credential/session needs use opaque refs from the Source Acquisition design. ## Result contract Each provider/sub-source returns: ```text emailEnrichmentObservation { providerId, observationType, outcome: found|notFound|unknown|blocked|unavailable|failed, reason, subjectRef, publicProfileRef?, evidenceRef?, registryOrSourceVersion, observedAt, provenance } ``` Terminal aggregate: ```text success | partial | cancelled | failed ``` `partial` remains visible and preserves every provider outcome. ## Platform Probe Registry extension Do not create an email-only registry. Extend the #173 Platform Probe Registry model to support `identifierType=email` plus one required safety field: ```text sideEffectClass: readOnly mayNotifyTarget mayMutateRemoteState unknown ``` Default execution policy permits only `readOnly` probes. All other classes fail closed unless a separately authorized policy explicitly permits the exact effect. Simple Holehe-like checks lower into closed request + classification entries. Multi-step checks needing CSRF/preflight/session extraction reference a versioned StarLang source definition from #171 rather than arbitrary callbacks. ## Classification rules ### `found` Requires an explicit current positive condition from the provider definition. ### `notFound` Requires an explicit current negative condition. It cannot be inferred from network failure, timeout, parse failure, CAPTCHA, 403, 429, anti-bot response, unknown status, schema drift or missing optional fields. ### `unknown` A response exists but cannot safely classify account existence or profile state. ### `blocked` Challenge, policy denial, anti-enumeration gate or side-effect policy blocks execution/classification. ### `unavailable` The provider/sub-source cannot be contacted within policy/budget. ### `failed` Internal execution failure after a valid definition and adapter were accepted. ## Google enrichment boundary Google enrichment is not a boolean email-registration probe. It is a richer source under the same Identity capability. Potential typed observations are emitted independently for only evidence-supported/publicly exposed surfaces, including: - profile/person identity and stable IDs when returned; - public profile names/photos; - Play Games public profile; - Maps/public contribution observations; - public Calendar metadata/events. Every sub-source is independently typed. Missing Google fields or changed response structures yield `unknown` or aggregate `partial`, not a crash and not account absence. ## Credential/session boundary - no raw Google cookie, OAuth token, master token or password in StarLang source, registry data, normal actor messages, logs or provenance; - any authenticated collection uses a principal-scoped opaque credential/session ref from #171; - session refs have owner, expiry and fencing semantics; - no session sharing across principals by default; - cancellation/expiry fences late results and closes owned acquisition resources. ## StarLang-first ownership ### StarLang / existing planned extensions Reuse #171/#172 for: - source definitions and closed request/extraction steps; - bounded fan-out; - deadlines/cancellation; - typed partial outcomes; - opaque credential/session/artifact refs; - provenance propagation; - local/remote request lifecycle. No new email-specific StarLang syntax is required. ### Common Lisp Identity implementation Owns: - email normalization/validation; - Platform Probe Registry email-entry normalization; - side-effect policy; - provider selection; - classification into typed observations; - Google provider/sub-source normalization; - canonical ingest handoff; - run-scoped registry/source version freezing. ### External process No required Python runtime in the foundational implementation. A GHunt-compatible process adapter is allowed only as a later separately reviewed provider adapter if a valuable Google surface cannot reasonably be implemented through the approved HTTP/browser/source ports. It remains replaceable, isolated and non-authoritative. ## Execution 1. normalize/authorize email input; 2. freeze Platform Probe Registry/source-definition versions for the run; 3. select only enabled, authorized, policy-safe email probes; 4. enforce max provider count, total deadline, global/per-host concurrency, bytes, redirects and retries; 5. execute through #172 fan-out + #171 Source Acquisition; 6. optionally execute the Google public-profile source if requested/authorized; 7. normalize each outcome without collapsing uncertainty; 8. emit observations with provenance; 9. hand accepted observations to canonical ingest; 10. return terminal aggregate. ## Persistence / health / drift Reuse #173 registry health semantics. Health belongs to probe + version and, where relevant, credential/egress/principal context. A provider drift event may demote a probe but cannot rewrite historical observations or mutate an in-flight run's frozen registry/source version. Google schema drift must be observable by typed diagnostics and fixture validation rather than surfacing as an uncaught provider exception. ## Entity/expert boundary A found account/profile is an observation, not a person merge. The expert/entity-resolution layer may later correlate observations using corroborating attributes and provenance. Identity enrichment itself does not claim two accounts belong to the same human merely because they share an email input or a provider-derived name. ## Adversarial review Rejected: 1. **Holehe subprocess as the core implementation** — duplicates orchestration, registry and semantics. 2. **GHunt subprocess as the core implementation** — Google-specific, brittle and credential-sensitive. 3. **A second email registry** — duplicates #173. 4. **`exists=false` on transport/rate-limit failure** — false-negative semantics. 5. **Google parser exception = notFound** — schema drift is uncertainty. 6. **Recovery/reset probes by default** — may notify the target or mutate remote state. 7. **Registry-defined arbitrary Python/Lisp callbacks** — turns data into executable authority. 8. **One shared Google session** — cross-principal leakage. 9. **Automatic identity merge** — wrong ownership. 10. **Unlimited provider sweep** — resource/abuse risk. 11. **Provider-specific remote API** — local/remote must use the ordinary StarLang domain lifecycle. 12. **Persist raw provider responses by default** — unnecessary private-data retention. ## Dependency-ordered proposed implementation slices All remain **AWAITING_OPERATOR_IMPLEMENTATION_APPROVAL**. 1. **Platform Probe Registry email identifier + side-effect policy extension** — pure schema/normalizer fixtures; no production network. 2. **`identity.enrichEmail` typed request/result contract** — mixed outcomes, partial aggregate, provenance; fake providers only. 3. **Bounded email-probe execution** — depends on the relevant #171/#172 implementation slices and reuses #173 registry mechanics. 4. **Google public-profile provider contract + defensive normalizer** — fake responses first; missing-field/schema-drift handling. 5. **Canonical ingest integration** — account/profile observations with provider/source provenance; no auto-merge. 6. **Optional real Google acquisition adapter** — Common Lisp/Source Acquisition first; isolated GHunt-compatible adapter only if separately justified. ## Mandatory RED-first targets ### Slice 1 On untouched server/Identity code, add a valid email probe-registry fixture requiring: - `identifierType=email`; - source/version/digest/lineage; - explicit positive + negative classification; - closed acquisition strategy/source-definition ref; - `sideEffectClass`. It must fail because current code has no normalized email Platform Probe Registry implementation. Negative fixtures must reject unknown side-effect class, executable callback/function field, raw credential literal and missing lineage. ### Slice 2 A fake enrichment run with one `found`, one `unknown` and one `blocked` provider must require terminal `partial` and preserve all provider identities/provenance. Untouched code must fail because no `identity.enrichEmail` typed contract exists. ### Slice 3 A provider set larger than `maxProviders` must prove the implementation never starts excess probes and respects per-host/global concurrency/deadline ceilings. ### Slice 4 A fake Google People/profile response missing nested metadata that a previous provider version expected must produce `unknown`/`partial` without throwing. Raw Google credentials must be rejected in request/source fixtures. ### Slice 5 A fake `found` observation must be accepted only through canonical ingest with provider/source provenance; no direct Rabbit publication and no implicit same-person merge. ## Acceptance - email account existence and Google profile enrichment live under one Identity Domain Server; - no new email-specific registry or workflow engine exists; - `notFound` requires explicit negative evidence; - rate-limit/transport/challenge/drift never silently becomes absence; - side-effecting probes fail closed by default; - Google missing fields degrade to typed partial/unknown; - credentials/sessions use opaque scoped refs; - provider count/concurrency/deadline/resources are bounded; - registry/source versions are frozen per run; - observations enter canonical ingest with provenance; - entity correlation remains outside acquisition/enrichment; - no mandatory Python dependency is introduced; - implementation remains blocked until explicit operator approval. ## Implementation approval `PENDING / AWAITING_OPERATOR_APPROVAL` Only the operator may authorize implementation.

nsaspy referenced this issue

2026-08-29 10:28:07 +00:00

StarLang research: OSINT domain-server orchestration gaps #175