nsaspy/starintel-auto-research

Fork 0

ARDR research: Identity email-account existence and Google public-profile enrichment #160

New issue

Open

opened 2026-08-29 10:28:04 +00:00 by nsaspy · 0 comments

nsaspy commented

2026-08-29 10:28:04 +00:00

Owner

StarIntel problem solved

Advance concrete research seed #161 into canonical ARDR research for the StarIntel Identity Domain Server.

Owning future implementation surface: lost-rob0t/starintel-server.
Related Identity design: #173 (Platform Probe Registry / account enumeration).
Shared acquisition/orchestration designs: #171 and #172.
Current starintel-server/master inspected at e99aadd3f44505bc692708aad5e34ab0c4ab0036.

ARDR state

READY_FOR_DESIGN

Implementation approval remains operator-only.

Candidate/ranking decision

This is an unlabelled, dependency-ready Identity capability research gap. Higher-priority StarIntel candidates discovered in this cycle were either already research/design complete (#176/#177, #178/#179), explicitly blocked/deferred, or implementation proposals awaiting operator approval. #161 can be researched now without changing product code and reuses existing StarLang/Identity design instead of creating a new subsystem.

Current StarIntel implementation check

Current starintel-server/master has no Holehe/GHunt/email-enrichment implementation found by repository search. Existing user-hunt work is already being replaced by the Identity Domain Server design in #173; therefore email enumeration must extend that domain and its Platform Probe Registry rather than revive or duplicate user-hunt.

Evidence: Holehe reusable semantics

Current Holehe modules are per-site async probes that commonly use signup/registration availability endpoints, site-specific request templates, status/body predicates, and output fields such as site/domain, method, rate-limit state and existence.

Representative current modules also expose an important defect for StarIntel to avoid: transport/block/rate-limit paths often emit rateLimit=true together with exists=false. That representation can collapse uncertainty into a false negative if a caller only consumes exists.

Some probes are multi-step (fetch page/token, then submit an availability request), so the reusable StarIntel unit is not a Python callback. It is either:

a closed Platform Probe Registry entry for simple request/classification behavior; or
a bounded StarLang source/scraper definition from #171 for multi-step token/session/extraction behavior.

Evidence: GHunt reusable semantics

Current GHunt email enrichment composes several Google-facing lookups: Google person/profile data, Play Games profile, Maps/review-derived information, and public Google Calendar data, then emits structured JSON.

Recent upstream failures in 2026 show why StarIntel must treat this as a drift-prone provider rather than authoritative identity truth:

April 2026 reports that email lookup stopped after a likely Google-side change;
May/August 2026 failures around missing People-response metadata.container caused crashes in released versions;
August 2026 reports show additional crashes when expected photo fields are absent;
current repository code has already added defensive handling not present in the latest packaged release.

Therefore missing or changed upstream fields are unknown/partial evidence, never notFound and never a process crash.

GHunt also has credential/session requirements around Google access. StarIntel must never embed Google cookies/tokens/master-token material in StarLang definitions, fixtures, provenance or ordinary messages.

Research decision

Keep one Identity Domain Server. Add email as an identifier type and separate two capability families inside that domain:

1. Email account-existence probes

Use the same normalized Platform Probe Registry from #173, generalized for identifierType=email.

A probe records:

source registry/version/digest/lineage;
canonical provider/site identity;
email identifier validation;
closed request/source-definition strategy;
explicit positive evidence;
explicit negative evidence;
rate/block/challenge/drift classification;
capability requirement (http, dom, browser, etc.);
side-effect classification;
last validation/health metadata.

2. Google public-profile enrichment

Treat Google enrichment as a richer provider/source capability under Identity, not as a generic boolean email-existence probe.

It may emit typed observations for evidence-supported public/profile surfaces such as:

Google person/profile identity and stable IDs where actually returned;
public profile names/photos;
Play Games public profile;
Maps/public contribution references or aggregate observations where available;
public Calendar metadata/events where publicly exposed.

StarLang-first classification

Already designed/reusable

bounded fan-out / typed partial aggregation: #172;
HTTP/browser/source definitions, deadlines/cancellation/provenance: #171;
Platform Probe Registry + typed account outcomes: #173;
local/remote domain request lifecycle: existing StarLang runtime and #171.

StarLang/library work

No new email-specific language syntax is justified.

Email probes should be represented by the existing planned closed source/probe definitions. If a provider needs a bounded preflight/token/request sequence, express it through #171 rather than arbitrary host callbacks.

Common Lisp

Identity normalization, policy, registry validation, provider selection, evidence classification and canonical ingest belong in the Common Lisp server/domain implementation beneath StarLang contracts.

External process boundary

Do not make GHunt a required production dependency in the foundational slice. Its undocumented/brittle Google surfaces and credential flow do not justify Python-first architecture.

If later evidence shows a specific valuable Google surface cannot reasonably be implemented through documented/public HTTP/source capabilities, a separately reviewed isolated GHunt-compatible adapter may be used behind the typed capability port. It must receive only scoped opaque credential/session refs and return normalized observations. That is a last-resort provider adapter, not the Identity control plane.

Required outcome model

Per-provider/source result:

found | notFound | unknown | blocked | unavailable | failed

Terminal aggregate:

success | partial | cancelled | failed

Rules:

notFound requires an explicit, current negative condition defined by the provider probe;
timeout, connection error, anti-bot response, rate limit, challenge, parse drift or missing optional field never means notFound;
one broken Google sub-source does not erase successful observations from another;
account/profile observations never automatically merge identities.

Side-effect / privacy policy

Email-enumeration techniques can use signup/reset/account-recovery surfaces. StarIntel must classify probe side effects explicitly.

Default policy:

permit read-only/account-availability probes that do not create accounts, send recovery mail/SMS, alter remote state or notify a target;
fail closed on a probe whose behavior may trigger email/SMS/recovery/account mutation unless an explicitly authorized collection policy permits that exact effect;
never let registry data invent arbitrary endpoint actions or executable callbacks;
do not expose email-enumeration as an anonymous public capability;
preserve source/probe/version evidence without storing unrelated private response content.

Provenance

Every accepted observation preserves at least:

queried email digest/reference according to dataset policy;
provider/probe ID;
registry/source-definition version and digest;
observation class and exact classification reason;
retrieval time;
evidence/artifact ref or bounded digest where retained;
originating run/target/correlation identity;
public-profile source URL/identifier only when safe and needed.

Adversarial findings

Rejected:

Run Holehe as the Identity service. Duplicates StarLang orchestration and carries ambiguous negative semantics.
Run GHunt as the Identity service. Google-specific, credential-sensitive and demonstrably drift-prone.
Separate email registry from username registry. Duplicates the Platform Probe Registry; use identifier types.
Treat transport/rate-limit as account absence. Creates false negatives.
Treat missing Google fields as account absence. Upstream schema drift proves this unsafe.
Arbitrary signup/reset probes. Can notify targets or mutate remote systems.
Share one Google session across principals/runs. Credential and attribution boundary violation.
Automatically merge Google/profile observations into a person. Entity correlation belongs to the expert/entity-resolution layer.
Python-first provider callbacks. Closed StarLang source definitions and Common Lisp policy remain authoritative.
Unbounded scan-all. Provider count, concurrency, deadline, bytes, redirects and retries remain bounded.

Mandatory RED-first targets for eventual implementation

An email Platform Probe Registry fixture must fail on untouched Identity code because no normalized identifierType=email probe schema/normalizer exists.
A fixture where transport failure/rate limiting occurs must require unavailable/blocked/unknown, not notFound.
A side-effecting recovery/signup probe fixture must be rejected by default policy.
An identity.enrichEmail contract fixture must represent mixed provider outcomes and terminal partial with provenance.
A fake Google response omitting expected nested profile metadata must produce typed unknown/partial rather than crash or notFound.
A raw Google cookie/token literal in a source definition/request must fail; only opaque credential/session refs are accepted.
A found observation must go through canonical ingest and remain an observation, not an automatic same-person merge.

Decision

Evidence is sufficient to proceed directly to design. No unresolved architecture question requires human research validation.

Implementation approval: PENDING / AWAITING_OPERATOR_APPROVAL.

## StarIntel problem solved Advance concrete research seed #161 into canonical ARDR research for the StarIntel Identity Domain Server. Owning future implementation surface: `lost-rob0t/starintel-server`. Related Identity design: #173 (Platform Probe Registry / account enumeration). Shared acquisition/orchestration designs: #171 and #172. Current `starintel-server/master` inspected at `e99aadd3f44505bc692708aad5e34ab0c4ab0036`. ## ARDR state `READY_FOR_DESIGN` Implementation approval remains operator-only. ## Candidate/ranking decision This is an unlabelled, dependency-ready Identity capability research gap. Higher-priority StarIntel candidates discovered in this cycle were either already research/design complete (#176/#177, #178/#179), explicitly blocked/deferred, or implementation proposals awaiting operator approval. #161 can be researched now without changing product code and reuses existing StarLang/Identity design instead of creating a new subsystem. ## Current StarIntel implementation check Current `starintel-server/master` has no Holehe/GHunt/email-enrichment implementation found by repository search. Existing user-hunt work is already being replaced by the Identity Domain Server design in #173; therefore email enumeration must extend that domain and its Platform Probe Registry rather than revive or duplicate `user-hunt`. ## Evidence: Holehe reusable semantics Current Holehe modules are per-site async probes that commonly use signup/registration availability endpoints, site-specific request templates, status/body predicates, and output fields such as site/domain, method, rate-limit state and existence. Representative current modules also expose an important defect for StarIntel to avoid: transport/block/rate-limit paths often emit `rateLimit=true` together with `exists=false`. That representation can collapse uncertainty into a false negative if a caller only consumes `exists`. Some probes are multi-step (fetch page/token, then submit an availability request), so the reusable StarIntel unit is not a Python callback. It is either: - a closed Platform Probe Registry entry for simple request/classification behavior; or - a bounded StarLang source/scraper definition from #171 for multi-step token/session/extraction behavior. ## Evidence: GHunt reusable semantics Current GHunt email enrichment composes several Google-facing lookups: Google person/profile data, Play Games profile, Maps/review-derived information, and public Google Calendar data, then emits structured JSON. Recent upstream failures in 2026 show why StarIntel must treat this as a drift-prone provider rather than authoritative identity truth: - April 2026 reports that email lookup stopped after a likely Google-side change; - May/August 2026 failures around missing People-response `metadata.container` caused crashes in released versions; - August 2026 reports show additional crashes when expected photo fields are absent; - current repository code has already added defensive handling not present in the latest packaged release. Therefore missing or changed upstream fields are `unknown`/`partial` evidence, never `notFound` and never a process crash. GHunt also has credential/session requirements around Google access. StarIntel must never embed Google cookies/tokens/master-token material in StarLang definitions, fixtures, provenance or ordinary messages. ## Research decision Keep one **Identity Domain Server**. Add email as an identifier type and separate two capability families inside that domain: ### 1. Email account-existence probes Use the same normalized Platform Probe Registry from #173, generalized for `identifierType=email`. A probe records: - source registry/version/digest/lineage; - canonical provider/site identity; - email identifier validation; - closed request/source-definition strategy; - explicit positive evidence; - explicit negative evidence; - rate/block/challenge/drift classification; - capability requirement (`http`, `dom`, `browser`, etc.); - side-effect classification; - last validation/health metadata. ### 2. Google public-profile enrichment Treat Google enrichment as a richer provider/source capability under Identity, not as a generic boolean email-existence probe. It may emit typed observations for evidence-supported public/profile surfaces such as: - Google person/profile identity and stable IDs where actually returned; - public profile names/photos; - Play Games public profile; - Maps/public contribution references or aggregate observations where available; - public Calendar metadata/events where publicly exposed. Each sub-source can independently be `found | notFound | unknown | blocked | unavailable | failed`; the aggregate may be `partial`. ## StarLang-first classification ### Already designed/reusable - bounded fan-out / typed partial aggregation: #172; - HTTP/browser/source definitions, deadlines/cancellation/provenance: #171; - Platform Probe Registry + typed account outcomes: #173; - local/remote domain request lifecycle: existing StarLang runtime and #171. ### StarLang/library work No new email-specific language syntax is justified. Email probes should be represented by the existing planned closed source/probe definitions. If a provider needs a bounded preflight/token/request sequence, express it through #171 rather than arbitrary host callbacks. ### Common Lisp Identity normalization, policy, registry validation, provider selection, evidence classification and canonical ingest belong in the Common Lisp server/domain implementation beneath StarLang contracts. ### External process boundary Do **not** make GHunt a required production dependency in the foundational slice. Its undocumented/brittle Google surfaces and credential flow do not justify Python-first architecture. If later evidence shows a specific valuable Google surface cannot reasonably be implemented through documented/public HTTP/source capabilities, a separately reviewed isolated GHunt-compatible adapter may be used behind the typed capability port. It must receive only scoped opaque credential/session refs and return normalized observations. That is a last-resort provider adapter, not the Identity control plane. ## Required outcome model Per-provider/source result: ```text found | notFound | unknown | blocked | unavailable | failed ``` Terminal aggregate: ```text success | partial | cancelled | failed ``` Rules: - `notFound` requires an explicit, current negative condition defined by the provider probe; - timeout, connection error, anti-bot response, rate limit, challenge, parse drift or missing optional field never means `notFound`; - one broken Google sub-source does not erase successful observations from another; - account/profile observations never automatically merge identities. ## Side-effect / privacy policy Email-enumeration techniques can use signup/reset/account-recovery surfaces. StarIntel must classify probe side effects explicitly. Default policy: - permit read-only/account-availability probes that do not create accounts, send recovery mail/SMS, alter remote state or notify a target; - fail closed on a probe whose behavior may trigger email/SMS/recovery/account mutation unless an explicitly authorized collection policy permits that exact effect; - never let registry data invent arbitrary endpoint actions or executable callbacks; - do not expose email-enumeration as an anonymous public capability; - preserve source/probe/version evidence without storing unrelated private response content. ## Provenance Every accepted observation preserves at least: - queried email digest/reference according to dataset policy; - provider/probe ID; - registry/source-definition version and digest; - observation class and exact classification reason; - retrieval time; - evidence/artifact ref or bounded digest where retained; - originating run/target/correlation identity; - public-profile source URL/identifier only when safe and needed. ## Adversarial findings Rejected: 1. **Run Holehe as the Identity service.** Duplicates StarLang orchestration and carries ambiguous negative semantics. 2. **Run GHunt as the Identity service.** Google-specific, credential-sensitive and demonstrably drift-prone. 3. **Separate email registry from username registry.** Duplicates the Platform Probe Registry; use identifier types. 4. **Treat transport/rate-limit as account absence.** Creates false negatives. 5. **Treat missing Google fields as account absence.** Upstream schema drift proves this unsafe. 6. **Arbitrary signup/reset probes.** Can notify targets or mutate remote systems. 7. **Share one Google session across principals/runs.** Credential and attribution boundary violation. 8. **Automatically merge Google/profile observations into a person.** Entity correlation belongs to the expert/entity-resolution layer. 9. **Python-first provider callbacks.** Closed StarLang source definitions and Common Lisp policy remain authoritative. 10. **Unbounded scan-all.** Provider count, concurrency, deadline, bytes, redirects and retries remain bounded. ## Mandatory RED-first targets for eventual implementation 1. An email Platform Probe Registry fixture must fail on untouched Identity code because no normalized `identifierType=email` probe schema/normalizer exists. 2. A fixture where transport failure/rate limiting occurs must require `unavailable`/`blocked`/`unknown`, not `notFound`. 3. A side-effecting recovery/signup probe fixture must be rejected by default policy. 4. An `identity.enrichEmail` contract fixture must represent mixed provider outcomes and terminal `partial` with provenance. 5. A fake Google response omitting expected nested profile metadata must produce typed `unknown`/`partial` rather than crash or `notFound`. 6. A raw Google cookie/token literal in a source definition/request must fail; only opaque credential/session refs are accepted. 7. A `found` observation must go through canonical ingest and remain an observation, not an automatic same-person merge. ## Decision Evidence is sufficient to proceed directly to design. No unresolved architecture question requires human research validation. Implementation approval: `PENDING / AWAITING_OPERATOR_APPROVAL`.

nsaspy referenced this issue

2026-08-29 10:28:05 +00:00

DESIGN_READY_FOR_OPERATOR_REVIEW: Identity Domain Server account enumeration #167

nsaspy referenced this issue

2026-08-29 10:28:07 +00:00

StarLang research: OSINT domain-server orchestration gaps #175