starintel-labs/starintel-gpt-auto-dig

Fork 0

[GH-#133] Scrape membership list: RAND Corporation — people directory #996

New issue

Open

opened 2026-09-08 17:05:44 +00:00 by nsaspy · 0 comments

nsaspy commented

2026-09-08 17:05:44 +00:00

Owner

Mirrored from GitHub https://github.com/lost-rob0t/starintel-gpt-auto-dig/issues/133 (GitHub is authoritative for this item).

Membership-list scraper target

Organization: RAND Corporation
Dataset: rand
URL: https://www.rand.org/about/people.html
Page label: People directory
Discovered from: config/dark-academia-targets.json

Required parser work

Verify and retain the official public RAND people directory parser.
Traverse the complete exposed pagination range and linked profiles.
Preserve names, public roles, research areas, organizational affiliations, source URLs, and retrieval provenance.
Parse RAND leadership, trustees, advisory boards, and RAND Europe public rosters separately when their fields differ.
Record only professional contact details explicitly published by RAND.
Import through the canonical StarIntel writer and validate the complete corpus.
Add regression coverage for pagination and profile extraction.

Current implementation

PR #105 contains the paginated RAND parser and the reusable roster-URL issue invariant.

Mirrored from GitHub https://github.com/lost-rob0t/starintel-gpt-auto-dig/issues/133 (GitHub is authoritative for this item). ---  ## Membership-list scraper target - **Organization:** RAND Corporation - **Dataset:** `rand` - **URL:** https://www.rand.org/about/people.html - **Page label:** People directory - **Discovered from:** `config/dark-academia-targets.json` ## Required parser work - [ ] Verify and retain the official public RAND people directory parser. - [ ] Traverse the complete exposed pagination range and linked profiles. - [ ] Preserve names, public roles, research areas, organizational affiliations, source URLs, and retrieval provenance. - [ ] Parse RAND leadership, trustees, advisory boards, and RAND Europe public rosters separately when their fields differ. - [ ] Record only professional contact details explicitly published by RAND. - [ ] Import through the canonical StarIntel writer and validate the complete corpus. - [ ] Add regression coverage for pagination and profile extraction. ## Current implementation PR #105 contains the paginated RAND parser and the reusable roster-URL issue invariant.

nsaspy added the

github-mirror

label

2026-09-08 17:05:44 +00:00

No Branch/Tag specified

main

spec/0.10.1-unify

fix/backlog-canonical-target-bulk

fix/backlog-legacy-source-normalization

fix/backlog-manual-dispatch-only

fix/backlog-resume-latest-ledger

osint/us-israel-military-integration-corpus

ops/migrate-v0.9-documents

ops/auto-dig-issue-worker

feat/receipt-embeds-findings

fix/forgejo-workflow-shadowing

agent/global-shapers-13k-directory

research/anarchist-violence-area-codes-a-219-231-recovery-2026-09-08

auto-dig/public-comms-pass4-2026-09-08

research/anarchist-violence-canton-fnb-expansion-2026-09-08

auto-dig/area-code-shard-c-248-251-254-20260908

area-code-hunt/b-253-256-262-2026-09-08

research/anarchist-violence-victoria-mutual-aid-expansion-2026-09-08

research/anarchist-violence-area-codes-a-270-303-2026-09-08

auto-dig/public-comms-2026-09-08-pass4

research/anarchist-violence-fnb-network-expansion-2026-09-08

auto-dig/area-code-shard-c-224-227-239-20260908

auto-dig/username-enumeration-2026-09-08

area-code-hunt/b-214-217-220-2026-09-08

research/anarchist-violence-area-codes-a-234-267-2026-09-08

research/anarchist-violence-abcf-expansion-2026-09-08

auto-dig/area-code-shard-c-212-215-218-20260908

auto-dig/public-comms-enum-2026-09-08-pass3

fix/durable-coverage-ledger-2026-09-08

auto-dig/public-comms-pass2-stats

research/anarchist-violence-area-codes-a-219-231-2026-09-08

auto-dig/public-comms-enum-2026-09-08-pass2

research/anarchist-violence-chapter-expansion-2026-09-08-b

auto-dig/public-comms-2026-09-08

research/anarchist-violence-area-codes-a-2026-09-08

research/2335-potateking-network-seeds

research/anarchist-violence-chapter-expansion-2026-09-08

feat/agent-zero-network-hunt-skills

feat/prolog-actor-starintel-tools

auto-dig-prolog/2026-09-06-svc-20260906T235958Z-1-ace520

auto-dig-prolog/2026-09-06-svc-20260906T235450Z-1-1a7fed

auto-dig-prolog/2026-09-04-33859349371-1

auto-dig-prolog/2026-09-01-33532131362-1

fix/canonical-report-prefix

fix/consume-deadline-typing-runtime

auto-dig-prolog/2026-09-01-33528962727-1

fix/auto-dig-transition-time-synthesis-2026-09-01

fix/auto-dig-synthesis-transition-2026-09-01

fix/auto-dig-explicit-synthesis-phase-2026-09-01

fix/auto-dig-synthesis-cutoff-240s-2026-09-01

fix/auto-dig-provider-headroom-2026-09-01

fix/auto-dig-bounded-synthesis-request

fix/auto-dig-deadline-synthesis-reserve

fix/auto-dig-context-alias-repair-2026-09-01

fix/auto-dig-context-alias-recovery-2026-09-01

fix/auto-dig-context-alias-recovery

fix/auto-dig-forced-repeat-live-test-2026-09-01

fix/auto-dig-forced-live-repeat

fix/auto-dig-forced-live-selection-2026-09-01

dogfood/real-output-glm53-flash

fix/auto-dig-real-output-gate-2026-09-01

auto-dig-prolog/2026-09-01-33502262680-1

fix/auto-dig-runtime-tool-cutoff-2026-09-01

fix/auto-dig-runtime-tool-cutoff-green-2026-09-01

test/auto-dig-live-synthesis-reserve-2026-09-01

fix/auto-dig-synthesis-reserve-2026-09-01

fix/importer-corpus-duplicate-contract

dogfood/prolog-rlm-native-tools

fix/import-counterevidence-duplicate

fix/ingest-db-precedence

fix/influencewatch-incomplete-read

feat/ingest-documents-ci

chore/update-prolog-rlm-2026-08-31

agent/auto-dig-backend

test/auto-dig-direct-run-2026-08-29

fix/use-prolog-rlm-304-context-peek

fix/use-prolog-rlm-298-schema-fix

fix/auto-dig-skip-unprojectable-llm-context

fix/auto-dig-direct-rlm-2026-08-29

task/beth-din-religious-court-corruption

fix/276-auto-dig-planner-budget

fix/auto-dig-rlm-observability-2026-08-27

noop

auto-dig-prolog/2026-08-27-latest-rlm

auto-dig-prolog/2026-08-24-bootstrap

rage/2293-quasar-lifecycle-worker

agent/request-2112-harvest-2026-08-25

agent/request-1904-harvest-2026-08-25

agent/request-2106-harvest-2026-08-25

agent/request-2286-harvest-oregon-2026-08-25

agent/request-2095-harvest-v2-2026-08-25

agent/harvest-rensselaer-1901-20260824

agent/siders-2026-08-21-indictment-update

agent/dark-academia-institutions-foundations-wave-3

agent/add-local-db-ingest-skill

agent/request-2095-harvest-2026-08-25

agent/dnc-key-people-wef-accountability

agent/request-2106-github-ci-authority

agent/bhr-partners-depth-12

agent/fbi-tsc-ai-procurement-lineage-depth-2

agent/osf-public-roster-depth-2

auto-dig/wef-ecp-depth-2-methodology

agent/trilateral-commission-research-pass

research/columbus-flock-hearing-2026-08-10

agent/flock-loyal-moses-x-conflict

agent/violent-offenders-local-news-corroboration

agent/request-2106-seattle-command-timeline

agent/request-2095-world-id-control-depth-1

agent/onondaga-county-network-1907

agent/troy-governance-audit-1905

agent/worldcoin-orb-smpc-depth-1-2095

agent/request-1903-syracuse-transition

agent/request-1902-stewarts-proposals

agent/request-2095-worldcoin-control-biometric-token

agent/violent-offenders-2026-08-19-news-scan

agent/prompt-injection-protection

agent/request-1901-rensselaer-sheriff-network

agent/request-1900-cornell-flock-controls

agent/request-1899-ithaca-offboarding

agent/cia-anduril-sparrows-point-primary-records-20260819

agent/request-1898-east-greenbush-flock-vendor

agent/request-1897-north-greenbush-flock-vendor

agent/request-1896-petersburgh-flock-deployment

agent/request-1895-poestenkill-flock-rejection

agent/auto-dig-request-queue

agent/gop-rnc-receipts-location-20260819

agent/influencewatch-one-america-location-semantics

agent/wef-palantir-identity-hq-staleness

agent/hourly-iiss-arundel-house-20260819

agent/siders-vinton-child-abuse

agent/hourly-bruegel-location-20260819

agent/rockefeller-hq-coo-20260819

agent/hourly-cloarida-strive-dallas-20260819

agent/usaid-former-hq-foreign-assistance-20260819

agent/offshore-leaks-energia-registration-20260819

agent/hourly-piie-location-news-20260819

agent/hourly-rand-headquarters-20260819

agent/hourly-hunter-biden-brownes-location

agent/hourly-milken-location-health-20260819

agent/hourly-kearns-upd-precinct-20260819

agent/chatham-house-hq-latin-america-programme

agent/hourly-boao-hq-2026-08-19

agent/hourly-graphika-hq-2026-08-19

agent/fed-nsa-eop-location-20260819

agent/dark-academia-ecfr-berlin-office

agent/drunk-drivers-kearns-court-geo-20260818-1951-v2

agent/drunk-drivers-kearns-court-geo-20260818-1951

agent/greater-israel-modiin-hq-20260818-1855

agent/hourly-cia-anduril-hq-semantics-20260818

agent/elijah-foster-violent-offender-packet

agent/hourly-blackrock-succession-hq-20260818

agent/hourly-blackrock-hq-stewardship-20260818

agent/hourly-graphika-hq-locality

agent/dnc-mur-7157-document-graph

agent/hourly-wef-columbus-shapers-archive-20260818

agent/hourly-palantir-vulcan-hq-osc-20260818

agent/hourly-devilcorp-credico-location-20260818

agent/hourly-cloarida-azoria-principal-place-20260818

agent/hourly-trump-netanyahu-netanya-ashkelon-20260818

agent/hourly-cia-anduril-principal-sparrows-20260818

agent/hourly-wef-columbus-andhealth-location-incentive

agent/hourly-reporting-provenance-20260818

agent/hourly-gop-c00096412-location-20260818

agent/hourly-dnc-suite-history-ice-resolutions-20260818

agent/hourly-cloarida-incubate-location-20260818

agent/hourly-greater-israel-trump-foundation-location-20260818

agent/hourly-tech-bro-capital-location-20260818

agent/hourly-graphika-offices-20260818

agent/fed-odni-clayton-handoff

agent/hourly-dnc-democracy-partners-hq-calendar

agent/hourly-ohio-development-hq-demolition

agent/hourly-jobsohio-hq-fury

agent/hourly-anduril-hq-thunder-20260817

agent/flock-287g-sharing-control-recursion

agent/wef-columbus-depth-6-primary-records

agent/canonicalize-all-dataset-roots

research/thiel-wef-crossmatch-depth-3-2026-08-12

agent/consolidate-flock-datasets

agent/add-flock-people-csv

research/flock-board-public-contacts-2026-08-17

agent/flock-vp-location-enrichment-pass-1

agent/flock-dunwoody-mjcca-autodig

research/canada-influence-network-2026-08-12

fix/2021-bilderberg-participant-parser

agent/dark-academia-rand-bilderberg-wave-2

agent/free-range-assistant-run

fix/dashboard-d3-scaling

fix/pin-large-graph-quasar-ui

fix/validate-pages-data-projection

fix/restore-bounded-data-projections

agent/mobile-dashboard-hardening

agent/fix-violent-offender-false-zeros

agent/restore-nim-dashboard

agent/violent-offender-live-stats-check

agent/restore-black-gold-default

agent/restore-default-theme-runtime

agent/usa-rare-earth-cantor-depth-3-2026-08-09

agent/violent-offender-localities-batch-2

agent/violent-offender-locality-scrapers

agent/violent-offenders-diquan-hunt

agent/fix-nim-pages-search-transport

agent/fix-pages-site-size-architecture

agent/sync-auto-dig-quasar-latest

agent/gop-dataset-100k-expansion

agent/black-gold-dashboard-polish

agent/fold-free-range-depth-2-3-mainline

agent/free-range-wef-trump-vance-palantir-depth-3

agent/fix-williamson-original-sources

agent/free-range-wef-trump-vance-palantir-depth-2

agent/free-range-wef-trump-vance-palantir-depth-1

agent/fix-williamson-legal-case-sources

agent/free-range-kratsios-ethics-ledger-v2

agent/free-range-ecp-2026-08-08

agent/free-range-kratsios-ethics-ledger

agent/free-range-exec-2026-08-08-2

agent/gop-nim-speed-work

agent/free-range-batch-3-pass-1

agent/nim-site-builder-performance

poster/real-corpus-stats-20260808

agent/gop-nim-speed-staging

agent/gop-nim-speed-staging-2

agent/adar-dashboard-ux

agent/repair-violent-offenders-24h-schema

agent/bump-auto-dig-version-0.9.1

agent/fit-pages-artifact

agent/fix-disabled-people-validation

agent/deploy-mobile-graph-recovery

agent/wef-columbus-depth-12-ordinance-payment-reconciliation

agent/wef-columbus-depth-11-smart-columbus-payment-census

agent/wef-columbus-depth-10-direct-recipient-ordinance-ledger

agent/wef-columbus-depth-10-membership-reconciliation

agent/wef-columbus-depth-10-po525642-deliverables

agent/wef-columbus-depth-9-fiscal-transactions

agent/wef-columbus-depth-8-membership-ledger

agent/wef-columbus-depth-7-public-records

agent/normalize-williamson-violent-offenders

agent/free-range-batch-2-pass-1

agent/ci-cost-control-v2

agent/hunter-biden-bhr-merged-corpus

agent/cryptographic-evidence-seal

agent/ci-cost-control

agent/wef-thiel-company-employee-enumeration-depth-4

agent/wef-thiel-company-employee-enumeration-depth-3

agent/free-range-action-trigger

agent/free-range-research-swarm

agent/remove-generated-people-directory

agent/scope-quasar-iframe-dataset

agent/graphika-info-ops-deep-dive

archive/pr-105-pre-fix-d1f1e80

agent/pr-78-takeover-fix

fix/site-corpus-integrity

agent/wef-columbus-depth-13-energi-ai-emissions-pilot

agent/bhr-partners-depth-11

agent/wef-columbus-depth-13-authority-gaps-records-request

agent/bhr-partners-depth-10

agent/bhr-partners-depth-6

agent/wef-columbus-depth-10-ride-drive-vendor-reconciliation

agent/wef-columbus-depth-10-po578849-current-status

agent/wef-columbus-depth-11-po525642-records-request

agent/wef-columbus-depth-11-membership-records-request

agent/wef-thiel-company-employee-enumeration-depth-2

agent/dnc-fec-project-veritas-seed

archive/pr-1912-pre-takeover-6965250

agent/start-violent-offenders-dataset

agent/cloarida-seed

agent/gop-fec-wef-depth-3-rebased

agent/gop-wef-palantir-aipac-depth-2-rebased

agent/gop-jfc-depth-1-rebased

agent/gop-national-depth-0-rebased

agent/alumni-people-directory

agent/trump-thiel-vance-palantir-rebased

agent/trump-thiel-circle-cabinet-depth-2-rebased

agent/bhr-partners-depth-1-5-rebased

agent/thiel-company-wef-enumeration-repaired

agent/brett-leatherman-fed-wef-rebased

agent/fed-executive-cabinet-advisers-repaired

agent/gop-fec-wef-depth-3

agent/issue-68-usaid-doge-mortality-rebased

agent/greater-israel-people-orgs-us-ties-rebased

agent/enumerate-palantir-lobbying-rebased

agent/trump-netanyahu-israel-business-autodig

agent/enumerate-palantir-lobbying

codex/palantir-graph-connectivity-rationale-20260731

agent/update-quasar-dataset-scope

archive/pr-105-pre-takeover-f8035f2

agent/dark-academia-rosters-wave-4

agent/membership-scraper-orchestrator

archive/pr-140-pre-takeover-34a0b40

agent/fed-executive-branch-secretaries-advisors

agent/membership-list-url-issue-invariant

agent/wef-depth-7-scrapers

agent/gop-privatization-leads-depth-3

agent/cia-inqtel-orgs-people

agent/trump-thiel-circle-seed

agent/lexisnexis-federal-contract-census-pass-8

agent/lexisnexis-brand-recipient-pass-7

agent/hunter-biden-bhr-partners

agent/lexisnexis-recipient-identity-pass-6

agent/wef-thiel-company-employee-enumeration-depth-1

fix/influencewatch-raw-hex-user-agent

agent/lexisnexis-contract-expansion-pass-5

agent/dark-academia-membership-recursion

agent/greater-israel-people-orgs-us-ties-2026-07-31

agent/influence-watch-db-importer

agent/investigate-brett-leatherman-fed

agent/lexisnexis-risk-state-surveillance

agent/trump-thiel-vance-fec-ngo-control-test

agent/gop-privatization-network-depth-3

agent/gop-wef-palantir-aipac-depth-2

agent/gop-jfc-network-depth-1

agent/wef-annual-meeting-partners-2015-2026

agent/seed-gop-depth-0

agent/palantir-depth-7-irs-portfolio-controls

agent/fix-site-hunter-wef-shapers

archive/pr-78-pre-takeover-a43c37e

agent/hunter-4plebs-recursive-auto-dig

agent/issue-68-usaid-doge-mortality

agent/import-wef-global-shapers-alumni

feat/use-new-quasar

docs/4plebs-iphone-backup-usage

agent/iphone-backup-extractor

agent/hunter-email-message-person-docs

agent/hunter-biden-dataset

agent/add-starintel-discord-footer

agent/wef-trump-vance-fink-palantir

agent/add-drunk-drivers-dataset

agent/complete-corpus-manifest

agent/us-israel-military-integration

auto-dig/usc4am-event-expense-verification-2026-07-27

agent/pin-quasar-input-map

agent/fix-quasar-route-selection

auto-dig/usc4am-local-governance-accountability-2026-07-27

agent/pin-quasar-pan-density-fix

auto-dig/wef-oakland-county-centre-finance-2026-07-27

agent/quasar-graph-editor-only

auto-dig/wef-smartstart-state-local-implementation-2026-07-27

auto-dig/wef-lattice-ami-us-government-capture-depth-4

auto-dig/wef-smartstart-government-bridges-2026-07-27

auto-dig/wef-lattice-ami-depth-3-2026-07-27

auto-dig/wef-smartstart-final-verification-depth-5

auto-dig/wef-smartstart-new-york-curriculum-depth-4

auto-dig/wef-smartstart-launch-coalition-depth-3

auto-dig/wef-lattice-ami-depth-2-2026-07-27

auto-dig/wef-smartstart-implementation-depth-2

auto-dig/wef-ecp-requisition-graph-depth-1

auto-dig/wef-lattice-ami-2026-07-27

research/wef-ecp-reddit-recursive-seed

agent/research-node-spec

agent/embed-auto-dig-quasar

agent/research-node-spec-clean

agent/flock-shotspotter-locality-recursion

agent/flock-columbus-cpd-control-plane

agent/flock-cpd-next-level

agent/complete-corpus-download

agent/starintel-v090-conformance

agent/aipac-us-israeli-enablement

agent/tech-bro-seed

agent/topic-datasets-offshore-aleph

agent/wef-depth-5-policy-contract-implementation

agent/devilcorp-seed-clean

agent/devilcorp-research-seed

agent/devilcorp-parent-network-seed

agent/fix-auto-dig-graph-workspace

agent/palantir-depth-6-controls-ethics-ledgers

agent/theme-system

agent/schema-org-v0.9

agent/flock-national-state-locality-seed

agent/use-canonical-schema-origin

fix/use-spec-starintel-actor

agent/starintel-v0.9-field-expansion

agent/wef-columbus-depth-5-documents

agent/palantir-depth-5-controls-chronology

agent/wef-columbus-depth-4-documents

agent/palantir-depth-4-contracts-ethics-portfolio

agent/wef-columbus-depth-3-documents

agent/palantir-related-orgs-loop

agent/render-relations-as-edges

agent/palantir-vance-thiel-executive-loop

agent/revert-unauthorized-tooling

agent/wef-columbus-depth-2-v090

agent/enforce-scripted-schema-writes

agent/fix-gao-c3ai-cdc-citation

agent/shareable-pathfinder

agent/starintel-v0.9.0

agent/mobile-pinch-zoom

agent/graph-physics-layouts

agent/larry-fink-fec-ledger

agent/wef-columbus-influence

agent/publish-palantir-starintel

agent/larry-fink-q1-lobbying-update

agent/larry-fink-q2-governance-update

agent/agent-research-ledger

agent/larry-fink-research-site

agent/larry-fink-exploration-site

agent/flock-safety-columbus-local-government

agent/starlang-loader-correctness-loop

No labels

github-mirror

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starintel-labs/starintel-gpt-auto-dig#996

No description provided.

Rows
Columns