Port curated legacy recon scripts into typed Common Lisp provider actors #2

New issue

Open

opened 2023-12-02 05:12:01 +00:00 by lost-rob0t · 1 comment

lost-rob0t commented

2023-12-02 05:12:01 +00:00

(Migrated from github.com)

Problem

Shell glue from the former lost-rob0t/hackmode-scripts repository captures useful recon workflows, but it bypasses Hackmode's typed assets, operation store, event lifecycle, and future actor supervision.

The source migration itself is complete in #5. This issue now tracks replacing that compatibility/reference source with maintainable Common Lisp capabilities.

Current behavior

Curated non-executable migration sources live under tools/legacy/recon/:

alt.py — domain word/permutation logic
bbot.sh — BBOT orchestration
dex.sh — fan-out recon workflow prototype
dns-up.sh — massdns resolution/filtering
dork.sh — domain-scoped search wrapper
parse-nmap.py — Nmap XML conversion
photon-scan.sh — crawler wrapper

They are intentionally not invoked by Hackmode.

Desired behavior

Port useful behavior into Common Lisp provider/actor implementations that consume and emit typed Hackmode/StarIntel-compatible assets through the canonical lifecycle:

asset -> provider actor -> normalized result -> discover-asset -> local Tek9 store -> asset event

No provider should write ad-hoc canonical state to findings text files.

Architecture / ownership

Suggested port targets:

DNS/massdns -> DomainSupervisor / DNS provider
Nmap XML import -> HostSupervisor parser/import adapter
BBOT -> ProviderSupervisor adapter that emits typed domains/URLs/services/findings
Photon/crawling -> WebSupervisor crawler provider
word/permutation generation -> DomainSupervisor
domain-scoped public search -> ProviderSupervisor
dex.sh -> decompose into asset-event subscriptions and recon jobs; do not reproduce its FIFO shell manager literally

Reuse the canonical operation/asset protocol from #6. Provider implementations should eventually be reusable by Quasar rather than copied into separate Quasar-specific modules.

Dependencies

#4 monorepo migration — complete
#5 script source curation — complete
#6 canonical operation/asset protocol — first runtime slice merged; issue remains open
#9 reproducible Common Lisp CI for executable regression tests

Acceptance criteria

no runtime dependency on tools/legacy/recon/*
each retained script has a Common Lisp replacement or an explicit not-planned decision
provider failure is isolated and observable
results enter the local operation store through discover-asset
repeated results do not create duplicate assets/jobs
provider-specific credentials/config remain backend-only
shell and Emacs clients dispatch capabilities instead of embedding provider implementations

Verification

For each port:

deterministic fixture/provider test
provider failure test
typed asset output test
dedupe/repeated-result test
verify no write path bypasses the canonical operation store
mark the corresponding entry in docs/migration/hackmode-scripts.org as replaced

## Problem Shell glue from the former `lost-rob0t/hackmode-scripts` repository captures useful recon workflows, but it bypasses Hackmode's typed assets, operation store, event lifecycle, and future actor supervision. The source migration itself is complete in #5. This issue now tracks replacing that compatibility/reference source with maintainable Common Lisp capabilities. ## Current behavior Curated non-executable migration sources live under `tools/legacy/recon/`: - `alt.py` — domain word/permutation logic - `bbot.sh` — BBOT orchestration - `dex.sh` — fan-out recon workflow prototype - `dns-up.sh` — massdns resolution/filtering - `dork.sh` — domain-scoped search wrapper - `parse-nmap.py` — Nmap XML conversion - `photon-scan.sh` — crawler wrapper They are intentionally not invoked by Hackmode. ## Desired behavior Port useful behavior into Common Lisp provider/actor implementations that consume and emit typed Hackmode/StarIntel-compatible assets through the canonical lifecycle: `asset -> provider actor -> normalized result -> discover-asset -> local Tek9 store -> asset event` No provider should write ad-hoc canonical state to findings text files. ## Architecture / ownership Suggested port targets: - DNS/massdns -> DomainSupervisor / DNS provider - Nmap XML import -> HostSupervisor parser/import adapter - BBOT -> ProviderSupervisor adapter that emits typed domains/URLs/services/findings - Photon/crawling -> WebSupervisor crawler provider - word/permutation generation -> DomainSupervisor - domain-scoped public search -> ProviderSupervisor - `dex.sh` -> decompose into asset-event subscriptions and recon jobs; do not reproduce its FIFO shell manager literally Reuse the canonical operation/asset protocol from #6. Provider implementations should eventually be reusable by Quasar rather than copied into separate Quasar-specific modules. ## Dependencies - #4 monorepo migration — complete - #5 script source curation — complete - #6 canonical operation/asset protocol — first runtime slice merged; issue remains open - #9 reproducible Common Lisp CI for executable regression tests ## Acceptance criteria - no runtime dependency on `tools/legacy/recon/*` - each retained script has a Common Lisp replacement or an explicit not-planned decision - provider failure is isolated and observable - results enter the local operation store through `discover-asset` - repeated results do not create duplicate assets/jobs - provider-specific credentials/config remain backend-only - shell and Emacs clients dispatch capabilities instead of embedding provider implementations ## Verification For each port: 1. deterministic fixture/provider test 2. provider failure test 3. typed asset output test 4. dedupe/repeated-result test 5. verify no write path bypasses the canonical operation store 6. mark the corresponding entry in `docs/migration/hackmode-scripts.org` as replaced

lost-rob0t commented

2026-08-16 14:18:07 +00:00

(Migrated from github.com)

Progress: PR #19 merged (24a276d) with the first executable provider slice.

Landed:

typed capability/provider registry and structured provider-job-result
shared Sento runtime with async dispatch-capability -> Future
short-lived provider worker actors for concurrent backend I/O
ordered supervisor completion plus a persistence lock protecting Tek9 dedupe/write state
canonical provider results persist only through discover-asset
dns-up.sh replaced by hackmode-provider-dns / :dns-resolve / :massdns
deterministic MassDNS parser/provider/dedupe/failure tests; no live DNS dependency in CI
migration inventory marks DNS/massdns DONE

#2 remains open. Next implementation boundary: add parent-aware provider emissions (overlaps #3 parent/child asset relationships) so Nmap XML can emit host plus parented port/service assets through the same lifecycle. Then port Nmap, followed by BBOT/Photon/etc.

Progress: PR #19 merged (`24a276d`) with the first executable provider slice. Landed: - typed capability/provider registry and structured `provider-job-result` - shared Sento runtime with async `dispatch-capability` -> Future - short-lived provider worker actors for concurrent backend I/O - ordered supervisor completion plus a persistence lock protecting Tek9 dedupe/write state - canonical provider results persist only through `discover-asset` - `dns-up.sh` replaced by `hackmode-provider-dns` / `:dns-resolve` / `:massdns` - deterministic MassDNS parser/provider/dedupe/failure tests; no live DNS dependency in CI - migration inventory marks DNS/massdns DONE #2 remains open. Next implementation boundary: add parent-aware provider emissions (overlaps #3 parent/child asset relationships) so Nmap XML can emit `host` plus parented `port`/service assets through the same lifecycle. Then port Nmap, followed by BBOT/Photon/etc.