IPX passive traffic stack: capture proxy -> Hackpert graph -> StarIntel ingest #50
Labels
No labels
bug
documentation
duplicate
enhancement
feature
good first issue
help wanted
invalid
question
refactor
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
nsaspy/hackmode#50
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Outcome
Add the IPX stack to Hackmode: a local proxy/capture boundary records network/application traffic as durable evidence, Hackpert analyzes the capture passively, Common Lisp validates the resulting graph/KB deltas, and accepted observations are projected through Hackmode's canonical StarIntel outbox.
This is a passive-intelligence path. Observing traffic must never implicitly trigger active scanning, exploitation, replay, mutation, or provider execution.
Related expert architecture: #24
Foundation PR: #25
Core flow
Ownership / authority
First protocol: HTTP(S)
Start with HTTP request/response exchanges captured through the proxy. Preserve enough evidence to reconstruct provenance without forcing all raw bodies into the KB.
Typed exchange metadata should cover at least:
Later protocol adapters may add WebSocket, SSE, raw TCP, DNS, or other application traces without changing the graph/KB authority model.
Passive Hackpert analysis
Hackpert should be able to derive graph/KB candidates such as:
The result is a graph delta with provenance back to the exact capture record(s).
Operational / long-term / global KB
Traffic-derived knowledge follows #24's KB lifecycle:
Secret / privacy boundary
Raw proxy captures may contain credentials and sensitive application data, so they require stronger handling than derived graph facts.
Requirements:
Authorization, cookies, API keys, common secret fields, etc.);File/spool semantics
StarIntel ingest
Hackpert does not write StarIntel documents directly.
Flow:
capture evidence -> typed exchange -> accepted graph/asset/finding -> Hackmode StarIntel projection -> durable outbox -> starintel-serverIngest should reuse canonical StarIntel document identities and provenance. A Rabbit/HTTP publish is not proof of acceptance; keep the existing durable outbox/ack semantics.
First implementation slices
Acceptance proof
Non-goals for the first slice
Database/execution-graph prerequisite merged via #84 as
d39fdafe627bc3363ef89fc02a96fa931c9b07f6.Completed owned slice:
:http-exchangeevidence records persist in the existing operation-scoped Tek9 execution graph; no second graph/database;:http-exchangefiltering;RED
0b1cde84cf1986588d8aea42a6a650327295e247: core failed at the newly required database regression while monorepo and agent-framework-boundary passed.GREEN exact implementation head
e29ad25cb3592ad52071f535ef5633423ecb5ccb: core, monorepo, and agent-framework-boundary all passed; PR was mergeable with zero reviews/threads before expected-head squash merge.Consumer-side capture/parser/Hackpert wiring remains outside the database worker.
Hackpert passive HTTP-evidence projection merged via #86 at
a858a6f4fd3e2192dc0a3d98901f62d29784672d.Database #84 supplied the canonical typed HTTP exchange record. Hackpert now projects that record into a dedicated bounded
http_exchange(...)Prolog fact carrying operation/capture/exchange identity plus method/scheme/host/port/path/status/body digests/observed time/duration. The raw capture evidence reference and generic payload/provenance serialization are intentionally excluded from Prolog.This is projection-only passive reasoning input: no provider execution, capture implementation, database mutation, StarIntel product work, or second scheduler/effect path was added.
RED
1dfcedf3c41173078d47cf4c58da87b7fb757552: exact-head core failed on the missing typed HTTP fact while monorepo and agent-framework-boundary passed.GREEN exact head
750ea18f813909b53cef4ec3d89be13888b74923: core, monorepo, and agent-framework-boundary all passed twice across the identical-head draft/non-draft PR transition. PR #86 was mergeable with zero reviews and zero unresolved review threads before expected-head squash merge.Draft #85 was closed only because the ready-for-review GraphQL mutation hit the connector's known
fullDatabaseIdincompatibility; #86 reused the identical green head.Database handoff: PR #87 merged the durable replay-checkpoint prerequisite into the canonical operation-scoped Tek9 execution graph. Capture checkpoints are immutable typed records scoped by operation + capture session + source, carry framing/provenance plus optional last-record evidence, replay idempotently, and expose a deterministic latest-checkpoint read by greatest durable offset. Same source/session/offset with changed content is protected by the existing replay-conflict boundary. Parser/tailer ownership can now resume only after its last committed framed record; parser actor, capture format, quarantine, and Hackpert consumption remain separate slices.
Database handoff: PR #93 merged malformed/truncated capture quarantine into the canonical operation-scoped Tek9 execution graph as
b2ffadaf951038d742774061915cdc531d425eff. Quarantine records are immutable typed:capture-quarantineevidence scoped by operation + capture session + source, retain offset/length, failure reason, raw-evidence reference, framing version, and parser provenance, replay idempotently, and read back in durable offset order. Exact PR head38ed0b4633eca913b6310f9a420639b1159c8188passed core, monorepo, and agent-framework-boundary; master was unchanged, the PR was mergeable, and it had zero reviews/threads before expected-head squash merge. Parser/tailer logic remains outside this database slice.Database slice merged via #96 (
f67a114d66856699a75ae04f5000844049635712). Canonical Tek9 execution-graph persistence now includes immutable, operation/capture-scoped capture-source rotation lineage with deterministic replay identity and durable-offset ordering. Parser/tailer ownership stays outside the database boundary: consumers should persist the rotation record when a predecessor source closes/rotates and then resume the successor source using the existing checkpoint/quarantine contracts. No shadow persistence authority was introduced.Database/graph progress: merged #105 as
3ca1da553db6c2218a16524f7b8e7b204686b25dto reject divergent capture-source rotation forks inside the canonical Tek9 execution-graph write transaction.Contract:
persistence-replay-conflict;RED regression:
96a7c2d3fb58056a4969d8eb824619190213adf3.GREEN exact PR head:
340dde0d0d798ba65e5a68d52565111681fa228b;core,monorepo, andagent-framework-boundaryall passed, with zero reviews/threads before expected-head squash merge.Parser/tailer handoff: treat the predecessor->successor rotation relation as a single-valued durable lineage edge. Replaying the same edge is safe; discovering a contradictory successor is an explicit persistence conflict to quarantine/escalate rather than silently choosing one.
Operator direction recorded in #133: make one operation-scoped mitmproxy/mitmdump service the default HTTP capture path, with HTTP-capable providers routed through it where protocol semantics permit. Keep this issue's IPX spool -> checkpoint/quarantine/rotation -> typed
:http-exchangepersistence as the single capture authority; the proxy addon/process writes the spool and never Tek9 directly.Important transport invariant added in #133: full TLS interception and end-to-end curl/browser TLS impersonation are mutually different modes. Capture provenance must say
interceptvstunnel/direct rather than claiming both decrypted traffic and the original client's origin-side TLS fingerprint.ZAP should reuse this capture path for outbound traffic instead of introducing a second HTTP-history ingestion pipeline. ZAP-specific alert/spider/scan control remains #21/#22.
Concrete implementation children now exist for the capture half:
Database checkpoint/quarantine/rotation/http-exchange persistence remains canonical; provider/addon workers must not write Tek9 directly.
Database/capture persistence slice landed via #174 (
6f40a4461809d6509ea8c7f02afcc1db0ce58b31).RED: test-only
a438518b59adb7c437f6f84c22e65631a5baa4c1failed the repository-native Common Lisp gate after dependencies installed becauseHACKMODE:REPLAY-IPX-HTTP-SPOOLwas absent.GREEN: exact implementation head
898b5f85d35947cec9debd3b05f597a119cc130apassed Common Lisp core, hygiene, and the product/framework boundary. The landed replay boundary reads binary byte offsets, validates operation/capture-session/source identity, persists canonical typed:http-exchangerecords, resumes from durable checkpoints, quarantines malformed complete frames while advancing past them, and quarantines truncated final frames without advancing the checkpoint. Persistence failures propagate rather than being mislabeled as corrupt source evidence.The lossless IPX spool remains source evidence authority; no second operation DB/graph/KB/scheduler was introduced.
#26 remains open for its broader remaining tailer/analysis/operator-control work.