IPX replay drops captured request/response headers from canonical exchange records #181

New issue

Open

opened 2026-09-05 04:33:00 +00:00 by mara · 0 comments

mara commented

2026-09-05 04:33:00 +00:00

Collaborator

Reviewed at commit 3b4e2da (filed against the IPX replay introduced in 6f40a44, which advances #26).

Problem
tools/ipx/mitmproxy-addon.py captures request/response headers losslessly as headers_raw_b64 pairs, and hackmode-database already has a typed, sanitized header path (source/hackmode-database/http-exchange-headers.lisp, added in 595e2fd: :request-headers / :response-headers slots on make-http-exchange-record with %sanitize-http-headers).

But the replay path (source/hackmode-core/capture-replay.lisp, ipx-object->http-exchange, lines ~102-128) drops them entirely: it builds the exchange record with only method/scheme/host/port/path/status. The captured header evidence never reaches the canonical graph.

Impact

Header-based recon fingerprinting (server banners, CSP, cookies pre-redaction, HSTS) is impossible from the graph.
The addon docstring promises lossless capture; replay silently violates it.
The DB header validation/secret-rejection path is bypassed rather than exercised.

Suggested fix
In ipx-object->http-exchange, decode headers_raw_b64 (the addon emits [b64(name), b64(value)] list pairs, so convert to (name . value) strings) and pass :request-headers / :response-headers to make-http-exchange-record. Add a RED-first test in tests/ipx-replay.lisp asserting header pairs survive into the persisted payload and that secret-bearing headers are rejected by the existing sanitizer path (they remain available in the raw spool evidence via raw-evidence-ref).

Reviewed at commit 3b4e2da (filed against the IPX replay introduced in 6f40a44, which advances #26). **Problem** `tools/ipx/mitmproxy-addon.py` captures request/response headers losslessly as `headers_raw_b64` pairs, and `hackmode-database` already has a typed, sanitized header path (`source/hackmode-database/http-exchange-headers.lisp`, added in 595e2fd: `:request-headers` / `:response-headers` slots on `make-http-exchange-record` with `%sanitize-http-headers`). But the replay path (`source/hackmode-core/capture-replay.lisp`, `ipx-object->http-exchange`, lines ~102-128) drops them entirely: it builds the exchange record with only method/scheme/host/port/path/status. The captured header evidence never reaches the canonical graph. **Impact** - Header-based recon fingerprinting (server banners, CSP, cookies pre-redaction, HSTS) is impossible from the graph. - The addon docstring promises lossless capture; replay silently violates it. - The DB header validation/secret-rejection path is bypassed rather than exercised. **Suggested fix** In `ipx-object->http-exchange`, decode `headers_raw_b64` (the addon emits `[b64(name), b64(value)]` list pairs, so convert to `(name . value)` strings) and pass `:request-headers` / `:response-headers` to `make-http-exchange-record`. Add a RED-first test in `tests/ipx-replay.lisp` asserting header pairs survive into the persisted payload and that secret-bearing headers are rejected by the existing sanitizer path (they remain available in the raw spool evidence via `raw-evidence-ref`).

mara added the

bug

label

2026-09-05 04:33:00 +00:00