Typed curl-impersonate HTTP requester with IPX/capture integration #34

New issue

Closed

opened 2026-09-02 17:24:36 +00:00 by nsaspy · 1 comment

nsaspy commented

2026-09-02 17:24:36 +00:00

Owner

Outcome

Add a typed curl-like HTTP capability that prefers a maintained/pinned curl-impersonate-class backend when available, integrates with Hackmode capture policy, and normalizes results before they reach canonical state.

Owning RAGE lane

Provider/runtime/tool worker.

Fence

Do not parse arbitrary subprocess stdout directly into KB state. Do not implement Hackpert selection policy here. Do not claim browser TLS impersonation is preserved through a decrypting intercept proxy when it is not.

Dependencies

#137 client/transport profiles
#135 capture provider
#26 typed HTTP evidence

Research gate

Verify current curl-impersonate maintenance/packaging state. If unsuitable, document why and choose the closest maintained equivalent with comparable browser-profile semantics instead of blindly adding an abandoned dependency.

Typed interface

Request: URL/method, bounded headers/body reference, redirect policy, timeout/deadline, operation/run identity, selected client profile, capture mode.
Result: final URL/status, bounded/redacted headers, body digest/evidence reference, timing/error classification, backend/version, capture/provenance metadata.

Runtime

capability detection on startup;
graceful fallback to an explicitly supported ordinary HTTP backend when policy permits;
operation budget/deadline enforcement;
deterministic redirect behavior;
proxy/IPX integration for intercept/tunnel modes;
no secret leakage to process logs or command diagnostics.

RED-first tests

Backend unavailable fallback, profile propagation, proxy configuration, timeout, redirect policy, secret redaction, normalized result, and execution provenance.

Acceptance proof

Execute deterministic local HTTP fixtures using an impersonating backend and fallback backend, prove typed-equivalent normalized results, and show execution provenance identifies backend/profile/capture mode without raw provider output becoming canonical state.

## Outcome Add a typed curl-like HTTP capability that prefers a maintained/pinned `curl-impersonate`-class backend when available, integrates with Hackmode capture policy, and normalizes results before they reach canonical state. ## Owning RAGE lane Provider/runtime/tool worker. ## Fence Do not parse arbitrary subprocess stdout directly into KB state. Do not implement Hackpert selection policy here. Do not claim browser TLS impersonation is preserved through a decrypting intercept proxy when it is not. ## Dependencies - #137 client/transport profiles - #135 capture provider - #26 typed HTTP evidence ## Research gate Verify current `curl-impersonate` maintenance/packaging state. If unsuitable, document why and choose the closest maintained equivalent with comparable browser-profile semantics instead of blindly adding an abandoned dependency. ## Typed interface Request: URL/method, bounded headers/body reference, redirect policy, timeout/deadline, operation/run identity, selected client profile, capture mode. Result: final URL/status, bounded/redacted headers, body digest/evidence reference, timing/error classification, backend/version, capture/provenance metadata. ## Runtime - capability detection on startup; - graceful fallback to an explicitly supported ordinary HTTP backend when policy permits; - operation budget/deadline enforcement; - deterministic redirect behavior; - proxy/IPX integration for intercept/tunnel modes; - no secret leakage to process logs or command diagnostics. ## RED-first tests Backend unavailable fallback, profile propagation, proxy configuration, timeout, redirect policy, secret redaction, normalized result, and execution provenance. ## Acceptance proof Execute deterministic local HTTP fixtures using an impersonating backend and fallback backend, prove typed-equivalent normalized results, and show execution provenance identifies backend/profile/capture mode without raw provider output becoming canonical state.

nsaspy closed this issue

2026-09-02 17:24:36 +00:00

nsaspy commented

2026-09-02 17:24:50 +00:00

Author

Owner

Merged contract slice in #162 as 8d23204bf95fc7053a4c4ac7ac1b27a97330b50a on current master. This adds the operation/run-scoped typed HTTP request, explicit redirect/timeout policy, coherent client-profile/capture provenance, and tests. #138 remains open intentionally. Next acceptance work: maintained impersonating backend integration (research gate currently favors curl_cffi 0.16.2), capability detection + explicit fallback, proxy/IPX execution, normalized typed results, actual deadline/redirect enforcement, and deterministic local fixture proof. Superseded diverged draft #159 was closed.

Merged contract slice in #162 as `8d23204bf95fc7053a4c4ac7ac1b27a97330b50a` on current `master`. This adds the operation/run-scoped typed HTTP request, explicit redirect/timeout policy, coherent client-profile/capture provenance, and tests. #138 remains open intentionally. Next acceptance work: maintained impersonating backend integration (research gate currently favors `curl_cffi` 0.16.2), capability detection + explicit fallback, proxy/IPX execution, normalized typed results, actual deadline/redirect enforcement, and deterministic local fixture proof. Superseded diverged draft #159 was closed.