Current Python transport admission: bounded queue + rate limits + hard 429 before client timeout #91

New issue

Open

opened 2026-09-20 01:19:24 +00:00 by nsaspy · 0 comments

nsaspy commented

2026-09-20 01:19:24 +00:00

Owner

Supersedes/reconciles the stale Common Lisp transport implementation work in #70/#74/#75/#76/#77 for the current architecture, where Python owns HTTP/SSE/WebSocket transport.

Required behavior

one admission owner per configured quota group;
max active requests;
requests/minute + burst token bucket;
bounded FIFO queue;
default maximum queue wait 10 seconds (configurable);
when admission cannot occur by the queue deadline, return HTTP 429 before opening upstream;
queue-full returns immediate 429;
local 429 includes Retry-After and Cache-Control: no-store;
cancellation removes waiter and consumes no request token;
no upstream work after queue deadline;
shared aliases/OpenCode presentation cannot multiply provider capacity.

Client-timeout safety

Keep the default 10s admission deadline below the normal OpenCode/Codex/GPT request timeout envelope. Do not hold a transparent request indefinitely merely because upstream quota is exhausted.

Streaming requests use the same pre-response admission gate. No SSE/body bytes are sent before admission in transparent mode.

Tests

Deterministic monotonic clock tests + aiohttp integration tests for queue/rate/full/deadline/cancel/SSE/no-upstream-contact.

Supersedes/reconciles the stale Common Lisp transport implementation work in #70/#74/#75/#76/#77 for the current architecture, where Python owns HTTP/SSE/WebSocket transport. ## Required behavior - one admission owner per configured quota group; - max active requests; - requests/minute + burst token bucket; - bounded FIFO queue; - default maximum queue wait 10 seconds (configurable); - when admission cannot occur by the queue deadline, return HTTP 429 before opening upstream; - queue-full returns immediate 429; - local 429 includes Retry-After and Cache-Control: no-store; - cancellation removes waiter and consumes no request token; - no upstream work after queue deadline; - shared aliases/OpenCode presentation cannot multiply provider capacity. ## Client-timeout safety Keep the default 10s admission deadline below the normal OpenCode/Codex/GPT request timeout envelope. Do not hold a transparent request indefinitely merely because upstream quota is exhausted. Streaming requests use the same pre-response admission gate. No SSE/body bytes are sent before admission in transparent mode. ## Tests Deterministic monotonic clock tests + aiohttp integration tests for queue/rate/full/deadline/cancel/SSE/no-upstream-contact.

nsaspy referenced this issue

2026-09-20 06:13:35 +00:00

Validate tool arguments against supplied JSON Schema #101