nsaspy/llm-log
1
0
Fork 0
Code Issues 34 Pull requests 19 Projects Releases Packages Wiki Activity Actions

Add provider-wide request scheduling and 429 backpressure #74

Open
nsaspy wants to merge 14 commits from feat/70-request-scheduler into main
pull from: feat/70-request-scheduler
merge into: nsaspy:main
Conversation 2 Commits 14 Files changed 9 +654 -41
nsaspy commented 2026-09-12 20:08:46 +00:00
Owner
Copy link

Closes #70.

Implements the first llm-log traffic-scheduler slice:

  • provider-keyed active-request limits;
  • bounded FIFO queues;
  • queue wait deadlines;
  • 429 Too Many Requests with Retry-After when full or expired;
  • no upstream socket is opened for rejected requests;
  • slot release is protected across relay success/failure;
  • scheduler TOML configuration with bounded defaults;
  • unit coverage for limits, FIFO ordering, queue full/timeout, and provider isolation;
  • raw HTTP integration coverage for 429 and Retry-After.

Transparent queued requests do not emit SSE body heartbeats because doing so would commit the downstream HTTP response before the upstream status/headers exist. The explicit smart-client streaming protocol is tracked in #73; Agent Zero retry/queue integration remains #72.

Closes #70. Implements the first llm-log traffic-scheduler slice: - provider-keyed active-request limits; - bounded FIFO queues; - queue wait deadlines; - `429 Too Many Requests` with `Retry-After` when full or expired; - no upstream socket is opened for rejected requests; - slot release is protected across relay success/failure; - scheduler TOML configuration with bounded defaults; - unit coverage for limits, FIFO ordering, queue full/timeout, and provider isolation; - raw HTTP integration coverage for 429 and Retry-After. Transparent queued requests do not emit SSE body heartbeats because doing so would commit the downstream HTTP response before the upstream status/headers exist. The explicit smart-client streaming protocol is tracked in #73; Agent Zero retry/queue integration remains #72.
fix: wake idle Woo loop during graceful shutdown
Some checks failed
Common Lisp proxy contracts / proxy-contracts (pull_request) Failing after 3s
Details
ac26a689c5
nsaspy commented 2026-09-21 20:12:52 +00:00
Author
Owner
Copy link

Current-head audit at ac26a689c5:

  • The scheduler's existing unit assertions passed in Actions run 34715742893, job 103612546205.
  • The full gate is NOT green: after the first HTTP transport test passed at 19:59:49Z, execution stalled until 20:14:49Z and terminated with TYPE-ERROR in the Woo event-loop callback (NIL is not WOO.EV.SOCKET:SOCKET). This is a transport/lifecycle blocker, not evidence that rate limiting is fully verified.
  • The workflow step named "Checkout exact head" actually checked out synthetic merge e838ed4fafe2d93198324ad013feb00f60a29fe3, not the PR head. Exact-head validation must be explicit.
  • Active request concurrency is bounded, but there is no request-start rate/burst budget yet. A rapid sequence of short requests can still exhaust a provider's rate allowance.

The user explicitly prioritized rate limiting above profile work. #71 now records applied, versioned outbound privacy/compatibility profiles for Hermes/OpenClaw/Agent Zero with one shared limiter independent of profile. A bounded follow-on will add request-rate enforcement and the header-profile foundation without claiming this failing full transport gate is green. No merge/deployment claim.

Current-head audit at ac26a689c56b1cf2b6c4999ad012b9bd2ced8497: - The scheduler's existing unit assertions passed in Actions run 34715742893, job 103612546205. - The full gate is NOT green: after the first HTTP transport test passed at 19:59:49Z, execution stalled until 20:14:49Z and terminated with TYPE-ERROR in the Woo event-loop callback (`NIL` is not `WOO.EV.SOCKET:SOCKET`). This is a transport/lifecycle blocker, not evidence that rate limiting is fully verified. - The workflow step named "Checkout exact head" actually checked out synthetic merge e838ed4fafe2d93198324ad013feb00f60a29fe3, not the PR head. Exact-head validation must be explicit. - Active request concurrency is bounded, but there is no request-start rate/burst budget yet. A rapid sequence of short requests can still exhaust a provider's rate allowance. The user explicitly prioritized rate limiting above profile work. #71 now records applied, versioned outbound privacy/compatibility profiles for Hermes/OpenClaw/Agent Zero with one shared limiter independent of profile. A bounded follow-on will add request-rate enforcement and the header-profile foundation without claiming this failing full transport gate is green. No merge/deployment claim.
nsaspy commented 2026-09-21 20:12:52 +00:00
Author
Owner
Copy link

Validation blocker confirmed from the existing parent head ac26a689c5 (before rate implementation):

  • Actions run 34715742893 / job 103612546205, nix build -L .#checks.x86_64-linux.llm-log-config-contract.
  • Production/test compilation and the config/scheduler assertions succeeded.
  • http-method-path-and-query-are-preserved returned 200 and passed its method/path/query assertions at 19:59:49 UTC.
  • Execution then hung for 15 minutes. At 20:14:49 UTC, the Woo proxy thread exited with TYPE-ERROR: NIL is not of type WOO.EV.SOCKET:SOCKET, in the TCP/libev callback (src/ev/tcp.lisp).

Investigate socket/watch ownership and shutdown/startup lifecycle; do not treat this as a rate-limiter regression or an infrastructure-only failure. The exact cause still needs isolation.

PR #76 is stacked here and adds time-based admission. It preserves all full-suite tests and adds an independent rate-contract job so rate assertions can be evaluated without hiding this HTTP blocker. Neither PR should merge on partial green.

Validation blocker confirmed from the existing parent head ac26a689c56b1cf2b6c4999ad012b9bd2ced8497 (before rate implementation): - Actions run 34715742893 / job 103612546205, `nix build -L .#checks.x86_64-linux.llm-log-config-contract`. - Production/test compilation and the config/scheduler assertions succeeded. - `http-method-path-and-query-are-preserved` returned 200 and passed its method/path/query assertions at 19:59:49 UTC. - Execution then hung for 15 minutes. At 20:14:49 UTC, the Woo proxy thread exited with TYPE-ERROR: `NIL` is not of type `WOO.EV.SOCKET:SOCKET`, in the TCP/libev callback (`src/ev/tcp.lisp`). Investigate socket/watch ownership and shutdown/startup lifecycle; do not treat this as a rate-limiter regression or an infrastructure-only failure. The exact cause still needs isolation. PR #76 is stacked here and adds time-based admission. It preserves all full-suite tests and adds an independent rate-contract job so rate assertions can be evaluated without hiding this HTTP blocker. Neither PR should merge on partial green.
Some checks failed
Common Lisp proxy contracts / proxy-contracts (pull_request) Failing after 3s
Details
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin feat/70-request-scheduler:feat/70-request-scheduler
git switch feat/70-request-scheduler

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff feat/70-request-scheduler
git switch feat/70-request-scheduler
git rebase main
git switch main
git merge --ff-only feat/70-request-scheduler
git switch feat/70-request-scheduler
git rebase main
git switch main
git merge --no-ff feat/70-request-scheduler
git switch main
git merge --squash feat/70-request-scheduler
git switch main
git merge --ff-only feat/70-request-scheduler
git switch main
git merge feat/70-request-scheduler
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
accessibility
Barrier affecting people with disabilities

  
bug
Something isn't working

  
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
invalid
This doesn't seem right

  
question
Further information is requested

  
wontfix
This will not be worked on

No labels
accessibility
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
nsaspy/llm-log!74
No description provided.