nsaspy/llm-log
1
0
Fork 0
Code Issues 34 Pull requests 19 Projects Releases Packages Wiki Activity Actions

Rate admission: time-based quotas shared across client aliases #77

Open
nsaspy wants to merge 4 commits from feat/75-rate-admission into feat/70-request-scheduler
pull from: feat/75-rate-admission
merge into: nsaspy:feat/70-request-scheduler
Conversation 0 Commits 4 Files changed 10 +588 -73
nsaspy commented 2026-09-12 23:21:00 +00:00
Owner
Copy link

Tracks #75; stacked on #74 (feat/70-request-scheduler) rather than duplicating the scheduler foundation.

Implemented

  • Monotonic-clock request token bucket (refill RPM + bounded burst), enforced atomically with active-slot and FIFO queue limits.
  • Successful/failed completed requests release active slots only; request quota is never refunded. Rejections and cancelled/expired waiters spend none.
  • Trusted provider-prefix -> quota-group mapping; explicitly mapped Agent Zero/OpenCode/native aliases share rate, active and queue limits. Original client identity and outbound presentation do not choose quota ownership.
  • Configuration validation and immutable scheduler policy snapshots, including mutable group/alias strings.
  • Deadline-first admission, expiry pruning, unwind-safe waiter cleanup and optional cooperative cancellation.
  • Scheduler-computed rounded-up Retry-After plus Cache-Control: no-store on local 429 responses, before opening upstream connections.
  • 12 socket-free rate regressions plus one retained HTTP rate regression. All existing tests remain registered.
  • Independent rate-contract CI job plus the unchanged full proxy test command; both check out the actual PR head and have bounded job timeouts.
  • Configuration/semantics/limitations documented in research/LLM-LOG-rate-admission-75.org.

Executed validation

Exact current head: 0011613cde0ceb0d101425a7606a3029f5105c2d.

PASS: all 12 socket-free rate tests. Actions run 34724938422, rate-contracts job 103637318354, completed successfully September 12 at 23:16:06 UTC. Checked the decoded execution log: production and full test-system compilation succeeded, the runner executed every one of the 12 named tests, and Rove reported All 12 tests passed. This includes the 64-thread test admitting exactly eight requests from an eight-permit burst, shared-alias enforcement, no quota refund, exact refill/deadline boundaries, mutable-policy isolation and cancellation/error cleanup. Actual execution used SBCL 2.6.7 through the pinned Nix package closure, not local structural checks.

At the last inspection, the same run's complete proxy-contracts job 103637318468 was still in progress. The HTTP rate test has therefore NOT been certified passing. No full integration green is claimed. No local Lisp runtime execution occurred in the editing environment.

Tests-first evidence: at e81be9508efb7785df210af0948d3502fccf8682, real SBCL/Nix run 34724484407 / job 103636097666 failed test compilation because REQUESTS-PER-MINUTE/BURST/PROVIDER-GROUPS APIs did not yet exist. This is an executed missing-API compile RED, not a behavioral assertion RED.

Implementation commit: 04c323b0cd4b913368c83ff39fc605990c0d71c9.
Independent gate commit: 0011613cde0ceb0d101425a7606a3029f5105c2d.

Existing full-suite blocker

Parent #74 at ac26a689c56b1cf2b6c4999ad012b9bd2ced8497 already fails run 34715742893 / job 103612546205: first HTTP assertions pass, then a 15-minute hang and a Woo callback TYPE-ERROR (NIL is not WOO.EV.SOCKET:SOCKET). Detailed evidence is recorded on #74. The precise lifecycle defect still needs isolation. Keep this PR draft; the verified rate green does not authorize merging an unverified transport stack.

Deliberate boundaries

This is process-local refill-rate/burst admission, not a strict rolling-minute, TPM, five-hour or weekly provider quota. Defaults (60 RPM refill, burst 4) are local policy, not provider entitlement. Every group currently uses the same settings. All aliases for shared credentials must be explicitly mapped by trusted configuration. Restarts/multiple instances need a later durable/shared owner. No provider 429 cooldown ingestion, automatic retries, credential rotation or socket-disconnect cancellation integration is added. Inbound socket/thread counts are not globally bounded by this upstream scheduler.

The user's Agent Zero -> OpenCode upstream presentation requirement is explicitly recorded in #71, but the wire-profile implementation is NOT part of this slice. It must preserve internal provenance and share the same quota group; a changed User-Agent alone is not proof of full harness equivalence or live provider acceptance. Smart-client SSE queue protocol remains #73.

Commands

Full existing gate:

nix build -L .#checks.x86_64-linux.llm-log-config-contract

Verified independent rate gate:

nix shell --no-write-lock-file .#llm-log-cl-sbcl --command \
  sbcl --noinform --no-userinit --no-sysinit --non-interactive \
  --load proxy/tests/rate-runner.lisp
Tracks #75; stacked on #74 (`feat/70-request-scheduler`) rather than duplicating the scheduler foundation. ## Implemented - Monotonic-clock request token bucket (refill RPM + bounded burst), enforced atomically with active-slot and FIFO queue limits. - Successful/failed completed requests release active slots only; request quota is never refunded. Rejections and cancelled/expired waiters spend none. - Trusted provider-prefix -> quota-group mapping; explicitly mapped Agent Zero/OpenCode/native aliases share rate, active and queue limits. Original client identity and outbound presentation do not choose quota ownership. - Configuration validation and immutable scheduler policy snapshots, including mutable group/alias strings. - Deadline-first admission, expiry pruning, unwind-safe waiter cleanup and optional cooperative cancellation. - Scheduler-computed rounded-up Retry-After plus Cache-Control: no-store on local 429 responses, before opening upstream connections. - 12 socket-free rate regressions plus one retained HTTP rate regression. All existing tests remain registered. - Independent rate-contract CI job plus the unchanged full proxy test command; both check out the actual PR head and have bounded job timeouts. - Configuration/semantics/limitations documented in `research/LLM-LOG-rate-admission-75.org`. ## Executed validation Exact current head: `0011613cde0ceb0d101425a7606a3029f5105c2d`. **PASS: all 12 socket-free rate tests.** Actions run 34724938422, `rate-contracts` job 103637318354, completed successfully September 12 at 23:16:06 UTC. Checked the decoded execution log: production and full test-system compilation succeeded, the runner executed every one of the 12 named tests, and Rove reported `All 12 tests passed.` This includes the 64-thread test admitting exactly eight requests from an eight-permit burst, shared-alias enforcement, no quota refund, exact refill/deadline boundaries, mutable-policy isolation and cancellation/error cleanup. Actual execution used SBCL 2.6.7 through the pinned Nix package closure, not local structural checks. At the last inspection, the same run's complete `proxy-contracts` job 103637318468 was still in progress. The HTTP rate test has therefore NOT been certified passing. No full integration green is claimed. No local Lisp runtime execution occurred in the editing environment. Tests-first evidence: at `e81be9508efb7785df210af0948d3502fccf8682`, real SBCL/Nix run 34724484407 / job 103636097666 failed test compilation because REQUESTS-PER-MINUTE/BURST/PROVIDER-GROUPS APIs did not yet exist. This is an executed missing-API compile RED, not a behavioral assertion RED. Implementation commit: `04c323b0cd4b913368c83ff39fc605990c0d71c9`. Independent gate commit: `0011613cde0ceb0d101425a7606a3029f5105c2d`. ## Existing full-suite blocker Parent #74 at `ac26a689c56b1cf2b6c4999ad012b9bd2ced8497` already fails run 34715742893 / job 103612546205: first HTTP assertions pass, then a 15-minute hang and a Woo callback TYPE-ERROR (`NIL` is not `WOO.EV.SOCKET:SOCKET`). Detailed evidence is recorded on #74. The precise lifecycle defect still needs isolation. Keep this PR draft; the verified rate green does not authorize merging an unverified transport stack. ## Deliberate boundaries This is process-local refill-rate/burst admission, not a strict rolling-minute, TPM, five-hour or weekly provider quota. Defaults (60 RPM refill, burst 4) are local policy, not provider entitlement. Every group currently uses the same settings. All aliases for shared credentials must be explicitly mapped by trusted configuration. Restarts/multiple instances need a later durable/shared owner. No provider 429 cooldown ingestion, automatic retries, credential rotation or socket-disconnect cancellation integration is added. Inbound socket/thread counts are not globally bounded by this upstream scheduler. The user's Agent Zero -> OpenCode upstream presentation requirement is explicitly recorded in #71, but the wire-profile implementation is NOT part of this slice. It must preserve internal provenance and share the same quota group; a changed User-Agent alone is not proof of full harness equivalence or live provider acceptance. Smart-client SSE queue protocol remains #73. ## Commands Full existing gate: ```sh nix build -L .#checks.x86_64-linux.llm-log-config-contract ``` Verified independent rate gate: ```sh nix shell --no-write-lock-file .#llm-log-cl-sbcl --command \ sbcl --noinform --no-userinit --no-sysinit --non-interactive \ --load proxy/tests/rate-runner.lisp ```
Extend #74 with atomic active-slot + token-bucket admission, trusted provider alias groups, immutable policy snapshots, deadline-first cleanup and cooperative cancellation.

Tests-first e81be950 failed real SBCL/Nix compilation for missing rate API (run 34724484407). Preserve all tests; split socket-free rate contracts before the pre-existing Woo lifecycle blocker. No full green or live provider acceptance is claimed.
ci: isolate rate contracts without weakening the full proxy gate
Some checks failed
Common Lisp proxy contracts / rate-contracts (pull_request) Failing after 3s
Details
Common Lisp proxy contracts / proxy-contracts (pull_request) Failing after 4s
Details
0011613cde 
Execute the 12 registered socket-free rate contracts independently and fail the process on assertion failure or missing contracts. Keep the complete proxy suite unchanged. Checkout actual PR head and bound both job durations.
Some checks failed
Common Lisp proxy contracts / rate-contracts (pull_request) Failing after 3s
Details
Common Lisp proxy contracts / proxy-contracts (pull_request) Failing after 4s
Details
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin feat/75-rate-admission:feat/75-rate-admission
git switch feat/75-rate-admission

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch feat/70-request-scheduler
git merge --no-ff feat/75-rate-admission
git switch feat/75-rate-admission
git rebase feat/70-request-scheduler
git switch feat/70-request-scheduler
git merge --ff-only feat/75-rate-admission
git switch feat/75-rate-admission
git rebase feat/70-request-scheduler
git switch feat/70-request-scheduler
git merge --no-ff feat/75-rate-admission
git switch feat/70-request-scheduler
git merge --squash feat/75-rate-admission
git switch feat/70-request-scheduler
git merge --ff-only feat/75-rate-admission
git switch feat/70-request-scheduler
git merge feat/75-rate-admission
git push origin feat/70-request-scheduler
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
accessibility
Barrier affecting people with disabilities

  
bug
Something isn't working

  
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
invalid
This doesn't seem right

  
question
Further information is requested

  
wontfix
This will not be worked on

No labels
accessibility
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
nsaspy/llm-log!77
No description provided.