nsaspy/llm-log
1
0
Fork 0
Code Issues 34 Pull requests 19 Projects Releases Packages Wiki Activity Actions

Rate-budget and outbound-profile foundation (stacked on #74) #76

Open
nsaspy wants to merge 2 commits from feat/71-rate-profile-foundation into feat/70-request-scheduler
pull from: feat/71-rate-profile-foundation
merge into: nsaspy:feat/70-request-scheduler
Conversation 0 Commits 2 Files changed 10 +741 -190
nsaspy commented 2026-09-12 23:21:00 +00:00
Owner
Copy link

Scope

Next bounded slice for #70 / #71, stacked on #74. Rate limiting remains the highest priority. Hermes, OpenClaw and Agent Zero can select the same operator-configured outbound application-header profile without replacing their prompts/tools or getting separate capacity.

Implementation

  • Add an explicit request-start token bucket (requests_per_minute, burst) in addition to the active-request limit and bounded FIFO.
  • Atomically consume start budget only on admission; completing/failing a request releases concurrency but does not refund its start token.
  • Canonicalize upstream origins so route aliases share the same budget, independent of profile, source harness, path and credentials.
  • Return a positive integer Retry-After based on refill time and configured fallback.
  • Remove abandoned queued waiters on non-local exit; preserve FIFO/deadline-before-admission checks.
  • Add typed/versioned [profiles.NAME] and [provider_profiles] operator configuration, integrated into the real relay before admission/upstream connection.
  • Support explicit User-Agent/X-Title normalization and bounded metadata-header removal. Reject CR/LF, unknown keys, dangling profile references, and credential/framing header removal.
  • Preserve original input headers, authentication, message bodies, prompts and tool schemas. Unmapped providers remain transparent.
  • Document an operator example and limitations in research/rate-profile-foundation.org.

RED-first evidence

Test-only head 72901bf58dc8e7caf292a988ee1b1c2e84c3bbf5 ran BEFORE runtime changes. Actions run 34724841525, job 103637062682: 2 passed, 12 failed, 14 total, with failures on missing rate configuration/profile support. The checkout log confirms the exact head, not a synthetic merge.

Implementation head: 1c0869912bb26ddb46f22982701867939076a3aa. Its policy check is being evaluated separately; no green assertion is made until the result is available.

Verification boundary / remaining work

The existing full Common Lisp proxy check remains enabled and is NOT green on the parent: run 34715742893 stalls after the first HTTP test and crashes in a Woo socket callback. The isolated policy job does not replace that full transport gate. The new boundary test substitutes only the final network operation; it is not live HTTP/SSE/provider conformance.

This is process-local request-start rate enforcement, not distributed/persistent account quota, TPM accounting, a five-hour credit budget, or upstream-429 cooldown. Rate zero preserves backward compatibility, so a nonzero operator rate must be configured. Different origins sharing one account will need explicit quota-group mapping.

Profiles are header-only normalization, not full anonymization/indistinguishability. OpenCode identity values in tests are explicitly SYNTHETIC, not a verified capture of a released version. Real capture/export/diff, provenance persistence, and provider conformance remain in #71. No permission/entitlement change is implied.

No merge or deployment while the full transport/lifecycle gate remains blocked. This PR does not close #71.

## Scope Next bounded slice for #70 / #71, stacked on #74. Rate limiting remains the highest priority. Hermes, OpenClaw and Agent Zero can select the same operator-configured outbound application-header profile without replacing their prompts/tools or getting separate capacity. ## Implementation - Add an explicit request-start token bucket (`requests_per_minute`, `burst`) in addition to the active-request limit and bounded FIFO. - Atomically consume start budget only on admission; completing/failing a request releases concurrency but does not refund its start token. - Canonicalize upstream origins so route aliases share the same budget, independent of profile, source harness, path and credentials. - Return a positive integer `Retry-After` based on refill time and configured fallback. - Remove abandoned queued waiters on non-local exit; preserve FIFO/deadline-before-admission checks. - Add typed/versioned `[profiles.NAME]` and `[provider_profiles]` operator configuration, integrated into the real relay before admission/upstream connection. - Support explicit User-Agent/X-Title normalization and bounded metadata-header removal. Reject CR/LF, unknown keys, dangling profile references, and credential/framing header removal. - Preserve original input headers, authentication, message bodies, prompts and tool schemas. Unmapped providers remain transparent. - Document an operator example and limitations in `research/rate-profile-foundation.org`. ## RED-first evidence Test-only head `72901bf58dc8e7caf292a988ee1b1c2e84c3bbf5` ran BEFORE runtime changes. Actions run `34724841525`, job `103637062682`: **2 passed, 12 failed, 14 total**, with failures on missing rate configuration/profile support. The checkout log confirms the exact head, not a synthetic merge. Implementation head: `1c0869912bb26ddb46f22982701867939076a3aa`. Its policy check is being evaluated separately; no green assertion is made until the result is available. ## Verification boundary / remaining work The existing full Common Lisp proxy check remains enabled and is NOT green on the parent: run `34715742893` stalls after the first HTTP test and crashes in a Woo socket callback. The isolated policy job does not replace that full transport gate. The new boundary test substitutes only the final network operation; it is not live HTTP/SSE/provider conformance. This is process-local request-start rate enforcement, not distributed/persistent account quota, TPM accounting, a five-hour credit budget, or upstream-429 cooldown. Rate zero preserves backward compatibility, so a nonzero operator rate must be configured. Different origins sharing one account will need explicit quota-group mapping. Profiles are header-only normalization, not full anonymization/indistinguishability. OpenCode identity values in tests are explicitly SYNTHETIC, not a verified capture of a released version. Real capture/export/diff, provenance persistence, and provider conformance remain in #71. No permission/entitlement change is implied. No merge or deployment while the full transport/lifecycle gate remains blocked. This PR does not close #71.
test: define rate-budget and outbound-profile contracts before implementation
Some checks failed
Rate and profile policy contracts / policy-contracts (pull_request) Failing after 3s
Details
Common Lisp proxy contracts / proxy-contracts (pull_request) Failing after 4s
Details
72901bf58d
feat: enforce shared request-rate budgets and versioned outbound header profiles
Some checks failed
Common Lisp proxy contracts / proxy-contracts (pull_request) Failing after 5s
Details
Rate and profile policy contracts / policy-contracts (pull_request) Failing after 5s
Details
1c0869912b 
RED evidence: exact test-only head 72901bf, Actions run 34724841525,
job 103637062682: 2 passed, 12 failed before runtime changes.

Keep full Woo transport gate separate and unmodified; no merge claim.
Some checks failed
Common Lisp proxy contracts / proxy-contracts (pull_request) Failing after 5s
Details
Rate and profile policy contracts / policy-contracts (pull_request) Failing after 5s
Details
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin feat/71-rate-profile-foundation:feat/71-rate-profile-foundation
git switch feat/71-rate-profile-foundation

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch feat/70-request-scheduler
git merge --no-ff feat/71-rate-profile-foundation
git switch feat/71-rate-profile-foundation
git rebase feat/70-request-scheduler
git switch feat/70-request-scheduler
git merge --ff-only feat/71-rate-profile-foundation
git switch feat/71-rate-profile-foundation
git rebase feat/70-request-scheduler
git switch feat/70-request-scheduler
git merge --no-ff feat/71-rate-profile-foundation
git switch feat/70-request-scheduler
git merge --squash feat/71-rate-profile-foundation
git switch feat/70-request-scheduler
git merge --ff-only feat/71-rate-profile-foundation
git switch feat/70-request-scheduler
git merge feat/71-rate-profile-foundation
git push origin feat/70-request-scheduler
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
accessibility
Barrier affecting people with disabilities

  
bug
Something isn't working

  
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
invalid
This doesn't seem right

  
question
Further information is requested

  
wontfix
This will not be worked on

No labels
accessibility
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
nsaspy/llm-log!76
No description provided.