nsaspy/llm-log
1
0
Fork 0
Code Issues 34 Pull requests 19 Projects Releases Packages Wiki Activity Actions

Enforce configurable OpenRouter quantization presets #61

Open
nsaspy wants to merge 11 commits from rage/openrouter-quantization-presets into rage/4-unified-config
pull from: rage/openrouter-quantization-presets
merge into: nsaspy:rage/4-unified-config
Conversation 0 Commits 11 Files changed 8 +602 -3
nsaspy commented 2026-09-07 03:17:09 +00:00
Owner
Copy link

Stacked on #5 (rage/4-unified-config).

What this adds

  • fail-closed OpenRouter quantization routing
  • default high-precision preset: fp32, fp16, bf16, fp8
  • built-in balanced and all-known presets
  • unknown omitted from every built-in preset
  • TOML, CLI, and Home Manager configuration for named quantization allowlists
  • request-level enforcement via OpenRouter provider.quantizations
  • caller-supplied quantizations are intersected with the active preset, so callers may narrow policy but cannot widen it
  • empty intersections fail locally with HTTP 400 instead of silently routing to a lower/unknown precision endpoint
  • OpenRouter preset-capture inference skins are covered as well, so presets captured through llm-log record the enforced quantization allowlist
  • research decision recorded in research/LLM-LOG-RESEARCH-003-openrouter-quantization-policy.org

Defaults

high-precision is active by default:

fp32 fp16 bf16 fp8

Endpoints reported by OpenRouter as unknown are not eligible. fp4 and int4 are also excluded by the default preset.

Explicit opt-in remains possible by defining a custom preset containing unknown.

OpenRouter contract

OpenRouter documents provider.quantizations as a provider endpoint filter and exposes unknown as a distinct quantization value. The implementation relies on that reported metadata rather than inferring precision from provider reputation or hardware.

Validation

  • added pure policy tests for injection, intersection, preservation of existing provider routing fields, local rejection, and explicit unknown opt-in
  • added configuration tests for safe defaults, TOML presets, CLI selection, CLI-defined presets, invalid values, and missing presets
  • branch audit shows only expected config/policy/tests/docs/Home Manager files changed
  • pure policy helper contract was exercised in this environment
  • full local python -m unittest discover -s tests -v and nix flake check still need to run in a normal repo checkout; this repository intentionally does not use GitHub Actions as its development loop
Stacked on #5 (`rage/4-unified-config`). ## What this adds - fail-closed OpenRouter quantization routing - default `high-precision` preset: `fp32`, `fp16`, `bf16`, `fp8` - built-in `balanced` and `all-known` presets - `unknown` omitted from every built-in preset - TOML, CLI, and Home Manager configuration for named quantization allowlists - request-level enforcement via OpenRouter `provider.quantizations` - caller-supplied quantizations are intersected with the active preset, so callers may narrow policy but cannot widen it - empty intersections fail locally with HTTP 400 instead of silently routing to a lower/unknown precision endpoint - OpenRouter preset-capture inference skins are covered as well, so presets captured through llm-log record the enforced quantization allowlist - research decision recorded in `research/LLM-LOG-RESEARCH-003-openrouter-quantization-policy.org` ## Defaults `high-precision` is active by default: ```text fp32 fp16 bf16 fp8 ``` Endpoints reported by OpenRouter as `unknown` are not eligible. `fp4` and `int4` are also excluded by the default preset. Explicit opt-in remains possible by defining a custom preset containing `unknown`. ## OpenRouter contract OpenRouter documents `provider.quantizations` as a provider endpoint filter and exposes `unknown` as a distinct quantization value. The implementation relies on that reported metadata rather than inferring precision from provider reputation or hardware. ## Validation - added pure policy tests for injection, intersection, preservation of existing provider routing fields, local rejection, and explicit `unknown` opt-in - added configuration tests for safe defaults, TOML presets, CLI selection, CLI-defined presets, invalid values, and missing presets - branch audit shows only expected config/policy/tests/docs/Home Manager files changed - pure policy helper contract was exercised in this environment - full local `python -m unittest discover -s tests -v` and `nix flake check` still need to run in a normal repo checkout; this repository intentionally does not use GitHub Actions as its development loop
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin rage/openrouter-quantization-presets:rage/openrouter-quantization-presets
git switch rage/openrouter-quantization-presets

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch rage/4-unified-config
git merge --no-ff rage/openrouter-quantization-presets
git switch rage/openrouter-quantization-presets
git rebase rage/4-unified-config
git switch rage/4-unified-config
git merge --ff-only rage/openrouter-quantization-presets
git switch rage/openrouter-quantization-presets
git rebase rage/4-unified-config
git switch rage/4-unified-config
git merge --no-ff rage/openrouter-quantization-presets
git switch rage/4-unified-config
git merge --squash rage/openrouter-quantization-presets
git switch rage/4-unified-config
git merge --ff-only rage/openrouter-quantization-presets
git switch rage/4-unified-config
git merge rage/openrouter-quantization-presets
git push origin rage/4-unified-config
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
accessibility
Barrier affecting people with disabilities

  
bug
Something isn't working

  
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
invalid
This doesn't seem right

  
question
Further information is requested

  
wontfix
This will not be worked on

No labels
accessibility
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
nsaspy/llm-log!61
No description provided.