[direct] add an explicit native-call batch cardinality budget (max_native_calls_per_batch) #305

New issue

Closed

opened 2026-09-02 17:25:36 +00:00 by nsaspy · 2 comments

nsaspy commented

2026-09-02 17:25:36 +00:00

Owner

Context

Direct mode classifies the complete native tool-call batch returned by one provider response (classify_calls/4 in prolog/rlm_direct.pl). Recoverable per-call preflight faults (issue #313, hardened after #316) never execute and are never charged against max_tool_calls / max_context_ops, so one provider response can request an arbitrarily large all-fault batch:

native_tool_calls_normalize/2 (prolog/rlm_native_tool.pl) accepts any list length (require_call_list/1 only checks list-ness).
No classification, budget, or admission step caps the number of calls in one batch.
The only implicit bounds are provider max_tokens (output size of the single response) and max_output_bytes (bounded fault observations per call) plus max_model_calls/max_iterations across turns.

Problem

A hostile or degenerate model response can force the runtime to normalize and classify a very large call list in one turn. Each faulted call still costs normalization work, one bounded observation, and observation bytes. There is no explicit, testable runtime knob bounding per-response native-call cardinality; the bound today is an emergent property of provider output limits.

Requested behavior

Add an explicit budget field equivalent to max_native_calls_per_batch (trusted, operator-configured, with a bounded default) enforced during classification against the ORIGINAL requested batch:

enforce before per-call classification work (fail closed, deterministic kind such as native_batch_too_large);
count every call in the requested batch, including calls that would fail preflight;
document it in docs/direct-runtime.md alongside the other budget admission rules;
add regression tests for oversize batches in both all-valid and all-fault shapes.

Non-goals

Do not silently repurpose max_tool_calls / max_context_ops to cover rejected calls: charged counts must keep meaning executed operations. This is a separate admission limit, not a charge change.

Related: #313, #316.

## Context Direct mode classifies the complete native tool-call batch returned by one provider response (`classify_calls/4` in `prolog/rlm_direct.pl`). Recoverable per-call preflight faults (issue #313, hardened after #316) never execute and are never charged against `max_tool_calls` / `max_context_ops`, so one provider response can request an arbitrarily large all-fault batch: - `native_tool_calls_normalize/2` (`prolog/rlm_native_tool.pl`) accepts any list length (`require_call_list/1` only checks list-ness). - No classification, budget, or admission step caps the number of calls in one batch. - The only implicit bounds are provider `max_tokens` (output size of the single response) and `max_output_bytes` (bounded fault observations per call) plus `max_model_calls`/`max_iterations` across turns. ## Problem A hostile or degenerate model response can force the runtime to normalize and classify a very large call list in one turn. Each faulted call still costs normalization work, one bounded observation, and observation bytes. There is no explicit, testable runtime knob bounding per-response native-call cardinality; the bound today is an emergent property of provider output limits. ## Requested behavior Add an explicit budget field equivalent to `max_native_calls_per_batch` (trusted, operator-configured, with a bounded default) enforced during classification against the ORIGINAL requested batch: - enforce before per-call classification work (fail closed, deterministic kind such as `native_batch_too_large`); - count every call in the requested batch, including calls that would fail preflight; - document it in `docs/direct-runtime.md` alongside the other budget admission rules; - add regression tests for oversize batches in both all-valid and all-fault shapes. ## Non-goals Do not silently repurpose `max_tool_calls` / `max_context_ops` to cover rejected calls: charged counts must keep meaning executed operations. This is a separate admission limit, not a charge change. Related: #313, #316.

nsaspy closed this issue

2026-09-02 17:25:36 +00:00

nsaspy referenced this issue

2026-09-02 17:26:32 +00:00

[CI failure] CI on fix/304-context-peek-selector-contract #317

nsaspy referenced this issue

2026-09-02 17:26:32 +00:00

[CI failure] Paid OpenRouter on fix/304-context-peek-selector-contract #316

nsaspy commented

2026-09-02 17:26:35 +00:00

Author

Owner

Fix in #332 (max_native_calls_per_batch admission: original-batch cardinality, fail-closed native_batch_too_large, executed-operation accounting unchanged).

nsaspy commented

2026-09-02 17:26:35 +00:00

Author

Owner

Completed by #332 (merged as 3a4c2d1): max_native_calls_per_batch admission — original-batch cardinality, fail-closed native_batch_too_large, executed-operation accounting unchanged, docs/direct-runtime.md updated.

Completed by #332 (merged as 3a4c2d1): max_native_calls_per_batch admission — original-batch cardinality, fail-closed native_batch_too_large, executed-operation accounting unchanged, docs/direct-runtime.md updated.