Responses forced tool_choice failure from upstream (NextBit) crashes the turn: no recovery path in TransportPolicy #122

New issue

Closed

opened 2026-09-08 11:45:26 +00:00 by mara · 0 comments

mara commented

2026-09-08 11:45:26 +00:00

Collaborator

Summary

Responses endpoints whose upstream cannot honor a forced tool call fail mid-stream with response.failed and kill the whole agent turn. The transport's TransportPolicy.recover() has no recovery path for this error class (it is neither a reasoning-effort, provider-state, builtin-tool, nor Responses-unsupported error), so unified_turn raises and the loop crashes with:

RuntimeError: Upstream error from NextBit: upstream model did not return a valid tool call for the requested tool_choice

Observed with openrouter/z-ai/glm-5.3-flash (NextBit is the OpenRouter upstream provider).

Environment

Repo: nsaspy/a0-symbolics@main bba5dcba
prolog-rlm pin (a0-symbolics flake): not applicable
Command: any agent turn with tool_choice (required) routed through a provider/upstream that does not reliably emit native tool calls under forcing

Repro

Configure a model routed through an upstream that reports did not return a valid tool call for the requested tool_choice (e.g. openrouter/z-ai/glm-5.3-flash).
Run an agent turn that carries tools (Responses API default forces one required native call via tool_choice: "required").
Upstream emits a response.failed event; ResponsesEventParser.parse raises RuntimeError from helpers/litellm_transport.py:1218.
TransportPolicy.recover() matches nothing and returns RAISE; the turn dies.

Expected / Actual

Expected: the transport degrades gracefully (retry once without the forced tool_choice) and the turn completes with model-chosen tools or text.
Actual: unhandled RuntimeError crashes the monologue loop.

Evidence

Pre-fix detector check against the exact error text:

reasoning: False
not_supported: False
state_unsupported: False
builtin: False
transient(models): False

No matching classifier → TransportRecovery.RAISE.

Fix (branch `fix/responses-upstream-tool-choice`, commit `af938ed6`)

New TransportRecovery.RETRY_DROP_TOOL_CHOICE: on first occurrence, retry the request once with tool_choice removed.
New classifier _is_tool_choice_not_honored_error() matching tool_choice + did not return a valid tool call.
a0_drop_tool_choice internal kwarg honored by the Responses request builder; surfaced as tool_choice_dropped capability metadata; stripped from outbound requests.
Regression test test_transport_retries_without_tool_choice_when_upstream_fails_forced_call in tests/test_responses_architecture.py.
tests/test_responses_architecture.py (13 passed), test_stream_tool_early_stop.py + test_model_call_extensions.py (50 passed).

## Summary Responses endpoints whose upstream cannot honor a forced tool call fail mid-stream with `response.failed` and kill the whole agent turn. The transport's `TransportPolicy.recover()` has no recovery path for this error class (it is neither a reasoning-effort, provider-state, builtin-tool, nor Responses-unsupported error), so `unified_turn` raises and the loop crashes with: ``` RuntimeError: Upstream error from NextBit: upstream model did not return a valid tool call for the requested tool_choice ``` Observed with `openrouter/z-ai/glm-5.3-flash` (NextBit is the OpenRouter upstream provider). ## Environment - Repo: nsaspy/a0-symbolics@main bba5dcba - prolog-rlm pin (a0-symbolics flake): not applicable - Command: any agent turn with `tool_choice` (required) routed through a provider/upstream that does not reliably emit native tool calls under forcing ## Repro 1. Configure a model routed through an upstream that reports `did not return a valid tool call for the requested tool_choice` (e.g. `openrouter/z-ai/glm-5.3-flash`). 2. Run an agent turn that carries tools (Responses API default forces one required native call via `tool_choice: "required"`). 3. Upstream emits a `response.failed` event; `ResponsesEventParser.parse` raises `RuntimeError` from `helpers/litellm_transport.py:1218`. 4. `TransportPolicy.recover()` matches nothing and returns RAISE; the turn dies. ## Expected / Actual Expected: the transport degrades gracefully (retry once without the forced `tool_choice`) and the turn completes with model-chosen tools or text. Actual: unhandled `RuntimeError` crashes the monologue loop. ## Evidence Pre-fix detector check against the exact error text: ``` reasoning: False not_supported: False state_unsupported: False builtin: False transient(models): False ``` No matching classifier → `TransportRecovery.RAISE`. ## Fix (branch `fix/responses-upstream-tool-choice`, commit af938ed6) - New `TransportRecovery.RETRY_DROP_TOOL_CHOICE`: on first occurrence, retry the request once with `tool_choice` removed. - New classifier `_is_tool_choice_not_honored_error()` matching `tool_choice` + `did not return a valid tool call`. - `a0_drop_tool_choice` internal kwarg honored by the Responses request builder; surfaced as `tool_choice_dropped` capability metadata; stripped from outbound requests. - Regression test `test_transport_retries_without_tool_choice_when_upstream_fails_forced_call` in `tests/test_responses_architecture.py`. - `tests/test_responses_architecture.py` (13 passed), `test_stream_tool_early_stop.py` + `test_model_call_extensions.py` (50 passed).

mara added the

bug

label

2026-09-08 11:45:26 +00:00

mara referenced this issue from a pull request that will close it,

2026-09-08 11:45:57 +00:00

fix(transport): retry without tool_choice when upstream fails forced tool call #123