[direct][integration] expose model context limits and trusted imported-tool projection #325

New issue

Open

opened 2026-09-02 17:25:38 +00:00 by nsaspy · 0 comments

nsaspy commented

2026-09-02 17:25:38 +00:00

Owner

Why

Auto-Dig is moving the research worker off the typed planner and onto native direct mode after plan_parse_failed in consumer run starintel-gpt-auto-dig #33124557266 / Prolog-RLM #285.

The consumer can be made to work now, but it currently has to own two pieces of runtime policy that belong in Prolog-RLM:

Model context-window metadata. The Auto-Dig worker needs a runtime budget equal to 30% of the selected model's context limit. The currently routed GPT-5.6 Luna/Terra/Sol models expose a 1,050,000-token context window, so the consumer workaround is a 315,000-token total budget. Prolog-RLM has no provider-neutral API for resolving the selected model's context window / output limit, so the consumer must temporarily pin those values.
Trusted imported-tool projection. Direct mode can already expose capability-filtered native tools, but a host that imports read-only MCP tools still has to hand-maintain the capability list. Auto-Dig wants to give direct mode the full trusted read-only research inventory without bypassing effect classification, authority, confinement, or the prompt compiler.

Requested Prolog-RLM additions

Model limits

Add a provider-neutral model metadata contract, e.g.:

provider_model_limits(+Provider, +Model, -Outcome).

with a closed result containing at least:

model_limits{
    context_window: Tokens,
    max_output_tokens: Tokens
}

The OpenRouter implementation may obtain/cache provider model metadata, but callers should not parse OpenRouter JSON themselves.

Also add a small helper for fractional host budgets so consumers can express policy like 30% of model context without duplicating arithmetic.

Trusted MCP capability projection

Add a helper that derives callable capability terms from the already imported, host-classified tool set, filtered by trusted effect/policy, e.g. read-only only. It must not turn raw server inventory into authority. The desired flow is:

MCP import
-> trusted effect classification
-> trusted capability projection
-> normal direct-mode prompt compiler selection
-> normal tool_invoke_execute authority/confinement path

The model must not be able to widen this set.

Acceptance

Unit tests for known and unknown model metadata.
Fractional budget helper rejects invalid fractions and never exceeds the model limit.
OpenRouter model limits are cached/bounded and failures stay structured.
Imported read-only MCP tools can be projected without manually duplicating every remote tool name.
Write/unknown-effect tools are excluded unless host policy explicitly allows them.
Direct-mode schema projection remains deterministic and capability-filtered.

Consumer workaround

starintel-gpt-auto-dig will temporarily:

pin a current Prolog-RLM commit with native direct mode;
use rlm_direct/4 instead of root typed-planner completion for the research loop;
set the worker token budget to 315,000 (30% of 1,050,000) for the currently routed GPT-5.6 models;
expose the trusted read-only Brave + Fetch research tools through direct mode;
keep the allow-list until the trusted imported-tool projection API exists.

Once the APIs above land, remove the duplicated model-limit/tool-projection policy from the consumer.

## Why Auto-Dig is moving the research worker off the typed planner and onto native direct mode after `plan_parse_failed` in consumer run `starintel-gpt-auto-dig` #33124557266 / Prolog-RLM #285. The consumer can be made to work now, but it currently has to own two pieces of runtime policy that belong in Prolog-RLM: 1. **Model context-window metadata.** The Auto-Dig worker needs a runtime budget equal to 30% of the selected model's context limit. The currently routed GPT-5.6 Luna/Terra/Sol models expose a 1,050,000-token context window, so the consumer workaround is a 315,000-token total budget. Prolog-RLM has no provider-neutral API for resolving the selected model's context window / output limit, so the consumer must temporarily pin those values. 2. **Trusted imported-tool projection.** Direct mode can already expose capability-filtered native tools, but a host that imports read-only MCP tools still has to hand-maintain the capability list. Auto-Dig wants to give direct mode the full trusted read-only research inventory without bypassing effect classification, authority, confinement, or the prompt compiler. ## Requested Prolog-RLM additions ### Model limits Add a provider-neutral model metadata contract, e.g.: ```prolog provider_model_limits(+Provider, +Model, -Outcome). ``` with a closed result containing at least: ```prolog model_limits{ context_window: Tokens, max_output_tokens: Tokens } ``` The OpenRouter implementation may obtain/cache provider model metadata, but callers should not parse OpenRouter JSON themselves. Also add a small helper for fractional host budgets so consumers can express policy like `30% of model context` without duplicating arithmetic. ### Trusted MCP capability projection Add a helper that derives callable capability terms from the **already imported, host-classified** tool set, filtered by trusted effect/policy, e.g. read-only only. It must not turn raw server inventory into authority. The desired flow is: ```text MCP import -> trusted effect classification -> trusted capability projection -> normal direct-mode prompt compiler selection -> normal tool_invoke_execute authority/confinement path ``` The model must not be able to widen this set. ## Acceptance - Unit tests for known and unknown model metadata. - Fractional budget helper rejects invalid fractions and never exceeds the model limit. - OpenRouter model limits are cached/bounded and failures stay structured. - Imported read-only MCP tools can be projected without manually duplicating every remote tool name. - Write/unknown-effect tools are excluded unless host policy explicitly allows them. - Direct-mode schema projection remains deterministic and capability-filtered. ## Consumer workaround `starintel-gpt-auto-dig` will temporarily: - pin a current Prolog-RLM commit with native direct mode; - use `rlm_direct/4` instead of root typed-planner completion for the research loop; - set the worker token budget to 315,000 (30% of 1,050,000) for the currently routed GPT-5.6 models; - expose the trusted read-only Brave + Fetch research tools through direct mode; - keep the allow-list until the trusted imported-tool projection API exists. Once the APIs above land, remove the duplicated model-limit/tool-projection policy from the consumer.

nsaspy referenced this issue from a commit

2026-09-04 20:53:53 +00:00

test: reproduce raw native argument batch abort (#325)

nsaspy referenced this issue from a commit

2026-09-04 20:53:53 +00:00

fix: isolate malformed native argument parsing per call (#325)

nsaspy referenced this issue from a commit

2026-09-04 20:53:53 +00:00

fix: keep partial normalization entries wire-safe (#325)

nsaspy referenced this issue from a commit

2026-09-04 20:53:53 +00:00

docs: state wire-text retention boundary for classify (#325)