feat: unify task-first native model runtime

nsaspy commented

2026-09-04 22:14:07 +00:00

Owner

Summary

consolidate the #277 planner repair, direct-or-plan root decision, task-first framing, and 40k conversation retrieval work
add a provider-native direct loop with correlated tool calls, opaque result contexts, registered-tool/effect reuse, SPEC lifecycle operations, and typed-plan execution
preserve canonical query-aware prompt compilation in direct mode, with deterministic per-session reuse and an explicit all_tools cache profile
run routine pull-request CI once, skip paid provider tests while the PR is draft, and retain manual plus ready-for-review exact-head paid gates

Runtime invariants

Prolog continues to own capabilities, schema validation, authority, durable effects, budgets, cancellation, context lifetime, accounting, and verification. Provider call IDs are correlation only. Registered operations re-enter the canonical tool/effect runtime, and potentially large results stay behind per-call opaque contexts.

Cost, compiler, and cache policy

Direct mode defaults to prompt_compile_mode(compiled), matching root completion. The canonical prompt compiler uses the real query, capability set, and context budget to select provider-visible registered tools and skills once per direct session; that exact selected projection is then reused on continuation turns within the session. prompt_compile_mode(all_tools) remains an explicit compatibility/cache profile for workloads where a stable warm inventory is measurably cheaper. Hosts should optimize provider-reported cached tokens, total prompt tokens, and actual cost together rather than maximizing hit percentage alone.

The credentialed cache gate deliberately selects the explicit all_tools profile, constructs ten fresh sessions, and requires provider-reported cache hits on at least 80 percent of the nine warm requests. Draft updates skip this paid suite; it runs when the PR becomes ready or is manually dispatched.

Review fixes in the consolidated head

restore contextual compiler selection as the direct default and pass the real query into skill activation
prove same-input compiler determinism while allowing relevant tools and skills to change with the query
reject assistant continuations whose normalized native call batch differs from the response envelope
charge nested typed-plan steps against the enclosing direct iteration budget
replace unrestricted private completion predicate dispatch with an explicit support ABI
eliminate duplicate branch-push plus pull-request workflow runs

Verification

Complete local gate at 0c9c2d509435971ead2a0804dd9730f5229139d2:

runtime check: pass on SWI-Prolog 9.2.9
production module load and research-approval validation: pass
focused direct runtime: 22/22
focused compiler visibility: 4/4
deterministic PlUnit: 1052/1052 across 90 suites
deterministic benchmark: 16/16
deep experiment: 15/15
workflow YAML parse and diff whitespace check: pass

Exact-head gate at 8eddb58b30cbdcb35283f0bd4cb5c332400bd20e:

runtime, production-module load, and research-approval validation: pass
focused nested usage/accounting: 6/6
deterministic PlUnit: 1052/1052 across 90 suites
deterministic benchmark: 16/16
deep experiment: 15/15
clean pack install, Nix package/load, and Tree-sitter FFI: pass
diff whitespace check: pass

Exact-head gate at fac93b782508f47e9a6aa3b0b884549830497f36 (typed-plan native model steps):

runtime check, production-module load, and standalone rlm_completion/rlm_direct/rlm_plan loads: pass
focused plan runtime: 23/23 (native session contract, atomic overspend rejection, partial-failure usage preservation, shared token budget across sessions, standalone one-raw-call fallback)
focused direct runtime: 25/25 (compiler-selected step schemas, tool-plus-context continuation to final text, schema bundle stability across continuation turns, exact call-ID correlation, nested counts in the runtime trace, reserved-option rejection before provider dispatch, budget exhaustion blocking an extra provider call)
focused completion runtime: 53/53 (root completion native model step with usage accounting)
focused final handoff: 13/13 (task-step requests carry scoped direct-agent framing, not the planner protocol)
deterministic PlUnit: 1061/1061 across 90 suites (discovered = planned = completed = passed; zero failures/timeouts/blocked/fixme)
deterministic benchmark: 16/16
deep experiment: 15/15
JSON CLI demo and diff whitespace check: pass
paid provider: not called during development; paid checks stay skipped while the PR is draft

Paid evidence and current boundary

Earlier 40k managed-conversation evidence passed locally with two HTTP 200 generations, exact context-search to model to final dataflow, two model calls, 7422 provider-native tokens, and runtime cost 0.007575160 USD.

The automatic paid run on superseded head f7d93e6 is not a passing gate. Its direct native context and registered-tool cases passed with HTTP 200 responses, but the typed-plan root planner lane exhausted three attempts with missing_field(op) and the 40k managed-conversation lane also failed. The corrected head intentionally has no paid claim while this PR remains draft.

First-class typed-plan integration

Typed plans remain a first-class, fully supported execution strategy with typed context, tool, model, recursive, parallel, retry, checkpoint, and final operations. The model operation now runs through the same provider-native session executor as direct mode whenever the runtime supplies the canonical rlm_direct_model_step/10 handler — root completion and direct-mode typed_plan_execute always do — while standalone plan_run/5 without a handler keeps its one-raw-call compatibility profile and llm_query/3 stays exactly one raw call. The plan reserves one step and one model call, then charges actual continuation iterations, model calls, tool calls, context operations, and observation bytes atomically against the shared plan budget; the final response and every provider response are recorded so plan_usage/2 and execution errors account for all spent calls, tokens, and cost. Model-supplied messages, tools, tool_choice, and streaming controls are rejected before provider dispatch; runtime-selected schemas remain authoritative. The native path is canonical, not legacy.

Non-goals

No product-specific Auto-Dig or coding-agent runtime, ambient shell/filesystem/network authority, new scheduler, authority engine, effect journal, verifier, persistence layer, or provider-specific core cache database.

Refs #277
Refs #279

## Summary - consolidate the #277 planner repair, direct-or-plan root decision, task-first framing, and 40k conversation retrieval work - add a provider-native direct loop with correlated tool calls, opaque result contexts, registered-tool/effect reuse, SPEC lifecycle operations, and typed-plan execution - preserve canonical query-aware prompt compilation in direct mode, with deterministic per-session reuse and an explicit `all_tools` cache profile - run routine pull-request CI once, skip paid provider tests while the PR is draft, and retain manual plus ready-for-review exact-head paid gates ## Runtime invariants Prolog continues to own capabilities, schema validation, authority, durable effects, budgets, cancellation, context lifetime, accounting, and verification. Provider call IDs are correlation only. Registered operations re-enter the canonical tool/effect runtime, and potentially large results stay behind per-call opaque contexts. ## Cost, compiler, and cache policy Direct mode defaults to `prompt_compile_mode(compiled)`, matching root completion. The canonical prompt compiler uses the real query, capability set, and context budget to select provider-visible registered tools and skills once per direct session; that exact selected projection is then reused on continuation turns within the session. `prompt_compile_mode(all_tools)` remains an explicit compatibility/cache profile for workloads where a stable warm inventory is measurably cheaper. Hosts should optimize provider-reported cached tokens, total prompt tokens, and actual cost together rather than maximizing hit percentage alone. The credentialed cache gate deliberately selects the explicit `all_tools` profile, constructs ten fresh sessions, and requires provider-reported cache hits on at least 80 percent of the nine warm requests. Draft updates skip this paid suite; it runs when the PR becomes ready or is manually dispatched. ## Review fixes in the consolidated head - restore contextual compiler selection as the direct default and pass the real query into skill activation - prove same-input compiler determinism while allowing relevant tools and skills to change with the query - reject assistant continuations whose normalized native call batch differs from the response envelope - charge nested typed-plan steps against the enclosing direct iteration budget - replace unrestricted private completion predicate dispatch with an explicit support ABI - eliminate duplicate branch-push plus pull-request workflow runs ## Verification Complete local gate at `0c9c2d509435971ead2a0804dd9730f5229139d2`: - runtime check: pass on SWI-Prolog 9.2.9 - production module load and research-approval validation: pass - focused direct runtime: 22/22 - focused compiler visibility: 4/4 - deterministic PlUnit: 1052/1052 across 90 suites - deterministic benchmark: 16/16 - deep experiment: 15/15 - workflow YAML parse and diff whitespace check: pass Exact-head gate at `8eddb58b30cbdcb35283f0bd4cb5c332400bd20e`: - runtime, production-module load, and research-approval validation: pass - focused nested usage/accounting: 6/6 - deterministic PlUnit: 1052/1052 across 90 suites - deterministic benchmark: 16/16 - deep experiment: 15/15 - clean pack install, Nix package/load, and Tree-sitter FFI: pass - diff whitespace check: pass Exact-head gate at `fac93b782508f47e9a6aa3b0b884549830497f36` (typed-plan native model steps): - runtime check, production-module load, and standalone `rlm_completion`/`rlm_direct`/`rlm_plan` loads: pass - focused plan runtime: 23/23 (native session contract, atomic overspend rejection, partial-failure usage preservation, shared token budget across sessions, standalone one-raw-call fallback) - focused direct runtime: 25/25 (compiler-selected step schemas, tool-plus-context continuation to final text, schema bundle stability across continuation turns, exact call-ID correlation, nested counts in the runtime trace, reserved-option rejection before provider dispatch, budget exhaustion blocking an extra provider call) - focused completion runtime: 53/53 (root completion native model step with usage accounting) - focused final handoff: 13/13 (task-step requests carry scoped direct-agent framing, not the planner protocol) - deterministic PlUnit: 1061/1061 across 90 suites (discovered = planned = completed = passed; zero failures/timeouts/blocked/fixme) - deterministic benchmark: 16/16 - deep experiment: 15/15 - JSON CLI demo and diff whitespace check: pass - paid provider: not called during development; paid checks stay skipped while the PR is draft ## Paid evidence and current boundary Earlier 40k managed-conversation evidence passed locally with two HTTP 200 generations, exact context-search to model to final dataflow, two model calls, 7422 provider-native tokens, and runtime cost 0.007575160 USD. The automatic paid run on superseded head `f7d93e6` is not a passing gate. Its direct native context and registered-tool cases passed with HTTP 200 responses, but the typed-plan root planner lane exhausted three attempts with `missing_field(op)` and the 40k managed-conversation lane also failed. The corrected head intentionally has no paid claim while this PR remains draft. ## First-class typed-plan integration Typed plans remain a first-class, fully supported execution strategy with typed context, tool, model, recursive, parallel, retry, checkpoint, and final operations. The `model` operation now runs through the same provider-native session executor as direct mode whenever the runtime supplies the canonical `rlm_direct_model_step/10` handler — root completion and direct-mode `typed_plan_execute` always do — while standalone `plan_run/5` without a handler keeps its one-raw-call compatibility profile and `llm_query/3` stays exactly one raw call. The plan reserves one step and one model call, then charges actual continuation iterations, model calls, tool calls, context operations, and observation bytes atomically against the shared plan budget; the final response and every provider response are recorded so `plan_usage/2` and execution errors account for all spent calls, tokens, and cost. Model-supplied `messages`, `tools`, `tool_choice`, and streaming controls are rejected before provider dispatch; runtime-selected schemas remain authoritative. The native path is canonical, not legacy. ## Non-goals No product-specific Auto-Dig or coding-agent runtime, ambient shell/filesystem/network authority, new scheduler, authority engine, effect journal, verifier, persistence layer, or provider-specific core cache database. Refs #277 Refs #279

nsaspy added 16 commits

2026-09-04 22:14:07 +00:00

fix: add corrective planner context 10da62c5bb

test: add live planner context acceptance 56bbf50b65

fix: close direct-or-plan root decision in rlm_completion e86b3f1b40

The root model may now answer directly through the strict
{"mode":"direct","answer":"<nonempty final text>"} envelope and
finish in exactly one model call with plan:none, empty transitions and
bindings, zero recursion, and truthful usage/trajectory data. A valid
typed plan still wins, so context, tool, model, and recursive plan
semantics are unchanged.

Fail-closed boundary: envelopes with unapproved fields, non-direct
modes, or empty/nontext answers fail explicitly as
invalid_root_decision; prose, malformed plans, and native provider tool
calls are never accepted as either root-decision form. The root prompt,
the mandatory rlm-operate skill, and planner repair diagnostics state
both forms and prefer direct completion when runtime operations add no
value; repair diagnostics remain bounded and never echo rejected
provider output.

The deep-experiment report now records the actual root decision and
claims plan_parsed/plan_validated only when a plan actually executed.
The live planner-context acceptance test uses neutral repository
fixtures (README and completion/tools runtime docs) instead of
SPEC/RAGE workflow records.

Fixes the paid harness_guided_depth_2 failure where the forced-planner
protocol left no textual final model output, and removes the second
model call ordinary questions previously paid just to echo the goal.

fix: make registry tool-envelope field hops repairable 8f530a3071

Live CI at c813df0 showed the real model executing context retrieval and
all project_read reads but composing the final as a one-hop field
reference over the tool binding (missing the tool_result envelope value
hop) and failing unrecoverably at execution.

Root-plan validation now rejects, as a repairable structural fault, any
first-hop field key outside the closed tool_result envelope over a
registry-tool binding, with a bounded tool_result_envelope_field(Key,Bind)
diagnostic routed through the existing planner retry loop. Direct
trusted host tools(tools([...])) are exempt because their result shape is
host-defined. The rlm-operate skill documents the envelope result shape
with a concrete final-step example.

The live planner-context acceptance now states its audit semantics
(retrieval must execute in-run), asserts on bound values/transitions
without over-constraining model bind naming, and raises planner
reasoning effort to medium for the composition task. Assertions are
unchanged in strength; no gate was weakened.

fix: fit direct root answers in planner token ceiling fc676a6194

The deep-integration harness truncated direct root answers at
planner_max_tokens(2400) because the direct envelope carries the full
final answer, not a small plan echo. Raise the ceiling to 3600 and
planner attempts 2->3, and state in the root prompt that any
non-envelope output shape (prose, markdown, provider-native tool
call) is rejected without execution.

feat: attribute OpenRouter requests to the runtime cb02fa8732

openrouter_provider/2 now carries app_title('prolog-rlm') and
app_referer('https://github.com/lost-rob0t/prolog-rlm'); the
OpenAI-compatible transport sends them as X-OpenRouter-Title and
HTTP-Referer. Attribution is descriptive identity only: absent keys
send no headers, invalid values fail closed as configuration_error
before network IO, and downstream provider terms override both.
Deterministic coverage uses a local HTTP server capturing the real
request headers (5 tests). Adds scripts/openrouter_completion.sh for
per-generation debugging and a repo-local debug-io skill. Live
verification: generation gen-1787858510-MAIPUiC1ViAP9FPPwWXf shows
app_id 4845152 with origin github.com/lost-rob0t/prolog-rlm.

docs: restructure rlm-operate skill as a plan-tool catalog f50de13b6c

The rlm-operate body was dense interleaved prose. Render the plan
protocol as an explicit tool catalog: one section for the two root
decisions, one 'Plan tools' section where final/model/context/tool
each get a one-line purpose plus their exact JSON shape, then
references and the prefer-direct rule. All JSON protocol shapes,
test markers, and fail-closed rules are byte-identical.

docs: frame rlm-recurse as the rlm plan tool, tighten facts/constraints 0347271e9b

Present the recursive step as a plan tool with its exact JSON shape
and capability gate; clarify facts (query exposed evidence first,
derive deterministically, judge only residual uncertainty) and
constraints (prefer an exposed solver, never invoke an ungranted
one). Markers and protocol shapes unchanged.

feat: give executed model steps a scoped system message ee0f5135fd

Model steps inside typed plans previously sent a bare user message,
so the step model received zero role or contract context while the
root planner held all of it. Execute-step model requests now carry a
static scoped system message: the user message is the complete step
task, answer directly with the step result, follow task-requested
formats, never emit plan JSON or provider-native tool calls. It
deliberately does not inherit the planner protocol or skill catalog;
the raw llm_query facade keeps its pinned single-user-message shape.
Regression uses the local HTTP handoff server to assert what the
transport actually received.

feat: give root planner calls a scoped agent identity system message b4c9d6e749

Direct rlm_completion roots now always open with an identity system
message: 'You are the root agent inside <app>.' The app name comes
from the trusted agent_name option (default prolog-rlm) so downstream
products format their own identity; the message states the library
boundary and that only core context inputs and granted tools are
loaded. agent_scope(root|delegated) is trusted runtime data:
rlm_subagent structurally marks every delegated child agent_scope
delegated, dropping any caller-provided scope, so children never
carry root identity. Both options validate fail-closed before any
planner call. Raw llm_query keeps its pinned single-user shape; the
delegated child and skill-opt-out shapes are pinned in tests.

perf: bound large conversation context packing 2a3c1413df

test: add live 40k conversation retrieval fee0a36c09

feat: add provider-native direct runtime f7d93e6f78

fix: preserve contextual compilation in direct mode 0c9c2d5094

docs: treat typed plans as first-class 8eddb58b30

feat: run typed-plan model steps through native provider sessions

Tree-sitter FFI / Direct SWI-Prolog Tree-sitter FFI (pull_request) Successful in 2m47s

Details

CI / REAL OpenRouter integration (pull_request) Has been skipped

Details

Nix flake / Flake package and clean runtime load (pull_request) Failing after 16s

Details

Clean SWI pack install / Install and load copied pack (pull_request) Successful in 1m46s

Details

Paid OpenRouter / Pinned paid OpenRouter integration (pull_request) Failing after 1m56s

Details

CI / Deterministic unit and load checks (pull_request) Successful in 3m56s

Details

fac93b7825

Typed-plan model operations are first-class. The plan runtime now accepts a
trusted model_step_handler runtime option, and both root completion and
direct-mode typed_plan_execute supply a module-qualified
rlm_direct_model_step/10 closure bound to the already-acquired context
handle, exact provider, capabilities, cancellation token, and remaining
budget (outer spent usage netted out).

The child model session is the existing direct loop entered through
rlm_direct_execute/4 from inside the bounded worker: canonical query-aware
prompt/compiler selection on the step prompt, provider-native tools and
tool_choice, exact assistant/tool call-ID correlation across role:tool
continuations, and the selected schema bundle reused unchanged per session.
The plan reserves one step and one model call, then charges the reported
native execution atomically: iterations-1 steps, model_calls-1 model calls,
all tool/context calls, and observation bytes. The final response and every
provider response are recorded into plan results and errors so usage
accounting stays complete on partial failure. Model-supplied messages,
tools, tool_choice, and streaming controls are rejected before provider
dispatch; runtime-selected schemas remain authoritative.

Standalone plan_run/5 without a handler keeps the one-raw-call compatibility
path, and llm_query/3 stays one raw call. The direct loop now retains every
provider response in direct_result, direct_error, and internal state.

Non-goals: no second scheduler or native loop, no handler/callable data
through model-controlled channels, no widening of child capabilities, and no
paid-provider calls in development.

nsaspy closed this pull request

2026-09-04 22:55:41 +00:00

nsaspy referenced this pull request

2026-09-08 01:29:20 +00:00

[semantic-ir] General semantic knowledge algebra v1: domain-neutral kernel, typing, modality, provenance, and extensions #406

nsaspy referenced this pull request

2026-09-08 01:29:20 +00:00

[EPIC][library/tool] General semantic knowledge compiler: arbitrary natural language → extensible validated symbolic knowledge #410

nsaspy referenced this pull request

2026-09-08 01:29:20 +00:00

[EPIC][experts] Closed-loop integration: Requirement→Knowledge→Plan→Experts→VERIFY→Repair→Review→completion #411

nsaspy referenced this pull request