nsaspy/prolog-rlm

Fork 0

[EPIC] Wire prompt compiler tool projection into the real rlm_completion planner surface #450

New issue

Closed

opened 2026-09-10 21:18:23 +00:00 by nsaspy · 1 comment

nsaspy commented

2026-09-10 21:18:23 +00:00

Owner

Goal

Make rlm_prompt_compiler actually control the tool schemas the root LLM sees in the production rlm_completion path, while keeping the full trusted runtime bindings available for execution and authority checks.

This closes the current gap between:

prompt compiler says: selected model-visible tools

and:

rlm_completion currently says: tool_discover(Registry, AllSchemas)
                               -> planner_prompt(..., AllSchemas, ...)

The implementation must preserve the architectural rule:

model visibility != runtime registration != execution authorization

No new tool registry or model-facing interface should be created.

Research: current main behavior

1. `prolog/rlm_prompt_compiler.pl` is implemented

The compiler already provides:

prompt_catalog_create/1
prompt_catalog_register/3
prompt_catalog_register_tool_registry/4
prompt_compile/4
prompt_recompile/4
prompt_render/3
prompt_compiler_tool_schemas/2

It produces a structured compiled_context{...} containing selected/rejected units, context units, tool schemas, token ledger, active units, reasons and a fingerprint.

It performs bounded evidence-driven selection, dependency closure, capability/discovery filtering, conflicts/supersession, and shared context packing.

research/RLM-RESEARCH-010-symbolic-prompt-compiler.org and research/RLM-RESEARCH-011-managed-context-tool-discovery.org define this as the provider-visible projection boundary.

2. Current `rlm_completion` bypasses it

In prolog/rlm_completion.pl, completion_with_handle/... currently does:

runtime_tools(Options,
              Capabilities,
              RuntimeTools,
              ToolSchemas),
planner_prompt(Query,
               MetadataRef.metadata,
               Capabilities,
               ChildCapabilities,
               ToolSchemas,
               Options,
               Prompt),

and runtime_tools/4 currently resolves registry schemas with:

tool_registry_runtime_tools(Registry, Capabilities, RegistryTools),
tool_discover(Registry, Schemas)

Therefore every schema returned by the runtime registry is still handed directly to the root planner when a registry is supplied.

That means prompt-compiler contextual activation/deactivation does not yet govern the actual production planner tool view.

3. `planner_prompt/7` serializes those schemas directly

The current planner prompt contains:

Registered tool schemas: ~q

using the raw ToolSchemas returned by tool_discover/2.

So the actual LLM-facing tool projection today is still owned by rlm_completion, not by rlm_prompt_compiler.

4. Compiler tests are currently isolated

Repository search of prompt_compile/4 shows compiler implementation/research/tests and other schema work, but no production rlm_completion call site on current main.

The compiler test context_deactivation_does_not_unregister_runtime_tool correctly proves that a tool may disappear from compiled context without runtime unregistering, but the production completion path does not yet consume that compiled tool list.

5. Existing research already called for this integration

RLM-RESEARCH-011-managed-context-tool-discovery.org explicitly states the intended direction:

rlm_completion should stop giving every discovered registry schema to the planner by default. The runtime bindings used by plan_run remain the full trusted set, while the model receives only selected schemas. all_tools preserves the old projection for regression comparison.

This epic implements that already-researched contract rather than inventing a new architecture.

Design contract

Split the current ambiguous runtime_tools responsibility into two explicit concepts:

trusted executable bindings
        !=
provider-visible compiled projection

Conceptually:

Tool Registry
    |
    +--> full trusted RuntimeTools -----------------> plan execution
    |
    +--> declarative catalog metadata
             |
             v
      rlm_prompt_compiler
             |
             v
      VisibleToolSchemas + CompiledText
             |
             v
          root LLM

Selection does not unregister tools. Selection does not grant authority. A tool hidden from the planner may remain registered and executable by trusted host code; a tool visible to the planner still must pass normal capability/schema/preflight/authority/effect checks when invoked.

Completion API / options

Do not add a second completion API.

Use existing rlm_completion(..., Options, ...) configuration.

Add/normalize compiler-related host options conceptually as:

prompt_catalog(Catalog)
prompt_compile_options(CompilerOptions)
prompt_compile_mode(compiled|all_tools)

Exact option naming may follow current conventions, but ownership must remain host-side.

Default mode

Target default after migration:

prompt_compile_mode(compiled)

all_tools remains an explicit compatibility/debug/benchmark mode as already specified by RLM-RESEARCH-010/011.

If backwards-compatibility risk requires a staged rollout, implementation may temporarily require an explicit compiler option for one release/slice, but the issue must document that staging and include a test proving the intended eventual compiled default. Do not leave two indefinite ambiguous defaults.

Catalog ownership

Preferred behavior:

if host supplies prompt_catalog(Catalog), use it directly;
otherwise, when tool_registry(Registry) exists, build a bounded ephemeral catalog projection from that registry for the completion call using prompt_catalog_register_tool_registry/4;
destroy only an ephemeral compiler catalog created by the completion call;
never destroy a host-owned supplied catalog.

A long-lived host catalog remains preferable when Skills/instructions/MCP/resources need to coexist with tool metadata across turns.

Do not store executable handlers inside the prompt catalog.

Exact `rlm_completion` flow

Refactor completion_with_handle/... so the order becomes conceptually:

resolve provider
resolve capabilities
resolve full runtime tool bindings
resolve/build prompt catalog
compile provider-visible projection from current Query + trusted signals
render compiled projection
build planner prompt from:
    - user Query
    - context metadata
    - capabilities
    - child capabilities
    - compiler-rendered text
    - compiler-selected active tool schemas
call planner
execute validated plan against FULL trusted RuntimeTools

The important invariant is:

Planner sees compiled schemas.
Executor retains full trusted capable runtime bindings.

Do not replace RuntimeTools with only selected tools unless a separate authority design explicitly requires that later.

Exact implementation files

1. `prolog/rlm_completion.pl`

A. Imports

Import/use rlm_prompt_compiler through the module-qualified style consistent with current architecture.

B. Split `runtime_tools/4`

Current predicate:

runtime_tools(Options, Capabilities, Tools, Schemas)

currently conflates executable tools and model-visible schemas.

Refactor into explicit responsibilities, conceptually:

runtime_tool_bindings(+Options,
                      +Capabilities,
                      -RuntimeTools,
                      -Registry).

provider_tool_projection(+Query,
                         +Registry,
                         +Capabilities,
                         +Options,
                         -Projection).

or equivalent names following current conventions.

runtime_tool_bindings owns the full trusted execution list.

provider_tool_projection owns catalog creation/import/compile/render and returns at least:

provider_projection{
    text:CompiledText,
    tool_schemas:VisibleToolSchemas,
    active_units:ActiveUnits,
    fingerprint:Fingerprint,
    compiled:Compiled
}

Do not pass raw handlers/callables into the compiler projection.

C. Compile input

The minimum compile input is the current normalized user Query.

Preferred shape:

prompt_input{
    text:Query,
    signals:TrustedSignals,
    needs:Needs,
    selected:HostSelected,
    denied:HostDenied
}

For the first integration slice, TrustedSignals, Needs, HostSelected, and HostDenied may default empty unless already available in Options.

Expose host options for these only if they can be cleanly normalized through the existing compiler API. Do not parse arbitrary model output into trusted selected/denied fields.

D. Compiler mode

For compiled mode call prompt_compile/4 normally.

For all_tools, use the compiler's existing compatibility mode instead of bypassing the compiler and directly calling tool_discover/2 for planner exposure. This keeps one projection path and makes comparison deterministic.

E. Planner prompt

Change planner_prompt/... so it no longer labels raw registry results as:

Registered tool schemas

Use language that reflects actual model visibility, e.g.:

Active tool schemas

Add compiler-rendered text in one explicit section, e.g.:

Compiled runtime guidance:
<CompiledText>

Do not serialize the same full tool schema twice. If prompt_render/3 already includes full schema text in CompiledText, decide one canonical representation for the planner path:

provider/tool schema channel if supported by this planner abstraction; or
the explicit Active tool schemas field in the typed-planner prompt.

The token ledger must charge the actual representation used. Do not count one representation and render another.

F. Completion result observability

Add non-secret compiler metadata to the completion result/trajectory if a stable extension point exists:

prompt_projection:_{
    fingerprint:Fingerprint,
    active_units:ActiveUnits,
    tool_count:Count
}

Do not expose rejected hidden tool schemas or host-only catalog internals by default.

This makes it possible to prove which projection the planner actually received.

G. Cleanup

If completion creates an ephemeral prompt catalog, wrap it in setup_call_cleanup/3 and destroy it on success, failure, timeout, cancellation, or planner error.

Host-owned catalogs supplied through Options must never be destroyed by completion cleanup.

2. `prolog/rlm_prompt_compiler.pl`

A. Provider tool schemas must correspond to active packed units

Audit selected_tool_schemas/2, active_units_from_pack/2, and prompt_render/3 together.

Current tool_schemas are derived from pre-pack SelectedEntries, while active_units come from the context pack.

For mandatory imported tools these normally coincide, but a custom tool with mandatory_context:false could theoretically be selected then omitted by packing while remaining in Compiled.tool_schemas.

The provider-visible contract must be exact:

if a tool schema is sent to the model, its tool unit is active in the final packed projection

Implement one canonical helper, conceptually:

active_tool_schemas(+SelectedEntries,
                    +ActiveUnits,
                    -Schemas).

and ensure prompt_render/3 returns only schemas corresponding to active tool/mcp_tool units.

If pack(false) is used for inspection, preserve a clearly documented pre-pack state rather than pretending it is a provider-ready render.

B. #174 integration

Issue #174 adds declarative activation:always|relevant to the same prompt_unit interface.

This completion epic must consume whatever final compiler contract #174 lands. It must not add its own special always-visible list.

If #176 lands before #174, tests should use current relevance behavior and add the always-visible integration regression when #174 lands.

3. `test/rlm_completion_test.pl` or canonical completion test file

Add production-path tests proving the root planner sees compiled schemas rather than raw registry discovery.

If current completion tests use support planners that capture planner prompt text, extend those fixtures instead of building a separate test harness.

4. `test/rlm_prompt_compiler_test.pl`

Add/adjust active-schema packing regressions if tool_schemas derivation changes.

5. `docs/completion-runtime.md`

Replace the current statement that the root planner receives registered tool schemas with the precise contract:

The runtime keeps registered/authorized execution bindings separately. The root planner receives the current compiled provider-visible tool projection.

Document compiled vs all_tools mode and catalog ownership.

6. New stable runtime doc

Create:

docs/prompt-compiler.md

This was already identified as missing in #174.

Required sections:

Purpose
Unit algebra
Registration vs availability vs activation vs authorization
Compilation inputs
Relevant vs always activation (#174)
Dependency/conflict/supersession closure
Provider-visible packing
Tool schema projection
Completion integration
Catalog ownership/lifecycle
all_tools compatibility mode
Explainability and fingerprints
Security/authority boundary

The doc must explicitly show:

Registry handlers ---------> executor
Registry schemas -> catalog -> compiler -> LLM

7. Research/design source of truth

Update:

research/RLM-RESEARCH-011-managed-context-tool-discovery.org

Add an Implementation closure / completion integration section recording the exact landed path.

Do not create another competing architecture research note for this integration; RLM-RESEARCH-011 already researched and required it.

Required regression matrix

completion_planner_hides_irrelevant_registered_tool
- registry contains git_diff and project_search;
- query only matches git review;
- capture planner prompt/projection;
- assert git schema visible, project_search absent;
- assert both remain registered in runtime.
completion_executor_retains_hidden_runtime_binding
- after compilation hides an irrelevant tool from planner visibility, inspect trusted runtime-tool set/test support and prove compiler did not unregister/delete binding.
completion_all_tools_mode_preserves_compatibility_projection
- same registry/query;
- all_tools mode;
- assert both schemas visible.
completion_capability_denied_tool_not_visible
- registered tool without allowed capability;
- assert absent/rejected from model projection.
completion_compiler_projection_fingerprint_is_observable
- identical material input => identical fingerprint;
- changed query/tool activation => changed fingerprint.
completion_prompt_uses_active_not_registered_wording
- regression against accidentally restoring raw registry semantics.
completion_does_not_duplicate_full_schema_rendering
- inspect captured planner payload and token representation;
- each schema has one canonical provider-visible representation.
completion_ephemeral_catalog_is_cleaned_on_success
completion_ephemeral_catalog_is_cleaned_on_planner_failure
completion_ephemeral_catalog_is_cleaned_on_timeout_or_cancel
completion_host_owned_catalog_is_not_destroyed
completion_compiled_projection_can_include_instruction_or_skill_context

register one non-tool provider-visible unit matching query;
assert compiler-rendered guidance reaches planner.

completion_pack_budget_applies_to_tool_projection

use constrained policy;
prove mandatory active tool cannot silently disappear; structural budget failure when impossible.

provider_tool_schemas_are_subset_of_active_units

compiler-level invariant test.

optional_selected_but_packed_out_tool_schema_is_not_sent

if optional tool representations are supported;
selected before packing but omitted after packing;
assert absent from provider tool schemas.

always_visible_tool_reaches_completion_planner_on_unrelated_query

dependent on #174;
configure ordinary tool unit activation:always;
unrelated query;
assert tool appears in actual captured planner surface.

relevant_tool_still_deactivates_between_completion_turns

Turn A relevant -> visible;
Turn B unrelated -> hidden;
Turn C relevant -> visible again;
never re-register runtime handler.

hidden_tool_cannot_be_model_selected_by_test_planner_without_schema_visibility

test planner should only choose from supplied active schemas; ensure fixtures do not accidentally inject global registry knowledge.

Backwards compatibility

tool registry registration/execution APIs remain unchanged;
plan execution continues using trusted runtime bindings;
selection never grants authority;
host callers may use all_tools for old projection behavior;
compiler catalog may be host-owned or ephemeral;
direct tools(...) host bindings remain supported, but their model-visible schema story must be documented explicitly instead of silently bypassing compilation;
no arbitrary callable enters compiler IR.

Non-goals

no new tool registry;
no new planner/interpreter;
no physical tool unload on contextual deactivation;
no authorization based solely on compiler selection;
no model-supplied handler names;
no embedding-only router as authority;
no duplicated context-budget optimizer;
no hard-coded core-tool visibility list (use #174's ordinary prompt_unit activation field);
no provider-specific compiler fork.

Dependencies / ordering

Base compiler implementation is already on main.
#174 defines declarative persistent visibility and should be consumed by this path when available.
This epic is the production integration gate that makes compiler visibility decisions observable to the real root planner.

Acceptance gate

Complete only when:

production rlm_completion no longer gives raw tool_discover/2 output directly to the planner in normal compiled mode;
full trusted execution bindings remain separate and intact;
planner receives only final active compiler-selected schemas;
contextual deactivation is proven through the real completion path;
#174 always-visible behavior is proven through the real completion path when available;
token packing and emitted schema representation cannot drift;
catalog lifecycle is leak-free;
projection fingerprint/active-unit metadata is inspectable without exposing secrets/handlers;
all_tools remains an explicit compatibility mode through the same compiler path;
docs/prompt-compiler.md, docs/completion-runtime.md, and RLM-RESEARCH-011 reflect the landed contract;
focused compiler/completion tests and deterministic aggregate tests pass.

Implementation agents must re-read live main before editing, but they must preserve this separation of runtime execution bindings from provider-visible compiled tool schemas.

## Goal Make `rlm_prompt_compiler` actually control the **tool schemas the root LLM sees** in the production `rlm_completion` path, while keeping the full trusted runtime bindings available for execution and authority checks. This closes the current gap between: ```text prompt compiler says: selected model-visible tools ``` and: ```text rlm_completion currently says: tool_discover(Registry, AllSchemas) -> planner_prompt(..., AllSchemas, ...) ``` The implementation must preserve the architectural rule: ```text model visibility != runtime registration != execution authorization ``` No new tool registry or model-facing interface should be created. --- # Research: current main behavior ## 1. `prolog/rlm_prompt_compiler.pl` is implemented The compiler already provides: ```prolog prompt_catalog_create/1 prompt_catalog_register/3 prompt_catalog_register_tool_registry/4 prompt_compile/4 prompt_recompile/4 prompt_render/3 prompt_compiler_tool_schemas/2 ``` It produces a structured `compiled_context{...}` containing selected/rejected units, context units, tool schemas, token ledger, active units, reasons and a fingerprint. It performs bounded evidence-driven selection, dependency closure, capability/discovery filtering, conflicts/supersession, and shared context packing. `research/RLM-RESEARCH-010-symbolic-prompt-compiler.org` and `research/RLM-RESEARCH-011-managed-context-tool-discovery.org` define this as the provider-visible projection boundary. ## 2. Current `rlm_completion` bypasses it In `prolog/rlm_completion.pl`, `completion_with_handle/...` currently does: ```prolog runtime_tools(Options, Capabilities, RuntimeTools, ToolSchemas), planner_prompt(Query, MetadataRef.metadata, Capabilities, ChildCapabilities, ToolSchemas, Options, Prompt), ``` and `runtime_tools/4` currently resolves registry schemas with: ```prolog tool_registry_runtime_tools(Registry, Capabilities, RegistryTools), tool_discover(Registry, Schemas) ``` Therefore every schema returned by the runtime registry is still handed directly to the root planner when a registry is supplied. That means prompt-compiler contextual activation/deactivation does **not yet govern the actual production planner tool view**. ## 3. `planner_prompt/7` serializes those schemas directly The current planner prompt contains: ```text Registered tool schemas: ~q ``` using the raw `ToolSchemas` returned by `tool_discover/2`. So the actual LLM-facing tool projection today is still owned by `rlm_completion`, not by `rlm_prompt_compiler`. ## 4. Compiler tests are currently isolated Repository search of `prompt_compile/4` shows compiler implementation/research/tests and other schema work, but no production `rlm_completion` call site on current main. The compiler test `context_deactivation_does_not_unregister_runtime_tool` correctly proves that a tool may disappear from compiled context without runtime unregistering, but the production completion path does not yet consume that compiled tool list. ## 5. Existing research already called for this integration `RLM-RESEARCH-011-managed-context-tool-discovery.org` explicitly states the intended direction: ```text rlm_completion should stop giving every discovered registry schema to the planner by default. The runtime bindings used by plan_run remain the full trusted set, while the model receives only selected schemas. all_tools preserves the old projection for regression comparison. ``` This epic implements that already-researched contract rather than inventing a new architecture. --- # Design contract Split the current ambiguous `runtime_tools` responsibility into two explicit concepts: ```text trusted executable bindings != provider-visible compiled projection ``` Conceptually: ```text Tool Registry | +--> full trusted RuntimeTools -----------------> plan execution | +--> declarative catalog metadata | v rlm_prompt_compiler | v VisibleToolSchemas + CompiledText | v root LLM ``` Selection does not unregister tools. Selection does not grant authority. A tool hidden from the planner may remain registered and executable by trusted host code; a tool visible to the planner still must pass normal capability/schema/preflight/authority/effect checks when invoked. --- # Completion API / options Do not add a second completion API. Use existing `rlm_completion(..., Options, ...)` configuration. Add/normalize compiler-related host options conceptually as: ```prolog prompt_catalog(Catalog) prompt_compile_options(CompilerOptions) prompt_compile_mode(compiled|all_tools) ``` Exact option naming may follow current conventions, but ownership must remain host-side. ## Default mode Target default after migration: ```prolog prompt_compile_mode(compiled) ``` `all_tools` remains an explicit compatibility/debug/benchmark mode as already specified by RLM-RESEARCH-010/011. If backwards-compatibility risk requires a staged rollout, implementation may temporarily require an explicit compiler option for one release/slice, but the issue must document that staging and include a test proving the intended eventual `compiled` default. Do not leave two indefinite ambiguous defaults. ## Catalog ownership Preferred behavior: 1. if host supplies `prompt_catalog(Catalog)`, use it directly; 2. otherwise, when `tool_registry(Registry)` exists, build a bounded ephemeral catalog projection from that registry for the completion call using `prompt_catalog_register_tool_registry/4`; 3. destroy only an ephemeral compiler catalog created by the completion call; 4. never destroy a host-owned supplied catalog. A long-lived host catalog remains preferable when Skills/instructions/MCP/resources need to coexist with tool metadata across turns. Do not store executable handlers inside the prompt catalog. --- # Exact `rlm_completion` flow Refactor `completion_with_handle/...` so the order becomes conceptually: ```text resolve provider resolve capabilities resolve full runtime tool bindings resolve/build prompt catalog compile provider-visible projection from current Query + trusted signals render compiled projection build planner prompt from: - user Query - context metadata - capabilities - child capabilities - compiler-rendered text - compiler-selected active tool schemas call planner execute validated plan against FULL trusted RuntimeTools ``` The important invariant is: ```text Planner sees compiled schemas. Executor retains full trusted capable runtime bindings. ``` Do not replace `RuntimeTools` with only selected tools unless a separate authority design explicitly requires that later. --- # Exact implementation files ## 1. `prolog/rlm_completion.pl` ### A. Imports Import/use `rlm_prompt_compiler` through the module-qualified style consistent with current architecture. ### B. Split `runtime_tools/4` Current predicate: ```prolog runtime_tools(Options, Capabilities, Tools, Schemas) ``` currently conflates executable tools and model-visible schemas. Refactor into explicit responsibilities, conceptually: ```prolog runtime_tool_bindings(+Options, +Capabilities, -RuntimeTools, -Registry). provider_tool_projection(+Query, +Registry, +Capabilities, +Options, -Projection). ``` or equivalent names following current conventions. `runtime_tool_bindings` owns the full trusted execution list. `provider_tool_projection` owns catalog creation/import/compile/render and returns at least: ```prolog provider_projection{ text:CompiledText, tool_schemas:VisibleToolSchemas, active_units:ActiveUnits, fingerprint:Fingerprint, compiled:Compiled } ``` Do not pass raw handlers/callables into the compiler projection. ### C. Compile input The minimum compile input is the current normalized user `Query`. Preferred shape: ```prolog prompt_input{ text:Query, signals:TrustedSignals, needs:Needs, selected:HostSelected, denied:HostDenied } ``` For the first integration slice, `TrustedSignals`, `Needs`, `HostSelected`, and `HostDenied` may default empty unless already available in Options. Expose host options for these only if they can be cleanly normalized through the existing compiler API. Do not parse arbitrary model output into trusted `selected`/`denied` fields. ### D. Compiler mode For `compiled` mode call `prompt_compile/4` normally. For `all_tools`, use the compiler's existing compatibility mode instead of bypassing the compiler and directly calling `tool_discover/2` for planner exposure. This keeps one projection path and makes comparison deterministic. ### E. Planner prompt Change `planner_prompt/...` so it no longer labels raw registry results as: ```text Registered tool schemas ``` Use language that reflects actual model visibility, e.g.: ```text Active tool schemas ``` Add compiler-rendered text in one explicit section, e.g.: ```text Compiled runtime guidance: <CompiledText> ``` Do not serialize the same full tool schema twice. If `prompt_render/3` already includes full schema text in `CompiledText`, decide one canonical representation for the planner path: - provider/tool schema channel if supported by this planner abstraction; or - the explicit `Active tool schemas` field in the typed-planner prompt. The token ledger must charge the actual representation used. Do not count one representation and render another. ### F. Completion result observability Add non-secret compiler metadata to the completion result/trajectory if a stable extension point exists: ```prolog prompt_projection:_{ fingerprint:Fingerprint, active_units:ActiveUnits, tool_count:Count } ``` Do not expose rejected hidden tool schemas or host-only catalog internals by default. This makes it possible to prove which projection the planner actually received. ### G. Cleanup If completion creates an ephemeral prompt catalog, wrap it in `setup_call_cleanup/3` and destroy it on success, failure, timeout, cancellation, or planner error. Host-owned catalogs supplied through Options must never be destroyed by completion cleanup. ## 2. `prolog/rlm_prompt_compiler.pl` ### A. Provider tool schemas must correspond to active packed units Audit `selected_tool_schemas/2`, `active_units_from_pack/2`, and `prompt_render/3` together. Current `tool_schemas` are derived from pre-pack `SelectedEntries`, while `active_units` come from the context pack. For mandatory imported tools these normally coincide, but a custom tool with `mandatory_context:false` could theoretically be selected then omitted by packing while remaining in `Compiled.tool_schemas`. The provider-visible contract must be exact: ```text if a tool schema is sent to the model, its tool unit is active in the final packed projection ``` Implement one canonical helper, conceptually: ```prolog active_tool_schemas(+SelectedEntries, +ActiveUnits, -Schemas). ``` and ensure `prompt_render/3` returns only schemas corresponding to active tool/mcp_tool units. If `pack(false)` is used for inspection, preserve a clearly documented pre-pack state rather than pretending it is a provider-ready render. ### B. #174 integration Issue #174 adds declarative `activation:always|relevant` to the same `prompt_unit` interface. This completion epic must consume whatever final compiler contract #174 lands. It must not add its own special always-visible list. If #176 lands before #174, tests should use current relevance behavior and add the always-visible integration regression when #174 lands. ## 3. `test/rlm_completion_test.pl` or canonical completion test file Add production-path tests proving the root planner sees compiled schemas rather than raw registry discovery. If current completion tests use support planners that capture planner prompt text, extend those fixtures instead of building a separate test harness. ## 4. `test/rlm_prompt_compiler_test.pl` Add/adjust active-schema packing regressions if `tool_schemas` derivation changes. ## 5. `docs/completion-runtime.md` Replace the current statement that the root planner receives `registered tool schemas` with the precise contract: ```text The runtime keeps registered/authorized execution bindings separately. The root planner receives the current compiled provider-visible tool projection. ``` Document `compiled` vs `all_tools` mode and catalog ownership. ## 6. New stable runtime doc Create: ```text docs/prompt-compiler.md ``` This was already identified as missing in #174. Required sections: ```text Purpose Unit algebra Registration vs availability vs activation vs authorization Compilation inputs Relevant vs always activation (#174) Dependency/conflict/supersession closure Provider-visible packing Tool schema projection Completion integration Catalog ownership/lifecycle all_tools compatibility mode Explainability and fingerprints Security/authority boundary ``` The doc must explicitly show: ```text Registry handlers ---------> executor Registry schemas -> catalog -> compiler -> LLM ``` ## 7. Research/design source of truth Update: ```text research/RLM-RESEARCH-011-managed-context-tool-discovery.org ``` Add an `Implementation closure / completion integration` section recording the exact landed path. Do not create another competing architecture research note for this integration; RLM-RESEARCH-011 already researched and required it. --- # Required regression matrix 1. `completion_planner_hides_irrelevant_registered_tool` - registry contains `git_diff` and `project_search`; - query only matches git review; - capture planner prompt/projection; - assert git schema visible, project_search absent; - assert both remain registered in runtime. 2. `completion_executor_retains_hidden_runtime_binding` - after compilation hides an irrelevant tool from planner visibility, inspect trusted runtime-tool set/test support and prove compiler did not unregister/delete binding. 3. `completion_all_tools_mode_preserves_compatibility_projection` - same registry/query; - `all_tools` mode; - assert both schemas visible. 4. `completion_capability_denied_tool_not_visible` - registered tool without allowed capability; - assert absent/rejected from model projection. 5. `completion_compiler_projection_fingerprint_is_observable` - identical material input => identical fingerprint; - changed query/tool activation => changed fingerprint. 6. `completion_prompt_uses_active_not_registered_wording` - regression against accidentally restoring raw registry semantics. 7. `completion_does_not_duplicate_full_schema_rendering` - inspect captured planner payload and token representation; - each schema has one canonical provider-visible representation. 8. `completion_ephemeral_catalog_is_cleaned_on_success` 9. `completion_ephemeral_catalog_is_cleaned_on_planner_failure` 10. `completion_ephemeral_catalog_is_cleaned_on_timeout_or_cancel` 11. `completion_host_owned_catalog_is_not_destroyed` 12. `completion_compiled_projection_can_include_instruction_or_skill_context` - register one non-tool provider-visible unit matching query; - assert compiler-rendered guidance reaches planner. 13. `completion_pack_budget_applies_to_tool_projection` - use constrained policy; - prove mandatory active tool cannot silently disappear; structural budget failure when impossible. 14. `provider_tool_schemas_are_subset_of_active_units` - compiler-level invariant test. 15. `optional_selected_but_packed_out_tool_schema_is_not_sent` - if optional tool representations are supported; - selected before packing but omitted after packing; - assert absent from provider tool schemas. 16. `always_visible_tool_reaches_completion_planner_on_unrelated_query` - dependent on #174; - configure ordinary tool unit `activation:always`; - unrelated query; - assert tool appears in actual captured planner surface. 17. `relevant_tool_still_deactivates_between_completion_turns` - Turn A relevant -> visible; - Turn B unrelated -> hidden; - Turn C relevant -> visible again; - never re-register runtime handler. 18. `hidden_tool_cannot_be_model_selected_by_test_planner_without_schema_visibility` - test planner should only choose from supplied active schemas; ensure fixtures do not accidentally inject global registry knowledge. --- # Backwards compatibility - tool registry registration/execution APIs remain unchanged; - plan execution continues using trusted runtime bindings; - selection never grants authority; - host callers may use `all_tools` for old projection behavior; - compiler catalog may be host-owned or ephemeral; - direct `tools(...)` host bindings remain supported, but their model-visible schema story must be documented explicitly instead of silently bypassing compilation; - no arbitrary callable enters compiler IR. --- # Non-goals - no new tool registry; - no new planner/interpreter; - no physical tool unload on contextual deactivation; - no authorization based solely on compiler selection; - no model-supplied handler names; - no embedding-only router as authority; - no duplicated context-budget optimizer; - no hard-coded core-tool visibility list (use #174's ordinary `prompt_unit` activation field); - no provider-specific compiler fork. --- # Dependencies / ordering - Base compiler implementation is already on main. - #174 defines declarative persistent visibility and should be consumed by this path when available. - This epic is the production integration gate that makes compiler visibility decisions observable to the real root planner. # Acceptance gate Complete only when: - production `rlm_completion` no longer gives raw `tool_discover/2` output directly to the planner in normal compiled mode; - full trusted execution bindings remain separate and intact; - planner receives only final active compiler-selected schemas; - contextual deactivation is proven through the real completion path; - #174 always-visible behavior is proven through the real completion path when available; - token packing and emitted schema representation cannot drift; - catalog lifecycle is leak-free; - projection fingerprint/active-unit metadata is inspectable without exposing secrets/handlers; - `all_tools` remains an explicit compatibility mode through the same compiler path; - `docs/prompt-compiler.md`, `docs/completion-runtime.md`, and RLM-RESEARCH-011 reflect the landed contract; - focused compiler/completion tests and deterministic aggregate tests pass. Implementation agents must re-read live main before editing, but they must preserve this separation of **runtime execution bindings** from **provider-visible compiled tool schemas**.

~~nsaspy referenced this issue 2026-09-10 21:20:50 +00:00~~

[EPIC] Wire prompt compiler tool projection into the real rlm_completion planner surface #176

nsaspy commented

2026-09-10 21:21:37 +00:00

Author

Owner

Duplicate of #176 (pre-existing Forgejo mirror). Closing this accidental duplicate created by today's open-state sync; #176 stays canonical on Forgejo.

nsaspy closed this issue

2026-09-10 21:21:37 +00:00

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

nsaspy/prolog-rlm#450

No description provided.

Rows
Columns