nsaspy/prolog-rlm

Fork 0

[EPIC] Durable tool-result projection with public visibility presets #211

New issue

Closed

opened 2026-08-25 11:23:41 +00:00 by lost-rob0t · 3 comments

lost-rob0t commented

2026-08-25 11:23:41 +00:00

(Migrated from github.com)

Goal

Separate tool execution, durable retention, and provider-visible projection for tool results.

Core invariant:

REGISTERED != EXECUTABLE != EXECUTED != STORED != PROJECTED

Tool results remain authoritative and retrievable even when they are no longer automatically injected into model context.

Public presets

Expose a small public API of presets that compile to an internal canonical projection policy:

full.
once.
reference.
hidden.

Canonical meanings:

full      => initial=full,      after_consumption=full
once      => initial=full,      after_consumption=none
reference => initial=reference, after_consumption=reference
hidden    => initial=none,      after_consumption=none

The internal representation should be explicit and extensible, e.g.:

result_projection{
    initial: Initial,
    after_consumption: After,
    retention: durable,
    retrievable: true
}.

Future presets such as summarized may map onto the same internal policy without changing the public tool invocation API.

visibility is a public/API convenience term. Internally this is projection policy; it must never imply deletion or weakened auditing.

Architecture

tool invocation
      |
      v
execute + authority/effect boundary
      |
      v
durable tool result + trace + result ref
      |
      v
projection policy
  |       |         |          |
 full    once    reference    hidden

The immediate tool consumer may need the full result while later turns do not. once is therefore the architectural center rather than hidden.

Slices

Canonical projection metadata on tool messages/results, with normalized public presets.
Add result_visibility(full|once|reference|hidden) to tool invocation options and persist the complete result regardless of projection.
Teach managed context projection not to make every recent tool result mandatory.
Track consumption for once: full in the immediate continuation, omitted afterward.
Preserve assistant-tool-call/tool-result causal integrity when providers require paired messages.
Add durable result references and bounded lazy retrieval for omitted results.
Add compact reference projection containing non-secret identity/status/size metadata rather than payload.
Make the token ledger charge the actual projected representation, never the retained payload when omitted.

Safety / authority invariants

Projection never grants execution authority.
Projection never unregisters a runtime tool.
Projection never deletes the authoritative tool result.
Hidden/omitted results remain auditable and retrievable subject to normal capabilities/authority.
Provider-visible data must not expose host-only handlers, credentials, or secret traces.
Result projection must not create invalid provider tool-call histories.

Required regressions

projection_preset_full_is_canonical
projection_preset_once_is_canonical
projection_preset_reference_is_canonical
projection_preset_hidden_is_canonical
invalid preset fails structurally
canonical tool-result/message metadata is ground and round-trippable
once is visible to the immediate continuation only
retained once result is retrievable after projection expires
reference projection contains no full payload
hidden result is never automatically projected
full preserves old behavior
large tool output does not poison subsequent context
provider-required tool-call/result causal pairs remain valid
token ledger reflects projected rather than retained bytes/tokens

Relation to #176

#176 handles tool schema visibility: which registered tools the model sees.

This epic handles the mirror-image problem: tool result visibility after execution.

Together they establish a single provider-projection boundary for both tool affordances and tool observations.

## Goal Separate tool execution, durable retention, and provider-visible projection for tool results. Core invariant: ```text REGISTERED != EXECUTABLE != EXECUTED != STORED != PROJECTED ``` Tool results remain authoritative and retrievable even when they are no longer automatically injected into model context. ## Public presets Expose a small public API of presets that compile to an internal canonical projection policy: ```prolog full. once. reference. hidden. ``` Canonical meanings: ```text full => initial=full, after_consumption=full once => initial=full, after_consumption=none reference => initial=reference, after_consumption=reference hidden => initial=none, after_consumption=none ``` The internal representation should be explicit and extensible, e.g.: ```prolog result_projection{ initial: Initial, after_consumption: After, retention: durable, retrievable: true }. ``` Future presets such as `summarized` may map onto the same internal policy without changing the public tool invocation API. `visibility` is a public/API convenience term. Internally this is projection policy; it must never imply deletion or weakened auditing. ## Architecture ```text tool invocation | v execute + authority/effect boundary | v durable tool result + trace + result ref | v projection policy | | | | full once reference hidden ``` The immediate tool consumer may need the full result while later turns do not. `once` is therefore the architectural center rather than `hidden`. ## Slices 1. [ ] Canonical projection metadata on tool messages/results, with normalized public presets. 2. [ ] Add `result_visibility(full|once|reference|hidden)` to tool invocation options and persist the complete result regardless of projection. 3. [ ] Teach managed context projection not to make every recent tool result mandatory. 4. [ ] Track consumption for `once`: full in the immediate continuation, omitted afterward. 5. [ ] Preserve assistant-tool-call/tool-result causal integrity when providers require paired messages. 6. [ ] Add durable result references and bounded lazy retrieval for omitted results. 7. [ ] Add compact `reference` projection containing non-secret identity/status/size metadata rather than payload. 8. [ ] Make the token ledger charge the actual projected representation, never the retained payload when omitted. ## Safety / authority invariants - Projection never grants execution authority. - Projection never unregisters a runtime tool. - Projection never deletes the authoritative tool result. - Hidden/omitted results remain auditable and retrievable subject to normal capabilities/authority. - Provider-visible data must not expose host-only handlers, credentials, or secret traces. - Result projection must not create invalid provider tool-call histories. ## Required regressions - `projection_preset_full_is_canonical` - `projection_preset_once_is_canonical` - `projection_preset_reference_is_canonical` - `projection_preset_hidden_is_canonical` - invalid preset fails structurally - canonical tool-result/message metadata is ground and round-trippable - `once` is visible to the immediate continuation only - retained `once` result is retrievable after projection expires - reference projection contains no full payload - hidden result is never automatically projected - full preserves old behavior - large tool output does not poison subsequent context - provider-required tool-call/result causal pairs remain valid - token ledger reflects projected rather than retained bytes/tokens ## Relation to #176 #176 handles **tool schema visibility**: which registered tools the model sees. This epic handles the mirror-image problem: **tool result visibility** after execution. Together they establish a single provider-projection boundary for both tool affordances and tool observations.

lost-rob0t commented

2026-08-25 11:27:11 +00:00

(Migrated from github.com)

Slice 1 is implemented in #213 on agent/tool-result-projection-presets.

Landed API/data shape in the PR:

result_visibility_preset/2
result_projection_normalize/2
tool_message_projection/3
tool_result_projection/3

Public presets compile to canonical result_projection{initial,after_consumption,retention:durable,retrievable:true} data. This PR deliberately does not change invocation or context packing yet; that remains slice 2+.

Slice 1 is implemented in #213 on `agent/tool-result-projection-presets`. Landed API/data shape in the PR: ```prolog result_visibility_preset/2 result_projection_normalize/2 tool_message_projection/3 tool_result_projection/3 ``` Public presets compile to canonical `result_projection{initial,after_consumption,retention:durable,retrievable:true}` data. This PR deliberately does not change invocation or context packing yet; that remains slice 2+.

lost-rob0t commented

2026-08-26 15:41:59 +00:00

(Migrated from github.com)

Slice 1 recovery — deterministic failure diagnosed and existing PR rebased forward without force

Recovered existing PR #213 rather than creating a replacement transaction.

Regression evidence

Old exact head fced85495700a0746ce49d7f697db3aab39f408d had one deterministic failure out of 889 tests:

rlm_chain_message_metadata:tool_result_projection_is_canonical_and_ground

All other deterministic tests passed (888/889), and REAL/Paid OpenRouter, Nix, clean-pack and Tree-sitter lanes were green on that old transaction.

Root cause: canonical_tool_result/2 required ground(Result0) before retagging. The public contract/test passes a normal anonymous SWI dict (_{...}). Anonymous dict tags are variables, so a dict whose key/value payload is fully ground is still not ground/1 solely because of its tag. Rejecting that shape accidentally made the canonicalizer require callers to invent a named input tag.

Design / adversarial decision

GO with the narrower semantic boundary:

require is_dict(Result0);
extract Pairs with dict_pairs/3;
require ground(Pairs) so no model/runtime variable survives in payload data;
rebuild with fixed tool_result tag;
attach canonical projection policy.

This accepts anonymous input tags while preserving the security/data invariant that projected tool-result payload data is ground. A new negative regression explicitly proves a genuinely nonground payload is still rejected.

Recovery realization

Existing branch agent/tool-result-projection-presets was fast-forwarded, without force, to merge/recovery commit 99dfc12e6651f69f2e2cac5bf23d1ac224df3e7c with parents old #213 head fced854... and current canonical main 267697bef10a3fffff7c093e1435ece770e7444b.

The resulting branch is directly based on current main (behind_by:0) and the complete diff is still only the owned slice:

prolog/rlm_tool_projection.pl
test/rlm_chain_message_metadata_test.pl
docs/tool-result-projection.md

No invocation, authority/effect, managed-context, provider serialization, token-accounting, or downstream product behavior was pulled into slice 1.

Exact-head verification is now pending on 99dfc12e...; no prior green evidence is being reused for this changed head.

## Slice 1 recovery — deterministic failure diagnosed and existing PR rebased forward without force Recovered existing PR #213 rather than creating a replacement transaction. ### Regression evidence Old exact head `fced85495700a0746ce49d7f697db3aab39f408d` had one deterministic failure out of 889 tests: `rlm_chain_message_metadata:tool_result_projection_is_canonical_and_ground` All other deterministic tests passed (888/889), and REAL/Paid OpenRouter, Nix, clean-pack and Tree-sitter lanes were green on that old transaction. Root cause: `canonical_tool_result/2` required `ground(Result0)` before retagging. The public contract/test passes a normal anonymous SWI dict (`_{...}`). Anonymous dict tags are variables, so a dict whose key/value payload is fully ground is still not `ground/1` solely because of its tag. Rejecting that shape accidentally made the canonicalizer require callers to invent a named input tag. ### Design / adversarial decision GO with the narrower semantic boundary: 1. require `is_dict(Result0)`; 2. extract `Pairs` with `dict_pairs/3`; 3. require `ground(Pairs)` so no model/runtime variable survives in payload data; 4. rebuild with fixed `tool_result` tag; 5. attach canonical projection policy. This accepts anonymous input tags while preserving the security/data invariant that projected tool-result payload data is ground. A new negative regression explicitly proves a genuinely nonground payload is still rejected. ### Recovery realization Existing branch `agent/tool-result-projection-presets` was fast-forwarded, without force, to merge/recovery commit `99dfc12e6651f69f2e2cac5bf23d1ac224df3e7c` with parents old #213 head `fced854...` and current canonical `main` `267697bef10a3fffff7c093e1435ece770e7444b`. The resulting branch is directly based on current `main` (`behind_by:0`) and the complete diff is still only the owned slice: - `prolog/rlm_tool_projection.pl` - `test/rlm_chain_message_metadata_test.pl` - `docs/tool-result-projection.md` No invocation, authority/effect, managed-context, provider serialization, token-accounting, or downstream product behavior was pulled into slice 1. **Exact-head verification is now pending on `99dfc12e...`; no prior green evidence is being reused for this changed head.**

lost-rob0t commented

2026-08-26 16:12:03 +00:00

(Migrated from github.com)

Slice 1 exact-head verification — GO / ready for review

Recovered candidate 99dfc12e6651f69f2e2cac5bf23d1ac224df3e7c has now completed the fresh gate against unchanged canonical main 267697bef10a3fffff7c093e1435ece770e7444b.

Exact-head results:

canonical CI: success, including deterministic and credential-backed REAL OpenRouter jobs;
Paid OpenRouter: success;
Nix flake: success;
clean SWI pack install: success;
Tree-sitter FFI: success;
submitted PR reviews: none;
PR conversation comments: none;
unresolved review threads: none.

PR #213 remains mergeable and was promoted from draft to ready for review without changing its head. The original anonymous-dict regression is fixed without weakening the ground-payload invariant; the explicit nonground-payload rejection remains covered.

RAGE decision for slice 1 only: GO. Keep #211 open: invocation wiring, managed-context projection, once consumption, durable references, provider causal-pair preservation, and projected-token accounting remain later slices. Repository merge authorization still applies; this verification record is not a merge action.

## Slice 1 exact-head verification — GO / ready for review Recovered candidate `99dfc12e6651f69f2e2cac5bf23d1ac224df3e7c` has now completed the fresh gate against unchanged canonical `main` `267697bef10a3fffff7c093e1435ece770e7444b`. Exact-head results: - canonical CI: success, including deterministic and credential-backed REAL OpenRouter jobs; - Paid OpenRouter: success; - Nix flake: success; - clean SWI pack install: success; - Tree-sitter FFI: success; - submitted PR reviews: none; - PR conversation comments: none; - unresolved review threads: none. PR #213 remains mergeable and was promoted from draft to ready for review without changing its head. The original anonymous-dict regression is fixed without weakening the ground-payload invariant; the explicit nonground-payload rejection remains covered. RAGE decision for **slice 1 only: GO**. Keep #211 open: invocation wiring, managed-context projection, `once` consumption, durable references, provider causal-pair preservation, and projected-token accounting remain later slices. Repository merge authorization still applies; this verification record is not a merge action.