Add first-class managed conversation runtime with bounded rolling context #440

New issue

Closed

opened 2026-09-10 21:18:14 +00:00 by nsaspy · 1 comment

nsaspy commented

2026-09-10 21:18:14 +00:00

Owner

Problem

rlm_completion/4 is intentionally a bounded fresh-root primitive. Interactive AgentProlog-style sessions need durable conversational continuity without replaying lifetime history into every model request.

Canonical contract

full durable transcript
        |
        +--> bounded rolling hot context
        +--> existing warm artifacts
        +--> synthetic cold-history boundary
        |      tells the model how to recover omitted turns
        `--> lazy cold-history RLM context
             peek / slice / search on demand

Full durable conversation API: create/open/list, append, exact lookup, ordered history/ranges, search, export/introspection.
Preserve the complete original transcript; runtime projection must never rewrite old user/assistant/tool messages.
Managed turn API layered above rlm_completion/4; keep the stateless primitive public and unchanged.
Explicit max_context_tokens, output reserve, safety margin, minimum recent turns, and fail-closed overflow behavior.
Provider hard window is a physical ceiling, not the working target. A 300k operator cap remains 300k on a 1M-token model.
Existing warm artifacts participate in the normal managed context pack when a warm store is configured.
A small synthetic cold-history boundary tells the model how to recover history outside the guaranteed hot tail.
RLM cold-history retrieval makes old turns addressable without replaying the full transcript.
Token accounting covers every provider-visible compilation stage plus host-only metadata separately.
Prolog constraints/backtracking choose the highest-utility admissible context pack under the hard ceiling.
Tool/MCP/skill/project context eventually participates in that same budget.

Cold-history boundary

The runtime does not edit an old message to replace its contents with retrieval instructions. Instead, rlm_conversation_runtime inserts a mandatory provider-visible managed_cold_history_boundary context unit when the projected conversation extends beyond min_recent_turns.

The boundary reports the prefix outside the guaranteed hot tail and instructs the planner to retrieve authoritative original history before guessing, using bounded operations such as:

context(input(context), search("query"), Result).
context(input(context), slice(Start, Length), Result).
context(input(context), peek(item(Index)), Result).

It is charged in the normal token ledger and exposed through cold_history_boundary status. Specialized callers can disable it with cold_history_boundary(false); default is enabled.

Warm context versus compaction

Warm context is wired into the managed runtime. Automatic compaction is not.

When warm_store(ArtifactStore) is present, the public runtime loads/ranks existing warm artifacts, converts them into multi-representation context units, and admits them into the same CLP(FD) packer as hot context and the cold boundary. Callers do not need manual warm context_units/1 plumbing.

A normal managed turn does not choose a range, call a summarizer, publish a new warm artifact, or trigger compaction under token pressure. Warm production remains an explicit API.

Current implementation slices

#102: managed conversation store/API, token policy, ledger, CLP(FD) packer.
#104: durable warm-context derivation/versioned artifacts.
#106: warm-aware managed orchestration, synthetic cold-history boundary, and lazy conversation-backed cold-history retrieval.

Remaining work

provider/model tokenizer registry and final rendered-request counting;
prompt-compiler accounting for tools, MCP, skills, project instructions, and rendering overhead;
bounded hot-candidate selection for very large histories;
indexed cold-history retrieval/storage optimization;
async managed-turn and streaming surfaces for AgentProlog/frontends;
adapter metadata-size hardening (#107).

Invariants

no transcript deletion or rewriting;
existing warm state is reusable without caller-side context-unit plumbing;
no hidden compaction/summarization calls;
old history remains explicitly discoverable from active attention through the synthetic boundary;
no hidden authority widening through context selection;
no provider request may exceed the effective hard cap;
unknown tokenization is marked estimated/unknown, never fabricated as exact;
writes remain deterministic under retry/backtracking and use the existing effect/authority boundaries where applicable.

## Problem `rlm_completion/4` is intentionally a bounded fresh-root primitive. Interactive AgentProlog-style sessions need durable conversational continuity without replaying lifetime history into every model request. ## Canonical contract ```text full durable transcript | +--> bounded rolling hot context +--> existing warm artifacts +--> synthetic cold-history boundary | tells the model how to recover omitted turns `--> lazy cold-history RLM context peek / slice / search on demand ``` - Full durable conversation API: create/open/list, append, exact lookup, ordered history/ranges, search, export/introspection. - Preserve the complete original transcript; runtime projection must never rewrite old user/assistant/tool messages. - Managed turn API layered above `rlm_completion/4`; keep the stateless primitive public and unchanged. - Explicit `max_context_tokens`, output reserve, safety margin, minimum recent turns, and fail-closed overflow behavior. - Provider hard window is a physical ceiling, not the working target. A 300k operator cap remains 300k on a 1M-token model. - Existing warm artifacts participate in the normal managed context pack when a warm store is configured. - A small synthetic cold-history boundary tells the model how to recover history outside the guaranteed hot tail. - RLM cold-history retrieval makes old turns addressable without replaying the full transcript. - Token accounting covers every provider-visible compilation stage plus host-only metadata separately. - Prolog constraints/backtracking choose the highest-utility admissible context pack under the hard ceiling. - Tool/MCP/skill/project context eventually participates in that same budget. ## Cold-history boundary The runtime does **not** edit an old message to replace its contents with retrieval instructions. Instead, `rlm_conversation_runtime` inserts a mandatory provider-visible `managed_cold_history_boundary` context unit when the projected conversation extends beyond `min_recent_turns`. The boundary reports the prefix outside the guaranteed hot tail and instructs the planner to retrieve authoritative original history before guessing, using bounded operations such as: ```prolog context(input(context), search("query"), Result). context(input(context), slice(Start, Length), Result). context(input(context), peek(item(Index)), Result). ``` It is charged in the normal token ledger and exposed through `cold_history_boundary` status. Specialized callers can disable it with `cold_history_boundary(false)`; default is enabled. ## Warm context versus compaction Warm context is wired into the managed runtime. Automatic compaction is not. When `warm_store(ArtifactStore)` is present, the public runtime loads/ranks existing warm artifacts, converts them into multi-representation context units, and admits them into the same CLP(FD) packer as hot context and the cold boundary. Callers do not need manual warm `context_units/1` plumbing. A normal managed turn does **not** choose a range, call a summarizer, publish a new warm artifact, or trigger compaction under token pressure. Warm production remains an explicit API. ## Current implementation slices - #102: managed conversation store/API, token policy, ledger, CLP(FD) packer. - #104: durable warm-context derivation/versioned artifacts. - #106: warm-aware managed orchestration, synthetic cold-history boundary, and lazy conversation-backed cold-history retrieval. ## Remaining work 1. provider/model tokenizer registry and final rendered-request counting; 2. prompt-compiler accounting for tools, MCP, skills, project instructions, and rendering overhead; 3. bounded hot-candidate selection for very large histories; 4. indexed cold-history retrieval/storage optimization; 5. async managed-turn and streaming surfaces for AgentProlog/frontends; 6. adapter metadata-size hardening (#107). ## Invariants - no transcript deletion or rewriting; - existing warm state is reusable without caller-side context-unit plumbing; - no hidden compaction/summarization calls; - old history remains explicitly discoverable from active attention through the synthetic boundary; - no hidden authority widening through context selection; - no provider request may exceed the effective hard cap; - unknown tokenization is marked estimated/unknown, never fabricated as exact; - writes remain deterministic under retry/backtracking and use the existing effect/authority boundaries where applicable.

nsaspy commented

2026-09-10 21:23:36 +00:00

Author

Owner

Duplicate of #101 (pre-existing Forgejo mirror with GitHub number parity). Closing this accidental duplicate created by today's open-state sync; #101 stays canonical on Forgejo.

nsaspy closed this issue

2026-09-10 21:23:37 +00:00