Add durable state graphs, checkpoints, and resumable execution (#12) #31

Merged

lost-rob0t merged 30 commits from feature/issue-12-durable-graph into main

2026-08-12 18:15:04 +00:00

lost-rob0t commented

2026-08-12 18:13:33 +00:00

(Migrated from github.com)

Closes #12.

Durable Prolog state graphs

Replace the rlm_graph stub with a validated graph runtime built around declarative state schemas, nodes, fixed/conditional edges, closed reducers, bounded loops, subgraphs, interrupts, checkpoints, history, streaming events, resume, wall-time limits, and cancellation.

Graph specs contain only IDs and declarative data. Trusted node/router callables and compiled subgraphs live in a separate host registry; model-produced data never becomes a callable.

Compile-time validation

The compiler rejects invalid graphs before execution, including duplicate state fields/nodes, reserved node names, missing registry entries, invalid edge targets, duplicate route keys, unreachable nodes, nodes without a structural path to end, and malformed start/end topology.

State and reducers

Schemas declare field type/default/reducer. Supported types are any, atom, string, integer, number, boolean, list, and dict. Reducers are intentionally closed to replace, append, and sum.

Runtime bounds

Execution is independently bounded by total node steps, per-node visits, and hard wall time. A router may create runtime loops, but those loops cannot execute without bound.

Interrupt / resume

A node can return interrupt(Reason, Patch). The patch is applied and the next edge is resolved before checkpointing, so resume continues from the next node rather than repeating the interrupting side effect. Resume values are exposed as Context.resume; resume(Value, Patch) may also atomically update state.

Checkpoints and history

Add memory and SWI library(persistency) backends. Persistent checkpoints/events contain only ground serializable graph state—not trusted closures. Event payload dict tags are canonicalized before persistence/streaming so JSON-style anonymous tags cannot make otherwise-ground events non-ground.

The acceptance suite proves both in-process detach/reattach and a literal process restart: one fresh swipl process pauses/persists and exits; a second fresh swipl process loads the graph definition/registry from scratch, reattaches the journal, resumes, and completes.

Cancellation

Graph cancellation tokens actively signal a currently executing graph thread. Nested inline subgraphs share the same token and use reference-counted thread registration, preventing a child graph from removing its parent's cancellation boundary.

Subgraphs

Compiled subgraphs can be registered as node implementations. Completed child state is converted to reducer-aware deltas before merging into the parent; sum and append therefore preserve semantics instead of double-applying absolute child state.

Public API

Expose graph defaults, compilation, backend lifecycle, run/resume, checkpoint/history, and graph cancellation through the main rlm module.

Deterministic acceptance

Exact-head deterministic CI is green and covers:

conditional branching and bounded loops;
unreachable and no-path-to-end rejection;
per-node visit exhaustion;
reducer/state semantics;
memory interrupt/resume;
persistent detach/reattach resume;
persistent resume across two different SWI processes;
ordered event streaming/history;
hard wall-time interruption;
cancellation of blocking work;
subgraph reducer-aware state merging;
static loading and whitespace validation.

Safety / cleanup

Temporary branch-only write-capable patch workflows/helpers used during connector-only development are removed from the final tree. Normal CI remains contents: read.

Do not merge until exact-head deterministic CI and the same-repository REAL OpenRouter regression job are both green, the PR is mergeable, and the final diff/credential scan is clean.

Closes #12. ## Durable Prolog state graphs Replace the `rlm_graph` stub with a validated graph runtime built around declarative state schemas, nodes, fixed/conditional edges, closed reducers, bounded loops, subgraphs, interrupts, checkpoints, history, streaming events, resume, wall-time limits, and cancellation. Graph specs contain only IDs and declarative data. Trusted node/router callables and compiled subgraphs live in a separate host registry; model-produced data never becomes a callable. ## Compile-time validation The compiler rejects invalid graphs before execution, including duplicate state fields/nodes, reserved node names, missing registry entries, invalid edge targets, duplicate route keys, unreachable nodes, nodes without a structural path to `end`, and malformed start/end topology. ## State and reducers Schemas declare field type/default/reducer. Supported types are `any`, `atom`, `string`, `integer`, `number`, `boolean`, `list`, and `dict`. Reducers are intentionally closed to `replace`, `append`, and `sum`. ## Runtime bounds Execution is independently bounded by total node steps, per-node visits, and hard wall time. A router may create runtime loops, but those loops cannot execute without bound. ## Interrupt / resume A node can return `interrupt(Reason, Patch)`. The patch is applied and the next edge is resolved before checkpointing, so resume continues from the next node rather than repeating the interrupting side effect. Resume values are exposed as `Context.resume`; `resume(Value, Patch)` may also atomically update state. ## Checkpoints and history Add memory and SWI `library(persistency)` backends. Persistent checkpoints/events contain only ground serializable graph state—not trusted closures. Event payload dict tags are canonicalized before persistence/streaming so JSON-style anonymous tags cannot make otherwise-ground events non-ground. The acceptance suite proves both in-process detach/reattach and a literal process restart: one fresh `swipl` process pauses/persists and exits; a second fresh `swipl` process loads the graph definition/registry from scratch, reattaches the journal, resumes, and completes. ## Cancellation Graph cancellation tokens actively signal a currently executing graph thread. Nested inline subgraphs share the same token and use reference-counted thread registration, preventing a child graph from removing its parent's cancellation boundary. ## Subgraphs Compiled subgraphs can be registered as node implementations. Completed child state is converted to reducer-aware deltas before merging into the parent; `sum` and `append` therefore preserve semantics instead of double-applying absolute child state. ## Public API Expose graph defaults, compilation, backend lifecycle, run/resume, checkpoint/history, and graph cancellation through the main `rlm` module. ## Deterministic acceptance Exact-head deterministic CI is green and covers: - conditional branching and bounded loops; - unreachable and no-path-to-end rejection; - per-node visit exhaustion; - reducer/state semantics; - memory interrupt/resume; - persistent detach/reattach resume; - persistent resume across two different SWI processes; - ordered event streaming/history; - hard wall-time interruption; - cancellation of blocking work; - subgraph reducer-aware state merging; - static loading and whitespace validation. ## Safety / cleanup Temporary branch-only write-capable patch workflows/helpers used during connector-only development are removed from the final tree. Normal CI remains `contents: read`. Do not merge until exact-head deterministic CI and the same-repository REAL OpenRouter regression job are both green, the PR is mergeable, and the final diff/credential scan is clean.