nsaspy/prolog-rlm

Fork 0

feat: Prolog-RLM as a self-hosted language — stage0 kernel, stage1/stage2, RLM semantics, actors, MCP, RSI #464

Open

nsaspy wants to merge 4 commits from feat/prolog-rlm-language into main

nsaspy commented

2026-09-19 05:40:25 +00:00

Owner

Runtime invariant implemented

Prolog-RLM becomes a self-hosted LANGUAGE whose RLM semantics are part of the language design, not a harness layered on top:

Stage0 kernel (prlm_kernel): our own Prolog machine — explicit '$v'(Name) variable representation, functional unification over immutable environments, an explicit choicepoint stack (standard cut scope, if-then-else commit, catch/throw ball unwinding), arithmetic, standard order, dynamic DB, modules with exports/imports, source loading with line provenance, and step budgets as a machine concept. Host usage is confined to a declared whitelist (prlm_host_primitive/1). No-escape is structural: program goals contain '$v'(_) and are never host-callable, so the engine cannot delegate program execution to SWI.
Stage1 (lang/prlm_std.prlm, lang/prlm_meta.prlm, lang/prlm_ctx.prlm, lang/prlm_mode.prlm, lang/prlm_plan.prlm): standard library, meta-interpreter, context-compiler operations, mode-selection and planning rules — written IN the language and executed by the engine.
Stage2: stage1 executed by stage1 (msolve); prlm_selfhost_gate/1 fails closed unless stage0-direct and stage2-interpreted runs agree solution-by-solution over a probe corpus, with stage1 proven loaded.
RLM loop (prlm_rlm): SPEC → PLAN → EXECUTE → VERIFY → LEARN → CONTINUE with in-language mode selection (/direct, /symbolic, /symbolic-recursive, /auto, /learn); failed verification causes bounded repair/replanning — the spec object is never weakened. Supervised recursion: depth budget 500 shared per session, cancellation checked per step.
Actors/agents: mailboxes, supervision (restart|stop|escalate), subtree cancellation, checkpoint/restore; agents are actor kinds with authority/capability narrowing (child ⊆ parent, never wider) and structured results.
Tools: capability-gated with args schema, authority, effects, result schema, evidence, provenance; projections both ways (tool→predicate, predicate→tool).
Providers: proposals are data; prlm_validate_proposal/3 admits only whitelisted plan-step shapes. Prolog-only mode needs no model.
Events: one normalized stream (18 canonical types) consumed via JSONL — the same surface OpenCode/Zara/MCP UIs consume.
Introspection → MCP: usage facts (calls, errors, slow ops, pass/fail patterns, missing capabilities) from the event stream; MCP surface generated from mcp_export/1 directives + <name>_schema/1 companions; external MCP tools consumed as capability-gated tools over a fail-closed transport.
Learn: verified outcomes become knowledge with provenance{source,time,confidence,scope}; LLM text goes only to an untrusted store and can never surface as trusted knowledge.
RSI: bounded OBSERVE→PROPOSE→IMPLEMENT→TEST→BENCHMARK→VERIFY→ACCEPT/REJECT over isolated workspaces with experiment/2, experiment_metric/4, experiment_change/2, experiment_result/2, experiment_failure/2 facts; promotion only through gated tools; rejected candidates leave the runtime untouched (rollback check) and stay in the KB.

Non-goals

No model-weight changes; RSI evolves language/runtime facts only.
Real git worktree execution and live network MCP transports are tool-boundary plugs (directory-isolated core, deterministic tests); production mapping documented.
Network LLM adapters stay deferred hooks; provider correctness is proven with deterministic adapters.

Tests / evidence

New plunit suites (all registered in the deterministic corpus): kernel 61, session 22, context 25, tools 26, providers 17, actors 19, rlm 13, selfhost 11, usage 8, mcp_export 14, learn 14, rsi 14, cli 17.
Full repo gates from this branch: test/run_tests.pl → 1496/1496 passed, 0 failed; test/check_runtime.pl ok; benchmark/run.pl -- deterministic 16/16; bin/prolog-rlm.pl -- demo --json pass; bin/prlm.pl -- selfhost gate ok (17/17 equivalence); git diff --check clean.

Remaining follow-up scope

Move more of the RLM layer (verify rules, repair policy) from host Prolog into lang/*.prlm.
Real git worktree-backed RSI execution behind the process tool boundary.
Live MCP transport adapters (stdio/HTTP) behind the existing fail-closed hook.

## Runtime invariant implemented Prolog-RLM becomes a self-hosted LANGUAGE whose RLM semantics are part of the language design, not a harness layered on top: - **Stage0 kernel (`prlm_kernel`)**: our own Prolog machine — explicit `'$v'(Name)` variable representation, functional unification over immutable environments, an explicit choicepoint stack (standard cut scope, if-then-else commit, catch/throw ball unwinding), arithmetic, standard order, dynamic DB, modules with exports/imports, source loading with line provenance, and step budgets as a machine concept. Host usage is confined to a declared whitelist (`prlm_host_primitive/1`). No-escape is structural: program goals contain `'$v'(_)` and are never host-callable, so the engine cannot delegate program execution to SWI. - **Stage1** (`lang/prlm_std.prlm`, `lang/prlm_meta.prlm`, `lang/prlm_ctx.prlm`, `lang/prlm_mode.prlm`, `lang/prlm_plan.prlm`): standard library, meta-interpreter, context-compiler operations, mode-selection and planning rules — written IN the language and executed by the engine. - **Stage2**: stage1 executed by stage1 (`msolve`); `prlm_selfhost_gate/1` fails closed unless stage0-direct and stage2-interpreted runs agree solution-by-solution over a probe corpus, with stage1 proven loaded. - **RLM loop (`prlm_rlm`)**: SPEC → PLAN → EXECUTE → VERIFY → LEARN → CONTINUE with in-language mode selection (`/direct`, `/symbolic`, `/symbolic-recursive`, `/auto`, `/learn`); failed verification causes bounded repair/replanning — the spec object is never weakened. Supervised recursion: depth budget 500 shared per session, cancellation checked per step. - **Actors/agents**: mailboxes, supervision (restart|stop|escalate), subtree cancellation, checkpoint/restore; agents are actor kinds with authority/capability narrowing (child ⊆ parent, never wider) and structured results. - **Tools**: capability-gated with args schema, authority, effects, result schema, evidence, provenance; projections both ways (tool→predicate, predicate→tool). - **Providers**: proposals are data; `prlm_validate_proposal/3` admits only whitelisted plan-step shapes. Prolog-only mode needs no model. - **Events**: one normalized stream (18 canonical types) consumed via JSONL — the same surface OpenCode/Zara/MCP UIs consume. - **Introspection → MCP**: usage facts (calls, errors, slow ops, pass/fail patterns, missing capabilities) from the event stream; MCP surface generated from `mcp_export/1` directives + `<name>_schema/1` companions; external MCP tools consumed as capability-gated tools over a fail-closed transport. - **Learn**: verified outcomes become knowledge with provenance{source,time,confidence,scope}; LLM text goes only to an untrusted store and can never surface as trusted knowledge. - **RSI**: bounded OBSERVE→PROPOSE→IMPLEMENT→TEST→BENCHMARK→VERIFY→ACCEPT/REJECT over isolated workspaces with `experiment/2`, `experiment_metric/4`, `experiment_change/2`, `experiment_result/2`, `experiment_failure/2` facts; promotion only through gated tools; rejected candidates leave the runtime untouched (rollback check) and stay in the KB. ## Non-goals - No model-weight changes; RSI evolves language/runtime facts only. - Real `git worktree` execution and live network MCP transports are tool-boundary plugs (directory-isolated core, deterministic tests); production mapping documented. - Network LLM adapters stay deferred hooks; provider correctness is proven with deterministic adapters. ## Tests / evidence - New plunit suites (all registered in the deterministic corpus): kernel 61, session 22, context 25, tools 26, providers 17, actors 19, rlm 13, selfhost 11, usage 8, mcp_export 14, learn 14, rsi 14, cli 17. - Full repo gates from this branch: `test/run_tests.pl` → **1496/1496 passed, 0 failed**; `test/check_runtime.pl` ok; `benchmark/run.pl -- deterministic` 16/16; `bin/prolog-rlm.pl -- demo --json` pass; `bin/prlm.pl -- selfhost` gate ok (17/17 equivalence); `git diff --check` clean. ## Remaining follow-up scope - Move more of the RLM layer (verify rules, repair policy) from host Prolog into `lang/*.prlm`. - Real `git worktree`-backed RSI execution behind the process tool boundary. - Live MCP transport adapters (stdio/HTTP) behind the existing fail-closed hook.

nsaspy added 4 commits

2026-09-19 05:40:25 +00:00

feat(language): stage0 kernel — self-hosted Prolog machine (prlm_kernel) af30e636df

Prolog-RLM as a language needs its own engine: a complete Prolog machine
implemented in Prolog with explicit '$v'(Name) variables, functional
unification over immutable environments, an explicit choicepoint stack
(standard cut scope, if-then-else commit, catch/throw ball unwinding),
arithmetic, standard order of terms, dynamic DB (assertz/retract/clause),
modules with exports/imports, file loading with line provenance, and
budget charging (steps) as a first-class machine concept.

The no-escape property: program terms containing '$v'(_} are never
host-callable, so the engine structurally cannot delegate program
execution to SWI. Host primitives are an explicit whitelist
(prlm_host_primitive/1); everything above it must be (and in stage1 is)
implemented in the language itself.

Untrusted text becomes inert data at prlm_read_goal/3 (parse only,
reserved '$v' functor rejected).

61 plunit tests: terms, unification, occurs-check option, DFS order,
cut scope, if-then-else commit, negation, once, forall, recursion
(incl. depth 500), arithmetic, findall, call/1-7, catch/throw,
term builtins, dynamic DB, budgets, modules, use_module, load errors,
provenance, ground solution rows.

feat(language): RLM runtime substrate — sessions, context compiler, tools, providers 58df9b9b77

Four language-runtime subsystems on the stage0 kernel:

- prlm_session: the ONE normalized event stream (seq-stamped, JSONL
  exportable, subscriber fan-out) consumed by OpenCode/Zara/MCP UIs;
  shared budgets (steps/tool_calls/depth 500/repairs) so recursive
  agents cannot escape accounting; canonical authority tiers
  (approve_diff < allow_once < allow_session < dangerous) with
  narrowing-only child derivation; fail-closed cancellation; session
  KVS as the state substrate.

- prlm_context + lang/prlm_ctx.prlm: the native context compiler.
  Deterministic inspect/search/select/slice/transform/delegate/
  aggregate/compile operations implemented IN THE LANGUAGE and
  executed by the engine — contexts are budgeted, ordered
  (priority desc, id asc), deduplicated, and never assembled by a
  model call.

- prlm_tools: tools as native capabilities with explicit args schema,
  capability, authority, effects, result schema, evidence and
  provenance; gated invocation order (cancel -> lookup -> schema ->
  caps -> authority -> budget -> event -> execute -> evidence ->
  observation); projections both ways (tool->predicate, engine
  predicate->tool); MCP/actor handlers via fail-closed hooks.

- prlm_providers: LLMs as replaceable proposal generators (prolog
  no-model provider, echo, fake, fake_fn, deferred http/process/actor
  adapters). Proposals are data: prlm_validate_proposal/3 admits only
  whitelisted plan-step shapes; everything else is rejected. Prolog-
  only mode needs no model at all.

Combined suites: session 22, context 25, tools 26, providers 17 — all
green; kernel 61 untouched.

feat(language): native actors + agents built on actor primitives 4386253c90

Actors are runtime primitives of the language: identity, state, FIFO
mailbox, messages, supervision (restart|stop|escalate), lifecycle,
capabilities, cancellation (subtree) and child actors. Deterministic
round-robin scheduler — no threads, fully reproducible traces.

Agents are KINDS of actor, not a separate architecture: profiles,
delegation with authority/capability narrowing (child subset of parent,
never wider), depth charged against the shared session budget (default
500), checkpoint/restore, and structured results
(agent_result{status,value,evidence,usage}) — never prose. The goal
hook (prlm_agent_run_goal/5) defaults to a deterministic prolog-only
implementation; the RLM layer replaces it.

MCP connections and worker processes appear as external actors via the
fail-closed prlm_actor_external_ask/5 hook.

Kernel addition: prlm_call_first/5 — the single sanctioned bridge for
trusted host code whose goals contain host variables (the machine
unifier fails closed on them otherwise).

19 tests green.

feat(language): stage1/stage2 self-hosting, RLM loop, introspection+MCP, learn, RSI, CLI

CI / REAL OpenRouter integration (pull_request) Has been skipped

Details

CI / Deterministic unit and load checks (pull_request) Failing after 1m51s

Details

Clean SWI pack install / Install and load copied pack (pull_request) Successful in 57s

Details

Nix flake / Flake package and clean runtime load (pull_request) Successful in 4m58s

Details

Tree-sitter FFI / Direct SWI-Prolog Tree-sitter FFI (pull_request) Successful in 4m17s

Details

Paid OpenRouter / Pinned paid OpenRouter integration (pull_request) Successful in 7m28s

Details

e31004f330

Completes the language layer on the stage0 kernel:

- prlm_selfhost + lang/prlm_std.prlm + lang/prlm_meta.prlm: the standard
  library and the msolve meta-interpreter are WRITTEN IN THE LANGUAGE and
  executed by the engine. Stage2 runs stage1 under msolve; the gate
  (prlm_selfhost_gate/1) fails closed unless stage0-direct and
  stage2-interpreted execution agree solution-by-solution over the probe
  corpus (17 probes), with stage1 proven loaded (a fresh engine failing
  identically on both sides does not count). No-escape is structural:
  program terms contain '$v'(_) variables and are never host-callable.
  Kernel additions: module-qualified clause(M:Head, Body), call/N
  unwrapping, defining-module clause bodies, prlm_import/4,
  prlm_call_first/5 host-variable bridge.

- prlm_rlm + lang/prlm_mode.prlm + lang/prlm_plan.prlm: the SPEC ->
  PLAN -> EXECUTE -> VERIFY -> LEARN -> CONTINUE loop as language
  semantics. Mode selection and planning run in-language. Failed
  verification causes bounded repair/replanning (planned_reorder /
  add_model_step / retry) — the specification object is never weakened.
  Modes: /direct (provider passthrough, no effects), /symbolic,
  /symbolic-recursive (agents allowed), /auto (in-language rules),
  /learn. spawn_agent in symbolic mode is forbidden_in_mode, not
  silently allowed.

- prlm_usage + prlm_mcp_export: the language inspects its own usage from
  the one normalized event stream (calls, errors, ops, slow ops,
  pass/fail patterns, missing-capability signals) and generates its MCP
  surface from mcp_export/1 directives + the <name>_schema/1 companion
  convention; registered tools execute real engine predicates with
  evidence and provenance. MCP consumption projects external tool
  descriptors into capability-gated tools over a fail-closed transport.

- prlm_learn: /learn converts completed verified outcomes into knowledge
  facts with provenance{source, time, confidence, scope}; LLM text goes
  only to the untrusted store (confidence 0.3, scope untrusted) and can
  never surface through prlm_knowledge/3.

- prlm_rsi: bounded OBSERVE -> PROPOSE -> IMPLEMENT -> TEST -> BENCHMARK
  -> VERIFY -> ACCEPT/REJECT loop over isolated workspaces with the
  experiment/2, experiment_metric/4, experiment_change/2,
  experiment_result/2, experiment_failure/2 fact shapes; promotion of a
  candidate to the runtime happens only through gated tools after tests
  and metrics pass; rejected candidates keep the runtime untouched
  (rollback check) and stay in the KB.

- bin/prlm.pl CLI (demo, stage0, stage1, stage2, selfhost, run --mode,
  mcp, usage, rsi, events; JSON lines, exit-code matrix) + literate docs
  (docs/prlm-language.md, self-hosting.md, rsi.md, docs/org/.

All prlm suites registered in the deterministic corpus.
Full repo gates: run_tests 1496/1496 passed (0 failed), check_runtime ok,
benchmark deterministic 16/16, demo --json pass, git diff --check clean.

CI / REAL OpenRouter integration (pull_request) Has been skipped

Details

CI / Deterministic unit and load checks (pull_request) Failing after 1m51s

Details

Clean SWI pack install / Install and load copied pack (pull_request) Successful in 57s

Details

Nix flake / Flake package and clean runtime load (pull_request) Successful in 4m58s

Details

Tree-sitter FFI / Direct SWI-Prolog Tree-sitter FFI (pull_request) Successful in 4m17s

Details

Paid OpenRouter / Pinned paid OpenRouter integration (pull_request) Successful in 7m28s

Details

This pull request can be merged automatically.

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin feat/prolog-rlm-language:feat/prolog-rlm-language

git switch feat/prolog-rlm-language

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main

git merge --no-ff feat/prolog-rlm-language

git switch feat/prolog-rlm-language

git rebase main

git switch main

git merge --ff-only feat/prolog-rlm-language

git switch feat/prolog-rlm-language

git rebase main

git switch main

git merge --no-ff feat/prolog-rlm-language

git switch main

git merge --squash feat/prolog-rlm-language