[P1] Add declarative MCP server facts with install and run predicates

lost-rob0t commented

2026-08-16 04:05:40 +00:00

(Migrated from github.com)

Goal

Represent MCP servers as ordinary inspectable Prolog facts and provide explicit predicates to install and run each declared server through a hard host-controlled execution boundary.

This belongs with the external tool/MCP integration layer, not as hard-coded concrete tool behavior in prolog-rlm core.

Final state — 2026-08-17

PR #60 (3db30d832f9b124d6b6e6aa3cbe2808914f4ae6e) landed the initial declarative lifecycle.

PR #66 (9ba763d248f6e087b3ff01c651762997f02ee726) landed the inert external mcp category/loading boundary.

PR #72 finishes the remaining declaration-security and host execution-policy acceptance:

inert multifile mcp_server/2 facts plus sanitized discovery;
explicit install/run/stop/restart/connect lifecycle with canonical execute -> async Future -> sync-await direction;
first-class env_ref/1 and config_ref/1 configuration references;
resolved secret values are fetched only inside the exact authority-permitted trusted continuation and are excluded from discovery, pending authority records, runtime handles, traces, and structured policy errors;
closed host-controlled installer and stdio execution profiles;
declarations select a trusted profile/package/version rather than arbitrary executable/argv or shell text;
structural package/version/profile/config/cwd validation occurs before authority and before process spawn;
dangerous skips interactive approval only and cannot bypass execution-profile allow-lists or hard validation;
exact normalized working directory participates in authority effect identity/fingerprinting;
loader-facing MCP discovery exposes safe reference/profile/package metadata only and does not install/start/connect/import anything;
capability grants remain independent from category loading and lifecycle authority;
deterministic stdio and non-stdio declaration fixtures cover inert loading, config references, missing config, allowed/unallowed profiles, malformed/injection inputs, legacy direct-process/direct-stdio rejection, authority separation, capability separation, and owned stdio lifecycle semantics.

The final PR #72 head 8e9ce1a053409f82deffe26f9f4c93e868725935 passed the complete deterministic matrix and, on unchanged-head rerun after one transient free-model core failure, the complete configured REAL OpenRouter matrix using openrouter/free and selected :free models.

Acceptance criteria

MCP servers are represented as ordinary queryable Prolog facts/data.
Add the repository-convention equivalent of rlmInstallMcpServer(+Server) with structured outcome and canonical async variant.
Add the repository-convention equivalent of rlmRunMcpServer(+Server) with structured outcome/handle and canonical async variant.
Provide server discovery/inspection predicates.
Unknown server names return structured errors.
Missing/unsupported install recipes return structured outcomes rather than falling through to arbitrary execution.
Install and run recipes are validated against the final host-controlled allow-listed execution schema.
Secrets are referenced by configuration/environment name and are not stored directly in server facts, traces, discovery results, pending authority records, or structured policy errors.
Running a server returns enough structured metadata to stop/restart/inspect it later.
MCP tools discovered from a running/connected server can be registered into the external tool registry path.
Loading the mcp tool category exposes sanitized MCP management/discovery functionality without automatically starting configured servers.
Capability grants remain separate from server installation, startup, category loading, and tool discovery.
Final deterministic declarative fixtures cover at least one stdio MCP server and one non-stdio server definition.

Non-goals retained

No arbitrary shell command supplied by model output.
No auto-install/auto-run from rlmLoadTools(mcp) or rlmLoadAllTools.
No fixed third-party MCP catalog in core.
No general process_run implementation in this issue.

Boundary

server facts          = declarative host configuration
loader/category       = availability only
capabilities          = invocation permission
authority             = human mediation
installer/run policy  = hard execution confinement

These boundaries are intentionally independent.

## Goal Represent MCP servers as ordinary inspectable Prolog facts and provide explicit predicates to install and run each declared server through a hard host-controlled execution boundary. This belongs with the external tool/MCP integration layer, not as hard-coded concrete tool behavior in `prolog-rlm` core. ### Final state — 2026-08-17 PR #60 (`3db30d832f9b124d6b6e6aa3cbe2808914f4ae6e`) landed the initial declarative lifecycle. PR #66 (`9ba763d248f6e087b3ff01c651762997f02ee726`) landed the inert external `mcp` category/loading boundary. PR #72 finishes the remaining declaration-security and host execution-policy acceptance: - inert multifile `mcp_server/2` facts plus sanitized discovery; - explicit install/run/stop/restart/connect lifecycle with canonical execute -> async Future -> sync-await direction; - first-class `env_ref/1` and `config_ref/1` configuration references; - resolved secret values are fetched only inside the exact authority-permitted trusted continuation and are excluded from discovery, pending authority records, runtime handles, traces, and structured policy errors; - closed host-controlled installer and stdio execution profiles; - declarations select a trusted profile/package/version rather than arbitrary executable/argv or shell text; - structural package/version/profile/config/cwd validation occurs before authority and before process spawn; - `dangerous` skips interactive approval only and cannot bypass execution-profile allow-lists or hard validation; - exact normalized working directory participates in authority effect identity/fingerprinting; - loader-facing MCP discovery exposes safe reference/profile/package metadata only and does not install/start/connect/import anything; - capability grants remain independent from category loading and lifecycle authority; - deterministic stdio and non-stdio declaration fixtures cover inert loading, config references, missing config, allowed/unallowed profiles, malformed/injection inputs, legacy direct-process/direct-stdio rejection, authority separation, capability separation, and owned stdio lifecycle semantics. The final PR #72 head `8e9ce1a053409f82deffe26f9f4c93e868725935` passed the complete deterministic matrix and, on unchanged-head rerun after one transient free-model core failure, the complete configured REAL OpenRouter matrix using `openrouter/free` and selected `:free` models. ## Acceptance criteria - [x] MCP servers are represented as ordinary queryable Prolog facts/data. - [x] Add the repository-convention equivalent of `rlmInstallMcpServer(+Server)` with structured outcome and canonical async variant. - [x] Add the repository-convention equivalent of `rlmRunMcpServer(+Server)` with structured outcome/handle and canonical async variant. - [x] Provide server discovery/inspection predicates. - [x] Unknown server names return structured errors. - [x] Missing/unsupported install recipes return structured outcomes rather than falling through to arbitrary execution. - [x] Install and run recipes are validated against the final host-controlled allow-listed execution schema. - [x] Secrets are referenced by configuration/environment name and are not stored directly in server facts, traces, discovery results, pending authority records, or structured policy errors. - [x] Running a server returns enough structured metadata to stop/restart/inspect it later. - [x] MCP tools discovered from a running/connected server can be registered into the external tool registry path. - [x] Loading the `mcp` tool category exposes sanitized MCP management/discovery functionality without automatically starting configured servers. - [x] Capability grants remain separate from server installation, startup, category loading, and tool discovery. - [x] Final deterministic declarative fixtures cover at least one stdio MCP server and one non-stdio server definition. ## Non-goals retained - No arbitrary shell command supplied by model output. - No auto-install/auto-run from `rlmLoadTools(mcp)` or `rlmLoadAllTools`. - No fixed third-party MCP catalog in core. - No general `process_run` implementation in this issue. ## Boundary ```text server facts = declarative host configuration loader/category = availability only capabilities = invocation permission authority = human mediation installer/run policy = hard execution confinement ``` These boundaries are intentionally independent.

lost-rob0t commented

2026-08-16 23:54:07 +00:00

(Migrated from github.com)

PR #62 authority-hardening update relevant to MCP:

imported MCP tools continue through the canonical normal tool contract;
owned MCP lifecycle mutations (install / process-backed start / stop / restart) use the shared rlm_authority infrastructure rather than an MCP-specific approval subsystem;
borrowed protocol connections remain distinct from owned lifecycle mutation;
canonical option-bearing sync/async lifecycle facades were restored after the hardening audit caught their accidental removal;
allow_once lifecycle execution now records completion/replay instead of leaving the exact fingerprint stuck in_progress;
lifecycle connect now returns the canonical mcp_client_connect_execute/5 outcome directly rather than expecting a nonexistent wrapper result.

#52 should remain open: final host allowlisted install/run schema, external secret/config references, companion MCP tool-pack/category work, and the remaining lifecycle acceptance surface are not all complete. Also see #63/#64, which block declaring the authority core final.

PR #62 authority-hardening update relevant to MCP: - imported MCP tools continue through the canonical normal tool contract; - owned MCP lifecycle mutations (install / process-backed start / stop / restart) use the shared `rlm_authority` infrastructure rather than an MCP-specific approval subsystem; - borrowed protocol connections remain distinct from owned lifecycle mutation; - canonical option-bearing sync/async lifecycle facades were restored after the hardening audit caught their accidental removal; - `allow_once` lifecycle execution now records completion/replay instead of leaving the exact fingerprint stuck `in_progress`; - lifecycle connect now returns the canonical `mcp_client_connect_execute/5` outcome directly rather than expecting a nonexistent wrapper result. #52 should remain open: final host allowlisted install/run schema, external secret/config references, companion MCP tool-pack/category work, and the remaining lifecycle acceptance surface are not all complete. Also see #63/#64, which block declaring the authority core final.

lost-rob0t commented

2026-08-17 05:55:26 +00:00

(Migrated from github.com)

Authority/lifecycle integration dependency is now complete on canonical main via PR #62, squash merge 88b64db41a4c474379a1808b95c761ffc78d9fc2. MCP install/run/stop/restart/connect remain mediated by the shared host authority model and preserve canonical execute -> async Future -> sync-await direction.

Keeping #52 open intentionally. Remaining acceptance is still the production-facing declarative layer: final host-controlled installer/package-manager allow-list, first-class config/environment secret references, concrete external mcp loader/category integration without auto-starting servers, and any final stdio/non-stdio declarative fixture/metadata gaps. Do not duplicate those as a new issue unless live implementation shows a genuinely separate defect.

Authority/lifecycle integration dependency is now complete on canonical `main` via PR #62, squash merge `88b64db41a4c474379a1808b95c761ffc78d9fc2`. MCP install/run/stop/restart/connect remain mediated by the shared host authority model and preserve canonical execute -> async Future -> sync-await direction. Keeping #52 open intentionally. Remaining acceptance is still the production-facing declarative layer: final host-controlled installer/package-manager allow-list, first-class config/environment secret references, concrete external `mcp` loader/category integration without auto-starting servers, and any final stdio/non-stdio declarative fixture/metadata gaps. Do not duplicate those as a new issue unless live implementation shows a genuinely separate defect.

lost-rob0t commented

2026-08-17 06:52:24 +00:00

(Migrated from github.com)

PR #66 implements the loader-facing MCP category portion of #52 on top of the existing #60/#62 lifecycle architecture.

Current PR behavior:

advertises an external mcp category through the hardened loader manifest contract;
loading mcp registers only inert read-only MCP server discovery/inspection tool schemas;
loader-facing MCP inspection sanitizes trusted fixture handlers/existing transport handles rather than exposing callables;
category loading never installs, runs, connects to, restarts, stops, or imports tools from any declared server;
category loading grants no capabilities and changes no authority;
explicit remote tool import remains in rlm_mcp_tool after an explicit host connection;
deterministic tests use an inert declaration whose transport handler throws if it is ever touched by the loader.

#52 should remain OPEN after #66 unless further work lands in this PR. The live implementation still permits the older restricted direct process(Executable, Args, Options) installer representation and raw env(Env) install options. Finishing the host-controlled installer/package-manager allow-list plus first-class environment/config secret references is a distinct execution/declaration policy slice and should not be smuggled into the loader PR. The final declarative stdio fixture/metadata acceptance should be reconciled with that slice as well.

PR #66 implements the loader-facing MCP category portion of #52 on top of the existing #60/#62 lifecycle architecture. Current PR behavior: - advertises an external `mcp` category through the hardened loader manifest contract; - loading `mcp` registers only inert read-only MCP server discovery/inspection tool schemas; - loader-facing MCP inspection sanitizes trusted fixture handlers/existing transport handles rather than exposing callables; - category loading never installs, runs, connects to, restarts, stops, or imports tools from any declared server; - category loading grants no capabilities and changes no authority; - explicit remote tool import remains in `rlm_mcp_tool` after an explicit host connection; - deterministic tests use an inert declaration whose transport handler throws if it is ever touched by the loader. #52 should remain OPEN after #66 unless further work lands in this PR. The live implementation still permits the older restricted direct `process(Executable, Args, Options)` installer representation and raw `env(Env)` install options. Finishing the host-controlled installer/package-manager allow-list plus first-class environment/config secret references is a distinct execution/declaration policy slice and should not be smuggled into the loader PR. The final declarative stdio fixture/metadata acceptance should be reconciled with that slice as well.

lost-rob0t commented

2026-08-17 06:55:16 +00:00

(Migrated from github.com)

Security-audit follow-up on PR #66: loader-facing MCP inspection now deliberately redacts legacy execution-detail values that may contain inline secrets before #52's first-class config-reference work lands. It reports execution shape (transport/install kind, executable where applicable, argv count, whether cwd/env are present) but not stdio/install argv contents, raw HTTP endpoint text, cwd value, raw env values, or runtime options. A dedicated regression fixture embeds sentinel secret values in stdio argv, installer argv, cwd and env and proves none appear in model-facing mcp_servers output.

This reduces exposure at the new loader boundary, but does NOT satisfy the remaining #52 declaration/lifecycle acceptance: raw secret-bearing declarations are still representable internally today, and the installer/package-manager allow-list is still unfinished. #52 remains intentionally open.

Security-audit follow-up on PR #66: loader-facing MCP inspection now deliberately redacts legacy execution-detail values that may contain inline secrets before #52's first-class config-reference work lands. It reports execution shape (transport/install kind, executable where applicable, argv count, whether cwd/env are present) but not stdio/install argv contents, raw HTTP endpoint text, cwd value, raw env values, or runtime options. A dedicated regression fixture embeds sentinel secret values in stdio argv, installer argv, cwd and env and proves none appear in model-facing `mcp_servers` output. This reduces exposure at the new loader boundary, but does NOT satisfy the remaining #52 declaration/lifecycle acceptance: raw secret-bearing declarations are still representable internally today, and the installer/package-manager allow-list is still unfinished. #52 remains intentionally open.

lost-rob0t commented

2026-08-17 06:58:14 +00:00

(Migrated from github.com)

PR #66 is now merged on canonical main as 9ba763d248f6e087b3ff01c651762997f02ee726.

The external mcp loader category acceptance is landed and validated: category load exposes only inert, capability-gated, sanitized management/discovery schemas; it does not install/start/connect/import servers/tools, grant capabilities, or change authority. The merged security regression also redacts legacy inline-sensitive declaration values from loader-facing inspection.

Keeping #52 OPEN. Remaining acceptance is now sharply limited to:

replace/augment the current raw secret-bearing declaration representation with first-class configuration/environment references so secrets are not embedded in server facts or traces;
replace/finalize the current restricted generic process installer representation with a host-controlled installer/package-manager allow-listed schema and prove unallowlisted recipes fail before process spawn;
reconcile the final declarative stdio + non-stdio fixture gate against that new declaration schema.

Do not reopen the loader/category work in the next slice unless a regression is found.

PR #66 is now merged on canonical main as `9ba763d248f6e087b3ff01c651762997f02ee726`. The external `mcp` loader category acceptance is landed and validated: category load exposes only inert, capability-gated, sanitized management/discovery schemas; it does not install/start/connect/import servers/tools, grant capabilities, or change authority. The merged security regression also redacts legacy inline-sensitive declaration values from loader-facing inspection. Keeping #52 OPEN. Remaining acceptance is now sharply limited to: 1. replace/augment the current raw secret-bearing declaration representation with first-class configuration/environment references so secrets are not embedded in server facts or traces; 2. replace/finalize the current restricted generic process installer representation with a host-controlled installer/package-manager allow-listed schema and prove unallowlisted recipes fail before process spawn; 3. reconcile the final declarative stdio + non-stdio fixture gate against that new declaration schema. Do not reopen the loader/category work in the next slice unless a regression is found.

lost-rob0t commented

2026-08-17 07:22:04 +00:00

(Migrated from github.com)

PR #72 is the active implementation slice for the remaining #52 acceptance from canonical main 9ba763d248f6e087b3ff01c651762997f02ee726.

Current draft architecture:

server facts use first-class env_ref/1 / config_ref/1 references rather than raw environment values;
installer declarations are closed package(Profile, Package, Version) recipes selected against trusted host profiles;
stdio run declarations likewise select trusted profiles instead of declaring arbitrary executable/argv;
package/version/config/cwd/profile preflight occurs before authority and before spawn;
resolved config values are fetched only inside the exact authority-permitted trusted continuation;
dangerous still cannot bypass profile allow-list or structural validation;
loader-facing MCP inspection exposes safe reference/profile/package metadata only and remains inert;
deterministic fixtures cover stdio + non-stdio declarations, missing config, unallowlisted profiles, injection attempts, legacy direct process/stdio rejection, capability separation, authority separation, and owned stdio lifecycle.

The stdio arbitrary-command weakness is being handled here rather than as a duplicate issue because #52's existing acceptance already calls for the final host-allowlisted install/run schema.

PR remains draft. Exact-head deterministic and REAL OpenRouter validation are still required before #52 can close.

PR #72 is the active implementation slice for the remaining #52 acceptance from canonical main `9ba763d248f6e087b3ff01c651762997f02ee726`. Current draft architecture: - server facts use first-class `env_ref/1` / `config_ref/1` references rather than raw environment values; - installer declarations are closed `package(Profile, Package, Version)` recipes selected against trusted host profiles; - stdio run declarations likewise select trusted profiles instead of declaring arbitrary executable/argv; - package/version/config/cwd/profile preflight occurs before authority and before spawn; - resolved config values are fetched only inside the exact authority-permitted trusted continuation; - `dangerous` still cannot bypass profile allow-list or structural validation; - loader-facing MCP inspection exposes safe reference/profile/package metadata only and remains inert; - deterministic fixtures cover stdio + non-stdio declarations, missing config, unallowlisted profiles, injection attempts, legacy direct process/stdio rejection, capability separation, authority separation, and owned stdio lifecycle. The stdio arbitrary-command weakness is being handled here rather than as a duplicate issue because #52's existing acceptance already calls for the final host-allowlisted install/run schema. PR remains draft. Exact-head deterministic and REAL OpenRouter validation are still required before #52 can close.

lost-rob0t commented

2026-08-17 07:31:22 +00:00

(Migrated from github.com)

Second-pass security audit on PR #72 found and fixed two additional #52-scoped acceptance gaps before calling the head final:

malformed declaration/profile payloads could be echoed inside structured policy errors, which meant a raw value mistakenly placed where a config reference belongs could leak through mcp_server_inspect; policy errors now retain safe structural reason/reference metadata but redact malformed raw payloads, with deterministic assertions for raw config/package/version data;
install/run/restart authority fingerprints did not include the exact declared working directory even though cwd is execution-relevant; normalized working_directory is now part of the authority operation args so approval/session fingerprints track that exact effect.

Issue search found no separate existing defect better scoped than #52 for either item, so they are fixed coherently in #72 rather than duplicated. Exact-head validation is rerunning after these audit fixes.

Second-pass security audit on PR #72 found and fixed two additional #52-scoped acceptance gaps before calling the head final: 1. malformed declaration/profile payloads could be echoed inside structured policy errors, which meant a raw value mistakenly placed where a config reference belongs could leak through `mcp_server_inspect`; policy errors now retain safe structural reason/reference metadata but redact malformed raw payloads, with deterministic assertions for raw config/package/version data; 2. install/run/restart authority fingerprints did not include the exact declared working directory even though cwd is execution-relevant; normalized `working_directory` is now part of the authority operation args so approval/session fingerprints track that exact effect. Issue search found no separate existing defect better scoped than #52 for either item, so they are fixed coherently in #72 rather than duplicated. Exact-head validation is rerunning after these audit fixes.