nsaspy/prolog-rlm

Fork 0

[EPIC] Add durable scoped state and Prolog-native project policy configuration #74

New issue

Open

opened 2026-08-17 07:39:38 +00:00 by lost-rob0t · 0 comments

lost-rob0t commented

2026-08-17 07:39:38 +00:00

(Migrated from github.com)

Goal

Add a reusable core substrate for durable scoped operator state and trusted policy/configuration so downstream agents and tool packs can build project-local Prolog configuration and remembered permissions without turning prolog-rlm into a coding-agent-specific configuration product.

The motivating user experience is semantically equivalent to:

Allow once
Allow for session
Allow for this project persistently
Deny

The persistent project choice must survive restart, remain isolated to the project, and authorize only the deliberately matched operation or bounded operation class. It must not become machine-global authority and must never silently become dangerous.

Analysis basis when this epic was created: main at 7b527750a64d5cbd9fff79413a399a8c69e90c75 (Harden declarative MCP execution policy (#72)).

Problem / motivation

The repository now has several intentionally different kinds of state, but no generic substrate for durable trusted operator settings/policy:

#53 / rlm_authority owns mutable runtime authority state, exact pending-operation fingerprints, approval lifecycle, and the four authority tiers. allow_session is intentionally non-persistent.
#16 / rlm_artifact owns immutable/versioned durable reasoning artifacts, blackboards, summaries, findings, provenance, and fresh-root handoff.
#52 owns declarative MCP server/configuration facts plus hard host-controlled execution profiles. Server presence/configuration does not grant capability or authority.
#68-#71 resolve and compile epistemic repository/research/design context and verified workflows.
#48/#49/#50 establish the extension lesson that core owns contracts while external libraries own concrete catalogs and downstream experiences.

None of those means:

“The trusted operator selected this setting or bounded permission for this project; remember it across restarts.”

Trying to stuff that meaning into artifact records, MCP facts, context facts, authority event history, or a generic fact/7 table would erase security/provenance distinctions that the current architecture deliberately preserves.

Architectural boundary

prolog-rlm core should own:

scoped durable-state API
trusted policy/config API
scope + precedence semantics
project identity contract
schema validation
persistence backend boundary
provenance + revocation semantics
authority integration hooks
safe provider/registration ABI

Downstream hosts / agents / tool packs should own:

user-facing preference names
project-specific preferences
project instructions
agent-specific configuration
project permission UX wording
optional project-local Prolog policy/config pack convention
concrete coding-agent experience

This is analogous in role to project-local agent configuration systems, but it must be designed natively for Prolog and the existing prolog-rlm security model rather than copying another product's file format or directory convention.

Required state classes and scopes

At minimum establish explicit scopes equivalent to:

host
user
project
session

Exact public terms may follow repository conventions.

These scopes are not all interchangeable. Ordinary settings may use deterministic overlay/precedence semantics. Authority-affecting state remains subordinate to current hard runtime policy and the trusted host ceiling.

Project identity

Project-scoped durable state must use a stable, explicit project identity contract. Persistent authority must not be keyed directly by the process current-working-directory string.

The design must address, without necessarily overbuilding the first slice:

repository-root discovery;
filesystem relocation;
symlinks/canonical paths;
Git worktrees;
multiple clones;
non-Git projects;
stale identity records;
explicit identity/version metadata.

The implementation issues should decide what is identity versus current location metadata and define deterministic behavior for ambiguous or stale cases.

Persistent operator settings and permissions

The core must be able to represent durable operator-selected state independently from model-generated context.

For persistent authorization, support at least two deliberately different matching classes:

Exact normalized operation — all security-relevant executable fields match.
Explicit bounded operation profile/pattern — a closed, validated matcher declared through trusted policy, not arbitrary model/project code.

A changed executable payload must not inherit an exact grant merely because rendered text looks similar.

Current #53 fingerprints include the authority context. #76 must therefore define deliberate composition with #53 rather than persisting an ephemeral runtime-context hash and hoping it means the same thing after restart.

Persistent permission is not a fifth authority tier. dangerous remains explicit and separate.

Precedence and authority

Do not treat preference overlay and security authority as the same algorithm.

The security direction must preserve the invariant:

hard validation / confinement / runtime restrictions
        >
trusted host ceiling and policy
        >
eligible persisted scoped policy
        >
session / operation mediation

Exact terms may change after reconciling with #53, but persisted state can never overpower a stricter current host/runtime policy.

A persistent grant also never substitutes for:

tool existence/loading;
capability possession;
schema/argument validation;
path confinement;
process/network policy;
budgets/timeouts;
effect identity/accounting (#57 where applicable);
cancellation/linearization guarantees (#63);
lifecycle ownership.

Revocation is first-class.

Prolog-native project policy/config packs

Downstream project configuration should be able to use Prolog-native data/rules instead of forcing JSON/YAML merely because it is fashionable to make every configuration language worse than the host language.

However, Prolog-native does not mean arbitrary auto-consulted repository code.

#77 must evaluate a safe boundary such as:

schema-validated ground declarations;
restricted multifile declarations from already trusted modules;
explicit trusted module/provider registration;
host allow-listed policy providers;
another bounded mechanism consistent with the external-tool ABI.

Repository/model terms must never be passed to unrestricted call/1. Mere presence of a project file must not grant capabilities or widen authority.

The concrete project-local filename/directory convention should remain downstream/extensible unless core genuinely requires a tiny provider discovery ABI.

Provenance and auditability

Active persisted state must be inspectable. Records should carry the repository-convention equivalent of:

scope;
project identity when applicable;
setting/policy key or rule identity;
normalized value/rule;
schema/version;
origin/trust class;
trusted actor/host source where representable;
creation/update metadata where consistent with current conventions;
current/superseded/revoked state.

Follow #64's lesson: active executable policy and retained audit/history are separate concerns. Do not turn an append-only history stream into the live authority registry or retain executable callables merely for auditability.

Security invariants

All children must preserve these invariants:

Model output does not become arbitrary executable Prolog.
Model/agent output cannot grant itself durable permissions.
Project files do not automatically widen host authority because they exist.
Loading configuration does not imply capability granting.
Persistent permission does not bypass capability checks, schemas, confinement, process/network policy, budgets, effect identity/accounting, cancellation, or lifecycle ownership.
allow_session remains intentionally non-persistent.
dangerous remains an explicit separate authority mode.
Malformed, stale, ambiguous, or untrusted persistent state fails closed where security is affected.
Secrets are references to trusted external configuration, not ordinary committed project facts/traces/policy values; follow the separation hardened by #52/#72.
Arbitrary project/model terms are never fed to unrestricted call/1.
Trusted host/operator code owns persistent mutation. A model may request authorization but cannot turn its request into durable authority by itself.

Public API direction

The child issues should converge on host-facing APIs with semantics equivalent to:

open/configure scoped store
resolve/create canonical scope identity
read / enumerate / query scoped state
trusted set/update
revoke/delete
resolve effective ordinary setting
inspect provenance/current/stale/revoked state
reload/close
resolve eligible persisted authority policy
register/discover validated project policy providers

Exact predicate names and records must follow current prolog-rlm conventions after implementation design.

Immediate/pure state operations do not need decorative async twins. If a future backend is genuinely latency-bearing, follow #54:

canonical execute semantics
  -> async Future
  -> sync facade awaits the same Future

Interaction with existing modules/issues

#53 — authority

Reuse the existing authority boundary. Do not build authority_v2 or turn persistent project permission into allow_session persistence.

#16 — artifacts

Artifacts remain durable task knowledge/fresh-root handoff. Publishing an artifact must never be equivalent to changing security policy.

#52 / #72 — MCP declarations and hard execution policy

MCP server/profile/config-reference facts remain a different class of trusted host configuration. Presence of an MCP server does not grant install/run/use permission. The new substrate may eventually hold user/project choices that refer to MCP configuration, but it must not collapse the categories.

#68-#71 — context/workflow pipeline

Context resolution may inspect effective project policy/config as operator state. Repository observations, research findings, design authority, and operator policy must retain distinct provenance/authority classes.

#48/#49/#50 — external tools

Reuse the architectural lesson: core owns stable contracts/security; external libraries and downstream agents own concrete catalogs/UX.

#54 — async contract

Do not invent duplicate sync/async business logic. Only latency-bearing backends require async surfaces.

#57 — effect identity

Persistent authorization matching must compose with canonical effect identity/accounting when applicable; it must not create a retry path that executes a side effect twice.

Child issues and dependency graph

#75 — [P1] Add backend-neutral durable scoped state and policy APIs
#76 — [P1] Add persistent project-scoped authorization rules to the authority boundary
#77 — [P1] Define a safe Prolog-native project policy/config pack boundary

#74 epic
  |
  +--> #75 scoped durable-state substrate
  |       |
  |       +--> #77 safe project policy/config provider boundary
  |
  +--> #76 persistent project authorization
          +--> #75 scoped state
          +--> #53 authority
          +--> #57 effect identity where applicable

#76 does not depend on #77: trusted UI/host-created project grants must work through #75 without requiring a project policy file/provider. Conversely, #77 may declare bounded policy profiles/configuration, but discovery/loading alone never creates a #76 grant.

Integration acceptance criteria

A host can persist an ordinary project setting, restart the Prolog process, resolve the same project identity, and recover the effective setting with provenance.
Project-scoped state for project A does not appear as effective state for project B.
A project-scoped persistent exact authorization created in project A does not authorize the same operation in project B.
Restart reloads an eligible persistent project authorization without changing the runtime authority mode to dangerous and without making allow_session persistent.
Changing a security-relevant executable payload invalidates an exact persistent grant unless an explicit trusted bounded policy rule matches the new operation.
A stricter current host/runtime policy can suppress or reject an otherwise stored project grant.
Revocation prevents future authorization after reload while retaining appropriate non-executable audit/provenance history.
Malformed/untrusted persisted state fails closed at security boundaries.
A downstream package can provide two projects with different Prolog-native settings/policy declarations through the safe provider boundary without arbitrary auto-executed repository code.
Loading/discovering project configuration alone grants no capability and widens no authority.
Artifact publication, MCP declaration, repository-context discovery, and persistent operator-policy mutation remain demonstrably distinct operations in tests/docs.

Non-goals

No coding-agent-specific .claude clone in core.
No UI/TUI or button wording in core.
No arbitrary repository consult/1 + call/1 configuration execution path.
No machine-global “allow forever” shortcut for a project request.
No persistence of allow_session.
No use of rlm_artifact as the permission database merely because it already persists ground terms.
No generic undifferentiated fact store for artifacts, context, MCP declarations, and operator authority.
No secrets stored directly in ordinary committed project configuration/policy state.

Dependencies / references

#16 durable artifacts
#48 external tool-library boundary
#49 companion tool library
#50 standard tool pack
#52 declarative MCP server/configuration facts
#53 host-controlled authority
#54 sync/async contract
#56 proof/evidence/verifier policy
#57 effect identity / once-only observations
#63 cancellation-linearizable approval handoff
#64 bounded terminal authority state
#68-#71 compiler/context/workflow pipeline
research/RLM-RESEARCH-003-typed-symbolic-execution.org
research/RLM-RESEARCH-005-swi-agent-runtime.org
research/RLM-RESEARCH-006-mcp-dual-version-runtime.org
research/RLM-RESEARCH-007-langchain-langgraph-port.org
research/RLM-RESEARCH-009-durable-artifact-context.org

RLM-RESEARCH-010 is referenced by #56/#57 and exists on draft PR #58, but was not present on main when this epic was designed; this epic does not invent or depend on unmerged contents.

## Goal Add a reusable core substrate for **durable scoped operator state and trusted policy/configuration** so downstream agents and tool packs can build project-local Prolog configuration and remembered permissions without turning `prolog-rlm` into a coding-agent-specific configuration product. The motivating user experience is semantically equivalent to: ```text Allow once Allow for session Allow for this project persistently Deny ``` The persistent project choice must survive restart, remain isolated to the project, and authorize only the deliberately matched operation or bounded operation class. It must **not** become machine-global authority and must never silently become `dangerous`. Analysis basis when this epic was created: `main` at `7b527750a64d5cbd9fff79413a399a8c69e90c75` (`Harden declarative MCP execution policy (#72)`). ## Problem / motivation The repository now has several intentionally different kinds of state, but no generic substrate for durable trusted operator settings/policy: - #53 / `rlm_authority` owns mutable runtime authority state, exact pending-operation fingerprints, approval lifecycle, and the four authority tiers. `allow_session` is intentionally non-persistent. - #16 / `rlm_artifact` owns immutable/versioned durable reasoning artifacts, blackboards, summaries, findings, provenance, and fresh-root handoff. - #52 owns declarative MCP server/configuration facts plus hard host-controlled execution profiles. Server presence/configuration does not grant capability or authority. - #68-#71 resolve and compile epistemic repository/research/design context and verified workflows. - #48/#49/#50 establish the extension lesson that core owns contracts while external libraries own concrete catalogs and downstream experiences. None of those means: > “The trusted operator selected this setting or bounded permission for this project; remember it across restarts.” Trying to stuff that meaning into artifact records, MCP facts, context facts, authority event history, or a generic `fact/7` table would erase security/provenance distinctions that the current architecture deliberately preserves. ## Architectural boundary `prolog-rlm` core should own: ```text scoped durable-state API trusted policy/config API scope + precedence semantics project identity contract schema validation persistence backend boundary provenance + revocation semantics authority integration hooks safe provider/registration ABI ``` Downstream hosts / agents / tool packs should own: ```text user-facing preference names project-specific preferences project instructions agent-specific configuration project permission UX wording optional project-local Prolog policy/config pack convention concrete coding-agent experience ``` This is analogous in *role* to project-local agent configuration systems, but it must be designed natively for Prolog and the existing `prolog-rlm` security model rather than copying another product's file format or directory convention. ## Required state classes and scopes At minimum establish explicit scopes equivalent to: - `host` - `user` - `project` - `session` Exact public terms may follow repository conventions. These scopes are not all interchangeable. Ordinary settings may use deterministic overlay/precedence semantics. Authority-affecting state remains subordinate to current hard runtime policy and the trusted host ceiling. ### Project identity Project-scoped durable state must use a stable, explicit project identity contract. Persistent authority must **not** be keyed directly by the process current-working-directory string. The design must address, without necessarily overbuilding the first slice: - repository-root discovery; - filesystem relocation; - symlinks/canonical paths; - Git worktrees; - multiple clones; - non-Git projects; - stale identity records; - explicit identity/version metadata. The implementation issues should decide what is identity versus current location metadata and define deterministic behavior for ambiguous or stale cases. ## Persistent operator settings and permissions The core must be able to represent durable operator-selected state independently from model-generated context. For persistent authorization, support at least two deliberately different matching classes: 1. **Exact normalized operation** — all security-relevant executable fields match. 2. **Explicit bounded operation profile/pattern** — a closed, validated matcher declared through trusted policy, not arbitrary model/project code. A changed executable payload must not inherit an exact grant merely because rendered text looks similar. Current #53 fingerprints include the authority context. #76 must therefore define deliberate composition with #53 rather than persisting an ephemeral runtime-context hash and hoping it means the same thing after restart. Persistent permission is not a fifth authority tier. `dangerous` remains explicit and separate. ## Precedence and authority Do not treat preference overlay and security authority as the same algorithm. The security direction must preserve the invariant: ```text hard validation / confinement / runtime restrictions > trusted host ceiling and policy > eligible persisted scoped policy > session / operation mediation ``` Exact terms may change after reconciling with #53, but persisted state can never overpower a stricter current host/runtime policy. A persistent grant also never substitutes for: - tool existence/loading; - capability possession; - schema/argument validation; - path confinement; - process/network policy; - budgets/timeouts; - effect identity/accounting (#57 where applicable); - cancellation/linearization guarantees (#63); - lifecycle ownership. Revocation is first-class. ## Prolog-native project policy/config packs Downstream project configuration should be able to use Prolog-native data/rules instead of forcing JSON/YAML merely because it is fashionable to make every configuration language worse than the host language. However, **Prolog-native does not mean arbitrary auto-consulted repository code**. #77 must evaluate a safe boundary such as: - schema-validated ground declarations; - restricted multifile declarations from already trusted modules; - explicit trusted module/provider registration; - host allow-listed policy providers; - another bounded mechanism consistent with the external-tool ABI. Repository/model terms must never be passed to unrestricted `call/1`. Mere presence of a project file must not grant capabilities or widen authority. The concrete project-local filename/directory convention should remain downstream/extensible unless core genuinely requires a tiny provider discovery ABI. ## Provenance and auditability Active persisted state must be inspectable. Records should carry the repository-convention equivalent of: - scope; - project identity when applicable; - setting/policy key or rule identity; - normalized value/rule; - schema/version; - origin/trust class; - trusted actor/host source where representable; - creation/update metadata where consistent with current conventions; - current/superseded/revoked state. Follow #64's lesson: **active executable policy and retained audit/history are separate concerns**. Do not turn an append-only history stream into the live authority registry or retain executable callables merely for auditability. ## Security invariants All children must preserve these invariants: - Model output does not become arbitrary executable Prolog. - Model/agent output cannot grant itself durable permissions. - Project files do not automatically widen host authority because they exist. - Loading configuration does not imply capability granting. - Persistent permission does not bypass capability checks, schemas, confinement, process/network policy, budgets, effect identity/accounting, cancellation, or lifecycle ownership. - `allow_session` remains intentionally non-persistent. - `dangerous` remains an explicit separate authority mode. - Malformed, stale, ambiguous, or untrusted persistent state fails closed where security is affected. - Secrets are references to trusted external configuration, not ordinary committed project facts/traces/policy values; follow the separation hardened by #52/#72. - Arbitrary project/model terms are never fed to unrestricted `call/1`. - Trusted host/operator code owns persistent mutation. A model may request authorization but cannot turn its request into durable authority by itself. ## Public API direction The child issues should converge on host-facing APIs with semantics equivalent to: ```text open/configure scoped store resolve/create canonical scope identity read / enumerate / query scoped state trusted set/update revoke/delete resolve effective ordinary setting inspect provenance/current/stale/revoked state reload/close resolve eligible persisted authority policy register/discover validated project policy providers ``` Exact predicate names and records must follow current `prolog-rlm` conventions after implementation design. Immediate/pure state operations do not need decorative async twins. If a future backend is genuinely latency-bearing, follow #54: ```text canonical execute semantics -> async Future -> sync facade awaits the same Future ``` ## Interaction with existing modules/issues ### #53 — authority Reuse the existing authority boundary. Do **not** build `authority_v2` or turn persistent project permission into `allow_session` persistence. ### #16 — artifacts Artifacts remain durable task knowledge/fresh-root handoff. Publishing an artifact must never be equivalent to changing security policy. ### #52 / #72 — MCP declarations and hard execution policy MCP server/profile/config-reference facts remain a different class of trusted host configuration. Presence of an MCP server does not grant install/run/use permission. The new substrate may eventually hold user/project choices that refer to MCP configuration, but it must not collapse the categories. ### #68-#71 — context/workflow pipeline Context resolution may inspect effective project policy/config as operator state. Repository observations, research findings, design authority, and operator policy must retain distinct provenance/authority classes. ### #48/#49/#50 — external tools Reuse the architectural lesson: core owns stable contracts/security; external libraries and downstream agents own concrete catalogs/UX. ### #54 — async contract Do not invent duplicate sync/async business logic. Only latency-bearing backends require async surfaces. ### #57 — effect identity Persistent authorization matching must compose with canonical effect identity/accounting when applicable; it must not create a retry path that executes a side effect twice. ## Child issues and dependency graph - [ ] #75 — **[P1] Add backend-neutral durable scoped state and policy APIs** - [ ] #76 — **[P1] Add persistent project-scoped authorization rules to the authority boundary** - [ ] #77 — **[P1] Define a safe Prolog-native project policy/config pack boundary** ```text #74 epic | +--> #75 scoped durable-state substrate | | | +--> #77 safe project policy/config provider boundary | +--> #76 persistent project authorization +--> #75 scoped state +--> #53 authority +--> #57 effect identity where applicable ``` #76 does **not** depend on #77: trusted UI/host-created project grants must work through #75 without requiring a project policy file/provider. Conversely, #77 may declare bounded policy profiles/configuration, but discovery/loading alone never creates a #76 grant. ## Integration acceptance criteria - [ ] A host can persist an ordinary project setting, restart the Prolog process, resolve the same project identity, and recover the effective setting with provenance. - [ ] Project-scoped state for project A does not appear as effective state for project B. - [ ] A project-scoped persistent exact authorization created in project A does not authorize the same operation in project B. - [ ] Restart reloads an eligible persistent project authorization without changing the runtime authority mode to `dangerous` and without making `allow_session` persistent. - [ ] Changing a security-relevant executable payload invalidates an exact persistent grant unless an explicit trusted bounded policy rule matches the new operation. - [ ] A stricter current host/runtime policy can suppress or reject an otherwise stored project grant. - [ ] Revocation prevents future authorization after reload while retaining appropriate non-executable audit/provenance history. - [ ] Malformed/untrusted persisted state fails closed at security boundaries. - [ ] A downstream package can provide two projects with different Prolog-native settings/policy declarations through the safe provider boundary without arbitrary auto-executed repository code. - [ ] Loading/discovering project configuration alone grants no capability and widens no authority. - [ ] Artifact publication, MCP declaration, repository-context discovery, and persistent operator-policy mutation remain demonstrably distinct operations in tests/docs. ## Non-goals - No coding-agent-specific `.claude` clone in core. - No UI/TUI or button wording in core. - No arbitrary repository `consult/1` + `call/1` configuration execution path. - No machine-global “allow forever” shortcut for a project request. - No persistence of `allow_session`. - No use of `rlm_artifact` as the permission database merely because it already persists ground terms. - No generic undifferentiated fact store for artifacts, context, MCP declarations, and operator authority. - No secrets stored directly in ordinary committed project configuration/policy state. ## Dependencies / references - #16 durable artifacts - #48 external tool-library boundary - #49 companion tool library - #50 standard tool pack - #52 declarative MCP server/configuration facts - #53 host-controlled authority - #54 sync/async contract - #56 proof/evidence/verifier policy - #57 effect identity / once-only observations - #63 cancellation-linearizable approval handoff - #64 bounded terminal authority state - #68-#71 compiler/context/workflow pipeline - `research/RLM-RESEARCH-003-typed-symbolic-execution.org` - `research/RLM-RESEARCH-005-swi-agent-runtime.org` - `research/RLM-RESEARCH-006-mcp-dual-version-runtime.org` - `research/RLM-RESEARCH-007-langchain-langgraph-port.org` - `research/RLM-RESEARCH-009-durable-artifact-context.org` `RLM-RESEARCH-010` is referenced by #56/#57 and exists on draft PR #58, but was not present on `main` when this epic was designed; this epic does not invent or depend on unmerged contents.