nsaspy/prolog-rlm

Fork 0

[EPIC][experts] Closed-loop integration: Requirement→Knowledge→Plan→Experts→VERIFY→Repair→Review→completion #411

New issue

Open

opened 2026-09-08 01:29:20 +00:00 by nsaspy · 5 comments

nsaspy commented

2026-09-08 01:29:20 +00:00

Owner

Parent: #376
Depends on: #377, #378, #379, #380, #381, #382, #383, #384, #385, #386
Integrates with: #288, #355, #68–#71, #56, #93

Goal

Wire the complete expert family into one canonical reusable closed-loop expert workflow and prove that the components compose under real runtime invariants rather than existing as isolated APIs.

This epic owns integration and conformance, not another scheduler or another specialist implementation.

Canonical flow

operator requirement / existing Frozen Spec
        ↓
Requirement / SPEC Expert (#378)
        ↓
Frozen Spec + fingerprint
        ↓
Project Knowledge Expert (#380)
        ↓
current coherent symbolic observations
        ↓
Planner / Coordinator Expert (#379)
        ↓
typed dependency graph + expert assignments
        ↓
Git / Retrieval / Write / other specialist execution
(#381 #382 #383)
        ↓
plan-native deterministic ops where required (#355)
        ↓
fresh post-effect observations
        ↓
VERIFY Expert (#384)
        ↓
┌──────────── satisfied ────────────────┐
│                                       ↓
│                              Review/Critic (#386)
│                                       ↓
│                         findings? -> Repair if needed
│                                       ↓
│                              final fresh VERIFY
│                                       ↓
│                          structured completion/evidence
│
└─ violated / unknown / blocked
        ↓
Repair / Diagnosis Expert (#385)
        ↓
repair/evidence subgoal
        ↓
Planner replan against SAME Frozen Spec
        └─────────────────────────────────→ loop

Zero-model reference path

The primary conformance fixture MUST complete a non-trivial two-pass repair loop with model providers disabled and model_calls = 0.

Experts are ordinary local computation and are not charged model usage merely for invocation or nesting.

Required deterministic fixture

Build a fixture representing a small project and requirement where:

a structured requirement is validated/frozen;
current project knowledge shows the initial requirement is not satisfied;
Planner selects a deterministic Write Expert strategy;
first change is intentionally incomplete but validly applied;
source/project state is refreshed;
VERIFY reports a concrete remaining violation;
Repair Expert diagnoses the missing part;
Planner constructs a bounded repair subplan;
Write Expert applies a second expected-preimage change;
state refreshes again;
VERIFY succeeds on the exact original Frozen Spec;
Review finds no unresolved blocking finding;
final VERIFY/acceptance emits structured proof/evidence;
total model/provider calls remain zero.

This fixture must exercise expert→expert lineage, not directly call every specialist from the test harness in an artificial order.

Alternate fallback fixture

Add a second fixture where deterministic Write reasoning deliberately reaches an unsupported generative case and explicit model fallback is enabled with a deterministic fake provider.

Prove:

expert invocations: free / no synthetic model charge
fallback model call: exactly the provider usage reported
candidate fallback edit: validated before effect
rest of loop: same canonical path

State and identity

Preserve stable identities across the loop:

run/session;
Frozen Spec ref/fingerprint;
project snapshot/source generations;
plan/replan generations;
expert invocation parent/child lineage;
tool/effect attempt identity;
verification runs/evidence refs;
review findings;
repair attempts.

A final result should be replayable/explainable from structured records without reconstructing truth from conversation prose.

Progress / termination

Define one bounded loop policy using existing runtime machinery:

recursion/depth bound;
aggregate expert work/inference bound;
tool/effect bound;
wall time;
repeated-state/no-progress fingerprint;
max repair attempts/strategy exhaustion;
cancellation.

Do not invent an independent scheduler. The existing graph/workflow runtime remains authoritative.

Failure semantics

Integration tests must prove:

unknown evidence remains unknown;
blocked authority remains blocked and is not retried as code repair;
stale source snapshot forces refresh before verification;
stale write preimage forces re-resolution/replan;
partial write/effect result cannot be treated as complete;
repeated equivalent failed repair stops;
cancelled loop dispatches nothing further;
Review finding must be resolved/accepted by configured policy before final completion;
final success is emitted only by evidence-backed VERIFY against the unchanged Frozen Spec.

Public workflow surface

Expose a reusable host-facing call equivalent to:

expert_workflow_run(+RequirementOrFrozenSpec, +Context, +Options, -Outcome).

with async/cancellation variant as appropriate using existing runtime conventions.

Outcome should include bounded references/summaries for:

Frozen Spec
final status
final evidence map
plan/replan generations
expert invocation lineage
repair count
review findings
local work counters
actual provider usage only when fallback occurred
tool/effect usage
stop reason

Exact API naming follows repo conventions.

Acceptance

Full zero-model repair loop fixture passes with provider disabled and model_calls = 0.
Expert nesting produces no synthetic model/token/cost charge.
Explicit fallback fixture charges exactly its fake-provider usage and nothing for pure expert calls.
Exact Frozen Spec fingerprint never changes across replan/repair.
D6-11 operations execute only via plan-native path.
edit/create route through Write Expert.
Post-write freshness is observed before VERIFY.
VERIFY alone owns requirement acceptance.
Review findings participate in the loop without becoming a second verifier.
No-progress and cancellation terminate deterministically.
Same workflow can be called by a headless host without AgentProlog product code.
Trace/explanation can answer why each expert/step was chosen and what evidence led to completion.
Full deterministic repository gate remains green.

Non-goals

No product UI.
No second scheduler.
No concrete AgentProlog tool implementation.
No requirement weakening to obtain green.

Refs #376 #377 #378 #379 #380 #381 #382 #383 #384 #385 #386 #288 #355 #68 #69 #70 #71 #56 #93

Parent: #376 Depends on: #377, #378, #379, #380, #381, #382, #383, #384, #385, #386 Integrates with: #288, #355, #68–#71, #56, #93 ## Goal Wire the complete expert family into one canonical reusable **closed-loop expert workflow** and prove that the components compose under real runtime invariants rather than existing as isolated APIs. This epic owns integration and conformance, not another scheduler or another specialist implementation. ## Canonical flow ```text operator requirement / existing Frozen Spec ↓ Requirement / SPEC Expert (#378) ↓ Frozen Spec + fingerprint ↓ Project Knowledge Expert (#380) ↓ current coherent symbolic observations ↓ Planner / Coordinator Expert (#379) ↓ typed dependency graph + expert assignments ↓ Git / Retrieval / Write / other specialist execution (#381 #382 #383) ↓ plan-native deterministic ops where required (#355) ↓ fresh post-effect observations ↓ VERIFY Expert (#384) ↓ ┌──────────── satisfied ────────────────┐ │ ↓ │ Review/Critic (#386) │ ↓ │ findings? -> Repair if needed │ ↓ │ final fresh VERIFY │ ↓ │ structured completion/evidence │ └─ violated / unknown / blocked ↓ Repair / Diagnosis Expert (#385) ↓ repair/evidence subgoal ↓ Planner replan against SAME Frozen Spec └─────────────────────────────────→ loop ``` ## Zero-model reference path The primary conformance fixture MUST complete a non-trivial two-pass repair loop with **model providers disabled** and `model_calls = 0`. Experts are ordinary local computation and are not charged model usage merely for invocation or nesting. ## Required deterministic fixture Build a fixture representing a small project and requirement where: 1. a structured requirement is validated/frozen; 2. current project knowledge shows the initial requirement is not satisfied; 3. Planner selects a deterministic Write Expert strategy; 4. first change is intentionally incomplete but validly applied; 5. source/project state is refreshed; 6. VERIFY reports a concrete remaining violation; 7. Repair Expert diagnoses the missing part; 8. Planner constructs a bounded repair subplan; 9. Write Expert applies a second expected-preimage change; 10. state refreshes again; 11. VERIFY succeeds on the exact original Frozen Spec; 12. Review finds no unresolved blocking finding; 13. final VERIFY/acceptance emits structured proof/evidence; 14. total model/provider calls remain zero. This fixture must exercise expert→expert lineage, not directly call every specialist from the test harness in an artificial order. ## Alternate fallback fixture Add a second fixture where deterministic Write reasoning deliberately reaches an unsupported generative case and explicit model fallback is enabled with a deterministic fake provider. Prove: ```text expert invocations: free / no synthetic model charge fallback model call: exactly the provider usage reported candidate fallback edit: validated before effect rest of loop: same canonical path ``` ## State and identity Preserve stable identities across the loop: - run/session; - Frozen Spec ref/fingerprint; - project snapshot/source generations; - plan/replan generations; - expert invocation parent/child lineage; - tool/effect attempt identity; - verification runs/evidence refs; - review findings; - repair attempts. A final result should be replayable/explainable from structured records without reconstructing truth from conversation prose. ## Progress / termination Define one bounded loop policy using existing runtime machinery: - recursion/depth bound; - aggregate expert work/inference bound; - tool/effect bound; - wall time; - repeated-state/no-progress fingerprint; - max repair attempts/strategy exhaustion; - cancellation. Do not invent an independent scheduler. The existing graph/workflow runtime remains authoritative. ## Failure semantics Integration tests must prove: - unknown evidence remains unknown; - blocked authority remains blocked and is not retried as code repair; - stale source snapshot forces refresh before verification; - stale write preimage forces re-resolution/replan; - partial write/effect result cannot be treated as complete; - repeated equivalent failed repair stops; - cancelled loop dispatches nothing further; - Review finding must be resolved/accepted by configured policy before final completion; - final success is emitted only by evidence-backed VERIFY against the unchanged Frozen Spec. ## Public workflow surface Expose a reusable host-facing call equivalent to: ```prolog expert_workflow_run(+RequirementOrFrozenSpec, +Context, +Options, -Outcome). ``` with async/cancellation variant as appropriate using existing runtime conventions. Outcome should include bounded references/summaries for: ```text Frozen Spec final status final evidence map plan/replan generations expert invocation lineage repair count review findings local work counters actual provider usage only when fallback occurred tool/effect usage stop reason ``` Exact API naming follows repo conventions. ## Acceptance - [ ] Full zero-model repair loop fixture passes with provider disabled and `model_calls = 0`. - [ ] Expert nesting produces no synthetic model/token/cost charge. - [ ] Explicit fallback fixture charges exactly its fake-provider usage and nothing for pure expert calls. - [ ] Exact Frozen Spec fingerprint never changes across replan/repair. - [ ] D6-11 operations execute only via plan-native path. - [ ] `edit/create` route through Write Expert. - [ ] Post-write freshness is observed before VERIFY. - [ ] VERIFY alone owns requirement acceptance. - [ ] Review findings participate in the loop without becoming a second verifier. - [ ] No-progress and cancellation terminate deterministically. - [ ] Same workflow can be called by a headless host without AgentProlog product code. - [ ] Trace/explanation can answer why each expert/step was chosen and what evidence led to completion. - [ ] Full deterministic repository gate remains green. ## Non-goals - No product UI. - No second scheduler. - No concrete AgentProlog tool implementation. - No requirement weakening to obtain green. Refs #376 #377 #378 #379 #380 #381 #382 #383 #384 #385 #386 #288 #355 #68 #69 #70 #71 #56 #93

nsaspy commented

2026-09-08 01:46:54 +00:00

Author

Owner

Machine Spirit #401D integration/conformance handoff

Depth 4 is design-complete with DPEC — Demand-Driven Procedural Execution & Conformance Graph. Add these integration requirements when #387 is implemented:

Heterogeneous zero-model fixture: at least two procedural/formalism adapter paths + multiple Prolog experts + one observation branch + one monitor/repair path -> final fresh VERIFY with providers disabled and model_calls = 0.
One-scheduler instrumentation: every executable task dispatch must pass through #288/rlm_plan; no adapter/native engine may start an ambient second scheduler.
Reference/differential adapter conformance: executable HTN/HDDL, temporal-PDDL, BT, workflow/BPMN and Golog/ConGolog subsets carry profile/version + comparison relation + preserved properties/loss receipt.
Replay equivalence: for identical canonical inputs/retained nondeterministic observations:

cold == warm-cache == cache-rebuilt
incremental == full demanded recomputation
recorded trace replay == original normalized semantic outcome

Concurrent determinism: physical interleavings of independent ready steps may differ, but normalized semantic results/receipts may not.
Cancellation race semantics: cancellation after external-effect dispatch preserves observed/indeterminate effect state; it cannot fabricate rollback or ordinary failure.
Resource exhaustion: procedure/decomposition/diagnosis/conformance ceilings return structured resource_limited; they never trigger implicit LLM fallback.

The current rlm_plan_graph.pl already provides the key architectural anchor: inert validated graph data, rlm_plan as sole step executor, explicit cancellation, aggregate budget, and no alternate effect path. Preserve that boundary.

## Machine Spirit #401D integration/conformance handoff Depth 4 is design-complete with **DPEC — Demand-Driven Procedural Execution & Conformance Graph**. Add these integration requirements when #387 is implemented: 1. **Heterogeneous zero-model fixture:** at least two procedural/formalism adapter paths + multiple Prolog experts + one observation branch + one monitor/repair path -> final fresh VERIFY with providers disabled and `model_calls = 0`. 2. **One-scheduler instrumentation:** every executable task dispatch must pass through #288/`rlm_plan`; no adapter/native engine may start an ambient second scheduler. 3. **Reference/differential adapter conformance:** executable HTN/HDDL, temporal-PDDL, BT, workflow/BPMN and Golog/ConGolog subsets carry profile/version + comparison relation + preserved properties/loss receipt. 4. **Replay equivalence:** for identical canonical inputs/retained nondeterministic observations: ```text cold == warm-cache == cache-rebuilt incremental == full demanded recomputation recorded trace replay == original normalized semantic outcome ``` 5. **Concurrent determinism:** physical interleavings of independent ready steps may differ, but normalized semantic results/receipts may not. 6. **Cancellation race semantics:** cancellation after external-effect dispatch preserves observed/indeterminate effect state; it cannot fabricate rollback or ordinary failure. 7. **Resource exhaustion:** procedure/decomposition/diagnosis/conformance ceilings return structured `resource_limited`; they never trigger implicit LLM fallback. The current `rlm_plan_graph.pl` already provides the key architectural anchor: inert validated graph data, `rlm_plan` as sole step executor, explicit cancellation, aggregate budget, and no alternate effect path. Preserve that boundary.

nsaspy commented

2026-09-08 01:46:54 +00:00

Author

Owner

Machine Spirit #405A closed-loop handoff

Depth 8A keeps #387 as the canonical executable expert-loop gate inside COVENANT-MS. The synthesis adds no second loop; it adds exact cross-layer run/receipt identity around the existing one.

The north-star run must bind the same Frozen Spec, evidence/world cut, procedure projection, expert-registry generation, capability/authority policy and verifier generation from planning through final VERIFY. Fresh post-effect observation is mandatory; predicted effects and worker/model self-report remain inadmissible substitutes.

#387's provider-disabled two-pass repair fixture is therefore a required component of the final known-symbolic Machine Spirit demonstration, but it is not sufficient alone: #405D must also connect semantic ingestion/durable world model/epistemics/bounded attention/learning/replay.

Refs #397 #405 #376 #384 and symbolic-memory#45.

## Machine Spirit #405A closed-loop handoff Depth 8A keeps #387 as the canonical executable expert-loop gate inside **COVENANT-MS**. The synthesis adds no second loop; it adds exact cross-layer run/receipt identity around the existing one. The north-star run must bind the same Frozen Spec, evidence/world cut, procedure projection, expert-registry generation, capability/authority policy and verifier generation from planning through final VERIFY. Fresh post-effect observation is mandatory; predicted effects and worker/model self-report remain inadmissible substitutes. #387's provider-disabled two-pass repair fixture is therefore a required component of the final known-symbolic Machine Spirit demonstration, but it is not sufficient alone: #405D must also connect semantic ingestion/durable world model/epistemics/bounded attention/learning/replay. Refs #397 #405 #376 #384 and symbolic-memory#45.

nsaspy commented

2026-09-08 01:46:54 +00:00

Author

Owner

#405B AEGIS-MS integration handoff: add cross-component race/composition fixtures to the closed loop. Required cases include authority revoked after plan but before dispatch; semantic/host target binding or preimage changes before effect; cancellation racing dispatch; two concurrent effects with similar result payloads where observations must remain attempt/object-bound; and a stack of locally valid receipts that omits one mandatory influence dependency and therefore fails end-to-end capsule validation. Preserve #288 as the sole scheduler and #384 as acceptance boundary. The effect path should use a current dispatch-time seal over principal/action/stable host object/policy/preimage rather than treating plan-time authority as durable.

nsaspy commented

2026-09-08 01:46:54 +00:00

Author

Owner

#405C CONCORDAT-MS expert-loop handoff

Expert cooperation stays target-local across federation.

Foreign procedure/expert metadata may establish semantic compatibility or provide evidence, but it cannot import a handler, principal, capability or authority. Required path remains:

foreign semantic/procedure/expert artifact
 -> directional federation/PSC contract
 -> target-local appraisal
 -> local expert/handler binding
 -> local capability + authority
 -> effect-time AEGIS seal

Likewise, a foreign VERIFY/pass receipt is evidence about that verifier/profile/version, not local #384 satisfaction. Domain-qualified control/influence labels must be explicitly mapped by target-owned policy before becoming control-eligible.

Canonical research: #405C.

## #405C CONCORDAT-MS expert-loop handoff Expert cooperation stays **target-local** across federation. Foreign procedure/expert metadata may establish semantic compatibility or provide evidence, but it cannot import a handler, principal, capability or authority. Required path remains: ```text foreign semantic/procedure/expert artifact -> directional federation/PSC contract -> target-local appraisal -> local expert/handler binding -> local capability + authority -> effect-time AEGIS seal ``` Likewise, a foreign VERIFY/pass receipt is evidence about that verifier/profile/version, not local #384 satisfaction. Domain-qualified control/influence labels must be explicitly mapped by target-owned policy before becoming control-eligible. Canonical research: #405C.

nsaspy commented

2026-09-08 01:46:54 +00:00

Author

Owner

Machine Spirit #405D executable-conformance handoff: the #376-#387 closed-loop design is still a required gate, not executable Machine Spirit evidence on current main; the canonical expert_register family was not found in the current code search. The implementation/conformance target is now MSD-PROC-* plus MSD-ZERO-*: one deterministic Requirement→Knowledge→Plan→Experts→effect fixture→fresh observation→VERIFY→Repair→success path, unchanged Frozen Spec, one scheduler, cancellation/no-progress/fault behavior, exact evidence lineage, providers disabled and model_calls = 0. This comment does not reopen architecture research; it records #405D's failed executable gate.

Machine Spirit #405D executable-conformance handoff: the #376-#387 closed-loop design is still a required gate, not executable Machine Spirit evidence on current `main`; the canonical `expert_register` family was not found in the current code search. The implementation/conformance target is now `MSD-PROC-*` plus `MSD-ZERO-*`: one deterministic Requirement→Knowledge→Plan→Experts→effect fixture→fresh observation→VERIFY→Repair→success path, unchanged Frozen Spec, one scheduler, cancellation/no-progress/fault behavior, exact evidence lineage, providers disabled and `model_calls = 0`. This comment does not reopen architecture research; it records #405D's failed executable gate.