[EPIC][expert] Repair / diagnosis expert: failure hypotheses, evidence gaps, bounded repair, and no-progress detection #413
Labels
No labels
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
nsaspy/prolog-rlm#413
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Parent: #376
Depends on: #377, #384
Integrates with: #379, #380, #381, #382, #383
Research input: #397 #401A
Goal
Add a Repair / Diagnosis Expert that consumes structured VERIFY failures/unknowns/blocked states, performs explicit bounded diagnosis where ambiguity matters, and derives the smallest defensible next repair or evidence-acquisition subgoal without weakening the Frozen Spec.
This expert is the core of the closed-loop retry path:
It does not directly declare success and does not own the scheduler.
MACHINE-SPIRIT #401A refinement — diagnosis is not repair selection
Depth 4A adopts the model-based diagnosis lesson from Reiter/de Kleer-style systems: candidate explanation and corrective action are distinct reasoning products.
Preserve explicit diagnostic state equivalent to:
Hard distinctions:
Where no adequate behavioral model exists, heuristic rules and verified prior cases may propose hypotheses, but their weaker basis remains explicit.
Responsibilities
need_evidence,diagnosed,repair,ambiguous,blocked,exhausted,resource_limited, orunsupportedresults;Failure/repair knowledge
Support extensible rules/models for cases such as:
The expert must distinguish repairable defects from policy/authority denial and missing operator information. Do not retry an authority denial as if it were a coding bug and do not create a diagnostic fault solely to explain policy denial.
Hypothesis / conflict model
Represent diagnosis as inspectable data, for example:
Do not collapse all diagnosis into one opaque prose explanation or one magic confidence score.
Multiple hypotheses may remain live. If structural/resource bounds prevent resolution, return explicit ambiguity/resource-limited status rather than first-found diagnosis.
Discriminator selection
When more than one materially different diagnosis is live, permit a typed evidence-acquisition decision before repair:
Exact information-gain/cost optimization is profile/policy work for later Machine Spirit subpasses. The invariant here is that discriminating evidence is first-class and may be preferable to blind repair.
Procedural repair integration
Repair output should refer to or propose a typed procedural task/method candidate consumed by #379/#401A rather than directly executing arbitrary corrective code.
A successful repair may later become an episodic case/precedent with its verification receipt. It does not auto-promote itself into a trusted generic method/rule; #404 owns promotion.
No-progress / loop detection
Integrate with existing loop/detector work where appropriate. At minimum detect:
Stop structurally rather than spin forever.
LLM fallback
Normal diagnosis/repair selection uses Prolog rules/models and zero model calls.
An explicitly enabled model fallback may propose a candidate diagnosis, discriminator, or repair for unmodeled failures. That proposal is inert data and must be reconciled with current evidence, diagnosis semantics, expert/procedure applicability, Frozen Spec and capability rules before use.
Only the fallback call is metered.
Hard invariants
unknownintosatisfiedwithout new admissible evidence.edit/createor plan-native semantics for D6-11 ops.Acceptance
Non-goals
Refs #376 #377 #379 #380 #381 #382 #383 #384 #130 #397 #401.
Machine Spirit #401B canonical handoff — diagnosis under nondeterminism, exogenous change, and risky tests
Depth 4B strengthens this diagnosis contract:
predicted != observedtofault(Action).retry until successis not a guarantee.Full rationale, sources and B1-B28 fixtures are in #401 Depth-4B.
Machine Spirit #401D handoff — bounded incremental diagnosis/repair
Depth 4D makes diagnosis execution explicitly query/run-local, bounded, incrementally reusable derived state.
resource_limited, never arbitrary first diagnosis;Relevant primary work: Jannach/Schmitz/Shchekotykhin AAAI 2015 parallel hitting sets without losing diagnoses (https://ojs.aaai.org/index.php/AAAI/article/view/9389) and Rodler/Herold StaticHS for sequential diagnosis (https://ojs.aaai.org/index.php/SOCS/article/view/18449).
Full DPEC design + D26/D27 diagnosis fixtures are on #401.
MACHINE-SPIRIT #404A handoff — diagnosis/repair experience
EPOCH consumes #385 outcomes as immutable learning experience, but preserves:
A verified repair may seed EBG/case/ILP candidate generation only with its exact diagnosis assumptions, observations, discriminators, repair method/version, VERIFY receipt, Review findings and replay lineage. Promotion remains a separate #404 policy decision with counterexample/independence checks. Regressions against an admitted learned repair strategy can append suspension/supersession evidence without rewriting the historical episode.
Ref #404A EPOCH.
Machine Spirit #404B / CITADEL-EPOCH handoff
Learning labels must consume the diagnosis/failure class rather than raw success/failure:
Preserve whether the episode was caused/blocked by method defect, plan-instance defect, stale evidence, authority denial, resource exhaustion, exogenous/transient failure, verifier uncertainty, or unresolved multi-fault diagnosis. Only sufficiently supported causal attribution may feed the corresponding learning lane as a counterexample/positive example; unresolved attribution stays explicit.
This prevents authority/resource/transient failures from poisoning procedural learning and prevents a successful repair from retrospectively proving the original diagnosis. Refs #404B CITADEL fixtures B21-B24.
MACHINE-SPIRIT #404D / CLOCKWORK handoff
Diagnosis remains upstream evidence-labeling input, not permission to rewrite a frozen evaluation. CLOCKWORK requires learning relevance to preserve the exact diagnosis/failure-class receipt used when an experience becomes positive/negative/context evidence.
After an EvaluationSnapshot is frozen, later diagnosis repair/reclassification is a new evidence/lifecycle event and may invalidate/restart evaluation; it cannot mutate old labels in place to rescue a candidate. Authority denial, stale evidence, resource exhaustion and exogenous/transient failure remain distinct from method/rule failure.
Counterexample/minimized-failure witnesses should link back to the exact diagnostic problem and observation generations when applicable. Refs #397 #404 #384 #395.