nsaspy/prolog-rlm

Fork 0

[RESEARCH/IDEA] Reusable evolutionary-search library for RLM callers #142

New issue

Open

opened 2026-08-21 04:49:42 +00:00 by lost-rob0t · 2 comments

lost-rob0t commented

2026-08-21 04:49:42 +00:00

(Migrated from github.com)

Companion to lost-rob0t/agentProlog#2.

Mission

Research and design a domain-neutral evolutionary-search library in Prolog that can be used with prolog-rlm and consumed by downstream harness integrations, especially the AgentProlog DeepSeek Harness plugin.

This is intentionally split across two repositories:

DeepSeek Harness
  -> AgentProlog out-of-tree Cordis plugin
  -> thin TypeScript/Cordis <-> Prolog adapter
  -> reusable prolog-rlm evolution library
  -> existing RLM runtime/Future/authority/verification/tracing

The algorithmic/evolutionary machinery belongs here when it is reusable beyond AgentProlog. DeepSeek-specific lifecycle, UI, service registration, plugin packaging, and adapter glue belong downstream in lost-rob0t/agentProlog.

Why a library, not product code

The reusable layer should support RLM callers without depending on DeepSeek Harness, Node, Cordis, a particular coding-agent product, or a frontend.

Likely library responsibilities include:

typed candidate/genotype representation;
bounded mutation and crossover operators;
selection / population management;
lineage and provenance;
fitness/evidence records;
promotion, rejection, rollback and reproducibility;
benchmark/evaluator hooks;
async evaluation through existing RLM Future semantics where evaluation is latency-bearing;
cost/token/time budgets;
tracing and structured outcomes;
immutable constraints that evolution cannot weaken;
optional experience/feedback memory for mutation guidance.

Do not invent a second scheduler, authority system, effect ledger, verifier stack, or async model. Reuse existing prolog-rlm contracts.

Initial hypothesis

Start with configuration-space evolution, not model-weight evolution.

A candidate could represent a typed agent/program configuration such as:

candidate(Id,
          topology(Topology),
          roles(Roles),
          skills(Skills),
          model_policy(ModelPolicy),
          tool_policy(ToolPolicy),
          loop_policy(LoopPolicy),
          verifier_policy(VerifierPolicy),
          context_policy(ContextPolicy),
          budget(Budget)).

Lineage/evidence should be first-class data:

parent(Child, Parent).
mutation(Child, Operator, Evidence).
crossover(Child, ParentA, ParentB).
fitness(Candidate, Benchmark, Metric, Score).
rejected(Candidate, Constraint, Evidence).
promoted(Candidate, Generation, Evidence).

Exact API/schema must follow repository conventions after inspecting current code.

Research basis

Compare at minimum:

Agentic ESOpt (2026-08-18): long-horizon agent evolution strategies and trajectory-level reward: https://arxiv.org/abs/2608.17310
EvoMAS: structured multi-agent system evolution with selection, mutation/crossover, execution traces and experience memory: https://arxiv.org/abs/2602.06511
EvoAgent (2024): evolving expert agents into diverse multi-agent systems: https://arxiv.org/abs/2406.14228
EvoAgent skill-learning work (2026): structured skill/delegation evolution: https://arxiv.org/abs/2604.20133
GEPA/DSPy reflective prompt/program evolution as an adjacent optimization model.

The design should also reconcile against existing prolog-rlm runtime contracts and current AgentProlog migration work, rather than treating papers as architecture authority.

Hard invariants

Evolution must never be able to mutate away or bypass:

Frozen-Spec semantics where applicable;
host authority/capability ceilings;
durable effect identity/adoption;
cancellation;
path/project confinement;
structured schema validation;
verifier requirements;
budget ceilings;
trace/evidence requirements;
stale-preimage protections;
any other security/runtime invariants already enforced by core.

Generated candidates are data. Do not pass arbitrary generated terms to unrestricted call/1 or expose mutation as ambient code execution.

Required upstream/downstream split

`prolog-rlm`

Own reusable evolutionary primitives and RLM integration hooks.

`agentProlog`

Own the DeepSeek Harness plugin, Cordis service/event/effect lifecycle, TypeScript-to-Prolog bridge, product presets, coding-agent genotype fields, UX, and benchmark/product composition.

DeepSeek Harness is currently developer preview and warns that compatibility-breaking changes are expected, so keep the adapter narrow/versioned and prevent DeepSeek-specific types from leaking into this library.

Research questions

What is the smallest reusable public API for evolutionary search in Prolog?
Which genotype constraints are generic versus AgentProlog product schema?
Can Prolog generate only valid candidates by construction rather than generate-then-reject?
Which mutation/crossover operators are safe and useful for symbolic agent configurations?
How should evaluator hooks compose with RLM Futures, cancellation, usage, tracing and structured outcomes?
What fitness representation supports multi-objective correctness/cost/latency/robustness without collapsing everything into one magic scalar?
How should lineage and benchmark evidence persist reproducibly?
How should failed trajectories feed mutation hints without becoming trusted policy?
Which current prolog-rlm modules can be reused unchanged, and which minimal generic APIs are actually missing?
Should model-weight/parameter-space ES remain a separate optional package rather than part of the initial library?

Proposed research deliverable

Produce a design note that defines:

module/library boundary and proposed public predicates;
typed genotype/candidate abstraction;
generic versus downstream schema split;
mutation/crossover/selection contracts;
evaluator/Future interface;
immutable constraint interface;
lineage/evidence/persistence model;
multi-objective fitness representation;
benchmark protocol;
promotion/rollback rules;
cost budget;
compatibility strategy for downstream harness adapters;
explicit GO / HOLD / REJECT recommendation for a first implementation slice.

Research/design only until that gate is complete. Do as much useful research as possible per loop rather than stopping after one paper summary.

Companion to `lost-rob0t/agentProlog#2`. ## Mission Research and design a **domain-neutral evolutionary-search library in Prolog** that can be used with `prolog-rlm` and consumed by downstream harness integrations, especially the AgentProlog DeepSeek Harness plugin. This is intentionally split across two repositories: ```text DeepSeek Harness -> AgentProlog out-of-tree Cordis plugin -> thin TypeScript/Cordis <-> Prolog adapter -> reusable prolog-rlm evolution library -> existing RLM runtime/Future/authority/verification/tracing ``` The algorithmic/evolutionary machinery belongs here when it is reusable beyond AgentProlog. DeepSeek-specific lifecycle, UI, service registration, plugin packaging, and adapter glue belong downstream in `lost-rob0t/agentProlog`. ## Why a library, not product code The reusable layer should support RLM callers without depending on DeepSeek Harness, Node, Cordis, a particular coding-agent product, or a frontend. Likely library responsibilities include: - typed candidate/genotype representation; - bounded mutation and crossover operators; - selection / population management; - lineage and provenance; - fitness/evidence records; - promotion, rejection, rollback and reproducibility; - benchmark/evaluator hooks; - async evaluation through existing RLM Future semantics where evaluation is latency-bearing; - cost/token/time budgets; - tracing and structured outcomes; - immutable constraints that evolution cannot weaken; - optional experience/feedback memory for mutation guidance. Do not invent a second scheduler, authority system, effect ledger, verifier stack, or async model. Reuse existing `prolog-rlm` contracts. ## Initial hypothesis Start with **configuration-space evolution**, not model-weight evolution. A candidate could represent a typed agent/program configuration such as: ```prolog candidate(Id, topology(Topology), roles(Roles), skills(Skills), model_policy(ModelPolicy), tool_policy(ToolPolicy), loop_policy(LoopPolicy), verifier_policy(VerifierPolicy), context_policy(ContextPolicy), budget(Budget)). ``` Lineage/evidence should be first-class data: ```prolog parent(Child, Parent). mutation(Child, Operator, Evidence). crossover(Child, ParentA, ParentB). fitness(Candidate, Benchmark, Metric, Score). rejected(Candidate, Constraint, Evidence). promoted(Candidate, Generation, Evidence). ``` Exact API/schema must follow repository conventions after inspecting current code. ## Research basis Compare at minimum: - Agentic ESOpt (2026-08-18): long-horizon agent evolution strategies and trajectory-level reward: https://arxiv.org/abs/2608.17310 - EvoMAS: structured multi-agent system evolution with selection, mutation/crossover, execution traces and experience memory: https://arxiv.org/abs/2602.06511 - EvoAgent (2024): evolving expert agents into diverse multi-agent systems: https://arxiv.org/abs/2406.14228 - EvoAgent skill-learning work (2026): structured skill/delegation evolution: https://arxiv.org/abs/2604.20133 - GEPA/DSPy reflective prompt/program evolution as an adjacent optimization model. The design should also reconcile against existing `prolog-rlm` runtime contracts and current AgentProlog migration work, rather than treating papers as architecture authority. ## Hard invariants Evolution must never be able to mutate away or bypass: - Frozen-Spec semantics where applicable; - host authority/capability ceilings; - durable effect identity/adoption; - cancellation; - path/project confinement; - structured schema validation; - verifier requirements; - budget ceilings; - trace/evidence requirements; - stale-preimage protections; - any other security/runtime invariants already enforced by core. Generated candidates are data. Do not pass arbitrary generated terms to unrestricted `call/1` or expose mutation as ambient code execution. ## Required upstream/downstream split ### `prolog-rlm` Own reusable evolutionary primitives and RLM integration hooks. ### `agentProlog` Own the DeepSeek Harness plugin, Cordis service/event/effect lifecycle, TypeScript-to-Prolog bridge, product presets, coding-agent genotype fields, UX, and benchmark/product composition. DeepSeek Harness is currently developer preview and warns that compatibility-breaking changes are expected, so keep the adapter narrow/versioned and prevent DeepSeek-specific types from leaking into this library. ## Research questions 1. What is the smallest reusable public API for evolutionary search in Prolog? 2. Which genotype constraints are generic versus AgentProlog product schema? 3. Can Prolog generate only valid candidates by construction rather than generate-then-reject? 4. Which mutation/crossover operators are safe and useful for symbolic agent configurations? 5. How should evaluator hooks compose with RLM Futures, cancellation, usage, tracing and structured outcomes? 6. What fitness representation supports multi-objective correctness/cost/latency/robustness without collapsing everything into one magic scalar? 7. How should lineage and benchmark evidence persist reproducibly? 8. How should failed trajectories feed mutation hints without becoming trusted policy? 9. Which current `prolog-rlm` modules can be reused unchanged, and which minimal generic APIs are actually missing? 10. Should model-weight/parameter-space ES remain a separate optional package rather than part of the initial library? ## Proposed research deliverable Produce a design note that defines: - module/library boundary and proposed public predicates; - typed genotype/candidate abstraction; - generic versus downstream schema split; - mutation/crossover/selection contracts; - evaluator/Future interface; - immutable constraint interface; - lineage/evidence/persistence model; - multi-objective fitness representation; - benchmark protocol; - promotion/rollback rules; - cost budget; - compatibility strategy for downstream harness adapters; - explicit `GO / HOLD / REJECT` recommendation for a first implementation slice. Research/design only until that gate is complete. Do as much useful research as possible per loop rather than stopping after one paper summary.

lost-rob0t commented

2026-08-21 07:31:36 +00:00

(Migrated from github.com)

Packaging worker research handoff while #145 revalidates. I compared the paired downstream agentProlog#2 / PR #3 design with the requested research basis and current core contracts.

Recommendation: GO for a small generic configuration-space evolution kernel after the design note is committed; HOLD model-weight/parameter ES and HOLD any DeepSeek/Cordis-specific implementation here.

Why:

EvoMAS (arXiv:2602.06511) directly supports structured configuration-space evolution with selection + feedback-conditioned mutation/crossover over execution traces, which matches our typed-data/Prolog strengths and avoids generated-code executability failures.
GEPA (arXiv:2507.19457) is useful specifically for preserving a Pareto frontier and using reflective trajectory feedback as untrusted mutation guidance, not as policy or verifier truth.
Agentic ESOpt (arXiv:2608.17310) supports trajectory-level black-box optimization for long horizons, but its full-parameter perturbation machinery is a different package/problem. The useful upstream lesson for v1 is evaluator/trajectory interfaces and multi-objective evidence, not weight mutation.

Smallest reusable API direction:

evolution_candidate_validate/3 over a closed typed candidate schema + immutable host constraint envelope.
evolution_mutate/5 and evolution_crossover/6 accepting only registered code-owned operator IDs; generated candidates remain data, never arbitrary call/1.
evolution_evaluate_async/4 over the existing Future/runtime execution path, with sync facade awaiting the same Future.
evolution_record_fitness/5 preserving a vector/objective record (correctness, verification, cost, latency, robustness/resource evidence) instead of one magic scalar.
evolution_select/4 using deterministic policy IDs; first implementation should support Pareto/non-dominated selection plus explicit tie policy.
lineage/provenance records link parent(s), operator/version, candidate fingerprint, benchmark/evaluator identity, runtime trace/result/evidence refs, and usage.
promotion/rejection is separate from evaluation; host-required verifier/Frozen-Spec/authority/budget ceilings are immutable constraints and cannot be candidate dimensions.

Reuse rather than duplicate: rlm_future for latency-bearing evaluation/cancellation, existing structured outcomes/traces/usage, #56 verifier/evidence boundary as it lands, #57/#79 effect semantics for externally effectful evaluations, and #144's eventual bounded subagent result contract. No second scheduler/authority/effect ledger.

Generic vs downstream split: generic core can evolve references such as topology/roles/prompt refs/skill refs/model-policy refs/tool-policy refs/loop/verifier/context/budget profiles under host ceilings. AgentProlog owns the coding-specific schema and benchmark composition. DeepSeek Harness PR #3 already correctly keeps experiment.* unavailable until this upstream capability exists and only transports passive parent/subagent correlation.

Suggested first implementation gate after a design note: deterministic pure candidate validation + code-owned mutation/crossover + lineage + Pareto fitness/selection fixtures, with evaluator integration as the next slice. This can be tested without a provider and gives downstream #2 a stable capability to target without prematurely inventing distributed experiment orchestration.

Packaging worker research handoff while #145 revalidates. I compared the paired downstream `agentProlog#2` / PR #3 design with the requested research basis and current core contracts. **Recommendation: GO for a small generic configuration-space evolution kernel after the design note is committed; HOLD model-weight/parameter ES and HOLD any DeepSeek/Cordis-specific implementation here.** Why: - EvoMAS (arXiv:2602.06511) directly supports structured configuration-space evolution with selection + feedback-conditioned mutation/crossover over execution traces, which matches our typed-data/Prolog strengths and avoids generated-code executability failures. - GEPA (arXiv:2507.19457) is useful specifically for preserving a Pareto frontier and using reflective trajectory feedback as *untrusted mutation guidance*, not as policy or verifier truth. - Agentic ESOpt (arXiv:2608.17310) supports trajectory-level black-box optimization for long horizons, but its full-parameter perturbation machinery is a different package/problem. The useful upstream lesson for v1 is evaluator/trajectory interfaces and multi-objective evidence, not weight mutation. Smallest reusable API direction: 1. `evolution_candidate_validate/3` over a closed typed candidate schema + immutable host constraint envelope. 2. `evolution_mutate/5` and `evolution_crossover/6` accepting only registered code-owned operator IDs; generated candidates remain data, never arbitrary `call/1`. 3. `evolution_evaluate_async/4` over the existing Future/runtime execution path, with sync facade awaiting the same Future. 4. `evolution_record_fitness/5` preserving a vector/objective record (correctness, verification, cost, latency, robustness/resource evidence) instead of one magic scalar. 5. `evolution_select/4` using deterministic policy IDs; first implementation should support Pareto/non-dominated selection plus explicit tie policy. 6. lineage/provenance records link parent(s), operator/version, candidate fingerprint, benchmark/evaluator identity, runtime trace/result/evidence refs, and usage. 7. promotion/rejection is separate from evaluation; host-required verifier/Frozen-Spec/authority/budget ceilings are immutable constraints and cannot be candidate dimensions. Reuse rather than duplicate: `rlm_future` for latency-bearing evaluation/cancellation, existing structured outcomes/traces/usage, #56 verifier/evidence boundary as it lands, #57/#79 effect semantics for externally effectful evaluations, and #144's eventual bounded subagent result contract. No second scheduler/authority/effect ledger. Generic vs downstream split: generic core can evolve references such as topology/roles/prompt refs/skill refs/model-policy refs/tool-policy refs/loop/verifier/context/budget profiles under host ceilings. AgentProlog owns the coding-specific schema and benchmark composition. DeepSeek Harness PR #3 already correctly keeps `experiment.*` unavailable until this upstream capability exists and only transports passive parent/subagent correlation. Suggested first implementation gate after a design note: deterministic pure candidate validation + code-owned mutation/crossover + lineage + Pareto fitness/selection fixtures, with evaluator integration as the next slice. This can be tested without a provider and gives downstream #2 a stable capability to target without prematurely inventing distributed experiment orchestration.

lost-rob0t commented

2026-08-21 11:33:16 +00:00

(Migrated from github.com)

Hey GPT-5.6 Sol here.

The first GO slice from this design gate is now merged via #148 at 23d919628a449acc3ced08fc9bd8469cc2cbf8ea.

Executable core now owns the pure provider-free configuration-space kernel: closed candidate validation, allow-listed deterministic mutation/crossover, lineage fingerprints/provenance, vector fitness input, and deterministic Pareto selection. Exact-head CI, Nix flake, clean SWI pack install, and Tree-sitter were all green before merge; no unresolved review threads existed.

Next generic slice remains the one identified here: compose latency-bearing evaluation with the existing Future/cancellation/runtime contracts rather than introducing another scheduler. Keep effectful evaluator execution behind existing authority/effect boundaries, and keep DeepSeek/Cordis/product genotype and lifecycle downstream in agentProlog.

Hey GPT-5.6 Sol here. The first GO slice from this design gate is now merged via #148 at `23d919628a449acc3ced08fc9bd8469cc2cbf8ea`. Executable core now owns the pure provider-free configuration-space kernel: closed candidate validation, allow-listed deterministic mutation/crossover, lineage fingerprints/provenance, vector fitness input, and deterministic Pareto selection. Exact-head CI, Nix flake, clean SWI pack install, and Tree-sitter were all green before merge; no unresolved review threads existed. Next generic slice remains the one identified here: compose latency-bearing evaluation with the existing Future/cancellation/runtime contracts rather than introducing another scheduler. Keep effectful evaluator execution behind existing authority/effect boundaries, and keep DeepSeek/Cordis/product genotype and lifecycle downstream in `agentProlog`.