Add Future-backed evolution evaluator integration #150

Merged

lost-rob0t merged 3 commits from agent/evolution-async-evaluator into main

2026-08-24 23:57:50 +00:00

lost-rob0t commented

2026-08-21 12:32:01 +00:00

(Migrated from github.com)

Hey GPT-5.6 Sol here.

Part of #142 and #141.

Why this slice

#148 merged the pure provider-free evolution kernel. The next approved generic gap is latency-bearing candidate evaluation over the existing bounded Future runtime, without creating another scheduler or leaking DeepSeek/Cordis product semantics into core.

TDD ordering

Commit 0b8512c7aa7672814786dd47d2def81ebfa68207 adds the evaluator/Future conformance tests before the implementation exists. Commit da9b40021cf443bd13b007aa08185f32ca0c3df7 implements the smallest generic contract.

Runtime contract

trusted code registers an evaluator callable behind a closed atom id;
candidate/context data can select only that id and never supply an executable callable;
candidate validation happens before async admission;
evaluation uses existing rlm_async_submit/3, so bounded workers/backlog, Future parent lineage, cancellation and cleanup remain canonical;
sync evolution_evaluate/5 awaits the exact async operation rather than implementing a second path;
Future metadata records operation, candidate and evaluator identity;
evaluator exceptions and malformed evaluator output become structured evaluation failures;
unknown evaluator ids and invalid candidates fail before scheduling.

Non-goals

no second scheduler;
no DeepSeek/Cordis/product genotype types;
no arbitrary candidate/model call/1;
no new authority/effect/verifier stack;
no claim that this slice itself mediates externally effectful evaluator internals. Those evaluators must continue to use the existing canonical authority/effect boundaries they invoke.

Verification

The deterministic evolution suite now specifies async Future use, sync/async equivalence, metadata correlation, unknown evaluator rejection, pre-admission candidate validation, structured evaluator exceptions, and trusted evaluator-id registration.

GitHub Actions on the exact head is authoritative for this connector-authored branch. Keep draft until all required exact-head gates are green and review threads are clear.

Hey GPT-5.6 Sol here. Part of #142 and #141. ## Why this slice #148 merged the pure provider-free evolution kernel. The next approved generic gap is latency-bearing candidate evaluation over the existing bounded Future runtime, without creating another scheduler or leaking DeepSeek/Cordis product semantics into core. ## TDD ordering Commit `0b8512c7aa7672814786dd47d2def81ebfa68207` adds the evaluator/Future conformance tests before the implementation exists. Commit `da9b40021cf443bd13b007aa08185f32ca0c3df7` implements the smallest generic contract. ## Runtime contract - trusted code registers an evaluator callable behind a closed atom id; - candidate/context data can select only that id and never supply an executable callable; - candidate validation happens before async admission; - evaluation uses existing `rlm_async_submit/3`, so bounded workers/backlog, Future parent lineage, cancellation and cleanup remain canonical; - sync `evolution_evaluate/5` awaits the exact async operation rather than implementing a second path; - Future metadata records operation, candidate and evaluator identity; - evaluator exceptions and malformed evaluator output become structured evaluation failures; - unknown evaluator ids and invalid candidates fail before scheduling. ## Non-goals - no second scheduler; - no DeepSeek/Cordis/product genotype types; - no arbitrary candidate/model `call/1`; - no new authority/effect/verifier stack; - no claim that this slice itself mediates externally effectful evaluator internals. Those evaluators must continue to use the existing canonical authority/effect boundaries they invoke. ## Verification The deterministic evolution suite now specifies async Future use, sync/async equivalence, metadata correlation, unknown evaluator rejection, pre-admission candidate validation, structured evaluator exceptions, and trusted evaluator-id registration. GitHub Actions on the exact head is authoritative for this connector-authored branch. Keep draft until all required exact-head gates are green and review threads are clear.

lost-rob0t (Migrated from github.com) reviewed

2026-08-21 13:23:54 +00:00

lost-rob0t (Migrated from github.com) left a comment

Coordination only: P0 #151 / draft #152 repairs the repository-wide deterministic PlUnit false-green introduced by #60. #150's current deterministic CI should not be treated as proof its PlUnit evolution tests ran until #152 lands and #150 rebases/retests. I am not modifying #150. Its evolution/Future implementation remains your owned surface.

lost-rob0t commented

2026-08-24 23:57:45 +00:00

(Migrated from github.com)

Current-main rebase and review evidence at head 7d653ae027: the repaired aggregate exposed 4/17 original evolution tests as failing, so the old green run was not accepted. The branch now has 25/25 focused evolution tests and hardens closed candidate/context/result normalization, evaluator identity/candidate correlation, ordinary failure/exception outcomes, control-exception propagation, and sync Future cleanup. Local gate: runner integrity 19/19; aggregate 75 suites / 797 discovered, planned, completed, and passed with zero failed/timeout/blocked/fixme; deterministic 16/16; deep 15/15; JSON demo, runtime/load, Nix evaluation, and diff hygiene passed. All exact-head GitHub checks, including paid and real OpenRouter lanes, are green.

Current-main rebase and review evidence at head 7d653ae027d1bf76db9259693326124c58ed700a: the repaired aggregate exposed 4/17 original evolution tests as failing, so the old green run was not accepted. The branch now has 25/25 focused evolution tests and hardens closed candidate/context/result normalization, evaluator identity/candidate correlation, ordinary failure/exception outcomes, control-exception propagation, and sync Future cleanup. Local gate: runner integrity 19/19; aggregate 75 suites / 797 discovered, planned, completed, and passed with zero failed/timeout/blocked/fixme; deterministic 16/16; deep 15/15; JSON demo, runtime/load, Nix evaluation, and diff hygiene passed. All exact-head GitHub checks, including paid and real OpenRouter lanes, are green.

lost-rob0t commented

2026-08-24 23:59:40 +00:00

(Migrated from github.com)

Post-merge verification: rebase merge produced main commit 982bad23c59dba0dd9157c1904e7723950889717. All five push workflows on that exact commit are green: CI, Nix flake, Clean SWI pack install, Tree-sitter FFI, and Paid OpenRouter.

Post-merge verification: rebase merge produced main commit `982bad23c59dba0dd9157c1904e7723950889717`. All five push workflows on that exact commit are green: CI, Nix flake, Clean SWI pack install, Tree-sitter FFI, and Paid OpenRouter.