[P1] Define dual synchronous/asynchronous API contract across RLM libraries

lost-rob0t commented

2026-08-16 04:15:19 +00:00

(Migrated from github.com)

Goal

Make blocking-capable prolog-rlm library operations available through both synchronous and asynchronous APIs, following one asynchronous execution path plus a synchronous bridge for callers that want blocking behavior.

This is a cross-library contract, not an agentProlog/-only feature.

Current state — 2026-08-18

Canonical main inspected for this reconciliation: abfc30ebb9f335d5841c1f7910bd474da905ebcf.

The core migration is substantially complete:

PR #55 added the bounded reusable Future/task substrate.
PR #59 made completion/provider/chain execution canonical async-first.
PR #60 migrated tool invocation and MCP latency-bearing operations.
PR #61 migrated agent and graph latency-bearing operations.
PR #62 completed #53 host authority and the non-blocking pending-operation core without holding a scheduler worker while waiting for human approval.
PR #72 completed and closed #52, including first-class MCP configuration references and closed host-controlled installer/stdio execution policy.

The old status sentence saying “#52 remains open” and treating #53 as an unfinished dependency was stale. Both #52 and #53 are closed on canonical main.

This issue remains open for the parts of the cross-library contract that do not yet exist end to end: concrete process/test/network tool surfaces under #49/#50, equivalence coverage for those future blocking-capable surfaces, and downstream agentProlog/ approval/TUI responsiveness. The existence of non-blocking pending approval in core does not prove a TUI that has not been built.

Design principle

Do not build two independent implementations of every operation.

Prefer:

canonical execute operation
        |
        +--> async Future caller awaits/polls/composes it
        |
        +--> sync facade waits for the same Future

Code already running inside a bounded async worker calls the trusted execute ABI directly rather than nesting another Future wait.

Required task/future semantics

The reusable asynchronous result abstraction must support at least:

task/future creation;
completion with value;
completion with structured error;
non-blocking status inspection;
synchronous wait/resolve;
timeout-aware wait;
cancellation;
callback or continuation registration where useful;
composition of multiple tasks;
trace/session correlation;
deterministic cleanup.

Cross-library coverage

The dual API contract applies where blocking or latency is meaningful, including provider/model requests, streaming, retries, tool/process/network work, MCP lifecycle and invocation, supervised agent work, graph run/resume, approval resolution, and downstream interactive clients.

Immediate/pure operations do not need decorative async twins.

Sync/async equivalence

For the same operation and inputs:

sync and async surfaces must produce equivalent structured outcomes;
tracing and usage accounting must be equivalent;
authority/capability checks must be identical;
budgets/timeouts must be identical unless explicitly overridden;
cancellation must not leave duplicate or orphaned work;
retries must not execute twice merely because a sync wrapper is used.

Avoid nested blocking

The async implementation must never internally call the synchronous wrapper for an async operation. The sync wrapper may wait on the async implementation.

Streaming

Streaming model/tool operations should be naturally asynchronous, with a synchronous convenience consumer allowed to collect/iterate the same stream rather than creating a second provider implementation.

Acceptance criteria

Define one reusable task/future abstraction for the library family.
Define naming/convention for sync and async predicate pairs.
Sync wrappers use the same underlying execution path as async calls across all blocking-capable libraries.
Add async model/provider calls.
Add async tool invocation using the canonical async-first architecture.
Add async process/test/network tool support.
Add async MCP install/run/invoke lifecycle.
Add async agent request/result/cancellation support using the canonical async-first architecture.
Add async graph execution/resume where blocking applies using the canonical async-first architecture.
Approval/pending-diff workflows can be resolved without blocking the TUI.
agentProlog/ can remain interactive while one or more operations are active.
Multiple concurrent tasks can be supervised with bounded concurrency.
Sync/async outputs, accounting, traces, authority, and capability decisions are equivalent across all migrated blocking-capable libraries.
Timeouts and cancellation clean up Future workers/resources deterministically.
Tests cover concurrency, cancellation, timeout, failure propagation, sync/async equivalence, and no duplicate execution across the remaining library migrations.

Non-goals

Do not create separate sync and async business logic implementations.
Do not make the entire Prolog runtime globally asynchronous.
Do not require downstream libraries to expose async variants for operations that are inherently immediate/pure.
Do not let async execution bypass capability, authority, budget, trace, or confinement rules.

References

#49 / #50 external concrete tool work
completed #52 MCP lifecycle/config policy
completed #53 host authority and pending-operation core
#79 canonical effect-boundary adoption
docs/async-runtime.md
docs/authority-runtime.md

## Goal Make blocking-capable `prolog-rlm` library operations available through both synchronous and asynchronous APIs, following one asynchronous execution path plus a synchronous bridge for callers that want blocking behavior. This is a **cross-library contract**, not an `agentProlog/`-only feature. ## Current state — 2026-08-18 Canonical `main` inspected for this reconciliation: `abfc30ebb9f335d5841c1f7910bd474da905ebcf`. The core migration is substantially complete: - PR #55 added the bounded reusable Future/task substrate. - PR #59 made completion/provider/chain execution canonical async-first. - PR #60 migrated tool invocation and MCP latency-bearing operations. - PR #61 migrated agent and graph latency-bearing operations. - PR #62 completed #53 host authority and the non-blocking pending-operation core without holding a scheduler worker while waiting for human approval. - PR #72 completed and closed #52, including first-class MCP configuration references and closed host-controlled installer/stdio execution policy. The old status sentence saying “#52 remains open” and treating #53 as an unfinished dependency was stale. Both #52 and #53 are closed on canonical `main`. This issue remains open for the parts of the cross-library contract that do **not yet exist end to end**: concrete process/test/network tool surfaces under #49/#50, equivalence coverage for those future blocking-capable surfaces, and downstream `agentProlog/` approval/TUI responsiveness. The existence of non-blocking pending approval in core does not prove a TUI that has not been built. ## Design principle Do not build two independent implementations of every operation. Prefer: ```text canonical execute operation | +--> async Future caller awaits/polls/composes it | +--> sync facade waits for the same Future ``` Code already running inside a bounded async worker calls the trusted execute ABI directly rather than nesting another Future wait. ## Required task/future semantics The reusable asynchronous result abstraction must support at least: - task/future creation; - completion with value; - completion with structured error; - non-blocking status inspection; - synchronous wait/resolve; - timeout-aware wait; - cancellation; - callback or continuation registration where useful; - composition of multiple tasks; - trace/session correlation; - deterministic cleanup. ## Cross-library coverage The dual API contract applies where blocking or latency is meaningful, including provider/model requests, streaming, retries, tool/process/network work, MCP lifecycle and invocation, supervised agent work, graph run/resume, approval resolution, and downstream interactive clients. Immediate/pure operations do not need decorative async twins. ## Sync/async equivalence For the same operation and inputs: - sync and async surfaces must produce equivalent structured outcomes; - tracing and usage accounting must be equivalent; - authority/capability checks must be identical; - budgets/timeouts must be identical unless explicitly overridden; - cancellation must not leave duplicate or orphaned work; - retries must not execute twice merely because a sync wrapper is used. ## Avoid nested blocking The async implementation must never internally call the synchronous wrapper for an async operation. The sync wrapper may wait on the async implementation. ## Streaming Streaming model/tool operations should be naturally asynchronous, with a synchronous convenience consumer allowed to collect/iterate the same stream rather than creating a second provider implementation. ## Acceptance criteria - [x] Define one reusable task/future abstraction for the library family. - [x] Define naming/convention for sync and async predicate pairs. - [ ] Sync wrappers use the same underlying execution path as async calls across all blocking-capable libraries. - [x] Add async model/provider calls. - [x] Add async tool invocation using the canonical async-first architecture. - [ ] Add async process/test/network tool support. - [x] Add async MCP install/run/invoke lifecycle. - [x] Add async agent request/result/cancellation support using the canonical async-first architecture. - [x] Add async graph execution/resume where blocking applies using the canonical async-first architecture. - [ ] Approval/pending-diff workflows can be resolved without blocking the TUI. - [ ] `agentProlog/` can remain interactive while one or more operations are active. - [x] Multiple concurrent tasks can be supervised with bounded concurrency. - [ ] Sync/async outputs, accounting, traces, authority, and capability decisions are equivalent across all migrated blocking-capable libraries. - [x] Timeouts and cancellation clean up Future workers/resources deterministically. - [ ] Tests cover concurrency, cancellation, timeout, failure propagation, sync/async equivalence, and no duplicate execution across the remaining library migrations. ## Non-goals - Do not create separate sync and async business logic implementations. - Do not make the entire Prolog runtime globally asynchronous. - Do not require downstream libraries to expose async variants for operations that are inherently immediate/pure. - Do not let async execution bypass capability, authority, budget, trace, or confinement rules. ## References - #49 / #50 external concrete tool work - completed #52 MCP lifecycle/config policy - completed #53 host authority and pending-operation core - #79 canonical effect-boundary adoption - `docs/async-runtime.md` - `docs/authority-runtime.md`

lost-rob0t commented

2026-08-16 12:07:11 +00:00

(Migrated from github.com)

State audit after merged PR #55: the bounded Future runtime is useful and should be retained, but #54's core directionality is not yet satisfied. rlm_completion_async/*, provider/chain async facades, and the public rlm async facade currently submit synchronous public predicates to the worker pool (async -> sync). The intended invariant is the reverse: canonical async/task execution with sync as await+cleanup wrappers. This slice will refactor completion and provider/chain first, extend Future metadata/continuations/cancellation propagation, add directionality/no-duplicate regression tests, then update this issue again with the exact remaining tool/MCP/agent/graph/authority work. #54 remains open.

State audit after merged PR #55: the bounded Future runtime is useful and should be retained, but #54's core directionality is **not yet satisfied**. `rlm_completion_async/*`, provider/chain async facades, and the public `rlm` async facade currently submit synchronous public predicates to the worker pool (`async -> sync`). The intended invariant is the reverse: canonical async/task execution with sync as await+cleanup wrappers. This slice will refactor completion and provider/chain first, extend Future metadata/continuations/cancellation propagation, add directionality/no-duplicate regression tests, then update this issue again with the exact remaining tool/MCP/agent/graph/authority work. #54 remains open.

lost-rob0t commented

2026-08-16 12:34:59 +00:00

(Migrated from github.com)

Post-merge verification for PR #59 is complete. Merge commit ded8c2734a0c716c6074457b45fad5f475cd16c9 passed main CI run #509 (31947378064): the full deterministic unit/load/benchmark/conformance/deep-recursion/CLI/persistence gate and the complete REAL OpenRouter core, structured-repair, benchmark, depth 0/1/2 recursion, and CLI smoke gate all succeeded. #54 remains open for the tool/MCP/agent/graph canonical async migrations, authority pending-operation integration, and later agentProlog/ responsiveness work.

Post-merge verification for PR #59 is complete. Merge commit `ded8c2734a0c716c6074457b45fad5f475cd16c9` passed `main` CI run #509 (`31947378064`): the full deterministic unit/load/benchmark/conformance/deep-recursion/CLI/persistence gate and the complete REAL OpenRouter core, structured-repair, benchmark, depth 0/1/2 recursion, and CLI smoke gate all succeeded. #54 remains open for the tool/MCP/agent/graph canonical async migrations, authority pending-operation integration, and later `agentProlog/` responsiveness work.

lost-rob0t commented

2026-08-16 23:53:23 +00:00

(Migrated from github.com)

Async/pending-operation update from PR #62:

The core async direction remains intact: execute ABI -> async Future; sync facade starts the same async operation and awaits it. Human approval uses deferred/manual Futures and does not occupy an rlm_async worker while waiting.

This hardening pass also fixed a core Future bug: a terminal Future whose stored result does not unify with a caller's pre-bound expected result now fails deterministically instead of falling back into the await polling loop forever. The focused MCP timeout exposed this because lifecycle connect was incorrectly expecting a wrapper shape; that connect adapter now returns the canonical MCP execute outcome directly.

Current deterministic CI is green across the canonical PlUnit suite, benchmark/conformance, deep recursion, CLI/trace, fresh-process graph resume, artifact handoff, and whitespace.

#54 remains open intentionally. Outstanding work includes:

#63 cancellation-linearizable approval -> execution handoff
#64 bounded terminal pending-operation/Future retention
remaining concrete process/test/network standard-tool/library work already tracked by the async/tool boundary issues
downstream approval presentation/TUI work (not owned by core)

REAL OpenRouter is currently externally rate-limited with HTTP 429 free-models-per-day-high-balance; do not weaken the live gate to close this umbrella issue.

Async/pending-operation update from PR #62: The core async direction remains intact: execute ABI -> async Future; sync facade starts the same async operation and awaits it. Human approval uses deferred/manual Futures and does not occupy an `rlm_async` worker while waiting. This hardening pass also fixed a core Future bug: a terminal Future whose stored result does not unify with a caller's pre-bound expected result now fails deterministically instead of falling back into the await polling loop forever. The focused MCP timeout exposed this because lifecycle connect was incorrectly expecting a wrapper shape; that connect adapter now returns the canonical MCP execute outcome directly. Current deterministic CI is green across the canonical PlUnit suite, benchmark/conformance, deep recursion, CLI/trace, fresh-process graph resume, artifact handoff, and whitespace. #54 remains open intentionally. Outstanding work includes: - #63 cancellation-linearizable approval -> execution handoff - #64 bounded terminal pending-operation/Future retention - remaining concrete process/test/network standard-tool/library work already tracked by the async/tool boundary issues - downstream approval presentation/TUI work (not owned by core) REAL OpenRouter is currently externally rate-limited with HTTP 429 `free-models-per-day-high-balance`; do not weaken the live gate to close this umbrella issue.

lost-rob0t commented

2026-08-17 05:53:39 +00:00

(Migrated from github.com)

Authority/pending-operation integration from this umbrella is now complete on canonical main via PR #62, squash merge 88b64db41a4c474379a1808b95c761ffc78d9fc2.

The core authority workflow uses deferred/manual Futures for human latency, canonical execute -> async Future -> sync-await direction, cancellation-linearizable approved execution, bounded terminal pending state, and agent/graph/MCP composition. Exact-head deterministic and complete configured REAL OpenRouter CI passed before merge.

Keeping #54 open intentionally: its remaining acceptance still includes concrete async process/test/network tool support, remaining cross-library equivalence coverage where those tools land, and downstream agentProlog/ responsiveness integration. Those should not be falsely marked complete by the authority merge.

Authority/pending-operation integration from this umbrella is now complete on canonical `main` via PR #62, squash merge `88b64db41a4c474379a1808b95c761ffc78d9fc2`. The core authority workflow uses deferred/manual Futures for human latency, canonical execute -> async Future -> sync-await direction, cancellation-linearizable approved execution, bounded terminal pending state, and agent/graph/MCP composition. Exact-head deterministic and complete configured REAL OpenRouter CI passed before merge. Keeping #54 open intentionally: its remaining acceptance still includes concrete async process/test/network tool support, remaining cross-library equivalence coverage where those tools land, and downstream `agentProlog/` responsiveness integration. Those should not be falsely marked complete by the authority merge.

lost-rob0t commented

2026-08-17 07:38:51 +00:00

(Migrated from github.com)

Post-merge reconciliation after PR #72: the Current state text above is stale where it says #52 remains open. #52 closed on 2026-08-17 via squash merge 7b527750a64d5cbd9fff79413a399a8c69e90c75 after exact-head deterministic + REAL validation. MCP install/run/configuration now uses first-class config references plus closed host-controlled installer/stdio execution profiles while preserving the canonical execute -> async Future -> sync-await direction.

#54 remains intentionally open for its actual unchecked acceptance: process/test/network external-tool async work, approval/pending workflow integration, remaining cross-library sync/async equivalence coverage, and later agentProlog responsiveness. Do not treat #52 as a remaining dependency.

Post-merge reconciliation after PR #72: the `Current state` text above is stale where it says `#52 remains open`. #52 closed on 2026-08-17 via squash merge `7b527750a64d5cbd9fff79413a399a8c69e90c75` after exact-head deterministic + REAL validation. MCP install/run/configuration now uses first-class config references plus closed host-controlled installer/stdio execution profiles while preserving the canonical execute -> async Future -> sync-await direction. #54 remains intentionally open for its actual unchecked acceptance: process/test/network external-tool async work, approval/pending workflow integration, remaining cross-library sync/async equivalence coverage, and later agentProlog responsiveness. Do not treat #52 as a remaining dependency.