[EPIC] Give rlm_subagent runtime-defaulted deadlines with optional model override #175
Labels
No labels
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
nsaspy/prolog-rlm#175
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Goal
Make the existing model-visible
rlm_subagenttool own a sane task timeout by default while allowing the model to request a different timeout explicitly when needed.The model must normally be able to call:
with no timeout argument at all.
It may optionally call:
The default, maximum, validation, and final effective deadline remain host/runtime policy. The model is requesting a duration, not granting itself unbounded execution.
Do not create a second model-facing task API. Extend the existing
rlm_subagenttool and existing completion/tool budget machinery.Research: current implementation
prolog/rlm_subagent.plrlm_subagent_register/7currently registers a normalrlm_toolschema:The model therefore sees only
query; there is no model-visible timeout override.The trusted handler captures
CompletionOptionsat registration time and later calls:with those host-owned options unchanged.
prolog/rlm_completion.plThe completion supervisor already owns a semantic wall-time budget:
completion_budget/2merges hostbudget(Dict)overrides into that default and validatestime_limitas a positive number.The actual completion is bounded with:
so the completion timeout is an execution budget, not merely a waiting timeout.
prolog/rlm_tool.plTool schemas already carry trusted runtime limits.
perform_tool_effect/...executes the handler through:which itself uses
call_with_time_limit/2and returns a structuredtool_error{kind:timeout,...}.Therefore
rlm_subagentcurrently has two 30-second layers:Schema.limits.time_limit = 30.0);completion_budget.time_limit = 30.0).This happens to align at the default, but there is no explicit policy relationship between them.
prolog/rlm_async.plrlm_future_await(Future, Timeout, Outcome)is only a waiter timeout. The async docs explicitly state that an await timeout does not cancel or restart the underlying task.This epic must not confuse Future-await timeouts with task execution deadlines.
test/rlm_subagent_test.plCurrent tests cover:
There is currently no timeout-policy coverage.
Design decision
Keep one interface
Continue using the existing ordinary tool:
Do not add
rlm_task,spawn_task, a second task registry, or any parallel timeout API just to express this policy.Extend the existing schema with one optional field:
Canonical model-visible schema:
queryremains the only required argument.Host-owned timeout policy
rlm_subagent_register/7currently capturesCompletionOptions; use the same host-controlled configuration path rather than adding another global subsystem.Define a normalized host policy derived from registration options, conceptually:
Exact field names may follow existing option conventions, but the semantics must be explicit.
Preferred registration options:
If no new registration option list is desired because
CompletionOptionsis already the canonical host configuration value, these may live insideCompletionOptionsas dedicated host-only options. Do not expose the max/default policy to model control.Resolution order
For each invocation:
Do not silently convert an over-max request into the max value. Return a structured policy error so traces and tests can distinguish:
Suggested error shape:
Invalid zero/negative/non-number values should be rejected structurally before child creation.
Effective completion budget
The handler must derive the child completion options without mutating the host-captured base options.
Conceptually:
completion_options_with_timeout/...must update only:while preserving every other host-controlled completion budget field:
If
CompletionOptionsalready contains:merge the effective
time_limitinto that dict rather than replacing the entire budget option.The model must never be able to alter other budget fields through
rlm_subagentarguments in this epic.Outer tool-handler deadline
The tool runtime already enforces
Schema.limits.time_limitaround the entire trusted handler.That outer limit must not remain fixed at 30 seconds if the host allows a 120/300-second model request, otherwise the outer tool wrapper will terminate the task before the inner completion budget can honor the requested timeout.
At registration time derive the tool-handler limit from host policy:
For example:
The model still normally gets 30 seconds because the inner completion budget is set to the default. The larger outer limit is only a safety envelope allowing legitimate host-approved overrides plus cleanup.
Do not use an infinite outer tool limit.
Enclosing parent deadline
A subagent is invoked from inside a parent operation that may itself have a smaller remaining wall-time budget.
The implementation must document and test this invariant:
If the runtime already exposes a reliable enclosing remaining deadline by implementation time, compute:
and record the parent-bound reduction explicitly in trace/envelope metadata.
If no reliable remaining-deadline API exists yet, do not invent one inside
rlm_subagent. Preserve the existing parentcall_with_time_limitbehavior and document that the enclosing completion may terminate first. File a separate generic deadline-propagation issue only if live source review demonstrates it is needed across multiple operations.This epic must not grow into an
rlm_asyncredesign merely to expose an optional subagent timeout.Result / trace observability
Successful or failed subagent envelopes should expose enough non-secret policy metadata to explain timeout behavior without leaking host internals.
Preferred addition:
Do not expose callable handlers or unrelated host configuration.
Timeout completion failures should retain the existing structured completion error:
and the subagent envelope should remain:
Avoid translating the same timeout through several incompatible error vocabularies.
Exact implementation files
1.
prolog/rlm_subagent.plrlm_subagent_register/7Change the generated
rlm_subagentschema:queryrequired;timeout_secondsnumeric property;limits.time_limitfrom host maximum + cleanup grace rather than hard-coded30.0;The model must still see this as the same normal
rlm_subagenttool.rlm_subagent_handler/...Extend the trusted handler closure/arity as needed to receive the normalized host timeout policy.
Before
agent_spawn/5:rlm_completion.Add private helpers with single ownership, conceptually:
Exact names may follow module conventions, but timeout parsing/merging must not be duplicated across tests or callers.
subagent_after_spawn/...Pass the effective completion options rather than the untouched registration options.
subagent_completion_envelope/...Add timeout provenance/effective value to the envelope if this can be done without breaking the existing result shape contract. If backwards compatibility requires avoiding new required keys, add it as an additional optional dict key.
2.
test/rlm_subagent_test.plAdd the full regression matrix described below.
3.
docs/agent-runtime.mdAdd a dedicated subsection under the model-visible subagent/tool discussion:
Document:
Include JSON examples with and without
timeout_seconds.4.
docs/completion-runtime.mdClarify that
completion_budget.time_limitis the execution wall-time used by subagent task policy after host validation, and that model-facing tools must not directly mutate arbitrary completion-budget fields.5.
docs/async-runtime.mdAdd one explicit cross-reference in
### Timeout behavior:Do not change
rlm_future_await/3semantics in this epic.6. Research/design artifact
Create:
This is currently the next unused research number after
RLM-RESEARCH-025-lem-ui.orgon main.Required sections:
The note must explicitly distinguish:
and record why this slice extends the existing
rlm_subagentinterface rather than creating a new task subsystem.Required tests
Add at least these tests to
test/rlm_subagent_test.pl:subagent_timeout_omitted_uses_host_defaultquery;source:defaultand expected effective value.subagent_timeout_explicit_override_is_honoredsubagent_timeout_above_host_max_is_rejected_before_spawnsubagent_timeout_zero_is_rejected_before_spawnsubagent_timeout_negative_is_rejected_before_spawnsubagent_timeout_non_number_is_schema_rejectedsubagent_timeout_does_not_replace_other_completion_budget_fieldssubagent_default_timeout_does_not_require_model_field[query].subagent_schema_exposes_optional_timeout_secondsouter_tool_limit_covers_host_maximum_plus_gracesubagent_completion_timeout_returns_structured_failed_envelopestatus:failedanderror.kind == timeout(within existing completion error envelope).timeout_request_does_not_widen_capabilitiestimeout_policy_is_host_ownedbudgetkeys because schema hasadditional_properties:false.sync_tool_timeout_does_not_depend_on_future_await_timeoutrlm_future_await/3waiter behavior.subagent_timeout_cleanup_does_not_leave_running_child_workBackwards compatibility
{query:...}remain valid.CompletionOptionsremain accepted.rlm_toolschema/result enforcement is unchanged.rlm_future_await/3behavior is unchanged.subagent_resultfields remain available; timeout metadata should be additive.Non-goals
rlm_async;infinite;rlm_toolschema/handler interface.Acceptance gate
This epic is complete when:
rlm_subagentremains the single normal model-facing delegation tool;completion_budget.time_limitis changed without mutating other host budget fields;research/RLM-RESEARCH-026-task-deadlines.orgrecords the design rationale and rejected alternatives;Implementation agents must inspect live main before editing and adjust exact helper names if main has changed, but must preserve this contract rather than inventing a parallel task subsystem.
RAGE reconciliation — deadline policy vs current
mainLive re-audit against canonical
main49d86f88730b4a0f5dc4a7cf300b8f09ef5845b0found that PR #212 is now a diverged transaction, not a merge candidate. Its head0109d25e2b22190d6d3339528f76c04748462c45is two commits ahead of the old merge base36cb418b833c77e24438fa94e596062a90088fa3and one commit behind currentmain.The conflict is semantic, not just textual: #172 landed typed/compiler-authenticated delegation policy in the same
rlm_subagent.plpath. Current main now includesrlm_subagent_register_command/8,prompt_command_subagent_options/3, authenticatedsubagent_delegation_source(...), fingerprint/prompt-id validation, uniqueness checks for explicit skills, and preservation of delegation source in child/result provenance. #212's deadline implementation predates those contracts and cannot replace them.Correct realization boundary
Port the existing deadline semantics into the current #172 subagent path, preserving all of these current-main invariants:
The model-visible schema may add optional
timeout_seconds, but command-authenticated role/skill/source policy must continue to come from the trusted compiler/host path, not model arguments.CI evidence
Exact #212 head
0109d25e...currently has REAL OpenRouter, Paid OpenRouter, Nix, clean SWI pack and Tree-sitter green, but the deterministic PlUnit job fails; all subsequent deterministic gates are skipped. That is a real blocker, not desired TDD state. Negative timeout/rejection contracts should assert the expected structured failure while the suite remains green; no intentional-red/xfail/skip convention.Decision
GO to reconcile/port #212 onto current main, HOLD merge. Do not merge or overwrite current #172 semantics. Re-run focused subagent tests and the complete exact-head repository gate after the semantic port; any new head invalidates the old green provider/Nix evidence.
RAGE update — deterministic regression isolated against post-#216 main
Canonical
mainis now267697bef10a3fffff7c093e1435ece770e7444b. Re-reading #175 and exact stale #212 head0109d25e2b22190d6d3339528f76c04748462c45isolated the deterministic failure precisely:test/rlm_subagent_test.pl:365 child_completion_enforces_wall_time_budgetexpectserror.kind == timeout, but getscapability_denied.This is not capability flakiness. The stale timeout implementation rewrites an existing
budget(completion_budget{time_limit:0.02})to the dedicated timeout-policy default (30.0). The slow planner therefore survives the old 20ms wall-time expectation, reaches its deliberateunusedtool plan, and correctly fails capability authorization. #175's current design confirms that dedicatedsubagent_timeout_default/max/gracepolicy owns subagent task lifetime and only rewritesbudget.time_limit; other budget fields must remain preserved. Therefore the old generic wall-time regression must be migrated to set a tiny hostsubagent_timeout_defaultwhen testing subagent task timeout. Capability checks must not be weakened.Analyze / research
rlm_subagent_register_command/8, trusted command-derived role/skill/source, delegation-source fingerprint/prompt-id validation, uniqueness checks, and child/result provenance.rlm_subagent.plwholesale.await timeout != task execution timeout != cancellationremains the vocabulary boundary.Design
Recover #175 from current main on a new issue-scoped branch rather than force-moving the shared stale #212 history. Port only the deadline policy into the current #172 path:
timeout_secondsin the existingrlm_subagentschema;budget.time_limit, preserving every other budget field;Adversarial review
time_limitmust survive;Decision
GO for a semantic recovery branch from exact
267697be…; keep stale #212 as preserved regression/history evidence and supersede it only after the current-main port has its own focused + full exact-head green evidence.RAGE adversarial gate — HOLD current #221 head
Re-audited #175 against current
main267697bef10a3fffff7c093e1435ece770e7444b, PR #221 exact head134a87d1654ce79ce6520eba1d32cd6229ffb07e, its patch, downstream ownership, and exact-head CI.What is good
rlm_subagent_register_command/8path and role/skill/source provenance rather than replacing it with stale #212 code.timeout_seconds.budget.time_limitis preserved as the legacy default when no dedicated subagent default exists, fixing the deterministic #212 precedence failure without weakening capability checks.Adversarial blocker
The current green head does not yet satisfy #175's declared acceptance contract. #175 explicitly requires the full regression matrix and public runtime docs. #221 currently changes only:
prolog/rlm_subagent.pltest/rlm_subagent_test.plresearch/RLM-RESEARCH-026-task-deadlines.orgThe branch adds coverage for schema exposure, outer max+grace, omitted default, explicit override, and over-max pre-spawn rejection, while existing tests cover some generic timeout/capability/cancellation behavior. But the issue still explicitly requires focused contracts for zero/negative/schema-invalid input, preservation of non-time budget fields, host-owned rejection of max/default/grace/budget model fields, structured completion-timeout envelope, timeout-with-capability denial, Future-await independence, and cleanup/no-running-child semantics.
It also explicitly requires updates to
docs/agent-runtime.md,docs/completion-runtime.md, anddocs/async-runtime.md. None are present in the current PR, despite this being a public behavior change.Decision
HOLD promotion/merge of
134a87d1…. Keep #221 draft. Do not weaken the issue checklist because the aggregate suite is green. Add the missing deterministic contracts and required docs, then invalidate this exact-head evidence and rerun the complete gate on the new SHA.The stale CI report #220 for test-only head
f4902c32…has been closed because the branch's current head resolves that specific reported failure.RAGE regression evidence on #221 exact head
9bf2431337d2604e4ad95ed2dc5a5d1464df9560:The newly registered deadline acceptance suite made canonical deterministic CI fail 2/900 tests. Both failures are zero/negative
timeout_secondscases. Therlm_subagentschema declarestimeout_seconds: {type:number, exclusiveMinimum:0}, but the genericrlm_toolschema validator currently checks onlynumber(Value)and ignoresexclusiveMinimum. As a result, zero/negative values cross the schema boundary and are rejected later by timeout policy as a successful tool execution carryingsubagent_result{status:failed,error.kind:invalid_timeout,child:none}instead of a schema-level invocation rejection.Analyze/research:
validate_schema_definition/1accepts numeric schemas;validate_schema_value/3 -> validate_type/4enforces only the base type. This is a generic tool-schema correctness gap, not a subagent-only design problem. Non-number and closed-schema host-option rejection are already behaving correctly.Design: teach the canonical
rlm_toolvalidator the smallest required numeric keyword,exclusiveMinimum, including schema-definition validation (numeric bound, only valid for integer/number schemas) and runtime enforcement after the base numeric type check. Keep the #175 tests unchanged; zero/negative must be rejected before capability/authority/child creation. Do not move this intorlm_subagentpreflight or accept the later failed envelope, because that would leave the declared schema contract false.Adversarial review: this narrows accepted model data only; it cannot widen capability/authority/effect semantics. It also applies uniformly to edited operations because both invocation and edit validation use
validate_schema/4. Invalid schema definitions should fail at registration rather than silently advertise unsupported constraints.Decision: GO on the generic validator fix within the existing #221 transaction. The branch stays draft/red until the unchanged deadline regressions and full exact-head gate pass.
RAGE design correction after reconciling the exact CI evidence with this issue's acceptance text:
My prior note over-constrained cases 4/5. #175 says zero and negative requests must be rejected before spawn, with the suggested failure
subagent_error{phase:timeout_policy,kind:invalid_timeout,...}. Exact head9bf24313...returned precisely that structured failedsubagent_result, withcorrelation.child == none; it did not create a child. Only case 6 explicitly requires schema rejection for a non-number, and that case passes.So the two CI failures are test-contract bugs in the newly added acceptance helper, not a #175 runtime failure. Per RAGE semantics I am not patching production around a disproven test assumption. I will change only the zero/negative assertions to require the exact structured timeout-policy failure plus no child creation. The non-number and host-only closed-schema cases continue to require tool/schema rejection.
Separate finding retained for later backlog work: generic
rlm_toolcurrently advertises but does not enforce numericexclusiveMinimumat invocation validation. That is a real generic schema-feature gap, but #175 already has an explicit host-side positive-number guard and does not require zero/negative to fail specifically at schema phase. It should not be smuggled into this deadline PR unless separately scoped/TDD'd.Exact-head verification / promotion update:
Candidate
1dc3fc0f4f8b4e6dd51819e744e57af0cb70c63dis now green on every returned required workflow. Canonical deterministic CI reports 79 suites / 900 discovered / 900 planned / 900 completed / 900 passed, with 0 failed, timeout, blocked, or fixme. Deterministic benchmark/conformance is 16/16; deep recursion is 15/15. Credential-backed REAL OpenRouter passed core, structured repair, benchmark, depth 0/1/2 and CLI smoke. Paid OpenRouter, Nix flake, clean SWI-pack install, and Tree-sitter FFI are also successful.The deadline-specific acceptance matrix and all three required public docs are now present. PR #221 is based directly on current
main(behind_by:0), GitHub reports it mergeable, and it has no reviews/comments/unresolved threads. Adversarial decision is now GO; PR #221 has been promoted from draft to ready-for-review.I am not merging in this step because the repository-level
AGENTS.mdmerge rule still requires explicit merge-on-green authorization for the transaction.