[P2] Preserve nested model events and true depths in completion trajectories #45

New issue

Closed

opened 2026-08-13 07:08:22 +00:00 by lost-rob0t · 0 comments

lost-rob0t commented

2026-08-13 07:08:22 +00:00

(Migrated from github.com)

Problem

Depth >1 execution now has correct usage accounting through the plan-level model_responses ledger, but rlm_completion trajectory construction still derives model events from the top-level plan plus top-level Result.vars.

For nested depth >1 trees, child lexical vars are restored on return. Intermediate model responses can therefore be absent from completion_trajectory.events, and a final nested response can be represented at a shallower depth than where the provider call actually occurred.

This is an observability problem rather than an execution/budget problem: #20's recursion stats and usage accounting remain correct, but the trajectory should eventually be a faithful call tree.

Desired behavior

record one event for every executed model response, including nested plans and retries;
preserve true depth and stable parent/child identity;
keep provider, selected model, HTTP status, reason, and usage per event;
retain deterministic ordering suitable for trace export;
avoid reconstructing nested events from lexical result vars after execution.

Suggested direction

Extend the append-only execution ledger/event stream in rlm_plan (or a closely related structured execution record) so trajectory construction consumes authoritative execution events rather than inferring them from final variable scope.

Discovered by

PR #43 / issue #20 depth >1 observability audit.

## Problem Depth >1 execution now has correct usage accounting through the plan-level `model_responses` ledger, but `rlm_completion` trajectory construction still derives model events from the top-level plan plus top-level `Result.vars`. For nested depth >1 trees, child lexical vars are restored on return. Intermediate model responses can therefore be absent from `completion_trajectory.events`, and a final nested response can be represented at a shallower depth than where the provider call actually occurred. This is an observability problem rather than an execution/budget problem: #20's recursion stats and usage accounting remain correct, but the trajectory should eventually be a faithful call tree. ## Desired behavior - record one event for every executed model response, including nested plans and retries; - preserve true depth and stable parent/child identity; - keep provider, selected model, HTTP status, reason, and usage per event; - retain deterministic ordering suitable for trace export; - avoid reconstructing nested events from lexical result vars after execution. ## Suggested direction Extend the append-only execution ledger/event stream in `rlm_plan` (or a closely related structured execution record) so trajectory construction consumes authoritative execution events rather than inferring them from final variable scope. ## Discovered by PR #43 / issue #20 depth >1 observability audit.