[GH-#2332] auto-dig service: crash after successful run-branch push left a green run recorded as failed (fixed on test branch, needs landing) #8

New issue

Open

opened 2026-09-08 16:59:52 +00:00 by nsaspy · 0 comments

nsaspy commented

2026-09-08 16:59:52 +00:00

Owner

Mirrored from GitHub https://github.com/lost-rob0t/starintel-gpt-auto-dig/issues/2332 (GitHub is authoritative for this item).

Summary

A live harness test (tmp worktree test/gateway-llm-run, run svc-20260906T235450Z) exposed a service-runner bug: the pass crashed after a successful run-branch push, so a fully successful research run was recorded as failed and durable state never advanced.

Failure

Sequence in agents/auto_dig_service.py run_one_pass:

research completed (8 model calls, report validated, exit 0);
push_run_branch created the run branch, committed the artifacts, pushed, and switched the worktree back to the original branch — which removes the run directory from the worktree;
the post-push manifest update update_run_manifest(run_dir, {"status": "pushed", ...}) then raised [Errno 2] No such file or directory: '.../run.json';
the pass-level handler recorded the pass as failed; state was not advanced and no receipt was posted.

Evidence branch (pushed before the crash, research is intact): auto-dig-prolog/2026-09-06-svc-20260906T235450Z-1-1a7fed (a072ac97).

Fix

Landed on the test branch:

no run-directory writes after the branch push: the manifest records everything knowable pre-push; pushed/receipt state is durably recorded in the actor state and the issue receipt instead;
push_run_branch restores the original branch in a finally block so a push failure cannot wedge the worktree for subsequent passes;
the sanitized failure report is always written to disk, even without a Forgejo client.

Verification after the fix (run svc-20260906T235958Z-1-ace520): exit 0, report validated, branch pushed (b767f5a), state advanced correctly, drain finished: 1 passes (0 failed).

Follow-ups tracked here

The fixes live on the tmp test branch and an uncommitted main checkout; they must be landed on main together with the new standalone service runner and gateway provider support.
Companion failure record: the llm.starintel.actor gateway run failed in the provider stage — blocked by starintel-labs/starintel-infra#115 and starintel-labs/starintel-infra#116; the OpenRouter path is green, ruling the harness out as the cause.
Prolog-RLM runtime issue for the other observed failure: nsaspy/prolog-rlm#381 (out-of-range context(slice) selector is terminal).

Mirrored from GitHub https://github.com/lost-rob0t/starintel-gpt-auto-dig/issues/2332 (GitHub is authoritative for this item). --- ## Summary A live harness test (tmp worktree `test/gateway-llm-run`, run `svc-20260906T235450Z`) exposed a service-runner bug: **the pass crashed after a successful run-branch push**, so a fully successful research run was recorded as failed and durable state never advanced. ## Failure Sequence in `agents/auto_dig_service.py` `run_one_pass`: 1. research completed (8 model calls, report validated, exit 0); 2. `push_run_branch` created the run branch, committed the artifacts, pushed, and switched the worktree back to the original branch — which removes the run directory from the worktree; 3. the post-push manifest update `update_run_manifest(run_dir, {"status": "pushed", ...})` then raised `[Errno 2] No such file or directory: '.../run.json'`; 4. the pass-level handler recorded the pass as failed; state was not advanced and no receipt was posted. Evidence branch (pushed before the crash, research is intact): `auto-dig-prolog/2026-09-06-svc-20260906T235450Z-1-1a7fed` (`a072ac97`). ## Fix Landed on the test branch: - no run-directory writes after the branch push: the manifest records everything knowable pre-push; pushed/receipt state is durably recorded in the actor state and the issue receipt instead; - `push_run_branch` restores the original branch in a `finally` block so a push failure cannot wedge the worktree for subsequent passes; - the sanitized failure report is always written to disk, even without a Forgejo client. Verification after the fix (run `svc-20260906T235958Z-1-ace520`): exit 0, report validated, branch pushed (`b767f5a`), state advanced correctly, `drain finished: 1 passes (0 failed)`. ## Follow-ups tracked here - The fixes live on the tmp test branch and an uncommitted main checkout; they must be landed on `main` together with the new standalone service runner and gateway provider support. - Companion failure record: the llm.starintel.actor gateway run failed in the provider stage — blocked by starintel-labs/starintel-infra#115 and starintel-labs/starintel-infra#116; the OpenRouter path is green, ruling the harness out as the cause. - Prolog-RLM runtime issue for the other observed failure: nsaspy/prolog-rlm#381 (out-of-range `context(slice)` selector is terminal).

nsaspy added the

github-mirror

label

2026-09-08 16:59:52 +00:00

No labels

github-mirror

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starintel-labs/starintel-gpt-auto-dig#8

No description provided.

Rows
Columns