[P2-01] Integrate resumable auto-research and analysis jobs as a bounded service subsystem #49

Open
opened 2026-07-22 00:20:46 +00:00 by lost-rob0t · 0 comments
lost-rob0t commented 2026-07-22 00:20:46 +00:00 (Migrated from github.com)

Objective

Turn the auto-research design corpus into an executable, resumable job model inside the importable library/service architecture without making the server recursively load the research repository or run arbitrary agent code.

Required job model

  • job id, kind, owner principal, tenant/dataset, target/case scope
  • immutable input manifest and normalized parameters
  • actor/component plan and required capabilities
  • lifecycle: queued, leasing, running, waiting, retrying, paused, completed, failed, cancelled
  • checkpoint/version, current step, completed steps, partial outputs, errors, attempt history
  • target lease ids/fencing tokens
  • trace/provenance, evidence references, budgets, deadlines, and cancellation

Runtime architecture

  • Research control actor on :platform/research control capacity.
  • Bounded router pools per task class, not one thread or actor system per job.
  • Keyed routing for stateful jobs by job id/target.
  • Persistence-backed checkpoints and an outbox for emitted documents/events.
  • Resume after process restart without repeating accepted side effects.
  • Admission control for concurrent jobs, tool calls, tokens, documents, elapsed time, and storage.
  • Jobs call declared service/component handles; they do not access raw actors, sockets, databases, or credentials.

HTTP/library API

Create authenticated operations for create, inspect, list, pause, resume, cancel, retry failed step, and stream/read events/results. Enforce dataset/target/program authorization and leases.

Acceptance tests

  • A multi-step fixture job survives restart and resumes at the correct checkpoint.
  • Cancellation stops future side effects and releases/marks leases.
  • Partial failures remain inspectable and retryable.
  • Concurrent job count and total worker/thread use remain within budgets.
  • Replayed steps do not duplicate accepted documents or events.
## Objective Turn the auto-research design corpus into an executable, resumable job model inside the importable library/service architecture without making the server recursively load the research repository or run arbitrary agent code. ## Required job model - job id, kind, owner principal, tenant/dataset, target/case scope - immutable input manifest and normalized parameters - actor/component plan and required capabilities - lifecycle: queued, leasing, running, waiting, retrying, paused, completed, failed, cancelled - checkpoint/version, current step, completed steps, partial outputs, errors, attempt history - target lease ids/fencing tokens - trace/provenance, evidence references, budgets, deadlines, and cancellation ## Runtime architecture - Research control actor on `:platform/research` control capacity. - Bounded router pools per task class, not one thread or actor system per job. - Keyed routing for stateful jobs by job id/target. - Persistence-backed checkpoints and an outbox for emitted documents/events. - Resume after process restart without repeating accepted side effects. - Admission control for concurrent jobs, tool calls, tokens, documents, elapsed time, and storage. - Jobs call declared service/component handles; they do not access raw actors, sockets, databases, or credentials. ## HTTP/library API Create authenticated operations for create, inspect, list, pause, resume, cancel, retry failed step, and stream/read events/results. Enforce dataset/target/program authorization and leases. ## Acceptance tests - A multi-step fixture job survives restart and resumes at the correct checkpoint. - Cancellation stops future side effects and releases/marks leases. - Partial failures remain inspectable and retryable. - Concurrent job count and total worker/thread use remain within budgets. - Replayed steps do not duplicate accepted documents or events.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
nsaspy/starintel-server#49
No description provided.