Adopt Pykka actors with ZeroMQ transport #45

Merged

lost-rob0t merged 3 commits from agent/issue-3-zmq into main

2026-07-18 02:09:51 +00:00

lost-rob0t commented

2026-07-18 01:52:29 +00:00

(Migrated from github.com)

What changed

Reopened issue #3 and corrected the merged concurrency design before implementation begins.
Replaced the authoritative AnyIO actor-bus design with Pykka ThreadingActor components and PyZMQ/ZeroMQ transport.
Added ADR-0006, which supersedes ADR-0002 and amends the runtime toolchain decision.
Updated the ADR index to preserve the older decision as superseded history.
Defined Pykka as the actor lifecycle/state framework and ZeroMQ as the bounded data-plane and owner-only local IPC transport.
Defined inproc:// for raw/redacted internal pipeline traffic and ipc:// for local control/query clients.
Explicitly prohibited RabbitMQ, Kafka, persistent message queues, filesystem queue fallbacks, pickle, and default TCP listeners.
Added socket ownership, framing, HWM, application-credit, timeout, finite-retry, IMMEDIATE, and zero-linger requirements.
Added a bounded Pykka/ZeroMQ pump pattern so ZeroMQ receivers cannot drain into an unbounded actor inbox.
Updated supervision, cancellation, overload, failure, source-layout, traceability, and test requirements for Pykka and ZeroMQ.

Architecture split

Pykka

Pykka provides the fixed set of Python actors, actor-owned state, lifecycle hooks, actor references, request/reply futures, registry support, and low-volume control/supervision messages.

ZeroMQ

ZeroMQ provides typed stage transport and local IPC:

inproc:// for raw and redacted pipeline data inside the daemon;
ipc:// for owner-only CLI, status, and query traffic;
no tcp:// listener by default;
no persistent broker or message spool.

Pykka inboxes are not used as an unbounded bulk-data queue. Each data edge uses a bounded number of pump/worker actors, socket high-water marks, application credits, deadlines, and typed overload results.

Security impact

Raw pixels and OCR cannot cross an ipc:// or tcp:// endpoint.
Every ZeroMQ socket is owned by exactly one actor thread.
Saturation cannot trigger an alternate queue, file, broker, infinite retry, or false success.
PUB/SUB is advisory only and cannot carry authoritative lifecycle commands.
Pykka stop and socket closure accelerate cancellation, while final generation validation remains the authoritative stale-write barrier.
Actor or required transport failure faults capture rather than widening permissions.

Validation

Compared the branch directly against merged main.
The correction changes only docs/architecture.md, the ADR index, and new ADR-0006.
Confirmed the authoritative architecture points to ADR-0006 and states that ADR-0002 is superseded.
Confirmed issue #3 now explicitly requires Pykka, ZeroMQ, inproc:// raw transport, owner-only ipc://, and no RabbitMQ or AnyIO actor bus.
No runtime implementation was started under the superseded design.

Closes #3

## What changed - Reopened issue #3 and corrected the merged concurrency design before implementation begins. - Replaced the authoritative AnyIO actor-bus design with Pykka `ThreadingActor` components and PyZMQ/ZeroMQ transport. - Added ADR-0006, which supersedes ADR-0002 and amends the runtime toolchain decision. - Updated the ADR index to preserve the older decision as superseded history. - Defined Pykka as the actor lifecycle/state framework and ZeroMQ as the bounded data-plane and owner-only local IPC transport. - Defined `inproc://` for raw/redacted internal pipeline traffic and `ipc://` for local control/query clients. - Explicitly prohibited RabbitMQ, Kafka, persistent message queues, filesystem queue fallbacks, pickle, and default TCP listeners. - Added socket ownership, framing, HWM, application-credit, timeout, finite-retry, `IMMEDIATE`, and zero-linger requirements. - Added a bounded Pykka/ZeroMQ pump pattern so ZeroMQ receivers cannot drain into an unbounded actor inbox. - Updated supervision, cancellation, overload, failure, source-layout, traceability, and test requirements for Pykka and ZeroMQ. ## Architecture split ### Pykka Pykka provides the fixed set of Python actors, actor-owned state, lifecycle hooks, actor references, request/reply futures, registry support, and low-volume control/supervision messages. ### ZeroMQ ZeroMQ provides typed stage transport and local IPC: - `inproc://` for raw and redacted pipeline data inside the daemon; - `ipc://` for owner-only CLI, status, and query traffic; - no `tcp://` listener by default; - no persistent broker or message spool. Pykka inboxes are not used as an unbounded bulk-data queue. Each data edge uses a bounded number of pump/worker actors, socket high-water marks, application credits, deadlines, and typed overload results. ## Security impact - Raw pixels and OCR cannot cross an `ipc://` or `tcp://` endpoint. - Every ZeroMQ socket is owned by exactly one actor thread. - Saturation cannot trigger an alternate queue, file, broker, infinite retry, or false success. - PUB/SUB is advisory only and cannot carry authoritative lifecycle commands. - Pykka stop and socket closure accelerate cancellation, while final generation validation remains the authoritative stale-write barrier. - Actor or required transport failure faults capture rather than widening permissions. ## Validation - Compared the branch directly against merged `main`. - The correction changes only `docs/architecture.md`, the ADR index, and new ADR-0006. - Confirmed the authoritative architecture points to ADR-0006 and states that ADR-0002 is superseded. - Confirmed issue #3 now explicitly requires Pykka, ZeroMQ, `inproc://` raw transport, owner-only `ipc://`, and no RabbitMQ or AnyIO actor bus. - No runtime implementation was started under the superseded design. Closes #3