Own server lifecycle and expose truthful readiness #103

Merged

lost-rob0t merged 6 commits from agent/runtime-lifecycle-readiness into master

2026-08-17 05:31:39 +00:00

lost-rob0t commented

2026-08-17 05:27:30 +00:00

(Migrated from github.com)

Purpose

Start the production-runtime hardening work from canonical master for issues #37 and #42 without overlapping the target-lease/fencing work in #101.

What changed

replace server/handler's bt:all-threads join with one explicit StarIntel runtime control loop;
add a runtime object that owns the lparallel kernel, actor system, Rabbit consumers, event consumer, and Clack server handle;
start HTTP last and roll back already-started components on partial startup failure;
install SIGTERM/SIGINT handlers that request shutdown on the owning main loop instead of performing complex work in the signal handler;
stop HTTP intake first, then Rabbit consumers, actors/timers/producer, and finally lparallel;
retain and stop the Clack server handle instead of discarding it;
remove the unused HTTP-owned Rabbit connection from startup ownership: publication remains actor-owned;
make Rabbit consumer reads finite (250 ms idle poll) so owner threads can observe :stopping, close their own streams, and be joined without violating the stream-owner invariant;
join only consumer-owned thread handles, never arbitrary Lisp process threads;
retain the actor GC timer and add bounded actor/timer/producer shutdown;
add public /live and /ready endpoints and make /health use readiness semantics rather than unconditional OK;
readiness fails closed on CouchDB probe failure, Rabbit producer failure, stopped consumer workers, missing actor system/kernel, or missing HTTP server handle.

Scope boundary

This is a bounded first slice of #37/#42, not a claim that those issues are fully complete. It does not implement reconnect/restart supervision, component generations/opaque handles, full pool teardown, target lease fencing, or target execution safety. Those remain separate production-hardening work.

PR #101 remains authoritative for target lease API/fencing work and is intentionally not modified here.

Tests added

The mandatory hermetic suite now verifies:

stopping a consumer joins only its retained owned thread and leaves an unrelated Lisp thread alive;
readiness is true only when required runtime components and dependency probes are healthy;
CouchDB or Rabbit probe failure makes readiness fail closed;
liveness tracks runtime lifecycle state;
/health, /live, and /ready are public operational endpoints.

CI is the authoritative execution environment for the full Common Lisp/Nix/container validation.

Refs #37
Refs #42

## Purpose Start the production-runtime hardening work from canonical `master` for issues #37 and #42 without overlapping the target-lease/fencing work in #101. ## What changed - replace `server/handler`'s `bt:all-threads` join with one explicit StarIntel runtime control loop; - add a runtime object that owns the lparallel kernel, actor system, Rabbit consumers, event consumer, and Clack server handle; - start HTTP last and roll back already-started components on partial startup failure; - install SIGTERM/SIGINT handlers that request shutdown on the owning main loop instead of performing complex work in the signal handler; - stop HTTP intake first, then Rabbit consumers, actors/timers/producer, and finally lparallel; - retain and stop the Clack server handle instead of discarding it; - remove the unused HTTP-owned Rabbit connection from startup ownership: publication remains actor-owned; - make Rabbit consumer reads finite (250 ms idle poll) so owner threads can observe `:stopping`, close their own streams, and be joined without violating the stream-owner invariant; - join only consumer-owned thread handles, never arbitrary Lisp process threads; - retain the actor GC timer and add bounded actor/timer/producer shutdown; - add public `/live` and `/ready` endpoints and make `/health` use readiness semantics rather than unconditional OK; - readiness fails closed on CouchDB probe failure, Rabbit producer failure, stopped consumer workers, missing actor system/kernel, or missing HTTP server handle. ## Scope boundary This is a bounded first slice of #37/#42, not a claim that those issues are fully complete. It does not implement reconnect/restart supervision, component generations/opaque handles, full pool teardown, target lease fencing, or target execution safety. Those remain separate production-hardening work. PR #101 remains authoritative for target lease API/fencing work and is intentionally not modified here. ## Tests added The mandatory hermetic suite now verifies: - stopping a consumer joins only its retained owned thread and leaves an unrelated Lisp thread alive; - readiness is true only when required runtime components and dependency probes are healthy; - CouchDB or Rabbit probe failure makes readiness fail closed; - liveness tracks runtime lifecycle state; - `/health`, `/live`, and `/ready` are public operational endpoints. CI is the authoritative execution environment for the full Common Lisp/Nix/container validation. Refs #37 Refs #42