RED: seed Tek9/Common Lisp/Prolog expert runtime #18

Merged

lost-rob0t merged 120 commits from rage/10-expert-runtime-red into main

2026-08-30 04:51:43 +00:00

lost-rob0t commented

2026-08-29 12:52:54 +00:00

(Migrated from github.com)

Starts #10 under #8 and consumes the zero-Python correction in #20.

Canonical research/design

research/LLM-LOG-RESEARCH-004-symbolic-expert-plane.org
research/LLM-LOG-RESEARCH-005-persistent-prolog-supervisor.org
research/LLM-LOG-RESEARCH-006-runtime-reconciliation.org
research/LLM-LOG-RESEARCH-007-expert-service-transaction.org
research/LLM-LOG-RESEARCH-008-reasoner-crash-recovery.org
research/LLM-LOG-RESEARCH-009-reasoner-timeout.org
research/LLM-LOG-RESEARCH-010-reasoner-result-validation.org
research/LLM-LOG-RESEARCH-011-zero-python-rewrite.org
research/LLM-LOG-RESEARCH-012-common-lisp-transport-red.org
research/LLM-LOG-RESEARCH-013-common-lisp-red-runner.org
research/LLM-LOG-RESEARCH-014-hosted-common-lisp-red-execution.org
research/LLM-LOG-RESEARCH-015-http-fidelity-black-box-red.org
research/LLM-LOG-RESEARCH-016-http-bootstrap-green.org
research/LLM-LOG-RESEARCH-017-immutable-http-capture-redaction.org
research/LLM-LOG-RESEARCH-018-single-writer-recorder-crash-contract.org

Implementation/test evidence is kept separately under evidence/.

Architecture

Common Lisp owns the accepted llm-log runtime: transparent HTTP/SSE/WebSocket proxying, provider routing, capture/redaction, recorder/configuration, typed expert orchestration, Tek9 lifecycle/bounded retrieval, SWI-Prolog supervision and result validation.
Tek9 is the canonical durable document/graph KB for symbolic state/provenance.
SWI-Prolog is the sole facts/rules/inference engine.
Python owns nothing in final acceptance. Existing Python implementation/tests/package wiring are migration debt until equivalent Common Lisp gates are GREEN and can be removed safely.
Raw capture evidence remains immutable and separate from derived assertions.

Existing expert substrate

Retained #10 behavior includes persistent supervised SWI-Prolog, closed named operations, Tek9 event projections/KB revision, idempotent replay with explicit contradiction conflicts, bounded point retrieval, crash/timeout/malformed-result handling, and provenance-bearing typed results. No Tek9 API change is currently justified.

RED → GREEN ordinary HTTP

Authoritative pre-implementation RED:

head 0359d92e4e637d888079ddf55e72d2c9124de81a
hosted run 33282707089

The bounded Common Lisp HTTP implementation is GREEN for lifecycle, provider selector routing, method/path/query fidelity, end-to-end request headers, binary bodies with NUL, upstream status, and distinct duplicate Set-Cookie response fields.

Evidence: evidence/LLM-LOG-EVIDENCE-002-http-fidelity-green.org.

RED → GREEN immutable finite-HTTP capture/redaction

Canonical design: research/LLM-LOG-RESEARCH-017-immutable-http-capture-redaction.org.

Valid pre-implementation RED:

head 31426657f5bb68871e66a723cde33ce4a845a4bc
hosted run 33287657011

Implementation:

e3830e76a4713fa331c67e71739453247a9c8a01
create-only event artifacts beneath configured data directory
exact binary request/response body evidence
ordered request/response header evidence
persistence-only credential redaction while original credential values continue upstream
capture persistence failure is fail-open for forwarding in this bounded slice

GREEN:

hosted run 33287732659, job 99193780462
exact evidence-only head 1041c2267a8edc9a2a94f86bbdb79a3e289ca629 also GREEN in run 33287815155

Evidence: evidence/LLM-LOG-EVIDENCE-003-http-capture-red-green.org.

Recorder concurrency/crash RED

Canonical design: research/LLM-LOG-RESEARCH-018-single-writer-recorder-crash-contract.org.

Executed pre-implementation RED:

head b0773366f830424ce49ab955ab22860a49803dec
hosted run 33289740093, job 99199157484
gate: nix build -L .#checks.x86_64-linux.common-lisp-transport-contract

All prior HTTP and immutable-redaction contracts remained GREEN. The new recorder concurrency and crash contracts failed for the intended missing production boundary:

SIMPLE-ERROR: Missing recorder contract function LLM-LOG-EXPERT::START-RECORDER

Evidence: evidence/LLM-LOG-EVIDENCE-004-recorder-concurrency-crash-red.org.

No recorder production mutation followed this RED yet. The selected design requires a Sento/cl-gserver single-writer actor and atomic staging→committed publication. Reproducible Sento/cl-gserver consumption through the pinned Nix/ASDF build must be established before implementation; do not substitute ad-hoc shared locking merely to make the contract GREEN.

Still RED/not implemented

This PR remains draft. Current work does not claim:

recorder concurrency / single-writer actor GREEN
crash-consistent multi-file event commit GREEN
HTTPS/TLS provider upstreams
chunked/SSE streaming/backpressure and streaming capture
WebSocket forwarding/frame evidence
provider-family conformance
expert fail-open/fail-closed invocation on the new transport
Nix/Home Manager final service migration
zero-Python repository/package completion

lost-rob0t/dotfiles remains consumer-only and pinned to merged llm-log main with ~/Documents/AI/proxy; PR #5 remains a separate configuration transaction.

Do not merge until the exact final head is GREEN through the remaining #10 gates, required discussions/reviews are satisfied, and Python migration debt is removed.

Starts #10 under #8 and consumes the zero-Python correction in #20. ## Canonical research/design - `research/LLM-LOG-RESEARCH-004-symbolic-expert-plane.org` - `research/LLM-LOG-RESEARCH-005-persistent-prolog-supervisor.org` - `research/LLM-LOG-RESEARCH-006-runtime-reconciliation.org` - `research/LLM-LOG-RESEARCH-007-expert-service-transaction.org` - `research/LLM-LOG-RESEARCH-008-reasoner-crash-recovery.org` - `research/LLM-LOG-RESEARCH-009-reasoner-timeout.org` - `research/LLM-LOG-RESEARCH-010-reasoner-result-validation.org` - `research/LLM-LOG-RESEARCH-011-zero-python-rewrite.org` - `research/LLM-LOG-RESEARCH-012-common-lisp-transport-red.org` - `research/LLM-LOG-RESEARCH-013-common-lisp-red-runner.org` - `research/LLM-LOG-RESEARCH-014-hosted-common-lisp-red-execution.org` - `research/LLM-LOG-RESEARCH-015-http-fidelity-black-box-red.org` - `research/LLM-LOG-RESEARCH-016-http-bootstrap-green.org` - `research/LLM-LOG-RESEARCH-017-immutable-http-capture-redaction.org` - `research/LLM-LOG-RESEARCH-018-single-writer-recorder-crash-contract.org` Implementation/test evidence is kept separately under `evidence/`. ## Architecture - **Common Lisp** owns the accepted llm-log runtime: transparent HTTP/SSE/WebSocket proxying, provider routing, capture/redaction, recorder/configuration, typed expert orchestration, Tek9 lifecycle/bounded retrieval, SWI-Prolog supervision and result validation. - **Tek9** is the canonical durable document/graph KB for symbolic state/provenance. - **SWI-Prolog** is the sole facts/rules/inference engine. - **Python owns nothing in final acceptance.** Existing Python implementation/tests/package wiring are migration debt until equivalent Common Lisp gates are GREEN and can be removed safely. - Raw capture evidence remains immutable and separate from derived assertions. ## Existing expert substrate Retained #10 behavior includes persistent supervised SWI-Prolog, closed named operations, Tek9 event projections/KB revision, idempotent replay with explicit contradiction conflicts, bounded point retrieval, crash/timeout/malformed-result handling, and provenance-bearing typed results. No Tek9 API change is currently justified. ## RED → GREEN ordinary HTTP Authoritative pre-implementation RED: - head `0359d92e4e637d888079ddf55e72d2c9124de81a` - hosted run `33282707089` The bounded Common Lisp HTTP implementation is GREEN for lifecycle, provider selector routing, method/path/query fidelity, end-to-end request headers, binary bodies with NUL, upstream status, and distinct duplicate `Set-Cookie` response fields. Evidence: `evidence/LLM-LOG-EVIDENCE-002-http-fidelity-green.org`. ## RED → GREEN immutable finite-HTTP capture/redaction Canonical design: `research/LLM-LOG-RESEARCH-017-immutable-http-capture-redaction.org`. Valid pre-implementation RED: - head `31426657f5bb68871e66a723cde33ce4a845a4bc` - hosted run `33287657011` Implementation: - `e3830e76a4713fa331c67e71739453247a9c8a01` - create-only event artifacts beneath configured data directory - exact binary request/response body evidence - ordered request/response header evidence - persistence-only credential redaction while original credential values continue upstream - capture persistence failure is fail-open for forwarding in this bounded slice GREEN: - hosted run `33287732659`, job `99193780462` - exact evidence-only head `1041c2267a8edc9a2a94f86bbdb79a3e289ca629` also GREEN in run `33287815155` Evidence: `evidence/LLM-LOG-EVIDENCE-003-http-capture-red-green.org`. ## Recorder concurrency/crash RED Canonical design: `research/LLM-LOG-RESEARCH-018-single-writer-recorder-crash-contract.org`. Executed pre-implementation RED: - head `b0773366f830424ce49ab955ab22860a49803dec` - hosted run `33289740093`, job `99199157484` - gate: `nix build -L .#checks.x86_64-linux.common-lisp-transport-contract` All prior HTTP and immutable-redaction contracts remained GREEN. The new recorder concurrency and crash contracts failed for the intended missing production boundary: `SIMPLE-ERROR: Missing recorder contract function LLM-LOG-EXPERT::START-RECORDER` Evidence: `evidence/LLM-LOG-EVIDENCE-004-recorder-concurrency-crash-red.org`. No recorder production mutation followed this RED yet. The selected design requires a Sento/cl-gserver single-writer actor and atomic staging→committed publication. Reproducible Sento/cl-gserver consumption through the pinned Nix/ASDF build must be established before implementation; do not substitute ad-hoc shared locking merely to make the contract GREEN. ## Still RED/not implemented This PR remains **draft**. Current work does **not** claim: - recorder concurrency / single-writer actor GREEN - crash-consistent multi-file event commit GREEN - HTTPS/TLS provider upstreams - chunked/SSE streaming/backpressure and streaming capture - WebSocket forwarding/frame evidence - provider-family conformance - expert fail-open/fail-closed invocation on the new transport - Nix/Home Manager final service migration - zero-Python repository/package completion `lost-rob0t/dotfiles` remains consumer-only and pinned to merged llm-log main with `~/Documents/AI/proxy`; PR #5 remains a separate configuration transaction. Do not merge until the exact final head is GREEN through the remaining #10 gates, required discussions/reviews are satisfied, and Python migration debt is removed.

lost-rob0t commented

2026-08-29 13:12:22 +00:00

(Migrated from github.com)

Next bounded slice landed on this existing transaction:

26df268 implements llm_log.expert_protocol; the exact tests/test_expert_protocol.py contract was reproduced locally with Python 3.13 and is 6/6 green.
e886999 documents the Common Lisp expert-runtime ownership boundary under expert/README.org.
1b194a4, 2da0602, 40b5f54 establish the llm-log-expert ASDF/package/Tek9 lifecycle skeleton. start-expert-host owns one reusable Tek9 environment; no per-query Prolog process behavior was introduced.
Upstream Tek9 packaging gap is now tracked as lost-rob0t/tek9#7: its current flake has a dev shell but no consumable package output, so llm-log will not assume a developer Quicklisp checkout.

Truthful gate state: Python protocol contract is green. The black-box CL/Tek9/SWI service contract remains intentionally RED/unexecuted here because this execution environment has Python but no SBCL, SWI-Prolog, Nix, or network-resolvable checkout. Do not merge yet.

Next implementation slice: persistent supervised SWI-Prolog session + named JSON-lines operation dispatcher, then llm-log-expert executable/Nix wiring once Tek9 package consumption is available.

Next bounded slice landed on this existing transaction: - `26df268` implements `llm_log.expert_protocol`; the exact `tests/test_expert_protocol.py` contract was reproduced locally with Python 3.13 and is **6/6 green**. - `e886999` documents the Common Lisp expert-runtime ownership boundary under `expert/README.org`. - `1b194a4`, `2da0602`, `40b5f54` establish the `llm-log-expert` ASDF/package/Tek9 lifecycle skeleton. `start-expert-host` owns one reusable Tek9 environment; no per-query Prolog process behavior was introduced. - Upstream Tek9 packaging gap is now tracked as `lost-rob0t/tek9#7`: its current flake has a dev shell but no consumable package output, so llm-log will not assume a developer Quicklisp checkout. Truthful gate state: Python protocol contract is green. The black-box CL/Tek9/SWI service contract remains intentionally RED/unexecuted here because this execution environment has Python but no SBCL, SWI-Prolog, Nix, or network-resolvable checkout. Do not merge yet. Next implementation slice: persistent supervised SWI-Prolog session + named JSON-lines operation dispatcher, then `llm-log-expert` executable/Nix wiring once Tek9 package consumption is available.