#8 - ARADR-004: Symbolic expert plane — Tek9 + Common Lisp + Prolog - nsaspy/llm-log

lost-rob0t commented

2026-08-29 12:48:55 +00:00

(Migrated from github.com)

Goal

Add a first-class symbolic expert-system plane to llm-log for request rewriting, request classification/analysis, task/cost tracking, and success/failure dataset labeling.

Canonical architecture

Current authority is research/LLM-LOG-RESEARCH-033-expert-authority-regression-reconciliation.org and active PR #24.

LLM clients
      |
      v
Python llm-log capture/transport
  |-- transparent HTTP/SSE/WebSocket forwarding
  |-- provider/upstream compatibility
  |-- immutable raw capture + redaction
  `-- expert availability policy
      |
      v
Common Lisp expert service
  |-- typed expert runtime/lifecycle
  |-- Tek9 access + bounded fact materialization
  |-- SWI-Prolog supervision
  `-- result validation + durable derived assertions
          |                    |
          v                    v
       Tek9                SWI-Prolog

Python llm-log remains capture/transport. Do not rewrite transport in Common Lisp and do not remove Python.
Common Lisp owns the expert runtime: Tek9 access, typed objects/contracts, bounded fact materialization, SWI-Prolog supervision, result validation, and durable derived assertions.
Tek9 is the canonical durable document/graph KB for symbolic state and provenance.
SWI-Prolog owns facts/rules/inference. Do not implement a second Python or Common Lisp rule engine.
Raw captured traffic remains immutable source evidence. Expert outputs are separate derived records linked by provenance.
Existing Tek9 public APIs are preferred; modify Tek9 only if a proven missing generic primitive blocks the expert plane.

Historical zero-Python material from #20 and older research is superseded by the current explicit architecture.

Expert backlog / order

#10 Common Lisp Tek9 store + typed Prolog shim — first substrate and current hard priority.
#12 request classifier in analysis/shadow mode.
#13 task graph + usage/cost accounting.
#15 success/failure/partial/etc. outcome expert.
#17 Beagle rewrite expert in shadow mode.
Beagle apply mode only after a separate conformance gate.

Provider census #6 remains backlog and must not displace #10.

Canonical KB model

Start with durable Tek9 entities/edges for:

user_message
request
response
websocket frame/turn linkage
session
task/subtask
provider/model
usage observation
immutable pricing snapshot
rewrite decision
classification assertion
outcome assertion
evidence/provenance
rule/rule-version

Common Lisp materializes only bounded indexed facts needed for a Prolog query and persists derived results with rule/version/source/expert provenance. Never load the entire corpus into Prolog per request.

Expert boundary

Use versioned typed operations only. No call_prolog, arbitrary goal string, callable term, term_string/2, read_term/3, or equivalent model-controlled execution escape hatch.

Proxy forwarding remains fail-open when the expert service is unavailable unless explicit require_expert_plane mode is configured.

Beagle constraints

begin in shadow mode;
preserve the exact original user message;
persist rewrite/rule/evidence provenance;
current explicit user-message constraints outrank conflicting older KB facts;
apply requires its own RED-first conformance gate.

Hard invariants

raw capture is immutable evidence;
derived assertions link source IDs + rule/version + expert version;
bounded indexed Tek9 retrieval only;
declared Prolog predicates only;
classification metadata never grants authority;
unknown pricing stays unknown and cost assertions bind to immutable pricing snapshots;
outcome evidence is separate from labels; transport 200 does not imply task success;
historical assertion revisions are superseded, not deleted;
credentials do not enter Tek9/Nix/prose/logs unless explicitly intended and safely handled.

Nix / non-Nix

Keep the existing flake. Reproducibly package the expert service with SBCL, Tek9 and SWI-Prolog, configurable through Home Manager/systemd. Non-Nix users need a normal config/launch path. Reusable state remains below ~/.llm-proxy; consumer dotfiles may explicitly preserve capture storage at ~/Documents/AI/proxy.

RED-first

Every production behavior mutation requires a legitimate executed RED against the untouched pre-implementation head. Never substitute unavailable test results or same-author bookkeeping for evidence/review.

## Goal Add a first-class symbolic expert-system plane to `llm-log` for request rewriting, request classification/analysis, task/cost tracking, and success/failure dataset labeling. ## Canonical architecture Current authority is `research/LLM-LOG-RESEARCH-033-expert-authority-regression-reconciliation.org` and active PR #24. ```text LLM clients | v Python llm-log capture/transport |-- transparent HTTP/SSE/WebSocket forwarding |-- provider/upstream compatibility |-- immutable raw capture + redaction `-- expert availability policy | v Common Lisp expert service |-- typed expert runtime/lifecycle |-- Tek9 access + bounded fact materialization |-- SWI-Prolog supervision `-- result validation + durable derived assertions | | v v Tek9 SWI-Prolog ``` - **Python llm-log remains capture/transport.** Do not rewrite transport in Common Lisp and do not remove Python. - **Common Lisp owns the expert runtime:** Tek9 access, typed objects/contracts, bounded fact materialization, SWI-Prolog supervision, result validation, and durable derived assertions. - **Tek9** is the canonical durable document/graph KB for symbolic state and provenance. - **SWI-Prolog** owns facts/rules/inference. Do not implement a second Python or Common Lisp rule engine. - Raw captured traffic remains immutable source evidence. Expert outputs are separate derived records linked by provenance. - Existing Tek9 public APIs are preferred; modify Tek9 only if a proven missing generic primitive blocks the expert plane. Historical zero-Python material from #20 and older research is superseded by the current explicit architecture. ## Expert backlog / order 1. #10 Common Lisp Tek9 store + typed Prolog shim — first substrate and current hard priority. 2. #12 request classifier in analysis/shadow mode. 3. #13 task graph + usage/cost accounting. 4. #15 success/failure/partial/etc. outcome expert. 5. #17 Beagle rewrite expert in shadow mode. 6. Beagle apply mode only after a separate conformance gate. Provider census #6 remains backlog and must not displace #10. ## Canonical KB model Start with durable Tek9 entities/edges for: - user_message - request - response - websocket frame/turn linkage - session - task/subtask - provider/model - usage observation - immutable pricing snapshot - rewrite decision - classification assertion - outcome assertion - evidence/provenance - rule/rule-version Common Lisp materializes only bounded indexed facts needed for a Prolog query and persists derived results with rule/version/source/expert provenance. Never load the entire corpus into Prolog per request. ## Expert boundary Use versioned typed operations only. No `call_prolog`, arbitrary goal string, callable term, `term_string/2`, `read_term/3`, or equivalent model-controlled execution escape hatch. Proxy forwarding remains fail-open when the expert service is unavailable unless explicit `require_expert_plane` mode is configured. ## Beagle constraints - begin in `shadow` mode; - preserve the exact original user message; - persist rewrite/rule/evidence provenance; - current explicit user-message constraints outrank conflicting older KB facts; - `apply` requires its own RED-first conformance gate. ## Hard invariants - raw capture is immutable evidence; - derived assertions link source IDs + rule/version + expert version; - bounded indexed Tek9 retrieval only; - declared Prolog predicates only; - classification metadata never grants authority; - unknown pricing stays unknown and cost assertions bind to immutable pricing snapshots; - outcome evidence is separate from labels; transport 200 does not imply task success; - historical assertion revisions are superseded, not deleted; - credentials do not enter Tek9/Nix/prose/logs unless explicitly intended and safely handled. ## Nix / non-Nix Keep the existing flake. Reproducibly package the expert service with SBCL, Tek9 and SWI-Prolog, configurable through Home Manager/systemd. Non-Nix users need a normal config/launch path. Reusable state remains below `~/.llm-proxy`; consumer dotfiles may explicitly preserve capture storage at `~/Documents/AI/proxy`. ## RED-first Every production behavior mutation requires a legitimate executed RED against the untouched pre-implementation head. Never substitute unavailable test results or same-author bookkeeping for evidence/review.

lost-rob0t commented

2026-08-29 20:15:11 +00:00

(Migrated from github.com)

Supersession note: #20 and canonical Org research/LLM-LOG-RESEARCH-011-zero-python-rewrite.org replace this issue's older Python capture/transport ownership statement. Accepted architecture is now Common Lisp for the whole llm-log runtime (HTTP/SSE/WebSocket proxy, capture/redaction, recorder/config/provider routing, expert orchestration/Tek9 access/Prolog supervision), SWI-Prolog for facts/rules/inference, Tek9 for the canonical durable symbolic KB, and zero Python in the final implementation. The expert invariants and #10 dependency ordering remain in force.

Supersession note: #20 and canonical Org `research/LLM-LOG-RESEARCH-011-zero-python-rewrite.org` replace this issue's older Python capture/transport ownership statement. Accepted architecture is now Common Lisp for the whole llm-log runtime (HTTP/SSE/WebSocket proxy, capture/redaction, recorder/config/provider routing, expert orchestration/Tek9 access/Prolog supervision), SWI-Prolog for facts/rules/inference, Tek9 for the canonical durable symbolic KB, and zero Python in the final implementation. The expert invariants and #10 dependency ordering remain in force.

lost-rob0t commented

2026-08-30 10:17:35 +00:00

(Migrated from github.com)

Architecture authority reconciliation: the current accepted boundary is Python capture/transport; Common Lisp expert runtime; Tek9 canonical durable expert KB; SWI-Prolog declared inference. The zero-Python/Common-Lisp-transport wording in the older issue body is superseded by the current explicit program direction and canonical research/LLM-LOG-RESEARCH-023-expert-plane-authority-reconciliation.org plus research/LLM-LOG-RESEARCH-024-expert-ci-authority-boundary.org. PR #24 now enforces that boundary and exact head 7bd8abe66b5708dc6304e4b7c32a2abefb21d641 is GREEN on hosted run 33306067089. No Tek9 or dotfiles mutation was required.

Architecture authority reconciliation: the current accepted boundary is **Python capture/transport; Common Lisp expert runtime; Tek9 canonical durable expert KB; SWI-Prolog declared inference**. The zero-Python/Common-Lisp-transport wording in the older issue body is superseded by the current explicit program direction and canonical `research/LLM-LOG-RESEARCH-023-expert-plane-authority-reconciliation.org` plus `research/LLM-LOG-RESEARCH-024-expert-ci-authority-boundary.org`. PR #24 now enforces that boundary and exact head `7bd8abe66b5708dc6304e4b7c32a2abefb21d641` is GREEN on hosted run `33306067089`. No Tek9 or dotfiles mutation was required.

lost-rob0t commented

2026-08-30 22:15:08 +00:00

(Migrated from github.com)

Authority reconciliation: the zero-Python passages in this issue are stale relative to the current explicit architecture. Canonical decision is now research/LLM-LOG-RESEARCH-033-expert-authority-regression-reconciliation.org on rage/10-expert-runtime-red (commit 07b92274d4312cbfe228be56524b430348687d22). Current ownership remains Python for transparent HTTP/SSE/WebSocket capture/transport; Common Lisp for expert runtime/Tek9/typed bounded materialization/Prolog supervision/result validation; Tek9 durable KB; SWI-Prolog facts/rules/inference. No second Python rule engine. #10 remains the implementation priority.

Authority reconciliation: the zero-Python passages in this issue are stale relative to the current explicit architecture. Canonical decision is now `research/LLM-LOG-RESEARCH-033-expert-authority-regression-reconciliation.org` on `rage/10-expert-runtime-red` (commit `07b92274d4312cbfe228be56524b430348687d22`). Current ownership remains Python for transparent HTTP/SSE/WebSocket capture/transport; Common Lisp for expert runtime/Tek9/typed bounded materialization/Prolog supervision/result validation; Tek9 durable KB; SWI-Prolog facts/rules/inference. No second Python rule engine. #10 remains the implementation priority.