ARADR-004: Expert-system substrate — Tek9 + Common Lisp Prolog shim #7

New issue

Closed

opened 2026-08-29 12:48:48 +00:00 by lost-rob0t · 1 comment

lost-rob0t commented

2026-08-29 12:48:48 +00:00

(Migrated from github.com)

Goal

Add a first-class expert-system plane to llm-log for symbolic rewriting, request analysis, task-cost accounting, and success/failure dataset labeling.

The expert system is not embedded as ad-hoc Python conditionals. The durable architecture is:

llm-log capture/proxy
        |
        v
Common Lisp expert host
  |              |
  |              +--> SWI-Prolog rule engine
  |
  +--> Tek9 embedded document/graph database

Ownership boundary

Tek9 owns durable facts, graph relationships, task/accounting records, outcomes, labels, evidence, and historical KB state.
Prolog owns rules, derivation, classification, rewrite decisions, confidence/evidence predicates, and expert-system reasoning.
Common Lisp is the stable host/shim around both: lifecycle, typed API, Tek9 access, Prolog process/session supervision, schema translation, transactions, and query/result normalization.
Python llm-log proxy remains transport/capture infrastructure. It forwards normalized events into the expert host and consumes typed results; it must not become a second rule engine.

Tek9 is already a Common Lisp embedded document+graph DB on LMDB, so the expert host should keep one Tek9 environment open for its lifetime and use indexed documents + graph edges rather than duplicating the KB in Python.

First expert families

This epic owns four initial experts, each tracked separately:

Rewrite expert — rewrite/normalize/augment requests based on user messages and an advanced evolving KB.
Request-analysis expert — classify requests primarily for analysis, routing metadata, dataset segmentation, and later symbolic policy.
Task-cost expert — identify/track logical tasks and attribute provider/model/token/cost/tool usage to each task and subtask.
Outcome expert — classify success/failure/partial outcomes and preserve evidence suitable for training/evaluation datasets.

Common data model

Start with stable IDs for:

event
conversation
message
request
response
task
subtask
provider-call
tool-call
rewrite
classification
outcome
evidence
rule-version
kb-revision

Relationships belong in Tek9 graph storage, e.g.:

message -> belongs_to -> conversation
request -> derived_from -> message
request -> assigned_to -> task
provider-call -> incurred_for -> task
response -> produced_by -> provider-call
outcome -> evaluates -> task
rewrite -> rewrites -> request
classification -> classifies -> request
classification -> supported_by -> evidence

Prolog shim requirements

Research/design a Common Lisp shim that exposes typed operations such as:

assert/retract durable fact projections
run named expert query
run rewrite expert
classify request
classify outcome
explain derivation / return evidence
load/version rules
query current KB revision

The shim should prefer a supervised persistent SWI-Prolog process/session over spawning a new interpreter per query. The transport must be explicit, typed, bounded, and testable. Do not allow model-controlled strings to become arbitrary Prolog callable terms.

Dataset principle

Every derived label/rewrite/outcome must retain:

input event/task IDs
expert/rule version
KB revision
evidence IDs
timestamp
confidence/status where applicable
whether result was deterministic, heuristic, model-assisted, or human-confirmed

This is required so later fine-tuning/evaluation datasets are reproducible instead of becoming a pile of unlabeled guesses.

Auto-ARADR work

Inspect current llm-log event/frame schemas and current Prolog classifier.
Inspect current Tek9 API/index/graph semantics and package boundary.
Design the CL expert-host package, lifecycle, protocol, schemas, and failure semantics.
Design Tek9 document/index/graph layout for the first four experts.
Design Prolog rule-module/versioning conventions and explanation output.
Adversarial review: crash recovery, malformed events, duplicate delivery, stale rules, partial writes, model-controlled Prolog injection, classification drift, dataset contamination, and cost-attribution ambiguity.
Produce canonical Org research/design artifacts and bounded RED-first implementation handoffs.

First implementation slice

Do not implement all experts at once. First realization should establish only the reusable substrate:

Common Lisp expert-host package/service
Tek9 dependency + database lifecycle
persistent supervised Prolog bridge
typed event ingestion
stable entity IDs
idempotent event projection into Tek9
one trivial end-to-end expert query proving llm-log event -> CL -> Tek9/Prolog -> typed result

Then implement each expert independently behind the same boundary.

RED-first acceptance for substrate

Before production implementation, tests must fail proving the baseline lacks:

a CL expert host that opens/reuses a Tek9 DB;
typed event ingestion with idempotency;
persistent Prolog query/session lifecycle;
safe rejection of arbitrary callable Prolog terms from untrusted event data;
typed expert result including rule/KB version and evidence references;
end-to-end round trip from captured event to expert result.

Non-goals for the substrate slice

autonomous prompt mutation without an explicit rewrite-policy gate
using Tek9 as a replacement for raw append-only capture files
allowing Prolog to directly own LMDB lifecycle
storing provider secrets in Tek9/Prolog
training models in this slice
automatic rule synthesis without provenance/review/versioning

## Goal Add a first-class expert-system plane to `llm-log` for symbolic rewriting, request analysis, task-cost accounting, and success/failure dataset labeling. The expert system is not embedded as ad-hoc Python conditionals. The durable architecture is: ```text llm-log capture/proxy | v Common Lisp expert host | | | +--> SWI-Prolog rule engine | +--> Tek9 embedded document/graph database ``` ### Ownership boundary - **Tek9** owns durable facts, graph relationships, task/accounting records, outcomes, labels, evidence, and historical KB state. - **Prolog** owns rules, derivation, classification, rewrite decisions, confidence/evidence predicates, and expert-system reasoning. - **Common Lisp** is the stable host/shim around both: lifecycle, typed API, Tek9 access, Prolog process/session supervision, schema translation, transactions, and query/result normalization. - **Python `llm-log` proxy** remains transport/capture infrastructure. It forwards normalized events into the expert host and consumes typed results; it must not become a second rule engine. Tek9 is already a Common Lisp embedded document+graph DB on LMDB, so the expert host should keep one Tek9 environment open for its lifetime and use indexed documents + graph edges rather than duplicating the KB in Python. ## First expert families This epic owns four initial experts, each tracked separately: 1. **Rewrite expert** — rewrite/normalize/augment requests based on user messages and an advanced evolving KB. 2. **Request-analysis expert** — classify requests primarily for analysis, routing metadata, dataset segmentation, and later symbolic policy. 3. **Task-cost expert** — identify/track logical tasks and attribute provider/model/token/cost/tool usage to each task and subtask. 4. **Outcome expert** — classify success/failure/partial outcomes and preserve evidence suitable for training/evaluation datasets. ## Common data model Start with stable IDs for: - `event` - `conversation` - `message` - `request` - `response` - `task` - `subtask` - `provider-call` - `tool-call` - `rewrite` - `classification` - `outcome` - `evidence` - `rule-version` - `kb-revision` Relationships belong in Tek9 graph storage, e.g.: ```text message -> belongs_to -> conversation request -> derived_from -> message request -> assigned_to -> task provider-call -> incurred_for -> task response -> produced_by -> provider-call outcome -> evaluates -> task rewrite -> rewrites -> request classification -> classifies -> request classification -> supported_by -> evidence ``` ## Prolog shim requirements Research/design a Common Lisp shim that exposes typed operations such as: - assert/retract durable fact projections - run named expert query - run rewrite expert - classify request - classify outcome - explain derivation / return evidence - load/version rules - query current KB revision The shim should prefer a supervised persistent SWI-Prolog process/session over spawning a new interpreter per query. The transport must be explicit, typed, bounded, and testable. Do not allow model-controlled strings to become arbitrary Prolog callable terms. ## Dataset principle Every derived label/rewrite/outcome must retain: - input event/task IDs - expert/rule version - KB revision - evidence IDs - timestamp - confidence/status where applicable - whether result was deterministic, heuristic, model-assisted, or human-confirmed This is required so later fine-tuning/evaluation datasets are reproducible instead of becoming a pile of unlabeled guesses. ## Auto-ARADR work 1. Inspect current `llm-log` event/frame schemas and current Prolog classifier. 2. Inspect current Tek9 API/index/graph semantics and package boundary. 3. Design the CL expert-host package, lifecycle, protocol, schemas, and failure semantics. 4. Design Tek9 document/index/graph layout for the first four experts. 5. Design Prolog rule-module/versioning conventions and explanation output. 6. Adversarial review: crash recovery, malformed events, duplicate delivery, stale rules, partial writes, model-controlled Prolog injection, classification drift, dataset contamination, and cost-attribution ambiguity. 7. Produce canonical Org research/design artifacts and bounded RED-first implementation handoffs. ## First implementation slice Do not implement all experts at once. First realization should establish only the reusable substrate: - Common Lisp expert-host package/service - Tek9 dependency + database lifecycle - persistent supervised Prolog bridge - typed event ingestion - stable entity IDs - idempotent event projection into Tek9 - one trivial end-to-end expert query proving `llm-log event -> CL -> Tek9/Prolog -> typed result` Then implement each expert independently behind the same boundary. ## RED-first acceptance for substrate Before production implementation, tests must fail proving the baseline lacks: - a CL expert host that opens/reuses a Tek9 DB; - typed event ingestion with idempotency; - persistent Prolog query/session lifecycle; - safe rejection of arbitrary callable Prolog terms from untrusted event data; - typed expert result including rule/KB version and evidence references; - end-to-end round trip from captured event to expert result. ## Non-goals for the substrate slice - autonomous prompt mutation without an explicit rewrite-policy gate - using Tek9 as a replacement for raw append-only capture files - allowing Prolog to directly own LMDB lifecycle - storing provider secrets in Tek9/Prolog - training models in this slice - automatic rule synthesis without provenance/review/versioning

lost-rob0t commented

2026-08-30 00:39:25 +00:00

(Migrated from github.com)

Closed as duplicate. Canonical corrected epic is #8; #20 supersedes the older Python-owned transport architecture described here.