nsaspy/zara

Fork 0

P0 — replace the minimal Prolog LLM wrapper with a full embedded async-capable client #124

New issue

Closed

opened 2026-08-16 03:03:51 +00:00 by lost-rob0t · 1 comment

lost-rob0t commented

2026-08-16 03:03:51 +00:00

(Migrated from github.com)

Goal

Build a first-class Prolog-native LLM client subsystem inside the Zara repository. This replaces the current minimal modules/llm_client.pl transport wrapper as the long-term provider boundary for Zara's Prolog-side model access.

There must be no runtime dependency on lost-rob0t/prolog-rlm for this subsystem. Zara owns the implementation, API, tests, configuration, lifecycle, and compatibility surface.

Why

The current Prolog client is too small for Zara's direction. It mixes provider selection, request serialization, HTTP transport, retries, response parsing, conversation history, and error normalization in one module. Manual testing also exposed a concrete regression: SWI-Prolog DNS socket_error/2 is currently misclassified as malformed_response (#17).

Zara needs a reusable client that can support ordinary completion, rewriting, structured output, tools, streaming, cancellation, and optional asynchronous execution without forcing callers into Python.

Required architecture

Create an embedded module family under a clear namespace, for example:

modules/llm/
  client.pl
  request.pl
  response.pl
  transport.pl
  errors.pl
  async.pl
  stream.pl
  history.pl
  providers/
    openai.pl
    anthropic.pl
    ollama.pl

Exact filenames may differ, but responsibilities must be separated. Keep modules/llm_client.pl as a compatibility facade until all existing callers migrate.

Core synchronous API

Provide one provider-neutral request/result model. Example direction:

llm_complete(+Request, +Options, -Result).
llm_chat(+Messages, +Options, -Result).
llm_stream(+Request, +Options, :OnEvent, -Result).

Results must be typed terms, not provider dictionaries leaking upward. Include at least:

llm_result(success, Response)
llm_result(error, Error)

Response metadata should be able to represent text/content, finish reason, provider/model, usage metadata, tool calls, structured output, and request/trace identifiers.

Optional asynchronous API

Where SWI-Prolog threading is available, add an optional non-blocking job interface using bounded worker threads/message queues rather than spawning an unbounded thread per request.

Suggested semantics:

llm_submit(+Request, +Options, -Job).
llm_poll(+Job, -State).
llm_await(+Job, -Result).
llm_await(+Job, +Timeout, -Result).
llm_cancel(+Job).
llm_job_events(+Job, -QueueOrSubscription).

States should be explicit: queued/running/streaming/completed/failed/cancelled.

Requirements:

bounded worker pool and bounded event queues;
cancellation is idempotent;
completed/cancelled jobs release resources deterministically;
no unbounded thread/job accumulation;
sync API remains available and may reuse the same internal execution engine;
async must be optional/configurable and must not break single-threaded/headless use.

Streaming

Support provider streaming where practical with provider-neutral events such as text_delta/1, tool-call deltas, usage updates, completed, failed, and cancelled. Do not expose raw SSE/provider frames above provider adapters. Streaming must obey cancellation and total/output bounds.

Providers

Initial supported providers remain OpenAI-compatible HTTP APIs, Anthropic, and Ollama. OpenAI-compatible endpoints must support configurable base URLs so local routers and compatible servers work.

Provider adapters own request serialization, authentication headers, streaming codec, response normalization, and provider-specific finish/tool/usage mapping. Transport must not contain provider-specific response parsing.

Request features

Support a provider-neutral subset sufficient for Zara:

system + user/assistant messages;
temperature/top-p where supported;
max output tokens;
stop sequences;
JSON/structured response mode;
tool/function definitions and tool-call results;
streaming toggle;
per-request provider/model override;
timeout/retry policy;
metadata/trace ID.

Unsupported provider features must fail explicitly or be ignored only when the contract documents that behavior. Do not silently reinterpret semantics.

Errors

Create a single normalized error taxonomy covering configuration, authentication, DNS/name resolution, connection refused/unreachable, timeout, cancellation, generic network/socket, rate limit, provider HTTP error, malformed JSON/body, provider schema mismatch, empty response, invalid request/options, structured-output validation failure, tool-call decoding failure, and internal client failure.

This work must satisfy reopened #17. In particular, SWI exception shapes including socket_error/1 and socket_error/2 must normalize correctly and must never fall through as malformed_response merely because their arity differs.

Do not leak API keys, authorization headers, secrets, or raw environment dumps into error terms/logs.

Retries and deadlines

Support bounded connect timeout, read/inactivity timeout, total request deadline, retry count, retryable status/error classification, and bounded backoff. Do not retry deterministic request/schema/auth failures. Cancellation/deadline must stop retries.

Connection/resource lifecycle

Reuse HTTP connections where SWI's HTTP stack permits it. Provide explicit client shutdown/cleanup that closes keep-alive resources, cancels/drains async jobs, destroys message queues/worker threads deterministically, and clears transient history/job state.

Conversation/history

Preserve compatibility with Zara's existing llm_query_with_history/2, but move history into its own bounded component. History must enforce configurable turn/message limits, preserve provider-valid ordering, avoid persisting transient rewrite/tool context accidentally, and support reset plus explicit session IDs where useful.

Compatibility

Existing predicates exported by modules/llm_client.pl should keep working during migration where practical:

llm_query/2
llm_query/3
llm_query_result/3
llm_query_with_history/2
llm_query_with_history_result/2
reset_llm_history/0
close_llm_client/0

Implement them as a thin facade over the new subsystem rather than retaining two independent transports.

Tests

Use local fake HTTP servers/fixtures only for the default gate. Cover provider golden requests and normalization, socket_error/1 and /2, connection failures, deadlines/timeouts, HTTP error classes, malformed/schema-invalid/empty responses, retry bounds, structured output, tool calls, streaming, cancellation, async state transitions, bounded worker/mailbox behavior, deterministic shutdown, and history compatibility.

Add a focused script such as scripts/test-prolog-llm-client.sh and include it in the full deterministic gate.

Acceptance criteria

Zara contains one canonical Prolog LLM client subsystem.
Sync completion works through the new client.
Async submission/await/cancel works when enabled without unbounded threads or queues.
Streaming is normalized into provider-neutral events.
Provider-specific wire formats are isolated behind adapters.
Errors are typed and the #17 DNS regression is fixed.
Existing Zara Prolog callers continue working through a compatibility facade.
No prolog-rlm runtime dependency is introduced.
The subsystem is suitable as the provider foundation for #122.

Ordering

Implement this issue and satisfy reopened #17.
Implement #122 rewriter on top of this client.

## Goal Build a first-class **Prolog-native LLM client subsystem inside the Zara repository**. This replaces the current minimal `modules/llm_client.pl` transport wrapper as the long-term provider boundary for Zara's Prolog-side model access. There must be **no runtime dependency on `lost-rob0t/prolog-rlm`** for this subsystem. Zara owns the implementation, API, tests, configuration, lifecycle, and compatibility surface. ## Why The current Prolog client is too small for Zara's direction. It mixes provider selection, request serialization, HTTP transport, retries, response parsing, conversation history, and error normalization in one module. Manual testing also exposed a concrete regression: SWI-Prolog DNS `socket_error/2` is currently misclassified as `malformed_response` (#17). Zara needs a reusable client that can support ordinary completion, rewriting, structured output, tools, streaming, cancellation, and optional asynchronous execution without forcing callers into Python. ## Required architecture Create an embedded module family under a clear namespace, for example: ```text modules/llm/ client.pl request.pl response.pl transport.pl errors.pl async.pl stream.pl history.pl providers/ openai.pl anthropic.pl ollama.pl ``` Exact filenames may differ, but responsibilities must be separated. Keep `modules/llm_client.pl` as a compatibility facade until all existing callers migrate. ## Core synchronous API Provide one provider-neutral request/result model. Example direction: ```prolog llm_complete(+Request, +Options, -Result). llm_chat(+Messages, +Options, -Result). llm_stream(+Request, +Options, :OnEvent, -Result). ``` Results must be typed terms, not provider dictionaries leaking upward. Include at least: ```prolog llm_result(success, Response) llm_result(error, Error) ``` Response metadata should be able to represent text/content, finish reason, provider/model, usage metadata, tool calls, structured output, and request/trace identifiers. ## Optional asynchronous API Where SWI-Prolog threading is available, add an optional non-blocking job interface using bounded worker threads/message queues rather than spawning an unbounded thread per request. Suggested semantics: ```prolog llm_submit(+Request, +Options, -Job). llm_poll(+Job, -State). llm_await(+Job, -Result). llm_await(+Job, +Timeout, -Result). llm_cancel(+Job). llm_job_events(+Job, -QueueOrSubscription). ``` States should be explicit: queued/running/streaming/completed/failed/cancelled. Requirements: - bounded worker pool and bounded event queues; - cancellation is idempotent; - completed/cancelled jobs release resources deterministically; - no unbounded thread/job accumulation; - sync API remains available and may reuse the same internal execution engine; - async must be optional/configurable and must not break single-threaded/headless use. ## Streaming Support provider streaming where practical with provider-neutral events such as `text_delta/1`, tool-call deltas, usage updates, completed, failed, and cancelled. Do not expose raw SSE/provider frames above provider adapters. Streaming must obey cancellation and total/output bounds. ## Providers Initial supported providers remain OpenAI-compatible HTTP APIs, Anthropic, and Ollama. OpenAI-compatible endpoints must support configurable base URLs so local routers and compatible servers work. Provider adapters own request serialization, authentication headers, streaming codec, response normalization, and provider-specific finish/tool/usage mapping. Transport must not contain provider-specific response parsing. ## Request features Support a provider-neutral subset sufficient for Zara: - system + user/assistant messages; - temperature/top-p where supported; - max output tokens; - stop sequences; - JSON/structured response mode; - tool/function definitions and tool-call results; - streaming toggle; - per-request provider/model override; - timeout/retry policy; - metadata/trace ID. Unsupported provider features must fail explicitly or be ignored only when the contract documents that behavior. Do not silently reinterpret semantics. ## Errors Create a single normalized error taxonomy covering configuration, authentication, DNS/name resolution, connection refused/unreachable, timeout, cancellation, generic network/socket, rate limit, provider HTTP error, malformed JSON/body, provider schema mismatch, empty response, invalid request/options, structured-output validation failure, tool-call decoding failure, and internal client failure. This work must satisfy reopened #17. In particular, SWI exception shapes including `socket_error/1` and `socket_error/2` must normalize correctly and must never fall through as `malformed_response` merely because their arity differs. Do not leak API keys, authorization headers, secrets, or raw environment dumps into error terms/logs. ## Retries and deadlines Support bounded connect timeout, read/inactivity timeout, total request deadline, retry count, retryable status/error classification, and bounded backoff. Do not retry deterministic request/schema/auth failures. Cancellation/deadline must stop retries. ## Connection/resource lifecycle Reuse HTTP connections where SWI's HTTP stack permits it. Provide explicit client shutdown/cleanup that closes keep-alive resources, cancels/drains async jobs, destroys message queues/worker threads deterministically, and clears transient history/job state. ## Conversation/history Preserve compatibility with Zara's existing `llm_query_with_history/2`, but move history into its own bounded component. History must enforce configurable turn/message limits, preserve provider-valid ordering, avoid persisting transient rewrite/tool context accidentally, and support reset plus explicit session IDs where useful. ## Compatibility Existing predicates exported by `modules/llm_client.pl` should keep working during migration where practical: ```prolog llm_query/2 llm_query/3 llm_query_result/3 llm_query_with_history/2 llm_query_with_history_result/2 reset_llm_history/0 close_llm_client/0 ``` Implement them as a thin facade over the new subsystem rather than retaining two independent transports. ## Tests Use local fake HTTP servers/fixtures only for the default gate. Cover provider golden requests and normalization, `socket_error/1` and `/2`, connection failures, deadlines/timeouts, HTTP error classes, malformed/schema-invalid/empty responses, retry bounds, structured output, tool calls, streaming, cancellation, async state transitions, bounded worker/mailbox behavior, deterministic shutdown, and history compatibility. Add a focused script such as `scripts/test-prolog-llm-client.sh` and include it in the full deterministic gate. ## Acceptance criteria - Zara contains one canonical Prolog LLM client subsystem. - Sync completion works through the new client. - Async submission/await/cancel works when enabled without unbounded threads or queues. - Streaming is normalized into provider-neutral events. - Provider-specific wire formats are isolated behind adapters. - Errors are typed and the #17 DNS regression is fixed. - Existing Zara Prolog callers continue working through a compatibility facade. - No `prolog-rlm` runtime dependency is introduced. - The subsystem is suitable as the provider foundation for #122. ## Ordering 1. Implement this issue and satisfy reopened #17. 2. Implement #122 rewriter on top of this client.

nsaspy referenced this issue

2026-08-29 10:40:44 +00:00

[ORDER] ZARA-000 — Repository repair and real-time voice roadmap #1

nsaspy commented

2026-09-02 17:27:54 +00:00

Owner

Closing as stale/not planned under the current architecture. This issue requires Zara to own a full embedded Prolog LLM client and explicitly forbids a prolog-rlm runtime dependency. PR #233 reversed that decision on master: Zara now pins Prolog-RLM and uses its direct mode for the optional Prolog rewrite path. Keeping #124 open would send workers toward the architecture that current master deliberately replaced. Any remaining provider/streaming/rewrite safety gaps should be tracked against the current Prolog-RLM/OpenRouter architecture rather than resurrecting this obsolete no-RLM client plan.

Closing as stale/not planned under the current architecture. This issue requires Zara to own a full embedded Prolog LLM client and explicitly forbids a `prolog-rlm` runtime dependency. PR #233 reversed that decision on `master`: Zara now pins Prolog-RLM and uses its direct mode for the optional Prolog rewrite path. Keeping #124 open would send workers toward the architecture that current master deliberately replaced. Any remaining provider/streaming/rewrite safety gaps should be tracked against the current Prolog-RLM/OpenRouter architecture rather than resurrecting this obsolete no-RLM client plan.

nsaspy closed this issue

2026-09-04 14:39:03 +00:00

nsaspy referenced this issue

2026-09-04 14:39:03 +00:00

P1 — RAGE research the post-client-split semantic intent architecture and freeze an IntentFrame contract #154

nsaspy referenced this issue

2026-09-08 02:03:28 +00:00

P0 — harden the bounded utterance rewriter on the current Prolog-RLM direct-mode path #122

nsaspy referenced this issue

2026-09-08 02:03:34 +00:00

[EPIC] Model-independent Zara identity and graceful intelligence degradation #336

nsaspy referenced this issue