[P0] Implement provider-neutral model API with a real OpenAI-compatible backend #5

New issue

Closed

opened 2026-08-12 01:17:44 +00:00 by lost-rob0t · 0 comments

lost-rob0t commented

2026-08-12 01:17:44 +00:00

(Migrated from github.com)

Goal

Make real model inference work directly from SWI-Prolog without a Python runtime dependency, and prove the production provider path with real OpenRouter requests in GitHub Actions.

Scope

define canonical model request/response/message terms;
implement provider behavior/capability predicates;
implement OpenAI-compatible HTTP transport using SWI HTTP/JSON libraries;
implement OpenRouter as the first real integration target using https://openrouter.ai/api/v1/chat/completions;
read the CI credential only from OPENROUTER_API_KEY and never persist the resolved value in config, traces, fixtures, logs, exceptions, or artifacts;
default live CI model to openrouter/free, overridable by OPENROUTER_TEST_MODEL;
support configurable endpoint, model, timeout, safe headers, and basic generation options;
normalize text, structured tool calls, finish reason, token usage, selected model, and provider metadata;
include other OpenAI-compatible endpoints such as Ollama/vLLM where they conform to the canonical API;
retain a deterministic fake provider only for unit tests.

CI requirements

There must be two distinct test paths:

Deterministic CI

runs without credentials or network dependency;
uses the fake provider;
runs on every PR/push;
tests parsing, normalization, provider errors, timeouts, and secret redaction.

Live OpenRouter integration CI

consumes the repository Actions secret OPENROUTER_API_KEY via ${{ secrets.OPENROUTER_API_KEY }};
performs an actual network request to OpenRouter; no mock/fake provider may satisfy this job;
uses a tiny prompt/output budget and OPENROUTER_TEST_MODEL (default openrouter/free);
fails if the secret is configured but the real provider request fails;
never silently falls back to the fake provider;
verifies a successful response is normalized into canonical Prolog response terms;
checks selected model/provider metadata and usage fields when returned;
must not echo authorization headers or the secret;
must not upload logs/artifacts containing request headers;
for PR execution, do not expose the repository secret to untrusted fork PR code. Same-repository branch PRs may run the live job; external fork PRs must skip it.

Acceptance criteria

a credentialed GitHub Actions run makes at least one real OpenRouter inference call and passes only when that call succeeds;
deterministic unit tests remain green with no OPENROUTER_API_KEY;
no Python process or Python package is required;
API/network failures return structured Prolog errors rather than raw exceptions;
usage metadata is available to later budget accounting;
git grep/review confirms no OpenRouter key or bearer token is committed;
live test output proves which provider/model handled the request without revealing credentials.

Dependencies

#4
parent #3

Research

See RLM-RESEARCH-002-agentic-harness.org and the llmpl findings recorded in the research set.

## Goal Make real model inference work directly from SWI-Prolog without a Python runtime dependency, and prove the production provider path with real OpenRouter requests in GitHub Actions. ## Scope - define canonical model request/response/message terms; - implement provider behavior/capability predicates; - implement OpenAI-compatible HTTP transport using SWI HTTP/JSON libraries; - implement OpenRouter as the first real integration target using `https://openrouter.ai/api/v1/chat/completions`; - read the CI credential only from `OPENROUTER_API_KEY` and never persist the resolved value in config, traces, fixtures, logs, exceptions, or artifacts; - default live CI model to `openrouter/free`, overridable by `OPENROUTER_TEST_MODEL`; - support configurable endpoint, model, timeout, safe headers, and basic generation options; - normalize text, structured tool calls, finish reason, token usage, selected model, and provider metadata; - include other OpenAI-compatible endpoints such as Ollama/vLLM where they conform to the canonical API; - retain a deterministic fake provider only for unit tests. ## CI requirements There must be two distinct test paths: ### Deterministic CI - runs without credentials or network dependency; - uses the fake provider; - runs on every PR/push; - tests parsing, normalization, provider errors, timeouts, and secret redaction. ### Live OpenRouter integration CI - consumes the repository Actions secret `OPENROUTER_API_KEY` via `${{ secrets.OPENROUTER_API_KEY }}`; - performs an actual network request to OpenRouter; no mock/fake provider may satisfy this job; - uses a tiny prompt/output budget and `OPENROUTER_TEST_MODEL` (default `openrouter/free`); - fails if the secret is configured but the real provider request fails; - never silently falls back to the fake provider; - verifies a successful response is normalized into canonical Prolog response terms; - checks selected model/provider metadata and usage fields when returned; - must not echo authorization headers or the secret; - must not upload logs/artifacts containing request headers; - for PR execution, do not expose the repository secret to untrusted fork PR code. Same-repository branch PRs may run the live job; external fork PRs must skip it. ## Acceptance criteria - a credentialed GitHub Actions run makes at least one real OpenRouter inference call and passes only when that call succeeds; - deterministic unit tests remain green with no `OPENROUTER_API_KEY`; - no Python process or Python package is required; - API/network failures return structured Prolog errors rather than raw exceptions; - usage metadata is available to later budget accounting; - `git grep`/review confirms no OpenRouter key or bearer token is committed; - live test output proves which provider/model handled the request without revealing credentials. ## Dependencies - #4 - parent #3 ## Research See `RLM-RESEARCH-002-agentic-harness.org` and the `llmpl` findings recorded in the research set.