Validate requested structured outputs against JSON Schema #100

Open

nsaspy wants to merge 3 commits from feat/58-structured-output-anomaly into feat/58-tool-schema-anomaly

nsaspy commented

2026-09-20 06:13:35 +00:00

Owner

Tracks the third bounded detector family from #58, stacked on #92.

RED-first scope:

support Chat Completions response_format.type=json_schema contracts and Responses API text.format.type=json_schema contracts;
reconstruct streamed Chat delta.content and Responses response.output_text.delta text before parsing/validation;
emit structured_output_invalid_json only for complete outputs under a valid supplied schema contract;
emit structured_output_schema_violation with safe validator/path evidence for valid JSON that violates the supplied schema;
explicit refusals, Chat finish_reason=length, Responses incomplete/failed states, ordinary text mode, and invalid/unresolvable request schemas are non-blame states;
reuse the bounded local-only JSON Schema validator from #92;
raw output text, prompt text and schema bodies must not enter anomaly evidence;
quantization annotation remains orthogonal and cannot change detector score.

This slice is observe-only model-behavior evidence. Repetition/looping, output-length baselines, encoding corruption, self-correction/canary drift and notification integration remain separate #58 slices.

Tracks the third bounded detector family from #58, stacked on #92. RED-first scope: - support Chat Completions `response_format.type=json_schema` contracts and Responses API `text.format.type=json_schema` contracts; - reconstruct streamed Chat `delta.content` and Responses `response.output_text.delta` text before parsing/validation; - emit `structured_output_invalid_json` only for complete outputs under a valid supplied schema contract; - emit `structured_output_schema_violation` with safe validator/path evidence for valid JSON that violates the supplied schema; - explicit refusals, Chat `finish_reason=length`, Responses `incomplete`/failed states, ordinary text mode, and invalid/unresolvable request schemas are non-blame states; - reuse the bounded local-only JSON Schema validator from #92; - raw output text, prompt text and schema bodies must not enter anomaly evidence; - quantization annotation remains orthogonal and cannot change detector score. This slice is observe-only model-behavior evidence. Repetition/looping, output-length baselines, encoding corruption, self-correction/canary drift and notification integration remain separate #58 slices.

nsaspy added 3 commits

2026-09-20 06:13:35 +00:00

test: define RED structured-output schema anomaly contract ca4c5e6f98

feat: expose safe request schema usability check 0f26c31873

feat: analyze requested structured outputs safely

Quota telemetry exact-head contract / quota-contracts (pull_request) Failing after 3s

Details

Analytics API / test (pull_request) Failing after 4s

Details

Analytics API / package (pull_request) Has been skipped

Details

d9674780b0

Quota telemetry exact-head contract / quota-contracts (pull_request) Failing after 3s

Details

Analytics API / test (pull_request) Failing after 4s

Details

Analytics API / package (pull_request) Has been skipped

Details

This pull request can be merged automatically.

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin feat/58-structured-output-anomaly:feat/58-structured-output-anomaly

git switch feat/58-structured-output-anomaly

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch feat/58-tool-schema-anomaly

git merge --no-ff feat/58-structured-output-anomaly

git switch feat/58-structured-output-anomaly

git rebase feat/58-tool-schema-anomaly

git switch feat/58-tool-schema-anomaly

git merge --ff-only feat/58-structured-output-anomaly

git switch feat/58-structured-output-anomaly

git rebase feat/58-tool-schema-anomaly

git switch feat/58-tool-schema-anomaly

git merge --no-ff feat/58-structured-output-anomaly

git switch feat/58-tool-schema-anomaly

git merge --squash feat/58-structured-output-anomaly