[P0-04] Remove the broken duplicate server document encoder and decoder #12

New issue

Closed

opened 2026-07-22 00:09:31 +00:00 by lost-rob0t · 2 comments

lost-rob0t commented

2026-07-22 00:09:31 +00:00

(Migrated from github.com)

Confirmed defects

dev:source/databases/couchdb.lisp implements a second codec with multiple correctness failures:

as-json calls slot-value without checking slot-boundp.
Lists are converted to a JSON string and then stored as a JSON value, double-encoding arrays.
Nested/unknown values call an unrelated to-json path.
from-json uses when value, so present JSON false is treated as absent.
Type specs such as (or null string) are passed through eval and recursively treated as classes.
Slots whose effective type is t, including ordinary untyped fields, are recursively decoded as class T.
Nested collections and optional object types are not handled.
The implementation is SBCL-specific through direct sb-mop use.

Related actor-side decode defects:

with-json reads dataset from the misspelled key "datast".
parse-doc interns untrusted dtype text into the SPEC package and treats it as a class selector, without a registered dtype/class allowlist.
Unknown or malicious dtype values can intern permanent symbols or select unsupported classes before validation.

Normal StarIntel documents containing dtype or _rev can therefore fail to decode or change meaning during round-trip.

Required changes

Delete the duplicate codec or turn it into a thin compatibility wrapper over one canonical typed codec.
Preserve JSOWN objects versus serialized JSON strings as distinct API types.
Handle slot presence separately from value truthiness.
Support optional, collection, nested-object, boolean, null, revision, and unknown-field policies explicitly.
Use portable MOP access through closer-mop where introspection remains necessary.
Replace ad hoc intern class selection with a versioned dtype→document-class registry and reject unknown types before decode.
Remove/fix the with-json key macros and make them call the canonical document API.

Acceptance tests

Start with failing regression tests for every defect listed above. All shared conformance fixtures from the preceding issue must pass. Add explicit tests for the dataset key, unknown dtype rejection, symbol-table non-growth from rejected dtypes, and registered class selection.

## Confirmed defects `dev:source/databases/couchdb.lisp` implements a second codec with multiple correctness failures: - `as-json` calls `slot-value` without checking `slot-boundp`. - Lists are converted to a JSON string and then stored as a JSON value, double-encoding arrays. - Nested/unknown values call an unrelated `to-json` path. - `from-json` uses `when value`, so present JSON `false` is treated as absent. - Type specs such as `(or null string)` are passed through `eval` and recursively treated as classes. - Slots whose effective type is `t`, including ordinary untyped fields, are recursively decoded as class `T`. - Nested collections and optional object types are not handled. - The implementation is SBCL-specific through direct `sb-mop` use. Related actor-side decode defects: - `with-json` reads dataset from the misspelled key `"datast"`. - `parse-doc` interns untrusted `dtype` text into the `SPEC` package and treats it as a class selector, without a registered dtype/class allowlist. - Unknown or malicious dtype values can intern permanent symbols or select unsupported classes before validation. Normal StarIntel documents containing `dtype` or `_rev` can therefore fail to decode or change meaning during round-trip. ## Required changes - Delete the duplicate codec or turn it into a thin compatibility wrapper over one canonical typed codec. - Preserve JSOWN objects versus serialized JSON strings as distinct API types. - Handle slot presence separately from value truthiness. - Support optional, collection, nested-object, boolean, null, revision, and unknown-field policies explicitly. - Use portable MOP access through `closer-mop` where introspection remains necessary. - Replace ad hoc `intern` class selection with a versioned dtype→document-class registry and reject unknown types before decode. - Remove/fix the `with-json` key macros and make them call the canonical document API. ## Acceptance tests Start with failing regression tests for every defect listed above. All shared conformance fixtures from the preceding issue must pass. Add explicit tests for the dataset key, unknown dtype rejection, symbol-table non-growth from rejected dtypes, and registered class selection.

lost-rob0t commented

2026-07-26 13:57:37 +00:00

(Migrated from github.com)

Active implementation: PR #64 (agent/issue-12-remove-duplicate-codec).

The branch replaces the active database codec with canonical wrappers, adds a versioned dtype→CLOS class registry, removes request-controlled interning from actor parsing, fixes dataset access, and adds executable regression coverage. Waiting on required conformance CI.

Active implementation: PR #64 (`agent/issue-12-remove-duplicate-codec`). The branch replaces the active database codec with canonical wrappers, adds a versioned dtype→CLOS class registry, removes request-controlled interning from actor parsing, fixes dataset access, and adds executable regression coverage. Waiting on required conformance CI.

lost-rob0t commented

2026-07-26 14:05:19 +00:00

(Migrated from github.com)

Completed via merged PR #64.

Validation:

fixture-contract: passed
clos-conformance: passed
arrays, false, null, nested objects, CouchDB revisions, registered class selection, unknown dtype rejection, symbol-table non-growth, v0.8 read-only decoding, and actor macro behavior are covered

Merge commit: 786d292c7a28661281c55118197c3708d8b8f61e.

Completed via merged PR #64. Validation: - fixture-contract: passed - clos-conformance: passed - arrays, false, null, nested objects, CouchDB revisions, registered class selection, unknown dtype rejection, symbol-table non-growth, v0.8 read-only decoding, and actor macro behavior are covered Merge commit: `786d292c7a28661281c55118197c3708d8b8f61e`.