Add declarative Project source and grammar registry #115

Merged

lost-rob0t merged 1 commit from agent/95-project-source-registry into main

2026-08-19 16:44:46 +00:00

lost-rob0t commented

2026-08-19 04:24:18 +00:00

(Migrated from github.com)

Summary

Implements #95 above the merged #94 direct Tree-sitter FFI.

adds an isolated Prolog Project/File/Language source registry
keeps file identity distinct from path and carries hash/generation plus excluded/vendor/generated/embedded-region metadata
represents extension and shebang detection as inspectable evidence
distinguishes known, unknown, ambiguous, unsupported, and explicit host-override language states
selects tree_sitter for supported polyglot source and swi_native for Prolog
adds declarative Tree-sitter grammar records with version, ABI declaration, provenance, content-derived durable refs, and no language-specific C changes
keeps grammar registration inert; native code loading is a separate explicit trusted-host activation operation
validates actual #94 runtime/grammar ABI compatibility before publishing a grammar as active/ready
distinguishes missing grammar, configured grammar, and activated/ready grammar states
keeps native language handles internal to the registry rather than exposing raw addresses or using them as durable identity
adds deterministic pure-registry tests plus native C/Lua/Tree-sitter Query activation/ABI tests in the existing Tree-sitter CI lane
documents the #95 boundary and updates the PrologAgent roadmap while preserving the merged UI/conversation/effect work

Identity and authority boundary

Project identities here are epistemic source identities supplied as closed ground data. A host may reuse/reference #75's future canonical security-sensitive ProjectIdentity, but this module does not derive authorization identity from paths, cwd, Git metadata, parser output, or model text.

Grammar registration does not grant filesystem, process, network, tool, or execution authority. ts_grammar_activate/3 only works when the trusted host has already loaded rlm_tree_sitter; ordinary core module loading remains usable without Tree-sitter installed.

Source-analysis layering

#94 direct SWI-Prolog <-> Tree-sitter FFI
        |
        v
#95 Project/File/Language + grammar selection
        |
        v
#96 versioned CST observations
        |
        v
#97 query/capture API
        |
        v
#98 semantic source relations
        |
        v
#99 incremental freshness

This PR deliberately stops before CST traversal/materialization, query packs, symbols/references, and incremental reparsing.

Tests

Pure deterministic coverage includes:

two-project isolation with identical relative paths
file identity vs path and hash/generation provenance
extension evidence
unknown language
conflicting extension/shebang ambiguity
explicit host override
unsupported override
missing grammar vs unknown language
swi_native Prolog selection
Python/JavaScript/Nim plus a fourth grammar through one generic registry
grammar identity not reducible to raw library path
registration without native activation
unregister/re-register version identity
excluded/vendor/generated/embedded-region metadata
custom backend registration
destroyed registry errors

The native Tree-sitter lane additionally covers generic activation of C, Lua and Tree-sitter Query fixtures, active parser selection, deactivate/reactivate, active unregister cleanup, incompatible grammar ABI, and declared-ABI mismatch.

Verification

GitHub Actions on the exact final head is authoritative. The normal CI lane must pass the complete deterministic suite and REAL OpenRouter integration; the Tree-sitter lane must build the native FFI/fixtures and run both #94 and #95 native tests.

Closes #95

## Summary Implements #95 above the merged #94 direct Tree-sitter FFI. - adds an isolated Prolog Project/File/Language source registry - keeps file identity distinct from path and carries hash/generation plus excluded/vendor/generated/embedded-region metadata - represents extension and shebang detection as inspectable evidence - distinguishes known, unknown, ambiguous, unsupported, and explicit host-override language states - selects `tree_sitter` for supported polyglot source and `swi_native` for Prolog - adds declarative Tree-sitter grammar records with version, ABI declaration, provenance, content-derived durable refs, and no language-specific C changes - keeps grammar registration inert; native code loading is a separate explicit trusted-host activation operation - validates actual #94 runtime/grammar ABI compatibility before publishing a grammar as active/ready - distinguishes missing grammar, configured grammar, and activated/ready grammar states - keeps native language handles internal to the registry rather than exposing raw addresses or using them as durable identity - adds deterministic pure-registry tests plus native C/Lua/Tree-sitter Query activation/ABI tests in the existing Tree-sitter CI lane - documents the #95 boundary and updates the PrologAgent roadmap while preserving the merged UI/conversation/effect work ## Identity and authority boundary Project identities here are epistemic source identities supplied as closed ground data. A host may reuse/reference #75's future canonical security-sensitive ProjectIdentity, but this module does not derive authorization identity from paths, cwd, Git metadata, parser output, or model text. Grammar registration does not grant filesystem, process, network, tool, or execution authority. `ts_grammar_activate/3` only works when the trusted host has already loaded `rlm_tree_sitter`; ordinary core module loading remains usable without Tree-sitter installed. ## Source-analysis layering ```text #94 direct SWI-Prolog <-> Tree-sitter FFI | v #95 Project/File/Language + grammar selection | v #96 versioned CST observations | v #97 query/capture API | v #98 semantic source relations | v #99 incremental freshness ``` This PR deliberately stops before CST traversal/materialization, query packs, symbols/references, and incremental reparsing. ## Tests Pure deterministic coverage includes: - two-project isolation with identical relative paths - file identity vs path and hash/generation provenance - extension evidence - unknown language - conflicting extension/shebang ambiguity - explicit host override - unsupported override - missing grammar vs unknown language - `swi_native` Prolog selection - Python/JavaScript/Nim plus a fourth grammar through one generic registry - grammar identity not reducible to raw library path - registration without native activation - unregister/re-register version identity - excluded/vendor/generated/embedded-region metadata - custom backend registration - destroyed registry errors The native Tree-sitter lane additionally covers generic activation of C, Lua and Tree-sitter Query fixtures, active parser selection, deactivate/reactivate, active unregister cleanup, incompatible grammar ABI, and declared-ABI mismatch. ## Verification GitHub Actions on the exact final head is authoritative. The normal CI lane must pass the complete deterministic suite and REAL OpenRouter integration; the Tree-sitter lane must build the native FFI/fixtures and run both #94 and #95 native tests. Closes #95