nsaspy/prolog-rlm

Fork 0

[EPIC][library/tool] General semantic knowledge compiler: arbitrary natural language → extensible validated symbolic knowledge #410

New issue

Open

opened 2026-09-08 01:29:20 +00:00 by nsaspy · 3 comments

nsaspy commented

2026-09-08 01:29:20 +00:00

Owner

Parent: #376
Downstream consumer: lost-rob0t/symbolic-memory
Integrates with: #377, #380, #381, #70, #93, #219

Mission

Implement a reusable general semantic knowledge compiler in prolog-rlm that transforms the meaning of arbitrary natural-language material into a safe, validated, extensible symbolic representation that Prolog can query and reason over.

The target is deliberately broader than entity extraction, claim extraction, or how-to compilation:

arbitrary natural-language knowledge
        ↓
semantic interpretation
        ↓
canonical semantic IR / knowledge algebra
        ↓
validation + normalization + ambiguity preservation
        ↓
validated symbolic package
        ↓
safe Prolog reasoning representation
        ↓
caller-selected sink
  - transient RLM context
  - expert-system KB
  - Project/source knowledge
  - symbolic-memory durable semantic store
  - another application

prolog-rlm owns the compiler, semantic IR, normalization and safe reasoning semantics. It does not own the long-term memory product or require a particular durable store.

Design principle: universal kernel, extensible vocabulary

Do not attempt to freeze one giant ontology containing every domain concept.

Instead define:

a small immutable/versioned knowledge kernel that captures general semantic forms;
typed/open domain vocabularies that may introduce new entity types, relation names and concepts as inert symbolic data;
strict safe lowering rules controlling what can become executable/queryable Prolog;
provenance and interpretation metadata on every material record.

This should make the compiler able to represent unfamiliar domains without redesigning the runtime while preventing arbitrary generated Prolog from becoming executable authority.

Knowledge classes the core MUST represent

The IR/kernel must be expressive enough to encode at least the following classes, including combinations of them.

1. Entities, identity and typing

entities / instances
classes / concepts / types
identity / aliases / synonyms
same-as / different-from
part-whole / membership
roles
attributes / properties

Examples:

instance_of(mara, researcher).
alias(prolog_rlm, 'Prolog-RLM').
part_of(component_a, system_x).
role(alice, maintainer, project_x).

2. Taxonomic / definitional knowledge

is-a hierarchies
definitions
necessary/sufficient conditions
category membership
subtype/supertype
concept equivalence/distinction

Example:

subclass(researcher, person).
definition(stale_preimage, expected_hash_mismatch).

3. Declarative facts and relations

observed facts
n-ary relations
properties
ownership
location
association
composition
status/state

Facts must retain scope, time and provenance where supplied rather than being treated as timeless universal truth by default.

4. General rules and implication

Support rule knowledge equivalent to:

Head :- Body.

including conjunction, disjunction/alternatives where safely represented, scoped variables, guards, exceptions and explicit assumptions.

Examples:

If an expected hash differs, the preimage is stale.
Every current project fact must match the current source generation.

5. Procedures / skills / how-to knowledge

Represent:

goals
procedures
ordered/partially ordered steps
preconditions
postconditions
branch conditions
loops/repetition descriptions
required resources
success criteria
failure conditions
fallback/recovery
alternatives

Example:

To recover X: check A; if B do C; if C fails do D; success requires E.

must become usable procedural knowledge, not a bag of extracted nouns.

6. Events and state transitions

Represent:

events
actions
participants/roles
before/after state
state transitions
creation/destruction/change
trigger/result

This is required for logs, incident histories, conversations, operations and narratives.

7. Temporal knowledge

Represent:

instants / intervals
event time
before / after / during / overlaps
sequence
frequency / recurrence
deadlines
valid-from / valid-until
historical vs current

Relative temporal language may remain unresolved/anchored when the source lacks an absolute reference.

8. Causal / diagnostic knowledge

Represent:

cause / effect
possible cause
necessary cause vs contributing factor
symptom
failure mode
diagnosis
test/discriminator
repair/recovery
prevention

Do not collapse correlation, possibility and asserted causality into one predicate.

9. Constraints and mathematical/quantitative knowledge

Represent:

equality / inequality
ranges
cardinality
ordering constraints
finite-domain constraints
arithmetic relations
units / dimensions
rates / quantities
set membership
resource limits

Lower suitable closed forms into CLP(FD), CLP(Q/R), or other trusted constraint facilities rather than asking an LLM to solve them repeatedly.

10. Spatial knowledge

Represent general spatial relations when present:

inside / contains
adjacent
near / far
above / below
left / right
between
route/connectivity
coordinates/regions where explicitly supplied

Domain-specific geometry is an extension, not a reason to redesign the kernel.

11. Normative / policy / deontic knowledge

Represent semantic statements such as:

required
permitted
forbidden
recommended
discouraged
obligation
exception
policy condition

Important: remembered/compiled normative knowledge is epistemic data. It does not itself grant runtime authority or become trusted host policy.

12. Goals, plans, intentions and preferences

Represent:

goals
subgoals
intentions
plans
preferences
priorities
tradeoffs
success measures

Keep someone said/wants X distinct from X is objectively true.

13. Epistemic knowledge / claims / beliefs

Represent attribution explicitly:

source S claims X
agent A believes X
source S denies X
uncertain X
unknown X
hypothesis X
assumption X

This is essential for LLM logs and web research. The compiler must not flatten attributed claims into unqualified world facts.

14. Uncertainty, confidence and alternatives

Represent at least qualitative uncertainty:

certain / asserted
probable / likely
possible
unlikely
unknown
ambiguous
competing interpretations

Where a source supplies numeric probabilities/confidence, preserve them as data without pretending model confidence is calibrated truth.

15. Defaults, exceptions and defeasible knowledge

Natural language frequently means:

normally P
unless Q
by default R
except when S

The IR must preserve this distinction from strict classical implication. Design an explicit safe default/exception representation rather than abusing negation-as-failure silently.

16. Negative knowledge

Represent explicit negation/denial separately from absence:

explicitly not(P)
unknown(P)
no evidence for P

Missing knowledge must not automatically become false unless a declared closed-world scope says so.

17. Comparative / ranking knowledge

Represent:

better/worse
more/less
preferred over
higher/lower priority
similar/different
ranking criteria

including the basis/context for a comparison where present.

Represent reusable concepts such as:

person / group / organization
membership
role
responsibility
ownership
dependency
communication/reporting relation

without baking one organization's ontology into core.

19. Technical/system knowledge

The generic representation must support system concepts such as:

components
interfaces
capabilities
dependencies
resources
configurations
versions
compatibility
failure paths
invariants
APIs
inputs/outputs

Project/source-specific authoritative observations remain owned by #93/#380; this compiler can represent prose knowledge about systems.

20. Examples, cases and precedents

Represent examples/cases independently from universal rules:

case/example instance
input/state
chosen action
outcome
lesson/rule candidate

This permits later case-based reasoning without mistaking one anecdote for a universal rule.

21. Hypotheses and counterfactuals

Represent:

hypothesis
support/counterevidence
if X were true, Y would follow
candidate explanation
scenario

Counterfactual/scenario knowledge must stay scoped and must not leak into asserted current-world facts.

22. Meta-knowledge

The KB must be able to state things about knowledge itself:

source provenance
trust class
schema/ontology version
rule origin
confidence
conflict
supersession
retraction
scope
validity interval
assumptions
whether a record is observed, extracted or derived

Meta-knowledge is essential, not decorative metadata.

Compositionality

A single passage may contain multiple kinds simultaneously.

Example:

Normally deploy after tests pass, but emergency hotfixes may bypass the full
suite if the on-call lead approves. If a hotfix is used, run the complete
suite within 24 hours.

contains:

procedure/order;
default;
exception;
permission/authorization claim;
role attribution;
temporal obligation;
condition;
follow-up procedure.

The compiler must preserve that structure rather than force the passage into one record type.

Semantic IR requirements

The IR should provide a small set of composable constructors rather than one predicate per English sentence pattern.

Conceptual forms may include:

entity(...).
type(...).
relation(...).
assertion(...).
negation(...).
rule(...).
default_rule(...).
exception(...).
event(...).
state(...).
transition(...).
procedure(...).
step(...).
condition(...).
constraint(...).
temporal(...).
causal(...).
modal(...).
claim(...).
hypothesis(...).
preference(...).
quantity(...).
provenance(...).

Exact representation is a design task. Prefer normalized ground IR with explicit IDs/references and a separate validated lowering layer over model-produced nested callable terms.

Open domain vocabulary

The compiler must support new domain concepts without core changes.

For example a biology source may introduce:

species
predation
habitat

while a software source introduces:

predicate
module
commit
service

These should be registered/represented as symbolic concepts/relations under the same kernel. Unknown domain predicates are data unless explicitly mapped to a trusted semantic extension.

Semantic compilation pipeline

Use multi-pass compilation where needed:

source text
  -> segmentation/discourse structure
  -> entity/concept detection
  -> coreference/alias resolution
  -> proposition/event extraction
  -> rule/condition/modal/temporal interpretation
  -> procedure/causal/constraint induction
  -> cross-sentence linking
  -> candidate semantic IR
  -> type/schema validation
  -> ambiguity/conflict preservation
  -> canonicalization/fingerprinting
  -> safe lowering

An LLM may implement one or more semantic frontend passes. Its output remains candidate data until validation.

Literal vs inferred semantics

Preserve a distinction between:

explicitly stated
strongly implied by linguistic construction
model-inferred/generalized
host/ontology-derived
Prolog-derived

A generalized rule induced from examples should not masquerade as a rule explicitly stated by the source.

Context / discourse

Support knowledge that depends on discourse context:

pronouns/coreference;
ellipsis;
section/topic scope;
quoted speech;
speaker/author attribution;
local definitions;
examples vs assertions;
hypothetical passages;
instructions vs descriptions.

Large documents must be compilable in bounded segments while preserving cross-segment entity/discourse links.

Compiler API direction

Provide semantics equivalent to:

semantic_compile(+Text, +Options, -Outcome).
semantic_compile_segment(+SourceRef, +Span, +Text, +Options, -Outcome).
semantic_compile_document(+Document, +Options, -Outcome).
semantic_validate(+CandidateIR, +Options, -Outcome).
semantic_normalize(+ValidatedIR, +Options, -Outcome).
semantic_lower(+ValidatedIR, +Options, -Outcome).
semantic_explain(+CompiledRecordOrPackage, -Explanation).
semantic_capability(?KnowledgeClass, ?SupportLevel).

Expose the same compiler through a normal capability-gated tool adapter.

The compiler result must be useful with no persistent database installed.

Safe lowering / execution boundary

No arbitrary model/source term may be handed to unrestricted consult/1 or call/1.

Separate:

semantic knowledge
        !=
trusted runtime predicate/capability

A compiled procedure may contain restart(service_x) as a symbolic action. It does not become executable unless a trusted host mapping explicitly binds that semantic action to an admitted capability.

Only closed safe rule constructs are lowered into directly queryable executable Prolog. Other knowledge remains interpreted symbolic data operated on by trusted generic reasoning predicates.

Provenance

Every material semantic record must be able to retain:

source identity/hash
exact source span(s)
document/segment/discourse identity
compiler version
semantic IR version
model/provider/extractor identity if used
compile-run fingerprint
explicit-vs-inferred classification
confidence/uncertainty
scope/context

Downstream stores may add lifecycle/history metadata but must not lose compiler provenance.

Contradiction and plurality

The compiler is not a truth oracle.

Given incompatible sources or ambiguous language, preserve multiple records plus conflict/alternative relationships. Do not force last-write-wins or choose whichever interpretation the model prefers.

Model usage

For arbitrary prose, an LLM may act as semantic parser/compiler frontend. Only actual provider calls are metered.

Prefer deterministic parsers/templates for structured inputs and deterministic normalization/logic after semantic parsing.

The long-term goal is that prose is semantically compiled once and repeated downstream reasoning is primarily Prolog/symbolic.

Child issue set

Create/implement focused children for:

general semantic knowledge algebra / IR v1 and extension mechanism;
multi-pass semantic induction and discourse/entity/coreference pipeline;
safe lowering + defaults/exceptions/negation/uncertainty reasoning semantics;
cross-domain conformance corpus and semantic-loss evaluation;
tool/library integration and large-document compilation.

Required cross-domain acceptance corpus

The test corpus must include materially different knowledge, not ten variants of software troubleshooting. At minimum include fixtures for:

operational how-to/troubleshooting;
software/system architecture;
scientific explanatory text;
historical narrative/events;
policy/rules/exceptions;
quantitative constraints;
temporal scheduling;
spatial description;
comparative/preference text;
attributed claims from conflicting sources;
uncertain/hypothetical knowledge;
definitions/taxonomy;
organization/roles;
case/example-based knowledge;
mixed document containing multiple classes.

For each fixture define expected semantic invariants, not necessarily one brittle exact serialization.

Semantic-loss evaluation

Add evaluation that asks whether material meaning disappeared during compilation.

Metrics/checks should cover at least:

entity/coreference preservation
relation preservation
rule antecedent/consequent preservation
quantifier preservation
negation preservation
modality/uncertainty preservation
time/ordering preservation
causal direction preservation
procedure branch/order preservation
exception/default preservation
attribution preservation
source-span provenance
unsupported hallucinated knowledge

Use deterministic fixture expectations plus optional model-assisted adversarial evaluation; deterministic acceptance remains authoritative.

North-star tests

General knowledge

Compile a heterogeneous multi-domain document set. Remove original prose from context. Ask symbolic queries that require chaining different knowledge classes. Prolog must recover the material semantics from the compiled representation.

Procedural knowledge

Feed several pages of operational prose once. Later ask How do I recover X when Y occurs? with original prose absent. Prolog derives the applicable ordered procedure, branches, conditions and recovery path.

Conflicting research

Compile two sources that disagree. Query what is known. The result identifies both attributed claims, supporting provenance, conflict and unresolved truth status rather than overwriting one source.

Non-goals

no claim that one closed ontology enumerates all domain concepts;
no durable long-term memory store in this epic;
no arbitrary generated Prolog execution;
no assumption that model extraction equals truth;
no requirement that every domain extension become a core predicate;
no collapse of source-code authoritative Project KB into prose-derived semantics.

Refs #376 #377 #380 #381 #70 #93 #219 and downstream lost-rob0t/symbolic-memory semantic-memory epic.

Parent: #376 Downstream consumer: lost-rob0t/symbolic-memory Integrates with: #377, #380, #381, #70, #93, #219 ## Mission Implement a reusable **general semantic knowledge compiler** in `prolog-rlm` that transforms the meaning of arbitrary natural-language material into a safe, validated, extensible symbolic representation that Prolog can query and reason over. The target is deliberately broader than entity extraction, claim extraction, or `how-to` compilation: ```text arbitrary natural-language knowledge ↓ semantic interpretation ↓ canonical semantic IR / knowledge algebra ↓ validation + normalization + ambiguity preservation ↓ validated symbolic package ↓ safe Prolog reasoning representation ↓ caller-selected sink - transient RLM context - expert-system KB - Project/source knowledge - symbolic-memory durable semantic store - another application ``` `prolog-rlm` owns the compiler, semantic IR, normalization and safe reasoning semantics. It does **not** own the long-term memory product or require a particular durable store. ## Design principle: universal kernel, extensible vocabulary Do **not** attempt to freeze one giant ontology containing every domain concept. Instead define: 1. a small immutable/versioned **knowledge kernel** that captures general semantic forms; 2. typed/open domain vocabularies that may introduce new entity types, relation names and concepts as inert symbolic data; 3. strict safe lowering rules controlling what can become executable/queryable Prolog; 4. provenance and interpretation metadata on every material record. This should make the compiler able to represent unfamiliar domains without redesigning the runtime while preventing arbitrary generated Prolog from becoming executable authority. ## Knowledge classes the core MUST represent The IR/kernel must be expressive enough to encode at least the following classes, including combinations of them. ### 1. Entities, identity and typing ```text entities / instances classes / concepts / types identity / aliases / synonyms same-as / different-from part-whole / membership roles attributes / properties ``` Examples: ```prolog instance_of(mara, researcher). alias(prolog_rlm, 'Prolog-RLM'). part_of(component_a, system_x). role(alice, maintainer, project_x). ``` ### 2. Taxonomic / definitional knowledge ```text is-a hierarchies definitions necessary/sufficient conditions category membership subtype/supertype concept equivalence/distinction ``` Example: ```prolog subclass(researcher, person). definition(stale_preimage, expected_hash_mismatch). ``` ### 3. Declarative facts and relations ```text observed facts n-ary relations properties ownership location association composition status/state ``` Facts must retain scope, time and provenance where supplied rather than being treated as timeless universal truth by default. ### 4. General rules and implication Support rule knowledge equivalent to: ```prolog Head :- Body. ``` including conjunction, disjunction/alternatives where safely represented, scoped variables, guards, exceptions and explicit assumptions. Examples: ```text If an expected hash differs, the preimage is stale. Every current project fact must match the current source generation. ``` ### 5. Procedures / skills / how-to knowledge Represent: ```text goals procedures ordered/partially ordered steps preconditions postconditions branch conditions loops/repetition descriptions required resources success criteria failure conditions fallback/recovery alternatives ``` Example: ```text To recover X: check A; if B do C; if C fails do D; success requires E. ``` must become usable procedural knowledge, not a bag of extracted nouns. ### 6. Events and state transitions Represent: ```text events actions participants/roles before/after state state transitions creation/destruction/change trigger/result ``` This is required for logs, incident histories, conversations, operations and narratives. ### 7. Temporal knowledge Represent: ```text instants / intervals event time before / after / during / overlaps sequence frequency / recurrence deadlines valid-from / valid-until historical vs current ``` Relative temporal language may remain unresolved/anchored when the source lacks an absolute reference. ### 8. Causal / diagnostic knowledge Represent: ```text cause / effect possible cause necessary cause vs contributing factor symptom failure mode diagnosis test/discriminator repair/recovery prevention ``` Do not collapse correlation, possibility and asserted causality into one predicate. ### 9. Constraints and mathematical/quantitative knowledge Represent: ```text equality / inequality ranges cardinality ordering constraints finite-domain constraints arithmetic relations units / dimensions rates / quantities set membership resource limits ``` Lower suitable closed forms into CLP(FD), CLP(Q/R), or other trusted constraint facilities rather than asking an LLM to solve them repeatedly. ### 10. Spatial knowledge Represent general spatial relations when present: ```text inside / contains adjacent near / far above / below left / right between route/connectivity coordinates/regions where explicitly supplied ``` Domain-specific geometry is an extension, not a reason to redesign the kernel. ### 11. Normative / policy / deontic knowledge Represent semantic statements such as: ```text required permitted forbidden recommended discouraged obligation exception policy condition ``` **Important:** remembered/compiled normative knowledge is epistemic data. It does not itself grant runtime authority or become trusted host policy. ### 12. Goals, plans, intentions and preferences Represent: ```text goals subgoals intentions plans preferences priorities tradeoffs success measures ``` Keep `someone said/wants X` distinct from `X is objectively true`. ### 13. Epistemic knowledge / claims / beliefs Represent attribution explicitly: ```text source S claims X agent A believes X source S denies X uncertain X unknown X hypothesis X assumption X ``` This is essential for LLM logs and web research. The compiler must not flatten attributed claims into unqualified world facts. ### 14. Uncertainty, confidence and alternatives Represent at least qualitative uncertainty: ```text certain / asserted probable / likely possible unlikely unknown ambiguous competing interpretations ``` Where a source supplies numeric probabilities/confidence, preserve them as data without pretending model confidence is calibrated truth. ### 15. Defaults, exceptions and defeasible knowledge Natural language frequently means: ```text normally P unless Q by default R except when S ``` The IR must preserve this distinction from strict classical implication. Design an explicit safe default/exception representation rather than abusing negation-as-failure silently. ### 16. Negative knowledge Represent explicit negation/denial separately from absence: ```text explicitly not(P) unknown(P) no evidence for P ``` Missing knowledge must not automatically become false unless a declared closed-world scope says so. ### 17. Comparative / ranking knowledge Represent: ```text better/worse more/less preferred over higher/lower priority similar/different ranking criteria ``` including the basis/context for a comparison where present. ### 18. Social / organizational knowledge Represent reusable concepts such as: ```text person / group / organization membership role responsibility ownership dependency communication/reporting relation ``` without baking one organization's ontology into core. ### 19. Technical/system knowledge The generic representation must support system concepts such as: ```text components interfaces capabilities dependencies resources configurations versions compatibility failure paths invariants APIs inputs/outputs ``` Project/source-specific authoritative observations remain owned by #93/#380; this compiler can represent prose knowledge *about* systems. ### 20. Examples, cases and precedents Represent examples/cases independently from universal rules: ```text case/example instance input/state chosen action outcome lesson/rule candidate ``` This permits later case-based reasoning without mistaking one anecdote for a universal rule. ### 21. Hypotheses and counterfactuals Represent: ```text hypothesis support/counterevidence if X were true, Y would follow candidate explanation scenario ``` Counterfactual/scenario knowledge must stay scoped and must not leak into asserted current-world facts. ### 22. Meta-knowledge The KB must be able to state things about knowledge itself: ```text source provenance trust class schema/ontology version rule origin confidence conflict supersession retraction scope validity interval assumptions whether a record is observed, extracted or derived ``` Meta-knowledge is essential, not decorative metadata. ## Compositionality A single passage may contain multiple kinds simultaneously. Example: ```text Normally deploy after tests pass, but emergency hotfixes may bypass the full suite if the on-call lead approves. If a hotfix is used, run the complete suite within 24 hours. ``` contains: - procedure/order; - default; - exception; - permission/authorization claim; - role attribution; - temporal obligation; - condition; - follow-up procedure. The compiler must preserve that structure rather than force the passage into one record type. ## Semantic IR requirements The IR should provide a small set of composable constructors rather than one predicate per English sentence pattern. Conceptual forms may include: ```prolog entity(...). type(...). relation(...). assertion(...). negation(...). rule(...). default_rule(...). exception(...). event(...). state(...). transition(...). procedure(...). step(...). condition(...). constraint(...). temporal(...). causal(...). modal(...). claim(...). hypothesis(...). preference(...). quantity(...). provenance(...). ``` Exact representation is a design task. Prefer normalized ground IR with explicit IDs/references and a separate validated lowering layer over model-produced nested callable terms. ## Open domain vocabulary The compiler must support new domain concepts without core changes. For example a biology source may introduce: ```text species predation habitat ``` while a software source introduces: ```text predicate module commit service ``` These should be registered/represented as symbolic concepts/relations under the same kernel. Unknown domain predicates are data unless explicitly mapped to a trusted semantic extension. ## Semantic compilation pipeline Use multi-pass compilation where needed: ```text source text -> segmentation/discourse structure -> entity/concept detection -> coreference/alias resolution -> proposition/event extraction -> rule/condition/modal/temporal interpretation -> procedure/causal/constraint induction -> cross-sentence linking -> candidate semantic IR -> type/schema validation -> ambiguity/conflict preservation -> canonicalization/fingerprinting -> safe lowering ``` An LLM may implement one or more semantic frontend passes. Its output remains candidate data until validation. ## Literal vs inferred semantics Preserve a distinction between: ```text explicitly stated strongly implied by linguistic construction model-inferred/generalized host/ontology-derived Prolog-derived ``` A generalized rule induced from examples should not masquerade as a rule explicitly stated by the source. ## Context / discourse Support knowledge that depends on discourse context: - pronouns/coreference; - ellipsis; - section/topic scope; - quoted speech; - speaker/author attribution; - local definitions; - examples vs assertions; - hypothetical passages; - instructions vs descriptions. Large documents must be compilable in bounded segments while preserving cross-segment entity/discourse links. ## Compiler API direction Provide semantics equivalent to: ```prolog semantic_compile(+Text, +Options, -Outcome). semantic_compile_segment(+SourceRef, +Span, +Text, +Options, -Outcome). semantic_compile_document(+Document, +Options, -Outcome). semantic_validate(+CandidateIR, +Options, -Outcome). semantic_normalize(+ValidatedIR, +Options, -Outcome). semantic_lower(+ValidatedIR, +Options, -Outcome). semantic_explain(+CompiledRecordOrPackage, -Explanation). semantic_capability(?KnowledgeClass, ?SupportLevel). ``` Expose the same compiler through a normal capability-gated tool adapter. The compiler result must be useful with no persistent database installed. ## Safe lowering / execution boundary No arbitrary model/source term may be handed to unrestricted `consult/1` or `call/1`. Separate: ```text semantic knowledge != trusted runtime predicate/capability ``` A compiled procedure may contain `restart(service_x)` as a symbolic action. It does not become executable unless a trusted host mapping explicitly binds that semantic action to an admitted capability. Only closed safe rule constructs are lowered into directly queryable executable Prolog. Other knowledge remains interpreted symbolic data operated on by trusted generic reasoning predicates. ## Provenance Every material semantic record must be able to retain: ```text source identity/hash exact source span(s) document/segment/discourse identity compiler version semantic IR version model/provider/extractor identity if used compile-run fingerprint explicit-vs-inferred classification confidence/uncertainty scope/context ``` Downstream stores may add lifecycle/history metadata but must not lose compiler provenance. ## Contradiction and plurality The compiler is not a truth oracle. Given incompatible sources or ambiguous language, preserve multiple records plus conflict/alternative relationships. Do not force last-write-wins or choose whichever interpretation the model prefers. ## Model usage For arbitrary prose, an LLM may act as semantic parser/compiler frontend. Only actual provider calls are metered. Prefer deterministic parsers/templates for structured inputs and deterministic normalization/logic after semantic parsing. The long-term goal is that prose is semantically compiled once and repeated downstream reasoning is primarily Prolog/symbolic. ## Child issue set Create/implement focused children for: 1. general semantic knowledge algebra / IR v1 and extension mechanism; 2. multi-pass semantic induction and discourse/entity/coreference pipeline; 3. safe lowering + defaults/exceptions/negation/uncertainty reasoning semantics; 4. cross-domain conformance corpus and semantic-loss evaluation; 5. tool/library integration and large-document compilation. ## Required cross-domain acceptance corpus The test corpus must include materially different knowledge, not ten variants of software troubleshooting. At minimum include fixtures for: - operational how-to/troubleshooting; - software/system architecture; - scientific explanatory text; - historical narrative/events; - policy/rules/exceptions; - quantitative constraints; - temporal scheduling; - spatial description; - comparative/preference text; - attributed claims from conflicting sources; - uncertain/hypothetical knowledge; - definitions/taxonomy; - organization/roles; - case/example-based knowledge; - mixed document containing multiple classes. For each fixture define expected semantic invariants, not necessarily one brittle exact serialization. ## Semantic-loss evaluation Add evaluation that asks whether material meaning disappeared during compilation. Metrics/checks should cover at least: ```text entity/coreference preservation relation preservation rule antecedent/consequent preservation quantifier preservation negation preservation modality/uncertainty preservation time/ordering preservation causal direction preservation procedure branch/order preservation exception/default preservation attribution preservation source-span provenance unsupported hallucinated knowledge ``` Use deterministic fixture expectations plus optional model-assisted adversarial evaluation; deterministic acceptance remains authoritative. ## North-star tests ### General knowledge Compile a heterogeneous multi-domain document set. Remove original prose from context. Ask symbolic queries that require chaining different knowledge classes. Prolog must recover the material semantics from the compiled representation. ### Procedural knowledge Feed several pages of operational prose once. Later ask `How do I recover X when Y occurs?` with original prose absent. Prolog derives the applicable ordered procedure, branches, conditions and recovery path. ### Conflicting research Compile two sources that disagree. Query what is known. The result identifies both attributed claims, supporting provenance, conflict and unresolved truth status rather than overwriting one source. ## Non-goals - no claim that one closed ontology enumerates all domain concepts; - no durable long-term memory store in this epic; - no arbitrary generated Prolog execution; - no assumption that model extraction equals truth; - no requirement that every domain extension become a core predicate; - no collapse of source-code authoritative Project KB into prose-derived semantics. Refs #376 #377 #380 #381 #70 #93 #219 and downstream `lost-rob0t/symbolic-memory` semantic-memory epic.

nsaspy commented

2026-09-08 01:46:57 +00:00

Author

Owner

Downstream durable-memory consumers are now tracked in lost-rob0t/symbolic-memory:

symbolic-memory#4 parent semantic-memory epic
#5 compiler adapter/projection lifecycle
#6 append-only projection ledger
#7 symbolic recall/query
#8 large-corpus/backfill ingestion
#9 native/MCP exposure and Prolog-RLM integration

Keep this issue compiler/library/tool-only. Do not move symbolic-memory namespace, retention, durable projection ledger, corpus backfill, or memory transport semantics into Prolog-RLM.

Downstream durable-memory consumers are now tracked in `lost-rob0t/symbolic-memory`: - symbolic-memory#4 parent semantic-memory epic - #5 compiler adapter/projection lifecycle - #6 append-only projection ledger - #7 symbolic recall/query - #8 large-corpus/backfill ingestion - #9 native/MCP exposure and Prolog-RLM integration Keep this issue compiler/library/tool-only. Do not move symbolic-memory namespace, retention, durable projection ledger, corpus backfill, or memory transport semantics into Prolog-RLM.

nsaspy commented

2026-09-08 01:46:57 +00:00

Author

Owner

Machine Spirit #405A synthesis handoff

Depth 8A baseline synthesis (#405) preserves this compiler boundary exactly. COVENANT-MS treats #388/#392-#396 as the semantic/meaning side of a typed contract spine; it does not introduce a durable-memory dependency into Prolog-RLM.

Baseline handoff from compiler to the rest of Machine Spirit must identify the exact source/version, semantic schema/profile, compiler run, package digest, export/license receipts, material ambiguity/gaps and provider usage. A validated package is still semantic candidate/represented knowledge, not world truth, host authority, VERIFY success or learned-policy admission.

The corresponding end-to-end conformance capsule begins with #395 meaning/commitment invariants and then follows the exact package into durable projection/retrieval/epistemics/procedure/VERIFY. Any later component must preserve compiler provenance and declared losses rather than reconstructing stronger semantics from a flattened view.

Refs #397 #405 #392-#396 and symbolic-memory#4/#5/#11.

## Machine Spirit #405A synthesis handoff Depth 8A baseline synthesis (`#405`) preserves this compiler boundary exactly. **COVENANT-MS** treats `#388/#392-#396` as the semantic/meaning side of a typed contract spine; it does not introduce a durable-memory dependency into Prolog-RLM. Baseline handoff from compiler to the rest of Machine Spirit must identify the exact source/version, semantic schema/profile, compiler run, package digest, export/license receipts, material ambiguity/gaps and provider usage. A validated package is still **semantic candidate/represented knowledge**, not world truth, host authority, VERIFY success or learned-policy admission. The corresponding end-to-end conformance capsule begins with #395 meaning/commitment invariants and then follows the exact package into durable projection/retrieval/epistemics/procedure/VERIFY. Any later component must preserve compiler provenance and declared losses rather than reconstructing stronger semantics from a flattened view. Refs #397 #405 #392-#396 and symbolic-memory#4/#5/#11.

nsaspy commented

2026-09-08 01:46:57 +00:00

Author

Owner

Machine Spirit #405D executable-conformance handoff: Cycle-1 acceptance failed because the canonical #388/#392-#396 semantic_compile family is not yet present as executable evidence on current main. This is not a redesign of #388. The implementation gate now needs to produce MSD-SEM-* evidence capsules covering heterogeneous compile→source-free query, forbidden semantic overpromotion, canonical fingerprints, bounded ambiguity/resource behavior, from-scratch vs incremental equivalence, and provider-disabled deterministic lanes. Parent #397 now contains the proposed Cycle-2 ranking; do not treat issue design completion as Machine Spirit evidence.

Machine Spirit #405D executable-conformance handoff: Cycle-1 acceptance failed because the canonical #388/#392-#396 `semantic_compile` family is not yet present as executable evidence on current `main`. This is not a redesign of #388. The implementation gate now needs to produce `MSD-SEM-*` evidence capsules covering heterogeneous compile→source-free query, forbidden semantic overpromotion, canonical fingerprints, bounded ambiguity/resource behavior, from-scratch vs incremental equivalence, and provider-disabled deterministic lanes. Parent #397 now contains the proposed Cycle-2 ranking; do not treat issue design completion as Machine Spirit evidence.