nsaspy/agentProlog

Fork 0

[EPIC] Personal skill refinery: observe workflows, synthesize candidates, RAGE-evaluate, and import external skills #6

New issue

Open

opened 2026-08-22 01:53:25 +00:00 by lost-rob0t · 1 comment

lost-rob0t commented

2026-08-22 01:53:25 +00:00

(Migrated from github.com)

Companion to lost-rob0t/prolog-rlm#167.

Mission

Build the downstream product layer that reviews real operator work, detects repeated workflows/failures/preferences, proposes custom native Prolog skill candidates, evaluates them through the reusable prolog-rlm SKILLS runtime, and imports skills from the operator's existing agent environments.

Initial source environments:

Claude / Claude Code
OpenCode
Agent Zero
AgentProlog native skills

Portability direction is intentionally into AgentProlog/prolog-rlm:

Claude / OpenCode / Agent Zero / standard SKILL.md
                     |
                     v
               import adapters
                     |
                     v
          canonical native Prolog skill
                     |
              prolog-rlm runtime

AgentProlog does not need to export its richer native skills back into Claude/OpenCode/Agent Zero formats. Do not constrain the native schema for reverse portability.

Keep generic skill loading, normalization, prompt selection, graph construction, evaluation, lifecycle, evolution, evidence, context budgeting and bounded subagent semantics upstream in prolog-rlm.

Runtime/library flow:

prolog-rlm -> AgentProlog

In dependency terms, AgentProlog depends on and consumes prolog-rlm.

Target RAGE loop

selected work/session sources
-> normalize observations
-> detect repeated patterns / friction / failures
-> classify candidate improvement
-> synthesize native Prolog skill candidate
-> build positive + negative trigger cases
-> baseline vs candidate eval
-> adversarial/reviewer evaluation
-> evolve/revise when useful
-> trusted promotion decision
-> activate native skill
-> observe future outcomes

Separately, external skill sources can enter through import adapters and normalize into the same native skill representation before evaluation/activation.

Observation adapters

Implement bounded adapters for available operator-authorized sources, including where practical:

AgentProlog session/trace history;
OpenCode sessions/artifacts/configuration;
Claude/Claude Code sessions and skill directories where accessible;
Agent Zero sessions/projects/skills where accessible;
imported ChatGPT/user-provided archives or transcripts as explicit inputs;
repository issues/PRs/commits and local project evidence when relevant to workflow inference.

Adapters should normalize observations into product-owned records with provenance rather than feeding raw lifetime transcripts into every model call.

Do not assume every external harness exposes the same history API. Prefer explicit import or documented storage contracts over brittle scraping.

External skill import

Consume the upstream import/normalization contract from prolog-rlm#173.

AgentProlog should be able to discover/import skills from supported external agent environments and normalize them into the canonical native Prolog representation.

Requirements:

preserve source provenance;
do not require external skills to contain Prolog-specific metadata;
fill missing native routing/graph semantics through documented deterministic defaults or later trusted enrichment;
never grant authority/capabilities merely because an imported skill requested them;
imported skills enter the same evaluation/lifecycle path as generated native candidates where appropriate;
avoid keeping multiple independently authoritative copies after import.

Reverse export to external agents is a non-goal.

Candidate classification

Before creating a native skill, classify an observed pattern into the right mechanism:

skill
project instruction
reference/knowledge
script/tool
hook/policy
agent role
AgentProlog config
memory/preference
ignore / insufficient evidence

The classifier must preserve evidence and confidence. One unusual session should not automatically become a global skill.

Skill synthesis

For genuine skill candidates:

generate concise discovery metadata;
generate a lean operational native SKILL.md;
populate useful Prolog routing/dependency/graph semantics;
split large reusable knowledge into references/resources;
move deterministic repeated procedures into scripts/tools when appropriate;
create positive trigger examples and adversarial negative examples;
record provenance showing which observed workflow patterns motivated the candidate;
scope candidates globally, per project, or per environment where appropriate.

Evaluation / RAGE integration

Consume upstream prolog-rlm#167 contracts rather than implementing local scoring engines.

Use:

selection evals for trigger precision/recall;
baseline-vs-skill task evals;
verifier/evidence-backed success where possible;
token/cost/latency/tool/subagent metrics;
bounded evolution and candidate lineage;
explicit lifecycle promotion/rejection/rollback.

A candidate should be promotable only after evidence meets configured policy. Model preference alone is not promotion evidence.

Privacy / provenance boundary

Observation is limited to explicitly configured/available operator sources.
Preserve source provenance and scope for inferred patterns.
Do not put secrets/credentials or unrelated private content into generated skills.
Product-specific redaction/filtering belongs here, before candidate material leaves the observation corpus.
Native promoted skills contain only distilled operational material, not raw source conversations.

Acceptance

at least two different source adapters normalize workflow observations into one provenance-preserving schema;
repeated behavior can be distinguished from one-off behavior;
observations can be classified into skill/tool/policy/config/memory/etc. rather than forcing everything into SKILL.md;
one real workflow produces a native Prolog skill candidate plus positive/negative trigger cases;
candidate is evaluated through upstream prolog-rlm skill eval contracts;
candidate can be rejected without polluting the active skill set;
standard/Claude/OpenCode/Agent Zero skills can be imported through supported adapters into the canonical native representation;
imported skill provenance remains inspectable;
accepted native skills can become active without any requirement to export them back to external agents;
lifecycle/evaluation provenance can explain why a skill became active;
no duplicate generic prompt compiler/evolution/verifier runtime is added downstream.

Non-goals

no reverse skill export to Claude/OpenCode/Agent Zero;
no requirement that native Prolog skills remain externally portable;
no duplicate upstream loader/compiler/graph/eval/evolution implementation;
no blind ingestion of every available transcript without configured scope.

Coordination

Upstream parent/runtime: lost-rob0t/prolog-rlm#167.
Native format/import contract: lost-rob0t/prolog-rlm#173.

Consume stable public seams from #117/#168/#169/#170/#171/#172/#173 as they land; do not fork private copies merely to move faster.

No coding in the current RAGE/design slice.

Companion to `lost-rob0t/prolog-rlm#167`. ## Mission Build the downstream product layer that reviews real operator work, detects repeated workflows/failures/preferences, proposes custom native Prolog skill candidates, evaluates them through the reusable `prolog-rlm` SKILLS runtime, and imports skills from the operator's existing agent environments. Initial source environments: ```text Claude / Claude Code OpenCode Agent Zero AgentProlog native skills ``` Portability direction is intentionally **into AgentProlog/prolog-rlm**: ```text Claude / OpenCode / Agent Zero / standard SKILL.md | v import adapters | v canonical native Prolog skill | prolog-rlm runtime ``` AgentProlog does **not** need to export its richer native skills back into Claude/OpenCode/Agent Zero formats. Do not constrain the native schema for reverse portability. Keep generic skill loading, normalization, prompt selection, graph construction, evaluation, lifecycle, evolution, evidence, context budgeting and bounded subagent semantics upstream in `prolog-rlm`. Runtime/library flow: ```text prolog-rlm -> AgentProlog ``` In dependency terms, AgentProlog depends on and consumes `prolog-rlm`. ## Target RAGE loop ```text selected work/session sources -> normalize observations -> detect repeated patterns / friction / failures -> classify candidate improvement -> synthesize native Prolog skill candidate -> build positive + negative trigger cases -> baseline vs candidate eval -> adversarial/reviewer evaluation -> evolve/revise when useful -> trusted promotion decision -> activate native skill -> observe future outcomes ``` Separately, external skill sources can enter through import adapters and normalize into the same native skill representation before evaluation/activation. ## Observation adapters Implement bounded adapters for available operator-authorized sources, including where practical: - AgentProlog session/trace history; - OpenCode sessions/artifacts/configuration; - Claude/Claude Code sessions and skill directories where accessible; - Agent Zero sessions/projects/skills where accessible; - imported ChatGPT/user-provided archives or transcripts as explicit inputs; - repository issues/PRs/commits and local project evidence when relevant to workflow inference. Adapters should normalize observations into product-owned records with provenance rather than feeding raw lifetime transcripts into every model call. Do not assume every external harness exposes the same history API. Prefer explicit import or documented storage contracts over brittle scraping. ## External skill import Consume the upstream import/normalization contract from `prolog-rlm#173`. AgentProlog should be able to discover/import skills from supported external agent environments and normalize them into the canonical native Prolog representation. Requirements: - preserve source provenance; - do not require external skills to contain Prolog-specific metadata; - fill missing native routing/graph semantics through documented deterministic defaults or later trusted enrichment; - never grant authority/capabilities merely because an imported skill requested them; - imported skills enter the same evaluation/lifecycle path as generated native candidates where appropriate; - avoid keeping multiple independently authoritative copies after import. Reverse export to external agents is a non-goal. ## Candidate classification Before creating a native skill, classify an observed pattern into the right mechanism: ```text skill project instruction reference/knowledge script/tool hook/policy agent role AgentProlog config memory/preference ignore / insufficient evidence ``` The classifier must preserve evidence and confidence. One unusual session should not automatically become a global skill. ## Skill synthesis For genuine skill candidates: - generate concise discovery metadata; - generate a lean operational native `SKILL.md`; - populate useful Prolog routing/dependency/graph semantics; - split large reusable knowledge into references/resources; - move deterministic repeated procedures into scripts/tools when appropriate; - create positive trigger examples and adversarial negative examples; - record provenance showing which observed workflow patterns motivated the candidate; - scope candidates globally, per project, or per environment where appropriate. ## Evaluation / RAGE integration Consume upstream `prolog-rlm#167` contracts rather than implementing local scoring engines. Use: - selection evals for trigger precision/recall; - baseline-vs-skill task evals; - verifier/evidence-backed success where possible; - token/cost/latency/tool/subagent metrics; - bounded evolution and candidate lineage; - explicit lifecycle promotion/rejection/rollback. A candidate should be promotable only after evidence meets configured policy. Model preference alone is not promotion evidence. ## Privacy / provenance boundary - Observation is limited to explicitly configured/available operator sources. - Preserve source provenance and scope for inferred patterns. - Do not put secrets/credentials or unrelated private content into generated skills. - Product-specific redaction/filtering belongs here, before candidate material leaves the observation corpus. - Native promoted skills contain only distilled operational material, not raw source conversations. ## Acceptance - [ ] at least two different source adapters normalize workflow observations into one provenance-preserving schema; - [ ] repeated behavior can be distinguished from one-off behavior; - [ ] observations can be classified into skill/tool/policy/config/memory/etc. rather than forcing everything into SKILL.md; - [ ] one real workflow produces a native Prolog skill candidate plus positive/negative trigger cases; - [ ] candidate is evaluated through upstream prolog-rlm skill eval contracts; - [ ] candidate can be rejected without polluting the active skill set; - [ ] standard/Claude/OpenCode/Agent Zero skills can be imported through supported adapters into the canonical native representation; - [ ] imported skill provenance remains inspectable; - [ ] accepted native skills can become active without any requirement to export them back to external agents; - [ ] lifecycle/evaluation provenance can explain why a skill became active; - [ ] no duplicate generic prompt compiler/evolution/verifier runtime is added downstream. ## Non-goals - no reverse skill export to Claude/OpenCode/Agent Zero; - no requirement that native Prolog skills remain externally portable; - no duplicate upstream loader/compiler/graph/eval/evolution implementation; - no blind ingestion of every available transcript without configured scope. ## Coordination Upstream parent/runtime: `lost-rob0t/prolog-rlm#167`. Native format/import contract: `lost-rob0t/prolog-rlm#173`. Consume stable public seams from #117/#168/#169/#170/#171/#172/#173 as they land; do not fork private copies merely to move faster. No coding in the current RAGE/design slice.

nsaspy referenced this issue

2026-09-20 01:40:01 +00:00

[RAGE-10][P0] Add deterministic end-to-end RAGE golden corpus and CI regression gate #38

nsaspy referenced this issue

2026-09-20 01:40:01 +00:00

[RAGE-9][P0] Ship headless refinery commands and one resumable RAGE pipeline over AgentProlog #39

nsaspy referenced this issue

2026-09-20 01:40:01 +00:00

[RAGE-8][P1] Discover and import external skills into the canonical native Prolog skill path #40

nsaspy referenced this issue

2026-09-20 01:40:01 +00:00

[RAGE-7][P1] Add bounded candidate revision/evolution and trusted lifecycle promotion workflow #41

nsaspy referenced this issue

2026-09-20 01:40:01 +00:00

[RAGE-6][P0] Orchestrate candidate RAGE evaluation through upstream selection, outcome, and reviewer contracts #42

nsaspy referenced this issue

2026-09-20 01:40:01 +00:00

[RAGE-5][P0] Synthesize native Prolog skill candidates plus positive/negative trigger corpora #43

nsaspy referenced this issue

2026-09-20 01:40:01 +00:00

[RAGE-4][P0] Mine repeated workflow patterns and classify the correct improvement mechanism #44

nsaspy referenced this issue

2026-09-20 01:40:01 +00:00

[RAGE-3][P1] Add bounded external workflow-history adapters for OpenCode, Claude, Agent Zero, and explicit transcript imports #45

nsaspy referenced this issue

2026-09-20 01:40:02 +00:00

[RAGE-2][P0] Ingest AgentProlog sessions, traces, issues, PRs, commits, and project evidence #46

nsaspy referenced this issue

2026-09-20 01:40:02 +00:00

[RAGE-1][P0] Freeze downstream observation, provenance, scope, and redaction records #47

nsaspy commented

2026-09-20 01:40:38 +00:00

Owner

RAGE implementation tree created on 2026-09-19.

Dependency-ordered downstream slices:

#38 — RAGE-1: observation/provenance/scope/redaction contract
#39 — RAGE-2: AgentProlog + Git/project evidence adapters
#40 — RAGE-3: external workflow-history adapters
#41 — RAGE-4: repeated-pattern mining + mechanism classifier
#42 — RAGE-5: native skill candidate synthesis + positive/negative trigger corpora
#43 — RAGE-6: upstream-backed selection/outcome/reviewer evaluation orchestration
#44 — RAGE-7: bounded revision/evolution + trusted lifecycle workflow
#45 — RAGE-8: external skill discovery/import
#46 — RAGE-9: headless resumable refinery commands/pipeline
#47 — RAGE-10: deterministic golden corpus + CI regression gate

Critical boundary preserved:

AgentProlog
  owns observation/mining/classification/candidate construction/import/product orchestration
      |
      v
prolog-rlm
  owns generic skill selection/evals/lifecycle/evolution/delegation/runtime

Upstream dependencies remain #168/#169/#170/#171/#172/#173 in lost-rob0t/prolog-rlm. Downstream slices should consume those public contracts and stop at a real upstream blocker rather than growing a parallel runtime.

RAGE implementation tree created on 2026-09-19. Dependency-ordered downstream slices: 1. #38 — RAGE-1: observation/provenance/scope/redaction contract 2. #39 — RAGE-2: AgentProlog + Git/project evidence adapters 3. #40 — RAGE-3: external workflow-history adapters 4. #41 — RAGE-4: repeated-pattern mining + mechanism classifier 5. #42 — RAGE-5: native skill candidate synthesis + positive/negative trigger corpora 6. #43 — RAGE-6: upstream-backed selection/outcome/reviewer evaluation orchestration 7. #44 — RAGE-7: bounded revision/evolution + trusted lifecycle workflow 8. #45 — RAGE-8: external skill discovery/import 9. #46 — RAGE-9: headless resumable refinery commands/pipeline 10. #47 — RAGE-10: deterministic golden corpus + CI regression gate Critical boundary preserved: ```text AgentProlog owns observation/mining/classification/candidate construction/import/product orchestration | v prolog-rlm owns generic skill selection/evals/lifecycle/evolution/delegation/runtime ``` Upstream dependencies remain #168/#169/#170/#171/#172/#173 in `lost-rob0t/prolog-rlm`. Downstream slices should consume those public contracts and stop at a real upstream blocker rather than growing a parallel runtime.