Skip to content

[IMPROVEMENT] Convert experiential memory into an evidence-grounded, trigger-monitored skill library #3210

Description

@Lexus2016

Improvement Description

Problem Statement

Memory failures are a recurring top failure bucket in Hermes pipeline metrics. Hermes already stores memories and already has a SKILL.md corpus plus a SkillCrystallizer that can turn successful traces into provisional skill candidates. What is missing is the operational link: experiential memory mostly sits as passive context the model must re-reason over, rather than being promoted into skills with explicit trigger conditions that are monitored at runtime and injected when the state matches.

Proposed Solution

Evolve the memory-to-skill path into an evidence-grounded co-evolution governance layer. For each validated memory/trace, produce a skill entry with an explicit trigger function μ(state, task-kind). At runtime, the agent monitors the current state and, when a trigger fires, retrieves the crystallized skill into context to guide action instead of re-deriving the workflow from scratch.

Concrete steps:

  1. Evidence bar — only promote memories that pass the existing meets_evidence_bar gates (reusability ≥ 0.6, ≥ 2 distinct tools, ≥ 3 actions, verified success). This guardrail already exists in evolution/lib/skill_crystallizer.py and must remain hard.
  2. Trigger extraction — when crystallizing a skill, extract and store the trigger conditions under which it should fire (task kind, tool constellation, error class, user intent signal). Store them in the skill frontmatter so they are inspectable.
  3. Runtime trigger monitor — add a lightweight monitor that, before each model turn, scores current state against stored trigger conditions and retrieves matching skills when the score exceeds a calibrated threshold.
  4. Co-evolution loop — when a triggered skill is used, record outcome; demote or revise skills whose triggered use repeatedly fails, so the library improves rather than fossilizes.

Value Proposition

  • Impact: High (0.8)
  • Effort: Medium (0.5)
  • Priority Score: 1.28

Research Evidence

Implementation Plan

  1. Extend evolution/lib/skill_crystallizer.py to require and validate trigger metadata in skill frontmatter.
  2. Add a trigger matcher/scorer (state + skill trigger → relevance score) in a new small module under evolution/lib/.
  3. Wire the trigger monitor into the agent turn loop so retrieved skills are injected transparently and respect prompt-caching constraints (skills are part of the stable prefix).
  4. Add feedback logging: triggered skill usage + outcome is written back to the skill library for periodic re-evaluation.

Success Criteria

  • Crystallized skills carry explicit, validated trigger conditions.
  • Runtime monitor retrieves skills only when triggers fire, with logged confidence scores.
  • Pipeline memory-failure rate trend shows reduction within a 14-day window.
  • Triggered skills that repeatedly fail are automatically demoted to provisional status.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestneeds-splitWanted, but exceeds one cycle — needs decompositionneeds-workBlocked by code-review (dead code / not integrated)proposalEvolution-generated improvement proposalresearch-generatedCreated by the evolution research cycle

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions