Skip to content

feat: Improve quality of rpi-agent.agent.md#1949

Open
jkim323 wants to merge 17 commits into
mainfrom
improve-rpi-agent
Open

feat: Improve quality of rpi-agent.agent.md#1949
jkim323 wants to merge 17 commits into
mainfrom
improve-rpi-agent

Conversation

@jkim323

@jkim323 jkim323 commented Jun 14, 2026

Copy link
Copy Markdown
Collaborator

Pull Request

Description

Refactors verbose wording in the RPI Agent instructions while preserving the same workflow rigor and behavior. This change tightens repeated guidance across the agent file, especially around validation and discovery, so the instructions are easier to maintain and read without changing when or how the agent performs Research, Plan, Implement, Review, or Discover phases.

Following updates have been made:

  • New Execution Model by Phase table (under Difficulty Levels): Consolidates the per-difficulty guidance that was previously scattered as inline bullet lists across every phase. The table content matches what was already present; it centralizes rather than changes the rules.
  • Tracking Artifacts section restructured into a table: Replaced seven separate Artifact Name subsections (each with a "Path:" line, "Create this..." paragraph, and bullet list) with a single table listing all artifacts with their paths and creation conditions.
  • Terminology normalization: "document-backed" → "artifact-backed" and "heavier-weight workflow" → "artifact-backed model" throughout. Consistent naming, no semantic shift.
  • Phase 5 discovery - Step 2: Reason About Next Work: Collapsed 6 verbose question-pair bullets into 5 concise category statements. All six original discovery categories are preserved — they're just expressed more concisely.
  • Removed duplicate bullet: The duplicated validation list in Phase 4 Step 2 now references Phase 3 Step 3 instead of repeating the same four bullets.

Related Issue(s)

Fixes #1939

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with prompt-builder agent and addressed all feedback
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)

Note for AI Artifact Contributors:

  • Agents: Research, indexing/referencing other project (using standard VS Code GitHub Copilot/MCP tools), planning, and general implementation agents likely already exist. Review .github/agents/ before creating new ones.
  • Skills: Must include both bash and PowerShell scripts. See Skills.
  • Model Versions: Only contributions targeting the latest Anthropic and OpenAI models will be accepted. Older model versions (e.g., GPT-3.5, Claude 3) will be rejected.
  • See Agents Not Accepted and Model Version Requirements.

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Sample Prompts (for AI Artifact Contributions)

User Request:

Execution Flow:

Output Artifacts:

Success Indicators:

For detailed contribution requirements, see:

Testing

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

AI Artifact Contributions

  • Used /prompt-analyze to review contribution
  • Addressed all feedback from prompt-builder review
  • Verified contribution follows common standards and type-specific requirements

Required Automated Checks

The following validation commands must pass before merging:

  • Markdown linting: npm run lint:md
  • Spell checking: npm run spell-check
  • Frontmatter validation: npm run lint:frontmatter
  • Skill structure validation: npm run validate:skills
  • Link validation: npm run lint:md-links
  • PowerShell analysis: npm run lint:ps
  • Plugin freshness: npm run plugin:generate
  • Docusaurus tests: npm run docs:test

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues
  • Security-related scripts follow the principle of least privilege

Additional Notes

This update is not intended to change the behavior of the agent - only to improve readability and quality.

@jkim323 jkim323 marked this pull request as ready for review June 14, 2026 18:34
@jkim323 jkim323 requested a review from a team as a code owner June 14, 2026 18:34
@codecov-commenter

codecov-commenter commented Jun 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.82%. Comparing base (a847cfa) to head (2325f0a).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1949      +/-   ##
==========================================
- Coverage   80.82%   80.82%   -0.01%     
==========================================
  Files         117      117              
  Lines       19095    19100       +5     
==========================================
+ Hits        15433    15437       +4     
- Misses       3662     3663       +1     
Flag Coverage Δ
pester 84.64% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bindsi bindsi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved: the RPI agent guidance cleanup is consistent with the surrounding terminology and documentation patterns. No actionable issues found.

@katriendg katriendg left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good refactoring that achieves its readability goal with a net -71 lines. The terminology normalization, Execution Model table, and Tracking Artifacts table are all clear improvements.

Requesting changes for one required edit (RI-4: missing explicit simple/medium bullet in Phase 1 Step 1) and two recommended fixes where semantic content was lost during compression (RI-1: Review Log content, RI-2: Iterate/Escalate display guidance).

Priority order:

  1. RI-4 — Required: restore explicit simple/medium bullet in Phase 1 Step 1
  2. RI-1 — Recommended: restore Review Log content specification
  3. RI-2 — Recommended: add display guidance to Iterate/Escalate bullets
  4. RI-3 — Non-blocking: abstraction tradeoff observation

Comment thread .github/agents/hve-core/rpi-agent.agent.md
Comment thread .github/agents/hve-core/rpi-agent.agent.md
Comment thread .github/agents/hve-core/rpi-agent.agent.md Outdated
Comment on lines +67 to 77

| Phase | Direct (Simple/Medium) | Artifact-backed (Medium-hard/Challenging) |
|-------------------------|---------------------------------------------------------|------------------------------------------------------------------------------------------------------------|
| Research | Investigate in-context; no research files or subagents | Create research documents; use `Researcher Subagent` selectively (medium-hard) or as default (challenging) |
| Plan | Record requests, order, and approach in working context | Create plan artifacts in `.copilot-tracking/plans/`; use subagents for especially complex planning |
| Implement | Execute directly from in-context plan | Execute from plan artifacts; use `Phase Implementor` selectively (medium-hard) or as default (challenging) |
| Track (Phase 3, Step 4) | Keep internal record of changes and validation | Update all `.copilot-tracking/` artifacts (plan checkboxes, changes log, planning log) |
| Review | Keep findings in working context | Compile review log in `.copilot-tracking/reviews/` |

### Intent Detection

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking observation: abstraction tradeoff

The "Under the direct model..." / "Under the artifact-backed model..." phrasing throughout the phases requires the agent to cross-reference the Execution Model by Phase table at every decision point. The original was more verbose but self-contained at each step.

This is a reasonable tradeoff for maintainability and reduced line count — just flagging it in case testing reveals degraded agent behavior at inference time due to the indirection. No change required unless functional regression is observed.

@jkim323 jkim323 Jun 15, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good observation. I used the prompt-tester to test the new agent file against couple scenarios.

Input: "Fix a typo in README.md"
Expected: Agent stays in direct model, creates no artifacts, invokes no subagents.
Result: Pass. All 5 phases resolved cleanly to the direct path. The difficulty classification acted as a persistent gate with zero pull toward over-delegation

Input: "Rename validate:frontmatter to lint:frontmatter" (initially medium, reveals 14 cross-cutting references + CI impact during research)
Expected: Agent upgrades from medium to medium-hard mid-Phase 1 and consistently follows artifact-backed path for all subsequent branch points.
Result: Pass (11/12 branch points clean, 1 with moderate tension). The upgrade trigger ("immediately" + "before continuing") is unambiguous. Post-upgrade branch points are stateless and resolve correctly regardless of when the classification changed.

Through this testing, one vulnerability identified: Phase 2 Step 2's "Choose the lightest planning mechanism" phrasing introduces a cognitive backdoor where an upgraded agent could rationalize skipping artifacts despite being in artifact-backed mode. I think it should be fixed to Choose the planning mechanism matching the current execution model that still gives the implementation phase enough structure.

What are your thoughts on updating the identified vulnerability?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think my comment was more about the cross referencing back to the table summarizing all file locations within their phases, where now you have the single table summarizing all, and this requires a phase to cross-reference the table again for identifying where to store artifacts.
In the previous version, the phase and type (like subagent) sections each contained their output locations, now it will require the agent to re-read the table and infer the artifact location, then continue on the phase it's at. I'm really not sure this is going to provide the intended optimization. That is the main concern.

@bindsi bindsi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved: the RPI agent refactor preserves the intended guidance while improving readability and consistency. I did not find actionable schema or artifact issues in the current head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Improve rpi agent file

5 participants