Skip to content

[Agents] [Agent Skills] Add PR review agent and skills#2643

Open
bhapas wants to merge 2 commits into
elastic:mainfrom
bhapas:pr_review_agent
Open

[Agents] [Agent Skills] Add PR review agent and skills#2643
bhapas wants to merge 2 commits into
elastic:mainfrom
bhapas:pr_review_agent

Conversation

@bhapas

@bhapas bhapas commented May 6, 2026

Copy link
Copy Markdown
Contributor

Summary: ECS PR quality review agent

What this PR does

Adds an automated PR quality review for ECS schema contributions, complementary to PR Triage (routing). It focuses on field-level quality: naming, descriptions, types, examples vs pattern / expected_values, OTel hints, and overlap/conflicts against a base-branch schema inventory. Findings use High / Medium / Low severity and are posted as a PR comment with a summary and collapsible <details> sections.

Changes

New workflow

  • .github/workflows/pr-review.yml
    • Triggers: pull_request_target (opened, synchronize, ready_for_review); workflow_dispatch (PR number, optional model).
    • Phase A: Checkout base SHA; generate schema-inventory.tsv from schemas/*.yml; gh pr diff; OpenCode + LiteLLM (same secrets pattern as pr-triage.yml); write pr-review-report.md.
    • Phase B: Post PR comment from artifact.
    • Concurrency: pr-review-${{ pr_number }}, cancel in-progress.

New skill — .agents/skills/ecs-pr-review/

  • SKILL.md — Execution steps for agents (scope, diff, rules, inventory, report).
  • quality-rules.md — Rule IDs, severities, detection, remediation.
  • report-template.md — Comment layout (title, Overall counts, ### Summary details, severity <details>).

Rule update

  • .agents/rules/ecs-schema-standards.mdc — Links to ecs-pr-review skill, quality-rules, and pr-review.yml.

Configuration

  • LITELLM_API_KEY required (same as PR Triage). Optional: LITELLM_MODEL, LITELLM_BASE_URL, repo var LITELLM_MODEL_DEFAULT.

Notes

  • Comment-only; does not approve or block merge.
  • Does not replace routing/triage (RFC vs direct PR).

@bhapas bhapas self-assigned this May 6, 2026
@github-actions

github-actions Bot commented May 6, 2026

Copy link
Copy Markdown
Contributor

🤖 GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@bhapas bhapas marked this pull request as ready for review May 8, 2026 11:56
@bhapas bhapas requested a review from a team as a code owner May 8, 2026 11:56
@github-actions

github-actions Bot commented May 8, 2026

Copy link
Copy Markdown
Contributor

ECS PR Triage (automated)

PR Triage Report

PR: #2643 — [Agents] [Agent Skills] Add PR review agent and skills
Classification: Direct PR
Change type: Tooling
Scope: Moderate

Summary

This PR adds a new automated PR quality review agent alongside the existing PR triage agent. It introduces a new GitHub Actions workflow (.github/workflows/pr-review.yml), a new agent skill (ecs-pr-review) with quality rules and a report template, and a small cross-reference update to the existing ecs-schema-standards.mdc rule. No schema YAML files are modified, no fields are added or changed, and no breaking changes are introduced. The change is purely tooling/CI infrastructure, making it appropriate for a direct PR.

Files changed

  • Schemas: none
  • Generated: none
  • Tooling/scripts/tests: none (no changes under scripts/)
  • Docs (hand-authored): none
  • CI / GitHub: .github/workflows/pr-review.yml (new, +439 lines)
  • RFCs: none
  • Agent infrastructure (.agents/):
    • .agents/rules/ecs-schema-standards.mdc (modified, +5 lines — added cross-reference to new skill)
    • .agents/skills/ecs-pr-review/SKILL.md (new, +75 lines)
    • .agents/skills/ecs-pr-review/quality-rules.md (new, +112 lines)
    • .agents/skills/ecs-pr-review/report-template.md (new, +81 lines)

Routing decision

Direct PR is appropriate. This change is entirely tooling and CI automation — it adds a new GitHub Actions workflow and agent skill definitions. No schemas/*.yml files are touched, no fields are introduced or modified, and no ECS semantics are altered. Per classification-rules §2, "CI / automation" and "Tooling only" changes are low-risk direct PR patterns. No RFC triggers from §1 apply (no new field sets, no breaking changes, no new reuse topology, no novel use case, no ECS-wide scope changes, no new fields).

Risk notes

  • Breaking / deprecation: No. The workflow and skill are purely additive. The workflow uses pull_request_target which runs in the base-repo context — this is an intentional security pattern to access secrets for fork PRs, consistent with the existing pr-triage.yml workflow. Reviewers should verify that the workflow does not check out or execute untrusted PR code (it checks out the base SHA and reads PR diff via gh, which appears correct).
  • OTel / semconv: N/A — no schema fields involved.
  • Scope / reuse: N/A — no field sets, reuse, or categorization fields affected.

Completeness checklist

  • PR description (all sections) — The PR body does not follow the standard 7-section PR template (.github/PULL_REQUEST_TEMPLATE.md). However, it provides thorough documentation of what changed, why, and configuration details using a custom format. The "what", "why", and "notes" are well covered. Sections 4–6 (documentation, make build, tests) are not explicitly answered, though they are less relevant for a tooling-only PR with no schema changes.
  • CHANGELOG.next.md — Not required. No schemas/ or scripts/ files are changed. Agent infrastructure under .agents/ and .github/workflows/ does not require a changelog entry per repo conventions.
  • make + committed generated outputs — Not applicable. No schema changes; no regeneration needed.
  • OTel otel: on new/changed semconv-related fields — Not applicable. No schema fields changed.
  • Tests / make check — Not explicitly confirmed in the PR description. For a CI workflow change, manual testing of the workflow (e.g., via workflow_dispatch) would be the appropriate verification. Reviewers may want to confirm the workflow was tested.
  • CLA (contributor) — Author (@bhapas) appears to be an Elastic employee based on contribution history.

Recommended next actions

  1. Reviewer: Verify the pull_request_target workflow security model — confirm that no untrusted code from the PR branch is checked out or executed (the workflow checks out github.event.pull_request.base.sha, which is correct).
  2. Reviewer: Confirm the workflow was tested via workflow_dispatch on a test PR to validate the end-to-end flow (schema inventory generation, OpenCode invocation, comment posting).
  3. Reviewer: Consider whether the PR description should be updated to use the standard PR template format for consistency, or whether the custom format is acceptable for agent/tooling PRs.
  4. Contributor: No blocking issues identified. The PR is well-documented and appropriately scoped as a direct PR.

Posted by PR Triage workflow

# pull_request_target runs in the base-repo context so secrets are available for
# fork PRs. We checkout the base revision (skills, rules, schema inventory base);
# the PR diff is read via `gh`.
pull_request_target:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit hesitant to use pull_request_target if there are other options available. Considering that we've just had a prompt injection attempt (unsuccessful) recently, is there another way we can do this?

Looking at the other workflow, it seems to use the same mechanism, so maybe this isn't actually an issue?

@andrewkroh, any thoughts on this?

@github-actions

Copy link
Copy Markdown
Contributor

Hi!

We just realized that we haven't looked into this PR in a while. We're
sorry!

We're labeling this PR as Stale to make it hit our filters and
make sure we get back to it as soon as possible. In the meantime, it'd
be extremely helpful if you could take a look at it as well and confirm its
relevance. A simple comment with a nice emoji will be enough :+1.

If there is no activity on this PR within the next 2 weeks, it will be
automatically closed.

Thank you for your contribution!

@github-actions github-actions Bot added the stale Stale issues and pull requests label Jun 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale Stale issues and pull requests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants