docs(agents): forbid unsolicited e2e suite timeout bumps#5527
Conversation
PR Summary by QodoDocs: prohibit unauthorized e2e/behaviour suite-timeout increases
AI Description
Diagram
High-Level Assessment
Files changed (2)
|
|
🤖 Finished Review · ✅ Success · Started 12:36 PM UTC · Completed 12:51 PM UTC |
Site previewPreview: https://9caf63b7-site.fullsend-ai.workers.dev Commit: |
Code Review by Qodo
1. Authorization rule inconsistent
|
| **Do not** increase e2e or behaviour **suite** timeouts without **explicit human authorization** in the current session (or an issue/PR comment that clearly authorizes that bump). Suite ceilings are job `timeout-minutes` in `.github/workflows/e2e.yml` for the `e2e` / `behaviour` jobs, the matching `go test -timeout` values in the `e2e-test` / `behaviour-test` Make targets, and the default `E2E_LOCK_TIMEOUT`. Scenario-level wait or assertion windows (for example dispatch detection) are out of scope when an issue asks for them — those are not suite ceilings. On timeout failures, diagnose and fix the root cause (slow scenarios, unnecessary waits, lock contention); do not open a PR whose primary change is raising the suite timeout. | ||
|
|
||
| **When reviewing PRs:** Flag unauthorized suite-timeout increases as an **important-severity** finding (policy violation). Explicit human authorization in the linked issue or PR description is the only exception. |
There was a problem hiding this comment.
1. Authorization rule inconsistent 🐞 Bug ⚙ Maintainability
The new suite-timeout policy text allows authorization via an issue/PR comment but later restricts the exception to only a linked issue or PR description, creating an ambiguous rule. This ambiguity can lead to inconsistent enforcement of the “unauthorized bump” policy during reviews.
Agent Prompt
## Issue description
The suite-timeout policy defines two different sources of acceptable “explicit human authorization” (issue/PR comment vs. linked issue or PR description). This creates an ambiguous rule for reviewers/agents.
## Issue Context
The first paragraph allows authorization via an issue/PR comment, while the review guidance later says the only exception is authorization in the linked issue or PR description. These are not equivalent.
## Fix Focus Areas
- docs/contributing/go-code.md[36-38]
## Suggested fix
Update the review guidance sentence to match the earlier definition (or update the earlier definition to match the later restriction), so there is exactly one clearly stated set of acceptable authorization sources (e.g., “linked issue/PR description or an explicit comment on the issue/PR”).
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
ReviewFindingsHigh
Previous runReviewFindingsHigh
Labels: PR modifies contributing documentation and adds e2e suite-timeout policy |
2744349 to
afd8db6
Compare
Stop agents from raising e2e/behaviour suite ceilings without explicit human authorization; flag unauthorized bumps on review. Signed-off-by: Barak Korren <bkorren@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com>
afd8db6 to
6b4fcd4
Compare
|
🤖 Finished Review · ✅ Success · Started 7:49 PM UTC · Completed 8:03 PM UTC |
| | File | When to read | | ||
| |------|-------------| | ||
| | [Go Code](docs/contributing/go-code.md) | Changing Go code under `cmd/` or `internal/` — covers mint sync, coverage, vet, e2e tests, concurrency testing | | ||
| | [Go Code](docs/contributing/go-code.md) | Changing Go code under `cmd/` or `internal/` — covers mint sync, coverage, vet, e2e tests, concurrency testing, and suite-timeout policy | |
There was a problem hiding this comment.
[high] protected-path
This PR modifies AGENTS.md, a protected governance file, without a linked issue authorizing the change. The modification adds ", and suite-timeout policy" to the Go Code entry in the topic-specific guidance table. Human approval is required for all protected-path changes regardless of change size.
Suggested fix: Link a tracking issue that authorizes the AGENTS.md modification, or obtain explicit human approval on this PR.
Stop agents from raising e2e/behaviour suite ceilings without explicit human authorization; flag unauthorized bumps on review.
Summary
Related Issue
Changes
Testing
make lintpasses (stage changes first, then run)Checklist
!for breaking changes)