Skip to content

Add E2E triage test case for not-affected resolution (RHEL-174694) #556

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
TomasTomecek wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add E2E triage test case for not-affected resolution (RHEL-174694) #556

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions ymir/agents/tests/e2e/test_triage.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
load_all_fixture_configs,
setup_mock_repos,
)
from ymir.common.models import BackportData, RebaseData, Resolution, TriageOutputSchema
from ymir.common.models import BackportData, NotAffectedData, RebaseData, Resolution, TriageOutputSchema

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -150,6 +150,16 @@ def testing_factory(gateway_tools, local_tool_options=None):
),
),
),
TriageAgentTestCase(
input="RHEL-174694",
expected_output=TriageOutputSchema(
resolution=Resolution.NOT_AFFECTED,
data=NotAffectedData(
explanation="not-implemented",
jira_issue="RHEL-174694",
),
),
),
]


Expand Down Expand Up @@ -234,12 +244,17 @@ def test_triage_agent(test_case: TriageAgentTestCase):
real_output = test_case.finished_state.triage_result
expected_output = test_case.expected_output
assert real_output.resolution == expected_output.resolution
assert real_output.data.package == expected_output.data.package
assert real_output.data.jira_issue == expected_output.data.jira_issue
assert real_output.data.fix_version == expected_output.data.fix_version

if expected_output.resolution == Resolution.BACKPORT:
assert real_output.data.package == expected_output.data.package
assert real_output.data.fix_version == expected_output.data.fix_version
assert real_output.data.patch_urls == expected_output.data.patch_urls
assert real_output.data.cve_id == expected_output.data.cve_id
elif expected_output.resolution == Resolution.REBASE:
assert real_output.data.package == expected_output.data.package
assert real_output.data.fix_version == expected_output.data.fix_version
assert real_output.data.version == expected_output.data.version
elif expected_output.resolution == Resolution.NOT_AFFECTED:
# TODO: validate justification_category against a known set of values
assert real_output.data.explanation
Comment on lines +258 to +260
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To ensure robust validation of the NOT_AFFECTED resolution, we should explicitly verify that real_output.data is an instance of NotAffectedData (preventing Pydantic union coercion issues) and assert that justification_category is populated and non-empty, as it is a required metadata field for Red Hat's triage process.

Suggested change
elif expected_output.resolution == Resolution.NOT_AFFECTED:
# TODO: validate justification_category against a known set of values
assert real_output.data.explanation
elif expected_output.resolution == Resolution.NOT_AFFECTED:
assert isinstance(real_output.data, NotAffectedData)
assert real_output.data.explanation
assert real_output.data.justification_category

Loading