[CI] Parity: fix flaky-test misattribution in log-based detection

[ethanwee1]

Summary

The log-based flaky-test detector in detect_log_failures.py misattributes flakiness to the wrong test.

It inferred flakiness from the last individual ... PASSED line seen before a Test succeeded in new process marker. That heuristic tags whichever test happened to pass most recently — usually an unrelated test that passed normally much earlier in the shard — rather than the test that actually crashed/failed and then passed on rerun.

Concrete example

From autoparity run 29255582284 (mi350, inductor/test_compiled_autograd):

• WrapTestClassTests::test_wrap_recreates_contexts_for_repeated_runs passed normally at ~16% into the shard (log line 4486), yet was reported as flaky.
• The test that truly failed-then-passed on rerun was FuncTorchHigherOrderOpTestsWithCompiledAutograd::test_jacfwd (named on log line 5244).

Fix

run_test.py already prints an authoritative per-shard summary naming the exact tests:

The following tests failed and then succeeded when run in a new process
['test/inductor/test_compiled_autograd.py::...::test_jacfwd']

Parse that line directly instead of guessing from nearby PASSED lines. Scoped entirely to the flaky-detection path; no other behavior changes.

Links / evidence

• Autoparity run that flagged the false positive: https://github.com/ROCm/pytorch/actions/runs/29255582284
• Upstream ROCm shard job (shard 5/8) whose log was parsed: https://github.com/pytorch/pytorch/actions/runs/29231370182/job/86759068151
• Upstream CUDA shard job (shard 3/5) whose log was parsed: https://github.com/pytorch/pytorch/actions/runs/29231370182/job/86760098157
• Raw ROCm shard log (the exact file the detector reads): https://ossci-raw-job-status.s3.amazonaws.com/log/86759068151
• Raw CUDA shard log: https://ossci-raw-job-status.s3.amazonaws.com/log/86760098157
• Upstream PR that introduced the rerun-in-new-process behavior we parse: Add multigpu pytest marker to partition distributed tests by GPU count pytorch/pytorch#189232

In the raw ROCm log above: line 4486 is ... test_wrap_recreates_contexts_for_repeated_runs PASSED (normal pass), and line 5244 is The following tests failed and then succeeded when run in a new process['...::test_jacfwd'] (the real flaky test).

Testing

1. py_compile

python -m py_compile .automation_scripts/pytorch-unit-test-scripts/detect_log_failures.py   # passes

2. Regression check against the real logs from run 29255582284

Download the two upstream shard logs that produced the false positive (rocm job 86759068151, cuda job 86760098157) and run the parser directly:

# rocm shard 5/8 (job 86759068151) and cuda shard 3/5 (job 86760098157)
for job in 86759068151 86760098157; do
  curl -s "https://ossci-raw-job-status.s3.amazonaws.com/log/${job}" | gunzip -c > ${job}.txt
done

python3 - <<'PY'
import detect_log_failures as d
for label, path in [("rocm", "86759068151.txt"), ("cuda", "86760098157.txt")]:
    _, _, flaky, _ = d.parse_log_file(path)
    print(label, [(f["cls"], f["method"], f["test_shard"]) for f in flaky])
PY

Before this change (buggy heuristic):

rocm [('WrapTestClassTests', 'test_wrap_recreates_contexts_for_repeated_runs', '5/8')]
cuda [('WrapTestClassTests', 'test_wrap_recreates_contexts_for_repeated_runs', '4/8')]

After this change:

rocm [('FuncTorchHigherOrderOpTestsWithCompiledAutograd', 'test_jacfwd', '1/2')]
cuda [('FuncTorchHigherOrderOpTestsWithCompiledAutograd', 'test_jacfwd', '1/1')]

• test_wrap_recreates_contexts_for_repeated_runs (which passed normally) is no longer flagged
• test_jacfwd (the test that actually failed-then-passed on rerun) is correctly reported on both rocm and cuda, with the correct test shard
• py_compile passes

3. End-to-end

Ran a full parity dispatch on the fork with this change and confirmed flaky_tests_mi350.csv reports test_jacfwd (not test_wrap_...) for inductor.test_compiled_autograd.

Deployment status

Merged to the fork ethanwee1/pytorch:main so it is live on autoparity while this PR is under review. This PR remains the upstream landing target for ROCm/pytorch:develop.

[rocm-repo-management-api]

Jenkins build for d4754e65a316ed2984ec89f2a077f8e79d97b842 commit finished as FAILURE
Links: Pipeline Overview / Build artifacts / Test Results