[cuda.compute]: Add smoke tests for benchmarks#9885
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughSummary by CodeRabbit
WalkthroughBenchmark execution gains profile mode and failure propagation. Host benchmark wrappers generalize native-compute bypassing. Linux, Windows, and cooperative CUDA CI jobs install required benchmark dependencies and run benchmark smoke tests. ChangesBenchmark smoke testing
Suggested reviewers: Comment |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
CI is failing because we need 3.14 wheels for cuda-bench. These should be released soon |
This comment has been minimized.
This comment has been minimized.
| # Install cuda_cccl, plus CuPy which the cuda.compute examples require, plus the | ||
| # benchmark-smoke deps: pytest-benchmark (host-overhead), and cuda-bench + PyYAML | ||
| # (throughput; PyYAML is needed by run_benchmarks.py --quick). |
There was a problem hiding this comment.
This dependency stuff is usually be specified in a pyproject.toml as an optional configuration (like we appear to have for the tests). Can we not do that here as well?
There was a problem hiding this comment.
Running the examples as tests requires extra dependencies (like CuPy + others mentioned here). We want to keep the existing optional configuration for testing minimal (without including these), and we don't want to add an extra test-examples configuration
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
python/cuda_cccl/benchmarks/compute/run_benchmarks.py (1)
407-413: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winsuggestion: Add focused tests for the new CLI and exit-status contract: verify
--profilereaches both child command builders, non-zero C++/Python statuses exit with code 1, execution errors withreturncode=Noneare failures, and skipped sides remain neutral. As per coding guidelines, validate changes with relevant targeted builds and tests, and report the results.Also applies to: 513-525
Source: Coding guidelines
ci/test_cuda_coop_python.sh (1)
32-37: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winsuggestion: Lines 32-37 use
--profilewithout a quick mode or axis filter. Profile mode reduces iterations, but a future expansion of the benchmark configuration space could make this smoke job unnecessarily expensive. Add a reduced configuration if supported, or verify that the current axis space is intentionally bounded.As per path instructions, "For CI and build scripts, focus on matrix correctness, targeted build/test behavior, cache/artifact handling, environment setup, GPU availability assumptions, clear failures, and avoiding unnecessary expensive jobs."
Source: Path instructions
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: a95743d6-d216-4d33-b9cb-0b7b24865a55
📒 Files selected for processing (4)
ci/test_cuda_cccl_examples_python.shci/test_cuda_coop_python.shpython/cuda_cccl/benchmarks/compute/host/host_benchmark_cases.pypython/cuda_cccl/benchmarks/compute/run_benchmarks.py
🚧 Files skipped from review as they are similar to previous changes (1)
- python/cuda_cccl/benchmarks/compute/host/host_benchmark_cases.py
😬 CI Workflow Results🟥 Finished in 2h 52m: Pass: 98%/515 | Total: 4d 12h | Max: 2h 38m | Hits: 99%/666593See results here. |
2 participants
No description provided.