Skip to content

Reject zero benchmark resamples on the command line - #3218

Open
r4venrudy wants to merge 1 commit into
catchorg:develfrom
r4venrudy:fix/reject-zero-benchmark-resamples
Open

r4venrudy wants to merge 1 commit into
catchorg:develfrom
r4venrudy:fix/reject-zero-benchmark-resamples

Conversation

@r4venrudy

Copy link
Copy Markdown

Description

--benchmark-resamples=0 currently parses successfully, even though the bootstrap analysis needs a nonempty set of resamples. Reject zero at the command-line boundary, mirroring the existing validation of --benchmark-samples.

Use parseUInt to validate the value before updating ConfigData, and report a specific error for malformed input. Add command-line regressions for zero and nonnumeric input, retain the existing positive-value test, document the constraint, and update the SelfTest approval baselines. No amalgamated files are changed.

Validation on Windows with Clang 23.1.3/libc++, C++14, and the basic-tests development preset:

  • Before the fix, the new zero-value test fails because parsing succeeds; the malformed-value test fails because the parser returns the generic conversion error.
  • After the fix, the command-line group passes all 263 assertions in 4 test cases.
  • All 82 CTest entries pass, including approval tests after regenerating and reviewing the expected output with tools/scripts/approve.py.
  • Linux, macOS, and sanitizer builds were not run locally.

GitHub Issues

No linked issue; this change is limited to command-line validation of the resample count.

@codecov

codecov Bot commented Oct 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.19%. Comparing base (0809213) to head (89e5cd5).

Additional details and impacted files
@@            Coverage Diff             @@
##            devel    #3218      +/-   ##
==========================================
+ Coverage   91.18%   91.19%   +0.01%     
==========================================
  Files         205      205              
  Lines        9101     9110       +9     
==========================================
+ Hits         8298     8307       +9     
  Misses        803      803              
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant