Skip to content

[BUG] fix the RandomizedSearchCV backend API mismatch.#1015

Open
MaazAli8460 wants to merge 2 commits intosktime:mainfrom
MaazAli8460:fix-issue-1013
Open

[BUG] fix the RandomizedSearchCV backend API mismatch.#1015
MaazAli8460 wants to merge 2 commits intosktime:mainfrom
MaazAli8460:fix-issue-1013

Conversation

@MaazAli8460
Copy link
Copy Markdown

Reference Issues/PRs

Fixes #1013.

What does this implement/fix? Explain your changes.

This PR fixes the RandomizedSearchCV backend API mismatch.

Changes made:

Added backend="loky" to RandomizedSearchCV.init in _tuning.py
Passed backend=backend through to the BaseGridSearch constructor so backend selection works as documented.

Does your contribution introduce a new dependency? If yes, which one?

No.

What should a reviewer concentrate their feedback on?

  1. Is backend left out intentionally in RandomizedSearchCV constructor?
  2. Is the problem in RandomizedSearchCV documentation?

Did you add any tests for the change?

I did not add tests in the commit, but this script is running without errors.

import pandas as pd
from sklearn.model_selection import KFold

from skpro.model_selection import RandomizedSearchCV
from skpro.regression.dummy import DummyProbaRegressor

# Minimal data
X = pd.DataFrame({"x": [1, 2, 3, 4]})
y = pd.DataFrame({"y": [1.0, 2.0, 3.0, 4.0]})


rscv = RandomizedSearchCV(
    estimator=DummyProbaRegressor(),
    cv=KFold(n_splits=2),
    param_distributions={"strategy": ["empirical"]},
    n_iter=1,
)

print(rscv)
print("Backend:", rscv.backend)
rscv.fit(X, y)
print(rscv.cv_results_)

Output
(.venv) D:\D drive\Projects_GSoC_ESoC\skpro>python x.py
RandomizedSearchCV(cv=KFold(n_splits=2, random_state=None, shuffle=False),
                   estimator=DummyProbaRegressor(), n_iter=1,
                   param_distributions={'strategy': ['empirical']})
Backend: loky
   mean_test_CRPS  mean_fit_time                     params  rank_test_CRPS
0            1.75       0.006672  {'strategy': 'empirical'}             1.0

PR checklist

For all contributions
  • I've added myself to the list of contributors with any new badges I've earned :-)
    How to: add yourself to the all-contributors file in the skpro root directory (not the CONTRIBUTORS.md). Common badges: code - fixing a bug, or adding code logic. doc - writing or improving documentation or docstrings. bug - reporting or diagnosing a bug (get this plus code if you also fixed the bug in the PR).maintenance - CI, test framework, release.
    See here for full badge reference
  • The PR title starts with either [ENH], [MNT], [DOC], or [BUG]. [BUG] - bugfix, [MNT] - CI, test framework, [ENH] - adding or improving code, [DOC] - writing or improving documentation or docstrings.
For new estimators
  • I've added the estimator to the API reference - in docs/source/api_reference/taskname.rst, follow the pattern.
  • I've added one or more illustrative usage examples to the docstring, in a pydocstyle compliant Examples section.
  • If the estimator relies on a soft dependency, I've set the python_dependencies tag and ensured
    dependency isolation, see the estimator dependencies guide.

If scoring= None, set it to CRPS().
Added backend parameter in RandomizedSearchCV constructor.
Copilot AI review requested due to automatic review settings March 30, 2026 10:46
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes the documented backend parameter mismatch in RandomizedSearchCV so callers can select a parallel backend as intended.

Changes:

  • Exposes backend in RandomizedSearchCV.__init__ and forwards it to BaseGridSearch.
  • Sets a default scoring metric (CRPS) when scoring=None to avoid runtime failures.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

# scoring = check_scoring(self.scoring, obj=self)
scoring = self.scoring
if scoring is None:
scoring = CRPS()
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scoring is documented as accepting str/callable, but _fit assumes a metric object with .name and .get_tag() (used for scoring_name and ranking). Passing a callable or string will raise at runtime. Consider coercing/validating self.scoring to a BaseMetric (or raising a clear ValueError if unsupported) so the implementation matches the documented API.

Suggested change
scoring = CRPS()
scoring = CRPS()
# validate scoring: BaseGridSearch requires a metric-like object
# with at least a ``name`` attribute and ``get_tag`` method for
# result naming and ranking. Passing a bare string or callable will
# otherwise fail at runtime.
if not (hasattr(scoring, "name") and hasattr(scoring, "get_tag")):
raise ValueError(
f"'scoring' must be a metric-like object with 'name' and "
f"'get_tag' attributes when used with {self.__class__.__name__}. "
"Passing strings or generic callables is not supported "
"in this tuner; please pass a metric object instead."
)

Copilot uses AI. Check for mistakes.
Comment thread skpro/model_selection/_tuning.py
Comment on lines 675 to 678
verbose=verbose,
return_n_best_estimators=return_n_best_estimators,
backend=backend,
error_score=error_score,
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a small regression test that instantiates RandomizedSearchCV(..., backend="threading"/"loky") and asserts the attribute is set and used (e.g., by checking no error and that backend is forwarded into parallelize via a simple fit). This prevents future regressions of the documented backend parameter support.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Collaborator

@fkiraly fkiraly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks - to prevent regression, could you add a parameter set to get_test_params that sets backend explicitly to be the "none" backend (to avoid nested parallelization)?

@fkiraly fkiraly added bug module:regression probabilistic regression module labels Apr 22, 2026
@fkiraly fkiraly changed the title [BUG] - bugfix, fixes the RandomizedSearchCV backend API mismatch. [BUG] fix the RandomizedSearchCV backend API mismatch. Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug module:regression probabilistic regression module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] RandomizedSearchCV documentation includes backend parameter but not in the constructor.

3 participants