Skip to content

[BUG] fix the RandomizedSearchCV backend API mismatch. #1015

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
MaazAli8460 wants to merge 2 commits into
base: main
Choose a base branch
from
+5 −0
Open

[BUG] fix the RandomizedSearchCV backend API mismatch. #1015

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
df22721
[Fix] GridSearchCV scoring=None defaults to CRPS
MaazAli8460 Mar 30, 2026
e81b17d
[Fix] Supports backend in RandomizedSearchCV
MaazAli8460 Mar 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions skpro/model_selection/_tuning.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import pandas as pd
from sklearn.model_selection import ParameterGrid, ParameterSampler, check_cv

from skpro.metrics import CRPS
from skpro.benchmarking.evaluate import evaluate
from skpro.regression.base._delegate import _DelegatedProbaRegressor
from skpro.utils.parallel import parallelize
Expand Down Expand Up @@ -108,6 +109,8 @@ def _fit(self, X, y, C=None):

# scoring = check_scoring(self.scoring, obj=self)
scoring = self.scoring
if scoring is None:
scoring = CRPS()
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scoring is documented as accepting str/callable, but _fit assumes a metric object with .name and .get_tag() (used for scoring_name and ranking). Passing a callable or string will raise at runtime. Consider coercing/validating self.scoring to a BaseMetric (or raising a clear ValueError if unsupported) so the implementation matches the documented API.

Suggested change
scoring = CRPS()
scoring = CRPS()
# validate scoring: BaseGridSearch requires a metric-like object
# with at least a ``name`` attribute and ``get_tag`` method for
# result naming and ranking. Passing a bare string or callable will
# otherwise fail at runtime.
if not (hasattr(scoring, "name") and hasattr(scoring, "get_tag")):
raise ValueError(
f"'scoring' must be a metric-like object with 'name' and "
f"'get_tag' attributes when used with {self.__class__.__name__}. "
"Passing strings or generic callables is not supported "
"in this tuner; please pass a metric object instead."
)

Copilot uses AI. Check for mistakes.
scoring_name = f"test_{scoring.name}"

backend = self.backend
Expand Down Expand Up @@ -660,6 +663,7 @@ def __init__(
verbose=0,
return_n_best_estimators=1,
random_state=None,
backend="loky",
error_score=np.nan,
backend_params=None,
Comment thread
fkiraly marked this conversation as resolved.
):
Expand All @@ -670,6 +674,7 @@ def __init__(
cv=cv,
verbose=verbose,
return_n_best_estimators=return_n_best_estimators,
backend=backend,
error_score=error_score,
Comment on lines 675 to 678
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a small regression test that instantiates RandomizedSearchCV(..., backend="threading"/"loky") and asserts the attribute is set and used (e.g., by checking no error and that backend is forwarded into parallelize via a simple fit). This prevents future regressions of the documented backend parameter support.

Copilot uses AI. Check for mistakes.
backend_params=backend_params,
)
Expand Down
Loading