Skip to content

feat: add SklearnTransformAdapter (wrap any scikit-learn transformer) - #1017

Open
Valyrian-Code wants to merge 3 commits into
OpenSTEF:mainfrom
Valyrian-Code:feat/sklearn-transform-adapter-683
Open

feat: add SklearnTransformAdapter (wrap any scikit-learn transformer)#1017
Valyrian-Code wants to merge 3 commits into
OpenSTEF:mainfrom
Valyrian-Code:feat/sklearn-transform-adapter-683

Conversation

@Valyrian-Code

Copy link
Copy Markdown
Contributor

What

Adds SklearnTransformAdapter, a TimeSeriesTransform that adapts any scikit-learn transformer so it can be used in a feature-engineering pipeline. Resolves #683.

Per @lschilders's note on the issue, the transformer is carried as a serializable spec (import path + constructor params) rather than an object, so the workflow config stays save/load-able:

SklearnTransformAdapter(
    transformer_class="sklearn.decomposition.PCA",
    transformer_params={"n_components": 5},
    selection=Include("temperature", "load"),
)

How it works

  • Implements the TimeSeriesTransform interface (fit / transform / features_added / is_fitted) following the same pattern as Scaler and DimensionalityReducer.
  • The transformer is constructed in model_post_init from the class path + params.
  • fit / transform run it over the FeatureSelection; the output replaces the selected columns and the remaining columns pass through unchanged.
  • Output column names come from the transformer's get_feature_names_out(), so shape-changing transforms (PCA, one-hot encoders) and shape-preserving ones (scalers) are both handled. features_added() reports the net-new columns (populated after fit).

Tests

test_sklearn_adapter.py covers: a shape-preserving transform (StandardScaler; columns unchanged, features_added() == []), a shape-changing one (PCA; inputs replaced by components), passthrough of unselected features, NotFittedError before fit, an unimportable class failing fast, and a config serialization round-trip that rebuilds an equivalent transformer.

ty, ruff, ruff format, the module doctest, and the full openstef-models general-transforms suite (82 tests) all pass.

Adapts any scikit-learn transformer to the TimeSeriesTransform interface. The
transformer is specified by its import path and constructor params so the config
stays serializable. It is fitted on the selected features and its output replaces
them; the remaining columns pass through. Output column names come from
get_feature_names_out(), so shape-changing transforms (PCA, one-hot) are handled
the same as shape-preserving ones (scalers).

Resolves OpenSTEF#683.

Signed-off-by: RAJVEER42 <irajveer.bishnoi2310@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new SklearnTransformAdapter to openstef-models so scikit-learn transformers can be used as TimeSeriesTransforms inside OpenSTEF feature-engineering pipelines, with accompanying unit tests and package exports.

Changes:

  • Introduces SklearnTransformAdapter that dynamically imports and applies a configured scikit-learn transformer over a FeatureSelection.
  • Exposes the adapter via openstef_models.transforms.general public API.
  • Adds unit tests covering shape-preserving vs shape-changing transformers, passthrough behavior, pre-fit errors, and config round-tripping.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
packages/openstef-models/src/openstef_models/transforms/general/sklearn_adapter.py Implements the new adapter transform (dynamic import, fit/transform integration, feature name handling).
packages/openstef-models/src/openstef_models/transforms/general/init.py Exports SklearnTransformAdapter from the general transforms package.
packages/openstef-models/tests/unit/transforms/general/test_sklearn_adapter.py Adds unit coverage for adapter behavior and config serialization/validation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/openstef-models/tests/unit/transforms/general/test_sklearn_adapter.py Outdated
- Restrict transformer_class to the sklearn.* namespace so the dynamic import
  cannot load arbitrary modules.
- Raise MissingExtraError when scikit-learn is not installed and a clear
  ValueError for an unknown class name.
- Prefer get_feature_names_out() but fall back to the input feature names for
  shape-preserving transformers instead of hard-requiring it.

Signed-off-by: RAJVEER42 <irajveer.bishnoi2310@gmail.com>
@Valyrian-Code

Copy link
Copy Markdown
Contributor Author

Addressed the automated review comments:

  • Arbitrary import: transformer_class is now restricted to the sklearn.* namespace, so the dynamic import can't load arbitrary modules. Happy to widen this to an explicit allowlist if you'd like to support sklearn-compatible libraries (e.g. category_encoders) too.
  • Missing dependency: constructing with scikit-learn absent now raises MissingExtraError("sklearn", package="openstef-models"), and an unknown class name raises a clear ValueError.
  • get_feature_names_out: now preferred but not required. It falls back to the input feature names for shape-preserving transformers instead of hard-failing.

Tests updated to cover the non-sklearn and unknown-class cases.

@LeandervdBijl

Copy link
Copy Markdown

Dear @Valyrian-Code,

Thank you for your contribution.

We'd like to clarify compliance with our contribution policy. This project requires contributors to disclose any use of AI-assisted tools during the development of a pull request.

Could you please confirm whether AI tools were involved in producing any part of this change set? This is a routine policy check and not a judgment of the contribution itself.

@Valyrian-Code

Copy link
Copy Markdown
Contributor Author

Hi @LeandervdBijl, thanks for checking, happy to answer directly.

Yes, AI tools were involved in this PR. I use coding assistants case by case: to explore design options faster (for example weighing a serializable class path plus params spec against carrying a live sklearn instance), to work through unfamiliar API surface like get_feature_names_out behaviour across different transformers, and to draft code and tests that I then review and rework. They are also how I learn. I am newer to this codebase, and working this way gets me to the point where I can reason about OpenSTEF internals properly much faster than I otherwise would.

The part I hold myself to is the ownership side of the guidelines: I remain fully responsible for the contribution. I reviewed every line, chose the test cases (shape changing vs shape preserving transformers, passthrough, the pre fit error, the serialization round trip), ran ruff, ty and the full transform suites locally, and I can explain any part of the diff. Nothing in the change set is copied from third party code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[OpenSTEF 4.0] Create adapter for sklearn transforms

3 participants