Repository navigation
fix: support non-float polars columns in is_nan_or_none - #297
Open
Raashish Aggarwal (raashish1601) wants to merge 2 commits into
Open
Raashish Aggarwal (raashish1601) wants to merge 2 commits into
Raashish Aggarwal (raashish1601) wants to merge 2 commits into
Conversation
José Morales (jmoralez)
requested changes
Oct 9, 2026
|
|
||
|
|
||
| def is_nan_or_none(s: Series) -> Series: | ||
| if isinstance(s, pl_Series) and not s.dtype.is_float(): |
Contributor
There was a problem hiding this comment.
please move this to the is_nan function. in the polars branch we can check if its float and call is_nan and otherwise return a series filled with false
There was a problem hiding this comment.
Done in a4c1d45. is_nan now calls is_nan only for polars float columns and returns an all-false series otherwise, and is_nan_or_none is back to is_nan(s) | is_none(s).
| is_nan_or_none(pl.Series([np.nan, 1.0, None])).to_numpy(), | ||
| np.array([True, False, True]), | ||
| ) | ||
| for values in (["a", None, "b"], [True, None, False], [1, None, 2]): |
Contributor
There was a problem hiding this comment.
please move these to the is_nan test
There was a problem hiding this comment.
Moved them to test_is_nan in a4c1d45. They check is_nan on string, boolean, integer and categorical series, and is_nan_or_none on the same series.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
is_nan_or_nonefails on polars columns that aren't numeric, because polars only supportsis_nanon numeric dtypes:This shows up in neuralforecast when fitting on a polars frame with a string categorical exogenous column:
_check_nancallsis_nan_or_noneon every column (see Nixtla/neuralforecast#1634, where it was suggested to fix it here).For polars series that are not floats,
is_nan_or_nonenow returnsis_null(), since only float columns can hold NaN. Float columns and pandas are unchanged.Extended
test_is_nan_or_nonewith polars string, boolean, integer and categorical series containing a null. It fails on main and passes with this change.tests/test_processing.pypasses apart from three polars concat/backtest tests that also fail on main with the polars version I have locally (2.0, above the<=1.31pin).