Skip to content

[FIX] Handle polars string columns in the missing-value check - #1634

Closed
Raashish Aggarwal (raashish1601) wants to merge 1 commit into
Nixtla:mainfrom
raashish1601:fix/polars-categorical-nan-check
Closed

Raashish Aggarwal (raashish1601) wants to merge 1 commit into
Nixtla:mainfrom
raashish1601:fix/polars-categorical-nan-check

Conversation

@raashish1601

Copy link
Copy Markdown
Contributor

No existing issue.

Fitting on a polars frame that has a string categorical column (declared in cat_exog_list) fails before the vocabulary is even built:

polars.exceptions.InvalidOperationError: `is_nan` operation not supported for dtype `str`

_check_nan runs ufp.is_nan_or_none on every temporal and static column. For polars that calls Series.is_nan(), which only works on float columns. pandas is fine because it uses isna(). The categorical encoding already has a polars branch, so this check was the only thing blocking it.

The check now uses is_null() for non-float polars columns and keeps the old path for everything else. Nulls in a string column are still reported as missing values.

Added test_polars_string_categoricals_run in tests/test_categorical.py. It fits and predicts with hist, futr and static string categoricals on polars, and checks that a null in a string column still raises. It fails on main and passes with this change. tests/test_categorical.py passes locally, and so do the mask/missing-value tests in tests/test_core.py.

@CLAassistant

CLAassistant commented Oct 7, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@marcopeix

Copy link
Copy Markdown
Contributor

This fix should be in utilsforecast, because the issue is really from the function is_nan_or_none.

@raashish1601

Copy link
Copy Markdown
Contributor Author

Makes sense, is_nan_or_none is the right place. I'll open the fix in utilsforecast and close this PR once that is in.

@raashish1601

Copy link
Copy Markdown
Contributor Author

Opened the fix in utilsforecast: Nixtla/utilsforecast#297. is_nan_or_none now uses is_null() for non-float polars columns.

@raashish1601

Copy link
Copy Markdown
Contributor Author

Closing in favor of Nixtla/utilsforecast#297, which fixes this in is_nan_or_none.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants