Skip to content

Explain pandas 3 limitation for distributed forecasting - #1267

Closed
AhmetHakanBezir (AHMETHAKANBEZIR1) wants to merge 1 commit into
Nixtla:mainfrom
AHMETHAKANBEZIR1:fix/distributed-pandas3-guidance
Closed

AhmetHakanBezir (AHMETHAKANBEZIR1) wants to merge 1 commit into
Nixtla:mainfrom
AHMETHAKANBEZIR1:fix/distributed-pandas3-guidance

Conversation

@AHMETHAKANBEZIR1

Copy link
Copy Markdown

Closes #1265.

When a Dask, Ray, or Spark input reaches Fugue under pandas 3, Fugue can fail deep in dataframe schema conversion. Check the pandas version at the two public distributed entry points and raise a clear error explaining the temporary pandas<3 requirement, while preserving local pandas/polars forecasting.

Validation:

  • Reproduced the original failure with pandas 3.0.5, Fugue 0.9.4, and Dask 2026.8.0; local forecast and cross-validation passed.
  • Confirmed both distributed entry points now return the guidance error with the same Dask input.
  • tests/test_distributed_core.py: 6 passed under both pandas 2.3.3 and pandas 3.0.5.
  • git diff --check passed.

Implementation and tests were prepared with Codex AI assistance and verified against the reported failure.

Co-authored-by: Codex <noreply@openai.com>
@jmoralez

Copy link
Copy Markdown
Contributor

Thanks for your interest in fixing the issue. The idea is that fugue will support pandas 3 soon and merging this would make the next statsforecast release unable to use the distributed interface regardless of whether or not fugue supports pandas 3, which would then require us to make another release removing this. We're going to leave the issue open for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Distributed backends (fugue) are not supported with pandas 3

2 participants