Unify missing values display - #749
Open
meubleancien wants to merge 51 commits into
Open
Conversation
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: milton-minervino <milton.minervino@inria.fr>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…e-vignal/shapash into fix_pandas_3_object_str
plot_categorical_distribution referenced an undefined in its no-hue branch, raising a NameError whenever it was called without hue
ProjectReport read from the raw argument (which can be None) instead of the coerced when looking up max_points, display_interaction_plot, nb_top_interactions and title_story
server_instance.kill = _kill overwrote CustomThread's real kill() method with an incompatible closure, which mypy correctly flagged as unsound. Give CustomThread a proper on_kill callback extension point instead of patching over one of its methods at runtime.
Annotate implicit-Optional defaults, replace bare with , add missing var-annotations, correct several webapp callback return-type signatures to match actual behavior, type Dash options lists against their own Options TypedDicts, and otherwise satisfy mypy with no behavior change.
Enable mypy type checking
…numpy-constraints fix issue 733
substitute ignore with return_nan
substitute ignore with return_nan
Fix category encoders 2.10.0 issue
…_str Add pandas 3 compatibility for string dtype handling
Null feature values were rendered inconsistently (raw "nan", "None", blank cells) depending on the component. They are now displayed as "missing" everywhere, via a shared format_missing_value helper: - contribution plot: extend NaN handling to object/category columns (missing modality + "x" marker) and fix violin point hover customdata - local plot: bar labels and hover text, incl. grouped features - compare plot: hover text - cluster plot: point hover text in classification and regression branches; also guard all-null clusters (mode()[0] IndexError, NaN mean/std formatting) and allow null as top modality - webapp dataset table: cells and tooltips through a shared get_datatable_data_and_tooltips helper; identity card label Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Title: feat: unify display of missing values across plots and webapp
Description
Enhancement: #721
Null/missing values from training datasets were displayed inconsistently across the web application: raw nan in some hover texts, None or in table tooltips, blank cells in the dataset table, and invisible points in the contribution plot for non-numeric features. Since some models (e.g. CatBoost) natively support missing values, null entries are legitimate feature values and deserve a consistent, unambiguous representation.
This PR unifies the display of null feature values as missing everywhere, extending the convention introduced in #700 for the violin plot. A shared helper format_missing_value / MISSING_VALUE_DISPLAY is added in shapash/utils/utils.py.
Changes by component
Component: Contribution plot (scatter, object/category features)
Before: null points not rendered, hover shows raw nan
After: rendered as a missing modality with x marker, hover shows missing
────────────────────────────────────────
Component: Contribution plot (violin)
Before: axis label says missing but point hover shows nan
After: point hover shows missing
────────────────────────────────────────
Component: Local plot
Before: y-axis label and hover show nan (incl. grouped features)
After: missing
────────────────────────────────────────
Component: Compare plot
Before: hover shows nan
After: missing
────────────────────────────────────────
Component: Cluster plot (classification & regression)
Before: point hover shows nan; all-null cluster raises IndexError on mode()[0]; all-null numeric
cluster crashes on NaN mean/std formatting
After: hover shows missing; all-null clusters handled; null can be reported as top modality, e.g.
feature top: missing (53.3%)
────────────────────────────────────────
Component: Dataset table
Before: cells blank, tooltips show nan/None
After: cells and tooltips show missing via shared get_datatable_data_and_tooltips helper
────────────────────────────────────────
Component: Identity card
Before: empty label for null values
After: missing
Tests
Two adjacent bugs surfaced during this work and are left out of scope (happy to open follow-up issues): the dtype check if type is float in smart_app.py makes the table rounding block dead code, and the webapp filters drop nulls entirely so missing values can't be filtered on.
PS: