!!! note "How the feature types are chosen"
Without features_specs, KDP reads a sample of the file and infers a type
per column:
| Column looks like | Inferred type |
|---|---|
| Floats, or whole numbers with many distinct values | `FLOAT_NORMALIZED` |
| Whole numbers with at most 20 distinct values | `INTEGER_CATEGORICAL` |
| Strings that parse as dates (`YYYY-MM-DD` or `YYYY/MM/DD`) | `DATE` |
| Strings of at most three words | `STRING_CATEGORICAL` |
| Longer strings | `TEXT` |
A high-cardinality single-token column such as an id stays categorical, so
hashing can keep it bounded rather than building a vocabulary the size of
the dataset. Pass `features_specs` for any column you want decided
differently — explicit specs are used as given.
from kdp import auto_configure, PreprocessingModel
# Analyze data and get recommendations
config = auto_configure("customer_data.csv")
# Review the recommendations
recommendations = config["recommendations"]
code_snippet = config["code_snippet"]
# Create your preprocessor using the code snippet as a guide
# Note: You'll need to manually implement the suggestions
💡
Suggests appropriate feature types and transformations based on data analysis
| Data Characteristic | Example | What It Detects |
|---|---|---|
| Distribution Types | Log-normal income, bimodal age | Statistical distribution patterns |
| Feature Statistics | Mean, variance, skewness | Basic statistical properties |
| Data Ranges | Min/max values, outliers | Value boundaries and extremes |
| Value Patterns | Discrete vs continuous | How values are distributed |
# Basic auto-configuration analysis
config = auto_configure(
"customer_data.csv", # Your dataset
batch_size=50000, # Process in batches of this size
save_stats=True # Save computed statistics
)
# Review the recommendations
for feature_name, recommendation in config["recommendations"].items():
print(f"Feature: {feature_name}")
print(f" Type: {recommendation['feature_type']}")
print(f" Preprocessing: {recommendation['preprocessing']}")
# Get the suggested code snippet
print(config["code_snippet"])</div>
# Example results structure
config = {
"recommendations": {
"income": {
"feature_type": "NumericalFeature",
"preprocessing": ["NORMALIZATION"],
"detected_distribution": "log_normal",
"config": {
# Specific configuration recommendations
}
},
# More features...
},
"code_snippet": "# Python code with recommended configuration",
"statistics": {
# If save_stats=True, contains computed statistics
}
}# Auto-configuration with options
config = auto_configure(
data_path="customer_data.csv", # Path to your dataset
features_specs=None, # Optional: provide existing features specs
batch_size=50000, # Batch size for processing
save_stats=True, # Whether to include statistics in results
stats_path="features_stats.json", # Where to save/load statistics
overwrite_stats=False # Whether to recalculate existing stats
)
👀
Always review the recommendations before blindly applying them
# Inspect the recommendations first
config = auto_configure("data.csv")
# Review before implementing
for feature, recommendation in config["recommendations"].items():
print(f"{feature}: {recommendation['detected_distribution']}")</div>
🧠
Use the recommendations alongside your domain expertise
# Get recommendations
config = auto_configure("data.csv")
# Create your features dictionary, informed by recommendations
features = {
"income": FeatureType.FLOAT_RESCALED, # Based on recommendation
"age": FeatureType.FLOAT_NORMALIZED, # Based on domain knowledge
}</div>
🔄
Rerun when your data distribution changes
# Update statistics with new data
new_config = auto_configure(
"updated_data.csv",
overwrite_stats=True # Force recalculation with new data
)</div>