Surrogate Interpretable Graph for Random Decision Forests
To install it, run the following command:
pip install -e .Spin up a Jupyter notebook server to try out the package:
jupyter notebookUn-comment the variable decision_paths to see the decision paths from sigRF.
While we can extract decision paths from the Random Forest, these methods only tell us how a specific instance is classified but fail to provide a global understanding of how the model behaves as a whole.
- Instance-Specific Insights Only: Decision paths typically focus on one data point at a time, which means we do not get a clear picture of the general decision-making patterns across all trees.
- High Variability: Each tree in the Random Forest may have different paths, leading to inconsistencies in interpretation.
- No Feature Interaction Visualization: We cannot see how different features interact across multiple trees to contribute to a final decision.
Thus, decision path analysis alone does not provide a holistic or human-friendly understanding of how features work together in an ensemble model.
Extracting decision rules from sub-trees allows us to:
- Capture Local Decisions: Rules extracted from individual decision trees show the conditions under which a sample is classified.
- Analyze Feature Importance Locally: Instead of just a feature importance score from the whole model, we see why a feature is important at specific decision points.
- Provide Explainability at a Granular Level: Instead of relying on a single surrogate model, we analyze the raw logic used within the ensemble.
Extracting rules directly from the entire Random Forest classifier does not give us meaningful insights because:
- Rules Become Too Complex: With hundreds of trees, the rule set becomes too large and unmanageable.
- Contradictory Rules: Different trees may have conflicting rules, making direct interpretation harder.
- Feature Interactions Are Hard to See: Individual trees do not always show how features interact across the whole forest.
Instead, extracting rules from sub-trees, then clustering them, helps us find patterns and common decision-making structures within the model.
- Random Forest consists of multiple decision trees that collectively make a prediction.
- We cannot visualize all trees at once, making it hard to understand feature interactions globally.
- The ensemble nature hides how individual trees contribute to the final decision.
The interpretable surrogate graph serves as a structural blueprint of how features interact across the entire Random Forest.
✅ Capturing Global Feature Interactions:
- Unlike single decision paths, a graph aggregates how features interact across all trees.
- Nodes represent features, and edges represent decision relationships learned by the model.
✅ Identifying the Most Important Relationships:
- We can quantify feature interactions based on how often two features appear together in decision rules.
- This helps identify dominant decision patterns rather than just relying on feature importance scores.
✅ Optimizing and Simplifying Interpretability:
- By applying Mixed Integer Linear Programming (MILP) optimization, we can prune unnecessary connections and highlight only the most crucial feature relationships.
- This allows us to remove noise and present a concise yet powerful explanation of the model.
✅ A More Trustworthy, Human-Friendly Model Representation:
- Instead of just a long list of extracted rules, a graph-based approach provides a visual and structured way to communicate how decisions are made.
- This is useful for regulatory applications, medical AI, finance, and critical decision-making models where trust is important.
Extracting rules from every tree in the Random Forest leads to a large number of overlapping and redundant rules. Clustering them helps to:
- Group Similar Decision Rules: Instead of analyzing thousands of rules, we summarize them into a few clusters that represent key decision patterns.
- Understand the Most Common Decision Strategies: Some trees may follow a similar decision logic. Clustering helps reveal these common strategies.
- Reduce Complexity for Interpretation: Instead of a list of hundreds of rules, we summarize them into just a few representative ones.
Without clustering, we would end up with hundreds or thousands of rules that are:
- Redundant
- Hard to navigate
- Difficult to summarize
Clustering condenses these rules, preserving essential information while making them easier to analyze.
✅ Going beyond decision paths: Individual decision paths are limited; global interpretability requires deeper insights.
✅ Graph representation is powerful: It enables visualization of complex feature interactions across all trees.
✅ Clustering rules helps: It summarizes thousands of rules into clear decision strategies.
✅ Optimization via MILP makes things cleaner: Reducing complexity ensures interpretability without losing essential insights.
✅ Our approach advances interpretability research: By combining multiple interpretability techniques, Our work is making AI more trustworthy, structured, and practical.
In summary: Existing methods are too simple, while our research enhances how we understand complex models.
our method:
- Extracts actual paths (rules) from each tree in the Random Forest.
- Creates a global feature interaction graph, where:
- Nodes = Features.
- Edges = Interactions (based on average usage and co-occurrence across subtrees).
- Prunes edges based on
max_edges_per_nodefor clarity.
- Clusters and summarizes rules for interpretability.
- Builds a surrogate decision tree from these summarized rules for global reasoning.
| Aspect | sigRDF (Surrogate Interpretable Graph) | TreeSHAP |
|---|---|---|
| Transparency | ✅ High – Extracts and visualizes actual decision rules and feature paths. | |
| Structure Awareness | ✅ Preserves the full path, split logic, and feature hierarchy from decision trees. | |
| Global Interpretability | ✅ Excellent – Shows how features co-occur, dominate, and interact across the forest. | |
| Rule-Based | ✅ Yes – Clusters and summarizes real, interpretable rules from trees. | ❌ No – Does not provide rule logic or extract decision paths. |
| Visual Simplicity | ✅ High – Graphs are pruned, layout-optimized, and modular. | |
| Faithfulness to Trees | ✅ Directly reflects true logic used by trees. | |
| Customizability | ✅ High – Flexible rule depth, clustering, interaction metrics, and layout. | ❌ Low – Fixed algorithm with minimal parameter tuning. |
| Scalability | ✅ Handles high-dimensional data with pruning and abstraction. | |
| Interaction Mapping | ✅ Builds full feature interaction DAG with co-occurrence weights. |
- Structural Fidelity – Maintains the original tree structure, making interpretations faithful and auditable.
- Human-Centered Design – Visuals are optimized for interpretability: interaction DAGs, rule clusters, and surrogate simplifications.
- Rule Compression – Transforms a forest into a few summarized, human-readable logic chains.
- Flexibility – Allows domain-specific adjustments: clustering, pruning depth, visual abstraction, etc.
- TreeSHAP abstracts rules – It gives numerical attributions but no visibility into decision paths.
- Limited global view – TreeSHAP’s interactions are pairwise and don’t scale well globally.
- Poor auditability – Attribution values are hard to trace back to specific training logic.
- Low visual and structural context – TreeSHAP lacks a structured, topological explanation of feature interactions.
If you found this work useful, please consider citing the following research work:
Dubey, A., Anžel, A., & Hattab, G. (2025). Surrogate Interpretable Graph for Random Decision Forests. arXiv:2506.01988. https://arxiv.org/abs/2506.01988
@misc{dubey2025surrogateinterpretablegraphrandom,
title={Surrogate Interpretable Graph for Random Decision Forests},
author={Akshat Dubey and Aleksandar Anžel and Georges Hattab},
year={2025},
eprint={2506.01988},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2506.01988}
}