.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "generated/gallery/examples/plot_comparison_pdp_ale.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_generated_gallery_examples_plot_comparison_pdp_ale.py: Model Interpretation: PDP vs ALE ================================ In this notebook, we compare **Partial Dependence Plots (PDP)** and **Accumulated Local Effects (ALE)**. The Problem with PDP ^^^^^^^^^^^^^^^^^^^^ PDPs work by marginalizing over the distribution of the features. If features :math:`X_1` and :math:`X_2` are highly correlated, the PDP will calculate predictions for points that are impossible (e.g. a 100 :math:`\mathrm{m}^2` apartment with 10 bedrooms). This leads to extrapolation bias. The ALE Solution ^^^^^^^^^^^^^^^^ ALE plots, proposed by :footcite:t:`apley2020accumulatedlocaleffects`, calculate the model's behavior based on conditional distributions. They only look at how the prediction changes when a feature varies locally, given the values of other features. .. GENERATED FROM PYTHON SOURCE LINES 23-37 Creating a Synthetic Correlated Dataset --------------------------------------- To easily witness this phenomenon, we will generate a synthetic dataset containing a deliberate statistical trap: 1. We generate two features, :math:`X_1` and :math:`X_2`, which are strongly correlated. 2. We define the target variable :math:`y` to depend only on :math:`X_1` via a quadratic function: .. math:: y = X_1^2 + \varepsilon **The Goal:** Because :math:`y` has no direct connection to :math:`X_2`, an ideal interpretation method must return a completely **flat line at zero** for :math:`X_2`. .. GENERATED FROM PYTHON SOURCE LINES 37-62 .. code-block:: Python import matplotlib.pyplot as plt import numpy as np import pandas as pd n_samples = 5000 generator = np.random.default_rng(92) X1 = generator.uniform(-1, 1, n_samples) X2 = X1 + generator.normal(0, 0.1, n_samples) # Strong correlation with X1 y = X1**2 + generator.normal(0, 0.05, n_samples) X = pd.DataFrame({"X1": X1, "X2": X2}) plt.figure(figsize=(6, 4)) plt.scatter(X["X1"], X["X2"], c=y, cmap="viridis", alpha=0.4, s=15) cbar = plt.colorbar() cbar.set_label("Target Value (y)", rotation=270, labelpad=15) plt.title("Strongly Correlated Features") plt.xlabel("X1") plt.ylabel("X2") plt.show() .. image-sg:: /generated/gallery/examples/images/sphx_glr_plot_comparison_pdp_ale_001.png :alt: Strongly Correlated Features :srcset: /generated/gallery/examples/images/sphx_glr_plot_comparison_pdp_ale_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 63-67 Training the model ------------------ We train a `RandomForestRegressor` from scikit-learn. Because :math:`X_1` and :math:`X_2` are similar, the model might use both to predict :math:`y`, especially when we set `max_features='sqrt'`. .. GENERATED FROM PYTHON SOURCE LINES 67-82 .. code-block:: Python from sklearn.ensemble import RandomForestRegressor from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.3, random_state=92 ) model = RandomForestRegressor(max_features="sqrt", random_state=92) model.fit(X_train, y_train) r2_test_score = model.score(X_test, y_test) print(f"R² score on the test set: {r2_test_score:.2f}") .. rst-class:: sphx-glr-script-out .. code-block:: none R² score on the test set: 0.97 .. GENERATED FROM PYTHON SOURCE LINES 83-87 Comparison: PDP vs ALE for Feature :math:`X_2` ---------------------------------------------- Now, we plot the interpretation curves for :math:`X_2` using both methods to observe how they process the correlation. .. GENERATED FROM PYTHON SOURCE LINES 87-108 .. code-block:: Python from hidimstat.visualization import ALE, PDP # 1. Partial Dependence Plot (PDP) pdp = PDP(model, feature_names=X.columns) pdp_axes = pdp.plot(X_test, features=1) # 2. ALE Plot ale = ALE(model, feature_names=X.columns) ale_axes = ale.plot( X_test, features=1, grid_resolution=100, confidence_level=0 ) mean_pred = model.predict(X_test).mean() pdp_ymin, pdp_ymax = pdp_axes[1].get_ylim() ale_axes[1].set_ylim(pdp_ymin - mean_pred, pdp_ymax - mean_pred) ale_axes[1].figure.axes[2].set_ylim(pdp_ymin, pdp_ymax) plt.show() .. rst-class:: sphx-glr-horizontal * .. image-sg:: /generated/gallery/examples/images/sphx_glr_plot_comparison_pdp_ale_002.png :alt: plot comparison pdp ale :srcset: /generated/gallery/examples/images/sphx_glr_plot_comparison_pdp_ale_002.png :class: sphx-glr-multi-img * .. image-sg:: /generated/gallery/examples/images/sphx_glr_plot_comparison_pdp_ale_003.png :alt: plot comparison pdp ale :srcset: /generated/gallery/examples/images/sphx_glr_plot_comparison_pdp_ale_003.png :class: sphx-glr-multi-img .. GENERATED FROM PYTHON SOURCE LINES 109-118 Conclusion ^^^^^^^^^^ - The PDP shows a prominent U-shaped parabola for :math:`X_2` because of extrapolation. An analyst looking at this plot could falsely conclude that increasing or decreasing :math:`X_2` directly increases the target variable. This is an error since it is actually the result of out-of-distribution artifacts. - ALE shows a mostly flat line (near zero), isolating the unique contribution of :math:`X_2` by blocking the shadow effect of :math:`X_1`. This isolates the direct non-impact, which is ideal for understanding pure mechanisms, though it leaves out indirect operational levers. .. GENERATED FROM PYTHON SOURCE LINES 120-123 References ---------- .. footbibliography:: .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 1.873 seconds) **Estimated memory usage:** 250 MB .. _sphx_glr_download_generated_gallery_examples_plot_comparison_pdp_ale.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_comparison_pdp_ale.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_comparison_pdp_ale.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_comparison_pdp_ale.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_