Skip to content

shap-monitor

Production ML explainability toolkit for monitoring SHAP values over time. Track how your model's explanations evolve, detect explanation drift, and maintain interpretability at scale.

Work in Progress Python Version License codecov

Overview

Most SHAP tooling focuses on development and analysis. shap-monitor bridges the gap for production monitoring, helping ML teams understand when and how their models' explanations change over time, compare reasoning across model versions, and detect shifts in feature importance patterns.

Key Features

  • Explanation Logging: Automatically log SHAP values for production predictions with configurable sampling
  • Flexible Storage: Parquet-based storage with pluggable backend support for efficient storage and retrieval
  • Multiple Explainers: Works with TreeExplainer, LinearExplainer, and other SHAP explainers
  • Time-Series Analysis: Compare SHAP explanations across different time periods to detect drift
  • Feature Importance Tracking: Monitor how feature importance evolves over time
  • Model Version Comparison: Compare explanations across different model versions

Quick Example

from shapmonitor import SHAPMonitor
import shap
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification

# Train a simple model
X, y = make_classification(n_samples=1000, n_features=10, random_state=42)
model = RandomForestClassifier(random_state=42)
model.fit(X, y)

# Create SHAP explainer
explainer = shap.TreeExplainer(model)

# Initialize the monitor
monitor = SHAPMonitor(
    explainer=explainer,
    data_dir="./shap_logs",
    sample_rate=0.1,  # Log 10% of predictions
    model_version="v1.0",
    feature_names=[f"feature_{i}" for i in range(10)]
)

# In your prediction loop
predictions = model.predict(X[:100])
monitor.log_batch(X[:100])

Why shap-monitor?

For ML Engineers

  • Monitor model explanations in production alongside performance metrics
  • Detect when your model's reasoning changes unexpectedly
  • Debug model behavior by analyzing SHAP values over time
  • Set up alerts for explanation drift

For Data Scientists

  • Understand how feature importance evolves as data distributions shift
  • Compare model versions based on their explanations, not just accuracy
  • Identify features that become more or less important over time
  • Validate that deployed models reason as expected

For ML Platform Teams

  • Provide teams with explanation monitoring infrastructure
  • Enable reproducible analysis of model behavior
  • Support model debugging and incident response
  • Integrate with existing MLOps tooling

Current Status

This project is in early development (v0.1). The core functionality is being actively developed.

Roadmap

  • v0.1 (Current): Core synchronous monitoring, Parquet storage
  • v0.2 (Planned): Drift detection, asynchronous processing, MLflow integration
  • v0.3+ (Future): Dashboard/visualization, additional framework integrations, advanced alerting

See the Roadmap for detailed information.

Getting Started

Ready to start monitoring your model's explanations?

Acknowledgments

Built on top of the excellent SHAP library by Scott Lundberg and the SHAP community.

License

This project is licensed under the Apache License 2.0. See the LICENSE file for details.