MLOps & Testing

modeltest

[ Open Source ][ Beta ]

Unit tests for machine learning models — like pytest, but for your model

modeltest is a testing framework for machine learning models: define contracts for quality, robustness, fairness, and data invariants, and run them automatically in your CI/CD pipeline. Includes SHAP explainability tests, MLflow integration, prediction caching, and custom test registration. Works with scikit-learn (including Pipelines), PyTorch, and Keras/TensorFlow out of the box.

// Features

  • Declarative YAML suites — define your model's contract without writing code
  • `modeltest validate` CLI with JUnit XML output and exit-code gating, ready to report in CI
  • Per-suite prediction caching: every test reuses results instead of re-running the model
  • Fairness, drift, and distribution tests: equal opportunity, statistical parity, data drift, KS
  • SHAP explainability tests (feature dominance, top-feature attribution) and MLflow experiment logging
  • Multi-framework: scikit-learn Pipelines, PyTorch, Keras/TensorFlow, custom adapters, and dotted-path custom tests in YAML

// Technologies