Evaluation & Operations

MLOps

The practice of reliably building, deploying, monitoring, and maintaining machine learning models in production, like DevOps for ML.

MLOps brings engineering discipline to the full model lifecycle: versioning data and models, automating training and deployment, and monitoring models in production for drift and failures. It exists because a model in a notebook is very different from one serving real users reliably over time. The variant focused on large language models and their prompts, retrieval, and evals is often called LLMOps.

Reference: Wikipedia: MLOps