Evaluation & Operations
MLOps
The practice of reliably building, deploying, monitoring, and maintaining machine learning models in production, like DevOps for ML.
MLOps brings engineering discipline to the full model lifecycle: versioning data and models, automating training and deployment, and monitoring models in production for drift and failures. It exists because a model in a notebook is very different from one serving real users reliably over time. The variant focused on large language models and their prompts, retrieval, and evals is often called LLMOps.
Reference: Wikipedia: MLOps
Related terms
Model Serving
The infrastructure that runs a trained model in production and answers live requests reliably and at scale.
Evaluation
Measuring how good a model actually is, from accuracy on tests to human judgments of quality and safety, often called evals.
Inference
Actually running a trained model to get an answer, as opposed to training it in the first place.
Training Data
The examples a model learns from, whose size, quality, and bias largely determine how the model turns out.