Infrastructure
Knowledge Distillation
Training a smaller model to imitate a larger one, keeping much of the quality at a fraction of the cost.
Knowledge distillation transfers what a large, capable teacher model knows into a smaller student model by training the student to mimic the teachers outputs. The result is a compact model that runs faster and cheaper while retaining a lot of the original performance. It is a common way to turn an expensive frontier model into something practical to deploy at scale.
Reference: Wikipedia: Knowledge Distillation
Related terms
Quantization
Shrinking a model by storing its numbers at lower precision, so it runs faster and on smaller hardware.
Inference
Actually running a trained model to get an answer, as opposed to training it in the first place.
Fine-Tuning
Taking a pretrained model and training it further on a smaller, focused dataset to specialize it for a task or style.
Edge AI
Running AI directly on a local device like a phone or camera, instead of sending data to the cloud.