Infrastructure
Edge AI
Running AI directly on a local device like a phone or camera, instead of sending data to the cloud.
Edge AI means doing the AI computation on the device where the data is, rather than round-tripping to a remote server. That brings lower latency, offline capability, and better privacy, since the data never leaves the device. The constraint is limited hardware, which is why techniques like quantization and distillation, that make models small and fast, matter so much for edge deployment.
Related terms
Quantization
Shrinking a model by storing its numbers at lower precision, so it runs faster and on smaller hardware.
Knowledge Distillation
Training a smaller model to imitate a larger one, keeping much of the quality at a fraction of the cost.
Inference
Actually running a trained model to get an answer, as opposed to training it in the first place.