Models & Architecture
Vision Transformer
ViTA transformer adapted for images, which splits a picture into patches and treats them like a sequence of words.
The vision transformer applies the transformer architecture, originally built for text, to images by cutting a picture into a grid of patches and feeding them in as if they were tokens. Attention then lets every patch relate to every other, giving a global view of the image. It rivals and often beats convolutional networks on large datasets, and it underpins many modern multimodal models.
Reference: Wikipedia: Vision Transformer
Related terms
Transformer
The neural network design behind almost every modern language model, built around a mechanism called attention.
Convolutional Neural Network
A network design built for images, which scans for small visual patterns and builds them up into recognizable objects.
Computer Vision
The field of AI that lets computers interpret images and video, from recognizing objects to reading documents.
Multimodal AI
A model that can work with more than one type of input or output, such as text, images, audio, and video together.