Models & Architecture

Vision Transformer

ViT

A transformer adapted for images, which splits a picture into patches and treats them like a sequence of words.

The vision transformer applies the transformer architecture, originally built for text, to images by cutting a picture into a grid of patches and feeding them in as if they were tokens. Attention then lets every patch relate to every other, giving a global view of the image. It rivals and often beats convolutional networks on large datasets, and it underpins many modern multimodal models.

Reference: Wikipedia: Vision Transformer