Models & Architecture

Multimodal AI

A model that can work with more than one type of input or output, such as text, images, audio, and video together.

Multimodal AI handles several kinds of data at once rather than being limited to text alone. A multimodal model can look at a photo and answer questions about it, or take a spoken prompt and reply with an image. Combining modalities tends to make models more useful and more grounded, since the information in a picture or a chart is often hard to express in words.

Reference: Wikipedia: Multimodal AI