A multimodal model handles more than one type of input.
Text, images, audio and video are projected into a shared representation.
This lets the model reason across modalities in one pass.
It can describe an image, read a chart or answer questions about a video.
Most frontier models are now multimodal by default.
A multimodal model handles more than one kind of input in the same conversation.
Text, images, audio and video can arrive together.
The trick is projecting each into a shared representation the model can reason over.
An image is encoded and its features are mapped into the same space as text tokens.
The language model then treats them much like words.
That is why you can ask a question about a picture in ordinary language.
Reasoning across modalities in one pass is the real gain.
Reading a chart, describing a photograph, extracting a table from a screenshot.
Document understanding is one of the most commercially useful applications.
A model that reads a scanned invoice end to end replaces a fragile OCR pipeline.
Training needs paired data, such as images with captions or video with transcripts.
Quality of that pairing sets the ceiling on how well the modalities align.
Most frontier models are now multimodal by default.
Images cost tokens too, often several hundred to over a thousand each.
Fine detail is still missed, so small text and precise measurements are unreliable.
One question on this card. Every option explains itself once you pick, right or wrong.
18 concepts in this module, then a graded test with every answer explained.