Meta's release of Llama 3.2 marks the first time a genuinely capable multimodal model has been available as open source. With variants from 1 billion to 400 billion parameters, it spans deployment targets from smartphones to data centers. It it arrives at a moment when the field is running into the limits of text-only training.
The motivation for adding vision to language models is partly practical necessity. The total supply of high-quality public text data is finite. Estimates suggest that all publicly available text, concatenated, would fill roughly 67 libraries. For the largest current models, this corpus has been effectively exhausted. Further scaling requires either synthetic data generation or new modalities.
There is also a deeper conceptual limitation. A model trained exclusively on text cannot develop genuine understanding of inherently visual concepts. How a letter looks, how objects relate spatially, what a face expresses: these are things that text can describe but not fully convey. Adding visual input goes beyond increasing the data supply; it gives the model access to a fundamentally different kind of information.
Building a multimodal model is not as simple as feeding images alongside text. The model must learn the correspondence between visual content and linguistic descriptions, and this requires training data where the two modalities are precisely aligned. A random image with a loosely related caption is not sufficient; the model needs examples where text and image describe the same thing at the same level of detail.
News broadcasts are close to ideal: professionally produced video paired with spoken narration and often subtitles, all describing the same events in real time. But this kind of tightly aligned data is rare. Most of the world's image and video content lacks corresponding text of sufficient quality, which makes curating multimodal training sets a significant bottleneck.
The range of model sizes in Llama 3.2 (from 1 billion to 400 billion parameters) is itself a statement about where the field is heading. The smallest variants can run on a smartphone, handling tasks like text completion and basic image analysis. They are limited in conversational depth and visual reasoning, but the fact that any meaningful multimodal capability fits on a mobile device is a notable engineering achievement.
The larger variants, requiring data center hardware, offer correspondingly deeper capabilities. The practical consequence is that developers and researchers can choose their trade-off between capability and compute cost, and that on-device inference for multimodal tasks is no longer a theoretical possibility but a shipping product.
A parallel development in image generation is the discovery of diffusion illusions: images that show different subjects depending on viewing orientation. These are produced by manipulating the denoising process in diffusion models, which work by iteratively converting random noise into coherent images. By intervening at specific steps with different conditioning signals, researchers can create images that display, for example, a dog when viewed upright and a penguin when inverted.
Diffusion illusions are more than a curiosity. They expose the internal representations that image generation models learn, providing a window into how these systems encode and reconstruct visual information. Understanding these representations has potential applications in medical imaging, quality assurance, and any domain where interpretability of generative models matters.
The practical value of diffusion illusions is not immediately obvious, and that is precisely the point. Fundamental research on how models encode visual information tends to find applications years after the initial discovery. The study of electrons was not motivated by a desire to build the internet, but the understanding it produced made the internet possible. Research into the internal mechanics of generative models is likely to follow a similar trajectory. The insights are accumulating, and the applications will emerge as the understanding deepens.
There are two reasons. First, text-only models are hitting a data ceiling the total supply of high-quality public text data is finite and largely exhausted for training purposes. Second, some concepts are inherently visual and cannot be adequately learned from text descriptions alone. A model that has never seen the letter 'M' cannot truly understand its shape through words alone.
Aligned datasets require images and text that are meaningfully paired (content where the visual and textual information correspond precisely, beyond simple images with captions. News broadcasts with synchronized video, narration, and subtitles are an ideal example, but such perfectly aligned data represents only a small fraction of all available image and video data.
Llama 3.2 is the first capable multimodal open-source model available to the research community. It spans model sizes from 1 billion to 400 billion parameters, enabling deployment from smartphones to data centers. This range makes advanced vision-language capabilities accessible for both research and practical applications.
Diffusion illusions exploit the image generation process of diffusion models, which convert random noise into coherent images through iterative denoising steps. By intervening in this process at specific points, researchers can produce images that show different subjects depending on viewing angle for example, a dog when viewed upright and a penguin when inverted.
Small models in the 1-3 billion parameter range can handle text completion and basic image analysis on mobile devices, but they lack the capacity for complex conversations or nuanced visual reasoning. They represent a useful trade-off between capability and hardware constraints rather than a replacement for larger models.
.
Copyright 2026 - Joel P. Barmettler