Meta's[1] post, published at Connect on September 25, 2024, says 'Today, we're releasing Llama 3.2'; the vision model card lists 'Model Release Date: Sept 25, 2024', TechCrunch[4] and VentureBeat[5] reported it that day and OpenRouter[6] lists the 1B and 3B as created 2024-09-25.
- Start
- Sep 25, 202490% CONFIDENCEfrom the source
Llama 3.2 1B, 3B, 11B and 90B Released
Meta releases Llama 3.2 at Connect 2024: lightweight 1B and 3B text models for phones and edge devices and 11B and 90B vision models, its first multimodal Llama, all with 128K context and open weights; the vision models were not offered in Europe.
- Meta released Llama 3.2 on 25 September 2024 at Connect: 'small and medium-sized vision LLMs (11B and 90B)' and 'lightweight, text-only models (1B and 3B)' that fit on phones and edge devices, each in pretrained and instruction-tuned form[1][4]
- It was the first Llama family with vision, arriving a little over two months after Llama 3.1; Zuckerberg called it 'our first open-source multimodal model' and open source 'the Linux of AI', and VentureBeat noted Meta said Llama usage had grown 10x since 3.1[1][5]
- Meta claims the vision models are competitive with Claude 3 Haiku and GPT-4o-mini on image recognition and visual understanding, the 3B beats Gemma 2 2.6B and Phi 3.5-mini on instruction following, summarization, prompt rewriting and tool use, and the 1B is competitive with Gemma; the claims rest on over 150 benchmark datasets Meta ran itself[1][5]
- The small models were made by pruning and distillation: the card says logits from Llama 3.1 8B and 70B were used as token-level targets in pretraining on up to 9T tokens and knowledge distillation recovered quality after pruning, then SFT, rejection sampling and DPO rounds finished them[2]
- The 11B and 90B are Llama 3.1 text models plus a separately trained cross-attention vision adapter, trained on 6 billion image-text pairs, and are drop-in replacements for 3.1 on text; image plus text is supported in English only[3][4]
- Europe was the catch: Meta kept the 11B and 90B out of the EU, blaming an 'unpredictable' regulatory environment, which also disabled some Meta AI image features there, while the 1B and 3B launched in Europe[4]
- Open licensing continued under the Llama 3.2 Community License, with platforms above 700 million monthly users needing a special license from Meta[2][4]
Notable features
- All four sizes take 128K-token context (the quantized small models 8K), roughly 100,000 words at once[1][2][4]
- More than 25 partners launched on day one, among them AMD, AWS, Databricks, Dell, Google Cloud, Groq, IBM, Intel, Microsoft Azure, NVIDIA, Oracle Cloud and Snowflake, with Arm, MediaTek and Qualcomm on device[1]
- Llama Stack distributions for single-node, on-prem, cloud and on-device use (PyTorch ExecuTorch on device, Ollama for single node) were released alongside[1]
- Safeguards: Llama Guard 3 11B Vision for text and image inputs, and Llama Guard 3 1B pruned and quantized from 2,858 MB to 438 MB[1]
- Quantized 1B and 3B variants (SpinQuant and QLoRA) followed, with an 8K context, and the text card was updated on 24 October 2024[2]
- OpenRouter lists the 1B at $0.027 per 1 million input tokens and $0.201 per 1 million output tokens and the 3B at $0.05 and $0.33[6]
Meta's instruction-tuned benchmarks[2][3]
| Benchmark | Llama 3.2 1B | Llama 3.2 3B | Llama 3.1 8B | Llama 3.2 11B | Llama 3.2 90B |
|---|---|---|---|---|---|
| MMLU (5-shot) | 49.3 | 63.4 | 69.4 | n/a | n/a |
| IFEval | 59.5 | 77.4 | 80.4 | n/a | n/a |
| GSM8K (CoT) | 44.4 | 77.7 | 84.5 | n/a | n/a |
| MATH (CoT) | 30.6 | 48.0 | 51.9 | n/a | n/a |
| ARC-C | 59.4 | 78.6 | 83.4 | n/a | n/a |
| BFCL V2 | 25.7 | 67.0 | 67.1 | n/a | n/a |
| MMMU (val, CoT) | n/a | n/a | n/a | 50.7 | 60.3 |
| MathVista (testmini) | n/a | n/a | n/a | 51.5 | 57.3 |
| ChartQA (test, CoT) | n/a | n/a | n/a | 83.4 | 85.5 |
| AI2 Diagram (test) | n/a | n/a | n/a | 91.1 | 92.3 |
| DocVQA (test) | n/a | n/a | n/a | 88.4 | 90.1 |
| VQAv2 (test) | n/a | n/a | n/a | 75.2 | 78.1 |
Buy onMeta AI