Estimate from the version's published specs, benchmarks and reviews, 0-5; scores are only comparable within one criteria year. Scored as launched; family pin, flagship Llama 3.1 405B named (8B and 70B ship alongside; 23 Jul 2024). Reasoning/coding null: no AA Intelligence Index or Terminal-Bench figure at launch; Meta's headline benchmarks (e.g. MMLU, GPQA) are not comparable to the anchors. Context 128k (Meta blog) -> 1. Price: Meta published no list price; hosted launch-week price for 405B was Together Turbo $5.00 per 1M in/out (Wayback 30 Jul 2024; Fireworks listed $3.00) -> 3.5. Speed null. Modalities 1: text only (TechCrunch). Reviews are launch coverage and commentary relaying Meta's benchmarks (Simon Willison, Interconnects, The Register, Tom's Guide); no hands-on test of the 405B opened.[1]
it really is benchmarking in that GPT-4+ class, competitive with both GPT-4o and Claude 3.5 Sonnet.
The 405 billion parameter model is directly in the ballpark of Anthropic's Claude 3 and OpenAI's GPT-4o.
The result is a model that, at least according to Meta's benchmarks, is ahead of larger, more proprietary systems from OpenAI and Anthropic on a variety of benchmarks.
quickly gained a reputation as one of the most powerful and useful models available
| Introducing Llama 3.1: Our most capable models to date · 2024年7月23日 | 0.70 |
|---|---|
| Llama 3.1 405B, Meta's AI strategy, and the new, open frontier model ecosystem · 2024年7月23日 | 0.50 |
| Meta says all new Llama 3.1 405B model bests OpenAI's GPT-4 · 2024年7月23日 | 0.60 |
| Meta Llama 3.1 is out now — here's how to try it for free · 2024年7月24日 | 0.80 |
读者上传的照片,与图钉带来源的媒体分开显示。你可以为吃到或看到的内容评分,在餐厅图钉上还能直接在照片上标记菜品。
第一项始终是图钉的来源。总体可信度是各份资料对上方所用开始和结束时间支持程度的加权平均;资料每比最新的一份旧 180 天,权重减半。
Meta的博文 'Introducing Llama 3.1: Our most capable models to date' 的日期为2024年7月23日,文中称模型当天即可下载;扎克伯格的公开信和TechCrunch[6]的报道日期相同。
Meta's[1] model card: 15T+ pretraining tokens, 39.3M H100-80GB GPU hours (30.84M for the 405B), December 2023 knowledge cutoff, Llama 3.1 Community License and the instruction-tuned benchmark table.
Dates the line: Llama 3 on 18 April 2024 with 8B and 70B, Llama 3.1 on 23 July 2024 with 8B, 70B and 405B, Llama 4 in April 2025.[1]
The technical paper: dense 405B Transformer with a context window of up to 128K tokens, performance comparable to GPT-4 on many tasks, plus Llama Guard 3 and the not-yet-released multimodal work.[1]
Mark Zuckerberg's open letter published with the release: 405B is the first frontier-level open source model, inference at roughly 50% of the cost of closed models like GPT-4o, and the partners (Amazon, Databricks, NVIDIA, Groq, Dell, Scale.AI).[1]
有遗漏或错误吗?用你自己的话说明:能佐证此图钉的链接、不同的开始或结束日期及理由,或缺失、有误的信息。AI 会对照此图钉的来源进行核实,搜索更好的来源,并添加任何支持你说法的页面。图钉自身的来源仍然最重要。AI 也会查看图片:显示的是别的东西或显示效果差的图片会被移到后面或替换。
原标题: Llama 3.1 8B, 70B and 405B Released
| 基准 | Llama 3.1 8B Instruct | Llama 3.1 70B Instruct | Llama 3.1 405B Instruct |
|---|---|---|---|
| MMLU (5-shot) | 69.4 | 83.6 | 87.3 |
| MMLU (CoT, 0-shot) | 73.0 | 86.0 | 88.6 |
| MMLU-Pro (CoT, 5-shot) | 48.3 | 66.4 | 73.3 |
| IFEval | 80.4 | 87.5 | 88.6 |
| ARC-C (0-shot) | 83.4 | 94.8 | 96.9 |
| GPQA (0-shot) | 30.4 | 46.7 | 50.7 |
| HumanEval (pass@1) | 72.6 | 80.5 | 89.0 |
| MBPP++ base (pass@1) | 72.8 | 86.0 | 88.6 |
| MultiPL-E HumanEval (pass@1) | 50.8 | 65.5 | 75.2 |
| MultiPL-E MBPP (pass@1) | 52.4 | 62.0 | 65.7 |
| GSM-8K (CoT, 8-shot) | 84.5 | 95.1 | 96.8 |
| MATH (CoT, 0-shot) | 51.9 | 68.0 | 73.8 |
| API-Bank (0-shot) | 82.6 | 90.0 | 92.0 |
| BFCL (0-shot) | 76.1 | 84.8 | 88.5 |
| Gorilla API-Bench (0-shot) | 8.2 | 29.7 | 35.3 |
| Nexus (0-shot) | 38.5 | 56.7 | 58.7 |
| Multilingual MGSM (CoT, 0-shot) | 68.9 | 86.9 | 91.6 |