Entri pertama sentiasa sumber pin. Keyakinan keseluruhan ialah purata berpemberat bagi sejauh mana setiap rujukan menyokong masa mula dan tamat yang digunakan di atas; sesebuah rujukan diambil kira separuh kurang bagi setiap 180 hari lebih lama daripada yang terbaharu.
Siaran bertarikh "Oct 22, 2024" dan menyatakan "The upgraded Claude 3.5 Sonnet is now available for all users. Starting today, developers can build with the computer use beta"; TechCrunch[4] dan CNBC[5] melaporkan keluaran itu "on Tuesday" pada hari yang sama.
Anthropic's[1][3] model card addendum: the upgraded 3.5 Sonnet sets new state-of-the-art results in agentic coding (SWE-bench Verified), agentic tasks (TAU-bench) and computer use from screenshots (OSWorld), with a 14.9% average OSWorld success rate from screenshots alone.
Anthropic's[1][2] research post on the skill: Claude looks at screenshots and "counts how many pixels vertically or horizontally it needs to move a cursor in order to click in the correct place", and was trained on simple software such as a calculator and a text editor without internet access.
TechCrunch, 2024-10-22: "Anthropic[1][2][3] on Tuesday released an upgraded version of its Claude 3.5 Sonnet model that can understand and interact with any desktop app" through a Computer Use API in open beta on the API, Bedrock and Vertex AI.
CNBC, 2024-10-22: chief science officer Jared Kaplan says it can do tasks with "tens or even hundreds of steps"; Amazon had early access and beta testers included Asana, Canva and Notion; released Tuesday in public beta for developers.
Ada yang tertinggal atau salah? Nyatakan dengan kata-kata anda sendiri: pautan yang menyokong pin ini, tarikh mula atau tamat yang berbeza beserta sebabnya, atau fakta yang tiada atau tersilap. AI menyemaknya berbanding sumber pin ini, mencari yang lebih baik, dan menambah mana-mana halaman yang menyokong anda. Sumber pin itu sendiri tetap paling diambil kira. Gambar yang menunjukkan sesuatu yang lain, atau menunjukkannya dengan buruk, turut dilihat, dan dialihkan ke bawah atau diganti.
Tajuk asal: Claude 3.5 Sonnet Upgrade Released
| Penanda aras | Claude 3.5 Sonnet (new) | Claude 3.5 Haiku | Claude 3.5 Sonnet | GPT-4o* | GPT-4o mini* | Gemini 1.5 Pro | Gemini 1.5 Flash |
|---|---|---|---|---|---|---|---|
| Penaakulan peringkat pascasiswazah (GPQA Diamond) | 65.0% (0-shot CoT) | 41.6% (0-shot CoT) | 59.4% (0-shot CoT) | 53.6% (0-shot CoT) | 40.2% (0-shot CoT) | 59.1% (0-shot CoT) | 51.0% (0-shot CoT) |
| Pengetahuan peringkat sarjana muda (MMLU Pro) | 78.0% (0-shot CoT) | 65.0% (0-shot CoT) | 75.1% (0-shot CoT) | — | — | 75.8% (0-shot CoT) | 67.3% (0-shot CoT) |
| Kod (HumanEval) | 93.7% (0-shot) | 88.1% (0-shot) | 92.0% (0-shot) | 90.2% (0-shot) | 87.2% (0-shot) | — | — |
| Penyelesaian masalah matematik (MATH) | 78.3% (0-shot CoT) | 69.2% (0-shot CoT) | 71.1% (0-shot CoT) | 76.6% (0-shot CoT) | 70.2% (0-shot CoT) | 86.5% (4-shot CoT) | 77.9% (4-shot CoT) |
| Pertandingan matematik sekolah menengah (AIME 2024) | 16.0% (0-shot CoT) | 5.3% (0-shot CoT) | 9.6% (0-shot CoT) | 9.3% (0-shot CoT) | — | — | — |
| Soal jawab visual (MMMU) | 70.4% (0-shot CoT) | — | 68.3% (0-shot CoT) | 69.1% (0-shot CoT) | 59.4% (0-shot CoT) | 65.9% (0-shot CoT) | 62.3% (0-shot CoT) |
| Pengekodan secara ejen (SWE-bench Verified) | 49.0% | 40.6% | 33.4% | — | — | — | — |
| Penggunaan alat secara ejen (TAU-bench), retail | 69.2% | 51.0% | 62.6% | — | — | — | — |
| Penggunaan alat secara ejen (TAU-bench), airline | 46.0% | 22.8% | 36.0% | — | — | — | — |
* Jadual penilaian Anthropic tidak merangkumi keluarga model o1 milik OpenAI, kerana ia bergantung pada masa pengiraan yang panjang sebelum memberi jawapan, tidak seperti model biasa.