Estimate from the version's published specs, benchmarks and reviews, 0-5; scores are only comparable within one criteria year. Scored as launched (Claude 3 Haiku, 13 Mar 2024): 200K-token context; list price $0.25 in / $1.25 out per 1M tokens = $0.50 blended at 3:1; text and image input. Speed null: Anthropic published only input throughput (21K tokens, about 30 pages, per second for prompts under 32K), not output tokens/s. Reasoning null (no launch-version AA index; Anthropic's family charts have no anchor). Coding null. Reviews: Zvi, Willison (family pricing post, written before Haiku shipped), Keywords AI (vendor test of ~100 prompts vs GPT-4 Turbo, 25 Mar 2024), Vellum (vendor comparison vs GPT-3.5 Turbo / GPT-4o mini, 19 Jul 2024). The Keywords AI and Vellum pieces are vendor blogs but report genuine tests.[1]
Claude Haiku is cheap, but it is tiny, and releasing cheap tiny models below the capabilities curve is fine.
Haiku's average generation time is 78% faster than GPT-4-preview-0125 (2.283s vs 10.475s)
GPT-4o Mini performs worse than GPT-3.5 Turbo and Claude 3 Haiku, sometimes missing the mark entirely.
Sonnet is cheaper than all of the GPT-4 models (including GPT-4 Turbo), and Haiku (which has not yet been released to the Claude API) will be cheaper even than GPT-3.5 Turbo.
| The new Claude 3 model family from Anthropic · Mar 4, 2024 | 0.50 |
|---|---|
| On Claude 3.0 · Mar 6, 2024 | 0.20 |
| Test Results: Claude Haiku can substitute GPT-4 in 95% of AI tasks at 4% Cost · Mar 25, 2024 | 0.60 |
| GPT-4o Mini v/s Claude 3 Haiku v/s GPT-3.5 Turbo: A Comparison · Jul 19, 2024 | 0.00 |
Photos from readers, kept apart from the pin's sourced media. Rate what you ate or saw, and on a restaurant pin tag the dishes right on your photo.
The first entry is always the pin's source. Overall confidence is a weighted average of how firmly each reference supports the start and end times used above; a reference counts half as much for every 180 days older than the newest.
Anthropic's[5] post is dated "Mar 13, 2024": "Today we're releasing Claude[2] 3 Haiku ... The model is now available alongside Sonnet and Opus in the Claude API and on claude.ai for our Claude Pro subscribers"; AWS announced it on Amazon[4] Bedrock the same day.
Anthropic[1][5] notified developers on 19 February 2026 that Claude Haiku 3 (claude-3-haiku-20240307) would be retired; it was retired on 20 April 2026, with claude-haiku-4-5-20251001 as the replacement.
SiliconANGLE's report: Haiku processes up to 21,000 tokens (about 30 pages) per second for prompts under 32,000 tokens, uses a 1:5 input-to-output pricing ratio, and joins Sonnet on Amazon[4] Bedrock, with Google Cloud Vertex AI coming soon.
AWS News Blog, published 2024-03-13T15:27-07:00: "Today, we are announcing the availability of Claude[2] 3 Haiku on Amazon Bedrock"; like Sonnet and Opus it has image-to-text vision, multilingual skills and a 200K context window.
The 4 March 2024 family launch, where Haiku was announced but not yet released ("Haiku will be available soon"): $0.25 per million input and $1.25 per million output tokens, a 200K context window, and it reads a ~10k-token arXiv paper with charts in under three seconds.[1]
Something missing or wrong? Say it in your own words: a link that backs this pin up, a different start or end date and why, or a fact it lacks or gets wrong. The AI checks it against this pin's sources, searches for better ones, and adds any page that backs you up. The pin's own sources still count most. A picture that shows something else, or shows it badly, is looked at too, and moved down or replaced.
| Benchmark | Claude 3 Haiku | GPT-3.5 | Gemini 1.0 Pro |
|---|---|---|---|
| Price, input / output | $0.25 / $1.25 | $0.50 / $1.50 | $0.47 / $1.42 |
| MMLU (undergraduate level knowledge) | 75.2% 5-shot | 70.0% 5-shot | 71.8% 5-shot |
| GPQA, Diamond (graduate level reasoning) | 33.3% 0-shot CoT | 28.1% 0-shot CoT | — |
| GSM8K (grade school math) | 88.9% 0-shot CoT | 57.1% 5-shot | 86.5% Maj1@32 |
| MATH (math problem-solving) | 40.9% 4-shot | 34.1% 4-shot | 32.6% 4-shot |
| MGSM (multilingual math) | 76.5% 8-shot | — | 63.5% 8-shot |
| HumanEval (code) | 75.9% 0-shot | 48.1% 0-shot | 67.7% 0-shot |
| DROP, F1 score (reasoning over text) | 78.4 3-shot | 64.1 3-shot | 74.1 Variable shots |
| BIG-Bench-Hard (mixed evaluations) | 73.7% 3-shot CoT | 66.6% 3-shot CoT | 75.0% 3-shot CoT |
| ARC-Challenge (knowledge Q&A) | 89.2% 25-shot | 85.2% 25-shot | — |
| HellaSwag (common knowledge) | 85.9% 10-shot | 85.5% 10-shot | 84.7% 10-shot |
| Vision-enabled | Yes | No | Yes |