New Claude beats GPT-4o

Anthropic has revealed its newest model, called Claude 3.5 Sonnet (which is available now for Claude web and iOS users), and says it outperforms its predecessor (Claude 3 Sonnet), its largest model, Opus, OpenAI’s GPT-4o, Google’s Gemini 1.5 Pro, and Meta’s Llama 3.

Claude 3.5 Sonnet’s performance

Anthropic says Claude 3.5 Sonnet is twice as fast as Claude 3 Sonnet and outscored its existing models and the newest models from OpenAI, Gemini, and Meta in 7 out of 9 industry benchmark tests for reading, coding, and math, and 4 out of 5 benchmark vision tests.

It reportedly understands humor and nuanced and complex instructions better, can more accurately interpret charts and graphs, transcribes text from “imperfect” images that have distortions and visual artifacts, and is better at writing and translating code, and handling multistep workflows.

Anthropic has also released a new feature–Artifacts–which is powered by Claude 3.5 Sonnet, that enables users to edit and add to content that’s been generated by Claude, in real-time, within the app.

For example, if you ask Claude to write an email, Artifact will allow users to edit the email, without needing to copy it into a separate editor.

Why does Claude 3.5 Sonnet perform so well?

Anthropic's product lead, Michael Gerstenhaber, says that the improvements made stem from architectural changes, new text and image training data, including some generated by AI, and feedback from human testers to “align the model with users’ intentions.”

What is Anthropic's long-term plan?

Anthropic has always focused on building models and features for business use cases and believes that this new release, coupled with the introduction of Artifacts,  will turn Claude into a tool that can “securely centralize knowledge, documents, and ongoing work in one, shared space”, which is ideal for Enterprise customers.

“What matters to [businesses] is whether or not AI is helping them meet their business needs, not whether or not AI is competitive on a benchmark, and from that perspective, I believe Claude 3.5 Sonnet is going to be a step function ahead of anything else that we have available — and also ahead of anything else in the industry.” – Michael Gerstenhaber, Anthropic’s Product Lead

‍