Anthropic’s Claude can now interact with computers like a human in new upgrade to its model
Claude 3.5 makes automating complex tasks easier for developers, while the upcoming Claude 3.5 Haiku promises faster, more efficient AI performance.
Imagine delegating repetitive computer tasks to an AI that can move the cursor, click buttons, and fill out forms for you—just like a human would.
Well, with Anthropic’s latest update to its Claude 3.5 Sonnet which introduces a new 'computer use' feature, that doesn't seem like a futuristic dream anymore.
Now in public beta, this update brings developers one step closer to automating more complex tasks across devices by allowing Claude to mimic human interactions like moving the cursor, typing, and clicking buttons.
Emmanuel Oyedeji
Although still in its experimental stage, Anthropic says the feature is already proving useful with companies like Canva, Asana, and DoorDash already using it to automate their tasks.
However, there are some limitations, as the AI model still struggles with drag-and-drop tasks or misses brief notifications. These issues stem from Claude’s reliance on piecing together screenshots instead of observing a continuous video stream, meaning it occasionally misses short-lived actions. Despite these early-stage hurdles, the potential for AI-driven automation seems likely to grow.
This development also sets Claude 3.5 apart from its competitors. Compared to other systems like OpenAI’s ChatGPT and Microsoft’s Copilot Vision which can "see" your computer screen and interpret content, Anthropic says Claude can actively execute tasks, making it more hands-on in terms of automation.
Even though Anthropic acknowledges that the feature is still in development, neither OpenAI nor Microsoft have rolled out similar capabilities on this scale yet.
Kelechi Edeh
In addition to its computer-use feature, Anthropic reports that the updated Claude 3.5 Sonnet has made notable strides in tool use and coding, with its accuracy on the SWE-bench rising from 33.4% to 49%, outperforming models like OpenAI’s o1-preview.
Building on this progress, Anthropic also revealed a new AI model—Claude 3.5 Haiku–which it claims will be faster and more cost-effective, particularly in tasks that require quick responses and precise instruction-following.
Haiku is expected to surpass not only its predecessor, Haiku 3, but also other models like Claude 3 Opus, offering better efficiency in handling large-scale tasks and coding tasks over OpenAI GPT-40.
To ensure safety, Anthropic says it has also implemented measures to prevent Claude from accessing sensitive areas, such as social media and government websites. This move could align Claude with high-security standards and make it a reliable tool for enterprises and developers alike.
As Anthropic continues to refine these features, whether in coding, automation, or computer interactions, I believe Claude 3.5 is positioning itself as a leader in the AI space.
The progress being made suggests that seamless, AI-driven automation may be closer than expected, with Claude 3.5 leading the charge towards this new frontier of efficiency.
