Can Anthropic’s Claude control your PC?

Anthropic unveiled an updated Claude 3.5 Sonnet (which is available in public beta) with a new feature, called ‘Computer Use,’ which completes tasks on a computer like a human does. 

‘Computer Use’ can understand and interact with any PC desktop app, and can control a PC—by moving the cursor, clicking buttons, and typing text—based on human prompts. 

For example, when Claude was asked to plan an outing with two friends, to see the Golden Gate Bridge, it opened Chrome, searched for the ideal viewing spot and times, and then used the calendar app to create and send invites out. All by itself. 

Humans remain fully in control, by telling Claude what it wants it to do, through prompts like ‘use data from my computer and online to fill out this form,” for example. But Claude can now see what’s on the PC screen (something it couldn't do, previously) and break the prompt down into a list of computer commands, like move the cursor, click here, or type this, to complete the task.

Anthropic has warned users that the new feature can be “cumbersome and error-prone” at times, and because it can struggle with basic actions like zooming or scrolling, has advised users to start exploring the feature with ‘low-risk’ easy tasks first. 

Anthropic has taken steps to minimize the potential misuse of the feature, especially with elections looming. It’s already been tested by the US AI Safety Institute and UK Safety Institute, and it has restrictions in place to stop it from completing high-risk tasks, such as generating and posting social media content, registering web domains, or interacting with government sites.