Synopsis
The feature, now in research preview, lets Claude open apps, browse the web, and complete tasks across the desktop. It can click, type, and move through apps on the desktop just like a human user. However, Claude is designed to seek permission before taking any action, and users must approve each request. Certain sensitive categories, including financial and trading platforms, are blocked by default. Users can view, edit, or delete what Claude remembers at any time.“When Claude doesn't have a connector or tool for what you need, it navigates your screen directly—clicking, typing, and opening apps just like you would. It can work in your browser, open files, and run your dev tools automatically, with no setup required,” the company said on its website.
At present, the feature is available through Claude Cowork and Claude Code and is limited to the macOS desktop app. Support for Windows has not yet been launched.
How the system works
Claude decides how to complete a task by choosing the most efficient method first. If a task is assigned, it follows a clear order of tools.
First, it checks for connectors. If tools like Gmail, Google Drive, or Slack are available, Claude uses them directly. This is the quickest and most reliable option.
If no connector is available, it moves to the browser, where it can use Chrome to carry out tasks online.
If that still does not work, Claude uses screen interaction, meaning it can click, type, and move through apps on your desktop just like a human user.
The company says it always tries to use the fastest option available. “For example, pulling messages through your Slack connection takes seconds, but navigating Slack through your screen takes much longer and is more error-prone.”
What tasks can Claude handle?
With this capability, Claude can work across apps and files on your computer to complete real tasks.
For example, it can gather information from different sources and compile a competitive analysis into a structured report. It can also open a phone simulator, interact with an app, and help identify user experience issues.
In another use case, Claude can fill in spreadsheets using data from multiple sources, format the information, and save it to a shared location.
It can also handle tools that do not have built-in connectors, such as internal dashboards or specialised software used within teams.
“If your work involves a physical machine, Claude keeps working while you step away. Your computer just needs to be on,” the company said.
Safety and safeguards
While the feature is powerful, it comes with several safety controls. Claude is designed to ask for permission before taking any action, and users must approve each request, the company said.
Claude takes screenshots of the screen to understand what is happening and how to navigate apps. This means it may see any visible content on your screen, including personal or confidential data.
Some sensitive categories, including financial and trading platforms, are blocked by default. The company also advises against using the feature with sensitive information at this stage.
“Be mindful of what's visible when using Claude, especially on apps containing confidential information. Close files or apps with sensitive information before using,” the company cautioned.
