What is a computer use agent, in plain terms?

A computer use agent is an AI model that works a computer through the same surface you do: it looks at the screen, moves the pointer, clicks and types. It needs no special connection to each app, which is its great strength, because almost every program ever written has a screen and accepts clicks.

Where the name comes from

"Computer use" is the name AI labs gave this ability in their developer tools. Anthropic's computer use tool hands a model screenshot capture and mouse and keyboard control of a desktop environment. OpenAI's computer use guide shows a model driving a browser through Playwright or a desktop through PyAutoGUI, from a loop your own code runs. In both, the model proposes an action and something on the computer performs it.

Researchers measure the skill with benchmarks such as OSWorld, which sets agents hundreds of real tasks across web and desktop apps, file handling and workflows that span several programs, and checks whether each one was actually completed.

What it is good for

• Software with no API or integration, such as an old accounting tool or a company dashboard.
• Jobs that cross several apps, where copying between windows is the real work.
• Testing your own app the way a user would see it.

What it is not good for, yet

Speed and certainty. Reading a screenshot and deciding where to click takes longer than calling a service directly, which is why Anthropic's desktop app tries a connector or a browser before the screen. For a task with a clean integration, the integration wins.

From developer tool to everyday app

The first computer use agents were demos and code samples that needed a virtual machine to run safely. They now ship inside consumer apps. Claude's desktop app offers it on macOS and Windows in beta. Our own agent, superbot, can drive a computer and show its screen in the chat so you can follow along, and it asks before it uses your computer. Open-source projects cover the ground in between for people who want to build their own.

The one question to ask any of them

Whose computer is it acting on? An agent on your own screen sees what you see. An agent on a separate cloud machine sees only what you give it. Neither is wrong; they suit different work.