Definition
Computer use is Computer use is an AI capability where a model looks at screenshots of a computer screen and controls the mouse and keyboard to operate software the way a person would, rather than through a purpose-built connection.
Computer use means an AI model operating a computer through its screen. Instead of calling a clean, purpose-built interface, the model looks at what is displayed, decides what to click or type, and acts, then looks again. It is how an AI can use software that was never designed for AI.
Anthropic's documentation describes the pattern. The model is given a screenshot and decides on an action, such as moving the cursor, clicking, dragging, typing text or pressing a keyboard shortcut. It returns that action as a tool call. The software around the model, the agent harness, carries out the action in a computer or virtual machine, takes a fresh screenshot and sends it back. This loop repeats until the task is done or the model gives up.
Positions on screen are usually expressed as pixel coordinates, so the model has to read the image accurately: find the right button, notice an error message, tell a loading screen from a finished one.
Other providers offer similar abilities under different names. Some focus only on web pages, in which case the product is a browser agent. Computer use is the broader idea: any app on the screen, not just a browser.
Plenty of useful software has no API. Old internal tools, desktop apps and websites that block automated access can only be used through their screens. Computer use lets an agent work with those tools without anyone building an integration.
The trade-off is reliability and speed. Reading a screen and clicking one step at a time is slower and more error-prone than calling a direct interface. A pop-up, a layout change or a slow page can derail it. Where a proper connection exists, such as one built on the Model Context Protocol, it is usually the better choice.
Giving a model control of a screen is powerful, and it brings risk. Text on a web page or in a document can contain hidden instructions aimed at the model, which is prompt injection. A mistaken click can send a message, delete a file or spend money.
Vendors' guidance is to run computer use inside a dedicated virtual machine or container with minimal privileges, an agent sandbox, and to keep it away from sensitive accounts and data. Requiring human approval before consequential steps is a common safeguard.
You ask an agent to move this month's invoices from a legacy accounting program into a spreadsheet. The program has no export feature. With computer use, the agent opens the program, reads each invoice on screen, switches to the spreadsheet and types the values in, checking its work with screenshots as it goes.
Computer use is the fallback that works when nothing cleaner exists. Use it in a sandbox, with limited access, and keep a person in the loop for anything that cannot be undone.
It is when an AI model views screenshots of a computer and controls the mouse and keyboard to use software like a person would.
It carries risks such as prompt injection and mistaken actions. Vendors advise running it in an isolated virtual machine with minimal access and requiring approval for sensitive steps.
Amazon is in talks to buy Decart, whose software ports AI models between chips, for about $7 billion, the Wall Street Journal reported Sunday. Read it as a bet on Amazon's Trainium chips, and on making your workload easier to move onto them.
Reflection AI introduced Beam on Monday, a 501-billion-parameter mixture-of-experts model with 23 billion active, and promised Apache 2.0 weights later in October. Until the weights ship, it is a waitlist and a claim about efficiency.
From October 9, the free Gemini app offers only Flash-Lite, and Google AI Plus subscribers lose Pro on a date Google will email them, per Google's help center. Free Gemini is now a sample. Find out this week whether the sample does your work.
OpenAI published 722 manuscripts in 372 result families on GitHub on Tuesday, produced mostly by an unreleased internal model, and not all are formally verified. The bottleneck in AI math has moved from finding proofs to checking them.
Iran's top security official warned of attacks on cables, data centers and AI facilities in the region if intense fighting resumes, the AP reported Sunday. Know which region your AI runs in, and where you would fail over.