Anthropic has released a new Browser Use tool that allows its Claude AI to interact with web pages through their underlying structure rather than just visual screenshots. Announced Thursday, the tool taps into a page's accessibility tree, enabling Claude to locate and manipulate specific elements directly instead of calculating their position from screen images. The release also moves Computer Use, the Skills API, and Files API into general availability.
The new approach replaces coordinate-based navigation with element references. When Claude calls read_page, it receives a text representation of the accessibility tree, where links, buttons, and input fields carry tags such as ref_3. If Claude wants to click that button later, it sends the reference along with the action rather than targeting coordinates like x: 640, y: 320. The system operates within the browser itself, where it can access page structure that would be hard to extract reliably from pixels alone. However, if a tab navigates to a new page or the content shifts enough, a reference that pointed to a button moments earlier may stop working, requiring the executor to recognize the mismatch, reject the action, and have Claude re-read the page before proceeding.
Anthropic is also cutting back-and-forth by letting Claude request multiple actions in one turn. Actions now arrive together as several tool_use blocks, which the application executes in sequence and returns as a batch, avoiding another model call between every click and keystroke. The company says this approach can lower latency and costs, especially as workflows grow from a few interactions to dozens or hundreds. Loading the default Browser Use toolset adds roughly 6,600 input tokens to a request before counting screenshots, accessibility trees, and other results sent back to Claude, though developers can disable unneeded operations to reduce that overhead. The tool is currently limited to the Claude API and isn't available inside Claude Managed Agents, and it exposes 27 browser operations by default when added to a Messages API request.
Anthropic recommends running the browser in an isolated container or virtual machine with minimal access because Claude can still encounter prompt injections in web content or be redirected to unexpected locations. JavaScript and file uploads should stay disabled unless necessary, since code generated by Claude runs with the page's privileges and can reach data or make requests available to that page. Batching makes approval more complicated because several actions can arrive at once, and a routine click at the start of a sequence could eventually lead to something requiring user permission, meaning the executor has to check actions as they happen and pause for approval when needed. Browser sessions, along with their downloads and uploaded files, remain in the developer's environment rather than Anthropic's infrastructure. Organizations building AI agents into customer-facing workflows will need to balance automation speed against the expanded surface for unintended actions, particularly as batched commands reduce human checkpoints. The shift from visual parsing to structural references may accelerate agent reliability, but it also transfers more security and session management responsibility to development teams.