Anthropic has created Claude Duet, an AI assistant that watches your connected applications and handles tasks in the background, determining what deserves your attention without waiting for instructions. A week following its October 21 launch, the frontier lab—which has an IPO on the horizon—disclosed to The New Stack how it's evaluating whether Duet makes the right priority calls. The company has built internal assessment tools to judge how Duet ranks tasks, though the urgency of any given item can be subjective. The risks grow when an agent oversees work spanning email, calendars, and business-to-business applications.
Powered by Claude Opus 5.5, Duet can ready work and highlight items it believes require attention without awaiting commands. Anthropic has been evaluating that capability with its own staff, relying on their everyday work to observe how Duet manages conflicting priorities. Duet can identify tasks such as an imminent deadline or an email needing a response, but it doesn't always recognize which task should take precedence, and the company concedes the agent can misinterpret exchanges or misjudge importance. The company's public launch materials don't reveal how frequently those errors occur. Duet consumes the same allowance as standard Claude conversations, meaning background work draws against a subscriber's plan caps. Larger jobs—such as constructing a financial model or a presentation deck—require more model turns and background effort. Even Duet's routine scans across connected applications count toward the five-hour session and weekly thresholds. Those scans happen every few hours rather than continuously, in part to prevent exhausting a subscriber's allowance.
"After using Duet for our day-to-day work at Anthropic, we calibrated graders that assess correctness in task prioritization," a company representative told The New Stack. The graders measure whether Duet can separate what merits attention from what can be postponed, though Anthropic recognizes the challenge of evaluating those choices. "Urgency is an inherently subjective call, so we found building an eval for it imperfect but quite helpful," a company representative noted. Anthropic maintains the component of Duet that executes work separate from the component that verifies whether the work aligns with what the user actually requested. "A separate, automated check runs alongside each action, built on the auto mode classifier from Claude Code and extended for Claude Duet," the representative explained. "It assesses whether the action matches your intent."
The separation aims to limit prompt injection, where hostile instructions buried in emails, documents, or webpages attempt to reroute an agent. Researchers have already investigated this threat by planting hidden messages for rogue agents and watching how they reacted. Because that verification runs independently of the model carrying out the task, Anthropic says instructions concealed in a webpage can't directly influence the classifier's verdict. Still, Anthropic's support materials advise users to examine drafts and approval requests because Duet may face prompt injection. The company says an outside firm penetration-tested Duet, but its public materials don't indicate how reliably the separate check intercepts actions beyond the user's intent. Related research shows the difficulty of policing agent scope: Researchers discovered that stripping a prompt section defining authorized and out-of-bounds actions in Claude Code tests raised out-of-bounds actions, although that study didn't test Duet's separate classifier. Duet is built to request approval before actions such as making payments or deleting items, and sharing private health or financial data requires consent—though an explicit request to share it can qualify as that consent. Trading securities, transferring money between bank accounts, and accessing another person's account remain prohibited entirely. Duet's research preview is open by waitlist to U.S. Pro and Max subscribers. Anthropic employees have used it to clear to-do lists while away and discuss workloads on walks, though Anthropic admits the agent isn't as dependable with consumer tasks like booking tickets. The company plans to continue refining the product as testing proceeds. Organizations deploying autonomous agents will need to weigh the efficiency of delegated prioritization against the opacity of decisions made without human oversight, particularly when subjective judgment shapes operational outcomes. The boundary between helpful automation and unaccountable discretion remains difficult to define in practice.