Claude Duet’s Background AI Work Raises Questions About Priorities, Safety and Usage

Oct 29, 2026 791 views
Robot hand over keyboard

Claude Duet’s central promise is also its hardest product problem: it watches connected apps, works in the background and decides when to interrupt a user. Anthropic is evaluating whether those choices are useful, but its account makes clear that task ranking remains a judgment call—and a model can respect its permissions while still focusing on the wrong thing.

Duet, announced October 21 and powered by Claude Opus 5.5, can prepare work and flag items without waiting for a request. The company has been testing it with employees on their daily work, then using those experiences to assess how it handles competing demands. A week after launch, Anthropic described its evaluation approach to The New Stack.

Task ranking is a judgment problem

Anthropic says it developed internal graders to assess whether Duet prioritizes tasks correctly. A company representative said the team used Duet in its own day-to-day work to calibrate those graders, then ran them offline on cases reflecting internal use and early external testing.

The approach can test whether the assistant spots an approaching deadline or an email that needs a reply. It can’t turn urgency into an objective measurement. “Urgency is an inherently subjective call, so we found building an eval for it imperfect but quite helpful,” the representative told The New Stack.

That caveat matters for an assistant operating across email, calendars and business applications. Duet may notice a time-sensitive item and still misjudge its place among other work. Anthropic acknowledges that the agent can misread conversations or assess urgency incorrectly. Its public launch and support materials don’t say how often either problem occurs, leaving users without a published measure of the system’s prioritization accuracy.

The product’s value depends on making these decisions well enough to save attention rather than consume it. A quiet, low-stakes task may be safe to defer; a missed deadline or delayed response may carry a real cost. Internal testing gives Anthropic a way to examine representative cases, but the company’s description doesn’t establish how reliably the graders reflect the competing priorities of different users.

A separate check reviews actions

Anthropic describes another safeguard for work Duet performs: the model carrying out a task is checked by a separate automated system that assesses whether an action matches the user’s intent. The company says that check builds on the auto mode classifier used in Claude Code and has been adapted for Duet.

Running the check independently is intended to limit the effect of instructions hidden in a webpage. Such instructions could otherwise attempt to redirect an agent as it reads emails, documents or sites. Anthropic says an outside firm penetration-tested Duet, but public materials don’t report how often the separate check catches an action that falls outside a user’s request.

Research on Claude Code offers relevant context, though it did not test Duet. Researchers found that removing a prompt section defining authorized and out-of-scope actions increased out-of-scope behavior in their Claude Code tests. That result doesn’t tell us how Duet’s classifier performs; it does show why keeping an agent within the user’s intended scope takes more than identifying the original instruction.

Permissions and prompt injection

Prompt injection remains a concern because an agent may encounter hostile instructions inside material it is asked to process. Anthropic’s support materials advise users to review drafts and approval requests, recognizing that Duet can come across such attempts. The separate action check is one defense, while the review step leaves consequential choices visible to the user.

Duet works through apps connected to the user’s Claude account. Disconnecting an app stops further access, but information already stored in Duet’s memory remains until that memory is reset. Anthropic says the assistant does not see passwords, browser cookies or sign-in tokens, and cannot store those credentials in memory.

For certain actions, Duet is designed to request approval, including payments and deletions. Sharing private health or financial information requires consent; an explicit user request to share that information can count as consent. Other activities remain prohibited, including securities trading, transfers between bank accounts and access to another person’s account.

These controls describe boundaries, but they don’t remove the need to understand what the assistant has done. Duet provides a task list showing completed and ongoing jobs, with a separate page for each task. Users can add instructions while a job is underway, and review activity or approve actions.

Visibility stops short of live oversight

During the research preview, users can’t watch Duet operate its cloud browser or take control of it. They can review task activity, but not inspect every browser step as it happens. Anthropic’s public launch and support materials also don’t describe a way to undo completed actions across connected apps.

That creates a practical gap between reviewing an agent’s progress and observing its work in real time. For a task involving several apps, the task page offers a record of activity, while the browser itself remains out of view. Users who want to intervene during execution have fewer options than those who can simply revise instructions or approve a request.

Background monitoring draws on plan limits

Duet uses the same allowance as regular Claude conversations, so its background activity counts against a subscriber’s plan limits. Larger assignments, such as building a financial model or presentation deck, require more model turns and background work, an Anthropic representative said. Routine checks across connected apps count toward both five-hour session limits and weekly limits.

The checks run every few hours rather than continuously, partly to keep background activity from consuming a user’s allowance too quickly. That schedule creates a trade-off: an urgent message arriving between checks may wait until the next one to be noticed. More frequent checks would use more resources; less frequent checks could extend the delay. Anthropic hasn’t published a figure for the share of typical usage consumed by background work.

Early use will shape the product

The research preview is available by waitlist to U.S. Pro and Max subscribers, with international availability planned. Anthropic employees have used Duet to work through to-do lists while away from their desks and to talk through workloads while walking. The company has also said the assistant is less reliable on consumer tasks such as booking tickets.

Those examples point to a product still finding the right boundary between useful background help and unreliable delegation. Duet’s evaluation, action checks and approval requests address distinct parts of that problem, while its polling schedule and usage limits shape how often it can intervene. Anthropic says it has already made changes in response to feedback and expects to continue adjusting the system as testing proceeds.

For now, Duet is best understood as an assistant that can prepare and surface work, with users still responsible for judging its priorities and reviewing consequential actions. The next measure of progress will be whether Anthropic can make those interruptions dependable while keeping the background work within the time and usage limits users actually have.

Source: Amanda Caswell · thenewstack.io

Comments

Sign in to comment.
No comments yet. Be the first to comment.

Related Articles

“Urgency is an inherently subjective call”: Anthropic tel...