AI That Controls a Computer: Two Years Since the First Demos and the Question of Trust

Two years ago, the idea of artificial intelligence clicking through an app and filling out a form on its own seemed like an experiment. Today, it’s becoming a product for everyday users. What has changed since the first demonstrations, what are the limitations, and why is security more important than the mere ability to click?

How It All Began

Anthropic introduced the computer control feature on October 22, 2024, as a public beta version for developers via an API. The model was given screenshots and was able to decide where to move the cursor, what to click on, and what to type. The company openly acknowledged even then that this was an early stage. The model had trouble with common tasks, such as scrolling through content, dragging elements, or zooming in. At the same time, the company stated that it had added tools to detect abuse, such as spam or fraud.

What has changed since then

In March 2026, a similar feature was made available to regular users as a research preview. Initially, it was available only to subscribers of higher-tier plans and only on macOS. The ability to assign tasks from a phone while the computer is working elsewhere was also added. The company reiterated that the feature is still in its early stages and may have bugs.

So the trend is clear. What started as a demo for developers is becoming a tool designed to work on a person’s own device.

How such an agent works

The principle is simple. The agent repeats a cycle: it looks at the screen, decides what to do, performs one step, and checks the result again. It doesn’t need a special interface for each application. It uses the same graphical environment as a human. That’s why it can be deployed even where there is no API.

At the same time, this is its weakness. Each step requires an assessment of the screen, and slow or time-consuming tasks can take longer. If the window changes unexpectedly, the agent may make a mistake.

Security Risks

• Embedded instructions: A website or document may contain text that attempts to provide the agent with new instructions. The agent must distinguish between instructions from the user and content that it is merely reading.
• Broad access: If the agent has access to the browser, files, and logged-in accounts, an error can have more serious consequences than with a chatbot.
• Irreversible actions: Sending a message, making a payment, or deleting files cannot always be undone.
• Privacy: Screenshots may contain sensitive information that the user did not intend to share.

How to Protect Yourself

• Grant the agent access only to the necessary applications and folders.
• Require confirmation for sensitive actions, such as payments or sending messages.
• Do not run the agent on high-value accounts without supervision.
• Check what the agent has done, especially for long tasks.
• Use a separate profile or device for testing.

No More Clicking?

Not yet. Today’s agents can handle repetitive and clearly defined tasks, but they are unreliable in unusual situations. A more realistic scenario is that they will take over the tedious routine tasks, while humans oversee and approve them. The key factor won’t be whether an agent can click, but whether we can trust it to click correctly.