Agent access

Agent access lets an AI agent connect to a GaiaDesk computer and operate it the way a person does — it takes screenshots, moves the mouse, clicks and types. There is no special API on the far machine and no automation framework installed there. The agent is just another participant in a session, sending the same mouse and keyboard events a human controller sends.

It exists for test agents: an agent that opens a real application on a real machine, works through a real task, and reports whether it worked.

What it is not

• It is not a hosted agent. GaiaDesk does not run models, hold your API key or bill you for inference. You bring your own key (BYOK) or your own agent, and your model provider bills you directly. Screen contents go to the model provider you chose and to nobody else.

• It is not a way into a machine that has not opted in. Agent access is off on every computer until its owner turns it on, and turning it on without issuing a token still admits nobody.

• It does not change human sessions. Nothing about the ordinary connect path moves.

• It is not in the browser. A web page cannot hold a key safely or run an unattended loop. Agent access is the desktop app and the gaiadesk-cli command-line tool only — and gaiadesk-cli ships inside the desktop app on macOS, Windows and Linux. There is nothing extra to install or build.

• It never evicts a person. An agent that finds somebody already on a desk is told host busy and has to wait. It cannot take a machine from a human, or from another agent.

The two front doors

Everything runs through one implementation — the agent bridge — which owns the seventeen computer-use actions, the coordinate maths, the wait-for-settle logic and the audit trail. Two doors lead to it:

Because they share one bridge, a fix to scaling, key mapping or settling lands for both, and there is exactly one place the far machine is touched.

If you are driving Claude, prefer the BYOK runner. It speaks the native computer-use toolset, which is what the model is trained on, and Anthropic's prompt-injection classifiers run only over that toolset's returns. Connecting Claude to the MCP server instead gives up both.

Status note. Both doors ship, in the installed app, and both routes to a desk are live: directly (LAN or GaiaDesk Mesh) and over the internet through the signaling server's agent socket (wss://gaiadesk.net/ws), which is deployed. gaiadesk-cli agent run is wired: it dials, runs the loop, writes an audit log and two reports, and exits with the verdict's own code. See Limits and known gaps.

Quick start: an AI assistant drives a computer

Everything here comes with the installed app. There are two computers in the story, and they can be the same one:

• the desk — the computer the assistant will see and drive, and

• the assistant's computer — where Claude Desktop, Claude Code, Cursor or any other MCP client runs.

The desk is the only one that has to say yes. Settings → Agent access → Let AI agents connect to this computer is that yes, and it is needed only on the desk.

(a) Drive another computer, anywhere

1. On the desk, install GaiaDesk, open Settings → Agent access and turn on Let AI agents connect to this computer. Then open the Agents tab, find Connect an AI assistant, press Issue a token for an assistant to drive this computer, and Copy token. It is shown once.

2. On the assistant's computer, install GaiaDesk and open Agents → Connect an AI assistant. Agent access can stay off here: the paste half of the card issues nothing and lets nothing in. Paste the token and press Set up this computer's assistant.

3. The card now shows the MCP config, filled in: the absolute path of the gaiadesk-cli that shipped with GaiaDesk on this computer, the GaiaDesk server it falls back to (--server, this install's own signaling setting — wss://gaiadesk.net/ws unless yours is self-hosted), where each session is recorded, and the token. Then either

   • press Add to Claude Desktop… — the card shows the exact file it will write and every byte of it, merged with what is already there (other servers and settings are kept exactly as written, and the old file is copied to claude_desktop_config.json.gaiadesk-backup first). Nothing is written until you press Write this to Claude Desktop's config; or

   • press Copy config and paste it into your client's MCP settings yourself (Claude Desktop: Settings → Developer → Edit Config).

4. Quit the client completely and open it again, then ask it: "Use GaiaDesk to open desk 481902774 and take a screenshot." — with the desk id the card showed.

The two computers do not need to be paired or on one network. The assistant dials the desk directly when it can (paired on the same LAN, or both on GaiaDesk Mesh) and through the GaiaDesk server otherwise — see How a desk is reached. The desk only has to be online.

(b) Drive this computer (Claude Desktop here, driving this screen)

1. Turn on Settings → Agent access → Let AI agents connect to this computer — here, because this computer is the desk.

2. Agents → Connect an AI assistant → Issue a token for an assistant to drive this computer, then Set up an assistant here.

3. Add to Claude Desktop… → Write this to Claude Desktop's config, quit Claude Desktop completely, reopen it, and ask it to "Use GaiaDesk to open desk N and take a screenshot" with this computer's desk id.

This goes through the GaiaDesk server, so this computer must be online: a computer is never paired to itself, so there is no direct route from a computer to itself, and GaiaDesk does not invent one. It is still an agent session in every respect — the token, the Agent access switch, your supervision policy, the on-screen disclosure and the recording all apply exactly as for an assistant elsewhere. The card also says so if something on this computer would refuse it (Agent access off, or an allow-list set).

Either way, the desk announces the session on its own screen, and the session appears on its Agents tab, where it can be watched, paused and stopped — see Supervision. Revoke the token on the desk's Agent tokens list to end it for good.

Where gaiadesk-cli is

It is installed beside the GaiaDesk app itself, so its path depends on where GaiaDesk is installed — which is why the card computes it on each computer rather than printing a fixed one. Typical locations:

The commands on the rest of this page write it as plain gaiadesk-cli. In a terminal, use the full path, or add its folder to your PATH.

Turning it on

Agent access is per computer, and the computer being driven is the one that grants it.

1. The switch

On the machine you want an agent to drive, open Settings and find Agent access:

Let AI agents connect to this computer

An agent holding a token issued here can drive this computer without anyone approving the connection — the same trust as your unattended password, scoped to agent sessions and revocable. Every session is announced on this computer's own screen — a notification and a banner that say an AI agent is driving, in different words from a human session and not anything the agent can turn off — recorded, and watchable live.

That switch is the whole consent step. An agent connect never prompts anyone — there is no "allow this connection?" dialog for an agent. A valid token plus this switch is enough, which is why it is worded the way it is.

Turning it off refuses the next agent connect immediately and disconnects any agent session already running.

Once the switch is on, an Agents tab appears in the top navigation. That is where tokens and running sessions live.

2. Issue a token

An agent authenticates with a scoped agent token, never the machine's unattended password. Go to Agents → Agent tokens and fill in:

Press Issue token. The token appears once, in a panel headed Token for "<name>", with a Copy token button:

Copy this token now. It is shown once and never again: this computer keeps only a hash of it, so it cannot be looked up or recovered. If you lose it, revoke it and issue another.

That is literally true — the computer stores a SHA-256 of the token and nothing else. A token looks like gdagt_<desk id>_ followed by 64 hex characters. The desk id in the middle is not secret — it is the number you hand out — and it is there so that the tools can tell, before they connect to anything, that a desk they have been asked to open is not the desk this token is for.

3. The desk id

An agent connects to a desk id — the numeric address GaiaDesk shows for that computer, the same one a human controller types. A token is scoped to the desk that minted it, and that scope binds twice:

• gaiadesk-cli and the MCP server read the desk out of the token and refuse to connect at all to a different one — nothing is sent. This matters because gaiadesk_open_session takes the desk id from the model, and a model reading a hostile web page can be told to name a desk you have never heard of.

• The machine itself refuses a token minted elsewhere, including on a clone of the same VM, with agent token refused: not valid for this host.

4. Revoking

Each token row on the Agents tab shows its state, desk, tier, when it was issued, when it expires and when it was last used. A live token offers Revoke, which asks once:

Revoking refuses this token immediately and disconnects the agent sessions it let in. Other agents on this computer keep running.

Revoking one token stops the runs that token admitted, and no others. The desk records which token admitted each session — an id derived from the stored hash, never the token and never the hash itself — so a revocation can find exactly those runs. Two CI jobs on two tokens are independent: revoking one leaves the other working.

Two things that have not changed. Turning the Agent access switch off still stops every agent session on the desk, because that is not a statement about a credential — it is the desk saying it runs no agents. And the disconnected device is refused for a few minutes afterwards, so if you revoke one token and immediately reissue another to the same CI runner, that runner is locked out for the cooldown even with a perfectly good new token. Rotate with that in mind.

5. Expiry, while a session is running

An expiry is not only a rule about new connections. A session that is already running ends when its token runs out — within about two seconds of the moment on the token's own row, on every one of this machine's host roles, including the app itself. Nothing has to be clicked and nothing has to reconnect for the expiry to bind.

It has the same reach as a revoke, and for the same reason: an expiry is a token no longer being admitted, so it stops the sessions that token admitted and leaves the rest alone. A long CI run whose token expires halfway through is dropped mid-run, so give a token a life longer than the runs it is for.

Headless and locked machines

On a Mac with unattended access installed, three processes can hold the same desk: the app, a login-window agent and a root daemon. All three have to agree about who may drive the desk, and they do not all trust the same file.

• The app and the login-window agent run as you, and read the grant the Settings switch publishes in your own Application Support folder.

• The root daemon takes its authority from a second copy, in /Library/Application Support/GaiaDesk/, which only an administrator can write. Your own copy can only ever narrow what that one allows.

The practical shape of that:

• Revoking a token, letting one expire, or switching Agent access off reaches every process within about two seconds, with no password prompt. Taking access away is not a privilege.

• Turning Agent access on for the background services, or issuing a token for them, needs an administrator password once. The app notices as soon as the two disagree and offers Repair background service; until you run it, the always-on daemon keeps the narrower of the two, so nothing is admitted that you have not authorised. Everything else — the app itself, and the desk while you are logged in — works immediately.

• If no administrator copy has ever been installed, the root daemon refuses every agent session. That is deliberate: a file your own account can rewrite must not be able to hand an agent a session running as root.

If you already have unattended access installed, the app will offer that one-time repair after upgrading. If a machine's own grant ever goes missing or stale, opening the app once republishes it.

On Linux the unattended service is a per-user systemd --user unit and runs as you, so there is nothing above your own account to protect; Windows has no unattended service yet.

Running an agent

Prove the path first

Before wiring anything up, check that an agent can reach the desk at all:

export GAIADESK_AGENT_TOKEN=gdagt_…
gaiadesk-cli agent-connect --desk-id 481902774

This opens a real agent session, takes one screenshot to prove the whole path works, prints the screenshot's size, and exits. It is the connectivity check to put at the top of a CI job.

Exit codes: 0 the session opened and the screenshot came back; 254 the host answered and said no (its own words are on stderr); 255 the desk was not reachable at all.

How a desk is reached

An agent reaches a desk two ways, and the choice is automatic:

1. Directly — over the LAN or over GaiaDesk Mesh, with certificate pinning and no signaling server involved.

2. Through the signaling server's agent socket — used only when the desk is not directly reachable.

A desk you can reach directly is dialled directly even if you passed --server. A direct dial needs no server deployed, survives the server being down, and — see Security — is the only way an agent can reach a host that has an allow-list set.

Either way the host's own authentication runs. A token is not authorisation on its own: it is checked below the removal cooldown, the room cap and the allow-list, and instead of the access code or password. A bad token never falls back to guessing a code.

Door 1 — the BYOK runner

The runner sends your task to Claude with the native computer-use toolset (computer_toolset_20260801) plus one extra tool for the verdict, runs each action the model asks for through the bridge, feeds the results back, and stops on a verdict or a budget.

Your key is read from ANTHROPIC_API_KEY in the environment of the machine running the loop — a developer laptop or a CI runner — and goes nowhere else. It is never sent to gaiadesk.net, never written to the audit log, and never included in a report.

Defaults it runs with: model claude-opus-5; 60 steps (one step is one model turn); 30 minutes wall clock; screenshots scaled to 1280 px on the long edge (the ceiling is 2576); an optional token budget. Old screenshots are cleared by the server, through the Messages API's context-editing (clear_tool_uses), so prompt caching survives a long run and the history the runner sends is append-only. It has to be: newer models reject a request whose earlier turns have been rewritten, and trimming the history client-side is exactly that rewrite.

The command

gaiadesk-cli agent run --desk-id <id> (--task <text> | --task-file <path>) [options]

What a CI job runs, with every flag it is likely to set:

ANTHROPIC_API_KEY=sk-ant-…  GAIADESK_AGENT_TOKEN=gdagt_…  \
gaiadesk-cli agent run \
  --desk-id 481902774 \
  --task-file field-check/click-grid.md \
  --name click-grid \
  --model claude-opus-5 \
  --max-steps 30 \
  --max-duration 600 \
  --long-edge 1280 \
  --allow-domain example.com \
  --out-dir out/run-1

Where the evidence lands

One run is one directory. It is created before the desk is dialled — a run whose report could never be written must not drive somebody's machine first — and it is named with a UTC timestamp so two runs of the same job never overwrite each other and gaiadesk-run-* collects them all with one glob.

gaiadesk-run-481902774-20260921-141320/
├── report.html      the step-by-step report, with the screenshots embedded
├── junit.xml        one suite, one test case, for CI
├── audit.jsonl      one JSON line per action: what, where, how long, outcome
├── run.json         outcome, exit code, steps, actions, elapsed, tokens
└── screenshots/     frame-<id>.png — the picture the model was looking at

--report and --junit still point wherever you like; the audit log, its screenshots and run.json stay in the run directory.

Old run directories are cleaned up. Each one holds a frame-by-frame recording of a real machine's screen, and a nightly job would otherwise keep every frame of every night forever. So when a run starts, it keeps the 20 most recent gaiadesk-run-* directories beside it — itself included — and deletes the older ones. --keep-runs <n> changes the number and --keep-runs 0 turns it off entirely, which is what you want if something else is archiving them.

Three things it never touches: the run in progress; anything not named exactly gaiadesk-run-<desk>-<YYYYMMDD>-<HHMMSS> (your own gaiadesk-run-notes folder stays); and anything at all when you passed --out-dir, because then you chose the layout. It also refuses to delete through a symbolic link, and a failure to remove a directory is logged rather than failing the run.

Exit codes

The exit code is the verdict, not the command's status. Only a report_result call with passed: true is zero.

What it reports about cost

Tokens, not dollars. run.json carries all four counters the API returns — input, output, cache-write, cache-read — and their total, and the same totals are printed when the run ends. The dollar figure is absent, not zero: pricing tokens would mean a hardcoded price table inside a binary that ships for months with no way to learn that a price changed, and a CI report that says $1.24 when the real figure is $3.30 is worse than one that says nothing. The live panel's budget row says the same thing, which is why it reads "Cost not tracked" rather than $0.00.

Door 2 — the MCP server

gaiadesk-cli mcp is a Model Context Protocol server that exposes a desk as thirteen tools. Any MCP-capable client can drive it: it speaks the standard lifecycle (initialize → notifications/initialized → requests, and ping) at protocol versions 2025-11-25, 2025-06-18, 2025-03-26 and 2024-11-05, and the stateless 2026-07-28 revision (per-request _meta, server/discover) beside it. serverInfo.version is the app's release. Tool names are gaiadesk_<tool> (gaiadesk_screenshot, gaiadesk_click, …); the dotted names from before 0.10.324 (gaiadesk.click) are still accepted in a call for one release. It holds no logic of its own — every tool call becomes exactly one bridge action, so the pause gate, the supervision policy, the guardrails and the activity feed all apply identically.

The server takes one agent token, from $GAIADESK_AGENT_TOKEN. That single token is both what the server presents to the desk and what a caller must present to the server. With no token in the environment, stdio still serves and refuses every call — that is how the client that launched it learns why — but --http does not open its port at all, because a listening socket that can serve nobody is one nobody meant to leave running.

Every session records what it did. A session opened on desk 481902774 writes to gaiadesk-mcp-481902774-<UTC stamp>/ under the audit directory — by default mcp-audit in your GaiaDesk data folder (~/Library/Application Support/com.gaiadesk.app/mcp-audit on a Mac, %LOCALAPPDATA%\com.gaiadesk.app\mcp-audit on Windows, ~/.local/share/com.gaiadesk.app/mcp-audit on Linux), never the directory the client started the server in:

gaiadesk-mcp-481902774-20260922-141320/
├── audit.jsonl      one JSON line per action: what, where, how long, outcome
└── screenshots/     frame-<id>.png — the picture the model was looking at

It is the same audit log agent run writes, with the same redaction, and it is written after every action rather than at the end — an MCP server may be killed at any moment, and a log only written at the end is a log that is usually never written. --audit-dir <dir> moves it; there is no flag that turns it off, and a session whose directory cannot be created is closed again without taking a single action. The twenty-directory retention applies here too, counting gaiadesk-mcp-* separately from a run's gaiadesk-run-*.

What an MCP session does not write is the HTML report, the JUnit XML and run.json: those need a task, a verdict and an ending, and a session has none of the three — the client opens it, drives it, and closes it or lets it expire.

Transport A — stdio (the default; your client launches the process). The Connect an AI assistant card writes this for you with the real values; by hand, it is:

{
  "mcpServers": {
    "gaiadesk": {
      "command": "/Applications/GaiaDesk.app/Contents/MacOS/gaiadesk-cli",
      "args": ["mcp", "--audit-dir", "/Users/you/Library/Application Support/com.gaiadesk.app/agent-sessions"],
      "env": { "GAIADESK_AGENT_TOKEN": "gdagt_…" }
    }
  }
}

command is an absolute path: an MCP client does not search your shell's PATH. --audit-dir is optional — without it the sessions are recorded in your GaiaDesk data folder (above), whatever working directory the client gives the server; a session whose audit directory cannot be created is closed before it takes a single action.

Transport B — Streamable HTTP, bound to loopback. The card shows this command, with the full path, for your platform's terminal:

GAIADESK_AGENT_TOKEN=gdagt_… gaiadesk-cli mcp --http
# listening on http://127.0.0.1:7333/mcp
# recording each session under /Users/you/Library/Application Support/com.gaiadesk.app/mcp-audit

{
  "mcpServers": {
    "gaiadesk": {
      "type": "http",
      "url": "http://127.0.0.1:7333/mcp",
      "headers": { "Authorization": "Bearer gdagt_…" }
    }
  }
}

Over HTTP the token is required on every single request, compared in constant time. A request without it is refused even if the request before it on the same connection had one — loopback is not a credential, because any local process, browser page or container neighbour can reach 127.0.0.1. The Origin header is parsed and validated on every request, and GET and DELETE are rejected. A client that opens with initialize gets an Mcp-Session-Id and sends it (and MCP-Protocol-Version) on later requests instead of _meta; the session id is not a credential — the token is still checked every time. A 2026-07-28 client sends _meta and the mirrored MCP-Protocol-Version, Mcp-Method and Mcp-Name headers, which must agree with the body.

The door also has two ceilings, for the same reason: any local process can reach it.

• 32 connections at once. One past that is answered 503 Service Unavailable and closed — never queued. A real client holds one connection and keeps it alive, so this is far above anything legitimate.

• 10 seconds for a request's headers, counted from its first byte. This is separate from the one-minute idle timeout on a kept-alive connection waiting for its next request: an idle client is not a slow one. Without it, a byte a minute held a connection open indefinitely.

• Eight open sessions per server, as before. Close one with gaiadesk_close_session to open another.

The tools. Call gaiadesk_open_session first; it returns the opaque handle every other tool takes. A handle expires after 15 minutes without a call.

x and y are always pixels in the most recent full screenshot, origin top-left. A crop never changes that coordinate space.

A note on keys. Key names are xdotool-style and are translated through one explicit table with no guessing — an unrecognised name is an error and nothing is sent, rather than a plausible keystroke that might be silently wrong on a machine nobody is watching. Nothing is silently remapped across operating systems either: ctrl+c on a Mac means Control-C, not Command-C. super is the Command key on macOS and the Windows key on Windows; the host-specific spellings (cmd, option, win) are accepted only on the host that uses them.

Why coordinates just work

The classic computer-use bug is scaling: the model clicks at a pixel in a screenshot, the real screen is a different size, and on a Retina display it is a different size again. GaiaDesk's wire protocol already speaks fractions of the screen, so a coordinate in a screenshot of known size maps onto the host exactly, whatever its real resolution or backing scale. The host scales the screenshot before sending it, so a 5K frame does not cross the network every turn.

A coordinate that cannot be mapped — no screenshot yet, out of bounds, not a number — is an error returned to the model, with nothing sent. There is no click at a guessed spot.

Waiting for the screen

After a click, a screen animates. Rather than sleeping a fixed second, a screenshot request can ask the host to answer once the display has been still for a short window (250 ms by default) or after a timeout (3 s by default), whichever comes first. The result always says which happened, so a never-settling screen — a spinner, a video — is visible rather than hidden.

Credentials for the system under test

Put credentials for the application being tested inside <robot_credentials> tags in the task text:

Sign in as <robot_credentials>qa@example.com / hunter2hunter2</robot_credentials>
and add two items to the basket.

The model can use them. Everywhere else — the audit log, the HTML report, the JUnit XML, the live activity feed a watcher reads, log lines — they are replaced with [redacted].

Use test accounts, never real ones. Three reasons:

1. Screenshots cannot be redacted. A password typed into a visible field is in the pixels. GaiaDesk's conservative rule is that once an action has carried a credential, every later before-screenshot is marked as possibly showing it, and the HTML report withholds those images with a stated reason — but the files still exist on the runner's own disk, and a screen can show a credential the agent never typed (a password manager, a page that echoes it), which no rule can see.

2. Redaction is literal. The exact string is matched. A URL-encoded, base64-encoded or HTML-entity-escaped form of the same secret is not caught.

3. A credential shorter than 4 characters refuses the run outright, rather than leaving you believing redaction is on when it cannot work.

A <robot_credentials> block that is never closed, or a second one opened before the first closes, also refuses the run rather than guessing where the credential ends.

The domain allowlist

--allow-domain example.com (repeatable) restricts where a browser-driven task may navigate. It is off by default, and it fails closed in every direction: a list that parses to nothing refuses the run rather than meaning "everything is allowed"; one malformed entry refuses the whole list; a target whose host cannot be parsed is refused rather than allowed.

Matching is by label boundary, never substring: example.com admits example.com and app.example.com, and refuses evil-example.com, example.com.evil.net and example.como. Entries must be bare host names — schemes, paths, ports, userinfo, wildcards, leading dots and non-ASCII are all refused (supply the punycode form). Read Security before relying on it.

Supervision

An agent driving a machine through screenshots would, by default, be invisible until its report arrived. That is the wrong default for software operating a computer, so every agent session is a live stream a human can watch while it happens, with the agent's actions drawn on top.

Watchable or Watched

There are two supervision tiers. The host's setting and the token's tier are both read live, and the stricter of the two wins.

Watchable is the intended default. A computer whose policy cannot be read — missing, corrupt, not loaded yet — is treated as Watched, never Watchable.

Under Watched, the watcher count is recomputed from the room's roster and each member's live transport at the instant it is asked. A count that cannot be determined refuses: an unknown watcher is not a watcher. The tier is not latched either — when a watcher joins, the agent can act again without anyone pressing Resume.

One thing to know before choosing Watched: a watcher and an agent are two participants in one room, so Watched can never open on a desk that only supports a single connection at a time. It needs multi-device sessions.

Tightening binds in seconds; loosening needs a control. The setting lives in a file on the machine — and an agent's vocabulary is "type and click anything on this desktop", which includes rewriting that file. So the background services watch it in one direction only: switching a machine to Watched reaches every process within about two seconds and holds a session that is already running, while switching it to Watchable takes effect in the app at once and reaches the always-on services the next time they start. Nothing that merely rewrites the file can release a running agent from being watched.

Where you watch

• The Agents tab lists the agent sessions this machine can see: which host, what state, how long, whether anyone is watching, and steps, time and spend used. Each row has a Watch button.

• A banner announces a session starting — "An agent session started on host" — with Watch and Dismiss. Nobody should find out after the fact that software was driving their machine.

• The watch panel follows you into a session, so the supervision surface is still there while you are connected to something. Actions → Agent panel brings it back if you close it; closing the panel does not end the watch.

What the person at the machine sees

Nobody should discover after the fact that software was driving their machine, so this is a property of the host and not of any screen the agent can reach.

• A system notification, in different words from a human session. A person connecting produces "Remote session started — Someone is now connected to this computer via GaiaDesk." An agent produces "An AI agent is controlling this computer — An automated agent session has started via GaiaDesk. Everything it does is recorded. Move this computer's mouse to pause it." Different title, so a glance at a notification centre tells the two apart.

• A persistent banner across the top of the screen for as long as an agent session is live: "An AI agent is controlling this computer." It is click-through, so it takes nothing from you and you can keep working under it, and it disappears when the session ends.

• The agent cannot suppress, delay or reword either. Both are decided from the room's own record of what the connection was admitted as — a fact set from the agent token before the session was built — and both texts are constants compiled into the app. Nothing carried on the connection reaches the decision or the words. The banner is excluded from screen capture, so the agent cannot see it, aim at it, or find anything on it to click.

• If the host cannot tell what is driving it, it says an agent. A live session whose participants this app never learned of is announced as an agent, not as a person. Being startled about a colleague is better than being reassured about a machine.

• Every participant is labelled in the host panel, so a machine is never listed as a person.

On an unattended machine. The notification and the banner come from the GaiaDesk app in the signed-in desktop session — they need no window open and no view visited. A desk served only by the background service or the lock-screen host has no desktop session to show them in; there the disclosure is the audit log and the Agents view, not the screen.

What a watcher sees

• The live screen, exactly as the agent is operating it.

• A marker where it clicked. The marker is drawn from the coordinate the agent reported. If that coordinate is off-screen, non-finite or malformed the view says "Position refused — off-screen or malformed, marker not drawn" — it never clamps a bad coordinate into a plausible-looking one.

• What it typed, with anything marked as a credential shown as a fixed-width mask (••••••••) that carries no length. On the BYOK door a credential is marked by the task file's <robot_credentials> block; on the MCP door, by secret: true on gaiadesk_type_text. Either way the text is still typed in full, and no watcher and no log ever sees it.

• Anything the guardrails refused, said out loud at the moment it happens: "Refused: Clicked — evil.example.net is not on this run's allowed domains." A refused action never reached the machine, so it draws no click marker, and the row is styled apart from the ones that did happen.

• What it is trying to do — the step number and a short plain-language summary of the model's reasoning. The runner asks for summarized thinking; the raw chain of thought is never available and is not needed.

• Its budget — steps used, time elapsed, and approximate spend. Spend that is not being tracked reads "Cost not tracked", never a fake $0.00.

Everything in that feed is text derived from pixels on a machine you do not own, so it is treated as hostile input: redacted, then stripped of control characters and bidirectional overrides, then length-bounded, then rendered as text and never as markup. The action name is checked by the host against its own table of the seventeen real actions, so a compromised agent cannot write "formatted the disk" into a feed row.

Acting, not just looking

Four controls are available to any human in the room, including a view-only one — being able only to look is not a reason to be unable to stop software. An agent may never send them.

A person physically moving the host's own mouse also pauses every agent running on it, with the reason "a person touched this machine's mouse".

Approving one action

Some things an agent might click are not undoable: paying, agreeing to terms, sending, deleting. When the agent's own words for what it is about to do match one of those categories, the run stops and asks a person, and the action happens only if somebody says yes.

What you see, in the Agents view of whatever device is watching:

The agent is asking to pay for something. left_click: Place order and pay now [ Approve this action ] [ Refuse ] Approving allows this one action. Anything else the agent tries is asked again, and a question nobody answers is refused.

The second line is the agent's own text, shown as text — markup in it is displayed, not run — and if it carried a credential it arrives as a fixed-width mask that reveals nothing, not even the length.

It fails closed, in every direction.

Three things worth being explicit about:

• An approval is for one action. It is not a mode, a session or a window. The next consequential action asks again, with a new question, and an old answer replayed against it does nothing.

• Only a person can answer. The answer must come from a watcher the host authenticated. An agent cannot answer its own question or another run's — the host refuses to relay an approval that came from an agent at all.

• A view-only watcher can answer. Approving is supervision, not control, exactly like Pause and Stop.

If you are running unattended — CI with nobody watching — a consequential action is simply refused, and the run reports it. That is the same behaviour as before this existed; what is new is that a watched run can now get past it.

The hold is enforced on the host, not just asked of the agent. The bridge checks its state before every single action, so a pause lands between two actions rather than after the model's next turn — and independently, the host drops a held agent's input at the machine, releases anything it was holding down, and refuses its clipboard writes, file transfers and terminal too. An agent that ignored every pause message it was sent would still move nothing. Reading is not acting: a screenshot is never blocked, because a paused agent that could not see the screen could not even report why it stopped.

A pause can also land in the middle of one action. Several actions are many events on the wire — a drag is a press, eight moves and a release; a key with repeat is up to two hundred keystrokes; typing goes out in short runs of characters. The host stops those one event at a time, and the pause is then reported to the agent as a refusal rather than as a completed action, so the run's log and the model's own view both say the drag did not finish. One type carries at most 4096 characters; longer is refused outright, never half-typed.

A control names one session

A control carries the id of the one session it is aimed at. If a control arrives that cannot be resolved to exactly one run, the host refuses it out loud and holds nobody:

• naming a run that is not on this host → "that agent session is not running on this host — it may have ended"

• naming nothing, with no agent running → "there is no agent session on this host to control"

• naming nothing, with two or more running → "this host is running N agent sessions and the control did not say which one — update the controller, which is too old to name a session"

The refusal goes back to the one watcher who asked. It is never treated as "apply it to everything".

Reports and exit codes

A test agent's "it worked" cannot be parsed out of prose. The runner gives the model one extra tool, report_result, with a strict schema ({ passed: boolean, reason: string, evidence: string[] }, all required, no extra fields), and the run's outcome comes from that call and nothing else.

A run that ends without calling it fails, reported as "no verdict". Prose saying "the test passed" is not a verdict. A model that calls report_result with passed: false under cheerful prose has failed the test. A malformed call is an error, never coerced into a default.

Outputs are what CI already understands:

Plus JUnit XML for a CI test report and a self-contained HTML report with the step-by-step screenshots. Every string in both is redacted first and escaped second. Underneath is a per-session audit log recording every action, its coordinates in both screenshot and normalized space, the screenshot taken before it, how long it took and its result — the report's raw material, and the record if an agent ever does something it should not have. The log is written before the action happens, so a run that dies mid-action still leaves an entry saying what was attempted. An action the guardrails refused is on the record too — the off-allowlist navigation, the click on "Place order" — with the refusal as its outcome and no screenshot, because nothing happened. It is the one event a supervisor most needs to be able to find afterwards, and it also reaches a live watcher as it happens.

A screenshot taken after the agent typed a credential is saved with the run like any other, at 0600, but the HTML report withholds it and says so in its place: the field it was typed into may still be showing it, and a report is something people forward.

What it costs

GaiaDesk charges nothing for this. Your model provider bills you.

Screenshots dominate the cost, and every turn re-sends the retained history, so the real number depends almost entirely on whether prompt caching is working. A worked estimate for a 30-step run on Claude Opus 5, at $5/MTok input, $25/MTok output, cache writes at 1.25× and cache reads at 0.1×:

At the default 60-step limit, expect roughly $2.50 – $3.50 with caching working and $7+ without. Cost rises slightly faster than linearly with steps, because the per-turn history grows.

This is an estimate, not a measurement — no run against the live API has been made. Re-check current prices before quoting any figure. The step limit is the cost ceiling, which is why it exists, alongside a wall-clock limit and an optional token budget.

If you want to know whether caching is working, watch cache_read_input_tokens in the per-turn usage. It should be above zero on every turn after the first. Stuck at zero means something in the prompt prefix changes every turn.

Security, stated honestly

An AI with control of a real machine can do anything that machine's user can. The primary control is where you run it: a dedicated VM or test machine with no sensitive data on it and no credentials worth stealing. Everything below is a second line, not a first.

Prompt injection is the defining risk

Reading untrusted screens is a test agent's whole job. The layers available:

• The native toolset's classifiers. Anthropic runs prompt-injection classifiers over what the computer toolset returns, including screenshots — but only for the native toolset, which is the BYOK door. This is the single best reason to prefer BYOK over MCP when the model is Claude.

• The run's configuration cannot be reached from the model's vocabulary at all. This is structural, not a sentence in a prompt asking the model to please refuse: the guardrails are built once, nothing in them is mutable, and none of the eighteen names the model can speak is a configuration verb.

• A consequential-action approval. Words like "confirm order" in an action's text stop the run and put the question in front of a watcher, who approves or refuses it — see Approving one action. It is a keyword list over the model's own action text, so a click by coordinate on a "Pay" button with no text is not matched. It is a helpful stop, not a complete one: what it catches, it stops dead; what it does not match, it never sees.

What the domain allowlist actually blocks

It refuses a navigation the agent itself plans — a URL the model named, text it is about to type into an address bar. That is the whole of it. It cannot stop:

• a redirect the page performs on its own, or a <meta refresh>

• a script setting location

• a link clicked by pixel coordinate, which carries no URL at all

• an iframe, a cross-origin form action, a pop-up, a fetch or an XHR

• a URL assembled across several separate actions

• a bare host name inside a sentence — type "go to evil.com now" is prose, not a navigation

We observe a screen. We are not a proxy and there is no URL-bar hook. A real boundary is egress control on the host: a proxy, a DNS policy or a firewall on the test VM. Treat the flag as a guard on the agent's own choices and nothing more.

The access rules

• An agent never evicts a human. An agent meeting an occupied desk is told host busy and waits — including when the occupant is another agent.

• A token is not authorisation. It is checked below the removal cooldown, the room cap and the allow-list, and it never falls back to an access code or password.

• A host with an allow-list cannot be reached by an agent over the signaling door. An agent socket carries no account, so an allow-list — which matches on account — refuses it with the host's own "not authorized for this host". The same host on the same LAN or mesh is reachable directly, which is exactly why a reachable desk is always dialled directly. A fleet that uses allow-lists will otherwise read this as "agents don't work".

• A failed agent token does not lock out a human. Wrong tokens count towards no brute-force lockout, so a misconfigured CI job cannot lock a desk's owner out of their own machine.

• Tokens travel inside wss, in the clear to the signaling server, exactly as an access code does. Hashing at rest does not help in transit. A compromised signaling server would see every agent token it relays. The direct door does not involve it at all.

Limits and known gaps

Current as of this build. None of this is hidden in a footnote elsewhere.

Not yet wired

• An assistant driving the computer it runs on needs the GaiaDesk server. The direct door refuses a computer dialling itself — a desk is never pinned to itself, so the certificate check fails on both ends — and the dial falls back to the server's agent socket, which admits it like any other agent. So self-drive works online only, and not on a computer with an allow-list (an agent carries no account). The card says both.

• With Agent access off, nothing on the Agents tab issues a token. The tab is still there (for the assistant card's paste half, and to revoke tokens), but both issue controls — the card's and the Agent tokens form — appear only with the switch on, because issuing is for being driven.

• Neither door writes an HTML report for an MCP session. Both doors write the audit log and its screenshots; the report, the JUnit XML and run.json are agent run's, because they need a task, a verdict and an ending. If you need a CI test report, use the BYOK door.

• There is no stop control on the agent run process itself. The seam exists and is tested (the run's own pause gate, held between actions), but nothing wires it to a signal or a key: a run is stopped from the host — a watcher's stop control, or turning Agent access off — not from the terminal that launched it. Ctrl-C kills the process, which ends the session and leaves the artifacts unwritten.

• Nothing has ever billed a cent. The runner has only ever run against a local fake of the Messages API. The token counts it reports are real arithmetic over what the API returned; no run against the live API has happened, so no cost figure in this document or the runbook is a measurement.

Not proven on real hardware

Nothing in this feature has yet run against a real Retina panel, a real non-integer Windows display scale, a real relay hop, a real second participant, a real lock screen, or the live Messages API. Every claim below is from tests, not from a machine.

• Double- and triple-click fidelity is a timing bet. Two press/release pairs 60 ms apart are counted as a double click by the operating system, against the host user's own double-click interval. Worse: one input injector is shared by every participant, so a human clicking on the host 200 ms before the agent clicks can silently promote the agent's single click to a double click.

• Drag recognition by real toolkits (eight intermediate moves, 15 ms apart) is unproven.

• Settle detection on Windows and Linux hashes one byte per pixel, so a change confined to the other three colour channels is invisible to it. A blinking caret or a taskbar clock can also keep a screen "changing" forever.

• A screenshot every turn over a relayed path has never been measured.

Known behaviours that will surprise someone

• The MCP server's path must be a stable one. Opened straight from Downloads, macOS runs GaiaDesk from a temporary copy; as an AppImage on Linux, its files live in a mount that moves on every launch. The card warns in both cases. Move GaiaDesk to Applications, or install the .deb/.rpm.

• An MCP client reads its config only when it starts. After adding GaiaDesk, quit the client completely — not just its window — and reopen it.

• Claude Desktop from the Microsoft Store may keep its config somewhere other than %APPDATA%\Claude, which is where Add to Claude Desktop… writes. If GaiaDesk does not appear, paste the config through Claude Desktop's own Settings → Developer → Edit Config instead.

• macOS may ask whether your MCP client may find devices on your local network the first time it starts GaiaDesk's server: the server first tries the desk over the LAN, and macOS attributes that to the app that launched it. Allow it for the direct route; refused, the server route still works.

• The Agents view's own banner is not the disclosure. It is raised when the app starts and when the view is opened or refreshed, so an owner elsewhere learns of a session only if their app happens to be looking. What the person AT the driven machine is told does not depend on it — see What the person at the machine sees.

• A revoked device is refused for a few minutes, so revoke-and-reissue locks a CI runner out for the cooldown.

• Typing on the host's own keyboard does not pause the agent. Only the mouse is watched. A person typing without touching the mouse shares the keyboard with the agent.

• "Watched" cannot open on a single-connection desk, because the watcher and the agent need to be two participants in one room.

• A token that expires mid-run drops the run, and the other runs that same token admitted, for the same reason a revoke does. Runs on other tokens are left alone.

• A consequential action with nobody watching is refused, not queued. An unattended run cannot get past one; see Approving one action.

• A revoke or a switch-off reaches the other host roles within ~2 seconds, not instantly — and if the app cannot write to disk, the other roles keep the previous grant until the app next starts.

• "Last used" on a token only reaches disk when the app next saves, and a fleet driven entirely through a headless role will show "Never used" for tokens in daily use.

• A watcher on a remote host cannot always name a session. If that host is running two agents, its controls are refused out loud rather than guessing. Watching one agent per host works today.

• A run's task description is not on the wire, so a session row reads "No task described".

• The Agents tab lists this machine's own sessions plus whatever it observes on a live connection — not every agent on your account. There is no account-wide session listing.

• Elapsed time mixes two clocks — the host stamps the start, the viewer computes the elapsed — so a badly skewed host clock makes the number wrong.

• On the native-video connection path there is no video stream this panel can host, so the watch picture reads "Waiting for this host's screen…". The feed, the controls and the reasoning summary still work.

• hold_keys and wait accept up to 300 seconds each, during which nothing the idle timer counts happens. Agent sessions are exempt from idle disconnect for that reason; a host configured otherwise is a hazard.

• On Windows, ctrl+alt+Delete does nothing. The secure attention sequence cannot be injected by anything, and this is not something GaiaDesk can fix.

• Uninstalling does not delete the published grant file. It holds no plaintext — labels, host ids and hashes — but a "reset" machine still carries its old revocation record, and so does a cloned VM (harmlessly: a clone's tokens are refused as belonging to another host).