API

The Action agent is a local WebSocket server inside Action.app. This page documents its raw RPC protocol. Most agents use the Action MCP server instead (see Agent setup). The MCP server adds sessions, target resolution, recording status, stage, and private layers on top of this protocol.

Source of truth: ActionAgentProtocol.swift and ActionAgentRuntime.swift.

Transport

status reports authentication: "capability-token" and authTokenFile, the path of a token file that the agent writes at each launch. The file is user-only (0600) in a user-only directory (0700).

Privileged requests must include protocolVersion and authToken in their params. The privileged requests are:

• drive.begin with mode: "attention";
• workspace.dragFile;
• workspace.cancelOperation.

Every method requires an authenticated connection.

Private connection

Action 0.4.0 uses the private connection below. Install its matching runtime and Lattices CLI 0.15.1. Earlier TCP clients do not automatically fall back.

Every connection now needs the token, not only the privileged methods. The privileged methods above still check authToken in their params as well. The Action runtime and lats action call read the token for you.

Remote machines cannot reach this socket. A paired tailnet client can call a reviewed subset of Action methods through the Lattices gateway, one exact grant per method, for example --grant action:status. The gateway opens one Action connection per call. It therefore denies drive.begin, drive.touch, drive.release, workspace.dragFile and workspace.cancelOperation, which create or use a lease owned by a persistent connection. drive.status remains available with its own read grant. See Private IPC and the gateway.

The receive side stays armed while a request runs, so a cancellation can arrive during a gesture. Responses are serialized and correlated by request id.

Request

{
  "id": "4f2c9b1e",
  "method": "capture.screenshotScreen",
  "params": { "output": "/tmp/screen.png" }
}

• id: request id. Native clients default to a UUID.
• method: a method name from the table below.
• params: a flat map of string keys to string values. Send numbers and booleans as strings ("1440", "true").

Response

{
  "id": "4f2c9b1e",
  "ok": true,
  "result": { "outputPath": "/tmp/screen.png", "status": "screenshot" },
  "error": null
}

• ok: success flag.
• result: a flat string map. Structured values, such as a drive lease, arrive as JSON strings inside this map.
• error: an error string when ok is false.

Methods

Runtime and permissions

Apps, windows, and input

calculator.buttons, calculator.clickButton, and calculator.displayValue exist for the Calculator demo scenario. They are not a general API.

Drive leases

A drive lease tells other clients that an agent is acting on the desktop. The lease belongs to the connection that created it. Closing the connection cancels that connection's leases only.

Workspace drag

workspace.dragFile is not a generic coordinate drag. It rejects unknown params and accepts no source point, destination point, duration, command, or executable. Action resolves exactly one Finder item by AXURL inside exactly one titled Finder window. Action resolves exactly one run-scoped AX drop-zone identifier inside exactly one titled app.openscout.scout window. All resolved frames and points must be inside the supplied display bounds.

An approved attention drive.begin acquires the global system-pointer resource and grants one workspace.drag-file capability. The capability is bound to the connection, run, and workspace, and is consumed before the gesture. During the drag, Action checks the lease and operation stop signals on every step. Cleanup always attempts mouse-up and restores the saved pointer position.

Capture

Recording contract

The agent does not record in its own process. A record method launches a fresh Action.app instance in recording-probe mode, and that probe performs the ScreenCaptureKit capture.

• A successful reply means the probe started.
• The recording ends when the stopFile appears.
• Completion is the movie at output plus the finishedFile marker. The marker can also contain a recording error.
• Pass debugLog to collect probe logs.

See Recording.

What is not in this protocol

• Private layers (layer_*), stage, observation, target resolution, and session artifacts are MCP and runtime features. The raw agent does not orchestrate them. See Agent layer.
• The browser_* tools belong to the Lattices MCP. See Browser.
• computer.* methods belong to the Lattices core daemon on lattices.sock, not to this agent. See the Lattices API.

From the Lattices CLI

lats action call <method> [params-json] sends one request and prints the result. In released builds it does not attach a capability token, so it cannot call privileged methods. The matching runtime reads and attaches the token automatically; see Private connection.

lats action call status
lats action call window.getFrame '{"bundleId":"com.apple.calculator"}'

Remote Linux hosts

The raw agent protocol is local to the Mac. For a Linux desktop, the Action runtime connects to lattices-host instead. See Remote engine.