What can an automation tool actually see?

A screenshot shows appearance. An accessibility tree supplies the interface information the app exposes. Each answers a different question.

Visual comparison · 4 min · Reviewed 2026-10-02

Two views of the same window

A screenshot contains pixels: layout, colors, artwork, and visible text. An accessibility query can return structured information about exposed controls, including roles and supported actions. A screenshot alone does not carry a button’s AX action; an AX label alone does not describe an illustration.

Try a question before choosing an approach

Use the selector below to compare finding a named button with checking a drawing. The example is a teaching model. Real results depend on the app, permissions, and the automation tool’s implementation.

What do you want to know?

Accessibility information

Window
└─ Button
   label: Save
   action: AXPress

The example app names the button and supplies an action. The tool can use this information to identify it.

Screenshot

Save

The button is visible as pixels. Recognizing its text and location requires image interpretation; its AX action is not part of the image.

Both views are illustrative. No screenshot of your Mac is captured.

Why both can be useful

A tool can use accessible information to identify a control, then examine the visible result after acting. For a custom canvas, pixels might show detail that the accessibility tree omits. Neither view, by itself, proves that the intended task succeeded.

Choose tools around the job

Start with the outcome: arrange your workspace, automate a repeatable task, or interact with an interface. Lattices focuses on the workspace. Action is the separate Lattices product for computer use. The tool directory explains where other Mac utilities fit.

Watch the animated explanation

See why an accepted action still needs a result check

Go to the source

Documentation reviewed 2026-10-02. Examples are illustrative; this is not a hands-on product benchmark.