Interaction commands

Control macOS applications by clicking, typing, and manipulating UI elements

Interaction commands perform actions on UI elements using either element references from snapshots or filter-based selection.

click

Click an element by performing the AXPress action.

agent-native click <target> [options]

Arguments

target · string · required

Target element: @ref (from snapshot) or app name

Options

--role · string

Filter by accessibility role

--title · string

Filter by title

--label · string

Filter by label

--identifier · string

Filter by identifier

--index · integer · default: "0"

Which matching element to click (0-indexed)

--json · boolean

Output as JSON

Examples

Click by reference

agent-native click @n15

Output

OK Clicked: Button "Submit"

The command attempts AXPress first, then falls back to AXConfirm if press is not available.

---

fill

Clear a text field and type new text.

agent-native fill <target> <text> [options]

Arguments

target · string · required

Target element: @ref or app name

text · string · required

Text to fill into the field

Options

--role · string

Filter by role (defaults to TextField when using app name)

--title · string

Filter by title

--label · string

Filter by label

--index · integer · default: "0"

Which matching element to fill

--json · boolean

Output as JSON

Examples

Fill by reference

agent-native fill @n8 "[e-mail]"

Output

OK Filled: TextField "Email" with "[e-mail]"

The fill command focuses the element, clears existing content, then sets the new value.

---

type

Type text into an element without clearing existing content.

agent-native type <target> <text> [options]

Arguments

target · string · required

Target element: @ref or app name

text · string · required

Text to type

Options

Same as fill command.

Examples

Type into field

agent-native type @n12 "additional text"

Output

OK Typed "additional text" into: TextField "Notes"

Use type to append text, or fill to replace existing content.

---

check

Check a checkbox or toggle (idempotent - only clicks if not already checked).

agent-native check <target> [options]

Arguments

target · string · required

Target element: @ref or app name

Options

--role · string

Filter by role (defaults to CheckBox when using app name)

--title · string

Filter by title

--label · string

Filter by label

--index · integer · default: "0"

Which matching element to check

--json · boolean

Output as JSON

Examples

Check by reference

agent-native check @n20

Output

OK Checked: CheckBox "Enable notifications"

---

uncheck

Uncheck a checkbox or toggle (idempotent - only clicks if currently checked).

agent-native uncheck <target> [options]

Arguments and options

Identical to check command.

Examples

Uncheck by reference

agent-native uncheck @n21

Output

OK Unchecked: CheckBox "Show toolbar"

---

select

Select an option from a popup button or combo box.

agent-native select <target> <option> [options]

Arguments

target · string · required

Target element: @ref or app name

option · string · required

Title of the option to select

Options

--role · string

Filter by role (defaults to PopUpButton when using app name)

--title · string

Filter by title

--label · string

Filter by label

--index · integer · default: "0"

Which matching element to use

--json · boolean

Output as JSON

Examples

Select by reference

agent-native select @n18 "Dark Mode"

Output

OK Selected "Dark Mode" in: PopUpButton "Theme"

The command clicks the popup button, waits briefly for the menu to appear, then clicks the menu item matching the option title.

---

focus

Set keyboard focus to an element.

agent-native focus <target> [options]

Arguments

target · string · required

Target element: @ref or app name

Options

Standard filter options: --role, --title, --label, --identifier, --index, --json

Examples

Focus by reference

agent-native focus @n10

Output

OK Focused: SearchField "Search"

---

hover

Move the mouse cursor to the center of an element.

agent-native hover <target> [options]

Arguments

target · string · required

Target element: @ref or app name

Options

Standard filter options (no --identifier support)

Examples

Hover by reference

agent-native hover @n25

Output

OK Hovered: Button "Downloads"

Hovering can trigger tooltips and hover states but does not click the element.

---

action

Perform an arbitrary accessibility action on an element.

agent-native action <target> <action> [options]

Arguments

target · string · required

Target element: @ref or app name

action · string · required

Accessibility action to perform (e.g., AXPress, AXConfirm, AXIncrement)

Options

Standard filter options: --role, --title, --label, --identifier, --index, --json

Examples

Custom action

agent-native action @n30 AXIncrement

Output

OK AXIncrement on: Stepper "Volume"

If the specified action is not available for the element, a warning is shown but the command still attempts the action.

Common actions

• AXPress - Click/activate
• AXConfirm - Confirm/submit
• AXShowMenu - Show context menu
• AXIncrement - Increase value
• AXDecrement - Decrease value
• AXRaise - Bring to front
• AXCancel - Cancel operation
• AXPick - Pick/select item

Use inspect command to see which actions are available for an element.

Workflow examples

Login form automation

# Get snapshot
agent-native snapshot MyApp --interactive

# Fill credentials
agent-native fill @n5 "[e-mail]"
agent-native fill @n6 "mypassword"

# Check "Remember me"
agent-native check @n7

# Submit
agent-native click @n8

Form with dropdown

# Fill text fields
agent-native fill @n10 "John Doe"
agent-native fill @n11 "[e-mail]"

# Select from dropdown
agent-native select @n12 "United States"

# Submit form
agent-native click @n13

Multi-step wizard

# Step 1
agent-native fill @n5 "Company Name"
agent-native click @n6  # Next button

# Wait for next page
agent-native wait MyApp --role Button --title "Previous"

# Step 2
agent-native check @n10
agent-native check @n11
agent-native click @n12  # Next button

# Step 3
agent-native fill @n15 "Additional notes"
agent-native click @n16  # Finish button