> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/ericclemmons/agent-native/llms.txt
> Use this file to discover all available pages before exploring further.

# Workflow

> The snapshot → interact → re-snapshot pattern for agent-native automation

Agent-native is designed around a simple workflow: **snapshot** the UI to get refs, **interact** with elements using those refs, then **re-snapshot** when the UI changes. This pattern works for any macOS app and handles dynamic interfaces gracefully.

## The core pattern

Here's the fundamental cycle:

<Steps>
  <Step title="Snapshot">
    Capture the current UI state and assign refs to elements:

    ```bash theme={null}
    $ agent-native snapshot Safari --interactive
    Snapshot: Safari (pid 1234) -- 8 elements
    AXWindow "Safari" [ref=n1]
      AXTextField [ref=n2]
      AXButton "Go" [AXPress] [ref=n3]
    ```
  </Step>

  <Step title="Interact">
    Use refs to click, type, or inspect elements:

    ```bash theme={null}
    $ agent-native click @n2
    OK Clicked: AXTextField

    $ agent-native type @n2 "example.com"
    OK Typed 11 chars into AXTextField

    $ agent-native click @n3
    OK Clicked: AXButton title="Go"
    ```
  </Step>

  <Step title="Re-snapshot">
    After the page loads (or any UI change), take a fresh snapshot:

    ```bash theme={null}
    $ agent-native snapshot Safari --interactive
    Snapshot: Safari (pid 1234) -- 15 elements
    # New UI state, new refs
    ```
  </Step>
</Steps>

<Info>
  This pattern mirrors how humans use computers: observe → act → observe again. The snapshot gives you a moment-in-time view of what's possible.
</Info>

## When to re-snapshot

You need a fresh snapshot whenever the UI changes significantly:

### After navigation

Browsers, settings panels, and multi-screen apps:

```bash theme={null}
# Initial screen
$ agent-native snapshot "System Settings" --interactive
$ agent-native click @n5  # Click "Network"

# ⚠️ UI changed - new screen loaded
$ agent-native snapshot "System Settings" --interactive
$ agent-native click @n3  # Now this refers to a button on the Network pane
```

### After dialogs open

Dialogs, sheets, and popovers add new elements:

```bash theme={null}
$ agent-native snapshot Safari --interactive
$ agent-native key Cmd+O  # Open file dialog

# ⚠️ Dialog appeared
$ agent-native snapshot Safari --interactive
# Now refs include the file picker elements
```

### After content loads

Dynamic content loading in browsers or document apps:

```bash theme={null}
$ agent-native snapshot Safari --interactive
$ agent-native click @n3  # Click "Go"
$ agent-native wait 2      # Wait for page load

# ⚠️ Page content changed
$ agent-native snapshot Safari --interactive
```

### When refs fail to resolve

If you get an error like this:

```bash theme={null}
$ agent-native click @n8
Error: Could not re-resolve @n8. The UI may have changed -- run `snapshot` again.
```

**Solution:** Take a new snapshot. The element either moved, changed attributes, or was removed.

<Tip>
  When in doubt, snapshot. Extra snapshots don't hurt, but stale refs will cause errors.
</Tip>

## Working with dynamic UIs

Dynamic interfaces require careful timing:

### Wait for changes to settle

After triggering an action, wait before snapshotting:

```bash theme={null}
$ agent-native click @n5
$ agent-native wait 1.5  # Wait for animation/load
$ agent-native snapshot MyApp --interactive
```

The `wait` command (from `WaitCommand.swift`) simply sleeps for the specified duration.

### Check element state

Use `get` to read element values without re-snapshotting:

```bash theme={null}
$ agent-native get @n2 --attr value
{"value": "example.com"}
```

Or use `is` to check boolean conditions:

```bash theme={null}
$ agent-native is @n4 --focused
true
```

### Use --interactive for cleaner snapshots

Large apps have hundreds of structural elements. Focus on what matters:

```bash theme={null}
$ agent-native snapshot "Final Cut Pro" --interactive --compact
```

This filters to only interactive elements and removes empty containers. From `SnapshotCommand.swift:66-74`:

```swift theme={null}
if interactive && !Self.interactiveRoles.contains(node.role) {
    continue  // Skip non-interactive elements
}
if compact && node.title == nil && node.label == nil
    && node.value == nil && node.actions.isEmpty
    && node.childCount > 0
{
    continue  // Skip empty structural containers
}
```

## Element resolution strategies

You don't always need refs. agent-native supports two resolution strategies:

### Ref-based (snapshot first)

```bash theme={null}
$ agent-native snapshot Safari --interactive
$ agent-native click @n3
```

**Pros:**

* Fast interactions after initial snapshot
* You see exactly what you're clicking
* Refs work across multiple commands

**Cons:**

* Requires a snapshot step
* Refs become stale when UI changes

### Filter-based (direct search)

```bash theme={null}
$ agent-native click Safari --role button --title "Back"
```

From `ElementResolver.swift:6-52`, this searches the tree on-demand using filters:

```swift theme={null}
static func resolve(
    app: String?,
    ref: String?,
    role: String?,
    title: String?,
    label: String?,
    identifier: String?,
    index: Int = 0
) throws -> (element: AXUIElement, node: AXNode, appName: String)
```

**Pros:**

* No snapshot needed
* Works with dynamic UIs where refs would be stale
* Good for one-off commands

**Cons:**

* Slower (searches the tree every time)
* Less visibility into what exists
* Ambiguous if multiple elements match

<Note>
  For agent workflows, prefer **ref-based** resolution. It's faster and gives your agent visibility into the full UI context.
</Note>

## Best practices

### 1. Start with an interactive snapshot

```bash theme={null}
$ agent-native snapshot MyApp --interactive
```

This gives you a clean, actionable view of the UI.

### 2. Use wait after UI-changing actions

```bash theme={null}
$ agent-native click @n7
$ agent-native wait 1  # Let the UI settle
$ agent-native snapshot MyApp --interactive
```

Don't snapshot too soon—animations and loads need time.

### 3. Re-snapshot liberally

Whenever you're unsure if the UI has changed, just snapshot again:

```bash theme={null}
$ agent-native snapshot MyApp --interactive
```

Snapshots are cheap and prevent ref resolution errors.

### 4. Use --compact for complex UIs

```bash theme={null}
$ agent-native snapshot "Adobe Photoshop" --interactive --compact
```

This removes noise and focuses on actionable elements.

### 5. Check snapshots in JSON for parsing

```bash theme={null}
$ agent-native snapshot Safari --interactive --json > snapshot.json
```

JSON output is structured and easy to parse for agents:

```json theme={null}
[
  {
    "ref": "n1",
    "role": "AXWindow",
    "title": "Safari",
    "enabled": true,
    "actions": ["AXRaise"],
    "depth": 0
  },
  {
    "ref": "n2",
    "role": "AXTextField",
    "value": "example.com",
    "enabled": true,
    "actions": ["AXConfirm"],
    "depth": 1
  }
]
```

### 6. Handle ref resolution failures gracefully

Always be ready to re-snapshot if a ref fails:

```python theme={null}
try:
    run(["agent-native", "click", "@n5"])
except RefNotFoundError:
    # Ref stale, refresh and retry
    run(["agent-native", "snapshot", "MyApp", "--interactive"])
    # Parse new snapshot to find the element again
```

## Example: Multi-step workflow

Here's a complete example of opening a file in TextEdit:

<Steps>
  <Step title="Open the app">
    ```bash theme={null}
    $ agent-native open TextEdit
    Opened: TextEdit (pid 1234)
    ```
  </Step>

  <Step title="Snapshot to see what's available">
    ```bash theme={null}
    $ agent-native snapshot TextEdit --interactive
    Snapshot: TextEdit (pid 1234) -- 6 elements
    AXMenuBar [ref=n1]
      AXMenuBarItem "File" [ref=n2]
    AXWindow "Untitled" [ref=n3]
      AXTextArea [ref=n4]
    ```
  </Step>

  <Step title="Click the File menu">
    ```bash theme={null}
    $ agent-native click @n2
    OK Clicked: AXMenuBarItem title="File"
    ```
  </Step>

  <Step title="Re-snapshot to see the menu">
    ```bash theme={null}
    $ agent-native snapshot TextEdit --interactive
    Snapshot: TextEdit (pid 1234) -- 12 elements
    AXMenuBar [ref=n1]
      AXMenuBarItem "File" [ref=n2]
        AXMenu "File" [ref=n3]
          AXMenuItem "Open..." [ref=n4]
          AXMenuItem "Save" [ref=n5]
    ```
  </Step>

  <Step title="Click 'Open...'">
    ```bash theme={null}
    $ agent-native click @n4
    OK Clicked: AXMenuItem title="Open..."
    ```
  </Step>

  <Step title="Wait for the file dialog">
    ```bash theme={null}
    $ agent-native wait 1
    ```
  </Step>

  <Step title="Snapshot the file picker">
    ```bash theme={null}
    $ agent-native snapshot TextEdit --interactive
    Snapshot: TextEdit (pid 1234) -- 23 elements
    # Now includes file picker buttons, text fields, etc.
    ```
  </Step>
</Steps>

Notice the pattern: snapshot → interact → wait → snapshot → interact. This is how you navigate complex UIs reliably.

## See also

<CardGroup cols={2}>
  <Card title="Accessibility tree" icon="sitemap" href="/concepts/accessibility-tree">
    Understanding the tree structure
  </Card>

  <Card title="Refs and snapshots" icon="camera" href="/concepts/refs-and-snapshots">
    Deep dive into the ref system
  </Card>
</CardGroup>