> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/ericclemmons/agent-native/llms.txt
> Use this file to discover all available pages before exploring further.

# Accessibility tree

> Understanding macOS Accessibility tree and how agent-native uses it

The macOS Accessibility (AX) tree is a hierarchical representation of every UI element in an application. It's how agent-native "sees" and interacts with apps—similar to how a DOM represents a web page, the AX tree represents native macOS interfaces.

## What is the Accessibility tree?

The Accessibility tree is provided by macOS through the Accessibility APIs (Carbon's `AXUIElement`). Every running application exposes its UI hierarchy through this system, which was originally designed for assistive technologies like VoiceOver.

Each node in the tree represents a UI element—buttons, text fields, windows, menus, etc. The tree structure mirrors the visual hierarchy: windows contain groups, groups contain buttons, and so on.

<Info>
  The Accessibility tree is built dynamically by querying macOS APIs. agent-native doesn't modify the tree—it's a read-only view of the application's current UI state.
</Info>

## AX roles

Every element has a **role** that describes what kind of UI component it is:

* `AXWindow` - Application windows
* `AXButton` - Clickable buttons
* `AXTextField` - Text input fields
* `AXStaticText` - Read-only text labels
* `AXGroup` - Container elements
* `AXMenuBar` - Menu bars
* `AXMenuItem` - Individual menu items
* `AXCheckBox`, `AXRadioButton`, `AXSlider`, etc.

See the complete list in `SnapshotCommand.swift:26-33`.

## AX attributes

Beyond role, elements have **attributes** that provide additional information:

```swift theme={null}
// From AXNode.swift:4-19
struct AXNode {
    let role: String          // AXButton, AXTextField, etc.
    let subrole: String?      // More specific role variants
    let title: String?        // AXTitle - button text, window title
    let value: String?        // AXValue - text field contents, slider values
    let label: String?        // AXDescription - accessibility label
    let identifier: String?   // AXIdentifier - programmatic ID
    let enabled: Bool         // AXEnabled - can the user interact?
    let focused: Bool         // AXFocused - has keyboard focus?
    let x, y: Double?         // AXPosition - screen coordinates
    let width, height: Double? // AXSize - element dimensions
    let actions: [String]     // Available actions (AXPress, etc.)
}
```

<Note>
  Not all attributes exist on every element. A button might have `title` but no `value`, while a text field has `value` but might not have `title`.
</Note>

## AX actions

Elements expose **actions** that can be performed on them:

* `AXPress` - Click a button
* `AXConfirm` - Activate/confirm
* `AXCancel` - Cancel/dismiss
* `AXRaise` - Bring window to front
* `AXShowMenu` - Show context menu

From `AXEngine.swift:188-193`, agent-native queries available actions:

```swift theme={null}
static func actions(_ element: AXUIElement) -> [String] {
    var names: CFArray?
    let result = AXUIElementCopyActionNames(element, &names)
    guard result == .success, let arr = names as? [String] else { return [] }
    return arr
}
```

## Tree structure and traversal

The tree is built by recursively walking from the application root element down through children. From `AXEngine.swift:236-275`:

```swift theme={null}
static func walkTree(
    _ element: AXUIElement,
    path: String = "",
    depth: Int = 0,
    maxDepth: Int = 5
) -> [(node: AXNode, depth: Int)] {
    // Build path like /AXWindow[@title="Settings"]/AXButton[@title="OK"]
    let fullPath = path.isEmpty ? "/\(segment)" : "\(path)/\(segment)"
    let node = nodeFrom(element, path: fullPath)
    var results: [(AXNode, Int)] = [(node, depth)]
    
    // Recurse through children
    guard depth < maxDepth else { return results }
    for child in children(element) {
        results += walkTree(child, path: fullPath, depth: depth + 1, maxDepth: maxDepth)
    }
    return results
}
```

<Warning>
  Deep trees can be expensive to traverse. Use `--depth` to limit how far agent-native walks. The default is 5-8 levels depending on the command.
</Warning>

## How agent-native uses the AX tree

Agent-native accesses the tree through these operations:

<Steps>
  <Step title="Create application element">
    Connect to a running app by PID:

    ```swift theme={null}
    // AXEngine.swift:138-140
    static func appElement(pid: Int32) -> AXUIElement {
        AXUIElementCreateApplication(pid)
    }
    ```
  </Step>

  <Step title="Read attributes">
    Query element properties:

    ```swift theme={null}
    // AXEngine.swift:142-158
    let title = stringAttr(element, kAXTitleAttribute)
    let enabled = boolAttr(element, kAXEnabledAttribute)
    let position = position(element)  // CGPoint
    ```
  </Step>

  <Step title="Find elements">
    Search the tree by role, title, label, or identifier (see `AXEngine.swift:279-356`).
  </Step>

  <Step title="Perform actions">
    Execute actions like `AXPress`:

    ```swift theme={null}
    // AXEngine.swift:361-363
    static func performAction(_ element: AXUIElement, action: String) -> Bool {
        AXUIElementPerformAction(element, action as CFString) == .success
    }
    ```
  </Step>
</Steps>

## Permissions required

Accessing the Accessibility tree requires explicit user permission. agent-native checks and requests access:

```swift theme={null}
// AXEngine.swift:379-386
static func checkAccess() -> Bool {
    AXIsProcessTrusted()
}

static func requestAccess() {
    let options = [kAXTrustedCheckOptionPrompt.takeRetainedValue(): true] as CFDictionary
    AXIsProcessTrustedWithOptions(options)
}
```

<Warning>
  You must grant Accessibility permissions in **System Settings > Privacy & Security > Accessibility**. Without this, all commands will fail with `accessDenied`.
</Warning>

## See also

<CardGroup cols={2}>
  <Card title="Refs and snapshots" icon="camera" href="/concepts/refs-and-snapshots">
    How agent-native assigns refs to tree elements
  </Card>

  <Card title="Workflow" icon="arrows-spin" href="/concepts/workflow">
    The snapshot → interact → re-snapshot pattern
  </Card>
</CardGroup>