Troubleshooting

Common errors, solutions, and debugging techniques for agent-native

Common errors

Accessibility permission denied

Error message:

Accessibility access denied. Enable in System Settings > Privacy & Security > Accessibility

Cause: Your terminal or IDE doesn’t have Accessibility permissions.

Solution:

1. Open System Settings

   open "x-apple.systempreferences:com.apple.preference.security?Privacy_Accessibility"
   

   Or manually: System Settings > Privacy & Security > Accessibility

2. Add your terminal

   Click the + button and add:

   • Terminal.app
   • iTerm.app
   • VS Code
   • Your IDE

   If the app is already in the list, try toggling it off and on.
3. Restart your terminal

   Quit and relaunch your terminal completely for permissions to take effect.

4. Test access

   agent-native apps
   

   If this works, permissions are correctly set.

See AXEngine.swift:378-386 for the access check implementation.

Changes to Accessibility permissions require a full restart of the app. Closing a window is not enough.

App not found

Error message:

App not found: Slak

Cause: Typo in app name, or app is not running.

Solutions:

Check running apps

agent-native apps

This lists all running GUI applications. Verify the exact name.

Launch the app first

agent-native’s open command launches apps if they’re not running:

agent-native open Slack  # Launches if needed

Use bundle ID

If the app name is ambiguous, use the bundle ID:

agent-native open com.tinyspeck.slackmacgap

Find bundle IDs with:

agent-native apps --format json | jq '.[] | {name, bundleId}'

Check case sensitivity

App names are case-insensitive, but try exact capitalization:

agent-native open "System Settings"  # Correct
agent-native open "system settings"  # Also works

See AXEngine.swift:56-63 for app finding logic.

Ref not resolving

Error message:

Ref @n42 not found. It may be stale. Re-snapshot to get fresh refs.

Cause: The UI structure changed since the last snapshot, invalidating the ref.

Why this happens:

• You clicked a navigation element (changed pane/view)
• App loaded new content
• A modal opened or closed
• Window was resized

Solution:

# Re-snapshot to get new refs
agent-native snapshot "System Settings" -i

# Now use the new refs
agent-native click @n5

Always re-snapshot after actions that change the UI structure.

Best practice:

# Take snapshot
agent-native snapshot App -i > /tmp/snap1.txt

# Interact with refs from snap1.txt
agent-native click @n5

# Re-snapshot after UI change
agent-native snapshot App -i > /tmp/snap2.txt

# Use refs from snap2.txt for next interactions
agent-native click @n10

See RefStore.swift for ref storage implementation.

Element not found

Error message:

Element not found: AXButton[@title="Submit"]

Cause: The element doesn’t exist, or is hidden/disabled.

Debugging steps:

1. Check if element exists

   # Full snapshot (no filters)
   agent-native snapshot App > full-tree.txt
   
   # Search for your element
   grep -i "submit" full-tree.txt
   
2. Check interactive-only flag

   The -i flag filters to interactive elements. Try without it:

   agent-native snapshot App  # Full tree
   
3. Increase depth

   Default depth is 8. Some deep elements need more:

   agent-native snapshot App -i -d 15
   
4. Wait for element to appear

   Elements may load asynchronously:

   agent-native wait App --title "Submit" --timeout 5
   
5. Use find command

   # Search by title
   agent-native find App --title "Submit"
   
   # Search by role
   agent-native find App --role AXButton
   
   # Combine filters
   agent-native find App --title "Submit" --role AXButton
   

See FindCommand.swift and AXEngine.swift:278-356 for element finding logic.

Element not responding

Error message:

Action 'AXPress' failed on: /AXWindow/AXButton

Cause: Element is disabled, hidden, or doesn’t support the action.

Solutions:

Check if element is enabled

agent-native is enabled @n10

If this returns false, the element can’t be interacted with.

Inspect element attributes

agent-native inspect @n10

Check:

• AXEnabled: Should be true
• Actions: Should include AXPress or relevant action
• AXPosition: Element should be on-screen

Try alternative action

# Instead of click
agent-native action @n10 AXPress

# Or focus then press space
agent-native focus @n10
agent-native key App space

Wait for element to be ready

# Wait 1 second
sleep 1
agent-native click @n10

See InspectCommand.swift for element inspection.

Timeout waiting for element

Error message:

Timeout: Element with title="Submit" not found after 5s

Cause: Element never appeared within the timeout period.

Solutions:

Increase timeout

agent-native wait App --title "Submit" --timeout 10

Check if element exists

Take a snapshot to see what’s actually there:

agent-native snapshot App -i

Verify filter criteria

Your filters might be too specific:

# Try less specific filter
agent-native wait App --role AXButton --timeout 5

Check page/pane loaded correctly

# Check window title
agent-native get title App

# Take screenshot
agent-native screenshot App /tmp/debug.png

See WaitCommand.swift for wait implementation.

Debugging techniques

Capture full snapshots

When debugging, capture the full tree without filters:

# Save to file
agent-native snapshot App > /tmp/full-tree.txt

# Search for your element
grep -i "keyword" /tmp/full-tree.txt

# View in editor
code /tmp/full-tree.txt

Use JSON output for parsing

# Get JSON snapshot
agent-native snapshot App -i --json > snapshot.json

# Query with jq
cat snapshot.json | jq '.[] | select(.title == "Submit")'

# Extract specific ref
REF=$(cat snapshot.json | jq -r '.[] | select(.title == "Submit") | .ref')
agent-native click "@$REF"

Compare snapshots

See how UI changes between actions:

# Before action
agent-native snapshot App -i > /tmp/before.txt

# Perform action
agent-native click @n5
sleep 1

# After action
agent-native snapshot App -i > /tmp/after.txt

# Compare
diff /tmp/before.txt /tmp/after.txt

Use screenshots for visual debugging

# Capture current state
agent-native screenshot App /tmp/state1.png

# Perform actions
agent-native click @n5

# Capture after state
agent-native screenshot App /tmp/state2.png

# Compare visually
open /tmp/state1.png /tmp/state2.png

Inspect element details

# Full element inspection
agent-native inspect @n10

# Or by filter
agent-native inspect App --title "Submit"

Output includes:

• Role and subrole
• All attributes (title, label, value, etc.)
• Available actions
• Position and size
• Enabled/focused state

Test commands incrementally

Build complex workflows step by step:

# Step 1: Open app
agent-native open App
echo "Opened app"
sleep 1

# Step 2: Snapshot
agent-native snapshot App -i > /tmp/snap.txt
echo "Snapshot captured"

# Step 3: Find element
REF=$(grep -i "submit" /tmp/snap.txt | grep -o 'ref=n[0-9]*' | sed 's/ref=//' | head -1)
echo "Found ref: @$REF"

# Step 4: Click
if [[ -n "$REF" ]]; then
  agent-native click "@$REF"
  echo "Clicked"
else
  echo "Element not found"
  exit 1
fi

Enable verbose output

While agent-native doesn’t have a verbose flag, you can wrap commands in debug scripts:

#!/usr/bin/env bash
set -x  # Print commands as they execute
set -euo pipefail  # Exit on error

agent-native open App
agent-native snapshot App -i
agent-native click @n5

Performance issues

Snapshots are slow

Cause: Deep trees with many elements take time to traverse.

Solutions:

Limit depth

agent-native snapshot App -i -d 5

Use compact flag

Remove empty structural elements:

agent-native snapshot App -i -c

Use interactive-only flag

Always use -i to filter to interactive elements:

agent-native snapshot App -i

Use filter-based commands

Skip snapshots when you know what you’re looking for:

agent-native click App --title "Submit"

See SnapshotCommand.swift:26-33 for interactive role filtering.

App becomes unresponsive

Cause: Too many rapid commands can overwhelm the app.

Solution:

Add delays between commands:

agent-native click @n5
sleep 0.5
agent-native fill @n10 "text"
sleep 0.3
agent-native click @n15

Platform-specific issues

macOS version compatibility

Issue: Some features require macOS 13+.

Check version:

sw_vers -productVersion

Solution: Upgrade to macOS 13 (Ventura) or higher.

Accessibility API changes

Apple occasionally changes Accessibility APIs between macOS versions.

Solution:

1. Update agent-native to the latest version:
   brew upgrade agent-native
   
2. Check for known issues on GitHub

Getting help

Check GitHub Issues

Search for similar problems and solutions

Run tests

Verify your installation:

cd /path/to/agent-native
make test-quick

Check accessibility

Use macOS Accessibility Inspector:

• Open Xcode
• Xcode > Open Developer Tool > Accessibility Inspector
• Inspect app elements manually

File a bug

Include:

• agent-native version (agent-native --version)
• macOS version (sw_vers)
• Full error message
• Minimal reproduction steps

Quick reference

Essential debugging commands

# List running apps
agent-native apps

# Full tree snapshot
agent-native snapshot App > debug.txt

# Interactive elements only
agent-native snapshot App -i

# JSON output
agent-native snapshot App -i --json

# Find elements
agent-native find App --title "Submit"

# Inspect element
agent-native inspect @n10

# Check element state
agent-native is enabled @n10
agent-native is focused @n10

# Get element attributes
agent-native get text @n10
agent-native get value @n10

# Screenshot
agent-native screenshot App /tmp/debug.png

# Window title
agent-native get title App

Common error patterns

Error	Likely Cause	Quick Fix
”access denied”	No Accessibility permission	Add terminal to System Settings > Accessibility
”App not found”	Typo or not running	Check agent-native apps
”Ref @nX not found”	UI changed	Re-snapshot
”Element not found”	Wrong filter or not loaded	Try snapshot without -i, or increase -d
”Action failed”	Element disabled	Check is enabled, wait longer
”Timeout”	Element never appeared	Increase --timeout, verify filter

Next steps

System Settings guide

Learn System Settings automation patterns

Electron apps guide

Automate Slack, Discord, VS Code

Safari automation

Interact with web content

API reference

Complete command reference