Automating System Settings

Complete walkthrough for navigating and configuring macOS System Settings

System Settings (formerly System Preferences) is a native macOS app with a rich Accessibility tree that works well with agent-native.

Overview

System Settings exposes excellent Accessibility support, making it ideal for automation. Common use cases include:

• Configuring Wi-Fi networks
• Adjusting display settings
• Managing accessibility permissions
• Toggling system features
• Checking system information

Core workflow

The standard workflow for automating System Settings:

1. Open System Settings

   agent-native open "System Settings"
   

   See AXEngine.swift:65-119 for the app launching implementation.

2. Snapshot to find elements

   agent-native snapshot "System Settings" -i
   

   The -i flag shows only interactive elements (buttons, checkboxes, text fields). This keeps the output focused and manageable.

   See SnapshotCommand.swift:14-19 for the snapshot flags.

3. Interact using refs

   agent-native click @n5
   

   Every element in the snapshot has a ref (like @n5) that you can use for interactions.

4. Re-snapshot after navigation

   agent-native snapshot "System Settings" -i
   

   Important: Always re-snapshot after clicking navigation elements. The UI structure changes, and old refs become invalid.

Refs are invalidated when the UI structure changes. Always re-snapshot after navigation actions.

Example: Configuring Wi-Fi

Here’s a complete example of connecting to a Wi-Fi network:

#!/usr/bin/env bash
set -euo pipefail

# Open System Settings
agent-native open "System Settings"
sleep 1

# Find and click Wi-Fi in sidebar
agent-native snapshot "System Settings" -i > /tmp/snapshot.txt
WIFI_REF=$(grep -i "Wi-Fi" /tmp/snapshot.txt | grep -o 'ref=n[0-9]*' | sed 's/ref=//' | head -1)

if [[ -n "$WIFI_REF" ]]; then
  agent-native click "@$WIFI_REF"
  sleep 1
else
  echo "Wi-Fi not found in sidebar"
  exit 1
fi

# Re-snapshot to see Wi-Fi pane
agent-native snapshot "System Settings" -i > /tmp/wifi-pane.txt

# Find and click network name
NETWORK_NAME="MyWiFiNetwork"
NETWORK_REF=$(grep "$NETWORK_NAME" /tmp/wifi-pane.txt | grep -o 'ref=n[0-9]*' | sed 's/ref=//' | head -1)

if [[ -n "$NETWORK_REF" ]]; then
  agent-native click "@$NETWORK_REF"
  sleep 1
else
  echo "Network $NETWORK_NAME not found"
  exit 1
fi

# Enter password
agent-native snapshot "System Settings" -i > /tmp/password-dialog.txt
PASSWORD_REF=$(grep -i "Password" /tmp/password-dialog.txt | grep "AXTextField" | grep -o 'ref=n[0-9]*' | sed 's/ref=//' | head -1)

if [[ -n "$PASSWORD_REF" ]]; then
  agent-native fill "@$PASSWORD_REF" "MySecurePassword123"
  sleep 0.5
else
  echo "Password field not found"
  exit 1
fi

# Click Join button
JOIN_REF=$(grep -i "Join" /tmp/password-dialog.txt | grep "AXButton" | grep -o 'ref=n[0-9]*' | sed 's/ref=//' | head -1)

if [[ -n "$JOIN_REF" ]]; then
  agent-native click "@$JOIN_REF"
  echo "Successfully connected to $NETWORK_NAME"
else
  echo "Join button not found"
  exit 1
fi

See test/integration.sh:1-286 for more examples of automated testing patterns.

Common patterns

Finding sidebar items

System Settings uses a sidebar for navigation. Items are typically AXStaticText or AXButton elements:

agent-native snapshot "System Settings" -i | grep -i "Wi-Fi\|Bluetooth\|Network"

AXStaticText "Wi-Fi" [ref=n12]
AXStaticText "Bluetooth" [ref=n13]
AXStaticText "Network" [ref=n14]

Using search

System Settings has a search field that can quickly navigate to settings:

# Find the search field
agent-native snapshot "System Settings" -i > /tmp/snap.txt
SEARCH_REF=$(grep -i "Search" /tmp/snap.txt | grep "AXTextField" | grep -o 'ref=n[0-9]*' | sed 's/ref=//' | head -1)

# Type search query
agent-native fill "@$SEARCH_REF" "Display"
sleep 0.5

# Press return to navigate
agent-native key "System Settings" return

Toggling checkboxes

Use check and uncheck commands, which are idempotent:

# Find checkbox
CHECKBOX_REF=$(agent-native snapshot "System Settings" -i | \
  grep -i "FileVault" | grep "AXCheckBox" | \
  grep -o 'ref=n[0-9]*' | sed 's/ref=//' | head -1)

# Check it (won't fail if already checked)
agent-native check "@$CHECKBOX_REF"

# Or uncheck it
agent-native uncheck "@$CHECKBOX_REF"

See InteractionCommands.swift for implementation details.

Reading current values

Use the get command to read element states:

# Get current window title
agent-native get title "System Settings"

# Get text from an element
agent-native get text @n42

# Get input value
agent-native get value @n15

# Check if element is enabled
agent-native is enabled @n20

See GetCommand.swift and IsCommand.swift for details.

Example: Adjusting display settings

#!/usr/bin/env bash
set -euo pipefail

# Open System Settings and navigate to Displays
agent-native open "System Settings"
sleep 1

# Click Displays in sidebar
agent-native snapshot "System Settings" -i > /tmp/snap.txt
DISPLAY_REF=$(grep -i "Displays" /tmp/snap.txt | \
  grep -o 'ref=n[0-9]*' | sed 's/ref=//' | head -1)

agent-native click "@$DISPLAY_REF"
sleep 1

# Re-snapshot to see Display settings
agent-native snapshot "System Settings" -i > /tmp/display.txt

# Find and adjust brightness slider (example)
BRIGHTNESS_REF=$(grep -i "Brightness" /tmp/display.txt | \
  grep "AXSlider" | grep -o 'ref=n[0-9]*' | sed 's/ref=//' | head -1)

if [[ -n "$BRIGHTNESS_REF" ]]; then
  # Get current value
  current=$(agent-native get value "@$BRIGHTNESS_REF")
  echo "Current brightness: $current"
  
  # Set new value (0.0 to 1.0)
  agent-native fill "@$BRIGHTNESS_REF" "0.75"
  echo "Set brightness to 75%"
fi

# Find Night Shift checkbox
NIGHT_SHIFT_REF=$(grep -i "Night Shift" /tmp/display.txt | \
  grep "AXCheckBox" | grep -o 'ref=n[0-9]*' | sed 's/ref=//' | head -1)

if [[ -n "$NIGHT_SHIFT_REF" ]]; then
  agent-native check "@$NIGHT_SHIFT_REF"
  echo "Enabled Night Shift"
fi

Example: Checking system information

#!/usr/bin/env bash

# Open System Settings
agent-native open "System Settings"
sleep 1

# Navigate to General > About
agent-native snapshot "System Settings" -i | grep -i "General"
# Click General, then About...

# Snapshot the About pane
agent-native snapshot "System Settings" -i > /tmp/about.txt

# Extract information
grep -i "macOS\|Version\|Chip\|Memory" /tmp/about.txt

Using JSON output

For programmatic access, use --json flag:

agent-native snapshot "System Settings" -i --json > snapshot.json

[
  {
    "ref": "n1",
    "role": "AXButton",
    "title": "Wi-Fi",
    "label": null,
    "value": null,
    "enabled": true,
    "actions": ["AXPress"],
    "depth": 2
  },
  {
    "ref": "n2",
    "role": "AXCheckBox",
    "title": "Show Wi-Fi status in menu bar",
    "label": null,
    "value": "1",
    "enabled": true,
    "actions": ["AXPress"],
    "depth": 3
  }
]

Parse with jq:

# Find Wi-Fi button ref
WIFI_REF=$(agent-native snapshot "System Settings" -i --json | \
  jq -r '.[] | select(.title == "Wi-Fi") | .ref')

echo "Wi-Fi ref: @$WIFI_REF"

# Click it
agent-native click "@$WIFI_REF"

See SnapshotCommand.swift:105-111 for JSON encoding implementation.

Filter-based commands

You can interact with elements by filter instead of refs:

# Click by title
agent-native click "System Settings" --title "Wi-Fi"

# Fill by label
agent-native fill "System Settings" "MyNetwork" --label "Network Name"

# Check by role and title
agent-native check "System Settings" --title "FileVault" --role AXCheckBox

This is useful for one-off commands, but refs are more reliable for multi-step workflows.

See ElementResolver.swift:6-51 for filter resolution logic.

Best practices

Always use -i flag

The -i flag filters to interactive elements only, making snapshots much more readable:

agent-native snapshot "System Settings" -i

Re-snapshot after navigation

UI changes invalidate old refs. Always re-snapshot after clicking:

agent-native click @n5
agent-native snapshot "System Settings" -i

Add delays

System Settings animations take time. Add small delays:

agent-native click @n5
sleep 1
agent-native snapshot "System Settings" -i

Use search for speed

The search field is the fastest way to navigate:

agent-native fill @n1 "Display"
agent-native key "System Settings" return

Common settings to automate

Wi-Fi

• Connect/disconnect networks
• View available networks
• Configure network settings
• Toggle Wi-Fi on/off

Display

• Adjust brightness
• Change resolution
• Enable Night Shift
• Arrange displays

Sound

• Adjust volume
• Select input/output devices
• Configure alert sounds

Privacy & Security

• Grant accessibility permissions
• Manage app permissions
• Configure FileVault

Network

• Configure firewall
• Manage VPN connections
• View network status

Troubleshooting

Sidebar item not found

Problem: Can’t find the setting in the sidebar.

Solutions:

1. Use the search field instead:
   agent-native fill @n1 "Wi-Fi"
   agent-native key "System Settings" return
   
2. Check if you need to expand a group first (e.g., General > About)
3. Try searching with different keywords

Ref not resolving

Problem: @n5 doesn’t resolve after navigation.

Cause: Old refs are invalid after UI changes.

Solution: Re-snapshot after each navigation:

agent-native click @n5
sleep 1
agent-native snapshot "System Settings" -i  # Get new refs

Permission dialogs

Problem: System Settings shows permission dialogs.

Solutions:

1. Use wait to wait for dialogs:
   agent-native wait "System Settings" --role AXSheet --timeout 5
   
2. Snapshot the dialog and interact with it:
   agent-native snapshot "System Settings" -i
   # Find "Allow" or "OK" button
   agent-native click @n10
   

Elements not found in snapshot

Problem: Expected element doesn’t appear in snapshot.

Solutions:

1. Remove -i flag to see full tree:
   agent-native snapshot "System Settings"
   
2. Increase depth with -d flag:
   agent-native snapshot "System Settings" -i -d 10
   
3. Use find to search:
   agent-native find "System Settings" --title "Wi-Fi"
   

Next steps

Electron apps

Learn to automate Slack, Discord, and VS Code

Safari automation

Interact with web content in Safari

API reference

Explore all available commands

Troubleshooting

Fix common issues and errors