> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/ericclemmons/agent-native/llms.txt
> Use this file to discover all available pages before exploring further.

# Working with Electron apps

> Automate Electron apps like Slack, Discord, and VS Code using keyboard shortcuts and screenshots

<Info>
  **Electron apps** include Slack, Discord, VS Code, Notion, Figma, and many other popular desktop applications.
</Info>

## Why Electron apps are different

Electron apps expose a minimal Accessibility tree compared to native macOS apps. Inner UI elements often appear as opaque `AXGroup` elements with no labels or structure, making traditional AX-based automation challenging.

When you run `snapshot -i` on an Electron app, you may see very few interactive elements:

```bash theme={null}
agent-native snapshot Slack -i
```

```text theme={null}
Snapshot: Slack (pid 1234) -- 3 elements
---------------------------------------------
AXWindow "Slack" [ref=n1]
  AXGroup [ref=n2]
    AXGroup [ref=n3]
```

This minimal tree structure means you need alternative automation strategies.

## Automation strategies

### Use keyboard shortcuts

Most Electron apps have rich keyboard support. This is the **primary method** for automating Electron apps.

<Steps>
  <Step title="Identify available shortcuts">
    Check the app's menu bar or documentation for keyboard shortcuts. Common patterns:

    * **Cmd+K**: Quick switcher (Slack, Discord, VS Code)
    * **Cmd+N**: New window/message
    * **Cmd+T**: New tab
    * **Cmd+W**: Close window/tab
    * **Cmd+F**: Search
  </Step>

  <Step title="Send keystrokes with agent-native">
    Use the `key` command to send keyboard shortcuts:

    ```bash theme={null}
    agent-native key Slack cmd+k                    # Open quick switcher
    agent-native key Slack "#general" return        # Navigate to channel
    agent-native key Slack "Hello team!" return     # Send message
    ```

    See KeyCommand.swift:36-44 for implementation details.
  </Step>

  <Step title="Chain commands for complex workflows">
    Combine multiple keystrokes to accomplish tasks:

    ```bash theme={null}
    agent-native key Slack cmd+k                    # Open switcher
    agent-native key Slack "@john" return           # Find user
    agent-native key Slack "Meeting at 3pm" return  # Send DM
    ```
  </Step>
</Steps>

### Use screenshots for visual context

When the AX tree doesn't provide enough information, capture screenshots to understand what's on screen:

```bash theme={null}
agent-native screenshot Slack /tmp/slack.png
```

The screenshot command captures the app's frontmost window (see ScreenshotCommand.swift:29-42):

```bash theme={null}
# With custom path
agent-native screenshot Slack ~/Desktop/slack-state.png

# Default auto-generated path
agent-native screenshot Slack
# Output: /tmp/agent-native-abc123.png

# JSON output for parsing
agent-native screenshot Slack --json
# {"path": "/tmp/...", "width": 1920, "height": 1080}
```

<Tip>
  Use screenshots to verify navigation state before and after keyboard actions.
</Tip>

### Check window titles

The window title often reflects the current navigation state:

```bash theme={null}
agent-native get title Slack
```

```text theme={null}
Chad Donohue (DM) - Ryan Florence Fan Club - Slack
```

This is useful for:

* Confirming successful navigation
* Determining which channel or conversation is active
* Verifying modal states

```bash theme={null}
# Navigate and verify
agent-native key Slack cmd+k
agent-native key Slack "#engineering" return
title=$(agent-native get title Slack)
if [[ "$title" == *"#engineering"* ]]; then
  echo "Successfully navigated to #engineering"
fi
```

### Paste files

For file uploads, use the `paste` command to copy a file to the clipboard and paste it:

```bash theme={null}
agent-native paste Slack /path/to/screenshot.png
```

This is equivalent to:

1. Copying the file to the clipboard
2. Sending Cmd+V to the app

See SKILL.md:60-67 for the paste command details.

## Example: Automating Slack

Here's a complete workflow for sending a message with an image to a Slack channel:

<Steps>
  <Step title="Open and activate Slack">
    ```bash theme={null}
    agent-native open Slack
    ```
  </Step>

  <Step title="Navigate to the channel">
    ```bash theme={null}
    agent-native key Slack cmd+k              # Quick switcher
    agent-native key Slack "#general" return  # Select channel
    ```
  </Step>

  <Step title="Verify navigation">
    ```bash theme={null}
    title=$(agent-native get title Slack)
    echo "Current view: $title"
    ```
  </Step>

  <Step title="Type message">
    ```bash theme={null}
    agent-native key Slack "Check out this screenshot:"
    ```
  </Step>

  <Step title="Paste image">
    ```bash theme={null}
    agent-native paste Slack /tmp/report.png
    ```
  </Step>

  <Step title="Send message">
    ```bash theme={null}
    agent-native key Slack return
    ```
  </Step>
</Steps>

## Common keyboard shortcuts

### Slack

| Shortcut      | Action          |
| ------------- | --------------- |
| `cmd+k`       | Quick switcher  |
| `cmd+u`       | Upload file     |
| `cmd+n`       | New message     |
| `cmd+f`       | Search          |
| `cmd+/`       | Show shortcuts  |
| `cmd+shift+k` | Direct messages |
| `cmd+shift+a` | All unreads     |

### VS Code

| Shortcut      | Action             |
| ------------- | ------------------ |
| `cmd+p`       | Quick open         |
| `cmd+shift+p` | Command palette    |
| `cmd+b`       | Toggle sidebar     |
| `cmd+j`       | Toggle terminal    |
| `cmd+k cmd+s` | Keyboard shortcuts |

### Discord

| Shortcut      | Action             |
| ------------- | ------------------ |
| `cmd+k`       | Quick switcher     |
| `cmd+n`       | Create/join server |
| `cmd+shift+t` | Create thread      |
| `cmd+i`       | Toggle inbox       |

## Best practices

<Warning>
  **Always re-snapshot or verify state** after navigation actions. Electron app states change frequently, and keyboard shortcuts may have unexpected effects.
</Warning>

<CardGroup cols={2}>
  <Card title="Use delays between actions" icon="clock">
    Add small delays between rapid keystrokes to ensure the app processes each action:

    ```bash theme={null}
    agent-native key Slack cmd+k
    sleep 0.5
    agent-native key Slack "#general" return
    ```
  </Card>

  <Card title="Verify with screenshots" icon="camera">
    Capture screenshots before and after actions to debug automation failures:

    ```bash theme={null}
    agent-native screenshot Slack /tmp/before.png
    agent-native key Slack cmd+k
    agent-native screenshot Slack /tmp/after.png
    ```
  </Card>

  <Card title="Check window titles" icon="window">
    Use window titles to confirm navigation:

    ```bash theme={null}
    agent-native get title Slack
    ```
  </Card>

  <Card title="Test shortcuts manually" icon="keyboard">
    Verify keyboard shortcuts work manually before automating them.
  </Card>
</CardGroup>

## Troubleshooting

<AccordionGroup>
  <Accordion title="Keyboard shortcuts don't work">
    **Possible causes:**

    * App is not focused/activated
    * Shortcut requires different modifiers on your system
    * App has custom keybindings

    **Solutions:**

    1. Ensure the app is activated:
       ```bash theme={null}
       agent-native open Slack  # Activates the app
       ```
    2. Check the app's keyboard shortcuts in Preferences
    3. Try alternative shortcuts (e.g., `cmd+shift+k` instead of `cmd+k`)
  </Accordion>

  <Accordion title="Text not typing correctly">
    **Possible causes:**

    * Special characters require escaping
    * App processes input asynchronously

    **Solutions:**

    1. Add delays between keystrokes:
       ```bash theme={null}
       agent-native key Slack "Hello"
       sleep 0.2
       agent-native key Slack "world"
       ```
    2. Use quotes for text with spaces or special characters:
       ```bash theme={null}
       agent-native key Slack "Message with spaces"
       ```
  </Accordion>

  <Accordion title="Can't verify navigation state">
    **Solutions:**

    1. Use window titles:
       ```bash theme={null}
       agent-native get title Slack
       ```
    2. Use screenshots:
       ```bash theme={null}
       agent-native screenshot Slack /tmp/verify.png
       ```
    3. Add delays to allow UI to update:
       ```bash theme={null}
       agent-native key Slack cmd+k
       sleep 0.5
       agent-native get title Slack
       ```
  </Accordion>
</AccordionGroup>

## Next steps

<CardGroup cols={2}>
  <Card title="System Settings" icon="gear" href="/guides/system-settings">
    Automate System Settings and native macOS apps
  </Card>

  <Card title="Safari automation" icon="browser" href="/guides/safari-automation">
    Interact with web content in Safari
  </Card>
</CardGroup>