> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/ericclemmons/agent-native/llms.txt
> Use this file to discover all available pages before exploring further.
# Refs and snapshots
> Understanding the ref system that makes element interactions possible
Refs are how you reference UI elements across commands. A snapshot assigns refs like `@n1`, `@n2`, `@n3` to elements in the Accessibility tree. You then use these refs in subsequent commands to click, type, or inspect specific elements.
## What are refs?
A **ref** (reference) is a short identifier like `@n1`, `@n2`, etc. that points to a specific UI element. Think of refs as temporary "handles" for elements discovered during a snapshot.
```bash theme={null}
# Take a snapshot
$ agent-native snapshot Safari --interactive
Snapshot: Safari (pid 1234) -- 12 elements
---------------------------------------------
AXWindow "Safari" [ref=n1]
AXButton "Back" [AXPress] [ref=n2]
AXButton "Forward" [AXPress] [ref=n3]
AXTextField [ref=n4]
AXButton "Go" [AXPress] [ref=n5]
# Click using a ref
$ agent-native click @n2
OK Clicked: AXButton title="Back"
```
Refs are numbered sequentially starting from `n1` in each snapshot. The `@` prefix distinguishes refs from app names.
## How snapshots work
The `snapshot` command walks the Accessibility tree and assigns refs to elements. From `SnapshotCommand.swift:5-133`:
Traverse the app's Accessibility tree to the specified depth:
```swift theme={null}
let element = AXEngine.appElement(pid: pid)
let tree = AXEngine.walkTree(element, maxDepth: depth)
```
Optionally filter to only interactive elements with `--interactive`, or use `--compact` to skip empty structural nodes:
```swift theme={null}
// SnapshotCommand.swift:26-33
static let interactiveRoles: Set = [
"AXButton", "AXTextField", "AXCheckBox",
"AXRadioButton", "AXPopUpButton", "AXLink",
"AXMenuItem", "AXTab", "AXSearchField", ...
]
```
Number each element sequentially:
```swift theme={null}
var refCounter = 0
for (node, depth) in tree {
refCounter += 1
let ref = "n\(refCounter)"
// Store for later resolution
}
```
Persist ref → element mappings to a temp JSON file:
```swift theme={null}
RefStore.save(refEntries)
```
## The RefStore
Refs are persisted to disk at `/tmp/agent-native-refs.json` so they survive between CLI invocations. Each command runs as a separate process, so refs must be saved externally.
From `RefStore.swift:4-35`:
```swift theme={null}
struct RefEntry: Codable {
let ref: String // "n1", "n2", etc.
let pid: Int32 // Process ID of the app
let role: String // AXButton, AXTextField, etc.
let title: String? // Button text, window title
let label: String? // Accessibility label
let identifier: String? // Programmatic identifier
let pathHint: String // XPath-like path in the tree
}
private static let storePath: URL = {
let tmp = FileManager.default.temporaryDirectory
return tmp.appendingPathComponent("agent-native-refs.json")
}()
static func save(_ entries: [RefEntry]) { ... }
static func load() -> [RefEntry] { ... }
```
The RefStore contains only the **metadata** needed to re-find elements, not live `AXUIElement` pointers (which can't be serialized).
## Ref resolution
When you use a ref like `@n5`, agent-native must **resolve** it back to a live `AXUIElement`. This happens by re-searching the tree using the stored attributes.
From `RefStore.swift:37-74`:
```swift theme={null}
static func resolve(ref: String) throws -> (element: AXUIElement, node: AXNode) {
let entries = load()
guard let entry = entries.first(where: { $0.ref == cleanRef }) else {
throw AXError.elementNotFound("Unknown ref: @\(cleanRef). Run `snapshot` first.")
}
let appElement = AXEngine.appElement(pid: entry.pid)
let results = AXEngine.findElements(
root: appElement,
role: entry.role,
title: entry.title,
label: entry.label,
identifier: entry.identifier,
maxDepth: 15
)
// Return first exact match
if let match = results.first(where: { /* exact attribute match */ }) {
return match
}
throw AXError.elementNotFound(
"Could not re-resolve @\(cleanRef). The UI may have changed -- run `snapshot` again.")
}
```
Read the entry from `agent-native-refs.json`.
Use `AXEngine.findElements()` to search for elements matching the stored `role`, `title`, `label`, and `identifier`.
Return the element that exactly matches all stored attributes.
If the element no longer exists or has changed, throw `elementNotFound`.
## Ref lifecycle and invalidation
Refs become invalid when:
1. **The UI changes** - If a button is removed or its text changes, the ref can't be resolved
2. **The app restarts** - PIDs change, invalidating all refs for that app
3. **A new snapshot is taken** - Each `snapshot` command overwrites the entire RefStore
Refs are **ephemeral**. They're valid for the current UI state only. If your interaction changes the UI significantly (e.g., navigating to a new screen), take a fresh snapshot.
### When refs fail
```bash theme={null}
$ agent-native click @n8
Error: Could not re-resolve @n8. The UI may have changed -- run `snapshot` again.
```
This means the element with ref `@n8` no longer exists or has different attributes. Solution: take a new snapshot.
## Interactive vs full snapshots
You can control what gets refs:
### Interactive snapshots
Use `--interactive` to only assign refs to elements you can interact with:
```bash theme={null}
$ agent-native snapshot Safari --interactive
```
This filters to `interactiveRoles` (buttons, text fields, checkboxes, links, etc.) and skips structural elements like groups and static text.
Use `--interactive` for agent workflows. It reduces noise and makes snapshots easier to parse.
### Compact snapshots
Use `--compact` to remove empty structural elements:
```bash theme={null}
$ agent-native snapshot Safari --compact
```
From `SnapshotCommand.swift:69-74`, this skips elements with:
* No `title`, `label`, or `value`
* No `actions`
* Has children (i.e., is just a container)
### Full snapshots
By default, every element gets a ref. This is useful for debugging or when you need to inspect structural elements:
```bash theme={null}
$ agent-native snapshot Safari
```
## Example workflow
```bash theme={null}
$ agent-native snapshot Safari --interactive
Snapshot: Safari (pid 1234) -- 8 elements
AXWindow "Safari" [ref=n1]
AXTextField [ref=n2]
AXButton "Go" [AXPress] [ref=n3]
```
```bash theme={null}
$ agent-native click @n2
OK Clicked: AXTextField
$ agent-native type @n2 "example.com"
OK Typed 11 chars into AXTextField
$ agent-native click @n3
OK Clicked: AXButton title="Go"
```
After the page loads, the UI is different. Take a new snapshot:
```bash theme={null}
$ agent-native snapshot Safari --interactive
Snapshot: Safari (pid 1234) -- 15 elements
# Now @n1, @n2, etc. refer to the NEW UI state
```
## See also
Understanding the tree that refs point into
The snapshot → interact → re-snapshot pattern