# Evidence

Canonical URL: https://autonom.yatsuk.me/docs/evidence/

Bring back proof, not a claim: screenshots that carry their own provenance, logs narrowed to your app and a report of every step.

**Try it**

```sh
autonom screenshot
```

**Needs** [a session on one target](https://autonom.yatsuk.me/docs/sessions/index.md)

## How it works

Autonom climbs an evidence ladder: code, then unit and widget tests, then an integration run on one target, then screenshot, tree and logs, then profile, memory and network, and last a before and after replay. Climb it; do not skip a rung.

![Six rungs rising left to right: code; unit and widget tests; integration on a target; screenshot, tree and logs; profile, memory and network; before and after replay.](https://autonom.yatsuk.me/docs/evidence/ladder-wide.svg)

Each rung is stronger proof than the one below it.

Every capture lands in the session’s folder, and every command goes into its [journal](https://autonom.yatsuk.me/docs/sessions/index.md), failures included. A screenshot carries its own provenance: the session, target, app, label and the number of active [mock rules](https://autonom.yatsuk.me/docs/network/index.md) are written into the PNG and into an index.

Two captures of the same screen can still differ by the battery glyph, the signal bars, a half-finished animation or an autocorrected word. Pins fix what the app does not own, so a before and after diff shows only what the app changed. The clock stays real unless you pin it on purpose.

## Examples

Every output below comes from Autonom’s test device, so you can run the same commands without an emulator. Open Reproduce these outputs at the end for the exact script.

### Capture a screen that never lands mid-animation

**Goal:** take a screenshot you can compare with tomorrow’s.

1. **Pin the status bar.** Full battery and signal, no notification icons, the real clock.

   ```sh
   autonom simulator status-bar pin
   ```

2. **Turn system animations off.** This is Android only; iOS has no switch for it.

   ```sh
   autonom simulator animations pin
   ```

3. **Wait for a still screen.** Two trees in a row must match.

   ```sh
   autonom ui wait --settled --timeout-ms 5000
   ```

4. **Take the screenshot.** Take it only once the wait reports `settled: true`.

   ```sh
   autonom screenshot
   ```

5. **Turn animations back on.** Each undo restores what the pin replaced.

   ```sh
   autonom simulator animations reset
   ```

6. **Unpin the status bar.** The device shows its real state again.

   ```sh
   autonom simulator status-bar clear
   ```

You should see the pin confirmed by reading the battery back, with a warning about the signal bars.

*Output of autonom simulator status-bar pin*

```
{
  "ok": true,
  …
  "verified_keys": [
    "battery"
  ],
  "verified": true,
  "verification": "read_back",
  "warnings": [
    {
      "code": "signal_unstable",
      …
```

**What to look at.** `verified` rests on a read-back, and `verified_keys` names what was read: the battery. The signal bars cannot be read back from the Mac, so they carry [`signal_unstable`](https://autonom.yatsuk.me/docs/safety/index.md#signal_unstable) instead of a claim.

**Show full output**

```
{
  "ok": true,
  "control": "status-bar",
  "action": "pin",
  "values": {
    "mode": "live",
    "battery": 100,
    "plugged": "false",
    "mobile_level": 4,
    "notifications": "hidden"
  },
  "observed_battery": 100,
  "verified_keys": [
    "battery"
  ],
  "verified": true,
  "verification": "read_back",
  "warnings": [
    {
      "code": "signal_unstable",
      "error": "the emulator's modem may re-report its own signal strength; the cellular bars are pinned but cannot be read back from the host",
      "hint": "The bars and battery are re-asserted automatically right before each 'autonom screenshot' and flow capture; pass hhmm=<HHMM> (demo mode, fixed clock) to hide the mobile icon instead."
    }
  ],
  "platform": "android",
  "target_id": "emulator-5554",
  "serial": "emulator-5554"
}
```

### Keep a mocked screenshot out of the proof

**Goal:** tell a capture of the real server from one taken while a mock rule could shape the screen.

1. **Take a labeled capture.** The label goes into the PNG with the session and target.

   ```sh
   autonom screenshot --label "after login"
   ```

2. **Add a mock rule.** It makes the login request fail, as in [Force a backend failure](https://autonom.yatsuk.me/docs/network/index.md#force-a-backend-failure).

   ```sh
   autonom network mock add --url '.../v1/login' --status 500 --json '{"error":"x"}'
   ```

3. **Capture again.** Group it under a task, so you can find it later.

   ```sh
   autonom screenshot --task login --label "500 error state"
   ```

4. **Switch the rule off.** Rules persist, so do not leave one on by accident.

   ```sh
   autonom network mock disable --all
   ```

You should see one active rule and a [`screenshot_shows_mocked_data`](https://autonom.yatsuk.me/docs/safety/index.md#screenshot_shows_mocked_data) warning on the second capture.

*Output of autonom screenshot --task login*

```
{
  …
  "metadata": {
    …
    "mocks_active": 1,
    "mocks": "m_1"
  },
  …
  "warnings": [
    {
      "code": "screenshot_shows_mocked_data",
      …
```

**What to look at.** The second capture says a rule was active, and the same fact is written into the PNG and the index. `shots list --mocked-only` finds it later, so it never passes for the real server’s behavior.

### Read a failed run as a report

**Goal:** turn a failed flow run into a report you can read and hand off.

1. **Run the flow.** A [flow](https://autonom.yatsuk.me/docs/flows/index.md) keeps evidence for every step it runs; name your own flow file or folder in place of this one.

   ```sh
   autonom flow run flows/sign-in.yaml
   ```

2. **Build the report.** One command writes the HTML report, a JUnit file and a bundle you can hand off.

   ```sh
   autonom report build
   ```

You should see `status: failed` and where the HTML report, the JUnit file and the bundle were written.

*Output of autonom report build*

```
{
  "ok": true,
  "run_id": "fr_74c1d92ee0",
  "status": "failed",
  "html": "/tmp/autonom-evidence/.autonom/sessions/s_004f33b833/flows/fr_74c1d92ee0/report.html",
  "junit": "/tmp/autonom-evidence/.autonom/sessions/s_004f33b833/flows/fr_74c1d92ee0/report.xml",
  "bundle": "/tmp/autonom-evidence/.autonom/sessions/s_004f33b833/flows/fr_74c1d92ee0/bundle-v2",
  …
}
```

![The top of an HTML report: Sign in with email failed, primary failure flow_assertion_timeout, first causal failure at step 4, and a timeline with three passed steps and a failed assertVisible.](https://autonom.yatsuk.me/docs/shots/report-overview@1x.webp)

The top of the report: the failure, the run and a timeline of its four steps.

![The failed step 4 of the report: flow_assertion_timeout with its test_failure class, the step record with the flow file, line and column, folded sections for screenshots, the hierarchy diff, device logs and network requests, and a replay-to-this-state command.](https://autonom.yatsuk.me/docs/shots/report-step-failure@1x.webp)

Step 4 opened: its record, with screenshots, the tree diff, logs and requests folded underneath.

**What to look at.** The report names the first failure that caused the others and links to its step. That step gives the file, line and column, the evidence Autonom kept and the command that replays the flow up to that point.

### Explain a crash with the app’s logs

**Goal:** save the app’s recent logs into the session and find the line that explains the failure.

1. **Read the app’s recent logs.** `--package` narrows the device log to your app.

   ```sh
   autonom logs tail --package com.example.app --since 60
   ```

2. **List crash reports.** On Android they come from the crash log buffer, on iOS from idb’s crash store.

   ```sh
   autonom crash list
   ```

You should see five lines: three from your app, the fatal exception among them, and two system lines that start and end its process. Then comes the filter Autonom used.

*Output of autonom logs tail*

```
{
  "ok": true,
  "count": 5,
  "lines": [
    …
      "line": "09-26 12:00:04.000  4242  4242 E AndroidRuntime: FATAL EXCEPTION: main"
    …
  ],
  …
  "filter": "uid",
  …
}
```

**What to look at.** `filter` is `uid`: on API 31 and later Autonom keeps the lines your app’s user id logged, plus the system lines that start or end its process. Another app’s lines never leak in.

**Show full output**

```
{
  "ok": true,
  "count": 5,
  "lines": [
    {
      "line": "09-26 12:00:00.000  4242  4242 I flutter : [Network] GET /catalog"
    },
    {
      "line": "09-26 12:00:01.000  1000  1100 I ActivityManager: Start proc 4242:com.example.app/u0a123 for top-activity"
    },
    {
      "line": "09-26 12:00:04.000  4242  4242 E AndroidRuntime: FATAL EXCEPTION: main"
    },
    {
      "line": "09-26 12:00:04.001  4242  4242 E AndroidRuntime: Process: com.example.app, PID: 4242"
    },
    {
      "line": "09-26 12:00:05.000  1000  1100 I ActivityManager: Process com.example.app (pid 4242) has died: fg TOP"
    }
  ],
  "platform": "android",
  "target_id": "emulator-5554",
  "serial": "emulator-5554",
  "filter": "uid",
  "saved": "/tmp/autonom-evidence/.autonom/sessions/s_004f33b833/logs/latest.json"
}
```

### Reproduce these outputs

From the root of an Autonom checkout at tag v0.36.1 (commit ca2a280), in bash or zsh. The target is Autonom’s own test device on its Yarn Shop sign-in screen; the log lines and app ids come from Autonom’s own tests. Ids, times and paths differ on every run. The report screens come from the same flow run.

*Script that reproduces the outputs*

```sh
S=$PWD W=/tmp/autonom-evidence
rm -rf $W && mkdir -p $W/flows && cd $W
export AUTONOM_HOME=$W/.autonom AUTONOM_FAKE_STATE=$W/fake-state.json
echo "{\"ui_dump\": \"$S/tests/fixtures/woolbox/android_sign_in.xml\"}" > $AUTONOM_FAKE_STATE
autonom() { python3 $S/scripts/autonom.py --platform android --serial emulator-5554 --adb $S/tests/fakes/fake_adb.py "$@"; }
autonom session start --app-id com.example.app
autonom simulator status-bar pin
autonom screenshot --label "after login"
autonom network mock add --url '.../v1/login' --status 500 --json '{"error":"x"}'
autonom screenshot --task login --label "500 error state"
autonom network mock disable --all
cat > flows/sign-in.yaml <<'Y'
schema: autonom.dev/flow/v1
id: sign-in-email-001
appId: com.example.app
name: Sign in with email
tags:
  - smoke
  - auth
evidence:
  mode: always
  collect:
    - screenshot
    - hierarchy
    - logs
---
- launchApp
- assertVisible:
    selector:
      visibleText: Welcome to Yarn Shop
      match: exact
- tapOn:
    selector:
      visibleText: Sign in with Email
      match: exact
    label: Open email sign in
- assertVisible:
    selector:
      text: Sign in to access all features
      match: exact
    timeoutMs: 1500
Y
autonom flow run flows/sign-in.yaml
autonom report build
cat > $AUTONOM_FAKE_STATE <<J
{"getprop": {"ro.build.version.sdk": "34"}, "pidof": {},
 "packages": {"com.example.app": 10123, "com.other": 10200, "com.example.app.debug": 10300},
 "pid_uids": {"4242": 10123, "5555": 10200, "1000": 1000, "6000": 10300},
 "logcat": [
  "09-26 12:00:00.000  4242  4242 I flutter : [Network] GET /catalog",
  "09-26 12:00:01.000  1000  1100 I ActivityManager: Start proc 4242:com.example.app/u0a123 for top-activity",
  "09-26 12:00:02.000  5555  5555 I chatty  : another app talking",
  "09-26 12:00:03.000  1000  1100 I ActivityManager: Start proc 5555:com.other/u0a200 for service",
  "09-26 12:00:04.000  4242  4242 E AndroidRuntime: FATAL EXCEPTION: main",
  "09-26 12:00:04.001  4242  4242 E AndroidRuntime: Process: com.example.app, PID: 4242",
  "09-26 12:00:05.000  1000  1100 I ActivityManager: Process com.example.app (pid 4242) has died: fg TOP",
  "09-26 12:00:06.000  1000  1100 I ActivityManager: Start proc 6000:com.example.app.debug/u0a300 for activity"]}
J
autonom logs tail --package com.example.app --since 60
```

## Good to know

- **Exit 0 is not proof.** A tap that returns ok has not been shown to do anything. Compare before and after trees or screenshots; the change is the proof.
- **Pin before the first capture you mean to compare.** Otherwise the battery glyph and the signal bars show up as a difference the app never made. `hhmm=0941` freezes the clock through demo mode; use it only when a fixed time matters more.
- **Take evidence with `autonom screenshot`, not a raw screen capture.** An emulator’s bars and battery drift within a minute of a pin, and Autonom sends the pin again right before every screenshot and flow capture.
- **Never present a mocked screenshot as the server’s behavior.** A capture taken while a rule is active carries `screenshot_shows_mocked_data` in its output, its PNG and the index.
- **A weaker log filter says so.** Below API 31, Autonom narrows logs to the app’s running process. If the app is not running, or the API level cannot be read, it keeps only lines that name the app. `filter` then reads `pid` or `none`, with a [`log_filter_degraded`](https://autonom.yatsuk.me/docs/safety/index.md#log_filter_degraded) warning, instead of passing the whole device log off as your app’s.
- **State the coordinate space.** `width` and `height` come from the PNG. On iOS that is pixels, while the tree and `ui tap` speak points.
- **A failed crash listing is an error, not an empty list.** On iOS, when idb has lost its companion, `crash list` fails instead of answering with no crashes.
- **Every live follow is bounded.** `logs follow` stops at `--max-seconds` or `--max-lines`.
- **A record that cannot be saved never repeats the input.** When an action ran but its detail file could not be written, the answer carries an [`evidence_write_failed`](https://autonom.yatsuk.me/docs/safety/index.md#evidence_write_failed) warning. Look at the screen before you act again.
- **Treat the session folder as sensitive.** It can hold screenshots, logs, pulled files and captured requests. It lives outside any repository; delete it when the investigation ends.

## Reference

Each command acts on the session’s one target and prints JSON.

| Command | What it does |
| --- | --- |
| `autonom screenshot` | Captures the screen with its provenance in the PNG. `--label` names it; `--task` groups it. |
| `autonom shots list` | Lists the session’s screenshots from the index; `--mocked-only` keeps those taken under a rule. |
| `autonom shots show <path>` | Shows one screenshot’s provenance and size. |
| `autonom record start` | Starts recording the screen into the session; `record stop` ends it. |
| `autonom logs tail` | Reads recent device logs. `--package` narrows them to one app and names the filter it used. |
| `autonom logs follow` | Prints new log lines as they arrive, until `--max-seconds` or `--max-lines`. |
| `autonom crash list` | Lists crash reports; `crash show` opens one. |
| `autonom simulator status-bar <action>` | `pin` sets full battery and signal and hides notification icons, with the real clock; `clear` restores. |
| `autonom simulator animations <action>` | Android: `pin` sets the three animation scales to 0; `reset` restores them. |
| `autonom simulator keyboard <action>` | iOS: `pin` turns autocorrect, prediction and auto-capitalization off and sets a locale; `reset` restores. |
| `autonom ui wait --settled` | Waits until two trees in a row match; exits 1 when the screen never settles. |
| `autonom report build` | Builds the HTML report, the JUnit file and the bundle of a flow run. |
| `autonom note add <text>` | Adds a note to the journal. |
| `autonom journal` | Reads the session’s timeline back: every command, its scrubbed arguments and the result. |

## Android and iOS differences

Logs are partial on both: Autonom narrows them to one app and says how.

| Capability | Android | iOS Simulator |
| --- | --- | --- |
| Screenshot | Yes | Yes, even without idb |
| Status bar pin | Yes; the signal is not read back | Yes, read back |
| System animations off | Yes | No |
| Keyboard and locale pin | No: the settings live in Gboard | Yes, on a shut-down Simulator |
| Logs | Partly: by user id on API 31 and later | Partly: by the app binary’s UUID |
| Crash reports | Partly: the crash log buffer | Yes: idb’s crash store |
| Screen recording | Yes | Yes |

On API 36 emulators the system UI may not draw the battery glyph at all, even when the pinned level is reported. A missing icon in a capture is the emulator’s.

## Next steps

You can bring back screenshots, logs and a report that prove what the app did.

- [Network](https://autonom.yatsuk.me/docs/network/index.md): See what the app sent, and make its server fail on purpose.
- [Flows](https://autonom.yatsuk.me/docs/flows/index.md): Keep evidence for every step of a replay.
- [Sessions and targets](https://autonom.yatsuk.me/docs/sessions/index.md): Read every step back from the journal.