# Agent report

Canonical URL: https://autonom.yatsuk.me/docs/agent-report/

Give your agent a run as compact JSON instead of an HTML report. The agent report names the failure, the steps and the evidence each one kept, and one step can be inspected on its own.

**Try it**

```sh
autonom agent export --bundle <dir> --out agent.json
```

**Needs** [a flow run](https://autonom.yatsuk.me/docs/flows/index.md)

## How it works

`agent export` writes one `autonom.agent-report/v1` file. `agent inspect` prints a single step, with the run's attempt and only that step's attachments. Both read the same report, from one of two sources:

- **A [report bundle](https://autonom.yatsuk.me/docs/report-bundle/index.md),** with `--bundle`. Autonom verifies every file of the bundle against its digest before it reads anything, so an edited bundle is refused instead of reported on.
- **A run of a session,** with `--run`, from the current session or the one `--session` names. A stopped session works too. With no `--run`, the session's latest run is read.

Both commands only read. They change nothing in the bundle or the session, and need no device; `agent export` writes only its `--out` file. A run records a secret's name, not its value, in the flow, the command line and the steps' arguments. Evidence of the screen is kept as it was, though: a typed secret that the screen shows is in the screen trees, the screenshots and the steps' target text, and so in the bundle and the report.

`agent inspect --step` takes the step's runtime index, such as `6`, or its stable step id, such as `src_4d88cd3b8ce7.1`. The stable id is the safer choice for anything your agent stores, because it names the same step after a flow gains a conditional branch.

## Examples

Every output here comes from Autonom's test emulator showing a PIN screen, so you can rerun it without a device. The run types a PIN from the `APP_PIN` secret, taps Continue, then waits for a Welcome screen that never comes, so step 6 fails. Reproduce these outputs, at the end of this section, has the exact commands.

### Export the report of a failed run

**Goal:** hand your agent the failure and its evidence in one file.

1. **Export the report from the run's bundle.** `autonom report bundle --run ID --out DIR` wrote the bundle when the run ended.

   ```sh
   autonom agent export --bundle bundle --out agent.json
   ```

2. **Read it.** The file is compact JSON; `python3 -m json.tool` indents it.

   ```sh
   python3 -m json.tool agent.json
   ```

You should see where the file went, then the report: the failure, the steps and a summary.

*Output of autonom agent export*

```
{
  "ok": true,
  "format": "agent",
  "out": "/tmp/autonom-agent/agent.json"
}
```

*agent.json, indented*

```
{
    "attachments": [
        …
    ],
    "capabilities": {
        …
        "device_class": "simulator",
        "provider": "local.android",
        "schema": "autonom.provider/v1",
        "target_id": "emulator-5554"
    },
    "failure": {
        "command": "assertVisible",
        "error": "{'text': 'Welcome', 'match': 'exact'} was not visible within 1000 ms",
        "error_code": "flow_assertion_timeout",
        "failure_class": "test_failure",
        "flow": "/tmp/autonom-agent/pin.yaml",
        "line": 15,
        "step_index": 6
    },
    "replay": {
        "mode": "baseline",
        "portable_restore": "replay-from-flow-start",
        "source_run_id": null
    },
    "schema": "autonom.agent-report/v1",
    "setup": {
        …
    },
    "steps": [
        …
        {
            "attachment_ids": [
                "att_…",
                "att_…"
            ],
            "command": "assertVisible",
            "delta": {
                …
            },
            "error": {
                "class": "test_failure",
                "code": "flow_assertion_timeout",
                "message": "{'text': 'Welcome', 'match': 'exact'} was not visible within 1000 ms"
            },
            "index": 6,
            "name": "assertVisible",
            "status": "failed",
            "step_id": "src_4d88cd3b8ce7.1"
        }
    ],
    "summary": {
        "attachments": 5,
        "attempt_id": "attempt_…",
        "case_id": "case_daaab23de6ec5430",
        "first_causal_failure": "src_4d88cd3b8ce7.1",
        "history_id": "hist_9d9270ba34dd43aa",
        "proof_verdict": "fail",
        "run_id": "fr_…",
        "schema": "autonom.summary/v1",
        "status": "failed",
        "step_status": {
            "broken": 0,
            "failed": 1,
            "passed": 5,
            "skipped": 0,
            "unknown": 0
        },
        "steps": 6
    }
}
```

**What to look at.** `failure` gives the code, the file and the line, so your agent can branch on [`flow_assertion_timeout`](https://autonom.yatsuk.me/docs/safety/index.md#flow_assertion_timeout) without parsing text. `summary.first_causal_failure` is the stable id of the step to inspect next. The failing step lists the attachments it kept, a screenshot and a screen tree, by id.

### Inspect the failing step

**Goal:** read one step in full, with only the evidence it kept.

1. **Inspect it by its index.**

   ```sh
   autonom agent inspect --bundle bundle --step 6
   ```

2. **Inspect it by its stable id.** The answer is byte for byte the same.

   ```sh
   autonom agent inspect --bundle bundle --step src_4d88cd3b8ce7.1
   ```

You should see the run's attempt, the step and its two attachments.

*Output of autonom agent inspect*

```
{
  "ok": true,
  "attempt": {
    "attempt_id": "attempt_…",
    "execution_command": "autonom flow run /tmp/autonom-agent/pin.yaml --secret APP_PIN",
    …
    "proof_verdict": "fail",
    "reproduction": "autonom replay --bundle <bundle>",
    "retry_of": null,
    "run_id": "fr_…",
    …
    "status": "failed"
  },
  "step": {
    …
    "command": "assertVisible",
    …
    "error": {
      "class": "test_failure",
      "code": "flow_assertion_timeout",
      "message": "{'text': 'Welcome', 'match': 'exact'} was not visible within 1000 ms"
    },
    …
    "index": 6,
    "name": "assertVisible",
    …
    "source_id": "src_4d88cd3b8ce7",
    …
    "status": "failed",
    "step_id": "src_4d88cd3b8ce7.1"
  },
  "attachments": [
    {
      …
      "blob": "blobs/sha256/da/da41…",
      "capture_kind": "hierarchy",
      "mime_type": "application/json",
      "name": "failure-step-6-hierarchy.json",
      "path": "flows/fr_…/failure-step-6-hierarchy.json",
      …
      "type": "ui_tree"
    },
    {
      …
      "blob": "blobs/sha256/…",
      "capture_kind": "screenshot",
      "mime_type": "image/png",
      …
      "path": "shots/fr_…/0002_…_failure-step-6.png",
      …
      "type": "screenshot"
    }
  ]
}
```

*Comparing the two answers*

```
index 6 and src_4d88cd3b8ce7.1 give the same answer
```

**What to look at.** `execution_command` is the command that made the run; the secret appears only as `--secret APP_PIN`. `reproduction` is how to rebuild it, with [Replay](https://autonom.yatsuk.me/docs/replay/index.md). To open an attachment from a bundle, join the bundle directory and its `blob`, such as `bundle/blobs/sha256/da/da41…`. `path` is where the file was in the session's directory, `flows/` or `shots/` under the session, and the bundle has no such folder. A report read from a session with `--run` has no `blob`; there, `path` is relative to the session's directory.

### Check that nothing changed and nothing leaked

**Goal:** confirm the report left the bundle as it was, and that the PIN was not recorded.

1. **Verify the bundle again.**

   ```sh
   autonom report verify bundle
   ```

2. **Search the bundle and the report for the PIN.**

   ```sh
   grep -rl 271828 bundle agent.json
   ```

You should see the bundle verified, and no file holding the secret, because the PIN field of this screen never shows what was typed.

*Output of autonom report verify and grep*

```
{
  "ok": true,
  "bundle": "/tmp/autonom-agent/bundle",
  "files": 18,
  "run_id": "fr_…"
}
the secret is in neither the bundle nor the report
```

**What to look at.** Every file still matches its digest after the export and both inspections. The second line comes from the script: `grep` found no file. The run kept only the name `APP_PIN`; the next example shows what changes when the screen shows the value.

### Know that screen evidence keeps what the screen showed

**Goal:** see where a secret goes when the app displays it, before you share a bundle or a report.

1. **Run the flow again on a screen that shows the PIN.** The script now shows `271828` in the PIN field, as a plain text field shows what was typed into it. It runs the flow, bundles the run into `shown` and exports its report to `shown-agent.json`.

   ```sh
   autonom flow run pin.yaml --secret APP_PIN
   ```

2. **Search the new bundle and report for the PIN.**

   ```sh
   grep -rl 271828 shown shown-agent.json
   ```

You should see the PIN in the report, in the bundle's screen trees and event stream, and in the files built from them.

*Files that hold the PIN*

```
shown-agent.json
shown/blobs/sha256/…
shown/blobs/sha256/…
shown/blobs/sha256/…
shown/model/report.json
shown/run.json
shown/steps/src_f8b360411243.1.json
shown/streams/events.ndjson
```

*Output of the verify check*

```
the shown bundle still verifies
```

**What to look at.** The three blobs are the two screen trees and the event stream; the step file is step 2, whose target text is the field's text. `report verify` still passes, because nothing was changed after the run: the value was recorded as screen content, not as the secret. On a real device the screenshots show it too. Treat a bundle or an agent report as sensitive when a secret can appear on screen: have the app mask the field, use a test credential, and do not publish such a bundle as a public artifact.

### Read the refusals

**Goal:** tell why a report was not produced, from the error code alone.

Each refusal is one error object on stderr, with exit code 2.

*A step the run does not have*

```
{"ok": false, "error_code": "flow_replay_step_not_reached", "error": "report has no step '7'"}
```

*No bundle and no session*

```
{"ok": false, "error_code": "no_active_session", "error": "no active session", "hint": "Start one with 'autonom session start', or pass a session id."}
```

*A file in the bundle was edited*

```
{"ok": false, "error_code": "report_integrity_failed", "error": "report bundle integrity verification failed", "failures": [{"path": "steps/src_4d88cd3b8ce7.1.json", "reason": "digest-mismatch"}]}
```

**What to look at.** [`flow_replay_step_not_reached`](https://autonom.yatsuk.me/docs/safety/index.md#flow_replay_step_not_reached) means the step index or id is not in this run. [`no_active_session`](https://autonom.yatsuk.me/docs/safety/index.md#no_active_session) asks for a source: pass `--bundle`, or name the stopped session with `--session` and its run with `--run`, as the script does after `session stop`. [`report_integrity_failed`](https://autonom.yatsuk.me/docs/safety/index.md#report_integrity_failed) lists each file whose digest no longer matches; nothing is read from that bundle.

### Reproduce these outputs

From the root of an Autonom checkout at tag v0.36.1 (commit ca2a280), in bash or zsh. The `autonom` function runs that checkout against its test emulator, and the script writes the PIN screen it shows, then a copy whose PIN field shows the PIN. Run ids, attempt ids, attachment ids, session ids, times and sizes differ on every run, and ids are shortened to … on the page; step, case and history ids stay the same, because the flow has an `id`. On macOS the paths inside the report start with `/private/tmp`.

*Script that reproduces the outputs*

```sh
# autonom below runs this checkout against Autonom's test emulator, showing a PIN screen the script writes.
S=$PWD W=/tmp/autonom-agent
rm -rf $W && mkdir -p $W
export AUTONOM_HOME=$W/.autonom AUTONOM_FAKE_STATE=$W/device.json
autonom() { python3 $S/scripts/autonom.py --adb $S/tests/fakes/fake_adb.py "$@"; }
cat > $W/pin.xml <<'XML'
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<hierarchy rotation="0">
  <node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.example.app" content-desc="" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" bounds="[0,0][1080,2400]">
    <node index="0" text="Enter your PIN" resource-id="com.example.app:id/title" class="android.widget.TextView" package="com.example.app" content-desc="" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" bounds="[80,200][1000,300]" />
    <node index="1" text="" resource-id="com.example.app:id/pin" class="android.widget.EditText" package="com.example.app" content-desc="PIN" clickable="true" enabled="true" focusable="true" focused="true" scrollable="false" bounds="[80,400][1000,520]" />
    <node index="2" text="Continue" resource-id="com.example.app:id/continue" class="android.widget.Button" package="com.example.app" content-desc="" clickable="true" enabled="true" focusable="true" focused="false" scrollable="false" bounds="[80,600][1000,720]" />
  </node>
</hierarchy>
XML
echo "{\"ui_dump\": \"$W/pin.xml\"}" > $AUTONOM_FAKE_STATE
cat > $W/pin.yaml <<'YAML'
schema: autonom.dev/flow/v1
appId: com.example.app
name: PIN sign-in
id: pin-sign-in
---
- launchApp
- tapOn:
    selector:
      description: PIN
- inputText: ${APP_PIN}
- checkpoint: pin-entered
- tapOn:
    selector:
      text: Continue
- assertVisible:
    selector:
      text: Welcome
    timeoutMs: 1000
YAML
autonom session start --serial emulator-5554 --app-id com.example.app > $W/session.json
SESSION=$(python3 -c 'import json, sys; print(json.load(open(sys.argv[1]))["session"]["session_id"])' $W/session.json)
export APP_PIN=271828
# A run that fails at step 6, kept as a report bundle
autonom flow run $W/pin.yaml --secret APP_PIN > $W/run.json
RUN=$(python3 -c 'import json, sys; print(json.load(open(sys.argv[1]))["run_id"])' $W/run.json)
autonom report bundle --run $RUN --out $W/bundle > /dev/null
# Export the report of a failed run
autonom agent export --bundle $W/bundle --out $W/agent.json
python3 -m json.tool $W/agent.json
# Inspect the failing step
autonom agent inspect --bundle $W/bundle --step 6 | tee $W/by-index.json
autonom agent inspect --bundle $W/bundle --step src_4d88cd3b8ce7.1 > $W/by-id.json
cmp $W/by-index.json $W/by-id.json && echo "index 6 and src_4d88cd3b8ce7.1 give the same answer"
# Check that nothing changed and nothing leaked
autonom report verify $W/bundle
grep -rl 271828 $W/bundle $W/agent.json || echo "the secret is in neither the bundle nor the report"
# Know that screen evidence keeps what the screen showed
sed 's/text="" resource-id="com.example.app:id\/pin"/text="271828" resource-id="com.example.app:id\/pin"/' $W/pin.xml > $W/shown.xml
echo "{\"ui_dump\": \"$W/shown.xml\"}" > $AUTONOM_FAKE_STATE
autonom flow run $W/pin.yaml --secret APP_PIN > $W/run-shown.json
SHOWN=$(python3 -c 'import json, sys; print(json.load(open(sys.argv[1]))["run_id"])' $W/run-shown.json)
autonom report bundle --run $SHOWN --out $W/shown > /dev/null
autonom agent export --bundle $W/shown --out $W/shown-agent.json > /dev/null
grep -rl 271828 $W/shown $W/shown-agent.json | sed "s|$W/||" | sort
autonom report verify $W/shown > /dev/null && echo "the shown bundle still verifies"
echo "{\"ui_dump\": \"$W/pin.xml\"}" > $AUTONOM_FAKE_STATE
# Read the refusals
autonom agent inspect --bundle $W/bundle --step 7
autonom session stop > /dev/null
autonom agent export --out $W/no-session.json
autonom agent export --session $SESSION --run $RUN --out $W/after-stop.json
cp -R $W/bundle $W/edited && echo '{}' >> $W/edited/steps/src_4d88cd3b8ce7.1.json
autonom agent inspect --bundle $W/edited --step 6
```

## Good to know

- **Branch on codes, not on messages.** `failure.error_code` and each step's `error.code` are stable codes; the messages are for people.
- **Start from `first_causal_failure`.** The summary names the first step that failed or broke, by its stable id; later failures often follow from it. Pass that id to `agent inspect`.
- **Store step ids, not indexes.** An index counts runtime steps, so a conditional or a retry can shift it. A step id stays with the step.
- **Read a bundle when the run matters later.** A bundle is verified on every read and keeps its own copy of the flow, so its report stays trustworthy after the session is gone.
- **Missing evidence is marked, not invented.** A step's `delta` says for the screen, logs and requests whether evidence is `available`, `partial` or `unavailable`, instead of showing an empty list as if nothing happened.
- **The snapshot travels with the run.** `capabilities` is the [capability snapshot](https://autonom.yatsuk.me/docs/capabilities/index.md) the run was checked against, and `setup` lists what the run applied and verified.

## Reference

Both commands take the same sources.

| Command | What it does |
| --- | --- |
| `autonom agent export --out FILE` | Writes the `autonom.agent-report/v1` file of the current session's latest run. |
| `autonom agent inspect --step STEP` | Prints one step by runtime index or stable step id, with the attempt and that step's attachments. |
| `--bundle DIR` | Reads a report bundle, after verifying every file against its digest. |
| `--run ID` | Reads one run of the session instead of its latest. |
| `--session ID` | Reads from that session, running or stopped, instead of the current one. |

The fields of `autonom.agent-report/v1`.

| Field | What it holds |
| --- | --- |
| `schema` | `autonom.agent-report/v1`. |
| `summary` | The run and attempt ids, `status`, `proof_verdict`, step counts by status, the attachment count, `first_causal_failure`, and the `case_id` and `history_id` that stay the same across runs of one flow. |
| `failure` | The run's first failure: `error_code`, `error`, `failure_class`, the command, the file, the line and the step index. `null` when the run passed. |
| `setup` | What the run's setup applied, selected, used and verified, and the capabilities that were available. |
| `capabilities` | The `autonom.provider/v1` snapshot the run was checked against. |
| `replay` | How the run can be rebuilt. A replay's own report adds the step it stopped at and `cleanup_hooks_skipped`. |
| `steps` | Every step: `step_id`, `index`, `name`, `command`, `status`, `error`, `delta` and `attachment_ids`. |
| `attachments` | Every piece of evidence: id, type, name, MIME type, `path` relative to the session's directory and, in a bundle, `blob` (the file's place in the bundle), `size` and `sha256`. |

The refusals, each with exit code 2.

| Error code | When |
| --- | --- |
| `flow_replay_step_not_reached` | `--step` names no step of the run. |
| `report_integrity_failed` | The bundle has no manifest, or a file in it no longer matches its digest. |
| `no_active_session` | There is no `--bundle`, no `--session`, and no current session. |
| `usage_error` | `--out` is missing from `agent export`, or `--step` from `agent inspect`. |

## Next steps

Your agent can read a run as compact JSON and inspect any step.

- [Replay](https://autonom.yatsuk.me/docs/replay/index.md): Rebuild the state at the failing step.
- [Evidence](https://autonom.yatsuk.me/docs/evidence/index.md): Open the same run as an HTML report.
- [Flows](https://autonom.yatsuk.me/docs/flows/index.md): Fix the flow the report points at.
- [Capabilities](https://autonom.yatsuk.me/docs/capabilities/index.md): See what the target could do during the run.