# CI campaigns

Canonical URL: https://autonom.yatsuk.me/docs/ci/

Run your flows as one campaign on a CI runner. Autonom keeps every run as a verified report bundle, gates the runs on explicit rules, can publish the whole campaign, and ends with an exit code your pipeline can act on.

**Try it**

```sh
autonom ci run flows --out ci-1
```

**Needs** [a session on one target](https://autonom.yatsuk.me/docs/sessions/index.md) · [flows](https://autonom.yatsuk.me/docs/flows/index.md)

## How it works

`autonom ci run PATH --out DIR` takes one flow file, or a directory of flows, and runs each flow on the current session's target. It works in phases, and writes the phase it has reached to `ci-state.json` in `--out` after each one:

- **`capture`:** the flows run one after another, with full evidence. Each run is finalized as a [report bundle](https://autonom.yatsuk.me/docs/report-bundle/index.md) under `bundles/` and packed as one shard under `packs/`, and `runs` records its flow, run id, status, bundle and pack. A flow that Autonom refuses to start is recorded under `errors`, and the flows after it do not run.
- **`merge`:** the shards are merged into `campaign/campaign.json`, which compares the shards it expected with the shards it received.
- **`finalize`:** every run is checked against the gate rules, and the verdict is written to `campaign/final.json`.
- **`publish`:** only with `--publish`. The finalized campaign is copied to an empty directory, or sent to an HTTP endpoint.
- **`complete`:** the campaign is done, and `ci-state.json` also holds its `final` verdict and its `publication`.

A campaign expects one shard per flow, unless `--expected-shards` says how many to wait for. A campaign that receives fewer is `incomplete` and fails, even when every run that did arrive passed. In v0.36.1, `missing_shards` is the expected number minus the number of distinct attempts received, not of distinct shard ids. Merge drops a pack only when its attempt id was already seen, so a runner that reruns `ci run` with the same `--shard-id` adds a new attempt that counts as one more shard, and can hide a runner that never reported.

`--env KEY=VALUE` sets a variable that a flow reads as `${KEY}`. `--secret NAME` reads a secret from the runner's environment by its name. Autonom records only the name, so the value is not written into the flow, the recorded command or the steps' arguments. Screen evidence keeps what the screen showed, as it does for [any run](https://autonom.yatsuk.me/docs/agent-report/index.md#screen-evidence).

Without `--rules`, the gate asks each run for status `passed`, a proof verdict of `pass` or `not_applicable`, no failed or broken step, and evidence for every failure. `--rules FILE` names a JSON object whose keys replace those defaults one by one; a key the file leaves out keeps its default.

The same work is split into commands for a campaign that runs on several machines: `ci pack` packs one bundle as a shard, `ci merge` merges the packs, `ci finalize` gates the merged campaign, and `ci publish` publishes a finalized one.

Of the `ci` commands, only `ci pack` verifies a bundle. `ci merge` reads `shard.json` and `bundle/model/report.json` from each pack and checks only the report's schema, not the bundle's digests, and `ci finalize` gates the reports merge copied. A pack changed between `ci pack` and `ci merge`, for example a report whose failed run was flipped to passed, is merged and gated as it is. When packs come from runners or channels you do not control, unzip each one and run `autonom report verify` on its `bundle/` directory before the merge, or compare the `sha256` that `ci pack` printed on the runner with the `sha256` that `campaign.json` records for that shard before you finalize. Even a verified bundle only shows that it is whole, not who made it, as [Report bundle](https://autonom.yatsuk.me/docs/report-bundle/index.md#how) explains.

`ci run` ends with one of three exit codes:

- **`0`,** status `passed`: every expected shard arrived, every run passed its gate, and the publication, if you asked for one, succeeded.
- **`1`,** status `failed`: a run failed its gate, a shard is missing, or a flow could not start.
- **`2`:** the publication failed, whatever the status, and `publication.publication_status` is `failed`. Exit code 2 with an error object on stderr instead means the campaign was refused before any flow ran.

The other commands keep to their own job. `ci merge` exits 1 while shards are missing. `ci finalize` exits 0 when the status is `passed` and 1 when it is not, even when its own `--publish` failed, so read `publication.publication_status` too. In v0.36.1 the top-level `publication_status` of `ci finalize`, and the one in `final.json`, always says `not_requested`, whether the publication failed, succeeded or was never asked for. `ci publish` exits 2 with [`ci_destination_failed`](https://autonom.yatsuk.me/docs/safety/index.md#ci_destination_failed) when it cannot publish.

Use a new `--out` directory for every campaign. In v0.36.1, `ci run` adds to what the directory already holds, and the gate checks every run left in `campaign/models/`, including the runs of earlier campaigns. The same goes for `ci merge --out`: a directory that held an earlier merge keeps its runs and its `final.json`.

## Examples

Every output here comes from Autonom's test emulator showing a PIN screen, so you can rerun it without a device. Two flows in `flows/` pass: `title.yaml` looks for the title it gets from `--env TITLE`, and `continue.yaml` looks for the Continue button. `sign-in.yaml` types a PIN from the `APP_PIN` secret, taps Continue, then waits for a Welcome screen that never comes, so it fails. On a real device or Simulator the runs take as long as their flows, and the bundles hold real screenshots, so times, sizes and attachment counts differ. Reproduce these outputs, at the end of this section, has the exact commands.

### Run a campaign

**Goal:** run every flow in a directory on the runner's device and get one verdict.

1. **Start a session on the runner's target.**

   ```sh
   autonom session start --serial emulator-5554 --app-id com.example.app
   ```

2. **Run the flows as one campaign.**

   ```sh
   autonom ci run flows --out ci-1 --env TITLE="Enter your PIN"
   ```

3. **Read what the campaign wrote, and the phase it reached.**

   ```sh
   ls ci-1 python3 -c 'import json; s = json.load(open("ci-1/ci-state.json")); print(json.dumps({k: s[k] for k in ("schema", "phase", "runs", "errors")}, indent=2))'
   ```

You should see status `passed` and exit code 0, then the directory and its state.

*Output of autonom ci run and its exit code*

```
{
  "ok": true,
  "campaign_id": "campaign_…",
  "status": "passed",
  "execution": {
    "attempts": 2,
    "passed": 2,
    "failed": 0
  },
  "missing_shards": 0,
  "publication": {
    "publication_status": "not_requested"
  },
  "out": "/tmp/autonom-ci/ci-1",
  "state": "/tmp/autonom-ci/ci-1/ci-state.json"
}
exit 0
```

*Output of ls and of the ci-state.json fields*

```
bundles
campaign
campaign-spec.json
ci-state.json
packs
{
  "schema": "autonom.ci-state/v1",
  "phase": "complete",
  "runs": [
    {
      "flow": "/tmp/autonom-ci/flows/continue.yaml",
      "run_id": "fr_…",
      "status": "passed",
      "bundle": "/tmp/autonom-ci/ci-1/bundles/fr_…",
      "pack": "/tmp/autonom-ci/ci-1/packs/shard-1.zip"
    },
    {
      "flow": "/tmp/autonom-ci/flows/title.yaml",
      "run_id": "fr_…",
      "status": "passed",
      "bundle": "/tmp/autonom-ci/ci-1/bundles/fr_…",
      "pack": "/tmp/autonom-ci/ci-1/packs/shard-2.zip"
    }
  ],
  "errors": []
}
```

**What to look at.** `execution` counts the runs, and `status` is the gate's verdict on them. The flows ran in name order, each kept as a bundle and a pack, and the packs are named after the default `--shard-id`, `shard`. `phase: complete` says the campaign got to its end; a runner that stopped halfway leaves the phase it was in.

### When a flow fails

**Goal:** see which rules a failed run broke.

1. **Run the failing flow, with its secret.** `APP_PIN` is exported in the runner's environment.

   ```sh
   autonom ci run sign-in.yaml --out ci-2 --secret APP_PIN
   ```

2. **Read the gate's failures from the campaign's verdict.**

   ```sh
   python3 -c 'import json; f = json.load(open("ci-2/campaign/final.json")); print(json.dumps(f["gates"][0]["failures"], indent=2))'
   ```

You should see status `failed` with exit code 1, then three broken rules.

*Output of autonom ci run and its exit code*

```
{
  "ok": true,
  "campaign_id": "campaign_…",
  "status": "failed",
  "execution": {
    "attempts": 1,
    "passed": 0,
    "failed": 1
  },
  "missing_shards": 0,
  "publication": {
    "publication_status": "not_requested"
  },
  "out": "/tmp/autonom-ci/ci-2",
  "state": "/tmp/autonom-ci/ci-2/ci-state.json"
}
exit 1
```

*The gate failures in final.json*

```
[
  {
    "actual": "failed",
    "rule": "allowed_statuses"
  },
  {
    "actual": "fail",
    "rule": "allowed_proof_verdicts"
  },
  {
    "actual": 1,
    "maximum": 0,
    "rule": "max_failed_steps"
  }
]
```

**What to look at.** The run failed, its proof verdict is `fail`, and one step failed where none is allowed. `require_evidence_for_failures` is not in the list, because the failing step kept its screenshot and screen tree. The run's bundle is under `ci-2/bundles/`, ready for [`agent export`](https://autonom.yatsuk.me/docs/agent-report/index.md) or [`replay`](https://autonom.yatsuk.me/docs/replay/index.md).

### Keep secrets out of the campaign

**Goal:** check that the PIN the flow typed was not written into the campaign.

1. **Search every file of the campaign for the PIN's value.**

   ```sh
   grep -rl 271828 ci-2 || echo "the PIN is in no file under ci-2"
   ```

You should see that no file holds it.

*Output of grep*

```
the PIN is in no file under ci-2
```

**What to look at.** `--secret APP_PIN` passed the value from the environment, and the campaign recorded only the name. This test screen never shows what was typed. A real app that displays the PIN puts it into the screen trees and screenshots of its bundles, so use a test credential, and do not publish such a campaign as a public artifact.

### Split a campaign across runners

**Goal:** run a campaign on several machines and judge it as one.

1. **On each runner, run its share with its own shard id.** Each runner writes its pack as `packs/<shard-id>-1.zip`.

   ```sh
   autonom ci run flows/title.yaml --out runner-1 --shard-id one --env TITLE="Enter your PIN" autonom ci run flows/continue.yaml --out runner-2 --shard-id two
   ```

2. **Merge the packs that have arrived, out of the three expected, and finalize.**

   ```sh
   autonom ci merge runner-1/packs/one-1.zip runner-2/packs/two-1.zip --out merged-1 --expected-shards 3 autonom ci finalize merged-1
   ```

3. **Pack the third runner's bundle.** That runner ran its flow with `flow run` and kept it with `report bundle`.

   ```sh
   autonom ci pack three --out three.zip --shard-id three
   ```

4. **Merge every pack into a new directory, and finalize again.** The first pack is passed twice, as a retried upload would deliver it.

   ```sh
   autonom ci merge runner-1/packs/one-1.zip runner-2/packs/two-1.zip three.zip runner-1/packs/one-1.zip --out merged-2 --expected-shards 3 autonom ci finalize merged-2
   ```

You should see `incomplete` and exit code 1 while a shard is missing, then `complete` and `passed` with exit code 0.

*Output of autonom ci merge with a shard missing, shortened*

```
{
  "ok": true,
  "schema": "autonom.campaign-result/v1",
  "created_at": "2026-10-19T08:09:13.787Z",
  "expected_shards": 3,
  "received_shards": 2,
  "missing_shards": 1,
  "status": "incomplete",
  "shards": [
    …
  ],
  "history": {
    …
  },
  "summary": {
    "attempts": 2,
    "passed": 2,
    "failed": 0
  }
}
exit 1
```

*Output of autonom ci finalize with a shard missing, shortened*

```
{
  "ok": true,
  "schema": "autonom.campaign-final/v1",
  "finalized_at": "2026-10-19T08:09:13.963Z",
  "status": "failed",
  "execution_status": "incomplete",
  "publication_status": "not_requested",
  "missing_shards": 1,
  "gates": [
    …
  ],
  "publication": null
}
exit 1
```

*Output of autonom ci pack*

```
{
  "ok": true,
  "out": "three.zip",
  "shard_id": "three",
  "sha256": "14d382fcfdbad1999bda4f3d630cf56b461e32fbd8a1b0cdc21eefc602391af6",
  "run_id": "fr_…"
}
```

*Output of autonom ci merge and ci finalize with every shard, shortened*

```
{
  "ok": true,
  "schema": "autonom.campaign-result/v1",
  "created_at": "2026-10-19T08:09:14.819Z",
  "expected_shards": 3,
  "received_shards": 3,
  "missing_shards": 0,
  "status": "complete",
  …
  "summary": {
    "attempts": 3,
    "passed": 3,
    "failed": 0
  }
}
exit 0
{
  "ok": true,
  "schema": "autonom.campaign-final/v1",
  "finalized_at": "2026-10-19T08:09:15.032Z",
  "status": "passed",
  "execution_status": "complete",
  "publication_status": "not_requested",
  "missing_shards": 0,
  …
  "publication": null
}
exit 0
```

**What to look at.** Both runs that arrived first passed their gates, yet the campaign failed: `missing_shards: 1` holds it back until the third shard comes in. The pack passed twice counts once, because merge keeps one copy of each attempt, so a retried upload does not count twice. A rerun of `ci run` on the same runner is a new attempt, though, and counts as one more shard. `history` groups the runs by flow across shards, which is how a flow that passed on one runner and failed on another shows as flaky.

### Gate on your own rules

**Goal:** loosen one rule, and see what happens to a key Autonom does not know.

1. **Write a rules file.** `max_failed_steps` allows one failed step. `allowed_status` is a misspelling of `allowed_statuses`.

   ```sh
   echo '{"max_failed_steps": 1, "allowed_status": ["passed", "failed"]}' > rules.json
   ```

2. **Finalize the failed campaign again with it.**

   ```sh
   autonom ci finalize ci-2/campaign --rules rules.json
   ```

You should see status `failed` and exit code 1, now with two broken rules instead of three.

*Output of autonom ci finalize with rules, shortened*

```
{
  "ok": true,
  "schema": "autonom.campaign-final/v1",
  "finalized_at": "2026-10-19T08:09:15.228Z",
  "status": "failed",
  "execution_status": "complete",
  "publication_status": "not_requested",
  "missing_shards": 0,
  "gates": [
    {
      "schema": "autonom.gate-result/v1",
      "evaluated_at": "2026-10-19T08:09:15.227Z",
      "passed": false,
      "rules": {
        "allowed_statuses": [
          "passed"
        ],
        "allowed_proof_verdicts": [
          "pass",
          "not_applicable"
        ],
        "max_failed_steps": 1,
        "max_broken_steps": 0,
        "require_evidence_for_failures": true,
        "allowed_status": [
          "passed",
          "failed"
        ]
      },
      "failures": [
        {
          "rule": "allowed_statuses",
          "actual": "failed"
        },
        {
          "rule": "allowed_proof_verdicts",
          "actual": "fail"
        }
      ],
      …
    }
  ],
  "publication": null
}
exit 1
```

**What to look at.** `max_failed_steps: 1` replaced its default, so the one failed step no longer breaks a rule; the other defaults still apply. Autonom v0.36.1 ignores a key it does not know: `allowed_status` is copied into `rules` as if it applied, and checks nothing, so `allowed_statuses` still asks for `passed`. Compare your keys with the rules in the reference below before you rely on a file.

### Publish the result

**Goal:** keep the campaign where the pipeline can find it, and tell a failed upload from a failed test.

1. **Run a campaign and publish it to an empty directory.**

   ```sh
   autonom ci run flows --out ci-3 --env TITLE="Enter your PIN" --publish published
   ```

2. **Run the next campaign, publishing to the same, now full, directory.**

   ```sh
   autonom ci run flows --out ci-4 --env TITLE="Enter your PIN" --publish published
   ```

3. **Do the same with ci finalize, then publish to an endpoint that is down.**

   ```sh
   autonom ci finalize merged-2 --publish published autonom ci publish merged-2 http://127.0.0.1:9/upload
   ```

You should see `published` with exit code 0, then a failed publication with exit code 2 from `ci run`, but 0 from `ci finalize`, and an error from `ci publish`.

*Output of autonom ci run with --publish, twice*

```
{
  "ok": true,
  "campaign_id": "campaign_…",
  "status": "passed",
  "execution": {
    "attempts": 2,
    "passed": 2,
    "failed": 0
  },
  "missing_shards": 0,
  "publication": {
    "publication_status": "published",
    "destination": "/tmp/autonom-ci/published"
  },
  "out": "/tmp/autonom-ci/ci-3",
  "state": "/tmp/autonom-ci/ci-3/ci-state.json"
}
exit 0
{
  "ok": true,
  "campaign_id": "campaign_…",
  "status": "passed",
  "execution": {
    "attempts": 2,
    "passed": 2,
    "failed": 0
  },
  "missing_shards": 0,
  "publication": {
    "publication_status": "failed",
    "error_code": "ci_destination_failed",
    "error": "filesystem destination is not empty"
  },
  "out": "/tmp/autonom-ci/ci-4",
  "state": "/tmp/autonom-ci/ci-4/ci-state.json"
}
exit 2
```

*Output of autonom ci finalize with --publish, shortened, and of autonom ci publish*

```
{
  "ok": true,
  "schema": "autonom.campaign-final/v1",
  "finalized_at": "2026-10-19T08:09:20.451Z",
  "status": "passed",
  "execution_status": "complete",
  "publication_status": "not_requested",
  "missing_shards": 0,
  …
  "publication": {
    "publication_status": "failed",
    "error_code": "ci_destination_failed",
    "error": "filesystem destination is not empty"
  }
}
exit 0
{"ok": false, "error_code": "ci_destination_failed", "error": "campaign publication failed", "destination": "http://127.0.0.1:9/upload", "reason": "<urlopen error [Errno 61] Connection refused>"}
exit 2
```

**What to look at.** A failed publication never changes the verdict: the second campaign still says `passed`. `ci run` exits 2 so the pipeline notices the missing upload, while `ci finalize` exits by the verdict alone, so a script around it has to read `publication.publication_status`. The top-level `publication_status` above still says `not_requested` although the upload failed, so do not test that one. A directory destination must be empty or absent. An HTTP destination receives the finalized campaign as one ZIP file by `PUT`, with an `Idempotency-Key` header holding its SHA-256 digest. Autonom sends no authorization header, so an endpoint that needs one can only take it in the URL, and the URL is not hidden: a successful publication prints it in full as `destination`, `ci run` also stores it in `ci-state.json`, and the error of `ci publish` repeats it. Use a short-lived URL that can write only this one object, and treat the CI log and `ci-state.json` as holding it. On Linux the refused connection reads `[Errno 111]`.

### Use a fresh --out for every campaign

**Goal:** see why a reused directory gives a wrong verdict.

1. **Run a passing flow into ci-2, the directory of the failed campaign.**

   ```sh
   autonom ci run flows/title.yaml --out ci-2 --env TITLE="Enter your PIN"
   ```

2. **Count the runs the gate checked.**

   ```sh
   ls ci-2/campaign/models | wc -l
   ```

You should see status `failed` with exit code 1 for a campaign whose only run passed, and two runs.

*Output of autonom ci run into a reused directory, and of wc*

```
{
  "ok": true,
  "campaign_id": "campaign_…",
  "status": "failed",
  "execution": {
    "attempts": 1,
    "passed": 1,
    "failed": 0
  },
  "missing_shards": 0,
  "publication": {
    "publication_status": "not_requested"
  },
  "out": "/tmp/autonom-ci/ci-2",
  "state": "/tmp/autonom-ci/ci-2/ci-state.json"
}
exit 1
2
```

**What to look at.** `execution` counts only this campaign's run, which passed. The gate also checked the failed run of the earlier campaign, still in `campaign/models/`, and so the verdict is `failed`. This is how Autonom v0.36.1 behaves. Give every campaign its own directory, for example one named after the pipeline's run number.

### Read the refusals

**Goal:** tell why a command did nothing, from the error code alone.

Each refusal below is one error object on stderr, with exit code 2, and no flow runs.

*APP_PIN is not exported*

```
{"ok": false, "error_code": "flow_secret_undefined", "error": "--secret APP_PIN is not present in the process environment", "hint": "Export the variable before running; its value never enters the flow file or artifacts."}
```

*--env TITLE has no value*

```
{"ok": false, "error_code": "flow_command_invalid", "error": "--env takes KEY=VALUE, got 'TITLE'"}
```

*ci publish of a campaign that was merged but not finalized*

```
{"ok": false, "error_code": "ci_destination_failed", "error": "campaign must be finalized before publication"}
```

*No session is running*

```
{"ok": false, "error_code": "no_active_session", "error": "no active session", "hint": "Start one with 'autonom session start'."}
```

**What to look at.** [`flow_secret_undefined`](https://autonom.yatsuk.me/docs/safety/index.md#flow_secret_undefined) names the variable to export on the runner. [`flow_command_invalid`](https://autonom.yatsuk.me/docs/safety/index.md#flow_command_invalid) quotes the `--env` value that has no `=`. [`ci_destination_failed`](https://autonom.yatsuk.me/docs/safety/index.md#ci_destination_failed) from `ci publish` means nothing was published; run `ci finalize` first. [`no_active_session`](https://autonom.yatsuk.me/docs/safety/index.md#no_active_session) asks for a session on the runner's target.

### Reproduce these outputs

From the root of an Autonom checkout at tag v0.36.1 (commit ca2a280), in bash or zsh. The `autonom` function runs that checkout against its test emulator, and the script writes the PIN screen it shows and the four flows. Campaign ids, run ids, attempt ids, digests and times differ on every run, and ids are shortened to … on the page. On macOS the paths start with `/private/tmp`.

*Script that reproduces the outputs*

```sh
# autonom below runs this checkout against Autonom's test emulator, showing a PIN screen the script writes.
S=$PWD W=/tmp/autonom-ci
rm -rf $W && mkdir -p $W/flows && cd $W
export AUTONOM_HOME=$W/.autonom AUTONOM_FAKE_STATE=$W/device.json
autonom() { python3 $S/scripts/autonom.py --adb $S/tests/fakes/fake_adb.py "$@"; }
cat > pin.xml <<'XML'
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<hierarchy rotation="0">
  <node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.example.app" content-desc="" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" bounds="[0,0][1080,2400]">
    <node index="0" text="Enter your PIN" resource-id="com.example.app:id/title" class="android.widget.TextView" package="com.example.app" content-desc="" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" bounds="[80,200][1000,300]" />
    <node index="1" text="" resource-id="com.example.app:id/pin" class="android.widget.EditText" package="com.example.app" content-desc="PIN" clickable="true" enabled="true" focusable="true" focused="true" scrollable="false" bounds="[80,400][1000,520]" />
    <node index="2" text="Continue" resource-id="com.example.app:id/continue" class="android.widget.Button" package="com.example.app" content-desc="" clickable="true" enabled="true" focusable="true" focused="false" scrollable="false" bounds="[80,600][1000,720]" />
  </node>
</hierarchy>
XML
echo "{\"ui_dump\": \"$W/pin.xml\"}" > $AUTONOM_FAKE_STATE
cat > flows/title.yaml <<'YAML'
schema: autonom.dev/flow/v1
appId: com.example.app
name: PIN screen title
id: pin-title
---
- launchApp
- assertVisible:
    selector:
      text: ${TITLE}
YAML
cat > flows/continue.yaml <<'YAML'
schema: autonom.dev/flow/v1
appId: com.example.app
name: Continue button
id: continue-button
---
- launchApp
- assertVisible:
    selector:
      text: Continue
YAML
cat > sign-in.yaml <<'YAML'
schema: autonom.dev/flow/v1
appId: com.example.app
name: PIN sign-in
id: pin-sign-in
---
- launchApp
- tapOn:
    selector:
      description: PIN
- inputText: ${APP_PIN}
- tapOn:
    selector:
      text: Continue
- assertVisible:
    selector:
      text: Welcome
    timeoutMs: 1000
YAML
printf '%s\n' 'schema: autonom.dev/flow/v1' 'appId: com.example.app' 'name: Launch' 'id: launch' '---' '- launchApp' > launch.yaml
autonom session start --serial emulator-5554 --app-id com.example.app > /dev/null
export APP_PIN=271828
# Run a campaign
autonom ci run flows --out ci-1 --env TITLE="Enter your PIN"; echo "exit $?"
ls ci-1
python3 -c 'import json; s = json.load(open("ci-1/ci-state.json")); print(json.dumps({k: s[k] for k in ("schema", "phase", "runs", "errors")}, indent=2))'
# When a flow fails
autonom ci run sign-in.yaml --out ci-2 --secret APP_PIN; echo "exit $?"
python3 -c 'import json; f = json.load(open("ci-2/campaign/final.json")); print(json.dumps(f["gates"][0]["failures"], indent=2))'
# Keep secrets out of the campaign
grep -rl 271828 ci-2 || echo "the PIN is in no file under ci-2"
# Split a campaign across runners
autonom ci run flows/title.yaml --out runner-1 --shard-id one --env TITLE="Enter your PIN" > /dev/null
autonom ci run flows/continue.yaml --out runner-2 --shard-id two > /dev/null
autonom ci merge runner-1/packs/one-1.zip runner-2/packs/two-1.zip --out merged-1 --expected-shards 3; echo "exit $?"
autonom ci finalize merged-1; echo "exit $?"
autonom flow run launch.yaml > run.json
RUN=$(python3 -c 'import json, sys; print(json.load(open(sys.argv[1]))["run_id"])' run.json)
autonom report bundle --run $RUN --out three > /dev/null
autonom ci pack three --out three.zip --shard-id three
autonom ci merge runner-1/packs/one-1.zip runner-2/packs/two-1.zip three.zip runner-1/packs/one-1.zip --out merged-2 --expected-shards 3; echo "exit $?"
autonom ci finalize merged-2; echo "exit $?"
# Gate on your own rules
echo '{"max_failed_steps": 1, "allowed_status": ["passed", "failed"]}' > rules.json
autonom ci finalize ci-2/campaign --rules rules.json; echo "exit $?"
# Publish the result
autonom ci run flows --out ci-3 --env TITLE="Enter your PIN" --publish published; echo "exit $?"
autonom ci run flows --out ci-4 --env TITLE="Enter your PIN" --publish published; echo "exit $?"
autonom ci finalize merged-2 --publish published; echo "exit $?"
autonom ci publish merged-2 http://127.0.0.1:9/upload; echo "exit $?"
# Use a fresh --out for every campaign
autonom ci run flows/title.yaml --out ci-2 --env TITLE="Enter your PIN"; echo "exit $?"
ls ci-2/campaign/models | wc -l
# Read the refusals
(unset APP_PIN; autonom ci run sign-in.yaml --out ci-5 --secret APP_PIN)
autonom ci run flows --out ci-5 --env TITLE
autonom ci merge three.zip --out merged-3 > /dev/null
autonom ci publish merged-3 published-3
autonom session stop > /dev/null
autonom ci run flows --out ci-5
```

## Good to know

- **One directory per campaign.** Name `--out` after the pipeline's run, and merge into a new directory each time. A reused directory adds the runs it already holds to the verdict.
- **Branch on the exit code, then on the fields.** 0 means passed, 1 means a test or a shard is the problem, and 2 means the upload or the command line is. `ci finalize` is the exception: read its `publication.publication_status`, not the top-level `publication_status`, which stays `not_requested`.
- **Keep the bundles.** Every run of a campaign is a verified [report bundle](https://autonom.yatsuk.me/docs/report-bundle/index.md) under `bundles/`, with its own copy of the flow, so a failure can be [replayed](https://autonom.yatsuk.me/docs/replay/index.md) on another machine after the runner is gone.
- **Shard by flow, not by step.** A shard is one run of one flow. Give each runner its own `--shard-id`, and tell the merge how many to expect with `--expected-shards`, so a runner that died shows as `missing_shards` instead of a smaller green campaign. The count is of distinct attempts, so to retry a failed upload send the same pack again, which merge ignores, and after a rerun check that the `shard_id` values in `campaign.json` are the ones you expected.
- **Check packs before the merge.** `ci merge` does not verify bundles, so a pack edited on its way to the merge host is merged as it is. Run `autonom report verify` on each unzipped pack's `bundle/` directory, or compare each pack's `sha256` from `ci pack` with `campaign.json`, before you trust the verdict.
- **Secrets come from the runner's environment.** Set them as the pipeline's secret variables and pass their names with `--secret`. A screen that shows a typed value still puts it into the evidence.
- **Unknown rule keys are ignored.** v0.36.1 does not reject a misspelled key in `--rules`; check the gate result's `rules` the first time you use a file.

## Reference

The commands of a campaign.

| Command | What it does |
| --- | --- |
| `autonom ci run PATH --out DIR` | Runs one flow, or every flow in a directory, on the current session's target, then merges, gates and, with `--publish`, publishes. |
| `--shard-id ID` | Names this runner's packs `ID-1.zip`, `ID-2.zip` and so on. The default is `shard`. |
| `--expected-shards N` | The number of shards the campaign waits for. The default is one per flow. |
| `--campaign-id ID` | Sets the campaign id instead of a fresh one. |
| `--env KEY=VALUE` | Sets a variable the flows read as `${KEY}`. Repeat it for more. |
| `--secret NAME` | Reads a secret from the environment by name. Only the name is recorded. |
| `--rules FILE` | A JSON object of gate rules that replace the defaults key by key. |
| `--publish DEST` | Publishes the finalized campaign to an empty directory or an HTTP endpoint. |
| `autonom ci pack BUNDLE --out FILE` | Verifies a report bundle and packs it as one shard, a ZIP file. Takes `--shard-id`; the default is `shard-1`. |
| `autonom ci merge PACK… --out DIR` | Merges packs into a campaign, one copy per attempt, and counts `missing_shards` by distinct attempts. Checks each report's schema, not the bundle's digests. Takes `--expected-shards`; the default is the number of distinct attempts received. |
| `autonom ci finalize DIR` | Gates a merged campaign and writes `final.json`. Takes `--rules` and `--publish`. |
| `autonom ci publish DIR DEST` | Publishes a finalized campaign to an empty directory or by HTTP `PUT`. |

What `ci run` writes into `--out`.

| Path | What it holds |
| --- | --- |
| `ci-state.json` | `autonom.ci-state/v1`: the `phase`, the `runs` with their flow, run id, status, bundle and pack, and the `errors`. At the end, `final` and `publication` too. |
| `campaign-spec.json` | `autonom.campaign/v1`: the campaign id and the shards it expects. |
| `bundles/<run-id>/` | The report bundle of each run. |
| `packs/<shard-id>-<n>.zip` | The shard pack of each run: its bundle and a `shard.json` descriptor. |
| `campaign/campaign.json` | `autonom.campaign-result/v1`: expected, received and missing shards, the shards, the history by flow and the summary. |
| `campaign/models/` | The report of every run the gate checks. |
| `campaign/final.json` | `autonom.campaign-final/v1`: the verdict, the execution status, the missing shards and one gate result per run. |

The gate rules, with their defaults.

| Rule | Default | A run fails it when |
| --- | --- | --- |
| `allowed_statuses` | `["passed"]` | Its status is not in the list. |
| `allowed_proof_verdicts` | `["pass", "not_applicable"]` | Its proof verdict is not in the list. |
| `max_failed_steps` | `0` | More steps failed than this. |
| `max_broken_steps` | `0` | More steps broke than this. |
| `require_evidence_for_failures` | `true` | A failed or broken step kept no evidence. |

The exit codes.

| Command | 0 | 1 | 2 |
| --- | --- | --- | --- |
| `ci run` | Status `passed`, and the publication, if requested, succeeded. | Status `failed`. | The publication failed, or the command was refused. |
| `ci merge` | Every expected shard arrived. | Shards are missing. | Refused. |
| `ci finalize` | Status `passed`, whatever the publication did. | Status `failed`. | Refused. |
| `ci pack`, `ci publish` | Done. | Not used. | Refused, for example with `ci_destination_failed`. |

The error codes, each with exit code 2.

| Error code | When |
| --- | --- |
| `ci_destination_failed` | The destination is not empty, is not a directory, or did not accept the upload, or the campaign is not finalized. From `ci run` and `ci finalize` it comes inside `publication`, in a successful answer. |
| `flow_secret_undefined` | A `--secret` variable is not set in the environment. |
| `flow_command_invalid` | An `--env` value has no `=`. |
| `no_active_session` | `ci run` has no session to run on. |
| `report_integrity_failed` | `ci pack` was given a bundle that fails verification. |
| `usage_error` | `--out` is missing. |

## Next steps

You can run a campaign, gate it and publish it from a pipeline.

- [Report bundle](https://autonom.yatsuk.me/docs/report-bundle/index.md): Open, verify and annotate the bundle of any run in a campaign.
- [Flows](https://autonom.yatsuk.me/docs/flows/index.md): Write the flows a campaign runs.
- [Replay](https://autonom.yatsuk.me/docs/replay/index.md): Rebuild the state of a failed run on your own device.
- [Agent report](https://autonom.yatsuk.me/docs/agent-report/index.md): Hand a failed run to your agent as compact JSON.