Verdict overrides

An AI agent or judge decides whether each run passed. Sometimes it’s wrong, or the run failed for a reason that doesn’t count, such as a sandbox that was down. A verdict override lets a person correct the result without changing what the engine recorded.

The run keeps its original status. The override sets its effective status, and from then on the effective status is what counts everywhere: the CI gate, Triage, Analytics, Rerun failed and the status shown in the app.

Override a verdict

1. Open the finished run.
2. Under Verdict wrong? on the Overview tab, click Mark as passed (on a failed run) or Mark as failed (on a passed run). An errored run offers both.
3. Optionally say why. The note is shown to everyone and kept with the override.
4. Click the button again to confirm.

The run’s status now reads Passed with overridden · was failed next to it, and the override shows who made it, when, the original result and the note. Undo override puts the original result back.

Rules

• Only finished runs can be overridden. A pending or running run returns 409.
• The override status is passed or failed. There is no override to error.
• Marking a run as what it already is (for example passed on a passed run) removes any override.
• The note is optional, up to 1,000 characters.
• Every set and clear is logged in the run’s extras.override_history (the last 20 entries), with who and when.
• You need the editor role.

What an override changes

Place	Uses the effective status?
CI gate (GET /api/batches/{id})	Yes. An override wins over a score threshold, too.
Triage	Yes. A run marked as passed leaves its failure cause.
Rerun failed	Yes. A run marked as passed is not re-run.
Analytics	Yes, in every chart and count.
Run status pills and the run page	Yes
Runs page status filter	No. The filter uses the original status, so an overridden run is listed under its original status (its pill still shows the override).
Alerts	No. Alerts were sent when the run finished; an override doesn’t send or retract one.
Retries	No. A retry that was already queued still runs.

Approving a visual change does this for you

When a run fails only because its screenshot changed, approving the change as the new baseline also overrides the run to passed, with the note “Visual change approved as the new baseline”. See Assertions and visual regression.

Use the API

Endpoint	Body	Role
POST /api/run-actions/{run_id}/override	{"status": "passed" | "failed", "note": "…"}	editor
DELETE /api/run-actions/{run_id}/override	none	editor

Both return the full run, including override and effective_status.

bash

curl -fsS -X POST "$PLAINWRIGHT_URL/api/run-actions/$RUN_ID/override" \
  -H "Authorization: Bearer $PLAINWRIGHT_TOKEN" -H "Content-Type: application/json" \
  -d '{"status": "passed", "note": "Payment sandbox was down during this run"}'

PowerShell

Invoke-RestMethod -Method Post -Uri "$env:PLAINWRIGHT_URL/api/run-actions/$env:RUN_ID/override" `
  -Headers @{ Authorization = "Bearer $env:PLAINWRIGHT_TOKEN" } -ContentType 'application/json' `
  -Body '{"status": "passed", "note": "Payment sandbox was down during this run"}'

Part of the returned run

{
  "status": "failed",
  "effective_status": "passed",
  "override": {
    "status": "passed",
    "note": "Payment sandbox was down during this run",
    "by_user_id": "…",
    "by_name": "Priya Shah",
    "at": "2026-10-20T15:40:12.004Z",
    "original_status": "failed"
  }
}

Next steps

Root cause and run chatCheck whether a failure was the app's fault before you override it.

Judges and scoresHow judged verdicts and scores are produced.

CI webhook and gatesHow the gate reads overrides, retries and quarantine.