Platform / Spec2RunAI / User guide / Reading results

Spec2RunAI user guide · updated 22 October 2026

Reading results

A run opens on a live summary — pass, fail and error counts, elapsed time and progress. Click any test case for its detail. For a printable record of the whole run, click Evidence report — see Reports.

The step strip

Each step appears as a chip with its number, text, duration and a status dot. Click one to see the screenshot taken at that moment and the AI's reasoning. Badges flag anything notable — 🩹 self-healed, 🧠 resolved from memory, ⚠ ambiguity warning.

Screenshots

Every step is captured. Visual Diff and Side-by-Side compare consecutive steps, which is the fastest way to see what a click actually did.

The AI Decision Inspector

For each step this shows what the engine did and why: the action, the selector or coordinates it used, its confidence, its reasoning, and the alternatives it considered and rejected. When a step goes wrong, this is where you find out whether the engine misunderstood the step or the application misbehaved.

Failure triage

Every failure is classified automatically, with evidence:

VerdictMeaningWhat to do
APP_DEFECTThe application misbehaved — JavaScript exceptions, server errors, wrong screenRaise a defect. A ready-to-paste report is generated.
TEST_DEFECTThe test no longer matches a working applicationApply the suggested rewrite.
ENVIRONMENTNetwork, DNS or availability troubleCheck the environment and re-run.
DATAThe test data was rejected — validation, credentials, duplicatesFix the data.

Self-healing

When a test case fails, the engine retries it once with adaptations — rewording the failed step from its own diagnosis, bypassing cached selectors, waiting longer. A healed pass is labeled "PASS (self-healed after 1 adaptive retry)" and carries the suggested permanent fix.

Healing never changes the verdict criteria It changes how the engine tries to do something, never what counts as success. Your expected results are untouched. A retry that still fails reports a genuine failure.

Script-defect advisories

Sometimes a step passes for the wrong reason. If your step quotes 'Complete Purchase' but the button actually reads 'Proceed to checkout', the engine finds it anyway — and flags it. The step passes, and you get a warning with a suggested rewrite. Worth fixing: right now that step only works because the AI rescued it. These advisories are collected for review in Autonomous Tuning, with the evidence and a suggested correction.

Try it on your own tests

Ten test cases. Untouched. We run them and send the evidence report.