Platform / Spec2RunAI / User guide / Troubleshooting

Spec2RunAI user guide · updated 22 October 2026

Troubleshooting

A step clicked the wrong thing

Almost always ambiguity. Open the AI Decision Inspector for that step and read the reasoning and the rejected alternatives — it usually tells you immediately what it thought you meant. Then add a location or quote the exact label: "Click the 'Add to Cart' button on the 'Roaster's Choice Box' panel".

The test passes but I don't think it should

Check the overall expected result. If it is vague — "the page loads" — almost anything satisfies it. Make it specific and observable: "The order confirmation page shows 'Thank you for your order!'".

Also check for a script-defect advisory on the run. A step can pass because the engine rescued a label that no longer exists.

The test case timed out

The default cap is 300 seconds for the whole test case. Long flows on slow environments legitimately need more. Raise the TC timeout on New Execution. If it times out consistently at the same step, look at that step rather than the timeout.

A step failed waiting for text that is on the screen

The wait polls the rendered text. Check for a typo, and check the text is not inside an iframe or a closed shadow root. Try waiting for a shorter, distinctive fragment rather than a whole sentence.

Runs are failing since the balance ran out

When AI engagements are exhausted the header flashes red and runs fail. Contact support@agileailabs.com to replenish.

A run on Chrome or Edge ended in ERROR before the first step

The error says why. "did not provide Microsoft Edge on Windows 11 within 120 s" means every BrowserStack session was busy: run again, or run fewer test cases at once. A message about credentials means the saved BrowserStack key no longer works: use Test connection or Replace key on the Secrets page. The same run on the built-in browser is unaffected.

My data-driven run only ran once

Only test cases containing a {{data: token are iterated. If nothing was tokenized, everything runs once — by design, so a token-free login test does not burn N times the engagements for N identical results. The badge on the test case detail tells you how many steps actually used data.

The wrong value went into a field

Open the Datasets preview for that column. A dotted underline marks a cell that was reformatted on the way out — hover to see the original. If a ZIP or ID lost or gained digits, change the column type in the grid and re-save.

A local agent shows offline

Confirm the agent process is running and can reach the platform. Agents report a heartbeat; a stale one means the process stopped or lost network.

The evidence report shows steps as Not verified

Those steps ran and passed, but nothing confirmed their expected result. Most often the step has no expected result: add one to every step you need to sign off. If the report says the verifier could not complete its check, re-run the test case. Runs from before engine 2.10.131 show no verification record for some steps; a fresh run records them.

A report says "Screenshot no longer available"

Screenshots are kept for 90 days. The report still holds every step, state and finding. For evidence you need to keep longer, download the PDF within 90 days of the run.

Try it on your own tests

Ten test cases. Untouched. We run them and send the evidence report.