Running tests
Everything starts on New Execution.
1. Choose what to test
| Browse Project Tree | Test cases from the core product — pick a requirement, a story, or individual cases. |
| Test Executor Repository | Test cases you authored or imported here — pick a project, then a folder, then tick the ones you want. |
| From Suite | A saved suite. |
2. Target URL
The application under test. Recently used URLs appear as chips. Test Executor tells you whether the URL is publicly reachable — that determines whether you can use the cloud engine or need a local agent.
3. Device
Desktop (1280×900 by default) or a mobile/tablet viewport. Use Customize to set an exact desktop size; it is applied to the run whether it executes in the cloud, on a local agent or on a real browser. (Before version 2.11.84 a custom desktop size was shown but not applied: those runs used 1280×900.)
4. Execution mode
| Mode | Use when |
|---|---|
| Cloud | The application is reachable from the internet. Nothing to install. |
| Local Agent | The application is behind a VPN or on an internal network. Runs on your own machine. |
5. Browser
For a cloud run, choose which browser runs it.
| Browser | Use when |
|---|---|
| Built-in (default) | Most runs. A headless Chromium inside the cloud worker: the fastest option, and the one every earlier run used. |
| Chrome or Edge | You need evidence from a real browser on a real operating system, for example because your users are on Microsoft Edge on Windows. The run uses a real browser on BrowserStack, and you pick the platform, the browser version and the screen size. |
A run on a real browser takes a little longer: about five seconds more to start each test case, and each step is slightly slower because every screenshot travels over the network. macOS sessions are slower again, roughly twice as long per step as Windows. The engine, the steps and the verdicts work exactly as they do on the built-in browser.
With Chrome or Edge picked, three more choices appear. They list only what your BrowserStack account actually offers:
| Choice | Options |
|---|---|
| Platform | Windows 11 and 10, and current macOS releases back to Monterey. |
| Version | Latest, Previous or Two back (each shows the version number it means on that platform), Beta where BrowserStack offers one, or any specific version back to 100. Older macOS releases cannot run the newest browsers, so their lists stop earlier; a version a platform does not have is never offered. |
| Screen | The screen resolution of the remote machine, 1920×1080 unless you choose otherwise. Keep it at least as large as the desktop size under Device; the card warns you if it is smaller. |
The runs list, the run page and the report say where each cloud run ran: Cloud · AgileAI Labs (the built-in browser on Spec2RunAI's own cloud workers) or Cloud · BrowserStack (a real browser). For BrowserStack runs the report records the exact browser version that ran, the platform and the screen.
In a parallel run, each test case gets its own browser. If the run has more test cases than there are free BrowserStack sessions, the rest wait their turn, and each still starts in a fresh browser. The run page, the runs list and the report all show which browser was used, with its version.
6. Data (optional)
Bind a dataset to run the selected test cases once per row. See Data-driven testing. Leave it as None and each test case runs once.
7. Timeouts
Step timeout (default 30s) caps a single step. Test case timeout (default 300s) caps the whole test case. Raise the second one for long flows — a fifteen-step checkout on a slow environment can legitimately need more than five minutes.
8. Launch
The estimate above the button shows roughly how many AI engagements the run will use. It is an upper bound: steps resolved from memory skip the AI call entirely, so real usage is often lower.