What “plain English” actually means in test automation

The phrase is doing four different jobs

"Plain English" appears on nearly every codeless test automation site, including ours. It is a useful phrase because it points at something real: tests should be readable by the people who understand the business. It is a slippery phrase because it covers at least four different capabilities, and a tool can honestly claim it while delivering only one.

This piece separates the four, gives you a way to tell which one you are being sold, and is candid about which ones Spec2RunAI does and does not do.

Sense 1 — Readable output

The test is stored as code, but the tool renders it as prose. A recorder captures clicks and keystrokes and displays them as "Click the Submit button" rather than page.click('#btn-submit'). The readability is a view layer over an artifact that is still fundamentally a script.

This is genuinely useful for review. It does not change who can author or maintain the test, because the underlying artifact is still the selector-bound script, and when it breaks the fix happens below the prose.

Sense 2 — Constrained authoring vocabulary

You write in English, from a supported list of phrasings. The tool documents a set of statement patterns — navigate to, enter [value] into [field], verify [element] contains [text] — and tests are composed from them. A non-engineer can author, but only in the tool's dialect.

This is the most common meaning of the phrase in the category, and it is a real capability. The trade-off is at migration: an existing manual test written by a tester who never expected a machine to read it has to be translated into the vocabulary before it can run. That translation is per test, and it is the conversion project that stalls most migrations.

The test for this sense is a single question: can we see your command reference? If there is one, you are in sense 2. That is not a criticism — a documented grammar is a legitimate design — but it should be understood as what it is.

Sense 3 — Natural-language authoring with interpretation

You write in English as you would write it, and the tool interprets. No vocabulary to learn; an AI reads the step and works out what to do. This is the promise most of the category has moved toward since 2024, and it is a real advance over sense 2.

The question it raises is what happens when the interpretation is wrong, or when the step is ambiguous. "Check the account" could mean verify a balance, open an account page, or tick a checkbox. A tool in sense 3 has to do something with that, and the something varies: some guess silently, some ask, some refuse. Which one matters enormously for whether you can trust the result.

Sense 4 — Existing tests run as written

The test you already have runs without being changed. Not authored in a new tool, not translated into a vocabulary, not recorded fresh. The manual test case a tester wrote three years ago — business jargon, three actions on one line, no expected result — is the input, unmodified.

This is the narrowest sense and the one with the largest economic consequence, because it removes the conversion project rather than making it easier. It is also the hardest to deliver, because it requires sense 3's interpretation plus the ability to handle whatever a human actually wrote rather than what a tool would have preferred they write.

Senses 1 through 3 make writing new tests easier. Sense 4 makes the tests you already have usable. They are different products, and the phrase covers all of them.

How to tell which you are being sold

Three questions, in order. Each one eliminates a sense.

1. "What does the stored test look like?" If it is code with a prose view, sense 1. If it is prose, continue.
2. "Can we see the command reference?" If one exists, sense 2. If the answer is that there isn't one because the tool interprets free text, continue.
3. "Here are ten of our manual test cases, untouched. Run them." If they need to be reformatted, restructured, or rewritten first, sense 3. If they run, sense 4.

The third question is the one that matters and the one vendors least want asked, because it cannot be answered with a demo. It has to be answered with your tests.

Where Spec2RunAI sits, and where it does not

Spec2RunAI, the test executor inside Spec2TestAI, is built for sense 4. Existing manual test scripts run as written: no required keywords, no reformatting, business jargon and multi-action lines handled as the tester wrote them, expected results optional and per step. Ambiguous steps are flagged, rewritten, and offered for one-click acceptance rather than guessed at silently. Proven on real customer scripts, and the ten-untouched-test-cases challenge at the bottom of this page is the sense-4 test applied to us.

What it is not: a recorder, and not a tool for teams who want to author a new suite from scratch in a guided vocabulary. If you have no manual tests and want a structured authoring environment with autocomplete, sense 2 tools do that well and we would say so. Our advantage is specific to teams that already have the tests and cannot afford to rewrite them — which, per the survey data, is most teams that have started automating and stalled.

A note on the word "no-code"

"No-code" and "plain English" are used interchangeably and are not the same claim. No-code says you will not write a programming language. Sense 2 tools are no-code and still require learning a vocabulary. A test written in a constrained grammar is not code, but it is not the English a tester would have written either. When the phrases are stacked — "no-code, plain-English, AI-powered" — ask which of the four senses each one refers to. Usually it is one sense described three ways.

Sources

• This piece is an evaluation framework rather than a research summary; it cites no external figures. The claim that most teams with automation have stalled below half coverage draws on PractiTest, State of Testing Report 2024, discussed in our migration guide.