When you need a real browser, and when headless is enough

What headless is now

For most of its life, Chrome's headless mode was a separate, lighter implementation that shared Chrome's rendering engine but not the browser around it. Tests that passed headless and failed in a real window were common enough that "run it headed and see" became a reflex.

That ended with Chrome 112. In the Chrome team's words, "the new Headless mode on the other hand is the real Chrome browser, and is thus more authentic, reliable, and offers more features." The old implementation still exists, as a standalone binary called chrome-headless-shell, published through Chrome for Testing from Chrome 120 onward for jobs where speed matters more than fidelity — screenshots, scraping. For "high-accuracy end-to-end web app testing," Chrome's own guidance is the new mode.

So a modern headless run is Chrome without a window. Same engine, same JavaScript, same layout. If your application misbehaves in Chrome, it will almost always misbehave identically in headless Chromium, and a suite that runs there is not a compromise for the Chromium case. It is the fast, cheap, correct default.

Three things headless cannot tell you

1. What Edge does. Edge is Chromium underneath, so the rendering is the same. What differs is the browser around the engine: enterprise policies pushed by group policy, tracking-prevention defaults, Edge-only features, and the version your desktop fleet is actually on, which is set by IT rather than by the user. None of that is visible from headless Chromium. A run in real Microsoft Edge on real Windows is the only way to see the browser most enterprise employees open every morning.

2. What the operating system does. Fonts, scrollbars, native form controls, date pickers, file dialogs, and screen resolution all come from the OS and the windowing environment, not the engine. A layout that is fine at 1920×1080 on a Linux container can wrap, clip, or push a button below the fold at 1366×768 on a Windows laptop or on a macOS display with different font metrics. Headless runs in a container do not have those variables; a real browser on a real OS does, and a screenshot from one is evidence of what a user saw.

3. What other engines do. Safari runs WebKit. Firefox runs Gecko. They are different engines with different behaviors, and headless Chromium says nothing about them at all. If your audience has meaningful Safari or Firefox share, you need to run on those engines. Playwright drives Chromium, Firefox, and WebKit natively, and for teams already on Playwright that coverage is a real strength that a Chromium-only executor does not have.

Who actually has which browser

StatCounter's desktop figures for August 2026 put Chrome at 73.3% worldwide, Edge at 10.5%, Firefox at 5.3%, and Safari at 5.2%. In the United States the picture shifts: Chrome 65.6%, Edge 12.8%, Safari 10.8%, Firefox 5.9%. Edge is the Windows default, and an internal enterprise application whose users are all on managed Windows machines will see an Edge share far above the population average — how far, only your own analytics can say.

Two things follow. Chromium-family browsers cover the large majority of desktop users almost everywhere, so a suite that is thorough on headless Chromium and confirmed on real Edge has covered most of the risk for most enterprise applications. And Safari's share, especially in the US and on consumer-facing products, is large enough that "we tested on Chrome" is not an answer for a public checkout. Check your numbers before you decide; the population figures are a starting point, not a substitute.

A decision rule

Real browsers cost more per step — screenshots and commands travel across a network, and sessions are metered by the grid. That cost is worth paying deliberately, not on every run.

• Every run: headless Chromium. Fast, parallel, cheap, and, since Chrome 112, the real engine. This is where regressions get caught.
• Release candidates and sign-off: real Chrome and Edge on the OS your users have. Same tests, real browser, exact version recorded. This is where evidence gets produced.
• Where the audience justifies it: WebKit and Gecko. Through a framework that drives those engines or a grid that offers them. This is where engine-specific defects get caught, and it is a separate decision from the first two.
• Pin versions when the fleet is pinned. If IT holds desktops on a particular Edge release, test on that release, not on Latest. A grid that offers versions back several years exists for exactly this reason.

The mistake to avoid is treating the second and third as one requirement. "Cross-browser" bundles them, and teams end up either paying for real-browser runs on every commit or skipping real browsers altogether because the full matrix looked too expensive. Split them, and each is affordable.

What this looks like in Spec2RunAI

Spec2RunAI, the test executor inside Spec2TestAI, runs on headless Chromium in its own cloud workers by default. The same plain-English test cases, unchanged, can also run in real Google Chrome or Microsoft Edge on Windows 11, Windows 10, or macOS through the customer's own BrowserStack Automate account, with the version pinned anywhere from Latest back to 100 and the screen resolution chosen per run. The Run Evidence Report records the exact browser, version, platform, and screen that ran, alongside verification coverage for every step.

Two limits, stated plainly. Real browsers through this route are Chrome and Edge on desktop; Safari and Firefox are not offered, and a team that needs WebKit or Gecko evidence should get it from Playwright or a grid session, which we would say on a call. And real browsers run against public URLs; internal applications behind a firewall run through the local agent, in its own Chrome, which covers the Windows-desktop case for internal apps in a different way.

Sources

• Chrome for Developers, "Chrome's Headless mode gets an upgrade" and the chrome-headless-shell documentation — new Headless mode as the real Chrome browser from Chrome 112; chrome-headless-shell published via Chrome for Testing from Chrome 120.
• StatCounter Global Stats, "Desktop Browser Market Share," worldwide and United States, August 2026. Monthly figures; check the current month before quoting.
• Playwright documentation, "Browsers" — supported engines: Chromium, Firefox, and WebKit.