tldr: Every testing vendor now claims self-healing. Most of them mean "we retry a different selector." That is not healing, it is deferring the break, and the difference decides whether your suite survives its first real redesign.
The claim on every pricing page
Read five testing vendor sites in a row and you'll see the same phrase five times. Self-healing tests. Some attach a number to it: 78% of broken locators repaired, 95% self-healing accuracy. The claims sound interchangeable, which is exactly the problem. The word has been stretched over four different mechanisms, and only one of them deserves the name.
If you're evaluating tools this quarter, the mechanism matters more than the number. A suite that "heals" by guessing a new selector will hold up right until your team ships a redesign, and then every guess breaks at once.
The four things vendors call self-healing
1. Retries. The test failed, so run it again. Retries paper over timing flakiness and repair nothing. If this is the mechanism, you don't have self-healing, you have a slower CI pipeline and a flake tax you haven't measured yet.
2. Selector fallback chains. The tool records several locators per element: an ID, an XPath, a CSS path, maybe text content. When the primary fails, it tries the alternates. This works for small DOM shifts. It fails structurally on redesigns, because all the fallbacks anchor to the same DOM that just changed. You've bought redundancy, not resilience.
3. AI locator repair. When a selector breaks, a model looks at the new DOM and picks the node most similar to the old one. This is where the 90-something-percent claims usually live, and it's a real improvement over fallback chains. It also introduces the failure mode nobody advertises, which gets its own section below.
4. Intent re-resolution. The test never stored a selector in the first place. It stored what the step means, "add the item to the cart," and resolves that intent against the live page on every run that needs it. When the UI changes, there is nothing to repair, only a step to re-resolve. We've argued before that the selector was always the wrong foundation; this is what replaces it.
Only the fourth mechanism heals in any meaningful sense. The other three are ways of postponing the same break.
The failure mode nobody advertises
Here's the uncomfortable part of tiers 2 and 3. When a tool guesses a replacement element, it can guess wrong and keep going. The test clicks something, the assertion is loose enough to pass, and your suite stays green while the actual feature is broken.
A hard failure is annoying. A false green is expensive. Healing without an independent check converts visible failures into invisible ones, which is a bad trade for a system whose entire job is visibility.
So the question to ask about any self-healing claim is not "how often does it repair the test" but "what stops it from repairing over a real bug?"
What real self-healing requires
Four things have to be true at once. This is how Passmark, Bug0's open-source testing engine, is built, and the design generalizes to any tool you evaluate.
Steps are stored as intent. A step is a sentence, "select size S," not a locator. Intent survives refactors, framework migrations, and renames because it describes the outcome, not the implementation. This is the core of what agentic testing actually means.
Resolution reads the page semantically. Passmark resolves steps through the accessibility tree, the same structured view a screen reader gets. The sign-in button is found because it looks and behaves like a sign-in button, wherever it moved. (For steps whose meaning lives in pixels rather than structure, canvases and drag targets, there's a separate visual mode.)
Healing re-resolves, then re-caches. Successful steps are cached so repeat runs skip the AI entirely and execute at native Playwright speed. When a cached action fails because the UI changed, the engine falls back to fresh AI execution, re-resolves the intent, and caches the new action. The heal is a one-time cost, not a per-run tax.
An independent layer judges the outcome. This is the guard against false greens. Passmark runs assertions through two models, Claude and Gemini, in parallel, with an arbiter to break disagreements, or a strict mode that fails on any disagreement. A healed step still has to produce a result two models agree actually happened.
The honest caveat
Real self-healing has a boundary, and it's the same boundary all automation has: judgment. When a step heals because your checkout flow changed, something still has to decide whether that change was an intentional redesign or a regression that happens to leave a completable flow behind. No engine can make that call, because the answer lives in your sprint plan, not in the DOM.
This is exactly the kind of QA work that stays human. Under Bug0 Managed, that judgment belongs to your dedicated forward-deployed engineer, who builds your tests on Passmark and reviews every healed step and every failure before it's called a bug or a change. The engine keeps the suite alive. The engineer keeps it honest. That pairing is why Managed suites run at close to a 0% flake rate without drifting into false greens.
One question for your vendor
If you take a single thing from this piece, take the interview question. Ask your vendor: when a step heals, what stops it from healing over a real bug?
A real answer names a mechanism: an independent assertion layer, a consensus check, a human review step. A marketing answer repeats the healing percentage. The first vendor is selling you resilience. The second is selling you a quieter way to break.
FAQs
What are self-healing tests?
Self-healing tests are automated tests that continue working when the application's UI changes, without a human rewriting them. The label "self-healing test automation" covers everything from selector retries to full AI re-resolution of test intent, and the mechanisms differ enough that the label alone tells you little.
How does self-healing work in Passmark?
Passmark stores each step as a natural-language intent and caches the resolved action in Redis. When a cached action fails because the UI changed, the engine re-executes the step with AI against the live accessibility tree, then re-caches the new action. Assertions run through Claude and Gemini in parallel so a wrongly healed step can't silently pass.
Do self-healing tests eliminate test maintenance?
They eliminate the mechanical part: broken locators, timing waits, DOM drift. They don't eliminate judgment. Someone still decides whether a healed flow reflects an intentional product change or a regression, and whether coverage should grow with a new feature.
Can self-healing hide real bugs?
Yes, and this is the most underrated risk in the category. A tool that repairs a locator by picking the most similar element can pick the wrong one and keep the suite green. Look for an independent assertion layer and human review of healed steps before trusting any healing percentage.
What flake rate should a modern suite have?
Teams living with selector-based suites commonly see a quarter of failures come from flakiness rather than real defects. Intent-based resolution removes the main causes, brittle selectors and fixed waits, which is how Bug0 Managed suites run at close to 0% flake. If your suite flakes weekly, the cost is measurable and larger than most teams expect.
