When AI Fixes a Test, How Do We Know It Fixed the Right Thing?

The useful part

While exploring AI-assisted testing, I started thinking more about what actually happens when a test breaks. Tools like mabl, Testim, and Functionize are approaching self-healing in different ways, and I can see why it's becoming useful as test suites get bigger. I keep coming back to one thing: just because a healed test passes again, it doesn't always mean it was fixed correctly.

How it works

• While exploring these ideas through X360 AI Tech, I've also been looking at how much context is needed for a test change to actually make sense.
• If a locator breaks and AI finds another element that makes the test pass, we still need to know whether it's actually the element the test was meant to use.
• There's a big difference between "the test passed again" and "the test passed again, and we know it was fixed correctly." I think that difference becomes more important as test suites grow.

Details worth keeping

When that isn't clear, I think QA review is still important.