Developer Tools
flakestat
An open-source flaky test detector for any language. One static Go binary, no SaaS account, and test results never leave your machine.
Sep 2026
Details
Problem statement: a flaky test passes and fails on the same code. The good tooling for finding them is hosted and paid; the open-source alternatives are a scatter of per-language plugins.
- Reads the JUnit XML every major test runner already writes, so it works with pytest, Jest, go test, RSpec, PHPUnit and the rest with no plugin and no framework lock-in.
- Scores inconsistency rather than failure rate. A test that fails every time isn't flaky, it's broken, so it scores zero and is reported separately instead of topping the list and wasting the time of whoever is hunting nondeterminism.
- Every verdict carries a confidence level derived from how much evidence exists, and classification uses a lower bound on the score, so a small sample can never claim high confidence no matter how flaky a test looks.
- Validated against three external projects under a contract written before any experiment ran. The strongest result: at the commit before ConduitIO's own deflaking fix, three of the four tests their issue named scored flaky at 0.67 with high confidence; at the fix itself, all four were stable. Their contributors found the bug and wrote the repair — flakestat was handed observations from both sides and separated them.
