Flaky Tests

A flaky test is worse than a missing one. It teaches everybody to re-run the suite rather than read the failure, and once that habit exists a real failure gets re-run too.

Fix it or delete it. Leaving it is the one option that damages the suite.

The usual causes.

Time. A test that passes except at midnight, or during a daylight change. Freeze the clock.

Order. A test that only passes after another one ran, because it depends on leftover state. Each test must arrange its own world.

Async. Asserting before the interface settled. Wait for the condition rather than sleeping.

Shared state. A module-level cache, a real network call, a file on disk.

Randomness. Random data that occasionally produces a case you never considered, which is sometimes a genuine bug the test just found.