50: Flaky Tests and How to Deal with Them
Anthony Shaw joins Brian to discuss flaky tests and flaky test suites.
- What are flaky tests?
- Is it the same as fragile tests?
- Why are they bad?
- How do we deal with them?
- What causes flakiness?
- How can we fix them?
- How can we avoid them?
- Proactively rooting out flakiness
- Test design
- GUI tests
- Sharing solutions
Special Guest: Anthony Shaw.
Sponsored By:
Links:
- Dropbox article on flaky tests
- Microsoft article on flaky tests
- pytest-rerunfailures: a py.test plugin that re-runs failed tests up to -n times to eliminate flakey failures
- pytest-randomly: Pytest plugin to randomly order tests and control random.seed
- pytest-random-order: pytest plugin to randomise the order of tests with some control over the randomness
- math.isclose()
- numpy.isclose()
- pytest.approx() — approx
- Anthony's testing article on RealPython
- Ghost Inspector
- wily: A Python application for tracking, reporting on timing and complexity in tests
Creators and Guests
![50: Flaky Tests and How to Deal with Them](https://img.transistor.fm/GmBvN9TVwjbBQ8IVgvx3QkEMZpthq0tK-iJBh94ymf8/rs:fill:800:800:1/q:60/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS9zaG93/LzQzMjkxLzE2OTUy/NDgxMTctYXJ0d29y/ay5qcGc.webp)