A flaky automated test is one that intermittently fails for no obvious reason: there haven't been any changes to the test or to the tested application.
These inconsistent failures — which often result from unpredictable testing conditions, like a web page loading more slowly than usual — can be disruptive to the QA process and lower trust in your tests.
That's why we’re excited to introduce automatic test retries, the best way to save your team the time and frustration of dealing with flaky tests.
With test retries configured, Rainforest will automatically retry failed tests up to X times (defined by you) or until a test passes, whichever happens first.
You can configure test retries in global settings or when creating a test run or run group.
Other recent improvements:
- Added a warning when a test variable currently being tested is about to be previewed in the Visual Editor, potentially causing an inadvertent test failure.
- Updated test failure categorization tags to include the assigned reviewer.