Rainforest QA changelog
Rainforest QA changelog
app.rainforestqa.com

Schedule Run Groups up to multiple times per day

 

New

 

Improvement

  

You can now schedule Run Groups to run automatically up to multiple times per day, an option that was previously only available via the API.

image.png

Other recent improvements:

  • You can now perform bulk actions on the Tasks page
  • Released a new platform: iPhone 12 iOS 16 Beta
  • From test result details, you can now open all screenshot images in new tabs for closer analysis.
  • Fixed bug where some bulk operations (delete, tag) for larger test suites were timing out.

New workflows for managing test results

 

New

 

Improvement

  

When it comes to test automation, it’s inevitable that tests sometimes fail. It happens for many reasons: bugs, UI changes, environment issues, random errors. Successful teams use a clear process to keep their test suites maintained and healthy, investigating and addressing all failures quickly to prevent them from recurring.

We’re excited to announce new features that makes it easy for you to stay on top of test failures and collaborate on test results with your team.

Failure status image.png

In the Results page, see which of your test failures still need to be resolved in the new “Failure Status” field, and mark what’s been addressed as Resolved.

Assigning test failures

image.png

When failures happen, it’s important someone is responsible for investigating and resolving them. Assign (or re-assign) team members to resolve each failure.

New Tasks page

image.png

Keep track of all test failures that need attention on the new Tasks page, a built-in todo list.

Broken test reminders

reminders.gif

We always recommend disabling broken tests so they don’t cause noisy results. However, if you allow broken tests to keep running, we now remind you that the test has open failure tasks from previous runs. No more guesswork; you’ll immediately know the cause of the test failure.

Summaries of resolved run failures

image.png

See at a glance in the Results page which runs have been cleared by your team, and those that still have failures that need attention.

Other recent improvements:

  • You can now remove test priority tags and bulk-change test priority

Webhook error notifications

 

New

 

Improvement

  

When a test fails due to webhook (mis)configuration, Rainforest now shows a notification describing the issue and providing next steps to resolve the issue.

image.png

One click to enable screenshots

 

New

  

Formerly, when choosing an action like Click, you'd then have to click the Element icon to enable screenshot capture.

Now when you choose any action that requires a screenshot, screenshot capture will be enabled immediately upon choosing the action. No more having to click the Element icon.

One click screen shots.gif

Other recent improvements:

  • The UI has been updated to allow bulk deletion of Run Groups and Features.
  • In the Results page, you can now click the Pass, Fail, and No Result chicklets to filter the results.

image.png

Sentry integration for better error reporting insights

 

New

 

Improvement

  

You can now use the rainforest-run-info NPM package (also available on Github) to get the run ID and test ID in your client-side code for every test you execute.

This can connect events in Sentry and other logging tools to your Rainforest test runs. When investigating errors, your developers will be able to go directly to the appropriate reproduction videos in Rainforest.

We also now add HTTP headers (X-RAINFORESTQA-TEST-ID, X-RAINFORESTQA-RUN-ID, and X-RAINFORESTQA-RUN-TEST-URL) to all outgoing requests during testing so you can access this information in more places.

Freeze-frame VMs

 

New

 

Improvement

  

It’s difficult to grab screenshots of elements when they’re moving, temporary, or require a hovering cursor (like a drop-down menu).

We’ve made it easier with the introduction of VM freeze-frames. Press the Esc key or click the new Freeze Screen button to create a frozen image of the VM in which you can capture screenshots. To unfreeze the image, simply press Esc again or click the Unfreeze Screen button.

This feature freezes the image on screen, but not the VM itself. The VM will keep running in the background, so when you unfreeze, you’ll see the latest state of the machine.

Freeze Screen.gif

Add comments to test steps

 

New

 

Improvement

  

Comments are plain-text snippets you can add to your test steps to provide context, clarification, or explanation. They don’t change the way tests work — they simply provide information to make tests easier to read and understand.

You can add comments to your test steps just like you’d add a test action:

comments.png

Other recent improvements:

In the Tests page:

  • It’s easier to create and manage saved filters.
  • Searches entered in the Search test steps now search Visual Editor tests, not just Plain Text tests.
  • You can now use Shift + click to bulk-select items.

In the Run Groups page:

  • There are two new columns of information to help you see important information at a glance: Maximum Failure Retries and Environment.
  • It’s easier to edit settings for individual run groups.

Usability improvements

 

New

 

Improvement

  

Our product teams have spent a lot of the past month focused on improving the usability of the platform so you can manage your testing more easily and quickly.

2022-02-16_17-26-46 (1).gif

Usability improvements include:

  • Pre-fill the element name field if the captured screenshot contains text.
  • Redesigned the Tests page to make it easier to navigate and customize your list of tests.
  • Sped up the load times of the Tests and Results pages.
  • Added more bulk-edit options including setting tests to draft, removing tests from a feature, and more.
  • Added breadcrumbs so it’s easier to see where you are in your Run Groups and Features.
  • Updated failure categorization tags to include avatar of the test reviewer.
  • Consolidated toggles for Tests, Features, and Run Groups into a single view.
  • Updated tables in the Tests, Features, Run Groups, and TWS pages to show your total number of records.

Other recent improvements:

Simplified the Tests page

 

Improvement

  

We've simplified the appearance and controls of the Tests screen to make it easier for you to navigate and customize your list of tests.

simple-test-list.gif

Other recent improvements:

  • Added support for Android App Bundles (.aab files)

Automatically retry failed automated tests

 

New

 

Improvement

  

A flaky automated test is one that intermittently fails for no obvious reason: there haven't been any changes to the test or to the tested application.

These inconsistent failures — which often result from unpredictable testing conditions, like a web page loading more slowly than usual — can be disruptive to the QA process and lower trust in your tests.

That's why we’re excited to introduce automatic test retries, the best way to save your team the time and frustration of dealing with flaky tests.

With test retries configured, Rainforest will automatically retry failed tests up to X times (defined by you) or until a test passes, whichever happens first.

You can configure test retries in global settings or when creating a test run or run group.

2022-01-12_11-06-03.png

Other recent improvements:

  • Added a warning when a test variable currently being tested is about to be previewed in the Visual Editor, potentially causing an inadvertent test failure.
  • Updated test failure categorization tags to include the assigned reviewer.