New and Improved Test Designer

A new Test Designer interface that makes the process of submitting test writing requests as user-friendly and intuitive as possible.

With a revamp of the Test Designer, our goal is to make the experience of submitting test writing requests easy and fun for you, with minimal hiccups.

The new interface allows for a more streamlined experience from creating your first request to writing your test outlines to tracking the progress of your request. Improved messaging, helpful tips and guidelines, prominent calls-to-action and an easy-to-use interface are all part of the new experience.


With this, you can fill out test outlines very quickly, edit at your convenience and easily help test authors when they are blocked at any moment in-app itself.

tw2.png tw3.png

Cleaned Up All Runs Page

We are proud to announce cleaned up All Runs page

A new and improved All Runs page with an emphasis and call-to-action to categorize failures and other visual improvements.

You can now track the percentage of failures you've categorized in each run. This helps both ICs and PMs/ESs by calling out runs that have not been fully categorized. Categorizing failures is important as it allows you to improve your results workflow, both by enabling the most appropriate action to be taken against a failure, and by improving reporting through the QA Visibility report.

On the All Runs page, there is now a column called “Failures Categorized”. This column shows the percentage of failures that have been categorized in the run. Upon categorizing 100% of failures, a green check mark appears indicating that all failures have been reviewed and categorized in the given run.


Separate Email Notification under Settings

Users are now able to personalize what email notifications they want, and users who had previously opted out of ALL emails will now receive notifications on new features, including the QA Visibility Report.

The way we previously organized the hierarchy of Email Notifications, users could opt out of all Email Notifications, but this meant that any new functionality we added (Test Designer, Rewrite, QA Visibility Report, etc…) clients would not receive email notifications on the new features.

Now, users will start opted IN to new functionality and still have the ability to remove the notifications desired.

QA Visibility Report - Tests Edited and Tracking Links

The QA Visibility Report now shows Tests Edited and has tracked links to takes you directly into the Rainforest app.

Without "Tests Edited", readers of the Report weren't able to get a good sense of what happened with tests being edited. We only showed information for Tests Created, which only showed half of the story.

The report now has tracked links and a new section for Tests Edited.


Auto-Close Blocked Test Writing Runs

Previously, all test writing blockers needed to be resolved before a test writing request could be could be closed. This requirement led to slower turnaround time for test writing runs and subsequently prevented users from submitting their next request until the previous one was returned.

Now, runs will autocomplete after four days if any blocked tests remain. The four day turnaround time is calculated from the moment a user clicks ‘submit’ for their request. This benefits users in that they can submit their next test writing request to get tests built for their application while they also work to resolve any outstanding blockers.

Users still have the opportunity to address blocked test during a run via the Test Level Status functionality. For any tests that remain blocked after a run is completed, Rainforest has implemented the following to draw attention to each blocked test:

  • Each blocked test will be in a disabled state
  • The title of each blocked test is appended with "BLOCKED" to identify tests in need of attention
  • Internal notes of each blocked test will contain the blocker comment plus the original inputs

When users are ready to resolve blocked tests after a run has completed, they simply need to supply additional information to the test author by submitting the test to the rewrite flow. They can then track the individual progress of that test on the rewrite tab.

Updates to Slack Notifications

Improved Slack result notifications to call attention to failure categorization.

Today, Rainforest supports the ability to notify clients about results directly in Slack, as they happen.

Clients who have this setup enjoy having results come in live into a channel where multiple people can see them and comment - it contributes to Rainforest becoming an embedded part of the team’s workflow.

This update makes the run completion notifications more helpful. Rainforest cut out unnecessary info where it wasn’t needed, added tester comments for failed runs and included calls to action to make it more obvious what you should do next, i.e. categorize failures from failed runs. We also threw in some emoji, so we’re not so serious all the time.

All customers who have Slack set up will see these changes. This will be followed by in-app awareness of slack so that other customers integrate Rainforest with Slack as well.

New Slack Notification (Rainforest test account)


Old Slack Notification (Rainforest test account)


Improved Run Summary Page

An Improved Run Summary page, now with aggregated stats on the failure categories for a run, a “delightful” toucan that prompts you to categorize failures, as well as percentage stats on the results from the run.

The process of viewing, triaging and acting on results is a core part of the Rainforest workflow. It’s why you use us. An essential part of this workflow involves the Run Summary page.

The Run Summary page allows you to easily review and categorize failures, as well as view aggregated data on the health of the run. The improvements are all done to make this process smoother. You can now:

  • View the % of passing or failing tests within the run
  • View how many tests you've categorized within the run
  • View aggregated stats on the failure categories


Improved Run Completion Email Notifications

Improved run completion email notification - focus on results and needed actions after reviewing a run.

For many of our customers, getting notifications is the first step towards reviewing their Rainforest results. Email notifications are one of the key features, so we revamped the run completion emails to streamline the information shown and bring attention to the value of categorizing failures.


NEW: QA Visibility Email Report

*The Rainforest QA Visibility Report is a weekly email sent on Mondays that highlights the number of tests created, tests run, tests passed, tests failed, and bugs caught for the last 2 full weeks. *

It shows the number of tests created, tests run, tests passed, tests failed, and bugs caught (provided you mark bugs in Rainforest) and should act as a conversational piece for your team to connect coverage being created and run and triaged in Rainforest to your releases.


NEW: Search & Filter on All Runs page

Clients can now search and filter through their entire run history to better parse results and keep track of results over time.

The process of triaging and acting on test results is a core part of the Rainforest results workflow. An essential part of this workflow is the All Runs page. It’s where the process of parsing through test results starts within the UI.

While the All Runs page used to be a chronological list of runs that was often opaque and hard to parse through, the Search and Filter functionality will allow clients to parse through their entire run history in order to better navigate to runs that need to be acted on, and will allow them to see the success of runs over time.

Customers can access the Search and Filter functionality from the top right of the All Runs page. One may search for runs based on name, result, environment, source, run group and several other criteria. The All Runs page looks like this: search.jpg

No published changelogs yet.

Surely Rainforest QA will start publishing changelogs very soon.

Check out our other public changelogs: Buffer, Mention, Respond by Buffer, JSFiddle, Olark, Droplr, Piwik Pro, Prott, Ustream, ViralSweep, StartupThreads, Userlike, Unixstickers, Survicate, Envoy, Gmelius, CodeTree