New "Expected Result" field in the Test Designer

You can now specify the expected result of their test writing requests via the Test Designer.

To improve the quality of Test Designer output and help our test writing team write better tests for clients, we now ask clients to also specify the expected result of their request. This gives test authors complete clarity on what the outcome of the tests should be and results in fewer questions around instructions and consequently, fewer blockers.

On the test writing request form, you will now see and enter a new field titled “Expected result”. This data is sent directly to the test author so they can then write tests keeping the result or outcome in mind.


Test Designer - Run Level Blockers and UI Improvements

Visual updates to the Test Designer main pages to improve the user experience.

Our goal is to provide a better user experience by making the status of test requests more apparent. The visual updates call attention to blockers and will help distinguish the progress of each test request.

Previously, reviewers could block runs from the Rainforest admin which would require OB/CS intervention to facilitate unblocking. Now, you can unblock runs straight from the UI removing the need for OB/CSM intervention.

If a run is blocked, you can simply address the run-level blocker through the Rainforest interface. Additionally, we made visual updates to the status badges as well as updates to the main pages to reflect when requests were submitted, their status, and how many tests were returned.




Easier Failure Categorization

Categorize failures straight from the run summary page.

You can do mass categorizing of test runs that failed for the same reason and view tester comments from the step failure on the run summary page in order to categorize failures with fewer clicks.

The results triage process can be painful for users. By reducing the number of clicks it takes to review and categorize failures, we are aiming to reduce some of the pain associated with this process.

You can now categorize several test runs into the same failure category by selecting the “Mass Categorize” checkbox and choosing a failure category. This saves them time from having to select each failure and categorize it individually.

By hovering over the failed step number, you can also view the tester comments for specific failed steps in order to diagnose the failure and categorize it straight from the run summary page.

Assign Test Owner For New Tests

Rainforest now surfaces the test owner field when creating new tests, defaulting to the currently logged in customer or the customer kicking off the Test Designer run

By surfacing the test owner there is now a sense of accountability when it comes to Rainforest results. If the owner’s test failed, it’s now easier to know who to notify that the run needs to be categorized and appropriately acted on.

When clicking “New Test” and opening the new test modal, there’s a new dropdown to select a test owner for the test being written. It defaults to the logged in user, but can be changed to any team member.

For Test Designer runs, the test owner is automatically assigned to the user that kicked-off the Test Designer run. This can be changed later on from the test page.

Mobile App upload via the CLI

Use the CLI to upload your mobile app directly to Rainforest

To help new (and existing) users onboard better and easier, we now allow a direct mobile app upload to Rainforest via the command line interface so your app can be ready for testing. This way, we move away from using third party tools for uploads that have proven to be a cumbersome and non-intuitive process.

To kick this off, you’ll need to install the Rainforest command line interface (find helpful guides here).

Once set up, use the following command to view your list of environments.

rainforest environments --token REPLACE_ME

Next, use the following command to view your list of sites.

rainforest sites --token REPLACE_ME

Both sites and environments can be added and edited through the Settings page on our platform. Select the relevant site ID and environment ID and use in the following command while uploading your app.

rainforest mobile-upload --token REPLACE_ME --site-id 1234 --environment-id 5678 my_app.apk

Go back to the Settings page to view your newly uploaded app. Then, kick off your first test on this new app.


View original test outlines in the Test Designer

View your original test outlines after you submit a test writing request via the Test Designer

Earlier, you were unable to view the test outlines they submitted to the Test Designer once they run was published. This prevented you from referencing their original requests when required or comparing these requests to the newly created tests. With this feature, we now enable you to go back and view these original requests/outlines as needed.

Click into your test writing request to view the tests within that request. At the top of the page, click on the “View Outlines” button to switch to the test outlines view. The button acts like a toggle so you can use it to switch back and forth between the tests view and the outlines view.



Test Owner & Emailing Team Members

Notify team members about a failure categorization & Surface the test owner.

This feature adds further value to failure categorization by creating a sense of ownership over tests and enabling customers to ensure actions get taken on failures that have been categorized.

Before this feature, the value of failure categorization was primarily limited to reporting via the QA visibility report. Now, with email notifications to team members, you can get a lot more value out of the feature by ensuring actions get taken against failures that have been categorized.

You now clearly see who the test owner for a particular test is. Once they categorize a failure, you are prompted with a modal that allows you to provide some context on the categorization and select a team member to notify about it. The team member fields default to the test owner, but you can change that to email a developer about a bug, or email it to someone else about a test that needs to be refactored for example.


Next arrow to next uncategorized failure

You can now keep track of your progress in categorizing the failures in a run and easily go into the next uncategorized failure with a single click.

Prior to this update, you had to click back and forth from the test result view to the run summary page in order to go through a run and triage all the failures. This update reduces the workflow friction associated with categorizing failures and makes it more likely you engage with results and fully experience the value of Rainforest.

The next button and progress tracker encourages you to go from one failure to another instead of opening failures in individual tabs or going back and forth between test results and the run summary. By clicking the next arrow on the top right of the test result header, you are taken straight to the next uncategorized failure and can keep an eye on your progress with the tracker.


Test History on Test Result Page

Surfacing the test’s recent run history on the test result page to add more context and enable better results triage

You can now see the test’s recent run history while they are on the test result page and completing your results triage workflow. This allows you to gain more context on the test and determine the most appropriate category and action that needs to be taken on the result.

Under the title of the test run we now surface the 5 most recent results for that test. You can hover over each block and view the date, environment, and platforms used for the given run. By surfacing this information the goal is to make the results triage process easier and increase overall engagement with results.


New and Improved Test Designer

A new Test Designer interface that makes the process of submitting test writing requests as user-friendly and intuitive as possible.

With a revamp of the Test Designer, our goal is to make the experience of submitting test writing requests easy and fun for you, with minimal hiccups.

The new interface allows for a more streamlined experience from creating your first request to writing your test outlines to tracking the progress of your request. Improved messaging, helpful tips and guidelines, prominent calls-to-action and an easy-to-use interface are all part of the new experience.


With this, you can fill out test outlines very quickly, edit at your convenience and easily help test authors when they are blocked at any moment in-app itself.

tw2.png tw3.png

No published changelogs yet.

Surely Rainforest QA will start publishing changelogs very soon.

Check out our other public changelogs: Buffer, Mention, Respond by Buffer, JSFiddle, Olark, Droplr, Piwik Pro, Prott, Ustream, ViralSweep, StartupThreads, Userlike, Unixstickers, Survicate, Envoy, Gmelius, CodeTree