Migrate to V2

Background

To solve for current limitations, a large refactor was implemented to how test results data is stored. Changes are typically backwards compatible but in this case the improvements were too significant.

This update involved our Test Runs, Dashboard, and CLI features. Please check out our new documentation for more details.

• Test Runs
• Dashboard

How to migrate

Ensure you have upgraded to CLI v2.0.0+ which applies the new version. Follow our installation guide for a review on this process.

Re-publish your test reports once you have upgraded the CLI.

If your account was created when V1 reports were available, you will have the option to navigate back to the old version in your dashboard. I recommend you to stop relying on this before it is deprecated.

Closing remarks

Thank you for understanding and please provide any feedback or concerns regarding the new version. Do not worry about this being a common occurrence, since this new foundation will set us up for a lot of growth.

I've included the initial changes below to better understand the motivation, but you will continue to see the benefits over the coming months.

Test run changes

• Chart metrics are now rolled up at multiple intervals rather than just 5 seconds. This improves performance and readability for long tests. Intervals are 5 seconds, 30 seconds, 1 minute, 5 minutes, and 30 minutes.
• The rendering of metrics by operation has been optimized to improve support for tests with many operations.
• Naming has changed to refer to all performance test reports as Test Runs.

Dashboard changes

• Introduced a way to group tests, called scenarios. These scenarios are displayed in your new dashboard with some aggregated metrics across all its runs. This grouping can be used to help catch trends in performance results.

CLI changes

• The api-key flag is now required when publishing reports. These anonymous reports make it harder to understand and serve users.