How Toyota Drove Agile Load Testing to the Cloud

Tricentis sponsored this post.

What do Priuses have in common with agile, cloud-based load testing? Both are evidence of Toyota’s continued commitment to innovation and modernization.

Toyota Motors North America recently shared its journey to cloud load testing with and  at :

This keynote room session, delivered by Neeraj Tripathi and Hector Martinez from Toyota, was one of the most well attended and top-reviewed sessions of the conference.

As either a tease or a recap, here are some of the key points…

Why did Toyota Shift to Cloud Load Testing?

Toyota has a commitment to innovation that extends beyond its cars to all of the ways in which it deliver its products.  This includes a desire to maintain a lead in terms of development best practices and a drive for constant improvement in the way software is being created.  The main factors that drove Toyota to replace its legacy load testing tools with a more modern, cloud-based approach included:

• Desire to shift testing left: More and more organizations are moving towards , with the demand that more load testing be done earlier in a development cycle to catch performance regressions while they are still cheaper and easier to fix.
• Complex maintenance of on premises resources: Managing on premises load generators was becoming a costly and time-intensive exercise.  Additionally, being limited only to running tests on the physical servers that you own can limit the amount of parallel load testing which can be done, which can be a major bottleneck when moving to an agile and DevOps environment.
• Previous performance testing tool was not flexible enough to meet the needs of agile and DevOps: The closed source, proprietary scripting model was not being adopted well by developers, who were not able to contribute to scripts to load test their new features being added in each sprint.
• No solution for early testing: Testing was always happening late in the development cycle, often leaving very little time to get these issues fixed before needing to push code into production.

What Were Toyota’s Key Criteria in a Cloud Load Testing Offering?

Toyota had a number of options when evaluating potential solutions for its cloud load testing platform of the future.  When assessing the various tools on the market, its selection boiled down to a few key criteria:

• Scripting: Ability to support common open source tools like , with a tight native integration to this tool.
• Management: Ability to control the various teams, projects, cloud resources and integrations required when testing across a wide array of teams in an enterprise like Toyota.
• DevOps Integration: A well-documented for integrating load tests to the CI/CD pipeline for continuous load testing.
• Execution: Ability to execute load in your own to take advantage of the ability to use VPC’s and other security controls to test behind the firewall environments.
• Reporting: Ability to easily report on test executions, compare historical results and drill down into any major issues quickly.
• Customization: Ability to extend and customize the tool to fit into the particular CI/CD ecosystem at Toyota.

At the end of the day, Toyota Motors North America evaluated many potential cloud load testing solutions, but ended up choosing Flood for having the highest marks across the key criteria.

👁 Image

What Benefits has Toyota Seen with Cloud Load Testing So Far?

Since implementing this new approach to load testing, Toyota has seen tremendous benefits in its organization’s software delivery process.  Specifically, the main benefits of adopting this new approach include:

• Quick Adoption: Roughly 40 separate teams have already adopted this cloud-based load testing solution, which has been named the Agile Load Test Platform (ALTP).
• Frequent Execution: Teams are testing a combined 700 hours per month already, with some of the longest tests running up to 24 hours.
• Integration to the CI/CD pipeline: By using the Flood API, teams have been able to integrate closely to their CI/CD pipelines and trigger load tests with each and every build.
• Security Team approved: Through using its own AWS account, Toyota can securely access environments that are both publicly and privately hosted, with the blessing of corporate security.

We look forward to seeing Toyota take this solution even further in the future, as it expects to ramp up almost 100 teams on this new approach. With the addition of our new Teams Management feature in Flood, Toyota will be well positioned to control and administer the platform to this massive population of new users.

How Can I  Build My Own Cloud Load Testing Platform?

Teams interested to experiment with building their own solution like Toyota’s can access the downloads and trials of the various required components here:

If you have any questions about how you can get started with cloud load testing, please don’t hesitate to  — we’re always happy to help you get started.