Progress 360: How Panera Fixed Its Application Validation Gap

Boston, Mass. — Panera Bread’s development team had a problem: Its homegrown solution to deploy applications to its more than 2,000 locations nationwide worked about 85% of the time, creating a significant problem and validation gap for the restaurant chain.

“Our cafes, they’re little silos with 36,000 machines that we have to make sure we’re all on the same version of everything,” Katie Canty, vice president of Café Systems for Panera, told audiences at the Progress 360 conference in Boston on Monday. “It’s like press the button, and then you cross your fingers.”

It was a business-critical problem. Each café has a server that runs everything — from the system telling workers how to make the food, to managing the drive-thru timers. While the team would get log files back, the primary way for them to learn that a system didn’t update correctly was through the help desk calls that came in from stores.

Panera switched to Chef’s Habitat, an open source automation tool for rolling out applications across operating systems and platforms. Chef is a relatively new acquisition for Progress Software, which bought the company for $220 million in 2020. The solution saved brought them from 85% success to 99.999% success in rollouts, Canty said.

“The last thing I want is they can’t ring in an order or they can’t make a sandwich. So it has really enabled us to to just not even worry about getting those applications out there,” Canty said. “We can focus on actually developing cool new stuff and just testing that stuff.”

Managing a Roll Back

Habitat also saved the development team major headaches when the team had to roll back a change after deploying problematic code to 2,200 cafes, she said. Using Habitat, they were able to redeploy the old version within 10 minutes — which was a conservative time frame, Canty added.

“It could have been 30 seconds,” she said. “We wanted to make sure online orders were still coming in, so we spread it over that 10 minute period.”

Progress 360, Progress Software’s conference for developers and DevSec ops teams, is going on through Wednesday in Boston, Mass. Canty was one of three executives from large companies who shared testimonials at the event.

A main focus of the keynote was how the Covid pandemic had challenged businesses to leverage technology in new ways.

“Over the last three years, technology has helped people make it through these extremely challenging times. And we all understood — you all understood — why you were doing the work you were doing to help these people as they were trying to navigate their lives at a certain time,” Progress CEO Yogesh Gupta said. “So what you have done, truly has helped make the world a much better place.”

The New Landscape for IT

Gupta also shared his perspective on how IT needs to respond to the changes brought on by the emerging world landscape, including an expectation that technology will be personalized, hyper-local, and yet available 24/7/365 days, plus across the world, while dealing with new threats such as cyberattacks from nation states — all while being agile.

“You have to move as fast, as we need to make changes as quickly as our customers want,” Gupta said. “All of this together almost seems insurmountable. It almost seems impossible to do this.”

First, IT must look at the problem holistically — and that includes developing, deploying and managing in a holistic way, he said.

“It has to be done collectively, it has to be done together, and so it requires us to think of the people who are sitting next to us, and saying, how do we work together to make this happen,” he said. Of course, the challenge in today’s landscape is that that may require collaboration with teammates halfway around the world, he added, so technology tools must enable this collaboration.

Second, IT must have the right architecture, which includes support for any kind of device, channel, made of interaction — from chatbots to wearables — and emerging new channels. It must also support all forms of data, including streaming from diverse sources such as the Internet of Things.

“By the way, it has to be done in a way that your business systems don’t have to worry about what channel that you’re supporting,” he said. “So really, you need to think of this as an independent layer that continues to expand and grow as technology changes.”

Third, it has to run well while being secure, he said. Finally, IT needs to be able to monitor the full stack and track what’s going on and automate responses so the system can continue to function in the face of challenges, he said.

Disclosure: Progress Software paid for Loraine Lawson to attend its conference in Boston.