Infrastructure as Code: Modernizing for Faster Development

Before Matt Stephenson worked at Starburst Data, he used to work at Square. There, he learned some hard lessons about working with legacy Infrastructure as Code (IaC).

“We built an entire system that kind of did a lot of orchestration with Terraform and Helm, and integrated with some of our own backend services,” Stephenson, a senior principal software engineer at Starburst, told The New Stack.

It’s not a project he remembers fondly: “The experience of having to build and maintain that service made me take a look at what was available out in the industry, for not having to build that again.”

The problem isn’t Terraform per se, he said, but “it’s all of the code to execute Terraform, all the code to manage the inputs and outputs to Terraform itself.”

Legacy IaC can bring a number of challenges into the lives of an engineering team. Among them:

• It can get more complex to follow required conventions and standards when configurations are being defined, and the complexity grows as it scales.
• As a result, configuration drift is common, and can result in noncompliance and service outages. (And misconfigurations in general are a leading cause of security breaches.)
• Necessary integrations and features aren’t always available for specific use cases.
• Legacy IaC can bring significant maintenance needs, and it can be tough to recruit and retain engineers who have those skills.

“A lot of the legacy Infrastructure as Code products have their own language, they have their own environment, you have to kind of become a bit of an expert in them to be effective at them,” Stephenson said. “Or you have to have some kind of support, going into using one of those.”

At Starburst Data, he oversees the architecture for the company’s Galaxy product, a managed data lake analytics platform. His team has gradually swapped out legacy IaC for Pulumi, an open source IaC product that allows infrastructure to be built in any programming language.

Stephenson will be among the presenters at PulumiUP, a virtual user conference on June 15 dedicated to Infrastructure as Code, how it enables faster application development and how users can navigate the challenges of legacy systems.

At the conference, he’ll be talking about Pulumi’s automation API, he said. “That was a big driver for us, being able to orchestrate all of our Pulumi stacks without having to write that whole service that we had to write in the past.”

Empowering the Whole Team

One of the differences between Pulumi and legacy IaC solutions, Stephenson said, is that “it’s based in programming languages that people learn in college or learn really quickly when they join the industry.”

Pulumi allows developers to build infrastructure in common languages including any Java language (Java, Scala, Clojure, Groovy, Kotlin); .NET (C#, F#, PowerShell); Node.js (JavaScript, TypeScript); Go, Python and even YAML. This helps make provisioning infrastructure something that more members of an engineering team can do.

Before his experience using Pulumi, Stephenson said, “it was mostly more senior engineers that would be involved in setting up all of your infrastructures, your code environments. These days we have folks across the skill-set level working in it.

Now, he said, even people in his organization without infrastructure or site reliability engineering backgrounds, “when they’re doing product development, they’re able to just go in and make the changes they need. They don’t really have to worry about engaging an expert to be able to get something to happen that they want.”

As a result, Stephenson added, there’s less need for hiring IaC-specific experts for a team, and more people are empowered to handle problems.

“If there’s an incident that involves the infrastructure, a lot of times people can make the changes they need to execute our continuous delivery pipeline and get things fixed.”

A Search for Flexibility

Dennis Sauvé, the first DevOps engineer hired by Washington Trust Bank, will also be presenting at PulumiUP, talking about his company’s experience moving from an entirely on-premises system to one run on Microsoft Azure Cloud — with IaC, largely written in TypeScript, provided by Pulumi.

Before the bank hired Sauvé, it decided to start services to the cloud to move forward with innovations like a customer collaboration tool that will allow Washington Trust’s relationship managers to talk with clients directly. It had determined that Azure’s communications services would help it build that application more easily, Sauvé told The New Stack.

But the bank also wanted flexibility for applications it might build in the future, and for the clouds it might deploy those apps on.

Pulumi, Sauvé said, offered that flexibility and the options his team needed. “You can pick your cloud provider. And then once you have a cloud provider, you can pick a language you want to build that stack in, and they support it.

“And so we had that peace of mind that not only if we wanted to change the language we wrote our Infrastructure as Code with, we could also change our cloud provider as well. We could go to [Amazon Web Services] or Google Cloud, and we’d be able to take plenty right along with us. So that was a huge bonus when considering different providers.”

Saving Time and Toil

One of the biggest benefits of Pulumi for Washington Trust Bank, Sauvé said, has been the ability it gives his team to save time and toil. He and his development team have been creating best-practice templates for creating resources.

Instead of the back-and-forth that might have existed between developers and operations engineers, “the developers can now just go to our infrastructure package, find the resource that they want to build, choose that and set it up to deploy. And it really speeds up development and testing environments.”

Not only that, he added, but Pulumi has become a standardization tool, ensuring that resources are being created in the same way across the organization.

However, he added, moving to the cloud and onto Pulumi, hasn’t been without hiccups. Notably, the native Typescript package is, “from a file-size standpoint, just a massive package that is a little taxing on resources to use, but it works in production.”

Pulumi, he noted, will soon release a next-generation version of the TypeScript package that “should be very slimmed down and address some of the performance issues.”

Shifting away from legacy IaC can cause a bit of disruption on a team at first, Stephenson acknowledged. (“There’s always folks that really kind of hang their hat on being the expert in the room with specific things like Terraform,” he noted.)

But in the long run, he said, it empowers a broader set of people in the organization. He pointed to a colleague who joined Starburst Data soon after graduating from college: “Now he’s at a senior level; he’s basically gotten himself bumped a level twice, because he’s just so on top of everything. Pulumi was one of those things that he really dug into.”

Stephenson has heard similar stories from other companies. “You end up with people who might push back, but then at the end of the day, there’s a lot of people who excel and become the next rock stars as a result of making a shift like this.”