Struggling with IT Staff Leaving? Try Infrastructure as Code

One day a few years ago, Sushil Kumar, our head of DevOps, was worried. A key admin on his team in charge of private and public cloud infrastructure had just given notice that he was leaving in two weeks. Even after scheduling a knowledge transfer session to other team admins, Kumar saw that their operations would be at risk.

And he was right. Kumar and his team spent the next 18 months dealing with the fallout. “We had to deal with a lot of technical debt, and some important systems weren’t upgraded in a timely manner,” Kumar said.

Many of our customers are struggling with this same problem. The recent “Great Resignation” has created a near crisis for DevOps managers and staff, who are struggling to keep applications up and performing well in the face of constant employee churn.

When only select DevOps admins know how to manage and maintain business-critical systems, then those team members leave, the knowledge leaves with them. There are many factors that can drive knowledge silos among IT operations, engineering and developer teams. Configuring and managing different environments, especially for organizations managing application environments across private and public clouds, is still a manual process and often entails specialized knowledge of cloud and legacy platforms.

To complicate this further, more organizations deploying cloud native and modern apps with containers and microservices requires infrastructure and apps to be provisioned across many clusters, pods and cloud environments. This becomes even more complex when considering the security, compliance, performance and cost considerations for each app.

Infrastructure as Code (IaC) has emerged in recent years as a way out. With IaC, DevOps teams automate the provisioning and changes to cloud environments. It can also bring standardization to infrastructure, which simplifies cloud configuration and reduces manual tasks for teams.

Moreover, IaC can speed up the onboarding and training of new and existing employees because it provides DevOps teams with a single source of truth on how an organization’s infrastructure is configured and how services run. In turn, adopting IaC can help narrow the knowledge and skills gap and ensure an organization is still able to maintain business continuity and deliver value to its customers even in a tight labor market.

Getting Started with Infrastructure as Code

IaC first requires a mindset shift, where teams need to treat infrastructure and operations in the same way they treat application code. It requires infrastructure and operations teams to think like developers. DevOps best practices — such as continuous integration, version control, cost forecasting, footprint optimization and automated testing — are applied to the code that runs and manages an organization’s infrastructure.

IaC enables the management and provisioning of infrastructure through software and automated processes, rather than through hardware and manual processes. But how do you get started?

For organizations that intend to adopt IaC, a good starting point is to select a new application or cloud environment. For that project, define your desired state as much as possible, including network configurations, security group rules, security and IAM requirements, cost constraints, performance and reliability conditions.

Next, identify your IaC toolset. Many leading tools will allow you to simply describe the desired state, an approach referred to as “declarative.” This makes the process simpler: Declare your desired state, and the toolset does the work of getting there and maintaining it. In addition, look for a toolset that works across multiple environments and is extensible. Ideally, the toolset should be multicloud — able to establish infrastructure to any cloud environment, private, public or hybrid.

Benefits of IaC

Once you’ve “codified” your infrastructure, you’ll likely start to realize its many benefits:

• Mitigate the fallout from employee churn — Staff turnover is always a challenge. But with IaC, the organization captures and retains critical knowledge of deployment and updates in code repositories, which greatly lessens the impact of any one expert leaving the team.
• Gain huge leaps in speed and agility — IaC allows software developers to write and execute instructions for compute, storage and network requirements, and thus provision them more quickly and efficiently. If an infrastructure component causes a problem, admins can easily roll back to an IaC version before the change.
• Onboard new staff more quickly — IaC makes it easier for new, and perhaps less experienced, members to join the team. Learning how the IaC system works is much more efficient than learning through “team knowledge.”
• “Shift left” to developers — With IaC, developers can provision their own infrastructure that is secure and compliant without always relying on the IT department to provision resources. IaC allows developers to scale their applications up and down themselves, a huge boon for efficiency.
• Maintain standards and consistency – Using the same code base to provision infrastructure allows organizations to not only achieve consistency across hundreds or thousands of applications, but also create standardization.

Today, Kumar’s team spends their time coding all desired-state deployments through IaC templates, for both private and public clouds.

The entire environment definition, from deployment through change events, is done through IaC workflows. Any change event triggers a pipeline, which automatically ensures quality and security of the change. “Team knowledge is now embedded in the code,” says Kumar.

“Employee turnover is a fact of life in our industry. But with IaC, we’ve had no visible impact on our quality of operations, and I can sleep well at night.” This, in the end, may be the most satisfying benefit of IaC: peace of mind.