Your Platform Engineering Toolkit for Terraform and Beyond

Building a platform engineering toolkit tailored to your organization’s unique needs is a formidable challenge, whether you’re starting from scratch or piecing together existing materials and processes.

What’s in your toolkit? Most platform engineering teams start by creating a toolkit of Infrastructure as Code (IaC) modules to simplify and accelerate the deployment process. By providing a foundational set of resources such as APIs, key-value stores, topics, queues, schedules, secrets, storage and services/handlers, the platform allows developers to start quickly. Tools like Terraform and Docker have achieved widespread success due to their ability to provide consistent infrastructure provisioning and containerization, respectively.

Where do you go from there? Your IaC module toolkit may include core resources, but it also needs to be extensible and support new resources easily, essentially maintaining flexibility as applications grow. Plus, to effectively use all of these tools in your platform, you need a powerful deployment engine capable of orchestrating your tooling to ensure consistent and secure infrastructure for all your projects.

Here are three automation tools your platform team needs to achieve success with IaC as the foundation:

1. Extensible IaC modules
   Use Infrastructure as Code (IaC) to simplify, secure and customize resource deployment using tools like Terraform.
2. Resource specification generator
   Automate the gathering and synchronization of application requirements with infrastructure deployments, ensuring efficiency and reducing miscommunication.
3. Deployment orchestration engine
   Use orchestration engines to map resource specifications to IaC modules, enabling smooth and consistent infrastructure deployments.

Platform Tool 1: Extensible IaC Modules

You’re likely already implementing modules using popular IaC tools like Terraform. This ensures consistent and secure deployment of resources and allows platform engineering teams to encapsulate best practices and governance policies within each module.

For example, a Terraform module for an API gateway ensures that every instance of the gateway is configured correctly and securely.

Using Terraform, platform engineering teams can apply security and governance practices consistently across all deployments. This reduces the risk of misconfigurations and security vulnerabilities.

While a foundational toolkit provides a solid starting point, every application has unique requirements. Your IaC modules must be highly extensible so that platform engineers can customize and extend resources as needed. For instance, if a development team needs to customize the timeout settings for an API gateway, the IaC module can be extended to accommodate this requirement.

This modular approach prevents the pitfalls of monolithic Terraform projects that become tightly coupled to specific applications, making them difficult to manage and scale. Instead, you create Terraform modules that are specifically designed to provision resources based on developers’ needs.

Creating extensible IaC modules is the first step toward automation, and the good news is that, unless you’re working on a greenfield project, you already have IaC that is ready to be converted into modules, and the process is as simple as carving out the Terraform snippet that represents the resource you are trying to provision.

Platform Tool 2: Resource Specification Generator

One of the most challenging aspects for platform teams is gathering requirements from development teams about their applications and keeping the infrastructure in sync with those requirements every time there is a change. Developers often don’t know what information is needed, making communication challenging. This is where a requirements specification becomes incredibly helpful.

We can further streamline the communication of requirements from developers to the operations team and plug this directly into the deployment pipeline. This automation ensures that the process is efficient and reduces the potential for miscommunication.

👁 Diagram of how resource specifications can be generated from application code.

Here’s an example of how resource specifications can be generated from application code. In this code snippet, the developer has created an API that exposes a route “/url” which returns a pre-signed URL for uploading an image.

Using this code snippet, a resource specification generator like Nitric is able to gather the following information:

From the application code snippet above the framework gathers the following information.

1. Service resource:
   • ID: `hello-world_services-hello`
   • Config: Service with an image `hello-world_services-hello`, one worker and an environment variable `NITRIC_BETA_PROVIDERS` set to `true`.
2. Bucket resource:
   • ID: `images`
3. API resource:
   • ID: `main_api`
   • Config: OpenAPI 3.0.1 document for an API with a `GET` method at `/url`, handled by the function `hello-world_services-hello`.
4. Policy resource:
   • ID: `c26b107582b33de1660950c440ee2ef7`
   • Config: Policy allowing actions on the `images` bucket for the `hello-world_services-hello` service.

With Nitric, this specification then also acts as live documentation, detailing the runtime requirements of the application each time a deployment is previewed in a visualizations dashboard or provisioned to the cloud.

Platform Tool 3: Deployment Orchestration Engine

To automatically generate deployment scripts, we’ll need an orchestration engine that creates a mapping between our resource specification and our IaC modules. We can do this with Cloud Development Kit for Terraform (CDKTF), a development kit specifically designed to map IaC modules (known as TF constructions) to a resource specification programmatically. The end result of this orchestration is a Terraform project.

Our sample project above requires a storage bucket, as outlined in the specifications document. It also requires that the helloworld service be given a policy that allows read and write access to the bucket.

Nitric, which generated the resource specification above, can also be used as the deployment orchestration engine. Nitric orchestrates the linkage between the requirements specification and our Terraform modules by generating a Terraform project to perform the actual deployment.

This project is extremely large, so for illustrative purposes, here is a snippet of the configuration that will be used to provision our storage bucket in the cloud. In this example our Terraform modules are targeting AWS, therefore we’ll be using S3 buckets and IAM roles and policies.

If you’d like to see the full Terraform project, you can easily export your own by following this guide.

Automate Key Deployment Steps

Automation plays a crucial role in synchronizing application requirements with infrastructure deployment, ensuring efficiency and reducing the potential for miscommunication. Platform tools like the open source Nitric framework can help you automate key steps during your deployment workflow and assist you in crafting a platform engineering toolkit that not only accelerates deployment but also maintains the flexibility to adapt to unique application needs. As a result, you ensure a secure, efficient and consistent infrastructure for all projects.