The Pillars of Platform Engineering: Part 5 — Orchestration

This guide outlines the workflows and checklist steps for the six primary technical areas of developer experience in platform engineering. Published in six parts, part one introduced the series and focused on security. Part five addresses orchestration. The other parts of the guide are listed below, and you can download a full PDF version of The 6 Pillars of Platform Engineering for the complete set of guidance, outlines, and checklists:

1.   Security (includes introduction)
2.   Pipeline (VCS, CI/CD)
3.   Provisioning
4.   Connectivity
5.   Orchestration
6.   Observability (includes conclusion and next steps)

When it comes time to deploy your application workload, if you’re working with distributed applications, microservices, or generally wanting resilience across cloud infrastructure, it’s going to be much easier using a workload orchestrator.

Workload orchestrators such as Kubernetes and HashiCorp Nomad provide a multitude of benefits over traditional technologies. The level of effort may vary to achieve these benefits. For example, rearchitecting for containerization to adopt Kubernetes may involve a higher degree of effort than using an orchestrator like HashiCorp Nomad which is oriented more toward supporting a variety of workload types. In either case, workload orchestrators enable:

• Improved resource utilization
• Scalability and elasticity
• Multicloud and hybrid cloud support
• Developer self-service
• Service discovery and networking (built-in or pluggable)
• High availability and fault tolerance
• Advanced scheduling and placement control
• Resource isolation and security
• Cost optimization

Orchestrators provide optimization algorithms to determine the most efficient way to allocate workloads into your infrastructure resources (e.g. bin-packing, spread, affinity, anti-affinity, autoscaling, dynamic application sizing, etc.), which can lower costs. They automate distributed computing and resilience strategies without developers having to know much about how it works under the hood.

As with the other platform pillars, the main goal is to standardize workflows, and an orchestrator is a common way modern platform teams unify deployment workflows to eliminate ticket-driven processes.

When choosing an orchestrator, it’s important to make sure it’s flexible enough to handle future additions to your environments and heterogeneous workflows. It’s also crucial that the orchestrator can handle multitenancy and easily federate across multiple on-premises data centers and multicloud environments.

It is important to note that not all systems can be containerized, or shifted to a modern orchestrator such as vendor-provided monolithic appliances or applications, so it is important for platform teams to identify opportunities for other teams to optimize engagement and automation for orchestrators as per the tenets of the other platform pillars. Modern orchestrators provide a broad array of native features. While specific implementations and functionality vary across systems, there are a number of core requirements.

Workflow: Orchestration

A typical orchestration workflow should follow these eight steps:

1. Code: A developer commits code.
   1. Note: Developers may have direct network control plane access depending on the RBACs assigned to them.
2. Validate: The CI/CD platform submits a request to the IdP for validation (AuthN and AuthZ).
3. IdP response: If successful, the pipeline triggers common tasks (test, build, deploy).
4. Request: The provisioner executes requested patterns, such as building modules, retrieving artifacts, or validating policy against internal and external engines, ultimately provisioning defined resources.
5. Provision: Infrastructure is provisioned and configured, if not already available.
6. Configure: The provisioner configures the orchestrator resource.
7. Job: The orchestrator runs jobs on target resources based on defined tasks and policies.
8. Response: Completion of the provisioner request is provided to the CI/CD platform for subsequent processing and/or handoff to external systems that perform actions such as security scanning or integration testing.

Orchestration Requirements Checklist

Successful orchestration requires:

• Service/batch schedulers
• Flexible task drivers
• Pluggable device interfaces
• Flexible upgrade and release strategies
• Federated deployment topologies
• Resilient, highly available deployment topologies
• Autoscaling (dynamic and fixed)
• An access control system (IAM JWT/OIDC and ACLs)
• Support for multiple interfaces for different personas and workflows (GUI, API, CLI, SDK)
• Integration with trusted identity providers with single sign-on and delegated RBAC
• Functional, logical, and/or physical isolation of tasks
• Native quota systems
• Audit logging
• Enterprise support based on an SLA (e.g. 24/7/365)
• Configuration through automation (infrastructure as code, runbooks)

The sixth and final pillar of platform engineering is observability: Check back tomorrow!