How to Mature Your DevOps Automation Practices

Organizations are under more pressure than ever to deliver innovative apps, services and digital solutions. This burden is falling particularly hard on DevOps teams, who are also responsible for the security and quality of their code. Many organizations are subsequently turning to DevOps automation to dramatically boost their teams’ productivity without making compromises elsewhere.

DevOps automation holds incredible promise for many teams. For example, it can:

• Cut remediation time for performance problems and vulnerabilities.
• Reduce the scope for human error or manual interventions.
• Standardize processes for repeatability and scale.

DevOps automation seems to be delivering on these promises for organizations that have made significant investments in this approach. According to the DevOps Automation Pulse Report 2023, organizations that adopt DevOps automation have seen their average software quality rise by 61%. Likewise, they’ve seen satisfaction among DevSecOps teams increase by 58% and IT costs fall by 55%.

However, organizations face challenges as they look to extend DevOps automation to additional use cases and workflows. Among the IT leaders surveyed, more than half (54%) worry that extending DevOps automation will increase security risks because of the heightened pace of software delivery. Extending DevOps automation also poses practical problems: 54% report difficulties in operationalizing data to support automated workflows, and 53% face challenges with the complexity of DevOps automation toolchains.

Two key approaches can help address these problems: AI-driven automation and platform engineering.

Embrace Intelligent AI-Driven Automation

Many DevOps workflows have complex, real-time data needs. Consider release management, for example. Assessing the impact of a new release requires DevOps teams to interpret and act on real-time performance, stability and user experience metrics their systems record and report in different formats. Typically, these more complex workloads need a human in the loop, which makes them difficult to automate. Moreover, as organizations move more workloads to the cloud to take advantage of their agility, these manual practices aren’t scalable.

Modern AI offers a solution to this problem: More advanced AI models can automatically take real-time data from multiple sources while retaining the rich context and performing the traditionally human task of spotting patterns, events and trends. These insights, in turn, enable the AI to automatically trigger predefined workflows. In the release management scenario, an AI could detect that a new release is damaging performance and stability and trigger an automatic rollback to the most recent stable version.

To achieve this intelligent automation, teams should first ensure their AI models have access to all the observability and security data they need and can query it on demand, in full context. Teams can achieve this end-to-end observability by using a unified platform approach that breaks down the silos between all data relevant to DevOps workflows.

When deploying AI atop this data infrastructure, organizations should use a framework that combines three key types of AI essential to DevOps automation:

1. Predictive AI models, which use deterministic reasoning to forecast the results of a change within an IT stack based on patterns in historical data.
2. Causal AI models, which use deterministic reasoning based on fault-tree analysis to deduce the precise root cause of a phenomenon within a technology stack.
3. Generative AI models, which use probabilistic reasoning to convert prompts created from these other sources of data into outputs that reflect the precision and accuracy of those inputs.

With this foundation, DevOps teams can achieve reliable answers that enable them to extend automation to a wide range of new workflows.

Leverage Platform Engineering and ‘as Code’ Principles

The vast toolchains in modern IT environments needed to support DevOps automation can become overwhelming. In fact, there are now more than 1,000 tools in the Cloud Native Computing Foundation (CNCF) landscape for DevOps teams to wade through. Because of this, teams can feel overwhelmed by the number of solutions and tools, while still spending significant time developing new automation workflows from scratch.

The complexity of these toolchains often leads to automation brittleness, when multiple teams build and manage automation scripts in silos, with each team having its own approach. When a team changes — such as a personnel or management change — it can create siloed tools and automation scripts, which can be incredibly difficult to share, understand, or modify.

There are ways to address these problems while still allowing DevOps teams to automate new workflows. Many teams are turning to platform engineering disciplines, where they’re building internal developer platforms (IDPs) that provide a centrally governed and secure interface between developers and the tools and solutions in the backend of an organization’s stack. Along with reducing cognitive load for DevOps teams and enabling them to self-serve the tools and capabilities they need, IDPs also allow organizations to save time creating automation workflows by defining the components and templates needed to build them.

In addition to platform engineering, “as code” approaches can also help simplify the work around building new automation workflows. These approaches allow DevOps teams to standardize configuration, management and monitoring of many parts of the IT stack using centralized policies expressed in code.

The result is that DevOps teams can simplify, consolidate and abstract away concepts like infrastructure, deployment and observability. For example, instead of a workflow having to define which particular server it will run on, it can simply refer to the relevant Infrastructure as Code script. Teams can feed the data from as-code solutions into repeatable workflows in a standardized way that enables DevOps automation at scale.

Observability as a Foundation

Approaches like intelligent automation, platform engineering and as-code principles are essential for overcoming the most-cited problems in DevOps automation. However, teams also need to ensure high performance and a pace of innovation that doesn’t compromise quality and security. As a result, DevOps teams need to be extremely diligent when it comes to the data they use to power their automation.

To ensure the highest quality data and analysis, organizations need to unify observability and security data from across their IT stack and embrace an AI framework that can produce automation functions safely and at scale.