High-Performing DevOps Teams Build Self-Service Platforms

Fast and effective DevOps teams use internal developer platforms, but does platform engineering actually make organizations perform better? Humanitec’s 2023 DevOps study provides new data, that with the right context can provide some clarity into the relatively new world of platform engineering.

According to the study, just 16% said that developer self-service is enabled by internal platform tooling like an internal developer platform (IDP) that a platform team builds for the rest of the organization. Half of the top or high-performing organizations in the study had something like an IDP, compared to 3% among all other organizations. According to the study, these organizations have the following practices in place:

• Its platform team provides developer self-service platform or other tooling, such as an internal developer platform (50%).
• It manages application configurations in a standardized way across all of its applications (67%).
• It can deploy new code on demand (49%).
• Its devs can bootstrap an application in less than two hours (43%), independently.

The researchers surveyed 1,053 teams around the globe, of which 19% were in DevOps, site reliability engineering, or platform engineering roles; another 29% identified as engineers or developers. The answers were scored again the four DORA metrics (deployment frequency, change lead time, change failure rate, and service production restoration time) to create four segments (top, high, medium and low performing).

Just 6% fell into the top-performing segment; 20% are high performing, while the remaining fall into the medium- (44%) or low-performing (30%) groups.

Yet, both the latest State of DevOps reports from both Perforce's Puppet and Google's DevOps team found that breaking out an elite or "top" group was not justified, based on a statistical method called cluster analysis. To help you better benchmark your organization versus your peers, we recalculated Humanitec's findings to focus on what a combined top- and high-performing segment looks like.

‘Broken DevOps’ Setups

Humanitec's questionnaire appears to have unintentionally conflated self-service developer platforms, and the ability of DevOps teams to build a Platform as a Service (PaaS) replacement. This makes it difficult for us to know exactly what the impact of an IDP is on software delivery performance metrics.

• While only 5% use a PaaS-like solution, that figure may be influenced by the wording of the survey question, which tells the respondent that this type of PaaS has problems with scaling and configuration.
• Meanwhile, 16% of organizations have a platform team that builds an internal PaaS that enables developer self-service.
• "Platform team" is likely just the new name for what DevOps engineers have been doing for years. In fact, 80% of the study are relying on either developers or operations teams to do DevOps tasks. The way the questions were worded,  all developers who handle their own infrastructure provisioning are assumed to be overwhelmed. Furthermore, there is an assumption that anytime an operations engineer or team executes DevOps tasks, this results in developers being dependent on Ops.

If you take the results at face value, then most organizations are struggling because they have only superficially embraced a DevOps culture. Yet, when we look at specific characteristics, we see a dramatic difference between the top/high-performing organizations and everyone else, mostly centered on developer time.

• Ninety percent of respondents working at top- or high-performing organizations said someone on their team can autonomously deploy to development or staging environments, as compared to 44% of medium- and 17% of low-performing organizations.
• Seventy-one percent of top- and high-performing organizations need less than a day to move from a code commit to deployment into production, as compared to 44% of medium and 17% of low-performing organizations.
• Among organizations with an IDP, 71% have the ability to deploy on demand—  or at least several times a day — while 49% of all the organizations surveyed can do that.

There is a strong correlation between developer self-serve platforms and DORA performance because there is a wide variation in development frequency and lead time needed to make changes. The immediacy of the ability to make changes is correlated with all four of the DORA metrics, but in particular "lead time" from making a commit to deploying that into production.

While the meantime restoring an application to its previous state may improve because of this type of platform engineering, the jury is still out on whether or not it dramatically decreases the rate in which changes fail.