How Relevant Is the Gap Between the Elite and Low Performing IT Shops?

There is a widening gap between the best and worst-performing companies, according to the latest DORA DevOps report, produced by Nicole Forsgren. The report makes a big deal about a so-called elite group that represents 21% of respondents working at companies that have adopted continuous delivery and other best practices. Yet, if you look at both elite and high performers, that grouping actually declined 55% to 44% of the study’s respondents. Perhaps this gap of performance is not as dramatic as the report concludes?

The report uses cluster analysis to segment the respondents. Several metrics appear to improve gradually when progressively comparing the Low, Medium, High and Elite groups. However, the statistical approach does not actually predict this improvement, but instead just shows that these companies have a common set of traits.

Two measures of throughput prove to be problematic when comparing the groups. For a number of deployments, the difference between the elite and low performers jumped dramatically. The relevance of this metric may be declining. Instead, the lead time from commit to production deployment may be better because it actually impacts stability related metrics. However, the gap between the worst and best companies decreased dramatically from 2018 to 2019. Without access to the actual data, we can’t be sure, but this is most likely because even the lowest-performing companies have begun to use continuous integration techniques. Furthermore, the ability to be able automatically revert back to the previous version appears to be more valuable than quickly deploy a new release. In this regard, the time to recovery has not improved since last year because how much improvement can be made to “automatic”?

We noted a sharp difference in how the cluster’s adoption of automation and integration processes changed as they moved from low to the elite performers. As mentioned earlier, some metrics improved gradually, but the methods that did not are also interesting. There was little correlation between these groups and the existence of automated performance or security tests, which indicates that these practices are not associated as much with DevOps as some DevSecOps proponents would have us believe. Integration of chatbots and/or Slack with DevOps tools was used by 69% of elite performers, but by less of a third among everyone else. To us, this indicates that chatbot integration is more associated with general tech best practices and less with achieving DevOps related goals.

Feature image via Pexels.