Is DevOps Tool Complexity Slowing Down Developer Velocity?

The overwhelming majority of developers in a new survey — 84% — say they’re involved in DevOps activities. But despite this, devs haven’t gotten any faster at making code changes and putting them into production over the past two and a half years.

The increasing complexity of the projects DevOps teams work on may be slowing down developer velocity, suggested the report by SlashData and the CD Foundation, a project of the Linux Foundation.

Among the findings:

• Lead times to restore service after an outage have increased. According to the report, it takes up to a day for 44% of DevOps practitioners to restore services compared to 54% who said the same since data collection began in the third quarter of 2020.
• The time needed to implement code changes has not improved. Thirty-seven percent of DevOps practitioners said their lead time for code changes is less than a week. That’s the same percentage that gave that answer in Q3 2020.
• The proportion of organizations that the researchers define as low performers has increased since Q3 of 2020, while the share of high performers has decreased in that same period. This fits a pattern we have written about previously.

“It is good to see that there is still an increase in the adoption of CD and DevOps practices. However, there are signs that we still have work to do,” said Fatih Degirmenci, executive director of the CD Foundation, in an email response to The New Stack.

“CD and DevOps require organizations to change how they organize themselves, get their teams to embrace the cultural and mindset changes and adopt their product structures,” Degirmenci wrote, adding, “These changes usually take some time to implement and show their effects.”

He also noted the impact of the complexity involved, including “not just how the products are built (e.g. microservices) but also the surrounding environment — from infrastructure to development environments to CD technologies, and so on.”

The new report’s findings are derived from data collected for SlashData’s past six Developer Nation surveys, which reached more than 125,000 respondents from the third quarter of 2020 to the first quarter of 2023.

Use of Self-Hosted CI/CD Tools Declined

The study’s results showed how organizations are using DevOps technologies. Not much has changed since Q1 of 2022, including the average number of tools used, which has held steady at 4.5.

However, some trends emerged:

• The use of self-hosted CI/CD tools has dropped from 32%  in Q1 2022 to 23% in Q1 2023.
• Application monitoring/observability tools saw the second biggest decline in usage, going from 37% to 31% in the same period.
• Few areas of security saw increased use. The largest increase was for application security testing technologies, which increased from 25% to 28% over the past year.

CI/CD tools appear to be a decisive factor in how quickly it takes organizations to restore service: Fifty-eight percent of those that use CI/CD tools can restore within a day, versus 35% of those that don't.

The more CI/CD technologies organizations use, the less time it takes to restore. For example, it takes more than a week to restore services for about 55% of DevOps practitioners, but that steadily drops for each additional tool, going to approximately 20% among those using eight or more tools. But there’s a caveat: the more self-hosted tools developers use, the longer it takes to restore service after an incident.

More than half of organizations that use five or more self-hosted tools were defined as low performers by the researchers, with only 10% of organizations that use that many self-hosted tools are considered high performers.

“An increasing number of tools used having such a strongly negative impact on service restoration time has multiple possible explanations,” the report stated. “However, interoperability issues may be at the centre of many of them. Multiple tools may make it challenging to integrate all of them well, leading to a greater challenge to isolate the service-impacting issue at hand.

“Further, a lack of standardisation between tools may make it more difficult for all tools to work together well.”

As teams choose from among the ever-expanding landscape of DevOps tools, they should consider how well the tools they use play together, Degirmenci suggested.

“One thing organizations could do and benefit from,” he wrote The New Stack, “is including interoperability as a criteria during their technology evaluations, so they can reduce the complexity as well as reach greater flexibility when it comes to adding new tools to their environments.”