The xz Hack Revealed a Looming $8.8 Trillion Infrastructure Disaster

Imagine the security screening system used at your local airport has broken down for the second time this month. Unfortunately, nothing can be done because it is the responsibility of one guy, Bob, to ensure the security screening system is working. And Bob is an unpaid volunteer.

Bob has a full-time day job, so he can only do security screening repair on nights and weekends, and only when it isn’t soccer season because he coaches his girls in the evening during soccer season. So the security of our skies will have to wait until soccer practice is over.

Sound ludicrous? This is exactly the situation we’ve come to accept as normal when it comes to the software infrastructure that powers all of the world’s phones, computers and the applications that run on them.

This came into stark relief earlier this month when one obscure but heavily used piece of software called xz — which is maintained by a single unpaid volunteer developer — was compromised, likely by a well-funded state actor like China or Russia. The most alarming part of this hack, and what it made crystal clear, is that our global software infrastructure — used by large corporations to generate billions of dollars of wealth — is built on the backs of unpaid labor.

This is dangerous, and it needs to change.

A bit of background: Today, almost all technology is built using open source software. The xz project at the center of this hack is an example of open source software — freely available and highly depended-on code used by millions of people and organizations to compress data and make it take up less space.

Over the past 25 years, open source has gone from a niche concept to the de facto way of building software. This is primarily because it gives any organization using it a huge head start of billions of lines of code that they can use freely rather than writing their own code from scratch. In actuality, one study found that some commercial software products are made up of over 99.9% freely available open source software.

Open source has become a global public good — irreplaceable infrastructure that should be mentioned in the same sentence with clean water, dependable electric power, safe highways and, yes, airport security. In fact, a recent study from Harvard Business School estimated the demand-side value of the open source software infrastructure we all rely on at $8.8 trillion.

By comparison, the U.S. Interstate Highway System is valued at only $742 billion, and the entire U.S. electrical grid is valued at only $1.5 trillion to $2 trillion.

That’s why it is so interesting — and also frightening — that much of the open source software running our companies, our governments and even our weapons systems is written and maintained by a vast community of unpaid volunteers — like Lasse Collin, the maintainer of xz.

The attack that snared xz was an elaborate and devious social engineering hack that took years to carry out, involved multiple fake aliases and preyed upon a weak link: an overworked, unpaid open source maintainer.

Someone using the alias Jia Tan built trust with Collin by contributing useful code over a multiyear period. The other aliases harassed Collin, asking him why more work wasn’t being done, and why he wouldn’t let someone else take over who had the time to work on the project.

Eventually, after admitting to suffering from mental health issues, Collin gave in to the pressure and granted Jia Tan the rights to add their own code to xz. Jia Tan then used their access privileges to add a security backdoor that would allow unfettered access to execute malicious commands on any impacted computer. This backdoor was accidentally uncovered by a curious developer at Microsoft who was investigating why xz was running slowly.

While this hack was exceptionally sophisticated and discovered before it could wreak global havoc, attacks on open source infrastructure are becoming a more and more regular occurrence.

Action to address these looming threats to our software infrastructure can’t come soon enough.

In 2022, the U.S. government passed a bipartisan national infrastructure law that provided over $400 billion in funding to desperately needed infrastructure projects, including efforts to update bridges and highways, connect more people to the Internet, improve the reliability of the electrical grid and much more. But investing in our shared software infrastructure — including paying for the contributions of the open source developers who build it — was not included.

The following year, the White House unveiled a National Cybersecurity Strategy that, among other things, sought to improve the security of open source software. In the fall of 2023, the Office of the National Cyber Director put out a request for information about ways to improve open source software security, concluding, “It may be appropriate to make open-source software a national public priority to help ensure the security, sustainability, and health of the open-source software ecosystem.”

And as part of a response to the xz hack, Mark E. Green, Chairman of the House Committee on Homeland Security, called on the House of Representatives to bring the Securing Open Source Software Act (H.R. 3286) to a vote. Yet none of these initiatives directly addressed — or funded — the work by actual humans that is required to keep our open source infrastructure properly maintained.

Modern governments were created in part to fund public goods like health services, transportation and energy infrastructure, yet open source software is the only public good in the world that still relies on volunteer labor to keep it safe and secure.

The reality on the ground today is that unpaid developers are still the norm for open source projects, and these developers are being tasked with an ever-increasing amount of work to ensure the security of the software we all rely on as threats become more pervasive. That status quo is not sustainable, and the problem becomes more dire by the day.

We all benefit from the incredible resource that open source software has become. But just like our roads, bridges, electrical grid and airports will deteriorate without public investment, so will our software infrastructure. We need urgent action from both the government and the many industries built on open source software to ensure that doesn’t happen.