You’re Addicted to Curl, You Just Didn’t Know It

Amazon Web Services (AWS) sponsored this post.

You’re addicted to curl and didn’t even know it. Yes, it’s easy to read its description — “curl is used in command lines or scripts to transfer data” — and think, “That sounds really nerdy. I would never use that.” But you — and billions of others just like you — do. Every day. Curl, after all, is running on your Nintendo Switch, your Mac, your car, your mobile phone, and more. It’s the tool your car would use to download maps to its navigation system. It’s the tool your gaming device might use to upload data to a server.

Curl, in short, does internet transfers of data behind the scenes, every day, on billions of devices.

As such, you might think there’s a big business behind the development of curl, or that its founder is now spending their billions on a cure for COVID-19. Alas, no! It’s just Daniel Stenberg, working on curl since 1996 in his spare time as an open source project. Though 2019 saw him make curl his full-time job, it took him 21 years to get to that point.

Running an open source project can be rough. With that in mind, I caught up with Stenberg to better understand the origins and his ongoing motivation to maintain curl.

“A Very Tiny, Small Project”

When Stenberg set out to automatically fetch and provide currency exchange rates on his IRC channel with curl, he wasn’t trying to change the world. There was no expectation of billions of users for this open source project. In fact, open source hadn’t been “invented” yet. It was 1996 and the term “open source” wasn’t coined until February 1998. No, Stenberg just wanted to build a little utility (“how hard can it be?”) as a “very tiny, small project.” Starting with an existing tool (httpget), he made some adjustments and released what would eventually become known as cURL, a way to transfer data using a variety of protocols. Need your car to update maps? You’re likely using curl for the job.

Today Rust is what the cool kids are coding, but back in the 1990s Stenberg wrote curl in C. Among other advantages, this made it exceptionally portable. “You can basically run it on whatever operating system or whatever CPU you have; big or small, old or new, you can run curl on it,” he says. This turns out to be a differentiator between curl and other data transfer tools.

When Stenberg wrote this little project in C, no one was paying him for his trouble. It wasn’t his full-time job, or even his part-time job. “It was completely a side thing,” he said. “I did it for fun.”

Of course, not everything about curl development is fun. Curl is now used in billions of devices (your Apple iPhone or Samsung Galaxy phone, for starters), which means that Stenberg’s side project just might be mission-critical for others. As you can imagine, this has the potential for some friction. “The people [who] are actually nagging me about bug reports or asking me for features, [they’re the ones who] are getting paid for using my stuff that I do at home without getting paid,” Stenberg says.

Not that Stenberg is overly concerned. While he acknowledged this has always been a bit of an irritant, he stresses that most people are not annoying or troublesome at all.

Especially those who contribute.

A Community Curl-ing Together

Plenty of folks in the curl community do quite a bit to help. Stenberg said he gets a lot of good stuff back. How much? Curl operates on an eight-week release cycle and gets contributions from 50 to 60 people in every release.

This is impressive, given that the project started with a solo Stenberg scratching that currency rates “itch.” As he put it, “From the beginning, I was convinced that it would become much better with the help of others.” Not just better, but actually used. As Stenberg has written, making curl open source made people want to help out and add features that made the project attractive, and that helped make companies willing to use and trust it. Stenberg says that had the project not been released under an open source license, it would’ve died off in the ’90s, “forgotten and ignored.”

Some of the best code contributions have helped Stenberg expand curl’s utility well beyond his personal experience (and time) would allow. As such, he’s worked hard to make it easy for people to contribute to the project. At times, he said, it would be faster for him to just write a code contribution himself in a way that better fits curl, but he tries to think longer term. “If I help this person do it right, maybe he’ll do it even better next time and it will make it easier for us and improve the project long term more,” Stenberg says.

Twenty-one years later, Stenberg’s little side project has attracted hundreds of contributors and has billions of users. You’re already one of those users. Maybe it’s time to become a contributor? Visit the project website to learn how to become a contributor.

If you or someone you know needs help with a different open source project, visit the AWS Open Source site to learn how open source projects can apply for AWS promotional credits.

Feature image via Pixabay.