How Microsoft Finally Came Around to Open Source

Microsoft has come a long way in its effort to initially warm up to and later adopt open source software — from small skunkworks projects to full-fledged adoption and now supporting more than 10,000 engineers contributing to open source.

During a talk at the Linux Foundation’s Open Source Leadership Summit (OSLS) here, two “Jeffs” from Microsoft — Jeff McAffer, director of the Open Source Programs Office at Microsoft, and Jeff Wilcox, principal engineer in Microsoft’s Open Source Programs Office — provided a bit of insight into how things are going as the software giant has scaled to such a large number of people not only dabbling in, but avidly contributing to open source projects.

McAffer said there are 10,000 Microsoft engineers on GitHub, there are 7,000 repositories, 1,000 release requests each year and 27,000 use registrations, which amounts to an overall two times annual growth in Microsoft participation in GitHub.

One reason Microsoft developers are gung-ho about open source is because its CEO is big on open-source technology, Wilcox said.

“Satya [Nadella, Microsoft’s chief executive officer] is so big on open source that people felt like they should just go out and create repos,” he said, noting that Nadella’s thoughts on the importance of open source translated to many engineers as a chance to further their careers by jumping into the fray.

But of course, it wasn’t always this way. Back in the days of Bill Gates and early Steve Ballmer, open source was a dirty word. In fact, as McAffer noted, it was tantamount to a disease at Microsoft. Indeed, Ballmer likened Linux and the GNU General Public License to a “cancer.” This was a stance he later went back on, but at the time, Microsoft felt open source to be every bit of a threat as a debilitating illness. This point was made in more than a couple sessions at this year’s OSLS.

“By any measure, Microsoft’s relationship with open source is complicated,” said Charles King, principal analyst at Pund-IT.

👁 Image

When Linux emerged in the late 1990s, it was hailed by many as an antidote to Microsoft’s “poisonous” influence on personal computing, he said. “Not surprisingly, Microsoft’s leadership, particularly CEO Steve Ballmer, reacted vociferously, even melodramatically, to that threat long after it was apparent that the impact of Linux on desktop computing was minimal, at best,” King added.

Microsoft sort of eased its way into the open-source fold. After Microsoft issued the famed “Halloween” documents, starting with an anti-Linux, anti-open source memo that leaked, the company also started to investigate how to begin interacting with what was quickly becoming a movement in the developer world.

The company launched a Shared Source Initiative and installed Jason Matusow to oversee the effort. Shared Source allowed individuals and organizations to access certain Microsoft source code for reference and for development in some cases.

Matusow ran that project until he became senior director of interoperability at Microsoft. Along the way, Microsoft launched “Rotor,” also known as the Shared Source Common Language Infrastructure, a shared source implementation of the CLI, the core of Microsoft’s .NET framework.

“Rotor” was “A pure cut through the .NET runtime, the Base Class Library, and the C# compiler that aligned with the ECMA standards,” Stephen Walli, a consultant at Docker and former Microsoft executive, told The New Stack. “All the Windows-centric technology was removed (e.g. WinForms, ADO.NET, ASP.NET). It depended on the Platform Adaptation Layer (PAL) to run on Mac OSX and FreeBSD. Microsoft employees did all the work, except for the PAL which was built by Corel in Ottawa to fulfill contractual obligations between the companies. It was released under an academic research license as part of the Shared Source initiative. The license was not an open source license.”

At that point, Microsoft began hiring open-source developers such as Phil Haack (a leading developer of open-source software for Windows), John Lam (creator of IronRuby), Jim Hugunin (creator of IronPython) and others to help foster an open source mentality among the ranks. Other Microsoft luminaries including Mark Russinovich, Scott Guthrie, Soma Somasegar, Scott Hanselman and others helped push for greater open source adoption. And along the way, folks like Walli, who served as program manager for “Rotor,” and Bill Hilf, who served as general manager of Open Source and Platform Strategy at Microsoft helped further the cause.

Also along the way, Sam Ramji, currently vice president of product management at Google, served as senior director of platform strategy at Microsoft, where he led the company’s worldwide Open Source and Linux strategy including corporate strategy, technical research, marketing, and global execution.

McAffer said after Microsoft’s initial phase of denial and then testing the open source waters, the company launched a subsidiary, Microsoft Open Technologies, also known as MS OpenTech, to get even more serious about open source software. Led by Microsoft veteran Jean Paoli with Gianugo Rabellino as director of open source communities, MS OpenTech helped forge better relationships between Microsoft and the community.

One of the unit’s first deliverables was an updated version of Redis on Windows. Redis is an open-source, networked, in-memory, key-value data store.

“At MS Open Tech, we focus on providing our customers with even greater choice and opportunity to bridge Microsoft and non-Microsoft technologies together in heterogeneous environments,” reads a blurb on the MS OpenTech website. “We believe that openness is good for our customers, good for the community and good for our business.”

👁 Image

In 2015, Microsoft folded the MS OpenTech group back into the company because its role was no longer needed. That was just a year after folks said hell froze over and pigs began to fly, as Microsoft committed to open-source its beloved .NET and take it cross-platform to Linux and the Mac. And last year, Microsoft joined The Linux Foundation.

“Today, MS Open Tech has reached its key goals, and open source technologies and engineering practices are rapidly becoming mainstream across Microsoft,” Paoli wrote in a blog post at the time. “It’s now time for MS Open Tech to rejoin Microsoft Corp., and help the company take its next steps in deepening its engagement with open source and open standards.”

He added that “Through the .NET Foundation, Microsoft has open sourced .NET, and dozens of .NET-based projects, and is bringing them to platforms such as Linux. We’ve built a strong open source community around TypeScript, and partnered with projects such as Angular and Ember to bring it to new audiences of developers. We’ve open sourced technologies such as WinJS and the Windows Driver Frameworks.”

Still, bringing MS OpenTech back into the fold, “left a gap, and that’s where we come in,” McAffer said, noting that his office is responsible for helping to automate the process of introducing projects to open source. For instance, “We don’t have time to support 10,000 people on authentication,” Wilcox said.

“We want to use all the GitHub features and use it authentically,” he said. “We tried to replicate everything on GitHub through an internal portal.”

Because of its diverse GitHub user base, Microsoft developed an easy to use repo create a wizard to help users create new repositories. The company also pushes out guidance on when to use private repositories.

“It turns out you can use Git without GitHub,” Wilcox quipped.

In addition, Microsoft uses what Wilcox calls “playbooks” to help frame conversations about open source projects that employees propose. The company also gains insights on projects gathering data into a common place for analysis.

“We have this thing called the ‘Crawler’ that slurps in data and you can run analysis on it,” Wilcox said.

Known as the GHCrawler, the tool is a GitHub API crawler that walks a queue of GitHub entities transitively retrieving and storing their contents, the GHCrawler entry on GitHub says. “GHCrawler is great for: Retrieving all GitHub entities related to an org, repo, or user; efficiently storing and the retrieved entities; and keeping the stored data up to date when used in conjunction with a GitHub webhook to track events,” it said.

Meanwhile, back to Microsoft’s concern over Linux and the floodgates it would open, “Interestingly, the real Linux revolution occurred in the data center, not the desktop — a point that Ballmer and his supporters were never able to reconcile.

Fortunately for Microsoft, that’s an issue whose significance and irony is well understood by Satya Nadella, King noted.

“I believe that the company is working very hard to be regarded as a good citizen in numerous open source communities,” he said. “I’m not sure Microsoft has achieved the position and legitimacy in regards to open source that they obviously desire to achieve but they are in a far better position today than they have ever been before.”

Images: From a presentation given by Microsoft’s Jeff McAffer and Jeff Wilcox, at the Linux Foundation’s Open Source Leadership Summit, taken by Darryl Taft.