5 Key Learnings about AI and ChatGPT in the Enterprise

If 2022 was the year when AI broke through to become a society-changing technology, 2023 has been the year of AI breaking through in the enterprise. Put it this way: generative AI and large language models (LLMs) have become a familiar part of the lexicon for IT departments the world over. Your CIO is now more likely to namecheck ChatGPT than Kubernetes.

Already, it’s been a year of giant strides forward for AI in the enterprise, so here are five things we’ve learned so far.

1. ChatGPT Isn’t the Only Option for Enterprises

In March, OpenAI announced an enterprise version of ChatGPT. But in this case, OpenAI was a fast follower — not the first-to-market. Cohere, a Toronto-based company with close ties to Google, was already selling generative AI to businesses, as I discovered when I spoke to them that month.

“We’re working with developers in organizations, the AI/ML teams, to bring these capabilities into their organizations,” Cohere’s CEO Martin Kon told us. He added that its approach is fundamentally different from OpenAI’s. “OpenAI wants you to bring your data to their models, exclusive to Azure. Cohere wants to bring our models to your data, in whatever environment you feel comfortable in.”

So far, companies are mostly using generative AI to create semantic search engines for their own private data — whether for internal use or for external customers. A related use case is knowledge management (KM). Imagine an employee being able to have a conversation with an AI that was trained on large language models based on company data.

One of the many new AI companies trying to achieve KM in chatbot form is Vectara. Its CEO Amr Awadallah told me that “in five years, every single application — whether that be on the consumer side or in the business/enterprise side — will be re-architected in a way that is a lot more human in nature, in terms of how we express what we are trying to achieve and what we’re trying to do.”

2. The Big Tech Platforms Are Pushing Low-Code Tools

Last month, Google Cloud and Google’s Workspace division announced a raft of new AI functionality. These included Generative AI App Builder, “which allows organizations to build their own AI-powered chat interfaces and digital assistants,” and new generative AI features in Google Workspace.

Google has coined a typically awkward term for applications powered by generative AI: “gen apps.” It claims that gen apps will become a third major internet application category, after web apps and mobile apps.

I’m willing to wager that gen apps as a term will never take off but, regardless, Google’s AI-powered tools will be well used within enterprise companies.

Likewise, Microsoft has been releasing new AI tools, such as Semantic Kernel (SK), described as “an open-source project helping developers integrate cutting-edge AI models quickly and easily into their apps.” SK is in many ways a classic Microsoft low-code tool, only it focuses on helping users do “prompts” for AI chatbots.

3. LLMs Vary Greatly in Size, But Size Isn’t Everything

If you scan Stanford’s HELM website, which measures LLMs in a variety of ways, you’ll see that these models vary greatly in size. There are, simply put, tradeoffs between model size and the speed it can work at.

OpenAI has several models, ranging in size from 1.3 billion parameters (its Babbage model) to 175B parameters (its DaVinci model).

Cohere goes as far as differentiating its model sizes as if they were Starbucks cups: small, medium, large, and extra large.

However, Stanford also measures “accuracy” and in these statistics, size doesn’t seem to matter much:

4. Tools Like Ray Are Helping to Scale AI

In this new era of generative AI, frameworks like Ray are now just as important as Kubernetes in creating modern applications at scale. An open source platform called Ray offers a distributed machine-learning framework. It’s being used by both OpenAI and Cohere to help train their models. It’s also used in other highly-scaled products, such as Uber.

The company behind Ray is Anyscale, whose CEO Robert Nishihara told me earlier this year is very developer-centric. All of Ray’s functionality is designed to be easy to use for developers, he said. This differs from the Kubernetes user experience for devs, he noted, which is notoriously difficult. Ray was designed for Python developers, which is the main programming language being used in AI systems.

Anyscale co-founder Ion Stoica told me that Ray is “like an extension of Python” and — like Python — there are a set of Ray libraries that are targeted towards different use cases. The awkwardly named RLlib is for reinforcement learning, but there are similar libraries for training, serving, data pre-processing, and more.

5. Business Intelligence is Being Reinvented in the AI Age

Just as cloud computing ushered in a raft of “big data” solutions, generative AI is a catalyst for a new wave of data intelligence companies.

I recently spoke to Aaron Kalb, a co-founder of Alation, which styles itself as a “data intelligence” platform and promotes a concept it calls the “data catalog.” This combines “machine learning with human curation” to create a custom store of data for enterprise companies.

Kalb noted that both AI and the common enterprise acronym of BI (business intelligence) are “garbage in, garbage out.” He said that data intelligence is a layer “that precedes AI and BI, that makes sure you can find, understand and trust the right data to put into your AI and BI.”

In this context, he said, taking something like ChatGPT from the public internet and bringing it into the enterprise is very risky. He thinks that data needs to be, well, more intelligent before it is used by AI systems within an enterprise. Also, he doesn’t think that the “internet scale” of ChatGPT and similar systems is needed in the enterprise. Every organization has its own terminology, he explained — that could be industry terms, or things that are very specific to that company.

Summary: The AI Enterprise Has Arrived

It’s hard to believe that it’s been just a year since generative AI burst onto the scene. It all started with OpenAI’s DALL-E 2, which was announced last April and launched as a private beta in July. DALL-E 2 is an image generation service powered by deep learning models, and it was a significant step forward for the industry. Also, last July, a company called Midjourney released its eponymous text-to-image generator.

The AI hype really ramped up in August, though, with the release of Stable Diffusion — another deep-learning text-to-image generator. Unlike DALL-E 2 and Midjourney, Stable Diffusion had a permissive licensing structure. This was around the time people began paying more attention to the models behind all these services — the LLMs. DALL-E 2 used a version of GPT-3, OpenAI’s main LLM at the time.

But, in hindsight, we can see that the image generators were just the appetizer. At the end of November, OpenAI launched ChatGPT, a chatbot built on top of GPT-3.5. This was the catalyst for generative AI to enter the enterprise, and it has quickly taken over IT departments since then.