Should You Try Small Language Models for AI App Development?

Most enterprises share a common goal: to bring their most critical business operations as close as possible to the audiences they serve. This might mean making it as painless as possible for consumers to order groceries online; helping staff deal with customer queries and orders wherever they meet them on the shop floor; or demonstrating to regulators that a bank is swiftly identifying and dealing with fraud.

In a world that is increasingly embracing AI, this should be straightforward. Enterprises can create innovative applications that can address, and even predict, their audience’s needs. Yet creating these applications can be more challenging than ever. How can the enterprise ensure it creates applications quickly enough to meet demands that evolve as fast as, if not faster than, the technology? How can it ensure those applications will behave as expected and give users accurate, useful insights and experiences? And how can it do this at scale to reach thousands or millions of users?

While industries have been fixated on the potential of large language models (LLMs) powering generative AI (GenAI), small language models (SLMs) and other approaches are often a more effective solution.

The Right Tools for the Job

As with any project, when developing any new application, it’s important to understand its purpose and who it is for — and, increasingly, use this insight to select the right AI tool for the job. For instance, AI doomsayers are quick to repeat the downsides of GenAI powered by LLMs. They can be resource-intensive, and they require careful data management and training to ensure the AI doesn’t put sensitive data at risk, nor suffer from hallucinations or biases that make any conclusions unreliable at best.

But at the same time, there’s no rule saying an LLM is always the answer. While they provide powerful general capabilities, they are not equipped to answer every question pertaining to an organization’s specific business domain. A more specialized SLM, trained on domain-specific data, can be equally, if not more, effective.

Larger Isn’t Always Better

The benefits of SLMs are clear. Their focus on more specific data means they can more easily and accurately answer specific queries, either by themselves or in concert with an LLM. For applications expecting to fulfill a specialized function or answer specialized queries, this can be invaluable.

There are also the advantages of security and privacy. Unlike an LLM that may rely on public data, an SLM can more easily be made proprietary: the algorithm, the training and — most importantly — the data are all controlled by the owner organization.

Whether an organization is concerned about sharing business-critical or personal data beyond its walls, worried that it will accidentally access and use copyrighted or otherwise sensitive data from another source, or fears finding its language model has been corrupted through exposure to conflicting or irrelevant data, an SLM can offer some peace of mind.

Remember the Basics

SLMs have a number of advantages over other approaches to AI. But, like any other technology, this doesn’t mean they are perfect. Instead, enterprises need to understand what approach best fits their needs and then ensure they have everything in place to make that approach succeed.

For instance, an SLM’s effectiveness is still completely reliant on data. If the organization cannot manage data at speed and at scale, it will inevitably find its AI applications are compromised. A key expectation of modern AI is that it works in real time and can increasingly decipher and speak in human-like language. Any data architecture needs to be able to support this, or applications will be missing vital capabilities.

There is also the question of where and how the SLM processes data. SLMs can be implemented directly onto mobile devices, and the ability to operate directly at the edge will be essential in many use cases. This is not only to ensure applications can operate when disconnected from a central server, but also to minimize latency and ensure the fastest possible operation. For instance, take the drive to build more effective autonomous vehicles. Most manufacturers, not to mention passengers, will want this AI to operate immediately, with no lag.

SLMs have a number of advantages over other approaches to AI. But, like any other technology, this doesn’t mean they are perfect.

It is important to note that many foundational models also offer smaller versions. The smaller versions of foundational models offer several advantages, including increased efficiency, lower computational costs and reduced latency, making them ideal for deployment on edge devices and local machines. They are also more interpretable, easier to fine-tune and more environmentally friendly due to lower energy consumption. However, they come with trade-offs such as reduced accuracy, limited knowledge retention and weaker adaptability, often requiring more fine-tuning for specific tasks.

Finally, there is the question of who owns and manages the data itself. Maintaining control over data is critical, not only to ensure privacy and compliance with a range of data protection laws, but to ensure that the data any language model is learning from and using is accurate, complete and entirely trusted by the organization.

Choosing Your Model

Choosing the right model size depends on balancing performance needs with resource constraints. SLMs can offer businesses a host of opportunities and prove that when it comes to AI, bigger doesn’t always equal better.

Choosing models (large or small) is part of the workflow that Couchbase helps to support. Couchbase offers vector search across its entire product line, including support for vector similarity search on mobile devices. The company also launched Capella AI Services to facilitate AI-enabled agent development and deployment workflows. Sign up to try Couchbase Capella for free. A private preview of Capella AI Services is also available.