AI for Dev Tools: Create Software Requirements with Userdoc

AI coding tools are now an established part of software development, but generative AI is also spreading out to other parts of a developer’s workflow. Userdoc is a startup that has created an AI-assisted service for creating software requirements. At the AI Engineer Summit in San Francisco last week, I sat down with its co-founder and lead developer, Chris Rickard, to discuss why he created Userdoc and how it will help developers.

Requirements — determining what a piece of software will do for one or more user “personas” — is typically the first thing done in a new project.

“A very large part of building good software is getting good requirements and ensuring that you’re building the right thing,” Rickard told me. “I was really interested in how AI can help facilitate that process and essentially catch issues that might ultimately end up as bugs; or, even worse, giant features of systems that don’t solve the original problem and are not even needed.”

One of the benefits of getting AI to create requirements is that you can “prime” it with information about your business, so that the AI can use that as context for the requirements. Rickard showed me an example of a grocery delivery service using Userdoc to build requirements for a delivery driver. He said that in such a case, the AI provides a kind of “governance” for developers as they build their system.

“The artificial intelligence in this example is sort of acting like a Business Analyst,” said Rickard, “and it understands the other ramifications of doing this.”

Why Not Just Use ChatGPT?

This is all well and good, but how is using Userdoc’s AI to generate software requirements different from using, say, ChatGPT to do the same thing?

Rickard replied that there’s a risk that ChatGPT will “make something up.” He added that Userdoc uses GPT-4 under the hood, but that business context is the key.

“So when we’re talking to GPT-4,” he said, “we’re sending up [information about] all the other relevant features of your system.”

He admitted that ChatGPT could achieve the same thing, but you’d need to construct “a huge prompt” to get a similar response to Userdoc.

For each Userdoc project, there are a series of “user stories” that often equate to the number of features in a piece of software. So there could be hundreds of features, and that’s where it gets complex, says Rickard.

“One of the big pain points we are trying to solve here, is [that] if a system has hundreds of features […] there’s a lot you need to think about and remember; and a big cause of software overruns and software costs is essentially these really little things that people don’t figure out up front.”

How Userdoc Works

Setting up a new Userdoc project is done through a wizard, helping you list the users and their various goals. Often this set-up work is done by product owners, product managers and business analysts — and after that, the project is handed off to the developers. But Rickard says that Userdoc also helps developers directly.

“So, once the features and stuff have been created, […] you can synchronize this with project management tools. So it connects to Jira, or Azure DevOps, [or] anywhere that creates a work item [that] goes to ‘to do’, ‘in progress’ and ‘done’; and that synchronizes all this information of what the feature is.”

Rickard noted that in some situations, software developers help build the requirements as well. So it depends on the business needs as to when and how developers will use Userdoc. Often it’s a good reference when returning to a project, too.

“We work with a lot of agencies that have a lot of concurrent projects,” he said, “and the software developers — and even designers — end up using this [Userdoc] as the jumping back point on a project after two months. You’ve got this great description of what needs to be done.”

Rickard added that the information in Userdoc can also be used as “the long-term living documentation for the system.”

“So if you start in this fashion, of having quite detailed requirements that you then keep up to date, and that’s where the business refers back to, then this becomes a source of truth [about a software system].”

LangChain Thoughts

Since we were talking together at the AI Engineer Summit, I wanted to gauge how much Rickard was using the current hot tools of the AI engineering space. Since he’s using GPT-4, I asked whether he is using LangChain as a communication layer for OpenAI’s LLM?

“I did probably spend a couple of weeks with the first prototype [of Userdoc] using LangChain,” he replied. “But I found that I wanted to understand what was happening behind the scenes; and LangChain is amazingly powerful, but what it brings to the table […] is handy, tried and true methods of doing certain tasks — such as reading a document, breaking it into little bits, sending it over to an LLM, and asking questions. But […] I was learning LLMs at the same time I was learning LangChain, [and] it was hard to understand where one started and one finished. So I just wrote my own communication layer for talking to GPT-4, and it significantly helps.”

How Userdoc Might Expand

So if Userdoc can, over time, become the source of documentation for a piece of software, could it also be used to power a help chatbot on the customer side (for that grocery delivery service, for example)?

“The answer is yes, definitely,” Rickard replied. “The main concern is really the internal versus external knowledge, and what people want to expose.”

He pointed out that Userdoc is bootstrapped at the moment, but if he was to expand the team then his dream would be “to get more into the big boring part of compliance.” So he’d rather go in that direction than do what everyone else is doing — create a chatbot for a consumer-facing website or app.

“I really like the idea of ensuring [that] what was agreed by the business was actually what ended up being made,” he explained. He added that there are automated tests that developers can use today, but he’d like to use AI to augment that process.