Cody’s Bot Serves Enterprise Info in Slack Workspaces

Like many people in tech, I’ve been writing about AI recently, with a focus on the projects that have been spawned in the wake of Large Language Models (LLMs). I am particularly interested in the business pattern of an AI processing your own data, and then answering questions about it — which led me to look at Cody.

The company behind the technology, also called Cody, has a strong immediate pitch:

“Cody is an intelligent AI assistant like ChatGPT — with the added benefit of being able to train it on your business, your team, your processes, and your clients with your own knowledge base.”

So the win here is to have a ChatGPT-like interface answering questions about your business. The original source graph worked with code in source code repositories. But what else is a “knowledge base?”

Well, Slack has knowledge locked away in conversations — and extracting conclusions from English language is a strong point for GPT4. Normally, accessing your channel timeline is sufficiently limited that most people do little other than basic searches. You can search for the term “budget” back in time (if you are a fully paid-up Slacker) and that will work, but not much more than that. If Cody is able to ingest a busy channel (that is, convert the conversation into a queryable knowledge base), all the better.

Hence I was very happy that Cody could work with Slack — and then a little deflated when it was not immediately available. But recently the feature dropped, so I thought I’d check it out. Unfortunately, they don’t quite complete the job, and won’t actually read from the channel yet — they just allow access to existing Cody knowledge bases. This is a pity, but things move fast in the AI world. Slack themselves are clearly looking at channel-based learning, so it is clear there is a very solid application here. Either way, let’s cover adding a Cody bot to Slack as this is how all Slack integrations generally work.

I had an indifferent journey using Slack integrations in earlier times, but the basic idea is that Slack allows another platform to interact with a channel, usually via a bot. Many people who started collaborating with Slack back in the day could configure an inbuilt bot; for example, to make disapproving comments if a swear word was used in a conversation by your team.

So I joined Cody, and installed the integration:

👁 Image

The usual two-way trust dance of acceptance between the two platforms (I notice that Cody is not an approved app within Slack) ensued, with Cody wanting the user to create an API key:

👁 Image

Once the key is set within Slack (again, the two-way dance) we are all set to chat with Cody.

On the Slack side, you get to assign a Cody bot to the channel. A channel is already a defined set of members, so no further user management is needed. I don’t think Slack counts a bot as a separate participant when it charges your bill. But even if it doesn’t now, it might in the future — beware.

Once you choose your channel and type the “assign bot” command, you are met with a slightly surprising choice:

👁 Image

That would heavily imply that a different LLM is being used, a different account or perhaps different learning approaches. Some people may balk at this separation of “factual” and “creative” before they have even asked a question. I think it might imply that the creative bot was not drawing anything from your knowledge base. You cannot as yet continue a conversation in the same thread — you must start a new one.

Now, as I mentioned Cody doesn’t use your channel as a knowledge base yet, so what can we expect when someone hits this mark? I think a great deal of project know-how is caught up in Slack conversations, but it will take a good AI to extract. Mixed in with all the nuanced detail are jokes, comments on cake, and excitement about the latest streaming shows. If you tell people they must restrict all conversations to work, you will kill any channel instantly. Think of a query you might want to ask:

“Who in this channel knows most about our last deal with Skynet?”

This is the type of question that requires analysis of an entire channel, as well as understanding the mechanics of Slack (understanding who is talking, and about what, and when). It could be argued that Slack (i.e. Salesforce) would actually be foolish to let anyone else walk off with this prize. This detail of comprehension is still at the frontiers of GPT, but these frontiers are falling quite fast.

Privacy from the AI

Now the question of privacy does have to be looked at. Would people be happy chatting in a  channel knowing that AI will mine it later? Clearly, the platform processing the Slack data can see your information here, though it is in their best interest to take good care of it. I think I heard (from a comment on a changelog podcast) that everything can be done on the fly with nothing stored, but this is probably not an optimal solution – especially if you want to ask follow on questions.

The issue about trust and how your data is handled must be made in parallel with transparency, such as telling you what data has been ingested. This is going to matter more and more as we work with AI closer to our own documents and privacy boundaries. It will matter what has been seen as much as what hasn’t been read. You can imagine a future court trial, with an AI in the dock, having to confess to what it knows.

I promise to return to the area of mining Slack when solutions have grown and matured, as I guarantee this will be a fruitful area.