When it comes to AI, there's a lot about it that we don't really know. Reverse engineering AI models is a massive industry, and the field itself is called mechanistic interpretability. With that said, we have some understandings on a conceptual level of how some things happen, and an interesting one I came across recently was a strange hallucination from OpenAI's GPT-3.5 when I asked it to translate a word into Irish.

Hallucinations and relations

AIs "relate" topics and concepts together

As a brief primer, irish is one of the official languages of Ireland, and is commonly mislabeled as "Gaelic". Gaelic and Irish are not the same thing, and Irish is the native tongue of a small subset of the Irish population. I'm not from an Irish speaking region of the country though, and so I would struggle in a conversation with a native speaker. I decided to play around with ChatGPT to translate words and phrases into Irish, particularly as I was curious if it had been trained on Irish language data.

With that in mind, a rudimentary overview of how an LLM learns is that with a massive corpus of data, it builds up relationships and concepts between the data contained in it to understand what's inside. It won't typically understand something outside of the data, but it can make guesses based on what it learned in the past. It can then learn patterns and figure out how things typically relate to each other, which in turn enables prompts where it essentially goes back over what it's learned in its neural network and how those words relate, and then builds a response in relation to it.

When asked about words translated to Irish, ChatGPT fared more or less alright, and seemed to have a general grasp of the basics of the language. When I asked it to translate "sandwich" into Irish, though, I came across something pretty funny. It didn't just struggle, it straight up hallucinated, and I think I know exactly why it struggled. It's a great example of what can happen when a relationship is built up in the data that it learned, but a weak enough one that data from other places leaks in too.

First and foremost, the Irish word for "sandwich" is "ceapaire". Everything else in the above screenshot is wrong, but specifically look at "sangwich". "Sangwich" is an Irish slang term for sandwich, and it's just a silly way to say it. It doesn't really come from anything (except possible regional accents), but it's something that will crop up on online forums such as Reddit frequently.

Why it's interesting though is that in Irish subreddits, where the descriptive term for a person from Ireland is "Irish," and the language is also called "Irish," it seems fairly likely to me that ChatGPT managed to conflate the two in the context of this slang term. The Irish (people) referring to a sandwich as "sangwich" was essentially mistaken in the corpus of data as people referring to a sandwich, in Irish (the language) as "sangwich." From there, wires were crossed, and it learned that association.

It's a small example, and one that I haven't found anything quite similar to, but it's still an interesting hallucination that shows just one way that it can happen. The best way to identify these is to try spot inconsistencies and problems in what would be the "edge" of the training set, and smaller languages like Irish are the perfect way to expose them.

AI hallucination is real, and can have worse consequences

Be careful when using an AI

While this example is fun, AI hallucination is very real. When they aren't fully trained on what you're asking, they'll simply fill in the gaps with whatever makes the most "sense" to the AI. In this case, it had a loose relation that it had likely built from social media. The same can happen elsewhere. Just because it learned it doesn't make it correct, and it also doesn't mean that the answer you get is fully accurate either.

When you get an answer from an LLM on a topic you're not familiar with, make sure to research all of the claims that it makes. LLMs are great tools to find out the source of something or to use as a supercharged search engine, but never rely on them for all the right answers. They sometimes get things wrong, and if you don't challenge or verify those responses, you might make the same mistake as well.