We started this series by raising the question: are language models just parrots? In that context, we discussed the notion of non-deterministic, probabilistic, and stochastic (save this one for the next dinner party!). Let’s explore some of the implications of this.
As we touched upon in the first article, it is not easy to know how language flows: you cannot necessarily guess what your interlocutor is going to say next. Thus human speech, as used by humans, can be seen as inherently probabilistic itself.
Human speech is inherently unpredictable
The beauty of the transformer architecture / large language models (LLMs) is that we have successfully captured this essence in a machine. Well, fully successfully? Not quite, as we shall see.
We’ve used the term ‘next-word-predictor’ to describe an LLM, and in the strictest sense, that’s how the architecture is implemented - one word at a time. If you think about it, it seems quite miraculous that this alone can entail the generation, by these LLMs, of coherent, comprehensible, even cogent language!
But this whole next-word prediction concept does throw up a bunch of questions:
How can, given just a sequence of words, an algorithm reliably predict the next word?
And since this happens word after word, with the current sequence of words passed in as input, what if there’s one somewhat badly predicted word which then takes the rest of the sentence, even paragraph or the whole text, completely astray, hopelessly off track?
How does one-word prediction add up to complex reasoning and coherent arguments?
[Feel free to insert your own questions here!]
And you will be completely right to ask these questions. For, these go to the heart of some of the limitations of the current architecture.
Anyone who has used a chatbot such as ChatGPT for a considerable amount of time would have noticed this. Given a query about something, the model sometimes comes up, sounding cocksure and confident, with a completely nonsensical and non-factual answer. In the LLM space, an old word has been repurposed to describe this: hallucination.
No, AI has not been fed psychedelic substances, and the LLMs are certainly not having visions, at least, I’d say, in the way we understand human hallucination! It’s essentially the consequence of the probabilistic nature.
But again, if human language as used by humans is unpredictable, do we not as well ‘hallucinate’ in this sense of the meaning? This of course brings up another closely related point: bias.
How do we ensure the LLMs are, as a lot of AI commentators are wont to say, ‘not biased’? Which of course throws up a more fundamental question: what is bias? Don’t we all think of a politician who we disagree with as being ‘biased’? In the same way, those who oppose our views would say the same about politicians we support - again as being ‘biased’?
This is a thorny issue, and people may never agree on this. My main point is, given these language models are being built to engage in, via language, the complex and messy reality of human life, we should perhaps be a bit forgiving over the idea of LLM bias.
At least in subject matters where we ourselves barely agree with each other. But let’s return to subject matters where it is easier to find common ground, or are based in fact, rather than opinion.
Even here, LLMs can hallucinate. This is for a host of reasons, but let’s look at the main ones.
Lack of data - if the data the LLM has been trained on does not have anything about a particular topic or fact, the LLM is of course not in a position to come up with the correct answer.
Lack of sufficient context - if the question asked of the LLM does not provide it sufficient context, again it has no way of giving the correct response.
The current architecture, being probabilistic, means it just picks one of the next likely words, even if of a lower probability (who knows the right probability cutoff?) and thus ends up making stuff up. There hasn’t been a satisfactory resolution to this problem at the architectural level.
But even without that, we have some measures to mitigate the problem, as we saw in the last couple of articles, and we shall in some of the next ones.
