Effective use of generative AI is being blocked by the so-called “hallucination problem”, when Large Language Models (LLMs) make inaccurate predictions (flat-out fabrications). For regulated and high-risk applications, this is preventing companies from using Generative AI (e.g., ChatGPT) altogether.

This isn’t simply a matter of waiting for bigger, better large language models (LLMs) to arrive. LLMs have a fundamental limitation: they’re sequence predictors that generate tokens (pieces of words or sentences), not concepts. Each token has some non-zero chance of veering away from the realm of correct, factual, and sensible answers, even while appearing to retain plausibility. Once the LLM is off the right path, another similar error can occur, and this error compounds. 

LLMs are Sequence-Based: Limited to Pattern Recognition

At Jaxon, we’re deeply immersed in the nuances of AI, and today, I’m excited to demystify how LLMs operate. LLMs use sequence-based predictions to craft responses and, despite their brilliance, there’s always a non-zero chance of error in their outputs.

The Core of LLMs: Sequence-Based Predictions

At their heart, LLMs like GPT-4 are built on the principle of sequence modeling. Think of them as sophisticated predictors that excel at understanding and generating human-like text. The process begins with a sequence of words or tokens—each token representing a word or a part of a word. The model’s primary task is to predict the next token in the sequence, based on the context provided by the preceding tokens.

Imagine you’re writing a sentence: “The quick brown fox jumps over the lazy.” An LLM trained on vast amounts of text data recognizes the pattern and predicts the next word, “dog,” to complete the sentence. This prediction isn’t random; it’s based on the patterns and sequences it has learned during training.

Patterns in Natural Language

Human language is rich with patterns. We use grammar, syntax, idioms, and common phrases that make our communication predictable to some extent. LLMs are trained on massive datasets, encompassing a wide array of text sources—books, articles, websites, and more. Through this training, they internalize these patterns, enabling them to generate coherent and contextually appropriate text.

When you prompt an LLM, it leverages these learned patterns to generate a response. For example, given a prompt like “Once upon a time,” the model is likely to produce a continuation that fits the narrative style of a story, such as “in a faraway land.” This ability to mimic human-like text is what makes LLMs so powerful and versatile.

The Non-Zero Chance of Error

Despite their prowess, LLMs are not infallible. Every piece of generated text carries a non-zero chance of error. Why? Because the model’s predictions are based on probabilities. For each token in a sequence, the model calculates the probability of various possible next tokens and selects the one with the highest likelihood. However, this probabilistic approach means that there’s always a chance it might choose a less appropriate or contextually incorrect token.

Consider the example where an LLM predicts the next word in the sentence, “She walked into the room and saw a.” The most probable continuation might be “cat,” “dog,” or “man,” but the model could also generate less likely options like “unicorn” or “spaceship,” depending on the nuances of the context and the training data.

The Importance of Context

Context is king in the realm of LLMs. The accuracy of a model’s predictions heavily relies on the richness and clarity of the provided context. Ambiguous or incomplete prompts can lead to less accurate responses. For instance, the prompt “The capital of France is” will almost certainly yield the correct answer, “Paris.” However, a more vague prompt like “The best way to” could generate a wide range of responses, some of which might not align with the intended query.

Embracing the Imperfections

As we continue to harness the power of LLMs, it’s crucial to acknowledge their limitations. The non-zero chance of error underscores the importance of human oversight and context refinement. While LLMs are incredibly advanced, they are tools that augment human capabilities rather than replace them.

At Jaxon, we’re committed to advancing AI technology while maintaining a clear-eyed understanding of its strengths and limitations. By appreciating the mechanics behind sequence-based predictions and recognizing the inherent uncertainties, we can better utilize LLMs to drive innovation and solve complex problems.

Want to learn more? Contact us!