1. Mitigating model limitations
Hello again!
2. Introduction to model limitations
Every model, regardless of its sophistication, has its set of limitations. These limitations stem from the data it's trained on. By recognizing these, we can craft prompts that navigate around these pitfalls and evaluate outputs with a critical eye.
3. The reversal curse
ChatGPT’s knowledge base is imperfect and often quite strange. This is perfectly demonstrated with a recent viral example, highlighting the reversal curse.
Let’s ask GPT-4, the best language model currently available, “Who is Tom Cruise’s mother?” It will respond with “Mary Lee Pfeiffer”, which is correct.
4. The reversal curse
But if you ask “Who is Mary Lee Pfeiffer’s son?”, ChatGPT doesn’t know. As Andrej Karpathy recently highlighted, it shows that ChatGPT’s knowledge is almost one-dimensional. You have to ask questions from a certain direction to get the answer. Unlike understanding a car and its respective components, we still don’t know exactly how language models work.
5. Biases - a mirror of Society
What we do know is that language models learn from vast amounts of data sourced from the internet. Consequently, they can inherit and sometimes amplify biases present in the data. Recognizing biases in outputs is crucial to avoid perpetuating stereotypes or misinformation.
If you ask ChatGPT, "Who typically cooks in a household?" and it responds with a gendered answer, it showcases the bias it might have absorbed from historical or cultural data. A more neutral answer would acknowledge that anyone, regardless of gender, can cook in a household.
6. Hallucinations - when the model imagines
Hallucinations in the context of LLMs refer to instances when the model confidently provides information that isn't accurate. It might "imagine" details or facts, leading to incorrect outputs. As models like ChatGPT are improving over time, it can be challenging to show concrete examples. Nonetheless, let’s take a closer look at what a hallucination could look like.
7. Hallucinations - when the model imagines
If you ask: “Who was the only survivor of the sinking of the Titanic?”
ChatGPT responds: “The only survivor of the sinking of the Titanic who is widely recognized as such was Violet Jessop.” This is a hallucination.
Asking ChatGPT to provide sources of information often leads to correcting itself.
8. Hallucinations - when the model imagines
We can ask the follow-up question: “Can you provide sources? Really think about it.”
ChatGPT apologizes for the confusion and highlights the misconception.
Recognizing and cross-referencing ChatGPT’s answers is crucial to ensure factual correctness.
9. Overfitting - echoing the data
Overfitting occurs when a model is too closely tailored to its training data, making it less effective at generalizing to new, unseen data. In essence, the model might echo what it has seen in the past, rather than providing a balanced response.
10. Overfitting - echoing the data
An example of overfitting is humor. Comedy challenges large language models like ChatGPT. In June 2023, researchers found that if you ask ChatGPT to tell you a joke, 90% of 1,008 generations were the same 25 jokes. These responses were likely learned during ChatGPT’s training.
11. Overfitting - echoing the data
The model currently sucks at discontinuous tasks that require a creative leap in progress. Examples include writing jokes, developing scientific hypotheses or creating new writing styles. My advice is to stick to incremental tasks. These are solved sequentially, adding more information in a gradual setting. Examples include writing summaries, answering questions or imitating your writing style through techniques such as one-shot or few-shot learning.
12. Let's practice!
Dive into the exercises designed to challenge your understanding of biases, hallucinations, and overfitting. By understanding its limitations, we become better users, ensuring that our interactions with ChatGPT are informed, critical, and productive.