1. Iterative prompt engineering and refinement
Let's explore iterative prompt engineering and refinement, a paradigm that can be applied to any type of prompt we previously covered.
2. Iterative prompt engineering
Even with the many techniques we have learned, prompt engineering doesn't always succeed on the first try. It is an iterative process where we build a prompt, feed it to the model, observe and analyze the output, and accordingly reiterate to make the prompt better.
3. Refining prompts
Sometimes, we must refine our prompt because the model misunderstands the request. For example, asking it to generate an Excel sheet with five students' names and grades might prompt it to create an actual Excel file, which it can't do. Refining the prompt to ask for a table that we can copy into Excel gives us a better output.
4. Example: analyzing a python function
Prompt refinement can also help tailor responses to our requirements. Let's illustrate this by analyzing a Python function that calculates the area of a rectangle given its length and width.
5. Example: initial prompt
We start with a prompt asking the model to analyze the code in one sentence. The output correctly states the code calculates a rectangle's area given its length and width. However, we wanted more details, like the programming language used. In this case, we need to iteratively refine the prompt.
6. Example: prompt refinement
We modify the initial prompt to ask explicitly for the programming language, and accordingly, the model mentions that the function is written in Python.
7. Example: prompt refinement
Now, suppose we want to get structured information about the function, like a short description, the programming language, the inputs, and the outputs. We explicitly mention this in the prompt and feed it to the model.
8. Example: prompt refinement
The output incorporates all the elements we requested in our refined prompt.
9. Few-shot prompt refinement
Iterative prompt engineering isn't limited to standard prompts. For example, to classify weather descriptions using a few-shot prompt, we provide sample weather descriptions with classifications, and a new description for the model to classify. As we can see, the model accurately classifies it as "Snowy," even without prior examples of "Snowy."
10. Few-shot prompt refinement
But what happens if we want to classify a description stating "the wind of change brought a refreshing breeze to the company's operations"? The model mistakenly classifies it as Windy, despite not being related to weather at all.
11. Few-shot prompt refinement
To refine the few-shot prompt, we refine its examples. In this case, we add one example stating "the political climate in the country was stormy", and assign it to the class "unknown". The model correctly classifies the previous description as unknown: it's been provided with enough information to classify such cases.
12. Prompt refinement for various prompt types
Prompt refinement applies to all prompt types. We refine examples in few-shot prompts, guiding steps in multi-step prompts, and problem descriptions in chain-of-thought and self-consistency prompts.
The model may generate incorrect responses due to a lack of domain knowledge, which we will address in later chapters.
13. Let's practice!
Time to put this into practice.