Generating Structured Outputs

1. Generating Structured Outputs

You're doing great so far!

2. Coming up...

In this chapter, we'll be stepping it up a gear to look at some key design patterns for unlocking production usage - starting with structured outputs!

3. The Need for Structure

Imagine you're building an application for helping people learn languages. To improve the experience,

4. The Need for Structure

you want to use AI for grading quiz answers and providing feedback. Generating the text in different languages is something an LLM can do out-of-the-box, but to be able to render this text in the application user interface, the output

5. The Need for Structure

must be in a strict JSON format, common to many production applications. The slightest change in the JSON formatting could break the application and upset your new customers. How do we get Large Language Models, or LLMs, to generate a strict, structured output given that they are inherently non-deterministic?

6. Part 1: Defining the Output Schema

The first step for generating structured outputs is defining the output schema. This is effectively the template that the LLM populates with the generated content. We'll be using the pydantic Python library to do this, which is supported by the openai library and is great for data validation. We use pydantic's BaseModel class to define a class called QuizResult, which contains three fields: score, passed, and feedback. We also specify the field types so the model understands what type of data to include. We'll pass this class to the model, and it will return its output parsed into this form. So it's 100% clear to the model exactly what it should generate for each field, it's also good practice to

7. Part 1: Defining the Output Schema

annotate each field using pydantic's Field class. Now, the model knows how to score, what the pass mark is, and guidance for providing feedback.

8. Part 2: Sending the Request

Next, we create our request to the model. Up until now, we've been using client.responses.create(). With structured outputs, because we are looking to parse outputs in a particular format, we use client.responses.parse(). We specify the model, as before, and then our messages. Here, we've written an instruction on how the model should behave as an instruction, then the user's quiz results as the input. Finally, we pass our pydantic class to the text_format argument to enforce the output structure.

9. Part 3: Extracting the Results

From the response, we can extract the parsed output using the .output_parsed attribute. This extracted object has attributes for each of the fields we defined, so we can extract and print the score, passed, and feedback information. Our results as structured, predictable, and now in a form that could be used in a production application.

10. More Complex Data Structures

It might be that we need more detailed information on the mistakes that learners have made so we can send them specific practice resources and materials. We define a new Mistake class, again, passing it pydantic's BaseModel. This will store the word being tested, the student's answer, and the correct answer. We can then capture this information inside a quiz result class like we had before. We define DetailedQuizResult with the same fields before, but now we can add an additional field called mistakes, that will be a list of Mistake instances. In other words, we will not only get final results on the quiz, but a list of mistakes and their correct answers.

11. The Second Request

The request is the same as before, but sending our DetailedQuizResult class this time.

12. The Second Request

We'll start by extracting the parsed output as before, along with the quiz final score and passed boolean value. We'll also loop over the new mistakes list and extract the word, student answer, and correct answer using their respective attributes. Now, we get a clear output showing the mistakes made in the quiz.

13. Summary

Let's quickly recap the structured outputs workflow given its importance in production applications. First, define the output schema using pydantic classes, using fields to add descriptions to each input argument. Second send the results to the Responses API using the text_format argument. Finally, we get the structured output from the response using the .output_parsed attribute, and information can be extracted their specific attributes.

14. Let's practice!

Time to give this a try!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.