Prompting with Images from URLs

You're building an automated product cataloging system for an online marketplace. The system needs to analyze product images and generate accurate descriptions for the catalog. You'll use role-based messages to send an image URL to the model and request both a classification and a description.

A URL string to a photo of a winter jacket has been stored as image_url, and the visualize_image() function has been defined for you to compare the image with the generated description.

This exercise is part of the course

Working with the OpenAI Responses API

Exercise instructions

Add a user message containing both a text prompt asking to classify and describe the product, and the image URL (image_url) using the appropriate content types.
Create the request to the "gpt-5-mini" model using your messages list and extract the output text.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

messages = [{"role": "system", "content": "You are a product cataloging expert who provides concise classifications and descriptions."}]

# Add user message with text and image
messages.append({
    "role": "____",
    "content": [
        {"type": "____", "____": "Classify this product and write a brief but punchy description for our catalog."},
        {"type": "____", "image_url": "____"}
    ]
})

# Create the response
response = ____

print(response.output_text)
visualize_image(image_url)

Edit and Run Code

This exercise is part of the course

Working with the OpenAI Responses API

IntermediateSkill Level

5.0+

Start Course for Free

Building AI applications has never been easier with the Responses API! Learn how to prompt models using a simple interface, best practices for working with reasoning models, and how to build back-and-forth conversations with LLMs.

Exercise 1: The Responses API: a Developer's Best Friend Exercise 2: Your First Responses API Call Exercise 3: Adding Model Instructions Exercise 4: Extracting Information from the Response Exercise 5: Mastering Response Parameters Exercise 6: Experimenting with More Powerful Models Exercise 7: Reasoning About Reasoning Exercise 8: Conversation Chaining Exercise 9: From One Message to Another Exercise 10: Enabling Chat Conversations

LLMs are powerful, but they're even more powerful once tools are introduced! Learn how to integrate web search with your LLM systems with no additional API keys or credentials. Discover that tools can be, well... pretty much anything! Build your own tools for LLMs to call to go beyond the model's base capabilities.

Exercise 1: Response Items: The Building Blocks Exercise 2: Extracting Output Items and Their Contents Exercise 3: Using Items for Custom Handling Exercise 4: Web Search That Just Works Exercise 5: Combining LLMs and Web Search Exercise 6: Building Confidence with LLM Sources Exercise 7: Function-Calling Tools: Defining the Function Exercise 8: The Function-Calling Tool Workflow Exercise 9: Defining a Function for Converting Timezones Exercise 10: Function-Calling Tools: The Full Workflow Exercise 11: Building the OpenAI Tool Definition Exercise 12: Integrating Function-Calling Tools

Move your LLM systems from proof-of-concept to production with three key design paradigms. Learn how to generate consistent and reliable structured outputs, like JSON, using LLMs. You'll also use semantic events to stream updates to your users, creating interactive and enjoyable experiences. Finally, go multi-modal and learn how to combine text and images with OpenAI's GPT-5 models.

Exercise 1: Generating Structured Outputs Exercise 2: Consistent Outputs, Every Time!Exercise 3: Nesting Pydantic Classes Exercise 4: Streaming with Semantic Events Exercise 5: Streaming OpenAI Responses Exercise 6: Streaming Semantic Events Exercise 7: Streaming with Function Calls Exercise 8: Input Flexibility and Multimodality Exercise 9: Role-Based Inputs Exercise 10: Prompting with Images from URLs

Current Exercise

Exercise 11: Prompting with Local Image Files Exercise 12: Congratulations!