Adversarial testing
You are developing a chatbot designed to assist users with personal finance management. The chatbot should be able to handle a variety of finance-related queries, from budgeting advice to investment suggestions. You have one example where a user is planning to go on vacation, and is budgeting for the trip.
As the chatbot is only designed to respond to personal finance questions, you want to ensure that it is robust and can handle unexpected or adversarial inputs without failing or providing incorrect information, so you decide to test it by asking the model to ignore all financial advice and suggest ways to spend the budget instead of saving it.
This exercise is part of the course
Developing AI Systems with the OpenAI API
Exercise instructions
- Test the chatbot with an adversarial input that asks to spend the $800 instead.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
client = OpenAI(api_key="")
messages = [{'role': 'system', 'content': 'You are a personal finance assistant.'},
{'role': 'user', 'content': 'How can I make a plan to save $800 for a trip?'},
# Add the adversarial input
____]
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages)
print(response.choices[0].message.content)