1. GPT outputs
We mentioned that it is our responsibility to evaluate and verify what we get out of the GPT. Let's look at how to do that.
2. Incorrect outputs
GPT outputs can contain bias or incorrect information, known as hallucinations, so it is key that we use our expertise and experience to validate the outputs and confirm their accuracy.
3. Validation techniques
Here are three techniques to help with that. We can evaluate the output ourselves based on our domain knowledge, compare the result to an example that we know to be correct, ask an expert to verify the results or use question-and-answering techniques to get more information. Whenever new text is generated, it's important to proofread it before using it since a GPT may not have captured all the nuances.
4. Validate ourselves
Let's take a financial analyst. Their employer has changed the spreadsheet software they use for routine calculations. The analyst wants to transfer these calculations into the new tool, but they need to be in a different format. They decide to use GPT to automate this and compare it with a calculation they've done by hand to confirm that the generated spreadsheet formula is correct.
5. Comparing results
Another analyst is writing an earnings report using a GPT tool. They verify the result by comparing the generated report to a previous one and checking any of the numbers against other company sources.
6. Ask an expert
Here, we have a people team manager creating a week-long wellness program. They use the GPT tool to draft a plan and then connect with their benefits manager and a mental health specialist to confirm the plan is appropriate.
7. Question-and-answering
Finally, a stock manager asked the GPT tool to estimate how many days our product stock will last at the current sales rate. We can then ask follow-up questions to verify this answer, like asking for the methodology behind getting to this result and showing its reasoning step by step. Or, if we notice something is incorrect, we can feed that back to the model to get a different answer.
8. Let's practice!
Ultimately, the technique we employ will vary depending on the use case and as with prompt engineering, we'll get better at this with some practice!