Ragas context precision evaluation

To start your RAG evaluation journey, you'll begin by evaluating the context precision RAG metric using the ragas framework. Recall that context precision is essentially a measure of how relevant the retrieved documents are to the input query.

In this exercise, you've been provided with an input query, and the documents retrieved by a RAG application, and the ground truth, which was the most appropriate document to retrieve based on the opinion of a human expert. You'll calculate the context precision on these strings before evaluating an actual LangChain RAG chain in the next exercise.

The text generated by the RAG application has been saved to the variable model_response for brevity.

This exercise is part of the course

Retrieval Augmented Generation (RAG) with LangChain

View Course

Exercise instructions

Define a ragas context precision chain.
Evaluate the context precision of the retrieved documents provided to the input query; a "ground_truth" has already been inputted.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

from ragas.metrics import context_precision

# Define the context precision chain
context_precision_chain = ____(metric=____, llm=llm, embeddings=embeddings)

# Evaluate the context precision of the RAG chain
eval_result = ____({
  "question": "How does RAG enable AI applications?",
  "ground_truth": "RAG enables AI applications by integrating external data in generative models.",
  "contexts": [
    "RAG enables AI applications by integrating external data in generative models.",
    "RAG enables AI applications such as semantic search engines, recommendation systems, and context-aware chatbots."
  ]
})

print(f"Context Precision: {eval_result['context_precision']}")

Edit and Run Code