Sparse retrieval with BM25
Time to try out a sparse retrieval implementation! You'll create a BM25 retriever to ask questions about an academic paper on RAG, which has already been split into chunks called chunks. An OpenAI chat model and prompt have also been defined as llm and prompt, respectively. You can view the prompt provided by printing it in the console.
This exercise is part of the course
Retrieval Augmented Generation (RAG) with LangChain
Exercise instructions
- Create a BM25 sparse retriever from the documents stored in
chunks; configure it to return5documents in retrieval. - Create an LCEL retrieval chain to integrate the BM25 retriever with the
llmandpromptprovided.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Create a BM25 retriever from chunks
retriever = ____
# Create the LCEL retrieval chain
chain = ({"context": ____, "question": ____}
| ____
| ____
| StrOutputParser()
)
print(chain.invoke("What are knowledge-intensive tasks?"))