Embedding and storing documents
The final step for preparing the documents for retrieval is embedding and storing them. You'll be using the text-embedding-3-small
model from OpenAI for embedding the chunked documents, and storing them in a local Chroma vector database.
The chunks
you created from splitting the Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks paper recursively have been pre-loaded.
Creating and using an OpenAI API key is not required in this exercise. You can leave the <OPENAI_API_TOKEN>
placeholder, which will send valid requests to the OpenAI API.
This exercise is part of the course
Retrieval Augmented Generation (RAG) with LangChain
Exercise instructions
- Initialize the default embedding model from OpenAI.
- Embed the document
chunks
usingembedding_model
and store them in a Chroma vector database.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Initialize the OpenAI embedding model
embedding_model = ____(api_key="", model='text-embedding-3-small')
# Create a Chroma vector store and embed the chunks
vector_store = ____