Retrieving vectors

1. Retrieving vectors

Welcome back!

2. Recap...

Now that we've created a serverless index with an appropriate dimensionality,

3. Recap...

and added our records, including metadata, it's time to talk about fetching and querying the vectors.

4. Accessing vectors

There are two main methods for accessing vectors from a Pinecone vector database: fetching and querying. Fetching involves finding and returning a record using its ID. This is normally done to explore and verify particular records from the index. Querying involves sending a vector to the index, and returning the vector or vectors that are most similar to it. If vectors were books, fetching would be like using the ISBN, or International Standard Book Number, to find a book. Querying is like searching for books that are similar to one we already own. We'll focus on fetching vectors in this video and querying vectors in the next video.

5. Fetching vectors

To fetch records based on their IDs, we can call the .fetch() method, passing it a list of record IDs to return. We can see the result contains a dictionary under the 'vectors' key that contains one entry for each record returned. From these entries, we can access the vector values and associated metadata. There's also a 'usage' key, which shows we've used one read unit when fetching our vectors. Let's talk about read units.

6. Read units

Read units, or RUs, are a Pinecone-specific measure of the resources consumed during reading operations. Fetching, querying, and listing vectors all consume read units. For fetching, 1RU is consumed for every batch of 10 records retrieved. Fortunately, Pinecone's starter account provides a generous monthly quota of read units for developing and testing solutions before scaling.

7. Fetching vectors from namespaces

Finally, we can also fetch vectors from specific namespaces by specifying the namespace argument. We can see that the previously blank namespace output has now been populated. Later in the course, when we come to storing and querying vectors across multiple namespaces, this functionality will become invaluable.

8. Let's practice!

Time to practice!

This exercise is part of the course

Vector Databases for Embeddings with Pinecone

IntermediateSkill Level

4.8+

Start Course for Free

Explore the mechanics behind Pinecone's vector database, from pods and indexes to comparing it with other databases. Learn to differentiate pod types, acquire API keys, and initialise Pinecone connection using python. Finally, you’ll learn how to create Pinecone indexes, exploring different parameters such as dimensionality, distance metrics, pod types, and others.

Exercise 1: Introduction to Pinecone indexes Exercise 2: Creating a Pinecone client Exercise 3: Your first Pinecone index Exercise 4: Managing indexes Exercise 5: Connecting to an index Exercise 6: Deleting an index Exercise 7: The Pinecone ecosystem Exercise 8: Vector ingestion Exercise 9: Checking dimensionality Exercise 10: Ingesting vectors with metadata

Get hands-on with Pinecone in Python, where we explore the practical side of using Pinecone for managing indexes, adding vectors with metadata, searching and retrieving vectors, and making updates or deletions. Gain a solid grasp of the key functions and ideas to smoothly handle data in the Pinecone vector database.

Exercise 1: Retrieving vectors

Current Exercise

Exercise 2: Querying vs. fetching Exercise 3: Fetching vectors Exercise 4: Querying vectors Exercise 5: Returning the most similar vectors Exercise 6: Changing distance metrics Exercise 7: Metadata filtering Exercise 8: Filtering queries Exercise 9: Multiple metadata filters Exercise 10: Updating and deleting vectors Exercise 11: Updating vector values Exercise 12: Updating vector metadata Exercise 13: Deleting vectors

In this chapter, learners delve into optimizing Pinecone index performance, leveraging multi-tenant namespaces for cost reduction, building semantic search engines, and creating retrieval-augmented question answering systems using Pinecone with the OpenAI API. Through these lessons, learners gain practical skills in performance tuning, semantic search, and retrieval-augmented question answering, empowering them to apply Pinecone effectively in real-world AI applications.

Exercise 1: Batching upserts Exercise 2: Defining a function for chunking Exercise 3: Batching upserts in chunks Exercise 4: Batching upserts in parallel Exercise 5: Multitenancy and namespaces Exercise 6: Namespaces Exercise 7: Querying namespaces Exercise 8: Semantic search with Pinecone Exercise 9: Creating and configuring a Pinecone index Exercise 10: Upserting vectors for semantic search Exercise 11: Querying vectors for semantic search Exercise 12: RAG chatbot with Pinecone and OpenAI Exercise 13: Upserting YouTube transcripts Exercise 14: Building a retrieval function Exercise 15: RAG questions answering function Exercise 16: Congratulations!