Ingesting vectors with metadata

It's ingesting time! You'll be ingesting vectors, which is a list of dictionaries containing the vector values, IDs, and associated metadata. They're already been provided in a format that can be directly inserted into the index without further manipulation.

Here's another reminder about the structure of vectors.

vectors = [
    {
        "id": "0",
        "values": [0.025525547564029694, ..., 0.0188823901116848]
        "metadata": {"genre": "action", "year": 2024}
    },
        ...,
]

This exercise is part of the course

Vector Databases for Embeddings with Pinecone

Exercise instructions

Initialize the Pinecone connection with your API key.
Connect to your index called "datacamp-index".
Upsert vectors to the index.
Print the index's descriptive statistics.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Initialize the Pinecone client with your API key
pc = Pinecone(api_key="____")

# Connect to your index
index = pc.____("datacamp-index")

# Ingest the vectors and metadata
____

# Print the index statistics
print(____)

Edit and Run Code

This exercise is part of the course

Vector Databases for Embeddings with Pinecone

IntermediateSkill Level

4.8+

Start Course for Free

Explore the mechanics behind Pinecone's vector database, from pods and indexes to comparing it with other databases. Learn to differentiate pod types, acquire API keys, and initialise Pinecone connection using python. Finally, you’ll learn how to create Pinecone indexes, exploring different parameters such as dimensionality, distance metrics, pod types, and others.

Exercise 1: Introduction to Pinecone indexes Exercise 2: Creating a Pinecone client Exercise 3: Your first Pinecone index Exercise 4: Managing indexes Exercise 5: Connecting to an index Exercise 6: Deleting an index Exercise 7: The Pinecone ecosystem Exercise 8: Vector ingestion Exercise 9: Checking dimensionality Exercise 10: Ingesting vectors with metadata

Current Exercise

Get hands-on with Pinecone in Python, where we explore the practical side of using Pinecone for managing indexes, adding vectors with metadata, searching and retrieving vectors, and making updates or deletions. Gain a solid grasp of the key functions and ideas to smoothly handle data in the Pinecone vector database.

Exercise 1: Retrieving vectors Exercise 2: Querying vs. fetching Exercise 3: Fetching vectors Exercise 4: Querying vectors Exercise 5: Returning the most similar vectors Exercise 6: Changing distance metrics Exercise 7: Metadata filtering Exercise 8: Filtering queries Exercise 9: Multiple metadata filters Exercise 10: Updating and deleting vectors Exercise 11: Updating vector values Exercise 12: Updating vector metadata Exercise 13: Deleting vectors

In this chapter, learners delve into optimizing Pinecone index performance, leveraging multi-tenant namespaces for cost reduction, building semantic search engines, and creating retrieval-augmented question answering systems using Pinecone with the OpenAI API. Through these lessons, learners gain practical skills in performance tuning, semantic search, and retrieval-augmented question answering, empowering them to apply Pinecone effectively in real-world AI applications.

Exercise 1: Batching upserts Exercise 2: Defining a function for chunking Exercise 3: Batching upserts in chunks Exercise 4: Batching upserts in parallel Exercise 5: Multitenancy and namespaces Exercise 6: Namespaces Exercise 7: Querying namespaces Exercise 8: Semantic search with Pinecone Exercise 9: Creating and configuring a Pinecone index Exercise 10: Upserting vectors for semantic search Exercise 11: Querying vectors for semantic search Exercise 12: RAG chatbot with Pinecone and OpenAI Exercise 13: Upserting YouTube transcripts Exercise 14: Building a retrieval function Exercise 15: RAG questions answering function Exercise 16: Congratulations!