Defining a function for chunking
To be able to batch upserts in a reproducible way, you'll need to define a function to split your list of vectors into chunks.
The built-in itertools module has already been imported for you.
Cet exercice fait partie du cours
Vector Databases for Embeddings with Pinecone
Instructions
- Convert the
iterableinput into an iterator. - Slice
itinto chunks of sizebatch_sizeusing theitertoolsmodule. - Yield the current chunk.
Exercice interactif pratique
Essayez cet exercice en complétant cet exemple de code.
def chunks(iterable, batch_size=100):
"""A helper function to break an iterable into chunks of size batch_size."""
# Convert the iterable into an iterator
it = ____
# Slice the iterator into chunks of size batch_size
chunk = tuple(itertools.____(it, ____))
while chunk:
# Yield the chunk
____
chunk = tuple(itertools.islice(it, batch_size))