Defining a function for chunking
To be able to batch upserts in a reproducible way, you'll need to define a function to split your list of vectors into chunks.
The built-in itertools module has already been imported for you.
Este exercício faz parte do curso
Vector Databases for Embeddings with Pinecone
Instruções do exercício
- Convert the
iterableinput into an iterator. - Slice
itinto chunks of sizebatch_sizeusing theitertoolsmodule. - Yield the current chunk.
Exercício interativo prático
Experimente este exercício completando este código de exemplo.
def chunks(iterable, batch_size=100):
"""A helper function to break an iterable into chunks of size batch_size."""
# Convert the iterable into an iterator
it = ____
# Slice the iterator into chunks of size batch_size
chunk = tuple(itertools.____(it, ____))
while chunk:
# Yield the chunk
____
chunk = tuple(itertools.islice(it, batch_size))