Metadata filtering
1. Metadata filtering
Pinecone provides a unique infrastructure for efficiently serving vectors querys, but even with this, querying can become slow if the index contains a lot of vectors. One way to combat this is with metadata filtering.2. Metadata filtering
Vectors often have associated metadata stored in the index. This metadata can be strings, numbers, Booleans, or true/false values, and lists of strings. This metadata can be used to filter the vectors that are searched through during the query, reducing the search space and query latency.3. Metadata filtering
To perform metadata filtering, we need to use the filter parameter in the .query() method. This argument takes a dictionary where each key is the metadata to filter, and their values specify how to filter them. In this case, 'year' is set to 2019, so records with the 'year' metadata of 2019 will be filtered and queried. There's also a second metadata filter, where 'genre' is set to another dictionary where the key is the $eq operator, and the value is what the operation is performed on. $eq is the equality operator, so this filter selects records with a 'genre' equal to 'documentary'. The two filters here show two methods of performing an equality filter, so you might be wondering why the dictionary and operator version is useful.4. Metadata filters
Pinecone provides a number of different operators to perform sophisticated filtering operations. These include not equal to, greater than, less than, greater than or equal to, and others. Let's have a go at filtering for records with 'year' greater than 2019 with the $gt operator.5. Metadata filtering - greater than
We start with our filter argument and open a dictionary. Next, we use the 'year' key to specify that we want to filter using the 'year' metadata, and finally, use the $gt operator and 2019 inside another dictionary to filter for later years. We're also going to use the include_metadatas argument to include the metadata in the result, so we can verify that the filter worked. There we have it, from the metadata, we can see that the filter was successful in selecting a record from after 2019.6. Let's practice!
You're going great! Time to master metadata filtering with some exercises!Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.