1. Vector databases for embedding systems
Welcome back! In this chapter, you'll learn how to enable use cases with larger datasets by efficiently storing and querying vectors using a vector database.
2. Limitations of the current approach
So far, we've created embeddings using the OpenAI API and stored them in-memory. Because each embedding contains 1536 float values, this results in around 13kB of data for each embedding, which becomes impractical to load for 100,000s or millions of embeddings. We also recalculated these embeddings with every query rather than storing them for later use.
To compare similarity, we computed cosine distances for every embedded document and sorted the results, which are both slow processes that scale linearly. To enable embeddings applications with larger datasets in production, we'll need a better solution: vector databases!
3. Vector databases
Here's a typical embeddings application: the documents to query are embedded and stored in the vector database. A query is sent from the application interface, embedded, and used to query the embeddings in the database. This query can be a semantic search query or data to base recommendations on. Finally, these results are returned by to the user via the application interface.
Because the embedded documents are stored in the vector database, they don't have to created with each query or stored in-memory. Additionally, due to the architecture of the database, the similarity calculation is computed much more efficiently.
4. NoSQL databases vs. SQL databases
The majority of vector databases are what's called NoSQL databases, which contrasts conventional SQL, or relational, databases.
SQL databases store data in tables with rows and columns. NoSQL databases don't use tables, and can be structured in several different ways to enable faster querying. Three examples of NoSQL architectures are shown, including key:value, document, and graph databases.
5. Components to store
Vector databases aren't only used for storing the embeddings; the source texts are also commonly stored. For vector databases that don't support this, source texts must be stored in a separate database and referenced with an ID.
Metadata is also stored in the database, including IDs and external references, and additional data that could be useful for filtering the query results. It could be tempting to store the source texts in the metadata, but this should be avoided. Metadata must be small to be practically useful, so adding a large amount of text data will greatly degrade performance.
6. The vector database landscape
So what are the options when it comes to vector databases? Fortunately, there are plenty of options out there to suit almost every use case; here are some of the most popular.
When deciding which database solution to go with, there are several factors to consider.
7. Which solution is best?
Vector databases can be managed, which is a turnkey solution that will "take care of itself" while you concentrate on application features. It does come at a price, but can be a sensible investment. Alternatively, you can self-manage and deploy your own server and database, either in the cloud or on-premise. This is usually cheaper, but requires time and expertise.
Open-source solutions provide greater flexibility in usage, and can be more cost-effective if budgets are tight. In contrast, commercial solutions offer better support, more reliable performance, advanced features, and better regulatory compliance.
We should also ask ourselves if our data lends itself to a particular database type. As discussed, there are several NoSQL architectures utilized by different database providers.
Finally, does the use case depend on specific functionality, like embedding and storing both text and images for a multi-modal application?
In this course, we'll be using Chroma, as it's open-source and quick to set up. At the end of this course, we'll provide resources for learning more about other solutions.
8. Let's practice!
Time for some practice!