Introduction to querying AI Models with Databricks
1. Querying AI models with Databricks
Welcome back, let's discuss some of the AI functionality that Databricks provides.2. Foundation Models
The AI models that we will learn how to query are Databricks Foundation Models. Foundation Models are large, pre-trained neural networks that are trained on a broad range of data allowing them to be useful in multiple different subjects. They are designed to learn general patterns in language, images and other types of data and they can be hosted on Databricks or outside of Databricks.3. Popular chat completion models
A chat completion Model is a Large Language Model (LLM) that allows the user to engage in a chat conversation with an AI model. The user sends messages to the model and receives intelligent replies from the AI agent. ChatGPT, Claude, and Gemini are some chat completion models hosted outside of Databricks. In this course, we will be working with the Databricks Meta Llama model hosted on Databricks.4. Databricks Meta Llama
The model we use in this course is the Meta Llama Instruct Model. It was built and trained by Meta and it is hosted on Databricks infrastructure, making it easy to access using the Databricks SDK. It is competitive in performance with equivalent OpenAI GPT models.5. Serving endpoints
Databricks has a service called Serving Endpoints that allows us to make API requests to deployed machine learning models. They already exist for Foundation Models hosted on Databricks and they can be created for external and custom models. We can use the Databricks SDK to query a specified foundation model.6. Serving endpoints API
The Databricks SDK allows us to query AI models using serving endpoints. This can be accessed via the `serving_endpoints` attribute of the `WorkspaceClient`.7. Chat message roles
The `ChatMessageRole` provides context to the model on what kind of message we are sending. The roles we will use are "System" or "User". The "System" role is used when the user wants to provide the AI model with instructions to take into consideration when responding to queries. The "User" role is used when the user sends queries to the model.8. Querying chat models with Databricks
Here we use the `WorkspaceClient` from the Databricks SDK to query the serving endpoint of the meta llama model. Don't worry if you don't understand all of the details in this code block. In the next lesson, we will break down the different components that make up this request.9. Let's practice!
In this lesson, we've learned how Databricks exposes AI models to be queried and which library in the Databricks SDK to use to query these models. Let's put what we learned to the test.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.