Learn how to set up the Databricks workspace client using Python and authenticate it to your Databricks Workspace. You will explore fundamental API operations, gaining familiarity with key functions for interacting with Databricks clusters and jobs, including creating, listing, and deleting resources.

Getting started with the Databricks SDK

Default authentication

Basic operations with Databricks Python SDK

Listing Databricks jobs

Listing Databricks clusters

Introduction to the Databricks Python SDK

Dive into querying AI models using the Databricks Python SDK. You will discover how to integrate external data sources like CSV files, query the Databricks AI model called DBRX LLM through the Foundation Models API, and understand the roles of different chat message types in AI models, including “System” and “User” roles. You will also learn how to use Databricks catalog data as a data source that that AI agents use to answer your queries.

Introduction to querying AI Models with Databricks

Chat Message Roles

Query a Foundation Model

Query chatbot using Databricks Servings Endpoint API

Query a Chat Completion Model

Text summarization

Proofreading text

Query Databricks Catalog with Databricks SQL Agent

Create LangChain SQL Agent Connected to Databricks

Query NYC Taxi Dataset using SQL Agent

Query bakery sales data using SQL agent

 Querying LLM Models with the Databricks SDK

Discover how to manage Databricks clusters and jobs programmatically. You will learn how to create, delete, start, and validate the status of Databricks clusters and jobs using the Python SDK.

Cluster creation and management

Start a cluster

Check that a cluster is running

Job creation and management

Create and run a Databricks job

Schedule a Databricks job

Delete a Databricks job

Wrap Up

 Using the Databricks SDK to Manage Clusters and Jobs

Take control of your Databricks environment with Python. This course shows you how to use the Databricks SDK to authenticate your workspace, manage clusters and jobs, and interact with platform resources programmatically. You’ll also learn to query AI models, create intelligent agents that analyze data from your Databricks catalog, and automate workflows, building the skills to efficiently manage and scale Databricks with Python.

<h2>Getting Started with the Databricks Python SDK</h2>
This course introduces the Databricks platform through the Python SDK, focusing on practical, developer-friendly workflows. You’ll learn the essential tools and techniques to interact with Databricks resources and AI models, while managing clusters and jobs programmatically. The emphasis is on hands-on, reproducible steps you can reuse across projects.  
<br><br>

<h2>Authenticating and Exploring the Workspace Client</h2>
Begin by setting up and authenticating a Workspace Client in Python. You’ll practice core API operations to create, list, and delete resources, building confidence with the request–response patterns used throughout the SDK. Along the way, you’ll understand how authentication and environment variables enable secure, scriptable access to your Databricks Workspace.  
<br><br>

<h2>Querying AI Models with the Databricks SDK</h2>
Dive into AI by querying foundation models with the SDK. You’ll learn how chat completion models work, how to pass “System” and “User” messages, and how to integrate external data into your workflows. You’ll also explore the Foundation Models API and practice generating and summarizing text programmatically.  
<br><br>

<h2>Managing Clusters and Jobs Programmatically</h2>
Conclude by orchestrating compute and automation. You’ll create, start, validate, and delete clusters; then create, run, schedule, list, and remove jobs—all from Python. By the end, you’ll have practical skills to manage resources, query AI models, and automate repeatable workflows in Databricks with confidence.

Intermediate Python for Developers

Master Databricks with Python: learn to authenticate, manage clusters, automate jobs, and query AI models programmatically.

Databricks with the Python SDK

Software/Data Engineer

Why learn Databricks?

What is a Databricks Workspace?

Install Databricks SDK

Authentication environment variables

Authenticate using a Service Principal

Interacting with our Databricks workspace

Let's practice!

Introduction to the Databricks Python SDK - Getting started with the Databricks SDK

Getting started with the Databricks SDK

Create Your Free Account