Inside an Asset Bundle

1. Inside an asset bundle

You know why Asset Bundles exist. Now let's open one up and walk through its structure.

2. Bundle project structure

A bundle is just a folder with a specific structure. At the root is databricks-dot-yml - the central configuration file that ties everything together. The src folder holds your notebooks and Python scripts. The resources folder contains YAML definitions for jobs, pipelines, and dashboards. And optionally, a tests folder for your unit tests. Nothing here is Databricks-proprietary - it's standard files in a standard folder structure, all version-controlled in Git.

3. The databricks.yml file

The databricks-dot-yml file is where you define the big picture. The bundle section names your project. The workspace section points to your Databricks instance. And the targets section is where it gets powerful - you define separate deployment targets for dev, staging, and production. Each target can have its own workspace path, permissions, and variable overrides. When you deploy, you specify which target to deploy to. Same code, different environments, zero manual reconfiguration.

4. Defining resources

Resources are the things your bundle deploys - jobs, pipelines, and dashboards. A job resource defines everything Databricks needs to run it: the name, schedule, tasks, cluster configuration, and notification settings. In this example, we define a nightly ETL job that runs at 2 AM, executing a notebook from the src folder. The job definition is pure YAML - readable, diffable, and reviewable in a pull request. When you deploy the bundle, Databricks creates or updates this job automatically.

5. Bundle CLI commands

The bundle CLI gives you four essential commands. Validate checks your YAML for errors before you deploy - catch typos and misconfigurations early. Deploy pushes your code and resource definitions to the specified target. Run triggers a deployed job manually, which is useful for testing. And destroy tears down everything the bundle created, which is handy for cleaning up dev environments. In a CI/CD pipeline, you'd typically run validate, then deploy, and optionally trigger a test run.

6. Deployment workflow

Here's the end-to-end workflow. A developer edits code and resource definitions locally, runs bundle validate to check for errors, and commits to Git. A CI/CD pipeline picks up the change, deploys to staging with bundle deploy, and runs automated tests. If everything passes, the change promotes to production. Every step is automated and auditable. No one clicks through a UI to deploy. No one copies settings from one environment to another. The bundle file is the single source of truth.

7. Summary

To recap: a Databricks Asset Bundle is a folder with a databricks-dot-yml configuration file, your source code, and resource definitions. Targets let you deploy the same project to different environments. And the CLI gives you validate, deploy, run, and destroy. Together, this is everything you need for repeatable, automated deployment. In the next lesson, we'll bring everything from the course together in a capstone scenario.

8. Let's practice!

Time to inspect a real bundle. You'll open a databricks-dot-yml file, explore its structure, and answer a question about where components belong.

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.