Notebook Fundamentals

1. Notebook fundamentals

You need somewhere to write and run code. That's the Databricks notebook - an interactive environment supporting multiple languages.

2. What is a Databricks notebook?

A Databricks notebook looks and feels like a Jupyter notebook if you've used one before, but with a few important differences. Each notebook is attached to a cluster that executes your code. You can mix code cells, their output, and markdown documentation all in one file. And here's where it gets interesting - you're not locked into a single language. You can use Python, SQL, Scala, and R all within the same notebook.

3. Magic commands

Every notebook has a default language - say, Python. But you can switch languages in any cell using a magic command. Prefix a cell with percent-sql to write SQL, percent-md to write formatted documentation, or percent-sh to run shell commands. This is genuinely useful in practice. You might write your transformation logic in Python, run a quick validation query in SQL, and document what you found in Markdown - all without leaving the notebook.

4. Available magic commands

Here's the full list. The language magic commands - percent-python, percent-sql, percent-scala, percent-r - let you switch execution engines. Percent-md renders Markdown for documentation. And percent-sh drops you into a bash shell on the driver node, which is handy for checking installed packages or downloading files. One thing to note: all cells share the same cluster, but each language has its own execution context. A variable you define in Python isn't directly accessible from a SQL cell.

5. Running another notebook with %run

The percent-run command is one of the most practical features in notebooks. It executes another notebook and makes its functions, variables, and imports available in your current session. This is how teams build reusable utility libraries without formal packaging. You put your cleaning functions in a shared notebook, and every team member runs it at the top of their own notebook. It's simple and effective, though for larger projects you'll eventually want proper Python packages managed through Repos.

6. Interpreting results

When you run a cell, the output appears directly below it. SQL queries automatically render as interactive tables you can sort and filter. DataFrames get a display with built-in charting options - click the chart icon to switch between bar, line, and scatter plots without writing any visualization code. And when something goes wrong, you get a stack trace with line numbers pointing you to the problem. These aren't revolutionary features individually, but together they make notebooks a genuinely productive environment for data work.

7. Summary

To sum up: Databricks notebooks are your primary development environment. Magic commands let you switch between languages in any cell, which is particularly powerful for combining Python logic with SQL queries. Percent-run lets you load shared utilities from other notebooks. And the built-in result rendering saves you from writing boilerplate visualization code. Next, we'll look at how to share your notebooks and connect them to Git.

8. Let's practice!

Let's try it out.

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.