1. Table storage
In this video, we'll explore how to create and manage databases and tables in Databricks. We'll focus on using the LOCATION keyword to customize where your data is physically stored. Let's walk through this step by step.
First, we'll create a database as a logical container to organize related tables and views. In the Databricks notebook, type the following command:
CREATE DATABASE my_custom_database.
Run the command by pressing shift and enter or clicking the run button. You'll see confirmation that the database my_custom_database has been created.
Next, we'll create a simple table within this database. By default, Databricks manages the storage location for this table. To do this, type and execute the following commands.
Once the command is ready, run it and observe the output. Databricks has automatically stored this table in its default location, simplifying data management.
Now, let's explore how to customize the storage location for a table. Custom storage locations can help with compliance, cost management, or performance optimization. To create a table with a specified storage location, type and execute the following.
Run the command and note the confirmation message. This command creates custom_table and stores its data in a custom location specified in the LOCATION clause.
Let's now query both tables to demonstrate their functionality, regardless of storage location. To query the default table, use the following command:
SELECT * FROM default_table.
Run the query and display the results. Next, query the custom table using:
SELECT * FROM custom_table.
Execute the query and show the results. Both tables work seamlessly, but custom_table stores data in the specified custom location.
As we've seen, the LOCATION keyword allows you to control where your data resides, allowing for more adaptable data management strategy. This feature enables compliance with regulations, improves performance, and enhances cost-efficiency, making your Databricks environment more scalable and flexible.
2. Let's practice!