1. Setting up a Databricks workspace example
Hello there! In this video, I will be showing you an example of how to get started with your Databricks environment, as well as providing an overview of the platform. In this scenario, I will be acting the part of the lead cloud data architect at Sierra Publishing. I have already setup our Databricks workspace so that our teams can start to experiment with the platform and use it for their analytical workloads.
Now that I am in the Databricks UI, let us take a moment to explore what is available to us. Looking at the left-hand side of the screen, I can see all of the different components of the platform. On the top we have all of the general capabilities like our compute clusters or data access. Below that, I can see there are a variety of capabilities designed for different data workloads, such as Delta Live Tables or the Model Registry. In later videos and courses, we will be diving deeper into these other capabilities, so I will skip them for now.
First I will click on the Catalog button towards the top. This takes us to the Catalog Explorer, which is your main hub for looking at all of the data assets available to you in the Databricks environment. Here you will see different catalogs, schemas, tables, and other data artefacts that are all governed by Unity Catalog, which will be covered in a separate video.
Next, I will click on the Compute button, which allows me to see the different kinds of processing power I have available to me. Databricks creates these clusters and warehouses of compute nodes for me, and all I have to do is run my processes against them.
Now I will click on the Workspace button to see all of the code that I have been working on. I will select a notebook, and will go ahead and run it against my cluster. Databricks will take this code and efficiently process it across in the cluster, and will return the result. Note that I can write code in different languages within the same notebook, and I would also be able to collaborate directly with my teammates on this code.
Now that I am familiar with my overall Databricks environment, let's jump back into the course and see how we can get this setup in your own environment.
2. Let's practice!