BigQuery data organization

1. BigQuery data organization

In our final video in this chapter, we will learn how data is organized in BigQuery.

2. BigQuery table names

Here is a sample query from BigQuery. As we can see, the table name looks different from other SQL databases, so let's explore why.

3. Projects

Google Cloud uses projects as a top-level structure for every tool in Google Cloud which manage user and permission controls. A user can be assigned to multiple projects, and the project is the top-level structure within BigQuery. This is the first element within a BigQuery table name.

4. Datasets

The second level of organization within BigQuery is the dataset. We can think of this like a schema within other SQL databases, as it is a structure that contains tables underneath it. Datasets also have their own permissions, meaning a user with permissions to two datasets can query and join tables from each dataset. This is the second element in the full table name within BigQuery.

5. Tables

Finally, there are tables, which are where your data resides. This is the third and final element in the full table name in BigQuery.

6. Regions

Google Cloud regions are effectively data centers, some of which have several zones within them. You can see all the regions represented on this map. For example, the Iowa data center has four zones within it. In the case of BigQuery, there are two multi-regions, one for all data centers in the United States and one for all data centers in European Union member countries.

7. Working with data between regions

It is important to remember that once we create a dataset, its region is set permanently and cannot be changed later. Some methods exist to move data between regions and create data replicas in other regions. However, there are no methods to query data between two different regions. This has to do with how resources are allocated physically in the data centers.

8. Let's practice!

Now that we understand how data is structured and stored within BigQuery, let's practice some of the new concepts.

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.