How Fabric ingests data
1. Data ingestion and semantic models with Microsoft Fabric
Welcome to this course on building analytics in Microsoft Fabric! We'll cover two key stages: ingesting data into Fabric and turning it into semantic models that power dependable Power BI reports.2. Meet the instructor
The instructor for this course is Fiodar Sazanavets, a former senior software engineer at Microsoft, and a Microsoft MVP in development technologies.3. Prerequisites
This course builds on DataCamp's Introduction to Microsoft Fabric, so you should know the Fabric interface and items such as lakehouses and warehouses.4. Course goals
By the end of this course, you will be able to choose an appropriate Fabric ingestion tool and transform data with Dataflow Gen2. You will also turn raw data into analytical semantic models, then secure, refresh, and optimize those models.5. How Fabric ingests data
Let's start with the first step: getting data into Fabric. Fabric offers several routes into OneLake, each suited to a different job. You can copy data with a Copy job, orchestrate multi-step loads with a pipeline, transform data as it arrives with Dataflow Gen2, reference data without copying it by using a shortcut, replicate supported sources with mirroring, or stream continuous events with Eventstream.6. Creating your first Fabric items
Your assigned workspace is where your Fabric items live. In each hands-on exercise, you'll create the items you need from scratch. To create a Copy job, open the workspace and select the plus New item button. Search for Copy job, then select it from the results.7. Practicing with sample data
In the Copy job dialog, a sample data option appears in the top menu; on smaller screens, it may be hidden under the ellipsis. Sample data lets you practice the full Copy job workflow before you connect to an external source.8. Copying data with a Copy job
A Copy job is a simple way to ingest data. Choose a source and a Fabric destination, such as a new lakehouse you create from the wizard. In the Settings step, choose a read method. Full copy copies all the data on every run. Incremental copy copies all data on the first run, then only the changes. In the Map to destination step, map the source to a schema and table. The part before the dot is the schema and the part after it is the table, so dbo dot Diabetes is the Diabetes table in the dbo schema. You can run a Copy job on demand or on a schedule. Each run copies data in batches, rather than as a continuous stream like Eventstream.9. Lakehouse as copy destination
A lakehouse is a typical copy destination because it can retain raw files for later processing and can also store data in tables. The Explorer panel separates managed tables from stored files and folders. Opening the files shows how their structures differ, before later transformations use them.10. Lakehouse file formats
Inside a lakehouse, you'll typically see these file formats. CSV stores delimited text with no column types. Parquet stores compressed, column-oriented data and carries its own typed schema. JSON stores semi-structured records that do not always fit neatly into tables. Tables, in turn, are stored in Delta format: Parquet files plus a transaction log.11. Lakehouse sample data
The easiest way to get familiar with file formats in a lakehouse is to use the Start with sample data option in an empty lakehouse. It will automatically create files and tables for you.12. Let's practice!
Now let's practice creating a Copy job and reading files in lakehouse storage.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.