Microsoft Fabric: The Ultimate Guide

Warehouses, Lakehouses, Data Pipelines, Semantic Models, Power BI, Real Time Intelligence and more! (DP-600)

What you'll learn

Data Warehousing: Understand the principles of data warehousing, master SQL for querying and managing data, and learn how to design and implement data warehouse

Data Engineering: Develop robust data engineering pipelines using Spark (PySpark), manage large-scale data transformations, and automate workflows efficiently.

Data Factory: Master the use of Microsoft’s Data Factory for orchestrating and automating data movement and transformation.

Real-Time Intelligence: Leverage real-time data processing capabilities to gain instant insights and act swiftly in dynamic business environments.

Power BI: Create powerful, interactive reports and dashboards.

A comprehensive end-to-end understanding of Microsoft Fabric

Requirements

Azure account and Microsoft Fabric Free Trial

Description

** This course can be used as supplementary preparation for the DP-600 Microsoft Fabric Analytics Engineer exam**Mastering Microsoft Fabric is your ticket to staying ahead in today’s data-driven world. With data skills in high demand, learning this all-in-one platform will set you apart in the job market. From automating data processes to creating real-time insights, Microsoft Fabric equips you with the cutting-edge tools needed by top employers. Whether you’re looking to boost your career or lead your companies data strategy, becoming proficient in Microsoft Fabric is a game-changer. Don't miss out on the opportunity to become an in-demand expert in the fast-growing field of data analytics and engineering.This course offers a balanced mix of theory and practical application. You'll dive deep into the concepts behind each component of Microsoft Fabric, followed by hands-on walkthroughs that allow you to apply what you’ve learned. The multi-experience course projects will challenge you to integrate your knowledge across different Fabric tools, preparing you to handle real-world scenarios with confidence.What You'll Learn:Data Warehousing: Understand the principles of data warehousing, master SQL for querying and managing data, and learn how to design and implement data warehouses within Microsoft Fabric.Power BI: Create powerful, interactive reports and dashboards.Data Engineering: Develop robust data engineering pipelines using Spark (PySpark), manage large-scale data transformations, and automate workflows efficiently.Data Factory: Master the use of Microsoft’s Data Factory for orchestrating and automating data movement and transformation.Real-Time Intelligence: Leverage real-time data processing capabilities to gain instant insights and act swiftly in dynamic business environments.Course Features:Theory Lectures: Grasp the foundational concepts and methodologies behind each component of Microsoft Fabric.Hands-On Walkthroughs: Apply your learning in real-time with detailed, step-by-step walkthroughs of key tools and features.Multi-Experience Projects: Engage in comprehensive projects that require you to integrate multiple Fabric experiences, simulating real-world business challenges.By the end of this course, you will be proficient in using Microsoft Fabric to design, build, and manage comprehensive data solutions. Whether you're a data engineer, analyst, or business intelligence professional, this course will equip you with the skills needed become an expert with Microsoft Fabric.

Overview

Section 1: Introduction

Lecture 1 Welcome to the Course

Lecture 2 What is Microsoft Fabric?

Lecture 3 What is Microsoft Azure?

Lecture 4 Azure Account Set Up

Lecture 5 Azure Portal Overview, Resource Hierarchy and Cost Management

Lecture 6 Starting Free Trial on Fabric Using Personal Domain

Lecture 7 Starting Free Trial on Fabric Using Business Domain

Lecture 8 Fabric UI Overview

Lecture 9 Creating a Resource Group and ADLS Gen 2 Storage Account

Lecture 10 Parquet and Delta File Formats

Section 2: Data Warehousing and Power BI

Lecture 11 Section Overview

Lecture 12 Overview of Data Warehouses in Fabric

Lecture 13 Creating a Fabric Workspace for this Section

Lecture 14 Creating a Data Warehouse

Lecture 15 Overview of SQL Queries

Lecture 16 Adding Comments in your SQL Queries

Lecture 17 Message for Students with SQL Experience

Lecture 18 SQL Code used in this Section

Lecture 19 Create and Drop Schemas

Lecture 20 Data Types

Lecture 21 Create and Drop Tables

Lecture 22 Inserting Records into a Table (and first look at the Select Statement)

Lecture 23 Loading the Retail Dataset

Lecture 24 The Select Statement

Lecture 25 Selecting Distinct Records

Lecture 26 Functions and Expressions

Lecture 27 Ordering and Limiting your Results

Lecture 28 Filtering Records

Lecture 29 Grouping and Aggregating

Lecture 30 Joining Tables

Lecture 31 SQL Execution Order

Lecture 32 Create Table As Select

Lecture 33 Updating and Deleting Records

Lecture 34 Subqueries

Lecture 35 Views

Lecture 36 Visual Query Editor

Lecture 37 Zero Copy Clones and Time Travel

Lecture 38 Cross Warehouse Querying

Lecture 39 Query Monitoring

Lecture 40 Stored Procedures

Lecture 41 (Optional) Mirroring in Fabric - Snowflake Demo

Lecture 42 Semantic Modelling and Power BI

Lecture 43 Establishing Relationships in our Data Model

Lecture 44 Measures

Lecture 45 Preparing the Presentation Layer and Semantic Model

Lecture 46 Creating the Power BI Report

Lecture 47 Dashboards

Lecture 48 Apps

Section 3: Data Factory

Lecture 49 Section Overview

Lecture 50 Creating the Workspace for this Section

Lecture 51 Creating the Lakehouse for this Section

Lecture 52 Pipelines Overview

Lecture 53 Adding the Citibike Data to our Lakehouse

Lecture 54 Copy Data - Lakehouse Files

Lecture 55 Copy Data - Copy Behaviour

Lecture 56 Copy Data - Tables

Lecture 57 Copy Data - External Storage (ADLS Gen2)

Lecture 58 Lookup Activity

Lecture 59 Get Metadata Activity

Lecture 60 Dynamic Content and Expressions

Lecture 61 Parameters

Lecture 62 Variables

Lecture 63 ForEach Activity

Lecture 64 Switch Activity

Lecture 65 Invoke Pipeline Activity

Lecture 66 Dataflows Gen2

Lecture 67 Script Activity

Lecture 68 Stored Procedure Activity

Lecture 69 Scheduling

Section 4: Multi Experience Data Analyst/BI Engineering Project

Lecture 70 NYC Taxi Data Project: Overview and Solution Architecture

Lecture 71 Overview of NYC Taxi Data

Lecture 72 Creating the Data Lakehouse for Initital Data Storage

Lecture 73 Data Pipeline for Ingestion to Staging

Lecture 74 Dataflow Gen2 for Processing to Presentation

Lecture 75 End to End Pipeline Orchestration

Lecture 76 Power BI Reporting

Lecture 77 Replacing Dataflow Gen2 with a Stored Procedure

Section 5: Data Engineering

Lecture 78 Section Overview

Lecture 79 What is Apache Spark?

Lecture 80 Spark SQL and DataFrame API

Lecture 81 Creating the Workspace and Overview of the Spark Starter Pool

Lecture 82 Creating the Lakehouse for this Section

Lecture 83 Uploading the Retail Analytics Data Files

Lecture 84 Overview of Notebooks in Fabric

Lecture 85 Section Notebooks (Download and Import to Fabric)

Lecture 86 Adding Comments to Code Cells

Lecture 87 Built-In Functions and Modules

Lecture 88 Reading Parquet Files

Lecture 89 Reading CSV and JSON Files

Lecture 90 Specifying Schema using StructType and StructField

Lecture 91 Multi Line Strings

Lecture 92 Loading Files to DataFrames using the UI

Lecture 93 SQL Limitations in Spark

Lecture 94 Selecting Columns from DataFrames

Lecture 95 Selecting Columns from DataFrames (Supplementary)

Lecture 96 Writing DataFrames to Files

Lecture 97 Writing to and Reading DataFrames from Tables

Lecture 98 Adding, Updating and Removing DataFrame Columns

Lecture 99 Changing Data Types

Lecture 100 Renaming DataFrame Columns

Lecture 101 Filering Rows

Lecture 102 Sorting and Limiting

Lecture 103 Grouping and Aggregating

Lecture 104 Joining Data

Lecture 105 Union and Drop Duplicates

Lecture 106 SQL

Lecture 107 Temporary Views

Lecture 108 Shortcuts

Lecture 109 Creating Spark DataFrames from Python Data Structures

Lecture 110 Notebook Orchestration and Parameters

Lecture 111 Overview of Medallion Architecture

Lecture 112 Medallion Architecture Implementation

Lecture 113 Custom Spark Pools

Lecture 114 Environments in Microsoft Fabric

Section 6: Real Time Intelligence

Lecture 115 Section Overview

Lecture 116 Overview of Real Time Intelligence

Lecture 117 Creating the Workspace and Lakehouse for this Section

Lecture 118 Eventstream to Lakehouse

Lecture 119 Eventhouses, KQL Databases and KQL Querysets

Lecture 120 Eventstream to KQL Database

Lecture 121 Real Time Analysis and Dashboards

Lecture 122 Reflexes (Data Activator)

Lecture 123 Reflexes for Power BI Alerts

Lecture 124 Deleting the Workspace

Section 7: Multi Experience Data Engineering Project

Lecture 125 Project Overview and Solution Architecture

Lecture 126 Project Workspace and Lakehouse Creation

Lecture 127 Creating the ADLS Gen2 Container and Lakehouse Shortcut

Lecture 128 Assigning Storage Blob Data Contributor Access

Lecture 129 TLC Trip Record Data

Lecture 130 Creating the Medallion Schemas and Silver Lookup Table

Lecture 131 Overview of Data Processing Notebooks

Lecture 132 Creating the Orchestration Pipeline

Lecture 133 End to End Pipeline Run

Lecture 134 Automated Reflex File Arrival Trigger (Data Activator)

Lecture 135 Further Analysis

Section 8: Congratulations

Lecture 136 Bonus Lecture

Anyone interested in learning Microsoft Fabric,Data Analysts,Data Engineers,Business Intelligence Professionals