Databricks Certified Data Engineer Associate - Bootcamp

Databricks Data Engineer [2025] | Strong Hands-on for Databricks Certified Data Engineer Associate Certification Exam

What you'll learn

All the important topics you need to PASS the Certification - with deep hands-on practice

Understand key concepts like Lakehouse Federation, Lakeflow Connect, and the Medallion Architecture

Strong hands-on with Unity Catalog, Volumes, Metastore, Catalog UDFs, and utils

PySpark Big Data Crash Course - from basics to real-world use cases

Master Spark Structured Streaming using Auto Loader for INCREMENTAL real-time data ingestion

Learn the complete Delta Lake Architecture, its benefits, and how to implement & tune it for performance

Deploy and manage Databricks SQL Warehouses with parameterized queries, alerts, and query caching

Build streaming pipelines using Streaming Tables, Materialized Views, and Lakeflow Declarative Pipelines

Implement Slowly Changing Dimensions (SCDs) and add Data Quality checks using Delta Live Tables

Master Lakeflow Jobs to orchestrate your ETL pipelines like a pro

Understand and apply Row-Level Security, Data Masking, and Delta Sharing for secure data access

Learn Data Versioning, Time Travel, ZORDERING, Cloning, and Liquid Clustering

Requirements

Basic SQL knowledge will be required

Basic Python programming knowledge will be required

No DATABRICKS knowledge is required - Everything is covered from SCRATCH

Description

Are you ready to become a DATABRICKS DATA ENGINEER?Whether you're a beginner or a working professional who wants to level up, this course will guide you step by step with a hands-on, practical, and engaging approach.GAIN STRONG HANDS-ON WITH:Lakehouse Architecture, Lakehouse Federation, and Lakeflow Connect – Understand how Databricks handles structured and unstructured data, and how Lakehouse Federation lets you query external sources seamlessly.Unity Catalog, Metastore, Volumes, and UDFs – Learn how to manage data, permissions, and catalogs efficiently using Databricks’ built-in governance features.PySpark for Big Data – Master PySpark with real use cases, transformations, actions, joins, and more — all from a Data Engineer’s point of view.Structured Streaming + Autoloader – Build real-time pipelines using Spark Streaming and learn how Autoloader handles files in cloud storage.Delta Lake Architecture – Dive deep into Delta’s features like ACID transactions, time travel, schema evolution, and performance tuning.Databricks SQL Warehouses – Learn how to write parameterized queries, schedule dashboards, and set alerts using SQL Warehousing.LakeFlow Declarative Pipelines – Work with Streaming Tables, Materialized Views, and build low-code data pipelines.Delta Live Tables (DLT) – Build robust pipelines with SCD implementation, data quality checks, expectations, and monitoring.Orchestrate ETL with LakeFlow Jobs – Schedule, monitor, and manage your pipelines using LakeFlow Jobs end-to-end.Security and Sharing – Apply row-level security, data masking, and explore Delta Sharing for secure and scalable collaboration.What Makes This Course Different?Super Engaging Lectures – No boring theory here! I explain every concept in a clear and beginner-friendly way using real-life examples and visuals.Deep Dive into Every Topic – I don’t just scratch the surface. You'll understand the “why” and “how” behind every feature .Strong Hands-On Focus – Learn by doing! From pipelines to notebooks to warehouse, you’ll build real solutions step-by-step, just like a Databricks Data Engineer does.DISCLAIMER : This course is independently created and not affiliated with or endorsed by Databricks Inc. All content, including explanations and practice materials, is original and intended solely for educational purposes. It does not include any actual certification exam questions and is based on publicly available documentation, real-world scenarios, and personal experience. Product names, logos, and trademarks used are the property of their respective owners and are included only for identification and learning. Always refer to official Databricks documentation for the latest and most accurate information.

Overview

Section 1: Course Introduction & Resources

Lecture 1 Introduction

Lecture 2 Disclaimer

Lecture 3 Resources - Zipped_Folder (Main)

Lecture 4 Resources - Raw Data

Lecture 5 Resources - Course Slides

Lecture 6 Resources - Scripts, Notebook, & Files

Lecture 7 Resources - Databricks Source Code

Section 2: Lakehouse Core Fundamentals

Lecture 8 What is Lakehouse?

Lecture 9 Delta Lake - Backbone of Lakehouse

Lecture 10 What is Databricks? & Why Databricks?

Lecture 11 Databricks Architecture

Lecture 12 What is Medallion Architecture?

Section 3: Get Started With Databricks

Lecture 13 Databricks Free Edition - Newly Announced

Lecture 14 Databricks Overview

Lecture 15 Refer Resources Within Databricks

Lecture 16 Databricks Compute - All The Types

Section 4: Unity Catalog - The Complete Guide

Lecture 17 What is Unity Catalog?

Lecture 18 Evolution With Unity Catalog

Lecture 19 Managed VS External Tables

Lecture 20 Unity Catalog Volumes

Lecture 21 Managed and External Volumes

Lecture 22 DBUTILS - Databricks OS Module

Section 5: Big Data With Apache Spark

Lecture 23 PySpark Transformations - Silver Layer

Lecture 24 Structured Queries With SparkSQL

Lecture 25 PySpark UDF For Custom Functions

Section 6: Databricks Lakehouse Foreign Objects

Lecture 26 Work with Databricks Files

Lecture 27 Databricks Lakehouse Federation

Lecture 28 Foreign Catalogs and Foreign Tables

Section 7: AUTOLOADER - Spark Structured Streaming

Lecture 29 What is Spark Streaming?

Lecture 30 Autoloader - what's that?

Lecture 31 Autoloader Complete Architecture

Lecture 32 The power of Autoloader

Lecture 33 Idempotency with Autoloader

Lecture 34 Schema Evolution with Autoloader - Rescued Data

Lecture 35 Schema Evolution with Autoloader - AddNewColumns

Lecture 36 Incremental Load with COPY INTO

Lecture 37 The power of COPY INTO

Section 8: Databricks SQL Warehouse

Lecture 38 Databricks SQL : a sneak-peek

Lecture 39 What is Databricks SQL Warehouse?

Lecture 40 Databricks SQL Editor

Lecture 41 Parametrized Queries in Databricks SQL

Lecture 42 SQL Query Snippets - Reuse your code

Lecture 43 Common Table Expressions - CTEs

Lecture 44 Query Scheduling

Lecture 45 Query Profile for Monitoring

Lecture 46 Query Cache - Boost Query Performance

Lecture 47 SQL Alerts - Triggers

Lecture 48 Databricks Genie

Lecture 49 Databricks SQL Dashboards

Section 9: Orchestration With Lakeflow Jobs

Lecture 50 What are Lakeflow Jobs?

Lecture 51 Build your first Lakeflow Job

Lecture 52 Conditionals in Jobs

Lecture 53 Control Flow using ForEach Task

Lecture 54 Dynamic Value Reference and Tasks Values

Lecture 55 Quota Limit Exceed Error

Lecture 56 Set and Get Values in Jobs

Lecture 57 SQL rows as output

Lecture 58 SQL First Row as output

Lecture 59 ForEach with SQL Rows

Lecture 60 Passing Large Array With Notebook

Lecture 61 Large Array With SQL Table

Lecture 62 Schedules and Triggers

Lecture 63 Computes For Jobs

Lecture 64 Jobs Monitoring

Lecture 65 Jobs Notification

Section 10: DELTA LIVE TABLES - Lakeflow Declarative Pipelines

Lecture 66 What is DLT - Delta Live Tables?

Lecture 67 DLT Code Editor - New Environment

Lecture 68 Create Streaming Table

Lecture 69 Create Materialized View

Lecture 70 Create Streaming Views

Lecture 71 Build your first DLT pipeline

Lecture 72 Delta Live Table with Autoloader

Lecture 73 DLT Append Flow API

Lecture 74 DLT Auto CDC Flow API

Lecture 75 Slowly Changing Dimension Type 1

Lecture 76 Slowly Changing Dimension Type 2

Lecture 77 Data Quality with DLT Expectations

Lecture 78 Filter out Corrupted Data

Lecture 79 Parametrize your DLT Pipelines

Lecture 80 DLT End-To-End Project

Lecture 81 DLT Pipeline Modes

Lecture 82 Orchestrate your DLT Pipelines

Lecture 83 Monitor your DLT pipelines

Section 11: Unity Catalog User Defined Functions

Lecture 84 What are Unity Catalog Functions?

Lecture 85 What are Scalar Functions?

Lecture 86 Scalar Functions with SQL

Lecture 87 Scalar Functions with Python

Lecture 88 User Defined Table Functions - UDTFs

Lecture 89 Create custom UDTFs

Section 12: Data Access Control and Governance

Lecture 90 Why do we need Data Governance?

Lecture 91 Data Discoverability

Lecture 92 Track Data Quality

Lecture 93 Share your Data with Delta Sharing

Lecture 94 Data Access Control - Non negotiable

Lecture 95 Dynamic Data Masking

Lecture 96 Row Level Security - RLS

Section 13: Build Databricks Applications

Lecture 97 What is Databricks Apps?

Lecture 98 Get started with your first Databricks App

Section 14: Delta Lake Optimization

Lecture 99 Let's talk about this

Lecture 100 OPTIMIZE Command - Coalesce partitions

Lecture 101 ZORDERING - Collocate your data

Lecture 102 Liquid Clustering - Say Good Bye To Partitions

Lecture 103 Data Versioning with Delta Lake

Lecture 104 Time Traveling with Delta Lake - UNDO

Lecture 105 VACUUM Command - Clear the MESS

Lecture 106 CTAS - Copy Table As Select

Lecture 107 Deep Clone Your Data

Lecture 108 Shallow Clone is also GOOD

Section 15: YOUR NEXT STEPS

Lecture 109 All The Best

Anyone who WANTS to become a DATABRICKS DATA ENGINEER