Databricks Certified Data Engineer Associate Complete Guide

Databricks Certified Data Engineer Associate Full Exam Prep 2023 Version 3 | Databricks Exam Prep | Databricks Certified

What you'll learn

Pass the Databricks Data Engineer Associate exam

Understand and apply the concepts and topics covered in the Databricks Certified Data Engineer Associate certification.

Gain a comprehensive understanding of Databricks as a Lakehouse platform and the benefits and advantage it provides.

Discover the significance of Delta Lake in data engineering and its role in managing data.

Develop your skills in Databricks data engineering tasks such as data ingestion, transformation and integration.

Navigate and use the Databricks UI effectively, enabling efficient workflow and task execution.

Explore how to effectively use the Unity Catalog to manage and organize data in Databricks.

Create and configure a Databricks account, providing access to the platform's powerful data engineering capabilities.

Learn how to setup and manage Databricks clusters to optimize data processing and utilize Databricks notebooks for data analysis and exploration.

Understand key Delta Lake concepts such as schemas, tables, optimization and efficiently loading data.

Gain data engineering techniques such as querying, data extraction and cleaning, and complex transformations within the Databricks environment.

Explore Databricks advanced features such as structured streaming, Delta Live Tables, and SQL warehousing to enhance data processing and analysis capabilities.

Requirements

Basic understanding of cloud computing concepts is recommended as this will assist students to understand the integration and deployments aspects of Databricks within a cloud environment, but not necessary

Knowledge of ETL and ELT processes as Databricks is recommend but not required

Basic Python and SQL knowledge

Description

Are you ready to take Data Engineering to the next level?Are you looking for a course to help you master data engineering with Databricks, or perhaps a guide to help you pass your upcoming Databricks Certified Data Engineer Associate exam? Look no further!Join the Databricks revolution, where the world of data analytics and warehousing is being reshaped like never before.Our curriculum is meticulously designed to cover all aspects of the Databricks Certified Data Engineer Associate exam, including data ingestion, data transformation, ETL processes, data modeling, and more. Discover how to optimize data pipelines and extract valuable insights from massive amounts of data.Learn from over 8 hours of instructional video and practical samples created by a seasoned data engineering professional with real-world experience and a the Databricks Engineer Associate certification. Take advantage of their practical insights, best practices, and invaluable guidance as they guide you through the course.What is this course all about?Our primary goal is to equip you with the knowledge, skills, and strategies needed to ace the Databricks Certified Data Engineer Associate exam. We cover each topic mentioned in the guidelines in great detail, ensuring that you are well-prepared for every aspect of the exam. Our number 1 priority is to help you pass the exam.What is Databricks?Databricks is a cloud-based data and AI platform that provides a unified set of tools for developing, deploying, and maintaining large-scale data solutions. It works with cloud storage and security to manage and deploy cloud infrastructure on behalf of users. Databricks provides an integrated environment for data engineering, data science, and machine learning tasks. It was founded by the Apache Spark creators.What is Databricks Certified Data Engineer Associate Certification?The Databricks Certified Data Engineer Associate certification offered by Databricks Academy is intended to validate your skills in using the Databricks Lakehouse Platform for data engineering tasks. By obtaining this certification, you demonstrate your thorough understanding of the platform, its tools, and benefits. What does the Databricks Certified Data Engineer Associate Certification cover?Understand how to use and the benefits of using the Databricks Lakehouse Platform and its tools, including Data Lakehouses, Data Science and Engineering workspaces, Delta LakeBuild ETL pipelines using Apache Spark SQL and Python, including Relational Entities, ELT, Spark SQL, and PythonIncrementally process data, including Structured Streaming, Auto Loader, Multi-hop Architecture, Delta Live TablesBuild production pipelines for Data Engineering Applications and Databricks SQL queries and Dashboards, including Jobs and DashboardsUnderstand and follow best security practices, including Unity Catalog and Entity PermissionsWhy should you get the certification?Get recognized - the Databricks Certified Data Engineer Associate Certification provides you with a valuable credential that will set you apart in the competitive field of data engineering.Improve your data engineering skills - Databricks is the most widely used tool for unified data processing, analytics, and machine learning, with amazing performance and scalabilityIncrease your employability - Databricks has grown in popularity and widespread adoption across a wide range of industries; being proficient in Databricks places you at the forefront of industry demand and increases your marketability as a data professional.Who are the instructor for this course? This course will be taught by Henry Habib Learning:Henry Habib - data consultant and online instructor for 100K+ students, focused on Azure and Databricks Megan Ong - an Australian-based, data engineer and AI consultant who has a passion for teaching and helping people reach their goalsWhy choose this course?Comprehensive Guide:  This course provides comprehensive coverage of all the topics you need to know for the Databricks Certified Data Engineer Associate certification exam.  With over 8 hours of instructional content, it offers in-depth preparation for the exam.Tailored for Success:  The course is specifically designed to address the key details of the examination, ensuring that we cover all the topics, skills, and knowledge required for the Databricks Data Engineer Certification exam.Hands-on approach: In addition to covering key topics, our course takes a fully instructional approach. We don't just explain the theoretical aspects and features, we actively apply them by building applications together. Instructor assistance: If there's something you're struggling with, don’t worry! I'm here to provide support and guidance. Feel free to reach out to me through the contact options provided in the video, and I'll be more than happy to assist you.Resource Availability: Follow along with ease! We provide all the necessary reference materials to make your learning journey smoother.Practice exams: This course contains practice exams with questions that exactly mirror the types of questions found on the PL-900 exam. Use them to validate your knowledge and find weaker areas where you need to review.Community: When you enroll in this course, you join a Databricks community full of learners just like youCourse Overview:Introduction: Learn about Databricks, the certification, and how to succeedSection 1: Data Lakehouse and Delta Lake in DatabricksSection 2: Data Science and Engineering Workspaces in DatabricksSection 3: Relational entities (databases, tables, views) in DatabricksSection 4: ELT (Spark SQL and Python) in DatabricksSection 5: Structured Streaming and Delta Live Tables in DatabricksSection 6: Structured Streaming and Delta Live Tables in DatabricksSection 7: Navigating Workflows and Production Pipelines in DatabricksSection 8: Discover and manage data using Data Explorer in DatabricksSection 9: Navigating Databricks SQLPractice Exams: practice what you have learned to validate your knowledgeConclusion: take your exam, earn your certification, and next stepsTake the exam: time to roll your sleeves and take the real exam to earn your certification

Overview

Section 1: Introduction

Lecture 1 Introduction to this course

Lecture 2 Instructor overview

Lecture 3 Introduction to Databricks Data Engineer Associate

Lecture 4 Introduction to Databricks

Lecture 5 Course roadmap and curriculum

Lecture 6 Course and exam curriculum

Lecture 7 Ways to reach out

Lecture 8 Keys to success

Lecture 9 Leave a rating

Lecture 10 Watch in 1080p

Section 2: S1: Data Lakehouse and Delta Lake in Databricks

Lecture 11 Databricks the Lakehouse Platform overview

Lecture 12 Delta lake

Lecture 13 Introduction to Unity Catalog

Lecture 14 Databricks and cloud provider

Lecture 15 Data engineering in Databricks

Lecture 16 Slides

Section 3: S2. Data Science and Engineering workspaces in Databricks

Lecture 17 Create an account in Databricks

Lecture 18 Navigating the Databricks UI

Lecture 19 Databricks Cluster

Lecture 20 Create a Databricks Cluster

Lecture 21 Databricks Repos

Lecture 22 Databricks Repos UI

Lecture 23 Databricks Repos UI - PLS SEE THIS IF YOU CANNOT ACCESS THE REPO FROM THE LINK

Lecture 24 Databricks Notebooks

Lecture 25 Databricks Notebooks UI

Lecture 26 Databricks notebook tutorials introduction

Lecture 27 Databricks Notebooks DA tutorial

Lecture 28 DE 1.3: Getting started with Databricks lab

Lecture 29 Slides

Section 4: S3. Relational entities (databases, tables, views) in Databricks

Lecture 30 Databricks workflows

Lecture 31 DE 2.1: Querying files directly demo

Lecture 32 DE 2.2: Querying files directly demo

Lecture 33 Slides

Section 5: S4. ELT (Spark SQL and Python) in Databricks

Lecture 34 DE 2.3: Extract data

Lecture 35 DE 2.4: Cleaning data

Lecture 36 DE 2.5: Complex transformations

Lecture 37 DE 2.6: Reshape Data Lab

Lecture 38 DE 2.7A: SQL UDFs

Lecture 39 DE 2.99: Higher order functions

Lecture 40 DE 2.7B: Python UDFs

Lecture 41 Module 2.0 Summary

Lecture 42 DE 3.1: Schemas and Tables

Lecture 43 DE 3.2 - Version and Optimize Delta Tables

Lecture 44 DE 3.3L - Manipulate Delta Tables Lab

Lecture 45 DE 3.4 - Set Up Delta Tables

Lecture 46 DE 3.5 - Load Data into Delta Lake

Lecture 47 DE 3.6L - Load Data Lab

Lecture 48 Module 3.0 Summary

Lecture 49 Slides

Section 6: S5. Structured Streaming and Delta Live Tables in Databricks

Lecture 50 Structured Streaming

Lecture 51 Delta Live Tables Part 1

Lecture 52 Delta Live Tables Part 2

Lecture 53 DE 4.1 - DLT UI Walkthrough

Lecture 54 DE 4.1.1 - Orders Pipeline

Lecture 55 DE 4.1.2 - Customers Pipeline

Lecture 56 DE 4.1.3 - Status Pipeline

Lecture 57 DE 4.2 - Python vs SQL

Lecture 58 DE 4.3 - Pipeline Results

Lecture 59 DE 4.4 - Pipeline Event Logs

Lecture 60 DE 4.99 - Land New Data

Lecture 61 Module 4.0 Summary

Lecture 62 Slides

Section 7: S6. Navigating Workflows and Production Pipelines in Databricks

Lecture 63 DE 5.1.1 - Task Orchestration

Lecture 64 DE 5.2.1L - Lab Instructions Part 1

Lecture 65 DE 5.2.1L - Lab Instructions Part 2

Lecture 66 Module 5.0 Summary

Lecture 67 Slides

Section 8: S7. Unity Catalog & Entity Permissions in Databricks

Lecture 68 Understanding Unity Catalog

Lecture 69 External tables on Unity Catalog

Lecture 70 Slides

Section 9: S8. Discover and manage data using Data Explorer in Databricks

Lecture 71 Managing permissions with Data Explorer UI

Lecture 72 Slides

Section 10: S9. Navigating Databricks SQL

Lecture 73 How to create a SQL warehouse

Lecture 74 DE 7.1 - Navigating Databricks SQL and Attaching to Warehouses

Lecture 75 DE 7.2 - Last Mile ETL with DBSQL

Lecture 76 Module 7.0 Summary

Lecture 77 Slides

Section 11: Exam Preparation

Lecture 78 Official Databricks Practice Exam

Section 12: Conclusion

Lecture 79 Congratulations

Lecture 80 Next steps

Lecture 81 Ways to reach out and contact me

Section 13: Bonus

Lecture 82 Bonus

Data engineers and enthusiasts pursuing the Databricks Certified Data Engineer Associate certification as this course aligns with the certification requirements,Data engineers who want to enhance their knowledge and skills when working with Databricks for data engineering tasks.,Individuals who aspire to become data engineers and want to build a solid foundation in data engineering principles and techniques using Databricks,Individuals who want to get started on Databricks Lakehouse