Aws Certified Data Engineer Associate 2023 - Hands On!

AWS DEA-C01 certification prep course with exercises. S3, Glue, Kinesis, Redshift, Lambda, Athena, EventBridge, and more

What you'll learn

Maximize your odds of passing the AWS Certified Data Engineer - Associate (DEA-C01) exam

Design and implement data pipelines with AWS to ingest, store, and transform data

Choose and design data stores, data models, data schemas, and data lifecycles.

Maintain, operationalize, and orchestrate data pipelines with EventBridge, Airflow, AWS Step Functions,, and more

Apply security, governance, and privacy best practices to your AWS data pipelines.

Create data lakes with S3, Glue, Redshift and more

Process batch and streaming data with Kinesis, EMR, containers, Lambda, and more

Requirements

You must have an AWS account to follow along with the hands-on activities. The services used will cost a few dollars in AWS fees.

AWS recommends 2-3 years of experience in data engineering, and 1-2 years of hands-on experience with AWS services.

Description

[v2023: The course is based on the currently available exam guide and practice exam from AWS. This course will be kept updated as new beta and final exams are released by Amazon. Happy learning! ]The AWS Certified Data Engineer Associate Exam is one of the most challenging associate-level certification exams you can take from Amazon Web Services, and even among the most challenging overall. Passing it tells employers in no uncertain terms that your knowledge of data pipelines is wide and deep. But, even experienced technologists need to prepare heavily for this exam. This course sets you up for success, by covering all of the data ingestion, transformation, and orchestration technologies on the exam and how they fit together.Best-selling Udemy instructors Frank Kane and Stéphane Maarek have teamed up to deliver the most comprehensive and hands-on prep course we've seen. Together, they've taught over 2 million people around the world. This course combines Stéphane's depth on AWS with Frank's experience in wrangling massive data sets, gleaned during his 9-year career at Amazon itself. The world of data engineering on AWS includes a dizzying array of technologies and services. Just a sampling of the topics we cover in-depth are:Streaming and transforming data with Amazon Kinesis and Amazon Managed Streaming for Apache Kafka (MSK)Queuing messages with Simple Queue Service (SQS)Orchestrating data pipelines with Amazon AppFlow, Amazon EventBridge, AWS Step Functions, and Amazon Managed Workflows for Apache Airflow (MWAA)Transitioning from small to big data with the AWS Database Migration Service (DMS), AWS DataSync, Snow Family, Transfer Family, and moreStoring massive data lakes with the Simple Storage Service (S3) and managing data lifecyclesOptimizing transactional queries with DynamoDB, DocumentDB, Keyspaces, and MemoryDBTying your big data systems together with AWS LambdaMaking unstructured data query-able with AWS Glue, Glue DataBrew, and Lake FormationProcessing data at unlimited scale with Elastic MapReduce, including Apache SparkApplying advanced machine learning algorithms at scale with Amazon SageMakerSearching and analyzing petabyte-scale data with Amazon Opensearch (formerly Elasticsearch) ServiceQuerying S3 data lakes with Amazon AthenaHosting massive-scale data warehouses with Redshift and Redshift SpectrumIntegrating smaller data with your big data, using the Relational Database Service (RDS) Visualizing your data interactively with QuicksightKeeping your data secure with encryption, KMS, Macie, Secrets Manager, IAM, and moreManaging and governing your systems with CloudFormation, CloudTrail, CloudWatch, AWS Config, and moreThroughout the course, you'll have lots of opportunities to reinforce your learning with hands-on demos and practice questions.  We'll also arm you with some valuable test-taking tips and strategies along the way.Although this is an associate-level exam, it is one of the more challenging ones. AWS recommends having a few years of both data engineering experience and AWS experience before tackling it. This exam is not intended for AWS beginners.You want to go into the AWS Certified Data Engineer Associate Exam with confidence, and that's what this course delivers. Hit the enroll button, and we're excited to see you in the course… and ultimately to see you get your certification!- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -InstructorMy name is Stéphane Maarek, I am passionate about Cloud Computing, and I will be your instructor in this course. I teach about AWS certifications, focusing on helping my students improve their professional proficiencies in AWS.I have already taught 1,500,000+ students and gotten 500,000+ reviews throughout my career in designing and delivering these certifications and courses!With AWS becoming the centerpiece of today's modern IT architectures, I have decided it is time for students to learn how to be an AWS Data Analytics Professional. So, let’s kick start the course! You are in good hands!- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -InstructorHey, I'm Frank Kane, and I'm also co-instructing this course. I spent nine years working for Amazon from the inside as a senior engineer and senior manager, and I'm best known for my top-selling courses in "big data", data analytics, machine learning, AI, Apache Spark, system design, and Elasticsearch.I've been teaching on Udemy since 2015, where I've reached over 750,000 students all around the world!I've worked hard to keep this course up to date with the latest developments in AWS data analytics, and to make sure you're prepared for the latest version of this exam. Let's dive in and get you ready!- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -This course also comes with:Lifetime access to all future updatesA responsive instructor in the Q&A SectionUdemy Certificate of Completion Ready for DownloadA 30 Day "No Questions Asked" Money Back Guarantee!Join us in this course if you want to pass the AWS Certified Data Engineer - Associate Exam DEA-C01 and master the AWS platform!

Overview

Section 1: Introduction

Lecture 1 Course Overview - Services we will Cover

Lecture 2 Get the Course Materials

Lecture 3 Udemy 101

Section 2: Data Engineering Fundamentals

Lecture 4 Intro: Data Engineering Fundamentals

Lecture 5 Types of Data (Structured, Unstructured, Semi-Structured)

Lecture 6 Properties of Data (Volume / Velocity / Variety)

Lecture 7 Data Warehouses vs. Data Lakes (and Lakehouses)

Lecture 8 Managing and Orchestrating ETL Pipelines

Lecture 9 Common Data Sources and Data Formats

Lecture 10 Quick Review of Data Modeling, Data Lineage, and Schema Evolution

Lecture 11 Database Performance Optimization

Lecture 12 Data Sampling Techniques

Lecture 13 Data Skew Mechanisms

Lecture 14 Data Validation and Profiling

Lecture 15 SQL Review: Aggregations, Grouping, Sorting, Pivoting

Lecture 16 SQL JOIN types

Lecture 17 Git review: architecture and commands

Section 3: Storage

Lecture 18 Intro: Storage

Lecture 19 Amazon S3

Lecture 20 Amazon S3 - Hands On

Lecture 21 Amazon S3 Security - Bucket Policy

Lecture 22 Amazon S3 Security - Bucket Policy - Hands On

Lecture 23 Amazon S3 - Versioning

Lecture 24 Amazon S3 - Versioning - Hands On

Lecture 25 Amazon S3 - Replication

Lecture 26 Amazon S3 - Replication - Notes

Lecture 27 Amazon S3 - Replication - Hands On

Lecture 28 Amazon S3 - Storage Classes

Lecture 29 Amazon S3 - Storage Classes - Hands On

Lecture 30 Amazon S3 - Lifecycle Rules

Lecture 31 Amazon S3 - Lifecycle Rules - Hands On

Lecture 32 Amazon S3 - Event Notifications

Lecture 33 Amazon S3 - Event Notifications - Hands On

Lecture 34 Amazon S3 - Performance

Lecture 35 Amazon S3 - Select & Glacier Select

Lecture 36 Amazon S3 - Encryption

Lecture 37 About DSSE-KMS

Lecture 38 Amazon S3 - Encryption - Hands On

Lecture 39 Amazon S3 - Default Encryption

Lecture 40 Amazon S3 - Access Points

Lecture 41 Amazon S3 - Object Lambda

Lecture 42 Amazon EBS

Lecture 43 Amazon EBS - Hands On

Lecture 44 Amazon EFS

Lecture 45 Amazon EFS - Hands On

Lecture 46 Amazon EFS vs. Amazon EBS

Lecture 47 AWS Backup

Lecture 48 AWS Backup - Hands On

Section 4: Database

Lecture 49 Intro: Database

Lecture 50 Amazon DynamoDB

Lecture 51 Amazon DynamoDB - Hands On

Lecture 52 Amazon DynamoDB in Big Data

Lecture 53 Amazon DynamoDB - Throughput (RCU & WCU)

Lecture 54 Amazon DynamoDB - Throughput (RCU & WCU) - Hands On

Lecture 55 Amazon DynamoDB - Basic APIs

Lecture 56 Amazon DynamoDB - Basic APIs - Hands On

Lecture 57 Amazon DynamoDB - Indexes (LSI & GSI)

Lecture 58 Amazon DynamoDB - Indexes (LSI & GSI) - Hands On

Lecture 59 Amazon DynamoDB - PartiQL

Lecture 60 Amazon DynamoDB Accelerator (DAX)

Lecture 61 Amazon DynamoDB Accelerator (DAX) - Hands On

Lecture 62 Amazon DynamoDB - Streams

Lecture 63 Amazon DynamoDB - Streams - Hands On

Lecture 64 Amazon DynamoDB - Time To Live (TTL)

Lecture 65 Amazon DynamoDB - Patterns with S3

Lecture 66 Amazon DynamoDB - Security

Lecture 67 Amazon RDS

Lecture 68 Shared and exclusive locks in RDS

Lecture 69 Amazon RDS Best Practices

Lecture 70 Amazon DocumentDB

Lecture 71 Amazon MemoryDB for Redis

Lecture 72 Amazon Keyspaces (for Apache Cassandra)

Lecture 73 Amazon Neptune

Lecture 74 Amazon Redshift Intro & Architecture

Lecture 75 Redshift Spectrum and Performance Tuning

Lecture 76 Redshift Durability and Scaling

Lecture 77 Redshift Distribution Styles

Lecture 78 Redshift Sort Keys

Lecture 79 Redshift Data Flows and the COPY command

Lecture 80 Redshift Integration / WLM / Vacuum

Lecture 81 Redshift Resizing & Newer Features

Lecture 82 Redshift AQUA, Redshift ML

Lecture 83 Redshift Security

Lecture 84 Redshift Serverless

Lecture 85 Redshift Materialized Views

Lecture 86 Redshift Data Sharing / Data Shares

Lecture 87 Redshift Lambda UDF

Lecture 88 Redshift Federated Queries

Lecture 89 Redshift - Hands On

Section 5: Migration and Transfer

Lecture 90 Intro: Migration and Transfer

Lecture 91 Application Discovery Service & Application Migration Service

Lecture 92 AWS Database Migration Service (AWS DMS)

Lecture 93 AWS Database Migration Service (AWS DMS) - Hands On

Lecture 94 AWS DataSync

Lecture 95 AWS Snow Family

Lecture 96 AWS Snow Family - Hands On

Lecture 97 AWS Transfer Family

Section 6: Compute

Lecture 98 Intro: Compute

Lecture 99 EC2 in Big Data

Lecture 100 AWS Lambda

Lecture 101 Lambda Integration - Part 1

Lecture 102 Lambda Integration - Part 2

Lecture 103 AWS Lambda - File Systems Mounting

Lecture 104 AWS SAM

Lecture 105 AWS SAM - CLI Installation

Lecture 106 AWS SAM - Create Project

Lecture 107 AWS SAM - Deploy Project

Lecture 108 AWS SAM - with API Gateway

Lecture 109 AWS SAM - with DynamoDB

Lecture 110 AWS Batch

Section 7: Containers

Lecture 111 Intro: Containers

Lecture 112 What is Docker?

Lecture 113 Amazon ECS

Lecture 114 Amazon ECS - Create Cluster - Hands On

Lecture 115 Amazon ECS - Create Service - Hands On

Lecture 116 Amazon ECR

Lecture 117 Amazon EKS

Lecture 118 Amazon EKS - Hands On

Section 8: Analytics

Lecture 119 Intro: Analytics

Lecture 120 AWS Glue

Lecture 121 Glue, Hive, and ETL

Lecture 122 Modifying the Glue Data Catalog from ETL Scripts

Lecture 123 Glue ETL: Developer Endpoints, Running ETL Jobs with Bookmarks

Lecture 124 Glue Costs and Anti-Patterns

Lecture 125 AWS Glue Studio

Lecture 126 AWS Glue Data Quality

Lecture 127 AWS Glue DataBrew

Lecture 128 Handling PII in DataBrew Transformations

Lecture 129 AWS Lake Formation

Lecture 130 AWS Lake Formation Data Filters

Lecture 131 Amazon Athena

Lecture 132 Athena and Glue, Costs, and Security

Lecture 133 Athena Performance

Lecture 134 Athena ACID Transactions

Lecture 135 Athena Fine-Grained Access to AWS Glue Data Catalog

Lecture 136 Apache Spark

Lecture 137 Athena, Glue, and S3 Data Lakes - Hands On

Lecture 138 Spark Integration with Kinesis and Redshift

Lecture 139 Spark Integration with Athena

Lecture 140 Amazon EMR

Lecture 141 EMR, AWS integration, and Storage

Lecture 142 EMR Promises; Intro to Hadoop

Lecture 143 EMR Serverless; EMR on EKS

Lecture 144 Amazon Kinesis Data Streams

Lecture 145 Amazon Kinesis Data Streams - Producers

Lecture 146 Amazon Kinesis Data Streams - Consumers

Lecture 147 Amazon Kinesis Data Streams - Hands On

Lecture 148 Amazon Kinesis Data Streams - Enhanced Fan Out

Lecture 149 Amazon Kinesis Data Streams - Scaling

Lecture 150 Amazon Kinesis Data Streams - Handling Duplicates

Lecture 151 Amazon Kinesis Data Streams - Security

Lecture 152 Amazon Kinesis Data Firehose

Lecture 153 Amazon Kinesis with CloudWatch Subscription Filters

Lecture 154 Kinesis Data Stream Troubleshooting and Performance Tuning

Lecture 155 Kinesis Data Analytics / Amazon Managed Service for Apache Flink (MSAF)

Lecture 156 Kinesis Analytics Costs; RANDOM_CUT_FOREST

Lecture 157 Amazon MSK

Lecture 158 Amazon MSK - Connect

Lecture 159 Amazon MSK - Serverless

Lecture 160 Amazon Kinesis vs. Amazon MSK

Lecture 161 Amazon OpenSearch Service

Lecture 162 Amazon OpenSearch Service, Pt. 2

Lecture 163 OpenSearch Index Management and Designing for Stability

Lecture 164 Amazon OpenSearch Service Performance

Lecture 165 Amazon OpenSearch Serverless

Lecture 166 Amazon QuickSight

Lecture 167 QuickSight Pricing and Dashboards; ML Insights

Section 9: Application Integration

Lecture 168 Intro: Application Integration

Lecture 169 Amazon SQS

Lecture 170 Amazon Kinesis Data Streams vs. Amazon SQS

Lecture 171 Amazon SQS - Dead Letter Queues

Lecture 172 Amazon SQS - Dead Letter Queues - Hands On

Lecture 173 Amazon SNS

Lecture 174 Amazon SNS - with SQS Fan Out

Lecture 175 Amazon SNS - Hands On

Lecture 176 AWS Step Functions

Lecture 177 Amazon AppFlow

Lecture 178 Amazon EventBridge

Lecture 179 Amazon EventBridge - Hands On

Lecture 180 Amazon Managed Workflows for Apache Airflow (Amazon MWAA)

Lecture 181 Full Data Engineering Pipelines

Section 10: Security, Identity, and Compliance

Lecture 182 Intro: Security, Identity, and Compliance

Lecture 183 Principle of Least Privilege

Lecture 184 Data Masking and Anonymization

Lecture 185 Key Salting

Lecture 186 Preventing Backups or Replication to Disallowed AWS Regions

Lecture 187 IAM Introduction: Users, Groups, Policies

Lecture 188 IAM Users & Groups Hands On

Lecture 189 IAM Policies

Lecture 190 IAM Policies - Hands On

Lecture 191 IAM MFA

Lecture 192 IAM MFA - Hands On

Lecture 193 IAM Roles

Lecture 194 IAM Roles - Hands On

Lecture 195 Encryption 101

Lecture 196 AWS KMS

Lecture 197 AWS KMS - Hands On

Lecture 198 Amazon Macie

Lecture 199 AWS Secrets Manager

Lecture 200 AWS Secrets Manager - Hands On

Lecture 201 AWS WAF

Lecture 202 AWS Shield

Lecture 203 AWS Services Security Deep Dive - Part 1

Lecture 204 AWS Services Security Deep Dive - Part 2

Lecture 205 AWS Services Security Deep Dive - Part 3

Section 11: Networking and Content Delivery

Lecture 206 Intro: Networking and Content Delivery

Lecture 207 VPC, Subnets, Internet Gateway, NAT Gateway

Lecture 208 NACL, Security Groups, VPC Flow Logs

Lecture 209 VPC Peering, Endpoints, VPN, Direct Connect

Lecture 210 VPC Cheat Sheet & Closing Comments

Lecture 211 AWS PrivateLink

Lecture 212 What is DNS?

Lecture 213 Amazon Route 53

Lecture 214 Amazon Route 53 - Registering Domains - Hands On

Lecture 215 Amazon Route 53 - Creating DNS Records - Hands On

Lecture 216 Amazon CloudFront

Lecture 217 Amazon CloudFront - S3 as Origin - Hands On

Lecture 218 Amazon CloudFront - ALB as Origin

Lecture 219 Amazon CloudFront - Cache Invalidation

Section 12: Management and Governance

Lecture 220 Intro: Management and Governance

Lecture 221 Amazon CloudWatch - Metrics

Lecture 222 Amazon CloudWatch - Logs

Lecture 223 Amazon CloudWatch - Logs - Hands On

Lecture 224 Amazon CloudWatch - Logs Unified Agent

Lecture 225 Amazon CloudWatch - Alarms

Lecture 226 Amazon CloudWatch - Alarms - Hands On

Lecture 227 Amazon CloudTrail

Lecture 228 Amazon CloudTrail - Hands On

Lecture 229 AWS CloudTrail Lake

Lecture 230 AWS Config

Lecture 231 AWS Config - Hands On

Lecture 232 CloudWatch vs. CloudTrail vs. Config

Lecture 233 AWS CloudFormation

Lecture 234 AWS CloudFormation - Hands On

Lecture 235 SSM Parameter Store

Lecture 236 SSM Parameter Store - Hands On

Lecture 237 SSM Parameter Store - Lambda Integration

Lecture 238 AWS Well-Architected Framework & Tool

Lecture 239 Amazon Managed Grafana

Section 13: Machine Learning

Lecture 240 Intro: Machine Learning

Lecture 241 Amazon SageMaker

Lecture 242 SageMaker Feature Store

Lecture 243 SageMaker ML Lineage Tracking

Lecture 244 SageMaker Data Wrangler

Section 14: Developer Tools

Lecture 245 Intro: Developer Tools

Lecture 246 AWS Access Keys, CLI & SDK

Lecture 247 AWS CLI Setup on Windows

Lecture 248 AWS CLI Setup on Mac OS X

Lecture 249 AWS CLI Setup on Linux

Lecture 250 AWS CLI Hands On

Lecture 251 AWS Cloud9

Lecture 252 AWS Cloud9 - Hands On

Lecture 253 AWS CDK

Lecture 254 AWS CDK - Hands On

Lecture 255 AWS CodeDeploy

Lecture 256 AWS CodeCommit

Lecture 257 AWS CodeBuild

Lecture 258 AWS CodePipeline

Section 15: Everything Else

Lecture 259 Intro: Everything Else

Lecture 260 AWS Budgets

Lecture 261 AWS Budgets - Hands On

Lecture 262 AWS Cost Explorer

Lecture 263 Amazon API Gateway

Lecture 264 Amazon API Gateway - Hands On

Section 16: Wrapping Up

Lecture 265 Intro: Wrapping Up

Lecture 266 Reviewing the Exam Guide (and other AWS resources)

Lecture 267 General AWS Certification Exam Tips

Lecture 268 Exam Walkthrough and Signup

Lecture 269 Save 50% on your AWS Exam Cost!

Lecture 270 Get an Extra 30 Minutes on your AWS Exam - Non Native English Speakers Only

Lecture 271 AWS Certification Paths

Lecture 272 Thank you!

Lecture 273 BONUS Lecture

Technologists seeking certification in data engineering technologies on Amazon Web Services