Learn Aws Data Engineering

ETL & BI on AWS Cloud

What you'll learn

Data engineering concepts and AWS services

Fetching data from external Rest Api

Fetching data from SFTP server

Ingesting data into a Database

Creating Serverless Data Lake using S3 and Athena

Creating Glue ETL jobs and Workflows

Transforming data into Parquet format

Learn how to store data in S3 Data lakes using Parquet columnar file formats

Requirements

Hands on expertise on Python & Sql is a must

An active AWS account

Description

A hands on course that covers majority of the typical data engineering / ETL scenarios. In this course you will learn:Different services and concepts of AWS data engineeringCreating serverless data lake using S3, Glue and AthenaIngesting data using Rest ApiIngesting data using Sftp serverIngesting data into Database (AWS RDS - Postgre SQL)Incremental data loadingPrerequisites:An active AWS accountPython / SQL knowledge

Overview

Section 1: Introduction

Lecture 1 Introduction

Section 2: Data Engineering Services

Lecture 2 S3

Lecture 3 Athena

Lecture 4 Glue

Lecture 5 Lambda

Lecture 6 RDS

Lecture 7 SQS

Lecture 8 SNS

Lecture 9 Kinesis

Section 3: Live Demos

Lecture 10 Serverless Datawarehousing

Lecture 11 Ingest data from a Rest Api

Lecture 12 Ingest data into a SQL database (Postgre SQL)

Lecture 13 Incremental data Loading

Lecture 14 Ingest data from a SFTP server

Beginner python developers curious for data science,Data engineers,BI/ ETL Developers,Data Scientists / Analysts,Anyone from technical background who wants to learn Data engineering in AWS,Software developers who want to learn different ways to ingest data into AWS and build a datawarehouse