Etl On Cloud Platforms Azure, Aws & Gcp

Developing ETLs on Azure, Amazon Web Services (AWS) and Google Cloud Platform (GCP)

What you'll learn

General understanding of Data Integration in Azure, Amazon Web Services, Google Cloud Platform

To gain the ability to muster necessary services in Azure, Amazon Web Services, Google Cloud Platform to create a funtional ETL

To be able to justify the choice of Data Integration services selected for a particular use case

To be able to create a functional ETL in Azure, Amazon Web Services, Google Cloud Platform using the available services

Requirements

Previous ETL experience is needed. However a keen beginner could follow and be able to attain the learning outcomes stipulated above

Previous experience in the Data Ecosystem areas such Data Engineering, Data warehousing or Business Intelligence to whom ETL is not a new concept. However, those who are keen to get their hands dirty without the said prerequisites can also benefit from the course if they follow it closely and enrich its contents with other resources

Description

The course "ETLs on Cloud Platforms Azure, Amazon Web Services (AWS), and Google Cloud Platform (GCP)" is designed to provide a comprehensive understanding of Extract, Transform, Load (ETL) processes across the three most widely used cloud platforms. The primary objective is to equip learners with the knowledge and skills to effectively design, develop, and manage ETL pipelines on Azure, AWS, and GCP, each of which offers unique tools and services for data integration.The course starts by building a strong foundation in ETL fundamentals, helping learners understand the principles of extracting data from various sources, transforming it into usable formats, and loading it into target systems. It then delves into the specific ETL tools and services provided by each cloud platform—Azure Data Factory, AWS Glue, and Google Cloud Data Fusion—ensuring that students can navigate and utilize these platforms effectively.One of the key focuses of the course is on building scalable and efficient data pipelines that can handle large-scale data processing tasks. It also emphasizes optimization techniques for enhancing performance while minimizing costs, an essential aspect of cloud-based ETL operations. Moreover, the course covers best practices for ensuring data security and compliance with industry regulations, which is critical in today's data-driven world.Hands-on experience is a significant component of the course, with real-world scenarios that enable learners to apply what they've learned in practical settings. Additionally, the course explores cross-platform interoperability, teaching students how to design ETL processes that can operate seamlessly across Azure, AWS, and GCP.By the end of the course, participants will be equipped with the expertise to implement robust and efficient ETL solutions on any of these cloud platforms, preparing them for advanced roles in cloud data engineering and enhancing their career prospects in the rapidly evolving field of cloud computing.

Overview

Section 1: Introduction

Lecture 1 Introduction

Lecture 2 Course Breakdown

Section 2: Cloud Computing

Lecture 3 Cloud Computing Fundamentels

Section 3: Extract Transform & Load (ETL)

Lecture 4 ETL Process

Section 4: Portal Navigation

Lecture 5 Azure

Lecture 6 GCP

Lecture 7 AWS

Section 5: Course Data

Lecture 8 Data

Section 6: Azure (End-to-End ETLs)

Lecture 9 09 Azure Introducing ETL Structure and Creating Storage

Lecture 10 10 Azure Uploading Data and Creating Data Factory

Lecture 11 11 Azure Creating Copy Activity ETL Pipeline Part 1

Lecture 12 12 Azure Creating Copy Activity ETL Pipeline Part 2

Lecture 13 13 Azure Creating and Configuring Server and Database

Lecture 14 14 Azure Creating Database Objects

Lecture 15 15 Azure Creating Copy to Database ETL Pipeline

Section 7: Amzon Web Service (AWS End-to-End ETLs)

Lecture 16 16 AWS Introducing ETL Structure

Lecture 17 17 AWS Creating S3 Bucket and Uploading Data

Lecture 18 18 AWS Creating an IAM Role

Lecture 19 19 AWS Creating Glue ETL Part 1

Lecture 20 20 AWS Creating Glue ETL Part 2

Lecture 21 21 AWS ETL Correction and Veriying ETL Data Load

Section 8: Google Cloud Platform (GCP End-to-End ETLs)

Lecture 22 22 GCP Introducing ETL Structure.

Lecture 23 23 GCP Creating Cloud Storage and Upload Data

Lecture 24 24 GCP Creating Data Fusion and ETL'ng to BigQuery

Section 9: Conclusion

Lecture 25 The End

Intermediate to advanced Data Engineering, Data warehousing or Business Intelligence Developers