AWS ETL [Glue, Data Pipeline and Athena] Fundamentals

AWS ETL Fundamentals. Make Data Pipelines using AWS Glue and Data Pipeline. Query data with AWS Athena

What you'll learn
How to setup an AWS Data Pipeline
How to setup an AWS Glue Pipeline
How to work on ETL Pipelines on AWS Cloud
How to use AWS Data Brew
How to use AWS Glue Studio
Make workflows in AWS Glue Workflow
Understand what is the difference ETL and ELT
When to use Quick-sight, FireHose and Athena
How to monitor AWS Data Pipeline
When to use AWS EMR or AWS Glue
How to use AWS Glue Crawlers
Why we use AWS Athena
How to use AWS Athena

Description
Welcome to this AWS Data Pipeline Course

We are very excited to get this course out to you. This course will take you from being a beginner to an expert in creating ETL Pipelines on the cloud using AWS Data Pipeline and AWS Glue.

In this course you will Learn by example, where we demonstrate all the concepts discussed so that you can see them working, and you can try them out for yourself as well.

My step-by-step training will initiate you into the world of AWS ETL Pipelines.

Amazon provides ETL (Extract, Transform and Load) tools which integrate with different data storages. Once you prepare the data AWS allows you to query it using Athena and then build dashboards using Quicksight. What is important is you know which tool to use when. EMR is great when used with Hadoop but Glue is great for quick pipeline.

Why Learn AWS Data Pipeline/Glue/EMR/Athena?

Another Question: What tool should you use for Scalable Data Integration when you have data dispersed in multiple data sources and need it to be cleaned and transformed for analysis?

AWS Data Pipeline is a web service that allows customers to create automated data transport and transformation operations. In other words, it provides data extraction, loading, and transformation as a service. To use their data, users don't need to build an expensive ETL or ELT platform; instead, they may use Amazon's cloud environment. Data keeps growing as business activity grows, so the scalability of your pipeline is important.

AWS Glue coupled with other tools like Glue Studio and Glue Data Brew allows you to build pipelines with varying amount of customization. AWS Glue allows you to build custom ETL pipelines while Glue Studio provides a UI tool to monitor everything. AWS Glue Data Brew is a new tool which allows you to build the pipelines without any coding. It has more than 250 transformations you can apply with just a few clicks!

AWS Glue Data Classify provides a consistent view of your data, allowing you to appropriately clean, enrich, and catalogue it. This also assures that your data is searchable, queryable, and ETL-ready right away. Understand when to use AWS EMR and when to use AWS Glue. From engineering to data to analytics, it offers immense potential for teams across corporate businesses.

Finally we take a look at some real life examples to study where and how AWS Glue or Data Pipeline can help us. We ensure you learn the best practices at all times.

By the end of the course you will have set up and learnt to manage data transformations in AWS like a pro.

Everything is well documented and separated, so you can find what you need. Assignments and Quizzes will make sure you stay on track and test your knowledge. The course will have a combination of theory and practical examples.

Who this course is for:
Software Developers who want to understand how to integrate data efficiently using a ETL tool like AWS Data Pipeline
DevOps Engineers
CTO's looking to setup AWS for data integration
Anyone interested in how to query data using Athena
Developers who want to use AWS Glue or Data Pipeline as an ETL tool