Fundamentals of Apache Airflow
Fundamentals of Apache Airflow
.MP4, AVC, 1920x1080, 30 fps | English, AAC, 2 Ch | 2h 20m | 516 MB
Instructor: Ivan Mushketyk
.MP4, AVC, 1920x1080, 30 fps | English, AAC, 2 Ch | 2h 20m | 516 MB
Instructor: Ivan Mushketyk
Automate your data pipelines with Apache Airflow. This hands-on course starts with the basics and builds up to real-world orchestration, from retries to Spark and external data ingestion.
What you'll learn
- How to build and schedule workflows with Apache Airflow
- Core concepts of DAGs, tasks, operators, and scheduling
- Error handling, retries, and making workflows fault-tolerant
- Ensuring idempotency and robustness in data pipelines
- Using sensors to wait for external systems or events
- Orchestrating Apache Spark jobs within Airflow
- Connecting Airflow with external data sources
- Ingesting data into a data lake with automation
Getting data from point A to point B is only part of the job. Making sure it gets there reliably, automatically, and at the right time? That’s where Apache Airflow comes in.
This course teaches you how to turn messy, manual pipelines into clean, orchestrated workflows. You’ll start simple - by understanding what Airflow does and how it works—then step into more advanced techniques like retries, handling failures gracefully, using sensors, and working with Apache Spark. You'll also learn how to automate data ingestion from outside sources into your data lake.
Whether you’re just getting into data engineering or looking to sharpen your orchestration game, this course gives you the practical tools to make your data pipelines reliable and scalable.
Fundamentals of Apache Airflow
