Apache Druid : Complete Guide

Learn Druid Architecture, Kafka Ingestion, Schema Evolution, Tuning and Hive & Presto Integration

What you'll learn
In depth knowledge on Druid Components and it's Architecture
Realtime data ingestion from Apache Kafka using Twitter Producer application
Tuning Apache Druid for better throughput
Accessing Apache Druid Tables through Avatica JDBC driver
Learning Schema Evolution
Complete Druid Hive Integration with hands-on experience
Presto Druid Connector

Requirements
Basics of Apache Kafka, Apache Hive, Presto
Practical experience on Mysql, AWS

Description
What do you learn from this course ?In this course, we learn end-to-end apache druid salient features and integration with Apache Hive, Prestodb (Trino), Spark and Schema Registry one by one.We start  this course by gaining theoretical knowledge on Druid and its key features. We write our own Twitter Producer app which pulls the tweets from twitter in realtime and push the tweets to apache Kafka. We create a Kafka streaming task on Druid which pull tweets from Kafka and store it into Apache Druid. Also, we learn how to apply transformation, filter, schema configuration, tuning during kafka ingestion.In the 3rd module, we explore Native and SQL Batch ingestion methods in depth. In an  ETL pipeline after extract and transform step, if you want to load the dataset to druid, then you must checkout this section. We  will automate the entire loading to druid part.In the 4th module, we learn how to read druid tables using Spark and create Spark Dataframe from it. We also explore the predicate and aggregate pushdown spark features.Section 5, talk about the Schema Registry. We learn how druid talk to schema registry and achieve  the schema validation. Also how druid parse the Avro records.Section 6 & 7 exposes the out of box druid capabilities. which are hive and presto integration. If your organisation data resides in hive or presto and you would like to join with druid table, then you should accomplish hive or presto integration.

Who this course is for:
Data Engineers,Software Engineers