Apache Kafka for Beginners - Second Edition

What you'll learn

Understand all the Kafka concepts and Kafka core internals.
Learn Kafka internals through practice
Build Kafka Producer applications using Java
Learn the Kafka Producer Techniques for reliable data delivery
Build Kafka Consumer applications using Java
Advance Kafka Consumer Techniques to handle duplicate processing of records
Consumer Rebalance and different techniques to handle them
Consumer techniques to seek to a specific offset
Learn to build Custom Serializer and Deserializer in Kafka
By the end of the course you will have a complete understanding of Apache Kafka and build Kafka consumer applications using Java

Requirements

Basic commands in Unix/Windows Platform.
Basic Java Knowledge.
Basics of Windows/Mac Platform

Description

[MAJOR UPDATE : This course was completely redone on March 2020]

Description:

Learn the fundamentals and advanced concepts of Apache Kafka in this course. This course will give you a good understanding of all the concepts through Hands-On

This course will walk you through the Kafka concepts step by step

All the Kafka concepts will be taught through Hands-On

This course will increase your technical expertise in Kafka

This course will give you good exposure to build Kafka Client applications using Java

This course will be an eyeopener for people who think Kafka is a complex system and you will find yourself in a better place after you complete this course

This course will transfer you from no Kafka experience to an experienced Kafka user.

Course Overview:

Getting Started with Kafka

In this section, I will give you all a quick introduction to Apache Kafka, terminologies and different client APIs that are part of Kafka

Download and Install Kafka

In this section, we will download the Kafka distribution from the Kafka Website.

Understanding Kafka Components and its Internals - (Theory + Hands-On)

In this section, we will explore the Kafka internals from a theoretical perspective followed by hands-on exploring the Kafka internals.

Learn about the Topics and Partitions in Kafka

Setup a Local Kafka Cluster with Multiple Brokers

Producer/Consumer messages in the Kafka Cluster

Learn about Consumer Offsets and Consumer Groups

Commit Log and Retention Policy

Learn about Kafka Load Distribution and Fault Tolerance and Robustness

Kafka Producer API - Hands-On

In this section, we will learn about building the Kafka Producer to producer messages into Kafka using the producer API

Setting up the base-project for Producer

Understand how the producer API works

Produce messages to the Kafka topic using the Synchronous/Asynchronous approach

Understand how Kafka producer API works behind the scenes

Kafka Producer API - Guaranteed Message Delivery Configurations

In this lecture, we will learn about the different configurations that guarantee message delivery into Kafka.

Explore some of the producer API Configurations

Configuring the acks and min.insync.replicas for guaranteed message delivery

Override the retries and retry-back-off-ms value\

Kafka Consumer API - Hands On

In this section, we will learn about building Kafka consumers to read messages from Kafka using the Consumer API.

Setting up the base-project for Consumers

Consuming Messages using Consumer API - poll() loop

Build Kafka Consumer using the Consumer API

auto.offset.reset - earliest vs latest

Consumer Groups & Consumer Rebalance

In this section, we will learn about Consumer Groups and Consumer rebalance.

Scaling the Consumer using Consumer Groups

Consumer Rebalance

Consumer Rebalance & max-poll-interval-ms

Consumer Offsets - Default and Manual Offset Management - Hands-On

Committing Offsets - Default Offset Commit Strategy

Manually Committing Offsets

Committing Offsets Synchronously - using commitSync()

Committing Offsets Asynchronously - using commitAsync()

Committing Specific Offsets - using CommitSync()

By the end of this, you will have a complete understanding of how Apache Kafka works and how to build Kafka client applications using Producer and Consumer API.

Who this course is for:

Any one who is willing to learn about Kafka.