Kafka (In-Depth) | An Absolute Treat For Beginners

An Extensive, Exhaustive, Overwhelming, In-Depth Tutrial On Kafka (V3). A tutorial You Don't Wanna Miss.

What you'll learn

The Very First Thing You will learn is Installation, Configurations in Kafka, followed by the uderstanding, we will learn what Kafka really means.

Kafka Brokers, Topics, Producers, Consumers, Consumer Groups, Partition, ZooKeeper etc. The Kafka "Eco System"

We will also learn on concepts like: Leaders, Followers, Controllers, Kraft, Exactly Once Processing in Kafka

Marching Towards, Multiple Racks, In Sync Replica, Comitted Vs UnComitted, Idempotent

Going to Next level on: Log Segment, Time Index, Message Offset, Default Groups, Custom Messages, Publish & Consume

You will also learn about Maven, as it is required for Java Development

Java Programming 101 - Basics on How to code kafka in Java

Java Programming 102 - Kafka Producer API - Architecture, Message Time Stamp, IO Thread

Java Programming 103 - MultiThreading, Java Admin Client Program & Transaction Management

Java Programming 104 - Spring Boot, Serialization & DeSerialization

Quick Introduction on, Kafka Connect, Kafka Streams

Requirements

Java Programming is Preferred

Description

The Course "Kafka (IN-DEPTH) | An Absolute treat for Beginners", it was all started recording in the year 2023, November. The intention for this course was very clear from the day the recording was initiated, which is: "Keep it Simple, so that, the most complex puzzle/problem can be visualized very easily and effectively". The course "is" designed only for those who are starting their journey in Kafka for the very first time. This course "IS NOT" for experienced Kafka engineers/developers/administrators. The course covers the following topics: -Kafka Installation & Configuration(s) in Windows and UnixProducer, Consumer, Consumer Groups, Topic, Zookeeper, Partitions, Leaders, Followers, Controllers and many more concepts revolving around Kafka EcosystemScaling of Kafka Servers, RACK, ISRLog Segments, Default Custom GroupsMaven and it's Purpose in the web-development (This is required for Java Development)Java Programming - Basics Kafka Development, Producer and Consumer, Complete Kafka Customized Producer Code, Message Time Stamp, IO ThreadJava Programming - MultiThreading, Java Admin Client Program & Transaction ManagementJava Programming - Spring Boot, Serialization & DeSerializationA bunch of Utilities tool that has been built across, which are quite useful for the project deliverablesBasic introduction on Kafka Streams, Kafka Connect, KSQL & the list goes on…I would request you to make sure that, the course is learn't very slowly and steadily. Make sure to learn the course at your own pace. Do not indulge in learning the course to complete quickly. "Slow and Steady win's the race"Wishing you all the very best in learning Kafka. And yeah, I'll be there for your during and after your course completion. I will make sure that, you will be always in my RADAR, in your journey within IT World.

Overview

Section 1: Hail To Kafka :) - Welcome Aboard

Lecture 1 01. My name is Manju Kaushik

Lecture 2 02. How Do you Need to Visualize This Course

Lecture 3 03. Why This Course was created. Intention matters, than Outcome

Section 2: WSL Installation & Java Installation In Windows

Lecture 4 Welcome to WSL Installation

Lecture 5 01. Install WSL

Lecture 6 02. Install JDK

Lecture 7 03. Installing & Configuring Multiple JAVA - JDK

Lecture 8 04. Install Java JDK-20 in Windows 11

Section 3: Eclipse & Resource

Lecture 9 01. Install Eclipse in Windows

Lecture 10 02. Install Eclipse in WSL, Linux

Lecture 11 03. Github - Download the Resources

Section 4: Installation & Kafka Basics

Lecture 12 01. Download Kafka For Windows and Linux

Lecture 13 02. Download Kakfa In Linux and Unzip tar files using wget and curl

Lecture 14 03. Start Kafka In Windows and Linux

Lecture 15 04. Check Logs & Porperties file and Start Kafka

Lecture 16 05. Start Multiple Kafka Broker Servers

Lecture 17 06. What is Producer & Consumer and topic in short

Lecture 18 07. Create Kafka Topic, Setup df ports in Win & Linux, List, describe and delete

Lecture 19 08. What is Apache Kafka

Section 5: Kafka Theory & Practical Continued

Lecture 20 01- Topics, Partitions and Offset. Create Topic with 2p and 3p

Lecture 21 02 - Kafka Cluster & Zookeeper

Lecture 22 03 - Quorum, Partitons Divided Across Broker Server's

Lecture 23 04 - Replication factor

Lecture 24 05. Message Structure, Consumer Group, Key Null & Not null

Lecture 25 06. Produce & Consume Messages - Multiple Producer & Multiple Consumer

Section 6: Consumers, Leaders, Kraft, Controllers

Lecture 26 01 - Adding Consumers to Consumer Group

Lecture 27 02 - Set Kafka Path in Linux & Start Kafka

Lecture 28 03 - Leaders & Followers

Lecture 29 04 - Controllers

Lecture 30 05 - Kraft

Section 7: Rack, ISR

Lecture 31 01. Partitions Across Multiple Racks

Lecture 32 02. In Sync Replica List

Lecture 33 03. Comitted Vs UnComitted & InSync Replica

Lecture 34 04. Commands on ISR

Section 8: Log Segments, Default Custom Groups

Lecture 35 01. Log Segment

Lecture 36 02. Message Index, Time Index, Message Offset

Lecture 37 03. Read Messages with Specific Offset & Specific Partition

Lecture 38 04. Multiple Broker Servers in a topic command

Lecture 39 05. Publish & Consume Using all the brokers in one command

Lecture 40 06. Default Consumer Groups

Lecture 41 07. Custom Message Groups

Section 9: Maven & It's Setup

Lecture 42 01 - Introduction to Maven

Lecture 43 02 - Eclipse Maven Project

Lecture 44 03 - Executing Maven Commands

Lecture 45 04 - Maven Installation In Windows

Section 10: Java Programming 101 - Let's Get Kafka Basics

Lecture 46 01. Create Kafka Project

Lecture 47 02. First Kafka Code - Create Kafka Producer Instance

Lecture 48 03. Send The Record To Kafka Cluster

Lecture 49 04. Kafka Consumer

Lecture 50 05. Not A Bug, Add Log4j & logback properties file

Lecture 51 06. Multiple Kafka Producer & Consumer, Data Does Persist

Section 11: Java Programming 102 - Extended

Lecture 52 01. Kafka Producer API - Architecture

Lecture 53 02 - Producer and Consumer - Java Code

Lecture 54 03. Complete Kafka Customized Producer Code

Lecture 55 04. Message Time Stamp, IO Thread

Section 12: MultiThreading, Java Admin Client Program & Transaction Management

Lecture 56 01. MultiThread - Single Producer & Multiple Producer Instance

Lecture 57 02. Kafka- Message Delivery Guarantee

Lecture 58 03. Kafka Admin Java Programming

Lecture 59 04. Transaction Producer Management

Section 13: Spring Boot, Serialization & DeSerialization

Lecture 60 01 - Configure BootStrap Application

Lecture 61 02 - Code WalkThrough & Start Application

Lecture 62 03 - Serialization & DeSerializtion - 01 (Theory)

Lecture 63 04 - Serialization & DeSerializtion - 02 (Code 101)

Lecture 64 05 - Serialization & DeSerializtion - 02 (Code 102) - Custom

Section 14: End Of Journey - So What's Next

Lecture 65 01 - Kafka Connect

Lecture 66 02 - Kafka Streams 101

Lecture 67 03 - Kafka Streams 102

Lecture 68 04 - KSQL

Lecture 69 05 - End Of Journey - Thanks a lot for taking care of me, I owe you one

Beginner With Java Programming skills is preferred OR Beginner with a Good Knowledge on one of the Object Oriented Programming