Working With Hadoop [Dec-22]

Learn the Advance Features of Hadoop Ecosystem with Hands-On

What you'll learn

Importing Incremental data from RDBMS to HDFS and from RDBMS to Hive

Hive Partitioning, Bucketing and Indexing

Exporting Incremental Data from hive to RDBMS and from HDFS to RDBMS

Creating Hive Tables for Different file formats

Developing the Pig Latin Scripts in Pig

Scheduling the OOZIE Workflow using Coordinator

Scheduling the OOZIE Sub-Workflow using coordinator

Flume Integration with HDFS

Reading Data from HDFS to Spark 1.x

Reading and Loading data from Hive to spark 1.x using spark SQL

Requirements

Hadoop Fundamentals (one of our courses in Udemy)

Basic Python Programming Knowledge

Working Knowledge on Data Base Systems and Data Warehouses

Basic Java Programming Knowledge

Basic Linux Commands

Description

If you are looking for building the skills and mastering in Big Data concepts, Then this is the course for you.The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures. In this course, you will learn about the Hadoop components, Incremental Import and export Using SQOOP, Explore on databases in Hive with different data transformations. Illustration of Hive partitioning, bucketing and indexing. You will get to know about Apache Pig with its features and functions, Pig UDF’s, data sampling and debugging, working with Oozie workflow and sub-workflow, shell action, scheduling and monitoring coordinator, Flume with its features, building blocks of Flume, API access to Cloudera manager, Scala program with example, Spark Ecosystem and its Components, and Data units in spark.What are you waiting for?Hurry up!!!!!!

Overview

Section 1: Introduction

Lecture 1 Introduction

Section 2: Lesson 1: Working with SQOOP

Lecture 2 Lesson 1: Working with SQOOP

Lecture 3 Practice 1-1: Import Incremental Data from RDBMS to HDFS and from RDBMS to Hive

Lecture 4 Practice 1-2: Export Incremental Data from HIVE to RDBMS and from HDFS to RDBMS

Section 3: Hive Concepts

Lecture 5 Lesson 2: Working with HIVE

Lecture 6 Practice 2-1: Working with HQL Scripts in HIVE

Section 4: Data Storage and Performance in HIVE

Lecture 7 Lesson 3: Data Storage and Performance in HIVE

Lecture 8 Practice 3-1: Hive Partitioning

Lecture 9 Practice 3-2: Hive Bucketing

Lecture 10 Practice 3-3: Hive Indexing

Lecture 11 Practice 3-4: Creating Hive Tables for Different File Formats

Section 5: Working with Pig

Lecture 12 Lesson 4: Working with Pig - Troubleshooting and Optimization

Lecture 13 Practice 4-1: Developing the Pig Latin Scripts in Pig

Section 6: Oozie Concepts

Lecture 14 Lesson 5: Working with Oozie

Lecture 15 Practice 5-1: Scheduling the OOZIE Workflow using Coordinator

Lecture 16 Practice 5-2: Scheduling the OOZIE Sub-Workflow using coordinator

Section 7: Flume Integration with HDFS

Lecture 17 Lesson 6: Integration of Flume with HDFS

Lecture 18 Practice 6-1: Flume Integration with HDFS

Section 8: Cloudera Administration

Lecture 19 Lesson 7: Cloudera Administration

Lecture 20 Practice 7-1: Creating the Dashboard in Cloudera Manager

Lecture 21 Practice 7-2: Verifying the Logs and status of Job in Cloudera Manager

Section 9: Scala and Apache Spark

Lecture 22 Lesson 8: Introduction to Scala and Apache Spark

Lecture 23 Practice 8-1: Read Data from HDFS to Spark 1.x

Lecture 24 Practice 8-2: Read and Load data from Hive to spark 1.x using spark SQL

Data Base and Data Warehouse Developers,Big Data Developers and Architects,Data Scientists and Analysts,Any technical personnel who are interested learning and Exploring the features of Big Data and Tools