Apache Spark Foundation

Apache Spark Foundation

What you'll learn

1. Apache Spark Foundation Introduction

2. Difference Between Apache Spark and Apache Hadoop - Apache Spark vs Apache Hadoop

3. Apache Spark Installation on Windows Operating System

4. Apache Spark Programming Introduction

5. Apache Spark with Java Introduction and Examples

6. Apache Spark & Scala Introduction and Examples

7. Apache PySpark Introduction and Examples

Requirements

1. Basic Computer Knowledge

2. Optionally Any Programming

3. Optionally Any RDBMS

4. Basic Cloud Computing Knowledge

Description

Apache Spark Foundation SyllabusApache Spark Introduction (What is Apache Spark, What are Apache Spark Components, What are Spark Opportunities/Job Roles)Apache Spark vs Apache HadoopApache Spark Installation on Windows Operating SystemApache Spark Programming IntroductionApache Spark with Java LanguageApache Spark with Scala LanguageApache Spark with Python (PySpark) LanguageApache Spark with R Language (SparkR)What is Apache Spark?Apache Spark is a Data Processing Framework.Apache Spark Software implemented using Scala Language.Apache Spark Applications/Programs we can write using 4 Languages. They are:1. Apache Spark with Java Language2. Apache Spark with Scala Language (Spark & Scala)3. Apache Spark with Python Language (PySpark)4. Apache Spark with R Language (SparkR)Apache Spark API Available for Java, Scala, Python and R LanguageAPI – Application Programming Interface (Contains Predefined classes, functions and Variables)Apache Spark is Pure Data Processing Framework. It does not have Storage. It can process any data.What are Apache Spark Components?Apache Spark CoreApache Spark SQLApache Spark StreamingApache Spark ML/MLibApache Spark GraphXApache Spark Core – RDD Programming (RDD – Resilient Distributed Dataset) (Transformations, Actions using either Java or Scala or Python)Apache Spark SQL – DataFrames/Tables/Datasets – We write SQL Kind Programming.Apache Spark Streaming – Streaming + Live AnalyticsApache Spark ML/MLib – Machine LearningApache Spark GraphX – Linked Data/Graph Data ProcessingApache Spark Application we can deploy onApache Spark Standalone Cluster orYARN Cluster (Hadoop Cluster)Mesos ClusterKubernetes ClusterWhat are Spark Opportunities/Job Roles?There are Two Kinds of Job Roles/ Opportunities in Apache Spark World. They are:Apache Spark DeveloperApache Spark Machine Learning Developer or Apache Spark Data ScientistApache Spark DeveloperStrong Apache Spark FoundationApache Spark Core ProgrammingApache Spark SQLApache Spark StreamingApache Spark Integration Like RDBSM, NoSQL, Streaming Frameworks and Cloud ComputingProgramming Language (Java or Scala or Python or R)SQL on Any RDBMSLinux Essentials.Any Cloud Computing like AWS or Azure or GCPApache Spark Machine Learning Developer or Apache Spark Data ScientistApache Spark DeveloperApache Spark ML/MLibApache Spark GraphXML/DL/Data Science AlgorithmsMathematics and Statistics

Overview

Section 1: Apache Spark Foundation

Lecture 1 Apache Spark Foundation Introduction

Lecture 2 Apache Spark Introduction

Lecture 3 Difference Between Apache Spark and Apache Hadoop

Lecture 4 Apache Spark Installation on Windows Operating System

Lecture 5 Apache Spark Programming Introduction

Lecture 6 Apache Spark with Java Language Introduction

Lecture 7 First Spark with Java Application or Program

Lecture 8 Spark and Scala Programming Introduction

Lecture 9 First Spark and Scala Program in Interactive Mode using Spark Shell

Lecture 10 First Spark and Scala Program in Script Mode using IntelliJ IDE

Lecture 11 PySpark Programming Introduction

Lecture 12 First PySpark Program in Interactive Mode using PySpark Shell

Lecture 13 First PySpark Program in Interactive Mode using Jupyter Notebook

Lecture 14 First PySpark Program in Script Mode using PyCharm IDE

• Any IT aspirant/professional willing to learn/Become Data Engineering using Apache Spark,• Python Developers who want to learn Spark to add the key skill to be a Data Engineer,• Who would like to learn Spark,• Who are Freshers/Experienced – Who Wants to Become Data Engineers,• Who are Programmers like Java, Scala, .Net, Python etc.. willing to learn/Become Data Engineering using Apache PySpark,• Who are Database Developer/DBA willing to learn/Become Data Engineering using Apache PySpark,• Who are Data Warehouse and Reporting People willing to learn/Become Data Engineering using Apache PySpark,• Non-Programmers like Test Engineers etc.. willing to learn/Become Data Engineering using Apache PySpark