Deep Learning With Apache Spark - Masterclass!

A fast-paced guide to implementing practical hands-on examples, streamlining Deep Learning with Apache Spark

What you'll learn

Explore deep learning neural networks such as RBM, RNN, and DBN using some of the most popular industrial deep learning frameworks.

Learn how to leverage big data to solve real-world problems using deep learning.

Understand how to formulate real-world prediction problems as machine learning tasks, how to choose the right neural net architecture for a problem, and how to train neural nets using DL4J.

Configure a Convolutional Neural Network (CNN) to extract value from images.

Create a deep network with multiple layers to perform computer vision.

Classify speech and audio data.

Get up-and-running and gain an insight into the deep learning library DL4J and its practical uses.

Train and test neural networks to fit your data model.

Requirements

Basic knowledge of Machine Learning and Big Data concepts is assumed.

Description

Deep learning has solved tons of interesting real-world problems in recent years. Apache Spark has emerged as the most important and promising Machine Learning tool and currently a stronger challenger of the Hadoop ecosystem. In this course, you’ll learn about the major branches of AI and get familiar with several core models of Deep Learning in its natural way. This comprehensive 3-in-1 course is a fast-paced guide to implementing practical hands-on examples, streamlining Deep Learning with Apache Spark. You’ll begin by exploring Deep Learning Neural Networks using some of the most popular industrial Deep Learning frameworks. You’ll apply built-in Machine Learning libraries within Spark, also explore libraries that are compatible with TensorFlow and Keras. Next, you’ll create a deep network with multiple layers to perform computer vision and improve cybersecurity with Deep Reinforcement Learning. Finally, you’ll use a generative adversarial network for training and create highly distributed algorithms using Spark.By the end of this course, you'll develop fast, efficient distributed Deep Learning models with Apache Spark.Contents and OverviewThis training program includes 3 complete courses, carefully chosen to give you the most comprehensive training possible.The first course, Deep Learning with Apache Spark, covers deploying efficient deep learning models with Apache Spark. The tutorial begins by explaining the fundamentals of Apache Spark and deep learning. You will set up a Spark environment to perform deep learning and learn about the different types of neural net and the principles of distributed modeling (model- and data-parallelism, and more). You will then implement deep learning models (such as CNN, RNN, LTSMs) on Spark, acquire hands-on experience of what it takes, and get a general feeling for the complexity we are dealing with. You will also see how you can use libraries such as Deeplearning4j to perform deep learning on a distributed CPU and GPU setup. By the end of this course, you'll have gained experience by implementing models for applications such as object recognition, text analysis, and voice recognition. You will even have designed human expert games.The second course, Apache Spark Deep Learning Recipes, covers over 35 recipes that streamline eep learning with Apache Spark. This video course starts offs by explaining the process of developing a neural network from scratch using deep learning libraries such as Tensorflow or Keras. It focuses on the pain points of convolution neural networks. We’ll predict fire department calls with Spark ML and Apple stock market cost with LSTM. We’ll walk you through the steps to classify chatbot conversation data for escalation. By the end of the video course, you'll have all the basic knowledge about apache spark.The third course, Mastering Deep Learning using Apache Spark, covers designing Deep Learning models to edge industrial-grade apps. You’ll begin with building deep learning networks to deal with speech data and explore tricks to solve NLP problems and classify video frames using RNN and LSTMs. You’ll also learn to implement the anomaly detection model that leverages reinforcement learning techniques to improve cybersecurity. Moving on, you’ll explore some more advanced topics by performing prediction classification on image data using the GAN encoder and decoder. Then you’ll configure Spark to use multiple workers and CPUs to distribute your Neural Network training. Finally, you’ll track progress, solve the most common problems in your neural network, and debug your models that run within the distributed Spark engine.By the end of this course, you'll develop fast, efficient distributed Deep Learning models with Apache Spark.About the Authors● Tomasz Lelek is a Software Engineer, programming mostly in Java and Scala. He has been working with the Spark and ML APIs for the past 5 years with production experience in processing petabytes of data. He is passionate about nearly everything associated with software development and believes that we should always try to consider different solutions and approaches before solving a problem. Recently he was a speaker at conferences in Poland, Confitura and JDD (Java Developers Day), and at Krakow Scala User Group. He has also conducted a live coding session at Geecon Conference. He is a co-founder of initlearn, an e-learning platform that was built with the Java language. He has also written articles about everything related to the Java world.

Overview

Section 1: Deep Learning with Apache Spark

Lecture 1 The Course Overview

Lecture 2 Review of Key Machine Learning Terminology and Fundamentals

Lecture 3 Fundamentals of Deep Networks: Feature Engineering

Lecture 4 The Building Blocks of Deep Learning

Lecture 5 Learning Path for Deep Learning

Lecture 6 Deep Learning Use Cases

Lecture 7 Pre-requisites and Installation

Lecture 8 Up and Running with DL4J on Spark

Lecture 9 Configuration and Test Run

Lecture 10 Up and Running with TensorFlow on Spark from Yahoo

Lecture 11 Understanding the Basics of Deep Learning

Lecture 12 ND4J for NumPy-like Arrays and Operations

Lecture 13 Data.Vec for Data Preparation Pipelines

Lecture 14 DL4J for Building Neural Network Architectures

Lecture 15 Understanding the Basics of GPU

Lecture 16 Parallel Training with Multiple GPUs

Lecture 17 Designing a Basic CNN

Lecture 18 Implement a Basic CNN on DL4J in Spark

Lecture 19 Basics and Design of RNN

Lecture 20 Implement a Basic RNN on DL4J in Spark

Lecture 21 Design a Basic LSTM

Lecture 22 Implement a Basic LSTM in Spark

Section 2: Apache Spark Deep Learning Recipes

Lecture 23 The Course overview

Lecture 24 Creating a Dataframes in Pyspark

Lecture 25 Manipulating Columns in a Pyspark Dataframes

Lecture 26 Converting a PySparkdataframe to an array

Lecture 27 Visualizing an Array in a Scatterplot

Lecture 28 Setting up Weights and Biases for Input into the Neural Network

Lecture 29 Normalizing the Input Data for the Neural Network

Lecture 30 Validating Array for Optimal Neural Network Performance

Lecture 31 Setting up the Activation Function with Sigmoid

Lecture 32 Creating the Sigmoid Derivative Function

Lecture 33 Calculating the Cost Function in a Neural Network

Lecture 34 Predicting Gender based on Height and Weight

Lecture 35 Visualizing Prediction Scores

Lecture 36 Pain Point #1: Importing MNIST Images

Lecture 37 Pain Point #2: Visualizing MNIST Images

Lecture 38 Pain Point #3: Exporting MNIST Images as Files

Lecture 39 Pain Point #4: Augmenting MNIST Images

Lecture 40 Pain Point #5: Utilizing Alternate Sources for Trained Images

Lecture 41 Pain Point #6: Prioritizing High-Level Libraries for CNNs

Lecture 42 Downloading the San Francisco Fire Department Calls Dataset

Lecture 43 Identifying the Target Variable of the Logistic Regression Model

Lecture 44 Preparing Feature Variables for the Logistic Regression Model

Lecture 45 Applying the Logistic Regression Model

Lecture 46 Evaluating the Accuracy of the Logistic Regression Model

Lecture 47 Downloading and Analyzing the Therapy Bot Session Dataset

Lecture 48 Visualizing Word Counts in the Dataset

Lecture 49 Calculating Sentiment Analysis of Text

Lecture 50 Removing Stop Words from the Text

Lecture 51 Training and Evaluating TF-IDF Model Performance

Lecture 52 Comparing Model Performance to a Baseline Score

Lecture 53 Downloading Stock Market Data for Apple

Lecture 54 Exploring and Visualizing Stock Market Data for Apple

Lecture 55 Preparing Stock Data for Model Performance

Lecture 56 Building the LSTM Model

Lecture 57 Evaluating the Model

Section 3: Mastering Deep Learning using Apache Spark

Lecture 58 The Course Overview

Lecture 59 Analyzing Input Text Data That Will Need to Be Classified

Lecture 60 Configuring Word Vectors That Will Be Used in Our Network

Lecture 61 Adding Layers to Deep Neural Network

Lecture 62 Asserting Classification of Input Sentences

Lecture 63 Generating Input Video Data

Lecture 64 Creating a Neural Network for Video Classification

Lecture 65 Adding RNN and LSTMs to Network to Perform a Task Better

Lecture 66 Testing and Validating Deep Learning Model

Lecture 67 Creating Paragraph Vectors

Lecture 68 Adding Labels to Non-Labelled Data

Lecture 69 Finding Similarity between Vectors

Lecture 70 Creating a Model That Can Guess the Meaning of The Word

Lecture 71 Anomaly Detection Problem Explained

Lecture 72 Extracting Features from Input Data Using Multi-Layer Approach

Lecture 73 Adding Layer That Finds an Actual Anomaly

Lecture 74 Testing and Validating Results from Our Deep Learning Model

Lecture 75 Creating Data Generator for GAN

Lecture 76 Adding Discriminator for Our Data

Lecture 77 Create Classifier for Generated Data

Lecture 78 Performing Validation of Our Model

Lecture 79 Configuring Spark for High Data Distribution

Lecture 80 Fetching Input Set into Distributed Data Set Using Spark API

Lecture 81 Creating Training Master That Supervise Computations on the Workers

Lecture 82 Evaluating Speed of Distributed Training Using Spark

Lecture 83 Monitoring of Models Using Spark UI

Lecture 84 Speeding Up Computations by Employing Caching

Lecture 85 Partitioning Deep Learning Data into Several Workers

Lecture 86 Tweaking Spark Workers Configuration

Data Scientist, Data Analysts, Big Data Architects, Anyone with a basic understanding of Deep Learning and Big Data concepts interested in developing fast, efficient distributed Deep Learning models with Apache Spark