Kubernetes Hands-On - Deploy Microservices To The Aws Cloud (updated 9/2022)

Use Kubernetes to deploy a Microservice architecture. You'll deploy, manage and monitor a live Kubernetes cluster.

What you'll learn
Deploy containers to a Kubernetes Cluster
Run Kubernetes in AWS using either EKS or Kops
Monitor a live Kubernetes cluster using Prometheus and Grafana
Analyse system-wide logs using the ELK Stack (ElasticStack); Kibana and ElasticSearch
Handle Alerts in a Kubernetes cluster by notifying Slack Channels
Understand how Requests and Limits work in Kubernetes
Use Kubernetes Horizontal Pod Autoscaling
Configure Ingress Control on a live Kubernetes deployment
Understand Kubernetes StatefulSets
Integrate Kubernetes with a Continuous Deployment System (CD)
Use Helm to Manage Charts and to Dynamically Update your Kubernetes YAML
Requirements
Some previous knowledge of Docker is useful, but an overview is provided as part of the course
Previous knowledge of AWS is useful for the later sections of the course; however all the concepts are explained in detail and this would serve as a great first project on AWS
We'll be using the terminal throughout the course so some (basic) familiarity of terminal operations is assumed
You will need a modern computer with at least 8Gb of RAM to run the practical sessions
Description
Kubernetes is one of the hottest topics right now, and engineers with Kubernetes skills are in big demand. Get those skills with this course! It's is a great chance to work on a real Kubernetes project, and to get yourself to a high professional standard on real projects.All the way through the course you'll be working on realistic requirements - but you don't need to be a coder or know any particular programming language - I've prepared for you a set of Docker images, and your job is to use Kubernetes to get these images running. The system is a Microservice-based architecture, and along the way, we'll look at design decisions and trade-offs you need to make when managing these complex systems. Note the course isn't about how to design Microservices (although we'll certainly be talking about that); the development work is done and we need to get the system running on a production cluster.We'll also discover that the developers have made some bad mistakes in their code, by analyzing the run time performance of the cluster!You can do the first part of the course on your local development computer (PC/Mac/Laptop). The second part (from Chapter 13 onwards) moves to the cloud. You'll use a real AWS account, and we go-ahead to set up monitoring with the ELK/Elastic Stack and monitor with Prometheus and Grafana.The course now supports EKS, the AWS Elastic Kubernetes Service, but we also show how to use the alternative system, called Kops. The pros and cons of each are explained in detail.I've designed this course for a wide audience - whether you're a DevOps engineer, a developer, or if you're quite new to the whole field, I'll explain everything along the way. Just some basic knowledge of working with computers, and maybe a bit of command-line experience will suffice.You will need an AWS account for a part of the course if you want to work on the system yourself. If you're new to AWS then don't worry, I've got you covered -but Amazon will charge you for running the system (as with all/most cloud providers). Expect to pay no more than around 10USD for this (this is a safe overestimate), but you are expected to manage this spend yourself and you must delete your Kubernetes cluster at the end of your session. Don't let that put you off, it's a great investment.You can absolutely just watch the AWS videos if you'd prefer not to incur these costs - you will still learn plenty!As always, I'm here to answer questions and I aim to respond within 48 hours.Good luck!

Overview

Section 1: Introduction

Lecture 1 Introduction to Kubernetes Microservices course

Lecture 2 Code for the course

Lecture 3 Support for Apple M1 users

Section 2: Welcome to Kubernetes

Lecture 4 Introducing Kubernetes

Lecture 5 Hands-on Project

Section 3: Installing Minikube for local Kubernetes Development

Lecture 6 Extra note for Linux Users

Lecture 7 If you already have Docker installed…

Lecture 8 Setting up Virtualization

Lecture 9 (Windows 10 Professional Only): Setting up Hyper-V

Lecture 10 Installing kubectl and minikube

Lecture 11 Troubleshooting Minikube

Section 4: (optional) Docker Quickstart

Lecture 12 Docker Overview

Lecture 13 Docker Containers vs Images

Lecture 14 Attention Apple M1 Users!

Lecture 15 Running Containers from DockerHub

Section 5: Kubernetes Pods

Lecture 16 Pods Overview

Lecture 17 Writing a Pod

Lecture 18 Running a Pod

Section 6: Services in Kubernetes

Lecture 19 Important note for Docker Desktop and Driver Users

Lecture 20 Services

Lecture 21 NodePort and ClusterIP

Lecture 22 Pod Selection with Labels

Section 7: Exercise: Deploy ActiveMQ as a Pod and Service to Kubernetes

Lecture 23 Exercise: Deploy ActiveMQ as a Pod and Service

Section 8: Kubernetes ReplicaSets

Lecture 24 ReplicaSets

Lecture 25 Writing a ReplicaSet

Lecture 26 Applying a ReplicaSet to Kubernetes

Section 9: Kubernetes Deployments

Lecture 27 Deployments Overview

Lecture 28 Managing Rollouts

Section 10: Networking and Service Discovery

Lecture 29 Networking Overview in Kubernetes

Lecture 30 A quick note about DNS pods

Lecture 31 Namespaces - kube-system

Lecture 32 Note for M1 Architectures

Lecture 33 Accessing MySQL from a Pod

Lecture 34 Cygwin extra - fixing the terminal with winpty

Lecture 35 Service Discovery

Lecture 36 Fully Qualified Domain Names (FQDN)

Section 11: Microservice Architectures

Lecture 37 WARNING - possible resource problems!

Lecture 38 An Introduction to Microservices

Lecture 39 Introduction to Microservices Part 2

Lecture 40 Fleetman Microservices - setting the scene

Lecture 41 Deploying the Queue

Lecture 42 Deploying the Position Simulator

Lecture 43 Inspecting Pod Logs

Lecture 44 Deploying the Position Tracker

Lecture 45 Deploying the API Gateway

Lecture 46 Deploying the Webapp

Section 12: Kubernetes Persistence and Volumes

Lecture 47 Persistence

Lecture 48 Upgrading to a Mongo Pod

Lecture 49 Mongo Service

Lecture 50 Expanding the Minikube VM

Lecture 51 Volume Mounts

Lecture 52 Volumes

Lecture 53 PersistentVolumeClaims

Lecture 54 StorageClasses and Binding

Section 13: Running Kubernetes on the AWS Cloud

Lecture 55 Warning

Lecture 56 Getting started with AWS

Lecture 57 Managing a Cluster in the Cloud

Lecture 58 EKS vs Kops - Which to Choose?

Lecture 59 Pricing Differences - EKS vs Kops (prices correct mid 2020)

Lecture 60 Choose Your Own Adventure!

Section 14: KOPS - Running Kubernetes on the AWS Cloud

Lecture 61 This Section is for Kops!

Lecture 62 Introducing Kops - Kubernetes Operations

Lecture 63 Installing the Kops Environment - 2021 Update

Lecture 64 Configuring your first cluster

Lecture 65 Running the Cluster

Section 15: EKS - Running Kubernetes on the AWS Cloud

Lecture 66 This section is for EKS

Lecture 67 Getting started with EKS

Lecture 68 Install eksctl and AWS CLI

Lecture 69 Configure the AWS credentials

Lecture 70 Install kubectl

Lecture 71 Start the cluster

Section 16: Operating your Cluster

Lecture 72 This section is for both EKS and Kops

Lecture 73 Provisioning SSD drives with a StorageClass

Lecture 74 Warning - problems with AWS LoadBalancers

Lecture 75 Note for Kops users

Lecture 76 Deploying the Fleetman Workload to Kubernetes

Lecture 77 Setting up a real Domain Name

Lecture 78 Surviving Node Failure

Lecture 79 Replicating Pods in Kubernetes

Section 17: Deleting the Cluster in Kops

Lecture 80 Deleting the Cluster

Lecture 81 Restarting the Cluster

Section 18: Deleting the Cluster in EKS

Lecture 82 How to delete your EKS cluster

Lecture 83 One last cluster deletion step

Section 19: Extra - how to run Kubernetes in Google Cloud

Lecture 84 How to deploy to Google Cloud Platform

Section 20: Logging a Kubernetes Cluster

Lecture 85 Introducing the ELK / ElasticStack

Lecture 86 Installing the Stack to Kubernetes

Lecture 87 Kibana - first look

Lecture 88 Setting Filters and Refreshes

Lecture 89 Demo: analysing a system failure

Lecture 90 Kibana Dashboards

Section 21: Monitoring a Kubernetes Cluster with Prometheus and Grafana

Lecture 91 Monitoring a Cluster (2020 update)

Lecture 92 Installing the kube-prometheus-stack

Lecture 93 Use the "Classic UI" in the next video

Lecture 94 Using the Prometheus UI

Lecture 95 Introducing Grafana

Lecture 96 (optional) How to Use NodePorts on a Cluster

Section 22: The Alert Manager

Lecture 97 Introducing Alerts - 2020 Update

Lecture 98 Preparing a Slack Channel

Lecture 99 Configuring the AlertManager

Lecture 100 Troubleshoot the alertmanager.yaml config

Lecture 101 Dealing with Firing Alerts

Lecture 102 The AlertManager UI and how to Silence Alerts

Lecture 103 How to handle the Watchdog

Lecture 104 Using PagerDuty

Lecture 105 Case Study

Lecture 106 Case Study: Troubleshooting a "Delinquent" node

Section 23: (Kops only): What Happens If The Master Node Crashes?

Lecture 107 What happens if the Master Node crashes? (Kops clusters only!)

Section 24: Going Further with Kubernetes

Lecture 108 Introduction to the "Advanced Section"

Lecture 109 Code / files for this section

Section 25: Kubernetes Requests and Limits

Lecture 110 Memory requests

Lecture 111 CPU Requests

Lecture 112 Memory and CPU Limits

Section 26: Metrics Profiling in Kubernetes

Lecture 113 Enabling the Metrics Server

Lecture 114 Viewing Metrics on the Dashboard

Lecture 115 Tuning Java Spring Boot Applications, Heap restriction

Lecture 116 Setting reasonable Requests

Section 27: Horizontal Pod Autoscaling

Lecture 117 Update: you will need to modify the yaml file in the next video

Lecture 118 Introducing Replication and Autoscaling

Lecture 119 Testing Autoscaling

Section 28: Readiness and Liveness Probes

Lecture 120 Demo: why readiness probes are needed

Lecture 121 Applying Liveness and Readiness Probes

Section 29: Quality of Service and Eviction

Lecture 122 Understanding the scheduler

Lecture 123 QoS labels

Lecture 124 Evictions

Lecture 125 Pod Priorities

Section 30: Kubernetes ConfigMaps and Secrets

Lecture 126 Creating a ConfigMap

Lecture 127 Consuming a ConfigMap as Environment Variables

Lecture 128 Do changes to a ConfigMap get propagated?

Lecture 129 How to consume multiple environments variables with envFrom

Lecture 130 Mounting ConfigMaps as Volumes

Lecture 131 Creating Secrets

Lecture 132 Using Secrets

Lecture 133 Where have we already used ConfigMaps and Secrets?

Lecture 134 (extra) Using Spring Cloud Kubernetes to Hot Reload ConfigMaps

Section 31: Ingress Controllers

Lecture 135 Introducing Ingress

Lecture 136 Defining Routing Rules

Lecture 137 Adding Routes

Lecture 138 Authentication

Lecture 139 Running Ingress on AWS

Lecture 140 Testing the Ingress Rules

Lecture 141 (Extra) setting up HTTPS with TLS termination at the load balancer

Section 32: Other Workload Types

Lecture 142 Batch Jobs

Lecture 143 Cron Jobs

Lecture 144 DaemonSets

Lecture 145 StatefulSets Overview

Lecture 146 StatefulSets for Database Replication

Lecture 147 Demo: Scaling out a Mongo Database

Section 33: Continuous Deployment on a Kubernetes Cluster

Lecture 148 Introducing CI/CD

Lecture 149 Establishing a GitHub organization

Lecture 150 Setting up a Basic Jenkins System

Lecture 151 Defining a Pipeline

Lecture 152 Update - creating a Personal Access Token from Github

Lecture 153 Another minor update!

Lecture 154 Running a Multibranch Pipeline

Lecture 155 Reviewing Builds

Lecture 156 Organization Pipelines

Lecture 157 Continuous Deployment into a Cluster

Section 34: Introducing Helm

Lecture 158 Getting Started with Helm

Lecture 159 How do I Install a Helm Chart

Lecture 160 How to Find Helm Charts

Lecture 161 Installing a Monitoring Stack with Helm

Lecture 162 Working with Chart Values

Lecture 163 Customising values.yaml

Lecture 164 Avoiding Snowflake Clusters!

Lecture 165 Using Helm Pull to Take Control of a Chart

Lecture 166 Generating yaml with "Helm Template"

Lecture 167 Why would you write your own Helm Charts?

Lecture 168 Writing Go Templates for Helm

Lecture 169 Helm Functions and Pipelines

Lecture 170 Flow Control in a Helm Template

Lecture 171 Named Templates

Lecture 172 Inspecting a Professional Helm Chart

Section 35: Goodbye!

Lecture 173 Goodbye!

Lecture 174 Bonus Lecture

Anyone wanting to use Kubernetes on live production projects,We will be using AWS in the later sections of the course (optional); all AWS concepts are explained so this is a great start if you're new to the cloud, but be aware that AWS do charge for usage.