Mastering Advanced Mlops On Gcp-Ci/Cd, Kubernetes Kubeflow

Simply streamline ML pipelines with GitHub Actions, GitLab CI, Jenkins, PostgreSQL, Grafana, Kubeflow & Minikube on GCP.

What you'll learn

Build and manage robust continuous integration and deployment pipelines using tools like GitHub Action and Jenkins tailored for machine learning s, GitLab CI/CD

Utilize containerization and orchestration tools such as Docker, Kubeflow, and Minikube to create scalable, production-ready ML systems on GCP.

Efficiently manage and secure ML data with PostgreSQL while implementing real-time monitoring and visualization dashboards using Grafana.

Apply best practices in scaling, resource management, and security compliance to ensure efficient and secure ML operations in cloud environments.

Requirements

Programming Proficiency: Basic to intermediate experience with programming, particularly in Python, which is widely used in machine learning and scripting for automation.

A basic understanding of machine learning principles, including data preprocessing, model training, and evaluation.

Prior experience with version control systems like Git, which is essential for managing code and collaborating on CI/CD pipelines.

An introductory understanding of cloud platforms (with a focus on GCP) and containerization (e.g., Docker) will help you grasp the orchestration concepts covered in the course.

Description

This course is designed for professionals looking to master advanced MLOps on Google Cloud Platform. It offers an in-depth exploration of the latest techniques and tools required to build, deploy, and manage scalable machine learning workflows in production environments.Throughout the course, learners will dive into the full lifecycle of MLOps, starting with the fundamentals of continuous integration and continuous delivery (CI/CD). You'll gain hands-on experience with industry-leading CI/CD tools such as GitHub Actions, GitLab CI, and Jenkins, learning how to automate testing, deployment, and version control for your ML models.Key components of the course include:CI/CD Pipelines: Understand the principles of CI/CD and learn how to implement automated workflows tailored for machine learning projects. You will configure pipelines that not only deploy code but also handle model training, testing, and validation seamlessly.Data Management with PostgreSQL: Learn best practices for integrating and managing databases in your ML projects. This section covers how to use PostgreSQL for storing and versioning data, ensuring data integrity and efficient retrieval during model training and inference.Monitoring & Visualization with Grafana: Gain insights into setting up real-time monitoring dashboards with Grafana. You’ll learn how to track model performance, system health, and resource utilization to maintain optimal operations in your ML systems.Containerization & Orchestration: Delve into containerization using Docker and master advanced orchestration tools with Kubeflow and Minikube. These sessions focus on deploying containerized ML workflows on GCP, enabling you to build scalable, production-grade systems that can easily be managed and scaled.Advanced GCP Integration: Explore the robust ecosystem of GCP services tailored for machine learning and data operations. You will understand how to integrate these services into your MLOps pipelines for enhanced performance, security, and scalability.By the end of this course, learners will have developed the expertise to build, manage, and optimize complex ML pipelines in a cloud-native environment. Practical labs and a comprehensive capstone project provide opportunities to apply these concepts in real-world scenarios, ensuring that you not only understand the theory but can also implement solutions in your own organization.Whether you are a Machine Learning Engineer, Data Scientist, DevOps specialist, or Cloud Architect, this course equips you with the skills necessary to drive innovation and efficiency in machine learning operations. Prepare to transform your approach to MLOps and leverage the full power of GCP combined with state-of-the-art tools like GitHub Actions, GitLab CI, Jenkins, PostgreSQL, Grafana, Kubeflow, and Minikube.

Overview

Section 1: COURSE INTRODUCTION

Lecture 1 INTRODUCTION

Section 2: Hotel Reservation Prediction with MLFlow, Jenkins and GCP Deployment

Lecture 2 Introduction to the Project

Lecture 3 Database Setup with GCP Buckets

Lecture 4 Project Setup

Lecture 5 Jupyter Notebook Testing

Lecture 6 Data Ingestion with GCP

Lecture 7 Data Processing

Lecture 8 Model Training and Experiment Tracking using MLFLOW

Lecture 9 Training Pipeline and Data & Code Versioning

Lecture 10 User App Building using Flask and CHATGPT

Lecture 11 CI-CD Deployment using Jenkins and Google Cloud Run

Section 3: Hybrid Anime Recommender System with Comet-ML , DVC , Jenkins and Kubernetes

Lecture 12 Introduction to the Project

Lecture 13 Database Setup using GCP Buckets

Lecture 14 Project Setup

Lecture 15 Data Ingestion with GCP

Lecture 16 Jupyter Notebook Testing - Part 1

Lecture 17 Jupyter Notebook Testing - Part 2

Lecture 18 Data Processing

Lecture 19 Model Architecture and Model Training

Lecture 20 Experiment Tracking using COMET-ML

Lecture 21 Building Training Pipeline

Lecture 22 Data Versioning using DVC & Code Versioning using GitHub

Lecture 23 Building Prediction Helper Functions

Lecture 24 User App Building with Flask and CHATGPT

Lecture 25 CI-CD Deployment using Jenkins and Google Kubernetes

Section 4: User Survival Prediction with Astro Airflow , SQL , Redis , Grafana & Prometheus

Lecture 26 Introduction to the Project

Lecture 27 Database Setup using GCP Buckets

Lecture 28 Project Setup

Lecture 29 Data Engineering ETL Pipeline using Airflow and PostgreSQL

Lecture 30 Data Ingestion using PSYCOPG2

Lecture 31 Jupyter Notebook Testing

Lecture 32 Building Feature Store using REDIS

Lecture 33 Data Processing with Feature Storing

Lecture 34 Model Training with Feature Extraction

Lecture 35 Training Pipeline and Data & Code Versioning

Lecture 36 User App Building using Flask and CHATGPT

Lecture 37 Data Drift Detection using ALIBI-DETECT

Lecture 38 ML Monitoring using Grafana and Prometheus with Setup

Section 5: Custom Guns Object Detection with Tensorboard, DVC, FastAPI and Postman

Lecture 39 Introduction to the Project

Lecture 40 Project Setup

Lecture 41 Data Ingestion using KAGGLE

Lecture 42 Jupyter Notebook Testing

Lecture 43 Data Processing

Lecture 44 Building Model Architecture

Lecture 45 Model Training

Lecture 46 Experiment Tracking using TensorBoard

Lecture 47 Training Pipeline using DVC

Lecture 48 Data and Code Versioning

Lecture 49 API Building and Testing using FastAPI , SwaggerUI and Postman

Section 6: Colorectal Cancer Prediction with Mlflow+DagsHUB ,Minikube Kubernetes & Kubeflow

Lecture 50 Introduction to the Project

Lecture 51 Project Setup

Lecture 52 Jupyter Notebook Testing

Lecture 53 Data Processing

Lecture 54 Model Training

Lecture 55 Experiment Tracking using MLFLOW and DAGSHUB

Lecture 56 User App Building Using Flask and ChatGPT

Lecture 57 KUBEFLOW and MINIKUBE Installation and Setup

Lecture 58 Building Kubeflow Pipelines

Section 7: MINOR MLOPS PROJECT - 1 using CIRCLE CI

Lecture 59 IMPORTANT NOTE

Lecture 60 Introduction to the Project

Lecture 61 Project Setup

Lecture 62 Jupyter Notebook Testing

Lecture 63 Data Processing

Lecture 64 Model Training

Lecture 65 User App Building using Flask and CHATGPT

Lecture 66 Training Pipeline and Data & Code Versioning

Lecture 67 CI-CD Deployment using CIRCLE-CI and Google Kubernetes

Section 8: MINOR MLOPS PROJECT - 2 using GITLAB CI/CD

Lecture 68 IMPORTANT NOTE

Lecture 69 Introduction to the Project

Lecture 70 Project Setup

Lecture 71 Jupyter Notebook Testing

Lecture 72 Data Processing

Lecture 73 Model Training

Lecture 74 User App Building using Flask and ChatGPT

Lecture 75 Training Pipeline and Data & Code Versioning using GITLAB

Lecture 76 Google Cloud Setup

Lecture 77 CI-CD Deployment using GITLAB CI/CD

Section 9: MINOR MLOPS PROJECT - 3 using GITHUB ACTIONS

Lecture 78 IMPORTANT NOTE

Lecture 79 Introduction to the Project

Lecture 80 Project Setup

Lecture 81 Jupyter Notebook Testing

Lecture 82 Data Processing

Lecture 83 Model Training

Lecture 84 User App Building using Flask and ChatGPT

Lecture 85 Google Cloud Setup

Lecture 86 Training Pipeline and Data & Code Versioning

Lecture 87 CI-CD Deployment using GITHUB ACTIONS

Section 10: Australia Weather Rain Prediction using Github Actions, Circle CI and GITLAB

Lecture 88 Introduction to the Project

Lecture 89 Project Setup

Lecture 90 Jupyter Notebook Testing

Lecture 91 Data Processing

Lecture 92 Model Training and Training Pipeline

Lecture 93 User App Building using Flask and ChatGPT

Lecture 94 Google Cloud Setup

Lecture 95 Dockerfile , Kubernetes Deployment file and Data & Code Versioning using GitHub

Lecture 96 CI-CD Deployment using GITHUB ACTIONS

Lecture 97 CI-CD Deployment using Circle CI

Lecture 98 CI-CD Deployment using GITLAB CI/CD

Section 11: Smart Manufacturing Machines Efficiency Prediction with GITOPS, ArgoCD & Jenkins

Lecture 99 Introduction to the Project

Lecture 100 Project Setup

Lecture 101 Jupyter Notebook Testing

Lecture 102 Data Processing

Lecture 103 Model Training and Training Pipeline

Lecture 104 User App Building using Flask and ChatGPT

Lecture 105 Data & Code Versioning , Dockerfile and Manifests Building

Lecture 106 Google Cloud VM Instance Setup and MINIKUBE Configurations

Lecture 107 Jenkins Installation and Configuration on VM

Lecture 108 GITHUB Integration with JENKINS

Lecture 109 CI Pipeline ( Continuous Integration )

Lecture 110 ArgoCD Installation and Configuration

Lecture 111 CD Pipeline ( Continuous Deployment )

Lecture 112 Full CI-CD Automation using Jenkins , ArgoCD and WebHooks

Machine Learning Engineers & Data Scientists: Those who want to bridge the gap between model development and scalable deployment.,DevOps & MLOps Practitioners: Individuals aiming to integrate CI/CD pipelines and container orchestration into ML workflows.,Cloud & Infrastructure Specialists: Professionals seeking to deepen their expertise in GCP and related cloud-native tools.,Technical Leaders & Architects: Decision-makers responsible for designing and maintaining robust, scalable ML systems in production.