Ai Quality Workshop: How To Test And Debug Ml Models

Supercharge your ability to drive ML performance with ML testing, drift detection, debugging, and AI bias minimization.

What you'll learn

Rapidly evaluate machine learning models for performance

Identify and address model drift

Debug production ML models

Identify and address possible ML bias issues

Requirements

This course is for data scientists and ML engineers, and assumes a working knowledge of Python and an introductory course in machine learning

Description

Want to skill up your ability to test and debug machine learning models? Ready to be a powerful contributor to the AI era, the next great wave in software and technology?Get taught by leading instructors who have previously taught at Carnegie Mellon University and Stanford University, and who have provided training to thousands of students from around the globe, including hot startups and major global corporations:You will learn the analytics that you need to drive model performanceYou will understand how to create an automated test harness for easier, more effective ML testingYou will learn why AI explainability is the key to understanding the key mechanics of your model and to rapid debuggingUnderstand what Shapley Values are, why they are so important, and how to make the most of themYou will be able to identify the types of drift that can derail model performanceYou will learn how to debug model performance challengesYou will be able to understand how to evaluate model fairness and identify when bias is occurring - and then address itYou will get access to some of the most powerful ML testing and debugging software tools available, for FREE (after signing up for the course, terms and conditions apply)Testimonials from the live, virtual version of the course: "This is what you would pay thousand of dollars for at a university." - Mike"Excellent course!!! Super thanks to Professor Datta, Josh, Arri, and Rick!! :D" - Trevia"Thank you so very much. I learned a ton. Great job!" - K. M. "Fantastic series. Great explanations and great product. Thank you." - Santosh"Thank you everyone to make this course available… wonderful sessions!" - Chris

Overview

Section 1: Welcome! Let's get set up

Lecture 1 Welcome - what you'll get from this course

Lecture 2 How to set up your free TruEra access

Lecture 3 How to use Google Colab for TruEra

Section 2: ML Testing

Lecture 4 Introduction to ML Testing

Lecture 5 Running and Interpreting Tests

Lecture 6 Creating New Tests

Section 3: ML Explainability

Lecture 7 Introduction to ML Explainability

Lecture 8 Overview of Feature Importance Methods

Lecture 9 Shapley Values - Query Definition

Lecture 10 Shapley Values - Comparing Model Outputs

Lecture 11 Shapley Values - Dealing with Feature Interactions

Lecture 12 Shapley Values - Summarization

Lecture 13 Overview - Gradient Based Explanations for Computer Vision

Lecture 14 Design - Gradient-Based Explanations for Computer Vision

Lecture 15 Evaluation - Gradient-Based Explanations for Computer Vision

Lecture 16 Hands-On Learning - Explainability

Lecture 17 Demonstration - Global and Local Explainability Analysis

Section 4: Drift

Lecture 18 Introduction to Drift

Lecture 19 Sources of Drift: Why Does Drift Happen?

Lecture 20 Identifying Drift: Metrics

Lecture 21 Identifying Drift: Challenges

Lecture 22 How to Mitigate Drift

Lecture 23 Hands-on Learning: Drift

Lecture 24 Demonstration - Going from the Model Summary to Drift Analytics

Section 5: ML Performance Debugging

Lecture 25 Introduction to ML Performance Debugging

Lecture 26 ML Peformance Debugging Methodology

Lecture 27 ML Performance Metrics - Classification

Lecture 28 ML Performance Metrics - Regression

Lecture 29 Narrowing Down the Scope of ML Performance Issues

Lecture 30 Hands-On Learning: Performance Debugging

Lecture 31 Demonstration - Performance Debugging

Section 6: Bias and Fairness in Machine Learning

Lecture 32 Introduction to Bias and Fairness in ML

Lecture 33 Worldviews of Fairness in Machine Learning

Lecture 34 How to Pick a Fairness Metric

Lecture 35 How Does Your ML Model Become Unfair?

Lecture 36 Demonstration: Fairness and Bias in ML

Lecture 37 Hands-On Learning: Bias and Fairness in ML

Data Scientists and ML Engineers who are looking to improve their ability to test, evaluate, and debug machine learning models.