Python Programming: Build A Recommendation Engine In Django

Collaborative Filtering with Python, Celery, Django, Worker Processes, Batch Predictions, SurpriseML, Keras, and more!

What you'll learn

Learn how to integrate Django & Celery

Learn how to use HTMX with Django for Dynamic Loading (no JavaScript Needed)

Training a Machine Learning Model with SurpriseML and an example in Keras

Build a rating system in Django with dynamic rating buttons. These ratings can be used on any Django Model

Learn how to run periodic background task and/or schedule functions to run exactly when needed

How to perform batch inference effectively using Django for *any* large workloads and/or ML packages

How to load large datasets into a SQL database through Django Models

Where to find great datasets online

How to implement an "infinite" review page that will always give a new item after rating.

So much more!

Requirements

Experience Python 3.6+ (such as the first 15/16 days from my course 30 Days of Python)

Django 3.2+ experience (such as my course Your First Django Web Project or any of the Try Django series)

Celery experience is a plus! (Such as my Time & Tasks 2 course)

Machine learning experience is a plus but not required (checkout my Hello World of Machine Learning Course)

Pandas basics is a plus but not required (checkout my Try Pandas Course)

Description

Build a recommendation engine using Django & a Machine Learning technique called Collaborative Filtering.Users will rate movies and the system will automatically recommend new ones. These recommendations will be done in batches (ie not in real time) to unlock a more scalable system for training and helping thousands and thousands of users.For this course, we'll use a real dataset called MovieLens; this dataset is downloaded in CSV and is used on all kinds of machine learning tutorials. What's special about this course is you'll load this dataset into a SQL database through a Django model. This alone might be worth watching the course as SQL databases are far more powerful than CSV files.To do the batch inference we implement the incredibly powerful background worker process called Celery. If you haven't used Celery before, this will be an eye opening experience and when you couple it with Django you have a truly powerful worker process that can run tasks in the background, run tasks on a schedule, or a combination of both. Tasks in Celery are simply Python functions with a special decorator.For rating movies, we'll be using HTMX. HTMX is a way to dynamically update content *without* reloading the page at all. I am sure you know the experience whenever you click "like" or "subscribe" , that's what HTMX gives us without the overhead of using 1 line of JavaScript. This course shows us a practical implementation of using HTMX not just for rating movies, but also sorting them, loading them, and doing much more. The recommendation engine in Django is really a collection of 3 parts:Web Process: Setup up Django to collect user's interest and provide recommendations once available.Machine Learning Pipeline: Extract data from Django, transform it, and train a Collaborative Filtering model.Worker Process: This is the glue. We'll use Celery to schedule/run the trained model predictions and update data for Django-related user recommendations.Recommended ExperiencePython 3.6+ (such as 30 Days of Python)Django 3.2+ (such as Your First Django Web Project or Try Django 3.2)Celery with Django (such as Time & Tasks 2 or this blog post)

Overview

Section 1: Introduction

Lecture 1 Welcome to Recommender

Lecture 2 Requirements & In-Depth Walkthrough

Lecture 3 Where to get help

Lecture 4 Setup Project

Lecture 5 Django as a ML Pipeline Orchestration Tool

Section 2: Handling the Dataset

Lecture 6 Generate Fake User Data

Lecture 7 Django Management Command to add Fake User Data

Lecture 8 Our Collaborative Filtering Dataset

Lecture 9 Load The Movies Dataset into the Movie Django Model

Lecture 10 Create Ratings Model with Generic Foreign Keys

Section 3: Running Calculations with Django

Lecture 11 Calculate Average Ratings

Lecture 12 Generate Movie Ratings

Lecture 13 Handling Duplicate Ratings with Signals

Lecture 14 Calculate Movie Average Rating Task

Section 4: Python Celery

Lecture 15 Setup Celery for Offloading Tasks

Lecture 16 Converting Functions into Celery Tasks

Section 5: Django Views & Auth

Lecture 17 Movie List & Detail View, URLs and Templates

Lecture 18 Django AllAuth

Lecture 19 Update the Movie Ratings Task

Section 6: User Ratings

Lecture 20 Rendering Rating Choices

Lecture 21 Display a User's Ratings

Section 7: Dynamic Django with HTMX

Lecture 22 Dynamic Requests with HTMX

Lecture 23 Rate Movies Dynamically with HTMX

Lecture 24 Infinite Rating Flow with Django & HTMX

Lecture 25 Rating Dataset Exports Model & Task

Section 8: Jupyter Notebooks with Django

Lecture 26 Using Jupyter with Django

Lecture 27 Load Real Ratings to Fake Users

Lecture 28 Update Movie Data

Lecture 29 Recommendations by Popularity

Section 9: Machine Learning & Collaborative Filtering

Lecture 30 What is Collaborative Filtering

Lecture 31 Collaborative Filtering with Surprise ML

Lecture 32 Surprise ML Utils & Celery Task For Surprise Model Training

Lecture 33 Batch User Prediction Task

Section 10: Handling Predictions in Django

Lecture 34 Storing Predictions in our Suggestion Model

Lecture 35 Updating Batch Predictions Based on Previous Suggestions

Lecture 36 ML-Based Movies Recommendations View

Lecture 37 Trigger ML Predictions Per User Activity

Lecture 38 Position Ranking for Movie Querysets

Lecture 39 Movie Embedding Idx Field and Task

Lecture 40 Movie Dataset Exports

Lecture 41 Schedule for ML Training, ML Inference, Movie IDX Updates, and Exports

Section 11: Wrap Up

Lecture 42 Overview of a Neural Network Colab Filtering Model

Lecture 43 Thank you and next steps

Beyond the basics Django Developers (ie you completed a Try Django course),Anyone interested in building powerful ML-heavy Web Applications,Anyone looking to learn about Python Celery for Worker processes,Anyone interested in building workflows that need to run along side of Django.