Building Ai Applications With Databricks And Gen Ai

Unlock the power of AI with Databricks & GenAI - Transforming data into intelligence, one application at a time!

What you'll learn

Use of Vector Search and LLM Models inside Databricks

Build Applications and deploy to Databricks Apps

Build End - End Data Refresh cycle

Automatic refresh data

Requirements

You should be good in Python Programming Language

Description

This comprehensive course will teach you how to develop cutting-edge AI applications by combining the power of Databricks and Large Language Models (LLMs). You will explore how to leverage Databricks for large-scale data processing, feature engineering, and model training, while integrating advanced LLMs for natural language processing (NLP) tasks such as text classification, summarization, semantic search, and conversational AI.Through hands-on labs and real-world projects, you will gain practical experience in building intelligent systems that can understand, process, and generate human language. This course is ideal for data scientists, machine learning engineers, and developers who want to stay ahead in the rapidly evolving world of AI.By the end of the course, you will have a strong understanding of how to architect end-to-end AI pipelines using Databricks and LLMs, deploy scalable NLP applications, and apply industry best practices for model integration and performance optimization.Key Highlights:Scalable data processing and ML using DatabricksNLP-powered applications with state-of-the-art LLMsPractical, project-based learning approachReal-world AI use cases and deployment strategiesUse Vector Search indexes to store indexesUse workflows to refresh the data end - end on schedule basisUse Serverless compute to refresh the dataUse Databricks Apps to deploy the application

Overview

Section 1: Architecture and Prerequiste

Lecture 1 Introduction

Lecture 2 Understanding of Dataset

Lecture 3 Databricks Workspace Setup in AWS

Lecture 4 Create S3 Bucket

Lecture 5 Download Dataset

Lecture 6 Prepare Source System - Upload Datasets in S3 Bucket

Section 2: Ingest Data from S3 to Unity Catalog

Lecture 7 Create Schema

Lecture 8 Understand Groups and Permission

Lecture 9 How to get access on S3 bucket and Create Scopes in AWS Databricks

Lecture 10 Ingest Data from S3 to Bronze Layer

Lecture 11 Create Repo and Link in Databricks

Lecture 12 Ingest other data into Bronze Layer and Setup Workflow

Section 3: Cleaning the bronze data

Lecture 13 Build Lakehouse Quality Dashboard Manually

Lecture 14 Create Monitor for all the Bronze Tables - Automation

Lecture 15 Create Clean Notebook for Patient Data

Lecture 16 Create Clean Notebook for Other Tables - Optimise

Lecture 17 Create Clean Notebook for Other Tables

Lecture 18 Update Workflow for Silver Notebooks

Section 4: Ingest data into Gold Layer

Lecture 19 Create Dimension Master Table Gold Layer and Add in Workflows

Section 5: Vector Search and Embedding Models

Lecture 20 Create Vector Search Endpoint

Lecture 21 Create Embedding Model and Serving EndPoint

Lecture 22 Create Embeddings and Create VS Index table

Lecture 23 Update Workflow - Add VS Notebooks

Section 6: Setup Model Serving Endpoint

Lecture 24 Create Serving Endpoint with RAG

Lecture 25 Query serving endpoint

Lecture 26 Run Workflows using SPN

Section 7: Build Streamlit Chatbot App

Lecture 27 Create Streamlit ChatBot Application

Intermediate Databricks Engineer