Modern Data Wrangling With Ai And Python - Beginner To Pro

Learn how to streamline your data processing and analysis with the power of AI and Python. From beginner to pro.

What you'll learn

Use AI and Python to increase the effectiveness of performing data-related tasks

Employ data wrangling to get to the answers, latent in data, quicker and more accurartely

Reduce the complexity and tedium of maintaining data products, such as adding new rows to a spreadsheet

Publish work done on data for other users to consume

Move from being frustrated with the limitations of spreadsheets to using Python with confidence

Understand the importance and benefits of data wrangling as part of the data lifecycle

Requirements

No programming experience needed but a familiarity with spreadsheets or database systems will be helpful

A willingness to take your career to the next level by learning 3 things: data wrangling, AI and Python

Description

Welcome to "Modern Data Wrangling with AI and Python: From Beginner to Pro." This comprehensive course is designed to equip participants with the essential skills and knowledge to effectively wrangle and manipulate data using the power of Python and integrate cutting-edge AI techniques.In today's data-driven world, the ability to wrangle and process data efficiently is fundamental for successful decision-making, predictive modelling, and gaining valuable insights. This course is structured to take learners on a journey from the basics of data wrangling to advanced AI-powered data manipulation, enabling them to become proficient practitioners in the field.This course begins by showing you how to install all the modern tools required for data wrangling. Next, we dive right into data exploration. We immediately start coding. After we've written our first code, we go over the concepts in a theoretical section. After data exploration, we cover structured data. After structured data comes unstructured data, including how to work with PDF files in data wrangling. Next, we cover, semi-structured data and web services, or APIs. In the structuring section, we try to answer the question - how do I make my data more useful? Next, we look at cleaning up our data. After we've cleaned up the data, we learn how to enrich data to make it more valuable. After enrichment comes data validation and we wrap up with publishing.In total, there are more than 180 videos in this course and you'll be well-versed in data wrangling once you've completed it.

Overview

Section 1: Welcome

Lecture 1 Welcome to the course

Lecture 2 Meet Gerhard

Lecture 3 Is this course for you?

Lecture 4 What will you get out of this course?

Lecture 5 What's in this course?

Section 2: Getting Started

Lecture 6 Introduction

Lecture 7 Visual Studio Code - Installing and getting Started

Lecture 8 Adding Extensions to VSC

Lecture 9 GitHub copilot

Lecture 10 Create a Github account

Lecture 11 GitHub Copilot chat

Lecture 12 GitHub Copilot continued

Lecture 13 Testing if Copilot is working

Lecture 14 Github copilot help content

Lecture 15 Anaconda

Lecture 16 Installing Github Desktop

Lecture 17 The GitHub Desktop Interface

Lecture 18 The repository for this course

Lecture 19 GIT (optional)

Lecture 20 Summary

Section 3: Data Exploration - What can you tell me about the data?

Lecture 21 Introduction

Lecture 22 A spreadsheet program

Lecture 23 The challenge with spreadsheets

Lecture 24 Follow along with me

Lecture 25 Welcome to Python (and AI)

Lecture 26 Markdown and Code fields in Jupyter

Lecture 27 What is Pandas?

Lecture 28 Read a file in Pandas and display it

Lecture 29 Describe the data

Lecture 30 Do more with Copilot, Jupyter and Python

Lecture 31 Summary

Section 4: Enough Python to start your journey

Lecture 32 Introduction

Lecture 33 Libraries

Lecture 34 More Pandas

Lecture 35 Objects

Lecture 36 Getting help from Large Language Models

Lecture 37 Methods

Lecture 38 Summary

Section 5: My journey to data wrangling

Lecture 39 My journey

Section 6: Structured Data - Tables

Lecture 40 Introduction

Lecture 41 Structured Data

Lecture 42 Structured Data in Python

Lecture 43 CSV files as structured data

Lecture 44 Excel data as structured data

Lecture 45 General methods for structured data - Excel and CSV

Lecture 46 SQL Tables

Lecture 47 SQL data in Python

Lecture 48 Summary

Section 7: More on Python - functions, properties and some other goodies

Lecture 49 Introduction

Lecture 50 Functions

Lecture 51 Function signatures

Lecture 52 Function bodies

Lecture 53 Using fucnctions

Lecture 54 A function in action

Lecture 55 Properties

Lecture 56 Properties in code

Lecture 57 For loops

Lecture 58 For loops in code

Lecture 59 Another example of for loops

Lecture 60 Getting help - Docstrings and signatures

Lecture 61 Getting help - online documentation

Lecture 62 Getting help - LLMs

Lecture 63 Getting help - Github copilot

Lecture 64 Summary

Section 8: Unstructured Data

Lecture 65 Introduction

Lecture 66 Installing Tabula in condas

Lecture 67 Tables in PDF documents

Lecture 68 Extracting a table from a PDF with Python

Lecture 69 Accessing particular cells in a dataframe

Lecture 70 Rename the columns of a dataframe

Lecture 71 Rename columns 2 and 3 of the dataframe

Lecture 72 Rename the remainder of the columns and concatenate strings

Lecture 73 Delete rows from a dataframe

Lecture 74 Split values in a column

Lecture 75 Drop columns in a dataframe

Lecture 76 PDFs with text

Lecture 77 Extract text from PDFs, using Python

Lecture 78 Summary

Section 9: Web services or Application Web Interfaces(APIs)

Lecture 79 Introduction

Lecture 80 Web services or Application Web Interfaces(APIs)

Lecture 81 The cat facts API

Lecture 82 HTTP response status codes

Lecture 83 JSON payloads

Lecture 84 More on JSON

Lecture 85 Import API call into Postman

Lecture 86 Making an API call in Postman

Lecture 87 Generate code in Postman

Lecture 88 Execute Postman code in a Jupyter Notebook

Lecture 89 Querying JSON objects in Python

Lecture 90 Accessing lists and nested values in JSON

Lecture 91 Converting JSON to Dataframes

Lecture 92 Summary

Section 10: Lists, dictionaries and data types in Python

Lecture 93 Introduction

Lecture 94 Lists

Lecture 95 More than numbers

Lecture 96 Lists - we’re only scratching the surface

Lecture 97 Lists and dictionaries in VSC

Lecture 98 Lists in action

Lecture 99 Dictionaries

Lecture 100 Dictionaries in action

Lecture 101 Data types

Lecture 102 Data types in Jupyter

Lecture 103 Lambdas

Lecture 104 Lambdas in action

Lecture 105 Conclusion

Section 11: How do make your data useful - Structuring

Lecture 106 Introduction

Lecture 107 Example dataset - A Canadian manufacturing company

Lecture 108 A data dictionary

Lecture 109 REF_DATE - changing the data type of a column

Lecture 110 GEO - getting the number of unique values in a column

Lecture 111 Dropping a column in a dataframe

Lecture 112 DGUID - Renaming and finding the meaning of a column

Lecture 113 Principal statistics - Filtering data

Lecture 114 Dropping mor than one column - UAM and UAM_ID

Lecture 115 NAICS, VECTOR and COORDINATE - grouping by more than one column

Lecture 116 Status - Getting the number of unique values in a column and it the dataframe

Lecture 117 Exporting the structured transformations to a CSV file

Lecture 118 Repeating the work we've done

Lecture 119 The datatime data type

Lecture 120 New methods used

Lecture 121 Filtering

Lecture 122 Adding a new column to a data frame

Lecture 123 Summary

Section 12: How do I clean data?

Lecture 124 Introduction

Lecture 125 The Netflix dataset

Lecture 126 Converting data and extracting digits from columns

Lecture 127 Missing rows in strings

Lecture 128 Replacing missing values in strings

Lecture 129 Replacing missing values in numbers

Lecture 130 Dropping missing rows

Lecture 131 Identifying and dropping duplicate rows

Lecture 132 Extracting numbers out of strings

Lecture 133 Getting parts of a string - slicing and substrings

Lecture 134 Getting the end of a string and finding help

Lecture 135 Getting words out of a string - splitting

Lecture 136 Advanced string extraction - regular expressions

Lecture 137 Getting help with regular expressions

Lecture 138 Applying functions to strings - mapping

Lecture 139 Summary

Section 13: Enrichment - Making data valuable

Lecture 140 Introduction

Lecture 141 Columns in dataframes - series

Lecture 142 Getting rows by their number - indexes

Lecture 143 Combining data - the concat function

Lecture 144 Adding columns together using the concat function

Lecture 145 Combining data by the same column name - merge

Lecture 146 Understanding joins

Lecture 147 Left join - returning all the rows in the left table

Lecture 148 Right join - all the rows in the right table

Lecture 149 Outer join - all the rows in both tables

Lecture 150 Joining tables by index - the Join method

Lecture 151 Adding a new row to the dataframe

Lecture 152 Removing the duplicates

Lecture 153 Adding multiple rows to a dataframe

Lecture 154 Changing the value of existing rows - update

Lecture 155 Updating rows based on a column - setting the indexes

Lecture 156 Updating a dataframe with merge

Lecture 157 Summary

Section 14: Validation - Making sure the data is correct

Lecture 158 Introduction

Lecture 159 The characteristics of good data

Lecture 160 Data lacking quality cause PR nightmares

Lecture 161 Data accuracy

Lecture 162 Getting help from GitHub co-pilot chat

Lecture 163 Identifying duplicate rows (reminder)

Lecture 164 Checking for missing values(reminder)

Lecture 165 Data completeness

Lecture 166 Data consistancy

Lecture 167 Data reliability

Lecture 168 Data relevance

Lecture 169 Data timeliness

Lecture 170 Summary

Section 15: Publishing - Querying and Presenting the data

Lecture 171 Introduction

Lecture 172 What is data publishing?

Lecture 173 Using Faker to generate fake data

Lecture 174 Querying the data

Lecture 175 Getting the total and average revenue by product

Lecture 176 Displaying revenue by product in a chart

Lecture 177 Use Matplotlib to generate a scattered plot

Lecture 178 Displaying data over time using Matplotlib and Pandas

Lecture 179 Use Seaborn for heatmaps

Lecture 180 Exporting results to PDF

Lecture 181 Exporting results to Excel

Lecture 182 Exporting to CSV

Lecture 183 Summary

Section 16: Conclusion

Lecture 184 Where to from here?

Lecture 185 Congratulations!

Data professionals, such as accountants and analysts, who want to learn about data wrangling,Data professionals who want to use AI to increase their productivity level significantly,Anyone curious about AI and how it can be used in the real world, right now