Pattern Detection With Python Regular Expressions (Regex)

Hands-On : Detect Patterns in Data, Verify Input, Improve Security, Data Cleanup

What you'll learn
Pattern Detection - Look for occurrences of a pattern using a concise language
Data Preparation - Use regex to locate and transform data of interest
Input Validation - Cross-Check Input and Improve Security by Preventing Injection Attacks
Learn Techniques to Write High-Performance Patterns
Complements Machine Learning Skills
Hands-on projects
Requirements
Familiarity with Python
Description
Hi, and welcome to the Pattern Detection with Python Regular Expressions (Regex) Course!This is a project driven course and in just a couple of hours, you will gain precise and relevant information that you can immediately apply to your projectsI am Chandra Lingam, and I am your instructor.Here are some typical uses of regular expressionPattern DetectionLook for occurrences of a pattern using a concise languageData PreparationData clean-up and preparation is often one of the most time-consuming activitiesYou can define the structure of data as a regex pattern and parse dataOne good application of this is AWS Glue and Athena.You can use regex to define the structure of a record in a plain text file, Create a table and query the file using SQLInput ValidationYou can implement a client-side check for input validationFor example, your app can guide the user to provide data in the correct format using regex.As part of the zero-trust architecture, you need to validate input to your microserviceWith regex, you can verify and validate data payloads in your serviceCloud ServicesSeveral cloud services use regex for advanced configuration.With the AWS web application firewall, you can allow or deny traffic based on a regex patternIn Google Workspace, you can use regex for content filtering, Gmail route configuration, and to search for content in google docsIn Google Analytics, you can use regex to locate and transform matching data in your data setRegex is also supported by several products such as SAP, Oracle, and SQL ServerCurriculumThe source code for this course is distributed using Github – so, you always have access to up-to-date codeAs part of resources, you will get this high-quality cheat-sheet for regex languageAnd an interactive regex tool to write patternsIn the Python Regex features section, you will get familiar with various regex methods, their purpose, and how to unit test your patternIn the regex language section, you will learn how to write patterns – starting from the simplest of patternsYou will also learn to incorporate regex in your HTML input types for validationRegex engine puts the onus on the developers, that is us, to write efficient patternsIn this section, you will gain knowledge of regular expression engine that will help you write optimal patternsThere are several exercises for you to apply your new skillsWe then look at performance and how poorly written patterns can degrade exponentiallyYou will learn how to optimize the patterns and address performance issuesThere are four hands-on projects in this courseYou will learn how to apply the regex for distinctly different data sets – unstructured log data, IoT sensor data, and parsing medical test data in HTML formatYou will get prompt support through the course Q&A forum and private messaging.I am looking forward to meeting youThank You!Chandra LingamCloud Wave LLC

Overview

Section 1: Introduction

Lecture 1 Introduction

Lecture 2 Increase the speed of learning

Lecture 3 Source Code Download

Lecture 4 Anaconda Python Environment Housekeeping

Lecture 5 Weekly Study Group - Live Q&A

Section 2: Python Regex Features

Lecture 6 Downloadable Resources

Lecture 7 Introduction to Regex Features

Lecture 8 RE Module, Match method, Unit Testing

Lecture 9 Regex Best and Worst Performance

Lecture 10 RE Module - Search, FindAll, FindIter, Groups

Lecture 11 RE Module - Find and Replace, Split

Lecture 12 Interactive Tool

Section 3: Python Regex Language

Lecture 13 Downloadable Resources

Lecture 14 Single Character Patterns

Lecture 15 Anchors

Lecture 16 Character Classes

Lecture 17 Quantifiers

Lecture 18 HTML input validation example

Lecture 19 Input Validation Example (Browser)

Section 4: Python Regex Engine - Behind the scenes

Lecture 20 Downloadable Resources

Lecture 21 One character at a time

Lecture 22 Left to Right

Lecture 23 Unusual Behavior When Using FindAll

Lecture 24 Lab - Left to Right

Lecture 25 Greedy, Lazy and Backtracking Analogy

Lecture 26 Greedy, Lazy and Backtracking Examples

Lecture 27 Lab - Greedy, Lazy and Backtracking

Lecture 28 Groups, Backreference, Replacement

Lecture 29 Lab - Groups, Backreference, Replacement

Lecture 30 Look Ahead

Lecture 31 What is a mark character?

Lecture 32 Look Behind

Lecture 33 Look Behind – Why does the pattern not work?

Lecture 34 Exercise - Currency Symbol

Lecture 35 Solution - Currency Symbol

Lecture 36 Exercise - Match a number

Lecture 37 Solution - Match a number

Lecture 38 Exercise - List all cars not made by Honda

Lecture 39 Solution - List all cars not made by Honda

Lecture 40 Exercise - Webserver Log Parser

Lecture 41 Solution - Webserver Log Parser

Lecture 42 Exercise - Filter by price

Lecture 43 Solution - Filter by price

Lecture 44 Exercise - List cars that meet specified criteria

Lecture 45 Solution - List cars that meet specified criteria

Lecture 46 Exercise - Password Validation

Lecture 47 Solution - Password Validation

Section 5: Python Regex Performance

Lecture 48 Downloadable Resources

Lecture 49 Exponential degradation - example of bad patterns and performance implication

Lecture 50 How to correct performance issues and optimize pattern

Lecture 51 Compiled versus Module Methods

Section 6: Project 1 - Log Parser

Lecture 52 Log Data Parser Objective

Lecture 53 Exercise 1 - Write a pattern to capture header information

Lecture 54 Exercise 2 - Write a pattern to capture error message

Lecture 55 Exercise 3 - Write a pattern to capture metrics

Lecture 56 Solution - How to write log parser regex patterns

Lecture 57 Solution - Log Data to JSON

Section 7: Project 2 - IoT Sensor Data

Lecture 58 Sensor Data Parser Objective

Lecture 59 Exercise 1 - Capture Date Value

Lecture 60 Exercise 2 - Capture Temperature and Humidity Value

Lecture 61 Solution - How to write sensor data patterns

Lecture 62 Solution - Sensor Data to JSON

Section 8: Project 3 - Health Care Data

Lecture 63 Health care Data Parser Objective

Lecture 64 Exercise 1- Cleanup pattern

Lecture 65 Exercise 2 - Write a pattern to capture a row

Lecture 66 Exercise 3 - Write a pattern to capture a cell

Lecture 67 Solution - How to write health care data patterns

Lecture 68 Solution - Health care data to CSV

Section 9: Project 4 - Network Configuration Parser

Lecture 69 Network Configuration Parser

Lecture 70 Network Configuration Parser - Answer

Section 10: Interesting Question and Answers from the Discussion Forum

Lecture 71 How to Remove Embedded Comma Inside Double Quotes

Lecture 72 How to Extract Unit Number from Postal Address

Lecture 73 Unusual Behavior When Using FindAll

Lecture 74 How to split text that uses comma and/or newline as separators

Section 11: Conclusion

Lecture 75 Congratulations!

Data Scientists,System Administrators,Data Analysts,Developers