Snowflake - Dynamic Table Masterclass & E2E Data Pipeline

Snowflake Dynamic Table Master Class To Build End To End Data Pipeline Declaratively Using Real Life Scenarios

What you'll learn

Understand The Challenges with Stream & Task And Future of Declarative Pipeline

Understand the purpose of Snowflake Dynamic Tables and Its Lifecycle

Master all the dynamic table fundamentals & advanced topics

What are the difference between Dynamic Tables & Stream/Task Objects

What are the limitations of Snowflake Dynamic Tables

How to implement a data pipeline & DAG using Snowflake Dynamic Tables

What are the difference between Dynamic Tables & Materialized Views

Requirements

Working knowledge with Snowflake Cloud Data Warehouse

Fundamental understanding of Snowflake stream Objects.

Fundamental understanding of Snowflake Task Objects.

Fundamental understanding of SQL Queries Using Snowflake

Description

Embark on an exciting journey into the world of Snowflake Dynamic Tables with our comprehensive hands-on course designed for snowflake data developers. This course is your key to mastering the construction of declarative data pipelines, providing you with the skills to streamline development processes and minimise object requirements.Explore the intricacies of Dynamic Tables and unlock their intelligent capabilities in determining whether data loading should follow a full or incremental approach. Through engaging hands-on tutorials, we delve into every aspect of Dynamic Tables, shedding light on operational metrics that make it an indispensable native object within the Snowflake ecosystem.This course is your gateway to answering critical questions such as:What is Dynamic Table in Snowflake?How to construct a continuous data pipeline declaratively in Snowflake.Challenges faced in building end-to-end data pipelines using stream and task objects.How Dynamic Tables identify whether data loading should be incremental or full.Understanding lag metrics in Dynamic Tables.Building a Directed Acyclic Graph (DAG) using Dynamic Tables.Support for cloning, time travel, data sharing, and stream features in Dynamic Tables.Join us on this transformative learning experience, where you'll not only find answers but gain the expertise to unlock the full potential of dynamic table development. Get ready to elevate your Snowflake skills and create efficient data pipelines. Enrol now for an immersive learning adventure! #SnowflakeDynamicTables #DataPipelineMastery

Overview

Section 1: Snowflake Dynamic Table Introduction

Lecture 1 Snowflake Dynamic Table Introduction & Declarative Pipeline Concept

Lecture 2 Prerequisite For This Course

Lecture 3 How To Access The SQL Scripts & Data Files

Lecture 4 What To Expect From This Snowflake Dynamic Table Master Class

Section 2: [Optional] Snowflake Widely Used Native Objections

Lecture 5 Introduction - Common Used Snowflake Native Objects

Lecture 6 What Is Snowflake Stage Object & How It Works

Lecture 7 What Is Snowflake File Format Objects & Query External Stage Using It

Lecture 8 What is Snowflake Sequence Object & How To Use Them

Lecture 9 What is Snowflake Stream Object & How Does It Track Changes

Lecture 10 What is Task and Task-Tree & How to Create a DAG Using Task Tree

Lecture 11 Quickly Understand The Common Table Expression

Lecture 12 Quickly Understand Window Function

Section 3: End To End Data Pipeline Requirement In Snowflake

Lecture 13 What Is End To End data flow

Lecture 14 Different Orchestration Options

Lecture 15 Challenges With Stream & Task As Orchestration Tool

Section 4: Build Data Pipeline (DAG) Using Stream/Task Object

Lecture 16 End To End Continuous Data Flow Objective

Lecture 17 Understand The Data (Customer & Order) Set

Lecture 18 Create Database & Schema Objects

Lecture 19 Data Loading Objective & Data Statistics

Lecture 20 Load Country Specific Customer/Order Data Set & Query Them

Lecture 21 RAW Schema Objective

Lecture 22 Continuous Data Ingestion To RAW Schema Layer & Task Tree

Lecture 23 CLEAN Schema Objective

Lecture 24 Continuous Data Ingestion To Clean Layer

Lecture 25 CONSUMPTION Schema Objective

Lecture 26 Continuous Data Ingestion To Consumption Layer

Lecture 27 Activate Pipeline Objective

Lecture 28 Resume Task & Monitor Continuous Customer/Order Data Flow

Lecture 29 Additional Continuous Data Load Objective

Lecture 30 Load Additional Customer/Data, Check Continuous E2E Flow & Monitor Progress

Lecture 31 Task Progress Tracking & Challenges

Lecture 32 Summary - Data Pipeline Using Stream/Task & Challenges

Section 5: All About Snowflake Dynamic Table & How To Build Pipeline Declaratively

Lecture 33 Introduction - What Is Covered In This Section

Lecture 34 Understand The Sample Data Set

Lecture 35 Load Data Set To Internal Stage Location

Lecture 36 Auto Ingest Using Task & Copy Command

Lecture 37 Create Our First Dynamic Table

Lecture 38 Update/Delete Operation On Underlying Table & Impact on Dynamic Table

Lecture 39 Dynamic Table On Stage Location & On Materialized View

Lecture 40 Alter Dynamic Table Paramters

Section 6: Dynamic Table On Multiple Tables

Lecture 41 Understand The Sample Data

Lecture 42 Ingest Data Using Task & Copy Command

Lecture 43 Create Dynamic Table On Multiple Tables

Lecture 44 Downstream Lag

Section 7: End To End Data Flow Using Dynamic Table (Customer/Order Example)

Lecture 45 Create Database & Schema For Dynamic Table E2E Flow

Lecture 46 Load Data Into Stage Location & Auto Ingest Using Task/Copy Objects

Lecture 47 Create Dynamic Tables For Clean Layer

Lecture 48 Create Dynamic Table For Consumption Layer

Lecture 49 Load Additional Data To Continuous Data Flow

Lecture 50 Increment Load Vs. Full Load Scenario

Section 8: Important Dynamic Table Features

Lecture 51 Introduction - Important Dynamic Table Features

Lecture 52 Dynamic Table Activity & Summary Dashboard (Snowsight WebUI)

Lecture 53 Dynamic Table Cost Calculation

Lecture 54 Dynamic Table & Cloning Support

Lecture 55 Dynamic Table & Time Travel Support

Lecture 56 Dynamic Table & Support For Stream Objects For Change Data Capture

Lecture 57 Dynamic Tables & Sharing Capabilities

Lecture 58 Dynamic Table & Table Functions For Run History

Section 9: Quick Dynamic Table Summary

Lecture 59 Dynamic Table & Course Summary

Snowflake Cloud Data Developer,Snowflake Cloud Data Engineer,Snowflake Cloud Data Architect,Snowflake Data Engineer