Fivertran Etl From Scratch

Learn Fivetran from basic to advanced | PostgreSql | Redshift | AWS S3 buckets | Sql Transformations

What you'll learn

ETL using Fivetran

Data Integration

Data Sync

Data Migration

Requirements

No programming experience is needed

Description

The building blocks of data organization are tables and schemas. You can think of a table as a file organized by rows and columns and of a schema as a folder that contains multiple tables. Each Fivetran connector creates and manages its own schema. Fivetran connector reaches out to your source, receives data from it, and writes it to your destination. Depending on the type of connector, Fivetran either collects data that the source pushes to us or sends a request to the source and then grabs the data that the source sends in response.Fivetran’s responsibilityIt is Fivetran’s responsibility to deliver up-to-date, accurate information in a cleaned and normalized schema - the canonical schema - at the lowest level of aggregation. It is our responsibility to regularly maintain the connector and evolve the canonical schema to reflect operational and product changes in the source systems. It is our responsibility to respond to any unknown operational breaking change in the extract and load from the source system to the destination schema.Fivetran connects to all of your supported data sources and loads the data from them into your destination. Each data source has one or more connectors that run as independent processes that persist for the duration of one update. A single Fivetran account, made up of multiple connectors, loads data from multiple data sources into one or more destinations.

Overview

Section 1: Introduction

Lecture 1 Introduction

Section 2: Designing Connector(PostgreSql) and Destination(Redshift)

Lecture 2 PostgreSql Scenario

Lecture 3 Connector

Lecture 4 Destination

Section 3: ETL - Sync up from PostgreSql to Redshift

Lecture 5 Sync Up

Lecture 6 Redshift Synced Objects

Lecture 7 Resync

Lecture 8 Metadata of Fivetran

Lecture 9 Connector Properties

Section 4: AWS S3 Buckets and Fivetran

Lecture 10 S3 Connector Configuration

Lecture 11 S3 to Redshift

Section 5: Working with the data

Lecture 12 Insert a record

Lecture 13 Delete a record in Source

Section 6: Transformations

Lecture 14 Transformations Overview

Lecture 15 Data Transformation using Sql

Lecture 16 Views using Transformation

ETL Architects,ETL Developers,Data Integration Developers,Data Migration Specialists,Data Architects,Data Engineers