System Design - Data Modeling For Olap Databases

Data modeling in a data warehouse

What you'll learn
Understand the internal architecture of an OLAP store
Designing the data model to support rich analytical queries
Understand distribution and sort keys and how to leverage them
Designing your aggregation queries in a column-oriented OLAP store
Optimize Redshift Query Performance
Requirements
You understand the basics of SQL and now want to extend your skills to designing the data model for data warehouses
Description
Designing a data model for a data warehouse is fundamentally different from designing the schema for your primary database. A data warehouse is meant for supporting aggregation queries that touch a huge number of records and are very expensive. The scale in terms of number of queries is at least an order of magnitude less than the number of queries seen by a consumer-facing primary database. This distinction leads to a fundamental different data layout and indexing model in the database used in a data warehouse. Such databases are called OLAP databases.This course will take you into the internal architecture of an OLAP database with specific focus on areas that help you how a query runs on an OLAP database. We will walk though the levers that an OLAP database provides to optimize your queries. We will then cover various scenarios that will help you design the data model for an OLAP database used in a data warehouse.Throughout this course, we will use AWS Redshift as the OLAP database. However, the principles covered in this course are general and can be applied in any data warehouse including Snowflake or  Hive. Some of these principles are also applicable in real-time column stores like HBase.

Overview

Section 1: Introduction

Lecture 1 Introduction

Lecture 2 OLTP Versus OLAP - part I

Lecture 3 OLTP versus OLAP - part II

Section 2: OLAP Architecture

Lecture 4 Data Layout and Distribution

Lecture 5 Query execution - I

Lecture 6 Query execution - II

Section 3: Data Modeling

Lecture 7 Avoiding Full Scans (Sort Keys) - 1

Lecture 8 Avoiding Full Scans (Sort Keys) - 2

Lecture 9 Distribution Keys

Lecture 10 Handling Joins - Part I

Lecture 11 Handling Joins - Part 2

Section 4: Beyond data modeling

Lecture 12 Beyond schema design

Data engineer and/or data architects