Kusto Query Language (Kql) - Part 1

A Deep Dive Into KQL Along With a Review of Azure Data Explorer (ADX)

What you'll learn
An overview of Azure Data Explorer (ADX)
Azure Data Explorer Web UI and Log Analytics Demo Site
A deep dive into the essentials of KQL
The most commonly used KQL operators and functions
Aggregating data with KQL
Exporting data to Excel and Power BI
Requirements
No knowledge of Azure required. Some knowledge of SQL would be helpful.
A Microsoft account will be required to use the Log Analytics demo site. There is no cost involved.
Description
There is a good chance you have already used Azure Data Explorer (ADX) to some degree without knowing it. If you have used Azure Security Center, Azure Sentinel, Application Insights, Resource Graph Explorer, or enabled diagnostics on your Azure resources, then you have used ADX. All these services rely on Log Analytics, which is built on top of ADX and is queried using KQL.Like many other tools and products, ADX was started by a small group of engineers in Israel around 2015. They needed to solve a problem. A group of developers from Microsoft's Power BI team needed a high-performing big data solution to ingest and analyze their logging and telemetry data. So, of course, they built their own because they could not find a service that met all their needs. This resulted in the Azure Data Explorer, also known as Kusto.So, what is ADX? It is a fully managed, append-only columnar store big data service capable of elastic scaling and ingesting literally hundreds of billions of rows daily. ADX offers:Low-latency ingestion and elastic scalingSecurityCost-efficient (pay as you consume)High availabilityTime Series AnalysisSuper fast query performance via KQLCustom built solutionsAs great as ADX is, this course is mostly centered around KQL (Kusto Query Language). KQL is the query language for managing all logging and telemetry data stored in ADX. Even if you do not use ADX directly, you will still use KQL for monitoring, analyzing logs, managing assets, exploring security data, and exploring Application Insights data. KQL is ADX's read-only query language that has many similarities with SQL, such as working with tables, columns, and providing functionality for filtering. KQL supports a subset of SQL, and SQL statements can be executed and converted to KQL using the EXPLAIN keyword, reducing the learning curve for engineers with an SQL background.This is part 1 of a two part series covering ADX (lightly) and the KQL language (mostly). The goal of this course is to cover the basics. At the end of this 5 hour course you will have a solid understanding of what KQL can do. And it can do a lot! In some respects I like it better than T-SQL which I have used for over 20 years.Part 2 of this course goes well beyond the basics and will cover many advanced KQL topics and scenarios (and some more ADX).

Overview

Section 1: Introduction

Lecture 1 Introduction

Section 2: Azure Data Explorer / ADX / Kusto

Lecture 2 Overview

Lecture 3 Creating an Azure Data Explorer Cluster

Lecture 4 Azure Data Explorer Web UI

Section 3: Kusto Query Language

Lecture 5 Contoso Dataset

Lecture 6 What is Kusto Query Language (KQL)?

Lecture 7 Would You Rather Use T-SQL?

Section 4: The Most Common Operators and Functions You Will Use

Lecture 8 Getting Started

Lecture 9 Project / Extend / Take

Lecture 10 Where / Ago

Lecture 11 Search

Lecture 12 Distinct

Lecture 13 Summarize / Bin

Lecture 14 Parse

Lecture 15 Order By

Lecture 16 Datetime / Timespans

Lecture 17 Datetime_Part / Datetime_Diff / Datetime_Add

Lecture 18 Format_Datetime / Format_Timespan

Lecture 19 StartOf / EndOf / Between

Lecture 20 IIF / Case / Split

Lecture 21 String Operators

Lecture 22 Strcat

Lecture 23 ToDynamic / Parse_Json

Lecture 24 Getschema

Section 5: Aggregating Data - Most Common Functions

Lecture 25 Count and DCount

Lecture 26 Arg_max and Arg_min

Lecture 27 Make_set / Make_list / Mv-expand

Lecture 28 Percentiles

Lecture 29 Pivot

Lecture 30 Top-Nested

Lecture 31 Any / Take_any

Lecture 32 Wrap Up

Section 6: Miscellaneous Statements, Operators and Functions

Lecture 33 Let

Lecture 34 Join

Lecture 35 Union

Lecture 36 Datatable

Lecture 37 Prev and Next

Lecture 38 Top-hitters

Lecture 39 Sample

Lecture 40 Render

Section 7: Exporting Data

Lecture 41 Exporting to Excel / CSV and Power BI

Section 8: Test Your Knowledge

Lecture 42 Test Your Knowledge

Anyone needing to analyze data from Azure Security Center, Azure Sentinel, Application Insights, Resource Graph Explorer, or enabled diagnostics on your Azure resources,Anyone wanting to learn the amazing Kusto Query Language