Information Theory Fundamentals

Pervasiveness of Information, Probability Axioms, Spaces and Fields

What you'll learn

Understand what is the prevalent definition of probability

Calculate simple relative frequency based probabilities

Identify domains where information theory is applicable

Be able to find the features of random variables such as their density

Requirements

Mathematical maturity

Description

Information theory is the brain child of American applied mathematician Claude Elwood Shannon who was a professor at the Massachusetts Institute of Technology. Embark on a journey into the world of information theory with "Information Theory Fundamentals." This course delves into the pervasive nature of information in our digital age, exploring its foundational concepts and significance. Begin your journey by understanding the pervasiveness of information in natural and man-made phenomena and systems and its impact on communication systems and technology. Dive into the core principles of probability, starting with the axioms that form the bedrock of probabilistic reasoning, founded in part by the Russian scientist A. N. Kolmogorov. Gain a comprehensive understanding of probability spaces and Borel fields, essential for analyzing random variables and processes. This course is designed for students and professionals seeking to grasp the fundamental concepts of information theory, providing a robust framework to tackle advanced topics in entropy, mutual information, asymptotic equipartition theorem, data compression, channel coding, and transmission schemes. Through clear explanations and practical examples, you'll build a solid foundation in information theory, preparing you for further studies and applications in various fields, including communications, computer science, and data science. Join us to unlock the mysteries of information theory and its applications.

Overview

Section 1: Introduction

Lecture 1 Pervasiveness of Information

Section 2: Probability Fundamentals

Lecture 2 Meaning of Probability

Lecture 3 Probability Spaces

Lecture 4 Probability Axioms

Lecture 5 Fields

Those who want a modern joint view of probability and information.