Information Theory Fundamentals - Part I

Set theory and the axioms of probability; the pervasiveness of information.

What you'll learn

The meaning of probability

The axiomatic approach to probability

The classical approach to probability

Basics of Set theory

The pervasiveness of information

Information theory and the basic sciences intersected

Requirements

Exposure to probability

Description

In this course, students will see that information is found in the stock market, in Brownian motion of pollen, in neural firing patterns, in the quantum realm of particle creation and annihilation, and also in intersection with many of the basic sciences. Information intersects with quantum physics, to yield quantum information theory, with neuroscience to yield computational neuroscience, and with economics to yield investment theory. This broad overview of information is accompanied by a discussion of the philosophical issues that information touches upon. We present an outline for a complete course on information theory of which the present course is the zeroth offering. In the lecture on probability, we start with a brief historical introduction to probability. We go on to define the meaning of probability via probability axioms, we introduce the classical definition, and the relative frequency definition of probability. We provide examples to illustrate these definitions. These are simple examples not requiring a calculator. Finally, we introduce the student to set theory fundamentals, explaining the notion of belonging, exclusion, disjointedness, intersection, union, complements. We also introduce partitions, De Morgan's laws and the properties of unions and intersections. All of the above makes this a strong foundation on which other course work can build.

Overview

Section 1: Introduction to Information

Lecture 1 Introduction: Pervasiveness of Information

Section 2: Introduction to Probability

Lecture 2 Meaning of Probability

Lecture 3 Probability Space and Terminology

Lecture 4 Axioms of Probability

Lecture 5 Fields and Borel Fields

Advanced learners having mathematical maturity, ready to challenge themselves to learn this very practical theory.