Linux System Programming Techniques & Concepts

For Developers - Build Linux Programmable Libraries, Makefiles, Memory Management, Compilation & Linking, C Programming

What you'll learn
Learn Advance Programming Concepts in C/C++
Develop and Integrate C/C++ Libraries
Automate Build Process using Makefile
Use of Function Pointers as Callbacks
Static and Dynamic Libraries and Linking
Compilation Process of a GCC Compiler
Organize the code into Header and Source files
Understanding Memory allocations, Heap Memory Management, Memory Leaks
Stack Memory, Stack overflow, Stack Corruption, Stack Registers
Paging Concepts and Page Tables
Concept of TLVs

Basic knowledge of Programming in C/C++ is required
Zeal and enthusiasm to learn and sharpen the skills
Good to have basic OS knowledge but not mandatory
Description
There are already innumerable courses/tutorials on the internet/Udemy which teach almost every aspect of C programming language. All those courses sound the same, talk the same and deliver more or less the same knowledge. How is this course different from others?

The intention of this course is to make you ready for System programming Technical interviews, interview level - from Beginner to Intermediate. This course is for (future) Developers, not for testers or System Administrators.

I choose to create this course to fill the gap between novice/beginner and intermediate/Advanced Programmers. This course assumes that you are at least above average in programming (in any programming language, but preferable in C/C++) - know memory allocations, all kinds of loops, function calls, pointers etc. In this course, I won't teach C/C++ (there are already many courses on it online) but would teach programming techniques and low-level details regarding how C program works behind the scenes - All topics are very important from an interview point of view. My Target is to grow this course into Linux/C Bible.

Yet, I always feel, you need to outsmart your fellow colleagues in this era of stiff competition, and therefore, I tried this attempt to present you the wisdom and knowledge which is of utmost importance for a programmer. I have seen though students could write good C/C++ programs, yet they lack the clarity on how one should write better organized, Manageable, extensible, and programmable codes in the form of libraries. For example, Students are too good at competitive programming, yet do not know how to write a simple Makefile, the reason being, Academics do not teach and students do not care to learn.

In this course, I will cover the topics related to creating Linux System Libraries (release 1) with Advanced language-agnostic Programming Concepts (release 2) which can be well applied if you happen to become a programmer in other languages tomorrow.

The Concepts like MultiThreading, Thread-Synchronization, Socket Programming, IPCs, etc demand a separate course on each topic altogether. Currently, I have a course on IPC, pls check it out. These topics are out of scope for this course.

The Course shall be delivered into two Releases :

Release 1 Building and Managing a Library: This covers the basics regarding how one should create and organize his code as a Library.

Release 2 Memory Management Concepts: This covers Advance concepts on Linux Memory Management specifically. Please check the Table of Contents for more info.

Why you should *NOT* do this course?

1. Please Excuse this course if you are the ultimate beginner in C programming !!

2. There is no point of doing this course if you don't like hitting the keyboard, and are lazy enough to watch lecture VDOs only

3. If you want everything cooked and served on your plate.

Course Pre-requisite :

Above Beginners Level in C/C++ programming

Good to have basic OS knowledge

Zeal to excel and Code

No Third-Party libraries

Whatever logic you implement, you need to implement it from scratch. Like all my other courses, this course does not suggest taking the help of any third-party library to get the jobs done. The use of external libraries completely defeats the purpose of the course.

Warning: This course has auto system-generated subtitles which may not be perfect. Please disable subtitles at your convenience.

Curriculum :

**********************************************

Release 1 Building and Managing a Library

**********************************************

Section 1 : Introduction to Libraries

What is Library

Relationship between Library and Application

Ex : Doubly linked list as a Library

Quick Compilation Steps

Summary

Section 2 : Header Files

Relationship between Source and header files

Text Substitution Method

Text Substitution Method - Example and Demo

Section 3 : Preprocessing Directives

Problem of Duplicate inclusion of Hdr files

Pre-processing Directives

Solution to Duplicate inclusion of Hdr files

Section 4 : Correct way of Using Structures and Functions

Structures - Define and Use thumb Rule

Functions - Declare and Use thumb rule

The problem of Recursive Dependency

A solution to Recursive Dependency

Section 5 : Creation of Static and Dynamic Libraries

Resuming with Doubly Linked List Library

Quick Creation of Static and Dynamic Libraries

Linking with Static Library

Linking with Dynamic Library

Section 7: Understanding four stages of Compilation

Four stages of C/C++ Compilation

How Dynamic Library works ?

Dynamic Linking : Linking with Dynamic Library

Comparison - Static Vs Dynamic Linking

Section 8 : Building using a Makefile

What are Makefiles and why do we need it

Functions of Makefile

Makefile Dependency tree

Steps of Writing a Makefile

Assignment on Makefile

Section 9 : Run-time Programmable libraries

What are Programmable Libraries?

Steps to Program the libraries

Registering of the callbacks with Libraries

key_match callback

comparison_fn callback

Delegation of Application-specific operations to Libraries

Section 10 : Writing Iterators using Macros

What are Iterative Macros ?

Why we need Iterative Macros ?

How to Write Iterative Macros - For Trees and Linked Lists

Exercises

Section 11 : Glue Based Libraries and Data structures

What are the Glue Based Libraries?

Introducing Glthreads - A Glued LinkedList

Glthreads Vs Traditional Linked List

Structure field offset

GLThread Operations

Code Walk

GLThread Benefits

Section 12 : Bit Level Programming

Logical Operators

Implementing BIT manipulating C macros

Using Enums as Bits

Bit Pattern Matching

BitMaps

************************************************

Release 2 Memory Management Concepts

************************************************

Section 13 : Memory Layout of Linux Process

Virtual Memory Basics

Memory Layout of Linux Process

Example: Memory Layout of Linux Process

Exercise on size command

Section 14 : Stack Memory Management

Stack Memory Basics and Contents

Stack-Overflow and Prevention

Stack Memory Corruption

Common Cpu Registers

Procedure Call Mechanism - Step by Step

Purpose of Base Pointer register (ebp)

Procedure Return Mechanism - Step by Step

Lab session

Section 15 : Heap Memory Management

Introduction and Goals

How Malloc Works

Top of Heap Memory region - break pointer

Heap Memory Mgmt Sys Calls - brk and sbrk

Meta and Data Blocks

How free() works

Block Splitting

Block Merging

Memory Illness - Problem of Fragmentation

Section 16 : Concept of Paging

Introduction to Paging

Byte Addressable Memory

32 bit and 64 bit Machine Architecture

Address Bus and Data bus

Physical Vs Virtual Address

Physical Memory Frames

Virtual Address Composition

Page Table

Paging In Action

Shared Physical Memory

Section 17 : Multilevel Paging

Section 18 : Demand Paging

Section 19 : Memory Management for Multi-threaded Process

The intention of this course is to make you ready for System programming Technical interviews from beginners to upto 8-9 yrs of experience.

Q. What are the frequently asked questions by interviewers in a technical round when someone writes C/C++/System Programming language on their resume?

Answer : If i am interviewer, what questions i would ask depends on his no of years of experience in C.

1–3 yrs of experience — I would have asked:

Double pointers

design a Macro to return the size of the structure

Two Dimensional Arrays, passing and returning arrays from a fn

Different stages of C program compilation

how fork() works

What are various ways to debug memory corruptions.

various IPCs

Heap and Stack memory-based Question

4–6 yrs of experience - I would have asked:

How memory is allocated by the OS

Internal and external fragmentation, what can be done to avoid it

System calls, strace()

Trade-of of one IPC over other

various ways to communicate with kernel and comparison

Data (De)Serialization in C

RPC in C

callbacks advanced application

typedef Vs #define

Generic programming in C using macros

Thread Synchronization

Heap and Stack memory-based Question

7+ yrs of experience - I would have asked

Have you designed any system module to solve any problem

Design thread library 0 what functionalities would you incorporate in and how?

What are Dos and Dont’s for writing a robust and flexible library

How to write generic code in C

Various ways to implement timers in C, and comparison of approaches

How do Interrupts work ?

IPCs and comparison

How would you convert a C code to C++ and vice versa

How to write a tool to detect memory leaks Or garbage collection

Design your own memory allocation tool. Why would you write your own memory allocation scheme?

When to go for Multi-process design over Multi-threaded design and vice versa

How ValGrind tool works

In production code, would you favor recursive but simple logic, Or Nonrecursive but complex logic, and why?

If you analyze the pattern,

Candidate with 1–3 yrs of experience, I would choose to ask more of a direct and straightforward Questions.

Candidate with 4–6 yrs of experience, I would choose to ask more advanced technical C Question plus some comparison of approaches based Questions

Candidate with 7+ yrs of experience, I would choose to ask more of a design and Analysis based Question.

Note that, since the Question pertains to C and System Programming, so I have not mentioned Data structures and Algorithms.

Who this course is for:
Students who have basic knowledge of C/C++ programming
Warning : Not for Absolute beginners in Programming
Who want to take their programming Skills from Beginner to Professional Level
Final year students looking to crack the Interview Questions on OS, C programming