Linux kernel programming : a comprehensive guide to kernel internals, writing kernel modules, and kernel synchronization / Kaiwan N Billimoria.

Format:

Book

Author/Creator:

Billimoria, Kaiwan N., author.

Language:

English

Subjects (All):

Linux.

Operating systems (Computers).

Physical Description:

1 online resource (xi, 734 pages) : illustrations

Place of Publication:

Birmingham ; Mumbai : Packt Publishing, 2021.

Biography/History:

Billimoria Kaiwan N. : Kaiwan N. Billimoria taught himself BASIC programming on his dad's IBM PC back in 1983. He was programming in C and Assembly on DOS until he discovered the joys of Unix, and by around 1997, Linux! Kaiwan has worked on many aspects of the Linux system programming stack, including Bash scripting, system programming in C, kernel internals, device drivers, and embedded Linux work. He has actively worked on several commercial/FOSS projects. His contributions include drivers to the mainline Linux OS and many smaller projects hosted on GitHub. His Linux passion feeds well into his passion for teaching these topics to engineers, which he has done for well over two decades now. He's also the author of Hands-On System Programming with Linux, Linux Kernel Programming (and its Part 2 book) and Linux Kernel Debugging. It doesn't hurt that he is a recreational ultrarunner too.

Summary:

Learn how to write high-quality kernel module code, solve common Linux kernel programming issues, and understand the fundamentals of Linux kernel internalsKey FeaturesDiscover how to write kernel code using the Loadable Kernel Module frameworkExplore industry-grade techniques to perform efficient memory allocation and data synchronization within the kernelUnderstand the essentials of key internals topics such as kernel architecture, memory management, CPU scheduling, and kernel synchronizationBook DescriptionLinux Kernel Programming is a comprehensive introduction for those new to Linux kernel and module development. This easy-to-follow guide will have you up and running with writing kernel code in next-to-no time. This book uses the latest 5.4 Long-Term Support (LTS) Linux kernel, which will be maintained from November 2019 through to December 2025. By working with the 5.4 LTS kernel throughout the book, you can be confident that your knowledge will continue to be valid for years to come. You’ll start the journey by learning how to build the kernel from the source. Next, you’ll write your first kernel module using the powerful Loadable Kernel Module (LKM) framework. The following chapters will cover key kernel internals topics including Linux kernel architecture, memory management, and CPU scheduling. During the course of this book, you’ll delve into the fairly complex topic of concurrency within the kernel, understand the issues it can cause, and learn how they can be addressed with various locking technologies (mutexes, spinlocks, atomic, and refcount operators). You’ll also benefit from more advanced material on cache effects, a primer on lock-free techniques within the kernel, deadlock avoidance (with lockdep), and kernel lock debugging techniques. By the end of this kernel book, you’ll have a detailed understanding of the fundamentals of writing Linux kernel module code for real-world projects and products.What you will learnWrite high-quality modular kernel code (LKM framework) for 5.x kernelsConfigure and build a kernel from sourceExplore the Linux kernel architectureGet to grips with key internals regarding memory management within the kernelUnderstand and work with various dynamic kernel memory alloc/dealloc APIsDiscover key internals aspects regarding CPU scheduling within the kernelGain an understanding of kernel concurrency issuesFind out how to work with key kernel synchronization primitivesWho this book is forThis book is for Linux programmers beginning to find their way with Linux kernel development. If you’re a Linux kernel and driver developer looking to overcome frequent and common kernel development issues, or understand kernel intervals, you’ll find plenty of useful information. You’ll need a solid foundation of Linux CLI and C programming before you can jump in.

Contents:

Cover

Title Page

Copyright and Credits

Dedication

Contributors

Table of Contents

Preface

Section 1: The Basics

Chapter 1: Kernel Workspace Setup

Technical requirements

Running Linux as a guest VM

Installing a 64-bit Linux guest

Turn on your x86 system's virtualization extension support

Allocate sufficient space to the disk

Install the Oracle VirtualBox Guest Additions

Experimenting with the Raspberry Pi

Setting up the software - distribution and packages

Installing software packages

Installing the Oracle VirtualBox guest additions

Installing required software packages

Installing a cross toolchain and QEMU

Installing a cross compiler

Important installation notes

Additional useful projects

Using the Linux man pages

The tldr variant

Locating and using the Linux kernel documentation

Generating the kernel documentation from source

Static analysis tools for the Linux kernel

Linux Trace Toolkit next generation

The procmap utility

Simple Embedded ARM Linux System FOSS project

Modern tracing and performance analysis with [e]BPF

The LDV - Linux Driver Verification - project

Summary

Questions

Further reading

Chapter 2: Building the 5.x Linux Kernel from Source - Part 1

Preliminaries for the kernel build

Kernel release nomenclature

Kernel development workflow - the basics

Types of kernel source trees

Steps to build the kernel from source

Step 1 - obtaining a Linux kernel source tree

Downloading a specific kernel tree

Cloning a Git tree

Step 2 - extracting the kernel source tree

A brief tour of the kernel source tree

Step 3 - configuring the Linux kernel

Understanding the kbuild build system

Arriving at a default configuration

Obtaining a good starting point for kernel configuration.

Kernel config for typical embedded Linux systems

Kernel config using distribution config as a starting point

Tuned kernel config via the localmodconfig approach

Getting started with the localmodconfig approach

Tuning our kernel configuration via the make menuconfig UI

Sample usage of the make menuconfig UI

More on kbuild

Looking up the differences in configuration

Customizing the kernel menu - adding our own menu item

The Kconfig* files

Creating a new menu item in the Kconfig file

A few details on the Kconfig language

Chapter 3: Building the 5.x Linux Kernel from Source - Part 2

Step 4 - building the kernel image and modules

Step 5 - installing the kernel modules

Locating the kernel modules within the kernel source

Getting the kernel modules installed

Step 6 - generating the initramfs image and bootloader setup

Generating the initramfs image on Fedora 30 and above

Generating the initramfs image - under the hood

Understanding the initramfs framework

Why the initramfs framework?

Understanding the basics of the boot process on the x86

More on the initramfs framework

Step 7 - customizing the GRUB bootloader

Customizing GRUB - the basics

Selecting the default kernel to boot into

Booting our VM via the GNU GRUB bootloader

Experimenting with the GRUB prompt

Verifying our new kernel's configuration

Kernel build for the Raspberry Pi

Step 1 - cloning the kernel source tree

Step 2 - installing a cross-toolchain

First method - package install via apt

Second method - installation via the source repo

Step 3 - configuring and building the kernel

Miscellaneous tips on the kernel build

Minimum version requirements

Building a kernel for another site

Watching the kernel build run.

A shortcut shell syntax to the build procedure

Dealing with compiler switch issues

Dealing with missing OpenSSL development headers

Chapter 4: Writing Your First Kernel Module - LKMs Part 1

Understanding kernel architecture - part 1

User space and kernel space

Library and system call APIs

Kernel space components

Exploring LKMs

The LKM framework

Kernel modules within the kernel source tree

Writing our very first kernel module

Introducing our Hello, world LKM C code

Breaking it down

Kernel headers

Module macros

Entry and exit points

Return values

The 0/-E return convention

The ERR_PTR and PTR_ERR macros

The __init and __exit keywords

Common operations on kernel modules

Building the kernel module

Running the kernel module

A quick first look at the kernel printk()

Listing the live kernel modules

Unloading the module from kernel memory

Our lkm convenience script

Understanding kernel logging and printk

Using the kernel memory ring buffer

Kernel logging and systemd's journalctl

Using printk log levels

The pr_&lt

foo&gt

convenience macros

Wiring to the console

Writing output to the Raspberry Pi console

Enabling the pr_debug() kernel messages

Rate limiting the printk instances

Generating kernel messages from the user space

Standardizing printk output via the pr_fmt macro

Portability and the printk format specifiers

Understanding the basics of a kernel module Makefile

Chapter 5: Writing Your First Kernel Module - LKMs Part 2

A "better" Makefile template for your kernel modules

Configuring a "debug" kernel

Cross-compiling a kernel module

Setting up the system for cross-compilation.

Attempt 1 - setting the "special" environment variables

Attempt 2 - pointing the Makefile to the correct kernel source tree for the target

Attempt 3 - cross-compiling our kernel module

Attempt 4 - cross-compiling our kernel module

Gathering minimal system information

Being a bit more security-aware

Licensing kernel modules

Emulating "library-like" features for kernel modules

Performing library emulation via multiple source files

Understanding function and variable scope in a kernel module

Understanding module stacking

Trying out module stacking

Passing parameters to a kernel module

Declaring and using module parameters

Getting/setting module parameters after insertion

Module parameter data types and validation

Validating kernel module parameters

Overriding the module parameter's name

Hardware-related kernel parameters

Floating point not allowed in the kernel

Auto-loading modules on system boot

Module auto-loading - additional details

Kernel modules and security - an overview

Proc filesystem tunables affecting the system log

The cryptographic signing of kernel modules

Disabling kernel modules altogether

Coding style guidelines for kernel developers

Contributing to the mainline kernel

Getting started with contributing to the kernel

Section 2: Understanding and Working with the Kernel

Chapter 6: Kernel Internals Essentials - Processes and Threads

Understanding process and interrupt contexts

Understanding the basics of the process VAS

Organizing processes, threads, and their stacks - user and kernel space

User space organization

Kernel space organization

Summarizing the current situation

Viewing the user and kernel stacks

Traditional approach to viewing the stacks.

Viewing the kernel space stack of a given thread or process

Viewing the user space stack of a given thread or process

[e]BPF - the modern approach to viewing both stacks

The 10,000-foot view of the process VAS

Understanding and accessing the kernel task structure

Looking into the task structure

Accessing the task structure with current

Determining the context

Working with the task structure via current

Built-in kernel helper methods and optimizations

Trying out the kernel module to print process context info

Seeing that the Linux OS is monolithic

Coding for security with printk

Iterating over the kernel's task lists

Iterating over the task list I - displaying all processes

Iterating over the task list II - displaying all threads

Differentiating between the process and thread - the TGID and the PID

Iterating over the task list III - the code

Chapter 7: Memory Management Internals - Essentials

Understanding the VM split

Looking under the hood - the Hello, world C program

Going beyond the printf() API

VM split on 64-bit Linux systems

Virtual addressing and address translation

The process VAS - the full view

Examining the process VAS

Examining the user VAS in detail

Directly viewing the process memory map using procfs

Interpreting the /proc/PID/maps output

The vsyscall page

Frontends to view the process memory map

The procmap process VAS visualization utility

Understanding VMA basics

Examining the kernel segment

High memory on 32-bit systems

Writing a kernel module to show information about the kernel segment

Viewing the kernel segment on a Raspberry Pi via dmesg

Macros and variables describing the kernel segment layout

Trying it out - viewing kernel segment details.

The kernel VAS via procmap.

Notes:

Includes index.

Description based on print version record.

ISBN:

9781789955927

1789955920

OCLC:

1240585605