The Linux Programming Interface Table of Contents Praise for The Linux Programming Interface Preface 1. History and Standards A Brief History of UNIX and C A Brief History of Linux The GNU Project The Linux Kernel Standardization The C Programming Language The First POSIX Standards X/Open Company and The Open Group SUSv3 and POSIX.1-2001 SUSv4 and POSIX.1-2008 UNIX Standards Timeline Implementation Standards Linux, Standards, and the Linux Standard Base Summary 2. Fundamental Concepts The Core Operating System: The Kernel The Shell Users and Groups Single Directory Hierarchy, Directories, Links, and Files File I/O Model Programs Processes Memory Mappings Static and Shared Libraries Interprocess Communication and Synchronization Signals Threads Process Groups and Shell Job Control Sessions, Controlling Terminals, and Controlling Processes Pseudoterminals Date and Time Client-Server Architecture Realtime The /proc File System Summary 3. System Programming Concepts System Calls Library Functions The Standard C Library; The GNU C Library (glibc) Handling Errors from System Calls and Library Functions Notes on the Example Programs in This Book Command-Line Options and Arguments Common Functions and Header Files Portability Issues Feature Test Macros System Data Types Miscellaneous Portability Issues Summary Exercise 4. File I/O: The Universal I/O Model Overview Universality of I/O Opening a File: open() The open() flags Argument Errors from open() The creat() System Call Reading from a File: read() Writing to a File: write() Closing a File: close() Changing the File Offset: lseek() Operations Outside the Universal I/O Model: ioctl() Summary Exercises 5. File I/O: Further Details Atomicity and Race Conditions File Control Operations: fcntl() Open File Status Flags Relationship Between File Descriptors and Open Files Duplicating File Descriptors File I/O at a Specified Offset: pread() and pwrite() Scatter-Gather I/O: readv() and writev() Truncating a File: truncate() and ftruncate() Nonblocking I/O I/O on Large Files The devfd Directory Creating Temporary Files Summary Exercises 6. Processes Processes and Programs Process ID and Parent Process ID Memory Layout of a Process Virtual Memory Management The Stack and Stack Frames Command-Line Arguments (argc, argv) Environment List Accessing the environment from a program Performing a Nonlocal Goto: setjmp() and long jmp() Summary Exercises 7. Memory Allocation Allocating Memory on the Heap Adjusting the Program Break: brk() and sbrk() Allocating Memory on the Heap: malloc() and free() Implementation of malloc() and free() Other Methods of Allocating Memory on the Heap Allocating Memory on the Stack: alloca() Summary Exercises 8. Users and Groups The Password File: etcpasswd The Shadow Password File: etcshadow The Group File: etcgroup Retrieving User and Group Information Password Encryption and User Authentication Summary Exercises 9. Process Credentials Real User ID and Real Group ID Effective User ID and Effective Group ID Set-User-ID and SetGroup-ID Programs Saved Set-User-ID and Saved SetGroup-ID FileSystem User ID and FileSystem Group ID Supplementary Group IDs Retrieving and Modifying Process Credentials Retrieving and Modifying Real, Effective, and Saved Set IDs Retrieving and Modifying FileSystem IDs Retrieving and Modifying Supplementary Group IDs Summary of Calls for Modifying Process Credentials Example: Displaying Process Credentials Summary Exercises 10. Time Calendar Time Time-Conversion Functions Converting time_t to Printable Form Converting Between time_t and Broken-Down Time Converting Between Broken-Down Time and Printable Form Timezones Locales Updating the System Clock The Software Clock (Jiffies) Process Time Summary Exercise 11. System Limits and Options System Limits Retrieving System Limits (and Options) at Run Time Retrieving File-Related Limits (and Options) at Run Time Indeterminate Limits System Options Summary Exercises 12. System and Process Information The /proc File System Obtaining Information About a Process: procPID System Information Under /proc Accessing /proc Files System Identification: uname() Summary Exercises 13. File I/O Buffering Kernel Buffering of File I/O: The Buffer Cache Buffering in the stdio Library Controlling Kernel Buffering of File I/O Summary of I/O Buffering Advising the Kernel About I/O Patterns Bypassing the Buffer Cache: Direct I/O Mixing Library Functions and System Calls for File I/O Summary Exercises 14. File Systems Device Special Files (Devices) Disks and Partitions File Systems I-nodes The Virtual File System (VFS) Journaling File Systems Single Directory Hierarchy and Mount Points Mounting and Unmounting File Systems Mounting a File System: mount() Unmounting a File System: umount() and umount2() Advanced Mount Features Mounting a File System at Multiple Mount Points Stacking Multiple Mounts on the Same Mount Point Mount Flags That Are Per-Mount Options Bind Mounts Recursive Bind Mounts A Virtual Memory File System: tmpfs Obtaining Information About a File System: statvfs() Summary Exercise 15. File Attributes Retrieving File Information: stat() File Timestamps Changing File Timestamps with utime() and utimes() Changing File Timestamps with utimensat() and futimens() File Ownership Ownership of New Files Changing File Ownership: chown(), fchown(), and lchown() File Permissions Permissions on Regular Files Permissions on Directories Permission-Checking Algorithm Checking File Accessibility: access() Set-User-ID, SetGroup-ID, and Sticky Bits The Process File Mode Creation Mask: umask() Changing File Permissions: chmod() and fchmod() I-node Flags (ext2 Extended File Attributes) Summary Exercises 16. Extended Attributes Overview Extended Attribute Implementation Details System Calls for Manipulating Extended Attributes Summary Exercise 17. Access Control Lists Overview ACL Permission-Checking Algorithm Long and Short Text Forms for ACLs The ACL_MASK Entry and the ACL Group Class The getfacl and setfacl Commands Default ACLs and File Creation ACL Implementation Limits The ACL API Summary Exercise 18. Directories and Links Directories and (Hard) Links Symbolic (Soft) Links Creating and Removing (Hard) Links: link() and unlink() Changing the Name of a File: rename() Working with Symbolic Links: symlink() and readlink() Creating and Removing Directories: mkdir() and rmdir() Removing a File or Directory: remove() Reading Directories: opendir() and readdir() File Tree Walking: nftw() The Current Working Directory of a Process Operating Relative to a Directory File Descriptor Changing the Root Directory of a Process: chroot() Resolving a Pathname: realpath() Parsing Pathname Strings: dirname() and basename() Summary Exercises 19. Monitoring File Events Overview The inotify API inotify Events Reading inotify Events Queue Limits and /proc Files An Older System for Monitoring File Events: dnotify Summary Exercise 20. Signals: Fundamental Concepts Concepts and Overview Signal Types and Default Actions Changing Signal Dispositions: signal() Introduction to Signal Handlers Sending Signals: kill() Checking for the Existence of a Process Other Ways of Sending Signals: raise() and killpg() Displaying Signal Descriptions Signal Sets The Signal Mask (Blocking Signal Delivery) Pending Signals Signals Are Not Queued Changing Signal Dispositions: sigaction() Waiting for a Signal: pause() Summary Exercises 21. Signals: Signal Handlers Designing Signal Handlers Signals Are Not Queued (Revisited) Reentrant and Async-Signal-Safe Functions Global Variables and the sig_atomic_t Data Type Other Methods of Terminating a Signal Handler Performing a Nonlocal Goto from a Signal Handler Terminating a Process Abnormally: abort() Handling a Signal on an Alternate Stack: sigaltstack() The SA_SIGINFO Flag Interruption and Restarting of System Calls Summary Exercise 22. Signals: Advanced Features Core Dump Files Special Cases for Delivery, Disposition, and Handling Interruptible and Uninterruptible Process Sleep States Hardware-Generated Signals Synchronous and Asynchronous Signal Generation Timing and Order of Signal Delivery Implementation and Portability of signal() Realtime Signals Sending Realtime Signals Handling Realtime Signals Waiting for a Signal Using a Mask: sigsuspend() Synchronously Waiting for a Signal Fetching Signals via a File Descriptor Interprocess Communication with Signals Earlier Signal APIs (System V and BSD) Summary Exercises 23. Timers and Sleeping Interval Timers Scheduling and Accuracy of Timers Setting Timeouts on Blocking Operations Suspending Execution for a Fixed Interval (Sleeping) Low-Resolution Sleeping: sleep() High-Resolution Sleeping: nanosleep() POSIX Clocks Retrieving the Value of a Clock: clock_gettime() Setting the Value of a Clock: clock_settime() Obtaining the Clock ID of a Specific Process or Thread Improved High-Resolution Sleeping: clock_nanosleep() POSIX Interval Timers Creating a Timer: timer_create() Arming and Disarming a Timer: timer_settime() Retrieving the Current Value of a Timer: timer_gettime() Deleting a Timer: timer_delete() Notification via a Signal Timer Overruns Notification via a Thread Timers That Notify via File Descriptors: The timerfd API Summary Exercises 24. Process Creation Overview of fork(), exit(), wait(), and execve() Creating a New Process: fork() File Sharing Between Parent and Child Memory Semantics of fork() The vfork() System Call Race Conditions After fork() Avoiding Race Conditions by Synchronizing with Signals Summary Exercises 25. Process Termination Terminating a Process: _exit() and exit() Details of Process Termination Exit Handlers Interactions Between fork(), stdio Buffers, and _exit() Summary Exercise 26. Monitoring Child Processes Waiting on a Child Process The wait() System Call The waitpid() System Call The Wait Status Value Process Termination from a Signal Handler The waitid() System Call The wait3() and wait4() System Calls Orphans and Zombies The SIGCHLD Signal Establishing a Handler for SIGCHLD Delivery of SIGCHLD for Stopped Children Ignoring Dead Child Processes Summary Exercises 27. Program Execution Executing a New Program: execve() The exec() Library Functions The PATH Environment Variable Specifying Program Arguments as a List Passing the Caller’s Environment to the New Program Executing a File Referred to by a Descriptor: fexecve() Interpreter Scripts File Descriptors and exec() Signals and exec() Executing a Shell Command: system() Implementing system() Summary Exercises 28. Process Creation and Program Execution in More Detail Process Accounting The clone() System Call The clone() flags Argument Extensions to waitpid() for Cloned Children Speed of Process Creation Effect of exec() and fork() on Process Attributes Summary Exercise 29. Threads: Introduction Overview Background Details of the Pthreads API Thread Creation Thread Termination Thread IDs Joining with a Terminated Thread Detaching a Thread Thread Attributes Threads Versus Processes Summary Exercises 30. Threads: Thread Synchronization Protecting Accesses to Shared Variables: Mutexes Statically Allocated Mutexes Locking and Unlocking a Mutex Performance of Mutexes Mutex Deadlocks Dynamically Initializing a Mutex Mutex Attributes Mutex Types Signaling Changes of State: Condition Variables Statically Allocated Condition Variables Signaling and Waiting on Condition Variables Testing a Condition Variable’s Predicate Example Program: Joining Any Terminated Thread Dynamically Allocated Condition Variables Summary Exercises 31. Threads: Thread Safety and PerThread Storage Thread Safety (and Reentrancy Revisited) One-Time Initialization Thread-Specific Data Thread-Specific Data from the Library Function’s Perspective Overview of the Thread-Specific Data API Details of the Thread-Specific Data API Employing the Thread-Specific Data API Thread-Specific Data Implementation Limits Thread-Local Storage Summary Exercises 32. Threads: Thread Cancellation Canceling a Thread Cancellation State and Type Cancellation Points Testing for Thread Cancellation Cleanup Handlers Asynchronous Cancelability Summary 33. Threads: Further Details Thread Stacks Threads and Signals How the UNIX Signal Model Maps to Threads Manipulating the Thread Signal Mask Sending a Signal to a Thread Dealing with Asynchronous Signals Sanely Threads and Process Control Thread Implementation Models Linux Implementations of POSIX Threads LinuxThreads NPTL Which Threading Implementation? Advanced Features of the Pthreads API Summary Exercises 34. Process Groups, Sessions, and Job Control Overview Process Groups Sessions Controlling Terminals and Controlling Processes Foreground and Background Process Groups The SIGHUP Signal Handling of SIGHUP by the Shell SIGHUP and Termination of the Controlling Process Job Control Using Job Control Within the Shell Implementing Job Control Handling Job-Control Signals Orphaned Process Groups (and SIGHUP Revisited) Summary Exercises 35. Process Priorities and Scheduling Process Priorities (Nice Values) Overview of Realtime Process Scheduling The SCHED_RR Policy The SCHED_FIFO Policy The SCHED_BATCH and SCHED_IDLE Policies Realtime Process Scheduling API Realtime Priority Ranges Modifying and Retrieving Policies and Priorities Relinquishing the CPU The SCHED_RR Time Slice CPU Affinity Summary Exercises 36. Process Resources Process Resource Usage Process Resource Limits Details of Specific Resource Limits Summary Exercises 37. Daemons Overview Creating a Daemon Guidelines for Writing Daemons Using SIGHUP to Reinitialize a Daemon Logging Messages and Errors Using syslog Summary Exercise 38. Writing Secure Privileged Programs Is a Set-User-ID or SetGroup-ID Program Required? Operate with Least Privilege Be Careful When Executing a Program Avoid Exposing Sensitive Information Confine the Process Beware of Signals and Race Conditions Pitfalls When Performing File Operations and File I/O Don't Trust Inputs or the Environment Beware of Buffer Overruns Beware of Denial-of-Service Attacks Check Return Statuses and Fail Safely Summary Exercises 39. Capabilities Rationale for Capabilities The Linux Capabilities Process and File Capabilities Process Capabilities File Capabilities Purpose of the Process Permitted and Effective Capability Sets Purpose of the File Permitted and Effective Capability Sets Purpose of the Process and File Inheritable Sets Assigning and Viewing File Capabilities from the Shell The Modern Capabilities Implementation Transformation of Process Capabilities During exec() Capability Bounding Set Preserving root Semantics Effect on Process Capabilities of Changing User IDs Changing Process Capabilities Programmatically Creating Capabilities-Only Environments Discovering the Capabilities Required by a Program Older Kernels and Systems Without File Capabilities Summary Exercise 40. Login Accounting Overview of the utmp and wtmp Files The utmpx API The utmpx Structure Retrieving Information from the utmp and wtmp Files Retrieving the Login Name: getlogin() Updating the utmp and wtmp Files for a Login Session The lastlog File Summary Exercises 41. Fundamentals of Shared Libraries Object Libraries Static Libraries Overview of Shared Libraries Creating and Using Shared Libraries—A First Pass Creating a Shared Library Position-Independent Code Using a Shared Library The Shared Library Soname Useful Tools for Working with Shared Libraries Shared Library Versions and Naming Conventions Installing Shared Libraries Compatible Versus Incompatible Libraries Upgrading Shared Libraries