30 Corporate Drive, Suite 400, Burlington, MA 01803, USA This book is printed on acid-free paper. Copyright © 2009 by Elsevier Inc. All rights reserved. Designations used by companies to distinguish their products are often claimed as trademarks or registered trademarks. In all instances in which Morgan Kaufmann Publishers is aware of a claim, the product names appear in initial capital or all capital letters. All trademarks that appear or are otherwise referred to in this work belong to their respective owners. Neither Morgan Kaufmann Publishers nor the authors and other contributors of this work have any relationship or affiliation with such trademark owners nor do such trademark owners confirm, endorse or approve the contents of this work. Readers, however, should contact the appropriate companies for more information regarding trademarks and any related registrations. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means—electronic, mechanical, photocopying, scanning, or otherwise— without prior written permission of the publisher. Permissions may be sought directly from Elsevier’s Science & Technology Rights Department in Oxford, UK: phone: (+44) 1865 843830, fax: (+44) 1865 853333, E-mail: perm Preface to the Second Edition When we wrote the first edition of this book, it was not very common for college courses on networking to include programming components. That seems difficult to believe now, when the Internet has become so important to our world, and the pedagogical benefits of hands-on programming and real-world protocol examples are so widely accepted. Although there are now other languages that provide access to the Internet, interest in the original C-based Berkeley Sockets remains high. The Sockets API (application programming interface) for networking was developed at UC Berkeley in the 1980s for the BSD flavor of UNIX—one of the very first examples of what would now be called an open-source project. The Sockets API and the Internet both grew up in a world of many competing protocol families—IPX, Appletalk, DECNet, OSI, and SNA in addition to Transmission Control Proto- col/Internet Protocal (TCP/IP)—and Sockets was designed to support them all. Fewer protocol families were in common use by the time we wrote the first edition of this book, and the num- ber today is even smaller. Nevertheless, as we predicted in the first edition, the Sockets API remains important for those who want to design and build distributed applications that use the Internet—that is, that use TCP/IP. And the interface has proven robust enough to support the new version of the Internet Protocol (IPv6), which is now supported on virtually all common computing platforms. Two main considerations motivated this second edition. First, based on our own experi- ence and feedback from others, we found that some topics needed to be presented in more depth and that others needed to be expanded. The second consideration is the increasing acceptance and use of IP version 6, which is now supported by essentially all current end sys- tem platforms. At this writing, it is not possible to use IPv6 to exchange messages with a large fraction of hosts on the Internet, but it is possible to assign an IPv6 address to many of them. Although it is still too early to tell whether IPv6 will take over the world, it is not too early to start writing applications to be prepared. ix x Preface Changes from the First Edition We have updated and considerably expanded most of the material, having added two chapters. Major changes from the first edition include: ■ IP version 6 coverage. We now include three kinds of code: IPv4-specific, IPv6-specific, and generic. The code in the later chapters is designed to work with either protocol version on dual-stack machines. ■ An additional chapter on socket programming in C++ (contributed by David B. Sturgill). The PracticalSocket library provides wrappers for basic socket functionality. These allow an instructor to teach socket programming to students without C programming back- ground by giving them a library and then gradually peeling back the layers. Students can start developing immediately after understanding addresses/ports and client/server. Later they can be shown the details of socket programming by peeking inside the wrapper code. Those teaching a subject that uses networking (e.g., OS) can use the library and only selectively peel back the cover. ■ Enhanced coverage of data representation issues and strategies for organizing code that sends and receives messages. In our instructional experience, we find that students have 1 less and less understanding of how data is actually stored in memory, so we have attempted to compensate with more discussion of this important issue. At the same time, internationalization will only increase in importance, and thus we have included basic coverage of wide characters and encodings. ■ Omission of the reference section. The descriptions of most of the functions that make up the Sockets API have been collected into the early chapters. However, with so many online sources of reference information—including “man pages”—available, we chose to leave out the complete listing of the API in favor of more code illustrations. ■ Highlighting important but subtle facts and caveats. Typographical devices call out important concepts and information that might otherwise be missed on first reading. Although the scope of the book has expanded, we have not included everything that we might have (or even that we were asked to include); examples of topics left for more comprehensive texts (or the next edition) are raw sockets and programming with WinSock. Intended Audience We originally wrote this book so that we would have something to hand our students when we wanted them to learn socket programming, so we would not have to take up valuable class time 1 We speculate that this is due to the widespread use of C++ and Java, which hide such details from the programmer, in undergraduate curricula. Preface xi teaching it. In the years since the first edition, we have learned a good deal about the topics that students need lots of help on, and those where they do not need as much handholding. We also found that our book was appreciated at least as much by practitioners who were looking for a gentle introduction to the subject. Therefore, this book is aimed simultaneously at two general audiences: students in introductory courses in computer networks (graduate or undergraduate) with a programming component, and practitioners who want to write their own programs that communicate over the Internet. For students, it is intended as a supplement, not as a primary text about networks. Although this second edition is significantly bigger in size and scope than the first, we hope the book will still be considered a good value in that role. For practitioners who just want to write some useful code, it should serve as a standalone introduction—but readers in that category should be warned that this book will not make them experts. Our philosophy of learning by doing has not changed, nor has our approach of providing a concise tutorial sufficient to get one started learning on one’s own, and leaving the comprehensive details to other authors. For both audiences, our goal is to take you far enough so that you can start experimenting and learning on your own. Assumed Background We assume basic programming skills and experience with C and UNIX. You are expected to be conversant with C concepts such as pointers and type casting, and you should have a basic understanding of the binary representation of data. Some of our examples are factored into files that should be compiled separately; we assume that you can deal with that. Here is a little test: If you can puzzle out what the following code fragment does, you should have no problem with the code in this book: typedef struct { int a; short s[2]; } MSG; MSG *mp, m = {4, 1, 0}; char *fp, *tp; mp = (MSG *) malloc(sizeof(MSG)); for (fp = (char *)m.s, tp = (char *)mp->s; tp < (char *)(mp+1);) *tp++ = *fp++; If you do not understand this fragment, do not despair (there is nothing quite so convo- luted in our code), but you might want to refer to your favorite C programming book to find out what is going on here. You should also be familiar with the UNIX notions of process/address space, command- line arguments, program termination, and regular file input and output. The material in Chapters 4 and 6 assumes a somewhat more advanced grasp of UNIX. Some prior exposure to networking concepts such as protocols, addresses, clients, and servers will be helpful. xii Preface Platform Requirements and Portability Our presentation is UNIX-based. When we were developing this book, several people urged us to include code for Windows as well as UNIX. It was not possible to do so for various reasons, including the target length (and price) we set for the book. For those who only have access to Windows platforms, please note that the examples in the early chapters require minimal modifications to work with WinSock. (You have to change the include files and add a setup call at the beginning of the program and a cleanup call at the end.) Most of the other examples also require very slight additional modifications. However, some are so dependent on the UNIX programming model that it does not make sense to port them to WinSock. WinSock-ready versions of the other examples, as well as detailed descriptions of the code modifications required, are available from the book’s Web site at www.mkp.com/socket. Note also that almost all of our example code works with minimal modifications under the Cygwin UNIX library package for Windows, which is available online. For this second edition, we have adopted the C99 language standard. This version of the language is supported by most compilers and offers so many readability-improving advantages—including line-delimited comments, fixed-size integer types, and declarations anywhere in a block—that we could not justify not using it. Our code makes use of the “Basic Socket Interface Extensions for IPv6” ?. Among these extensions is a new and different interface to the name system. Because we rely completely on this new interface (getaddrinfo()), our generic code may not run on some older platforms. However, we expect that most modern systems will run our code just fine. The example programs included here have all been tested (and should compile and run without modification) on both *NIX and MacOS. Header (.h) file locations and dependencies are, alas, not quite standard and may require some fiddling on your system. Socket option support also varies widely across systems; we have tried to focus on those that are most universally supported. Consult your API documentation for system specifics. (By API documentation we mean the “man pages” for your system. To learn about this, type “man man” or use your favorite web search tool.) Please be aware that although we strive for a basic level of robustness, the primary goal of our code examples is pedagogy, and the code is not production quality. We have sacrificed some robustness for brevity and clarity, especially in the generic server code. (It turns out to be nontrivial to write a server that works under all combinations of IPv4 and IPv6 protocol configurations and also maximizes the likelihood of successful client connection under all circumstances.) This Book Will Not Make You an Expert! We hope this second edition will be useful as a resource, even to those who already know quite a bit about sockets. As with the first edition, we learned some things in writing it. But becoming an expert takes years of experience, as well as other, more comprehensive sources ?, ?. Preface xiii The first chapter is intended to give “just enough” of the big picture to get you ready to write code. Chapter ?? shows you how to write TCP clients and servers using either IPv4 or IPv6. Chapter ?? shows how to make your clients and servers use the network’s name service, and also describes how to make them IP-version-independent. Chapter ?? covers User Datagram Protocol (UDP). Chapters ?? and ?? provide background needed to write more programs, while Chapter ?? relates some of what is going on in the Sockets implementation to the API calls; these three are essentially independent and may be presented in any order. Finally, Chapter ?? presents a C++ class library that provides simplified access to socket functionality. Throughout the book, certain statements are highlighted like this: This book will not make you an expert! Our goal is to bring to your attention those subtle but important facts and ideas that one might miss on first reading. The marks in the margin tell you to “note well” whatever is in bold. Acknowledgments Many people contributed to making this book a reality. In addition to all those who helped us with the first edition (Michel Barbeau, Steve Bernier, Arian Durresi, Gary Harkin, Ted Herman, Lee Hollaar, David Hutchison, Shunge Li, Paul Linton, Ivan Marsic, Willis Marti, Kihong Park, Dan Schmitt, Michael Scott, Robert Strader, Ben Wah, and Ellen Zegura), we especially thank David B. Sturgill, who contributed code and text for Chapter ??, and Bobby Krupczak for his help in reviewing the draft of this second edition. Finally, to the folks at Morgan Kaufmann/Elsevier— Rick Adams, our editor, assistant editor Maria Alonso, and project manager Melinda Ritchie— thank you for your patience, help, and caring about the quality of our book. Feedback We are very interested in weeding out errors and otherwise improving future editions/ printings, so if you find any errors, please send an e-mail to either of us. We will maintain an errata list on the book’s Web page. M.J.D. c h a p t e r 1 Introduction Today people use computers to make phone calls, watch TV, send instant messages to their friends, play games with other people, and buy most anything you can think of—from songs to automobiles. The ability of programs to communicate over the Internet makes all this possible. It’s hard to say how many individual computers are now reachable over the Internet, but we can safely say that it is growing rapidly; it won’t be long before the number is in the billions. Moreover, new applications are being developed every day. With the push for ever increasing bandwidth and access, the impact of the Internet will continue to grow for the forseeable future. How does a program communicate with another program over a network? The goal of this book is to start you on the road to understanding the answer to that question, in the context of the C programming language. For a long time, C was the language of choice for implementing network communication softward. Indeed, the application programming interface (API) known as Sockets was first developed in C. Before we delve into the details of sockets, however, it is worth taking a brief look at the big picture of networks and protocols to see where our code will fit in. Our goal here is not to teach you how networks and TCP/IP work—many fine texts are available for that purpose [1,3,10,15,17]—but rather to introduce some basic concepts and terminology. 1.1 Networks, Packets, and Protocols A computer network consists of machines interconnected by communication channels. We call these machines hosts and routers. Hosts are computers that run applications such as your Web 1 2 Chapter 1: Introduction browser, your IM agent, or a file-sharing program. The application programs running on hosts are the real “users” of the network. Routers (also called gateways) are machines whose job is to relay, or forward, information from one communication channel to another. They may run programs but typically do not run application programs. For our purposes, a communication channel is a means of conveying sequences of bytes from one host to another; it may be a wired (e.g., Ethernet), a wireless (e.g., WiFi), or other connection. Routers are important simply because it is not practical to connect every host directly to every other host. Instead, a few hosts connect to a router, which connects to other routers, and so on to form the network. This arrangement lets each machine get by with a relatively small number of communication channels; most hosts need only one. Programs that exchange information over the network, however, do not interact directly with routers and generally remain blissfully unaware of their existence. By information we mean sequences of bytes that are constructed and interpreted by pro- grams. In the context of computer networks, these byte sequences are generally called packets. A packet contains control information that the network uses to do its job and sometimes also includes user data. An example is information identifying the packet’s destination. Routers use such control information to figure out how to forward each packet. A protocol is an agreement about the packets exchanged by communicating programs and what they mean. A protocol tells how packets are structured—for example, where the destination information is located in the packet and how big it is—as well as how the infor- mation is to be interpreted. A protocol is usually designed to solve a specific problem using given capabilities. For example, the HyperText Transfer Protocol (HTTP) solves the problem of transferring hypertext objects between servers, where they are stored or generated, and Web browsers that make them visible and useful to users. Instant messaging protocols solve the problem of enabling two or more users to exchange brief text messages. Implementing a useful network requires solving a large number of different problems. To keep things manageable and modular, different protocols are designed to solve different sets of problems. TCP/IP is one such collection of solutions, sometimes called a protocol suite. It happens to be the suite of protocols used in the Internet, but it can be used in stand-alone private networks as well. Henceforth when we talk about the network, we mean any network that uses the TCP/IP protocol suite. The main protocols in the TCP/IP suite are the Internet Protocol (IP), the Transmission Control Protocol (TCP), and the User Datagram Protocol (UDP). It turns out to be useful to organize protocols into layers; TCP/IP and virtually all other protocol suites are organized this way. Figure 1.1 shows the relationships among the proto- cols, applications, and the Sockets API in the hosts and routers, as well as the flow of data from one application (using TCP) to another. The boxes labeled TCP and IP represent imple- mentations of those protocols. Such implementations typically reside in the operating system of a host. Applications access the services provided by UDP and TCP through the Sockets API, represented as a dashed line. The arrow depicts the flow of data from the application, through the TCP and IP implementations, through the network, and back up through the IP and TCP implementations at the other end. 1.1 Networks, Packets, and Protocols 3 Application Application Socket Socket TCP TCP IP IP IP Channel Channel (e.g., Ethernet) Host Router Host Figure 1.1: A TCP/IP network. In TCP/IP, the bottom layer consists of the underlying communication channels—for example, Ethernet or dial-up modem connections. Those channels are used by the network layer, which deals with the problem of forwarding packets toward their destination (i.e., what routers do). The single-network layer protocol in the TCP/IP suite is the Internet Protocol; it solves the problem of making the sequence of channels and routers between any two hosts look like a single host-to-host channel. The Internet Protocol provides a datagram service: every packet is handled and delivered by the network independently, like letters or parcels sent via the postal system. To make this work, each IP packet has to contain the address of its destination, just as every package that you mail is addressed to somebody. (We’ll say more about addresses shortly.) Although most delivery companies guarantee delivery of a package, IP is only a best-effort protocol: it attempts to deliver each packet, but it can (and occasionally does) lose, reorder, or duplicate packets in transit through the network. The layer above IP is called the transport layer. It offers a choice between two protocols: TCP and UDP. Each builds on the service provided by IP, but they do so in different ways to provide different kinds of transport, which are used by application protocols with different needs. TCP and UDP have one function in common: addressing. Recall that IP delivers packets to hosts; clearly, a finer granularity of addressing is needed to get a packet to a particular application program, perhaps one of many using the network on the same host. Both TCP and UDP use addresses, called port numbers, to identify applications within hosts. TCP and UDP are called end-to-end transport protocols because they carry data all the way from one program to another (whereas IP only carries data from one host to another). TCP is designed to detect and recover from the losses, duplications, and other errors that may occur in the host-to-host channel provided by IP. TCP provides a reliable byte-stream chan- nel, so that applications do not have to deal with these problems. It is a connection-oriented protocol: before using it to communicate, two programs must first establish a TCP connection,