Salomon-FM.qxd 10/19/05 9:18 AM Page i Foundations of Computer Security Salomon-FM.qxd 10/19/05 9:18 AM Page iii David Salomon Foundations of Computer Security With 45 Figures Salomon-FM.qxd 10/19/05 9:18 AM Page iv Professor David Salomon (emeritus) Computer Science Department California State University Northridge, CA 91330-8281 USA email: [email protected] British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library Library of Congress Control Number: 2005932091 ISBN-10: 1-84628-193-8 e-ISBN 1-84628-193-8 ISBN-13: 978-1-84628-193-8 Printed on acid-free paper © Springer-Verlag London Limited 2006 Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic repro- duction in accordance with the terms of licences issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers. The use of registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific state- ment, that such names are exempt from the relevant laws and regulations and therefore free for general use. The publisher makes no representation, express or implied, with regard to the accuracy of the information contained in this book and cannot accept any legal responsibility or liability for any errors or omissions that may be made. Printed in the United States of America (HAM) 9 8 7 6 5 4 3 2 1 Springer Science+Business Media springeronline.com Dedicated to the many anonymous users and experts who serve with zeal and dedication in the unending war of computer security. There isn’t an author who doesn’t take their [sic] books personally. —Muriel Spark, A Far Cry From Kensington (1988). Preface Gentle Reader. Your interest in this book is understandable. Computer security hasbecomeoneofthemostimportantareasintheentiredisciplineofcomputing. Computerstodayareusednotonlyinthehomeandoffice,butinamultitudeofcrucial and sensitive applications. Computers control long distance telephone conversations, the flow of information on the Internet, the distribution of electrical power to cities, and they monitor the operations of nuclear power plants and the performance of space satellites, to name just a few important applications. We have become used to these small, quiet machines that permeate our lives and we take them for granted, but from time to time, when they don’t perform their tasks, we immediately become aware that something has gone terribly wrong. Considering the complexity of today’s computers and their functions, and considering especially the physical hazards that abound in the world, it is a wonder that our computers function at all, yet we expect them to be reliable and we entrust them with more and more delicate, sensitive, and complex assignments. It is easy to disrupt a computer. Just brush your elbow accidentally against your desk and you may spill your cup of coffee on your computer. A power loss lasting a fraction of a second may lead to a head crash of the hard disk, resulting in a complete loss of the disk and all its data. Carelessness on the part of operators or administrators in a large computations center can cause a costly loss of data or even physical damage to expensive equipment. Yet all these dangers (and there are many more like them) pale in comparison with the many types of intentional criminal damage that we have come to expect and that we collectively associate with the field of computer security. A term closely related to computer security is computer crime. A computer crime is an incident of computer security in which a law is broken. Traditionally, computer crime has had a low profile. After all, in a computer crime there are no smoking guns, no blood-stained victims, and no getaway cars. Often, such a crime is solved just by sheer accident. In contrast, computer security is a high-visibility discipline because it involves most of us. Experience has shown that the more sophisticated a civilization is, the more vul- nerable it is to natural or man-made disruptions. A tree that fell on power lines in viii Preface Ohio in August 2004 plunged 50 million people from Detroit to New York into dark- ness. A computer glitch at an airport on 26 December 2004 (the day this paragraph was written) caused the cancellation of 1100 flights of Comair, a subsidiary of Delta Air Lines, and similar examples abound. Our civilization depends more and more on computers, which is why any disruption of our computers is at least inconvenient and at worst catastrophic. Inthepast,computersecurityviolations,suchasvirusesandDoS(denialofservice, Section 7.5) attacks were caused by hackers, most of whom were believed to be young adults who did this for fun or enjoyed the feeling of power and notoriety. However, it seems that this situation is rapidly changing. Security experts are warning that future attacks on computers may be planned and funded by terrorists (better called cyberterrorists) and may be devastating. A powerful hurricane, a huge earthquake, or a tsunami may kill many and wreak untold havoc, but a large-scale, concerted attack onkeycomputersmaybringtheeconomyofanentirecountrytoitsknees,eventhough no one may actually get killed. Thereasonforsuchdirepredictionsisourexperiencewithcomputersecurityinthe last two decades. We know that a single computer virus, perhaps written and released byateenagerlivinginaremotetowninadistantcountry,canpropagatequickly,infect a vast number of computers within hours, and cause economic damage in the billions (of Dollars, Euros, or whatever currency is affected). Today, computers are responsible for the distribution of electrical power and for routingtelephoneconversations. Theystoreinformationonpassengerandcargoflights, onlargecashtransfersbetweenbanks,andonmilitaryplans,tonamejustafewcrucial applications. It is generally agreed that a well-organized attack that takes over several important, sensitive computers may cause at least a temporary collapse of an entire country. What makes this kind of attack attractive to organized terrorists is that it can be carried out from the comfort of their homes. There is no need to actually go anywhere, to obtain and use dangerous nuclear or chemical materials, or to smuggle anything across international borders. The fact that we depend so much on computers may be crucial to our future survival, and the least that we can do now is to learn as much as possible about potential threats to computers and how to defend against them. Virus writing is a crazy activity. People who write viruses just don’t consider the consequences of their actions. At the same time, I believe in the American constitu- tion, and the first amendment, which gives people freedom to write and to talk, so I don’t have a problem in the larger sense of people discussing or studying viruses. —Peter Tippett (Symantec) in [Virus bulletin 05] May 1994 issue. Thereisanongoingdebateaboutwhethernewly-discoveredsecurityholesandvul- nerabilities in operating systems and communications software should be made public. Publicizing a security weakness allows users to avoid it until a patch is issued or a so- lution is found. On the other hand, it gives the bad guys ideas. So far, advocates of public exposure have had the upper hand, with the result that any item of news about a new computer security problem ignites a race between attackers and defenders. The following is a list of some of those races: Preface ix SNMP flaw. A flaw in the Simple Network Management Protocol (SNMP) leaves open many network devices to attack. The flaw has not been widely exploited. Microsoft SQL vulnerability. A hole in a common component of Microsoft’s SQL databasesoftwareleavesPCsopentoremoteattack. Sixmonthsafteritwasfound,the vulnerability was exploited by the slammer worm (see year 2003 in Appendix B). Microsoft RPC flaw. In July 2003, Microsoft published details of a flaw in the remote procedure call (RPC) functions of Windows. About three weeks later, the MSBlastwormarrivedandexploitedthisflawtoinfectasmanyas10millioncomputers. Microsoft LSASS flaw. A hole in Local Security Authority Subsystem Service (LSASS)exposedpersonalcomputersrunningtheWindowsoperatingsystem. Amonth afteritwasrevealed,thesasserwormhittheInternetandspreadamongcomputersthat still had this hole (see year 2004 in Appendix B). iFrame flaw. In late October 2004, a security researcher discovered the existence of a flaw in Internet Explorer, a popular Web browser (page 61). Hackers with nothing bettertodoimmediatelyexploitedthevulnerabilitytocompromisepersonalcomputers running this software. Three types of persons are involved in computer security: experts who study this fieldandrecommendpreventivemeasuresandsolutions,thegeneralpublic,whichsuffers from the breakdown of computer security, and the (mostly anonymous) perpetrators of the various misdeeds and attacks. Most of these perpetrators are known as hackers, which is why this important, popular term is discussed here. From the dictionary Expert: someone widely recognized as a reliable source of knowledge or skill whose judgement is accorded authority and status by the public or their peers. The Hacker Madame Curie once said “En science, nous devons nous int´eresser aux choses, non aux personnes [In science, we should be interested in things, not in people].” Things, however, havesincechanged, andtodaywehavetobeinterestednotjustinthefactsof computer security and crime, but in the people who perpetrate these acts. Hence this discussion of hackers. Over the centuries, the term “hacker” has referred to various activities. We are familiar with usages such as “a carpenter hacking wood with an ax” and “a butcher hacking meat with a cleaver,” but it seems that the modern, computer-related form of this term originated in the many pranks and practical jokes perpetrated by students at MIT in the 1960s. As an example of the many meanings assigned to this term, see [Schneier 04] which, among much other information, explains why Galileo was a hacker but Aristotle wasn’t. A hack is a person lacking talent or ability, as in a “hack writer.” Hack as a verb is used in contexts such as “hack the media,” “hack your brain,” and “hack your reputation.” Recently, it has also come to mean either a kludge, or the opposite of a x Preface kludge, as in a clever or elegant solution to a difficult problem. A hack also means a simplebut often inelegant solution or technique. Thefollowing tentativedefinitionsare quoted from the jargon file ([jargon 04], edited by Eric S. Raymond): 1. A person who enjoys exploring the details of programmable systems and how to stretch their capabilities, as opposed to most users, who prefer to learn only the minimum necessary. 2. One who programs enthusiastically (even obsessively) or who enjoys program- ming rather than just theorizing about programming. 3. A person capable of appreciating hack value. 4. A person who is good at programming quickly. 5. An expert at a particular program, or one who frequently does work using it or on it; as in “a Unix hacker.” (Definitions 1 through 5 are correlated, and people who fit them congregate.) 6. An expert or enthusiast of any kind. One might be an astronomy hacker, for example. 7. One who enjoys the intellectual challenge of creatively overcoming or circum- venting limitations. 8. [deprecated] A malicious meddler who tries to discover sensitive information by poking around. Hence “password hacker” and “network hacker.” The correct term for this sense is cracker (which stands for criminal hacker). Today’s computer hacker is often an expert in a computer-related field who finds a way to exploit a weakness or a vulnerability in a certain component of that field. This component may be a piece of hardware, part of the operating system, or a software application. Not all hackers are experts and not all are malicious. A notable example is Linus Torvalds, the creator of the well-known, free Linux operating system. Many Linux users will agree that this activity of Torvalds is a hack, but everyone (except commercial competitors) agrees that it is useful. I think any time you expose vulnerabilities it’s a good thing. —Janet Reno Somesecurityexpertsclaimthattoday’scomputerhackersshouldbetermedcrack- ersorintruders,butthegeneralpublicandthemediaseemtolovethetermhacker. The word “cracker” is used to designate someone who breaks the security code of software, so that it can be used without pay. The term “intruder” is commonly used to indicate a person who breaks into a remote computer. The following classification of the various hacker categories is informal and is by no means universally accepted. The highest category of hacker may be a brilliant programmer (although such a hacker may prefer the title of guru, cracksman, or wizard). Someone who is intimately familiarwithacertaincommunicationsprogram,protocol,operatingsystem,orencryp- tion algorithm. Such a person can identify weaknesses or vulnerabilities and then come upwithaclever,originalwayofpenetratingacomputerandinflictingdamage. Alterna- tively,suchanexpertmaydevelopwaysandmeanstoplugupsecurityholesinsoftware, or even completely rewrite a weak routine or procedure to make it invulnerable. Preface xi The next category is that of the good programmer. Such a person hears of a new security threat, for example, a new type of virus, and may decide to “improve” it. A good programmer can disassemble the code of a virus, read and understand it, and come up with more “efficient” ways of employing the basic principle of the virus. Such a person may also be a good guy (a white-hat hacker) and work as a security expert. Disassembling and reading the code of a virus uncovers the vulnerabilities the virus exploits and leads directly to eliminating them. A script kid is a hacker with little or no programming skills who simply follows directions created by a higher-rank hacker or who uses a cookbook approach without fully understanding the principles and details of what he is constructing. A hacktivist is an activist who employs hacking to promote a cause. In 1995, a virusattachedapoliticalmessage“StopallFrenchnucleartestinginthePacific”tothe footer of letters printed from Microsoft Word, so users who trusted the computer and didn’t check their printouts became unwilling supporters of a cause. A sneaker or a gray-hat is a hacker who breaks security for altruistic motives or other non-malicious reasons. The darker the hat, the more the ethics of the activity should be considered dubious. Theleastharmfulhackeristhewhite-hattype. Thistermisoftenusedtodescribe self-appointedsecurityguruswhoattempttobreakintocomputersornetworksinorder to find security flaws and inform the owners/administrators of the problem. The following is a list of “tools of the trade,” methods, approaches, and special software used by hackers to gain unauthorized access to data, to computers, and to entire computer installations: Rogue software. These are computer programs especially designed to propagate among computers and either inflict damage or collect data and send it back to the hacker. They are also known as malware. The chief types of rogue software are viruses, worms, Trojan horses, and the various kinds of spyware. Each is described in one paragraph below. Virus (Chapter 2, a term borrowed from biology). A program that invades a com- puter and embeds itself inside a host program, where it replicates and propagates from computer to computer, infecting each in turn. A virus spreads by infected removable disks, or over a network. Worm. A program that exploits weaknesses in an operating system or in commu- nicationssoftwareinordertoreplicateitselfonothercomputersonanetwork. Aworm does not reside in a host program. Worms are discussed in Chapter 3. Trojan horse. A program that seems useful, but has a backdoor, installed by its creator and employed later to gather information or to damage software. Examples are programsthatmimicloginsequencesorthatfoolauserintodownloadingandexecuting them by claiming to be useful applications. This type of rogue software is described in Chapter 4. Spyware is the general name assigned to a whole range of nasty software that runs on a computer, monitors its users’ activities, collects information such as keystrokes, xii Preface screen dumps, and file directories, and either saves this information or sends it to a remote location without the knowledge or consent of the computer owner. Spyware is described in Chapter 9. Scanning. This term refers to software and equipment that methodically probes computers on the Internet for vulnerabilities. Two of the main tools used for this purpose are a vulnerability scanner and a sniffer. They are described here. Vulnerabilityscanner. Aprogramdesignedtoquicklycheckcomputersonanetwork forknownweaknesses. Aportscanner(Section7.2)isaspecialcase. Itisaprogramthat attempts to find open ports on a target computer or ports that are available to access thecomputer. Afirewallisapieceofhardwareorsoftwarethatdefendscomputersfrom intruders by closing off all unused ports. Sniffer. A program that captures passwords and other data while the data is in transit either within the computer or between computers or routers on a network. Exploit. Aready-to-runprogramthattakesadvantageofaknownweakness. These can often be found in hackers’ newsgroups. Social engineering. A general term for methods that exploit human weaknesses. A hacker may discover someone’s password by calling and pretending to be an official, by looking over someone’s shoulder while a password is being typed, or by sending email that pauses as an official notice asking for sensitive information. Bribing and blackmailing are also included in this class. Even though no special software may be needed and no software weakness is exploited, this is still a powerful tool used by many miscreants. Social engineering (page 204) is a wide class that includes, among others, the following methods: Shoulder spying (or shoulder watching or surfing). A hacker enters a secure com- puterinstallationorarestrictedcomputerlab(oftendisguisedasapizzadeliveryman) and looks behind users’ shoulders for passwords typed by them or being taped to the sides of computer monitors. Optical spying. The hacker watches from a nearby room or building, perhaps with a binocular, and tries to read keystrokes typed by legitimate users. Scavenging (or dumpster diving). Hackers have been known to collect trash and examine it for passwords and credit card numbers (see also page 205). Side-channel attacks. A hacker can spy on a secure installation “from the side” by capturingandlisteningtoinformationthatiscontinuouslyandunintentionallyleakedby electronic devices inside. The basis of this approach is the well-known fact that people are nosy and machines are noisy. Side-channel methods are discussed in Section 1.1, but the following are typical examples. Eavesdropping. A hacker, often disguised as a telephone company repair man, entersacomputerroomandplantsdevicesthatlatertransmittohimusefuldataonthe activities of users. Such devices may include radio transmitters, acoustic microphones (Section 1.1.1), and cameras. Acoustic keyboard eavesdropping. This recent, sophisticated approach to spying employs the little-known fact that each key in a keyboard emits a slightly different sound when pressed. Recording the sounds of keys with a sensitive microphone may