01_574817 ffirs.qxd 3/16/05 8:37 PM Page ii 01_574817 ffirs.qxd 3/16/05 8:37 PM Page i Reversing: Secrets of Reverse Engineering 01_574817 ffirs.qxd 3/16/05 8:37 PM Page ii 01_574817 ffirs.qxd 3/16/05 8:37 PM Page iii Reversing: Secrets of Reverse Engineering Eldad Eilam 01_574817 ffirs.qxd 3/16/05 8:37 PM Page iv Reversing: Secrets of Reverse Engineering Published by Wiley Publishing, Inc. 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright © 2005 by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada Library of Congress Control Number: 2005921595 ISBN-10: 0-7645-7481-7 ISBN-13: 978-0-7645-7481-8 Manufactured in the United States of America 10 9 8 7 6 5 4 3 2 1 1B/QR/QU/QV/IN No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copy- right Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Legal Department, Wiley Publishing, Inc., 10475 Crosspoint Blvd., Indianapolis, IN 46256, (317) 572-3447, fax (317) 572-4355, e-mail: [email protected]. Limit of Liability/Disclaimer of Warranty:The publisher and the author make no repre- sentations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fit- ness for a particular purpose. No warranty may be created or extended by sales or promo- tional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in ren- dering any professional services. If professional assistance is required, the services of a com- petent professional person should be sought. Neither the publisher nor the author shall be liable for any damages arising herefrom. The fact that an organization or Website is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or Website may provide or recommendations it may make. Further, readers should be aware that Internet Websites listed in this work may have changed or disappeared between when this work was written and when it is read. For general information on our other products and services or to obtain technical support, please contact our Customer Care Department within the U.S. at (800) 762-2974, outside the U.S. at (317) 572-3993 or fax (317) 572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Trademarks: Wiley, the Wiley Publishing logo and related trade dress are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates, in the United States and other countries, and may not be used without written permission. All other trademarks are the property of their respective owners. Wiley Publishing, Inc., is not associated with any product or vendor mentioned in this book. 01_574817 ffirs.qxd 3/16/05 8:37 PM Page v Credits Executive Editor Graphics and Production Specialists Robert Elliott Denny Hager Jennifer Heleine Development Editor Lynsey Osborn Eileen Bien Calabro Mary Gillot Virgin Copy Editor Quality Control Technician Foxxe Editorial Services Leeann Harney Editorial Manager Proofreading and Indexing Mary Beth Wakefield TECHBOOKS Production Services Vice President & Executive Group Cover Designer Publisher Michael Trent Richard Swadley Vice President and Publisher Joseph B. Wikert Project Editor Pamela Hanley Project Coordinator Ryan Steffen v 01_574817 ffirs.qxd 3/16/05 8:37 PM Page vi 01_574817 ffirs.qxd 3/16/05 8:37 PM Page vii Foreword It is amazing, and rather disconcerting, to realize how much software we run without knowing for sure what it does. We buy software off the shelf in shrink- wrapped packages. We run setup utilities that install numerous files, change system settings, delete or disable older versions and superceded utilities, and modify critical registry files. Every time we access a Web site, we may invoke or interact with dozens of programs and code segments that are necessary to give us the intended look, feel, and behavior. We purchase CDs with hundreds of games and utilities or download them as shareware. We exchange useful programs with colleagues and friends when we have tried only a fraction of each program’s features. Then, we download updates and install patches, trusting that the vendors are sure that the changes are correct and complete. We blindly hope that the latest change to each program keeps it compatible with all of the rest of the programs on our system. We rely on much software that we do not understand and do not know very well at all. I refer to a lot more than our desktop or laptop personal computers. The concept of ubiquitous computing, or “software everywhere,” is rapidly putting software control and interconnection in devices throughout our envi- ronment. The average automobile now has more lines of software code in its engine controls than were required to land the Apollo astronauts on the Moon. Today’s software has become so complex and interconnected that the devel- oper often does not know all the features and repercussions of what has been created in an application. It is frequently too expensive and time-consuming to test all control paths of a program and all groupings of user options. Now, with multiple architecture layers and an explosion of networked platforms that the software will run on or interact with, it has become literally impossible for all vii 01_574817 ffirs.qxd 3/16/05 8:37 PM Page viii viii Foreword combinations to be examined and tested. Like the problems of detecting drug interactions in advance, many software systems are fielded with issues unknown and unpredictable. Reverse engineering is a critical set of techniques and tools for understand- ing what software is really all about. Formally, it is “the process of analyzing a subject system to identify the system’s components and their interrelation- ships and to create representations of the system in another form or at a higher level of abstraction”(IEEE 1990). This allows us to visualize the software’s structure, its ways of operation, and the features that drive its behavior. The techniques of analysis, and the application of automated tools for software examination, give us a reasonable way to comprehend the complexity of the software and to uncover its truth. Reverse engineering has been with us a long time. The conceptual Revers- ing process occurs every time someone looks at someone else’s code. But, it also occurs when a developer looks at his or her own code several days after it was written. Reverse engineering is a discovery process. When we take a fresh look at code, whether developed by ourselves or others, we examine and we learn and we see things we may not expect. While it had been the topic of some sessions at conferences and computer user groups, reverse engineering of software came of age in 1990. Recognition in the engineering community came through the publication of a taxonomy on reverse engineering and design recovery concepts in IEEE Softwaremagazine. Since then, there has been a broad and growing body of research on Reversing techniques, software visualization, program understanding, data reverse engi- neering, software analysis, and related tools and approaches. Research forums, such as the annual international Working Conference on Reverse Engineering (WCRE), explore, amplify, and expand the value of available tech- niques. There is now increasing interest in binary Reversing, the principal focus of this book, to support platform migration, interoperability, malware detection, and problem determination. As a management and information technology consultant, I have often been asked: “How can you possibly condone reverse engineering?” This is soon fol- lowed by: “You’ve developed and sold software. Don’t you want others to respect and protect your copyrights and intellectual property?” This discus- sion usually starts from the negative connotation of the term reverse engineer- ing, particularly in software license agreements. However, reverse engineering technologies are of value in many ways to producers and consumers of soft- ware along the supply chain. Astethoscope could be used by a burglar to listen to the lock mechanism of a safe as the tumblers fall in place. But the same stethoscope could be used by your family doctor to detect breathing or heart problems. Or, it could be used by a computer technician to listen closely to the operating sounds of a sealed disk drive to diagnose a problem without exposing the drive to