E 2 D N I T D I J IIDDAA PPRROO EBP MP TT HH EE ON DDEE--OOBBFFUUSSCCAATTEEDD SUB 2ND EDITION II DD AA PP RR OO J E M TT BP P HH B U S No source code? No problem. With IDA Pro, the inter- • Use code graphing to quickly make sense of cross- EE BB OO OO KK active disassembler, you live in a source code–optional references and function calls world. IDA can automatically analyze the millions of opcodes that make up an executable and present you • Extend IDA to support new processors and filetypes II using the SDK with a disassembly. But at that point, your work is just DD T H E U N O F F I C I A L G U I D E T O T H E beginning. With The IDA Pro Book, you’ll learn how • Explore popular plug-ins that make writing IDA scripts W O R L D ’ S M O S T P O P U L A R D I S A S S E M B L E R to turn that mountain of mnemonics into something you easier, allow collaborative reverse engineering, and AA can actually use. much more Hailed by the creator of IDA Pro as “profound, compre- • Use IDA’s built-in debugger to tackle hostile and C H R I S E A G L E PP hensive, and accurate,” the second edition of The IDA obfuscated code Pro Book covers everything from the very first steps to advanced automation techniques. You’ll find complete Whether you’re analyzing malware, conducting vulnerabil- RR ity research, or reverse engineering software, a mastery “I wholeheartedly recommend The coverage of IDA’s new Qt-based user interface, as of IDA Pro is crucial to your success. Take your skills to the IDA Pro Book to all IDA Pro users.” well as increased coverage of the IDA debugger, the OO next level with this 2nd edition of The IDA Pro Book. —Ilfak Guilfanov, Bochs debugger, and IDA scripting (especially using creator of IDA Pro IDAPython). But because humans are still smarter than ABOUT THE AUTHOR computers, you’ll even learn how to use IDA’s latest BB Chris Eagle is a Senior Lecturer of Computer Science interactive and scriptable interfaces to your advantage. at the Naval Postgraduate School in Monterey, CA. Save time and effort as you learn to: He is the author of many IDA plug-ins and co-author of OO • Navigate, comment, and modify disassembly Gray Hat Hacking (McGraw-Hill), and he has spoken • Identify known library routines, so you can focus your at numerous security conferences, including Blackhat, OO Defcon, Toorcon, and Shmoocon. analysis on other areas of the code KK J M THE FINEST IN GEEK ENTERTAINMENT™ www.nostarch.com E E A B G P $69.95($79.95CDN) L P This b o o k u“sIe sL aI Ela yF-LflAaTt .b”inding that won’t snap shut. SOFTWARE DEVELOPMPROGRAMMING/SHELVE IN: E B ENT U S PRAISE FOR THE FIRST EDITION OF THE IDA PRO BOOK “I wholeheartedly recommend The IDA Pro Book to all IDA Pro users.” —ILFAK GUILFANOV, CREATOR OF IDA PRO “A very concise, well laid out book. . . . The step by step examples, and much needed detail of all aspects of IDA alone make this book a good choice.” —CODY PIERCE, TIPPINGPOINT DVLABS “Chris Eagle is clearly an excellent educator, as he makes the sometimes very dense and technically involved material easy to read and understand and also chooses his examples well.” —DINO DAI ZOVI, TRAIL OF BITS BLOG “Provides a significantly better understanding not of just IDA Pro itself, but of the entire RE process.” —RYAN LINN, THE ETHICAL HACKER NETWORK “This book has no fluff or filler, it’s solid information!” —ERIC HULSE, CARNAL0WNAGE BLOG “The densest, most accurate, and, by far, the best IDA Pro book ever released.” —PIERRE VANDEVENNE, OWNER AND CEO OF DATARESCUE SA “I highly recommend this book to anyone, from the person looking to begin using IDA Pro to the seasoned veteran.” —DUSTIN D. TRAMMELL, SECURITY RESEARCHER “This book does definitely get a strong buy recommendation from me. It’s well written and it covers IDA Pro more comprehensively than any other written document I am aware of (including the actual IDA Pro Manual).” —SEBASTIAN PORST, SENIOR SOFTWARE SECURITY ENGINEER, MICROSOFT “Whether you need to solve a tough runtime defect or examine your application security from the inside out, IDA Pro is a great tool and this book is THE guide for coming up to speed.” —JOE STAGNER, PROGRAM MANAGER, MICROSOFT THE IDA PRO BOOK 2 N D E D I T I O N The Unofficial Guide to the World’s Most Popular Disassembler by Chris Eagle San Francisco THE IDA PRO BOOK, 2ND EDITION. Copyright © 2011 by Chris Eagle. All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. Printed in Canada 15 14 13 12 11 1 2 3 4 5 6 7 8 9 ISBN-10: 1-59327-289-8 ISBN-13: 978-1-59327-289-0 Publisher: William Pollock Production Editor: Alison Law Cover and Interior Design: Octopod Studios Developmental Editor: Tyler Ortman Technical Reviewer: Tim Vidas Copyeditor: Linda Recktenwald Compositor: Alison Law Proofreader: Paula L. Fleming Indexer: BIM Indexing & Proofreading Services For information on book distributors or translations, please contact No Starch Press, Inc. directly: No Starch Press, Inc. 38 Ringold Street, San Francisco, CA 94103 phone: 415.863.9900; fax: 415.863.9950; [email protected]; www.nostarch.com The Library of Congress has cataloged the first edition as follows: Eagle, Chris. The IDA Pro book : the unofficial guide to the world's most popular disassembler / Chris Eagle. p. cm. Includes bibliographical references and index. ISBN-13: 978-1-59327-178-7 ISBN-10: 1-59327-178-6 1. IDA Pro (Electronic resource) 2. Disassemblers (Computer programs) 3. Debugging in computer science. I. Title. QA76.76.D57E245 2008 005.1'4--dc22 2008030632 No Starch Press and the No Starch Press logo are registered trademarks of No Starch Press, Inc. Other product and company names mentioned herein may be the trademarks of their respective owners. Rather than use a trademark symbol with every occurrence of a trademarked name, we are using the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The information in this book is distributed on an “As Is” basis, without warranty. While every precaution has been taken in the preparation of this work, neither the author nor No Starch Press, Inc. shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in it. This book is dedicated to my mother. B R I E F C O N T E N T S Acknowledgments.........................................................................................................xix Introduction..................................................................................................................xxi PART I: INTRODUCTION TO IDA Chapter 1: Introduction to Disassembly..............................................................................3 Chapter 2: Reversing and DisassemblyTools....................................................................15 Chapter 3: IDA Pro Background......................................................................................31 PART II: BASIC IDA USAGE Chapter 4: Getting Started with IDA................................................................................43 Chapter 5: IDA Data Displays.........................................................................................59 Chapter 6: DisassemblyNavigation................................................................................79 Chapter 7: DisassemblyManipulation...........................................................................101 Chapter 8: Datatypes and DataStructures......................................................................127 Chapter 9: Cross-References andGraphing....................................................................167 Chapter 10: The Many Faces of IDA.............................................................................189 PART III: ADVANCED IDA USAGE Chapter 11: Customizing IDA.......................................................................................201 Chapter 12: Library Recognition Using FLIRT Signatures...................................................211 Chapter 13: Extending IDA’s Knowledge.......................................................................227 Chapter 14: Patching Binaries and Other IDA Limitations.................................................237 PART IV: EXTENDING IDA’S CAPABILITIES Chapter 15: IDA Scripting............................................................................................249 Chapter 16: The IDA Software Development Kit..............................................................285 Chapter 17: The IDA Plug-in Architecture.......................................................................315 Chapter 18: Binary Files and IDA LoaderModules..........................................................347 Chapter 19: IDA Processor Modules..............................................................................377 PART V: REAL-WORLDAPPLICATIONS Chapter 20: Compiler Personalities...............................................................................415 Chapter 21: Obfuscated Code Analysis.........................................................................433 Chapter 22: Vulnerability Analysis................................................................................475 Chapter 23: Real-World IDA Plug-ins.............................................................................499 PART VI: THE IDA DEBUGGER Chapter 24: The IDA Debugger....................................................................................513 Chapter 25: Disassembler/Debugger Integration............................................................539 Chapter 26: Additional Debugger Features....................................................................569 Appendix A: Using IDA Freeware 5.0...........................................................................581 Appendix B: IDC/SDK Cross-Reference..........................................................................585 Index.........................................................................................................................609 viii Brief Contents