Performance Evaluation and Benchmarking Performance Evaluation and Benchmarking Edited by Lizy Kurian John Lieven Eeckhout Boca Raton London New York A CRC title, part of the Taylor & Francis imprint, a member of the Taylor & Francis Group, the academic division of T&F Informa plc. 3622_Discl.fm Page 1 Friday, August 5, 2005 8:59 AM Published in 2006 by CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2006 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group No claim to original U.S. Government works Printed in the United States of America on acid-free paper 10 9 8 7 6 5 4 3 2 1 International Standard Book Number-10: 0-8493-3622-8 (Hardcover) International Standard Book Number-13: 978-0-8493-3622-5 (Hardcover) Library of Congress Card Number 2005047021 This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish reliable data and information, but the author and the publisher cannot assume responsibility for the validity of all materials or for the consequences of their use. No part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC) 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data John, Lizy Kurian. Performance evaluation and benchmarking / Lizy Kurian John and Lieven Eeckhout. p. cm. Includes bibliographical references and index. ISBN 0-8493-3622-8 (alk. paper) 1. Electronic digital computers--Evaluation. I. Eeckhout, Lieven. II. Title. QA76.9.E94J64 2005 004.2'4--dc22 2005047021 Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com Taylor & Francis Group and the CRC Press Web site at is the Academic Division of T&F Informa plc. http://www.crcpress.com Preface It is a real pleasure and honor for us to present you this book titled Perfor- mance Evaluation and Benchmarking. Performance evaluation and benchmark- ing is at the heart of computer architecture research and development. With- out a deep understanding of benchmarks’ behavior on a microprocessor and without efficient and accurate performance evaluation techniques, it is impossible to design next-generation microprocessors. Because this research field is growing and has gained interest and importance over the last few years, we thought it would be appropriate to collect a number of these important recent advances in the field into a research book. This book deals with a large variety of state-of-the-art performance evaluation and bench- marking techniques. The subjects in this book range from simulation models to real hardware performance evaluation, from analytical modeling to fast simulation techniques and detailed simulation models, from single-number performance measurements to the use of statistics for dealing with large data sets, from existing benchmark suites to the conception of representative benchmark suites, from program analysis and workload characterization to its impact on performance evaluation, and other interesting topics. We expect it to be useful to graduate students in computer architecture and to computer architects and designers in the industry. This book was not entirely written by us. We invited several leading experts in the field to write a chapter on their recent research efforts in the field of performance evaluation and benchmarking. We would like to thank Prof. David J. Lilja from the University of Minnesota, Prof. Tom Conte from North Carolina State University, Prof. Brad Calder from the University of California San Diego, Prof. Chita Das from Penn State, Prof. Brinkley Sprunt from Bucknell University, Alex Mericas from IBM, and Dr. Kishore Menezes from Intel Corporation for accepting our invitation. We thank them and their co-authors for contributing. Special thanks to Dr. Joshua J. Yi from Freescale Semiconductor Inc., Paul D. Bryan from North Carolina State University, Erez Perelman from the University of California San Diego, Prof. Timothy Sherwood from the University of California at Santa Barbara, Prof. Greg Hamerly from Baylor University, Prof. Eun Jung Kim from Texas A&M University, Prof. Ki HwanYum from the University of Texas at San Antonio, Dr. Rumi Zahir from Intel Corporation, and Dr. Susith Fernando from Intel Corporation for contributing. Many authors went beyond their call to adjust their chapters according to the other chapters. Without their hard work, it would have been impossible to create this book. We hope you will enjoy reading this book. Prof. L. K. John The University of Texas at Austin, USA Dr. L. Eeckhout Ghent University, Belgium Editors Lizy Kurian John is an associate professor and Engineering Foundation Centennial Teaching Fellow in the electrical and computer engineering department at the University of Texas at Austin. She received her Ph.D. in computer engineering from Pennsylvania State University in 1993. She joined the faculty at the University of Texas at Austin in fall 1996. She was on the faculty at University of South Florida, from 1993 to 1996. Her current research interests are computer architecture, high-performance microproces- sors and computer systems, high-performance memory systems, workload characterization, performance evaluation, compiler optimization techniques, reconfigurable computer architectures, and similar topics. She has received several awards including the 2004 Texas Exes teaching award, the 2001 UT Austin Engineering Foundation Faculty award, the 1999 Halliburton Young Faculty award, and the NSF CAREER award. She is a member of IEEE, IEEE Computer Society, ACM, and ACM SIGARCH. She is also a member of Eta Kappa Nu, Tau Beta Pi, and Phi Kappa Phi Honor Societies. Lieven Eeckhout obtained his master’s and Ph.D degrees in computer sci- ence and engineering from Ghent University in Belgium in 1998 and 2002, respectively. He is currently working as a postdoctoral researcher at the same university through a grant from the Fund for Scientific Research—Flanders (FWO Vlaanderen). His research interests include computer architecture, performance evaluation, and workload characterization. Contributors Paul D. Bryan is a research assistant in the TINKER group, Center for Embedded Systems Research, North Carolina State University. He received his B.S. and M.S. degrees in computer engineering from North Carolina State University in 2002 and 2003, respectively. In addition to his academic work, he also worked as an engineer in the IBM PowerPC Embedded Processor Solutions group from 1999 to 2003. Brad Calder is a professor of computer science and engineering at the Uni- versity of California at San Diego. He co-founded the International Sympo- sium on Code Generation and Optimization (CGO) and the ACM Transac- tions on Architecture and Code Optimization (TACO). Brad Calder received his Ph.D. in computer science from the University of Colorado at Boulder in 1995. He obtained a B.S. in computer science and a B.S. in mathematics from the University of Washington in 1991. He is a recipient of an NSF CAREER Award. Thomas M. Conte is professor of electrical and computer engineering and director for the Center for Embedded Systems Research at North Carolina State University. He received his M.S. and Ph.D. degrees in electrical engi- neering from the University of Illinois at Urbana-Champaign in 1988 and 1992, respectively. In addition to academia, he’s consulted for numerous companies, including AT&T, IBM, SGI, and Qualcomm, and spent some time in industry as the chief microarchitect of DSP vendor BOPS, Inc. Conte is chair of the IEEE Computer Society Technical Committee on Microprogram- ming and Microarchitecture (TC-uARCH) as well as a fellow of the IEEE. Chita R. Das received the M.Sc. degree in electrical engineering from the Regional Engineering College, Rourkela, India, in 1981, and the Ph.D. degree in computer science from the Center for Advanced Computer Studies at the University of Louisiana at Lafayette in 1986. Since 1986, he has been working at Pennsylvania State University, where he is currently a professor in the Department of Computer Science and Engineering. His main areas of interest are parallel and distributed computer architectures, cluster systems, com- munication networks, resource management in parallel systems, mobile computing, performance evaluation, and fault-tolerant computing. He has