OX F O R D L I B R A RY O F P S Y C H O L O G Y  .  Editor-in-Chief  . .  Editor, Organizational Psychology Th e Oxford Handbook of Evidence-Based Management Edited by Denise M. Rousseau 1 1 Oxford University Press, Inc., publishes works that further Oxford University’s objective of excellence in research, scholarship, and education. Oxford New York Auckland Cape Town Dar es Salaam Hong Kong Karachi Kuala Lumpur Madrid Melbourne Mexico City Nairobi New Delhi Shanghai Taipei Toronto With offi ces in Argentina Austria Brazil Chile Czech Republic France Greece Guatemala Hungary Italy Japan Poland Portugal Singapore South Korea Switzerland Th ailand Turkey Ukraine Vietnam Copyright © 2012 by Oxford University Press, Inc. Published by Oxford University Press, Inc. 198 Madison Avenue, New York, New York 10016 www.oup.com Oxford is a registered trademark of Oxford University Press All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior permission of Oxford University Press Library of Congress Cataloging-in-Publication Data Th e Oxford handbook of evidence-based management / edited by Denise M. Rousseau. p. cm. — (Oxford library of psychology) ISBN-13: 978-0-19-976398-6 (acid-free paper) ISBN-10: 0-19-976398-4 (acid-free paper) 1. Decision making. 2. Business intelligence. 3. Management. I. Rousseau, Denise M. HD30.23.O955 2012 658.4'03—dc23 2011036150 9 8 7 6 5 4 3 2 1 Printed in the United States of America on acid-free paper FOREWORD We have a very long way to go to make management practice evidence-based. A few years ago, while serving on the compensation committee for a publicly traded NASDAQ company, we were considering what to do about the CEOs’ stock options and our stock-option program in general. Just that day, articles appeared in the mainstream business press on Donald Hambrick’s research showing that stock options led to risky behavior (Sanders & Hambrick, 2007). Th at research added to the growing body of evidence demonstrating that many executive pay practices not only did not enhance company performance (Dalton, Certo, & Roengpitya, 2003), but led instead to misreporting of fi nancial results (Burns & Kedia, 2006). Th e fi nding that options led to riskier actions is logical. Once an option is out of the money, there is no further economic downside for executives. Th erefore, there is every incentive for managers to take big risks in the hope that the stock price will go up, thereby giving those once previously worthless options economic value. At the meeting, a vice president from Aon Consulting who was advising the compensation committee replied, “No,” without any hesitation or embarrassment when I asked him, fi rst, if he knew about this research, and, second, if he was interested in my sending him the original articles or other information about the extensive research on stock options and their eff ects. What is particularly telling is that many people from other compensation consulting fi rms to whom I have related this story said it could have been their fi rm, too—that the perspective refl ected is typical. Meanwhile, my colleagues on the compensation committee seemed to believe that the fact that our professional advisor was both unknowl- edgeable about and, even worse, uninterested in sound empirical research that might inform our compensation policy decisions should not in any way aff ect our continued reliance on his “advice.” Th is consulting example is all too typical of the perspective of numerous, particularly U.S., executives, many of whom value their “experience” over data, even though much research suggests that learning from experience is quite diffi cult and much of such presumed learning is just wrong (e.g., Denrell, 2003; Mark & Mellor, 1991; Schkade & Kilbourne, 2004). Yes, it is true that there is much recent interest in investing in “big data” start- ups and technology and in building software to comb through large data sets for analytical insights, such as how to improve margins through pricing optimiza- tion. However, as Jim Goodnight, the CEO of SAS Institute, a business analytics and business intelligence software company, recently told me, that analytic work is mostly done by professionals fairly far down in the company hierarchy who sometimes have trouble getting the CEO interested in implementing their results. Few companies or their leaders seem to see that information and data-driven vii business insight can be a competitive advantage. Although it may seem ironic that companies invest in analytical capabilities and software that they don’t fully use, such a circumstance is far from unusual. David Larcker, a well-known cost- accounting professor, has said that few companies turn all the data they have collected in their enterprise resource-planning systems into business intelligence but instead use most of that information simply for purposes of control. And purveyors of marketing research also tell a similar tale: companies purchase their services but then don’t act on the results when the recommendations confl ict with the preexisting views of senior management. Th e problems confronting building evidence-based practice in management are scarcely unique to this domain. It has taken more than 200 years for evidence- based medicine to become a normative standard for professional practice. Th e objections to applying standards of practice—protocols—in medicine are eerily similar to those encountered in advocating evidence-based management: that each situation is diff erent; that practitioners’ wisdom, experience, and insight is more valuable than aggregated data-based information; and that statistical evidence pertains to what happens “on average,” but that the particular individual speak- ing is, of course, much above average and not, therefore, bound by the results of aggregate information—a phenomenon reliably observed in psychology and called the above-average eff ect (e.g., Williams & Gilovich, 2008). Although there are evidence-based movements in various policy domains ranging from crimi- nology to education, resistance to implementing science, particularly when that science confl icts with belief and ideology, looms large. Failure to adhere to clinical guidelines in medicine, unfortunately still quite com- mon, costs lives and money (e.g., Berry, Murdoch, & McMurray, 2001). Pressures for cost containment and the documented instances of literally hundreds of thousands of lives lost from preventable medical errors have seemingly turned the tide so that evidence-based medical practice now seems to be inevitable. However, the absence of similar pressures in management means that the quest to bring evidence-based practice into organizations still faces a long and tortuous journey. In the meantime, damage occurs to both people and companies. According to Conference Board sur- veys, job satisfaction is at an all-time low; most studies of employee engagement and distrust of management portray a dismal picture of the typical workplace; and the failure to heed the evidence about the physiological eff ects of workplace stress, eco- nomic insecurity, and long working hours contributes to employee mortality and morbidity (Pfeff er, 2010). Companies suff er from having disengaged workforces with excessive turnover even as they neglect the science and information that could improve their operations and profi tability (e.g., Burchell & Robin, 2011). Simply put, there is a profound “doing-knowing” problem in management practice: many managers make decisions and take actions with little or no con- sideration of the knowledge base that might inform those decisions. Th e Oxford Handbook of Evidence-Based Management intends to change this situation. It con- tains more than just a review of the literature on evidence-based management (EBMgt). Th e Handbook recognizes the need to provide information on how to teach EBMgt in classes and executive programs and also the requirement to viii  provide role models—illustrations of practitioners who have embraced an evidence-based approach—so that practicing leaders can see how evidence-based principles might be applied in real organizations. Because EBMgt is relevant in both the for-profi t and nonprofi t worlds, the handbook has examples from both. With its focus on changing the practice of management, this work exam- ines the important topic of how to get research fi ndings implemented in practice and the barriers to connecting science with practice. Th e Oxford Handbook of Evidence-Based Management provides a comprehensive overview of the informa- tion required to understand what EBMgt is, how to teach it, how to apply it, and how to understand and overcome the barriers that stand in the way of basing management practice on the best relevant research. It represents an important step on the road to building an EBMgt movement. It is scarcely news that management is not a profession, even though it might and should be. Th ere are two elements that defi ne a profession, one of which is the development and adherence to specialized bodies of knowledge. Sometimes specialized knowledge is refl ected in licensing examinations. However, licensing is not the only path to ensuring that people know and implement knowledge and standards of professional practice. What may be even more important are social norms and public expectations and the sanctions for violating those expectations. As the opening example—and hundreds of others—make clear, there is, at the moment, neither the expectation that professionally educated practitioners will know relevant management research nor any sanctions that punish their igno- rance. Consequently, management, and indeed, much of the popular manage- ment literature, is beset with myths, dangerous half-truths (Pfeff er & Sutton, 2006), and rules of thumb often based on little more than publicity, repetition, and belief. Organizations and their leaders do a profound disservice to society and even to themselves by not being more committed—not just in rhetoric but in their behavior—to implementing management science. In this project of professionalization, the role of educational institutions looms large, which means that it is important that the handbook includes so much mate- rial on teaching EBMgt and on linking science with practice. At the moment, however, there is little to suggest that even in business-school classrooms, much attention is paid to EBMgt. Business schools are increasingly staff ed by part-time and adjunct faculty who have no requirement to know, let alone contribute, to the science of management. Courses often use case examples as a primary peda- gogic focus, and relatively few business schools impart the critical thinking and evaluation of ideas and the skills necessary to separate good management science from quackery. A recent review of more than 800 syllabi of required courses from some 333 programs found that only about one-quarter utilized scientifi c evidence in any form (Charlier, Brown, & Rynes, 2011). Th e fact that even business school instructors often inhabit the world of folk wisdom rather than the domain of sci- ence has sparked notice and commentary (e.g., Pearce, 2004). If EBMgt is to become a reality in professional practice, business schools at both the undergraduate and MBA levels will need to play an important role. Th at’s because it is in school where students not only begin to learn about the  ix relevant research and theory but also come to understand standards of science and what constitutes evidence and the sound basis for making decisions. It is inter- esting that business schools are currently evaluated in many ways—the extent to which they raise their graduates’ salaries, how “satisfi ed” they make their students and the companies that recruit them, their adherence to guidelines that specify what they must teach and the proportion of their faculty with terminal degrees, their reputations as perceived by their peers, and various other criteria. What is not measured is the extent to which they impart the science of management and critical-thinking skills to their graduates, or the extent to which the graduates actually practice EBMgt. If we are serious about implementing a science of man- agement, we need to measure and evaluate both schools and their graduates, at least to some degree, using these metrics. We live in an organizational world. Decisions made in and by organizations profoundly aff ect the working lives and economic well-being of people all over the globe. As the fi nancial crisis of 2007–2008 well illustrates, many of those decisions are all too frequently characterized not just by venality but also by profound incom- petence (Lewis, 2010). As scholars and as practitioners whose work aff ects so many lives in so many ways, we have a sacred obligation and responsibility to develop and use the best knowledge possible to make the world a better place. It is only through embarking on the goal of building an evidence-based body of knowledge linked to professional practice that this obligation can be fulfi lled. In that regard, Th e Oxford Handbook of Evidence-Based Management makes both a profoundly important con- tribution and lays out a statement of intent to make EBMgt a reality. —Jeff rey Pfeff er References Berry, C., Murdoch, D. R., & McMurray, J. J. V. (2001). Economics of chronic heart failure, European Journal of Heart Failure, 3, 283–291. Burchell, M., & Robin, J. (2011). Th e great workplace: How to build it, how to keep it, and why it matters. San Francisco, CA: Jossey-Bass. Burns, N., & Kedia, S. (2006). Th e impact of performance-based compensation on misreporting, Journal of Financial Economics, 79, 35–67. Charlier, S. D., Brown, K. G., & Rynes, S. L. (2011). Teaching evidence-based management in MBA programs: What evidence is there? Academy of Management Learning and Education, 10, 222–236. Dalton, D. R., Certo, S. T., & Roengpitya, R. (2003). Meta-analyses of fi nancial performance: Fusion or confu- sion? Academy of Management Journal, 46, 13–28. Denrell, J. (2003). Vicarious learning, undersampling of failure, and the myths of management, Organization Science, 14, 227–243. Lewis, M. (2010). Th e Big Short: Inside the Doomsday Machine, New York: W. W. Norton. Mark, M. M., & Mellor, S. (1991). Eff ect of self-relevance on hindsight bias: Th e foreseeability of a layoff , Journal of Applied Psychology, 76, 569–577. Pearce, J. (2004). What do we know and how do we really know it? Academy of Management Review, 29, 175–179. Pfeff er, J. (2010). Building sustainable organizations: Th e human factor, Academy of Management Perspectives, 24, 34–45. Pfeff er, J., & Sutton, R. I. (2006). Hard facts, dangerous half-truths, and total nonsense: profi ting from evidence- based management. Boston: Harvard Business School Press, 2006. Sanders, W. M., & Hambrick, D. C. (2007). Swinging for the fences: Th e eff ects of CEO stock options on company risk-taking and performance, Academy of Management Journal, 50, 1055–1078. Schkade, D. A., & Kilbourne, L. M. (2004). Expectation-outcome consistency and hindsight bias, Organizational Behavior and Human Decision Processes, 49, 105–123. Williams, E. F., & Gilovich, T. (2008). Do people really believe they are above average? Journal of Experimental Social Psychology, 44, 1121–1128. x  PREFACE What fi eld is this? • “Th ere is a large research-user gap.” • “Many practices are doing more harm than good.” • “Practitioners do not read academic journals.” • “Academics not practitioners are driving the research agenda.” • “Practice is being driven more by fads and fashions than research.” Th e fact is, these quotes all come from the early days of evidence-based medi- cine (Sackett et al., 2000). More recently the same comments have been about the (lack of) use of scientifi c evidence in management practice; no wonder, because management practice today is as poorly aligned with organization and manage- ment research as medicine and science were then. Scientifi c knowledge identi- fi es what we know and what we don’t know about the natural and human-made world. It is the bedrock of all evidence-based approaches to practice. From med- icine to education, each area of evidence-based practice has needed to confront these criticisms and work to make such comments descriptions of things past. Evidence-based management (EBMgt) is the science-informed practice of man- agement. It’s about using scientifi c knowledge to inform the judgment of manag- ers and the nature, content, and processes of decision making in organizations. Its practice reduces the costs of unaided judgment and limited and biased human information processes. EBMgt allows managers, consultants, and other practitio- ners to overcome these impediments through use of decision aids, practices, and frameworks to support better quality decisions. Th e notion of evidence-based practice is relatively new, but the vision of man- agement informed by science is not. Mary Parker Follett a renowned management consultant in the early twentieth century was perhaps the fi rst to recognize the power of social science to inform management. Much of her infl uence today has been channeled through the writings of Peter Drucker, her self-proclaimed disci- ple, who advised practitioners to pay attention to scientifi c research and to pursue feedback on the outcomes of their decisions more deliberately in order to learn and perform better. Herbert Simon, the quintessential scientist and founder of such fi elds as computer science, artifi cial intelligence, and robotics, identifi ed the limits of unaided human decision making as well as the potential for science and practice to partner in designing preferred conditions for our human-made world. As we shall explore in this handbook, EBMgt is an adaptive evidence-informed family of practices relevant to anyone seeking to improve how people and organi- zations are managed. Th is handbook is intended to promote EBMgt’s broad use in for-profi t businesses, nonprofi t organizations, and government. Management means getting people together to accomplish some objective. Th is handbook is xxiii intended for anyone in an organization who steps up to make things happen by organizing people, tasks, and processes. Th at includes executives, offi cers, and department heads as well as by proactive workers and volunteers. Th roughout history, management has refl ected fundamental beliefs about what people can and should control in organizations. Manufacturing fi rms had no des- ignated workplace safety function in the fi rst 100 years after the industrial revo- lution began. Accidents were viewed as an act of God and, hence, could not be reduced or managed. Periodically, through scientifi c and practice innovations, such beliefs have changed. In its day, the notion of a planful enterprise man- ager was a Utopian view, scarcely evident in the loosely tied small job shops that comprised pre-industrial age shipbuilding or textile manufacturing. Practices pro- moting effi ciencies in organizations became management’s job over subsequent centuries, and by the twentieth century its functions included innovation and marketing (Drucker, 2003). Th rough all these developments, the habits of mind and knowledge used in the practice of management also changed. Th e emergence of EBMgt in the twenty-fi rst century parallels in some ways the systematization of fi nance and accounting in the last century. Management concepts have evolved over time and with them the mental models of man- agers and other practitioners. Nineteenth-century organizations for the most part lacked fundamental concepts like the time-value of money. In the early twentieth century, the concept of return on equity was developed and its com- ponents explicated in order to better assess and analyze company performance. Finance and accounting today have well-specifi ed logics that lead to wider con- sistency in their practice and shared understandings of their meaning. In con- trast, the logics that executives, managers, human-resource staff , and others use in managing people, structuring organizations, and making strategic decisions are far more inconsistent and ad hoc (Boudreau, 2010). EBMgt off ers evidence- informed logics for eff ective management practice that complement other business disciplines. Why Now? Drivers of Evidence-Based Management We can never understand the total situation without taking into account the evolving situation. And when a situation changes we have not a new variation under the old fact, but a new fact. —Mary Parker Follett Th e time is ripe for the emergence of EBMgt. A confl uence of factors (in italics in the following sentence and the following paragraphs) provides unparalleled opportunity to reconsider the fundamentals of managing organizations. Since World War II, a large body of social science and management research has investigated the individual, social, and organizational factors that impact managerial performance and organizational eff ectiveness. It has produced hundreds of well-supported evi- dence-based principles relevant to organizational decisions and practices. Aware- ness of the large volumes of research relevant to real-world decisions is a major factor in the current zeitgeist of evidence-based practice across innumerable pro- fessions (e.g., Armstrong, 2010; Locke, 2009). xxiv  Th e Internet off ers broad access to scientifi c knowledge. From electronic libraries to listservs, the Internet off ers fi ndings from organizational and management science and opportunities to participate in communities interested in applying them. Medicine and nursing, among other fi elds, have undergone their own Internet- based transformations toward evidence-based practice. Using online sources, phy- sicians and nurses regularly fi nd information about problems and decisions they face. Th e result has been science-informed patient-care protocols, guidelines, and procedures, turning clinical research into better patient outcomes. In our fi eld, management research is becoming widely accessible through Google.scholar and other web-based resources. Broadly available management research and summa- ries provide the basis for more eff ective mental models, processes, and practices. Th ese make it easier for managers to apply the products of scientifi c knowledge daily in organizations. Increasing awareness of the consequences from managerial decisions prompts widespread concern with improving its quality. Advancements in human knowledge give people new resources for tackling what once appeared to be intractable prob- lems. Schools and criminal-justice institutions have been early adopters of evidence- based practice, perhaps no surprise given that their reliance on public dollars can require greater accountability (Orszag, 2010). Similarly, health-care managers are being challenged to use the same evidential approach that their clinical practitio- ners are using for patient care (Walshe & Randall, 2001). Th e movement toward accountability is likely to grow, fueled by widely available information regarding consequential decisions made with increasing global interdependence. An evidence-based practice zeitgeist results from these forces across fi elds as diverse as medicine, education, criminal justice, and public policy. Each of these evidence initiatives refl ects a recognition that the well-being of people, organiza- tions, and society in an increasingly vulnerable and interdependent world may depend on how eff ectively we act on evidence. In the context of management, the potential benefi ts from evidence use are no less. Th is Handbook’s Organization Th is handbook is designed with three goals in mind. First, it provides an overview of key EBMgt concepts and puts them in context of broader eff orts promot- ing evidence-based practice and the closing of research-practice gaps. Second, it addresses the roles, contributions, and concerns of EBMgt’s three core constitu- ents: practitioners, educators, and scholars, providing perspectives and resources for each. Th ird, it incorporates critical perspectives to raise awareness of alterna- tive views and possible unintended consequences and to stimulate future EBMgt innovations. EBMgt’s development, adoption, and future progress depend on the related eff orts of managers, consultants, and other practitioners who perform EBMgt; the educators who help develop the professional skills on which EBMgt rests; and the scholars who provide the basic research, summaries, and partner- ships with practitioners that make EBMgt possible. From health care to education, the many fi elds that now base core practices on research fi ndings all began with the early adoption of evidence-informed practices  xxv by a small percentage of professionals (Rogers, 1995; Proctor, 2004). For this reason, this handbook targets the early adopters in all three groups, that is, those self-improving practitioners, educators, and scholars interested in learning and innovating. Th e handbook is intended to inform all three communities, off ering perspective, food for thought, and guides to action. Th e Introductory section provides an overview of EBMgt and its facets and puts these in context. I fi rst present its central elements and functions in my chapter on envisioning EBMgt. Th en, because evidence-based practice began in the fi eld of medicine, Barends, ten Have, and Huisman compare and contrast EBMgt and its medical counterpart, addressing the commonalities as well as false beliefs and misapprehensions regarding their association. Th e introductory sec- tion concludes with van Aken and Romme’s chapter on design science, a broadly applicable process originally developed by Herbert Simon (1997) that enables the use of scientifi c knowledge to solve practical problems through the collaborations between practitioners and scientists. Th e Research section provides both resources and tools relevant to the role of scholars and academic research in promoting EBMgt. Its fi rst set of chapters pre- sent core domains of management and organizational research, addressing what we know now and what we can do in future to advance each domain’s uptake and use in EBMgt. “Micro” organizational behavior (OB), which I describe, is perhaps the most mature research fi eld with a 100+ year history and consider- able body of cumulative research. OB has yielded a host of evidence-based prin- ciples to guide practice. “Macro” organizational theory and strategy, as Madhavan and Mahoney detail, is a more recent fi eld of study characterized by disciplinary disputes regarding the value of novelty versus cumulative research. Th ey call for attention by macroscholars to moving this fi eld toward greater integration and synthesis. Lastly, Frese, Bausch, Schmidt, Rauch, and Kabst describe the fi eld of entrepreneurship, which has fostered cumulative research on critical issues related to creating and managing start-ups and developing entrepreneurial capabilities. Other Research chapters describe techniques and approaches to help make research more accessible and useful to practice. A core aspect of all evidence- based practice is systematic reviews (SRs). Th ese are summaries of a comprehen- sive body of research undertaken to address a specifi c managerial (and sometimes, too, scholarly) question. Briner and Denyer describe how SRs are conducted and their role in making useful knowledge more readily available. Th e next two chap- ters provide insights into how researchers can make their work more informative and useful to practice. Giluk and Rynes present lessons learned from evidence- based medicine to help us appreciate why practitioners might not believe, accept, or act on the best available evidence, and how to overcome these barriers. Leung and Bartunek address how scholarly research can be more eff ectively presented so that practitioners fi nd it more memorable and easy to use. Th e Practice section targets the realities of everyday management practice and the role EBMgt can play in improving decision making and well-being of employ- ees and organizations. Practicing EBMgt involves overcoming a variety of hurdles. A fi rst set of Practice chapters provides resources for understanding practice issues. xxvi 