The Institute for Health, Health Care Policy, and Aging Research Assessing Hospitals’ Use of State-Mandated Adverse Event Reporting Data Final Report to the National Patient Safety Foundation Kimberley Fox, M.P.A. Amy M. Tiedemann, Ph.D. Denise Davis Dr.P.H., M.P.A. Joel C. Cantor, Sc.D. Supported by a grant from the National Patient Safety Foundation June 2004 Assessing Hospitals’ Use of State-Mandated Adverse Event Reporting Data i ii Rutgers Center for State Health Policy, June 2004 Acknowledgements The authors of this paper thank the National Patient Safety Foundation (NPSF) for supporting this study. In particular, we would like to thank Asta Sorensen and Emily Wilson, our program managers at NPSF. We would like to extend our special thanks to our colleagues on our research advisory team – Dr Albert Siu, M.D., M.S.P.H, Professor of Medicine at Mount Sinai School of Medicine; Andrea Kabcenell, R.N., M.P.H., Deputy Director, Pursuing Perfection; David Frankford, LLD, Professor at Rutgers University School of Law and the Center for State Health Policy; and Kathy Ciccone, Vice President of Quality and Research Initiatives at the Healthcare Association of New York State (HANYS) for their insightful contributions on the research design, development of interview protocols, and their feedback on the final report. We also extend our thanks to Ellen Flink at the New York State Department of Health for providing background information on NYPORTS and to the Healthcare Association of New York State for providing hospital data as well as their endorsement of the study which helped facilitate hospital participation. At the Rutgers Center for State Health Policy, we would also like to thank Joann Donatiello, Information Specialist, who provided invaluable research assistance. Finally, this study would not have been possible without the cooperation of the hospital administrative and clinical leaders who took time out of their busy schedules to participate in our study. We are grateful to all of these individuals for agreeing to speak with us. Assessing Hospitals’ Use of State-Mandated Adverse Event Reporting Data iii iv Rutgers Center for State Health Policy, June 2004 Table of Contents Executive Summary...........................................................................................................vii Introduction.........................................................................................................................1 History and Description of NYPORTS.................................................................................2 Study Purpose......................................................................................................................3 Methods...............................................................................................................................4 Findings.............................................................................................................................12 Awareness...................................................................................................................12 Data Collection and Perceived Quality of Data Collected...........................................15 Analysis and Response – How Hospitals Use NYPORTS Data....................................20 Barriers to NYPORTS Reporting and Use....................................................................28 Perceived Effectiveness of NYPORTS in Improving Patient Safety............................35 Potential Improvements..............................................................................................38 Discussion.........................................................................................................................41 Endnotes............................................................................................................................47 Appendix A: Includes/Excludes List..................................................................................49 Appendix B: Hospital Interview Protocol..........................................................................61 Assessing Hospitals’ Use of State-Mandated Adverse Event Reporting Data v vi Rutgers Center for State Health Policy, June 2004 Assessing Hospitals’ Use of State-Mandated Adverse Event Reporting Data Kimberley Fox, M.P.A.; Amy M. Tiedemann, Ph.D.; Denise A. Davis, Dr.P.H.; Joel C. Cantor, Ph.D. EXECUTIVE SUMMARY State-mandated medical error reporting in hospitals and other health care facilities has become common since the release of the Institute of Medicine’s report To Err is Human in 2000. In 2002, Rutgers Center for State Health Policy conducted an exploratory study funded by the National Patient Safety Foundation to assess hospitals’ use of state-mandated medical error and adverse event data in New York State, which has one of the oldest and largest state- mandated hospital reporting systems in the country. Based on semi-structured telephone interviews with over 100 administrative and clinical leaders from a stratified random sample of 20 hospitals throughout New York State, the study investigates hospital leaders’ awareness and perceived purpose of the state-mandated reporting system, the process by which hospitals collect and use this data, the barriers to use, and perceived value by hospital leaders and its impact on patient safety. The study also sought to identify key factors that either facilitated or limited the use of data from the mandatory reporting system within New York State hospitals. This report presents the findings and highlights “best practices” for collecting and utilizing such data as well as barriers that may limit its usefulness. Key Findings Awareness and Purpose § Most hospital administrative and clinical leaders are familiar with NYPORTS to some extent. However, the degree of familiarity varies considerably across position within the hospital hierarchy. § Chiefs of surgery and pharmacy directors were least familiar with NYPORTS, despite the fact that a large number of NYPORTS-reportable incidents are surgery- related and pharmacy errors are a continuing area of concern. Assessing Hospitals’ Use of State-Mandated Adverse Event Reporting Data vii § Awareness of reporting less serious adverse events to the state was far lower than for major events that trigger a root cause analysis (RCA). Data Collection and Confidence in Data Reported § According to administrative and clinical leader respondents, nurses and case managers most commonly report NYPORTS cases. Physicians are much less likely to report due, in part, to fear of concerns about potential punitive actions by NYSDOH. § Three quarters of respondents were confident that most NYPORTS-reportable cases were being reported at their facility. However, many felt that the NYPORTS system failed to capture some important near-miss events that the hospital could equally learn from. In addition, respondents were less confident in the uniformity and comprehensiveness of reporting by other facilities. Analysis and Response § Most respondents indicated that they had participated in at least one root cause analysis (RCA) within the past year and nearly universally found the RCA process beneficial. § Most commonly, RCAs resulted in modifications to policies and procedures, and in-house training programs to re-educate nursing and resident staff. Only a few hospitals had made monetary investments in response to an RCA. § While many respondents understood that NYPORTS is intended to provide standardized data to compare performance with other hospitals, most hospital leaders were unaware that comparative reports could be generated from the NYPORTS online system. Those that were aware of them found them difficult to use. Nearly all respondents were familiar with aggregate annual state-level reports but felt that they were of minimal use. § Few hospitals disseminated information on NYPORTS to all staff at the hospital. Barriers to Reporting and Use § Despite efforts by facilities to move towards a non-punitive “systems” approach to addressing medical errors, the primary barrier to reporting cited by nearly all respondents was that reported errors would be used for punitive purposes either within the hospital or by the state. Physicians were identified as most resistant to reporting. viii Rutgers Center for State Health Policy, June 2004 § The large majority of hospital leaders, especially clinical leaders, were sympathetic to physicians’ concerns about external repercussions. The leaders believe that this concern is fueled in part by the mixed messages sent by the state including the requirement that physician license numbers be reported. § Most respondents felt that the state had made a concerted effort to clearly define what cases should be reported, although some respondents still were unclear about which events should be reported. § Many interviewees reported that hospitals could use more timely feedback from the state in the form of “best practices” and comparative error data. Perceived Impact of NYPORTS in Improving Patient Safety § The state mandate, while not always welcomed by these leaders, was generally credited with increased awareness and ‘raising the bar’ of accountability, which has helped leaders of quality departments garner resources. § Some felt that NYPORTS had a chilling effect on error identification due to the close affiliation of this process with the professional oversight by the Department of Health. § Most respondents found it difficult to identify a causal connection between modifications that may have resulted from a NYPORTS-related investigation and improved patient safety. § Most respondents felt that RCAs were very helpful in fostering a systems-focused culture to investigate errors. However, most of the changes made as a result of RCAs were changes in policies, procedures and in-service education, which are largely process versus systems modifications. Conclusion This study, the first of its kind, suggests that state-mandated hospital adverse event reporting in New York has been successful in raising awareness of patient safety among hospital leadership and promoting investigative processes of serious medical errors that hospitals have found to be useful. However, hospitals do not appear to be utilizing much of NYPORTS adverse event data because of insufficient comparative data feedback and lack of confidence in event reporting across hospitals. Although we set out to explore how hospital characteristics relate to NYPORTS use, we generally did not observe variations in patterns of use across hospital types. Our primary Assessing Hospitals’ Use of State-Mandated Adverse Event Reporting Data ix findings instead demonstrate the influence that one’s position in a hospital’s administrative and leadership structure has for perceptions of this adverse event reporting system. This study suggests that well-designed, state-mandated reporting systems can have positive impacts in raising awareness and accountability within hospitals, but also points to some barriers and burdens that designers of next-generation error reporting systems should address: § New York’s root cause analysis requirement for serious adverse events has been an important component of their state-mandated reporting system and appears to have had a positive impact on internal process improvements within hospitals. States that choose to mandate error reporting should require similar investigative processes. § Clear distinctions should be made between the state’s regulatory oversight function and adverse event reporting for patient safety improvement. § States need to dedicate more resources to providing timely feedback to hospitals and disseminating “best practices”. § It may not be possible to achieve comprehensive, consistent reporting of adverse events for comparative purposes. States should concentrate on mandating reporting for those events that result in the greatest harm (i.e. where root cause analyses are required). § Mandatory adverse event reporting may be insufficient to spur hospitals to invest in effective but costly means of error reduction, such as automated systems and greater staff resources. Direct financial incentives to support such interventions may be an important adjunct to mandatory reporting. x Rutgers Center for State Health Policy, June 2004