National Institute on Drug Abuse RESEARCH MONOGRAPH SERIES Drug Abuse Prevention Intervention Research: Methodological Issues 107 U.S. Department of Health and Human Services • Public Health Service • National Institutes of Health Drug Abuse Prevention Intervention Research: Methodological Issues Editors: Carl G. Leukefeld, D.S.W. William J. Bukoski, Ph.D. NIDA Research Monograph 107 1991 U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES Public Health Service Alcohol, Drug Abuse, and Mental Health Administration National Institute on Drug Abuse 5600 Fishers Lane Rockville, MD 20857 For sale by the Superintendent of Documents. U.S. Government Printing Office Washington, DC 20402 ACKNOWLEDGMENT This monograph is based on the papers and discussion from a technical review on “Drug Abuse Prevention intervention Research: Methodological Issues,” held on May 22-23, 1989, in Rockville, MD. The review meeting was sponsored by the National Institute on Drug Abuse. COPYRIGHT STATUS The National Institute on Drug Abuse has obtained permission from the copyright holders to reproduce certain previously published material as noted in the text. Further reproduction of this copyrighted material is permitted only as part of a reprinting of the entire publication or chapter. For any other use, the copyright holder’s permission is required. All other material in this volume except quoted passages from copyrighted sources is in the public domain and may be used or reproduced without permission from the Institute or the authors. Citation of the source is appreciated. Opinions expressed in this volume are those of the authors and do not necessarily reflect the opinions or official policy of the National Institute on Drug Abuse or any other part of the U.S. Department of Health and Human Services. The U.S. Government does not endorse or favor any specific commercial product or company. Trade, proprietary, or company names appearing in this publication are used only because they are considered essential in the context of the studies reported herein. NIDA Research Monographs are indexed in the Index Medicus. They are selectively included in the coverage of American Statistics Index, Biosciences Information Service, Chemical Abstracts, Current Contents, Psychological Abstracts, and Psychopharmacology Abstracts. DHHS publication number (ADM)91-1761 Printed 1991 ii Contents Page An Introduction to Drug Abuse Prevention Intervention Research: Methodological Issues 1 Carl G. Leukefeld and Wiilliam J. Bukoski A Framework for Drug Abuse Prevention Research 7 William J. Bukoski Prevention Intervention Research: Challenges and Opportunities 29 Richard R. Clayton and Anne Cattarello Contributions of Drug Epidemiology to the Field of Drug Abuse Prevention 57 Lloyd D. Johnston Methodological Issues in Drug Use Prevention Research: Theoretical Foundations 81 Brian R. Flay and John Petraitis Defining the Intervention and the Target Population 110 Lewayne D. Gilchrist Implementation Issues in Drug Abuse Prevention Research 123 Mary Ann Pentz and Elizabeth Trebow Experimental and Quasi-Experimental Designs in Prevention Research 140 David L. Snow and Jacob Kraemer Tebes iii Modeling of Intervention Effects 159 Peter M. Bentler Outcome Measurement Issues in Drug Abuse Prevention Studies 183 James H. Dwyer and David P. MacKinnon Assessing Effectiveness of Drug Abuse Prevention: Implementation Issues Relevant to Long-Term Effects and Replication 195 J. David Hawkins, Robert Abbott, Richard F. Catalano, and Mary R. Gillmore Subject Attrition in Prevention Research 213 Anthony Biglan, Donald Hood, Paul Brozovsky, Linda Ochs, Dennis Ary, and Carol Black Increasing the Validity of Self-Report Data in Effectiveness Trials Susan G. Forman and Jean Ann Linney Technology Transfer 248 Steven P. Schinke and Mario A. Orlandi Prevention Evaluation Research Methods: Findings 264 and Consensus Carl G. Leukefeld and William J. Bukoski List of NIDA Research Monographs 272 iv An Introduction to Drug Abuse Prevention Intervention Research: Methodological Issues Carl G. Leukefeld and William J. Bukoski INTRODUCTION With the renewed emphasis on drug abuse prevention, questions now are being asked about the effectiveness of those prevention interventions. Responses to these questions clearly suggest that drug abuse prevention interventions have been inconsistent in changing drug abuse and related behaviors. It has been suggested that more is known about what does not work than about what works in preventing drug abuse (Berberian et al. 1976; Goodstadt 1974; Schaps et al. 1981). Nevertheless, much is known about drug abuse prevention, and there are promising approaches (Donohew et al., in press: Glynn et al. 1983). Most questions related to the effectiveness of drug abuse prevention interventions center on research design and methodology and on the differences as well as the inconsistencies among study findings. An example is the choice of outcome measures (e.g., no drug use as contrasted with occasional drug use), which significantly affects a study’s findings and consequently a study’s importance. Additional methodological issues are important to the prevention practitioner and the researcher; these issues are the basis of this volume. Moreover, there are questions and discussion about the limitations of drug abuse prevention evaluation research and prevention evaluations in general (Biglan and Ary 1985; Leukefeld, in press) as well as recommendations that new research methodologies be developed to better understand prevention interventions. Some suggest that drug abuse prevention researchers talk more frequently with prevention practitioners about their expectations, opinions, and anecdotal experiences related to prevention program effectiveness. Drug abuse prevention has been controversial; consequently, drug abuse prevention research is a part of that controversy. Swisher (1979) identified the following controversial issues: evidence that prevention makes a difference; 1 difficulty in agreeing on how to demonstrate the effectiveness of prevention strategies; confusion regarding the differences among treatment, intervention, and prevention efforts; and concern about the purpose of prevention—ultimate use or nonuse by the target population. Silverman (in press) confronts this issue by suggesting that prevention research is evolving and will continue to develop as a direct consequence of more complex theoretical and conceptual thinking, more valid and reliable measures of drug-related problems, better understanding of individual risk factors, better identification of individuals and groups at high risk, better research design and long-term followup studies, and better integration across various settings—family, school, community, religious, and criminal justice. With this brief background and the overriding goal of providing greater clarity to research findings, the purpose of this monograph is to examine the state of the art of drug abuse prevention research methodology, to develop recommendations for refining current methodological approaches, and to develop an agenda for future research applications. Authors were asked to emphasize instrumentation, control/comparison groups, intervention specificity, clarification of outcome variables, replication issues, and measurement of long- term effects. Although this monograph does not review drug abuse prevention effectiveness research, chapter authors were asked to present data and research findings as examples of methodological issues. DRUG ABUSE PREVENTION RESEARCH Defining prevention is a first step in exploring drug abuse prevention methodology. Bukoski (in press) identifies three approaches or perspectives for drug abuse prevention. First, the public health model incorporates the concepts of primary, secondary, and tertiary prevention (Last 1980). Primary prevention is directed to preventing the onset of disease, including decreasing the incidence, new start, or onset. Many drug abuse prevention activities can be placed in this category of primary prevention. A common criticism of this conceptualization of prevention is the overlap between categories as well as the fact that all prevention, treatment, and rehabilitation services could be categorized within this definition of prevention. A second prevention conceptualization, the communicable disease model, focuses on the host, agent, and environment (Wilner et al. 1978). (The agent is the germ, virus, or other cause of a disease.) The host relates to the human susceptibility or resistance to disease and can be influenced by many hereditary and lifestyle factors. Environment refers to social or physical factors that may contribute to the initiation and spread of a disease. 2 A third conceptualization is the risk factor model, which is directed to identifying psychological, social, and biologic factors related to the emergence of a health problem (Arnold et al. 1981). Risk factors have been used extensively to depict increased risks for drug abuse using correlation research and other research findings. From one point of view these three ways of thinking provide some clarity about prevention, but from another point of view they also generate confusion because they are not completely compatible. Defining prevention research has not always been clear. Bukoski (1980) adds clarity with a description of a drug abuse prevention research evaluation model that includes three levels of evaluation: (1) process evaluation, which focuses on assessing the service operation of a prevention program and includes descriptions of the program’s prevention services, use of resources, and costs; (2) outcome evaluation, which is used to determine if a prevention program’s objectives were met by applying comparative evaluation designs; and (3) impact evaluation, which is used to assess macroindicators of drug abuse at the community level. The outcome evaluation design is probably the most frequently used design for evaluating drug abuse prevention. This controlled and comparative design of two or more groups is reviewed by Snow and Tebes in this volume. THE IMPORTANCE OF RESEARCH METHODOLOGY This volume is organized into five areas: introduction, documenting the prevention intervention, measuring the efficacy of prevention interventions, assessing effectiveness, and consensus development. Clayton and Cattarello provide an overview of drug abuse prevention research and discuss several methodological issues from their current research, which focus on assessing the Kentucky drug abuse resistance education prevention program. Johnston reviews drug trends among senior high school students and presents an overview of prevention impact in the United States. In the section on documenting prevention intervention, Flay and Petraitas provide a rationale and argue for the need to base prevention intervention programs on a strong theoretical foundation. History suggests that many prevention interventions, focused on drug abuse and other health promotion areas, have skipped this important part of documenting the intervention. Gilchrist discusses the parameters for defining a prevention intervention and for delineating the intervention target audience. Defining the prevention intervention as the intervening variable is not only essential for program replication but also is important for process evaluation and for training intervention staff. Specifying the target audience is also important in understanding the limitations of the intervention and the anticipated outcomes. 3 Pentz and Trebow present the final chapter in this section by reporting on issues related to program implementation. The implementation of drug abuse prevention program interventions is influenced by many environmental variables, which along with other factors make drug abuse prevention intervention research interesting and dynamic. The third section focuses on measuring the efficacy of prevention interventions. Snow and Tebes provide a review of experimental and quasi-experimental research designs that can be used in prevention intervention studies. Validity, basic threats to validity, and tradeoffs are examined. Bentler presents an overview of modeling and measurement issues related to measuring the effects of the prevention intervention. Using statistical controls is suggested when research designs break down. Dwyer and MacKinnon discuss outcome measures used in drug abuse prevention intervention research. Areas for consideration include potential variable type, issues related to validity, categorical variables, and pretest measures. In the section on assessing the effectiveness of drug abuse prevention interventions, Hawkins and colleagues examine the long-term effects of drug abuse prevention interventions and issues related to replication. Prevention intervention effects are temporally limited, and approaches need to be further refined to enhance impact (e.g., booster sessions). Biglan and coworkers review issues related to controlling and examining attrition. Study dropouts need to be better understood; generally, those with higher rates of problem behaviors, including drug abuse, tend to leave prevention studies. Likewise, the effects of those entering a study (drop-ins) need to be assessed. Forman and Linney present approaches for validating drug abuse self-reports, which include physical/chemical tests, behavioral observations, and peer ratings. Validating outcome measure is an issue that should be considered in all drug abuse prevention intervention research. Schinke and Orlandi present stages for transferring technologies that focus on drug abuse prevention. Technology transfer stages incorporate a range of possibilities from basic research to adaptation to obsolescence. REFERENCES Arnold, C.; Kuller, L.; and Greenlick, M., eds. Advances in Disease Prevention. Vol. 1. New York: Springer Publishing Company, 1981. Berberian, R.M.; Gross, C.; Lovejoy, J.; and Paparella, C. The effectiveness of drug education programs: A critical review. Health Educ Monog 4:377-398, 1976. 4 Biglan, A., and Ary, D.V. Methodological issues in research on smoking prevention. In: Bell, C.S., and Battjes, R., eds. Prevention Research: Deterring Drug Abuse Among Children and Adolescents. National Institute on Drug Abuse Research Monograph 63. DHHS Pub. No. (ADM)87-1334. Washington, DC: Supt. of Docs., U.S. Govt. Print. Off., 1985. pp. 170-195. Bukoski, W. Where drug education stands now. In: Edwards, G., and Arif, A., eds. Drug Problems in the Sociocultural Context. Geneva, Switzerland: World Health Organization, 1980. Bukoski, W. A definition of drug abuse prevention research. In: Donohew, L.; Sypher, H.; and Bukoski, W., eds. Persuasive Communication and Drug Abuse Prevention. Hillsdale, NJ: Lawrence Erlbaum Associates, in press. Donohew, L.; Sypher, H.; and Bukoski, W., eds. Persuasive Communication and Drug Abuse Prevention. Hillsdale, NJ: Lawrence Erlbaum Associates, in press. Glynn, T.J.; Leukefeld, C.G.; and Ludford, J.P., eds. Preventing Adolescent Drug Abuse: Intervention Strategies. National Institute on Drug Abuse Research Monograph 47. DHHS Pub. No. (ADM)83-1282. Washington, DC: Supt. of Docs., U.S. Govt. Print. Off., 1983. Goodstadt, M.S. Myths and methodology in drug education: A critical review of research evidence. In: Goodstadt, MS., ed. Research on Methods and Programs of Drug Education. Toronto, Ontario: Addictions Research Foundation of Ontario, 1974. Last, J. Public Health and Preventive Medicine. 11 th ed. New York: Appleton- Century-Croft, 1980. Leukefeld, C.G. The role of the National Institute on Drug Abuse in drug abuse prevention. In: Donohew, L.; Sypher, H; and Bukoski, W., eds. Persuasive Communication and Drug Abuse Prevention. Hillsdale, NJ: Lawrence Erlbaum Associates, in press. Schaps, E.; DiBartolo, R.; Moskowitz, J.; Palley, C.; and Churgin, S. Primary prevention evaluation research: A review of 127 impact studies. J Drug Issues 11:17-43, 1981. Silverman, M.M. Prevention Research: Impediments, Barriers, and Inadequacies. Proceedings of the First National Conference on Prevention Findings. Washington, DC: U.S. Govt. Print. Off., in press. Swisher, J.D. Prevention issues. In: DuPont, R.L.; Goldstein, A.; and O’Donnell, J., eds. Handbook on Drug Abuse. Washington, DC: Supt. of Docs., U.S. Govt. Print. Off., 1979. Wilner, D.; Walkley, R.; and O’Neil, E. lntroduction to Public Healfh. 7th ed. New York: MacMillan, 1978. 5