JEIBI 3.4 – 4.1 A Function-Based Intervention to Decrease Disruptive Behavior and Increase Academic Engagement Kathleen Lynne Lane, Rachel Smither, Rachel Huseman, Jennifer Guffey and James Fox Abstract A range of interventions exist to prevent and respond to disruptive classroom behavior. This study documents the efficacy of a function-based intervention conducted using a multiple baseline across settings design. Despite moderately variable levels of treatment fidelity, results suggest a functional relation between the introduction of a package intervention and corresponding increases in academic engagement and decreases in disruption. Limitations and implications for future research are presented. Keywords: function based intervention; treatment integrity. In reviewing the empirical status of our knowledge about the nature and management of the class of behavior known as “disruption,” Neary and Eyberg (2002) noted that: (a) disruption is a common behavior problem in children; (b) disruption presents itself in various specific ways including noncompliant, impulsive, attention-demanding, high rate and/or aggressive behaviors; (c) it is a common reason for referral to child mental health services; and, (d) left untreated, it is a highly stable and problematic pattern of behavior. Not surprisingly, various behavior intervention procedures and packages have been developed and evaluated in terms of their ability to effectively eliminate such behavior challenges in children and youth. Some interventions have been developed for application by a child’s parents such as Parent Child Interaction Therapy (e.g., Eisenstadt, Eyberg, McNeil, Newcomb, & Funderbunk, 1993; McNeil, Eyberg, Eisensadt, Newcomb, & Funderbunk, 1991; Neary & Eyberg, 2002). Others have developed interventions for application by teachers and other school personnel to address disruptive behavior in classroom and other educational settings (e.g., De Martini-Scully, Bray, & Kehle, 2000; Kelshaw-Levering, Sterling- Turner, Henry, & Skinner, 2000; Musser, Bray, Kehle, & Jenson, 2001; Theodore, Bray, Kehle, & Jenson, 2001). Furthermore with the addition of case management and consultation strategies, these procedures can be applied in a coordinated fashion to have good effect across a child’s school and home environments (e.g., Martin & Hagan-Burke, 2002). Although different in their specific applications and some of their conceptual underpinnings, these behavioral procedures and intervention packages also have underlying similarities. Generally, these approaches seek to alter maladaptive interaction patterns between the disruptive child and the adults (teachers, parents) in his/her environment. This re-arrangement is frequently accomplished by teaching the teacher or parent to modify his/her own behavior toward the child: (a) state behavior expectations clearly; (b) use direct or “precision” requests; (c) apply differential reinforcement procedures; and (d) employ response cost or time out procedures to decrease more serious disruptive behaviors of the child (Sulzer-Azaroff & Mayer, 1991). There is a considerable data base supporting the efficacy of these combined procedures in producing lasting decreases in disruptive behaviors and increasing alternative positive behaviors (e.g., Ayllon & Roberts, 1974; De Martini-Scully et .al. 2000; Friman, 1990; Kelshaw- Levering et al. 2000; Langland, Lewis-Palmer, & Sugai, 1998; Matheson, & Shriver 2005; McIntosh, Rizza, & Bliss, 2000; Moore, 1999; Musser et al. 2001; Theodore et al. 2001; Thomas, Becker & Armstrong, 1968). 348 JEIBI 3.4 – 4.1 When specific classes of behavior intervention procedures have been compared in terms of their relative effectiveness some procedures generally appear to be more effective than others. In a meta- analysis of 99 studies with over 5000 students, Stage and Quiroz (1997) found that the most effective behavioral interventions as determined by effect sizes were group contingencies, self-management, and differential reinforcement while the least effective were functional-based interventions (hypothesis-based interventions that altered classroom antecedent triggers and/or consequences that maintained disruptive behaviors) and cognitive-behavioral interventions (those that combine learning principles with cognitive factors such as anger control programs, relaxation training affective imagery, and social problem- solving). At the same time the use of packaged procedures, even those that have documented effectiveness for disruptive students in general, may have certain shortcomings in individual application. For example, a standard intervention protocol or package of interventions may be less effective or not effective at all for certain students, depending upon the maintaining function of their disruptive behaviors. In the last 20 years research on functional behavioral assessment has shown that students’ disruptive and other challenging behaviors have definable environmental triggers (setting events and antecedents) as well as consequences that maintain those behaviors (Fox & Gable, 2004). Furthermore, this research also indicates that interventions based on the assessed function of the behavior can be more effective than those not based on the behavior’s function (Iwata et al., 1994; Repp, Felce, & Barton, 1988; Repp, Karsh, Munk, & Dahlquist, 1995; Scotti, Evans, Meyer, & Walker, 1991). Interestingly, Stage and Quiroz (1997) reported in their meta-analysis that function-based interventions designed to address disruptive behavior did not produce as large an effect size as did other standard behavioral interventions (group contingencies, self-management, and differential reinforcement), and concluded that function-based interventions were less effective. In contrast, a number of single subject analyses have shown that, across various ages, grade levels and specific forms of disruption, function-based interventions effectively and substantially decreased disruption and/or increased more appropriate replacement behaviors (e.g., Aikman, Garbutt, & Furniss, 2003; Broussard, & Northup, 1995; Dunlap, White, Vera, Wilson, & Panacek, 1996; Ellis, & Magee, 1999; Field, Nash, Handwerk, & Friman, 2004; Grandy, & Peck, 1997; Hoff, Ervin, & Friman, 2005; LaBelle, & Charlop-Christy, 2002; March & Horner, 2002; Reese, Richman, Zarcone, & Zarcone, 2003; Sigafoos, & Meikle,1996; Sprague & Thomas, 1997; Sterling-Turner, Robinson, & Wilczynski, 2001; Umbreit, 1995; Umbreit, 1996; Umbreit, Lane, & Dejud, 2004). Why this apparent discrepancy should exist between Stage and Quiroz (1997) meta-analysis and single subject analyses of the effects of function-based interventions for disruptive behavior is unclear. One possible and quite likely explanation has to do with treatment integrity (Peterson, Horner, & Wonderlich, 1982; Lane & Beebe-Frankenberger, 2004), that is, the extent to which interventions are faithfully and consistently implemented by the intervention agents. In their analysis, Stage and Quiroz (1997) did not provide data evaluating the extent to which intervention agents had implemented the procedures. It is possible that such data may not have been available for many or all of those studies. Behavior analytic researchers have rightly critiqued the state of behavior intervention research in general (e.g., Kazdin, 1998; Peterson et al.1982; Gresham, Gansle, & Noell,1993) and in specific areas of behavior challenges such as autism (e.g., Wheeler, Baggett, Fox, & Blevins, 2006) and behavioral disorders (e.g., Fox & Gable, 2004; Gable, Hendrickson, & Van Acker, 2001; Lane, Umbreit, & Beebe- Frankenberger, 1999; Sasso, Conroy, Stichter, & Fox, 2001) for the relative lack of data on treatment integrity and the associated changes in student behavior. It might be argued that, although treatment integrity data were not included in Stage and Quiroz (1997) analysis, one might expect that function-based interventions would be likely to be implemented 349 JEIBI 3.4 – 4.1 with the same level of fidelity as standard behavior interventions. This line of reasoning, however, ignores a second related issue. While functional assessment and analysis procedures have a long history (Fox & Gable, 2004), the more specific applications of this approach to classroom analysis and intervention had only recently begun to be developed at the time of Stage and Quiroz (1997) meta-analysis. Indeed, the necessary and sufficient tactics and instruments of functional assessment and their adaptations to specific populations of students (e.g., low incidence disorders such as mental retardation and autism versus higher incidence disorders such as emotional and behavioral disorders and learning disabilities) continue to be the subject of debate and of applied research (e.g., Sasso et al, 2001). It may be that functional assessment based interventions were less precisely developed and implemented than standard behavior interventions at the time of Stage and Quiroz (1997) analyses. Finally, functional assessment-based interventions tend to be multi-component intervention packages involving modification of antecedent triggers; alterations of instructional procedures and curricular content; as well as changes in behavior consequences rather than interventions that modify a single aspect of the classroom environment. Given the multiple components, function-based intervention packages may be more likely to be implemented incompletely or insufficiently, particularly by natural environment agents. Consequently, treatment integrity measures are an increasingly important part of any research analysis of behavioral interventions, and are especially important in FBA-based interventions. In reviewing those studies since or not included in Stage and Quiroz (1997) that have addressed functional assessment and intervention for students with disruptive behaviors, we identified 14 studies (Aikman et al., 2003; Broussard & Northup, 1995; Dunlap et al. 1996; Ellis & McGee, 2000; Field et al.2004; Grandy & Peck, 1997; Hoff et al., 2005; LaBell & Charlop-Christy, 2002; March & Horner, 2002; Sigafoos & Meikle, 1996; Sprague & Thomas, 1997; Sterling-Turner et al., 2001; Umbreit, 1995; Umbreit, Lane, & Dejud, 2004), only five of which provided treatment integrity data (Broussard & Northup, 1995; Dunlap et al, 1996; Field et al, 2004; Umbreit, 1995; Umbreit et al, 2004), suggesting continued need for treatment integrity analyses. Purpose The purpose of the present single subject research study was three-fold. First, we sought to further replicate the effects of interventions that had proven effective in treating disruptive behavior, namely self-monitoring and differential reinforcement (Stage & Quiroz, 1997). Second, not only did we apply science-based interventions to the target student’s disruptive behavior, we chose these interventions and added components to them based on a systematic functional assessment of that behavior. A combination of indirect assessment and direct assessment instruments were systematically employed to gather data, analyze the function of the student’s disruption and to empirically identify intervention components. These functional assessment instruments and procedures included functional assessment interviews with the referring teacher and the student, the Social Skills Rating System, (Gresham & Elliott, 1990), the Motivation Assessment Scale (Durand & Crimmins, 1988), direct observations, and the Function Matrix (Umbreit, Ferro, Liaupsin, & Lane, 2007), an instrument for organizing functional assessment data and developing a hypothesis of the behavior function. Finally, a third purpose was to collect data on the fidelity of implementation of the intervention by research assistants and to examine levels of fidelity in relation to changes in student disruption using single case design methodology. Method Participant Harry was a six-year old, typically developing Caucasian male who received education services a general education kindergarten classroom. Harry did not qualify for special education services according 350 JEIBI 3.4 – 4.1 to Individuals with Disabilities Education Act (IDEA, 2004) nor did he have a diagnosed disorder per the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR; American Psychiatric Association, 2000). Harry was referred by his teacher for possible participation due to frequent disruptions throughout the school day that resulted in exclusionary time out and impeded her ability to instruct the class (i.e., hitting or kicking peers, refusing to comply with teacher instructions, being out of seat without permission, talking to peers without permission, making faces or gestures toward other students, touching others’ property without permission, and hitting his desk loudly with his hands or another object). Setting This study was conducted in a general education, public school kindergarten classroom in an inclusive school district located in Middle Tennessee. The school serves approximately 530 students in kindergarten through fourth grade. Harry’s classroom contained 20 students, one certified teacher, and one student teacher. The teacher followed a daily schedule that included all content areas and specialty classes (e.g., Spanish, music, and art) that were taught by another teacher. The teacher had a classroom management plan that involved four class rules: (a) follow directions first time given, (b) keep hands, feet, and objects to yourself, (c) respect others: treat others as you want to be treated, and (d) raise your hand for permission to speak. The class rules were posted in the classroom as were the rewards for meeting these expectations (e.g., a good feeling, ultimate recess, praise, good notes, stickers, and visits to the office) and negative consequences for violating these rules (e.g., name on board, check mark beside name, laps for half of recess, laps for all of recess, and contact parent or vice principal). In addition to participating in the above classroom management plan, other interventions such as in-school detention and visits to the guidance counselor had been attempted. Yet, the teacher stated that these interventions had not been effective in decreasing his disruption. Dependent Measure and Interobserver Agreement Total disruptive behavior (TDB). TDB was chosen as the target behavior because it encompassed many of the behaviors that were a concern for the teacher. TDB referred to the general class of behavior that interrupted classroom instruction and specifically included hitting or kicking peers, not complying with teacher instructions, being out of seat without permission, talking to peers without permission, making faces or gestures toward other students, touching others’ property without permission, and hitting his desk loudly with his hands or another object. TDB was assessed via total duration recording using a stopwatch. The stopwatch was started when Harry engaged in a disruptive behavior, and stopped when Harry was no longer engaging in disruptive behavior. The percentage of time Harry engaged in TDB was computed by dividing the total amount of time he was engaged in disruptive behavior by the total time observed and multiplying by 100 to yield a percentage of TDB. The observation sessions were 10 min. Academic engaged time (AET). The replacement behavior, AET, was also measured using a stopwatch and duration recording procedures as described above for TDB. Academic engaged time was defined as the time Harry spent actively engaged in classroom activities. Examples included looking at the teacher during instruction, working independently at his desk, working in a group or with a peer, following teacher directions, and raising his hand to ask questions. Interobserver agreement (IOA). Three masters’ level graduate students (GAs) collected all behavioral observations. Prior to collecting baseline data, the GAs completed successfully a direct observation training program that included developing operational definitions, reviewing procedures for conducting observations in a classroom setting, and achieving reliability of data collection procedures. 351 JEIBI 3.4 – 4.1 Specifically, GAs trained to a 95% criterion over three consecutive data collection sessions before collecting baseline data. IOA was collected for approximately 25% of the observations with IOA collected during each phase in each setting. IOA was computed by dividing the shorter duration by the longer duration and multiplying the quantity by 100. Mean IOA scores were as follows: TDB 90.52% in the classroom and 97.67% in specialty areas; AET 99.54% in the classroom and 99.70% in specialty areas. Social validity. Social validity was assessed from the teacher and student perspectives using the Intervention Rating Profile – 15 (IRP-15; Martens, Witt, Elliott, & Darveaux, 1985) and Children’s Intervention Rating Profile (CIRP; Martens et al.,), respectively. The IRP-15 is a 15-item, factor analytically-derived tool used to measure the significance of intervention goals, acceptability of treatment procedures, and social importance of intervention effects (Wolf, 1978) from the teacher perspective. Teachers rate each item on a 6-point Likert-type scale ranging from strongly disagree (1) to strongly agree (6). Scores range from 15 to 90, with high scores indicating high acceptability. Internal consistency reliability coefficients range from .88 to .98. The CIRP is a seven item, factor analytically-derived tool used to assess social validity from the student perspective (e.g., fairness and expected effectiveness) from the student perspective. Students rate the items on a 6-point Likert-type scale ranging from I agree (6) to I do not agree (1). Scores range from 7 to 42, with high scores indicating high acceptability. Internal consistency reliability estimates range from .75 to .89. The IRP-15 was administered at the onset of the intervention to inform intervention design and again at the end to obtain summative information. The CIRP was administered at the end of the intervention. Functional Assessment Procedures A functional assessment was conducted within the context of a collaborative consultation model with the authors and the classroom teacher serving as participants (Ervin, Ehrhardt, & Poling, 2001). Specifically, the research assistants, the classroom teacher, parent, and student worked together to (a) complete the functional assessment process; (b) design a function-based intervention that was socially valid to all parties; (c) implement the intervention as planned; and (d) examine intervention outcomes (Stahr, Cushing, Lane, & Fox, in press). Descriptive functional assessment tools and hypothesis development. After completing informal discussions with the teacher and conducting informal observations of the students’ target behavior, the formal functional assessment procedures were conducted. Specially, the following functional assessment tools were used: (a) a functional assessment interview using the Preliminary Functional Assessment Survey (Dunlap et al., 1993), (b) a functional assessment interview with the student using the Student Functional Assessment (Kern, Dunlap, Clarke, & Childs, 1994), (c) 10 hours of direct observations using A-B-C data (Bijou, Peterson, & Ault, 1968; Carr, Levin, McConnachie, Carlson, Kemp, & Smith, 1994), (d) a teacher-completed rating scale, the Motivation Assessment Scale rating scale (MAS; Durand & Crimmins, 1988), to determine the function of the target behavior, (e) the teacher version of the Social Skills Rating System, (SSRS; Gresham & Elliott, 1990) to determine if the student had acquisition or performance deficits; and (f) the School Archival Record Search (SARS; Walker, Block-Pedego, Todis, & Severson, 1991) to quantify information in the student’s cumulative file. Data from the above named functional assessment tools were analyzed using the function matrix (Umbreit et al. 2007) - a matrix of 2 columns of reinforcement functions (positive and negative) and 3 rows delineating three types of consequences (attention, tangibles/activities, and sensory). Data were placed into one of the six cells. Then, the function of the target behavior was determined by examining 352 JEIBI 3.4 – 4.1 the matrix and identifying the cell (e.g., positive reinforcement, attention) that contained the most data. Next, hypothesis statements were developed. During the functional assessment interview the teacher identified hitting and kicking his peers as well as failure to follow directions as the primary behaviors of concern. She stated that he appeared to engage in these disruptive and noncompliant behaviors to gain peer attention and that these behaviors often occurred when other students teased him. The teacher had attempted other interventions such as removing him from class, informing him of the consequences ahead of time, using signals to prompt Harry to stop and think before acting, and sending him to the guidance counselor which he seemed to enjoy. However, these interventions had been largely unsuccessful in decreasing the disruptive behaviors and increasing academic engagement. She indicated that Harry was a capable student, but that these behaviors were occurring quite often over the school day and in the after school child care program. SSRS data revealed that Harry’s academic skills fell within the average range (standard scores = 89) and his social skills were slightly below average (standard scores = 84). Harry’s problem behaviors exceeded normative performance as evidenced by a standard score of 125. These data suggest that Harry did not have academic skill deficits; however, he did appear to have interfering problem behaviors that required attention. His social skill deficits may also impede Harry’s ability to respond to teasing by peers. During the student interview Harry indicated that he had the fewest problems on Mondays and the most problems on Fridays because he did not like watching movies that were held on Fridays. When asked what caused him to engage in disruptive behaviors, Harry stated that he became most angry when people laughed at him and said things that he did not like. When asked what things could be done to help manage his disruptive behavior, Harry responded with the same consequences that the teacher had attempted (e.g., stop and think). Although he was well-versed in the consequences for disruptive behavior, Harry’s disruptive behavior did not decrease when these techniques were employed. Harry did suggest that his behavior would improve if he could earn more rewards for appropriate behavior. Direct observation data suggested that Harry’s disruptive behavior was maintained by attention in 11 instances and maintained by escape in only 2 instances of the target behavior. When Harry exhibited disruptive behavior, the consequence was often either lengthy conversations with the teacher in which she explained in detail why such behavior was inappropriate or arguments with his peers. Given that the clear majority of the observations fell into the attention/positive reinforcement cell of the matrix, it was hypothesized that Harry’s disruptive behavior was maintained by positive reinforcement in the form of teacher attention. Results of the MAS also indicated that Harry’s disruptive behavior was maintained by attention with mean score of 2.25 and a relative ranking of 1 followed by tangible reinforcers (relative ranking = 2). Escape and sensory categories both received mean scores of 1.75, indicating that they were the least motivating for Harry. Synthesis of FBA data using the function matrix. Data from the teacher and student interviews, direct observations, and ratings scales were placed into the function matrix to identify the hypothesized function of Harry’s disruptive behavior (Umbreit et al., 2007). All sources of data supported the hypothesis that Harry’s behavior was maintained by positive attention in the form of attention from teachers and peers. In most instances, Harry’s disruptive behavior would allow him to access attention from others. It was hypothesized that designing an intervention to allow Harry to receive attention on a consistent basis for appropriate behaviors would result in decreased levels of disruptive behaviors and increased levels of academic engagement. 353 JEIBI 3.4 – 4.1 Intervention development. Next, the function-based intervention decision model (Umbreit et al., 2007) was used to design an intervention linked to the function of the target behavior. The function-based decision tree is a flow chart comprised of two questions: (a) Can the student perform the replacement behavior? and, (b) Do antecedent conditions represent effective practices? The teacher interview, SSRS results, and student interview all suggested that Harry was capable of participating in instructional activities and completing successfully the other task demands. Consequently, it was determined that Harry demonstrated a performance deficit rather than a skill deficit as he could perform the replacement behavior, academic engagement. Next, it was determined that the classroom conditions could be improved by affording Harry higher rates of positive attention for demonstrating appropriate behaviors. It was hypothesized that if Harry received access to positive reinforcement for being academically engaged, then Harry would not demonstrate disruptive behavior to gain attention. A function-based intervention package was designed that included three components: (a) self-monitoring, (b) differential reinforcement, and (c) positive scanning. The intent of this intervention was to adjust the classroom conditions to prompt and reinforce academic engagement and extinguish disruptive behavior. Self-monitoring. Self-monitoring was chosen as the primary method of intervention to help direct both the student and the teacher to focus on the child’s appropriate on-task behavior. A small, laminated flip chart was attached to Harry’s desk with Velcro. One side of the chart listed questions, along with pictorial representations, related to on-task behavior to serve as behavior prompts for the student. The other side of the chart contained a student and a teacher column with happy and sad faces. A happy face was to be marked if student was on-task. A sad face was to be marked if student was disruptive. The intervention was first implemented during one half-hour of reading instruction. The teacher prompted Harry every five min to circle either a happy or sad face based on his behavior during that interval. The teacher was then to circle a happy or sad face. Praise was given anytime teacher marked a happy face and an explanation was provided as to why happy face was marked. A brief one-sentence explanation was provided if different faces or sad faces were marked. If the teacher marked four or more happy faces during this half-hour, then Harry earned special time with the teacher, a peer, or the Guidance Counselor later that day. This intervention was used for two weeks in the regular classroom during reading instruction before implementation in specials. Harry’s self-monitoring chart was portable so that he could take it with him to other settings. During specials, the participating teacher would keep Harry’s chart and approach him every five min of this half-hour class time and prompt him to mark a happy or sad face. This teacher would then mark a happy or sad face. The same reward was used, thus giving Harry two chances a day for special time with a teacher, a peer, or the Guidance Counselor. Differential reinforcement. Differential reinforcement of other behavior (DRO) was also implemented in this intervention. The teachers were instructed to ignore disruptive behavior as much as possible, and instead give attention to appropriate, on-task behavior. Any behavior that was harmful or destructive to others was to be handled quickly and with as little attention given to Harry as possible. In such situations, attention was to be given to the victim instead. Positive scanning. Positive scanning was implemented with the regular classroom teacher and then two weeks later with the specials teachers. Each teacher was to find one thing that Harry did well that day and write it on a special note card. This card was then shown to Harry at the end of the day and sent home with him. The positive scanning was intended to increase teacher awareness of Harry’s appropriate behaviors and also provide Harry with reinforcement from teachers and parents for exhibiting the desired behavior. 354 JEIBI 3.4 – 4.1 Summary of intervention package and links to assessed behavior function. In sum, the intervention contained three components: self-monitoring, differential reinforcement, and positive scanning. The self-monitoring was designed to provide Harry with a structure for attaining and sustaining academic engagement that was reinforced by immediate and delayed attention. Differential reinforcement was intended to place disruptive behavior on extinction and increase reinforcement for desired behavior. Positive scanning was also intended to reinforce positively Harry’s academic engaged behavior. Collectively, these three components (a) adjusted the antecedent conditions in the classrooms so that the conditions that set the stage for disruptive behavior were eliminated and academic engagement was more likely to occur, (b) provided positive reinforcement in the form of teacher, peer, and parent attention for demonstrating the replacement behavior – academic engagement, and (c) withheld the consequences (attention) that previously reinforced disruptive behavior when it occurred. Training of Staff and Harry in the Intervention Package. A 1 hr meeting was held after school to explain the intervention to the general education and special class teachers. The GAs provided examples of and checklists for each component were provided and all questions were answered regarding the intervention. Treatment integrity forms were given to both teachers to serve as reference points as needed. The classroom teacher was observed the first day of the intervention and appeared comfortable with the procedure as evidenced by a high treatment integrity scores (100%). The GAs held a separate meeting with Harry during the school day to explain the intervention and teach him to use the self-recording sheet. A series of questions that included demonstrations were posed to Harry to check for understanding. When Harry was able to answer all questions accurately, the training was concluded (approximately 45 min). Treatment integrity. Treatment integrity data were collected by the GAs on a daily basis. The treatment integrity checklists contained 8 questions that were rated on a present (1) or absent (0) basis. Mean session integrity scores were computed by dividing the sum of the present scores by 8 and multiplying the quantity by 100 to obtain a percentage. Intervention evaluation: Experimental design and analysis. A multiple baseline across setting (general education and specialty classes) design was used to examine intervention effectiveness. The intervention was implemented first in the general education setting after baseline data achieved stability and later introduced in the specialty area class once Harry’s disruptive behavior began to decrease in the general education setting. Data probes were collected twice weekly over the course of the study. Data were analyzed using visual inspection techniques, computing percentage of non-overlapping data points (PND), and examining mean changes by phase (Johnston & Pennypacker, 1993). Results Disruptive Behavior and Academic Engaged Time During the classroom baseline, the mean level of AET was low (M = 46.00; SD = 23.66), with data points ranging from 30.30 to 59.50% and a downward trend (slope = -5.92). TDB ranged from 9.3 to 35.8 (M = 22.15; SD = 13.69), with an increasing trend (slope = 5.02). Once the intervention was 355 JEIBI 3.4 – 4.1 introduced in the classroom setting, Harry demonstrated a marked increase in mean levels of AET (M = 83.63; SD = 11.82), as well as a significant decrease in the mean level of TDB (M = 6.66; SD = 9.38). Percentage of non-overlapping data points was 100% for AET and 67% for TDB, suggesting that the intervention was more successful in increasing academic engagement than it was in decreasing disruptive behavior. Treatment integrity data revealed considerable variability in treatment integrity during the classroom setting, with a mean fidelity rating of 62.92 (SD = 29.70; range = 0 to 100%; Median = 69.05.). Figure 1. Intervention Outcomes Across Settings AET and TDB data were highly variable during the special classes baseline phase, with AET scores ranging from 29.2 to 98% (SD = 13.69) and TDB scores ranging from 5.3 to 48.2% (SD = 17.51). Mean levels of performance were higher for AET (M = 69.54) and lower for TDB (M = 20.73) as compared to the general classroom setting. The intervention phase was brief in the specialty settings due to winter break and frequent schedule changes that occurred in the days prior to winter break. Once introduced, there was less variability in performance for AET (SD = 6.90) and TDB (SD = 2.80). However, it should also be noted that, relative to baseline, there were fewer number of sessions for variability to occur. Percentage of non-overlapping data points was low - only 33% for AET and TDB, possibly due to generalization from the initially trained condition. Treatment integrity data revealed more consistent implementation during the special classes setting, with a mean fidelity rating of 93.33 (SD = 11.55; range = 80 to 100%; Median = 100). 356 JEIBI 3.4 – 4.1 Despite the brevity of the intervention in the second setting, the data are suggestive of a possible functional relation between the introduction of the intervention and changes in mean levels and variability between baseline and intervention phases for both intervention settings. This relation is clearer in the first setting, the general education setting, than in the second (specialty class) setting. It is interesting to note that following the fourth baseline probe session overall AET appeared to increase and TDB decreased although both remained rather variable. This coincided with application of the intervention in the general classroom. These changes in performance may have been occurred as a result of generalization of treatment effects. While generalization is a desired outcome for intervention research, it can be problematic for demonstrating experimental control in multiple baseline designs as illustrated in the present study. Social Validity The teacher ratings on the IRP-15 were high (total = 76) prior to implementing the intervention and were even higher (total = 86) when the post-intervention IRP-15 was administered, suggesting that the intervention exceeded the general education teacher’s expectations. The teacher rated all aspects of the intervention has highly acceptable and stated that she would be willing to use the intervention again with other students of students who exhibited similar difficulties. Harry completed the CIRP at the end of the study. He, too, rated the intervention favorably indicated that it was fair, would help him do better in school, and would also be good for other students. However, he indicated that it bothered him when some students looked at the chart on his desk. While a total CIRP score could not be obtained as one item was not completed, Harry was very cooperative and accepting of the intervention. Discussion The present study sought to evaluate the effectiveness of an intervention package in reducing a 6 year old kindergarten boy’s frequent disruptive behaviors and increasing his task engagement. The intervention package was composed of empirically-validated (Stage & Quiroz, 1997) intervention procedures, self-monitoring and differential reinforcement. These particular intervention components and one additional procedure, positive scanning, were selected through a systematic functional behavior assessment of the student’s disruptive behavior. Treatment integrity and social validity data were also obtained. Despite some variability in the teachers’ application of the intervention package, observational measures of the subject’s academic engaged time and his disruptive behavior showed positive and substantial improvements. The internal validity of these results was suggested in that these behavioral changes were concurrent with the application of the intervention, first in the general classroom and subsequently in the specials classroom (i.e., a multiple baseline across settings). However, as discussed above, there is a possibility of generalization in the specials classrooms just prior to introducing the intervention. Social validity measures indicated that both teachers and the student found the procedures to be very helpful, effective, and acceptable in classroom application. Given these outcomes, the present single subject study represents further evidence that hypothesis-based interventions are effective in treating the challenging behaviors of disruptive students (Ellingson, Miltenberger, Stricker, Galensky, & Garlinghouse, 2000; Ingram, Lewis-Palmer, & Sugai, 2005; Meyer, 1999; Repp et al. 1995; Taylor & Miller, 1997). Furthermore, these results were achieved under naturalistic classroom conditions when the intervention was implemented by natural environment agents (Harry’s teachers as well as Harry himself), and they were effective across diverse classroom 357