Does Seeking to Create a Better Classroom Climate Lead to Student Success or Improved Teaching? Examining the Relationship Between Pedagogical Choices and Classroom Climate in Urban Secondary Schools John Shindler, Albert Jones, Clint Taylor and Herminia Cadenas California State University, Los Angeles There is growing evidence to suggest a strong link between achieving student affective outcomes and corresponding achievement outcomes (Hoy & Hannum, 1997; Hawkins, 1997; Levin & Wiens, 2003; Ryan & Patrick, 2001; Sweetland & Hoy, 2000; Wang, Haertel & Walberg, 1997). Moreover, research shows that students experience gains in both academic achievement and academic self-concept as teachers seek to develop students’ academic agency (Jackson, 2003). Jackson describes student academic agency as instructional approaches that are “identity-sensitive.” More than a decade of research supports the affective and academic benefits of what Glasser (1990) referred to as “needs-satisfying” curriculum. Building on Glasser’s work, Patterson (2003) found that the use of problem-based curricula that challenge students to grow as reflective learners and in their collective functioning result in the development of students who are self-directed, critical thinking learners. He concludes that schools “must include a vision of what kind of learners we are trying to develop and how we can assist students to become what we envisioned. We must put the educational needs of the students above the comfort of the adults.” Programs such as Facing History and Ourselves (FHAO) offer teachers a powerful history curriculum as well as problem-based pedagogical format that promotes both “identity-sensitivity” and what Schultz, Barr & Selman (2001) term “relational maturity” resulting in an increase in student agency and the group’s collective functioning. Nevertheless, the effectiveness of such programs will ultimately vary substantially depending on the teacher, their values, the pedagogy used before and during any such unit, various contextual variables and the pre-existing classroom climate (Sweetland & Hoy, 2000; Wang, 1997). Curricular approaches that promote what could be defined as critical pedagogy, and/or those that promote the collective social and emotional intelligence of students are typically endorsed on moral and altruistic grounds, yet rarely discussed as embodying what could be considered “effective teaching.” Shindler, Taylor, Jones & Cadenas (2003) found that efforts toward what could be termed “intentional climates” that were defined by a transformational or liberating practice did appear to produce greater student achievement. This research suggests that the choice that often seems to be presented in teacher education (implicitly, if not explicitly) between either teaching for depth of understanding, meaning making and affective growth, or that of preparation for demonstration of competency, may be a false choice. In fact, attaining high-levels of academic competency may be more likely within a classroom climate that is “identity-sensitive” and “needs-satisfying.” 27 Purpose and Guiding Questions for the Study The study was a pilot effort to examine the relationships among a series of factors within the practice of a teacher who is attempting to implement the Facing History and Ourselves (FHAO) curriculum into his/her classroom. The factors included the content being taught, the pedagogical choices made, the “intentionality” of the efforts related to classroom climate development, and their effects on student achievement and classroom climate. The pilot study attempted to examine four primary questions: 1. Do the pedagogical choices made by the teacher affect the degree to which the goals of a curriculum such as FHAO can be met? 2. How do certain pedagogical choices affect the quality of classroom climate? 3. What is the relationship between the degree of “intentionality” in the development of classroom climate and student learning? 4. What long-term impact does incorporating a critical curriculum such as Facing History and Ourselves have on a teacher? Study Methods The pilot study employed a qualitative emergent design to follow 2 teachers into the field that had recently completed a Facing History and Ourselves (FHAO) Training Institute. Participants were examined while in the midst of units that lasted from 2-4 weeks. Teacher participants were purposively sampled to be representative of the most “successful FHAO teachers.” Both teacher participants were experienced teachers of over 5 years and held graduate degrees. Two additional classes of the same course taught by non-FHAO teachers with equivalent population of students were selected from each of the schools to act as control samples. Data collection included observation of teacher practice during the unit, interviews with the teacher participants, a survey of FHAO goal accomplishment of both teacher and student participants, a survey of classroom climate by students, and focus group interviews of students near the end of the unit. The survey used to assess classroom climate incorporated an analytic trait format. The instrument format is depicted in below. Example A: Analytic Trait Scale Instrument Item Example: High Middle Low Teacher-student interactions could Teacher-student interactions could Teacher-student interactions are be typically described as be typically described as fair but mostly teacher-dominated and supportive and respectful. teacher-dominated. reactive. <-----(cid:31)----------- (cid:31) ---------- (cid:31) ------------- (cid:31) ----------- (cid:31) ------------ (cid:31) ------------ (cid:31) --------- (cid:31) ---------- (cid:31) ----(cid:198) 9 8 7 6 5 4 3 2 1 In a study of the efficacy of the WASSC system in an urban setting (Shindler, Taylor, Jones & Cadenas, 2003), significant advantages for a participant-driven, analytic-scale system were observed. The analytic-scale (rubric) instrument demonstrated greater soundness (validity, reliability, efficiency and benefit) than traditional inventories. 28 Data analysis was conducted in 3 phases. (cid:131) Phase 1 – Study protocol development o Observational data was collected from the participants in School 1, an analytical framework was established for further data collection in phases 2 and 3. (cid:131) Climate was confirmed as a salient factor and incorporated into subsequent phases. (cid:131) A “Social Studies Outcome” survey was developed to assess student report of success related to the intended outcomes of the FHAO program. (cid:131) A focus group protocol was developed for phase 2. (cid:131) Phase 2 – Data collection from Participant Teacher 1 including: (cid:131) In-depth examination of each area of the climate inventory (cid:131) In-depth examination of each item on the FHAO survey – with a special emphasis on judging causality for ratings (cid:131) Explore an analytical framework for a “successful FHAO teacher.” The framework includes 3 factors 1. The relationship of the teacher to the students 2. The pedagogy used before and during the unit 3. The FHAO curriculum content (cid:131) Phase 3, the data collection protocol was repeated with the participant and control classes in school 2. Re 29 Results The results of the survey data indicated that the Facing History and Ourselves (FHAO) teacher participants showed a significantly better performance across all measures on both classroom climate and FHAO/Social Studies outcomes. In school 1, the FHAO participant teacher’s class rated the overall classroom climate as a 7.1 (falling in the 3 or high level) as compared to the control class rating of 5.4 (falling in the middle or 2 levels). Likewise the students rated the FHAO teacher as much more successful in accomplishing goals such as providing a curriculum that “addressed issues of race and membership” (1.2 or “strongly agree” rating as compared to a 2.3 or “agree to neutral” rating), and providing a curriculum that encouraged students to take an active role as citizens (2.0 or “agree rating vs. a 3.2 or “neutral to disagree rating). In School 2, the findings were comparable. The FHAO teacher’s class rated the climate mean as 7.5 overall (e.g., “middle high level”/ 3 points) or a full level higher than the control class (X=4.5). Likewise on the FHAO/Social Studies Outcome ratings the FHAO teacher scored a 2.1 or a mean of an “Agree” response versus a 2.9 or a mean response of a ‘neutral’ response for the control class. This data suggests a clear relationship between the quality of the pedagogy and student perception of the success of the course. Students rated the curriculum as more effective in the FHAO classes. Yet, it is not clear if and/or to what extent the teaching methods contributed to the students’ perceptions that the curriculum was so meaningful or if it was that the curriculum could be seen as enhancing the perception of the methods. Nevertheless, participants in each focus group suggested the there was an interrelationship. This relationship could be seen in the correlation of scales 2 and 4 on the climate inventory. For both FHAO teacher participants the perceived effectiveness of their teaching methods (X=7.0,7.6) was related to high scores in the culture of the class (X=7.1,7.5). This relationship could also be seen in the control groups. However in the control groups the ratings were significantly lower (see Table 1). A causal relationship between the quality of the instruction and the way they felt about the culture of the class and their relationships to one another was reported by students in the focus group interviews. Students also reported a strong influence of the methods used on the efficacy and agency of the curriculum and content being covered. In both schools, the students suggested that the curriculum would have been much less meaningful without being delivered within a methodological framework that they experienced as successful. But the students also suggested that given the high level of trust created, the high quality of the relationship with the teacher and the engaging instructional climate, the pedagogical methods were enhanced by a curriculum that contributed to the relevance and interest level of their work and learning experience. 30 Table 1: Classroom Climate Survey Comparison of Means for Participant and Control samples S1 FHAO S1 control S2 FHAO S2 control (N= 33) (N=28) (N=34) (N=26) Scale 1 6.5 5.4 7.1 4.3 Student Interactions Scale 2 7.7 5.5 8.0 4.8 Discipline Environment Scale 3 7.0 5.6 7.6 4.7 Learning/Assessment Scale 4 7.1 5.1 7.5 4.4 Attitude & Culture Sub-scale A 7.7 5.4 8.2 4.5 Relationship Sub-scale B 7.2 5.7 7.5 4.9 Clarity Sub-scale C 7.4 5.5 7.5 4.2 Community Subscale D 7.8 5.4 7.8 4.4 Diversity Overall Mean 7.1 5.4 7.5 4.5 Scores are derived from a mean rating for each item. Items are rated 1-3 for low quality rating, 4-6 for middle level ratings and 7-9 for high level ratings (see Example A above). 31 Table 2: Facing History and Ourselves/Social Studies Goals Survey Results S1 FHAO S1 control S2 FHAO S2 control (N= 33) (N=28) (N=34) (N=26) 1. Self Understanding 2.4 2.6 2.8 3.6 2. Addresses issues of 1.2 2.3 2.0 2.6 race, class, membership 3. encourages citizen 2.0 3.2 1.9 3.0 participation 4. understand 1.8 2.0 1.7 2.5 broader implications of history 5. builds classroom 2.4 2.6 2.0 3.1 community 6. feels like 1.5 2.2 2.0 2.4 “authentic” history 7. effects my 1.8 2.0 2.4 2.8 behavior outside of class Overall Mean 1.9 2.4 2.1 2.9 Scores are derived from a mean rating for each of the 7 items. 1=Strongly Agree, 2=Agree, 3=Neutral, 4=Disagree, 5=Strongly Disagree. The results of the focus group interview helped clarify the causality of the survey data. Students were enthusiastically positive about the methods of the 2 FHAO teachers. Neither of the teachers experienced any classroom management problems even though the students reported that classroom discipline was a big issue in most of their other classes. This discrepancy was confirmed by researcher observations as well. The Students in both FHAO classes experienced a high degree of “community” while reporting very little of it their other classes. The focus group interviews provided more support for the significance of the relationships that were formed with each of the FHAO teachers. Asked why there were no discipline problems, why all students seemed to be prepared, motivated and excited to be in class, unanimously the students reported that it was due to what was taking place in the class and especially to their relationship with the teacher. They sited examples of the teacher asking them about their lives outside of school, taking a sincere interest in them as individuals and treating them with respect. Comparing the methods incorporated by the FHAO teachers to the control teachers, there were clear differences. The FHAO teachers used group work purposefully and taught the students how to function in collectives. The FHAO teachers found engaging interactive methods to present material such as presentations, role-plays, debates, and discussions. In both FHAO classes, the discussions that were observed were striking in that nearly all students felt welcome to participate, were engaged, and projected a supportive, egalitarian affect to their peers. This type of climate was not the case in other classes observed at either of the 2 schools, in fact it could be characterized as exceptional. 32 Within the context of this supportive interpersonal climate and the engaging pedagogical environment, the evocative and personally and socially challenging content of the FHAO curriculum flourished. The students took the material very seriously and were very respectful to the issues they were addressing. Absent was any hint of the students ridiculing or making light of those they were studying. It would have been expected to see some students approaching the material with a lack of maturity or compensating for a sense of “group/collective guilt” by joking about some of the emotionally challenging material. Yet, only a mature and respectful affect was evidenced. Asked to describe what it was that made the class “successful,” the students suggested that the teacher was the most significant variable, next was the methodology and then came the curriculum they selected to teach. Asked to give percentages for each, both FHAO classes reported about the same proportions. It averaged out to about 50,35,15 – relationship, pedagogy, curriculum. However, they felt that all 3 were interrelated, and critical to the level of success that there had experienced. In school 2, where the teacher used less formal FHAO curriculum, the students saw less of a critical role of the content for what they felt made the class successful. In school 1, where the students had been exposed to a longer unit and a more substantial and sustained use of the FHAO curriculum, the students seemed to feel the curriculum content was more responsible for what might be referred to as the “transformational quality” that they experienced in the course as a whole. An indication of the effect the teacher’s commitment in school 1 to the FHAO curriculum had could be seen in the rating of the class for item 2 in the Social Studies Survey, “The content of the class effectively addresses issues of race, membership and identity.” This FHAO class overwhelmingly rated this item with a “strongly agree” response (X=1.2), compared to an average response of “agree” in the FHAO class in school 2 (X=2.0), and much more neutral responses in the control groups (X=2.3,2.6). So while both FHAO teachers were rated as highly successful on all accounts by their students, and their methods translated into environments where students approached the material with care, thought and inquiring minds, the data seems to suggest that more time spent engaged in the FHAO curriculum did appear to produce more success in terms of the intended FHAO outcomes. When each FHAO class was asked how they would have hypothetically responded to such a challenging curriculum dealing with race, class and membership within a class where they did not have the kind of relationship with their teachers that they currently experienced, and if the teaching methods used were characterized by a traditional “sit and get,” lecture and test format, the students suggested that it would make the curriculum essentially academic. They felt that it would be, in effect, disconnected and not nearly as meaningful to their lives as what they currently experienced. Consistent with its effects on the students, the data suggests that the FHAO curriculum cannot be examined outside of its effects on the teachers and their practice. Both teachers reported growing pedagogically as a result of their involvement with FHAO. While each participant seemed committed to the creation of a highly effective class regardless of the curriculum they used, the FHAO curriculum provided a venue for each to examine their methods and their materials and make improvements. Each reported that they were more purposeful in how they tried to bring about affective domain learning outcomes with their students. They also tended to use more primary documents and felt more comfortable teaching for a greater depth of understanding rather than accepting such a heavy (district driven) emphasis on content coverage. 33 Discussion The results of the study suggest that these 2 FHAO teachers successfully achieved “intentional climates” in their classes. The results also suggest that in the control group classrooms what was created could best be characterized as “accidental climates” (Shindler, Jones, Taylor & Cedenas, 2003). Within these intentional climates, a sense of community, academic rigor and relational maturity were better able to take root. It appears that the intentionality of the climates, the quality of the relationships and use of highly effective pedagogy were all critical to promoting a receptivity, depth of processing and a seriousness of treatment with the FHAO content. In this climate, the FHAO curriculum flourished. It is not clear what would happen if a teacher attempted to incorporate the challenging FHAO curriculum in an unintentional/accidental climate, where there were not the relational or pedagogical requisite conditions present. Because of the relational maturity (Schultz, Barr & Selman, 2001) required for FHAO to gain a receptivity, in the absence of these requisite conditions it is very possible that the FHAO curriculum may be trivialized by students who are not emotionally prepared as individuals or as a collective to treat the material seriously. While this assertion was suggested by some student participants, and appears to make sense, it would require further study to confirm. This pilot study seems to supports the interdependency of factors related to the effects of pedagogy, the relationship environment, the intentionality of the climate, the quality of the curriculum and the level of student learning. Students in the FHAO classes, while similar to those in the control samples, reported learning more and having a higher degree of investment in their learning. These results support the contention that each of these areas needs to be approached within the broader whole. Moreover, efforts to improve one of these areas or to implement a program that does not consider the holistic relationships among the factors may be inherently limited in its efficacy. The study may shed light on the need to rethink the starting point in the teacher reform process. As Wagner (2003) and Levin (2003) report, other countries have found a great deal of success beginning the school restructuring process by defining the values and dispositions that they want students to possess at the end of their educational experience, rather than focusing on such outcomes as standardized test gains. If an emphasis on developing social and emotional capital in students is the critical piece to student success, then we as a field may want to shift the way we assess both micro and macro educational efforts. Since currently classroom climate is relegated to a peripheral concern in the process of teacher development, mostly taught in connection to classroom management and rarely as part of the pedagogical domain, this study may provide some support for a more prominent place for it in the teacher preparation process. If in fact there exists an interrelatedness of pedagogy and curriculum that creates “good climate” which in turn leads to better results with students, then understanding the process of constructing “good climate” early in one’s teaching career is essential for high teaching performance. Conclusion The current curricular climate in schools often creates a mindset in new teachers that they must choose between student academic achievement and student social and emotional development. There is growing research to suggest that this choice is not inherently necessary or valid. But the essence of bringing these two domains together seems to be more a matter of the 34 “hows” than the “whats” of teaching. This limited study supports the growing insight into how schools might approach the difficult task of attaining high standards, develop emotionally and socially healthy students, and supporting sound curricular choices, and the place an intentional approach to the area of classroom climate and the added dimensions provided by a curriculum such as Facing History and Ourselves can play in that effort. References Feshback, N. and Feshback, S. (1987) Affective Processes and Academic Achievement. Child Development, 51, pp.1149-56. Freiberg, J. (1999) School Climate : Measuring, Improving, and Sustaining Healthy Learning Environments: Student Outcomes and the Reform of Education. Falmer Press, New York. Haynes, Emmons & Comer (1994) School Climate Survey. Yale Child Development Center, School Development Program. Hoy, W. and Hannum, J. (1997) Middle School Climate: An empirical assessment of organizational health and student achievement. Educational Administration Quarterly, v. 33, pp. 290-311 Hawkins, J. (1997) Academic Performance and School Success: Sources and Consequences. Pp. 278-305. in Enhancing Children’s Wellness, ed. R. Weissberg. Thousand Oaks, CA, Sage. Glasser, W. (1990) The Quality School: Managing Students without Coercion. New York. HarperCollins. Jackson, B. (2003) Education Reform as if Student Agency Mattered: Academic Micro- cultures and Student Identity. Phi Delta Kappan. 84 (8) pp. 579-585 Levin, B. and Wiens, J. (2003) There is Another Way: A Different Approach to Education Reform. Phi Delta Kappan. 84 (9) pp. 658-664. Ostermann, K. (2001) Students’ Need for Belonging in the School Community. Review of Educational Research. 70, 323-367. Patterson, W. (2003) Breaking Out of Our Boxes. Phi Delta Kappan. 84 (8) pp. 568-574. Ryan, A. and Patrick, H. (2001) The Classroom Social Environment and Changes in Adolescents’ Motivation and Engagement During Middle School. American Educational Research Journal. 38(2): 437--60 Schultz, L., Barr, D., and Selman. R. (2001) The Value of a Developmental Approach to Evaluating Character Development Programs: an outcome study of Facing History and Ourselves. Journal of Moral Education, 30(1) pp. 3-27. Shindler, J. (2002) Which type of rubric is best: Exploring various structural options for rubric use. Forum Journal of Teacher Education v. 12, n. 2, pp.3-13. Shindler, J., Taylor , C., Jones, A., Cadenas, H (2003) Don't Smile 'til Christmas: Examining the Immersion of new teachers into existing school climates. Yearbook for the American Educational Research Association Urban Teaching and Learning SIG. 35 Shindler, J., Taylor C., Jones, A., Cadenas, H. (2003, April) Sharing the data along with the responsibility: Examining the efficacy of an analytic scale-based school climate assessment system for K-12 schools. Paper presented at the American Educational Research Association, Chicago, Il. April 2003. Stiggins, R. (2001) Student-Involved Classroom Assessment. Prentice Hall, Upper Saddle River. Sweetland, S. and Hoy, W. (2000) School Characteristics and Educational Outcomes: Toward an Organizational Model of Student Achievement in Middle Schools. Educational Administration Quarterly. 36 (5) pp. 703-729. Wagner, T. (2003) Reinventing America’s Schools. Phi Delta Kappan. 84 (9) pp. 665-668. Wang, M., Haertel, G. and Walberg H. (1997) Learning Influences. Pp. 199-211 in Psychology and Educational Practice, eds H. Walberg and G. Haertel. Berkley. CA. McCutchan. Wilson, D., Gottfredson, D. and Najaka S. (2001) School-Based Prevention Program Behaviors: A Meta Analysis. Journal of Quantitative Criminology 17; pp. 247-272. Zins, J., Weissberg, R., Wang, M. and Walberg H., eds. In Press. Building School Success through Social and Emotional Learning: Implications for Practice and Research. New York. Teachers College. 36