DOCUMENT RESUME ED 479 501 TM 035 139 Public Use Data Set User's Manual. TITLE INSTITUTION Consortium on Chicago School Research, IL. 2000-00-00 PUB DATE NOTE 97p. PUB TYPE Non-Classroom (055) Guides EDRS PRICE EDRS Price MF01/PC04 Plus Postage. DESCRIPTORS Data Analysis; Elementary Secondary Education; *Reports; *Surveys; Urban Schools; *User Needs (Information); Users (Information) IDENTIFIERS *Chicago Public Schools IL; *Data Files ABSTRACT In spring 1999 the Consortium on Chicago School Research conducted surveys of students in grades 6, 9, and 10 and all teachers 7, 8, and principals in the Chicago Public Schools. In all, 54,660 elementary school students, 18,187 high school students, 7,905 elementary school teachers, 2,009 high school teachers, and 315 principals completed the surveys. The results have been used to provide individual school reports. The purpose of this manual is to provide some basic background into the development, distribution, and collection of the surveys, and more detailed information about the creation of the constructs and other variables reported in the individual school reports. The public use data set contains student and teacher data and school-level aggregates. Any information that could be used to identify individual students or teachers has been removed. The manual contains discussions of developing the surveys and data collection procedures. Reports based on the data are described, and both teacher measures and student measures are outlined. An appendix, not included in the Web version of the manual, contains the coded surveys. (SLD) Reproductions supplied by EDRS are the best that can be made from the original document. Turn, 1999 Improving Chicago's Schools: The Teacher ,-. c) Improving Chicago's Schools: The Student's Speak, 1999 kr) A Survey of Students and Teachers in the Chicago Public Schools csr--- N .7r (:*-) W Public Use Data Set User's Manual June 2000 U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement PERMISSION TO REPRODUCE AND EDUCATIONAL RESOURCES INFORMATION DISSEMINATE THIS MATERIAL HAS CENTER (ERIC) This document has been reproduced as BEEN GRANTED BY received from the person or organization originating it. J. Q. Easton Minor changes have been made to improve reproduction quality. ° Points of view or opinions stated in this TO THE EDUCATIONAL RESOURCES document do not necessarily represent INFORMATION CENTER (ERIC) official OERI position or policy. 0.) The Consortium on Chicago School Research 1313 E. 60th Street ce) O Chicago, IL 60637 5 F (773) 702-3364 www.consortium-chicago.org BEST COPY AVAILABLE Table of Contents Introduction 3 The Public Use Data Set 3 Purpose of 1999 Student and Teacher Surveys 4 Chicago Annenberg Research Project 5 Background of The Consortium on Chicago School Research 6 Developing the Survey Instrument 7 Survey Development 7 Content of the Surveys 8 Student Survey 8 Teacher Survey 8 Principal Survey 9 The Analytic Samples 10 Survey Distribution and Data Collection 13 Advance Publicity and Survey Distribution 13 Data Confidentiality Agreements 15 Data Collection Procedures 15 Response Rates 18 Comparison of the Analytic Sample, Volunteer Sample, and Population of Schools 19 Reports Based on 1999 Surveys 24 Measures and Other Variables Included in the Public Data Set 25 A Primer of Rasch Analysis 25 Creating Categories for the Rasch Measures 27 Rasch Measures Included in the Public Data Set 29 Teacher Measures 30 CMTY Use of Community Resources 30 CNCM Ties to Community 32 CNST Knowledge of Students' Culture 33 COLG Peer Collaboration 34 COLR Collective Responsibility 35 INFL Teacher Influence 36 INNV Innovation 38 INST Instruction Leadership 39 NIDE Access to New Ideas 41 OUTR Teacher Outreach to Parents 42 PART Parent Involvement in School 44 PBSV Joint Problem Solving 45 PGMC Program Coherence 46 PRIN Inclusive Leadership 47 QPRD Quality Professional Development 48 REFD Reflective Dialogue 50 SCMT School Commitment 52 1 SLRN Focus on Student Learning 53 SPCH Support for Change 54 TRPR Teacher-Principal Trust 56 TRTE Teacher-Teacher Trust 58 UNPD Uncoordinated Professional Development 59 Teacher Measure Cutpoints 60 Teacher Background Survey Items 61 Student Measures 65 ACAD Press Toward Academic Achievement 65 ACNO Peer Support for Academic Work 67 COMP Social Competence 69 CONS Social Conscience 70 DISO Incidence of Disciplinary Action 71 ENGG Academic Engagement 72 HUMR Human and Social Resources in the Community 74 LIKE Liking School 76 PARS Parent Support for Student Learning 77 PEER Classroom Behavior 79 PERC Classroom Personalism 81 PSUP Parent Supervision 83 SAFE Safety 84 SELF Self-Efficacy 85 SHDM Student Influence in the Classroom 87 TRTS Student-Teacher Trust 88 Student Measure Cutpoints 90 Student Background Survey Items 91 Categorical Variables Includes in the Public Data Set 94 Appendix A: Coded Teacher and Student Surveys Elementary School Teacher Survey High School Teacher Survey Elementary School Student Survey High School Student Survey CODED SURVEYS NOT INCLUDED IN WEB VERSION OF MANUAL. TO OBTAIN CODED SURVEYS, PLEASE CONTACT THE CONSORTIUM AT [email protected]. 2 Introduction In the spring of 1999, the Consortium on Chicago School Research conducted surveys of sixth-, seventh-, eighth-, ninth-, and tenth-grade students and all teachers and principals in the Chicago Public Schools. Seventy-six percent of elementary schools and 71 percent of high schools participated. In total, 54,660 elementary school students, 18,187 high school students, 7,905 elementary school teachers, 2,009 high school teachers, and 315 principals in 378 elementary schools and 65 high schools completed the survey. To date, the results of these surveys have been used to produce individual school reports for most of the schools that participated in the survey (Improving Chicago's Schools: Harding School is an example of these reports). The purpose of this manual is to provide some basic background into the development, distribution, and collection of the surveys, as well as more detailed information about the creation of the constructs and other variables reported in the individual school reports. The Public Use Data Set The public use data set contains student and teacher data, as well as school-level aggregates. To ensure the confidentiality of the respondents, any information that could be used to identify individual teachers or students has been removed. This means unit number in the public data set have been replaced by random school identification numbers, and student identification numbers have been scrambled (teachers were never individually identified). The school identification 3 5 numbers used here are consistent with those used in the public use data sets from other surveys that the Consortium has conducted. The public data set currently consists of measures created for the individual school reports and some school-level demographic data the Consortium traditionally uses in its reporting. As more reports are produced, the data set will be updated to include any new measures or variables created. Purpose of 1999 Student and Teacher Surveys The Improving Chicago Schools surveys are essentially a continuation of the Charting Reform survey series the Consortium has been conducting since 1991. The revised name signals a change in focus away from governance reform and toward school development and improvement. While the major focus has changed from prior surveys, many previously used constructs were included in the Improving Chicago 's Schools surveys, allowing for comparisons across the years. The Improving Chicago 's Schools surveys were conducted with several purposes in mind. Primary among them was to investigate key elements of school organization, parent involvement, and the relationship between community resources and student learning. Specifically, questions were asked concerning the organizational structure of schools, human resources in schools, instructional quality, social support for learning, parent involvement and community resources, and student experiences with and attitudes toward schooling. 4 Also key was the desire to provide individual schools with useful information about themselves to help in their planning and self-analysis, and to guide their improvement efforts. Schools with more than 50 percent of their students or 42 percent of their teachers responding to the survey received a report that summarized their results, and compared them to citywide data. The reports were designed to correspond directly with the Five Essential Supports for Student Learning that are referenced in the SIPAAA. These reports are explained in greater detail later in this manual. Chicago Annenberg Research Project. While many of these topics represent themes the Consortium has been studying for some time, some have their origin in the Consortium's Chicago Annenberg Research Project. The project is a four-year program to document and study the activities of the Chicago Annenberg Challenge. In 1993, Ambassador Walter Annenberg announced a challenge grant to support school reform in the nation's largest cities. The Chicago Annenberg Challenge was organized to manage and distribute the funds from a five-year grant from the National Annenberg Challenge to Chicago public schools. The mission of the Chicago Challenge is to improve student achievement by supporting intensive efforts to reconnect schools to their communities, restructure education, and to improve teaching and learning. The Chicago Challenge funds school networks and partnerships that seek to create successful, community-based schools that address three critical education issues: school and teacher isolation, school size, and time for learning. Items were included in the survey to help address these issues. In addition, the sampling design was set so as to garner participation among a reasonable percentage of schools not receiving Annenberg funding to insure suitable comparisons. 5 7 Background of the Consortium on Chicago School Research The Consortium on Chicago School Research was initiated in 1990 as an independent federation of Chicago area organizations united to undertake research activities designed to advance school improvement in Chicago's public schools and to assess the progress of school reform. The Consortium aims to encourage: broad access to the research agenda-setting process; collection and reporting of systematic information on the condition of education in the Chicago Public Schools; high standards of quality in research design, data collection, and analysis; and wide dissemination and discussion of research findings. The Consortium views research not just as a technical operation of gathering data and publishing reports, but as a form of community education. It does not argue a particular policy position. Rather, it believes that good policy results from a genuine competition of ideas informed by the best evidence that can be obtained The Consortium works to produce such evidence and helps ensure that the competition of ideas remains vital. 6 8 Developing the Survey Instrument Survey Development Like previous Consortium surveys, these were developed through an extensive stakeholder consultation and review process. John Easton, Anthony Bryk, Melissa Roderick, Mark Smylie, Chris Kelly, and Sara Hallman led the survey development. This group collected survey items from across the country. They met with groups of teachers (through the Chicago Teachers Union), principals (through the Chicago Principals and Administrators Association), and central office staff to discuss the content and logistics of the surveys. High school students from Chicago public schools reviewed the student surveys. Researchers also met with members of the Annenberg evaluation committee and other Annenberg study researchers. Consistent with Consortium procedures, two of the Consortium's standing committees also provided advice: the Steering Committee (comprised of faculty from local universities, research staff from Chicago Teachers Union, researchers in education advocacy groups, and staff from the North Central Regional Educational Laboratory and the Illinois State Board of Education) and the Constituent Advisory Board (comprised of teachers, principals, parents, civic and political leaders). Many survey questions were drawn from previous Consortium surveys, especially the 1997 Improving Chicago's Schools student and teacher surveys, which also adhered to a comprehensive stakeholder process. Survey development lasted from the summer to the late fall. Two student surveys, two teacher surveys, and one principal survey were developed. Separate student surveys were developed for elementary and high school students. The high school survey had longer content 7 and included questions that would not be relevant for elementary school students. Separate surveys were created for elementary and high school teachers as well. Content of the Surveys While the content between the elementary and high school surveys did differ Student Surveys. somewhat, all students were asked about their academic experiences, including classroom activities, homework, and the behavior of other students. In addition to their academic experiences, students were also asked about issues of safety and discipline, their motivation and expectations for learning, the peer culture of the school, the community and their involvement in it, the degree to which their parents were involved in their education, supports for new students or students after absences, the summer school they attended in 1998, and their educational plans and aspirations. Background questions, such as whether English is the language spoken in the home, whether they were born in the United States, and questions about their parents' education were asked to allow researchers to examine differences among various groups of students. Many of the items used in the 1999 survey came from the Consortium's 1997 Improving Chicago's Schools surveys and the 1994 Charting Reform surveys. In addition, some were pulled from other surveys, such as the National Educational Longitudinal Study of 1988 and the First Follow-Up of 1990. Some items were also created specifically for these surveys. Teachers answered questions about leadership in their school, their use of Teacher Surveys. community resources, parent involvement in their student and the school, students' capability to learn, the professional community in their schools, their professional development activities, and 8 10