Running head: DESIGN AND IMPLEMENTATION OF AN SIS 1 From Data Poor, Information Poor to Data Rich, Information Rich Decision-Making: Design and Implementation of a Student Information System John M. Burgera, Anna Nadirovab aRocky View Schools, Alberta, Canada bCalgary Board of Education, Alberta, Canada DESIGN AND IMPLEMENTATION OF AN SIS 2 Abstract In this paper, we report on the design and implementation of a state-of-the-art Student Information System (SIS) being developed in a moderate size Alberta, Canada school jurisdiction. The SIS design is premised upon transparency, balanced, comprehensive, and multi-method assessments that link qualitative and quantitative data in support of enhanced, evidence-based instructional leadership. Practical examples of data analytic strategies are discussed in relationship to educational leadership enhancement. The article concludes that as SIS design evolves more optimal, data- informed supports for student learning will be realized. DESIGN AND IMPLEMENTATION OF AN SIS 3 Rocky View Schools (RVS) is a school jurisdiction with approximately 18,000 students in Alberta, Canada. The RVS 2011-14 Three-Year Education Plan directs staff to “Develop a system- wide, balanced, and integrated student information system that supports instant [timely] diagnostics of individual student achievement, as well as aggregated data at the classroom, school, and jurisdictional levels” (Rocky View Schools, 2011). Development of a comprehensive and balanced Student Information System (SIS) represents a fundamental investment of time and staff resources. The desired outcome related to the development of a SIS is enhanced school and system student diagnostics that will support educational leadership and action research focused on achieving higher degrees of student success. Demonstrating that this outcome is achievable is necessary to convince staff and stakeholders that the required implementation investment is warranted. Related Research The RVS Strategy to develop the SIS is premised upon an emerging focus of educational research pointing to the importance of student information systems that are transparent, multi- method, balanced, comprehensive, and directly linked to educational and instructional leadership. Educational excellence requires transparent and open education systems that use data and evidence to build partnerships with students, parents and communities in moving schools to higher performance (Fullan, Hill & Crevola, 2006). An example of timely, strategic and effective use of student data is suggested by Allignton, (2011, p. 41) who observes, “We could know on the second day of kindergarten who is at risk of becoming a struggling reader, but we typically do nothing with this information.” Black and Wiliam’s (1998) meta-analysis of student assessment studies builds on earlier research (Bloom, 1980) and has demonstrated that balanced student assessment premised on formative assessment linked to effective teaching methods and appropriate summative assessment with active feedback to students, teachers and parents are key to effective pedagogy. The DESIGN AND IMPLEMENTATION OF AN SIS 4 development of balanced models of student assessment is premised on capturing comprehensive student data generated by classroom assessment triangulated with other useful student data. Rocky View Schools is actively following up on this research with the view to institutionalize balanced assessment models through the development and implementation of Policy HK, Assessment and Communication of Student Learning (see http://www.rockyview.ab.ca/board_policies/boardpolicies/curriculum/HK-Assessment- CommunicationofStudentLearning20090604.pdf/view). Supportive of the RVS approach to more balanced and holistic student assessment models, Parrilla, McQuarrie, Klassen, Georgiou & Odishaw (2010) have suggested that contextually sensitive and comprehensive student data and evidence-informed decision-making are central components of effective school leadership. They have observed, when considering data on students’ literacy achievement, that “…the methodological concerns with the evidence underline the importance of local assessment of the effectiveness, and implementation quality, of any reading interventions that are undertaken in schools” (p.66). Essentially their observation aligns with Buzick and Cahalan-Laitusis`s research (2010) that supports the conclusion that organizational efficacy can be improved through proactive and transparent school and system response to accountability results – processes that are enhanced with strong student information systems. These research-based observations suggest that schools need rich data to make well informed decisions, not only about what is working or not working, but also to identify specific strategies premised on a full and balanced picture of students, including their learning needs, aspirations, abilities, attitudes and interests. As Couture (2011, p. 38) has observed, “to foster school cultures of creativity and ingenuity in Alberta we must build on the strength of an already strong teacher force by supporting ongoing school-based research and innovation, and continually pushing the limits of sound teaching practice, curriculum design and school development.” Extrapolating Couture’s DESIGN AND IMPLEMENTATION OF AN SIS 5 observations and building on the “action research” focus of Cycle Five of the Alberta Initiative on School Improvement would suggest that Alberta schools are entering a world where student data will fuel a new era of enhanced data generation and analysis. Supplemental student achievement data is readily available by tapping into a rich source of data - the classroom assessments that are compiled throughout the school year. It has been estimated that a student through 12 years of schooling will experience 600 classroom-based summative assessments compared to approximately 16 summative assessments based on external instruments. Incorporating classroom-based achievement data into a comprehensive student information database can open doors to greater data richness, added relevance, deeper meaning and resultant enhanced teacher ownership of student achievement improvement strategies (Burger, Nadirova, & Besenski, 2011). One caveat, however, is that comprehensive models of classroom assessment should be based on a balance between formative and summative as well as criterion-referenced and norm-referenced assessments, but the bulk of classroom assessment ideally should be formative, criterion-referenced and authentic (Burger & Krueger, 2003). Balanced Assessment Models Teachers need to know their students individually and in terms of group dynamics to better meet their students’ needs as learners. This means not only multiple sources of data are necessary, but also both quantitative and qualitative methods. Emerging assessment tools and technologies should be incorporated into student assessment practice in order to advance efficacy and establish meaningful and useful linkages between external assessment resources and ongoing, classroom- based assessment. External, often commercially produced, assessment instruments that can produce useful, triangulated data are discussed next. DESIGN AND IMPLEMENTATION OF AN SIS 6 External Achievement Tests Criterion referenced tests such as the Alberta Provincial Achievement Tests (PATs) or norm- referenced tests such as the Canadian Achievement Test provide useful benchmarks of student achievement for different purposes and provide complementary pictures of student learning. Being criterion-referenced, the Alberta PATs give a good indication of how well a student has learned the intended outcomes of the Alberta Program of Studies. Norm-referenced achievement tests, on the other hand, compare a student’s level of achievement in core curricular subjects compared to his or her peers. Also, classical test theory tells us that no test is perfectly reliable or valid; hence, all test data must be interpreted in light of the test confidence intervals and relative to triangulated and trend data to have a clearer sense of how accurate and informative any test result may be. Ability Tests Ability tests such as the Canadian Test Centre’s Insight (http://www.canadiantestcentre.com/insight/insight.asp) can be group administered. Group administered IQ tests are less reliable but less costly than individually administered tests such as the Wechsler Scales (WISC-III, WPPSI-R). However, as Salvia and Yesseldyke (2004) note, these instruments can provide useful information regarding whether student cohorts are achieving to their potential and can provide some preliminary insight into individual students’ relative verbal and performance abilities thus informing teaching and learning strategies. Aptitude Tests Aptitude tests are less widely recognized and used, but provide useful information on what programs may best fit a student in high school or post-secondary learning. The Differential Aptitude Test (DAT) is a good example of an instrument that provides timely DESIGN AND IMPLEMENTATION OF AN SIS 7 information at key points in a student’s program decision-making (http://www.creativeorgdesign.com/tests_page.htm?id=84). Interest Inventories Instruments that measure student interests, such as the Safran Student’s Interest Inventory (http://career.nelson.com/ceg/safran.html) or more sophisticated computer-based career planning-interest assessments such as the Web 2.0 based resource myBlueprint (http://www.myblueprint.ca/discovery-explorations/) provide important information that give insight into what turns students on to learning. When linked to attitude and aptitude assessment data, information on a student’s interests can enable teachers and counselors to evolve a more refined picture of how to optimize education programs to maximize a student’s motivation and success. One RVS high school, Bert Church, has completed a pilot using the myBlueprint resource and is reporting very positive results and a commitment to proceed to school-wide implementation. Attitude Assessment There is considerable accumulating evidence on the importance of emotions and emotional competencies in student learning (Collaborative for Academic, Social and Emotional Learning, 2003; Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011; Parker, Saklofske, Wood, & Collin, 2009). Recent research on the development and internal measurement properties of a new instrument, the Student Orientation to School Questionnaire (SOS-Q), designed to provide comprehensive, yet succinct assessment of core facilitators of engagement and persistence from the student-centered perspective was reported by Burger, Nadirova and Keefer (2012). In summary, when considered comprehensively, external assessment data that captures a student’s achievement in criterion and norm-referenced contexts in combination with ability DESIGN AND IMPLEMENTATION OF AN SIS 8 indicators, aptitude measures, and attitude and interest indicators can provide powerful data sets that complement and add enhanced understanding to a student’s assessment profile that emerges from solely classroom-based assessment. Recent Non-Cognitive Assessment Research Attitudes linked to motivation can affect achievement early in a student’s experience of schooling. In her recent ethnographic study of a southeastern United States kindergarten class, Hatt (2012) investigated students’ evolving concept of “smartness… tied to notions of academic identity” (p. 439). She observed that kindergarten students were “…more likely to be framed as smart if parental expectations closely align with those of the teacher.” Although Hatt’s qualitative methodology does not permit generalization, it is fascinating to read an account where students’ socio-economic and racial backgrounds contribute to a situation where they “…learn early on school is not where they belong or worth investing in, so they begin to disengage” (p. 456). Non-cognitive assessment of student attitude and affect is an emerging field in Canada. The province of Alberta is using the Tell Them From Me instrument (http://www.thelearningbar.com/about.php) applied to grade, school and jurisdiction cohort measures as part of a province-wide pilot project to inform high school completion strategies. The Student Orientation to School Questionnaire (SOS-Q), noted above, measures a range of students’ experiences in school and beyond that may affect the observable outcomes, such as low engagement, attendance, achievement and high school completion. The instrument makes it possible to evaluate student disposition toward school at multiple levels - individual students, school, or the jurisdiction. The SOS-Q was developed and tested by researchers at the Alberta Department of Education in cooperation with the Parkland School Division (Nadirova, Burger, Clarke and Mykula, 2008) and subsequently was acquired by the DESIGN AND IMPLEMENTATION OF AN SIS 9 Educational Testing Service (ETS) for marketing and administration support (see http://www.etscanada.ca/afl/sosprogram). Nadirova and Burger (2012) recently demonstrated that Grade 9 student achievement in Mathematics and Language Arts measured by standardized provincial achievement tests was significantly lower for students manifesting lower scores on the SOS-Q. Furthermore, their research demonstrates that student orientation to school measured by the SOS-Q with Grade 9 students has a strong predictive relationship to high school completion three years later. In this analysis, students scoring in the top quartile of the SOS-Q scale had an 85.9% high school completion rate three years after entering Grade 10 whereas students scoring at the bottom had a 61.3% completion rate – a 24.6% difference! Therefore, it is clear that diagnostically assessing and acting on student attitude at the individual or cohort levels can significantly support student academic success. Rocky View Schools (Burger, Cardinal, Hennig, Valerio, Ziegler, and Nadirova, 2012) is engaging in action research on innovative ways to measure students’ affective school experience using the SOS-Q. Initially focusing on cohort analysis, the SOS-Q was administered anonymously to students in five RVS schools. As one of the principals observed, While we use many sources of data to help us evaluate our success in the area of instructional design, measuring school culture, specifically how students perceive themselves fitting into the culture, is more elusive. The SOS Q has helped us by providing specific feedback in this area. Our goal to build a culture that allows students to thrive in a faith context has everything to do with building a safe and caring school, individual self- DESIGN AND IMPLEMENTATION OF AN SIS 10 confidence flowing from understanding themselves in light of faith and leadership, building both internal and external resilience, finding purpose in life while seeing a connection between school and that purpose, and experiencing positive peer relationships. These are all areas measured by the SOS Q and the feedback we receive from this instrument is one more tool we use to assess our success as a school. As we move forward with the use of this survey, we see potential value in using the instrument to identify specific students who are not experiencing the culture we intend for them. We will then be able to address the areas where they do not feel supported and work diligently to fill the gaps, ensuring that every student feels valued. The use of the SOS Q is all about ensuring we are actually living our mission. (Burger, et.al., 2012, p. 28) Current research, supported with a grant from the Alberta School Community Wellness Fund, involves over 4000 RVS upper elementary, junior and senior high students who have been administered the SOS-Q with an opportunity for voluntary self-identification to permit individual follow-up. A key objective of this research is to identify supports or strategies to enhance school connectivity for students who manifest weak orientation to school. Taines (2012) has reported positive results of a qualitative study where student involvement in school activism generated stronger school connectivity, especially for mild and moderately alienated students. The on-going RVS research with the SOS-Q will provide an opportunity to build on the research reported by Taines and others, while also providing information on the benefits and risks of including data on student affect as a standard resource in the RVS student information system.