Benchmarking Exponential Growth of Educational Reform: The Sustainability Index Ray Garcia California State University, East Bay AfterdecadesofreformeffortsinpublicschoolsystemsintheUnitedStates,thereis minimalstudentachievementprogresstomeasure.Thisarticleaddressestheongoing challengesandcomplexitiesofthesustainabilityofeducationalreformthroughare- viewoftheliteratureandtheproposalofaSustainabilityIndexasametrictobench- markthesustainabilitystatusofareform.Highlightinga2-yearstudyinanurban schoolsysteminCalifornia,thisworkanalyzesthenavigationofaneducationalre- forminitiativeusingaSustainabilityIndexasatoolforreformplanning.Theresults ofthestudyidentifyanumberofpotentimplicationsforreformers,schoolleaders and university preparation programs. Introduction Over the past three decades, educational reform efforts in the U.S. have been peppered with educators’ and politicians’ rhetoric of their commit- mentthatallchildrenwilllearn.Whileinnowayanindictmentofthiscom- mitment, the startling actuality is that there has been little progress to measure.Onecouldarguethatthiswidespreadcommitment,coupledwith considerable financial investments in education over this same period, shouldhaveresultedinsustainedimprovementofpublicschoolsystems. Conzemuisand O’Neill(2003)offerthissobering thought: Eventhemostoptimisticpersonwouldhavetoadmitthattherehasbeenlittleoverall improvementinthequalityofU.S.publiceducation.Statelegislatures,thefederal government,businessandindustry,andspecialinterestandcommunitygroupshave pumpedbillionsofdollarsintoeducationalreform.Theyhavespentcountlesshours indebateandstrategysessionsandhaveaddedtheirvoicestothegroundswellofcalls demandingbettereducation(p.4). Thecloseofthe20thcenturysawapushforincreasedaccountabilityin schools in the cry for longer school days, extended school years, fewer electives, high school exit exams, and competency standards (Keefe and Amenta,2005).Thefederalgovernmenthasaddeditsowntaketotheedu- cational reform arena. In 2001, the U.S. Congress approved a reauthorizationoftheElementaryandSecondaryEducationAct,renaming itastheNoChildLeftBehindAct(NCLB).Thislegislationincludesaper- formance-based accountability system built around student test results. CAPEAEducationLeadershipandAdministration,Vol.21—2009 72 ©2010DEStechPublications,Inc. BenchmarkingExponential Growth of Educational Reform 73 Thisincreasedemphasisonaccountabilityrepresentsanimportantchange frompastfederaleducationalinitiativesthatfocusedprimarilyonthepro- vision ofservices(Stecherand Kirby, 2004). Despite the accountability thrust, the outlook for the impact of public school reform is bleak. Nichols, Glass, and Berliner (2005) found that in most states, high stakes testing programs associated with accountability systemswereineffectiveinachievingtheirintendedpurposesandcaused severeunintendednegativeconsequences.Theeconomictimesfacingthe nationservetoexacerbatethechallengeofadvancingschoolreforminan eraofdeclining resources. Ifthisdilemmaistochange,publicschoolsystemsmustberesponsiveto whatittakestosustainschoolimprovement.Forsomeeducators,system- aticcustomizationofcurriculumandinstructionisthesolutiontosustain- ing school reform. For others, the need is on addressing the entrenched patterns in daily practice that are all too common in schools (Southworth andDuQuesnay,2005).Theseissuesarebeingcalledintoquestioninthe body ofresearch on reformsustainability. Sowhyhaveeducationalreformeffortsfailedtosustain?Afterdecades of countless reforms and billions of dollars invested into closing the stu- dent achievement gap, why have educators been unable to sustain school improvement efforts? This article introduces an approach to measure the sustainabilityofeducationalreforms.TheSustainabilityIndexintroduced in this work benchmarks the growth of reforms and is designed to help school leaders examine organizational structures for meaningful, lasting schoolimprovementreform. Highlighting a two-year study in an urban school system in California, thisarticleanalyzesthenavigationofaneducationalreforminitiativeus- ingtheSustainabilityIndexasatoolforreformplanning.Supplementedby an examination of the professional literature on sustainability, this paper addressesthefollowing questions: 1. What are the prevalent challenges in sustaining school reform efforts? 2. What role can the Sustainability Index play in the monitoring of educa- tional reform? Undergirdingtheseresearchquestionsisadiscussionoftheroleofuni- versity leadership preparation programs in the area of reform sustainability. To address these research questions, clarification of two concepts is in order.Firstandforemost,whatismeantbyschoolreform?Labelingreform as innovation, Goodson (2001, p. 45) offers the following elaboration on theconcept, “...educationalinnovationrepresentsa‘coalition’ofinterestsandprojectsbrought togetherunderacommonlabelataparticularpointintime.Stateddifferently,anin- novationcanbeconsideredanexpressionofpeople’svalues,beliefs,politicalopin- ions,andmorals—embeddedwithinaparticularpowercontext.”Thetermreform, 74 CAPEAEducation Leadership and Administration therefore,isusedinthisworktodenoteeducators’effortsand/orinitiativestoim- provestudentachievement. The second concept to be clarified is the term sustainability. Research unveils numerous attempts to define the concept of sustainability. HargeavesandFink(2000)assertthatsustainabilitydoesnotsimplymean whether a reform can endure. The concept expands to how initiatives are developedwithoutcompromisingthedevelopmentofotherreformsinthe surrounding environment. Century and Levy (2002) argue that sustainability is the capacity of a program to withstand shocks over time whilemaintainingcorebeliefsandvaluesandusingthemtoguideadapta- tiontochange.Inthiswork,theconceptofsustainabilityisusedtocapture educators’ actions at maintaining a reform in place for the duration re- quired to bring aboutschoolimprovementefforts. Sustainability of Educational Reforms: The Chronic Challenge Pressure for public school reform comes from the political, business and privatesectors,associetydemandsmoreofschoolsinaneraofdiminish- ing resources. With instant snapshots of a school’s performance at the touchofthepublic’sfingertipsviatheInternet,thisaccesstoinformation hasresultedinnational,stateandlocalgovernments’transparentdemands forschoolstoincreasestudentperformance.FlettandWallace(2005)con- tendthatalthoughthecallsforreformmaybemany,successfulreformsare farlessfrequent,andinsomecases,failureisalmostpredictable.Thefac- tors contributing to successful reform in one scenario do not necessarily work in another. Theresearchonthefailureofschoolreformisdocumentedinthelitera- ture.Noguera(2004)highlightsastudyof10schoolsintheBostonPublic School System that were undertaking a variety of reform strategies. A closerlook,however,revealedanalltoocommonpattern.Theresearcher discovered thatthereformeffortswerefragmented and managed ineffec- tively. Eightofthe10 schoolsdemonstrated no achievementgains. Inanotherstudy,Datnow(2005)offersapersuasiveprofileofreform’s inability to sustain. The researcher’s 3-year longitudinal study of six re- formmodelsin13schoolsinoneurbandistricthighlightsthechallengesof reform sustainability in the context of organizational flux. All of the re- formsinthedistrictcalledforsignificantchangesaffectingwholeschool arrangements.Theresearcherconcludedthatafterthreeyears,reformshad expiredinsixofthe13schoolsstudied;twootherschoolswerestillimple- menting reforms, but at low levels. Only five of the 13 schools were still continuingtoimplementtheirreformdesignswithmoderatetohighlevels ofintensity. Offering a structural perspective on the lack of reform sustainability, KeefeandAmenta(2005)positthatschoolshavebecomeincreasinglyob- solete.TheorganizationalstructureofthetypicalAmericanschoolisanti- BenchmarkingExponential Growth of Educational Reform 75 quatedandisbecomingmoreandmoreoutdated.Theinfrastructureofthe public school system was developed when the country was building a blue-collarworkforceandonlyafewstudentswereexpectedtogotocol- lege. Whilesomeevolution in curriculahasoccurred, thecorebelief sys- temsremain relatively unchanged in schools. Whatiscleartoresearchersandpractitionersalikeisthatschoolsaresuf- fering from the perils of reform stagnation. The institutional muscles in schools—thewillingnesstochangewhennecessary—haveatrophied(El- more, 2000). The failure to build organizational structures to sustain re- form is seen as a culprit behind the chronic nature of the sustainability challenge.Morethanever,inthiseraofdecliningresources,theinability of school reform to sustain is being called into question. A theoretical modelfromwhichtoanchorassertionsaboutreformsustainabilityfollows in thenextsection. The Sustainability Model Fullan (2005) suggests that bringing about reform means changing entire organizational school structures. Theresearcher counsels that setting tar- getsforannualyearlyprogress,asinthecaseofNoChildLeftBehind,will changeonlyatinysliceofthecontext,andisneitherlargeenoughnorpow- erfulenough to motivateorganizationsto reformin orderto succeed. Knight and Erlandson (2003) documented the importance of organiza- tionalstructuresinschoolsandtherolethosestructuresplayinsustaining reforms. Other researchers have drawn attention to incoherent school re- formswherediverseinitiativesareset up to serveimportant needs. How- ever, Newman, Smith, Allensworth, and Bryk, (2001) found that these initiativeslackthesustainedsupportofthemajorityofstaffwithinaschool and resultin no apparentimpacton improving studentachievement. Taylor(2006)remindsusthatexistingresearchhasarguedthatsustainabilityisasso- ciatedwithanarrayofplausiblevariables(i.e.,schoolcapacity,politicalcontext, funding, alignment, leadership, faculty, retention, professional development, etc.) buthasnotyetprovenconsistentlinkagesbetweenthesevariablesandsustainability. So what type of structural adjustments in the organization of schools can leaders maketocompensateforthelackofresourceswhilesustainingschoolreform?Inear- lierwork,thisauthorintroducedaSustainabilityModelreflectingtheinterrelation- ship of three school organizational structures or dimensions as a framework for conceptualizing the sustainability of educational reform (Garcia, 2008). The Sustainability Model (Figure 1) was intended to build on the reform framework craftedbyKnightandErlandson(2003)andtakingittoanewlevelofapplication. Figure1highlightsthetriangulationofthethreedimensionsofsustainabilitytoillus- tratetheinterrelationshipsamongthethree. TheSustainabilityModelreflectsasystem’sframeworkbasedonthree reciprocal, interdependent organizational dimensions: commitment, con- gruenceandcoherence.Thesethreedimensionswereselectedbasedonthe extensive, longitudinal research undertaken by Knight and Erlandson (2003)inexaminingthemultitudeofvariablesandcomplexitiesthathave 76 CAPEAEducation Leadership and Administration Figure 1.Triangulation of the Three Dimensions of theSustainability Model. frustratedreformersfordecades.Asaresultoftheirseminalwork,there- searchersidentified theattributes ofcommitment, congruenceand coher- enceasprimary variablesin theeducationalreformarena. TheSustainabilityModelinthisworkallowsfortheplottingandthenav- igation of school reformby having school leadersaddresssuch questions as:Whatisthelevelofstaffcommitmenttoagivenreform?Towhatdegree isthereformcongruentoralignedtotheschool’sgoals?Howcoherentis the reform in working with other initiatives in the school? In answering these questions using surveys, focus-groups, or one-on-one interviews, schoolleaderscanquantifyandplottheresultsontheSustainabilityModel affordingtheopportunitytodiscernthestrengtheningand/orweakeningof thebondsbetween thethreedimensionsofthemodel. Thetriangulationtechniquenotonlytellsleaderswhereareformis,but also,ittellsthemwhattodo.Usingthemodelasanavigationalmapallows reformers to diagnose and subsequently take action on any or all of the threedimensionsoftheSustainabilityModel.Thisreform-navigationtool is used to identify patterns and trends in the deployment of a reform and subsequently allow leaders to make necessary adjustments to extend, ac- celerate, or terminate a given reform. The implications of the tool are far reaching. How is the plotting and navigation of a given reform carried out? The survey in this study asked questions addressing each of the three dimen- sions of the Sustainability Model: commitment, congruence and coher- ence.Respondentswereaskedsurveyquestionsabouttheircommitmentto BenchmarkingExponential Growth of Educational Reform 77 thereformunderstudy;questionsoftheirperceptionsofthecongruenceof the reform to culture of the school; and, questions of their perceptions of thecoherenceofthereformtootherinitiativesorprogramsintheirschools. Themeanofsurveyresponsesundereachofthedimensionswascalculated and plotted on theSustainability Model. TheSustainabilityModelprovidesafoundationforanalyzinganddevel- oping discussions about reforms in the context of school organizational structures required for sustainability. An understanding of the Sustainability Model brings to the forefront the clear tensions among the three organizational dimensions. The three dimensions expand and con- tract, sometimes independently, sometimes dependently, and yet at other times interdependently, each contingent on the conditions of the reform. Throughadeeperanalysisofthismodel,thedegreeofsustainabilityofare- form can be quantified, plotted, and graphed to reflect the strengthening and/or the weakening of the bonds between the three dimensions. Subse- quently,schoolreformcanbemappedallowingleaderstomakestructural adjustmentsin schoolsforreformto sustain. Implicit within this Sustainability Model is the construct of reform fo- cus. In other words, when mapping the course of a given reform on the Sustainability Model, where does the center point of the reform reside? Thiscenterpointisdeterminedbycalculatingthemeanofthethreedimen- sions of the Sustainability Model. It is out of this calculation that the SustainabilityIndex(SI)hasitsinception.TheSustainabilityIndexcanbe used to determine the status of a reform. The next section profiles the Sustainability Index and itcalculation in moredetail. Practical Application of the Model: The Sustainability Index Thereisachorusofresearchersofferingstrategiesforframingbenchmarks forgaugingsustainability.KnightandErlandson(2003)provideaframe- workforexaminingfourvariables(commitment,congruence,coherence, andcontinuity)thatcompriseeducationalreform.Similarly,Goodmanand Steckler (1989) describe sustainability structures along two dimensions: Thebreadthofthereform’sintegrationandthedepthoftheinitiative.Cen- turyandLevy(2004)goasfarastoofferamathematicalequationincorpo- rating amultitudeofvariablesto gaugesustainability. Theseresearcherssuggestthatwhatwasonceperceivedtobeanincalcu- lable factor in reform sustainability is now a reality. Indeed the sustainabilityofareformcanandshouldbemeasuredifreformeffortsare tobesustained.HencearationalefortheSustainabilityIndexisunearthed. The fundamental premise of the Sustainability Index is based on re- search.Itseeksconsensustowardsareliablemetrictoidentifythedegree ofsustainingpowerofschoolreform.TheSustainabilityIndexisanaver- ageofthethreedimensionsofthemodel.TheSustainabilityIndexiscalcu- lated from the “mean of the means” of the three dimensions of sustainability. The index is agnostic to the reform or program it seeks to 78 CAPEAEducation Leadership and Administration measure.ThenextsectionprofilesastudywheretheSustainabilityIndex wasused asabasisfordecision-making in reformsustainability. An Urban Leadership Study Building school capacity for reform sustainability is not an easy task. Hallinger and Bridges (1997) note that school reformers must shift their roles toward supporting and developing the organization’s capacity for change.Cultivatorsofsustainabilityre-createaschoolculturethathasthe capacity to stimulate continuous improvement on a broad front. This ca- pacity in turn enables people to adapt to and prosper in their increasingly changing environments and hopefully sustain reform with flexibility (Capra,1997).Thebuildingofschoolcapacityrequiresleaderstoattendto the three dimensions of the Sustainability Model: commitment, congru- ence, and coherence. Directed by the literature on organizational structures that enhance re- form sustainability, a study was conducted in an urban school system in California.Thisstudyanalyzedtheimpactofaleadershipcapacitybuild- ing training initiativeconducted during the2006–07 and 2007–08 school years. This study focused on a descriptive analysis of the attitudinal sur- veysadministeredtoteachersandschoolleadersabouttheirperceptionsof the training reform initiative using the three dimensions of the Sustainability Model: commitment, congruence, and coherence. There were 77 schools, approximately 2,344 teachers and 164 administrators in thedistrictatthetimeofthestudy. Seeking to investigate the relationship of the three dimensions of the SustainabilityModelandtheirroleindeterminingthesustainabilityofthe systemic training initiative, the purpose of the study was to examine the practicality oftheSustainability Index. Methodology Beginning in the spring 2007 and throughout the 2007–2008 academic year, this researcher provided training and follow-up activities on Lam- bert’s(2003)modelforbuildingschoolleadershipcapacitytothedistrict’s 77 school leadership teams that were charged with ensuring that school structureswereinplaceforcontinuousimprovementofteachingandlearn- ingateachsite.Thethree-daytraininginitiativefocusedondevelopinga skillsetinleadershipteamstomonitorthequalityofclassroominstruction in theirrespectivecampuses. Earlyinthedevelopmentofthisinitiative,theschooldistrictexpressedaninterestin ensuringthesustainabilityoftheinitiativegiventhetimeandfinancialinvestmentof thereformeffort.UsingLambert’s(2003)30-itemsurvey,theinstrumentwasdis- tributedtoallteachersandschoolleadersinthe77schoolsinthedistrict.Usinga four-point(1–stronglydisagree;2–disagree;3-agree;4–stronglyagree)Likertscale, thesurveyitemsaskedparticipantstorespondtoeachofthesurveyitemsbasedon theirperceptionsoftheimpactofthetraininginitiative.Afactoranalysiswascon- BenchmarkingExponential Growth of Educational Reform 79 ductedontheresponsesofallofthesurveyitemsasadatareductionefforttocreate composite variables of the three categories on the Sustainability Model: commit- ment, congruence, and coherence. This factor analysis enabled the researcher to identifyunderlyingpatternsofrelationshipsbyrearrangingorreducingthedatato smallersetsoffactors. Inthefallof2006,thesurveyunderwentreviewbyschoollevelpracti- tionerswhocommentedonitsrelevancetothereformbeinginvestigated; by district policymakers to ensure that the instrument captured the dis- trict’spoliciesaccurately;andbyacademicstosecurefeedbackonthesur- vey’sfocusandwording.Afterthisinitialreview,thesurveywaspilotedin twelvedifferentelementaryschoolsinthedistrict.Principalsandteachers were requested to take the survey and then interviewed to glean their im- pressions of the instrument, what questions required clarification and to whatdegreethesurveycapturedwhatwasimportanttothemwithregardto thereforminquestion.Thisfield-testingallowedfortheadministrationof thesurvey in thesubsequentspring. Thefirstsurveywasadministeredinthespringof2007withthesecond surveyadministeredinthespringof2008.Thisstudyuseddatafromthe77 schoolsthatparticipatedinthesurveysinbothyears.Theresponseratefor the2007administrationwas67percentcomparedtotheresponserateof76 percent in the 2008 administration. Responses were averaged in each schooltoproduceameasureofeachorganizationallevel(elementary,mid- dle, high school, and district composite) in the three dimensions of the Sustainability Model. For the purposes of this study, the 2007 survey re- sultsareclassifiedasPre-Resultsandthe2008surveyresultsaredenoted as Post-Results. Table 1 reflects the means, standard deviations, and the Sustainability Index ofboth administrationsofthesurvey. In Figure 2, the means of the survey’s three factors are plotted on the SustainabilityModeltoillustratetheinterrelationshipamongthethreedi- mensionsofthemodel.Plottingthesurveyresultsbyorganizationallevels (elementary, middle, high school, and district composite) allowed for the identification of any fluctuations in the three dimensions of the SustainabilityModelfromthePre-ResultstothePost-Results.Thesefluc- tuations are described in the findings of this study. Furthermore, plotting the snapshots of the respondents’ perceptions on the three dimensions of theSustainabilityModelprovidedaninfrastructurefromwhichtohostfo- cused dialogues on adjustments to theimplementation of thetraining ini- tiative.Patternsandtrendsbetweenthepreandpostsurveyresultsamong theschoollevelgroupsofferedabasisfordiscussionaboutthestatusofthe reformin thisstudy. As the data were interpreted, two provisions were kept in mind. First, sensitivity was clearly established with both survey administrations that the data collection was not an evaluation mechanism of the respondents’ performance in schools. Secondly, it was important for the researcher to identify any distinguishing sustainability trends between elementary and secondary respondentsin orderto modify futuretraining approaches. 80 CAPEAEducation Leadership and Administration Table 1 Means, Standard Deviations, andSustainability Index for the Three Dimensions of theSustainability Model. Means, Standard Deviations, and Sustainability Index Sustainability Commitment Congruence Coherence Index Pre N=937 2.83/0.16 3.13/0.39 2.69/0.49 2.95 Elementary Mean/SD Schools Post N=1,047 3.14/0.12 3.39/0.32 2.94/0.51 3.22 Mean/SD Pre N=230 2.83/0.15 3.09/0.32 2.67/0.34 2.92 Middle Mean/SD Schools Post N=249 3.21/0.09 3.38/0.23 3.03/0.24 3.26 Mean/SD Pre N=396 2.56/0.21 2.96/0.30 2.72/0.48 2.78 High Mean/SD Schools Post N=482 2.96/0.09 3.23/0.25 3.02/0.39 3.10 Mean/SD Pre N=1,563 2.78/0.17 3.08/0.33 2.71/0.45 2.91 District Mean/SD Average Post N=1,778 3.11/0.11 3.34/0.26 2.96/0.41 3.19 Mean/SD Additionally, it should be noted that there are inherent limitations in a self-reportedsurveystudyofthiscaliber.Thesurveyresponseswereper- ceptionsofschoolstaffregardingtheiropinionsabouttheircommitment, coherenceandcongruenceofareforminstudyandthereforemaynotrep- resenttheactualreformsustainability. Findings Thequestionofthegeneralizabilityofthisstudycannotbedefinitivelyan- sweredsincethisresearchstudyreflecteddatafromasingleurbanschool system.Thisstudyhasunveiledkeyfindingswhichincludethefollowing: 1. The Sustainability Index (SI) increased at all school levels from the BenchmarkingExponential Growth of Educational Reform 81 Figure 2.Pre & Post Survey Results Plotted by School Level. Note:Pre-Results (2007) are denoted by dotted lines. Post-Results (2008) are reflected by broken lines. SI represents theSustainability Index. Pre-Results to the Post-Results. The highest SI in the post year’s results wasatthemiddleschoollevelwithanSIof3.26. Thisgradelevelgroup also had the highest increase (0.34) in the SI from the previous year. 2. The Commitment Dimension displayed its prevalence with increasesbe- tweenthePreandPostResultsacrossallschoollevels:elementary,middle, and high schools. The high school group reflected the highest increase fromthepretothepostresultsinthisdimensionwithadifferenceof0.40in the means.