Springer Texts in Education David Andrich Ida Marais A Course in Rasch Measurement Theory Measuring in the Educational, Social and Health Sciences Springer Texts in Education Springer Texts in Education delivers high-quality instructional content for graduates and advanced graduates in all areas of Education and Educational Research. The textbook series is comprised of self-contained books with a broad and comprehensive coverage that are suitable for class as well as for individual self-study. All texts are authored by established experts in their fields and offer a solid methodological background, accompanied by pedagogical materials to serve students such as practical examples, exercises, case studies etc. Textbooks published in the Springer Texts in Education series are addressed to graduate and advancedgraduatestudents,butalsotoresearchersasimportantresourcesfortheir education, knowledge and teaching. Please contact Natalie Rieborn at textbooks. [email protected] for queries or to submit your book proposal. More information about this series at http://www.springer.com/series/13812 David Andrich Ida Marais • A Course in Rasch Measurement Theory Measuring in the Educational, Social and Health Sciences 123 DavidAndrich IdaMarais Graduate Schoolof Education Graduate Schoolof Education TheUniversity of Western Australia TheUniversity of Western Australia Crawley, WA,Australia Crawley, WA,Australia ISSN 2366-7672 ISSN 2366-7680 (electronic) SpringerTexts inEducation ISBN978-981-13-7495-1 ISBN978-981-13-7496-8 (eBook) https://doi.org/10.1007/978-981-13-7496-8 LibraryofCongressControlNumber:2019935842 ©SpringerNatureSingaporePteLtd.2019 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar methodologynowknownorhereafterdeveloped. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained hereinorforanyerrorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregard tojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations. ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSingaporePteLtd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore Preface This book has arisen from two postgraduate level courses in Rasch measurement theory that have been taught both online and in intensive mode for over two decades at Murdoch University and The University of Western Australia. The theory is generally applied in the fields of education, psychology, sociology, marketing and health outcomes to create measures of social constructs. Social measurement often begins with assessments in ordered categories, with two cate- goriesbeingaspecialcase.Toincreasetheirreliabilityandvalidity,instrumentsare composed of multiple, distinct items which assess the same variable. Rasch mea- surementtheoryisusedtoassessthedegreetowhichthedesignandadministration of the instrument are successful and to diagnose problems which need correcting. Following confirmation that an instrument is working as required, persons may be measured on a linear scale with an arbitrary unit and arbitrary origin. Themainaudiencesforthebookaregraduatestudentsandprofessionalswhoare engaged in social measurement. Therefore, the emphasis of course is on first principles of both the theory and its applications. Because software is available to carry out analyses of real data, small hand-worked examples are presented in the book. The software used in the analysed examples, which is helpful in working throughthetext,isRUMM2030(Rasch unidimensional models for measurement). Although the first principles are emphasized, much of the course is based on research by the two authors and their colleagues. ThedistinctivefeatureofRaschmeasurementtheoryisthatthemodelstudiedin this book arises independently of any data—it is based on the requirement of invariant comparisons of objects with respect to instruments within a specified frame ofreferenceandvice versa. Thisisa feature ofallmeasurement.Deviations of the data from the model are taken as anomalies to be explained and the instrument improved. The approach taken is to provide the researcher with confi- dence to be in control of the analysis and interpretation of data, and to make professionalratherthanprimarilystatisticaldecisions.Becausestatisticalprinciples are necessarily involved, reviews of the necessary statistics are provided in Appendix D. v vi Preface Graduates and professionals are likely to encounter classical test theory. Therefore,introductorychaptersreviewtheelementsofthistheory.Theperspective on the relationship between Rasch measurement theory and classical test theory is thattheformerisanelaborationoftheidealsofthelatter,notthattheyareentirely in conflict. However, because the centrality of invariance as a requirement for measurement had been articulated by two giants of social measurement, L. L. Thurstone and L. Guttman, reference is made to their work. In particular, Thurstone had articulated the requirements of invariance in almost identical terms as G. Rasch, but did not express it in terms of a mathematical equation, and the elementary Guttman design which is introduced in the early chapters, is shown to be a deterministic form of the Rasch model. The distinctive contribution of Rasch compared to that of Thurstone and Guttman is that the model studied in this book has built into it the principle of invariance and is immediately probabilistic. Therefore, the deviation of data from the model implies some kind of deviation from invariance and measurement. Together with the relationships shown with classical test theory, the book provides a unified theme for approaches to social measurement, rather than as a compendium of techniques. Finally, the book stresses that the requirement of invariance, and its expression in the Rasch model, is necessary, but not sufficient to ensure sound measurement. All the principles of measurement, of experimental design and of statistical infer- ence must be applied in the process of constructing instruments that provide invarianceofcomparisonsandreliableandvalidmeasurement.Indeed,theexplicit requirementsofinvarianceintheRaschmodelcanattimesappearmoredemanding of the data than do other theories and approaches. Crawley, Australia David Andrich Ida Marais Acknowledgements RUMM2030, which is a Windows, menu-driven program, has been written pri- marilybyBarrySheridan.Hehaswrittentheprogramsothatitpermitsanefficient expositionofthetheoryandtheapproachemphasizedinthebookfordataanalyses. Alan Lyne contributed to the original programming and further contributions were madebyGuanzhongLuo.IreneStyleshasbeenacolleaguebothinresearchandin improving the courses on which this book is based. Many students have also providedfeedback,includingSoniaSapplwhohascontributedtotheeditingofthe book. Natalie Carmody has administered the courses for more than a decade and helpedpreparethebook.Thefirstauthoralsoacknowledgesthedeepinfluenceofa year of study with the Danish mathematician and statistician Georg Rasch in the 1970s when Rasch had turned to the philosophy of measurement. The first author also acknowledges the support of the Australian Research Council for a range of grants over more than 30 years that have helped him conduct research into Rasch measurement theory. vii Contents Part I General Principles and the Dichotomous Rasch Model 1 The Idea of Measurement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Latent Traits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Assessment: A Distinction Between Latent and Manifest. . . . . . . . . 4 Scoring Assessments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Dichotomous Items and Their Scoring . . . . . . . . . . . . . . . . . . . . . . 5 Polytomous Items and Their Scoring . . . . . . . . . . . . . . . . . . . . . . . 5 Key Features of Measurement in the Natural Sciences . . . . . . . . . . . . 6 Stevens’ Levels of Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Nominal Use of Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Ordinal Use of Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Interval Use of Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Ratio Use of Numbers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Reliability and Validity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Some Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 A Model of Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2 Constructing Instruments to Achieve Measurement . . . . . . . . . . . . 13 Constructing Tests of Proficiency to Achieve Measurements . . . . . . . . 15 Constructing Rating Scales to Achieve Measurements. . . . . . . . . . . . . 18 Number, Order and Wording of Response Categories . . . . . . . . . . . 19 An Example of the Assessment of Writing by Raters . . . . . . . . . . . 20 An Example of the Assessment of the Early Development Indicator Instrument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 The Measurement of Attitudes: Two Response Mechanisms . . . . . . . . 23 An Example: The Cumulative Mechanism . . . . . . . . . . . . . . . . . . . 23 ix x Contents An Example: The Unfolding Mechanism . . . . . . . . . . . . . . . . . . . . 24 A Practical Approach: Likert Scales. . . . . . . . . . . . . . . . . . . . . . . . 25 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3 Classical Test Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Elements of CTT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 The Total Score on an Instrument . . . . . . . . . . . . . . . . . . . . . . . . . 30 Reliability, True and Error Scores . . . . . . . . . . . . . . . . . . . . . . . . . 31 Statistics Reviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Item Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Facility of an Item. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Discrimination of an Item . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Person Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Notation and Assumptions of CTT. . . . . . . . . . . . . . . . . . . . . . . . . 35 Basic Equations of CTT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Reliability of a Test in CTT ryy . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 The Standard Error of Measurement se. . . . . . . . . . . . . . . . . . . . . . 37 Statistics Reviews. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Reference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4 Reliability and Validity in Classical Test Theory . . . . . . . . . . . . . . 41 Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Reliability in Terms of Items. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Coefficient Alpha ðaÞ: Estimating Reliability in CTT . . . . . . . . . . . . . 47 Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Factors Affecting the Reliability Index. . . . . . . . . . . . . . . . . . . . . . . . 48 Internal Factors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 External Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Common Factors Affecting Reliability and Validity . . . . . . . . . . . . . . 51 Causal and Index Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 5 The Guttman Structure and Analysis of Responses . . . . . . . . . . . . 55 The Guttman Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 Interpretations of the Continuum in the Guttman Structure . . . . . . . 57 Elementary Analysis According to the Guttman Structure in the Case of a Proficiency Example. . . . . . . . . . . . . . . . . . . . . . . . . 59