Using and Developing Measurement Instruments in Science Education A volume in Science & Engineering Education Sources Calvin S. Kalman, Series Editor Using and Developing Measurement Instruments in Science Education A Rasch Modeling Approach (Second edition) Xiufeng Liu State University of New York at Buffalo INFORMATION AGE PUBLISHING, INC. Charlotte, NC • www.infoagepub.com Library of Congress Cataloging-in-Publication Data A CIP record for this book is available from the Library of Congress http://www.loc.gov ISBN: 978-1-64113-934-2 (Paperback) 978-1-64113-935-9 (Hardcover) 978-1-64113-936-6 (E-Book) Copyright © 2020 Information Age Publishing Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written permission from the publisher. Printed in the United States of America Contents List of Tables ..........................................................................................ix List of Figures ........................................................................................xi Preface ................................................................................................xvii Preface to the Second Edition ..............................................................xxi 1 Essential Concepts and Skills for Using and Developing Measurement Instruments .............................................................1 Evolution of Measurement Instruments in Science Education .....2 Fundamental Theories for Using and Developing Measurement Instruments .......................................................................................5 Measurement Standards .................................................................11 Essential Skills for Developing Measurement Instruments .........12 Locating Measurement Instruments .............................................22 Chapter Summary ...........................................................................23 Exercises ..........................................................................................25 References .......................................................................................26 2 Approaches to Developing Measurement Instruments ................29 Classical Test Theory and Generalizability Theory .....................30 Rasch Models and Item Response Theory ....................................34 Using Rasch Modeling to Develop Measurement Instruments ...41 Chapter Summary ...........................................................................58 v vi  Contents Exercises ..........................................................................................59 References .......................................................................................60 3 Using and Developing Instruments for Measuring Conceptual Understanding ..........................................................63 What Is Conceptual Understanding? ............................................63 Instruments for Measuring Conceptual Understanding .............67 Developing Instruments for Measuring Conceptual Understanding ................................................................................89 Chapter Summary ..........................................................................114 Exercises ........................................................................................115 References ......................................................................................116 4 Using and Developing Instruments for Measuring Affective Variables .......................................................................121 What Are Affective Variables? ......................................................121 Instruments for Measuring Student Affective Variables ............125 Developing Instruments for Measuring Affective Variables ......167 Chapter Summary .........................................................................182 Exercises ........................................................................................184 References .....................................................................................184 5 Using and Developing Instruments for Measuring Science Inquiry and Practices ..................................................................193 What Is Science Inquiry? ..............................................................193 Instruments for Measuring Science Inquiry ...............................199 Developing Instruments for Measuring Science Inquiry and Practices .................................................................................221 Chapter Summary .........................................................................236 Exercises ........................................................................................237 References .....................................................................................238 6 Using and Developing Instruments for Measuring Learning Progression ..................................................................................243 What Is Learning Progression in Science? .................................244 Instruments for Measuring Student Learning Progression ......249 Developing Instruments for Measuring Learning Progression ...258 Chapter Summary .........................................................................273 Exercises ........................................................................................275 References .....................................................................................276 Contents  vii 7 Using and Developing Instruments for Measuring Science Learning Environments ...............................................................279 What Are Science Learning Environments? ...............................279 Instruments for Measuring Science Learning Environments .....285 Developing Instruments for Measuring Science Learning Environments ...............................................................300 Chapter Summary ..........................................................................314 Exercises ........................................................................................315 References .....................................................................................316 Author Index ........................................................................................321 Subject Index .......................................................................................331 List of Tables Table 2.1 Fit Statistics for an Elementary Science Concept Test ........48 Table 2.2 Item Choice Statistics ............................................................50 Table 2.3 Raw Score and Rasch Scale Score Conversion Table ..........58 Table 3.1 Revised Bloom’s Taxonomy: Cognitive Dimension .............66 Table 3.2 A Sample Table of Test Specification With Type of Items and Total Points ...........................................................94 Table 3.3 A Hypothetical MTMM Validation Design .........................110 Table 3.4 Raw Total Score to Rasch Scale Score Conversion Table .113 Table 4.1 A Test Specification for Measurement of Interest in School Science .................................................................169 Table 4.2 Raw Score to Rasch Scale Score Conversion Table for the Nanotechnology Attitude Scale .............................182 Table 5.1 Sample TIPS Questions .......................................................225 Table 5.2 Fit Statistics for Items on a Middle School Performance Assessment on Energy Transfer ...................232 Table 6.1 Student Behaviors of Diversity of Living Things ...............260 Table 6.2 Test of Specification With Values for the Three Forms of Assessment Instrument for Learning Progression of Diversity in Living Things ..............................................261 Table 6.3 Test Specification With Values for Assessment of Learning Progression of Diversity in Living Things ....262 Using and Developing Measurement Instruments in Science Education, pages ix–x Copyright © 2020 by Information Age Publishing All rights of reproduction in any form reserved. ix