DOCUMENT RESUME ED 354 228 SP U34 313 Lockledge, Ann; Hayn, Judith A. AUTHOR Preparing Teachers for Open Response Testing. TITLE PUB DATE 92 NOTE 9p. PUB TYPE Descriptive (141) Reports EDRS PRICE MFO1 /PCO1 Plus Postage. DESCRIPTORS Academic Achievement; *Constructed Response; *Content Validity; Elementary Secondary Education; Essay Tests; Inservice Teacher Education; Multiple Choice Tests; *Questioning Techniques; Standardized Tests; *Student Evaluation; *Teacher Workshops; Test Format *Open Ended Questions; *Question and Answer IDENTIFIERS Exercises; Question Answering; Question Types ABSTRACT This paper describes a workshop designed to lead teachers toward the use of open ended response questions rather than multiple choice items. The rationale was based on the assumption that assessment will become more authentic because it will measure depth of understanding and deal with both content and process. Authentic assessment is defined as an alternative to traditional multiple-choice standardized achievement tests involving direct examination of student performance on significant tasks that would be relevant to life outside of school. The focus of the workshop described was on participants constructing questions based on a group of generic questions or formulas into which the teacher could insert his or her own content. Workshop participants were then asked to write answers to the questions they had prepared and to construct an appropriate rubric. Three handouts guiding the workshop (Using Generic Open-Ended Response Questions, Let's Practice Writing and Scoring Open Response Questions, and Samples of Generic Questions for Adapting to Your Own Content) are included. (Author/LL) *********************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. *********************************************************************** Preparing Teachers for Open Response Testing Ann Lockledge School of Education University of North Carolina at Wilmington Wilmington, NC 28403-3297 919-395-3366 Judith A. Hayn School of Education University of North Carolina at Wilmington Wilmington, NC 28403-3297 919-350-4075 U S DEPARTMENT OF EDUCATION 'PERMISSION TO REPRODUCE THIS 0.1ce 01 Edu:at.nna. Reseych and .,hryo. eh-eh. MATERIAL HAS BEEN GRANTED BY EDUCATIONAL RESOURCES INFORMAT:ON CENTER /ERIC! Th.5 document has Deer 'WOO, 1...1 as .ece,ed f,on- The ve.son orgah ia, 0..y.nahng e :"' Mal°, changes ha...e beer made 'ed,oducl.on quauty Pcunts of ..uev. 0.* ocedens slated .n ihs dor u TO THE EDUCATIONAL RESOURCES 'rent do ...CI neCeSSaroy eorese,t n.lic °Ertl DOS.lit, INFORMATION CENTER (ERIC)" rt BEST itOPY AVAILABLE Preparing Teachers for Open Response Testing Ann Lockiedge and Judith A. Hayn Abstract Describes a workshop designed to lead teachers toward the use of open ended response questions rather than multiple choice items. Rationale was based authentic because it will on the assumption that the assessment will become more both content and process. Authentic measure depth of understanding and deal with alternative to traditional multiple- or alternative assessment was defined as an choice standardized achievement tests involving direct examination of student performance on significant tasks which would be relevant to life outside of school. The focus of the workshop described was on participants constructing questions based on a group of "generic questions" or formulas into which the teacher could Workshop participants were then asked to write insert his or her own content. appropriate rubric. answers to the questions they had prepared and to construct an The handouts and directions for the workshop are included. Preparing Teachers for Open Response Testing Ann Lock ledge and Judith A. Hayn Recently we have seen a move toward more authentic evaluation -- testing in the that involves students in answering the sorts of questions they might confront Educators want to know if the student can move beyond the school real world. teachers at all levels setting and consider problems in a larger context. Classrooms materials, manipulatives, projects, and are using response journals. hands-on national level there is a move toward the use of open ended At the portfolios. In some multiple choice items. response questions rather than relying solely on have been developed states such as North Carolina, standardized open-ended tests Accountability Services, at each grade level for the major subject areas (Division of 1992). Authentic or alternative assessment (Worthen, 1993) is viewed as an and alternative to traditional multiple-choice standardized achievement tests that are involves direct examination of student performance on significant tasks records relevant to life outside of school. Teachers have long kept anecdotal running legitimate and folders of student work before such records were recognized as an But when the demand for accountability became means of direct assessment. of coupled with increasing criticism of standardized multiple-choice tests, this way looking at what a stude.1 had accomplished was seen as holding considerable It is possible that the development of standards or rubrics for judging this promise. That is one of the type of performance has the potential. for becoming divisive. prepared to develop rubrics of reasons that the authors felt that teachers should be 1 their own and discus them with their students so that the children would understand the process. In assessing depth of understanding (Newman, 1992) we must be sure that students are asked to consider alternatives and to make choices based on critical ideas and use knowledge. They need to be working with substance and persuasion or take a theoretical stance. The problem they are asked to solve should be one that is likely to be addressed in the world beyond the school The use of open-ended questions asks students to use higher order thinking to organize or apply the knowledge they are acquiring. Open-ended responses differ from the traditional essay in that they are more pointed and require a concise answer that shows analysis of information presented, situations that develop, or problems to be confronted. This type of evaluation also requires the use of a rubric for grading. The rubric for a question sets out the standards a teacher could reasonably expect in It is usually based on a scale that goes from an answer from a prepared student. minimally acceptable to clearly understood and analyzed. The focus of this workshop was on participants constructing open-ended questions based on that which they in fact teach and then preparing rubrics to accompany each one. Only by the consistent use of such questions can students become comfortable with the techniques for answering and teachers with the techniques for grading. Even better is when students use prepared rubrics for scoring each others writing. The assessment will become authentic because it will measure depth of understanding and deal with both content and process. The training was based on a group of "generic questions" or formulas into which the teacher could insert his or her own content. The workshop participants were then asked to write answers to the questions they had prepared and to construct an appropriate rubric. Participants scored each other's answers in the same manner they might ask their students to try. It was suggested that another helpful technique might 2 c) be to use one generic question, teach the students the rubric and then use the same formula with different content. The first and second set of answers could be examined for progress in understanding quality answers. REFERENCES North Carolina End-of-Grade Testing. Division of Accountability Services. (1992). " .11 ^1 "IS'S II S Raleigh, North Carolina: -u Department of Public Instruction. Newman, Fred. (1992). Authentic Assessment. Address at the National Social Studies supervisors Association, Detroit. Worthen, B.R. Critical issues that will determine the future of alternative assessment. Phi Delta Kappan. Z4. E. 444-457. See the April 1989 and May 1992 issues of Educational Leadership and the May 1989, May 1991, and February 1993 issues of the Phi Delta Kappc.a. USING GENERIC OPEN-ENDED RESPONSE QUESTIONS Many open ended questions follow a pattern that may be used to fit the content of any Below you will find a generic question followed by possible ways to grade level. adapt it to each grade level. List three that 1. Suppose you were asked to interview . questions that you would ask him/her. has just closed a playground Mayor of your city Grade 3 who is running for reelection a State Senator Grade 4 saved the redwoods John Muir Grade 5 is British Prime Minister John Major Grade 6 spent 27 years in jail Nelson Mandela Grade 7 helped write the Constitution Thomas Jefferson Grade 8 For the grade which you teach write a response that you might hope to see. 2. Now score your response on a scale of 0-3 using the general Rubric at the bottom of the page. 3. Below is a another generic question. See how your group would go about adapting it to several grade levels. Tell why this (name holiday )_._ celebrate or observe (name people) . Tell how the people would holiday is important to the people that celebrate it. feel during this observance. Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 GENERAL RUBRIC FOR SCORING READING AND SOCIAL STUDIES RESPONSES Answer is unresponsive, unrelated or inappropriate. 0 Answer deals with material on a concrete, literal level that is accurate in most dimensions. 1. Answer deals with most aspects of the question and makes correct inferences, although 2. minor errors may exist. Comprehension is on an inferential level and the key skills are synthesis and analysis. Answer addresses most aspects of the question and uses sound reasons and cites and 3. explains appropriate examples. Uses skills of evaluation as well as analysis and synthesis. 4 fr- LET'S PRACTICE WRITING AND SCORING OPEN RESPONSE QUESTIONS Take the questions in your envelope and adapt one of them to the curriculum for each of the (For Social Studies: 3rd-Communities; 4th-Geography 5th-Regions of grade levels. U.S.; 6th Europe; 7th-Africa and Asia; 8th-United States History). Or take several questions and adapt them for the grade level you teach. After you have written the sample questions, choo:,, one and write an answer that you would consider acceptable from your students. Exchange answers and score using the rubric at the bottom of the page or on the first page: SAMPLE QUESTIONS: WRITE AN ANSWER TO ONE QUESTION: SCORE SOMEONE ELSE'S RESPONSE BY USING THIS RUBRIC: No response 0 Responded to question and attempted an explanation. Details are sparse. 1. Responded to question with a logical explanation. Includes specific details. 2. Responded to questions with extended explanation. Elaborates to back up logic. 3. WRITE A BEI 1.b,R RUBRIC FOR A SPECIFIC QUESTION YOU WROl E: 1- 2- 3- SAMPLES OF GENERIC QUESTIONS FOR ADAPTING TO YOUR OWN CONTENT SOCIAL STUDIES Pick one and explain its Recently there have been changes in _(name place or process) effect on the lives of people. changed the attitudes and viewpoints of the people in Make a list of ways that _(name event) _(name country or time period)_. Wars bring hardships. Tell why the __(name war)_was difficult for _(name people)_. Explain one way someone might have tried to cope with these hardships. Using the map(s) of _(tell type)__ resources in _(name area)__, circle an area where you would find a large city. Explain your answer. One problem tIrt must decide about is _(name problem)_. Tell what _(identify type of person) two different points of view might be. Name three ways You are assigned a report to compare the government of and . of doing things you would need to describe or compare. people do. Give two It has been said that religious belief affects almost everything the specific examples and explain. Place a check beside All of the following statements about _(name place, person, event) are true. the one you consider most important and explain your reasons for your selection. All places change over time. The following are examples of changes taking place in Choose one and explain how it will affect the lives of the people who live there. If you were going to _(name a change)__ in __(name place) what are some of the factors you , would have to consider? Tell how they have tried to Name two different ethnic groups who live in . maintain their traditional cultural identity. MATH The Using the map and the graph shown below, tell what were traveling by . a journey from would have been like. to SCIENCE In the picture of the items labeled. At least below, there are of them are necessary for it to operate properly. Explain why of the items are needed. Danny has samples of Tell two things he could do to find out which one . of them works best Explain why your test would give accurate information . . READING After reading a book about the life of tell why you think he chose to become , a(n) What things in his life made him become a great one. . 6