DOCUMENT RESUME CE 079 977 ED 440 230 Nagel, Elaine AUTHOR Making a Choice between the TABE and the CASAS. Action TITLE Research Monograph. PUB DATE 1999-00-00 10p.; In: Action Research Monographs. Complete Set. NOTE Pennsylvania Action Research Network, 1998-99. A Section 353 Project of the Pennsylvania Department of Education, Bureau of Adult Basic and Literacy Education. A Learning from Practice Project; see CE 079 962. Evaluative (142) PUB TYPE Reports MF01/PC01 Plus Postage. EDRS PRICE Action Research; Administrator Attitudes; Adult Basic DESCRIPTORS Education; Adult Literacy; Adult Students; *High School Equivalency Programs; *Literacy Education; Pilot Projects; Predictor Variables; Pretests Posttests; Program Development; Program Effectiveness; *Standardized Tests; Student Attitudes; *Student Evaluation; Teacher Attitudes; Teacher Researchers; Test Reliability; ;,Test Selection; Test Validity *Comprehensive Adult Student Assessment System; General IDENTIFIERS Educational Development Tests; Pennsylvania; *Test of Adult Basic Education; 353 Project ABSTRACT For seven years, the Test of Adult Basic Education (TABE) was the primary standardized assessment instrument used by the adult education program of the Midwestern Intermediate Unit IV (MIUIV) of the Pennsylvania Department of Education Bureau of Adult Basic and Literacy Education. While reviewing its current assessment procedures, the MIUIV's quality improvement team determined that the Comprehensive Adult Student Assessment System (CASAS) was identified as an "up and coming" adult assessment that warranted further review. The MIUIV's program improvement team therefore decided to conduct an action research project to choose between the TABE and CASAS as its primary standardized assessment instrument. During the project, two pilot sites in the MIUIV's adult education program used the TABE for four months and then switched to the CASAS for all pre- and posttesting for six months. Compared with the TABE, the CASAS took less time to administer and produced slightly less student test anxiety. Nevertheless, the TABE was deemed the assessment instrument of choice based on the following four factors: ability to predict performance on the General Educational Development test; usefulness in planning individual instruction; ease of scoring and communicating results to students; and feedback from students, staff, and the program administrator. (MN) Reproductions supplied by EDRS are the best that can be made from the original document. EURITAU OF AOULF BASIC A LITESACT EDUCATION able ENNSTIVANM DEPARTMENT Of EDUCATION %paid poject Action Research A Learning From Practice Project Monograph PENNSYLVANIA ACTION RESEARCH NETWORK 1998-99 Monograph Title: Making a Choice Between The TARE and The CASAS Action Researcher's Name: U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement Elaine Nagel EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) This document has been reproduced as received from the person or organization originating it. Minor changes have been made to improve reproduction quality. For. further project detail contact: Points of view or opinions stated in this document do not necessarily represent The Pennsylvania Action Research Network official OERI position or policy. do Adult Education Graduate Program Penn State University, McKeesport Campus University Drive McKeesport PA 15132 rc--- A Section 353 Project of the Pennsylvania Department of Education, Bureau of Adult Basic and Literacy Education Contact State Literacy Resource Center for Additional copies. &), by The Pennsylvania State University, under support from This monograph is a result of a Learning From Practice project developed Adult Basic and Litc:ney Education: the U.S. Dcpartmcnt of Education, through the Pennsylvania Department of Education. Bureau of U.S. Department of Education or however, the opinions expressed herein do not necessarily reflect the positions or policies of the inferred. the Pennsylvania Department of Education, and no official endorsement should be BEST COPY AVAILABLE P °DUCT "Pennsylvania Action Research Network: Staff Development Through Six Professional Development Centers" Project Number 099-99-9010 July 1998-June 1999 Project Director Dr. Gary Kuhne Assistant Professor and Regional Director of Adult Education The Pennsylvania State University Pennsylvania Action Research Monograph Note: Action Research is a process of systematic inquiry credited to Kurt Lewin who popularized it in the U.S. in the 1940"s. Today it is Considered a system of qualitative research. Typical of action research, none of the individual projects in this monograph series claims to have generalizable application beyond the specific project described. However, each monograph report can serve to be illustrative, instructive and provides the potential for replication in other locations. For a level of generalizability, it is recommended that the reader seek common patterns in the monograph reports in this series, and the wider literature, or contact the Action Research Network for assistance in this. I. ABSTRACT Midwestern Intermediate Unit IV's Adult Education Program used action research to make a decision between the Life Skill CASAS (Comprehensive Adult Student Assessment System) and the TABE.7/8 (Test of Adult Basic Education) for their primary standardized assessment instrument. As part of a continuous improvement effort, the intent was to select the assessment instrument which offered the highest level of satisfaction as rated by students, teachers, administration and the Pennsylvania Department of Education Bureau of Adult Basic and Literacy Education. MIU IV's Adult Education Program is primarily intermediate level ABE, upper level ABE and GED preparation. The assessment instrument has been the TABE for the last seven years. Two pilot sites used the TABE for four months, then switched to the CASAS for all pre and post testing for six months. The TABE was the assessment instrument of choice based on predictability to the GED test, usefulness in planning individual instruction, ease of scoring and communicating results to students. The CASAS took significantly less time to administer and produced slightly less student test anxiety. There was no measurable impact on student retention with one assessment over the other. Feedback from students, staff and program administrator favored the TABE. MIU IV will continue to use the TABE 7/8 for standardized assessment. II. PROBLEM With local program self-assessment under the EQUAL project, the Midwestern Intermediate Unit IV's program improvement team scrutinized many current procedures and explored other current best practice options. The CASAS had been deemed an "up and coming" adult assessment by many practitioners in the field and seemed to warrant further review. The purpose of this action research effort was to determine which was better suited, the Test of Adult Basic Education (TABE) or the Comprehensive Adult Student Assessment System (CASAS) for standardized assessment in MIU IV's Adult Education Program. MIU IV provides Adult Basic Education for intermediate and upper level students, as well as GED preparation classes. Lower level ABE (0-3 G. Eq.) are referred to other programs in the area, usually a local literacy council. The TABE has been the MIU 1V's assessment instrument for seven years. It is standardized and norm-referenced. It measures academic skills in reading, 1 mathematics, language and spelling. Two formats are available; the Complete Battery TABE, which takes approximately 3 V2 hours to complete and a shorter, but less diagnostic, Survey TABE. Both formats have advancing difficulty levels and pre and post test forms. Scores are available as scaled scores, stanine, percentile rank and grade level equivalents. The CASAS is standardized and criterion referenced. It is organized around over 300 functional context competencies in areas of basic communication, consumer economics, community resources, health, employment, government and law, computation, learning to learn and independent living skills. Results are reported as scaled scores. The assessment allows for advancing difficulty levels and has pre and post test forms. It offers a CASAS Curriculum Materials Guide for instructional use in accessing current trade workbooks and materials around the student's identified need. As part of a continuous improvement effort, MIU IV was seeking the standardized assessment instrument that offered the highest satisfaction level as rated by the users. The users included the students, the instructors, the program administrator and the Department of Education's Bureau of Adult Basic and Literacy Education. The program had concerns with satisfaction around issues such as student placement, instructional diagnosis, individual instructional planning, student test anxiety, student retention and monitoring progress. III. PLANNING The plan was to use the TABE for establishing the baseline data for comparing and contrasting with the CASAS. Two open-entry sties were to use the TABE for all pre and post testing with students from September through December. Starting in January and ending in June, the two pilot sites were to use the CASAS for all pre and post testing. Feedback was to be collected from four different stakeholder groups namely, the students, the instructional staff, the program administrator and the Bureau of Adult Basic and Literacy Education at the Pennsylvania Department of Education. The feedback and results were to be compared and contrasted using the following criteria: usefulness in planning individual instruction impact on student test anxiety impact on student retention communicating results to students 2 6 length of time to administer ease of scoring acceptability of use for required PDE information predictability to the GED test CASAS testing materials were to be purchased, approvals obtained, CASAS training scheduled, and the pilot sites inserviced. The plan was to determine if the CASAS or the TABE was better suited for use in the MIU IV Adult Education Program. IV. ACTION The MIU IV program administrator participated in a one day CASAS training. Materials were purchased, teachers and counselors inserviced and two pilot sites selected. Four teachers and two counselors worked with the TABE for the first four months of the program year, then switched and used the CASAS for the last six months. Both pilot sites were open-entry, open-exit with experienced teachers and counselors. 28 CASAS assessments were given during this time. The two assessment instruments were then compared and evaluated in terms of the planned criteria from the perspective of the four primary stakeholders. V. RESULTS Preferences of all four stakeholders were taken into consideration. The students admitted slightly more test anxiety with the TABE but also more satisfaction. The TABE appeared to be in line with their test expectations and for measuring the specific skills required for a GED diploma. Several students expected grade equivalents or percentile ranks as required for job training programs. There seemed to be an established paradigm that the TABE was a "real test" and provided helpful information on what was needed before stepping up to the GED test. The teaching staff all expressed prior interest and support for 1) criteria referenced assessment with emphasis on the level of mastery rather than the relationship with a norming group, 2) the CASAS claim of reduced test anxiety and better student retention, and 3) accurate but shorter testing times. However, after using the CASAS for six months, all six staff involved preferred to test their students with the TABE. They found the diagnostic and prescriptive information from the TABE profile more helpful for interpreting and planning meaningful and 3 specific skill instruction with students. They were as frustrated as their students with the seemingly poor predictability of the CASAS to the GED test. They suggested that the CASAS may be a very beneficial assessment for lower level ABE participants which make up less than 5% of current enrollments. Staff found the CASAS Instructional Material Guide, which provided teaching resources from trade workbooks to be professionally questionable in a few areas. They preferred to self- design instruction and self-select materials on an individual student need basis. They also found maintaining an adequate supply of the numerous instructional materials very cumbersome with their sites limited space and shelving. The program administrator found the CASAS technical assistance cumbersome to access. While the technical assistance staff were congenial and enthusiastic in support of the CASAS, it was difficult to call in CASAS scores for a GED predictability on an individual student by student basis as they suggested. A conversion scale was not available. The program administrator preferred the TABE as the better assessment for aligning and measuring progress with the programs taught curriculum. The Bureau of Adult Basic Education and Literacy Education has listed both the TABE and the CASAS as acceptable standardized testing instruments for their progress measuring and federal reporting requirements. In summary, the feedback from all four stakeholder groups strongly supported keeping the TABE as the MIU IV assessment instrument. However, there was agreement on CASAS strength in the areas of shorter testing time requirement, slightly less student test anxiety, and favorable support for use with lower level ABE participants. 4 RESULTS SUMMARY Criteria Assessment CASAS TABE Preferred TABE Usefulness in Aligned with life skills Significantly more planning individual competencies but not with comprehensive and instruction objectives voiced by aligned with program . participants or with program curriculum and the curriculum. Instructional actual GED test guide is cumbersome and somewhat unclear. CASAS Slightly less anxiety Impact on student Slightly more anxiety test anxiety producing producing Neutral Impact on student No differences No differences retention TABE Not aligned to GED skills Communicating Aligned with specific results to students that most students are objectives as voiced by students. Good seeking diagnostic profile for presenting to student. Lengthy 3+ hours Length of time to Very acceptable length CASAS administer Ease of scoring Pretest answer sheet was TABE Familiar and easy to difficult to score, too many score once trained answer booklets Neutral Acceptability to Acceptable Acceptable PDE/ABLE TABE Low predictability that is Predictability to the Solid, useful GED test difficult to obtain predictability VI. REFLECTION Participation in action research to determine which standardized assessment to use made several observations noteworthy. First, an assessment needs to align with the curriculum and the instruction. Clearly, the TABE does that for MIU IV better than the CASAS. Second, change happens slowly. If MIU IV decides to transition into a life skill type curriculum as our student population becomes more difficult to serve and requires lower level ABE instruction, then a considerable amount of staff inservice and pre-change activity would be needed. We are very comfortable with the familiar and time must be scheduled to shift paradigms and build a comfort level with a new framework. Third, some students perceived a CASAS assessment as a "lowering 5 of the bar" when compared to the TABE. They obviously need communication and instruction to help bridge and see as valuable their life experiences and skills. Fourth, while "best practices" discourage grade equivalent scores for adult education programs, we need to acknowledge that our students may need scores of this nature for employment attainment opportunities. Fifth and last, this was a very limited and subjective evaluation based on MIU IV teacher, counselor, student and administrator feedback. This action research is program specific and should not be generalized. 6 I0