DOCUMENT RESUME FL 800 551 ED 352 854 Bell, James H.; Johnson, Reta E. AUTHOR Effect of Lowering the Reading Level of a Health TITLE Education Pamphlet on Increasing Comprehension by ESL Adults. PUB DATE [92] 32p.; This study was supported by a grant from the NOTE Research and Development Committee, Calgary General Hospital (Alberta, Canada). Research/Technical (143) PUB TYPE Reports HF01/PCO2 Plus Postage. EDRS PRICE *Adult Literacy; Comparative Analysis; *English DESCRIPTORS (Second Language); Foreign Countries; Health Education; Readability; Reading Achievement; *Reading Comprehension; *Reading Improvement; Reading Materials; Testing; Vocabulary ABSTRACT A study investigated whether lowering the reading level of a health education pamphlet would increase comprehension by adult English-as-a-Second-Language (ESL) readers. The reading level of a general pharmacy education handout was lowered by changing vocabulary, sentence structure, and organization, by highlighting the main idea of each point, and by writing an introduction designed to catch attention and focus reading. Low-intermediate and advanced ESL college students read either the unaltered (n=25) or altered (n=24) versions of the pamphlet, which were estimated to be at the grade 12 and grade 7 level, respectively. Subjects then answered a free written test and a short-answer test. There was no statistically significant difference in the reading comprehension scores of the two groups on either test. The results suggest that health educators and ESL professionals should be wary of using readability formulas to estimate the suitability of materials for ESL readers, and of assuming that lowering the reading level of materials means :.ncreasing comprehension even when some changes beyond the lexical and syntactical are made. Excerpts from the tests are appended. (Contains 12 references.) (Author/LB) (Adjunct ERIC Clearinghouse on Literacy Education) *********************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. *********************************************************************** krZ Effect of Lowering the Reading Level of a Health Education Pamphlet on Increasing Comprehension by ESL Adults C.T4 James H. Bell Calgary Adult Literacy Awareness Project U.S. DEPARTMENT OF EDUCATION Office oi Educational Ritsaich "PERMISSION TO REPRODUCE THIS am Improvement EDUCATIONAL RESOURCES INFORMATION Reta E. Johnson MATERIAL HAS BEEN RANTED BY CENTER (ERIC) document has been reproduced as 0112.X owed from the Gerson or organization Calgary General Hospital originating it Minor changes have been made to improve reproduction quality Points of vie* or opinions stated In this dot. merit do not necessarily represent official TO THE EDUCATIONAL RESOURCES OERI position or policy INFORMATION CENTER (ERIC)." This study was supported by a grant from the Research and Development Committee. Calgary General Hospital. We gratefully acknowledge the help of D. George. D. Goring. J. Goring, P. Pankratz, R. Raughton. A. Trussler. D. Van Staalduine, D. Warkentin. D. Wood. H. Holley for her assistance with the statistical analysis. and the Department of Psychiatry. Calgary General Hospital, for access to computers and statistical software. Running head: Reading Levels 2 BEST COPY AMBLE Reading Levels 2 Abstract Does lowering the reading level of a health education pamphlet actually increase the comprehension by adult readers for whom English is a second language? We lowered the reading level of a general pharmacy education handout by changing vocabulary, sentence structure, and organization, by highlighting the main idea of each point, and by writing an introduction designed to catch attention and focus reading. Low-intermediate and advanced English as a Second Language (ESL) college students read either the unaltered (25 students) or altered (24 students) version of the pamphlet, which were estimated to be at the grade 12 and grade 7 level respectively according to the Flesch-Kincaid readability formula. Subjects then answered a free written test and a short-answer test. There was no statistically'significant difference in the reading comprehension scores of the two groups on the free written test p-0.141 or on the short- answer test (p-0.59). Health educators and ESL professionals should be wary of using readability formulas to estimate the suitability of materials for ESL readers, and of assuming that lowering the reading level of materials means increasing comprehension even when some changes beyond the lexical and syntactical are made. Reading Levels 3 Effect of Lowering the Reading Level of a Health Education Pamphlet on Increasing Comprehension by ESL Adults One of the most important kinds of materials that English as a Second Language adults encounter is general health education material. Not understanding can be dangerous; misunderstanding can be just as bad. Health educators are becoming aware that much material is too difficult for many L2 readers. Often health professionals turn to ESL professionals for help with"lowering the readability level" of materials. The level is typically calculated with readability formulas. The writing or revision of materials is based on the assumption that lowering the readability level will increase comprehension. This assumption is not as common-sensical as it might first appear. We tested it in an experiment involving adult ESL students at an urban. Canadian college. Review of the Literature This study stems from concerns about, and research on. readability in health education and in English as a Second Language. Readability and Health Education Health education materials are often too difficult for many readers to comprehend. By comparing the readability levels of materials analyzed by readability formulas with the estimated average reading level of adults (grade 8-9), it becomes clear that 'much patient education material is potentially incomprehensible to a large portion of the adult population" (Vivian & Robertson. 19801. 4 Reading Levels 4 Spadero (1983) found that of the 55 health brochures reviewed, only 14 (25.510 were written at or below the grade nine level. La Pierre and Mallet (1987) found the average reading level of 190 Canadian and American health education pamphlets to be grade 11.8 + 1.7. Rather than estimating the average adult reading level to be grade 8-9, some studies actually measure the reading level of the target audiences for their health education materials. Doak and Doak found that the mean reading level of their subjects was approximately grade 7 as estimated by the Wide Range Achievement Test, but that the mean reading level of 100 samples of patient education material was approximately grade 10 (Doak, Doak, & Root, 1985). Davis, Crouch, Wills, Miller, and Abdehou (1990) found that most health education materials they examined "required average reading comprehension grade levels of 11th to 14th" grade (p. 535), while the 151 patients averaged 6th grade 5th month on the reading sections of the Peabody Individual Achievement Test. Because of the wide discrepancy between the reading ability of the citizenship and the readability levels of the health education materials, writers recommend lowering the readability grade level of materials to less than grade 9 (Spadero, 1983), 8-9 (Lange, 1988), 8 or less (La Pierre & Mallet, 1987), less than 8 (Farrell-Miller & Gentry,1989), 7 (Davis et al.,1990), or 6.5-8.5 (Matthews, Thornton & McLean, 1985). It is assumed that lowering the readability level of the text will increase comprehension by the readers. 5 Reading Levels 5 A decade ago, advice emphasized relying heavily on readability formulas. For example, Vivian and Robertson (1980) wrote: "Producers of patient education materials should apply readability formulas to ascertain the comprehensibility of the offered material" (p. 135). Vivian and Robertson advise shorter words and sentences for increased comprehension. But because readability formulas rely on estimates of only two variables--word difficulty and sentence difficulty--patient education experts now advise considering other variables affecting comprehension: the reader, content, organization, and format. They advise, for example, stimulating the reader's interest (Lange,1988), defining semi-technical terms (Gibbs, Gibbs, & Henrich, 1987), ensuring an introduction, body, and summary (Lohr, Ventura, Crosby, Burch, & Todd. 1987), as well as making the changes necessary to lower the readability level as measured by the formulas. However. the literature does not address a key question: Is the relationship between simplification and comprehension incremental or all-or- , nothing? Is it true that the more readability tips implemented in writing a text the more comprehensible the text becomes? Or must nearly all readability advice be followed to engender significantly improved comprehension? Nor does the literature seriously question the assumption that lowering the readability level of health education materials to about the grade 7 level as measured by readability formulas will increase comprehension significantly. Reading Levels 6 Readability and Formulas While the literature on health education now recommends more than simplifying vocabulary and shortening sentences, the readability formula score is still the imprimatur. This may place unwarranted confidence in readability formulas. A readability formula is "a predictive device intended to provide quantitative, objective estimates of reading difficulty" (Klare, 1984, p. 684). All rely on estimates of word and sentence difficulty. The Flesch-Kincaid, for example. uses average sentence length and word length, as the following simplified version of the formula shows: Grade Level - .4(words/sentence) + 12(syllables/word) 16 (Klare, 1984, p. 693). Klare reminds us that such formulas are designed to predict readability, not to produce it. In developing readability formulas, researchers establish a correlation between reader comprehension (usually measured by doze tests) of different passages and some elements of the passages. Estimates of word and sentence difficulty in passages give the highest convenient and reliable correlation with comprehension. This research does not claim that the relationship is causal. It is worth noting that while such formulas predict readability, they do not measure readability. While formulas or expertise applied to a text yield predictions of readability, tests of reader comprehension measure readability. 7 Reading Levels 7 Of the most common readability formulas, the Flesch seems well suited to our research. The Spache formula is designed for very elementary materials (grades 1-3). The Da le-Chall is based on a dated list of common words (Dale & Chall, 1948). The Raygor yields essentially the same results as the Fry, counting letters in words rather than syllables (Baldwin & Kaufman, in Klare, 1984). Although the raw scores from the Fry, Flesch, and SMOG formulas correlate highly, when these scores are converted to grade equivalents, SMOG yields grade levels higher than other formulas (Spadero, 1983) and Fry yields scores lower than average (Guidry & Knight, 1976). According to Klare, author of the readability chapter in Handbook of Reading Research(1984), the Flesch is the most widely used, and more computer programs have been developed to apply [it] than any other formula" (p. 690). Even used properly, readability formulas have many weaknesses. Formulas have high correlations with passage difficulty when they are tested with a wide range of readers on a large number of texts of varying content and difficulty. But in normal application, formulas are often used for a narrow range of reading ability and a small number of passages on a particular subject and with a restricted range of difficulty. This use is legitimate, but the predictive power of the formulas decreases. It is also well to remember that the predictive power of formulas was generally developed with school 8 Reading Levels 8 pupils and the extrapolation to adults, speakers of English as a second language, and especially ESL adults must be viewed critically. Yet readability formulas will continue to be used widely because alternatives are not promising. Testing--that is, actually measuring readability--is time consuming, requiring creating valid and reliable tests, administering them to appropriate subjects under suitable conditions, and calculating and interpreting results. Judging, a common method, is risky for non-experts. Even people who are usually considered to be good judges of readability may not be. Klare (1984) cites research showing school and public librarians rating one book from grade 3 to grade 12 in difficulty, professional writers rating five passages at all but one of the five possible levels of difficulty, and teachers' judgements varying by six to nine grade levels. Individuals who are not specially trained are not reliable judges of readability. Readability and ESL Although readability formulas are less commonly used in the ESL field than the health education field, readability in ESL often has a similar focus: vocabulary size and the difficulty of grammatical structures (Hetherington,1985). Numerous authorities argue eloquently for including consideration of content, discourse features, format, and, most importantly, the reader, as well as linguistic variables (Carrell, 1987; Hetherington, 1985; Lotherington- Woloszyn, 1988). The research is sparse and inconclusive, but it calls into question focusing heavily on the lexical and syntactical. 9 Reading Levels 9 Johnson's (1981) study indicated by multivariate analysis of variance that "the level of syntactic and semantic complexity [of an English language text] had a lesser effect than did the cultural origin of the text on the reading comprehension" (p. 171) of 46 Iranian intermediate/advanced ESL students at ar American university. Nonetheless, the Iranian students understood the simplified version of an American folk story better than the unaltered original. Blau (1982) developed three versions of 18 paragraphs, each version differing in the degree of sentence combining. Analysis of covariance indicated no significant difference in comprehension for 85 Puerto Rican ESL college students among version 1 (primarily simple sentences), version 2 (complex sentences with surface clues to relationships), and version 3 (complex sentences with sophisticated, subtle clues to relationships). Post hoc, says Blau, one could reason as follows. If one were to use readability levels as a guide in formulating a directional research hypothesis, a logical statement would be that comprehension of version 1 will be superior to comprehension of version 2, a one-tailed t-test. This hypothesis would not be supported (p <0.45). Floyd and Carrell (1985) were able to achieve more definite results by making the variables extreme. Twenty intermediate-level ESL students were divided into an experimental group which experienced two carefully planned, dynamic lessons on Independence Day in Boston, and a control group which did its normal school work.