English Language Teaching; Vol. 8, No. 5; 2015 ISSN 1916-4742 E-ISSN 1916-4750 Published by Canadian Center of Science and Education An Investigation into Pragmatic Knowledge in the Reading Section of TOLIMO, TOEFL, and IELTS Examinations Alireza Karbalaei1 & Mehrnaz Kashkooli Rahmanzade2 1 Farhangian University, Tehran, Iran 2 Department of English, Kish International branch, Islamic Azad University, Kish, Iran Correspondence: Mehrnaz Kashkooli Rahmanzade, Department of English, Kish International branch, Islamic Azad University, Kish, Iran. E-mail: [email protected] Received: December 24, 2014 Accepted: March 18, 2015 Online Published: April 23, 2015 doi:10.5539/elt.v8n5p208 URL: http://dx.doi.org/10.5539/elt.v8n5p208 Abstract The present study focused on the analysis of listening sections of two international English proficiency tests, i.e. IELTS and TOEFL tests, and one local English proficiency test, i.e. TOLIMO from pragmatic perspective. An attempt was made to explore the areas of pragmatic knowledge presented, and to assess test takers’ pragmatic knowledge. For this purpose, 250 items from each of these three proficiency tests were collected and analyzed. IELTS and TOLIMO tests were taken from preparation textbooks available in the market, and TOEFL tests were taken from the tests administered from 2000 to 2004. To elucidate what areas of pragmatic knowledge was involved in each item, Jung’s (2002) classification of components of pragmatic knowledge was used. In this study, pragmatic knowledge, includedthe ability to perform speech acts, the ability to convey and interpret non-literal meanings, the ability to perform politeness functions, the ability to perform discourse functions, and the ability to use cultural knowledge. The results of the study showed that TOLIMO, TOEFL and IELTS tests are able to assess test takers’ pragmatic knowledge; however, higher instances of pragmatic knowledge components were involved in TOEFL and TOLIMO test. Keywords: listening comprehension, communicative competence, pragmatic knowledge, IELTS, TOEFL 1. Introduction Reading comprehension is the application of a skill that evolved for other purposes (listening or oral comprehension) to a new form of input (text). Whereas oral comprehension seems to develop “naturally” with minimal intentional involvement, reading comprehension is more challenging and requires deliberate instruction. Human beings have been using oral form of languages for 100,000 years or more (Donald, 1991), and almost all humans do it; reading comprehension has only been practiced for 5,000 years, and for most of that time. Most human beings did not do it (Olson, 1994). Thus, one of the sources of the difficulty of reading comprehension is its novelty (Kirby, 2006). Reading comprehension is the process of constructing meaning from the text. The goal of all reading instructions is ultimately targeted at helping a reader to comprehend a given text. Reading comprehension involves at least two people; the reader and the writer. The process of comprehending involves decoding the writer’s words and then using background knowledge to construct an approximate understanding of the writer’s message (Kirby, 2006, p. 161). One of the components of language proficiency knowledge is pragmatic knowledge. To be proficient in a language, EFL learners need to be pragmatically competent. Pragmatic knowledge, previously a neglected area in the realm of SLA, has increasingly taken more and more attention in recent years (Bachman, 1990; Garcia, 2004). That is, because being considered as a proficient second language, a user means not only having grammatical, lexical and phonological aspects but also having pragmatic knowledge (Corsetti, 2010). According to Van Dijk (1977), the pragmatic comprehension is different from linguistic comprehension because it calls for contextual informationsuch as the role played by interlocutors and status of them, the physical setting of the conversation, and the types of communicative acts that may occur in that context. Crystal (2008) defines pragmatics as “the study of language from the point of view of the users, especially of the choices they make, the constraints they encounter in using language in social interaction, and the effects their use of language has on the 208 www.ccsenet.org/elt English Language Teaching Vol. 8, No. 5; 2015 other participants in an act of communication” (p. 379). Pragmatics is concerned with rules and principles of language use in social contexts (Tan, 1994). Also, pragmatics involves using the language to show the linguistic roles increased by the social functions of language (Brown & Levinson, 1987). According to Flowerdew and Miller (2005), in order to comprehend a spoken message, four main types of knowledge are needed: phonology; syntax; semantics; and pragmatics. Pragmatics means to understand the meaning of produced utterances in particular situations. In this study, the reading section of Test of English as a Foreign Language (TOEFL), International English Language Testing System (IELTS), and a local English proficiency, TOLIMO (The Test of Language by the Iranian Measurement Organizations), were taken into accountregarding the involvement of components of pragmatic knowledge. Uso-Juan and Martinez-Flor (2006) have elaborated the role of reading in communicative competence framework. They believe that linguistic, strategies, intercultural, pragmatic competence, as well as discourse competence are related to reading ability. Another type of competence, which is the most pertinent component, is discourse competence which involves “the knowledge of written discourse features such as markers, cohesion and coherence as well as formal schemata with reference to the particular communicative goal and context of the written text” (p. 268). Linguistic competence is reported to be lowest level of ability required to understand a piece of text. Vocabulary is a subsection of this kind of competence, which is of high importance in the reading ability. Strategic competence deals with both learning and communication strategies. The former deals with metacognitice, cognitive and socioaffective measures which are taken by the reader to have a better understanding of the text, and the latter has to do with using reading strategies to make up for interpretation deficiencies. Intercultural competence refers “to the knowledge of how to interpret written texts appropriately within their sociocultural context” (270). The last and the most relevant competence to the present study is the pragmatic competence, which involves the understanding of the meaning which is beyond the locutionary; in other words, it deals with the illocutionary force of utterances. The knowledge of context is of paramount significance here to attach the right illocutionary force to an utterance. As it was already mentioned, one of the language proficiency components, which is influential in the process of comprehending a text is the pragmatic knowledge. Having syntactic and semantic knowledge does not guarantee students’ second language communicative competence. Students also need to possess pragmatic knowledge in order to be considered as a proficient speaker of a second language. “Inability to match utterances with contexts in which they are appropriate has affected students’ overall communicative competence” (Fakeye David, 2008, p.73). Not having enough pragmatic knowledge leads to misunderstanding and misinterpretation, which may break down the whole communication process? TOEFL, IELTS, and TOLIMO are said to be able to assess testees’ pragmatic competence. But, to what extent and what areas of pragmatic knowledge are assessed in these three proficiency tests? Therefore, themain objective of this study is to investigate to what extent pragmatic knowledge is assessed in reading sections of TOEFL, IELTS, and TOLIMO. The present study tries to fill the gap in the literature by probing into the pragmatic knowledge assessed in three proficiency tests, namely TOEFL, IELTS, and TOLIMO. No study has ever investigated the ability of reading sections of the international proficiency tests in assessing test takers’ pragmatic knowledge. 1.1 Research Questions The present study tries to answer the following questions: 1) To what extent can reading section of TOEFL assess pragmatic knowledge? 2) What areas of pragmatic knowledge are assessed in reading section of TOEFL? 3) To what extent can reading section of IELTS assess pragmatic knowledge? 4) What areas of pragmatic knowledge are assessed in reading section of IELTS? 5) To what extent can reading section of TOLIMO assess pragmatic knowledge? 6) What areas of pragmatic knowledge are assessed in reading section of TOLIMO? 7) Which test assesses pragmatic knowledge of test takers more? IELTS, TOEFL, or TOLIMO? 2. Literature Review According to Jung (2002), a person to be considered as a pragmatically competent language user needs to have the following aspects: 1) The Ability to Perform Speech Acts: speech act was developed by Austin (1962) and later more elaborated by 209 www.ccsenet.org/elt English Language Teaching Vol. 8, No. 5; 2015 Searle (1969). At the beginning of “How to Do Things with Words”, J. L. Austin asserts that “the business of a [sentence] can only be to ‘describe’ some state of affairs, or to ‘state some fact’, which it must do either truly or falsely” (1962, p. 6). A few years later in 1969, Searle worked more on speech acts and give functions to them. He then classified them into five categories, which according to Yule (2000) are as follows: a) Declaratives, which are those kinds of speech acts that change the world through the utterances. For example: • I now pronounce you man and wife. b) Representatives, which are those kinds of speech acts that state what the speaker believes to be the case or not. Statements of fact, assertions, and conclusions belong to this category. For example: • William Faulkner wrote The Sanctuary. c) Expressives, which are those kinds of speech acts that state what the speaker feels. They express various psychological states such as likes, dislikes, joy, sorrow, etc. For example: • Wow, how beautiful you are! • I really love your new dress. • The meal was delicious. d) Directives, which are those kinds of speech acts through which the speaker gets someone to do something. Commands, orders, requests, and suggestions belong to this category. For example: • Don’t hit your sister. e) Commissives, which are those kinds of speech acts that speakers use to commit themselves to some future action such as promises, threats, refusals, etc. For example: • I’ll help you. 2) The Ability to Convey and Interpret Non-literal Meanings: This ability is directly related to the Grice’s cooperative principles and the meaning of implicature. According to Stanford Encyclopedia of Philosophy by Wayne Davis (forthcoming), “Implicature denotes either (i) the act of meaning, implying, or suggesting one thing by saying something else, or (ii) the object of that act.” Conversational implicatures have become one of the main issues in the study of pragmatics. Grice (1975, p. 45) maintains that the dominant principle in conversation is cooperative principle: “Make your conversational contribution such as is required, at the stage at which it occurs, by the accepted purpose or direction of the talk exchange in which you are engaged.” Grice, who coined the term “implicature,” and classified the phenomenon, developed an important theory to explain and predict conversational implicatures, and describe how they used and are understood. 3) The Ability to Perform Politeness strategies: According to Holmes (2008), being polite is a complicated business in any language. It is difficult to learn because it involves understanding not just the language, but also the social and cultural values of the community. She also mentions that “generally speaking politeness involves taking into account the feeling of others.” (p. 281). According to major studies of politeness (Brown & Levinson, 1987; Lakoff, 1973; Leech, 1983), linguistic expressions display different degrees of politeness. Politeness, sometimes, is attached to indirectness (Austin, 1962; Brown & Levinson, 1987; Srinarawat, 2005). And, Indirectness is prevalent in all human languages (Srinarawat, 2005). Brown and Levinson (1987) classify direct speech acts as Face Threatening Acts (FTAs). In the direct speech act, the speaker employs a sentence whose meaning explicitly provides the hearer with the content of the intended act. Indirectness, on the other hand, refers to the speech act in which the expressed meaning of an utterance does not match the speaker’s implied or intended meaning (Srinarawat, 2005). An indirect speech act necessitates the speaker’s and the hearer’s common background knowledge and the ability to make inferences on the listener’s part (Rabinowitz, 1993). According to Watts (2003), investigation of directness or indirectness in speech act realizations is pertinent to politeness. Leech (1983) argues that indirectness usually comes hand in hand with politeness and that the “more indirect an illocution is, the more diminished and tentative its force tends to be” (p. 108). In probing into directness/indirectness and its effect on politeness, Blum-Kulka (1987) concludes that direct strategies are often taken as impolite since they lack a concern for face. On the contrary, Brown and Levinson (1987) maintain that the level of directness is to be ruled by universal principles of politeness. 4) The ability to carry out discourse functions. Since the last few decades, linguists have recognized that 210 www.ccsenet.org/elt English Language Teaching Vol. 8, No. 5; 2015 communication is not based solely on sentence-level (lexis and sentence structure) and that the investigation of language and language learning should incorporate longer stretches of text or what is known as discourse. Many linguists have investigated the relationship between sentences in a text and labeled this relationship as texture. A set of sentences constitute a text if there is a relationship within and between the sentences; otherwise, they would be only a group of unrelated sentences. “The connections between sentences are called cohesive relations” (Yule & Brown, 1989, p. 191). Different parts of a text (or a conversation or any stretch of language) are interrelated in different forms. Sometimes the underlying semantic relations between the sentences and propositions have the cohesive power and indicate texture (coherence); however, in many cases there are some linguistic elements which manifest the relationship between the facts and propositions in a text (cohesion). The most frequently studied markers signaling coherence relations are discourse markers. Fraser (1999) has defined DMs “as a pragmatic class, lexical expressions drawn from the syntactic classes of conjunctions, adverbials, and prepositional phrases.” He asserts that with certain exceptions, they signal a relationship between the segment they introduce, S2, and the prior segment, S1. Their core meaning is not conceptual but procedural, and their more specific interpretation is ‘negotiated’ by the context, both linguistic and conceptual. Fraser (1999) has divided DMs in four important groups: Contrastive Discourse Markers (CDM), Elaborative Discourse Markers (EDM), Inferential Discourse Markers (IDM), and Temporal Discourse Markers (TDM). 5) Cultural knowledge: Culture is what people “must know in order to act as they do, make the things they make, and interpret their experience in the distinctive way they do” (Quinn & Holland, 1987, p. 4). According to Bloch (1991), culture, which is studied in the realm of anthropology, is defined as what people must be acquainted with in order to function reasonably and effectively in their social environment. Social environment consists of social organizations and behaviors that are the instruments through which people relate to each other. “A society’s culture consists of whatever it is one has to know or believes in order to operate in a manner acceptable to its members, and to do so in any role that they accept for anyone one of themselves” (Goodenough, 1957, p. 167). So, cultural knowledge is “socially acquired: the necessary behaviors are learned and do not come from any kind of genetic endowment” (Wardhaugh, 2008, p. 216). When exploring culture, we have to consider concepts such as schema (plural, schemata), frame, and script. Yule (2000) argues that “a schema is a pre-existing knowledge structure in memory, (p. 85)” which can be either static or dynamic. When schema is fixed and static, it is called a frame. A frame is shared within a social group, which is like a prototypical version. For instance, in a frame for a restaurant, tables, plates, fork, spoon, and food will be come into mind. According to Yule (2000), a more vibrant type of schemata is called scrip, which is a pre-existing knowledge arrangement involving event sequences. Scripts are used to construct interpretations of accounts of what happened, and is a way of recognizing some expected sequence of actions in an event. Since this study is the first in the pragmatic area of evaluating reading sections in TOEFL, IELTS, and TOLIMO tests, the researcher did not find any related empirical studies neither in TOEFL, IELTS, and TOLIMO, nor in any other type of universal language proficiency tests. 3. Methodology In order to collect data for the purpose of this study, qualitative method was used. Regarding the design, descriptive design was taken into account for data collection. 3.1 Material Materials used in this study included reading sections of two internationally-accepted English proficiency tests, IELTS and TOEFL, and a local proficiency test, TOLIMO. IELTS Reading Section: Because the original IELTS tests were not accessible, the researcher used and analyzed IELTS preparation tests available in the market. Five series of reading comprehension tests were adopted from Practice Tests for IELTS 1 (Jakeman & McDowell, 1997), Practice Tests for IELTS 3 (2004), and insight into IELTS (Jakeman & McDowell, 2002). In general, 250 items were taken and analyzed in this study. TOEFL Reading Section: The Reading Comprehension section included 50 questions onreading passages. To determine test takers’ ability to understand written English, they must answer multiple questions about the ideas and the meanings of words in reading passages. In this study, five series of TOEFL from 2000 to 2004 were investigated. As a whole, 250 questions were used and analyzed. TOLIMO Reading Section: This test is similar to TOEFL. The Reading Comprehension section included 50 questions about reading passages. There are 5-6 passages and 8-12 questions about each passage. To determine test 211 www.ccsenet.org/elt English Language Teaching Vol. 8, No. 5; 2015 takers’ ability to understand written English, they should answer multiple-choice questions about the ideas and the meanings of words in reading passages. In this study, 5 series of preparation tests for TOLIMO were investigated. As a whole, 250 questions were under full scrutiny. 3.2 Procedure In this study, the reading sections of IELTS, TOEFL, and TOLIMO tests were analyzed according to the five areas of pragmatic knowledge mentioned in Jung (2002). Each item of these tests was analyzed one by one. In this section, the five areas of pragmatic knowledge are explained and exemplified. One of the main features of a research is a high degree of consistency. To estimate the inter-coder reliability, Spearman Correlation test was run. These tests were analyzed by three experts to make sure that the coding process is done precisely. 3.3 Data Analysis To answer the seven researchquestions, a series of Chi-square was calculated. As we were dealing with frequency of types of levels, two types of chi-square tests were used. A chi-square test for goodness of fit was usedfor variables with one category and chi-square test of independence was utilized for variables with more than one category. 4. Results Before data analysis, we should know that, for all statistical analyses in this study, .05 was used as the alpha level at which findings were considered to be significant. Several statistical tests were employed to address the different research questions. The data were analyzed using the Statistical Package for the Social Sciences (SPSS 16). Analysis of Speech Acts in TOEFL: Speech acts were assessed in 49.6% of items (N=124). Commissives had the lowest number of speech acts (N=13). Representatives were used in 29 items and expressives in 40 items. And the highes test number of assessed speech acts belonged to dirrectives (N=42). Also, there was a significant difference in the frequency of speech act categories, χ2=17.097, df=3, p=0.001. Table 1. Chi-square for the distribution of speech acts in TOEFL Speech acts Chi-Square 17.097 Df 3 Asymp. Sig. .001 Analysis of Discourse Markers in TOEFL: Discourse markers were used in 132 items of TOEFL (52.8%). Elaborative discourse markers were assessed in 33 items (13.2%). Contrastive discourse markers had 64 instances in TOEFL test (25.6%). and temporal discourse markers had the frequency of 22 (8.8%). The least discourse markers used was inferential (N=13, 5.2%). There was a significant difference between the category of discourse markers used in TOEFL (χ2=44.909, df=3, p<0.05). Table 2. Chi-square tests for the distribution of discourse markers in TOEFL Discourse Markers Chi-Square 44.909 Df 3 Asymp. Sig. .000 Analysis of Politeness Strategies in TOEFL: In 7.2% of items in TOEFL, politeness functions were included (N=18). Direct strategies had been used in 8 items (3.2%) and indirect strategies in 10 items (4%). As a whole there were 18 instances of using politeness strategies in TOEFL tests. However, as can be seen in Table 3, there was no significant difference in inclusion of politeness strategies. (p>0.05) 212 www.ccsenet.org/elt English Language Teaching Vol. 8, No. 5; 2015 Table 3. Chi-square tests for the distribution of politeness strategies in TOEFL Politeness Chi-Square .222 df 1 Asymp. Sig. .637 Analysis of Implicature in TOEFL: In TOEFL test, only in 113 items non-literal meaning has been assessed (45.2%). The maxim of quantity was assessed in 23 items (9.2%), and the maxim of manner in 31 items (12.4%). Maxims of quality 22 (8.8%) and relevance 37 times were used in TOEFL tests (14.8%). Yet, no significant difference was found in the category of implicature (χ2=05.336, df=1, p>0.05). Table 4. Chi-square tests for the distribution of implicature in TOEFL Implicature Chi-Square 5.336 df 1 Asymp. Sig. .149 Analysis of Speech Acts in IELTS: Speech acts were assessed in 22.4% of items (N=56). Commissives had the highest number of speech acts (N=26). Representatives were used in 21 items and expressives in 6 items. And, the lowest number of assessed speech acts belonged to dirrectives (N=3). Also, there was a significant difference in the frequency of speech act categories (χ2=27.000, df=3, p=0.00). Table 5. Chi-square for the distribution of speech acts in IELTS Speech acts Chi-Square 27.000 df 3 Asymp. Sig. .000 Analysis of Discourse Markers in IELTS: Discourse markers were used in 33 items of IELTS (13.3%). Elaborative discourse markers were assessed in 15 items (6%). Contrastive discourse markers and inferential discourse markers each had the frequency of 8 (3.2%). The least discourse markers used was temporal (N=2, 0.8%). There was a significant difference between the category of discourse markers used in IELTS (χ2=10.723, df=3, p<0.05). Table 6. Chi-square tests for the distribution of discourse markers in IELTS Discourse Markers Chi-Square 10.723 df 3 Asymp. Sig. .016 Analysis of Politeness Strategies in IELTS: In 11.6% of items in IELTS, politeness functions were included (N=29). Direct strategies had been used in 16 items (6.4%) and indirect strategies in 13 items (5.2%). As a whole there were 29 instances of using politeness strategies in IELTS tests. However, as can be seen in Table 7, there was no significant difference in inclusion of politeness strategies. (p>0.05) 213 www.ccsenet.org/elt English Language Teaching Vol. 8, No. 5; 2015 Table 7. Chi-square tests for the distribution of politeness strategies in IELTS Politeness Chi-Square .310 df 1 Asymp. Sig. .577 Analysis of Implicature in IELTS: In IELTS test, only in 19 items non-literal meaning has been assessed (7.6%). The maxim of quantity was assessed in 10 items (4%), and the maxim of manner in 9 items (3.6%). Yet, no significant difference was found in the category of implicature (χ2=0.053, df=1, p>0.05). Table 8. Chi-square tests for the distribution of implicature in IELTS tests Implicature Chi-Square .053 df 1 Asymp. Sig. .819 Analysis of Speech Acts in TOLIMO: Speech acts were assessed in 45.6% of items (N=114). Commissives had the lowest number of speech acts (N=22). Representatives were used in 29 items and expressives in 30 items. And the highest number of assessed speech acts belonged to dirrectives (N=33). Also, there was no significant difference in the frequency of speech act categories (χ2=17.097, df=3, p=0426). Table 9. Chi-square for the distribution of speech acts in TOLIMO Speech acts Chi-Square 2.788 df 3 Asymp. Sig. .426 Analysis of Discourse Markers in TOLIMO: Discourse markers were used in 122 items of TOLIMO (48.8%). Elaborative discourse markers were assessed in 59 items (23.6%). Contrastive discourse markers had 20 instances in TOLIMO test (8%). and temporal discourse markers had the frequency of 24 (9.6%). The least discourse markers used was inferential (N=19, 7.6%). There was a significant difference between the category of discourse markers used in TOLIMO (χ2=35.967, df=3, p<0.05). Table 10. Chi-square tests for the distribution of discourse markers in TOLIMO Discourse Markers Chi-Square 35.967 df 3 Asymp. Sig. .000 Analysis of Politeness Strategies in TOLIMO: In 6.4% of items in TOLIMO, politeness functions were included (N=16). Direct strategies had been used in 9 items (3.6%) and indirect strategies in 7 items (2.8%). As a whole, there were 16 instances of using politeness strategies in TOLIMO tests. However, as can be seen in Table 11, there was no significant difference in inclusion of politeness strategies. (p>0.05) 214 www.ccsenet.org/elt English Language Teaching Vol. 8, No. 5; 2015 Table 11. Chi-square tests for the distribution of politeness strategies in TOLIMO Politeness Chi-Square .250 df 1 Asymp. Sig. .617 Analysis of Implicature in TOLIMO: In TOLIMO test, only in 110 items non-literal meaning has been assessed (44%). The maxim of quantity was assessed in 23 items (9.2%), and the maxim of manner in 31 items (12.4%). Maxims of quality 20 (8%) and relevance 36 times were used in TOLIMO tests (14.4%). Yet, no significant difference was found in the category of implicature (χ2=5.855, df=1, p>0.05). Table 12. Chi-square tests for the distribution of implicature in TOLIMO tests Implicature Chi-Square 5.855 df 1 Asymp. Sig. .119 5. Analysis of Different Categories in TOEFL, IELTS, and TOLIMO The following table reveals the frequency of different categories of speech acts, expressive, representatives, directives, and commissives, in TOEFL, IELTS, and TOLIMO. Table 13. Frequency of speech acts in TOEFL, IELTS, and TOLIMO Expressives Representatives Directives Commissives Total TOEFL 40 29 42 13 124 Test IELTS 6 21 3 26 56 TOLIMO 30 29 33 22 114 Total 76 79 78 61 294 A chi-square test for independence indicated that there was a significant difference between TOLIMO, TOEFL, and IELTS with regard to assessing speech acts, (χ2=46.674, df=6, p<0.05). Table 14. Difference between TOLIMO, IELTS, and TOEFL with regard to speech acts Value df Asymp. Sig. (2-sided) Pearson Chi-Square 46.674 6 .000 Likelihood Ratio 48.385 6 .000 Linear-by-Linear Association .924 1 .337 N of Valid Cases 293 5.1 Discourse Markers Table 15 shows a sharp contrast between frequency of items in TOLIMO, TOEFL, and IELTS with regard to discourse markers. As can be seen in Table 15, in TOEFL out of 250 tests 124 items assessed discourse markers, while only 56 out of 250 items of IELTS were involved in assessing test takers’ knowledge of discourse markers. Also, 114 instances of discourse markers were found in 250 TOLIMO tests. 215 www.ccsenet.org/elt English Language Teaching Vol. 8, No. 5; 2015 Table 15. Frequency of discourse markers in IELTS, TOEFL, and TOLIMO Discourse markers Total CDM EDM IDM TDM TOEFL 64 33 13 22 132 Test IELTS 8 15 8 2 33 TOLIMO 20 59 19 24 122 Total 62 44 19 25 287 A chi-square test for independence indicated that there was a significant difference between TOLIMO, IELTS, and TOEFL tests with regard to assessing discourse markers (χ2=38.238, df=6, p<0.05). Table 16. Difference between TOLIMO, IELTS, and TOEFL with regard to discourse markers Value df Asymp. Sig. (2-sided) Pearson Chi-Square 38.238 6 .000 Likelihood Ratio 39.564 6 .000 Linear-by-Linear Association 10.960 1 .001 N of Valid Cases 287 Table 17 depicts the frequency of each politeness strategies in IELTS and TOEFL. There are two kinds of politeness principles: direct and indirect. Table 17. Frequency of politeness strategies in IELTS, TOEFL, and TOLIMO Politeness Total Direct Indirect TOEFL 8 10 18 Test IELTS 16 13 29 TOLIMO 9 7 16 Total 33 30 63 A chi-square test for independence indicated no significant difference between the frequency of IELTS and TOEFL in terms of assessing politeness function χ2=0.641, df=2, p>0.05 (see Table 18). Table 18. Difference between TOLIMO, IELTS, and TOEFL with regard to politeness functions Value df Asymp. Sig. (2-sided) Pearson Chi-Square 0.641 2 .726 Likelihood Ratio 0.641 2 0.726 Linear-by-Linear Association 0.003 1 0.954 N of Valid Cases 63 5.2 Implicature Table 19 shows the frequency of category of non-literal meaning. 216 www.ccsenet.org/elt English Language Teaching Vol. 8, No. 5; 2015 Table 19. Frequency of implicature in IELTS, TOEFL, and TOLIMO Implicature Total Flouting Quality Flouting Quantity Flouting Relation Flouting Manner TOEFL 22 23 37 31 123 Test IELTS 0 10 0 9 19 TOLIMO 20 23 36 31 110 Total 42 56 73 71 242 A chi-square test for independence indicated that there was a significant difference between IELTS, TOEFL, and TOLIMO tests in terms of assessing non-literal meaning, χ2=17.790, df=6, p<0.05. Table 20. Difference between IELTS, TOEFL, and TOLIMO with regard to non-literal meaning Value df Asymp. Sig. (2-sided) Pearson Chi-Square 17.790 6 0.007 Likelihood Ratio 23.930 6 0.001 Linear-by-Linear Association 0.524 1 .469 N of Valid Cases 242 5.3 Pragmatic Knowledge Table 21 depicts the frequency of category of pragmatic knowledge in TOEFL and IELTS. Table 21. Frequency of component of pragmatic knowledge in IELTS, TOEFL, and TOLIMO Pragmatic knowledge Total Speech acts Discourse markers Politeness functions Implicature TOEFL 124 132 18 123 397 Test IELTS 56 33 29 19 137 TOLIMO 114 122 16 110 362 Total 204 287 63 253 896 A chi-square test for independence indicated that there was a significant difference between the TOEFL, IELTS, and TOLIMO with regard to assessing pragmatic knowledge, χ2=64.088, df=6, p<0.05. Table 22. Difference between IELTS, TOEFL, and TOLIMO with regard to pragmatic knowledge Value df Asymp. Sig. (2-sided) Pearson Chi-Square 64.088 6 .000 Likelihood Ratio 53.688 6 .000 Linear-by-Linear Association 2.843 1 .000 N of Valid Cases 896 6. Discussion Q1. To what extent can reading section of TOEFL assess pragmatic knowledge? The purpose of this research question was to find the frequency and the percentage of the involvement of pragmatic knowledge in reading section of TOEFL test. This research question tries to find that in how many of 217