THE BEHAVIOR ANALYST TODAY VOL. 12, NO. 1, 2011 ISSN: 1555-7855 SOTA, LEON, & LAYNG Thinking Through Text Comprehension II: Analysis of Verbal and Investigative Repertoires Melinda Sota, Marta Leon, and T. V. Joe Layng DYMO | Mimio – Headsprout Abstract Reading comprehension can be considered a complex human performance involving two integrated repertoires: a verbal repertoire and an investigative (generative) repertoire. This paper describes an analysis of these repertoires in terms which can ultimately inform the design of programs to teach them, using the analysis and design of Headsprout® Reading Comprehen- sion as an example. Keywords reading comprehension; content analysis; concepts; strategies If one, in front of a room full of people, wrote on the to teach. This paper describes the types of analysis that informed blackboard,“look at the ceiling,” some may verbalize,“look the design of Headsprout® Reading Comprehension and which at the ceiling,” while others may tilt their heads and look up. were based on an analysis of textual comprehension as described The ones looking up demonstrate comprehension. in Layng, et al. (2011). —Goldiamond and Dyrud (1966) „ CONTENT ANALYSIS as cited in Layng, Sota, and Leon (2011) The first problem in designing a program to teach reading com- T hough true, the above quote describes only the beginning of prehension involves determining what to teach. Several factors an analysis of comprehension which will ultimately lead to must inform this decision, including factors related to the cur- instruction that successfully teaches learners how to com- rent market, technologies, and resources, as well as factors re- prehend text. Comprehension is not a monolithic concept. It is lated to the repertoire itself. All of these factors were considered not something that someone either has or does not have. It is in the design of Headsprout® Reading Comprehension. The cur- not something that someone either can do or cannot do. Read- rent market was analyzed in terms of reading comprehension ing comprehension is what we call it when particular responses products available and the needs of schools, and technological are made in the presence of particular textual stimuli. Often, it and resource limitations and affordances were considered. Initial refers to public events as well as private events which we would learner analysis was also conducted in order to determine the typically call thinking or reasoning (see Layng, Sota, & Leon, entering repertoires of our target learners (see Twyman, Layng, 2011). When we say that a learner can comprehend what he or Stikeleather, & Hobbins, 2004, for a description of Headsprout’s she has read, we are making a generalization statement based design process). on a large pattern of stimulus-control topographies (Ray & Sid- The initial analyses of the program itself focused on two ma- man, 1970). These topographies vary across passages, questions, jor questions: (1) what is measured in school systems across and responses. For example, reading material may vary in terms states and publishers—or, more precisely, what are the stimulus- of the passage’s length, the reading level at which it is written, control topographies that, if present, will make an observer or and its subject matter, as well as its style, sentence structure, examiner conclude that the learner has demonstrated reading vocabulary, and so on. A question about the passage read may comprehension, and (2) what is the structure of the domain in vary in terms of length, structure, and vocabulary as well as the terms of composite and component repertoires, including pre- response required. Questions may be multiple-choice or open- requisite and coordinate skills (see Johnson & Layng, 1992). Answering these questions involves content analysis: identify- ended. They may require a spoken response or a written one. ing the repertoires to be taught and analyzing these repertoires These differences represent differences in stimulus-response re- into their components, including the type of learning involved, lations and, ultimately, in the programing involved in building the learning hierarchy, and the relations of one skill to another a reading comprehension repertoire. The first step in the design (Tiemann & Markle, 1990; Twyman, et al., 2004). of a program, then, is analysis of these stimulus-response rela- tions in terms which will lead to determining what repertoires ANALYSIS OF READING COMPREHENSION MEASURES Authorship is considered equal. We would like to thank Jay Thompson, Marleen We say that a learner comprehends what he or she reads when Adema, and Lauren Mahon for their critical comments. reading a text is followed by an evaluated change in referent be- 12 THINKING THROUGH TEXT COMPREHENSION II: ANALYSIS OF VERBAL AND INVESTIGATIVE REPERTOIRES 13 havior (see Layng et al., 2011). This behavior could involve se- lection response (multiple-choice) or a constructed response lecting an answer to a multiple-choice question, constructing an (open-ended). Answering questions across these two response answer to an open-ended question, or doing something (engag- categories involves different repertoires. For example, answer- ing in a task, following a procedure, and so on) that could not or ing open-ended questions involves skills of composition and would not have been done prior to reading the text. writing in addition to the skills needed to determine the answer One of the first steps in our analyses of content involved exam- to the question. When instruction is delivered via a computer, ining tests of reading comprehension currently used in schools typing skills are also necessary. Furthermore, answering open- to determine what was being measured. Thus, we gathered and ended questions involves absolute matching, which requires examined tests of reading comprehension for third and fourth a criterion to be set by a learner’s reinforcement history (see grade (the grade levels of our target population) from several Goldiamond & Thompson, 1967/2004). If learners do not have states in different regions across the country1. this history, then it would need to be provided for them—that We analyzed these tests to determine: is, built into the instructional program. These considerations as well as others, such as issues in evaluating varied responses 1. What stimuli and responses make up the stimulus-control with the current software capabilities available in the indus- topographies being evaluated? try, entered into the decision to focus on only multiple-choice 2. In what different ways can these relations be categorized? questions in the online program, with extension to open-ended 3. Which categorizations hold the most promise for program questions left for offline transfer and extension activities. There- development? fore, the analysis described in the current paper will focus on the class of questions requiring a selection response. When a student is reading, several stimulus-response relations are in effect (see Layng et al., 2011). In answering a question, the ANALYSIS OF THE DOMAIN STRUCTURE response is guided by stimuli including: (1) the passage read, (2) The initial analyses (the analysis of target learners’ entering the question asked, and (3) possible answers, if the question is a repertoires combined with initial content analysis) determined multiple-choice question. All of these serve to restrict response the overall objective of the program: to build the repertoires alternatives along specific dimensions—that is, if guidance by needed for learners to correctly answer multiple-choice ques- the passage and question is present, some classes of responses tions across literal/factual, inferential, summative, and derived- become more probable than other classes of responses upon meaning/vocabulary question types (see Leon, Layng, & Sota, reading. For example, upon reading about a birthday party, re- 2011, for a discussion of program design). We also determined sponses related to birthdays that are already in the repertoire that the reading level of the text in the program would range of the reader become more probable. A question about a cake from mid-second to mid-fourth grade. The next step was to fur- may increase the likelihood that responses related to cakes will ther analyze these skills in terms of the relations that make up be made (Skinner, 1957). In order for a learner to do well on a the necessary repertoire. test of reading comprehension, however, it is not enough that The matrix in Figure 1 displays categories of learning in terms reading about a birthday party results in an increase in related useful for closely analyzing a repertoire in order to design pro- responses. His or her behavior must meet the contingency re- grams to teach it. The columns encompass three broad types of quirements specified in the question. The questions, then, be- learning: differentiated relations (or psychomotor), discrimina- came our first basis for categorization. tive relations (or simple cognitive), and extended relations (or CATEGORIZATION OF READING COMPREHENSION QUESTIONS complex cognitive). The rows order these types by complex- ity—moving from single-skill units in the bottom row to com- Categorization based on question asked. The analysis of questions led plex combinations of units and sequences of units at the top. to defining several different categories, or types, of questions, Differentiated relations (the leftmost column) refers to relations each of which required a different response on the part of the chiefly involving the execution of muscle movements. The mid- learner. Questions could require the learner to (1) identify the dle column, discriminative relations, includes relations involv- most prominent theme (main idea) in a passage (e.g., “What ing specific stimuli and responses, as found in paired associate is this story mostly about?”), (2) derive the meaning of a word learning at a basic level and verbal repertoires at the highest or phrase from the surrounding context (e.g., “What does __ level. The rightmost column, extended relations, includes cat- most likely mean?”), (3) answer a question when the answer egorical relations as its basic unit and generative repertoires as has a point-to-point topographic correspondence with words its most complex. Extended relations involve responses to novel in the passage, or (4) answer a question when the answer has a stimuli—for example, those relations involved in abstraction thematic but not a point-to-point topographic correspondence would be considered to lie in the extended relations or com- with words in the passage. We categorized each of these types, plex cognitive column of the matrix (Skinner, 1957; Tiemann respectively, as (1) summative, (2) vocabulary or derived mean- & Markle, 1990). ing, (3) literal or factual, and (4) inferential. Many types of relations enter into a complex performance Categorization based on response type. Another major division be- such as that demonstrated by what we call reading comprehen- tween questions involved whether the question required a se- sion. Identifying the relations involved helps to determine what 1 Analyzing these tests as a starting point was not a judgment of the appropriateness to teach; what not to teach, and instead to assume the learner of these tests as measures of reading comprehension. Rather, it was a practical judg- can already do; and how to teach, in terms of the procedures ment related to the current needs of schools and learners. best suited to the type of learning involved. In designing Head- 14 SOTA, L EON, & LAYNG DIFFERENTIATED RELATIONS: DIFFERENTIATED RELATIONS: EXTENDED RELATIONS: PSYCHOMOTOR SIMPLE COGNITIVE COMPLEX COGNITIVE KINESTHETIC REPERTOIRES VERBAL REPERTOIRES GENERAL REPERTOIRES Response sequences combine Discriminative Sequences combine & Supplementary Verbal Stimulation Combined & recombine to produce recombine to produce new forms leading to analytic, practical, Units new forms or creative combinations or recombinations RESPONSE SEQUENCES DISCRIMINATION SEQUENCES ORDERED RELATIONS Conjunctive (criteria follow Successive (Conditional criteria follow Principles, Rule Applying, Complex varying response series) entire fixed or variable series) & Conditional Equivalences Linked Tandem (criteria follow fixed Units response series) Chain (criteria follow each Serial (Criteria response in a series) follow entire fixed or variable series) Present-Absent Present-Present Concepts, Functional & Stimulus Basic Relations (Paired Relations (Multiple Equivalence, Operations Units Associates) Discrimination) RESPONSES OCCASION-BEHAVIOR PAIRS CATEGORICAL RELATIONS EMOTIONS Figure 1. Types of learning as described by Tiemann and Markle (1990) and further refined by Layng Figure 1. Types of learning as described by Tiemann and Markle (1990) and further refined by Layng (2005, 2007). (2007). sprout® Reading Comprehension, we assumed that learners al- task for us in designing Headsprout® Reading Comprehension ready had a repertoire involving the motor skills necessary was to build on and take advantage of learners’ current verbal to move and click the computer mouse (the input device that repertoires in a way that increased the likelihood that learners would be used with the program). However, discriminative and would make correct responses to reading comprehension ques- extended relations are involved in reading comprehension in tions. different ways. Examining these relations and how they enter Extended relations: complex cognitive. The extended relations, or into reading comprehension is essential to designing programs complex cognitive, column is concerned with extension to nov- that will effectively establish the necessary repertoires. The fol- el stimuli. lowing sections examine each of these areas in greater detail. Categorical relations form the most basic unit of this column. Discriminative relations: simple cognitive. The basic units in the dis- Categorical relations include concepts (Markle & Tiemann, criminative relations column are occasion-behavior pairs. These 1969; 1974; Tiemann & Markle, 1990), functional (Goldia- occasion-behavior pairs are involved in paired associate and mond, 1962; 1966) and stimulus equivalence (Sidman, 1994), multiple discrimination learning. and operations (Engelmann & Carnine, 1991). A concept is a When occasion-behavior pairs are linked, they form serial or class of stimuli, each instance of which shares some proper- successive discrimination sequences. At the top of the simple ties with other instances of the class while varying across many cognitive column are verbal repertoires. A verbal repertoire in- other properties. The shared properties define the class. Rather volves a large number of occasion-behavior pairs and sequences than being essentially defined, these shared properties of a class of those pairs that combine and recombine in novel ways, but it are defined by contingency requirements resulting from the be- remains in the simple cognitive column because it is based on havior of the verbal community. They are not defined a priori specific occasion-behavior pairs. What is tested, in other words, except in cases such as the scientific definition of categories. We is the same as what is taught (Tiemann & Markle, 1990). say that a learner has learned a concept when he or she correctly Discriminative relations of major interest in reading compre- identifies novel instances of the concept and correctly discrimi- hension include intraverbal repertoires. By the time learners nates between instances that are and are not examples of the come in contact with the program, they have extensive intraver- concept—that is, when his or her behavior is guided by the same bal repertoires as a product of the contingencies set up by their instructional stimuli (SDi) across changing dimensional stimuli verbal communities. When learners read a passage and a com- (SDd) (see Goldiamond, 1966; Layng et al., 2011). We can clas- prehension question about it, changes in response probabilities sify this as an abstract tact (Skinner, 1957). occur based on stimulus guidance (after Donahoe & Palmer, Ordered relations form the next level of the extended rela- 2004) developed throughout the learners’ history. The ultimate tions column. Ordered relations include principles (statements THINKING THROUGH TEXT COMPREHENSION II: ANALYSIS OF VERBAL AND INVESTIGATIVE REPERTOIRES 15 of the relationship between concepts) and complex and condi- meaning. Because questions can be categorized in terms of their tional equivalences. type, they can be analyzed as abstract tacts. In concept analysis, Generative repertoires are at the top of the extended rela- instances of the concept are analyzed in terms of their critical tions column. This is the level at which creative problem solving features or attributes, or what attributes make the instance an occurs—for example, employing strategies that result in new example of that concept rather than a non-example, as well as forms in art and science (Tiemann & Markle, 1990). their varying attributes, or what attributes vary among examples of the concept. READING COMPREHENSION AS A COMPLEX COGNITIVE REPERTOIRE (EXTENDED RELATIONS) Critical attributes. The critical attributes of each question type are those attributes that make the question one type rather than Answering reading comprehension questions can be con- another. In review, the features include point-to-point cor- sidered a problem-solving task in which two major repertoires respondence between the words in the answer and the words are involved: (1) a verbal repertoire and (2) an investigative in the passage (literal/factual comprehension), thematic corre- or strategy (i.e., generative) repertoire. A verbal repertoire is spondence between the words in the passage and the words in built gradually through years of speaking, listening, reading, the answer (inferential comprehension), questions about theme and writing. Although a strategy repertoire is also complex, a (summative or main-idea comprehension), and questions about series of strategies focused on answering reading comprehen- the meaning of a word or phrase (derived-meaning/vocabulary sion questions can be learned relatively quickly and applied to a comprehension). variety of problem situations. A strategy is essentially an orga- When responding is guided by the critical features of the con- nized procedure or series of general steps to be taken to solve a cept, instructional guidance has been established (see Goldia- problem (see Robbins, 2004, 2011, for a more detailed discus- mond, 1966; Layng et al., 2011). Instructional guidance refers sion). Building a repertoire of strategies for use in situations in to guidance by instructional stimuli (SDi)—those stimuli which which the learner has to answer a question about what he or she restrict response alternatives along certain dimensions, making reads can vastly increase the likelihood that the learner—given some responses in a learner’s repertoire more likely than others. a passage–question–answer set that overlaps with the learner’s For example, when a learner tacts a question as a vocabulary verbal repertoire—answers that question correctly (see Layng question, some strategy-specific responses in his or her reper- et al., 2011). toire subsequently become more likely than other responses.2 Strategy analyses. Two sets of strategies were identified in ana- In addition to the features that distinguish one question type lyzing the steps involved in answering reading comprehension from another, questions were analyzed in terms of what features questions: a general set of steps that can be applied across all make an answer to a question correct. Because the focus of the questions and another set of steps specific to the type of question program is on multiple-choice questions, the possible answers being asked (literal/factual, inferential, summative, or vocabu- and the answers themselves were part of the targeted intersect- lary/derived meaning). The general steps involve (1) reading the ing stimulus-control topographies, along with the question and passage, (2) reading the question and possible answers, (3) de- the passage read. termining which specific strategy to use based on the question, Features which make one answer correct were analyzed for (4) applying the strategy by looking for specific information in each question type. For literal questions and inferential ques- the passage, and, finally, (5) answering the question. The analy- tions, those two features were the same: (1) the answer or an- sis of question types led to the creation of a specific strategy swer category must appear in the passage and (2) the answer for each type that formed the third and fourth steps within the must meet the requirements specified in the question. Consider overall strategy (determining which strategy to use and then ap- the following example: plying that strategy). In the third step, the learner asks him- or herself, “What is the question asking me to do?” The answer to Gus was working at the computer. He clicked, but nothing happened. this question is a categorization response—a tact—which serves He got worried and went to look for the teacher. as a supplemental stimulus that occasions the next step in that How did Gus feel when he clicked and nothing happened? particular strategy (see Layng et al., 2011; Leon et al., 2011). The design of specific strategies for each question type was A. He turned off the computer. based on concept analyses that identified both (1) the attributes B. He was a little upset. that differentiated one question type from another and (2) the C. He was a little happy. attributes that differentiated a correct answer from an incorrect answer within a particular question type. The former informed Answer A, “He turned off the computer,” has a thematic match programing related to the third step of the overall strategy— with the paragraph. There is categorical overlap with categories determining the type of question being asked and, therefore, such as “computers,” “what one does when a computer is mal- which strategy to apply. The latter informed programing related functioning,” and so on. In fact, turning off the computer could to the fourth and fifth steps of the overall strategy—applying the be a correct response to a question asking the learner to pre- strategy and selecting the correct response among alternatives. dict what the teacher might do or in what other ways Gus could The following sections describe these concept analyses. 2 Learners may not be able to describe the instructional stimuli that guide their be- Concept analyses. As previously noted, the questions on tests of havior. For example, a learner may be able to identify those items that are chairs and reading comprehension can be classified as four major types: those items that are not chairs, but may not be able to say why one item is a chair and literal/factual, inferential, summative, and vocabulary/derived one is not (see Layng et al., 2011). 16 SOTA, LEON, & LAYNG Table 1. The critical attributes and a sample of the variable attributes identified for inferential comprehension questions related to the passage–question–answer set. All attributes (both critical and variable) make up the dimensional stimuli. The critical attributes make up the instructional stimuli. Critical Attributes (SDi) 1. Answer category appears in passage 2. Answer meets criteria specified in question 3. Answer does not have topographic correspondence with words in passage Variable Attributes (SDi) Passage 4. Type 5. Length 6. Narrator a. Narrative a. One paragraph a. First person b. Expository b. Two–three paragraphs b. Second person c. Poetic c. Four+ paragraphs c. Third person Question 7. Type of characteristic asked 8. Question word 9. Phrasing about a. Why (reason for character’s a. Standard a. Personal quality action) b. Exclusion b. Sequence b. Why (cause/effect) c. Cloze c. Actions c. When (temporal) d. Spatial d. Objects d. When (conditional) e. People/animals e. What f. Places f. Where g. Events g. Which h. Time h. Who i. How (process) j. How (quantity) k. How (thoughts/feelings) Answer 10. Response topography a. Selection b. Construction have tried to solve his problem. “He turned off the computer” ple, the passage could be a narrative piece, an expository piece, has a predictive correspondence to the paragraph. The SDi guid- or a poem. It might be one paragraph, two paragraphs, or sev- ance exerted by the thematic match between Answer A and the eral paragraphs long. The question could begin with who, what, paragraph increases the probability that the learner will select when, where, why, how, or another word. The question could Answer A. ask about a character’s feelings or actions, the time an event The question, however, asks how Gus felt. The question speci- took place, the conditions under which something happened, fies that the answer falls within the category of feelings. When or what might happen next. One sentence from the passage may only the question is considered, either Answer B, “He was a little be sufficient to find or derive the answer, two sentences may need to be combined, or an entire paragraph may need to be upset,” or Answer C, “He was a little happy,” could be correct. considered. Ensuring that these variable attributes are varied Both fall into the category specified in the question. across the program is essential in programing for generic exten- The desired source of SDi guidance includes a combination sion across novel passages, questions, and answers. of the passage, the question, and the possible answers. Answer In analyzing concepts, the variable attributes are identified so B, “He was a little upset,” is the correct answer. The category that they may be varied systematically. This ensures that guid- “negative feelings” appears in the passage and meets the criteria ance by the instructional stimuli (SDi) is maintained across specified in the question. “A little upset” is a member of this changing dimensional stimuli (SDd) (Goldiamond & Thomp- category of feelings. Answer B has both attributes that make an son, 1967/2004; Layng et al., 2011). In the case of Headsprout® answer correct. The answer category appears in the passage and Reading Comprehension, two different aspects of variable attri- it meets the criteria specified in the question. The incorrect an- butes were identified. First, there were those attributes which swers each lack one of these attributes. The answer category of are relevant in establishing instructional guidance by question Answer A appears in the passage, but it doesn’t meet the criteria type. These variable attributes entered into the design of instruc- specified in the question. Answer C meets the criteria specified tion that teaches learners to identify the question type, increas- in the question, but the answer category does not appear in the ing the likelihood that they will use the correct strategy to an- passage. swer the question. Second, there were those attributes that vary Variable attributes. In addition to critical attributes, concept analy- systematically across passage–question–answer sets. Learners ses also identify variable attributes. Variable attributes are attri- must apply the strategy flexibly to new problems. Although this butes that vary across questions of a particular type. For exam- application is the same in that guidance by SDi is maintained THINKING THROUGH TEXT COMPREHENSION II: ANALYSIS OF VERBAL AND INVESTIGATIVE REPERTOIRES 17 Table 2. A sample of the variable attributes identified for inferential comprehension questions related to the relations between items in a passage–question–answer set. Passage-Question Relation 11. Degree to which question & passage share characteristics a. Literal b. Interpretive: word/phrase meaning c. Interpretive: sequence d. Interpretive: prediction e. Interpretive: likelihood f. Interpretive: author’s purpose g. Interpretive: category h. Interpretive: multi-step Passage-Answer Relation 12. Number of sentences in passage needed to 13. Degree to which answer & passage share derive full answer characteristics a. One a. One-category variation: word/phrase b. Two meaning only c. Three+ b. One-category variation: sequence c. One-category variation: prediction d. One-category variation: likelihood e. Two+ category variation f. Absence g. If-then determinations/computations across changing SDd, the response is different (i.e., identification required to derive the full answer, and the category of the an- of the strategy to use versus application of that strategy to the swer overlaps with the category of the relevant portion of the current problem). passage in terms of word meaning: “He was a little upset” and The variable attributes include attributes of the passage, the “He got worried.” question, and the answers, as well as attributes of the relations Now consider the following, slightly modified, set. The pas- between the passage and question and the passage and possible sage and answers are the same, but the question has changed: answers. Tables 1 and 2 list the critical attributes and some of Gus was working at the computer. He clicked, but nothing happened. the variable attributes identified for inferential comprehension He got worried and went to look for the teacher. questions. How did Gus feel when his mouse didn’t work? Let’s take a closer look at the previous example: A. He turned off the computer. Gus was working at the computer. He clicked, but nothing happened. B. He was a little upset. He got worried and went to look for the teacher. C. He was a little happy. How did Gus feel when he clicked and nothing happened? A. He turned off the computer. Here, the learner follows the same strategy—the same general B. He was a little upset. set of steps. However, these steps must be modified from the ones taken in the previous example. Nothing in the passage ex- C. He was a little happy. plicitly states that Gus’s mouse failed to work. The question is interpretive (see Passage-Question Relation in Table 2). That is, In this example, the passage is one paragraph long, it is a nar- the learner must locate words in the passage that are related to rative, and it is written in third person. The question is a “how” “when his mouse didn’t work.” In order to do so, the learner question about feelings, and it is written in a standard format must apply the steps of the strategy flexibly to the new problem. (i.e., it begins with a question word and ends with a question mark). BUILDING VERBAL REPERTOIRES Now, let’s look at the passage–question and passage–answer So far, the analysis described has focused on strategies. Of relations. In this question, there is a topographic correspon- course, in order to answer a comprehension question correctly, dence between some of the words in the question and words in the learner’s verbal repertoire must have extensive overlap with the passage. Learners read “when he clicked and nothing hap- the text (see Layng et al., 2011). Regardless of a learner’s abil- pened” in the question and can find those exact words in the ity to apply reading comprehension strategies, for instance, the passage. This makes it a “literal” inferential question in terms likelihood of the learner answering a question correctly if the of the question–passage pair (note that it is an inferential ques- passage, question, and answers were presented in an unfamiliar tion based on the answer not sharing a one-to-one topographic foreign language would be very low. correspondence with words in the passage). Two sentences are Building a verbal repertoire is an extensive and ongoing task. 18 SOTA, LEON, & LAYNG Table 3. Relations involved in vocabulary. Type of Relation Example Discriminative Paired Associates SD: word Response: say/write/select definition Extended Stimulus Equivalence SD: word Response: select definition SD: definition Response: select word SD: word Response: select picture SD: definition Response: select picture SD: picture Response: select word SD: picture Response: select definition Concepts (abstract tacts) SD: example including critical features of the concept S∆: non-example lacking one or more of the critical features Response: identify examples In designing Headsprout® Reading Comprehension, we felt it seeing the definition (B1), the relation between the word and the was important to build learners’ verbal repertoires in addition definition shows symmetry. If the learner then learns to select a to strategy repertoires and, also, to ensure that we provided the picture (C1) upon seeing the definition (B1) and as a result then necessary help on occasions where the text presented did not selects the picture (C1) upon seeing the word (A1), the relation overlap with the learner’s repertoire. The following sections between the word and the picture shows transitivity (Sidman, describe the analysis that formed the basis of programing de- 1994). The stimuli and responses that make up an equivalence signed to build and take advantage of learners’ entering verbal class are arbitrary. That is, the stimuli gain their relation to one repertoires. another from the pairing procedure, not from shared features. To teach vocabulary effectively, several different issues must The contingency defines the class, and the relations arise out of be considered. First, what is meant by “vocabulary?” What does associations between dimensional stimuli. it mean to know a word? The relations one is after must be speci- Concepts. The concept (or abstract tact) is a relation in which cer- fied. Table 3 presents a list of relations produced by an analysis of vocabulary, in terms useful for the programing of vocabulary instruction. Word DISCRIMINATIVE RELATIONS. (A1) Paired Associates. Words and their definitions may simply be oc- casion-behavior pairs. Upon seeing the word, the learner says or selects the definition. When the definition is only associated with that word, the word-definition pair remains a single paired associate. Upon the occasion of the word, the definition is said or selected. However, words in the definition may enter into many other relations already in the learner’s repertoire (for ex- ample, intraverbal relations and tactual relations). It is these re- lations which allow a program to telescope (Goldiamond, 1966) a learner’s history and rapidly establish guidance by a new word. Picture Definition EXTENDED RELATIONS. (C1) (B1) Stimulus Equivalence. Stimulus equivalence refers to the emergence of arbitrary relations which are not directly taught (see Figure Figure 2. Relations involved in stimulus equivalence among a word, definition, and 2). For example, if a learner learns to select a definition (B1) picture. Solid lines indicate taught relations, while dotted lines indicate emergent upon seeing a word (A1), and then selects the word (A1) upon relations. THINKING THROUGH TEXT COMPREHENSION II: ANALYSIS OF VERBAL AND INVESTIGATIVE REPERTOIRES 19 tain features of a stimulus (critical or defining features) guide structional guidance of the critical attributes. the response to that stimulus, while other features vary. In the Combinations of Discriminative and Extended Relations. Pairing a word presence of a novel instance of the concept, responding guided and a definition can have a substantial effect on the learner’s by the presence of these critical or defining features would be repertoire if the words in that definition enter into other rela- reinforced. In the presence of a stimulus that lacked one or more tions within the reader’s repertoire (see Figure 3). When they do, teaching a definition increases the likelihood that the tar- of these features, responding in the same manner would not be get vocabulary word also then enters into these relations. For reinforced. After a program of this type, if a learner respond- example, if the word is an abstract tact, and the words in the ed to new stimuli that included the critical features and not to definition are already part of the learner’s repertoire (i.e., the stimuli that lacked one or more of these features, the behavior words in the definition already guide responding by the relevant would constitute an abstract tact and would be under the in- instructional stimuli), then pairing the word and the defini- tion can tap into the instructional guidance that already exists. When the abstract tact is part of a conceptual network including Distant superordinate and subordinate concepts, defining the new con- cept in terms of an already established superordinate concept can allow the learner to respond to the new concept in terms of the features of its superordinate category. For example, clas- sifying a new animal as a “mammal” within a definition allows features of mammals to guide responding to the new animal (Markle, 1978). Far Away Figure 3 displays stimulus-equivalence relations, with poten- tial relations resulting from generic extension based on the defi- nition and the picture included in stimulus-equivalence train- ing. For example, imagine that the target vocabulary word is “distant.” The definition (likely to be in the learner’s verbal rep- ertoire) is “far away.” The picture illustrates both “distant” and “far away” in a scene that is likely to be familiar to the learner. Both “distant” and “far away” tact the same relation—specifi- Generic Extension cally, a difference between two points that is measured as greater He was from a distant land. relative to some standard. This difference is illustrated by the Her smile seemed distant. picture. Properties among instances of this concept also vary. “Far away,” for example, can tact the relation between two points It happened in the distant past in space, time, or social relations. To the extent that the features that define a concept enter into Word relations with already-known words, the concept itself is not completely “new” to the learner. In addition, to the extent that (A1) definitions and pictures overlap with the learner’s entering ver- bal repertoire, stimuli that enter into equivalence relations are not necessarily new. Therefore, instruction can be designed to test for, extend, and refine relations rather than build entirely new relations from scratch. This is an important distinction to make, as it ultimately guides the programing involved in build- Picture Definition ing the verbal repertoire. Where the learner’s verbal repertoire (C1) (B1) can be utilized in the program, it can be extended and reorga- nized with a minimum of instruction. Where the verbal rep- ertoire cannot be utilized (for example, in the case of certain scientific concepts for which learners have a less established, or no, entering repertoire), instruction would need to be very dif- ferent if the targeted outcome was establishing a tactual relation rather than a paired-associate or equivalence relation. Generic Extension „ CONCLUSION Analyzing a repertoire for the purpose of designing instruction begins with determination of the overall goal or objective of instruction. This goal or objective is then further analyzed in Figure 3. An illustration and example depicting potential generic extension as a result of a stimulus-equivalence procedure that includes a picture and definition already in terms of the relations that make up the necessary repertoire. In the learner’s repertoire. the case of reading comprehension, two integrated repertoires 20 SOTA, LEON, & LAYNG were identified and analyzed: a verbal repertoire and an investi- Markle, S. M., & Tiemann, P. W. (1969). Really understanding concepts: Or in frumious gative (generative) repertoire. These repertoires are made up of pursuit of the jabberwock. Champaign, iL: Stipes. both public and private events, such as those which we typically Markle, S. M., & Tiemann, P. W. (1974). Some principles of instructional design at higher cognitive levels. in R. Ulrich, T. Stachnik, & J. Mabry (Eds.), Control of human call “thinking” (Layng et al., 2011). Further analysis identified behavior: Vol. 3, (pp. 312–323). Glenview, iL: Scott, Foresman. the discriminative and extended relations involved in each of Ray, B. A. & Sidman, M. (1970). Reinforcement schedules and stimulus control. in these repertoires. W. N. Schoenfeld (Ed.) The theory of reinforcement schedules (pp. 187–214). NY: Although the current paper has focused almost exclusively on Appleton-Century-Crofts. analysis of content, it should be clear that analysis of learners’ Robbins, J. K. (2004). Problem solving, reasoning, and analytical thinking within the entering repertoires is also essential to programing instruction. Morningside model. in K. Johnson & L. Street (Eds.) The Morningside model of Instruction must start from where the learner is (the learner’s generative instruction: What it means to leave no child behind (pp. 126–144). current repertoire) and bring that learner to where he or she Concord, MA: Cambridge Center for Behavioral Studies. needs to be (the goal of the program). Analysis both of learners’ Robbins, J. K. (2011). Problem solving, reasoning, and analytical thinking in a class- entering repertoires and the content served as the basis for this room environment. The Behavior Analyst Today, 12, 42-50. programing, which is described in the next paper in this series Sidman, M. (1994). Equivalence relations and behavior: A research story. Boston: (Leon et al., 2011). Author’s Cooperative. Skinner, B. F. (1957). Verbal behavior. Englewood Cliffs, NJ: Prentice-Hall. „ REFERENCES Tiemann, P. W., & Markle, S. M. (1990). Analyzing instructional content: A guide to instruction and evaluation. Seattle, WA: Morningside Press. Donahoe, J. & Palmer, D. (2004). Learning and complex behavior. Richmond, MA: Ledgetop Publishing. Twyman, J. S., Layng, T. V. J., Stikeleather, G. and Hobbins, K. A. (2004). A non-linear approach to curriculum design: The role of behavior analysis in building an effec- Englemann, S. & Carnine, D. (1991). Theory of instruction: Principles and applications. tive reading program. in W. L. Heward et al. (Eds.), Focus on behavior analysis Eugene, OR: ADi Press. in education, Vol. 3 (pp. 55–68). Upper Saddle River, NJ: Merrill/ Prentice–Hall. Goldiamond, i. (1962). Perception. in A. J. Bachrach (Ed.), Experimental foundations of clinical psychology (pp. 280–340). New York: Basic Books. „ AUTHOR CONTACT INFORMATION Goldiamond, i. (1966). Perception, language, and conceptualization rules. in B. Klein- muntz (Ed.), Problem solving (pp. 183–224). New York: Wiley. MELINDA SOTA Goldiamond, i. & Dyrud, J. (1966). Reading as operant behavior, J. Money (Ed.) The DYMO | Mimio – Headsprout disabled reader: Education of the dyslexic child. Baltimore, MD: Johns Hopkins Press. 127 Broadway E. Goldiamond, i. & Thompson, D. ([1967] 2004). The blue books: Goldiamond & Suite 300 Thompson’s functional analysis of behavior. P. T. Andronis (Ed.) Cambridge, MA: Seattle, WA 98102 Cambridge Center for Behavioral Studies. email: [email protected] Johnson, K. & Layng, T. V. J. (1992). Breaking the structuralist barrier: Literacy and numeracy with fluency. American Psychologist, 47(11), 1475–1490. MARTA LEON Layng, T. V. J. (2005, May). The importance of understanding and “extending” Skin- DYMO | Mimio – Headsprout ner’s extended tacts for behavior analysis applications. invited tutorial presented 127 Broadway E. at the 31st Annual Conference of the Association for Behavior Analysis, Chicago, Suite 300 iL.Layng, T. V. J., Sota, M., & Leon, M. (2011). Seattle, WA 98102 Layng, T. V. J. (2007, August). Complex verbal relations and theory of mind. invited email: [email protected] workshop presented at the National Autism Conference, Pennsylvania State Univer- sity, University Park, PA. T. V. JOE LAYNG Thinking through text comprehension i: Foundation and guiding relations. The Behav- DYMO | Mimio – Headsprout ior Analyst Today, 12, 3–11. 127 Broadway E. Leon, M., Layng, T. V. J., & Sota, M. (2011). Thinking through text comprehension iii: Suite 300 The programing of verbal and investigative repertoires. The Behavior Analyst Today, 12, 21–32. Seattle, WA 98102 Markle, S. M. (1978). Teaching conceptual networks. NSPI Journal, (February) 4–7. email: [email protected]