DOCUMENT RESUME IR 017 958 ED 396 745 Taniar, David; Rahayu, Wenny AOTHOR Automatization of Student Assessment Using Multimedia TITLE Technology. 96 P"ii DATE 10p.; In: Learning Technologies: Prospects and N,"; Pathways. Selected papers from EdTech '96 Biennial Conference of the Australian Society for Educational Technology (Melbourne, Australia, July 7-10, 1996); see IR 017 931. Speeches/Conference PUB TYPE Descriptive (141) Reports Papers (150) EDRS PRICE MF01/PC01 Plus Postage. DESCRIPTORS Audiotape Recordings; *Computer Assisted Testing; Computer Graphics; Elementary Secondary Education; *Evaluation Methods; Foreign Countries; Graphs; Item Banks; Learning; *Multimedia Materials; Postsecondary Education; Videotape Recordings Object Oriented Programming; *Question Banks; IDENTIFIERS Questions ABSTRACT Most use of multimedia technology in teaching lnd learning to date has emphasized the teaching aspect only. An application of multimedia in examinations has been neglected. This paper addresses how multimedia technology can be applied to the automatization of assessment, by proposing a prototype of a multimedia question bank, which is able to handle questions containing not only text but also images, graphs, audio, and video recordings. One issue in automatization of student assessment is the availability of question banks. Some questions (text) can be grouped together with a single object (a graph, video recording, etc.), and some objects should only be displayed in portions relevant to the selected questions. Choosing which parts of the object to accompany questions has become a non-trivial task. An object-oriented technology provides a feature of message passing from one object to another. The object is stored as an aggregate of its components, not as an unbreakable unit, so the relevant parts of the object can be presented together with the corresponding questions. Section 1 of this paper is an introduction; section 2 describes the structure of multimedia questions, which consists of the object part and the question part; section 3 explains how to retrieve questions from the database, both through object based retrieval and question based retrieval; section 4 presents sample results; and section 5 provides a conclusion. (Contains 10 references.) (Author/SWC) *********************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. ***************.******************************************************* Automatization of Student Assessment Using Multimedia Technology David Taniar U.S. DEPARTMENT OF EDUCATION "PERMISSION TO REPRODUCE THIS OfteotEomonaiReseatchamimptovemm MATERIAL HAS BEEN GRANTED BY EDUCATIONAL RESOURCES INFORMATION Department of Information Studies CENTER (ERIC) Michelle Thompson El This document has been reproduced as Swinburne University of Technology received from the person or organization originating it. Hawthorn, Victoria, Australia O Minor changes have been made to [email protected] improve reproduction quality. TO THE EDUCATIONAL RESOURCES Points of view or opinions stated in this document do not necessarily represent INFORMATION CENTER (ERIC)." Wenny Rahayu official OERI position or policy. School of Computer Science and Computer Engineering LaTrobe University Bundoora, Victoria, Australia [email protected] Most use of multimedia technology in teaching and learning to date has emphasised the teaching aspect only. An application of multimedia in examinations has been neglected. In this paper, we present a prototype of multimedia question bank, which is able to handle questions consisting not only text but also images, video recordings, etc. We also describe retrieval techniques, and show some sample results. Introduction As multimedia technology has become more available, the benefits of using this technology in teaching and learning become more visible. Multimedia is often referred to as a combination of different medium (eg, images, graphs, audio, video) used in a software application (Christodoulakis and Koveos, 1995, Grosky, 1994). Incorporating these media in computer-b2sed tutorials (CBT), one of the devices in educational technology, re'sults in a much more flexible and interesting learning process. However, most of these tutorials concentrate on the delivery of the lesson only. Assessment, which is another part of teaching process, is often neglected. The use of multimedia technology in this area has not been considered in depth. In this paper, we would like to address how multimedia technology can be applied to the automatization of assessment. One issue in automatization of student assessment is the availabil'ty of question banks. Multimedia question banks typically consist of questions with multimedia features. This means that the questions can be presented not only in text, but also in any other forms, eg, images, video. Furthermore, some questions can be grouped and they refer to a common object (ie, image, video). For example, in the database there are ten questions which are based on the same video recording. A major problem arises when only a few of these questions are selected. A decision must be made whether to present the object as a whole or only parts of the object which directly relate to the questions. If the object is large, such as a video recording of 30 minutes, and only two or three questions are selected from this recording, it would be wiser if only relevant segments of the recording are played. Presenting the whole recording will not only distract, but also confuse examinees. Choosing which part(s) of the object to accompany the selected questions has become a non- trivial task. Our approach is to use the object-oriented technology which provides a feature of message passing from one object to another. In this way, the selected questions only need to send a message to the object, and let the relevant object partition and construct the segments to be presented together with the questions. This technique works because an object is stored as an aggregate of its BEST COPY AVAILABLE Automatization of Student Assessment Using Multimedia Technology 160 components, not as an unbreakable unit. Our previous work (Taniar and Rahayu, 1995) discussed the storage and retrieval techniques of mu1timedia questions in great detail. In this paper, we would like to present a prototype of a multimedia question bank. The rest of this paper is organised as tbliows. Section 2 describes the structure of multimedia questions. Section 3 explains how to retrieve questions from the database. Section 4 presents some sample results, and finally. Section 5 gives a conclusion. The Struf;ture of Multimedia Questions The two main parts of multimedia questions are object and question parts . Figure 1 shows a typical multimedia question. In the context of object-orientation, "object part" should not be confused with "object". Object part contains a "thing", which can be of any form, ie., image, graphs, recording, text (Taniar and Ranayu, 1995). In contrast, the term "object" in the object-oriented paradigm refers to an instance of a class (Dillon and Tan, 1993, Taniar, 1992). Questions 1 - 10 refer to the following object. QUESTION PART Figure 1. Multimedia Questions Object Part The object is the main focus to which questions in the question part refer. These objects can be either: temporal or spatial objects. Temporal objects are objects that span a linear dimension. An example of a temporal object is a recording of any type, such as audio, visual. These objects fill a single dimensional space which is the duration of the play. A key feature of this kind of object is that there is a starting point and an ending point. Spatial objects are static. They can be presented on x and y axis, or columns and rows. Some examples include images, graphics, text, etc. When the object does not fit into the provided space, a scroll bar is shown to enable users to scroll the object. Tanier and Rihayu 161 Question Part The question part consists of a set of questions referring to an object. The questions can be either: single point or multi point questions. A single point question is a question which has a reference to a single segment of an object. If the object is a temporal object, single point questions relate to a "moment" of the play. The "moment" can be interpreted as a section that cannot be divided into more sections. The question could be a question on what was said, or where it took place, etc. If the object is a spatial object, single point questions refer to a section of an image/picture. For example, a question can be asked about a part of an engine, or a block of a landscape. Multi point questions have reference to several sections of an object. The referred sections can be in serial or at random. Serial multi point questions associate with a series of adjacent sections. If it is in a temporal object, the sections are contiguous. However, in a spatial object, adjacent sections can be clustered based on columns or rows. On the other hand, random multi point questions link with several sections of an object in random order. In an extreme case, a question refers to the starting point and ending point of a recording. Therefore, when this question is selected, all parts of the recording must be played. This similar technique applies to spatial objects as well. Some Examples Figure 2 shows an example of questions with a spatial object. The object is a music score of Beethoven Symphony No. 5. Since the music score is very long, a scroll bar is provided at the bottom. The questions related to this object can be viewed by pressing the "Question" button. If the music is played, rather than displayed, the object becomes a temporal object. Figure 3 shows an example of a temporal object. The object itself is a video recording of an orchestra playing Beethoven Symphony No. 5. It is not possible to represent temporal objects on paper. Rather a moment of the recording is shown, where the orchestra is in action. The questions, which will be viewed by clicking the "Question" button, are suitable to test the listening comprehension of students studying at a conservatorium of music. Retrieval Techniques Question retrieval should be two-directional, ie. from the object parts or from the question parts. As a result, searching techniques should facilitate both object based and question based retrieval. Object Based Retrieval Content based searching of multimedia objects is considered immature as current video and audio management tools are based on pixel rather than the perceived contents (Smoliar and Zhang, 1994). Current research in the areas of image and pattern recognition is addressing this problem, and therefore is excluded from our implementation. In our prototype, each object part has a set of properties, ie., title, topic. Users are able to do a search based on information contained in these properties. Figure 4 shows an example of question retrieval based on objects. In this example, the user is able to search an object on a particular title. Once the object is found, all related questions will also be retrieved. If the number of objects in the database is small, searching can be done through a complete browsing of all objects. Once the objects are selected, the associated questions are then presented and chosen, as associated questions have been implemented as an aggregate of the object parts. Therefore, each object has a number of corresponding questions. Figure 5 shows a number of objects to be manually selected by double clicking a desired object. The title of the object itself appears at the top of each object. 4 air Beethoven Symphony No. 5 a smummiN 71imumMimmilmiwiEN killrallikaiiNENT'Ammemposila FJNM I Questions L. Orcbeeltel Fixagdras ZI .44 c ..-4 7 4r- Pi* - 7, ; S. I II .6.. 1 ' 163 Tanier and Rihayu . 144, Exit Eimmt$ Browsing Procrtres Bed Object gletrIems1 roc-1 Ergot is this: 4=m Figure 4. Property based object retrieval Figure 5. Object browsing Question Based Retrieval Selecting questions from a database can be done through a query facility within the database, which retrieves questions satisfying the selection criteria. This selection can be done through keywords or subject matching. At this stage of the implementation, content based searching for questions is not possible. Figure 6 shows a property based question retrieval. Th user has to enter the subject of questions to be retrieved. 6 Automatization of Student Assessment Using Multimedia Technology 164 2taperlies Elaratstag Egit Figure 6. Property based question retrieval Ouestion,Based Retrieval Figure 7. Questions browsing Another retrieval technique is based on database browsing. Users are presented with a complete list of questions in the database. The user can select a particular topic, and then within each topic, the questions can further be selected and chosen. This technique will be very useful when combined with the object-based retrieval, as the object based searching lists an object. Based on the selected object, a number of questions can be chosen either manually or randomly. Figure 7 shows an example of question browsing retrieval technique. Once the questions are selected, the accompanying objects are also retrieved. This mechanism is possible because each question is attached to an object. Sample Results Some retrieval results are presented as follows. Figure 8 shows an example of the result of multi point serial questions. The object presented shows a string part only (not a full orchestral part). Question 1 refers to the tempo of the music. The tempo is usually displayed at the beginning of a section. As the section concerned is the first movement of the symphony, only the very first few bars are displayed. Question 2 refers to the clef used by a particular instrument (ie., viola). Combining questions 1 and 2, only the first few bars of the symphony, which also include the viola part, are displayed. 7 165 Tanier and Rihayu Figure 9 shows an example of multi point random questions. Question 3 refers to the instruments playing on the few first bars on the symphony (in this case, they are Clarinet, Violin 1 and 2, Viola, Cello, and Double Bass). Question 4 refers to the Clarinet and Violin 1 sections. Both questions 3 and 4 are multi point random questions, because they refer to two or more different types of instrument which are not adjacent. Questions 1 and 2 ese based on the following music scow The tempo of this music is; 1, ff 0 3 flats ()Allegro con brio O 2/4 The clef used by Viola is: 2. 0 Bass Clef G Clel 0 Melody Clef O C Clef Figure 8. Multi point serial questions 166 Automatization of Student Assessment Using Multimedia Technology Music Examination Questions 3 and 4 are based on the following music scam: 3. In the beginning of Beethoven's Symphony V. only these instruments are playing: 0 Clarinet end Strings 0 Woodwinds and Strings 0 Full Orchestra *The Clarinet should have use 3 flats as in violin 1 section, instead of just one flat.' 0 True 0 False umpwwwwwwwwwwwwwwww.I.M......T.A..... Figure 9. Multi point random questions Conclusion A prototype of a multimedia question bank has been presented in this paper. The type of question that we concentrate on is an object referred to by several questions. Retrieving some of the questions can be very tricky because not all parts of the objects are concerned. Our system is able to select parts of the object which must be presented with the questions. Future work includes finalizing technical details of the system. It is also planned to have a usability testing of the system when available. 9 167 Tanier and Rihayu. References Christodoulakis, S. and Koveos, L. (1995). Multimedia Information Systems: Issues and Approaches, Modern Database Systems: The Object Model, Interoperability, and Beyond, W. Kim (ed), Addison-Wesley, ACM Press. Dillon, T., and Tan, P-L. (1993). Object-Oriented Conceptual Modeling, Prentice Hall of Australia Pty Ltd. Gemmell, D.J, Vin, H.M., Kandlur, D.D., Rangan, P.V, and Rowe, L.A. (1995). Multimedia Storage Servers: A Tutorial, IEEE Computer, May 1995, IEEE Computer Society. Grosky, W.I. (1994). Multimedia Information systems, IEEE Multimedia, Spring 1994, IEEE Computer Society. Rahayu, J.W, and Chang, E. (1993). A Methodology for Transforming an Object-Oriented Data Model to a Relational Model, Proceedings of the Twelfth International Conference TOOLS Pacific, Melbourne. Rahayu, W., Chang, E., and Dillon, T.S. (1995a). A Methodology for the Design of Relational Databases from Object-Oriented Conceptual Models Incorporating Collection Types, Proceedings of the TOOLS18 Pacific, Melbourne. Rahayu, W., Chang, E., and Dillon, T.S. (1995b). Transformation of Object-Oriented Collection Types into Relational Logical Models, Industrial and Poster Paper Proceedings of the 14th International Conference on Object-Oriented and Entity-Relationship Modelling, Gold Coast. Smoliar, S.W., and Zhang, H-J. (1994). Content-Based Video Indexing and Retrieval, IEEE Multimedia, Summer 1994, IEEE Computer Society. Taniar, D. (1992). Unifying Object-Oriented Features with Database Technology: A Survey, Masters Swinburne University of Technology, Melbourne. Thesis , Taniar, D., and Rahayu, W. (1995). Multimedia Question Banks: Storage and Retrieval Techniques, Proceedings of the Twelfth Annual Conference of the Australian Society for Computers in Learning in Tertiary Education ASCILITE'95, Melbourne. 1 0