TOJET: The Turkish Online Journal of Educational Technology – January 2012, volume 11 Issue 1 A CONCEPT-MAP INTEGRATED DYNAMIC ASSESSMENT SYSTEM FOR IMPROVING ECOLOGY OBSERVATION COMPETENCES IN MOBILE LEARNING ACTIVITIES Pi-Hsia Hung Graduate Institute of Measurement and Statistics, National University of Tainan 33, Sec. 2, Shu-Lin St. Tainan, Taiwan 700 [email protected] Gwo-Jen Hwang Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology 43, Sec.4, Keelung Rd., Taipei, 106, Taiwan [email protected] I-Hsiang Su Graduate Institute of Measurement and Statistics, National University of Tainan 33, Sec. 2, Shu-Lin St. Tainan, Taiwan 700 [email protected] I-Hua, Lin Graduate Institute of Measurement and Statistics, National University of Tainan 33, Sec. 2, Shu-Lin St. Tainan, Taiwan 700 [email protected] ABSTRACT Observation competence plays a fundamental role in outdoor scientific investigation. The computerized concept mapping technique as a Mindtool has shown the potential for enhancing meaningful learning in science education. The purposes of the present study are to develop a concept map integrated mobile learning design for ecology observation and to examine the implementation effect for elementary school children. Also, the difference in growth slopes between gifted and average students is investigated. Eighteen gifted students and thirty average students were included in this study. A computerized dynamic assessment system which combines a Computerized Ecology Observation Competence Assessment (CEOCA) and a concept map integrated ecology observation learning design were implemented. The results of the hierarchical linear model (HLM) analysis reveal that the overall growth slope is significant (µ=0.27, p<.01). The effect size is 0.53. The growth slope of the gifted students is a little higher than that of the average students. However, the slope difference between ability groups is not significant. The results suggest that a well designed concept map integrated learning system demonstrates very promising potential for enhancing both the gifted and average students’ mobile observation competence. The system developed in this study could be a useful resource for elementary school outdoor learning design. Keywords: concept map, dynamic assessment, observation competence, mobile learning, growth slope BACKGROUND AND OBJECTIVE The issue of ecological environmental protection becomes increasingly crucial for environmental education. Regrettably, passion in environment inquiry is rarely fostered in the majority of classrooms despite the clear interest in many children, especially those in the primary years in science. Science literacy is the creative exploration of meaning in the natural world. It requires both observing and thinking. Scientific observation is the process of gathering information about objects, events or processes in a careful, orderly way. Organisms are characterized by diversity and unity. Despite the diversity, biologists are able to group organisms based on shared similarities. Classification helps us understand diversity. Life has many levels of organization. Each level of organization is more complex and has more properties than the previous level. In other words, concept of hierarchy is important for ecology observation. Observation generally involves using the senses, particularly sight and hearing. Observations may lead to unanswered questions. Scientific discovery often takes place when a scientist observes something no one has noticed before (Hung, Lin, & Hwang, 2010). Outdoor teaching is widely recognized as one of the best alternative teaching methods for scientific observation learning. However, some outdoor teaching approaches are ineffective because students lack expert guidance and appropriate outdoor learning tools. With the advantages of portability and easy information access, the use of mobile technology is a growing trend in education. Therefore, the application of information technology in outdoor teaching has become an attractive research topic (Hung, Lin & Hwang, 2009). Outdoor teaching Copyright © The Turkish Online Journal of Educational Technology 10 TOJET: The Turkish Online Journal of Educational Technology – January 2012, volume 11 Issue 1 provides an excellent opportunity for scientific observation and inquiry learning. The success of environmental science education is not only dependent on the knowledge and understanding of environmental challenges, but also on impassioned participation in environmental conservation activities. Observation competence plays a fundamental role in outdoor scientific investigation. Through well-organized observation activities on field trips, students became more motivated and engaged in science learning to enhance problem-solving competence and meaningful learning. However, the key point of effective teaching depends on validated instruction design. In a conventional learning activity for outdoor teaching, the students are guided by a learning-mission sheet prepared by the teacher, and write down their findings on the sheet after visiting each of the learning objects. Such a learning activity allows the students to observe the real-world objects without personalized guidance or support; consequently, some students might fail to pay attention to the key features to be observed, or fail to complete the mission owing to a lack of sufficient information or guidance (Chu et al., 2008; Hwang, Kuo, Yin, & Chuang, 2010). The rapid progress and advances in innovative mobile, wireless and sensor technologies have substantially revolutionized the ways in which outdoor learning activities for science education can be carried out (Hwang, Tsai, & Yang, 2008: Chu, Hwang, & Tsai, 2010). Mobile devices, e.g. mobile phones and Personal Digital Assistants (PDAs), have also become more popular as cognitive tools in science learning (Hwang, Yang, Tsai, & Yang, 2009; Hwang, Kuo, Yin, & Chuang, 2010; Vogel, Spikol, Kurti, & Milrad, 2010). Due to the attractive features of handheld computers such as portability, adaptability, flexibility, intuitiveness, and comparatively cheap prices, ubiquitous/mobile learning which integrates handheld computers with wireless networks in teaching and learning has become one of the leading topics in educational research (Chen, Hwang, Yang, Chend, & Huang, 2009; Hwang, Kuo, Yin, & Chuang, 2010; Liu & Hwang, 2010; Shih, Chuang, & Hwang, 2010; Hwang & Chang, 2011). Especially, with the advantages of portability and easy information access, mobile technologies are now used frequently in outdoor scientific investigation activities. Several studies have been conducted to investigate the effectiveness of mobile and ubiquitous learning which can provide an opportunity for students to keep accessing digital resources while learning in real-world scenarios (Chen et al, 2003; Westerlund, 2008; Chu, Hwang, & Tsai, 2010). Although those new technologies seem to be promising, researchers have pointed out that the students’ learning achievements could be disappointing without the aid of effective learning strategies or tools to engage them in improving their knowledge structure (Chu, Hwang, Tsai, & Tseng, 2010; Hwang, Shi, & Chu, in press), which is regarded as an important component of understanding in a subject domain, especially in science (Novak, 1990). The knowledge structure of experts and successful learners is characterized by elaborate, highly integrated frameworks of related concepts (Mintzes, Wandersee, & Novak, 1997). A knowledge structure, then, might well be considered as an important but generally unmeasured aspect of science achievement. In order to understand the advanced scientific concepts of the various disciplines, students cannot rely on the simple memorization of facts or the enrichment of their naive, intuitive theories. They need to be able to restructure their prior knowledge which is based on everyday experience and lay culture. The restructuring is known as conceptual change (Vosniadou, 2007). Concept mapping techniques are interpreted as representative of students’ knowledge structures and so might provide one possible means of tapping into a student’s conceptual knowledge structure (Mintzes, Wandersee, & Novak, 1997). A concept map is a graph structure containing nodes that are interlinked by labelled, directed arcs. Concept maps can be used as a knowledge representation tool to reflect relationships that exist between concepts that reside within an individual’s long-term memory. When constructing a concept map, the focus is the relationships among concepts. The combination of two concepts connected by a linking line and labeled by a linking word creates a proposition, which is the smallest linguistic unit that carries meaning (Jacobs-Lawson & Hershey, 2002). Generally speaking, in the practice of concept mapping, weighted scores are assigned based on the organization of a map’s hierarchical structure, concept-links, and cross-links. Especially, the inclusion of cross-links is a significant characteristic of concept maps in representing creative thinking. Cross-links are important to show that the learner understands the conceptual relationships between the sub-domains in the map (Novak & Canas, 2008). The concept mapping tasks are categorized as either “fill-in the blank” tasks where learners are provided with a blank structure of a map and lists of concepts and linking phrases, or “construct a map” tasks where learners are free to make their own choices. Assessments based on “construct a map” tasks more accurately evaluate differences in learners’ knowledge structures and elicit more high-order cognitive processes (Anohina, Graudina, & Grundspenkis, 2007). “Construct a map” tasks are better than “fill-in the blank” tasks for capturing students’ partial knowledge. However, “fill-in the blank” tasks can be scored more efficiently than “construct a map” tasks. Based on their characteristics, if used as an assessment tool, “construct a map” tasks are more Copyright © The Turkish Online Journal of Educational Technology 11 TOJET: The Turkish Online Journal of Educational Technology – January 2012, volume 11 Issue 1 suitable for formative assessment, while “fill-in the blank” tasks are a better fit for large scale assessment (Yin et al., 2005). The inclusion of concept mapping in learning activities is also recognized as one way to summarize understandings acquired by students after they study instructional materials. Both summarization writing and construction of concept maps are appropriate ways to improve students’ knowledge growth. They share a substantial common cognitive process in correspondence with concept mapping to select concepts, make propositions, and to hierarchize and structuralize the key concepts. It has also been found that training in concept mapping should be beneficial for enhancing text comprehension, summarization and writing skills (Chang, Sung & Chen, 2002; Riley & Ahlberg, 2004). Moreover, concept mapping creates less cognitive load than summarization and essay writing. However, when constructing concept maps in the conventional pencil-paper way, students could need extra effort in deletion and revision of partial or even whole maps (Anderson-Inman & Ditson, 1999). Researchers (Chang, Chen, & Sung 2001) have identified some weaknesses which make traditional style concept mapping inconvenient for interaction and feedback between learners and instructors. In contrast, computerized concept mapping assessment provides a number of advantages: greater flexibility of revision, instant feedback to learners, extensive feedback to teachers, reduced errors in comparison with human marking, as well as decreased time needed for supervising and marking of assessments (Akkaya, Karakirik, & Durmus, 2005; Tsai et al., 2001). The purposes of this study are to develop a concept map integrated mobile learning system for ecology observation and to examine the implementation effects for elementary school children. The difference in learning growth between gifted and average students is also investigated. CONCEPT-MAP INTEGRATED DYNAMIC ASSESSMENT SYSTEM The Concept-map Integrated Dynamic Assessment System (CIDAS) combines the Computerized Ecology Observation Competence Assessment (CEOCA) and a concept map integrated ecology observation learning design. The concept-map is applied as a Mindtool to enhance students’ ecology observation competence. The purpose of developing CIDAS is to investigate the characteristics of students’ learning progress and the need for supportive feedback for students’ knowledge construction. The system includes a worksheet-embedded PDA learning design, an e-library, and online feedback as scaffolding to guide, clarify, stimulate, and monitor students’ observational inquiries. The CIDAS functions as both an instruction and an assessment tool. The online feedback and the e-library have been developed to facilitate effective learning. The Computerized Ecology Observation Competence Assessment (CEOCA) is developed for assessing students’ ecology observation competence. The test items are presented with real pictures, films or concept maps. The CEOCA consists of three facets, i.e. knowledge, observation and conceptual relationships. Totally there are 40 items in CEOCA. Table 1 shows the test specifications for CEOCA. The alpha coefficient of CEOCA is around 0.72. The correlations between CEOCA and school grades, i.e. science, mathematics and Chinese, are 0.54, 0.47 and 0.51, respectively. The CEOCA demonstrates moderate convergent and discriminant validity (Hung, Hwang, Lin, Hung, & Wu, 2010). Figure 1 provides an illustrative example of concept map task. The task is to measure students’ competence on feature identifications of ecology system. Students can drag an alternative from right hand side to fill in the blanks of a structured concept map. The item format is also function as a model of knowledge structure. Table 1. Test specifications for CEOCA Content Basic Proficient Advanced Total Knowledge 2 12 6 20 Observation 5 6 3 14 Relationship 5 1 0 6 Total 12 19 9 40 Copyright © The Turkish Online Journal of Educational Technology 12 TOJET: The Turkish Online Journal of Educational Technology – January 2012, volume 11 Issue 1 Figure 1. An illustration of concept map item on CEOCA The concept map integrated ecology observation system was designed based on the three-layer framework of the cognitive load-oriented ecology observation approach proposed by Hung, Lin and Hwang (2010). The concept map was integrated into this system to scaffold students’ knowledge structure. The learning tasks are transferred by wireless communication to students’ handheld PDAs in the field. The students’ responses to tasks and their observation records are also sent back to the learning system. In the learning system, the PDAs function as portable notebooks and walking encyclopedias. The learning system also provides guided tasks, immediate feedback, e-library search functions for mangrove wetland ecological systems, and an e-diary editor. Figure 2 presents the illustration of these supporting functions. The concept map tasks are embedded in the learning system to scaffold students’ ecology conceptual schema. The tasks in activity I present a structured one-level concept map to clarify students’ basic knowledge about the features and classifications of mangrove species. The tasks in activity II especially encourage students to take notes on two-level comparisons of different species. The tasks in activity III guide students to use what they have learned to draw three-level inferences about the relationships among species. observation worksheet edit e-diary PDA note e-library e-diary annotation Figure 2. The illustration of the supporting functions on PDA A preliminary e-library consisting of a database of various species in mangrove wetland ecological system was developed for CIDAS. The content of the e-library is divided into two parts, i.e. plants (e.g. main plants and associated plants) and animals (e.g. birds, crabs, fish, and shellfish). Table 2 provides illustrative examples of e-library entries for animals. Copyright © The Turkish Online Journal of Educational Technology 13 TOJET: The Turkish Online Journal of Educational Technology – January 2012, volume 11 Issue 1 Table 2. An illustrative example of e-library entries for animals No. specie behavior feature habitat special note The length of the body is 74 – 85cm. The face and the beak are black. The More than 1,000 winter Black-faced end of the beak is wide and flat like a fish farm, birds immigrated B001 migratory Spoonbill spoon. The feathers are mostly white. swamp to Chiku wetland The wingspan is about 130-142 cm. this year. Legs are also black. METHOD Figure 3 presents the experiment design for evaluating the effects of using CIDAS on the students’ ecology observation competencies. The CEOCA is administered three times to generate the growth slope for participants. The growth slope of CEOCA is the criterion variable for examining the intervention effect of CIDAS. The tasks included in the worksheets are multiple-choice, short-answer, constructed responses and concept-map. The adaptive feedback is provided online to support the learning progress of CIDAS. Experimental group 1 Experimental group 2 11 gifted students 7 gifted students 16 average students 14 average students December, 2009 Pre-test of CEOCA 26th December, 2009 Applying the learning system of CIDAS to conduct three concept-map integrated to 16th January, 2010 mobile observation activities. Middle of January, 2nd test of CEOCA 2010 Applying the learning system of CIDAS to 12th to 28th conduct three concept-map integrated January, 2010 mobile observation activities. March, 2010 Follow-up test of CEOCA Figure 3. The CIDAS implementation flowchart Participants and Procedures Forty-eight fifth to sixth-grade students from six schools in Taiwan participated in this study. Eighteen participants were gifted students recruited from gifted classes while the other 30 were average students recruited from regular classes. Three mangrove wetland field observation trips were arranged within 3 months. During the field trips, each of the participants was equipped with a PDA, a digital camera, and a telescope. The observation learning objectives (Table 3) were mostly guided by the worksheets. On the first trip, the students were guided to carefully observe some target items. On the second trip, they were suggested to focus on comparison tasks. Copyright © The Turkish Online Journal of Educational Technology 14 TOJET: The Turkish Online Journal of Educational Technology – January 2012, volume 11 Issue 1 On the last trip, they were encouraged to draw the links among the animals and plants they observed. The participants used the equipment to record what they observed, and completed the worksheet tasks using the PDAs in the field. After each field trip, the participants were asked to finish learning diaries on the websites, based on their observations, within a week. Due to a lack of equipment, participants were divided into two experimental groups. The gifted and average students were mixed in these two groups. For each task completed on PDA, score or feedback will be provided immediately. Students are expected to conduct a scientific inquiry based on their observation records on PDAs after each field trip. They carried out further in-depth inquiry based upon the notes and questions raised autonomously during the field observations. The automatic scoring feedback system is developed to motivate students to elaborate their records, reflections and inquiries. Table 3. The descriptions of three outdoor activities for CIDAS Activity Site Time Objectives Tasks Multiple choice 1st Chiku Wetland 6 hr Observation on target items Short answer Concept-map Multiple choice Multiple true-false 2nd Mangrove Pond 3 hr Comparisons among targets Constructed responses Concept-map Multiple-choice Multiple true-false Constructed 3rd Sihcao Wetland 3 hr Relations of Ecosystem responses Concept-map Dynamic assessment instruments The concept map integrated ecology observation worksheet design was revised from the three-layer framework of observation worksheet design (Hung, Lin, & Hwang, 2010). Based on the guided inquiry learning theory and concept mapping theory, tasks are developed to sequentially scaffold students’ knowledge construction (Hung, Hwang, & Hung, 2010). The worksheet design arranged the learning objectives into three layers to balance the students’ cognitive load and challenge step by step. The three layers consisted of a) guided observation with one-level concept map tasks, b) autonomous observation with two-level concept map tasks, and c) scientific inquiry with three-level concept map tasks. Table 4 provides the specification table for the ecology observation formative assessment. The scoring rubrics for these concept map tasks have also been proposed and validated by Hung, Hwang and Hung (2010). Table 4. The specification table for ecology observation formative assessment Feature and Feature contrast Conceptual connection Item type Total classification of species of species and extension Multiple choice 25 11 7 43 Multiple true-false 0 4 3 7 Short-answer 3 3 3 9 Concept map 2 2 2 6 Total 30 20 15 65 FINDINGS AND DISCUSSION The ecology observation learning potential is defined as the slope of students’ performance on CEOCA in three month intervals. The hierarchical linear model (HLM) is applied for this analysis. Table 5 presents the descriptive statistics on three CEOCA z-scores for all students. For pre-test, the subjects included perform very close to the norm (0.04 vs. 0.00) of 5th and 6th graders with a smaller standard deviation(cid:31)(cid:31)vs.1.00). The results of student 2nd-test demonstrate substantial improvement on CEOCA. The follow-up test also suggests the subjects can maintain what they have learned quite well. In other words, after CIDAS intervention, students show persistent better observation competence. Table 6 provides the results of the students’ growth slope on the CEOCA. The results demonstrate that the overall growth slope is significant (µ=0.27, p<.01). In other words, for Copyright © The Turkish Online Journal of Educational Technology 15 TOJET: The Turkish Online Journal of Educational Technology – January 2012, volume 11 Issue 1 all the students included in this study, the CIDAS is effective for promoting their ecology observation competence. The effect size is 0.53 (about one half standard deviation, see Table 5). Relatively speaking, the effect size is noticeable compared to the difference between grades, namely 0.04 (Hung, Hwang, & Lin, 2010). Table 5. Descriptive statistics on three CEOCA z-scores for gifted and average students Gifted（N=18） Average（N=30） Total (N=48) Mean SD Mean SD Mean SD pre-test 0.27 0.51 -0.10 0.55 0.04 0.56 2nd-test 0.76 0.68 0.47 0.65 0.58 0.67 follow-up test 0.98 0.71 0.33 0.85 0.57 0.85 Table 6. Final estimation of fixed effects of unconditional model on CEOCA slope (N=48) Fixed Effect Coefficient Std. error T-ratio df Sig. INTRCPT, 0.13 0.08 1.67 47 0.10 B0 TIME slope, 0.27 0.06 4.69 47 0.00 B1 The contrast of the growth slopes for the two different ability groups on the CEOCA is provided in Figure 4 and Table 7. The growth slope of the gifted students is a little higher than that of the average students (about 0.14). However, the slope difference is not significant. Some concept mapping assessment has suggested that the concept maps constructed by high performing students are qualitatively and quantitatively superior to those of average performing students in science (Austin & Shore, 1993). Although gifted students inherently possess higher academic achievement, their scientific observation competence progress slopes may not significantly surpass those of the average students in a well-designed mobile learning system. However, the results of Table 7 also imply that the reservation effect between ability groups might be an important issue for further investigation. It seems that the average students forget some of the concepts or knowledge in follow-up test. On the other hand, the gifted students may keep on internalized what have been learned after the intervention. 1.2 1 y 0.8 abilit 0.6 gifted A C 0.4 average O E C 0.2 0 pre-test 2nd-test follow-up -0.2 Time Figure 4. Contrast of the growth profiles on CEOCA z-score for gifted and average students Table 7. Final estimation of fixed effects on CEOCA slopes for gifted and average students (N=48) Fixed Effect Coefficient Std.error T-ratio Df Sig. INTRCPT, Average student 0.02 0.10 0.17 46 0.86 B0 Gifted student 0.30 0.15 1.94 46 0.06 Average student 0.22 0.07 3.00 46 0.00 TIME slope, B1 Gifted student 0.14 0.12 1.20 46 0.24 CONCLUSION Copyright © The Turkish Online Journal of Educational Technology 16 TOJET: The Turkish Online Journal of Educational Technology – January 2012, volume 11 Issue 1 Scientific observation learning occurs through the learner’s participation in a “community of practice”. In other words, learners participate in complex, real world challenges. Outdoor observations are usually fun but superficial for most elementary school students. Any innovative instruction that ignores the structures which constitute human cognitive architecture is not likely to be effective. Novice observers should be provided with effective guidance on the concepts and procedures required for observations. To help learners understand the structure of what is to be learned is very crucial. Understanding leads to increased awareness and improved ability to remember. The interconnection of design of activity systems and learning cognition is very important. Formative assessment refers to assessment that is specifically intended to generate feedback on performance to improve and accelerate learning (Sadler, 1998). Black and William (1998) also claimed that well designed formative assessments have a great positive effect on students’ learning. Online feedback given as part of formative assessment helps learners to focus on and achieve their leaning goals in field. In other words, concept map integrated formative assessment of mobile learning plays a supportive role for novice learners. Ecology observation activity will be much more effective if mobile learning and assessment technologies could be appropriately integrated for elementary school students. In this study, a dynamic assessment approach is adopted to develop an ecology observation learning system for mobile learning. Moreover, concept map is integrated into three mobile observation activities to promote students’ ecology observation competences. Students are guided to observe the critical features of targeted species, to compare the similar species and to relate the species to the environment sequentially. Students are also encouraged to take notes and to pursue their own learning goal besides responding to the worksheet tasks. Online scoring and feedback are provided to support and monitor students’ learning progress. The results of this study suggest that the concept map integrated mobile learning design successfully enhances observation competencies for both average and gifted students. It is also found that the gifted students’ scientific observation performance is superior to that of the average students through all learning stages. However, there is no significant difference in the growth slope between the gifted and the average students. This suggests that the concept map integrated mobile learning demonstrates very promising potential for all students. Generally speaking, the intervention effect of CIDAS is quite substantial for both the average and gifted students. Without any extra effort, the difference of observation competence between the 5th and 6th graders is minor (Hung, Hwang, & Lin, 2010). In other words, a well designed instructional innovation can be beneficial for all students. The preliminary results suggest CIDAS is a valuable resource for mainstream science education. The structured concept map tasks are applied to demonstrate an effective way of knowledge organization. Students provide more scientific vocabularies, comparison and relational statements gradually in their diaries. The structural changes of students’ knowledge are also worth detailed discussions to reveal the characteristics of different growth profiles. So, using the formative assessment results (such as worksheet or diary) to crossed validate the inferences made by objective tests will be a promising direction for further studies. Moreover, the larger gap on the follow-up test on CEOCA between the gifted and the average students reveals an important issue for further investigations on the reservation effect for different ability groups. The differential effect of concept map embedded learning design also needs larger sample and longer intervention to clarify. ACKNOWLEDGMENTS This study is supported by the National Science Council of Taiwan under contract number NSC 99-2631-S-011 -002. REFERENCES Akkaya, R., Karakirik, E., & Durmus, S. (2005). A computer assessment tool for concept mapping. The Turkish Online Journal of Educational Technology, 4(3), 1-4. Anderson-Inman, L., & Ditson, L. (1999). Computer-based concept mapping: A tool for negotiating meaning. Learning and Leading with Technology, 26(8), 6-13. Anohina, A., Graudina, V., & Grundspenkis, J. (2007). Using Concept Maps in Adaptive Knowledge Assessment. Advances in information systems development: new methods and practice for the networked society, 469. Black, P. & Wiliam, D. (1998) Assessment and classroom learning. Assessment in Education. 5(1), 7-75. Chen, Y. S., Kao, T. C., & Sheu, J. P. (2003). A mobile learning system for scaffolding bird watching learning. Journal of Computer Assisted Learning, 19(1), 347-359. Chang, K. E., Chen, S-F. & Sung, T. C. (2001). Concept mapping system with scaffolding and guided learning. Journal of Computer Assisted Learning, 17(1), 21-33. Chang, K., Sung, Y. & Chen, I. (2002). The effect of concept mapping to enhance text comprehension and summarization. Journal of Experimental Education, 7(1), 5-23. Chen, C. H., Hwang, G. J., Yang, T. C., Chen, S. H., & Huang, S. Y. (2009). Analysis of a ubiquitous Copyright © The Turkish Online Journal of Educational Technology 17 TOJET: The Turkish Online Journal of Educational Technology – January 2012, volume 11 Issue 1 performance support system for teachers. Innovations in Education and Teaching International, 46(4), 421-433. Chu, H. C., Hwang, G. J., Huang, S. X., & Wu, T. T. (2008). A knowledge engineering approach to developing e-libraries for mobile learning. The Electronic Library, 26(3), 303-317. Chu, H. C., Hwang, G. J., & Tsai, C. C. (2010). A knowledge engineering approach to developing mindtools for context-aware ubiquitous learning. Computers & Education, 54(1), 289-297. Chu, H. C., Hwang, G. J., Tsai, C. C., & Tseng, J. C. R. (2010). A two-tier test approach to developing location-aware mobile learning system for natural science course. Computers & Education, 55(4), 1618-1627. Hung, P. H., Lin, Y. F., & Hwang, G. J. (2010). The formative assessment design for PDA integrated ecology observation. Educational Technology & Society, 13(3), 33-42. Hung P. H., Hwang, G. J., & Hung, X. T. (2010). A Concept Map Embedded Formative Assessment Design for Assessing Students’ Knowledge Structure in a Mobile Ecology Learning Environment. Paper presented at Asia-Pacific conference of Technologies Enhanced Learning (APTEL2010), Kansai University, Osaka, Japan. Hung P. H., Hwang, G. J., & Lin, I. H. (2010). The Implications of cognitive component analysis to the Computerized Ecology Observation Competence Assessment for Mobile Learning Design. Paper presented at Asia-Pacific conference of Technologies Enhanced Learning (APTEL2010), Kansai University, Osaka, Japan. Hung, P. H., Hwang, G. J., Lin, I. H., Hung, X. T., Wu T. H. (2010). The Development of a Computerized Ecology Observation Competence Assessment. Advanced Materials Research, 108-111, 860-865. Hwang, G. J., Tsai, C. C., & Yang, S. J. H. (2008). Criteria, Strategies and Research Issues of Context-Aware Ubiquitous Learning. Educational Technology & Society, 11(2), 81-91. Hwang, G. J., Yang, T. C., Tsai, C. C., & Yang, Stephen J. H. (2009). A context-aware ubiquitous learning environment for conducting complex experimental procedures. Computers & Education, 53, 402-413. Hwang, G. J., Kuo, F. R., Yin, P. Y., & Chuang, K. H. (2010). A heuristic algorithm for planning personalized learning paths for context-aware ubiquitous learning. Computers & Education, 54, 404-415. Hwang, G. J., Shi, Y. R., & Chu, H. C. (2010). A concept map approach to developing collaborative Mindtools for context-aware ubiquitous learning. British Journal of Educational Technology. doi: 10.1111/j.1467-8535.2010.01102.x Hwang, G. J., & Chang, H. F. (2011). A formative assessment-based mobile learning approach to improving the learning attitudes and achievements of students. Computers & Education, 56(1), 1023-1031. Jacobs-Lawson, J. M., & Hershey, D. A. (2002). Concept maps as an assessment tool in psychology courses. Methods & Techniques, 29(1), 25–29. Liu, G. Z., & Hwang G. J. (2010). A key step to understanding paradigm shifts in e-learning: Towards context-aware ubiquitous learning. British Journal of Educational Technology, 41(2), E1-E9. Mintzes, J. J., Wandersee, J. H., & Novak, J. D. (1997). Teaching science for understanding. San Diego, CA: Academic Press. Novak, J. D. (1990). Concept mapping: A useful tool for science education. Journal of Research in Science Teaching, 27(10), 937–949. Novak, J. D. (2002). Meaningful learning: The essential factor for conceptual change in limited or appropriate propositional hierarchies (liphs) leading to empowerment of learners. Science Education, 86(4), 548-571. Novak, J. D., & Canas, A. J. (2008). The theory underlying concept maps and how to construct and use them. Florida Institute for Human and Machine Cognition Pensacola Fl, Retrieved from http://cmap.ihmc.us/Publications/ResearchPapers/TheoryCmaps/TheoryUnderlyingConceptMaps.htm. Riley, N. R., & Ahlberg, M. (2004). Investigating the use of ICT-based concept mapping techniques on creativity in literacy tasks. Journal of computer assisted learning, 20(4), 244-256. Sadler, D. R. (1998). Formative assessment: Revisiting the territory. Assessment in Education: Principles, Policy & Practice, 5(1), 77-84. Shih, J. L., Chuang, C. W., & Hwang, G. J. (2010). An inquiry-based mobile learning approach to enhancing social science learning effectiveness. Educational Technology & Society, 13(4), 50-62. Tsai, C. C., Lin, S. S. J., & Yuan, S. M. (2001). Students' use of web-based concept map testing and strategies for learning. Journal of Computer Assisted Learning, 17(1), 72-84. Vosniadou, S. (2007). Conceptual change and education. Human Development, 50(1), 47-54. Vogel, B., Spikol, D., Kurti, A, & Milrad, M. (2010, April). Integrating Mobile, Web and Sensory Technologies to Support Inquiry-Based Science Learning. Proceedings of the 6th IEEE International Conference on Wireless, Mobile and Ubiquitous Technologies in Education (WMUTE 2010), Kaohsiung, Taiwan. Westerlund, J. (2008). Class size and student evaluations in Sweden. Education Economics, 16, 1928. Yin, Y., Vanides, J., Ruiz-Primo, M. A., Ayala, C. C., & Shavelson, R. J. (2005). Comparison of two concept mapping techniques: Implications for scoring, interpretation, and use. Journal of Research in Science Copyright © The Turkish Online Journal of Educational Technology 18 TOJET: The Turkish Online Journal of Educational Technology – January 2012, volume 11 Issue 1 Teaching, 42(2), 166-184. Copyright © The Turkish Online Journal of Educational Technology 19