International Journal of ePortfolio 2017, Volume 7, Number 1, 13-28 http://www.theijep.com ISSN 2157-622X Thinking Skills by Design: Using a Capstone ePortfolio to Promote Reflection, Critical Thinking, and Curriculum Integration Cathleen Morreale, Carol Van Zile-Tamsen, Cheryl A. Emerson, and Matthew Herzog University at Buffalo A capstone ePortfolio is a digital space where students can gather and integrate their learning experiences from their undergraduate careers into a meaningful whole, demonstrate their growth as learners, and connect their learning to the world. The process of creating a capstone ePortfolio equips students with the digital composition skills necessary for creating a professional career ePortfolio, helping them to showcase their strengths for future employment and for graduate or professional school applications. This project piloted an undergraduate capstone ePortfolio program designed to be the culminating experience for juniors upon completion of the general education program (i.e., core curriculum). Pilot program results, based on a group of 18 students from a variety of academic and demographic backgrounds, suggest that the capstone ePortfolio program can serve as a vehicle for promoting reflection, critical thinking, digital literacy and composition, and integration of curricular experiences. This article presents the results of a mixed-method assessment of the pilot and discusses how these results will be used to frame the semester-long capstone ePortfolio program for the undergraduate general education program at a large AAU research institution. This article describes pilot work for a capstone them well in educational and career endeavors. It course, requiring a culminating ePortfolio, within the includes: critical thinking, integrative learning, undergraduate general education curriculum (i.e., core quantitative reasoning, scientific reasoning, ethical curriculum) of a large AAU research institution in the reasoning, communication skills, and digital citizenship. northeast United States. Although many examples of The learning outcomes of the program include the the use of ePortfolios as educationally purposeful following: Through completion of the general education culminating learning experiences in academic majors curriculum, students will can be found (Cambridge, 2010; National Survey of Student Engagement, 2011), the capstone course and 1. attain and apply knowledge in written, oral, ePortfolio described here are required for all and visual communication; mathematics and undergraduate students in their junior year as part of the quantitative reasoning; and natural sciences; newly launched general education program (GEP), 2. acquire, apply, analyze, evaluate, and integrate including transfer students with significant credits of knowledge from a wide range of disciplines; general education completed elsewhere. Due to the 3. attain and apply critical thinking skills to scale of implementation and the significant impact this define and solve problems; new course will have on requirements for degree 4. demonstrate an understanding of human and completion, a pilot was conducted to assess practical cultural diversity within local and global aspects of course delivery, as well as the ability of the contexts; course content to help students achieve the desired 5. acquire the skills, technologies, knowledge, learning outcomes of the course and of the general ethical judgment, and personal responsibility education program. for effective citizenship, professional The new GEP, launching in fall 2016 with new and leadership, and lifelong learning. re-designed course offerings, is based on the curricular components of American Association of Colleges & The goal of the general education capstone and the University’s (AAC&U) Liberal Education and America’s required ePortfolio is to provide a mechanism through Promise (LEAP) initiative, developed to prepare students which students can critically review content from in broad thinking and communication skills and disparate general education classes and make emphasizing integrative learning (AAC&U, 2011). connections across them, integrating their work to make It also may be the first of its kind to incorporate broader knowledge connections that can be more easily several high-impact educational practices as a leveraged and applied in new learning situations purposeful, integrated package to improve student (Hauhart & Grahe, 2014; Kinzie, 2013; Mentkowsky & persistence (Carini, Kuh, & Klein, 2006; Kuh, 2008; Sharkey, 2011). Specifically, reflection within the Kuh, Cruce, Shoup, Kinzie, & Gonyea, 2008; NSSE, ePortfolio allows students to consider how they have 2007; Pascarella & Terenzini, 2005). been successful in their learning and how learning in This revised GEP is designed to help students learn the present situation relates to other contexts (Buyarski transferable higher-order thinking skills that will serve et al., 2015). From this reflection, they gain practice in Morreale, Van Zile-Tamsen, Emerson, Herzog Using a Capstone ePortfolio 14 metacognitive thinking, which they can use to help intended field of study, outlining their them self-regulate learning processes in future learning understanding of general education course topics situations (Flavell, 1979; Livingston, 2003). The and how these topics contributed to a deeper process of reflecting on learning in general education understanding of their intended major; and and integrating content from across general education 3. summarize the larger impact of the general courses increases the likelihood that students will education curriculum on their intellectual transfer knowledge and skills gained to study in the development during their time at the university major and to life outside the classroom (Mentkowski & delivered via the ePortfolio tool. Sharkey, 2011). Further, it raises the level of importance of the GEP by not allowing it to be This pilot project was designed to determine the something that students can just forget about as they extent to which the Capstone ePortfolio experience, in check off courses, but as an experience on which to practice, will promote student reflection, critical reflect (Mummalaneni, 2014). This cultivates the thinking, and curriculum integration and provide a realization that it has inherent value for them in later meaningful learning experience for all undergraduate studies and in their lives after college (Eccles & students, all of which are found in the literature of both Wigfield, 2002; Kinzie, 2013; Kruger, Holtzman, & ePortfolios and capstones to be productive outcomes of Dagavarian, 2013; Wigfield & Eccles, 2000). such an educational experience (e.g., Eynon & Gambino, 2016; Gardner & Van Der Veer, 1998; The Capstone ePortfolio Pilot as a Learning McGill, 2012; National Survey of Student Engagement, Experience 2007). In addition, the capstone provided an opportunity for assessment of the GEP (Berheide, A pilot was needed to ensure that the students in 2007). Artifacts for reflection and inclusion in the this newly revised GEP view the culminating ePortfolio ePortfolio were chosen by students, in consultations and the capstone course itself as a meaningful part of with instructors. Not all artifacts were required to their educational experience, where students could address every component of the general education apply “higher-order thinking, authentic learning, and curriculum, though all artifacts should address some multilayered decision-making while engaged in an component of the general education curriculum. experiential learning activity” (Buzzetto-More, 2013, p. 1), and bring a “holistic understanding to students’ Method educational journeys” (Kinzie, 2013, para. 2). A six- week mini-capstone was proposed. In alignment with Institutional Context and Participants the generally recognized purposes of capstones (Kinzie, 2013) and the use of reflective ePortfolios (Cambridge, This pilot program was conducted in a large public Cambridge, & Yancey, 2009; Eynon & Gambino, Research I institution in the Northeastern United States. 2016), the design of the pilot was meant to simulate the The institution will transition to the new GEP, conditions of a real capstone course and to assess the described above, in the fall of 2016 with the pilot ability of the online course design, as represented in the capstone program conducted in the spring semester capstone syllabus (Appendix A): (1) to engage students prior to the program launch. The shortened capstone ran in the capstone experience and promote their for six weeks during the midpoint of the semester as a perceptions of this experience as meaningful to their hybrid course (hybrid to provide additional feedback educations; (2) to promote the achievement of the opportunities in the development stage), with face-to- identified capstone learning outcomes (see Table 1); face opportunities for student engagement, peer support and (3) to be feasibly implemented across large groups, and feedback loops. (The actual capstone will numbers of students from disparate disciplines, be conducted solely online as proposed in the GEP and including students transferring in general education to meet capacity needs.) Students participated in the coursework from other institutions. program voluntarily through a proprietary product, the In the pilot, students were required to complete platform being both the course and the vehicle by three tasks: which they created and hosted their ePortfolios. Although the pilot students had not participated in the 1. upload examples of prior course work to new GEP, the pilot was adapted to allow for their own demonstrate the achievement of learning GEP to be incorporated into the new framework. outcomes of each component of the general The study personnel included two doctoral-level education curriculum into their ePortfolios; teaching assistants and two administration assessment 2. complete one essay in which they reflected on staff. Capstone instruction and course management the connections and meaningful integrations of responsibilities were handled by the teaching assistants, their general education coursework and their while the administrative assessment staff was Morreale, Van Zile-Tamsen, Emerson, Herzog Using a Capstone ePortfolio 15 Table 1 Capstone Learning Outcomes No. Outcomes CLO 1 Adapt and apply skills, abilities, theories or methodologies acquired in one situation to new situations CLO 2 Connect relevant experiences and academic knowledge CLO 3 Demonstrate an evolving sense of self as learner CLO 4 Integrate different forms of communication to enhance meaning (prose, sound, visual media) CLO 5 Formulate a concept of digital citizenship and be able to fashion an online identity that demonstrates an awareness of the public/private divide responsible for developing and executing the of the pilot, having completed the majority of their assessment plan. However, the whole team worked undergraduate general education curriculum. With the together, meeting regularly on the pilot project. In exception of one, these students were high achieving: addition, there were general education administrative all participants had cumulative grade point averages stakeholders funding and supporting the project. (GPAs) over 3.0, with nine (50%) having cumulative The participating students were recruited from a GPAs over 3.75. Among the majors represented were variety of undergraduate education programs, including single and double majors. including: accounting (n = 1), Ronald McNair and Student Support Services Federal psychology (n = 3), social sciences interdisciplinary TRiO Programs, the Honors College, and programs degree program (n = 3), political science (n = 1), targeted to support underrepresented and low income biochemistry (n = 1), biological sciences (n = 3), students, as well as via the transfer student listserv. biomedical science (n = 2), chemistry (n = 1), Participants were incentivized in several ways. First, geological sciences (n = 1), biomedical engineering (n they received expert guidance to help them develop = 1), and civil engineering (n = 1). This diversity their integrative and reflective ePortfolios, which has allowed for a wide range of curricular foci to be value in their career and academic development. represented in the capstone ePortfolios. Second, they were given an opportunity to present at a Access to digital technologies ranged across the prestigious campus event, the Celebration of Academic spectrum, with 64% indicating that they had some Excellence, which they could include on their resume access to digital technologies, 27% indicated they had or curriculum vitae as having been an active participant. nearly unlimited digital access, and only two students Finally, completion of the pilot and all ePortfolio indicating limited access with computer and internet components would result in one credit of successful access available only on campus or at public libraries. independent study being added to their transcripts and a Most of the students in the pilot (86%), had never taken deposit of $100 into their Campus Cash accounts. a class utilizing the ePortfolio platform. Their prior The “course” had an initial pool of 35 student experiences building ePortfolios varied, with the volunteers, with 25 of them committing to participation. majority (73%) having little to no experience, 18% with Eighteen students completed the capstone and all limited experience but considering themselves requirements and also provided consent to participate in beginners, and only two feeling comfortable with the research component of the pilot project. The ePortfolio platforms. primary reason offered for not completing the capstone Students utilized the Digication ePortfolio software, was “too many other demands on my time.” In the exit an online product, to produce their ePortfolios. They were survey, one respondent selected “the technology was provided a general template to follow but were allowed to too confusing.” Over two-thirds of the participants deviate from that template (which included a learning (68%) expressed a willingness to volunteer for a full philosophy and outline of GEP components as a guide) so semester pilot in the fall. that their ePortfolios reflected their own uniqueness, The final sample of 18 students included students creativity, and variety of artifacts. ePortfolios consisted of from many different backgrounds and majors. With text and multimedia, including pictures, video, and music, regard to gender and race/ethnicity, the sample was as well as PowerPoints and PDFs of assignments from a 72% female and 50% White and 50% Black. Five of the variety of disciplines. In addition, students included participants were international students. Academically, curricular and co-curricular experiences. Students were the students were diverse, as well. For those who gave directed to create a curricular ePortfolio, emphasizing permission to participate in the study, one was a special learning and development, as well as curriculum admittance student, one was a transfer student, and the integration, compared to a professional ePortfolio, which remaining 16 were admitted as freshmen. Four students is styled more on a resume format and aimed at had junior standing and 14 senior standing at the time employment goals. The Digication platform allows Morreale, Van Zile-Tamsen, Emerson, Herzog Using a Capstone ePortfolio 16 Table 2 Approaches to Learning Questionnaire Pre-test Post-test Scale α M SD M SD ES Self-regulated strategy use 0.60 4.17 0.42 4.16 0.65 0.02 Intrinsic learning motivation 0.56 4.28 0.54 4.33 0.58 0.14 Critical thinking 0.86 3.88 0.89 3.66 0.84 0.18 Integrative learning 0.72 4.19 0.50 4.27 0.51 0.18 multiple ePortfolios to be created, and students may adapt learning, and critical thinking (American Association of their capstone ePortfolio into a professional ePortfolio for Colleges & Universities, 2011). Cronbach’s alpha future uses. This was not explored in the capstone. internal consistency reliability estimates and descriptive statistics for the pre- and post- administrations are Study Design and Data Collection shown in Table 2. Evidence for the reliability of the scales indicates that Integrative Learning and Critical This study employed mixed methodology for both Thinking are much more reliable than Self-Regulated the development of the pilot and the research approach. Strategy use and Integrative Learning. However, all Mixed methods allowed the assessment team to results for this instrument should be considered triangulate results. Data were collected in the following primarily exploratory at this time. ways: (a) pre-post survey, (b) Approcahes to Learning Qualitative analysis of student portfolios. A Questionnaire, (c) qualitative analysis of student qualitative thematic analysis of student portfolio portfolios, (d) assessments of student assignments using content examined student use of ePortfolios to rubrics, and (e) student feedback via forums. reflect, think critically, and integrate their Instructor-created pre-post survey. The curriculum experiences. instructor-created pre-post survey contained a Assessments of student assignments using combination of open- and close-ended questions and was rubrics. As part of the instructional process, student administered via Google Forms. The pre-survey had ten assignments were assessed with rubrics developed by the questions, and the post-survey had 22 questions. This instructors. (Students provided feedback on usefulness of survey was designed to assess students’ understanding of rubrics). Student essays and artifacts were submitted to digital literacy, technical skills in digital composition, student ePortfolios by students through the ePortfolio and the purpose of a reflective capstone ePortfolio as platform. The platform was used for both peer and self compared to a professional presentation ePortfolio. In “grading,” as well as reflection on artifacts. See sample addition, in the post- version, students were asked to rubric adapted from the AAC&U Value Rubrics in identify aspects of the course that were most and least Appendix B. Rubrics were used to assess artifacts helpful to them in completing the weekly assignments individually and the ePortfolio holistically on learning and the culminating ePortfolio. outcomes and related to GEP components. Approaches to Learning Questionnaire. The 22- Student feedback via forums. The instructor led item online Approaches to Learning Questionnaire, three face-to-face and online forums to collect students’ developed by Van Zile-Tamsen and Livingston (1999) feedback. The instructors also maintained a detailed to assess students’ perceptions of growth in higher- record of interactions with students that occurred during order thinking skills as they progressed through the office hours and electronically. GEP, asked students to rate the extent to which statements describe themselves as learners on a five- Data Analysis point scale, from strongly disagree to strongly agree. This questionnaire contains subscales relating to Self- Approaches to Learning Questionnaire. Changes Regulated Strategy Use, Intrinsic Learning Motivation, in pre- and post-scores were examined to determine if Integrative Learning, and Critical Thinking. Students students changed in their perceptions of their responded to the questionnaire twice, during both the approaches to learning after completing the capstone first and last week of the pilot. This measure is still requirements. Since the sample size was so small (12 being piloted but has good concordance with agreed- students who completed both pre- and post- upon definitions of self-regulated strategy use (Van questionnaire), effect sizes were used to determine the Zile-Tamsen & Livingston, 1999), intrinsic motivation magnitude and direction of changes rather than for learning (Eccles & Wigfield, 2002), integrative traditional paired samples t tests. Morreale, Van Zile-Tamsen, Emerson, Herzog Using a Capstone ePortfolio 17 Qualitative analysis of student portfolios. picture at the Celebration of Excellence). Group Student ePortfolios were qualitatively analyzed for cohesion was surprising, as this was an online cohort reflection, critical thinking, and integration of the for a short six-week program. academic curriculum. Standards of qualitative analysis, From the pre-post instructor survey overall, the including thematic coding, were utilized. Each students’ understanding of digital literacy did not ePortfolio was analyzed individually and then the full change from the beginning to the end of the pilot. When set was re-analyzed as a whole with the set of codes for asked to define “digital literacy” on the opening survey, appropriate fit. Qualitative analysis was informally most students responded with “the ability to use triangulated with instructors’ understanding of rubric technology” or “to find information” on the Internet. assessment outcomes in areas of reflection and Only one student defined digital literacy as integration of curriculum. “maintaining some kind of profile or presence” on the Internet. When asked, “How has your understanding of Results digital literacy changed since completing the mini- course,” many responded “not a lot” or “I’m still not Instructor Pre-Post Survey, Student Forums, and sure what digital literacy means.” Of the few students Office Hours Feedback who noted a change in understanding, one wrote, “Being able to use such a tool is no longer sufficient Feedback was collected by the instructor through enough. Being able to maneuver such tool to present surveys, in-person and online student forums, and one's own thoughts and experiences as clearly as office hour discussions. The following is a summary of possible is my new understanding of ‘digital literacy.’” the surveys and those notes. Throughout the pilot The opening/exit surveys showed a greater course, students expressed appreciation for a moment to understanding of how an integrative capstone ePortfolio look back over their coursework, surprised by all they differs from a professional career ePortfolio. The had done and by the contrasts between who they were surveys, moreover, showed an improvement in as freshmen and the students they had become. technical skills using a digital media platform and a Intellectual growth was witnessed when students strong likelihood to use these new skills on digital realized that their positions on controversial social and media projects in the future. political issues had changed. In regards to introspection Furthermore, the survey indicated the order about their growth over their curriculum, one student students made greatest use of the following help stated, “I read my first research paper again and I resources: (1) ePortfolio startup guide (86%), (2) e- couldn’t believe those were my words. I totally mails to instructor (64%), (3) peer support (50%), (4) disagreed with everything I wrote as a freshman!” visual guide to ePortfolios (36%), (5) other online help Reflection could be found in other statements, as well. (e.g., ePortfolio platform videos; 21%), (6) e-mails to One student, for example, noted: “I haven’t looked back support technology support (14%), (7) scheduled office at the things I have done throughout my time here . . . in visits (7%), and (8) open lab walk-in hours (0%). a comprehensive and thoughtful way like this before.” Utilizing these resources and participating in the project Another student stated, “The sky’s the limit on all the helped the students develop their abilities. Of the 14 things that make you unique.” With regard to students who responded to the survey, a majority curriculum integration one student noted, indicated improvement in using a digital media platform (mean 3.36/4.0) and an increased likelihood of Reflecting back on my work made me realize how using their technical skills in other digital media a lot of it actually impacted me as a student, even projects in the future (mean 3.64/4.0). though I did not think it did at the time. I would not One resource students did use was the rubrics. be as well-rounded, open minded, or understanding Students made good use of the evaluation rubrics for as I am today it if were not for my general self-reflection in developing their ePortfolios, with most education courses. students having consulted the rubrics for two or more assignments prior to submission, M = 2.93, n = 14. Also, Another student said, “It made my gen ed courses on a scale of 1-5, a majority of students (57%) ranked actually mean something.” clarity of the rubrics at a 4 (5 = clearly articulated Unexpectedly, the instructors were impressed by learning outcomes). The mean was 3.57 (n = 14). the variety and depth of the stories, and narratives the Finally, the exit survey showed great satisfaction with students shared, providing insight into their lived the ways in which the pilot course fulfilled student experiences. The students reflected on and shared expectations, with 43% responding 5/5 (fully satisfied moments of discovery. Also, through peer groups and expectations) and 36% at 4/5 (nearly fulfilled), M = 4.21, n the pilot program, students developed a sense of = 14). While one student expressed surprise at the amount community (as evidenced by their desire for a group of work involved in the pilot, open responses to Morreale, Van Zile-Tamsen, Emerson, Herzog Using a Capstone ePortfolio 18 “unexpected outcomes” were overwhelmingly positive. online profile introductions; for example, “I’m an Many students noted surprise at realizing how much work international student from Ho Chi Minh, Vietnam (the they had actually done as an undergraduate, how many name ‘Saigon’ might ring a bell). I enjoy traveling, connections they were able to draw, how much their cooking Vietnamese cuisine, making crafts, and catching general education curriculum had actually impacted them up on current politics, human right[s] issues, and East as a student; they even realized the existence of “missed Asia/Southeast Asia’s news.” For others, the description opportunities” after looking back over their experiences. became more narrativized; for example, Approaches to Learning Questionnaire As a single mother of two teenage boys and a full- time student, I am in a unique class of adult learners . . As shown in Table 3, students’ scores on the self- . and I take great pride in both roles. As one can regulated strategy use scale remained quite similar from imagine, combining these two demanding roles pre- to post-questionnaire, increasing a negligible amount consumes the majority of my time, and leads to many (d = -0.02). In each instance, students rated themselves late nights and little sleep, but I wouldn’t give it up on the high end of the scale in terms of monitoring and for anything! My children are my world, but coming regulating their own learning. With regard to integrative back to school has given me something that I can be learning, students’ scores remained on the high end of proud of outside of being someone’s mother. the scale for both questionnaire administrations, but in terms of effect size, there was a small increase from pre Others presented their identities in resume format, to post (d = 0.18). In contrast, students’ critical thinking which blended the genre conventions of presentation scores and intrinsic motivation for learning scores and reflective portfolios. In addition, these were often decreased a small amount from pre- to post-questionnaire accompanied with pictures containing descriptions. (d = 0.18 and 0.14, respectively). Interestingly, they rated Their identity attributes were evident not only in their themselves initially lower in critical thinking at the introductory page but throughout their learning beginning, and they were even less confident in their philosophy and reflective essays when they reflected on Critical Thinking skills at the end of the pilot. In terms of their experiences while identifying who they were: “In initial ratings, intrinsic motivation for learning was my sophomore year I became a Teaching Assistant for highest at the beginning and also decreased. These this course. I recognized the need for a TA among my findings are not surprising in light of the qualitative classmates and was eager to volunteer as a TA.” analysis of their portfolios. Second was the use of ePortfolios to express emotion and values. Students would share their Qualitative Analysis of Student Portfolios viewpoints, standpoints, beliefs, and attitudes and portray their emotions and values through their In general, the reflective essays took the form of discussions and choice of visual media. For example, personal autobiographical narratives that were far more many students chose visual images of experiences that reflective than integrative. Even when present, critical held great passion for them in extracurricular realms of reflection tended toward autobiographical narrative, their education, such as study abroad experiences, with an inward focus upon self instead of greater, connecting with others, family, and friends, and global issues—again, with a few notable exceptions. pictures that represented their cultural and ethnic With regard to critical thinking, students made heritage. In one picture from a trip abroad, a student judgments, evaluations, and analyses of their own described the individual she was hugging and wrote, experiences, artifacts, and education as presented in “I’ve visited the Dominican Republic three times their ePortfolios. The majority of critical thinking was during spring breaks to teach English. While there, I represented through the individual artifacts, mainly discovered a passion for teaching that I brought back to submitted as unique assignments, and not represented in [university] with me!” Furthermore, when discussing the personal learning statement or reflective essay. their values related to learning they used words related However, the level of both reflection and integration of to passion and strong desire to engage in learning. In curriculum represented in the ePortfolios varied among another example, one student stated, the students. More specifically, several themes emerged from convergent theme analysis. These included I wanted my college experience to be more than this, attributes, emotion, values, narratives, reflections, and so I pushed myself . . . and spend more time early on integration of curriculum. learning the material . . . In order to truly learn I Attributes included students’ use of their ePortfolios understand that not only will it take time, but it will to share their identities in regards to their demographics also take a lot of motivation. I would say I motivate and academic data as direct points. For some students, myself to learn . . . Learning enables me to broaden this information was stated in language similar to other my perspectives and become a better educated, more Morreale, Van Zile-Tamsen, Emerson, Herzog Using a Capstone ePortfolio 19 well-rounded person; even if I am not particularly works and use that knowledge to help people better interested in the material, I recognize that there is their lives! . . . I also had a heavy scientific course still value in gaining knowledge. load including anatomy, chemistry, and nutrition. I have always found the biological sciences Another stated, “Learning has been a strength and passion attractive. I am unendingly intrigued by the inner of mine that has kept me going through the years.” workings of the human body and continue, to this Students also used their ePortfolios as a reflective day, to seek out opportunities to learn more about tool. Although specifically prompted, they presented how the body works! well-developed reflective essays and descriptions of artifacts that shared stories with a sense of connection Others spoke more broadly to the integration of the to something deeper or broader than the surface artifact curriculum connecting it to their personal and career or statement being displayed. One student included an lives: artifact that was her internship reflection journal. Most students demonstrated strong reflection skills. Only Through my general education requirements . . . I three of the 18 students had reflection skills that were was able to explore these empathy-driven interests. lacking or basic as demonstrated by the portfolio as a I explored cross-cultural understandings, whole. One student reflected, economically disadvantaged communities, race in America, and the legal system as it relates to I chose none of the samples for the mere fact that I morality. In each of these courses, I felt the fibers, did extremely well or utterly terrible on them or in the empathy which motivates me, resonate. As I the course; they were chosen to depict growth and explored the variety of stories so often forgotten or portray that there is always room for improvement overlooked by others, my curiosity in the human as one continues to pursue the undergraduate career experience only expanded, and it still expands with and even beyond that. my majors. Many reflective statements demonstrated that students Another student furthers this sentiment, stating that were truly able to make connections beyond surface observation. For example, By my senior year I really began to appreciate all the opportunities that I have had to learn. Three I think this essay is vital to include because it years ago, I thought my World Civilizations class explains my thoughts and concerns prior to my was futile to me as an Accounting major. As I service. I knew that my service would be a learning reflect back . . . I realize that the class did [affect] experience for me, but I was unsure what I would me, not necessarily the specific learning material, be taking away from it. but with the exposure I have gained by taking the course. I feel like I understand people better and Moreover, these reflective statements also showed their cultures. I understand others viewpoints and students developing more nuanced approaches to their the things that are meaningful to them. learning situations as can be seen in the following Specifically, this impacts my role as a global response: “Since the course, I have grown to learn that it citizen because it is important to have awareness of is okay to have different opinions than others; one what is going on in the world. person’s opinion is not necessarily greater than another’s.” Students also further established Student Artifacts Assessed by Rubrics developments in their understanding of the complexity of meaning: “Ever since writing this paper I have been able Rubrics, based on the LEAP rubrics, were utilized to dig deeper when researching and analyzing other to assess student work. Several themes emerged from topics. I have learned to look beyond the surface in order the use of these rubrics. to truly find the meaning behind certain things.” “Connections” took the shape of unfolding student Lastly, integration of curriculum was a theme self-discovery, with personal narratives of how students expressed in the discussion of academic experiences beyond discovered their major and style of learning, and single experiences or courses. Some students spoke to how recognized their personal growth as a student. It was their curriculum actually focused in the integration: difficult to identify specific evidence of connections between “examples, facts, or theories from more than Interestingly, my course choices foreshadowed the one field of study or perspective.” Because of the path I would eventually end up taking. Courses autobiographical nature of the reflections, the unifying such as developmental psychology and parenting or “connecting” factor was the individual student, displayed my inner desire to learn how the mind usually limited to one field of study. Morreale, Van Zile-Tamsen, Emerson, Herzog Using a Capstone ePortfolio 20 Evidence of ability to “adapt and apply skills” most critical thought regarding global-digital citizenship often appeared in community service, experiential since there was no single place for students to articulate learning, or alternate break experiences, where students explicitly their understanding of digital citizenship in used their language, math, and science skills to help the ePortfolio assignment. others, contribute to an internship assignment, or At this time, the rubric categories are not strongly participate in cultural exchange programs. Students supported by actual evidence in the ePortfolios. As a who had not participated in activities beyond the courtesy to our volunteer pilot students, the instructors classroom showed difficulty in fulfilling this portion of merged certain aspects (such as digital citizenship and the capstone. In applying this portion of the rubric, it collapsing the two connections categories into one) was difficult to distinguish between a level 3 and 4, as a when evaluating their work, pending further revisions judgment had to be made whether a student was solving to the program (see below). The current capstone a difficult problem (vs. a problem), a complex issue (vs. ePortfolios yielded meaningful results to the students, an issue), and whether or not they had done so in an but fell short when held against the rubric standards, as original way. In many cases, the students narrated currently worded. For many categories, it was difficult problems of a personal nature, such as locating student to determine what distinguished a 2 from a 3 or a 3 help services on campus or pursuing a social science from a 4. For example, what evidence would we look degree instead of entering the medical field. There was for to determine whether a student uses or adapts skills also evidence of ability to adapt and apply skills within to new situations, or to illuminate concepts vs. deepen students’ academic coursework samples, but these were understanding? applications to individual problems without evidence of Overall, the students had a more positive transfer to new situations. experience using the evaluation rubrics for self- How students “connect relevant experiences and evaluation than did the instructors. While the rubrics academic knowledge” revealed, through the rubrics, an apparently articulated the learning outcomes in a way amount of overlap between this category and that is helpful to the students, the instructors found it “Articulate Connections” above, especially since both difficult to align the rubrics with tangible evidence from categories emphasize connections between multiple the ePortfolios. That being said, the student self- fields of study. The added element of this category evaluations of the final ePortfolios were well in keeping seems to be an emphasis on experiences outside the with instructor evaluations. Few students scored classroom. If anything, this category more effectively themselves either significantly higher or lower than addressed the sort of reflections in the pilot. marks given by instructors. Demonstrating their “sense of self as an evolving learner” category was the easiest to validate, since both Discussion the learning philosophy statement and reflective essay offered an opportunity for students to address their The capstone course was designed to foster critical ongoing progress as learners and to project plans for thinking skills through a variety of tasks and processes themselves into the future. This category was especially within the capstone project. For example, students had well suited to the mode of personal reflection the the opportunity to maximize the use of hierarchy for students adopted in their essays. sections, pages, and modules, which promoted In the rubric theme “integrate different forms of conceptual understanding. For students who opted to communication,” there was much room for present their written content in PowerPoint format improvement, especially at the point of integrating (which helps to fulfill the multi-media requirement), the different forms of communication. While every student slides needed to be accompanied by either a written or at least attempted to create a multi-media design, few oral script that meets the general requirement for the actually “integrated” their visual and textual material. scope of the composition assignment (translated as Videos were rare and presented with little to no minimum word count). Depth and development of commentary to “enhance meaning, making clear the thought was lacking in all PowerPoint format interdependence of language and meaning, thought, and presentations. In addition, students own ratings of their expression.” Future iterations of the project may need critical thinking skills decreased a small amount after to place greater emphasis on citation of sources for completion of the pilot. images as well as verifying that embedded media Reflection was promoted throughout the project actually “works” (example: certain add-ons work for through several integral assignments, such as the Mac but not for PC). philosophy statement and the reflective essay. Choosing Finally, there were the criteria for “digital which artifacts to include over the course of their citizenship.” While the instructors were confident in the curriculum required critical reflective observation on students’ ability to present themselves in a responsible the part of the students. Better instruction for students is manner, it was difficult to find evidence of deeper needed on the rhetorical moves necessary for Morreale, Van Zile-Tamsen, Emerson, Herzog Using a Capstone ePortfolio 21 incorporating and developing examples in a thoughtful, connections or applied skills. General education reflective manner rather than simply naming a title of a outcomes would need to be made explicit in the course or mentioning hastily a relevant experience in evaluation rubric, in terms of what sorts of connections their essays. The current reflective essays and learning the students are to formulate, and whether they are philosophy statements tend to offer broad drawing upon academic coursework or experiences generalizations, with little support or development. beyond the classroom. For example, students’ ability to Individualization is important in reflection. It was think critically about global issues, even if clearly learned in the pilot that we must guide students through articulated in their writing, may or may not mean they the process of individualizing the general template into have achieved the stated learning outcomes of drawing their own design, reflective of their interests and connections or adapting skills to explore complex philosophy of learning. An opportunity for increasing problems. critical reflection might be in the philosophy statement; Digital literacy was another important element of for example, recasting the current learning philosophy the capstone ePortfolio. Building on their statement to address the meaning of digital literacy, communication skills in this digital platform made it digital citizenship, and the ethical challenges and easier for students to envision their readers. The obligations of lifelong learning in a digital world. The ePortfolio platform in and of itself allowed students to style of writing for this essay should be critical engage in digital citizenship. Students became part of a reflection rather than personal autobiographical digital community, which prompted instructors to narrative, apart from specific experiences related to discuss community membership and managing access ethical challenges of digital citizenship. to content. Visual rhetoric was crucial to the aspect of With regard to integration of the curriculum, the digital literacy in the capstone. Here, the ability to students did a thorough job of transferring their establish a guiding idea that unifies the ePortfolio undergraduate experiences into quality ePortfolios and visually and conceptually will be an important concept demonstrating concrete learning in the realm of the to develop further. Moving forward, there is a need for general education curriculum. Along these lines, direct engagement with topics of digital literacy, digital reflective ePortfolios lend themselves to such curricular citizenship, and ethical challenges in a digital developments in that they are open to metaphorical environment. What was thought would be an implicit conceptualization, which allows students to build outcome will need to be made explicit in learning connections and engage in high-order processes of modules or modifications to ePortfolio assignments. representation. Rather than optional supplementary material, “beyond the classroom” experiences should be Recommendations and Future Directions required as an integral component of the capstone ePortfolio. Coursework samples and experiences Overall, the pilot was a success in generating beyond the classroom can be accompanied by a brief feedback on how the capstone might provide the written introduction from the student that frames the institution with an understanding of how students can significance of the project, assignment, or experience. present knowledge, skills, and abilities. The capstone Even a simple criteria statement (why this sample was pilot provided evidence about how well the syllabus chosen) would be helpful to evaluators, as well as an was developed to guide the instructors in facilitating the important preliminary step towards the final reflective course, in addition to the logistics of administering the essays. Brief (i.e., 50 to 100 word), required capstone as an online course. explanations of each artifact promote synthesis and First, not all students have high autonomous access cohesion of the artifacts. One challenge with integration to information communications technology (Robinson, of curriculum in this pilot was found in the assessment 2009). Students’ ability to present higher order thinking through the rubric. In teaching the full capstone, stating skills through a technology such as an ePortfolio more clearly the parameters the reflective essay, which requires a technology skill-base. The instructors built in should draw connections across academic disciplines peer support groups and instruction, which should be and connect relevant experience with academic maintained, but 1:1 instruction and the platform knowledge, will be highly important. The essays should learning curve were time consuming and, early on in demonstrate application of skills to solving complex the pilot, were found to detract from the main learning problems, if the current rubric is to be an accurate outcomes of the course. In the actual implementation, reflection of outcomes. More explicit writing these problems may be mitigated by the fact that instruction will need to be developed for each students will be exposed to the platform as early as assignment, detailing style, tone, and rhetorical freshman year (with the exception of transfer students). conventions that will clearly locate students’ ability to However, support documentation will need to be more synthesize material. We currently cannot validate, for robust and may include video tutorials and alternative example, areas in which students have made active learning modes. Morreale, Van Zile-Tamsen, Emerson, Herzog Using a Capstone ePortfolio 22 Second, the rubrics will need to be adapted to be Buzzetto-More, N. A. (2013). Models to inform capstone more responsive to assessing critical thinking, program development. Issues in Informing Science & reflection, and integration of curriculum. The Information Technology, 10, 81-93. Retrieved from instructors struggled with using the rubrics, as currently http://iisit.org/Vol10/IISITv10p081- written, to assess these outcomes. Further refinement 093Buzzetto0270.pdf will be needed, as well as calibration with additional Cambridge, D. R. (2010). E-portfolios for lifelong instructors and teaching assistants. learning and assessment. San Francisco, CA: Third, additional assignments will be included in Jossey-Bass. the full semester capstone, further complicating the Cambridge, D., Cambridge, B. L., & Yancey, K. B. syllabus, measures of student learning, and (2009). Electronic portfolios 2.0: Emergent assessments. A full semester and the chance to provide research on implementation and impact. Sterling, multiple opportunities for students to present their work VA: Stylus. in ePortfolios will likely increase their abilities to Carini, R. M., Kuh, G. D., & Klein, S. P. (2006). Student demonstrate higher order thinking skills. However, engagement and student learning: Testing the while there will be more time to execute the activities linkages. Research in Higher Education, 47(1), 1-32. of the ePortfolio, caution will be needed for the Eccles, J. S., & Wigfield, A. (2002). Motivational instructors and students to have the required support to Beliefs, Values, and Goals. Annual Review of utilize the ePortfolio tool to represent these outcomes to Psychology, 53, 109-132. doi:10.1007/s11162- a greater degree and complexity. 005-8150-9 Eynon, B., & Gambino, L. M. (2016). Professional Conclusion development for high-impact ePortfolio practice. Peer Review, 18(3), 4-8. Retrieved from Although there is room for improvement in the https://www.aacu.org/peerreview/2016/summer/ execution of the curriculum design, both instructors and Eynon students found the capstone ePortfolio experience to be Flavell, J. H. (1979). Metacognition and cognitive fulfilling in meeting the goals intended of the pilot. monitoring: A new area of cognitive Multi-method assessment shows that a capstone developmental inquiry. American Psychologist, 34, ePortfolio course experience can be valuable in giving 906-911. doi:10.1037/0003-066X.34.10.906 students a chance to integrate their general education Gardner, J. N., Van Der Veer, G. (1998). The senior curriculum and demonstrate their higher-order thinking year experience: Facilitating reflection, skills in a digital space. Capstone ePortfolio integration, closure and transition. San Francisco, experiences offer excellent opportunities for students to CA: Jossey-Bass. reflect on their undergraduate careers as well as for Hauhart, R. C., & Grahe, J. E. (2015). Designing and institutions to assess the knowledge and skills that teaching undergraduate capstone courses. San students have gained throughout the curriculum. A Francisco, CA: Jossey-Bass. well-developed capstone curriculum design and rubrics Kinzie, J. (2013). Taking stock of capstones and integrative help guide these opportunities. learning. Peer Review, 15(4), 27-30. Retrieved from https://www.aacu.org/peerreview/2013/fall/kinzie References Kruger, E. J., Holtzman, D. M., & Dagavarian, D. A. (2013). Comprehensive education portfolio with a Association of American Colleges and Universities career focus. Journal of Continuing Higher Education, (AAC&U). (2011). The LEAP vision for learning 61(1), 46-53. doi:10.1080/07377363.2013.759494 outcomes, practices, impact, and employer’s views. Kuh, G. D. (2008). High impact educational practices: Washington, DC: Author. What they are, who has access to them, and why Berheide, C. W. (2007). Doing less work, collecting they matter. Washington, DC: AAC&U. Retrieved better data: Using capstone courses to assess from http://provost.tufts.edu/celt/files/High- learning. Peer Review, 9(2), 27-30. Retrieved from Impact-Ed-Practices1.pdf https://www.aacu.org/publications- Kuh, G. D., Cruce, T. M., Shoup, R., Kinzie, J., & research/periodicals/doing-less-work-collecting- Gonyea, R. M. (2008). Unmasking the effects of better-data-using-capstone-courses student engagement on first-year college grades Buyarski, C. A., Aaron, R. W., Hansen, M. J., and persistence. Journal of Higher Education, Hollingsworth, C. D., Johnson, C. A., Kahn, S., . . . 79(5), 540-563. Powell, A. A. (2015). Purpose and pedagogy: A Livingston, J. A. (2003). Metacognition: An overview. conceptual model for an ePortfolio. Theory into Retrieved from ERIC database. (ED474273) Practice, 54(4), 283-291. McGill, P. T. (2012). Understanding the capstone doi:10.1080/00405841.2015.1076691 experience through the voices of students. Journal