Student Attitudes toward Technology-Mediated Advising Systems Student Attitudes toward Technology-Mediated Advising Systems Hoori Santikian Kalamkarian & Melinda Mechur Karp Columbia University Abstract The literature on broad-access colleges suggests that low persistence and completion rates may be improved through better advising that employs a teaching-as-advising approach. While resource constraints have traditionally limited the ability of colleges to reform advising practices, technological advances have made it possible to implement technology-based advising tools, some of which can replace face-to-face services. Using focus group interview data from 69 students at six colleges, this study investigates students’ attitudes toward technology-mediated advising. More specifically, we seek to understand how students’ perceptions and experiences vary across different advising functions. We find that students are open to using technology for more formulaic tasks, such as course registration, but prefer in-person support for more complex tasks, such as planning courses for multiple semesters and refining their academic and career goals. Keywords: technology, advising, iPASS, student supports, higher education Kalamkarian, Hoori Santikian & Karp, Melinda Mechur (2017) Student attitudes toward technology-mediated advising systems. Online Learning 21 (2) doi: 10.24059/olj.v21i2.918 Introduction Amidst sweeping advances in information technology systems and increased higher education costs, a broad debate has emerged about the role of technology in higher education services (Blumenstyk, 2015; Tait, 2000). For some, the use of technology is framed as a “disruptive innovation” that has the capacity to displace many traditional business-as-usual processes in higher education (Blumenstyk, 2015). Proponents argue that replacing face-to-face services will bring cost savings and ensure that higher education continues to attract students who are increasingly comfortable and adept with technology. Critics maintain that using technology to displace processes that traditionally include substantial personal interaction with Student Attitudes toward Technology-Mediated Advising Systems students, such as instruction and advising, will dilute the student experience (Blumenstyk, 2015; Gaines, 2014; Tait, 2000). Recent focus on the role of technology in advising, in particular, mirrors the broader debate. While some argue that the increased use of technology will improve the information that students receive and strengthen student satisfaction (see, for example, Junco, 2010), others caution that advising is and ought to be dependent on human interaction (see, for example, Noonan & Stapley, 2015). These divergent perspectives, however, do not sufficiently capture how technology may be used in higher education. Rather than fully replacing current systems, it is more likely that technology will increasingly be used in ways that coexist with traditional face- to-face delivery of services. In the domains of both instruction and advising, some thus argue that technology can be strategically integrated with face-to-face services in a way that optimizes the attributes of both mediums (Gaines, 2014). Put differently, technology may have a student support role to play alongside in-person interaction. It may be useful in some contexts for some functions but not others. More research is needed on this topic, particularly in advising. Within the domain of technology-mediated teaching and learning, emerging research has begun to address the effects of increased use of technology. Recent empirical studies of online instruction have found that course grade and persistence vary across online courses depending on the subject area and the complexity of the course content; for example, studies have found that students struggle to persist in online versions of applied professions courses (Bambara, Harbour, Davies, & Athey, S. 2009; Xu & Jaggars, 2013). These studies contribute to a more nuanced understanding of the way technology may be used in higher education. Similar research is needed to better understand the potentially differentiated role that technology might play in technology-mediated advising services. It is important to consider the student perspective in such research. Proponents and critics of technology-mediated systems sometimes make assumptions about which platform for service delivery—technology-based or face-to-face—students prefer. Yet, empirical findings about the student perspective, particularly in the context of advising, remain largely absent (Gaines, 2014). Additional exploration of student attitudes may reveal important variation in student preferences regarding mode of delivery. Using focus group interview data from 69 students at six colleges, the current study investigates students’ attitudes toward technology-mediated advising. More specifically, we seek to understand how students’ perceptions and experiences vary across different advising functions. We find that students are open to using technology for more formulaic advising services, such as course registration, but prefer in-person support for more complex undertakings, such as planning courses for multiple semesters and refining their academic and career goals. This paper begins with a literature review of technology in higher education. Next, we present two conceptual frameworks that together guide our analysis of student attitudes toward technology for different types of advising and support functions. We then outline our findings and conclude with a discussion of implications for practice and for further research. Student Attitudes toward Technology-Mediated Advising Systems Background Advising and Counseling in Broad-Access Colleges National college persistence and completion rates remain alarmingly low, particularly among low-income students and students of color at broad-access colleges (Bound, Lovenheim, & Turner, 2012; Goodman, Hurwitz, & Smith, 2015). We define broad-access colleges as public or not-for-profit private institutions that accept 80 percent or more of applicants, including community colleges (Doyle, 2010). For example, only about 30 percent of students who attend public two-year colleges full-time complete a credential (either an associate degree or a certificate) within three years of initial enrollment. Degree attainment is even lower among socioeconomically disadvantaged students (Aud et al., 2012; Karp, 2013). The literature on broad-access colleges suggests that persistence and completion rates may be improved through institutional supports designed to foster student success, including advising and counseling. Theory suggests, and empirical research has found, that students are more likely to persist in college and complete a degree if they have clear goals, understand college processes, and engage with staff or other students on campus. In ideal circumstances, advisors can facilitate these conditions through personalized, holistic, and sustained guidance for students. Moreover, this ideal is achieved if advisors follow a “developmental” approach, meaning that they help students develop the self-awareness and problem-solving skills they need in order to clarify and fully commit to their aspirations (Karp, 2011). Put differently, advising that adheres to a developmental approach extends beyond information dissemination; instead, developmental advising supports student persistence by helping students develop the capacity to evaluate and make academic and vocational choices (Lowenstein, 2005). Structural limitations, however, make it difficult for advising services at broad-access colleges to achieve this ideal. Advising and counseling centers at these institutions are severely underfunded and, as a result, understaffed. Advising loads typically range from 800 to 1,200 students for every advisor (Jaggars & Fletcher, 2014; Karp, 2013). Moreover, broad-access colleges enroll a substantially larger proportion of socioeconomically and academically disadvantaged students compared with other types of institutions. These students need additional guidance and academic support, which further drains already limited staffing and financial resources (Karp, 2013; Karp, O’Gara, and Hughes, 2008). Given these constraints, colleges often struggle to offer the comprehensive and integrated support services necessary to facilitate personalized, consistent advising. While they do often target incoming students, institutions typically do not have the capacity to require advising of all students and instead rely on students to self-advocate and to take initiative to make use of advising services when they feel it is warranted. In addition, colleges do not generally assign advisors to specific students (Karp, 2013; Karp et al., 2008). Advising sessions are also often limited in time, especially during peak advising periods such as registration period. Moreover, colleges commonly seek economies of scale by splitting apart advising functions, creating separate “services” or offices rather than providing holistic student support. For example, academic advising usually functions independently from financial aid advising, career counseling, personal counseling, and other areas of student support. Students often have to piece together information from different sources, and, even for academic advising, they are not necessarily able to meet with the same advisor in subsequent sessions. Student Attitudes toward Technology-Mediated Advising Systems Under this structure, the advising experience often falls short of expectations, and the ideal advisor–advisee relationship is not established or maintained. Absent sustained interaction with the same student and the opportunity to engage with the student across advising functions, advisors are not able to offer personalized and holistic support. Instead, advising sessions often entail helping students more or less exclusively with the administrative function of registering for courses (Karp, 2013). As “registration clerks,” however, advisors are not able to extend their engagement with students into the realm of developmental support. This support may be crucial in helping students learn to make wise, well-founded academic and career choices. The literature on student perceptions of support services further underscores the limitations of typical advising practices (Grubb, 2006; Karp, 2011; Karp et al., 2008; Low, 2000). Survey and interview studies show that students generally rate their experiences with support services as negative to adequate (Jaggars & Fletcher, 2014; Low, 2000). These studies suggest that students are dissatisfied with advising services because the information they receive during advising sessions falls short of the guidance they expect to get from advisors. Most students note that the advisors they work with primarily provide basic information about registering for courses and fulfilling graduation requirements, a description that aligns with the registration clerk analogy. While students note that it is important to receive this information from advisors, most students also expect more in-depth and personalized advising that includes career planning. Moreover, studies have found that students are, at best, only moderately satisfied with even the basic advising they do experience; several students in these studies reported receiving inaccurate information from advisors about registration and academic requirements (Karp et al., 2008; Low, 2000; Smith & Allen, 2006). Technology-Mediated Advising To reform support services in the face of persistent resource constraints, several college administrators, policymakers, and education foundations are strongly advocating for the increased use of technology (Hornak, Akweks, & Jeffs, 2010; Leonard, 2008; Moneta, 2005; Yanosky, 2014). For example, the 2014 White House summit on higher education focused on data- and technology-driven reforms to improve degree and certificate completion. During his remarks at the summit, President Obama advocated using technology to identify and support students who may be deviating from their path toward a degree (Felton, 2014). Recently, technologies have emerged that offer robust information delivery and data analysis capabilities for student support services. These advising systems, sometimes referred to as Integrated Planning and Advising for Student Success (iPASS), seek to improve degree or certificate attainment by facilitating both intra-institutional coordination of student supports and data-driven academic decision-making for advisors and students (Yanosky, 2014). The growing suite of products in this space offers features that include: • automated communication (institution-wide or with a subset of students), • an institution-wide platform for identifying academically at- risk students, Student Attitudes toward Technology-Mediated Advising Systems • interactive multi-semester course planning modules customizable for each student, • shared staff access to notes from advising sessions (in accordance with privacy regulations), and • integration of existing technologies, including data and course management systems. This list suggests that iPASS products have the potential to make it easier for colleges and universities to coordinate across stakeholders within the institution, monitor student progress, and help students make wise choices (Yanosky, 2014). This paper refers to student support services that are offered through technology, including software or web-based hosts, as technology-mediated advising systems. This term is also used interchangeably with iPASS. The Impact of Higher Education Technology on Student Outcomes If and how iPASS technologies improve support services and, by extension, student outcomes, however, remain open empirical questions. Existing discussions of technology- mediated advising primarily take place through commentaries, such as online blogs and newspaper or journal editorials, that are often not based on empirical findings (Hornak et al., 2010). A handful of colleges that have implemented one or more of these emerging technologies have conducted internal evaluations. These evaluations, however, have been limited both in terms of scope and methodology. Generally, the accompanying evaluation reports offer descriptions of specific technologies and share trends in student outcomes that may be related to, but are not causally linked to, these technologies (Oblinger, 2012; Phillips, 2014). Among these internal evaluations, Purdue University’s study of the “Signals” early alert system offers the most methodologically robust framework. To evaluate this homegrown product, the university identified an experimental group of students who would receive alerts from Signals and compared their final course grades with those of students in a control group. Purdue University found that there was a lower proportion of Ds and Fs among the experimental cohort (Arnold & Pistilli, 2010). Yet this evaluation is limited to the context of a specific course and does not extend to longer-term student outcomes. One of the few studies of technology-mediated advising in the academic literature assessed the effect of student coaching on academic performance (Bettinger & Baker, 2014). Coaches used a variety of media, including social network and text messaging as well as telephone calls, to communicate with treatment students over the course of two semesters. Coaches offered encouragement and individualized advice about managing academic and personal challenges while in college; comparison students did not receive individualized coaching but did have access to typical student support services at their college. Coaching positively impacted retention, highlighting the powerful influence that individualized, long-term support can have on student success. However, the study, by design, did not address if and how the technology-mediated structure of the coaching influenced the observed effect on student outcomes (either positively or negatively); the observed positive impacts could have been due to factors other than technology, such as the sustained relationship between student and coach. Additional research that specifically considers the technology-mediated delivery of advising would help to clarify technology’s role in this process. Student Attitudes toward Technology-Mediated Advising Systems Moreover, to better understand how technology-mediated advising shapes student outcomes, it is important to first understand what students think of engaging with advising resources through technology. Technology-mediated systems can only be effective if students are willing to use the systems; what students think and feel about them should therefore be considered when designing and implementing these systems. Moreover, students may have the most intimate knowledge of the support they need and how they would like to receive such support. As previously stated, technology advocates argue that because technology is often integrated into the fabric of everyday life among younger generations (Hornak et al., 2010; Joslin, 2009; Moneta, 2005; Stephens, 2007), students expect support to be delivered through technology tools. In these discussions, students are commonly referred to as “digital natives” (Leonard, 2008, p. 293), “the net generation” (Gaines, 2014, p. 44), and “the iGeneration” (Hornak et al., 2010), underscoring their high propensity to use technology. These notions of student comfort with technology, however, are largely assumed to apply to the advising context and are not sufficiently informed by speaking directly with students. Findings from the few studies that have asked students about technology-mediated advising suggest that student attitudes may be more nuanced than technology advocates assume. In a survey of 167 undergraduate students, Gaines (2014) found that nearly half (47 percent) preferred to meet face-to-face with an advisor, while another one-third preferred to communicate with advisors through email. Similarly, in a study assessing the intake and orientation process at one community college, Jaggars and Fletcher (2014) found mixed responses from students regarding technology-mediated advising. While students preferred to meet face-to-face with advisors, they also indicated that they would like more access to information online. While their study highlights the complexity of student preferences, it is limited in scope to the orientation and intake components of advising. Additional research that further unpacks student attitudes toward technology across the spectrum of advising services, from course registration to education planning and career exploration, is needed. While the literature on student attitudes toward technology specifically in the context of advising is limited, select studies have investigated what students think of online courses. The existing literature suggests that student preferences regarding online courses vary depending on the complexity of the subject. In one study (Jaggars, 2014), students who enrolled in both online and face-to-face courses expressed a preference for face-to-face instruction when they perceived the subject to be “difficult”; in the study, students generally referred to math and science courses as difficult. Students also felt that certain subjects were better suited for an online format compared with others. For example, students felt that the online format was not suitable for studying foreign languages because it would not allow students to practice speaking the language to each other. The variation in student attitudes toward online classes across subject areas suggests that student attitudes toward technology-mediated advising may also vary depending on the nature and complexity of the advising function. Conceptual Framework To examine how students would like to engage with advising and student supports, this study relies on two theoretical frameworks. First, we take a particular point of view on the optimal approach to advising in broad-access colleges, that of “advising as a form of teaching” (Appleby, 2008). Second, we use Tait’s (2000) framework to unpack the different types of Student Attitudes toward Technology-Mediated Advising Systems advising and student support functions in which advisors and technology tools engage. The first framework enables us to ground our analyses in a specific theoretical point of view that underscores the foundational nature of advising in encouraging student success; the second enables us to explore which pieces of advising-as-teaching may be best suited to various methods of provision. Advising as a form of teaching is the dominant view of advising among education theorists, and it aligns with best practice guidelines put forth by the National Academic Advising Association, the leading professional association for college advisors (Appleby, 2008). The advising-as-teaching approach defines academic advising as a relationship between an advisor and an advisee that parallels the relationship between an instructor and a student. Both effective teaching and effective advising entail not only disseminating information, but also cultivating students’ higher-order reasoning skills. Exemplary instructors teach students analytic skills that they can then apply across subjects and contexts (Appleby, 2008; Lowenstein, 2005; Moore, 1993). Exemplary advisors guide students to develop the problem-solving and higher-order cognitive skills they need to successfully navigate their postsecondary trajectory (Appleby, 2008; Lowenstein, 2005). Ultimately, both effective instructors and effective advisors help students make meaning of their educational experiences. Instructors guide students to see connections between assignments within a class and thereby construct a cohesive understanding of the material. Similarly, advisors help students clarify the logic connecting the disparate courses that together comprise their overall college curriculum (Lowenstein, 2005). To facilitate the development of higher-order skills, advising sessions should mirror the “active learning” classroom model. According to this model, students develop analytic skills by engaging directly with the material instead of being passive recipients of information. Therefore, advisors ideally guide students through an interactive exchange to explore and evaluate alternative pathways and clarify education and career goals (Appleby, 2008; Lowenstein, 2005; Moore, 1993). Feedback from the advisor prompts the student to reflect, evaluate, and ultimately, arrive at a decision. By facilitating students’ active participation in decisions about their education, advisors help students develop the skills they need to make subsequent decisions about their education (Appleby, 2008; Lowenstein, 2005). Theoretical discussions assume that the advising occurs primarily in a face-to-face context (Appleby, 2008; Lowenstein, 2005). Though advising-as-teaching is the preferred approach of professional advisors, the realities of life in open-access colleges described earlier make this approach challenging to implement. To facilitate an interactive relationship with students, advisors need to invest substantial time and resources in working with each student. For example, Appleby (2008) offers a guide for designing an advising curriculum that calls for scheduled face-to-face advising sessions with a “question and answer” format; in addition, advisors are encouraged to ask questions to prompt less expressive students. Financially constrained and understaffed colleges struggle to allocate the resources to structure advising sessions in this staff-intensive format. Technology products may help institutions move closer to an advising-as-teaching approach by reducing the burden on advising services. For example, products that allow faculty to flag academically at-risk students allow advisors to more quickly and effectively identify students who need additional support. Advising-as-teaching emphasizes the need to teach higher-order thinking skills in navigating postsecondary education. However, not all advising functions necessarily require Student Attitudes toward Technology-Mediated Advising Systems extensive higher-order thinking. Some advising activities may require less intensive instruction and therefore may rely less heavily on the advising-as-teaching framework. To differentiate between advising functions, this study applies Tait’s (2000) conceptual framework for types of student support. This framework recognizes higher-order support functions and thus aligns well with the advising-as-teaching principle. However, this framework also differentiates among the various types of activities in which advisors and students engage and is therefore more representative of the realities of day-to-day advising activities. Moreover, this framework is widely used in the design of support services in the context of online and distance education, which parallels the technology-mediated context of this study. Tait (2000) identifies three categories of student supports in online courses: systemic, affective, and cognitive. Systemic support involves helping students navigate administrative tasks; in advising, this type of support includes clerical functions such as helping students register for courses. Affective support, both broadly and in the context of advising, involves strengthening students’ confidence and sense of self-efficacy by creating an encouraging environment. Finally, cognitive support includes facilitating learning through supplemental instruction and feedback. In advising, cognitive support involves helping students develop the skills to make decisions about their postsecondary pathway and achieve clarity in their educational and career goals. Together, these two frameworks allow us to investigate when and why students prefer technology-mediated support services. Data and Method This study addresses the following research questions: 1. What are students’ attitudes toward technology-mediated advising services? 2. How do students’ attitudes toward technology-mediated advising services vary across different types of advising functions? 3. What are the implications of students’ attitudes toward technology- mediated advising for the organization and promotion of these systems? The data presented in this study come from student focus group interviews conducted at six colleges that vary in urbanicity and sector type. Our sample includes two small non-urban community colleges, two urban community colleges, one mid-sized urban state college, and one rural mid-sized state college. The mid-sized urban college is also categorized as an historically black college or university, or HBCU. An HBCU is a college or university founded before 1964 for the primary purpose of serving African American students. There are currently 105 certified HBCUs in the country (U.S. Department of Education, n.d.). Sites were selected from volunteer colleges interested in deploying iPASS technologies. This study is part of a larger analysis of iPASS implementation at participating colleges. To collect data, the research team conducted three-day site visits to each college during fall 2013. Over the course of all six visits, we conducted 18 focus groups with 69 students. Student Attitudes toward Technology-Mediated Advising Systems Students were invited to participate in focus groups through a mass email sent by each institution’s office for student services. All students who were interested in participating were asked to contact the research team directly by phone or email. Students were selected on a first- come, first-serve basis. All participating students received a $25 retail gift card. In addition to the focus group interview, all students who participated completed a brief background questionnaire; we use data from this questionnaire to calculate descriptive statistics. Our sample included 25 male and 44 female students. While students in our sample ranged from 16 to over 60 years old, 75 percent were 18–25 years old. A large majority (81 percent) of the students in our sample attended college full-time; the remaining 19 percent of students were enrolled part-time at the time of data collection. Our sample included both new and more advanced students; 43 percent of students were in their first semester, while 57 percent were in their second or subsequent semesters. It is important to note that the survey data are not linked to focus groups; as a result, we cannot disaggregate the survey responses based on insights shared during focus group interviews. Table 1 Demographics of Sample Compared with National Statistics Our Sample National Statistics Characteristics (n = 69) (IPEDS 2012) Female 64% 57% Traditional-aged 75% 61% (18–25) Full-time 81% 39% The demographic make-up of our sample aligns moderately well with national demographic characteristics for community college students. According to IPEDS (U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, 2012), a majority of community college students nationally are women. However, while a majority of students in our sample attended college full-time at the time of data collection, nationally, slightly less than 40 percent of community college students attend full- time. The overrepresentation of full-time students in our sample reflects the challenges with reaching and engaging part-time students in campus and related activities. It is also important to note that, according the existing research, students often switch between part-time and full-time status throughout the course of their tenure at community colleges (Crosta, 2014); therefore, the limitations posed by the overrepresentation of full-time students in our study are somewhat mitigated. The student focus groups were designed to capture how students approached academic decisions and the extent to which advising resources contributed to their decision-making process. During semi-structured interviews students were asked to describe their experiences in each of five areas: course selection, course registration, major selection, multi-semester planning, and academic interventions from the college. Once students had described their general approach Student Attitudes toward Technology-Mediated Advising Systems in each of these areas, we probed about their use of various technological tools. For example, students who indicated that they planned for multiple semesters were asked to describe how they figured out which courses they needed to take and to what extent they used online or other electronic tools. We also probed for opinions about technology-mediated advising as the conversation warranted. Typically, a few students from the group described their experiences at length, and others noted if and how their experiences compared with those described by their peers. Focus group transcripts were coded and analyzed using Atlas.ti software. A preliminary code list was developed based on the research questions guiding the overarching study and initial impressions about possible themes. We organized codes into four overarching categories: context, service practices, service process, and service structure. Context codes captured student and institutional needs. The service practice category included 15 codes, one for each identifiable service function such as course selection, registration, and major selection. Service process codes specified whether students completed the task independently or with assistance from institutional support services, including iPASS products. Finally, the service structure codes captured the organizational setup of support services. Four rounds of test coding were conducted to refine the preliminary codebook. Inter-rater reliability was established through the test coding process and ongoing coding reviews conducted by the project lead for every fifth transcript; moreover, coders discussed codes for particular passages during weekly coding meetings. After all of the documents were coded, we used Atlas.ti tools to identify themes relevant to the focus of this paper. This process included running queries that identified concurrence of codes, in particular across the service practice and service process categories. For example, to understand when students used technology, including iPASS products, to plan their coursework, we queried the concurrence of “service practice—multi-semester planning” and “service process—technology-system-wide.” We read query outputs thematically to identify emerging themes and organized themes in an Excel spreadsheet, with each row representing a different theme and each column displaying a quote indicative of the corresponding theme. We used our conceptual framework to guide our analysis; where appropriate, we reorganized our emerging ideas into broader themes that mapped student preferences to the three categories of advising (systemic, affective, and cognitive) identified in our framework. For example, under the broad category of “systemic support” we organized excerpts by three themes: support for basic procedural questions, feedback on course logistics, and confirmation to ensure accurate course selection and registration. It is important to note that to maintain the anonymity of the students who participated in our study, we did not record any names. Our transcriptions thus do not specify which student made which comment. This presents methodological limitations. We are not able to trace the comments of specific students over the course of the interview. We are also not able to count the number of students who make a specific point across our sample. Findings Our data address both students’ attitudes toward advising generally and their perspectives on in-person and technology-based delivery of services. Overall, we find that students in our study sought an interactive relationship with advisors that is characteristic of the advising-as- teaching approach; in particular, students wanted to learn from advisors how to approach more