Journal of Technology Education Vol. 29 No. 1, Fall 2017 The MESA Study Cameron D. Denson Abstract This article examines the Mathematics, Engineering, Science Achievement (MESA) program and investigates its impact on underrepresented student populations. MESA was started in California during the 1970s to provide pathways to science, technology, engineering, and mathematics careers for underrepresented students and represents an exemplar model of informal learning environments. Using a mixed-method research design of investigation, this exploratory study looks at the relationship between MESA activities and underrepresented students’ self-efficacy, interests, and perceptions related to engineering. Evidences for this study includes data from focus-group interviews conducted and results from quantitative data collected using the Engineering, Self-Efficacy, Interests, and Perceptions Survey (ESIPS) instrument. Results from this study suggest that participation in MESA’s activities has a positive influence on underrepresented students’ self-efficacy, interests, and perceptions related to engineering. Keywords: Informal learning, underrepresented student populations, mixed methods research Broadening the participation of underrepresented populations in the science, technology, engineering, and mathematics (STEM) fields is a matter of national security and has become an emphasis for national policy (Strayhorn, 2015). Yet, recent studies have provided evidence that efforts to address these shortages in STEM areas have fallen short. In describing patterns of enrollment in STEM majors, studies revealed that “less than 15% of undergraduate degrees in engineering, math, and physical science were earned by African American, Latina/o, or Native American . . . students (NSF, 2013)” (MacPhee, Farro, & Canetto, 2013, p. 348). A recent report revealed a troubling trend for underrepresented student populations entering into STEM majors. Of all the bachelor degrees awarded in 2015, Black students represented only 3% of this population, Hispanic students a slightly better 8%, and female students only 19% of all engineering degrees awarded, numbers that are significantly lower than their representation in the general population (National Science Foundation, National Center for Science and Engineering Statistics, 2015). Continuing efforts have tried to address the proportion of participants in engineering who are women and underrepresented minorities, but the demographics of engineering enrollments continue to fall significantly short of the goals of reflecting the demographics of the overall population (Watson & Froyd, 2007). -66- Journal of Technology Education Vol. 29 No. 1, Fall 2017 The result is a STEM field that remains overwhelmingly White, male, and able- bodied, leaving the available pool of talented women, minorities, and persons with disabilities significantly underrepresented (May & Chubin, 2003). To meet this challenge, it is important to identify factors that may help encourage underrepresented student populations to choose careers in STEM fields. It can be argued that the lack of engineering understanding and a loss of interest in science and mathematics is contributing to the lack of underrepresented students pursuing engineering careers (Jeffers, Safferman, & Safferman, 2004). To effectively address this problem, educators have sought to create new and innovative pathways for attracting a talent pool to STEM professions that encompasses the diversity evident in the nation’s general population (Chubin, May, & Babco, 2005). The National Academy of Engineering’s Committee on K–12 Engineering Education released a report that detailed the status of engineering in K–12 education (Katehi, Pearson, & Feder, 2009). In this report, the committee stressed the importance of developing curricula with features that appeal to students from underrepresented groups (Katehi et al., 2009). Scientists, engineers, and scholars should not leave the job of recruiting underrepresented populations to STEM careers solely to K–12 teachers of math and science education (Jeffers et al., 2004). Studies show that formal learning environments have traditionally struggled to effectively introduce STEM content and STEM professions to underserved student populations (Denson, Austin, & Hailey, 2012). Currently, there is a lack of empirical research on the efficacy of intervention programs to influence underrepresented students (Dyer-Barr, 2014). It is important that comprehensive research studies are employed to help illuminate the practices that are particularly effective in recruiting underrepresented students to STEM careers. One way to address the dearth of literature on best practices for recruiting underrepresented students to STEM careers is to investigate the practices of informal learning environments, particularly those that have been effective in recruiting underrepresented students to STEM careers. Informal Learning Environments Informal learning environments may provide the milieu needed for introducing STEM content to all students, but even more importantly, they may provide a pathway to STEM careers for underrepresented students. It is estimated that students spend 86.7% of their time outside of a classroom (Gerber, Cavallo, & Edmund, 2001). This helps illustrate the importance of informal learning environments and the opportunities that they may provide for the teaching and learning of STEM content. Martin (2004) notes that informal learning environments have been an integral part of education for years and will be critical for transforming the teaching of STEM content in the 21st century. Although the merits of informal learning environments are duly noted, research -67- Journal of Technology Education Vol. 29 No. 1, Fall 2017 in this area is sparse and undecided on how these experiences benefit students (Gerber et al., 2001). Beyond anecdotal reporting on informal learning environment experiences, there is little research detailing specific activities and their effect on students. This highlights the need to investigate informal learning environments that effectively teach STEM-based concepts to students. Although there are aspects of the program that are conducted during school hours, MESA formally functions as an afterschool program complementing the work of formal STEM curricula. Inferential studies into the ways that informal learning environments are able to impact underrepresented student populations are of particular importance. The results of investigations that explicate how successful informal learning environments impact underrepresented students will provide insight into how the United States can attract diverse populations to STEM fields. Chubin, May, and Babco (2005) produced a review of engineering-based informal learning environments and concluded that effective engineering-based informal learning environment “must (1) promote awareness of the engineering profession, (2) provide academic enrichment, (3) have trained and competent instructors, and (4) be supported by the educational system of the student participants” (p. 79). Categorically, informal learning environments fall into three settings: (1) “everyday experiences,” (2) “designed settings,” and (3) “programmed settings” (Kotys-Schwartz, Besterfield-Sacre, & Shuman, 2011, p. 1). Program settings are characterized by “structures that emulate [or complement] formal school settings—planned curriculum, facilitators or mentors (taking a teaching role), and a group of students who continuously participate in the program [(National Academy of Sciences, 2009)]” (Kotys- Schwartz et al., 2011, p. 2). The learning environment featured in this study, the Mathematics, Engineering, Science Achievement (MESA) program has been identified as an effective informal learning environment and is categorically identified as a programmed setting (Mathematics, Engineering, Science Achievement [MESA], 2017a). Research has shown that students who participate in the MESA program “outperform California public high school students overall in the following categories: completion of advance mathematics and physics courses, grades and performance on college entrance exams [(Building Science and Engineering Talent, 2004)]” (Kotys-Schwartz et al., 2011, p. 2–3). Due to MESA’s success as an informal learning environment and its unmatched ability to recruit and retain underrepresented student populations to STEM careers (MESA, 2017a), researchers for this study were interested in examining the aspects of MESA that appealed to their underrepresented student populations. This article reports on the results of an investigation into the impact of the MESA program on underrepresented student populations. Using a sequential, exploratory, mixed-method research design, this article adds to the literature focused on underrepresented student populations and informal learning -68- Journal of Technology Education Vol. 29 No. 1, Fall 2017 environments. This article will first provide the reader with a brief history of the MESA organization followed by the research design framing this study. The article will follow with results from focus-group interviews conducted with MESA participants, which provided eight intriguing themes from the MESA organization and helped informed the design of the Engineering, Self-Efficacy, Interests, and Perceptions Survey (ESIPS) instrument used in this study. Finally, this article will provide results and conclusions from quantitative data collected from over 400 student participants using the ESIPS instrument. The MESA Organization The first MESA program was founded in 1970 at Oakland Technical High School in Oakland, California with a membership of 25 students. MESA’s goal was “to develop academic and leadership skills, raise educational expectations, and instill confidence in California’s students” from groups that were “historically underrepresented in engineering, physical science, or other math- based fields in order to increase the number of African American, Latino American and American Indian graduates from a four-year university” (MESA, 2017, para. 1). The MESA effort was supported by the California Public School System, the state Community College System, and the California College System. “There may be other established programs, or programs under development, designed to increase Latino academic achievement in mathematics and science, but none has the longevity, organizational structure, network, and academic rigor as does MESA” (Haro, 2004, pp. 218–219). MESA has been able to achieve these goals despite declining federal and state support. MESA supports educationally disadvantaged students and minority students in middle schools and high schools by providing pathways to help them succeed in science, mathematics and engineering (Kane, Beals, Valeau, & Johnson, 2004). MESA’s goals are to: (1) “increase the number of engineers, scientists, mathematicians, and related professionals at technical and management levels, and (2) serve as a driving force in encouraging minorities and females in achieving success in these fields” (Maryland MESA, 2012). MESA programs are based on a common co-curricular academic enrichment model that includes “academic planning, community service, family involvement, academic enrichment, hands-on engineering activities, career advising, field trips, competitions and workshops”(MESA USA, 2011). MESA programs represent an innovative way of linking a co-curricular learning environment to mathematics, engineering, and science programs within the formal public-school setting to enhance the STEM education of students. Over the past 40 years, the California MESA program has become a model for MESA-USA, a partnership that now involves MESA programs from nine states that are joined together to support disadvantaged and underrepresented students to improve their academic achievement in math, science, and engineering. MESA-USA members are active in Arizona, California, Colorado, -69- Journal of Technology Education Vol. 29 No. 1, Fall 2017 Maryland, New Mexico, Oregon, Utah, Washington, and Pennsylvania. Additional information about the history and status of MESA are available on their website: https://mesa.ucop.edu/about-us/. MESA “has demonstrated through statistics from their California statewide office that MESA students outperform California public school students overall in the following categories: completion of advanced mathematics and physics courses, grades and performance on college entrance exams [(Building Science and Engineering Talent, 2004)]” (Kotys-Schwartz et al., 2011, pp. 2–3). A review of evaluation reports from after-school science, technology, engineering, and mathematics (STEM) programs, both co-curricular and extracurricular, by the Afterschool Alliance found “that attending high-quality STEM afterschool programs yields STEM specific benefits that can be organized under three broad categories: improved attitudes toward STEM fields and careers; increased STEM knowledge and s‐kills; and higher likelihood of graduation and pursuing a STEM career” (Afterschool Alliance, 2011, p. 2). Further evidence of the program’s impact is that California MESA received the Presidential Award for Excellence in Science, Mathematics, and Engineering Mentoring in 2000, an award administered by the National Science Foundation on behalf of the President. The success of individual MESA programs has been well documented, including a recent external evaluation conducted by John Hopkins University with the Maryland MESA program (Corcoran, Eisinger, Reilly, & Ross, 2014). This study looked at MESA’s influence on students interest, but it was limited to one state and included just 77 participants (Corcoran et al., 2014), making the results hard to generalize. There is still a need for empirical research that identifies “appropriate content for informal learning models [or environments] or . . . assess[es] the degree to which these informal experiences impact students” (Kotys-Schwartz et al., 2011, p. 1), particularly across programs in different states. In response to this need, this article will present qualitative and quantitative data to illustrate the ways in which MESA is able to influence students’ self-efficacy, interest, and perceptions of engineering. Furthermore, relationships among students’ interest, perceptions, and self-efficacy will be explored, and qualitative data will be presented on the benefits of MESA for underrepresented students. This study was designed to examine students’ participation and involvement in five activities that are common among MESA programs: field trips, guest lecturers, design competitions, hands-on activities, and career and academic advisement. -70- Journal of Technology Education Vol. 29 No. 1, Fall 2017 Research Design Methodology This study utilized a mixed-method research design. The purpose of this study was to examine the MESA program and understand features of the program that appeal to underrepresented groups. This work complements the work of Tierney and Farmer (2002), by identifying student-oriented activities within the MESA program that have an influence on underrepresented students’ engineering self-efficacy, interest in engineering and perceptions of engineering. In addition, focus-group interviews were conducted in an effort to unpack activity variables within the MESA organization and understand the benefits of the program for underrepresented students. The study was conducted in four MESA-USA states: California, Maryland, Washington, and Utah. This study used qualitative and quantitative measures to answer the research questions. The first research question was addressed in the qualitative portion of the study. 1. What are the benefits of participating in MESA for underrepresented student populations? The second, third, and fourth research questions were addressed in the quantitative portion of the study. 2. What influences do MESA activities have on students’ engineering self-efficacy? 3. What influences do MESA activities have on students’ interest in engineering? 4. How are the students’ perceptions of engineering influenced by their participation in 
MESA activities? This study examined student-oriented activities which can be categorized into five distinct groups: (a) field trips, (b) guest lecturers, (c) design competitions, (d) hands-on activities, and (e) student advisement. These five MESA activities represented the independent variables for this study. The dependent variables for this study included students’ self-efficacy, interest, and perceptions related to engineering. The study also examined the influence of the MESA program on outcome factors. Mixed-method research designs are particularly advantageous when seeking to confirm and cross-validate findings within a single study (Creswell, 2009). This study employed an exploratory design of investigation. Exploratory designs begin with a primary qualitative phase, then the findings are validated or otherwise informed by quantitative results. This approach is usually employed to develop a standardized (quantitative) instrument in a relatively unstudied area. The qualitative phase identifies important factors, while the quantitative phase applies them to a larger -71- Journal of Technology Education Vol. 29 No. 1, Fall 2017 and/or more diverse sample (Creswell and Piano Clark, 2007). (Borrego, Douglas, & Amenlink, 2009, p. 59) In this study, focus-group interviews were used to help identify important features of MESA, which were applied to a larger sample during the quantitative phase. In this study, the qualitative results helped identify features within the MESA program that appealed to underrepresented student populations. In addition, the focus-group results informed instrument development of the ESIPS instrument. Theoretical Framework The theoretical framework that guided this study was social cognitive theory (SCT), which holds that knowledge acquisition is directly related to observing others within their context of social interactions, experiences, and outside media influences (Bandura, 1988). This framework proposes a relationship between outcome expectations and other behavioral factors such as self-efficacy and interest. SCT is based upon the assumption that human ability is a dynamic attribute and that competence in complex tasks requires both well- developed skills and a strong sense of efficacy to deploy one’s resources effectively. Social cognitive career theory (SCCT) provides a base for exploring the interaction among personal, environmental, and behavioral influences in career development (Lent, Brown, & Hackett, 1994). This framework is appropriate for this study because of SCT’s emphasis on the role that self- efficacy, beliefs, outcome expectations, and goals play in career selection. Self-efficacy. The first construct to be explored in this study is self-efficacy, as defined by Bandura (1977, 1986), which refers to the beliefs about one’s ability to execute a given task or behavior in order to attain designated performance. Research has provided evidence that the lack of participation of minorities in STEM careers is due in part to low self-efficacy in science and mathematics. Self-efficacy has been found to be a powerful contributor to the decision to pursue a career in STEM and a major predictor of success in STEM courses (Zeldin, Britner, & Parajes, 2008). Although studies have examined self- efficacy as it related to STEM fields, few have focused specifically on engineering (Lent et al., 1994). However, there is evidence that self-efficacy regarding scientific–technical tasks is predictive of student interest (Brown, Lent, & Larkin, 1989) and academic performance (Hackett, Betz, Casas, & Rocha-Singh, 1992) in STEM fields. Bandura (2006) states that “there is no all- purpose measure of perceived self-efficacy” (p. 307). Sherer et al. (1982) noted that self-efficacy has been primarily thought of as a task-specific belief. Thus, in order to measure engineering self-efficacy, a scale must be created specifically related to the engineering domain. “Self-efficacy scales must be tailored to activity domains and assess the multifaceted ways in which efficacy beliefs operate within the selected activity domain” (Bandura, 2006, p. 310). Although -72- Journal of Technology Education Vol. 29 No. 1, Fall 2017 some researchers have attempted to create an accurate measure of “general self- efficacy,” arguments still persist about the scales validity as a true measure (Chen, Gully, & Eden, 2001). Sherer et al. (1982) assert that when dealing with specific behaviors, more direct behavioral measures will increase the accuracy of the measurement. Interest in engineering. The second construct in this study is interest. If one seeks to account for the low numbers of underrepresented students in STEM careers (e.g., Babco, 2001), one need only look at the trend of tracking and the placement of minority students in lower academic tracks which has negatively impacted student interest in the sciences (Museus, Palmer, Davis, & Maramba, 2011). Multiple studies describe the importance of interest and its relationship to self-efficacy (e.g., Fouad & Smith, 1996; Hutchinson, Follman, Sumpter, & Bodner, 2006; Wender, 2004). Bandura (1986) suggested that perceived efficacy in people fostered the growth of intrinsic interest, which would remain consistent as long as those interests engaged their personal feelings and offered satisfaction. The decades old trend of placing minorities in lower academic tracks does not foster intrinsic interest and may contribute to shortages of minority representation in fields such as science and mathematics (Babco, 2001; Boyer, 1983). A lack of interest in learning science and engineering may come about if one does not see science or engineering as a viable career option. Researchers in science education have asserted that one reason students from low-income communities are not interested in science is that there is “a ‘disconnect’ between school and home/community life” (Basu & Calabrese Barton, 2007, p. 467). Currently, research offers few solutions on how to sustain these students’ interest. However, Basu and Calabrese Barton (2007) found a “strong connection between a sustained interest in science and science learning environments in which students were able to cultivate relationships with people and in ways that reflected their values of relationships and community” (p. 483). Carlone and Johnson (2007) found that interest in science or science-related fields had less to do with the subject of science than with the effect that their scientific competence would have on the world. The participants in their study were interested in humanitarian work such as health care—efforts that could change the world in a positive way. Interests, along with self-efficacy and outcome expectations, predict intentions, which in turn lead to choice behaviors including those about careers (Lent et al., 1994; Waller, 2006). Waller (2006) also found that African American students’ “math self-efficacy and outcome expectations predicted math interest” (p. 543). Brown et al. (1989) showed that even if there is strong interest in a pursuit, if another option is viewed as more attainable that will be the one to which students will strive. In addition to these findings, Fouad and Smith (1996) found that self-efficacy was a large influence on students’ interest. Math and science self-efficacy are included among the factors that impact students’ interest in engineering. -73- Journal of Technology Education Vol. 29 No. 1, Fall 2017 Perceptions of engineering. The final construct to be explored is students’ perceptions about engineering. A student’s perception of an occupation along with their self-efficacy in skills associated with that occupation greatly influence the likelihood that the student will pursue the occupation (Bandura, Barbaranelli, Caprara, & Pastorelli, 2001). For some individuals, their perceived efficacy rather than their actual achievement is a key determinant of their perceived occupational self-efficacy and preferred choice of work. In a study of African American females, Carlton Parsons (1997) found that 11 of the 20 interviewees imagined a scientist as an unattractive, nerdy, White male. Their image described the male as having a secondary social life with a perfect family. The image that they described did not represent what most African American students see on a daily basis. In fact, negative attitudes toward engineering and “less positive perceptions of the work engineers do” have been reported as key factors in high attrition rates for aspiring engineering students (Besterfield- Sacre, Atmn, & Shuman, 1997; as cited in Hirsch, Gibbons, Kimmel, Rockland, & Bloom, 2003, p. F2A-7). Changing the public’s perception of engineering was a major focus of a study reported in the National Academy of Engineering’s (2008) report Changing the Conversation: Messages for Improving Public Understanding of Engineering. These factors highlight the need to address the negative perceptions that underrepresented students have of sciences. Qualitative Study: Focus-Group Study Purpose Focus-group interviews were conducted with two goals in mind. First, the researchers were interested in understanding the nuances of the MESA organization by unpacking the activity variables of the informal learning environment. Second, findings from the focus-group interviews would inform instrument development for the quantitative phase of the study. The purpose of the focus-group interviews included determining the benefits of participating in MESA’s informal learning environment for underrepresented students. Methodology The research team used a focus-group protocol to guide the interview sessions. “Focus groups are used to gather opinions” (Krueger & Casey, 2009, p. 2). They consist of a series of interviews, conducted with five to 10 participants, wherein the researcher attempts to gain a certain perspective from a particular group (Krueger & Casey, 2009). Focus-group interviews are well suited for qualitative studies including grounded theory (Webb & Kevern, 2001). Members of the group are there for member checking, expounding upon participant responses, and adding clarity to group responses. Focus group interviews typically have five characteristics or features. These characteristics relate to the ingredients of a focus group: (1) people, who (2) -74- Journal of Technology Education Vol. 29 No. 1, Fall 2017 possess certain characteristics, (3) provide qualitative data (4) in a focused discussion (5) to help understand the topic of interest. (Krueger & Casey, 2009, p. 6) In order to ascertain a perspective that was reflective of the MESA program, it was important to establish a “consensus” among group members. For the purpose of this study, researchers felt that focus-group interviews were appropriate. The participants for this study were all members of MESA who provided qualitative data during a focused discussion in an effort to inform the researchers as to the aspects of MESA that were particularly beneficial to their experience. Focus-group interviews are particularly beneficial when seeking consensus: Interactions among participants enhance data quality because participants provide checks and balances on each other’s statements (Patton, 2002). A semi- structured interview technique was employed to collect data. During the focus- group interviews, the interviewer was allowed to digress and probe the students for richer descriptions of activities before returning back to the interview guide to maintain the integrity of the interview process (Krueger & Casey, 2009). Participant Selection Participants were selected for this study using purposeful sampling. Purposeful sampling is an effective strategy of sampling that allows for the collection of “information-rich” data (Glesne, 2006; Patton, 2002). Advisors for each MESA chapter participating in the study selected participants for the focus groups based on student attendance, achievement, and overall participation in the MESA program. Researchers for this study used a purposeful sampling technique in order to secure participants who could provide insight into the aspects of MESA that were beneficial and understand what students are gaining by their participation. Using a purposeful sample of successful MESA programs, researchers were keen in selecting settings and participants who could help illustrate characteristics of the MESA program that led to student recruitment and retention. It is important to note that this focus-group study was not done in an effort to evaluate the effectiveness of MESA; instead, researchers were investigating the aspects of the program that helped recruit and retain students. Participants were provided with food and refreshments as remuneration for their participation. A total of 28 MESA students from five different schools in the California area participated in the five focus-group interviews. Due to convenience, time constraints, and logistical challenges researchers limited their focus-group populations to schools in California. As an example, over a period of 1 week, researchers rented a car and travelled to six different schools in California to collect the data. The student distribution is as follows: Site 1 provided seven participants, Site 2 provided five participants, Site 3 provided six participants, Site 4 provided five participants, and Site 5 provided five -75-