Profiles of High-Performing STEM Majors Paul Westrick, PhD ACT ReseARCh RepoRT seRies 2017–2 Paul Westrick is a research scientist in Statistical and Applied Research specializing in postsecondary outcomes research and validity evidence for the ACT test. © 2017 by ACT, Inc. All rights reserved. R1621 Table of Contents Abstract .................................................................. vi Background ................................................................1 Methods ...................................................................2 Data ....................................................................2 Measures ................................................................2 Analyses .................................................................3 Results ....................................................................3 Descriptive Statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 Standardized Mean Differences in Precollege Academic Achievement Levels ...........7 Standardized Mean Differences in Measured Interests ............................13 Profiles of High-Performing STEM Majors ......................................20 Discussion ................................................................23 Limitations and Future Research. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24 Conclusion ................................................................25 References ................................................................25 iii ACT Research Report Profiles of High-Performing STEM Majors List of Tables Table 1. Descriptive Statistics for STEM-Biological Majors’ ACT Scores and HSGPAs ....4 Table 2. Descriptive Statistics for STEM-Quantitative Majors’ ACT Scores and HSGPAs ..4 Table 3. Descriptive Statistics for Non-STEM Majors’ ACT Scores and HSGPAs ........5 Table 4. Descriptive Statistics for STEM-Biological Majors’ ACT Interest Inventory Scores and Calculated Work Task Dimension Scores ............................5 Table 5. Descriptive Statistics for STEM-Quantitative Majors’ ACT Interest Inventory Scores and Calculated Work Task Dimension Scores ......................6 Table 6. Descriptive Statistics for Non-STEM Majors’ ACT Interest Inventory Scores and Calculated Work Task Dimension Scores ............................6 Table 7. Estimated Mean Effect Sizes for Comparisons between STEM-Biological Majors’ ACT Scores and HSGPAs ...........................................8 Table 8. Estimated Mean Effect Sizes for Comparisons between STEM-Quantitative Majors’ ACT Scores and HSGPAs ....................................10 Table 9. Estimated Mean Effect Sizes for Comparisons between Non-STEM Majors’ ACT Scores and HSGPAs ..........................................12 Table 10. Estimated Mean Effect Sizes Comparisons between STEM-Biological Majors’ Interest Inventory Scores and Calculated Work Task Dimension Scores .......14 Table 11. Estimated Mean Effect Sizes Comparisons between STEM-Quantitative Majors’ Interest Inventory Scores and Calculated Work Task Dimension Scores .......16 Table 12. Estimated Mean Effect Sizes Comparisons between Non-STEM Majors’ Interest Inventory Scores and Calculated Work Task Dimension Scores .............18 Table 13. Profiles of High-performing STEM Majors, Student Means and Interquartile Ranges for STEM Majors with Semester GPAs of 3.0 or Higher in Semesters 5 through 8 .............................................21 Table 14. Profiles of All Persisting STEM Majors, Student Means and Interquartile Ranges for Precollege Academic Achievement and Interest Measures ........22 Table 15. Profiles of High-Performing STEM Majors with Semester GPAs of 3.0 or Higher in Semesters 5 through 8, Medians and Interquartile Ranges for Institutional Means .............................................22 Table 16. Median and Interquartile Ranges of Institutional Means for Precollege Academic Achievement and Interest Measures ..........................23 iv List of Figures Figure 1. Data/Ideas and People/Things plots for STEM major, disaggregated by semester 1 GPA and semester 8 GPA. ................................20 v ACT Research Report Profiles of High-Performing STEM Majors Abstract Building upon the research findings in an earlier ACT Research Report (Westrick, 2016), this study used data from 119,131 students at 26 four-year institutions to make comparisons between STEM majors earning semester GPAs of 3.0 or higher and their STEM peers earning semester GPAs less than 3.0. The results indicate that the high-performing students entered college with higher mean ACT scores and HSGPAs than did their peers, though their measured interests were quite similar. For STEM majors who earned semester GPAs of 3.0 or higher consecutively in semesters five through eight, their mean ACT STEM scores exceeded the ACT STEM benchmark of 26. High school students considering a STEM major in college may benefit from knowing the level of precollege academic achievement required to perform at a high level in a STEM program in college. vi Background This study was a follow-up of the ACT Research Report 2016-5, Profiles of Persisting Fourth- Year STEM Majors (Westrick, 2016). That study examined the standardized mean differences (δ, Cohen, 1988) between students’ precollege academic achievement levels, measured by ACT test scores and high school grade point average (HSGPA), and their interests, as measured by their ACT Interest Inventory scores for STEM and non-STEM majors. Specifically, students were placed into one of three student major categories (SMC) based on their declared majors using the two-digit Classification of Instructional Program (CIP) codes (National Center for Education Statistics, 2002): STEM-Biological (CIP 26), STEM-Quantitative (CIPs 11, 14, 27, and 40), and non-STEM (all other CIP codes). The results indicated that STEM majors enter college with higher levels of precollege academic achievement than did non-STEM majors, and both STEM-Biological and STEM-Quantitative majors had interest profiles that distinguished them from non-STEM majors. Future research using this data set will focus on STEM migration, with comparisons made between students persisting in STEM majors and students who enter or leave STEM fields between the second and eighth semesters. However, the aim of the current study was to build upon the results of the first study by examining differences between those persisting in STEM majors who were performing well in their studies and lower-performing peers. In keeping with previous ACT research (Allen, 2013; Allen & Sconing, 2005; Radunzel & Noble, 2012), a postsecondary semester grade point average (SGPA) of 3.0 or higher, a B or better average, was defined as high performing. The first objective of the current study was to calculate the standardized mean differences between the ACT test scores, HSGPAs, and ACT Interest Inventory scores of “high performing” students persisting in STEM majors and the students who, though they were persisting in STEM majors, were not performing as well as their peers. Identical analyses were conducted for non-STEM majors. As admission test scores and HSGPA are positively correlated with grades earned through four years of study (Mattern & Patterson, 2011; Westrick, 2012), an expected outcome was the mean ACT scores and HSGPAs of STEM majors earning SGPAs greater than or equal to 3.0 would be higher than the means associated with students earning SGPAs less than 3.0. Biserial correlations can be calculated to show the strength of the relationship between a continuous variable (e.g., ACT scores) and a dichotomized variable SGPA of 3.0 or higher – SGPA less than 3.0. However, a biserial correlation can be converted to a standardized mean difference or d-value (Schmidt & Hunter, 2015). Standardized mean differences between two groups are calculated using the means for each group and the pooled standard deviations for the two groups, making group differences on different measures comparable. The advantage of using standardized mean differences is the ease of which they can be converted back to the original metric, something the general reader better understands. Interest Inventory profiles were also examined in this study. Given that the ACT Interest Inventory profiles of STEM majors showed little change over time in the first study (Westrick, 2016), differences in the measured interests of persisting STEM majors dichotomized by their SGPAs were expected to be small. A second objective of the current study was to provide profiles of highly successful STEM majors, where success was defined as earning a SGPA of 3.0 or higher. The first study (Westrick, 2016) presented the profiles of persisting fourth-year STEM majors regardless of their GPAs. The profiles for STEM-Biological and STEM-Quantitative majors were determined using data from all students who had been continuously enrolled for eight semesters and were 1 ACT Research Report Profiles of High-Performing STEM Majors in a STEM major in the eighth semester. In contrast, the profiles of high-performing STEM majors in the current study were based on a select group of persisting STEM students. The profiles for the two STEM categories were based on the students who earned an SGPA of 3.0 or higher in semesters five, six, seven, and eight, the time when students should have been specializing and the majority of their courses were related to their major. Students had to meet this standard in each of the four semesters. Furthermore, these students had to be in the same STEM category – STEM-Biological or STEM-Quantitative – over those four consecutive semesters. The objective was to base the profiles on students who consistently performed well in a STEM category over these four semesters. For reference, the profiles of all persisting STEM majors from the first study are presented with the high-performing STEM majors in current study. Methods Data Data for this study were the same as those used in the previous one (Westrick, 2016). Data came from 26 four-year institutions that had 120,612 students who enrolled as first-time students, of which 119,131 completed the first semester and 66,980 remained continuously enrolled through the eighth semester. As in the earlier study, in each semester students were classified according to their declared major into one of three SMCs – STEM-Biological, STEM- Quantitative, and non-STEM—though in the current study, each SMC is dichotomized based upon the students SGPAs (i.e., less than 3.0, greater than or equal to 3.0). Measures The measures in the current study were ACT test scores, HSGPA, ACT Interest Inventory scores, and SGPA. The ACT test is a battery of four tests – English (ACTE), Mathematics (ACTM), Reading (ACTR), and Science (ACTS) – with a Composite (ACTC) score that is the average score of the four tests. All scores are reported on a scale from 1 to 36. ACT has recently introduced a STEM score, which is the average of the mathematics and science scores (Mattern, Radunzel, & Westrick, 2015; Radunzel, Mattern, Crouse, & Westrick, 2015). The measure of HSGPA in this study was based on students’ self-reported high school grades in four core subject areas: English, mathematics, social science, and natural science. HSGPA was reported on a scale from 0 to 4. The ACT Interest Inventory is a wideband measure intended for use in career exploration. Data was collected when the students registered for the ACT test in high school. The inventory provides scores on six basic types of vocational interests paralleling six career types in Holland’s (1997) theory of careers. The six vocational interests, with Holland’s types in parentheses, are: Science & Technology (Investigative), Arts (Artistic), Social Service (Social), Administration & Sales (Enterprising), Business Operations (Conventional), and Technical (Realistic). ACT Interest Inventory scale scores range from 20 to 80. Research has shown that two dimensions (Data/Ideas and People/Things) underlie job analysis ratings and measured interests of Holland-type career groups (ACT, 2009). ACT Interest Inventory scores can be converted to Data/Ideas (DI) and People/Things (PT) scores.1 As in the previous study, DI and PT scores were examined in the current study. The 1 DI = 0(Realistic) – 1.73(Investigative) – 1.73(Artistic) + 0(Social) + 1.73(Enterprising) + 1.73(Conventional) and PT = 2(Realistic) + 1(Investigative) – 1(Artistic) – (Social) – 1(Enterprising) + 1(Conventional). Source: The ACT Interest Inventory Technical Manual (ACT, 2009). 2 only new measure in this study was SGPA, which was reported on a 0 to 4 scale by each institution. Analyses As in the previous study, standardized mean differences (δ) between the comparison groups regarding their ACT test scores, HSGPA, and ACT Interest Inventory scores were calculated at the institution level, and then meta-analytic techniques were used to provide the overall results across institutions (Schmidt & Hunter, 2015). Standardized mean differences greater than or equal to |0.20| with 80% credibility intervals that did not contain zero were considered to be of practical significance and are presented in bold text within the tables. As noted earlier, standardized mean differences can be converted to correlations and vice versa, and they provide validity evidence much as correlations do (Schmidt & Hunter, 2015). However, standardized mean differences allow comparisons between the means for different groups. For example, if the pooled standard deviation on a measure is 5 and the standardized mean difference between two groups is 0.60, multiplying the effect size (0.60) by the pooled standard deviation (5) indicates that the difference between the mean scores for the two groups is 3. For the semester-by-semester comparisons, high-performing students were defined as students who earned a SGPA of 3.0 or higher. These students were identified in each semester, and students may have changed from one classification to the other (higher- performing or lower-performing) from one semester to the next. As in the first study, students were allowed to change majors throughout the time they were enrolled. Students who were in one SMC in the first semester may have been in another SMC in a later semester. As discussed earlier, the profiles for the two STEM categories are based on the students who earned an SGPA of 3.0 or higher consecutively in semesters five, six, seven, and eight, the time when the majority of their courses would be related to their major. Students had to meet this standard in each of the four semesters, and they had to be in the same STEM category – STEM-Biological or STEM-Quantitative – over those four consecutive semesters. Results Descriptive Statistics The overall descriptive statistics for the higher-performing and lower-performing students are presented in Tables 1 to 3 (ACT scores and HSGPA) and 4 to 6 (Interest Inventory scores). In each semester, STEM and non-STEM majors who earned SGPAs of 3.0 or higher had higher mean ACT scores and HSGPAs when compared with the students within their SMC who earned SGPAs less than 3.0. As the number of students retained decreased between the first and eighth semester, the means generally rose for both the higher performing and lower performing persisting students. As was seen in the first study, the mean scores and standard deviations for the Interest Inventory scores and the Data/Ideas and People/Things work task dimensions changed only slightly over eight semesters, with some means increasing and others decreasing, and with the standard deviations also showing little change. 3 ACT Research Report Profiles of High-Performing STEM Majors Table 1. Descriptive Statistics for STEM-Biological Majors’ ACT Scores and HSGPAs Semester SMC GPA Sem. N ACTC ACTE ACTM ACTR ACTS HSGPA STEM- <3.0 1 4,247 22.3 (3.8) 22.2 (4.6) 21.7 (4.2) 22.7 (5.3) 22.1 (3.7) 3.47 (0.44) Biological 2 3,615 22.6 (3.9) 22.5 (4.6) 22.1 (4.3) 23.0 (5.3) 22.3 (3.9) 3.50 (0.43) 3 2,925 22.9 (3.9) 22.8 (4.6) 22.5 (4.4) 23.4 (5.4) 22.6 (3.9) 3.55 (0.41) 4 2,487 23.2 (3.9) 23.1 (4.7) 22.8 (4.4) 23.6 (5.4) 22.9 (3.9) 3.58 (0.40) 5 2,174 23.5 (3.9) 23.4 (4.7) 23.1 (4.4) 23.8 (5.4) 23.1 (3.9) 3.60 (0.40) 6 1,893 23.6 (3.9) 23.5 (4.7) 23.3 (4.4) 23.9 (5.4) 23.2 (4.0) 3.62 (0.39) 7 1,719 23.9 (4.0) 23.7 (4.8) 23.7 (4.4) 24.1 (5.5) 23.4 (4.0) 3.65 (0.37) 8 1,581 23.9 (3.9) 23.8 (4.7) 23.9 (4.4) 24.0 (5.4) 23.4 (3.9) 3.66 (0.37) STEM- ≥3.0 1 4,970 25.3 (3.9) 25.5 (4.7) 25.0 (4.4) 25.7 (5.3) 24.4 (4.0) 3.77 (0.31) Biological 2 4,838 25.5 (3.9) 25.7 (4.7) 25.2 (4.4) 25.8 (5.3) 24.6 (4.0) 3.79 (0.29) 3 4,453 25.7 (3.8) 26.0 (4.6) 25.5 (4.3) 26.1 (5.2) 24.8 (4.0) 3.80 (0.28) 4 4,377 25.8 (3.8) 26.1 (4.6) 25.6 (4.3) 26.1 (5.2) 24.8 (4.0) 3.80 (0.28) 5 3,921 25.9 (3.7) 26.2 (4.6) 25.9 (4.2) 26.2 (5.2) 24.9 (4.0) 3.81 (0.27) 6 3,857 26.0 (3.7) 26.2 (4.6) 25.9 (4.1) 26.3 (5.1) 25.0 (3.9) 3.82 (0.27) 7 3,588 26.0 (3.7) 26.3 (4.6) 26.0 (4.1) 26.4 (5.1) 25.1 (3.9) 3.82 (0.26) 8 3,579 26.1 (3.7) 26.2 (4.6) 26.0 (4.1) 26.4 (5.1) 25.1 (3.9) 3.82 (0.26) Note. SMC = student major category; Sem. = semester; ACTC = ACT Composite; ACTE = ACT English; ACTM = ACT Mathematics; ACTR = ACT Reading; ACTS = ACT Science; HSGPA = high school grade point average. Table 2. Descriptive Statistics for STEM-Quantitative Majors’ ACT Scores and HSGPAs Semester SMC GPA Sem. N ACTC ACTE ACTM ACTR ACTS HSGPA STEM- <3.0 1 7,368 23.7 (4.0) 22.8 (4.7) 24.3 (4.4) 23.4 (5.5) 23.7 (4.1) 3.47 (0.46) Quantitative 2 6,832 24.0 (4.1) 23.0 (4.8) 24.8 (4.6) 23.6 (5.5) 24.0 (4.2) 3.50 (0.44) 3 5,794 24.3 (4.0) 23.4 (4.8) 25.2 (4.5) 23.8 (5.5) 24.2 (4.2) 3.56 (0.41) 4 4,690 24.4 (4.0) 23.6 (4.8) 25.4 (4.4) 24.0 (5.5) 24.3 (4.2) 3.57 (0.41) 5 3,981 24.5 (4.0) 23.6 (4.8) 25.6 (4.4) 24.0 (5.4) 24.4 (4.2) 3.59 (0.40) 6 3,364 24.6 (4.0) 23.7 (4.8) 25.7 (4.4) 24.1 (5.5) 24.5 (4.2) 3.60 (0.40) 7 2,820 24.7 (4.0) 23.8 (4.8) 25.7 (4.4) 24.2 (5.5) 24.5 (4.2) 3.61 (0.39) 8 2,438 24.7 (4.1) 23.8 (4.8) 25.9 (4.4) 24.1 (5.5) 24.6 (4.3) 3.61 (0.39) STEM- ≥3.0 1 8,148 26.5 (4.1) 26.0 (4.9) 27.5 (4.5) 26.2 (5.5) 26.0 (4.4) 3.77 (0.31) Quantitative 2 6,743 26.7 (4.1) 26.2 (4.8) 27.7 (4.3) 26.4 (5.4) 26.2 (4.4) 3.79 (0.30) 3 5,262 27.2 (3.9) 26.6 (4.7) 28.2 (4.1) 26.8 (5.3) 26.5 (4.4) 3.81 (0.28) 4 5,293 27.0 (4.0) 26.4 (4.8) 28.2 (4.2) 26.6 (5.4) 26.4 (4.4) 3.80 (0.29) 5 4,819 27.3 (3.9) 26.6 (4.7) 28.4 (4.0) 26.9 (5.3) 26.6 (4.3) 3.81 (0.28) 6 4,833 27.2 (3.9) 26.5 (4.7) 28.3 (4.1) 26.7 (5.3) 26.6 (4.3) 3.82 (0.27) 7 4,773 27.1 (3.9) 26.5 (4.7) 28.4 (4.0) 26.7 (5.4) 26.6 (4.3) 3.81 (0.27) 8 4,885 27.1 (3.9) 26.4 (4.7) 28.3 (4.0) 26.6 (5.3) 26.5 (4.3) 3.81 (0.27) Note. SMC = student major category; Sem. = semester; ACTC = ACT Composite; ACTE = ACT English; ACTM = ACT Mathematics; ACTR = ACT Reading; ACTS = ACT Science; HSGPA = high school grade point average. 4