by Wendy G. Ford, PhD Evaluating the Effectiveness of College Web Sites for Prospective Students Introduction College Web sites are often the first structured encounter a student has with a prospec- tive college or university. Outside of serving as a marketing tool (Williams 2000), very little literature exists on the functional purpose of a college’s Web site. Almost all college sites show an informational and transactional tool for currently enrolled students. for prospective students, the same site is a connection to a possible education and lifestyle. For colleges, regarding prospective students, the college Web site Literature Review is increasingly becoming a sales tool. As with all effective mar- When prospective students visit a Web site, they are engaging keting and sales tools, understanding the customer is crucial to in information-seeking behavior. As Vassiliadis (2002) indicates, success. In addition, Mechitov (2001) raises the importance of success or failure with information seeking that is conducted via a developing a systematic approach to university Web site design, computer is significantly tied to interface design. Thus, one factor based on customer needs, as financial investments in university for determining the effectiveness of a Web site is to evaluate the Web sites increases. effectiveness or usability of the site’s organization and design. In line with this, Vassiliadis highlights the critical guidance that This research examined how well college and university Web sites library users supplied in the redesign of the library Web site at the meet the informational needs and expectations of prospective University of North Carolina at Chapel Hill. Web site usability and students. In addition, it is intended to provide a baseline assessment effectiveness is intricately tied to user needs. Mechitov (2001) of college and university Web sites that specifically identifies also demonstrates the link between successfully identifying user whether the sites are meeting prospective students’ needs and to needs and incorporating them into the design of Web sites. provide an exploratory benchmarking vehicle for further study in the area of college/university Web site effectiveness. This study Demonstrating the cyclical nature of Web site evaluation, Vassiliadis addresses how well college Web sites meet the needs of high (2002) indicates that evaluation “should be incorporated into school students, and which needs of high school students are Web management.” This can most efficiently be done through being addressed most and least effectively. the use of standardized evaluation and baseline assessment tools. 26 | SUMMER 2011 JOURNAL OF COLLEGE ADMISSION WWW.NACACNET.ORG The results of this research attempt to move colleges closer toward Table 1. The Ranked Importance of developing and improving these types of evaluative instruments. College Web site Characteristics Characteristic (Important or Very Important) % Rank Abrahamson (2000) and Steele (2002) profile the 60 million Content 97 1 highschool student population as growing up with the Internet and Organization/architecture 95 2 having a strong attraction to technology. For this generation, a Web Download speed 88 3 presence is a direct extension of the institution itself. To stress the Organization by functional topic (admissions, athletics, etc.) 84 4 point, Abrahamson observes that in the past, colleges marketed in a Organization by target audience (applicants, alumni, etc.) 78 5 free-for-all manner, with the admission office sending one message Friendliness 73 6 and each department creating its own literature and brochures to Distinctiveness 49 7 send its message. This community patchwork approach is evident in Graphics (major emphasis) 49 7 college Web sites where each department is encouraged to develop Graphics (minor emphasis) 40 8 and maintain its own pages. The result is a disjointed presence Table 1 - Poock (2001) of varying levels of technical sophistication and inconsistent information depth and tone. Abrahamson goes on to point out that Researchers developed a form to evaluate how important the top prospective college students will quickly project dissatisfaction with five characteristics were to high school students; the form, used a Web site to dissatisfaction with the actual institution. to gather data about 18 colleges in North Carolina, contained dichotomous yes/no questions. (The Poock study included high Steele (2002) identifies some of the Internet information-seeking school students from North Carolina and most examples in that behaviors of high school students and notes that their Internet study related to universities in North Carolina). According to 2000 attention span is streamlined to allow them to quickly discern, survey data, provided by the National Center for Educational sometimes at a glance, if a Web site is offering the information Statistics, 75 percent of undergraduate students attend college they want. Steele’s interviews with high school students have re- within their state of residence. Based on this data, researchers vealed that minimally, prospects are on a fact-finding mission that chose to map Web site perceptions of high school students in they expect to be direct and fruitful. Likewise, Hartman (1998) North Carolina to college Web sites in the same state to see if indicates the importance of developing Web sites targeted towards colleges were meeting the needs of their largest potential student prospective students that offer not only promotion but also sub- population base. stance. He challenges institutions to develop strategic Web sites that will allow for a competitive advantage as recruitment enters A systematic random sample was used to select 18 colleges for the Internet age. the study. The 54 colleges in North Carolina were listed alphabeti- Methodology cally and every third college was selected. To gather data for each For the purposes of this study, “college” refers to a minimally four- college, the researcher visited the Web site of each college over a year, degree-granting institution. Both public and private colleges six-week period. were included to provide a broad spectrum of analysis data. The findings and data Analysis subjects, prospective college students still in high school, were 1. Content chosen because they are the largest incoming applicant constitu- Ninety-seven percent of high school students rated content as ents for colleges and universities; and because this study is based important or very important. This characteristic was identified as on that of Poock (2001), who studied this same population’s per- the most important for prospective students. In examining content, ception of college Web sites. the researcher attempted to determine if specific categories of information appeared on each college Web site. The content In his study, Poock gathered data from 55 high school students items included in this study were 1) application fee 2) financial in North Carolina and Michigan. Poock’s findings, Table 1, are aid information 3) scholarship information 4) majors offered ranked by the importance to high school students. In this study, 5) admission criteria 6) tuition 7) faculty information 8) general the top five characteristics are examined to determine if they meet academic information 9) the application and 10) general college prospective students’ needs. (The remaining characteristics have contact information. Each college received one point for each inherent visual and subjective elements best studied with the content item on their Web site. direct participation of the target audience). WWW.NACACNET.ORG SUMMER 2011 JOURNAL OF COLLEGE ADMISSION | 27 Table 2 represents the points achieved for each As indicated in Table 3, general academic informa- Web site and Figure 1 depicts the overall content tion was not available on eight college Web sites and representation for the Web sites as a whole. Forty- tuition information was not available on three sites. two percent of the Web sites reviewed in the study Table 3. Content Items Missing contained all of the content that prospective college students rated as important or very important. Content Item Absence % frequency Only three of the Web sites, representing 14 General Academic Information 8 44% percent of the sample, were missing two or more Tuition 3 17% content items. Admission Criteria 2 11% Table 2. Content Points Earned Faculty Information 1 6% for each Web Site It is important Web Site # of missing Points Additional areas of concern relate to the consis- content to be able to items tency of information presented in terms of content. Some of the items which students look for currently Appalachian State University 0 10 evaluate the have a well-defined structure and definition. For ex- Davidson College 0 10 ample, of the content areas that were examined in many areas Elizabeth City State University 0 10 this study application fee, majors, tuition, and ap- High Point University 0 10 of functioning plication have a clear and common definition. Other Lenoir-Rhyne College 0 10 content areas are not as well defined. The financial that comprise Montreat College 0 10 aid information a student is seeking can vary widely. academic North Carolina A&T State 0 10 Is the student seeking information on how to apply University for financial aid or information on what type of aid is success in Belmont Abbey College 1 9 offered or on financial aid counseling? In several in- Campbell University 1 9 college students. stances, this information appeared on different pag- Mars Hill College 1 9 es within a Web site. The faculty information also Relying solely on Shaw University 1 9 varies widely. Are prospective students interested in St. Andrews Presbyterian College 1 9 faculty email addresses or vitae? Again, in several ability measures, University of North Carolina – 1 9 instances, this information was presented on differ- such as college Charlotte ent pages of the Web sites and also, in some in- University of North Carolina – 1 9 stances, in different organizational areas of the Web entrance exams Wilmington sites. Clarifying what prospective students’ needs or high school Western Carolina University 1 9 are in these loosely defined content areas would en- Gardner-Webb University 2 8 hance a college’s ability to organize and provide this grade point North Carolina Wesleyan College 2 8 information in an effective manner. averages will Piedmont Baptist College 3 7 2 and 4. Organization/Architecture/Functional Topic not result Along with available content, it is also important to Poock’s findings indicate that 95 percent of high understand which content items interest prospec- school students rated the organization/architecture in accurate tive students, but are not appearing on college Web of a Web site as important or very important, and 84 measurements. sites. Table 3 depicts the content items that were percent of high school students rated organization not located on various Web sites in the sample by functional topic as important or very important. and their omission frequency. If the Web site had The two categories are closely linked and include a search function, the search function was used to the visual structure of the site and the ease of navi- try to locate the item, in the absence of a specific gation, so are addressed together. These categories link to the information. If the result of the search were evaluated by examining the number of links was not successful, the item was deemed missing. that had to be followed to get to information for the 28 | SUMMER 2011 JOURNAL OF COLLEGE ADMISSION WWW.NACACNET.ORG selected content areas. For example, when seeking general aca- were presented either in the form of a visually consistent navigation demic information, although a link for academics may appear on bar or in the form of independent text or logo hyperlinks. McGillis the homepage, the student may have to click two additional links (2001) indicates that studies have shown that the navigation bar, before getting to a specific information item. In that case, a total although intended to be an aid for the user, is often an overlooked of three clicks were needed to obtain the information. item on a Web page. Two-thirds (12) of the Web sites examined utilized navigation bars that were consistent on the majority of the As previously indicated, some information areas were broadly de- content pages in the Web site. This gave the sites cohesive appear- fined. Financial aid can refer to how to apply for aid, what type ances and allowed for structured designs that facilitated the sites’ of aid is offered or what advisement is offered in this area. With navigation and organization. topics that were loosely defined, the number of clicks required to get to the first information item was counted. If a Web site had a Of the six sites that did not have navigation bars, three used a con- “prospective student” link, the “prospective student” page was sistent set of links, although they were not necessarily consistently treated as the homepage. Thus, if general academic information located across the content pages in the Web site. The other three appeared on the first page of the academic link and a college’s sites that did not have a navigation bar included fewer relevant academic link was on the homepage, the number of links needed content links and these links were not consistently placed on each to get to the information was counted as one. Likewise, if general information page. This organizational structure required the pro- academic information appeared on the first page of the academic spective student to visually search each page for links of interest. link, and a college’s academic link was on the prospective student page, the number of links needed to get to the information was A navigation bar on a Web site provides a visual focal point for the also counted as one. In this manner, having a prospective student user when navigating through a site. This is particularly useful for page is not regarded as a disadvantage for a college. college Web sites as it allows the major links or relevant content areas to be easily identified. The absence of a navigation bar re- Poock found that high school students wanted to obtain informa- quires the user to use the “back” button or visually hunt through tion within three clicks; if they did not, they lost the initial positive the available links to locate relevant content. feeling associated with the search. For the 18 Web sites exam- ined in this study, on average, two clicks were needed to get to a 3. Download Speed specific information item. Likewise, two clicks was also the most Eighty-eight percent of high school students rated download speed frequent number of clicks needed to get to any particular content as important or very important. The sites visited had an average item from the homepage or from the prospective student page. download speed of 6.17 seconds with two sites having a download Two-thirds (12) of the Web sites examined utilized navigation bars that were consistent on the majority of the content pages in the Web site. This gave the sites cohesive appearances and allowed for structured designs that facilitated the sites’ navigation and organization. The maximum number of clicks needed was five to obtain faculty speed of 15 seconds and three sites having a download speed of information at one college and contact information appeared in all two seconds. The most frequent download speed was five sec- but two instances on each Web site’s homepage, thus requiring onds. Download speeds are affected by network connection and zero clicks to obtain the information. the amount of and type of graphics depicted on a Web site. Ideally, if a Web site has an extensive amount of graphics, the site should Site organization was also evaluated by examining the number rel- offer the user a text-only version of the site. evant content links—links that took the student to one of the 10 content areas examined in this study—on each content informa- 5. Organization by Target Audience tion page. In addition, a link to the homepage or the prospective Poock’s findings concluded that 78 percent of high school stu- student page was counted as a relevant content link. These links dents feel that a site’s focus on the target audience is important WWW.NACACNET.ORG SUMMER 2011 JOURNAL OF COLLEGE ADMISSION | 29 or very important. Poock suggested that a college Web site was was absent from 44 percent of the Web sites. This represents targeted toward prospective college students if the Web site has eight out of the 18 sites visited. This high absence percentage a constituent link for prospective students and the Web site uses may be attributed to the broad definition of this content area and terminology that is familiar to high school students. For example, to how the presence of this item was determined. the word “major” is used instead of “program,” and the Web site presents its majors listed alphabetically. My findings revealed that General academic information was defined as academic 94 percent of the Web sites studied contained constituent links, information presented independent of a college department. The but only 56 percent of the sites contained prospective student absence of general academic information meant that the Web links. Other constituent links present were for alumni, current stu- site presented academic information only in the context of a dents, faculty, and visitors. college department. Thus, the high school student had to select a particular college department to access academic information. The prospective student link provides a user-targeted homepage Funneling the prospective student through a department at this where colleges can use terminology relevant and information perti- information seeking stage reduces their flexibility mentally, as well nent to these students. This creates a welcoming environment and as structurally. In addition, several content areas, though present implies that the college has made a space for the student and his on all Web sites, presented information at various levels of detail. or her needs. This was most often present in loosely defined content areas, such as financial aid and academic information. Clarifying what Seventy-eight percent of the Web sites used “major” when iden- prospective students’ needs are in these loosely defined content tifying their programs of study and only 50 percent of the sites areas would enhance a college’s ability to organize and provide this listed their majors alphabetically. If a site listed its majors alpha- information in a more effective manner. betically, within department or school, that site was not counted as listing its majors alphabetically. Requiring the student to be 2. and 4. Organization/architecture/functional topic familiar with which majors are associated with which departments In these areas, the needs of high school students are being met detracts from the Web site’s ability to serve the needs of prospec- when information is being presented in two links. In addition, tive students. Table 4 summarizes the findings related to target two-thirds of the Web sites examined utilized a navigation bar and audience, download speed and site organization. 67 percent of the Web sites visited provided search functionality. Consequently, high school students have the ability to perform Table 4. College Web site Organization Summary agile navigation and directed searches. Constituent Links Present 94% Prospective Students Link Present 56% 3. Download speed Term "Major" Used 78% The average download speed was 6.17 seconds, indicating that Majors Listed Alphabetically 50% high school students data download expectations are being met. Average Download Speed 6.17MB/sec 4. Organization by target audience Text-Only Option 17% In this area, findings indicate that the needs of high school Search Present 67% students are not being met. Only 56 percent of the Web sites Link to Application of Homepage or Prospective Student Page 28% visited contained prospective student links. In Poock’s study, prior to viewing the college Web sites, most of the students discussion indicated that a functional organization for the college Web site Research Question #1: How well do college Web sites meet the was preferred. However, after viewing and navigating several sites, needs of high school students? most of the students then indicated that organization by target audience was more important. This area warrants further research 1. Content to clarify student needs and to provide useful information and Eighty-six percent of the Web sites visited contained 90 percent of guidance to college Web site designers. the content that high school students expected to see. These find- ings indicate that the content expectations of students are being Another area examined was target audience terminology, specifi- met by the information present on college Web sites. The content cally regarding the use of the term “major” and how majors were that was not available most often, general academic information, 30 | SUMMER 2011 JOURNAL OF COLLEGE ADMISSION WWW.NACACNET.ORG presented to the students. Although the use of the term “major” The results clearly indicate that many colleges are missing a pro- was prevalent, only 50 percent of the Web sites visited listed the spective student Web site section—a section that is critical to majors alphabetically, independent of the college department un- recruitment. Because high school students are increasingly using der which the major is offered. Many high school students are not college Web sites to research where they’d like to apply, colleges familiar with the academic structure of college and departmental should think of Web sites not only as sources of information, but as designations. Primarily, majors should be listed alphabetically sales and marketing tools. This study has also established a base- and the academic structure in which the major resides should line assessment of prospective students’ needs and can be used as be presented as supplemental information. This structure allows benchmarking vehicle for further study in the area. high school students to focus on the information of interest, the major, and not get bogged down in the hierarchical structure of the college. WEndy g. foRd, Ph.D., assistant professor at Queensborough Community College – City University of New York (NY) in the Business Research Question #2: Which needs of high school students are Department, teaches computer information being addressed most effectively? systems. She received her doctorate from Long Island University (NY) and her research interests include Web design, information The research findings indicate that the content or informational systems, e-government, and small business needs and the organizational/architecture/functional topic needs information management. are being addressed most effectively. As previously stated, the majority of the Web sites visited presented between nine and 10 of the informational items sought by students. In addition, the REfERENCES student was able to locate information using navigation bars. This indicates that students were able to find the content that they Abrahamson, Thomas. 2000. Life and Misic, Mark M. and Kelsey L. Johnson. were looking for in an organizational structure that was intuitive death on the Internet: to Web or not to 1999. Benchmarking: a tool for Web site Web is no longer a question. Journal of evaluation and improvement. Internet and flexible. College Admission. 168. 6 – 11. Research: Electronic Networking Appli- cations and Policy. 9(5). 383 – 392. Breeding, Marshall. 2003. Benchmark- Research Question #3: Which needs of high school students are ing technology: a theory of penultimacy National Center for Educational being addressed least effectively? Computers-in-Libraries. 23(3). 44 – 46. Statistics. Mobility of College Stu- dents. http://nces.ed.gov/programs/ City University of New York Queens- coe/2008/section1/indicator10.asp. borough Community College. www.qcc. Accessed 8/2/2010. The data reveal that organization by target audience is least cuny.edu (accessed April 25, 2004). effectively addressed by college Web sites. College Web sites Ng, Choo Kiang, et al. 2003. Evaluation Deutsch, Paula and Barbara Silcox. of a graduate school web-site by gradu- should have prospective students links structure the information 2003. Learning from other libraries: ate assistants. College Student Journal. about majors in a format that does not require the student to select benchmarking to assess library per- 37(2). 242 – 259. formance. Information-Outlook. 7(7). or identify a department of study. Another finding, suggested by 18 – 25. Poock, Michael C. and Dennis Lefond. Poock, indicates that the college application should be available 2001. How college-bound prospects Hartman, Kenneth E. 1998. The internet perceive university Web sites: findings, on the Web site’s homepage or on the prospective student link & college admissions: implications and implications, and turning browsers page. This study revealed that only 28 percent of the Web sites opportunities. Change. 2(54). 5. into applicants. C & U Journal. Sum- mer. 15 – 21. visited placed the college application on the homepage or the Kim, Sung-Eon, et al. 2003. Web site design benchmarking within industry Steele, Jonathan. 2002. Media omni- prospective student link page. This simple change increases a groups. Internet Research: Electronic vores: understanding college-bound student’s likelihood for applying to a particular college. Networking Applications and Policy. students and communicating with them 13(1). 17 – 26. effectively. Journal of College Admis- sion. 175. 10 – 19. Conclusions and future Research McGillis, Louise and Elaine G. Toms. 2001. Usability of the academic library Vassiliadis, Kim and Lisa R. Stimatz. Due to the sample selection methods used, the results cannot be Web site: implications for design. College 2002. The instruction librarian’s role in generalized. Further research should explore other sections of the & Research Libraries. 62(4). 355-67. creating a usable Web site. Reference Services Review. 30(4). 338 – 342. student population. In addition, the study examined the needs of Mechitov, Alexander I., et al. 2001. high school students. While the largest number of prospective stu- Comparative analysis of academic Web Williams, Brian Gary. 2000. To the per- sites. Education. 121(4). 652 – 662. sonalized go the prospects. Journal of dents for a college is high school students, the needs of transfer College Admission. 166. 12-21. students and adult learners should not be ignored. WWW.NACACNET.ORG SUMMER 2011 JOURNAL OF COLLEGE ADMISSION | 31