Evaluation & Research Department January 2010 E&R Report No. 09.36 NOVANET 2008-09 Major Findings EVALUATION • Students Served: There were 1,920 enrollments in the fall and spring of 2008-09 n Author: Dina Bulgakov-Cooke, Ph.D. and 763 in the summer of 2009. Students on free or reduced price lunch (FRL) and Black/ Question 1: What need does NovaNET o African-American students were over- address? What are the represented compared to WCPSS overall. NovaNET objectives or goals? • Courses Offered: In 2008-09, 38 NovaNET i courses were offered districtwide. All high t NovaNET is an online individualized schools offered credit recovery (CR) courses. instructional program. Wake County Public Six schools offered remediation opportunities, a Schools System (WCPSS) utilizes NovaNET and 11 offered new credits. as an alternative opportunity for high school • Achievement: Based on summer courses, u students at risk of not meeting graduation NovaNET had a high success rate in helping students earn credits towards graduation. requirements to accrue credits. Nearly all students completed the Summer l 2009 courses attempted and high percentages NovaNET contributes to the WCPSS goal of a passed the coursework (from 83% to 100%). closing achievement gaps and creating Pass rates for End of Course (EOC) tests were opportunities for all students to graduate on v not high (from 17% to 46%), but overall EOC time. The long-term objective of NovaNET course pass rates were still high (82% to is for all students who take a NovaNET 100%). CR course pass rates were 24 E course during the school year or in the percentage points higher than for similar North summer to receive a passing course grade Carolina Virtual Public School (NCVPS) and a course credit. (See Figure 1.) courses. t • Expenditures: Total expenditures for Fall and NovaNET has been used in WCPSS since Spring 2008-09 were $1,553,847. Considering c 1996, when it only served students at risk of the 1,920 enrollments in 2008-09, an average failure at Cary and Mary Phillips High cost per student enrollment was $809 in the fall and spring. With $121,192 in summer salaries a Schools. In 1999, the funding for the and benefits, cost of 763 summer enrollments program was provided through the federal was lower, at about $159 per student. p grant called Safe Schools/Healthy Students. • Recommendations: Based on high summer With the increased funding, the program pass rates, encourage use of NovaNET to earn expanded from two schools to all high m course credits. Examine ways to enhance EOC schools and two alternative middle schools. performance. Designate use of NovaNET Information on NovaNET implementation in within course codes, streamline reporting the earlier years can be found on the WCPSS methods, and enhance monitoring of success. I Web site at http://www.wcpss.net/evaluation- Consider expanded use of NovaNET as a research/reports/. After the grant period was supplement for regular courses. Based on cost– over, WCPSS decided to support the effectiveness, expand student access to NovaNET in summer if feasible. program by funding it through local funds. (Continued on p. 2.) Impact Evaluation reports provide basic evaluative outcomes information on standard indicators. These reports may suggest a need for further study of program efficacy. NovaNET 2008-09 E&R Report No. 09.36 Currently, NovaNET provides alternative academic support to high school students who for various reasons do not succeed in traditional classrooms.1 In 2008-09, each high school offered semester- or year-long NovaNET courses and summer school with one exception (one school only offered summer school). NovaNET served several purposes: • Credit Recovery: Students could recover credits for courses previously failed in a regular classroom by retaking the courses through NovaNET. (See Figure 1.) • Remediation: Students enrolled in a regular class who were at risk of a course failure could simultaneously take individual modules at the NovaNET lab for remediation (while continuing the course in the regular course). • First-Time Credit: Students could take a NovaNET course for a first-time credit. This was reserved primarily for out-of-state or out-of-county transfer students who were behind on high school credits as an opportunity to catch up on their coursework and graduate on time. • Summer School: NovaNET, along with another online option, NCVPS, offered summer school courses, serving mainly senior and junior students who needed credits for on-time graduation. Underclassmen also took NovaNET summer courses for credit recovery, as space in the NovaNET lab was available. • Lesson Supplements: A few schools used individual lessons in NovaNET to supplement their regular materials. The anticipated outcomes are illustrated in the logic model in Figure 1. The short-term outcomes of the program are improved understanding of the NovaNET course material and receiving credits necessary for promotion and graduation. Enhancement of the processes is another short-term outcome that relates to student selection, teacher assignment, and credit reporting. NovaNET’s intermediate and long-term outcomes are increased number of credits recovered and ultimately for all students taking NovaNET courses to receive passing grades and course credits. Question 2: How many students were served by NovaNET overall and by purpose in 2008-09? NovaNET enrollment records during the year were only maintained at the schools and were not provided to the district NovaNET coordinator. Courses were listed using a regular course number, and it is therefore not possible to distinguish courses taken through NovaNET from a regular classroom course, except in the summer. At the evaluator’s request, to estimate the number of total NovaNET enrollments in 2008-09, summary enrollment data for fall and spring of 2008-09 were collected from school- based NovaNET coordinators. Individual student data were not available. The data submitted were summarized and showed the following: • In the fall and spring of 2009, high schools had 1,920 NovaNET enrollments. Because some students may have taken two courses, the total number of NovaNET students was likely to be a little lower than the number of total enrollments. 1 Only high schools were considered in this report. No data were received from one middle school that has a NovaNET lab. 2 NovaNET 2008-09 E&R Report No. 09.36 • Most enrollments during the fall and spring of 2008-09 were in credit recovery courses (1,591 enrollments). • NovaNET was less frequently used for remediation or for gaining a first time credit (123 and 206 enrollments). Two schools were an exception to the general trend and used NovaNET more broadly for remediation and earning new credits (Cary and Fuquay-Varina). Table 1 Number of Enrollments by Purpose in the 2008-09 School Year Fall and Spring 2008-09 Courses High School Credit Remediation First Time Total Recovery (Grade Repair) Credit Apex 96 0 1 97 Athens Drive 93 1 4 98 Broughton 78 2 24 104 Cary 129 21 81 231 Enloe 78 0 0 78 East Wake Schools 135 11 19 165 Fuquay-Varina 35 36 25 96 Garner 77 0 3 80 Green Hope 82 0 0 82 Holly Springs 41 0 0 41 Knightdale 27 0 0 27 Leesville Road 64 0 13 77 Millbrook* 184 0 0 184 Middle Creek 51 0 0 51 Phillips 100 0 18 118 Panther Creek 45 0 3 48 Sanderson 98 2 0 100 Southeast Raleigh 92 0 0 92 Wakefield 80 0 14 94 Wake Forest-Rolesville 56 0 1 57 Wake Early College 0 0 0 0 Total 1,591 123 206 1,920 *Only totals for the school were available, so all enrollments were tentatively assigned to credit recovery. Note: NovaNET enrollments at the four East Wake High Schools are all listed under East Wake High Schools. 3 NovaNET 2008-09 E&R Report No. 09.36 Figure 1 NovaNET Logic Model 4 NovaNET 2008-09 E&R Report No. 09.36 Unlike Fall and Spring 2008-09, Summer 2009 NovaNET rosters were centrally maintained. • Summer 2009 rosters listed 763 enrollments. • A total of 695 students took NovaNET courses, with 69 students taking two courses and 626 students taking one course. Table 2 shows the total number of students enrolled in NovaNET courses at each high school in Summer 2009. Number of enrollments ranged from 13 to 64. Table 2 Summer 2009 NovaNET Enrollments by School Number of Number of Number of Students Students High School Enrollments Enrolled in Enrolled in Two Courses One Course Apex 32 6 20 Athens Drive 30 0 30 Broughton 50 7 36 Cary 39 1 37 East Wake School of Health Sciences 16 4 8 East Wake School of Integrated Technology 13 1 11 East Wake School of Arts, Education, and 17 5 7 Global Studies East Wake School of Engineering Systems 16 0 16 Enloe 36 6 24 Fuquay-Varina 27 1 25 Garner 37 8 21 Green Hope 35 8 19 Holly Springs 29 0 29 Knightdale 64 1 62 Leesville Road 32 0 32 Middle Creek 31 0 31 Millbrook 58 10 38 Panther Creek 36 1 34 Phillips 13 1 11 Sanderson 40 2 36 Southeast Raleigh 28 4 20 Wake Early College 13 0 13 Wake Forest-Rolesville 29 1 27 Wakefield 42 1 40 Total 763 69 626 5 NovaNET 2008-09 E&R Report No. 09.36 Question 3: What resources were needed to implement the program in 2008-09? Was NovaNET cost effective? Resources needed to maintain NovaNET included annual renewal of software licenses, NovaNET teacher training costs, and maintaining a full-time position at each school. Considering that the total 2008-09 NovaNET budget excluding summer was $1,553,847, and there were 1,920 enrollments during the year, an average cost per enrollment was $809 during the year. This is comparable with the average cost of a regular course in WCPSS. Based on the 2008-09 district Budget, the cost of a typical course per student in the district was about $803. This calculation was made based on teacher salaries and the cost of instructional materials for an average class size of 22 students. There were 22 full-time NovaNET positions available for the year plus part-time positions for summer school. NovaNET teachers were funded through at-risk funding, one position per school (the East Wake High Schools shared one position). A single position at some schools was divided between two or three teachers. (See Table 4.) The 2008-09 NovaNET budget showed that teachers’ total salaries and benefits, were $1,294,890. This constituted $58,858 per teaching position including benefits. Based on teacher and software costs, the cost of 763 summer enrollments was much lower than in the fall or spring, at $159 per student, with $121,192 in summer salaries and benefits. (See Table 3.) Clearly, per student cost of a NovaNET course taken in summer was lower than a similar course taken during the year. Table 3 NovaNET Costs Description State Budget Local Budget Total 2008-09 NovaNET Teacher Salaries $ 935,100.97 $ 124,431.13 $1,059,532.10 2008-09 NovaNET Teacher Benefits $ 193,681.53 $ 41,676.03 $ 235,357.56 2008-09 NovaNET Cost for Ports* $ 258,958.18 $ - $ 258,958.18 Total NovaNET $1,387,740.68 $ 166,107.16 $1,553,847.84 2009 Summer School NovaNET Teacher Salaries $ 79,658.24 $ 14,269.75 $ 93,927.99 2009 Summer School NovaNET Teacher Benefits $ 12,162.11 $ 2,193.26 $ 14,355.37 2009 Summer School NovaNET Cost for Ports $ 12,909.40 $ - $ 12,909.40 Total 2009 Summer School NovaNET $ 104,729.75 $ 16,463.01 $ 121,192.76 Expenditure Totals 2008-09 $1,492,470.43 $ 182,570.17 $1,675,040.60 *Data Source: Annual Contract with Pearson, Inc. The amount includes renewal of annual license and support fee (ports) and annual training. Slight variations in costs are possible due to the specifics in tax calculations. With 15 simultaneous ports available per school and courses offered during three periods, each school could potentially handle as many as 45 students per semester. Student enrollments were only restricted to one NovaNET teacher and the number of computers in the lab. This was especially true in summer when the number of ports was not limited. Analysis of the enrollment in 2008-09 shows that high schools had a median of 93 NovaNET enrollments per year per school (See Table 1). With an exception of four schools, the ports have been fully utilized. 6 NovaNET 2008-09 E&R Report No. 09.36 Question 4. How was NovaNET implemented in the WCPSS high schools in the fall and spring of 2008-09 and in the summer of 2009? Each high school received the same resources: one full-time position to implement NovaNET, 15 simultaneous ports to access the courses, and annual teacher training. Implementation varied somewhat by school. To examine any differences in NovaNET implementation, an open-ended questionnaire was administered to NovaNET coordinators or teachers. Based on the responses from 16 NovaNET coordinators or teachers (66% response rate), some variations in implementation of NovaNET were identified at each high school. The differences were in management, teaching positions, courses offered, and student access to NovaNET. Coordination and Management In at least seven of the 16 responding high schools, assistant principals or deans of students served as NovaNET coordinators, i.e., as part of their duties they monitored and coordinated NovaNET implementation. This was especially important at the schools that had the single NovaNET teaching position divided between two or three teachers. Other schools had dedicated NovaNET coordinators, who were full-time NovaNET teachers with no other duties assigned. Number of teachers Except for the four small East Wake High Schools that shared one position, one full-time position per school was assigned to NovaNET for the fall and spring semesters. Schools could decide whether to assign one full-time teacher to the position or several teachers to teach a course or two. Seven schools reported assigning one teacher to NovaNET, and eight indicated more than one teacher was assigned—usually two or three (see Table 4). • At a school with one NovaNET teacher, she/he usually becomes a resident expert who has an in-depth knowledge of how NovaNET works from the perspective of the teacher and the student. • Using more than one teacher also had its benefits. For example, it allowed expanding use of NovaNET to more than three periods per day and to after-school use. After-school hours were especially beneficial for students who were returning after dropping out of school. As another example, three NovaNET teachers at one of the schools were able to develop supplemental materials for each NovaNET course they taught. Involving more than one teacher also meant each teacher was more likely to have expertise in the subject area of the course. Some additional staff also worked in the NovaNET lab that typically was funded through other sources. 7 NovaNET 2008-09 E&R Report No. 09.36 Table 4 Number of NovaNET Teachers per School in Fall and Spring of 2008-09 # of NovaNET Teachers Additional NovaNET Staff High Schools Sharing One (Regular Allotments) NovaNET Allotment Several other teachers delivered NovaNET instruction for one Cary 1.5 period per day. Holly Springs 1 East Wake * 1 Several teachers were assigned NovaNET courses and combined Fuquay-Varina 1 NovaNET with regular teaching. Wakefield 1 One additional after-school teacher. Enloe 1 Several additional teachers taught part-time. Phillips 2 Leesville 2 Green Hope 2 Apex 1.5 Middle Creek 3 Behavior support specialist taught NovaNET. Southeast Raleigh 3 Athens Drive 3 Garner 1 One teacher taught one block. Sanderson 1 *The four East Wake High Schools shared one position. Courses The length of courses offered through NovaNET varied by school. Typically, NovaNET courses were semester-long. However, several schools had year-long rather than semester-long courses. One school offered NovaNET only in spring and summer of 2008-09, and another offered only a summer NovaNET. Other differences were in the choice of available courses. One NovaNET lab offered a number of pre-intervention mathematics courses to prepare students for Algebra I. Another made available initial credit for technical mathematics. While English I, II, and III, as well as Algebra I and II were most frequently offered, the number of Non-EOC NovaNET course offerings varied by school. Access Student ability to gain access to NovaNET courses varied in terms of location and time of the day. A number of schools allowed NovaNET access not only from the lab, but from any other location at the school, while others restricted access to certain areas. A number of schools had opportunities for after-school or Saturday access, others had no such opportunities. 8 NovaNET 2008-09 E&R Report No. 09.36 Duties of NovaNET Coordinators and Teachers To gain a better understanding of NovaNET daily operations, the evaluator investigated the duties of NovaNET coordinators and teachers. Generally, NovaNET coordinators oversaw the program, maintained site coordination, and communicated with administrators and counselors. Coordinators who did not teach NovaNET courses guided NovaNET teachers. Typically coordinators: • discussed student screening and selection to the program, • coordinated student schedules, • addressed attendance and discipline issues, • maintained parent contacts when students were struggling, • discussed coursework with counselors and students, • administered diagnostic tests and administered testing for summer school, • completed summer school report cards and finalized summer school promotions and retentions, and • coordinated use of NovaNET for homebound students and with special programs. Some coordinators were NovaNET teachers. Teachers maintained the NovaNET lab and program and provided support to students. They worked with guidance counselors to admit students. They also performed the following duties: • assigned students to courses/curricula, and assigned modules to students, • prepared curriculum guides, • consulted with other teachers for assistance with content area questions, • addressed attendance and discipline issues, • interacted with students and assisted students with work and course content, • helped with questions, • monitored students and tutored course content, • ensured that students were on-task and managed behavior, • tracked student progress through NovaNET reports, and • recorded grades and calculated final grades. Number and Types of Courses Offered In 2008-09, based on data submitted by the coordinators, 38 different courses were offered in WCPSS through NovaNET. EOC courses were offered most frequently. In the fall and spring of 2008-09 and Summer 2009, students were most frequently enrolled in mathematics courses (Algebra I, Algebra II, Geometry, and Technical Math) but English (English I and III), Science (Biology, Earth Science, and Physical Science), and History (US History, and World History) were also frequently taken. Course names and the number of schools offering the courses are listed in Table 5, organized in descending order, from the highest to the lowest number of schools offering the course. In addition to core courses, a number of electives which are less typical for NovaNET were also each offered by a single school (e.g., Journalism, Physics, Advanced Functions, and Digital Communications). 9 NovaNET 2008-09 E&R Report No. 09.36 Table 5 NovaNET Courses Offered in 2008-09 Counts of Schools Offering Courses Courses Offered Fall and Summer Spring 2009 2008-09 Algebra I (including Algebra part 1 & 2) 20 25 English I 19 16 Biology 18 17 Physical Science 18 17 English III 18 26 U.S. History 17 25 Earth Science 17 15 World History 17 13 Civics & Economics 16 16 Algebra II 15 21 Geometry 15 16 English IV 13 18 Technical Math I 12 6 Intro to Math 10 7 Technical Math II 10 7 English II 8 18 Chemistry 6 5 Healthful Living 6 2 Study Skills 5 3 Environmental Science 4 5 World Civilizations 4 1 Geography 4 1 PreAlgebra 4 1 Career Exploration 3 0 Graduation Elective 3 0 Sociology/Psychology 2 0 Psychology 2 0 Child Development 2 0 Digital Communications 1 0 Print Graphics 0 1 Trigonometry 1 0 Sociology 1 0 Advanced Functions 1 1 Math Elective 1 0 Physics 1 0 Journalism 1 0 Reading 1 0 Computer Science 1 0 Data Source: NovaNET Coordinators’ Summary Data 10