DOCUMENT RESUME JC 010 075 ED 448 805 Instructional Program Review Process. TITLE Austin Community Coll., TX. INSTITUTION PUB DATE 1999-00-00 NOTE 17p. PUB TYPE Descriptive (141) Reports MF01/PC01 Plus Postage. EDRS PRICE College Outcomes Assessment; Community Colleges; Educational DESCRIPTORS Quality; *Evaluation Methods; *Institutional Evaluation; Program Effectiveness; *School Effectiveness; Two Year Colleges *Austin Community College TX IDENTIFIERS ABSTRACT The report describes the Instructional Program Review Process at Austin Community College (Texas). The report includes program review process timeline and schedule. The Instructional Program Review Process focuses on issues of need, cost, and effectiveness of instructional programs. It provides for identification of improvements, includes standards of academic quality, and allows for better decision-making, including modifications, budget, strategic planning. In consultation with the administration, Deans determine the order in which programs in their areas will be reviewed. Each dean will designate a self-study team for each instructional area. The selection will be based on recommendations from the Program Coordinator or Taskforce Chair. The Chair is the key member of the self-study team. The Chair's responsibility begins during the spring term prior to the self-study and continues through the next academic year until the final self-study report has been submitted. The basis of the program review process is the analysis of information relating to a set of core indicators. These core indicators have been identified by the Program Review Work Group and reflect the input from a variety of sources. They include indicators federal, state, and regional agencies, as well as "best practices" identified in similar processes at other community colleges. The core indicators are presented in three sets, categorized by need, cost, and program effectiveness. (JA) Reproductions supplied by EDRS are the best that can be made from the original document. kr) 00 00 .7r C21 Austin Community College Instructional Program Review Process PERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS BEEN GRANTED BY M. Osburn TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) 0 U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement 0 EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) This document has been reproduced as received from the person or organization V-7 originating it. 0 Minor changes have been made to improve reproduction quality. BEST COPY MAILABLE Points of view or opinions stated in this document do not necessarily represent official OERI position or policy. 2 Instructional Program Review Process Overview The Instructional Program Review Process focuses on aspects of need, cost and effectiveness of instructional programs. Specifically, is the instructional quality of the program high enough to meet institutionally set standards? What is a program? For purposes of program review, a program is an organizational unit within the college that provides instructional or support services. These include instructional disciplines, degree and credentialing, and non-instructional services, including non- classroom instructional services. The process outlined here focuses on instructional areas. Purpose of Program Review Process Continuous improvement of programs Assessment of strengths and weaknesses of a program as well as the opportunities and threats it faces. Determination of the capabilities of the program/area Break down barriers between areas of operation Ensure best uses of resourcesfiscal, human, facilities, equipment, technology Articulate program's mission and vision Provide the foundation for application to the Greater Austin Quality Council (Baldrige Criteria) Responds to Board policy Characteristics of Program Review Process The Instructional Program Review Process at Austin Community College: Provides for identification of improvements Includes standards of academic quality Allows for better decision-making, including modifications, budget, strategic planning Is integrated into the Institutional Effectiveness process Is integrated into other external review process, including the THECB site review, SACS accreditation, program-specific reviews required by other agencies Ensures customer focus integrated into program Creates an opportunity for broad-based input from constituencies including students, staff, faculty, employers 1 3 Is a cyclical process with the expectation that there will be continuous review within the program Assesses the degree to which the program is fulfilling its mission and accomplishing its goals Administration of the Instructional Program Review Process Oversight of the Program review process in the instructional areas is the responsibility of the Instructional Program Review Committee. Membership includes one faculty member from each dean area. Appointments are made biennially and members serve for 2 years. The Associate Vice President for Institutional Effectiveness serves as an ad hoc member. This Committee is chaired by a member of the Committee selected by the Executive Vice President for Instruction. The responsibilities of the Instructional Program Review Committee include: Overseeing the Program Review Process to ensure that it occurs within the stated timeline Monitoring of the implementation of the Program Review Process Providing assistance to units involved in the process Evaluating the overall process Developing modifications as necessary to ensure the process functions effectively Developing modifications of the Indicators of Effectiveness, including related criteria, as needed Reviewing self-study reports Guidelines for Implementation In consultation with the administration, Deans determine the order in which programs in their areas will be reviewed New programs will be incorporated into the timeline as they are developed. In most cases, program review will not occur during the first 3 years of program implementation. The Office of Institutional Effectiveness will provide data related to the program review. Specific types of information are indicated below. Self Study Team Each Dean will designate a Self-study Team for each instructional area. The selection will be based on recommendations from the Program Coordinator or Taskforce 2 Chair. The primary responsibilities of the Self-study Team are to collect and analyze the information, develop the Self-study Report, and make recommendations. It is anticipated that the Team will meet on a regular basis during the fall semester. These meetings may be augmented with discussions via email or telephone. The Chair is the key member of the Self-study Team. The Chair's responsibility begins during the Spring Term prior to the Self-study and continues through the next academic year until the final Self-study report has been submitted. The membership of the Self-study Team will include: Member Role # -Coordinates Self-study Chair 1 -Coordinates departmental surveys and focus groups -Provides motivation and direction to the Self-study Team -Sets meeting schedule -Moderates discussions -Coordinates development of the drafts of the Self-study Report -Submits report as required -Acts as liaison with the Dean, Task Force/Program, Instructional Program Review Committee and Office of Institutional Effectiveness Full time faculty- -Participates in discussions 1-3 -Assists with collection and analysis of information in discipline -Assists with the development of the Self-study Report, including writing sections as needed Adjunct faculty- -Participates in discussions 1-2 in discipline -Assists with collection and analysis of information -Assists with the development of the Self-study Report Full time faculty- -Participates in discussions 1-2 not in discipline -Provides insight into relationship of service courses to the instructional area under review, including the collection and analysis of information -Assists with the development of the Self-study Report Students (current -Participates in discussions 1-2 -Provides insight from student perspective and/or former) -Assists with the development of the Self-study Report -Participates in discussions Community 1-2 members/external -Provides insight from the perspective of business, industry, consultant or other college/university -Assists with the development of the Self-study Report Dean (ex officio) -Participates in discussions, but not in writing of Self-study 1 Report 3 External consultants may be included as part of the program review process. The Dean in consultation with the appropriate Associate Vice President and Taskforce Chair/Program Coordinator will make the determination. Program review process This timeline is to serve as a guide. Modifications may need to be made based on institutional constraints. Prior to starting review Instructional Program Review Committee members identified by instructional Executive Vice President Guidelines for Instructional Program Review developed Training for Instructional Program Review Committee members and Self-study Team developed Spring I Summer I Programs to undergo review notified (January) Instructional Program Review Committee meets with Program Coordinators/Taskforce Chairs to review process Program/Task Force identifies Self-study Team membership and makes recommendations to the Dean. Dean reports membership of Self-study Team to Instructional Program Review Committee Surveys of students, faculty, employers, etc. conducted by Program/Task Force and/or Self-study Team Data on indicators provided by Office of Institutional Effectiveness Fall I Self-study Team conducts self-study, with assistance from the Instructional Program Review Committee Self-study Team reviews draft Self-study report and recommendations with Program/Task Force and Dean Institutional Effectiveness Measures reviewed and revised Implications for strategic planning, operation planning, and budgeting determined Taskforce Chair/Program Coordinator proposes Action Plans to address recommendations 4 6 Self-study Report completed by December 15 Self-study sent to Instructional Program Review Committee for review of completeness and quality Self-study sent to Dean for review of results Spring II Preliminary Self-study Report reviewed with Dean and Associate Vice President (January) Associate Vice President and instructional Executive Vice President review the Self-study Report If necessary, Self-study Team revises Self-study Report Associate Vice President incorporates recommendations into institutional planning and budgeting processes By April 15, Dean files final Self-study Report Associate Vice President and the instructional Executive Vice President Summary of Self-study Report prepared by Self-study Chair and Dean presented to Board of Trustees (Information Item) Spring III Progress report completed by Taskforce Chair/Program Coordinator as part of institutional effectiveness process 5 7 Program Review Schedule In consultation with the administration, area Deans are responsible for determining the order in which programs will be evaluated. During the first cycle, programs will be reviewed as indicated in the chart below. The Instructional Program Review Committee will consult with the Deans each fall prior to notifying programs. Dean Area 2001-2002 2000-2001 2002-2003 1999-2000 Photography ACR Human Services BCT Applied Criminal Justice Welding Commercial Art Technical Technologies, Fire Protection Child Development Printing Communications Multimedia, Automotive Commercial Music and Public Service Quality Assurance SRV Computer EDG SMT CIS/CSC CER Studies and ELN Advanced Technology Travel Marketing Accounting Business OST Hospitality Fashion Legal FNM Studies Real Estate Culinary Arts BMG MGT PHR Health VNG DMS OTA FIT (PFT) NSG MLT SRG Sciences PTA DMR EMS AHS Mathematics (including Engineering Astronomy Biology Math and Developmental) Physical Science Health & Kinesiology Physics Sciences Chemistry Geology Psychology History Anthropology Government Social and Geography Sociology Behavioral Economics Human Development Sciences Women's Studies Art Developmental Reading Foreign Languages English Arts and Developmental Writing Radio-TV-Film Dance Speech Humanities English as a Second Journalism Drama Language Music Philosophy/ Religion Humanities Modifications to this schedule may be made in consultation with the instructional Executive Vice President. Where programs are closely related, one Self-study Team may be appointed to review multiple programs. This should be coordinated with the Dean and Associate Vice President. In these cases, the membership of the Self-study Team may be expanded and must include at least one full-time faculty member from each area under review. 6 Indicators of Effectiveness The basis of the Program Review Process is the analysis of information relating to a set of Core Indicators. These Core Indicators have been identified by the Program Review Work Group and reflect input from a variety of sources. They include indicators federal, state and regional agencies, including SACS, the LBB, and the THECB, as well as "best practices" identified in similar processes at other community colleges. As part of the Self-study process, Self-study Teams in consultation with the Dean and Instructional Program Review Committee will select the indicators on which their analysis will focus. The only exception to this is a set of required indicators that are defined by SACS, the LBB, the THECB, other agencies, and/or institutional priorities. These are noted in parenthesis in the tables below. Codes are as follows: SOURCE P = Provided in Documentation Notebook I = Instructional area must provide PRIORITY R = Required O = Optional A = All instructional areas W = Workforce T = Transfer The Core Indicators are organized into three sets--Need, Cost, and Program Effectiveness--defined as follows: Does the program address a verifiable need of students, the community, or Need: society? Within the context of the College's mission, is the cost of the program Cost: justified? Program Effectiveness: Are the teaching, learning, course, program, student and student support outcomes of this program of the best possible quality? NEED Does the program address a verifiable need of students, the community, or society? Need Pri Core Indicators Type of data/information Src R-A Enrollment trends (previous 5 years) Student enrollment in courses P Number of sections offered R-A P R-W Number of declared majors (beginning when Degree Audit is implemented) R-A Graduates (previous 3 years) Number of degrees conferred {THECB, LBB} P Outcomes (previous 3 years) R-W Percent of graduates who are employed with in P one year of graduation {THECB} R-A Number and Percent of graduates who P Transfer with in one year of graduation {THECB, LBB} R-W Percent of employed graduates who indicate P employment is related to training (ACC Survey of Graduates) Analysis of courses transferred O-A P/I Labor market information Labor market opportunities O-W I Employer inputSurvey, focus groups P/I Competition from other institutions Other options for students in the Service O-A I Delivery Area COST Within the context of the College's mission, is the cost of the program justified? Cost Core Indicators Type of data/information Budget analysis (previous 2 years) O-A Analysis of actual expenditure related to I program compared to budget R-A Ratio of non-faculty program costs to contact Comparative program costs (previous 2 P years) hours generated Cost per contact hour consistent with type of R-A Program costs (previous 2 years) P instruction 810