DOCUMENT RESUME CG 029 013 ED 426 329 Sampson, James P., Jr. AUTHOR Using the Internet To Enhance Test Interpretation. TITLE 1998-01-18 PUB DATE 9p.; Plenary paper presented at the ERIC Clearinghouse on NOTE Counseling and Student Services Conference: "Assessment '98: Assessment for Change-Changes in Assessment" (St. Petersburg, FL, January 16-18, 1998). -- Speeches/Meeting Papers (150) Research (143) Reports PUB TYPE MF01/PC01 Plus Postage. EDRS PRICE Computer Mediated Communication; Counseling; Distance DESCRIPTORS Education; *Internet; Lifelong Learning; *Test Interpretation; Test Use; Testing; Testing Problems ABSTRACT The Internet is now adding the potential for remote delivery of test administration, scoring, profiling, report writing, and multimedia functions, as well as cost-effective capability in communication and links to related information. Using the Internet to enhance test interpretation is the focus of this paper. A review of potential Internet applications in test interpretations is provided. One important advantage of using the Internet as a test interpretation resource is the ease with which the interpretive data can be kept current. Client preparation for test interpretation, generalized test interpretation, specific test interpretation, and supervision are covered. All of these applications can be accomplished with technology that is currently available. Computer-based test interpretation, two-way videoconferencing, moderated list servers, moderated computer conference, and follow-up resource links are potentially powerful features of the Internet available to test users. Issues discussed associated with Internet use in test interpretation include inappropriate use of counselor-mediated assessment, relationship issues, ethical issues, credentialing, and counselor training. Given the general direction of change, it would seem wise to experiment with appropriate applications of this technology and proactively deal with potential limitations while there is still time to shape the early adoption of the Internet in testing. (EMK) ******************************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. ******************************************************************************** The Internet and Test Interpretation 1 Using the Internet to Enhance Test Interpretation by James P. Sampson, Jr. Plenary paper presented at the Assessment '98 Conference: Assessment for ChangeChanges in Assessment, St. Petersburg, Florida, January 18, 1998 U.S. DEPARTMENT OF EDUCATION "PERMISSION TO REPRODUCE THIS Office of Educational Research and Improvement MATERIAL HAS BEEN GRANTED BY EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) 0 This document has been reproduced as received from the person or organization originating it. 0 Minor changes have been made to improve reproduction Quahty. Points of view or optrons stated in this docu- ment do not necessarily represent official TO THE EDUCATIONAL RESOURCES OERI position or pohcy. INFORMATION CENTER (ERIC)." James P. Sampson, Jr. is Professor and Co-Director of the Center for the Study of Technology in Counseling and Career Development at Florida State University. Appreciation is expressed to Thomas Clawson, Janet Lenz, Robert Reardon, and Sandra Sampson for their comments on initial drafts of this paper. Correspondence should be addressed to James P. Sampson, Jr., University Center, Suite A4100, Florida State University, Tallahassee, FL, 32306- 2490, (850) 644-2490 (voice) 644-3273 (FAX), [email protected], CD www.aus.fsu.eduftechcntr/ C\I 0 0 LE AVAI BEST COPY 2 The Internet and Test Interpretation 2 considerable knowledge or software Using The Internet to Enhance Test Interpretation enhancements accumulate before releasing a new version of the interpretive software. With the Internet, test publishers can update software Computer applications in assessment as the information becomes available, notifying have been in use for over forty years. Main users of recent changes as they access a frame computers made it cost-effective to score, password-protected site, and bill on a per use profile, and produce narrative interpretive reports basis or on an annual license basis. for traditional paper-and-pencil tests. Personal computers subsequently made it cost-effective to The Internet can be used in a variety of add test administration and multimedia elements ways to potentially enhance test interpretation. to the above functions. The Internet is now This section will deal with client preparation for adding the potential for remote delivery of test administration, scoring, profiling, report writing, test interpretation, generalized test interpretation, specific test interpretation, and supervision. All and multimedia functions, as well as adding of the following applications can be potential cost-effective capability in accomplished with technology that is currently communication and links to related information. available. Principles of effective and responsible Client Preparation for Test Interpretation test use are embodied in testing standards Effective test interpretation actually (AERA, APA, NCME, 1985; Joint Committee on begins before a test is administered. Orientation Testing Practices, 1988; & AMECD, 1989) and to testing provides a foundation for delivering a assessment competency statements (Garfield & subsequent test interpretation. Problems in test Prediger, 1994). These standards and competency statements refer to common interpretation can often be prevented if clients (and their parents or guardians, if appropriate) elements of the assessment process that include test selection, administration, scoring and are adequately informed of the purpose and process of testing. For example, the common interpretation, as well as communicating client perception that the "scientific" nature of effectively with test takers and parents or testing will provide an "answer to his or her guardians in the case of minors. In practice, these testing elements can be sequenced as problem can potentially be altered by information delivered during orientation. Intelligent follows: counselors, however, frequently become bored selection with repetitive tasks and as a result are attracted orientation administration to more intellectually challenging and interesting tasks. The problem is that test orientation often scoring involves presenting repetitive information. Since interpretation The focus of this particular paper will be on using computers do not become bored with repetitive the Internet to enhance test interpretation. The information delivery, they are likely to be a more paper begins with a review of potential Internet effective learning resource for general principles of test orientation. Years of experience in applications in test interpretation and concludes with issues associated with Internet use in test delivering computer-assisted instruction could be easily applied to the task of test orientation on interpretation. the Internet. Using a password to maintain Internet Applications in Test Interpretation security, clients could access the orientation at their convenience at home or at public locations, One important potential advantage of such as a public library. The counselor would using the Internet as a test interpretation add any necessary client-specific orientation that resource is the ease with which the interpretive is not covered by computer-assisted instruction. data can be kept current The costs associated with disk manufacture, shipping, and billing, Generalized Test Interpretation makes it expensive for publishers to update Similar to test orientation, test interpretive software as each new enhancement interpretation can also often be repetitive and is available from on-going practice and research. time consuming for the counselor. As a result, As a result, test publishers tend to wait until counselor performance can be compromised. 3 The Internet and Test Interpretation 3 Two negative outcomes may result for the client interpretations to clients. This section will deal First, the client may not receive information they with computer-based test interpretation, two-way need regarding basic terminology used in a videoconferencing, moderated list servers, particular test and what is being measured by computer moderated conferences for group scale and total scores. This lack of basic interpretation, and follow-up resource links. knowledge may make it more difficult for the client to understand and apply the more specific Computer-based test interpretation. interpretive information provided by the Computer-Based Test Interpretation (CBTI) can counselor. Second, if the counselor appears enhance the validity and reliability of testing by bored while delivering basic information, the providing counselors with an expanded and client may misperceive that the counselor is consistent knowledge base for test bored with him or her and the counseling interpretation. Accumulated research data and relationship may be harmed as a result. Even if practitioner experience expands the knowledge the counselor does a good job of communicating base for interpretation, while the standardized basic concepts, less time is available to help a nature of computing contributes to the client gain insights about factors that influence consistency of interpretation. In comparison with his or her behavior and to help integrate insights practitioner-developed reports, CBTI reports gained in assessment into a realistic plan for tend to be more comprehensive and objective behavior change. Using a computer to provide a and less subject to interpreter bias (Sampson, in generalized test interpretation of test results can press-a). Varying types of CBTI exist according help a client to be better prepared for a specific to the type of knowledge base that is used for test interpretation by "being aware of basic the software. CBTI can be categorized as terminology, concepts, and the general nature of descriptive, clinician-modeled (renowned their scores" (Sampson, 1983, p. 294). By clinician type), clinician-modeled (statistical allocating the repetitive computational and model type), and clinical actuarial (Roid & instructional tasks to the computer, the Gorsuch, 1984). CBTI has also been counselor can focus on interpersonal functions categorized into three levels: (a) the statement associated with helping clients understand and level contains data-based descriptions; (b) the apply test results to their individual narrative level adds the judgment of experts in circumstances (Sampson, in press-a). sequencing interpretive statements; and (c) the decision level adds prediction of client behavior Specific Test Interpretation (Lanyon, 1987). Building on the foundation of the generalized test interpretation, specific test Computer-based test interpretation via interpretation adds interpretation of individual the Internet can be used in three different scales and aggregate score profiles as well as modes. When using self-assessment recommendations for action based on test instruments, clients can independently access results. In the case of self-assessment CBTI from password-protected Internet sites instruments, such as the Self-Directed Search immediately after test administration is complete. (SDS, Holland, 1994), the measures are Given that the Self-Directed Search (SDS, designed to be administered and interpreted Holland, 1994) was designed to be used with without input from a counselor. As a result, self- little or no counselor intervention (Reardon & assessment instruments may be delivered on Lenz, in press), the SDS could be administered the Internet by using or adapting existing and interpreted over the Internet In this case personal computer-based interpretations, such generalized and specific test interpretation are as the interpretation for the Self-Directed Search combined for the user. When using counselor- (Reardon & PAR Staff, 1996). Although self- mediated assessment, the client first reviews a assessment instruments can be used without generalized test interpretation, then discusses counselor input, Reardon and Lenz (in press) his or her results with a counselor (face-to-face noted that experience with the SDS has shown or using a videoconference over the Internet), that counselor input enhances the effectiveness and then reviews a specific test interpretation of interpretations. In the case of counselor- delivered from a password-protected Internet mediated (traditional) assessment, the measures site as a homework assignment Some require that trained practitioners deliver test counselors might prefer for clients to review both 4 The Internet and Test Interpretation 4 way videoconferencing over the Internet This the generalized and the specific interpretation as preparation for counseling. In one case, the use of technology may be especially appropriate for clients in geographically remote locations and specific interpretation on the Internet reinforces clients with physical disabilities who could learning that occurs in counseling, while in the choose to receive services at their residence. E- second case, specific interpretation serves as an mail could be used to schedule test interpretation advance organizer for subsequent learning sessions. Documentation of the completion of a occurring in counseling. The current narrative test interpretation, including all test reports and interpretive report for the Strong Interest intervention (treatment) plans, could be Inventory (SII, Hansen, Harmon, Borgen, & automatically added to the client's case notes. Hammer, 1994) could be delivered over the Subsequent client questions or concerns could Internet in this manner. The third mode for be E-mailed to the counselor for immediate delivering CBTI occurs when the principal response or discussion at the next scheduled consumer of test data and reports is the counseling session (Sampson, Kolodinsky, & counselor, rather than the client In this case, no Greeno, 1997). generalized interpretation is provided to the client, and the counselor accesses a specific Delivering CBTI via two-way interpretation from a password-protected Internet site. For, example, the current narrative videoconferencing could be an option that individuals might select who have completed interpretive report for the MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, Hathaway, & self-assessment measures. CBTI for self- assessment could indicate the availability and McKinley, 1989) could be delivered over the potential benefits of two-way videoconferencing Internet in this manner. with a counselor trained in interpreting this By integrating CBTI, multimedia, and the specific test Individuals selecting this option Internet, it will be possible to better attend to could either pay for the time used (as is the case multicultural issues in test interpretation. The for many current telephone help lines for gender, age, race, and ethnicity of the individual computer software) or the service could be paid visually presenting information on test for by an organization with a mission to serve interpretation can be made to match the group specific populations (such as employment membership of the test taker. Keeping group service staff helping individuals make the membership constant should make it easier for transition from welfare to work). Self- the client to relate to and understand the assessment measures are increasingly being individual presenting the interpretation incorporated into comprehensive counseling (Sampson, 1990). Additional multicultural resources, such as computer-assisted career research on test content and test interpretation guidance systems. Two-way videoconferencing can be added to CBTI as the research becomes makes it possible for the counselor to intervene "in the moment," providing access to client available. perceptions and behavior as they occur, instead Much of the current Internet is text of subsequently discussing a client's intensive, data intensive, and structured. These reconstruction of a learning event that has characteristics make it more difficult for the many occurred in the past (Sampson et al., 1997). individuals with limited literacy skills to access and successfully use the Internet. Integrating Moderated list servers. Moderated CBTI and multimedia can make it easier for mailing lists assist individuals with common individuals with limited literacy skills to use the interests to communicate with each other (Offer Internet. Providing versions of test & Watts, 1997). A list server allows sequential interpretations with more video content and less public exchange of text-based information on a text has the potential to help both individuals with predetermined topic among a predetermined limited literacy and individuals with a more visual group of individuals (Sampson et al., 1997). learning style. Messages are distributed to all individuals who have registered to participate on the list server. Two-way videoconferencinq. The lack of interaction in "real time" is offset by Presentation and discussion of test the convenience of being able to view messages interpretations could be accomplished via two- at any time. The participant can choose to post 5 The Internet and Test Interpretation 5 messages or to simply read the available study skill behaviors could reveal links to web messages and maintain a degree of anonymity. sites delivering specific study skills instruction. The moderator is responsible for keeping the Supervision of Test Interpretation interaction focused, halting inappropriate The Internet has expanded opportunities information exchanges, and proactively dealing with potential ethical problems. This resource for delivering supervision, potentially enhancing would allow a counselor to respond to general both the quantity and quality of interaction (Casey, Bloom, & Moan, 1994). The Internet questions about test interpretation or specific can be used to facilitate supervision in several questions about individual test results (assuming that informed consent has been given when ways. A moderated list server could be used as joining the list server). Participants can a form of group supervision, with counselors potentially learn from the interpretive insights requesting assistance for difficult interpretive revealed by the counselor and other members of issues. The moderator could be selected on the the list server. The list server can also provide basis of specific interpretive expertise as well as some social support for confronting issues his or her group facilitative skills. In this case, revealed in test results and in taking action for the role of the moderator would be expanded to positive behavior change. include sharing his or her interpretive expertise and ensuring that the contributions of the other Moderated computer conferences. A participants are appropriate for a specific test. moderated computer conference allows Two-way videoconferencing could also be used simultaneous public interaction among a for individual supervision. The client's case predetermined group of individuals (Sampson et notes (Casey et al., 1994), test results, and CBTI al., 1997). The requirement of adhering to a report could be attached to an e-mail file and scheduled time for interaction is offset by the sent to the supervisor to help with preparation for higher level of interpersonal interaction that is supervision. The supervisor and supervisee possible in "real time." While not identical, the could then discuss a specific test interpretation group dynamics associated with group in depth from remote locations in "real time." A counseling are also operative in computer variation might include adding a consultant from conferences. As a result, the availability of a a remote location to an ongoing supervisory moderator can help keep the interaction among relationship when an unusual interpretive participants focused, ensure that all participants question requires highly specialized expertise. have the opportunity to contribute, halt inappropriate information exchanges, and Issues Associated with Internet Use in Test Interpretation proactively deal with potential ethical problems. A moderated computer conference can serve the same functions as the moderated list server While the Internet applications described described above with the exception that the "real above offer the potential to enhance the access time" interaction available has the potential to to and the quality of testing, issues also exist provide the opportunity to more quickly explore that have the potential to nullify the potential issues in greater depth and to provide a higher benefits of using this technology. This section level of social support. deals with inappropriate use of counselor- mediated assessment, relationship issues, Follow-up resource links. A potentially ethics, credentialing, and counselor training. powerful feature of the Internet is the ability to use links embedded in one Internet web site to Inappropriate use of Counselor-Mediated access related information and services Assessment available at other Internet web sites. Links can As stated previously, counselor- be used to promote additional learning related to mediated (traditional) assessment is designed to test interpretation. For example, an interest include trained practitioners in delivering test inventory interpretation could reveal links to interpretations to clients, while self-assessment occupational information web sites for a client's measures are designed to be self-interpreting. high scoring occupations as a stimulus for career The problem occurs when counselor-mediated exploration. The interpretation of a measure of measures are delivered on the Internet without practitioner intervention. The first problem is the 6 The Internet and Test Interpretation 6 assumption that the results of the Internet- clients who need a more personalized level of delivered version are equivalent in validity to the assistance. Inadequately trained or overworked results of the traditional measure. The second counselors may misuse or become dependent on software, such as CBTI. A lack of counselor problem is the assumption that the written interpretation offered on the Internet is awareness of important location-specific equivalent in validity to the interpretation offered circumstances may cause a counselor in a by a practitioner. Moving the interpretation of a remote location to misinterpret client data or fail to recognize relevant issues. Clients with limited self-assessment measure to the Internet is appropriate if the measure was originally financial resources may have difficulty gaining validated to be self-interpreting. Unless access to the Internet Finally, accessing the validation data are available, the interpretation of Internet from a residence shared with other counselor-mediated measures on the Internet individuals may not provide the auditory and without practitioner intervention is inappropriate visual privacy necessary for the client to (NCDA, 1997). establish and maintain a counseling relationship. Relationship Issues Initial issues associated with computer Videoconferencing and face-to-face networking were addressed in ethical standards interaction have been shown to be similar, but and practice guidelines adopted in the American not identical forms of communication. Association for Counseling and Development Videoconferencing resulted in a more intense (1988), the National Board for Certified task focus and greater participant awareness of Counselors (1989), the National Career their physical appearance in the visual recording Development Association (1991), and the process, in comparison with face-to-face American Psychological Association (1986). interactions (Oravec, 1996). The question, "Is These initial standards on computer networking remote videoconference interaction between a have recently been revised and expanded to counselor and a client in a helping relationship deal specifically with the provision of information equivalent to face-to-face interaction'?" is an and counseling services over the Internet interesting, but not crucial question. Given our current knowledge, the ultimate answer will likely The NBCC/CCE (1997) webcounseling be novideoconferencing and face-to-face standards address links to existing standards, interaction are different forms of communication. confidentiality, supervision, relationship issues, The more important question is, "Does remote release of information, record keeping, self- videoconference interaction between a disclosure, certification and licensure, research, counselor and a client in a helping relationship informed consent, impostor clients and assist clients in understanding and applying test counselors, security, local counseling support, results to solving problems and changing liability, counselor access off-line, inappropriate behavior'?" Development of initial Internet presenting concerns, assessment and intake, applications and subsequent research on communication problems, and relationship effectiveness is necessary to maximize the issues. The NCDA (1997) Internet standards benefits and minimize the limitations associated specifically deal with the qualifications of the with using this technology in counseling developer or provider, access to Internet sites, (Sampson, in press-b). counselor understanding of local environment, content of career counseling and planning Ethical Issues services, appropriateness of the client for receipt Numerous ethical issues have been of services, appropriate local support for the . raised related to delivering assessment, client, clarity of the contract with the client, information and counseling over the Internet inclusion of linkages to other web sites, use of (Bartram, 1997; Sampson, in press-b; Sampson assessment, job posting and searching, and et al., 1997). The confidentiality of client data unacceptable counselor behaviors. transmission and storage of assessment data can be compromised. It is possible to deliver Credentialinq interpretive information on the Internet that is The Internet poses some important attractively presented but inherently invalid. challenges regarding credentialing. At present, it There may be a lack of counselor intervention for is uncertain how state counselor licensure laws The Internet and Test Interpretation 7 will apply to a counselor delivering information choices will increasingly be made on the basis of distance guidance, and testing will likely continue and services out-of-state (Sampson et al., 1997). to play an important role in the guidance The same issue applies to delivering interpretive function. The speed at which these changes will information and services across national occur can be debated, but the general direction boundaries (Bartram, 1997). Also, counselors of the change seems clear. delivering interpretive information and services It would seem wise to carefully experiment with appropriate over the Internet need to clearly indicate their applications of this technology and to proactively credentials, including the complete name of the credential, as well as the name and address of deal with potential limitations while there is still the credentialing organization. Existing Internet time to shape the early adoption of the Internet in web sites often fail to indicate the credentials of testing. the service provider (Sampson et al., 1997). The References potential lack of client awareness of the role of credentialing in protecting the public encourages unqualified persons to offer assessment American Association for Counseling and Development. (1988). Ethical Standards. information and services. Journal of Counseling and Development, 67, 4-8. Counselor Training Preservice and inservice counselor American Educational Research training is essential if counselors are to use the Association, American Psychological Internet effectively to serve their clients. Association, National Council on Measurement Students graduating from counselor preparation in Education. (1985). Standards for educational programs, as well as experienced counselors, and psychological testing. Washington, DC: need to be competent in using Internet search American Psychological Association. engines, familiar with current web sites related to counseling, skilled in evaluating the quality of American Psychological Association. web sites, competent in integrating counseling (1986). Guidelines for computer-based tests and interventions with Internet use, knowledgeable of interpretations. Washington, DC: Author. the process for implementing Internet applications into counseling services, and aware Association for Measurement and of ethical issues and related professional Evaluation in Counseling and Development. standards. Students-in-training and practicing (1989). Responsibilities of users of standardized counselors who wish to take leadership in tests. Alexandria, VA: American Association for developing Internet applications need to Counseling and Development supplement the above preparation with instructional design competencies and web site Bartram, D. (1997). Distance design skills. assessment: Psychological assessment through the Internet. Selection and Development Review, Conclusion 13 15-19. While the move to a paperless society Butcher, J. N., Dahlstrom, W. G., has been less rapid than some futurists Graham, J. R., Tellegen, A., Hathaway, S. R., & predicted, there appears to be inexorable McKinlay, J. C. (1989). Minnesota Multiphasic movement in the direction of increased Internet Personality Inventory - 2. Minneapolis, MN: use. Several factors are encouraging this trend. University of Minnesota Press. First, the cost-effectiveness of computer technology continues to improve dramatically. Casey, J. A., Bloom, J. W., & Moan, E. Second, Internet applications in general are R. (1994). Use of technology in counselor growing exponentially despite the fact that the supervision. In L. D. Borders (Ed.). Counseling majority of Americans still do not have Internet supervision. Greensboro, NC: University of North access at home. Third, the pressure for distance Carolina at Greensboro, ERIC Clearinghouse on learning will continue as the life-long demand for Counseling and Student Services. (ERIC education and training increases and funding Document Reproduction Service No. ED 372 remains limited. Appropriate distance learning 357) 8 The Internet and Test Interpretation 8 Oravec, J. A. (1996). Virtual individuals, Garfield, N. J., & Prediger, D. J. (1994). virtual groups: Human dimensions of groupware Assessment competencies and responsibilities: A checklist for counselors. In J. T. Kapes, M. M. and computer networking. Cambridge, United Kingdom: Cambridge University Press. Mastie, & E. A. Whitfield (Eds.), A counselor's guide to career assessment instruments (3rd. Ed.) (pp. 41-47). Alexandria, VA National Career Reardon, R. C., & PAR Staff. (1996). Development Association. Self-Directed Search Form R computer version (Computer program]. Odessa, FL: Psychological Hansen, J. I., Harmon, L., Borgen, F., & Assessment Resources. Hammer, A. (1994). Strong Interest Inventory. Palo Alto, CA Consulting Psychologists Press. Reardon, R. C., & Lenz, J. G. (in press). The Self-Directed Search: A practitioner's guide to the SDS and other Holland-based career Holland, J. L. (1994). Self-Directed materials. Odessa, FL: Psychological Search. Odessa, FL: Psychological Assessment Assessment Resources. Resources. Roid, G. H., & Gorsuch, R. L. (1984). Joint Committee on Testing Practices. Development and clinical use of test-interpretive (1988). Code of fair testing practices in education. Washington, DC: American programs on microcomputers. In M. D. Schwartz Psychological Association. (Ed.), Using computers in clinical practice: Psychotherapy and mental health applications Lanyon, R. I. (1987). The validity of (pp. 141-149). New York: Haworth Press. computer-based personality assessment products: Recommendations for the future. Sampson, J. P., Jr. (1983). Computer- Computers in Human Behavior, 3, 225-238. assisted testing and assessment: Current status and implications for the future. Measurement and Evaluation in Guidance. 15, 293-299. National Board for Certified Counselors. (1989). Code of Ethics. Alexandria, VA: Author. Sampson, J. P., Jr. (1990). Computer- National Board for Certified Counselors assisted testing and the goals of counseling and the Council for Credentialing and Education. psychology. The Counseling Psychologist, 18, (1997). Standards for the ethical practice of 227-239. webcounseling. Greensboro, NC: Author. Sampson, J. P., Jr. (in press-a). National Career Development Computer applications. In C. E. Watkins, Jr. & V. Association. (1991). Ethical standards. In D. L. Campbell (Eds.), Using tests and assessment Engels (Ed.). The professional practice of career procedures in counseling (2nd. Ed). Hillsdale, counseling and consultation: A resource NJ: Lawrence Erlbaum Associates, Inc. document (2nd. Ed.) (pp. 26-33). Alexandria, VA Author. Sampson, J. P., Jr. (in press-b). The Internet as a potential force for social change. In National Career Development C. C. Lee & G. R. Walz (Eds.). Social action: A Association. (1997). NCDA guidelines for the mandate for counselors. Greensboro, NC: use of the Internet for provision of career University of North Carolina at Greensboro, information and planning services. Alexandria, ERIC Clearinghouse on Counseling and Student VA Author. Services. Offer, M., & Watts, A. G. (1997). The Sampson, J. P., Jr., Kolodinsky, R. W., Internet and careers work. NICEC Briefing. & Greeno, B. P. (1997). Counseling on the Cambridge, United Kingdom: National Institute information highway: Future possibilities and for Careers Education and Counselling, potential problems. Journal of Counseling and Sheraton House, Castle Park, Cambridge CB3 Development, 75. 203-212. OAX. AVAILABLE 9 BEST COPY 01/27/1939 08: 45 02 PAGE 1 ERIC-,CM4 141N-2.6-99 03!12 PM 336 S34 411C, P . 2 LERIC1 U.S. Department of Education of Educational Research end improvement (OER1) Offiae National Library of Education (NI-0) Educational Resource., information Center (ERIC) REPRODUCTION RELEASE (Speoifio Dociument) DOI MEW IDENTIFICATION; t. wW" TO: 11.111111111111, AUtticul Con= I Spume; IL RE RODUCTION RELEASE: mg If to afterninats es widely es possible timely elonlIkent materlsle of Murat to the educations! community, documents announCed In the (RIE), are usueny made available to users In mIcroliche. reproduced paper copy, Educatkin moony Mogi Aims, of the ERIC system, Reaourses m end 0. eto medic and mold through the ERIC Document OtsrproductIon Service (EORS). Credit IS Elven to the source Ouch EOCWROnt. end, it mprodu on release I. granted, one of the following nom la sf9xed to Mc document. itulon fg cpentcd to roprodups and disseminate Me identified document. please CHECK ONE of the rotiowIng three ottani end sir st its bottom If do I. elms IN The mOW MVO MOW Ilhoon MI* Th Nos stkew stew 041ow an be 710112MIO ASV Mao tiolow AI Ow Ma. 01201antil Mad to 40 la* t4 102~10 IM20112 to 40 Lael1 d FertremidION TO AlFROPuOf PERMISsIoN TO *PRODUCE AND DIESEMNATE TAO MATERIAL IN 1133!ON TO REPRODUCE AND 0110 DISSEMINATE THIS WATERIAL N MICROFICHE, ANO IN ELECTRONIC MEDIA oui %MATE THIS VAITRiAL HAS MICROFIC14; ONLY HAS BEEN GRANTED BY FOR ERIC COLLECTION EUBECRIEERS ONLY. SEEN GRANTED SY 1414 SEEN GRANTED SY =n1.1,11j ,11111176e To ME soueariONAL, itnotiRed; TO The 'EDUCATIONAL RESOURCE'S G COL/DATIONAL RESOURCES TO 1 INFORMATION CENTER (ERIC) INFORMATION CENTER (ERIC) ORNATION CENTER (MC) II 2B 2A 11101101, Wel CA Level 29 LIMO 1 1 LI /ED if lawl I rills, HAfflipg mrsiodten OW her Gag hors a Lail St Warn. omoNIA0 molsOLICOM DINS Mei IN L441 20 /Mem powgrInit wan dloWNItra lii nuIenifleis ind In costronio mos ROMMOlis or NW ERIC/MUNI tworoduotm 100 elaornInsan In ifkouldm alto (s.e., atortiald Anima Qua St SRC read mobenim a:WOW' only nw Pnyombp u..th tiu pmemend i.tiidInuè.d aw111441/444roduellew quality omiWi& MO:bails Mocked, dNuillmS WE OS Organs/ It Levy! poloOmIon to rep(otImMONAME, 1, (WC) nonexclusive permission to reprockae end dlasemlnete this document Intimation Centsr I hereby want to the Ectucetiornsi messurcee ea lndloated above, lfeproduutlen born the flfIC inicroitche or election4 media by persona other Man ERIC employe°. and Ifs systsm copysioht holden exception is mad* ltv nonwoftt reproduction by Wedge end other service men= otcrirsofors requires permicskIA him tne satisfy Infametfan needs a/ educators In response to discrete Neulties, hr nwrivrn9.kbreo.: Sago JR. P. .1A.mes P. SAAAPSota, hem, tudoNAtiorar atass4 dekrtedt S4.ct14E- AR1404o (A0 ta4/14% * title P1.00610 101LIA611,s a