AD-A249 995 AD 111I!1 1A ;llillllflW E ~lllliil~~1111111i MIPR NO. 90MM0528 TITLE: CONTINUED PERFORMANCE ASSESSMENT METHODOLOGY (PAM) RESEARCH (VORPET) 3 SUBTITLE: Refinement and implementation of the JWGD MILPERF- NAMRL Multidisciplinary Performance Test Battery (NMPTB) PRINCIPAL INVESTIGATOR: Efrain A. Molina CONTRACTING ORGANIZATION: Bioengineering Division Research Department Naval Aerospace Medical Research Lab DTIC Pensacola, Florida 32508-5700 S= CT REPORT DATE: January 14, 1992 A TYPE OF? REPORT: Annual/Final Report PREPARED FOR: U.S. ARMY MEDICAL RESEARCH AND DEVELOPMENT COMMAND Fort Detrick, Frederick, Maryland 21702-5012 DISTRIBUTION STATEMENT: Approved for public release; distribution unlimited The findings in this report are not to be construed as an official Department of the Army position unless so designated by otheL authorized documents. 92-12461 (cid:127),,... .(cid:127)Illii IIi II III 1I H REPORT DOCUMENTATION PAGE FoMormB ANop pr0ov 0e4d-0,88 Pubhc relx)rtlng burden (cid:127)or thls'(olledion of =nforrnat(cid:127)on ,s e(cid:127)tlmated to average t hour per response, m(ludmg the time for reviewing instructions, searching e(cid:127)stln9 data ",Ource(cid:127). gathermg and maintatnm9 the data needed, and (On.Dieting and reviewing th • (e(cid:127)i(.(llon of Information Send comments regarding this burden estimate Or any other aspect of thl(cid:127), ¢ollex'(cid:127)lon of Inlormauo,", including sugge'(cid:127)Uon', Ior re-duclng th,$ burden to Wa(cid:127),hmgton Headauarter$ Services. DlredOtat.e for Inforl'natlon O&erahon$ and Repodk 12 15 Jefferson Oavl(cid:127), Highway, Suite 1204, Arhngton, V(cid:127) 22202-4302, and to the Off<e Of Managelnent and Budget, Pa!(cid:127)'rwork Reduction Pro t=¢t (0 ;'04.01 &8), Wahlngton. DC 20503 4. AGENCY US{ ONLY (Leave (cid:127)an.k) 2. REPORT DATE 3. REPORT=TYPE AND 'DATES COVERED" 14 January 1992 A(cid:127)nual/Final Report (10/1/89 - 9/30/.91) 4, TITRLEe fAiNnDe mSUeBnTItT LEa nd Implementation of the JWGD3 MILPERF-;;;4.(cid:127)L! S. MFUINPRDI NNG oN.UMBERS Multidisciplinary Performance Test Battery (NMPTB) 90MM0528 0603002A 6. AUTHOR(S) 3M263002D995. BG. 047 Efrain A. Molina WUDA331006 7. PERFORMING ORGANIZATION NAME(S) AND'ADDRESS(ESi 8, PERFORMING ORGANI(cid:127);AI"ION REPORT NUMBER Naval Aerospace Medical Research Laboratory Pensacola, Florida 32508-5709 9. SPONS(cid:127))RING/MONITORING AGENCY Na.l(cid:127)'l:TS) AND AD'DRE'SS(ES) 10. SP6(cid:127)SORING / M'ON'ITORING AGENCY REPORT NUMBER U.S. Army Medical Research and Development C(cid:127)m(cid:127)nd Fort Derick Frederick, MD 21702-5012 11, SUPPLEMENTARY NOTES ...... Annual covers the period of time December i, 1990 - September 30, 1991. 12a. DISTRIBUTION i AVAILABILITY STATEI(cid:127)IENT ... .. 12b. DISTRll3UTION CODE A[43roved for public release; distribution unlimited 13. ABSTRACT (Maximum 200 words) ............. .The Vestibulo-Ocular-Reflex Performance Evaluation Test (VORPET) developed at the Naval Aerospace Medical Research Laboratory gives a measure of left- ana right- directed gaze shift threshold time used to assess the type of head/eye coordinatlon relevant to the aviator who routinely makes large (cid:127)hifts in gaze during his scan of cockpit instruments and the ouBside environment. Automation of the VORPET (cid:127)equires %he use of a voice recoqnition system for the collection and scorinq of ,Be subject's voice response. We tested the accuracy of the VOTAN voice recognition system in collecting subject's voice response to that of the present me, nod that uses a test administrator to listen and hand enter subject's responses when aaminis(cid:127)errng the VORPET. Thirty-six subjects were adminis£ered the VORPET test under three different conditions (a) direct viewinq of the stimulus digits presented on the CRT, involving no head movement. (cid:127)b) VORPET ad[ninistratron using a test operator for subject's voice recognihion and manual aata entry, and (c) VORPET adminlstration using the automatic voice recognition system for subject's voice acc(cid:127)uisition and recognition. Four, three, and two digits were used as visual stxmulus for each method used. The hypothesis to be tested was that there would not be a significant statistical difference in the results when methods (b) and (c) were used. Analysis of variance of test results indicate that there were significant differences Between the thresholds obtained when method {b) and (c) were used to administer the VORPET. The VOTAN automated voice recognrtron system cannot be used to automate the VORPET. Present speed and accuracy of automated voice recognition systems still need additional technological advancement or improvement in order to replace hhe Present "human based vol.ce .recoqnition s[stem.(cid:127)' " - - 14. SUBJECT TERMS lS. NUMBER OF PAGES VORPET, Gaze Shift, Scan, VOTAN voice Eecognition, (cid:127) V, MIPR, Performance, Test 4.6. PRICE CODE 17. SECURITY ((cid:127)LASSIFICATION '18. SECUI(cid:127)ITY CLASSIFICATION 19. SECURITY CLASSIFICATION" 20. LIMITATION((cid:127)FARSTRACT OF REPORT OF TH!S PAGE OF ABSTRACT Unclassified Unclassi f ied Unclassified Unlimited NSN 7540-01-280-5500 Standard I:orm 298 (Rev 2 89) Pt,'%(cid:127)rt(cid:127) by AN%, %Id ; 4'I "b 2'Jl%1.0 2 FOREWORD Opinions, interpretations, conclusions, and recommendations are those of the author and are not necessarily endorsed by the U.S. Army. _Where copyrighted material is quoted, permission has been obtained to use such material. Where naterial from documents designated for limited ds-stribution is qruoted, permission has been obtained to use the material. __Citations of commercial organizations and trade names in this report do not constitute an official Department of the Army endorsement or approval of the products or services of these organizations. in conducting research using animals, the investigator(s) adhered to the "Guide for the Care and Use of Laboratory Animals," prepared by the Committee on Care and Use of Laboratory Animals of the Institute of Laboratory A-nimal Resources, National Research Council (NIH Publication No. 86-23, Revised 1985). _For the protection of human subjects, the investigator(s) have adhered to policies of applicable Federal Law 45CFR46. In conducting research utilizing recombinant DNA technology, the investigator(s) adhered to current guidelines promulgated by the National Institute of Health. PFincipal Inve:§7ator's Sirnature Date NTIS 64, F DTIC TAB Unminounoed I Justif leation_ A-a la'bility Codes ....... '.Dist Vspaueoti llaurIdo TABLE OF CONTENTS INTRODUCTION ...................... ........................ 3 METHODS 5 ME'I'ODA......................................................... Test Administration................ 5 VORPET's Flow Chart Diagram .......... ............... 7 Experimental Design .......... .................. 8 Hypothesis ..................... ...................... 8 RESULTS AND DISCUSSION ................ ................... 8 RESULTS ...... ........ 12 Right direction gaze thresholds. . . . . . . . . ... 12 Left direction gaze thresholds ........ .... .. 22 Average left and right direction gaze thresholds 32 DISCUSSION ................... ....................... 42 CONCLUSIONS ................. ......................... 43 REFERENCES ............................................. 43 2 N 1. INTRODUCTION This Final Report covers work done at the Naval Aerospace Medical Research Laboratory during fiscal years 1990 and 1991. The Joint Working.Group on Drug Dependent Degradation in Military Performance (JWGD MILPERF) was established for the purpose of :eveloping and testing procedures to evaluate the effects of chemical defense pharmaceutical agents on military performance. The JWGD3 MILPERF has sponsored the performance assessment research effort at the Naval Aerospace Medical Research Laboratory (NAMRL) that has evolved from independent contributions by various scientific disciplines into an organized multidisciplinary test system called the NAMRL Multidisciplinary Performance Test Battery (NMPTB). The NMPTB has been designed for assessing aviator performance from a five-discipline perspective (i.e., cognitive, vestibular, visual, auditory, and physiological). Other products of sponsored research efforts from NAMRL are computational models of human performance inr operational tasks and in laboratory performance tests. The Vestibulo-Ocular-Reflex Performance Evaluation Test (VORPET) is one of the major component tests in the NMPTB. It allows assessment of the type of head/eye coordination that is required in a variety of military weapons systems (especially aircraft). It is most relevant to the aviator who must make large shifts in gaze during his scan of cockpit instruments and scenes in the outside environment such as horizon, targets, and wingmian position when doing formation flying. In a JWGD MILPERF sponsored validation study, the VORPET results indicated degradation of performance as a result of antihistamine administrations, indicating that the VORPET may be useful as a sensitive NMPTB performance test. The test requires the subject to directly face one display that provides visual fixation, and upon presentation of a simultaneous visual and auditory cue, to rapidly rotate his head toward the second display. At the onset of the cue, four digits are presented for a brief exposure time on the second display; the subject task is to call out as many as possible of the four digits that he can correctly identify (presently, the subject response is manually entered into the computer's keyboard by the operator). The exposure time of the second display is appropriately adjusted (increased or decreased) according to the accuracy of response of the previous trial. The procedure is repeated for a number of trials with the sole of the first and second displays sequentially interchanged resulting in bidirectional gaze shifts. The end result is a single threshold time for ea-;h direction of oaze shift, and a grand mean threshold time based on the simple mnean of the left- and right-directed gaze shift thresholds. 3 The main objectives of this research effort for the years FY90 and FY91 were: (a) To complete and refine the automation of test instructions and computerized scoring of the VORPET test using voice-actuated subject response and test instructions. (b) To construct two to four (depending on cost, space, and acoustical isolation requirements) automated VORPE"T test stations and install them in NAMRL's Mobile Field Laboratory trailer #5 after appropriate modification of the trailer required for additional test stations. (c) To conduct baseline validation studies such as test/retest reliability studies. (d) Perform systematic follow-on studies to incorporate appropriate additional tests derived from the NAMRL Multi-disciplinary Performance Test Battery, which has been developed for assessing performance effects of chemical defense antidote/pretreatment drugs. In order to accomplish objective (a), the VORPET(1) test initially designed and implemented to operate in the BASIC language running on a Hewlett-Packard based digital computer was upgraded to operate in a Zenith Z-248 environment, and adapted to use a voice-recognition interface in order to automate test administration. The automated VORPET was then compared to the manual VORPET method of data entry by the test operator. Statistical analysis of the data collected froin both methods and a thitd static VORPET test indicate that the automated VORPET test does not yield the same threshold measurements and that it will introduce unknown effects. State-of-the art tachnol - in voice recognition needs further improvement. Higher data acquisition sampling rate and more efficient algorithms are needed in order to provide the accuracy required. The use of IBM, and IBM compatible computers like the 386 and 486 systems with a much higher rate of instruction execution (33 MHz) in combination with faster Digital Signal Processors (DSP) to acquire and analyze human voice responses (50 MHz), and more efficient algorithms should make acquisition and analysis of rapid voice responses more accurate. The other listed milestones were not completed. Further work needs to be done in the area of digital signal processing. There is a need for the use of voice recognition systems in computer-based testing similar to the one needed to incorporate the VORPET test as part of the NMPTB. 4 2. METHODS The VORPET initially designed and implemented to operate in the BASIC language running on a Hewlett-Packard based digital computer was upgraded to operate in a Zenith Z-248 environment. The VORPET software programs were converted from the BASIC language to the C language and transported to run in Z-248 based IBM AT compatible computer. The programs were modified to incorporate the VOTAIT automatic voice-recognition system in order to automate test administration. Implementation of the software program for administration of the VOR.2ET was done in the C language for two main reasons: (a) code compactness, and (b) all the library routines to interface to the VOTAN voice-recognition interface are available only for the C language. Next, the accuracy of the new automated VORPET test was compared to that of the manually administered version. An experimental test design was prepared to measure the left- and right-directed gaze shift threshold times (in milliseconds) under three different test conditions: Method 1. VORPET test administration with the subject directly viewing the stimuli digits displayed on a CRT, with no head/eye motion required, and using only the vz'ice recognition system to collect the subject's responses. This method was considered to be a static VORPET test, to give a measure of accuracy of the voice recognition interface system. Method 2. VORPET test administration using the manual method where the subject's voice response collected by the test operator is used to determine the thresholds. Method 3. VORPET test administration using the VOTAN voice recognition system to collect and evaluate the subject's voice responses to determine the thresholds. The VORPET under the three different test conditions was administered using two, three, and four stimuli digits. Both the operator's manually entered response and the responses collected by the voice-recognition system were recorded in methods 2 and 3. Thirty-six naval aviator students waiting for flight training from the Naval Aviation School's Command participated in the experiment. Figure 1 shows the flow-chart diagram of how the VORPET test was administered. The test was configured in this manner to facilitate execution of the experimental design described below. Test Administration. As soon as hardware and software initialization is completed, prerecorded voice instructions are 5 __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ given to the subject by the VOTAN voice-recognition system (the voice recognition system can be programmed to provide voice instructions or messages) about the test to be administered. Next, subject and test-related information are entered by the test operator. Prior to its use to capture and to record the subject's voice responses, the VOTAN voice-recognition system is trained to the subject's vocal responses (the numbers 2 through 9 in the case of the VORPET test) that are used throughout testing. Instructions are given to the subject about the training period. At the end of VOTAN's voice-recognition training, a test or evaluation of the training is performed. Ins\ructions are given to the subject about the evaluation. At the end of the training test. a summary of percentage accuracy for individual numbers 2 through 9 as well as in groups of four numbers presented in random order is displayed on the CRT screen for the operator to see. Results are also printed and stored on magnetic media. The operator is then queried by the program about the need of repeating the training/evaluation. If the operator responds affirmative, training/evaluation is repeated, otherwise the VORPET test is then administered. There were three methods of VORPET test administration: 1 - No Head Movement Mode. In this mode, the stimuli numbers are displayed on the CRT screen. which the subject views with no gaze shifts required. In this mode, the voice- recognition system is used to record the responses and to interpret the results, which are used tn determine the gaze thresholds (VORPET thresholds). 2 - Manual Data Entry Mode. In this mode, the stimuli numbers are nresented in the standard fashion that the VORPET test is administered. Both voice-recognition system and manual data entry by the operator are used to record the subject's responses. However, only the manual data entered by the operator are used to determine the VORPET thresholds. 3 - Voice Recoqnition Mode. In this mode, the stimuli numbers are presented in the standard fashion that the VORPET test is administered. Both *voice-recogniticn system and manual data entry by the operator are used to recor:d the subject's responses. However, only the responses collected by the voice recognition system are used to determine the VORPET thresholds. The final f'(cid:127)rm of the VORPET will be the same as that shown by the flow-chart in Figure 1, with decision making for retraining of the voice-recognition system to be computer automated. 6 Start Test - Initialize variables. - Initialize display data. - Input subject information, - Initialize voice systeM. - Instructions to Voice Recognition training. Start of training, - Instructions to test Voice Recognition training done. Start oft esting. Yes eet < - Administration of VORPET test: i-No head MoveMent, 2-Manual data entry Mode, 3-Voice recognition mode. End Test Figure 1. VORPET Flow-Chart Diagram. 7 Experimental Design. Thirty-six subjects were administered the VORPET under each of the test modes described above and for four, three, and two stimuli digits. The sequence in which the three different modes were administered was randomized but balanced. The 36 subjects were split into 3 different groups. Seven subjects had missing test data. Hypothesis. The main hypothesis to be tested is that the mean of gaze thresholds for methods two and three are the same: H : I(cid:127)Li = (cid:127) The mean of the gaze thresholds are the same. 0 H : (cid:127) The mean of the gaze thresholds are different. 1 and that the introduction of the automated voice-recognition system does not alter the measurements of both right- and left- gaze thresholds. Each subject was given a total of 32 trials each time the VORPET was administered, with a 30-s rest interval at the end of the 16th trial. Tables I, II, and III illustrate the right-gaze threshold means, the left-gaze threshold means, and the grpnd-gaze threshold means (left and right threshold average), respectively, for each of the 36 subjects tested using methods 1, 2, and 3 described above, when 4, 3, and 2 digits were used as the stimuli. 3. RESULTS AND DISCUSSION The VORPET provides a measure of the right- and left-gaze threshold as well as the "grand mean threshold" defined to be the average of both the right- and left-threshold every time a subject is tested. The data tabulated in Tables I, II, and III were analyzed with the aid of the SuperANOVA(2) Statistical Package run on a Macintosh digital computer. Analysis results include Anova Type III Summary Table, Modeling, Interaction Plots, and a Type III Manova table of the following: (a) the variables Four-Digits, Three-Digits, and Two-Digits as a function of the methods used, namely, Method-i, Method-2, and Method-3. (b) the variables Method-l, Method-2, and Method-3 as a function of the number of digits used as stimuli, namely, Four- Digits, Three-Digits, and Two-Digits. 8