Technical Report 1141 Coding Verbal Interactions In A Prototype Future Force Command And Control Simulation Paula J. Durlach U.S. Army Research Institute Laticia D, Bowens and John-L. Neumann University of Central Florida Consortium Research Fellows Program Thomas J. Carnahan ‘Western Kentucky University Consortium Research Fellows Program Jury 2004 20040315 014 United States Army Research Institute for the Behavioral and Secial Sclences Appvd for pubic releese; dstibuion i uninited U.S. Army Research Institute for the Behavioral and Social Sciences A Directorate of the U.S. Army Human Resources Command ZITA M. SIMUTIS Director ‘Techaical Review By Wiliam R. Sanders, ARI Donald R. Lampton, ARI NOTICES DISTRIBUTION: Primary cistritution of this Technical Report has been made by ARI. Please address comespondence concoming distribution of raporis to: U.S. Army Research institute for the Behavioral and Social Sciences, Atin: DAPE-ARLPO, ‘001 Eisenhower Ave... Alexandria, VA 225044844 FINAL DISPOSITION: This Technical Report may be destroyed when itis no longer needed. Please do not return itto the U.S. Army Research Institute for the Behavioral and Social Sciences. NOTE; The findings in this Technical Repost are not to be construed as an official Department of the Army position, unless go designatad by other authorized documents. REPORT DOCUMENTATION PAGE ee en 2 gare conte a UFint ‘ee Soe aon | etna vert rains in frye tre Fre Sorta aid Gal Sndaten es same TARR roma ER Pata J Ouich (US Any Research itch tian. Fae Sores lain Neaneirumoom econo, 2 [ME ‘Thomas J, Camahan (Western Kentucky University). ‘Se, WORKUNIT NUMBER | ERR SRS Oia ORES 7 NG ORO 5. iy ie Rane anes oe aes a oecere on (8. SPONSORINIATINITORINE ABENCY RAME(S, ANT ANTRESS(ESH “20 MONFTORACRONYW ~ jgiuprocmacr mar trosgusaes a sees. " ‘Technical Report 1741 “BE TRGITIOWAVALABIETY STATEMENT Approved for public release; alstibatons unimiied ".SUPPLEWENTARY NOTES, 7. ABSTRACT per 260 wa Aste U.S. Amy undergoes traneformation, wil nsec reliable means of measuring and alung comping new Sts. Acauing those wil require analysis of hymen behaslor inthe context of human-n-theoop simulsions of Future Gombal System: (FCS) atl in te concept exsloration phase, The gnal ofthe present eat vas to estabish and measure command gravp behsvor observed in ugh humar-mthe ong simulation via analysis cf he verbal ‘traction of the command group. A coding scheme for command group verbal inferations was devised and pftad to several simulation runs In which a command group of 4 experienced Aimy Lt Cal's contoled an array of Sraulated unmanned sir and ground platorms to aocomplish s deizerate atack mission, This report decurents the development of te ceding scheme, the analysis osu, and considers varaus cer agpranshes for analyzing ‘ateal data. Tig Suncor TERNS 5, Vorol Anais. Verbal Cong, Command an Canto, Batofd Fentone, WETT-TC, Rabat, Fue Fose, AIS [SERRE Ty. aasTmE [STAI PRGE assTmACT orences Uneiseited | Unclassified | Unelassites : United Pula J. Durtae i | 497/384-9983, ‘Technical Report 1444 Coding Verbal Interactions in A Prototype Future Force Command And Control Simutation Paula J. Durlach U.S. Anny Research Institute Laticla D. Bowens and John L. Neumann University of Central Florida Consortium Research Fellows Program Thomas J. Carnahan Westem Kentucky University Consortium Research Fellows Program Simulator Systems Research Unit Stephen L. Goldberg, Chief U.S. Army Research institute for the Behavioral and Social Sciences 5001 Eisenhower Avenue, Alexandria, Virginia 22304-4841 January 2004 inal PropoCT NUMIBSP Personnel Performance 20262785A760 and Training Technology ‘Approved for subi retaze; deteution untites FORFWORD Asthe US. Army undergoes transformation, members of the future force will need to acquire ccmplex new skills in an enviconraen! of pid dectrine, organivaiion, traning, materiel, leadership, petsannel, nd facies changes and deervased deployeent reparation tine, A. good understanding of the required Command. Cantcol, Communications, Computers, Inielligence, Survcillanoc and Reconnaissance (C'4TSR) skills and a relinble- means af measuring and teining therp will be essential, Acquiring chose wil roquire analysis af hnman bebavioe io the context of humat-in-tve-loup simulations of Future Caraat Systems (FCS) sil in the conszpt exploration phase, This report documents the development of a coding scheme lo: the analysis of eowmand ‘group verbal interactions in suc a hurtus-in-tho-loop simplation. Command group members ‘were asked to “think out Loud” so that their thinking procostes could be eaptured atnong the ‘verbal interactions, The anlysis oftheir verbal interactions yielded a mewurement instrument ‘that cau be used to help estublish und assess eiuing ofthe CaTSR skulls required by the Faturs Sime ‘fhe work described in ihis report isa portion of the resvarch pouformed under ihe FCS Command and Contrel (C2) programa, lsd by the Defense Advanced Restatch Projects Agency (DARPA) and the U.S. Army Communisntions-Elecconics Command] (CECOM) Research and Develapment and Fnginecring Center (RDC). As aprogram partuer, the TLS, Amity Research, institute for the Behavioral ang Social Sciences (ART) focused on measuring harman performance to improve human system integrative, and tnining and to support the Seience and Technolagy Objective (STO) titled “Methods and Measures of Commander-Centsic Training,” Findings from this effort vere briefed to the Deputy Chicf of Staff for Operations and ‘Training from the ‘Training and Doutrine Command (TRADOC NCSOPSBT). Methods and findings ftom cach of the four experiments vere provided tothe Progrum Manager (°M) for FCS ‘C2as par of ARTs angoing support to £CS and Army rescerch and development efforts. ‘Hum porfomianee findings hy ART helped shape the C2 prointype showeased inthe FCS, Capstone Demonstration of C2 systems prior to the FCS Milesiome Ts devision, Pit bd lh - MICHAL G. RUMERY ‘Acting Technical Director ACKNOWLEDGEMENT ‘The authors would like to acknowledge the gnidanee und expertise prvvided hy Car] Lickteig during the curse ofthis research. CODING VERBAL INTERACTIONS IX 4 PROTOTYPE FUTURE FORCT: COMMAND: AKDCONTROI. SIMULATION EXECUTIVE SUMMARY ‘Resoich Requirement ‘As the U.S. Acmy undergoes transfozmiation, members uf The fate force will need to squire complex new skills in an environment of rapid doctrine, organization, ruining, matoticl Jeadership, personnel, and fuclities changes and decreased deployment preparation time. good understanding of vie required Command, Contra, Communications, Computers, Intelligence, Survetllance and Reconnaissance (C41SK) skills and a reliuhle means of racasuring und teuining them wall be essential. Acquiring those will require analysis of umn behavior in the cantext of tumain-the-loop simulations af Future Combat Systems (PCS) stil in the concept exploration ahuse, The FCS Command and Conteat (€2) program provides a working, example of sucht a simulation. ‘The progeum lends are the Defense Advanced Research Projects ‘Apeney (DARPA) and the U.S. Army Commorications~Electran'cs Commasil (CTCM) Research and Development and Eueineering Center (RIDEC), As a prograra partie, the U.S ‘Ariny Research Institue for the Behavioral and Social Sciences (ARI) focused on measuring human peeformance to improve human system intearation and training anu io supper the Scicnce and ‘leshnotogy Objective (STO) tiled “Methods and Measures af Commmander-Centiie Training.” Procodhre ‘The FCS C2 program created a transformation envvirenment for canpitical assessment oF command group performance at th: small ont level, the notimal Unil Cell, From Ostober 2001 fo March 2003, the program conducted u series of communder-in-the-foop experiments 31 Monmouth, NI. Over the course ofthe experiments the digital command and contil sysier— the Commandes's Support Envirocment (CSE)~became increasingly saphistcated, allowing the command group (of 4 people) to directly control £3 simulated unmanned si and grou whicle platforms, and z sot of unmanned ground sensors. The mission was consistently set in simulated desert terrain fiom the National ‘raining Center (NPC) and required the Linit Cello condnct Adaliborate artic missions agsinst an encmay battalion to clear pasnaye lanes fru follow-ent Tore. ‘The ARI partners developed! analyses of critical cammand group functions, und described player reuctions such as workload assessment, and CSE strengths and weaknesses. ‘The dats lor this work consisted of the verbal inferactions among the experiment participants im the Unit Cell, {heir computer in‘eraction activites, ané thir responses to questionnaires. ‘The prevent report is concerned specifically with documenting the voding scheme used to analyze the verbal interaction data. Unit cell members were asked 0 “chink out loud” xo thal their thinking processes could be captured among the verbal incractions, ‘The verbal interactions: wf Une Tie Cell far selected experimental sessions were runseribed and evnlent was anatyed for Function (Plax, Sec, Mave, Strike, Battle Damage Assessment), Source {Who was speaking), Type of communication (e.g., question, declaration, ete.) and Factor (Mission, Enemy, Tereuin, Troops, ‘Time, Civilians). Ag well as dacumenting the cnding scherne and presenting iMustative cesults, is reporcalzo includes recommendations for further refincment of the coding scheme. Findings: A striking finding ftom the analysis of verbal interactions arog the Tinil Cell members ‘a5 flat sharing of infomation was consistently the most frequent type of verbal interaction (as opposed, e-., t© questions or ditections). Despite the fact that al Unit Cell members did have access to the sam dative af information, it as still essential to verbalize abot data that were observed in order to mainizin a common operating picture. Analogously, Sex was always the ‘most quently observed Funetion (defined mx “Detect or identify enemy or fiendly positions, or significane terrain aspects"). Thin result has significant implications for communications zrequireutents for the FCS, intended to be a netocnaric system of systems supporting « distributed force, Speeificaly, ane itaplieation is that a nctwarked information system, allowing all users to aocess all inrmation may still aot be sufficient ta mnfain a common operating picture. uc te oT Uve data available, aud the fimitations on the human information processing 3g woers to fhe must relevant information is going to be required. Inthe present researc, this was accomplished by participants telling one unother whan they Trelieved they should kaow, It should be kept in mind however, that participants were instracled ‘n“think aul loud.” This may bave distorted the type of verbal inweactions observed, compared ‘with panticipents not no instructed. From a methodalogical point of view. there were seversl findings. workuble ending, scheme for analyzing command group vorbal teractions was established. This scheme proved sensitive enough te diseriminste the verbal behavior of an expert vs, a noviee command group, and to discriminate the verhal behavior observed in mission exceudion ws. mission planing. Several issues were tackled. such ay hw [inely fo analyze the verbal data (wast cconstintes a data point?), The issues iavolved in developing a coding scheme for verbal data, ‘were explorell und the alvantagcs and disadvantages of different approaches were illustrated. To ‘panicolas, we found a trade-off between the prawularity of analysis and the ability to capuure in ‘ended data, verbalizations reflecting command considerations and mnoce abstiact thinking and cision making, In the final conclusion, itis suggested thatthe most useiul coding scheme ‘ru he une specifically tallored to support hypothesis-driven research, Utilization of Finding: ‘This work established a coding schome for mialyring the verbal inveractions of a command group duting mission planning and execution. This meesurement tool can now be ‘used funther to measure, help establish, and ass0ss tsining of the C4TSR skaila required hy the fusute fore,