T E C H N O T E S Encryption System for Supporting Hard Real-Time Distributed Testing Kenneth G. LeSueur Subsystem Test and Analysis Branch, U.S. Army Redstone Technical Test Center, Huntsville, Alabama Trent Woods and Jack Carter ERC, Inc., Huntsville, Alabama I n the arena of distributed testing, there is outto40 km,whichisthetargetrangefortheRTTC a subset of applications that require hard real- solution.The RM system usedfor thisapplication has time distributed interfaces to accomplish the a400 nsnode-to-nodelatencyandhasa43–174 MB/s intendedmission.Withtheincreaseuseoffiber bandwidth depending on packet size (4–64 byte opticsatmanyinstallations,theabilitytoperformhard packets). The system can have a maximum of real-time distributed testing has become possible, that 256 nodes. is,forallthosethatarenotrunningclassifiedreal-time Most facilities/ranges that have classified operation operations.Thispaperpresentsthedesigninformation areas and require external connectivity typically do so and latency test resultsfrom a system developed at the by using Internet Protocol (IP)-based encryption/ RedstoneTechnicalTestCenter(RTTC)thatenables decryption systems and networks. This approach is distributed real-time classified testing applications. widelyutilizedformanystandardinterfaceapplications Hard real-time applications are those that have but cannot meet the demanding deterministic latency a given time period to complete an operation, and if requirements needed for the stated class of hard real- thetimelineisnotmet,theresultsareinvalid.Tomeet time target applications. the timing constraints of distributed hard real-time applications, the interface or network between the Solution separated applications must operate in a deterministic The newly developed RTTC system incorporates manner. The classes of hard real-time applications a high bandwidth, low latency encryption system with targeted by the new system are those that require long-haul, single mode, fiber-optic interfaces and has a round-trip latency of less than 500 microseconds plain text interfaces for RM systems, RS-422, low (msec)andneedtobeencryptedbyaNationalSecurity voltage differential signaling (LVDS), and emitter Agency (NSA)-approved class-1 encryption system coupled logic (ECL). The system is utilized in pairs, prior to exiting the controlled (classified) areas. This and each pair consists of a computer equipped with latency time requirement includes the encryption/ aRMinterfacecard,aKG-95encryptor,andacustom decryption times, speed of light over a 50 mile real-time peripheral component interconnect (PCI) round-trip, and all computer/electronic interface interfacecardthattranslates desired input formats into translations. properprotocolsforconnectiontotheplaintextsideof the KG-95 encryptors. The cipher text side of the Existing capability encryptorisinterfacedtoafiber-optictransmitterboard Many real-time facilities/ranges utilize reflective that translates the electronic connection to a single memory systems (RM) for intersystem communica- mode fiber-optic signal for real-time communication tions. This is especially true for hardware-in-the-loop betweentheremotesites.Nodetailedinformationabout (HWIL) facilities. RM systems are generally low theKG-95systemsispresentedinthispaper. latencydeterministicsystemsthatbroadcastspecialized An objective of this system development is to memory contents to all nodes in the network, in- provide a versatile connection type and protocol dependent of operating systems and software applica- allowingforawidevarietyofapplicationstoseamlessly tions. RM systems are ideally suited for real-time use the system. As seen in Figure 1, the user applications but do not have integrated class-1 application can make use of the system by adding it encryption systems and do not extend the node rings as a node on the RM ring or providing directly ITEAJournalN December2007/January2008 131 TheITEAJournalofTestandEvaluation jite-28-04-05.3d 28/12/07 11:59:56 131 Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 3. DATES COVERED 2008 2. REPORT TYPE 00-00-2008 to 00-00-2008 4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER Encryption System for Supporting Hard Real-Time Distributed Testing 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION Army Redstone Technical Test Center,Subsystem Test and Analysis REPORT NUMBER Branch,Huntsville,AL,35898 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S) 11. SPONSOR/MONITOR’S REPORT NUMBER(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited 13. SUPPLEMENTARY NOTES 14. ABSTRACT 15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF ABSTRACT OF PAGES RESPONSIBLE PERSON a. REPORT b. ABSTRACT c. THIS PAGE Same as 4 unclassified unclassified unclassified Report (SAR) Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18 T E C H N O T E S Figure1. Systemblockdiagram connectedelectricalsignalsfromthefamilyofstandard the interface system is totally transparent to the RS-422, LVDS, or ECL interfaces. The direct operator and connected equipment with exception of electrical connections allow the user to connect test the minimal added latency described below. equipment, sensors, tactical hardware, or other similar equipmenttothesystemcreatingalonghaulencrypted Technical description extension of the electrical signal that is totally Thissectionwilldescribethefunctionalityofeachof replicated on the remote end with minimal latency. the items in the system block diagram in Figure 1 The system can currently transmit a sustained above.Theuserapplicationbox[1]inthisfigureisany 33 Mbpsandhasthepotentialtoincreaseto50 Mbps user-supplied computer or electronic system that has with future releasesof the electronics.Alloperation of an integrated RM interface card or an external 132 ITEAJournalN December2007/January2008 TheITEAJournalofTestandEvaluation jite-28-04-05.3d 28/12/07 11:59:56 132 T E C H N O T E S be any single mode continuous or patched fiber-optic line up to 40 km, depending on line and connection attenuation. Items [7–12] are identical to items [1–6] providing the same functionality at the second user application location (Figure 1). The system is full duplex. Test results A test bed was developed to validate the perfor- mance of the new system and to collect system latency measurements. The RM interface was chosen for the testcase.Apairofencryptionsystemswassetupside- by-sidewithan8-ftlengthoffiberconnectingthetwo. A logic analyzer was connected to the appropriate Figure2. ConfigurablePCII/Ocard(CPIC) signals on both systems, and test messages were generated on both sides and sent through the systems. The logic analyzer captured the critical timing electrical interface compatible with one of the follow- parameter and the system latency is presented in ing standards: LVDS, RS-422, RS-232, transistor- Figure 3. transistor logic (TTL), low voltage TTL (LVTTL), The first block of time listed in Figure 3 is the time positive emitter-coupled logic (PECL), ECL, and it takes for the CPIC to poll the dirty bit in the RM opto-isolators. system. Currently the RM is polled at 500 kHz. The The system RM [2] card is integrated into the time from when the user application updates the computer[4]viaaPCIslotandprovidesalowlatency memory to when the CPIC reads the memory change external fiber optic interface to user applications canvarybetween0and2msecdependingonwherethe choosing to use this interface medium. update falls in the polling cycle. Test data show that The configurable PCI input/output (I/O) card the rest of the latency timeline has a variance of only (CPIC) [3] is integrated into the computer [4] via 70 ns. aPCIslotandprovidestheexternalelectricalinterface, The second latency time block is that required for translations to and from the RM, and interface to the the CPIC to perform a direct memory access (DMA) plain text side of the KG-95 Encryptor [5] (Figure 2). transferofthedatafromtheRMcard.Thenextblock The system currently has a sustained bandwidth limit is the time it takes the CPIC to process and serialize of33 Mbps.Thesystemcansupporthigherburstdata the data and transmit to the encryptor. The next five rates from the data source. That is, a short duration timing blocks, totaling 3.95msec, were measured 100 Mbps data stream could be supported. This is together because the encrypted data stream cannot be accomplished using the dual port random-access compared with the plain text source. The fiber-optic memory (RAM) in the CPIC. The computer [4] conversion time and the speed of light through the simply provides a housing and PCI bus for the fiber were measured asindividual components but add integration of the RM and CPIC cards. a very small amount of delay compared with the other TheKG-95Encryptor[5]isaNSAtype1certified, parts of the system. The last timing block is the time symmetric key encryption device that is used in the requiredfortheCPICtotransferthedecrypteddatato system to encrypt the incoming classified plain text the reflective memory card. data stream (represented by dashed lines, Figure 1) Thetotalsystemlatencywasmeasuredat9.203msec from theCPICandprovidetheencrypted output data usingthe8-fttestbedfiberconnection.Giventhespeed stream(representedbysoliddatalines,Figure 1)tothe of light in a fiber, after approximately 1.25 miles of fiber-optic converter module [6]. fiber-optics,thespeedoflightinthefiberwilldominate The fiber-optic module converter board (FOMCB) theoverallsystemlatency.Thefibertransmittersonthe providesanelectricalinterfacetotheKG-95encryptor FOMCB can transmit up to 25 miles (approx 40 km). and converts the electrical signal into a single mode The round-trip speed of light latency through 50 fiber-optic transmission source. The fiber connection miles of fiber would be approximately 1.4 ns/ft*50 betweentheconvertermodulesisusersuppliedandcan miles*5280 ft/mile5370msec. ITEAJournalN December2007/January2008 133 TheITEAJournalofTestandEvaluation jite-28-04-05.3d 28/12/07 11:59:59 133 T E C H N O T E S Figure3. Systemlatencytestmeasurements Adding the latency of the encryption/transmission RTTC has a patent pending on this system design. system of 9.203msec for each direction, the total Hardware components have been purchased to build round-trip latency for a scenario with test sites threecompletesystemsforusethroughoutthetestand separated by 25 miles would be 9.203msec+370 evaluation community as needed. The system can also msec+9.203msec5388.406msec. This fits well under be used for unclassified activities as well. The our design latency budge of 500msec for the system. encryptorscanbebypassedandtheuserhasaversatile, The performance of the system when operating using low latency tool to extend interfaces over great one of the direct electronic interfaces (RS-422, ECL, distances through single mode fibers. % LVDS, TTL) will be similar through most of the system. The main latency driver in these modes of operation will be the time to clock in a word through the serial interface from the user application. KENNETHG.LESUEURservesasthechieftechnologistin the Subsystems Test & Analysis Branch at RTTC. His Conclusion work&researchhavebeenconcentratedinHWILtesting, distributed testing, modeling and simulation, and high The newly developed RTTC-encrypted interface performance computing. He received his master degree in system has expanded the capability to conduct hard computer engineering at the University of Alabama in real-time distributed testing to include those applica- Huntsville and is currently working on his doctoral tions that are classified. The system is extremely dissertation. versatile with interfaces for RM systems and directly connected electronic interfaces. The latency of the JACK CARTER holds a master degree in electrical system is low enough to maintain closed-loop perfor- engineering from the University of Alabama in Hunts- mancearoundmostdemandingapplications.Usingthe ville. He is employed by ERC, Inc., as a senior engineer direct electronic connection configuration, the user responsible for firmware and embedded systems develop- application hardware components can be physically ment in support of ongoing test efforts at RTTC. separatedusingthissystem,andtheoperatorwillnever TRENT WOODS earned a bachelor degree in electrical have to do more than turn the system on. There is no engineering from Tennessee Technological University in code to write, no sampling of signals, and no network 1986. He is a senior systems engineer at ERC, Inc., and configurations to make. Just hook the point-to-point hassupportedthearmytestandevaluationcommunityfor fiber into the systems and start distributed testing. 20 years. 134 ITEAJournalN December2007/January2008 TheITEAJournalofTestandEvaluation jite-28-04-05.3d 28/12/07 12:00:00 134