Laboratory Information Management Systems for Forensic Laboratories: A White Paper for Directors and Decision Makers Principal Investigators Anthony R. Hendrickson, Ph.D. Brian Mennecke, Ph.D. [email protected] [email protected] Creighton University Iowa State University College of Business Administration College of Business 2500 California Plaza 2200 Gerdin Business Building Omaha, NE 68178 Ames, IA 50011-1350 (402) 280-2852 (515) 294-8100 Kevin Scheibe, Ph.D. Anthony M. Townsend, Ph.D. [email protected] [email protected] Iowa State University Iowa State University College of Business College of Business 2200 Gerdin Business Building 2200 Gerdin Business Building Ames, IA 50011-1350 Ames, IA 50011-1350 (515) 294-0545 (515) 294-7834 Research Staff Christopher S. Pilson, M.Sc. [email protected] Iowa State University College of Business 2200 Gerdin Business Building Ames, IA 50011-1350 (515) 294-2002 September 30, 2005 This work was funded by the National Institute of Justice, through the Midwest Forensics Resource Center at Ames Laboratory under interagency agreement number 2002-LP-R-083. The Ames Laboratory is operated for the US Department of Energy by Iowa State University, under contract No. W-7405-Eng-82. Table of Contents EXECUTIVE SUMMARY......................................................................................1 INTRODUCTION..................................................................................................8 OVERVIEW OF LABORATORY INFORMATION MANAGEMENT SYSTEMS.11 Log-In Functions...............................................................................................11 Operational Data.............................................................................................11 Evidentiary Data..............................................................................................12 Billing Data......................................................................................................12 Evidence Tracking............................................................................................12 Bar Coding and Evidence Tracking.................................................................13 Support for Analyst Functions........................................................................14 Analysis Request.............................................................................................14 Evidence Collection and Submission ..............................................................14 Evidence Login................................................................................................15 Distribution of Samples ...................................................................................15 Schedule of Analysis.......................................................................................15 Analysis...........................................................................................................15 Sample Preparation.........................................................................................15 Sample Measurement.....................................................................................16 Verification and Correction..............................................................................16 Reporting.........................................................................................................16 Interpretation...................................................................................................17 Disposal of Sample Materials..........................................................................17 Biometric Identification....................................................................................18 Results from Scientific Analysis.....................................................................18 Review of Scientific Analysis Results.............................................................19 Audit Trails........................................................................................................19 Reporting the Results.......................................................................................20 ADVANCED FEATURES OF LIMS....................................................................20 Evidence Analysis Scheduling........................................................................20 Instrumentation Validation & Integration........................................................21 Enhanced Data Quality.....................................................................................22 Data Entry Restrictions....................................................................................22 Double Data Entry Screens.............................................................................23 Range and Limit Checking..............................................................................23 Limit to List......................................................................................................23 Automatic Calculations....................................................................................24 Automatic Reporting........................................................................................24 Reduced Turnaround Time .............................................................................24 Supply Inventory Management........................................................................24 Human Resource Management........................................................................25 Data Archiving...................................................................................................26 Data Warehousing...........................................................................................27 Scheduling: .....................................................................................................28 OVERVIEW OF LIMS DEVELOPMENT.............................................................29 Standard Systems Development Life Cycle...................................................29 System Definition............................................................................................30 Feasibility Analysis..........................................................................................31 Functional Requirements.................................................................................35 Reporting Requirements .................................................................................36 Data Capture Requirements............................................................................36 Technical Requirements ..................................................................................36 Functional Design.............................................................................................37 Technical Design..............................................................................................37 Implementation.................................................................................................38 Personnel Training..........................................................................................38 Data Conversion..............................................................................................38 System Creation..............................................................................................39 System Validation............................................................................................40 System Integration..........................................................................................40 System Evaluation and Maintenance..............................................................40 Vendor Selection...............................................................................................41 Vendor Review................................................................................................41 Technical Issues..............................................................................................42 Assessment of In-House Personnel Skills.......................................................42 Resource Availability.......................................................................................42 Hardware & Software Considerations.............................................................42 ADDITIONAL LIMS COMPETENCIES...............................................................43 Regulatory Issues.............................................................................................43 ISO 9000.........................................................................................................43 ISO Guide 25...................................................................................................44 Good Automated Laboratory Practices (GALP)...............................................44 Electronic Signatures......................................................................................45 National Environmental Laboratory Accreditation Conference (NELAC).........45 ANALYSIS OF COMMERCIALLY-AVAILABLE LIMS......................................48 IN SITU EXAMINATION OF LIMS IMPLEMENTATIONS..................................49 Lab Structure and LIMS....................................................................................49 Absolute size...................................................................................................50 Differences in Users.........................................................................................51 Administrative Users .......................................................................................52 Scientific Users................................................................................................52 Evidence Technicians and Administrative Personnel......................................53 LIMS Development............................................................................................54 In-house development.....................................................................................54 Commercial Systems ......................................................................................56 Commercial versus In-house...........................................................................57 Process Engineering Issues............................................................................58 LIMS integration with Police Evidence Management Systems (PEMS) and other Requesting Agency Evidentiary Systems...............................................59 Within-Process Use of Evidence Technicians.................................................61 Evidence control as a driver of chain-of-custody and barcoding.....................63 Other Information System Issues....................................................................65 The paperless imperative................................................................................65 Laboratory Information Management Systems................................................66 QUANTITATIVE RESEARCH............................................................................66 An Overview of Conjoint Analysis...................................................................67 Conjoint Methodology......................................................................................69 Conjoint Survey Structure ...............................................................................72 Research Procedures.......................................................................................74 Results...............................................................................................................76 Conjoint Analysis.............................................................................................77 Conjoint Analysis: Aggregate Results.............................................................79 Conjoint Analysis: Laboratory Size..................................................................80 Conjoint Analysis: Personnel Level.................................................................85 Conjoint Analysis: LIMS Vendor/Source..........................................................89 IMPLICATIONS OF ANALYSIS.........................................................................93 REFERENCES...................................................................................................98 APPENDIX A – GENERIC LABORATORY PROCESS FLOW........................100 APPENDIX B – “TIGHTLY-COUPLED” LABORATORY PROCESS FLOW..101 APPENDIX C – “LOOSELY-COUPLED” LABORATORY PROCESS FLOW 102 APPENDIX D – LIMS PROVIDERS.................................................................103 APPENDIX E – FORENSIC LIMS VENDOR REVIEWS..................................135 Porter Lee Corporation...................................................................................135 Forensic Technology Inc................................................................................145 Justice Trax, Inc..............................................................................................152 Promadis .........................................................................................................156 StarLIMS Corporation.....................................................................................159 APPENDIX F – CONJOINT ANALYSIS...........................................................167 Aggregate Results..........................................................................................167 Average Utility Values: Aggregate.................................................................167 Average Utility Values: Aggregate (cont.)......................................................168 Average Importances....................................................................................168 Conjoint Analysis: LabSize............................................................................169 Average Utility Values: LabSize....................................................................169 Average Utility Values: LabSize (cont.)......................................................170 Average Importances by LabSize..................................................................170 Conjoint Analysis: Personnel Level..............................................................172 Average Utility Values: Personnel Level.......................................................172 Average Utility Values: Personnel Level (cont.).............................................173 Average Importances by Personnel Level.....................................................173 Conjoint Analysis: LIMS Vendor/Source ......................................................175 Average Utility Values: LIMS Vendor/Source................................................175 Average Utility Values: LIMS Vendor/Source (cont.).....................................176 Average Importances by LIMS Vendor/Source.............................................176 APPENDIX G – RESEARCH SOLICITATION LETTER...................................177 Table Index Table 1 - Types of Biometric Identification Systems................................................18 Table 2 – Forensic LIMS Vendors or Developers....................................................77 Table 3 – Importance: Aggregate Response............................................................80 Table 4 - Ideal LIMS Based on Aggregate Response..............................................80 Table 5 - Importance for Small-Sized Laboratories..................................................81 Table 6 - Ideal LIMS for Small-Sized Laboratories...................................................81 Table 7 - Importance for Medium-Sized Laboratories..............................................83 Table 8 - Ideal LIMS for Medium-Sized Laboratories...............................................83 Table 9 - Importance for Large-Sized Laboratories..................................................84 Table 10 - Ideal LIMS for Large-Sized Laboratories................................................84 Table 11 - Importance for Clerical / Evidence Technicians.....................................86 Table 12 - Ideal LIMS for Clerical / Evidence Technicians......................................86 Table 13 - Importance for Analysts / Scientists.........................................................87 Table 14- Ideal LIMS for Analysts / Scientists...........................................................88 Table 15 - Importance for Management.....................................................................89 Table 16 - Ideal LIMS for Managers............................................................................89 Table 17 - Importance for Respondents with In-House Systems...........................90 Table 18 - Ideal LIMS for Respondents with In-House Systems............................90 Table 19 - Importance for Respondents with Systems from External Vendors...92 Table 20 - Ideal LIMS Systems for Respondents with Systems from External Vendors............................................................................................................................92 EXECUTIVE SUMMARY Modern forensics laboratories need LIMS implementations that allow the lab to track evidentiary items through their examination lifecycle and also serve all pertinent laboratory personnel. The research presented here presents LIMS core requirements as viewed by respondents serving in different forensic laboratory capacities as well as different forensic laboratory environments. A product-development methodology was employed to evaluate the relative value of the key features that constitute a LIMS, in order to develop a set of relative values for these features and the specifics of their implementation. In addition to the results of the product development analysis, this paper also provides an extensive review of LIMS and provides an overview of the preparation and planning process for the successful upgrade or implementation of a LIMS. Analysis of the data indicate that the relative value of LIMS components are viewed differently depending upon respondents’ job roles (i.e., evidence technicians, scientists, and lab management), as well as by laboratory size. Specifically, the data show that: • Evidence technicians place the most value on chain of evidence capabilities and on chain of custody tracking • Scientists generally place greatest value on report writing and generation, and on tracking daughter evidence that develops during their analyses. 1 • Lab Managers place the greatest value on chain of custody, daughter evidence, and not surprisingly, management reporting capabilities. • Lab size affects LIMS preference in that, while all labs place daughter evidence tracking, chain of custody, and management and analyst report generation as their top three priorities, the order of this prioritization is size dependent. The following tables present a summary of the analyses in the larger paper. Aggregate Response: Importance of LIMS Abilities Total Daughter evidence 8.82 Management and Analyst Report Preparation 8.31 Chain of Custody Transfer 8.00 System Command Navigation 6.50 Generation of Analyst Summary Statistics 6.19 Pre-logging 6.07 Data Entry 6.01 Case Prioritization 5.51 Screen Manipulation 5.18 Case Evidence Status 5.15 Court system status 5.13 Case Grouping 5.08 Query Access to Management Data 4.77 Interface with analytical equipment 4.43 Terminal Mobility 4.13 Analyst Assignment 4.12 Asset Management 3.34 Personnel Certification Management 3.26 2 Ideal LIMS Based on Aggregate Response • Daughter Evidence items can be created as a new piece of evidence in a case with clear links to parent evidence items and the case • High level of Data Entry automation • When evidence is transferred within the laboratory, Chain of Custody information is automatically entered into the computer by scanning bar codes • Supports keystroke shortcuts for Navigation as well as typed commands and GUI for Navigation • The LIMS allows analysts to create or access Summary Statistics showing performance, backlog, and other case information Importance of LIMS attributes for… Preferred LIMS configuration for:l Small-Sized Laboratories Small-Sized Laboratories Daughter evidence 11.04 • Daughter Evidence items can be created Chain of Custody Transfer 8.50 as a new piece of evidence in a case with Management and Analyst Report 7.83 clear links to parent evidence items and Preparation the case. Pre-logging 6.74 • When evidence is transferred within the System Command Navigation 6.40 laboratory, Chain of Custody information is Case Grouping 6.33 automatically entered into the computer by Query Access to Management Data 5.93 scanning bar codes. Generation of Analyst Summary 5.84 • The LIMS provides templates for analyst Statistics Report Preparation and provides Case Prioritization 5.66 automatic field entry through drop-down Court system status 5.65 boxes and automatic word/phrase completion. Data Entry 5.35 • The LIMS supports Pre-Logging by Screen Manipulation 4.56 integrating with Agency Evidence Terminal Mobility 4.03 Management Systems for initial data input. Analyst Assignment 3.87 Case Evidence Status 3.79 • Supports keystroke shortcuts for Interface with analytical equipment 3.66 Navigation as well as typed commands and GUI for Navigation. Personnel Certification Management 2.60 Asset Management 2.22 3