BRAIN-COMPUTER INTERFACES Brain-Computer Interfaces An International Assessment of Research and Development Trends by THEODORE W. BERGER University of Southern California, Los Angeles, CA, USA JOHN K. CHAPIN State University of New York, Downstate Medical Center, Brooklyn, NY, USA GREG A. GERHARDT University of Kentucky, Lexington, KY, USA DENNIS J. MCFARLAND Wadsworth Centre, Albany, NY, USA JOSÉ C. PRINCIPE University of Florida, Gainesville, FL, USA WALID V. SOUSSOU University of Southern California, Los Angeles, CA, USA DAWN M. TAYLOR Case Western Reserve University, Cleveland, OH, USA and PATRICK A. TRESCO University of Utah, Salt Lake City, UT, USA Printed on acid-free paper This document was sponsored by the National Science Foundation (NSF) and other agencies of the U.S. Government under an award from the NSF (ENG-0423742) to the World Technology Evaluation Center, Inc. The Government has certain rights in this material. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the United States Goverment, the authors’ parent institutions, or WTEC, Inc. Copyright to electronic versions by WTEC, Inc. and Springer except as noted. WTEC, Inc., retains rights to distribute its reports electronically. The U.S. Government retains a nonexclusive and nontransferable license to exercise all exclusive rights provided by copyright. All WTEC final reports are distributed on the Internet at http://www.wtec.org. Some WTEC reports are distributed on paper by the National Technical Information Service (NTIS) of the U.S. Department of Commerce. ISBN: 978-1-4020-8704-2 e-ISBN: 978-1-4020-8705-9 Library of Congress Control Number: 2008930994 ⃝c 2008 Springer Science + Business Media B.V. No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written permission from the Publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Printed on acid-free paper. 9 8 7 6 5 4 3 2 1 springer.com WTEC Panel on Brain-Computer Interfaces Theodore W. Berger University of Southern California, Los Angeles, CA, USA John K. Chapin State University of New York, Downstate Medical Center, Brooklyn, NY, USA Greg A. Gerhardt University of Kentucky, Lexington, KY, USA Dennis J. McFarland Wadsworth Center, Albany, NY, USA José C. Principe University of Florida, Gainesville, FL, USA Walid V. Soussou University of Southern California, Los Angeles, CA, USA Dawn M. Taylor Case Western Reserve University, Cleveland, OH, USA Patrick A. Tresco University of Utah, Salt Lake City, UT, USA Acknowledgments WTEC staff members wish to extend their gratitude and appreciation to all the panelists for their knowledge, enthusiasm, and dedication to this international bench- marking study of brain-computer interface R&D, and to all workshop presenters and site visit hosts for so generously sharing their time, expertise, and facilities with us. We also wish to thank Professor Jiping He of Arizona State University, who provided guidance and contacts regarding brain computer interface research labo- ratories in China, and Dr. Jason J. Burmeister of the University of Kentucky, who provided background material on sensor technologies for Chapter 2 of the report. For their sponsorship of this unique study, our sincere thanks go to the National Science Foundation, the Telemedicine and Advanced Technology Research Center of the U.S. Army Medical Research and Materiel Command, the National Institute of Neurological Disorders and Stroke of the National Institutes of Health, the National Space Biomedical Research Institute, the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health, and the Margot Anderson Brain Restoration Foundation. Finally, our special thanks go to Dr. Semahat Demir for her vision and her unflagging support of the panel through all phases of this study. R. D. Shelton President, WTEC v Abstract Brain-computer interface (BCI) research deals with establishing communication pathways between the brain and external devices. BCI systems can be broadly classified depending on the placement of the electrodes used to detect and measure neurons firing in the brain: in invasive systems, electrodes are inserted directly into the cortex; in noninvasive systems, they are placed on the scalp and use electro- encephalography or electrocorticography to detect neuron activity. This WTEC study was designed to gather information on worldwide status and trends in BCI research and to disseminate it to government decisionmakers and the research community. The study reviewed and assessed the state of the art in sensor techno- logy, the biotic–abiotic interface and biocompatibility, data analysis and modeling, hardware implementation, systems engineering, functional electrical stimulation, noninvasive communication systems, and cognitive and emotional neuroprostheses in academic research and industry. The WTEC panel identified several major trends in current and evolving BCI research in North America, Europe, and Asia. First, BCI research throughout the world is extensive, with the magnitude of that research clearly on the rise. Second, BCI research is rapidly approaching a level of first-generation medical practice; moreover, BCI research is expected to rapidly accelerate in nonmedical arenas of commerce as well, particularly in the gaming, automotive, and robotics industries. Third, the focus of BCI research throughout the world is decidedly uneven, with invasive BCIs almost exclusively centered in North America, noninvasive BCI systems evolving primarily from European and Asian efforts, and the integration of BCIs and robotics systems championed by Asian research programs. In terms of funding, BCI and brain-controlled robotics programs have been a hallmark of recent European research and technological development. The range and investment levels of multidisciplinary, multinational, multilaboratory programs in Europe appear to far exceed that of most university and government-funded BCI programs in the United States and Canada. Although several U.S. government programs are advancing neural prostheses and BCIs, private sources have yet to make a major impact on BCI research in North America generally. However, the U.S. Small Business Innovative Research grants (SBIRs) and Small Technology Transfer Research grants (STTRs) have been effective in promoting transition from basic research to precommercialized prototypes. In Asia, China is investing heavily in biological sciences and engineering in general, and the extent of invest- ment in BCI and BCI-related research has grown particularly rapidly; still, the panel observed little coordination between various programs. Japanese universities, research institutes, and laboratories also are increasing their investment in BCI research. Japan is especially vigorous in pursuing nonmedical applications and exploiting its expertise in BCI-controlled robotics. The WTEC panel concludes that there are abundant and fertile opportunities for worldwide collaborations in BCI research and allied fields. vii WTEC Mission WTEC provides assessments of international research and development in selected technologies under awards from the National Science Foundation (NSF), the Office of Naval Research (ONR), and other agencies. Formerly part of Loyola College, WTEC is now a division of the World Technology Research Center, a sepa- rate nonprofit research institute. Michael Reischman, Deputy Assistant Director for Engineering, is NSF Program Director for WTEC. Sponsors interested in interna- tional technology assessments and related studies can provide support for the program through NSF or directly through separate grants or GSA task orders to WTEC. WTEC’s mission is to inform U.S. scientists, engineers, and policymakers of global trends in science and technology. WTEC assessments cover basic research, advanced development, and applications. Panels of typically six technical experts conduct WTEC assessments. Panelists are leading authorities in their field, techni- cally active, and knowledgeable about U.S. and foreign research programs. As part of the assessment process, panels visit and carry out extensive discussions with foreign scientists and engineers in their labs. The WTEC staff helps select topics, recruits expert panelists, arranges study visits to foreign laboratories, organizes workshop presentations, and finally, edits and publishes the final reports. WORLD TECHNOLOGY EVALUATION CENTER, INC. (WTEC) R. D. Shelton, President Michael DeHaemer, Executive Vice President Geoffrey M. Holdridge, Vice President for Government Services David Nelson, Vice President for Development Ben Benokraitis, Assistant Vice President Other WTEC Staff Members and Subcontractors Involved in this Study Hassan Ali, Director of International Study Operations Grant Lewison (Evaluametrics, Ltd.), Advance Contractor, Europe Gerald Hane (Globalvation), Advance Contractor, Asia Maria L. DeCastro, Manuscript Development and Support Patricia M.H. Johnson, Director of Publications Halyna Paikoush, Event Planner and Office Manager viii Foreword We have come to know that our ability to survive and grow as a nation to a very large degree depends upon our scientific progress. Moreover, it is not enough simply to keep 1 abreast of the rest of the world in scientific matters. We must maintain our leadership. President Harry Truman spoke those words in 1950, in the aftermath of World War II and in the midst of the Cold War. Indeed, the scientific and engineering leadership of the United States and its allies in the twentieth century played key roles in the successful outcomes of both World War II and the Cold War, sparing the world the twin horrors of fascism and totalitarian communism, and fueling the economic prosperity that followed. Today, as the United States and its allies once again find themselves at war, President Truman’s words ring as true as they did a half-century ago. The goal set out in the Truman Administration of maintaining leadership in science has remained the policy of the U.S. Government to this day: Dr. John Marburger, the Director of the Office of Science and Technology (OSTP) in the Executive Office of the President, made remarks to that effect during his confirmation hearings in October 2001.2 The United States needs metrics for measuring its success in meeting this goal of maintaining leadership in science and technology. That is one of the reasons that the National Science Foundation (NSF) and many other agencies of the U.S. Government have supported the World Technology Evaluation Center (WTEC) and its predecessor programs for the past 20 years. While other programs have attem- pted to measure the international competitiveness of U.S. research by comparing funding amounts, publication statistics, or patent activity, WTEC has been the most significant public domain effort in the U.S. Government to use peer review to evaluate the status of U.S. efforts in comparison to those abroad. Since 1983, WTEC has conducted over 50 such assessments in a wide variety of fields from advanced computing, to nanoscience and technology, to biotechnology. The results have been extremely useful to NSF and other agencies in evaluating ongoing research programs and in setting objectives for the future. WTEC studies also have been important in establishing new lines of communication and identi- fying opportunities for cooperation between U.S. researchers and their colleagues abroad, thus helping to accelerate the progress of science and technology generally within the international community. WTEC is an excellent example of coopera- tion and coordination among the many agencies of the U.S. Government that are involved in funding research and development: almost every WTEC study has 1 Remarks by President Harry S. Truman on May 10, 1950, on the occasion of the signing of the law that founded the National Science Foundation. Public Papers of the Presidents 120: p. 338. 2 http://www.ostp.gov/html/01_1012.html. ix x Foreword been supported by a coalition of agencies with interests related to the particular subject at hand. As President Truman said over 50 years ago, our very survival depends upon continued leadership in science and technology. WTEC plays a key role in deter- mining whether the United States is meeting that challenge, and in promoting that leadership. Michael Reischman Deputy Assistant Director for Engineering National Science Foundation Table of Contents Acknowledgments .................................................................................................... v Foreword ................................................................................................................. ix List of Figures ........................................................................................................ xv List of Tables........................................................................................................xvii Preface .................................................................................................................. .xix Executive Summary .......................................................................................... xxvii 1. Introduction ....................................................................................................... 1 Theodore W. Berger Background and Scope ........................................................................................ 1 Methodology........................................................................................................ 2 Overview of the Report ....................................................................................... 5 2. Sensor Technology............................................................................................. 7 Greg A. Gerhardt and Patrick A. Tresco Introduction ......................................................................................................... 7 BCI Sensor World Overview .............................................................................. 8 Major Types of Sensors for BCI Technology ..................................................... 9 Major Challenges for Producing BCI Sensors .................................................. 24 Summary and Conclusions ................................................................................ 25 References ......................................................................................................... 26 3. The Biotic–Abiotic Interface.......................................................................... 31 Patrick A. Tresco and Greg A. Gerhardt Introduction ....................................................................................................... 31 BCI Abiotic–Biotic Interface World Overview ................................................ 33 Strategies under Development to Improve Electrode Performance .................. 41 Summary and Conclusions ................................................................................ 42 References ......................................................................................................... 43 4. BMI/BCI Modeling and Signal Processing................................................... 47 José C. Principe and Dennis J. McFarland Introduction ....................................................................................................... 47 Multimicroelectrode Array Techniques ............................................................ 48 EEG/ECoG Recordings ..................................................................................... 54 Summary and Conclusions ................................................................................ 58 References ......................................................................................................... 60 5. Hardware Implementation..............................................................................65 John K. Chapin Introduction: Restoring Movement in Paralysis Patients.................................. 65 Different Approaches to BCI Research Worldwide.......................................... 67 xi xii Table of Contents Original Feasibility Demonstrations for Brain-Controlled Robotics................ 69 Brain Control of Multiple-Output Functions .................................................... 72 Biomimetic Robot Research at the Scuola Superiore Sant’Anna ..................... 74 References ......................................................................................................... 78 6. Functional Electrical Stimulation and Rehabilitation Applications of BCIs..............................................................................................................81 Dawn M. Taylor Overview of Functional Electrical Stimulation................................................. 81 FES Applications of BCI Technology around the World ................................. 84 How Different Types of BCI Command Signals can be Applied to FES......... 88 Application Areas of BCI-Controlled FES Systems......................................... 89 Practical Considerations .................................................................................... 93 References ......................................................................................................... 93 7. Noninvasive Communication Systems...........................................................95 Dennis J. McFarland Introduction ....................................................................................................... 95 Slow Cortical Potentials .................................................................................... 96 Steady-State Evoked Potentials......................................................................... 98 Online Evaluations .......................................................................................... 102 Prospects for Practical BCI Communication Systems .................................... 103 Summary and Conclusions.............................................................................. 104 References ....................................................................................................... 105 8. Cognitive and Emotional Neuroprostheses .................................................109 Walid V. Soussou and Theodore W. Berger Introduction ..................................................................................................... 109 Volitional Prostheses....................................................................................... 109 Emotional Computers and Robots................................................................... 112 Memory Prostheses ......................................................................................... 113 Neurofeedback................................................................................................. 118 Summary and Conclusions.............................................................................. 121 References ....................................................................................................... 122 9. Research Organization-Funding, Translation-Commercialization, and Education-Training Issues. ....................................................................125 Theodore W. Berger BCI Research Organization and Funding ....................................................... 125 Funding and Funding Mechanisms ................................................................. 133 Translation-Commercialization....................................................................... 134 Training-Education.......................................................................................... 139 References ....................................................................................................... 140