IETCONTROL,ROBOTICSANDSENSORSSERIES114 Signal Processing and Machine Learning for Brain--Machine Interfaces Othervolumesinthisseries: Volume8 AHistoryofControlEngineering,1800–1930S.Bennett Volume18 AppliedControlTheory,2ndEditionJ.R.Leigh Volume20 DesignofModernControlSystemsD.J.Bell,P.A.CookandN.Munro(Editors) Volume28 RobotsandAutomatedManufactureJ.Billingsley(Editor) Volume33 TemperatureMeasurementandControlJ.R.Leigh Volume34 SingularPerturbationMethodologyinControlSystemsD.S.Naidu Volume35 ImplementationofSelf-tuningControllersK.Warwick(Editor) Volume37 IndustrialDigitalControlSystems,2ndEditionK.WarwickandD.Rees(Editors) Volume39 ContinuousTimeControllerDesignR.Balasubramanian Volume40 DeterministicControlofUncertainSystemsA.S.I.Zinober(Editor) Volume41 ComputerControlofReal-timeProcessesS.BennettandG.S.Virk(Editors) Volume42 DigitalSignalProcessing:Principles,devicesandapplicationsN.B.JonesandJ.D.McK.Watson(Editors) Volume44 Knowledge-basedSystemsforIndustrialControlJ.McGhee,M.J.GrimbleandA.Mowforth(Editors) Volume47 AHistoryofControlEngineering,1930–1956S.Bennett Volume49 PolynomialMethodsinOptimalControlandFilteringK.J.Hunt(Editor) Volume50 ProgrammingIndustrialControlSystemsUsingIEC1131-3R.W.Lewis Volume51 AdvancedRoboticsandIntelligentMachinesJ.O.GrayandD.G.Caldwell(Editors) Volume52 AdaptivePredictionandPredictiveControlP.P.Kanjilal Volume53 NeuralNetworkApplicationsinControlG.W.Irwin,K.WarwickandK.J.Hunt(Editors) Volume54 ControlEngineeringSolutions:ApracticalapproachP.Albertos,R.StrietzelandN.Mort(Editors) Volume55 GeneticAlgorithmsinEngineeringSystemsA.M.S.ZalzalaandP.J.Fleming(Editors) Volume56 SymbolicMethodsinControlSystemAnalysisandDesignN.Munro(Editor) Volume57 FlightControlSystemsR.W.Pratt(Editor) Volume58 Power-plantcontrolandInstrumentation:ThecontrolofboilersandHRSGsystemsD.Lindsley Volume59 ModellingControlSystemsUsingIEC61499R.Lewis Volume60 PeopleinControl:HumanfactorsincontrolroomdesignJ.NoyesandM.Bransby(Editors) Volume61 NonlinearPredictiveControl:TheoryandpracticeB.KouvaritakisandM.Cannon(Editors) Volume62 ActiveSoundandVibrationControlM.O.TokhiandS.M.Veres Volume63 SteppingMotors,4thEditionP.P.Acarnley Volume64 ControlTheory,2ndEditionJ.R.Leigh Volume65 ModellingandParameterEstimationofDynamicSystemsJ.R.Raol,G.GirijaandJ.Singh Volume66 VariableStructureSystems:FromprinciplestoimplementationA.Sabanovic,L.FridmanandS.Spurgeon (Editors) Volume67 MotionVision:DesignofcompactmotionsensingsolutionforautonomoussystemsJ.Kolodkoand L.Vlacic Volume68 FlexibleRobotManipulators:Modelling,simulationandcontrolM.O.TokhiandA.K.M.Azad(Editors) Volume69 AdvancesinUnmannedMarineVehiclesG.RobertsandR.Sutton(Editors) Volume70 IntelligentControlSystemsUsingComputationalIntelligenceTechniquesA.Ruano(Editor) Volume71 AdvancesinCognitiveSystemsS.NeftiandJ.Gray(Editors) Volume72 ControlTheory:Aguidedtour,3rdEditionJ.R.Leigh Volume73 AdaptiveSamplingwithMobileWSNK.Sreenath,M.F.Mysorewala,D.O.PopaandF.L.Lewis Volume74 EigenstructureControlAlgorithms:Applicationstoaircraft/rotorcrafthandlingqualitiesdesign S.Srinathkumar Volume75 AdvancedControlforConstrainedProcessesandSystemsF.Garelli,R.J.MantzandH.DeBattista Volume76 DevelopmentsinControlTheorytowardsGlobalControlL.Qiu,J.Chen,T.IwasakiandH.Fujioka(Editors) Volume77 FurtherAdvancesinUnmannedMarineVehiclesG.N.RobertsandR.Sutton(Editors) Volume78 Frequency-DomainControlDesignforHigh-PerformanceSystemsJ.O’Brien Volume80 Control-orientedModellingandIdentification:TheoryandpracticeM.Lovera(Editor) Volume81 OptimalAdaptiveControlandDifferentialGamesbyReinforcementLearningPrinciplesD.Vrabie, K.VamvoudakisandF.Lewis Volume83 RobustandAdaptiveModelPredictiveControlofNonlinearSystemsM.Guay,V.AdetolaandD.DeHaan Volume84 NonlinearandAdaptiveControlSystemsZ.Ding Volume86 ModelingandControlofFlexibleRobotManipulators,2ndEditionM.O.TokhiandA.K.M.Azad Volume88 DistributedControlandFilteringforIndustrialSystemsM.Mahmoud Volume89 Control-basedOperatingSystemDesignA.Levaetal. Volume90 ApplicationofDimensionalAnalysisinSystemsModellingandControlDesignP.Balaguer Volume91 AnIntroductiontoFractionalControlD.ValérioandJ.Costa Volume92 HandbookofVehicleSuspensionControlSystemsH.Liu,H.GaoandP.Li Volume93 DesignandDevelopmentofMulti-LaneSmartElectromechanicalActuatorsF.Y.Annaz Volume94 AnalysisandDesignofResetControlSystemsY.Guo,L.XieandY.Wang Volume95 ModellingControlSystemsUsingIEC61499,2ndEditionR.LewisandA.Zoitl Volume96 Cyber-PhysicalSystemDesignwithSensorNetworkingTechnologiesS.ZeadallyandN.Jabeur(Editors) Volume99 PracticalRoboticsandMechatronics:Marine,SpaceandMedicalApplicationsI.Yamamoto Volume100OrganicSensors:MaterialsandApplicationsE.Garcia-BreijoandPCosseddu(Editors) Volume102RecentTrendsinSlidingModeControlL.FridmanJ.P.BarbotandF.Plestan(Editors) Volume104ControlofMechatronicSystemsL.Guvenc,B.A.Guvenc,B.DemirelandM.TEmirler Volume105MechatronicHands:ProstheticandRoboticDesignP.H.Chappell Volume107SolvedProblemsinDynamicalSystemsandControlD.Valério,J.TMachado,A.M.LopesandA.M.Galhano Volume108WearableExoskeletonSystems:Design,ControlandApplicationsS.Bai,G.S.VirkandT.G.Sugar Volume111TheInvertedPenduluminControlTheoryandRobotics:FromTheorytoNewInnovationsO.Boubaker andR.Iriarte(Editors) Volume112RFIDProtocolDesign,Optimization,andSecurityfortheInternetofThingsA.X.Liu,M.Shahzad,XLiu andK.Li Volume113DesignofEmbeddedRobustControlSystemsUsingMATLAB®/Simulink®P.H.Petkov,T.N.Slavovand J.K.Kralev Volume119SwarmIntelligenceVolumes1–3Y.Tan(Editor) Volume121IntegratedFaultDiagnosisandControlDesignofLinearComplexSystemsM.Davoodi,N.Meskinand K.Khorasani Signal Processing and Machine Learning for Brain--Machine Interfaces Edited by Toshihisa Tanaka and Mahnaz Arvaneh The Institution of Engineering and Technology PublishedbyTheInstitutionofEngineeringandTechnology,London,UnitedKingdom TheInstitutionofEngineeringandTechnologyisregisteredasaCharityinEngland& Wales(no.211014)andScotland(no.SC038698). ©TheInstitutionofEngineeringandTechnology2018 Firstpublished2018 ThispublicationiscopyrightundertheBerneConventionandtheUniversalCopyright Convention.Allrightsreserved.Apartfromanyfairdealingforthepurposesofresearch orprivatestudy,orcriticismorreview,aspermittedundertheCopyright,Designsand PatentsAct1988,thispublicationmaybereproduced,storedortransmitted,inany formorbyanymeans,onlywiththepriorpermissioninwritingofthepublishers,orin thecaseofreprographicreproductioninaccordancewiththetermsoflicencesissued bytheCopyrightLicensingAgency.Enquiriesconcerningreproductionoutsidethose termsshouldbesenttothepublisherattheundermentionedaddress: TheInstitutionofEngineeringandTechnology MichaelFaradayHouse SixHillsWay,Stevenage Herts,SG12AY,UnitedKingdom www.theiet.org Whiletheauthorsandpublisherbelievethattheinformationandguidancegiveninthis workarecorrect,allpartiesmustrelyupontheirownskillandjudgementwhenmaking useofthem.Neithertheauthorsnorpublisherassumesanyliabilitytoanyoneforany lossordamagecausedbyanyerrororomissioninthework,whethersuchanerroror omissionistheresultofnegligenceoranyothercause.Anyandallsuchliability isdisclaimed. Themoralrightsoftheauthorstobeidentifiedasauthorsofthisworkhavebeen assertedbytheminaccordancewiththeCopyright,DesignsandPatentsAct1988. BritishLibraryCataloguinginPublicationData AcataloguerecordforthisproductisavailablefromtheBritishLibrary ISBN978-1-78561-398-2(hardback) ISBN978-1-78561-399-9(PDF) TypesetinIndiabyMPSLtd PrintedintheUKbyCPIGroup(UK)Ltd,Croydon Contents Preface xiii 1 Brain–computerinterfacesandelectroencephalogram: basicsandpracticalissues 1 MahnazArvanehandToshihisaTanaka Abstract 1 1.1 Introduction 1 1.2 CorecomponentsofaBMIsystem 2 1.3 Signalacquisition 3 1.3.1 Electroencephalography 4 1.3.2 Positronemissiontomography 4 1.3.3 Magnetoencephalography 4 1.3.4 Functionalmagneticresonanceimaging 4 1.3.5 Near-infraredspectroscopy 5 1.3.6 CommonlyusedmethodinBMI—whyEEG? 5 1.4 MeasurementofEEG 6 1.4.1 PrincipleofEEG 6 1.4.2 HowtomeasureEEG 6 1.4.3 Practicalissues 7 1.5 NeurophysiologicalsignalsinEEGfordrivingBMIs 9 1.5.1 Evokedpotentials 9 1.5.2 Spontaneoussignals 10 1.6 CommonlyusedEEGprocessingmethodsinBMI 11 1.6.1 Preprocessing 11 1.6.2 Re-referencing 12 1.6.3 Featureextraction 13 1.6.4 Classification 14 1.7 Feedback 14 1.8 BMIapplications 15 1.9 Summary 16 References 16 vi Signalprocessingandmachinelearningforbrain–machineinterfaces 2 Discriminativelearningofconnectivitypattern ofmotorimageryEEG 23 XinyangLi,CuntaiGuan,andHuijuanYang Abstract 23 2.1 Introduction 23 2.2 Discriminativelearningofconnectivitypattern ofmotorimageryEEG 26 2.2.1 Spatialfilterdesignforvariancefeatureextraction 26 2.2.2 Discriminativelearningofconnectivitypattern 27 2.3 Experimentalstudy 28 2.3.1 Experimentalsetupanddataprocessing 28 2.3.2 Correlationresults 29 2.3.3 Classificationresults 35 2.4 Relationswithexistingmethods 36 2.5 Conclusion 36 References 37 3 AnexperimentalstudytocompareCSPandTSMtechniques toextractfeaturesduringmotorimagerytasks 41 MatteoSartori,SimoneFiori,andToshihisaTanaka Abstract 41 3.1 Introduction 42 3.2 Theoreticalconceptsandmethods 44 3.2.1 AveragingtechniquesofSCMs 44 3.2.2 SCMaveragesinCSPandTSMmethods 46 3.2.3 Multidimensionalscaling(MDS)algorithm 47 3.3 Experimentalresults 48 3.3.1 Classificationaccuracy 48 3.3.2 SCMsdistributionsontangentspaces 54 3.4 Conclusions 58 References 58 4 RobustEEGsignalprocessingwithsignalstructures 61 HiroshiHigashiandToshihisaTanaka Abstract 61 4.1 Introduction 61 4.2 Sourceanalysis 63 4.3 Regularization 65 4.4 Filteringingraphspectraldomain 67 4.4.1 GraphFouriertransform 67 4.4.2 SmoothinganddimensionalityreductionbyGFT 70 4.4.3 TangentspacemappingfromRiemannianmanifold 72 4.4.4 Smoothingonfunctionalbrainstructures 74 4.5 Conclusion 76 References 76 Contents vii 5 Areviewontransferlearningapproachesinbrain–computer interface 81 AhmedM.Azab,JakeToth,LyudmilaS.Mihaylova, andMahnazArvaneh Abstract 81 5.1 Introduction 81 5.2 Transferlearning 82 5.2.1 Historyoftransferlearning 82 5.2.2 Transferlearningdefinition 83 5.2.3 Transferlearningcategories 84 5.3 Transferlearningapproaches 85 5.3.1 Instance-basedtransferlearning 85 5.3.2 Feature-representationtransferlearning 85 5.3.3 Classifier-basedtransferlearning 85 5.3.4 Relational-basedtransferlearning 86 5.4 TransferlearningmethodsusedinBCI 86 5.4.1 Instance-basedtransferlearninginBCI 86 5.4.2 Feature-representationtransferlearning inBCI 88 5.4.3 Classifier-basedtransferlearninginBCI 92 5.4.4 Unsupervisedtransferlearning 96 5.5 Challengesanddiscussion 96 5.5.1 Instance-basedtransferlearninginBCI 96 5.5.2 Feature-representationtransferlearninginBCI 97 5.5.3 Classifier-basedtransferlearninginBCI 97 5.6 Summary 97 References 98 6 Unsupervisedlearningforbrain–computerinterfacesbasedon event-relatedpotentials 103 Pieter-JanKindermans,DavidHübner,ThibaultVerhoeven, Klaus-RobertMüller,andMichaelTangermann Abstract 103 6.1 Introduction 103 6.2 Event-relatedpotentialbasedbrain–computer interfaces 106 6.3 Decodingbasedonexpectationmaximisation 107 6.3.1 TheprobabilisticmodelforERPBCI 107 6.3.2 Trainingthemodel 108 6.4 Decodingbasedonlearningfromlabelproportions 110 6.4.1 Learningfromlabelproportions 110 6.4.2 AmodifiedERPparadigm 111 6.4.3 TrainingoftheLLPmodel 112 6.5 CombiningEMandLLPdecodersanalytically 113 6.5.1 TrainingtheMIXmodel 114 viii Signalprocessingandmachinelearningforbrain–machineinterfaces 6.6 Experimentalsetup 114 6.6.1 Data 114 6.6.2 Dataprocessing 115 6.6.3 Methodsandhyperparameters 115 6.7 Results 116 6.8 Conclusion 120 Acknowledgements 120 References 121 7 Covariateshiftdetection-basednonstationaryadaptation inmotor-imagery-basedbrain–computerinterface 125 HaiderRazaandDheerajRathee Abstract 125 7.1 Introduction 125 7.2 Background 127 7.2.1 CovariateshiftinEEGsignals 127 7.2.2 AdaptivelearningmethodsinEEG-basedBCI 127 7.3 Covariateshiftdetection-basednonstationaryadaptation (CSD-NSA)algorithm 129 7.3.1 Problemformulation 129 7.3.2 Covariateshiftdetection(CSD)test 130 7.3.3 SupervisedCSD–NSAalgorithm 131 7.3.4 UnsupervisedCSD-NSAalgorithm 132 7.4 ExperimentalvalidationoftheCSD-NSAalgorithms 135 7.4.1 EEGdataset 135 7.4.2 Signalprocessingandfeatureextraction 135 7.4.3 Featureselectionandparameterestimation 136 7.4.4 Empiricalresults 137 7.5 Discussionandfutureprospects 138 References 139 8 ABCIchallengeforthesignal-processingcommunity: consideringtheuserintheloop 143 FabienLotte,CamilleJeunet,JelenaMladenovic, BernardN’Kaoua,andLéaPillette Abstract 143 8.1 Introduction 144 8.2 Modelingtheuser 145 8.2.1 Estimatingandtrackingtheuser’smentalstatesfrom multimodalsensors 146 8.2.2 Quantifyingusers’skills 148 8.2.3 Creatingadynamicmodeloftheusers’states andskills 149 Contents ix 8.3 ImprovingBCIusertraining 155 8.3.1 Designingfeaturesandclassifiersthattheusercan understandandlearnfrom 156 8.3.2 Identifyingwhentoupdateclassifierstoenhance learning 157 8.3.3 DesigningBCIfeedbacksensuringlearning 158 8.4 Conclusion 163 Acknowledgments 164 References 164 9 Feedforwardartificialneuralnetworksforevent-related potentialdetection 173 HubertCecotti Abstract 173 9.1 Introduction 173 9.2 Event-relatedpotentials 175 9.3 Feedforwardneuralnetworks 176 9.3.1 Activationfunctions 177 9.3.2 Errorevaluation 178 9.3.3 Architectures 178 9.4 Methods 183 9.5 Experimentalprotocol 184 9.5.1 Convnets 184 9.5.2 Performanceevaluation 186 9.6 Results 186 9.7 Discussion 188 9.8 Conclusion 190 References 190 10 Signalmodelsforbraininterfacesbasedonevokedresponse potentialinEEG 193 YeganehM.Marghi,PaulaGonzalez-Navarro,FernandoQuivira, JamesMcLean,BrunaGirvent,MohammadMoghadamfalahi, MuratAkcakaya,andDenizErdogmus Abstract 193 10.1 ERP-basedBCIs 193 10.1.1 MultidimensionalEEGclassification 195 10.1.2 NonstationaritiesinEEGsignals 195 10.1.3 Noiseintheclasslabels 196 10.2 ERP-basedinference 197 10.2.1 ERPdetection 197 10.2.2 Linearmodelandcovariancematrixstructures 198 10.2.3 Nonstationaritiesdetection 203 10.2.4 DecouplingtheclasslabelfromERPdetection 205