EEG-Based Evaluation of Cognitive and Emotional Arousal when Coding in Different Programming Languages University of Oulu Information Processing Science Master’s Thesis Amit Rajendra Desai Date: 29/5/2017 2 Abstract Cognitive psychology is a study of the brain, an organ that behaves as a complex computing system. The brain signals generate electrical signals, which can be interpreted meaningfully in line with the actions performed by the brain using various computational devices and measures using electroencephalography methodology. In this thesis, the signals obtained from the brain are processed to quantitatively study and compare the brain activities of coders while programming in two different programming languages. In this research, we have chosen the structured programming language C and the scripting language Python for comparison. Previous empirical research comparing various programming languages in a controlled manner identified attributes such as correctness, robustness, syntax, efficiency, etc. as parameters that characterize those programming languages (see e.g., Nanz and Furia, 2015; Garcia Jarvi, Lumsdaine, Siek and Willcock, 2003). This thesis aims to build upon the previous findings and compare the psychological effects during programming tasks. Emotiv Epoc is a Brain Computer Interface device used for reading and analyzing brain signals in this study. Understanding the usage of the Emotiv device and the corresponding software tools is an essential part of this thesis work. Thus, in this thesis a pilot study is planned and a controlled experiment is conducted in order to collect and evaluate the data collected from the Emotiv Epoc device and self-reports to derive meaningful statistical results and interpret the emotional and cognitive activity of the participants. The pilot study aims to understand how emotions and/or cognitive load vary while coding in C and Python. Initial study involves understanding the EEG method, principles and complexities involved with the collection of data. The core part of the thesis consists of planning and conducting a lab experiment, which involves six participants performing predefined tasks in C and Python, and answering a series of questionnaires. The collected data is analyzed in SPSS tool based on factors like time, performance (correctness), and questionnaire-based self-reports. EEG signals are analyzed in this thesis up to the point of artifact removal (Filtering and ICA using MATLAB and EEGLAB). The results of the thesis reveal findings from emotional and EEG data analysis which have been captured from an experimental setup using the Emotiv device, software developers as participants and software programming tasks. The thesis helps to understand the steps to perform such an experimental study and to assimilate the emotional and cognitive load that affects the brain when performing programming. The particular comparison of C and Python as programming languages shows the high correlation between the programming language characteristics (such as syntax and time to code) and emotional and cognitive behavior of the programmers. Keywords Emotiv Epoc, quantitative, programming languages, EEG, emotions, cognitive load. Supervisor Dr. Dorina Rajanen 3 Foreword First and foremost, I would like to thank Dr. Dorina Rajanen for giving me the opportunity to write my Master thesis in the Software Engineering for the chair of Information Processing Science at University or Oulu. This thesis topic was proposed by Dr. Dorina Rajanen. I am very grateful for her valuable inputs, collaboration efforts and immense support that contributed to the successful completion of the thesis work. I wholeheartedly thank my mentor for her persistent motivation and the extensive advice and support throughout the course of this research work. I would also like to thank Dr. Mikko Rajanen for supporting me on all technical issues and important feedback during research. A special thanks to Lic.Phil. Jouni Lappalainen for providing a valuable feedback and reviewing the final thesis work. The thesis work was supported by the EMSE (European master of software Engineering) coordinators Prof Dr. Burak Turhan (University of Oulu, Finland) and Prof Dr. Dieter Rombach (Technical University Kaiserslautern, Germany), as well as Christian Wolschke, my EMSE coordinator at TUKL, Germany. I would like to epically thank for all the participants who spent their valuable time and contributed to the experiment. I want to specially thank all my friends who were directly or indirectly involved during the course of this thesis and took time to help me out and motivate me in my difficult times. I would like to specially thank my wife Ankitaa Bhowmick for her valuable inputs on technical issues and constant support. Also, I would like to extend my special token of thanks to my special friends Aniqa Rehman, Ayush Verma, Aprilia Hartami, Leevi Rantala and Samitha Jayathilaka; for being my strongest pillars of support. Last but not least, I thank my parents for always supporting and backing me up in all my endeavors. Amit Rajendra Desai Oulu, May 29, 2017 4 Contents 1. Introduction 6 1.1 Background and Motivation 6 1.2 Research Aim and Research Questions 7 2. Related Work 9 2.1 Psychophysiological research using electroencephalography 9 2.1.1 Electroencephalography (EEG) 10 2.1.2 Principles of EEG measurement 10 2.1.3 Measures obtained using EEG 11 2.1.4 Applications of EEG in IS/SE research 14 2.2 Emotiv Device 15 2.2.1 The design, features, and usage of BCI devices 16 2.2.2 Emotiv Epoc as a wireless EEG tool and BCI device: Methodology and characteristics 18 2.2.3 Applications of Emotiv Epoc in Software Engineering 18 3. Research methods 21 3.1 Questionnaire design 22 3.2 Measures 22 3.2.1 EEG based measures 22 3.2.2 Self-report measures 23 Background information 23 Questionnaire in Programming 24 Overall experiment 24 3.3 Planning the Experiment 25 3.4 Data analysis 25 4. Implementation 27 4.1 Experimental Design 27 4.2 Materials and Equipment 29 4.3 Questionnaire Data 31 4.3.1 Background information questionnaire 31 4.3.2 Programming Task Questionnaire 31 4.3.3 Overall Experience Questionnaire 32 4.4 EEG recording 32 4.5 Data Collection Problems 33 4.6 EEG Data Analysis 35 5. Results 39 5.1 Participants 39 5 5.2 Experimental Task Analysis 42 5.3 Overall Experience Analysis 45 6. Discussion 49 6.1 Answers to the Research Questions 49 6.2 Implication of the Research 50 7. Conclusion 52 8. References 53 APPENDIX A. Consent Form for Participants 59 APPENDIX B. C and Python Programming Tasks 60 APPENDIX C. Background Questionnaire 63 APPENDIX D. Programming Questionnaire 67 APPENDIX E. Overall Experience Questionnaire 68 APPENDIX F. Checklist 69 6 1. Introduction Humans are often very emotional; our brain generates signals for every small activity performed in our daily life, whether playing games on computer, watching movies or a play, interacting with computers, or even programming. Emotions are feelings derived from the current state of mind and these emotions can be recognized via speech, facial expressions, textual information or even a gesture. Emotions can be categorized as positive or negative emotional states whereas cognitive load is the amount of work that the brain puts in to perform certain activity. (Liu, Sourina & Nguyen, 2010; Ahmad et al., 2016.) The emotions of programmers during programming is an interesting study of human emotional states and cognitive load. To validate the emotions while programming, it is important to understand the programmer’s psychology and the amount of effort put into completing a programming task. Programming languages are complex. They can be categorized as simple as long as there is no debugging involved and can get very complex when an error is encountered. (Hoc, 2014.) Thus, a comparative study of programmer’s emotional states while programming in different programming languages gives an insight to the correlation between programming language complexity and the corresponding psychology of the programmers while programming in that particular language. The section 1.1 provides an overview of the research topic and motivation to pursue the study. The aim and scope of the study is set in section 1.2. 1.1 Background and Motivation In software engineering, we study several courses related to programming languages. A software engineer is intended to learn and use several programming languages. Programming languages can be categorized as structured, object-oriented, scripting, etc. Lahtinen, Ala-Mutka & Järvinen (2005) conducted a survey of novice programmers, which indicated that C++ was more difficult than Java. The survey results also stated that programmers were finding it more difficult to understand the basic concepts of programming rather than the application of the logic. Software developers’ emotions can be identified based on the progress of the programming tasks. If the positive emotions like happiness is observed one can assume that the task is in progress and if the negative emotions like frustration is observed, then it would seem that he is stuck in the task. (Muller & Fritz, 2015.) This thesis aims to study and understand the differences in programming languages based on the emotional and cognitive activities of programmers while programming. For this purpose, a pilot study comparing two programming languages is conducted, in order to understand the programmer’s emotions and cognitive load while performing the programing tasks. To limit the scope of this study, two widely-used programming languages are selected; C which is a low-level structured programming language and Python which is a higher-level scripting language with simpler syntactical constructs as compared to languages. Nanz and Furia (2015) made the comparison of programming languages in order to find out the differences between procedural, object-oriented, functional, and scripting languages. Some of the factors considered for pairwise comparison in that experiment were lines of code, compile time, run time, patch and merge. The various programming languages that were compared in this experiment were C, C#, Java, Python, Ruby, Go, etc. 7 The results of the comparative study showed a significant difference between C and Python for the pairwise comparison mentioned above. (Nanz and Furia, 2015.) Hence C and Python languages were selected for the comparative pilot study in this thesis. The motivation of this pilot study is to gain an in-depth understanding of the developers’ emotions and cognitive dimensions while they code. This analysis will help the software engineering community to understand the challenges like comprehension, attention, focus and memory load that may be encountered while programming. This research, which makes a comparative study, can provide valuable information to enhance the learning capabilities and skills of a programmer. Going further, in order to generalize this research a considerable number of participant from different backgrounds must participate in the research experiment. This would also eventually improve the results of the comparative study. The Emotiv Epoc device used in this research is a low cost commercially available device, but for further studies any other EEG device could be used. The steps formulated for analysis of EEG signals provides the basis for future application development using Emotiv Epoc device. 1.2 Research Aim and Research Questions The study of psychophysiological research combined with software engineering has an extended scope of study. The aim of this research is to examine the emotions and cognitive load of the developers during programming in two programming languages. In particular, this thesis aims to conduct an exploratory pilot experiment where EEG data as well as self-reports are collected in order to measure emotions and cognitive load during programming. The following research questions are addressed in this thesis. RQ1. What are the steps to be followed while using Emotiv Epoc device starting from recording of brain signals to analysis of the data? RQ2. What is the cognitive and emotional activity as measured by different EEG-based measures and self-report measures of the participants when programming in two of programming languages of software engineering (C and Python)? RQ3. What differences are observed between programming in C and Python with respect to programmer’s performance, emotions and cognitive load during programming? To conduct this study an empirical approach is adopted and an experiment is conducted in order to collect and analyze the data of interest. The experiment for data collection is carried out in a controlled environment and the data collected is analyzed using descriptive statistics, Pearson correlation coefficient and pie and bar charts using SPSS tool. The research method used in this study is a quantitative approach that provides an evaluation of the emotional states and cognitive dimensions measured during pre- defined programming tasks. In quantitative research the analysis is carried out in an unbiased way to deal with data, facts and figures. It relies on numbers/numerical values which are obtained when a phenomenon of interest is measured. (Muijs, 2010 and Jedlitschka & Rombach, 2016.) Thus, quantitative approach has been chosen in this research, based on the fact that the experiment provides numerical data that can be statistically analyzed in order to answer the research questions. 8 This thesis has the following structure: Chapter 2 aims to understand the background of psychophysiological research and the existing literature that provides the basic understanding like EEG signals and its measures for this thesis. Section 2.2 explains the features of Emotiv Epoc device and the feasibility of the device for this research. Chapter 3 presents the research methodology adopted for this pilot study and the measures which can be obtained for emotional and cognitive dimensions. In chapter 4 the implementation of experiment is explained in detail. Chapter 5 presents the results obtained from the study. In chapter 6 the results are discussed with respect to the research questions and the last chapter 7 presents the finding of the research work and future work related to this study. 9 2. Related Work Since the thesis is based on psychophysiological study and its application in the field of software engineering, it is important to understand the related measurement and analysis techniques like electroencephalography (EEG). This chapter provides background information on EEG principles, its current usage in emotional and cognitive study and its further application in software engineering studies. The related work chapter also aims to provide the details of data capturing and analysis techniques in psychophysiological software engineering applications, thereby underlining the features and efficiency of Emotiv Epoc for this research. In the first section 2.1, the existing research has been studied to understand the different psychophysiological methods to obtain the emotions and cognitive measures using EEG. In order to conduct a EEG-based research, it is important to understand how the previous studies were performed in relation to software engineering applications. The EEG is captured via Emotiv Epoc which is a commercially available, low cost BCI device. The section 2.2 shows the various research applications of Emotiv device and helps to draw a conclusion that this device fits the purpose of this study. 2.1 Psychophysiological research using electroencephalography Psychophysiological research is the study of physiology, which deals with the relationship between mental and physical phenomena of a person. The non-invasive recording of the psychophysiological responses primarily focuses on collection of electrical signals at the surface of the skin. There are distinct types of measures employed in psychophysiological research. (see e.g., Potter and Bolls, 2012; Hodges, 2010.) Accordingly, the heart rate is measured using the electrocardiogram (ECG). The measure of gastric mobility is measured using electrogastrogram, blood pressure is captured by finger pulse amplitude (FPA), and muscular tension can be studied by using the electromyography (EMG). Brainwaves are captured using electroencephalography (EEG). In this study, the focus is on measuring brain activity using EEG, and therefore the literature review details the background and applications of EEG. According to Potter and Bolls (2012), the psychophysiological measurements involve three primary approaches, namely self-reports, readings of the psychophysiological signals using specialized equipment, and observations of the external behavior of the study participants. The common usage of psychophysiological research is to study the emotions and to understand the cognitive processes such as attention and memory. The recording in psychophysiology focuses on collection of electrical signals or “biopotentials”, which are generated at the surface of the skin. Since these signals are measured at the scalp in the case of EEG, the strength of the EEG signals is very low. Therefore, the EEG equipment also involves the use of amplifiers in order to detect and record the actual signals. There has been significant research in improving the signals obtained from brain and preprocessing the data using EEGLAB, machine learning algorithms or Fast Fourier Transform for signal measurements (Lee et al., 2015; Khushaba et al., 2013; Khushaba et al., 2012; Trivedi, 2013). In the following sub- section 2.1.1, the emphasis is given on understanding the concepts of EEG. 10 2.1.1 Electroencephalography (EEG) The existence of human race is due to the fact that we have emotions. The ability to feel, touch, sense and act based on the emotional factor makes our life complete. Emotions in humans have a special meaning to life and it exists every second. The emotions cannot be expressed completely or measured using a survey or self-reports and hence an in-depth analysis can be performed by a psychophysiological study to validate the positive and negative emotions and cognitive measures like comprehension and attention of humans. (Potter and Bolls, 2012.) EEG is a short-form for electroencephalography, which measures the central nervous system activity evoked by brain electrical signals (Potter and Bolls, 2012). Hans Berger discovered this measurement technique in late 1920, which was the first of its kind to measure electrical signals from brain. The experiment was conducted by soaking two sponges in saline and connected to differential amplifier. Since then there is a significant improvement in ways of collecting the EEG data. The electrical signals which can be received from the scalp can be measured to a very low range of 10-6 V. (Potter and Bolls, 2012.) The brain is divided into four regions, which can be used for the measurement purpose; the frontal, parietal, temporal, and occipital lobes. The human brain is a huge mass of interconnected neurons. The neurons provide immense amount of biopotentials, which can be recorded by the EEG signals. (Potter and Bolls, 2012.) Two major types of brain waves that can be measured using EEG are: spontaneous EEG (referred as continuous EEG), and evoked potentials (referred as event-related potentials). Most of the research is done on the spontaneous EEG in clinical context. The spontaneous EEG signals are equally relevant in the field of cognitive neuroscience, psychophysiology, etc. (see e.g., Müller-Putz, Riedl and Wriessnegger, 2015.) The further section provides an insight of the measurement in EEG and electrode placement system. This contains general information of how to record the brainwaves and principles behind the measurement. 2.1.2 Principles of EEG measurement In psychophysiological research, typically the recording of the EEG is done using a stretch-lycra cap on which a number of electrodes are mounted (e.g., 32, 64, 128, or even more; Harmon-Jones and Amodio, 2012). The electrodes are made of tin or silver and silver chloride. With the cap, the electrodes are positioned over the entire scalp. Typically, the placement of electrodes is done according to established standards such as the 10-20 system or 10-10 system (Harmon-Jones & Amodio, 2012; Oostenveld & Praamstra, 2001). The number ‘10’ and ‘20’ primarily refers to the distances between the electrode placement which is either 10% or 20% of the total front-back or right-left length. As shown in Figure 1, there are several points where the electrodes are placed in order to collect the brainwaves. The letter is indicated based on the position of the electrode placed. The letter FP means frontal pole, F stands for the frontal lobe, C means the central region, P is parietal lobe, T is the temporal lobe and O is the occipital lobe. (Alshbatat, Vial, Premaratne & Tran, 2014; Harmon-Jones & Amodio, 2012; Oostenveld & Praamstra, 2001.) The electrodes placed in between these regions are often represented with two letters and it measures the difference in voltage between the neurons at this region. The letters can be associated with numbers. The letters with odd numbers represent left side of the brain and the letter with even number refers to the right part of the brain. This representation helps to locate the designated site for measurement.