AVOIDANCE-BASED PAVLOVIAN-INSTRUMENTAL INTERACTIONS by ANDREA HOUGHTLING LEWIS A Dissertation submitted to the Graduate School-Newark Rutgers, The State University of New Jersey in partial fulfillment of the requirements for the degree of Doctor of Philosophy Graduate Program in Psychology Written under the direction of Dr. Mauricio R. Delgado and approved by: Dr. Andrew Delamater ________________________________ Dr. Mauricio R. Delgado ________________________________ Dr. Michael Shiflett ________________________________ Dr. Elizabeth Tricomi ________________________________ Newark, New Jersey May 2016 ABSTRACT OF THE DISSERTATION Avoidance-based Pavlovian-instrumental interactions by ANDREA HOUGHTLING LEWIS Dissertation Director: Dr. Mauricio Delgado How are aversive associations formed in the brain, and how do they subsequently influence behavior? Imagine an individual who experienced a harrowing flight, and subsequently acquired an intense fear of flying. The sight of airports or planes in flight – previously innocuous stimuli – may now trigger an aversive response within the individual. If the individual continues to fly without facing additional negative experiences, the aversive response may be updated, or extinguished. However, this individual may choose to avoid air travel in order to relieve anxiety, even though future flights would likely not be coupled with negative events. Here, the temporary relief of anxiety renders the avoidance behavior adaptive. However, avoidance can also be maladaptive. For instance, continually avoiding air travel limits the ability to visit family and friends that live afar. While avoidance behaviors are often performed without consequence in everyday life, they also play a role in the persistence of many clinical disorders. The avoidance of an anxiety-provoking stimulus is a defining behavior in anxiety disorders. Similarly, negative reinforcement-based models of addiction posit that avoidance of withdrawal symptoms is a major factor in sustained drug-seeking and   ii relapse. In both of these cases, aversive Pavlovian conditioned stimuli (CS) modulate instrumental avoidance behaviors and vice versa. This dissertation sought to better understand the flexibility of aversive CS-US relationships and how these relationships can motivate avoidance behaviors. A combination of behavioral, neuroimaging and physiological measures were used. The first goal of this dissertation was to understand how aversive Pavlovian CS- US associations are formed and updated in the brain. The second goal was to examine the behavioral and neural correlates of aversive Pavlovian control over instrumental avoidance behavior using the Pavlovian-to-instrumental Transfer (PIT) task, which tests the ability of Pavlovian CS to motivate instrumental behavior. The third goal of this dissertation was to understand how stress, a real-world variable that is often comorbid with anxiety and addiction, affects the ability of aversive CS to motivate instrumental avoidance behavior. Overall, these studies shed light on clinical disorders involving extinction failure and excessive avoidance responses, such as drug addiction, anxiety and post-traumatic stress disorder (PTSD).  iii Dedication To Mom, for teaching me the importance of perseverance To Jamie, for your unconditional patience and love To Phoebe, for re-opening my eyes to the simple wonders of the world  iv Acknowledgements I am extremely grateful to all of the people that have played a role in my journey through graduate school. First and foremost, I would like to thank my advisor, Dr. Mauricio Delgado. Mauricio has been an amazing source of wisdom, knowledge and support over the past six years. He has devoted so much of his time to helping me learn, and has also been incredibly supportive when “life” has gotten in the way of my work. It has been a sincere privilege to be a part of his lab and to learn from his example of how to be a fabulous scientist and mentor. I would also like to sincerely thank my committee members, Dr. Andrew Delamater, Dr. Michael Shiflett, and Dr. Elizabeth Tricomi for their support and guidance throughout the past several years. Thank you to my former mentors, Dr. David Rubin at Duke University and Dr. Sharon Thompson-Schill at the University of Pennsylvania, for believing in my potential at the earliest stages of my career and for helping to instill within me a passion for understanding human behavior. I am grateful to all of the past and present members of the Delgado Lab - Jamil Bhanji, Swati Bhattacharya-Sharma, Catherine Cho, Katie Dickerson, Dominic Fareri, Meredith Johnson, Stephanie Kim, Vicki Lee, Lauren Leotti, Heena Manglani, Laura Martin, Mike Niznikiewicz, Tony Porcelli, Ana Rigney, Kamila Sip, David Smith, Meg Speer, Sally Wang and Noriya Watanabe - for being amazing colleagues and a great group of human beings. A special thank you to Andy, Mike, Heena and Tony for being wonderful and helpful collaborators on much of my graduate school research. To the faculty and graduate students in the Psychology Department at Rutgers – thank you for being a great group of educators and peers. To the departmental staff, your tireless work keeping the department running smoothly has not gone unnoticed. Last and most importantly, thank   v you to my family for always loving and supporting me throughout my life. Without all of you, I would never have gotten where I am today. I am forever grateful for all of the opportunities that you have provided, and for your constant encouragement. A special thanks to my mom and mother-in-law for all of your help over these last few years – I couldn’t have written this dissertation if it weren’t for your time spent helping to take care of my home and family. Finally, I have to thank the two most important people in my life, Jamie and Phoebe. Jamie, thank you for being the most selfless, kind, hardworking and hilarious person I have ever met. You will never understand how much confidence you have helped me to gain and how much your support has allowed me to get to this point. I truly believe that I am the luckiest girl in the world to have you as my husband! And Phoebe, thank you for being the most wonderful little blessing. I’m not sure how one toddler could possess so much personality and joy, but somehow you do. Thank you for opening my heart to a love I never knew existed. I can’t wait to watch you - and your baby sister, who will be arriving soon! - grow into beautiful, smart young women and make your mark on the world.  vi Table of Contents Abstract of the Dissertation ii Dedication iv Acknowledgements v Table of Contents vii List of Figures xii List of Tables xiii Chapter 1: Introduction 1 1.1 General Introduction 1 1.2 Learning about affective information in the environment 2 1.2.1 Basic principles of Pavlovian conditioning 2 1.2.2 Measuring conditioned responses 3 1.2.3 Pavlovian learning: neural and physiological correlates 3 1.2.4 Instrumental learning: behavior and neural correlates 5 1.3 Extinction learning 6 1.3.1 Basic principles of extinction learning 6 1.3.2 Neural correlates of extinction learning 7 1.3.3 Consideration of affective learning brain regions 8 in the context of large-scale neural networks 1.4 Interactions between Pavlovian and instrumental learning systems 10 1.5 The effects of stress on learning and motivated behavior 12 1.5.1 The biological basis of the human stress response 12 1.5.2 The effects of stress on Pavlovian and instrumental systems 13  vii 1.6 Clinical significance 15 1.6.1 Positive and negative reinforcement-based models of 15 addiction 1.6.2 Avoidance-based PIT as a clinical model 17 1.7 General description and significance of dissertation experiments 18 Chapter 2: Experiment 1: Neural activation and functional connectivity 22 during extinction learning with appetitive and aversive conditioned stimuli 2.1 Introduction 22 2.2 Materials and Methods 24 2.2.1 Participants 24 2.2.2 Experimental procedures 24 2.2.3 Learning criterion 26 2.2.4 Behavioral analysis 26 2.2.5 fMRI acquisition 27 2.2.6 fMRI preprocessing and analysis: general linear model 27 2.2.7 fMRI preprocessing: functional connectivity 28 2.2.8 fMRI analysis: functional connectivity 29 2.3 Results 31 2.3.1 Behavioral results 31 2.3.2 Neuroimaging results – GLM 32 2.3.3 Neuroimaging results – connectivity analysis 33 2.4 Discussion 33  viii Chapter 3: Experiment 2: Avoidance-based human Pavlovian-to-instrumental 40 transfer 3.1 Introduction 40 3.2 Materials and Methods 41 3.2.1 Participants 41 3.2.2 PIT task procedure 42 3.2.3 Behavioral analysis 46 3.2.4 fMRI acquisition and analysis 46 3.3 Results 47 3.3.1 Behavioral results: instrumental conditioning 47 3.3.2 Behavioral results: Pavlovian conditioning 48 3.3.3 Behavioral results: Pavlovian-to-instrumental transfer 48 3.3.4 Neuroimaging results: Pavlovian-to-instrumental transfer 50 3.3.5 Relationship between Pavlovian striatal activation and 51 behavioral PIT 3.4 Discussion 52 Chapter 4: Experiment 3: Effects of stress on avoidance-based Pavlovian-to- 62 instrumental transfer 4.1 Introduction 62 4.2 Materials and Methods 64 4.2.1 Participants 64 4.2.2 Timeline of experimental procedures 65 4.2.3 PIT task procedure 65  ix 4.2.4 Stress application 68 4.2.5 Cortisol collection and analysis 68 4.2.6 SCR acquisition and analysis 69 4.2.7 Behavioral analysis 69 4.3 Results 70 4.3.1 Cortisol results 70 4.3.2 Behavioral results: tone ratings 70 4.3.3 Behavioral results: Pavlovian conditioning 70 4.3.4 Behavioral results: instrumental conditioning 71 4.3.5 Behavioral results: stress or control procedure ratings 71 4.3.6 Behavioral results: Pavlovian-to-instrumental transfer – 72 control group 4.3.7 Behavioral results: Pavlovian-to-instrumental transfer – 73 stress group 4.3.8 Behavioral results: Pavlovian-to-instrumental transfer – 75 combined 4.3.9 Physiological results: Stress or control procedure 76 4.3.10 Physiological results: Pavlovian-to-instrumental transfer 76 4.4 Discussion 77 Chapter 5: General Discussion 83 5.1 Purpose and Summary of Dissertation Studies 83 5.2 Limitations 85 5.3 Future Directions and Implications 87 5.4 Overall Conclusions 90   x