Loughborough University Institutional Repository Numbers, words and anonymity in 360-degree feedback: a qualitative study ThisitemwassubmittedtoLoughboroughUniversity’sInstitutionalRepository by the/an author. Additional Information: • A Doctoral Thesis. Submitted in partial fulfilment of the requirements for the award of Doctor of Philosophy of Loughborough University. In agreement with one of the case study companies, chapter five has not been released for public access. Metadata Record: https://dspace.lboro.ac.uk/2134/12564 Publisher: (cid:13)c Amanda Harrington Please cite the published version. This item was submitted to Loughborough University as a PhD thesis by the author and is made available in the Institutional Repository (https://dspace.lboro.ac.uk/) under the following Creative Commons Licence conditions. For the full text of this licence, please go to: http://creativecommons.org/licenses/by-nc-nd/2.5/ Numbers, words and anonymity in 360-degree feedback: a qualitative study by Amanda L. Harrington Doctoral Thesis Submitted in partial fulfilment of the requirements for the award of The Degree of Doctor of Philosophy of Loughborough University Abstract Academic research in 360-degree feedback continues to be dominated by a positivist approach with analyses of the feedback ratings. In contrast, this qualitative study explores how people make sense of 360, across the ‘chain of meaning making’ involving not only raters and feedback recipients but also HR managers, facilitators and external consultants. Two corporate case studies in the pharmaceutical sector show how 360 evolves as a social process and carries a variety of meanings in different organisations and management contexts. Quasi- scientific rituals are revealed, demonstrating the existence of pseudo- anonymity and of complex use of numerical ratings and narrative comments. Woven alongside these corporate case studies is an autoethnography, which examines emotional and cognitive responses to two rounds of 360 asking for feedback on coaching performance. The autoethnographic thread allows insights into the dynamic relationship between academic and practitioner perspectives, as the researcher moves between both worlds. This PhD makes three contributions: the conceptualisation of 360 as a social process; the questioning of taken-for- granted customs within 360; and a methodological contribution to the development of autoethnographic practice. KEYWORDS: 360-degree feedback; multisource feedback; anonymous ratings; narrative comments; qualitative; McDonaldisation; encoding; meaning-making. - 2 - Acknowledgements Many moons ago, I went to Laurie Cohen to ask whether we could carry out some research together. She said she had no time, but that she would be happy to supervise me to do a PhD. That started me thinking, and this thesis is the result. My thanks, therefore, to Laurie for starting me on this journey and for seeing me through to the end. My thanks also to Peter Ackers for being steady as a rock. I was lucky enough to have not just one but two supervisors who gave me helpful guidance and feedback, were interested in my work and had unshakeable belief in my capability. As an occupational psychologist, it was important to me to have someone from that academic community overseeing my work. My thanks therefore to John Loan-Clarke, who took on this role, helping to ensure that the resulting work is at least potentially interesting for others in this field. There were many others without whom this thesis would never have been conceptualized or written:  Kerry Young, whose voice rang in my ears throughout, (“structure is everything”, “writing is re-writing”) and who taught me how to structure, and how to use all those joining words; and who generously read my drafts and helped me think how to restructure and re-write  Xiaozheng Zhang, who I sat next to on the first doctoral training day, and with whom I have shared many times of mutual support and encouragement  Sara Bosley, who helped me to see that things were simpler than I thought  Bernadette Cass, Rachel Phillips and Peta Sandison, who all coached me as I thought the next thoughts - 3 -  Sarah Dale of Creating Focus, who facilitated my first 360 as part of this research, inspired me with her own writing, gave me feedback on mine and encouraged me  David Cooper of Lumus, who facilitated my second 360 as part of this research, brought me back to the commercial world, gave me feedback on my autoethnography chapter and with whom I continue to discuss the details of 360 design and implementation  John Arnold for all those many sessions of listening to my complaints and not believing a word of them.  Lesley Chandler, Mel Cowell, Tyra Till, Alex Charlton, Greg Foister and many others who listened, and listened, laughed with me at my struggles and cheered me on at every step.  Marea O’Sullivan for being such a true and consistently encouraging friend, passionate in her own learning and always wanting more.  Pamela Castro, Melanie Uhlmann and Paul Linde for their companionship on the home straight, studying alongside me, inspiring me with their learning, and keeping me focused - 4 - TABLE OF CONTENTS ABSTRACT .............................................................................................. 2 ACKNOWLEDGEMENTS ........................................................................ 3 1 – INTRODUCTION .............................................................................. 11 2 - METHODOLOGY .............................................................................. 17 2.1 INTRODUCTION ............................................................................. 17 2.2 RELATIVISM AND INTERPRETIVISM ........................................... 18 2.3 NUMBERS: COUNTING PERFORMANCE .................................... 21 2.4 TRIANGULATION .......................................................................... 22 2.5 CREDIBILITY .................................................................................. 25 2.5.1 Theoretical position ................................................................... 26 2.5.2 Congruence between methodology and methods ..................... 28 2.5.3 Rigour ........................................................................................ 30 2.5.4 The analytic lens ........................................................................ 30 2.6 DEVELOPMENT OF RESEARCH QUESTIONS ............................ 31 3 – LITERATURE REVIEW .................................................................... 34 3.1 INTRODUCTION ............................................................................. 34 3.2 HISTORY AND DEVELOPMENT OF 360 ...................................... 34 3.3 PURPOSE AND VALUE OF 360 .................................................... 37 3.3.1 Assessment ............................................................................... 37 3.3.2 Development ............................................................................. 39 3.3.3 Organisational development ...................................................... 40 3.4 ANONYMITY AND ACCOUNTABILITY ......................................... 41 3.4.1 Disentangling the terminology ................................................... 41 3.4.1.1 Anonymity .......................................................................... 42 3.4.1.2 Confidentiality .................................................................... 43 3.4.1.3 Accountability ..................................................................... 44 3.4.2 The recipient .............................................................................. 45 3.4.3 The raters .................................................................................. 49 3.4.4 The impact of anonymity on participants’ understanding of the content and process of 360 ................................................................. 52 3.5 Numerical ratings .......................................................................... 52 3.5.1 Reliability and validity ................................................................ 53 3.5.2 Performance Improvement ........................................................ 54 3.5.3 Self-Awareness ......................................................................... 56 3.5.4 Averaging .................................................................................. 57 3.5.5 The influence of rater goals and intentions ................................ 60 3.6 NARRATIVE COMMENTS ............................................................. 63 3.6.1 Recipients’ use of narrative comments ...................................... 64 3.6.2 Raters’ use of narrative comments ............................................ 65 3.6.3 Number and favourability of narrative comments ...................... 66 3.6.4 Managers’ reactions to ratings and narrative comments ........... 67 3.6.5 A qualitative approach to 360 .................................................... 68 3.7 RESEARCH METHODS ................................................................. 70 3.8 McDONALDISATION OF FEEDBACK .......................................... 73 3.8.1 Weber and Ritzer ....................................................................... 73 3.8.2 Applying Ritzer’s four principles ................................................. 74 3.8.2.1 Efficiency ........................................................................... 75 3.8.2.2 Predictability ........................................................................ 76 3.8.2.3 Calculability .......................................................................... 76 3.8.2.4 Control through technology .................................................. 77 3.8.3 Irrationalities of a rational system .............................................. 78 3.8.4 Simplification and Complexification ........................................... 80 3.8.5 Critique of McDonaldisation ....................................................... 81 3.9 STUART HALL’S ENCODING AND DECODING THEORY ........... 82 3.9.1 The making of meaning in 360, with reference to Hall ............... 83 3.9.2 Encoding/Decoding ................................................................... 84 3.9.2.1 Preferred reading ................................................................. 87 3.9.2.2 Oppositional reading ............................................................ 90 3.9.2.3 Negotiated reading ............................................................... 90 3.10 SUMMARY .................................................................................... 91 4 – RESEARCH DESIGN AND METHODS ........................................... 93 4.1 INTRODUCTION ............................................................................. 93 4.2 THE RESEARCH DESIGN: THREE CASE STUDIES ................... 94 4.3 CORPORATE CASE STUDIES ...................................................... 96 4.3.1 Case study guidelines ................................................................ 97 4.3.2 Choice of methods within the corporate case studies ................ 98 4.3.3 Interviews ................................................................................ 100 4.3.4 Interview sample ...................................................................... 103 4.3.5 Roles in the 360 process ......................................................... 104 4.3.5.1 The consultant ................................................................... 104 4.3.5.2 Vice President – Company A ............................................. 104 4.3.5.3 Central manager of the company’s 360 – Company B ....... 105 4.3.5.4 HR Managers ..................................................................... 105 - 6 - 4.3.5.5 Learning & Development Managers and Leadership Coach – Company B .................................................................................... 105 4.3.5.6 Managers (who had received 360 and who had rated others) ....................................................................................................... 105 4.4 AUTOETHNOGRAPHIC CASE STUDY ....................................... 106 4.4.1 Reasons for choosing autoethnography .................................. 106 4.4.2 Boundarying the autoethnography ........................................... 107 4.4.3 Development of autoethnography ............................................ 108 4.4.4 Principles of analytic autoethnography .................................... 110 4.4.4.1 Complete member researcher (CMR) status ..................... 110 4.4.4.2 Uses analytic reflexivity ...................................................... 110 4.4.4.3 Has a visible narrative presence in the written text ............ 111 4.4.4.4 Engages in dialogue with informants beyond the self ........ 111 4.4.4.5 Is committed to an analytic research agenda focused on improving theoretical understandings of broader social phenomena. ....................................................................................................... 112 4.4.5 Analytic versus evocative autoethnography ............................. 112 4.4.6 Design of the autoethnographic case study ............................. 113 4.5 ANALYSIS OF THE DATA ........................................................... 115 4.5.1 Transcription of the interviews ................................................. 115 4.5.2 Reflexivity ................................................................................ 116 4.5.3 Thematic analysis .................................................................... 117 4.6 ETHICS ......................................................................................... 121 4.7 EVALUATION CRITERIA ............................................................. 122 4.7.1 Contribution to the literature .................................................... 123 4.7.2 Development of theory and epistemological integrity .............. 123 4.7.3 Data collection and analysis .................................................... 125 4.7.3.1 Validity and reliability ......................................................... 125 4.7.3.2 Rigour ................................................................................ 126 4.7.3.3 Credibility and trustworthiness ........................................... 127 4.7.3.4 Persuasiveness ................................................................. 128 5 – ANALYSIS OF CASE STUDY A .................................................... 130 6 - ANALYSIS OF CASE STUDY B ..................................................... 131 6.1 INTRODUCTION ........................................................................... 131 6.1.1 Attitudes to research, security and creativity ........................... 131 6.1.2 Themes and subthemes .......................................................... 133 6.2 PURPOSE AND VALUE OF 360 .................................................. 134 6.2.1 Internally developed and owned .............................................. 136 6.2.2 A tool for relationship development ......................................... 142 6.2.3 A ‘snapshot’ ............................................................................. 145 6.2.4 360 for measurement............................................................... 147 - 7 - 6.3 ANONYMITY ................................................................................. 152 6.3.1 Safety ...................................................................................... 152 6.3.2 Pseudo-anonymity ................................................................... 153 6.3.3 Accountability .......................................................................... 157 6.4 NUMERICAL RATINGS ............................................................... 163 6.4.1 Numbers drive clarity ............................................................... 163 6.4.2 Graphical representation of the data ....................................... 164 6.4.3 The meaning of ‘3’ ................................................................... 165 6.4.4 Averaging ................................................................................ 167 6.4.5 The meaning of agreement ...................................................... 167 6.4.6 The meaning of difference ....................................................... 170 6.4.7 Interpreting the numbers .......................................................... 172 6.5 NARRATIVE COMMENTS ........................................................... 173 6.5.1 The relationship between comments and numbers ................. 173 6.5.2 The meaning of agreement ...................................................... 177 6.5.3 The meaning of difference ....................................................... 179 6.5.4 A message of respect .............................................................. 180 6.5.5 Interpreting text comments ...................................................... 182 6.6 SUMMARY AND CONCLUSION .................................................. 189 7 – AUTOETHNOGRAPHY OF A COACH USING 360 ....................... 192 7.1 INTRODUCTION ........................................................................... 192 7.1.1 My work with 360 ..................................................................... 192 7.1.2 Themes and subthemes .......................................................... 192 7.2 PURPOSE AND VALUE ............................................................... 193 7.2.1 Conversations .......................................................................... 194 7.2.2 Relationships ........................................................................... 196 7.2.3 Credibility ................................................................................. 197 7.2.4 Practice .................................................................................... 198 7.2.5 Conversations with myself ....................................................... 200 7.3 ANONYMITY ................................................................................. 203 7.3.1 Frustration ............................................................................... 203 7.3.2 Identifying the raters ................................................................ 207 7.3.3 Changing raters ....................................................................... 210 7.3.4 Follow-up and accountability ................................................... 213 7.4 NUMERICAL RATINGS ............................................................... 215 7.4.1 Graphical representation of the data ....................................... 216 7.4.2 Drive for simplicity ................................................................... 217 7.4.3 Agreement and difference ....................................................... 218 7.4.4 Interpreting the numbers .......................................................... 220 7.5 NARRATIVE COMMENTS ........................................................... 223 7.6 SUMMARY AND CONCLUSION .................................................. 224 - 8 -