The response of art education to the challenges of a multimodal world by Pieter Arie Binsbergen A thesis submitted in partial fulfilment of the requirements for the degree PhD (Humanities) in the Faculty of Humanities UNIVERSITY OF PRETORIA 31 March 2016 SUPERVISOR: Prof. Adelia Carstens i © University of Pretoria ii © University of Pretoria ABSTRACT The response of art education to the challenges of a multimodal world The purpose of the research conducted for this thesis was to understand the combination of different modes of representation (multimodality) in contemporary art-making in order to better prepare arts education for the challenges of communicating in a multimodal world. Through personal experience as an arts educator in secondary and tertiary education, and an adjudicator of an important national competition for young contemporary artists, the Sasol New Signature Art Competition, I became aware of the discrepancy between curricular emphases at schools and universities, and the characteristics of pieces that are selected as winning works by the artistic society. Whereas traditional art education focuses on technical perfection and techniques bound to traditional art genres, contemporary society values meaning making through various modes of expression (multimodality) and by increasingly using technology as a vehicle of delivery. It was therefore the main aim of this study to explore empirically the processes by which multimodal art-making takes place in practice, in order to elicit principles of how multimodality may equip art educators to make aspects of a hidden curriculum visible, and thus better equip learners/students to cope with the demands and challenges of an increasing multimodal world of communication. Through the analyses of various art works it is shown that although the notion of multimodality originated among linguists and was theorised within the framework of Systemic Functional Linguistics, instances of the combination of multiple sense-based modes (sight, sound, movement, tactability, taste and olfaction) of which the visual mode is typically central, have been evident throughout the history of visual art. Theoretically I draw on socio-semiotics, which emphasises the construction of meaning by exploiting the affordances of multiple modes and media. Although the initial rigorous grammatical approach of Kress and Van Leeuwen (1996) gave momentum to the theorising of multimodal meaning making, it has since been replaced by a perspective that retains only the foundations of Systemic Functional Grammar, and further draws upon the notions of continuous design and re- design driven by social discourses and technology. The empirical research, aimed at establishing how and why nascent artists combine particular modes and media, was conducted within a multiple case study design, focused on the qualitative content analysis of questionnaires e-mailed to 10 winners of a national art competition, their winning art works, and the artist statements accompanying the art works. Four prominent themes emerged from the data, namely combination, layering, viewer response and validation of contemporary issues. These themes do not only support the sense-based taxonomy of modes that precipitated from a literature review of prominent iii © University of Pretoria sources on multimodality, but also served to underpin a curriculum framework for art education. iv © University of Pretoria TABLE OF CONTENT CHAPTER 1 1 Purpose, background and problem statement 1 1 PURPOSE AND BACKGROUND 1 1.1 National education policies and documents for arts education 7 1.2 Personal triggers 11 1.3 Origins and development of the notion of ‘multimodality’ 13 1.4 Previous research 14 2 PROBLEM STATEMENT 18 3 RESEARCH QUESTIONS 20 4 OBJECTIVES 21 5 SUMMARY AND CHAPTER PREVIEW 21 CHAPTER 2 23 Theoretical underpinnings: The linguistic foundations of a theory of multimodality 23 1 INTRODUCTION 23 2 HISTORICAL OVERVIEW OF MULTIMODALITY 24 3 STRUCTURALIST APPROACHES TO SEMIOTICS: SAUSSUREAN AND PEIRCEAN SEMIOTICS 26 4 SOCIAL AND CONSTRUCTIVIST APPROACHES TO SEMIOTICS 33 4.1 Social semiotics 33 4.2 Hallidayan systemic functional linguistics (SFL) 34 4.3 Post-Halidayan socio-semiotics 37 4.3.1 Introduction 37 4.3.2 Kress and Van Leeuwen’s ‘visual grammar’ 38 4.3.3 Scholarly debates on Kress and Van Leeuwen’s Reading images: The grammar of visual design (1996) 40 4.3.4 The influence of visual grammar on contemporary interdisciplinary research 43 4.3.4 Critical discourse analysis (CDA) 47 5 SUMMARY 49 CHAPTER 3 51 Modes and multimodality 51 1 INTRODUCTION 51 2 DEFINING AND DELIMITING MODES 51 3 CHARACTERISTICS OF MODES 54 3.1 Modes are an interface between the natural and cultural world 54 3.2 Modes have grammars 54 3.3 Modes have communicative effects 57 v © University of Pretoria 3.4 Modes have materiality 58 3.5 Modes can be realised in multiple media 58 3.6 Modes may be related to sensory possibilities of the body 59 3.7 Modes may be disseminated through technology 63 3.8 Towards a classification of modes 67 3.9 Multimodality 69 3.9.1 The challenges of theorising multimodality 70 3.9.2 The influence of technology on theorising multimodality 72 3.9.3 Attempts at theorising multimodality 75 3.9.4 Towards a theory of multimodality 76 3.9.4.1 The process of ‘design’ (Kress, 2010, 2014) 76 3.9.4.2 The new multimodal media landscape 82 3.9.4.3 The impact of ‘design’ on multimodality in visual art 87 3.9.4.4 Where rhetoric and semiosis meet 88 4 SUMMARY 89 CHAPTER 4 92 An overview of modalities and multimodality in the history of art 92 1 INTRODUCTION 92 2 MULTIMODALITY IN VISUAL ART 93 2.1 Pre-20th-century multimodality in visual art 93 2.2 Multimodality in 20th-century art 104 2.3 Multimodality in contemporary visual art 108 2.3.1 Introduction 108 2.3.2 Multimodal visual art genres 109 3 ELEVEN MODES AT WORK IN VISUAL ART 114 3.1 Still images and text 115 3.2 Image and sound 118 3.3 Image and movement 122 3.4 Image and action 124 3.5 Image and gesture 128 3.6 Image and 3D models 135 3.7 Image and touch 136 4.8 Image and smell 138 4.9 Image and taste 140 4 SUMMARY 142 CHAPTER 5 144 Research methodology 144 1 INTRODUCTION 144 vi © University of Pretoria 2 PHILOSOPHICAL WORLDVIEW 144 3 RESEARCH APPROACH 145 4 RESEARCH STRATEGY: CASE STUDY 146 5 METHODS OF SAMPLING, DATA GATHERING AND DATA ANALYSIS 148 5.1 Methods of sampling 148 5.2 Data types 149 5.3 Methods of data analysis: qualitative content analysis 151 5.3.1 Raw data 151 5.3.2 Defining and delimiting qualitative content analysis 151 5.3.3 Selecting the coding parameter 153 5.3.4 Coding display and procedure 154 6 REPORT BACK 155 6.1 Within-case analysis of the emergent themes 155 6.2 ‘Across-case analysis’ of the emergent themes 156 7 QUALITY CONTROL (STRATEGIES OF VALIDATION) 156 7.1 Triangulation of data 157 7.2 Clarification of researcher bias 158 7.3 Thick descriptions 158 8 ETHICAL CONSIDERATIONS 158 9 SUMMARY 159 CHAPTER 6 161 Data analysis 161 1 INTRODUCTION 161 2 BACKGROUND TO THE RESEARCH SITE 161 3 DATA COLLECTION AND ANALYSIS 163 3.1 Instruments and methods 163 3.2 Analysis of the e-mail questionnaire 163 3.2.1 Procedure 163 3.2.2 Responses to the questions of the e-mail questionnaire 165 3.3 Responses to the open questions of the e-mail questionnaire 170 3.3.1 Combining materials 172 3.3.2 Layered end product 173 3.3.4 The importance of viewer response 174 3.3.5 Validating contemporary issues 175 4 APPLICATION OF THE FOUR THEMES IN AN ANALYSIS OF THE ARTWORKS 178 4.1 Zane Wesley Lange, Joystick (Port Elizabeth) 178 4.2 Sikhumbuzo Makhandula, Ilolo (Umthatha) 181 4.3 Ingrid Jean Bolton, Un(SEA)N, (Pretoria) 184 4.4 Brendon Erasmus, It’s all going down the tube thanks to the media lube, (Pretoria) 187 vii © University of Pretoria 4.5 Zelda Stroud, Body Butter (Pretoria) 191 4.6 Dot Vermeulen, Desperately Disciplined (Bloemfontein) 195 4.7 Liberty Battson, Did you know? (Pretoria) 198 4.8 Claire Jorgensen, Seedlings (Cape Town) 201 4.9 Adelheid von Maltitz, Bodies (Bloemfontein) 204 4.10 Josua Strümpfer, There is death in the pot (Port Elizabeth) 207 5 SYNOPSIS OF THE THEMATIC ANALYSIS 210 6 SUMMARY 213 CHAPTER 7 214 Framework for multimodal teaching and learning in visual art 214 1 INTRODUCTION 214 2 OVERVIEW OF THE RESEARCH 214 3 FRAMEWORK FOR TEACHING MULTIMODALITY IN VISUAL ART 219 3.1 Alignment of the research themes with national policy documents 221 3.2 Curriculum and Assessment Policy Document (Visual Art) 2011 221 3.3 South African Qualifications Authority (SAQA) NQF level 7 222 4 INSTRUCTIONAL DESIGN FRAMEWORK FOR MULTIMODAL ART PRACTICES BASED ON THE ADDIE MODEL 223 4.1 Characteristics of the ADDIE instructional design model 223 4.2 Framework for multimodality in visual art based on the ADDIE model (meso- level) 225 5 CONCLUDING THOUGHTS 229 LIST OF REFERENCES 231 ADDENDUM A: E-MAIL QUESTIONNAIRE 247 ADDENDUM B: DATA-CODING TABLE 251 viii © University of Pretoria LIST OF FIGURES Figure 2.1 Peirce, triadic model 1931 (adapted from Peirce, 1931‒1958, (2)…. 28 Figure 2.2 Peirce’s classification of signs applied to visual art: Piet Mondrian, Broadway Boogie Woogie (1942–1943), oil on canvas; Chuck Close, Bob (1979), oil on canvas; Jacques-Louis David, The Death of Marat (1793), oil on canvas ……………………………………………………………… 30 Figure 2.3 Contemporary media advertisement for Audi 2013 ………………………… 31 Figure 2.4 Brett Murray, The Spear, oil on canvas, 2010, Johannesburg, South Africa …………………………………………………………………………………………………. 49 Figure 3.1 Constantine Brancusi, Sculpture for the Blind, marble, 1916, Paris, France ……………………………………………………………………………………………….. 62 Figure 3.2 Sense-based taxonomy of modes ……………………………………………………. 62 Figure 3.3 Google animated emoticons (still image) ………………………………………… 73 Figure 3.4 David R. Burns, Visit Us, still image from the virtual reality video, 2013 …………………………………………………………………………………………………… 74 Figure 3.5 Semiotic approach to multimodal meaning-making in visual art ……. 88 Figure 4.1 Egyptian art. Section from the ‘Book of the Dead’ of Nany, Third Intermediate Period, Dynasty 21, reigns of Psensennes I–II, c 1040–945 BC …………………………………………………………………………………….. 94 Figure 4.2 Example of an ordered visual language structure. Hall of the Bulls, the first third of the axial gallery, Lascaux Caves, France, 15 000 ‒ 13 000 BC ………………………………………………………………………………………….. 95 Figure 4.3 The birth of text. Counting tokens from the Neolithic Fertile Crescent, c 4000‒3200 BCE………………………………………………………………. 95 Figure 4.4 Two pages from an incunabulum of the 30th section of Al-Tasrif book of Abul-Qasim Al-Zahrawi (aka Abulcasis, Chirugia Abulcasum) as translated into Latin in the 12th century by Gerard of Cremona ………………………………………………………………………………………… 96 Figure 4.5 The Gutenberg Bible, extract with text and printed illumination (42- line Bible), 1455 ………………………………………………………………………………. 97 Figure 4.6 Gothic art. Miniatures from two different manuscripts based on the same model, Apocalypse, Oxford, Bodleian Library, 1100 to 1300 … 98 Figure 4.7 Early Renaissance. Leonardo da Vinci, Study for proportions, from Vitruvius’s De Architectura, 1490, Italy …………………………………………… 98 Figure 4.8 Early Renaissance. Albrecht Dürer, Celestial map of the southern sky, 1515, Germany ………………………………………………………………………….. 99 ix © University of Pretoria Figure 4.9 High Renaissance. Michelangelo, study, Casa Bounarroti, 1525, Florence, Italy ……………………………………………………………………………………. 99 Figure 4.10 High Renaissance. The Adoration of the Magi, stained-glass window, Circle of Peter Hemmel von Andlau, Strassburger Werkstattgemeinschaft, Munich, Germany, 1507 ……………………………. 100 Figure 4.11 Baroque. Francisco de Goya, The Sleep of Reason Produces Monsters, plate 43 of Los Caprichos (etching), 1799, Spain …………… 101 Figure 4.12 Rococo. Sir Joshua Reynolds, Self-portrait (oil on canvas), 1771 ….. 101 Figure 4.13 Neo-Classicism. Jacques-Louis David, The Death of Marat, oil on canvas, 1793, Belgium ………………………………………………………………………. 101 Figure 4.14 19th century. William Blake, Laocoön, line engraving, 1822, England................................................................................... 102 Figure 4.15 19th century. Telamaco Signorini, Leith, oil on canvas, 1881, England ……………………………………………………………………………………………… 102 Figure 4.16 Impressionism. Edouard Manet, Portrait d'Emile Zola, oil on canvas, 1867‒1868 ………………………………………………………………………………………… 103 Figure 4.17 Impressionism/Post-Impressionism. Henri de Toulouse-Lautrec, Troupe de Mlle Églantine, lithographic print, 1898, France …………….. 103 Figure 4.18 Man Ray, Indestructible Object, Readymade, 1928, Paris, France …. 105 Figure 4.19 Hannah Höch, Poetry Reading at the First International Dada Fair, 1920, Berlin ……………………………………………………………………………………….. 106 Figure 4.20 Marcel Duchamp, L.H.O.O.Q., 1919, Paris, France ………………………….. 106 Figure 4.21 Performance art. Joseph Beuys, I Love America and America Loves Me, still image from the performance, 1974, USA …………………………… 110 Figure 4.22 Installation art. Ernesto Neto, Hyper Culture Madness in the Vertigo World, knitted structure, 2011, Buenos Aires ………………………………….. 111 Figure 4.23 Multimedia. Berni Searle, Snow White, two-projector video, 2001, South Africa ……………………………………………………………………………………….. 112 Figure 4.24 New media. Jeffrey Walter Dooley, Untitled, digital animated painting using a web camera, 2010, South Africa …………………………… 114 Figure 4.25 Cubist collage. George Braque, Fruit Dish and Glass, 1912, France… 117 Figure 4.26 Dada collage. Hannah Höch, Cut with a Kitchen Knife, 1919–1920, Germany …………………………………………………………………………………………….. 117 Figure 4.27 Dada poetry performances. Hugo Ball opening the Cabaret Voltaire in Zurich in 1916 ……………………………………………………………………………….. 119 Figure 4.28 Dada poetry. Bernstein and Federman, The Dada poem for two face to face, written text/image, 1996 …………………………………………………….. 121 x © University of Pretoria