FINAL GRANT REPORT AFOSR-BAA-2008-1 Socio-Cultural Modeling of Effective Influence Grant No. FA2386-09-1-4008 AOARD 094008 Period of Performance: 17 Feb 2009 - 17 May 2010 Kay O’Halloran Multimodal Analysis Lab Interactive & Digital Media Institute (IDMI) National University of Singapore Kevin Judd School of Mathematics and Statistics University of Western Australia Perth, Western Australia 6009 Project Title Synthesis of Systemic Functional Theory & Dynamical Systems Theory for Socio- Cultural Modeling 1. Overview The overall research plan of this project is to apply methods and principles of dynamical systems theory (DST) to base data derived from systemic functional theory (SFT) analysis of text and multimedia resources, with the aim to identify and track evolving patterns, in particular those related to stability and instability. The goal of the project is to develop theory and algorithms, and demonstrate their validity and potential with case studies. In this first stage of the project, the goal was to perform detailed SFT analysis of sample texts to provide test-case base data for DST analysis. SFT case studies include online discourses about global financial crisis and climate change. For case studies concerning climate change, a particular focus is on events occurring around Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 2. REPORT TYPE 3. DATES COVERED 26 JAN 2011 FInal 17-02-2009 to 17-05-2010 4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER Synthesis of Systemic Functional Theory & Dynamical Systems Theory FA23860914008 for Socio-Cultural Modeling 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER Kay O’Halloran 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION National University Singapore,9 Prince George’s Park,National REPORT NUMBER N/A University of Singapore,Singapore 118408,SP,118408 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S) AOARD, UNIT 45002, APO, AP, 96337-5002 AOARD 11. SPONSOR/MONITOR’S REPORT NUMBER(S) AOARD-094008 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited 13. SUPPLEMENTARY NOTES 14. ABSTRACT The overall research plan of this multiyear project is to apply methods and principles of dynamical systems theory (DST) to base data derived from systemic functional theory (SFT) analysis of text and multimedia resources, with the aim to identify and track evolving patterns, in particular those related to stability and instability. The goal of the project is to develop theory and algorithms, and demonstrate their validity and potential with case studies. Progress toward that goal, including reports of six case studies. What has emerged from analyses available to date is essentially the confluence of particular linguistic choices with respect to content and text production. 15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF ABSTRACT OF PAGES RESPONSIBLE PERSON a. REPORT b. ABSTRACT c. THIS PAGE Same as 26 unclassified unclassified unclassified Report (SAR) Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18 the time of the United Nations Copenhagen Climate Change Summit 2009 (COP15) which took place in Copenhagen, Denmark on 7-18 December 2009. In parallel with the gathering of SFT base data has been the development of software tools for manipulation, analysis, and visualisation of the base data. The analysis of SFT base data mapped the ‘landscape’ on which the dynamics of the text plays out. This mapping of the landscape involved visualisation and understanding the distribution of information of high dimensionality. Standard mathematical methods of mapping were applied, such as principle component analysis, local linear embedding, recurrence analysis, and clustering. These preliminary experiments were designed to determine both the identifying features of the texts and what existing mapping methods are most useful, before developing new techniques for capturing the dynamics of the text in the second phase of the project. At present, work is under way to develop the DST analysis and tools on the extended SFT base data with the aim of revealing dynamics of meaning making in discourse. Research output has been three conference presentations [8, 9, 10], a paper submitted to a linguistics journal [5], a book chapter [6], and a Master’s thesis [7]. The application of the mathematical visualisation tools on the base data in the Masters thesis provided interesting analytical insights which are currently being written up as further journal submissions. There are no problems and issues concerning ethics as the project uses only publicly available resources and anonymous public comments. 2. Activities of Personnel Kay O'Halloran provides expert knowledge for assessing the strategic direction for the DST analysis. She oversees the SFT analysis of the case studies and supervises Research Assistant Marissa E and Masters student Loh Boon Liang. In addition, she is responsible for administration of the grant and negotiations with NUS. Kay O’Halloran presented work on the project during her plenary talk at the 6th Annual Conference of the Latin American Systemic-Functional Linguistics Association (ALSFAL) in Fortaleza, Brazil on 7-9 October 2010 [10]. 2 Kevin Judd has developed the software tools for SFT analysis and visualisation, and the tools for the mathematical analysis and visualisation methods. Kevin Judd works closely with other members of the research team in order to develop an understanding of SFT and how to best apply mathematical analysis techniques to the SFT base data. Marissa E is the Research Assistant who undertakes the SFT analysis of texts and applies the analysis and visualisation tools and techniques. Marissa has a Masters Degree in English Language Studies in the Department of English Language & Literature NUS with a focus on SFT. Marissa gave a conference paper on the results of her analysis at the 36th International Systemic Functional Congress (ISFC09) at Tsinghua University Beijing China on 14-18 July 2009 [8] and she presented a paper on recent findings with Kay O’Halloran at the Fifth International Conference on Multimodality (5ICOM) at the University of Technology Sydney on 1-3 December 2010 [9]. Loh Boon Liang, a postgraduate student who recently completed his Masters Degree in the Department of English Language & Literature NUS, undertook detailed SFT analyses of an on-line CNN article on the global financial crisis for his Masters project. Boon Liang has an Honours degree in physics, and he has also applied some of the preliminary DST analyses. Boon gave a conference paper on the results of his research at the 36th International Systemic Functional Congress (ISFC09) at Tsinghua University Beijing China on 14-18 July 2009 [8]. 3. Theory, Analytical Tools and Methods Systemic Functional Theory (SFT) The key terms in Systemic Functional Theory (SFT) are systemic and functional. That is, language and other resources (e.g. images and sound resources) are conceptualised as inter-locking systems of meaning which realise four functions. That is, the systems are used to (a) construct our ideas about the world (experiential meaning), (b) establish logical relations (logical meaning), and (c) enact social relations which create a stance towards the ideas which are expressed (interpersonal meaning). These three ‘metafunctions’ are enabled through textual meaning, the 3 fourth metafunction concerned with the organisation of the message. Comprehensive descriptions of the grammatical systems and structures for the four metafunctions (experiential, logical, interpersonal and textual) are provided according to hierarchical ranks and strata (e.g. sounds, word groups, clauses, and complex discourse structures in language, and elements, figures and episodes in images). This provides a conceptual framework for analysing ideas and informational content (configurations of agents, participants, processes and circumstances), the social relations which are established (power, status and affect), the orientation to the ideas which are presented (modality and truth value), and the ways in which the choices are organised to achieve specific purposes (e.g. points of departure, given and new information). The systemic functional descriptions of the grammatical systems and structures are semantic in nature, thus providing a powerful tool for mapping the meaning potential of resources as sets of inter-related systems, and for analysing the meaning arising from the actual choices made in the text. SFT provides a means for bridging the ‘semantic gap’ which exists between the text and its meaning. In addition, the metafunctional principle provides an integrating platform for describing how language and other resources (e.g. images and sound) work together to fulfil particular objectives. While there are alternative approaches to study texts and generate and extract essential data for mathematical analysis, SFT is chosen because: (1) In this approach, language is conceptualised as systems of meanings, accompanied by forms through which those meanings are realized in text. As such, SFT is designed to account for how language is used, and it provides tools for the interpretation of (a) the underlying linguistic systems, (b) the elements of linguistic structure, and (c) the text itself. (2) Linguistic choices are explained by reference to their function in the total linguistic system, resulting in a comprehensive description of essential information in the data extraction process. The systemic functional model 4 includes system choices for: (a) Content plane (word, word group, clause, clause complex and paragraph) (b) Expression plane (phonology, typography and graphology) (c) Context plane for register (field: what the text is about; tenor: the types of social relations; mode: spoken/written) and genre (text type). (3) Systemic functional analysis reveals subtle shifts across four strands of meaning (experiential meaning, logical reasoning, interpersonal orientation and textual organization). Configurations of processes, participants and circumstance (i.e. happenings involving agents, actors and other participants) and logical reasoning about those configurations are analysed. The representations are conceptualised as interactive events between ‘speakers’ where information and goods and services are exchanged. The textual organisation of the linguistic elements reveals the points of departure, and given and new information. (4) The systemic functional model has been extended to resources other than language to provide a unified approach to data analysis so that extracted information includes the co-contextualising or re-contextualising meanings arising from the interaction between linguistic, visual and aural resources. Systemics Software The main tool for creating the SFT base-data is the Systemics software. This software was originally developed by Kay O’Halloran and Kevin Judd in 1999-2002 for research and teaching in SFT. The original Systemics software provided a cross- platform GUI application for SFT annotation of text at the rank of word group, clause, clause complex, and discourse. These annotations are stored in a database. The software provided basic search functions based on tag count frequencies. The Systemics software has been extensively revised and extended for this project by adding new annotation features, more sophisticated search features, and scientific visualisation techniques. The new annotation features allow better analysis of embedded clause structures, discourse chains and lexical items. 5 Figure 1(a). SFT Clause Annotation Figure 1(b). SFT Clause Complex Annotation 6 Figure 1(c). SFT Discourse Annotation The new search features in Systemics include word-tag concordances, complex pattern-matching, and complex logical relations of tags across systems and different databases. The visualisation features combine a number of mathematical techniques for feature extraction, correlation analysis and cluster analysis. Mathematical Analysis and Visualisation of SFT Base-data The aim of the mathematical analysis is to reveal and understand how meaning is being made in texts, in particular the dynamic accumulation of meaning as the text unfolds. The SFT annotations of the text provide an extensive decomposition of the text into functional elements, typically word groups. The meaning potential of these functional elements is multidimensional in the sense that each element plays a role in the different SFT systems. This results in a complex data structure, where the text is decomposed in word groups, which are further grouped into larger and larger groups which are analysed multiple times according to their metafunctional roles. The data structure includes annotations that are attribute tags attached to each element, or group, where the attribute tags are options drawn from the hierarchically organised 7 SFT systems. One of the projections of this data structure we have extensively explored is clause- tag associations, which can be conveniently represented as a binary matrix. In this matrix representation each row is associated with a clause, each column is associated with a tag, so tags are attached to the corresponding clause and vice versa. In this data projection the text is represented as a cloud of points in a dual vector space, the clause-space and tag-space, corresponding to the row and column spaces of the binary matrix. The text can be investigated through examination of the dual space, for example, using singular value decomposition (SVD) and clustering techniques. The features of the text are visualised using various network diagrams and by projection of the features back onto the text using colour tints and font attributes. The various visual renderings are transformations and filterings of the underlying data structure. One of the key innovations of our work is that many of the qualitative aspects of meaning making in a text previously described by Halliday (1978, 1994), Halliday and, Matthiessen (2004), Martin (1992) and others, can be associated with quantifiable aspects of our data structures. For example, qualitative features can be identified with reference points in the clause-tag dual-space. The degree to which a text possesses a feature can be described in terms of barycentric coordinates with respect to predefined reference points and metrics. The dynamics of the unfolding meaning in a text can be quantified by the path of the text in clause-space as described by the barycentric coordinates. We have also examined the dynamics of texts in terms of the accumulation of quantitative measures over the logical structure of the text (Figure 2), and by state machines derived from projection and clustering of the underlying data structure (Figure 3). One of the key advantages of our quantitative description of the meaning making in texts is that it enables comparative analysis of texts, and the identification of features of a text that deviate from genre norms (Figure 4). In addition, it is possible to interpret covert messages (experiential, logical, interpersonal and textual) which are not immediately apparent. 8 Figure 2. Accumulation of features over logical structure Figure 3. A state machine based on clustering in dual-space 9