LEA is a publication of Leonardo/ISAST. vol 19 no 1 Volume editors lanfranco aceti and richard rinehart editors Özden Şahin, Jonathan Munro and catherine M. Weir This lea publication has a simple goal: surveying the current trends in augmented reality artistic interventions. There is no other substantive aca- demic collection currently available, and it is with a certain pride that lea presents this volume which provides a snapshot of current trends as well as a moment of reflection on the future of ar interventions. NOT HERE NOT THERE Copyright 2013 ISAST Editorial Address Leonardo Electronic Almanac Leonardo Electronic Almanac Volume 19 Issue 1 Sabanci University, Orhanli - Tuzla, 34956 date of publication January 15, 2013 Istanbul, Turkey issn 1071-4391 isbn 978-1-906897-20-8 Email The isbn is provided by Goldsmiths, University of London. [email protected] leonardo electronic almanac, Volume 19 issue 1 lea publishing & subscription inforMation Web » www.leoalmanac.org Not Here Not There Editor in Chief » www.twitter.com/LEA_twitts Lanfranco Aceti [email protected] » www.flickr.com/photos/lea_gallery » www.facebook.com/pages/Leonardo-Electronic- Co-Editor Almanac/209156896252 Özden Şahin [email protected] Volume editors lanfranco aceti and richard rinehart Managing Editor Copyright © 2013 John Francescutti [email protected] Leonardo, the International Society for the Arts, editors Sciences and Technology Özden Şahin, Jonathan Munro and catherine M. Weir Art Director Deniz Cem Önduygu [email protected] Leonardo Electronic Almanac is published by: Leonardo/ISAST Editorial Board 211 Sutter Street, suite 501 Peter J. Bentley, Ezequiel Di Paolo, Ernest Edmonds, Felice San Francisco, CA 94108 Frankel, Gabriella Giannachi, Gary Hall, Craig Harris, Sibel Irzık, USA Marina Jirotka, Beau Lotto, Roger Malina, Terrence Masson, Leonardo Electronic Almanac (LEA) is a project of Leonardo/ Jon McCormack, Mark Nash, Sally Jane Norman, Christiane The International Society for the Arts, Sciences and Technol- Paul, Simon Penny, Jane Prophet, Jeffrey Shaw, William ogy. For more information about Leonardo/ISAST’s publica- Uricchio tions and programs, see http://www.leonardo.info or contact [email protected]. Contributing Editors Nina Czegledy, Susan Collins, Leonardo Da Vinci, Anna Leonardo Electronic Almanac is produced by Dumitriu, Vince Dziekan, Darko Fritz, Marco Gillies, Davin Passero Productions. Heckman, Saoirse Higgins, Jeremy Hight, Denisa Kera, Frieder Nake, Vinoba Vinayagamoorthy Reposting of this journal is prohibited without permission of Leonardo/ISAST, except for the posting of news and events listings which have been independently received. The individual articles included in the issue are © 2013 ISAST. 2 LEONARDOELECTRONICALMANAC VOL 19 NO 1 ISSN 1071-4391 ISBN 978-1-906897-20-8 E D I T O R I A L Not Here, Not There: An Analysis Of An International The Leonardo Electronic Almanac Collaboration To Survey acknowledges the kind support Augmented Reality Art for this issue of Every published volume has a reason, a history, a ties and differences, contradictions and behavioral conceptual underpinning as well as an aim that ulti- approaches. The interviews add a further layer of mately the editor or editors wish to achieve. There documentation which, linked to the artists’ statements, is also something else in the creation of a volume; that provides an overall understanding of the hopes for is the larger goal shared by the community of authors, this new artistic playground or new media extension. artists and critics that take part in it. What I personally wanted to give relevance to in this volume is the artistic creative process. I also wanted to This volume of lea titled Not Here, Not There had a evidence the challenges faced by the artists in creat- simple goal: surveying the current trends in augment- ing artworks and attempting to develop new thinking ed reality artistic interventions. There is no other sub- and innovative aesthetic approaches. stantive academic collection currently available, and it is with a certain pride that both, Richard Rinehart and The whole volume started from a conversation that I myself, look at this endeavor. Collecting papers and had with Tamiko Thiel – that was recorded in Istanbul images, answers to interviews as well as images and at Kasa Gallery and that lead to a curatorial collabo- artists’ statements and putting it all together is per- ration with Richard. The first exhibition Not Here at haps a small milestone; nevertheless I believe that this the Samek Art Gallery, curated by Richard Reinhart, will be a seminal collection which will showcase the was juxtaposed to a response from Kasa Gallery with trends and dangers that augmented reality as an art the exhibition Not There, in Istanbul. The conversa- form faces in the second decade of the XXIst century. tions between Richard and myself produced this final volume – Not Here, Not There – which we both As editor, I did not want to shy away from more criti- envisaged as a collection of authored papers, artists’ cal essays and opinion pieces, in order to create a statements, artworks, documentation and answers to documentation that reflects the status of the current some of the questions that we had as curators. This is thinking. That these different tendencies may or may the reason why we kept the same questions for all of not be proved right in the future is not the reason for the interviews – in order to create the basis for a com- the collection, instead what I believe is important and parative analysis of different aesthetics, approaches relevant is to create a historical snapshot by focusing and processes of the artists that work in augmented on the artists and authors developing artistic practices reality. and writing on augmented reality. For this reason, Richard and I posed to the contributors a series of When creating the conceptual structures for this col- questions that in the variegated responses of the lection my main personal goal was to develop a link artists and authors will evidence and stress similari- – or better to create the basis for a link – between ear- 4 LEONARDOELECTRONICALMANAC VOL 19 NO 1 ISSN 1071-4391 ISBN 978-1-906897-20-8 ISSN 1071-4391 ISBN 978-1-906897-20-8 VOL 19 NO 1 LEONARDOELECTRONICALMANAC 5 E D I T O R I A L E D I T O R I A L lier artistic interventions in the 1960s and the current porary artists that use augmented reality as a medium. in order to gather audiences to make the artworks making use of their better knowledge of the ar media. artistic interventions of artists that use augmented Here, is not perhaps the place to focus on the role of come alive is perhaps a shortsighted approach that Nevertheless, this is knowledge born out of neces- reality. ‘publicity’ in art history and artistic practices, but a few does not take into consideration the audience’s neces- sity and scarcity of means, and at times appears to be words have to be spent in order to explain that pub- sity of knowing that interaction is possible in order for more effective than the institutional messages arriving My historical artist of reference was Yayoi Kusama licity for ar artworks is not solely a way for the artist that interaction to take place. from well-established art organizations. I should also and the piece that she realized for the Venice Bien- to gain notoriety, but an integral part of the artwork, add that publicity is functional in ar interventions to nial in 1966 titled Narcissus Garden. The artwork was which in order to come into existence and generate What perhaps should be analyzed in different terms the construction of a community – a community of a happening and intervention at the Venice Biennial; interactions and engagements with the public has to is the evolution of art in the second part of the XXth aficionados, similar to the community of ‘nudists’ that Kusama was obliged to stop selling her work by the be communicated to the largest possible audience. century, as an activity that is no longer and can no follows Spencer Tunic for his art events / human in- biennial’s organizers for ‘selling art too cheaply.’ longer be rescinded from publicity, since audience stallation. “By then, Kusama was widely assumed to be a public- engagement requires audience attendance and atten- “In 1966 […] she went uninvited to the Venice Biennale. ity hound, who used performance mainly as a way of dance can be obtained only through communication / I think what is important to remember in the analysis There, dressed in a golden kimono, she filled the lawn gaining media exposure.” 2 The publicity obsession, publicity. The existence of the artwork – in particular of the effectiveness both in aesthetic and participa- outside the Italian pavilion with 1,500 mirrored balls, or the accusation of being a ‘publicity hound’ could of the successful ar artwork – is strictly measured in tory terms of augmented reality artworks – is not which she offered for sale for 1,200 lire apiece. The be easily moved to the contemporary group of artists numbers: numbers of visitors, numbers of interviews, their publicity element, not even their sheer numbers authorities ordered her to stop, deeming it unaccept- that use augmented reality. Their invasions of spaces, numbers of news items, numbers of talks, numbers (which, by the way, are what has made these artworks able to ‘sell art like hot dogs or ice cream cones.’” 1 juxtapositions, infringements could be defined as of interactions, numbers of clicks, and, perhaps in a successful) but their quality of disruption. nothing more than publicity stunts that have little to not too distant future, numbers of coins gained. The The conceptualization and interpretation of this ges- do with art. These accusations would not be just ir- issue of being a ‘publicity hound’ is not a problem that The ability to use – in Marshall McLuhan’s terms – the ture by critics and art historians is that of a guerrilla relevant but biased – since – as in the case of Sander applies to artists alone, from Andy Warhol to Damien medium as a message in order to impose content by- action that challenged the commercialization of the Veenhof’s analysis in this collection – the linkage Hirst from Banksy to Maurizio Cattelan, it is also a passing institutional control is the most exciting ele- art system and that involved the audience in a process between the existence of the artwork as an invisible method of evaluation that affects art institutions and ment of these artworks. It is certainly a victory that a that revealed the complicit nature and behaviors of presence and its physical manifestation and engage- museums alike. The accusation moved to ar artists of group of artists – by using alternative methodological the viewers as well as use controversy and publicity as ment with the audience can only happen through being media whores – is perhaps contradictory when approaches to what are the structures of the capital- an integral part of the artistic practice. knowledge, through the audience’s awareness of arriving from institutional art forms, as well as galler- istic system, is able to enter into that very capitalistic the existence of the art piece itself that in order to ies and museums that have celebrated publicity as an system in order to become institutionalized and per- Kusama’s artistic legacy can perhaps be resumed in achieve its impact as an artwork necessitates to be element of the performative character of both artists haps – in the near future – be able to make money in these four aspects: a) engagement with audience’s publicized. and artworks and an essential element instrumental to order to make art. behaviors, b) issues of art economy and commercial- the institutions’ very survival. ization, c) rogue interventions in public spaces and d) Even if, I do not necessarily agree with the idea of a Much could be said about the artist’s need of fitting publicity and notoriety. ‘necessary manifestation’ and audience’s knowledge of The publicity stunts of the augmented reality interven- within a capitalist system or the artist’s moral obliga- the artwork – I believe that an artistic practice that is tions today are nothing more than an acquired meth- tion to reject the basic necessities to ensure an op- These are four elements that characterize the work unknown is equally valid – I can nevertheless under- odology borrowed from the second part of the XXth erational professional existence within contemporary practices and artistic approaches – in a variety of stand the process, function and relations that have to century. This is a stable methodology that has already capitalistic structures. This becomes, in my opinion, a combinations and levels of importance – of contem- be established in order to develop a form of engage- been widely implemented by public and private art question of personal ethics, artistic choices and ex- ment and interaction between the ar artwork and the institutions in order to promote themselves and their istential social dramas. Let’s not forget that the vast audience. To condemn the artists who seek publicity artists. majority of artists – and ar artists in particular – do 1. David Pilling, “The World According to Yayoi Kusama,” The not have large sums and do not impinge upon national Financial Times, January 20, 2012, http://www.ft.com/ Publicity and community building have become an budgets as much as banks, financial institutions, mili- cms/s/2/52ab168a-4188-11e1-8c33-00144feab49a. 2. Isabelle Loring Wallace and Jennie Hirsh, Contemporary Art artistic methodology that ar artists are playing with by taries and corrupt politicians. They work for years html#axzz1kDck8rzm (accessed March 1, 2013). & Classical Myth (Farnham; Burlington, VT: Ashgate, 2011), 94. 6 LEONARDOELECTRONICALMANAC VOL 19 NO 1 ISSN 1071-4391 ISBN 978-1-906897-20-8 ISSN 1071-4391 ISBN 978-1-906897-20-8 VOL 19 NO 1 LEONARDOELECTRONICALMANAC 7 E D I T O R I A L E D I T O R I A L Site, Non-site, and Website with small salaries, holding multiple jobs and making hundred dollars, and they lend themselves to collabo- In the 1960’s, artist Robert Smithson articulated the and virtual memorials. Moreover, public space is personal sacrifices; and the vast majority of them does rations based on global networks. strategy of representation summarized by “site vs. now truly open, as artworks can be placed any- not end up with golden parachutes or golden hand- non-site” whereby certain artworks were simultane- where in the world, without prior permission from shakes upon retirement nor causes billions of damage Problems though remain for the continued success of ously abstract and representational and could be site- government or private authorities – with profound to society. augmented reality interventions. Future challenges are specific without being sited. A pile of rocks in a gallery implications for art in the public sphere and the in the materialization of the artworks for sale, to name is an “abstract” way to represent their site of origin. discourse that surrounds it.” The current success of augmented reality interven- an important one. Unfortunately, unless the relation- In the 1990’s net.art re-de-materialized the art object tions is due in small part to the nature of the medium. ship between collectors and the ‘object’ collected and found new ways to suspend the artwork online ManifestAR develops projects using Augmented Real- Museums and galleries are always on the lookout for changes in favor of immaterial objects, the problem between website and non-site. In the 21st century, ity (ar), a new technology that – like photography be- ‘cheap’ and efficient systems that deliver art engage- to overcome for artists that use augmented reality new technologies suggest a reconsideration of the re- fore it – allows artists to consider questions like those ment, numbers to satisfy the donors and the national intervention is how and in what modalities to link the lationship between the virtual and the real. “Hardlinks” above in new ways. Unlike Virtual Reality, Augmented institutions that support them, artworks that deliver ar installations with the process of production of an such as Qr codes attempt to bind a virtual link to our Reality is the art of overlaying virtual content on top of visibility for the gallery and the museum, all of it with- object to be sold. physical environment. physical reality. Using ar apps on smart phones, iPads, out requiring large production budgets. Forgetting and other devices, viewers look at the real world that art is also about business, that curating is also Personally I believe that there are enough precedents Throughout the 1970’s, institutional critique brought around them through their phone’s camera lens, while about managing money, it means to gloss over an im- that ar artists could refer to, from Christo to Marina political awareness and social intervention to the site the app inserts additional images or 3d objects into portant element – if not the major element – that an Abramovich, in order develop methods and frame- of the museum. In the 1980’s and 90’s, street artist the scene. For instance, in the work Signs over Semi- artist has to face in order to deliver a vision. works to present ar artworks as collectable and such as Banksy went in the opposite direction, critiqu- conductors by Will Pappenheimer, a blue sky above sellable material objects. The artists’ ability to do so, ing the museum by siting their art beyond its walls. a Silicon Valley company that is “in reality” empty Augmented reality artworks bypass these financial to move beyond the fractures and barriers of insti- contains messages from viewers in skywriting smoke challenges, like daguerreotypes did by delivering a tutional vs. revolutionary, retaining the edge of their Sited art and intervention art meet in the art of the when viewed through an ar-enabled Smartphone. cheaper form of portraiture than oil painting in the aesthetics and artworks, is what will determine their trespass. What is our current relationship to the sites first part of the XIXth century, or like video did in the future success. we live in? What representational strategies are con- Ar is being used to activate sites ranging from Occupy 1970s and like digital screens and projectors have temporary artists using to engage sites? How are sites Wall Street to the art exhibition ManifestAR @ Zero1 done in the 1990s until now, offering cheaper systems These are the reasons why I believe that this collec- politically activated? And how are new media framing Biennial 2012 – presented by the Samek Art Gallery to display moving as well as static images. Ar in this tion of essays will prove to be a piece, perhaps a small our consideration of these questions? The contempo- simultaneously at Bucknell University in Lewisburg, pa sense has a further advantage from the point of view piece, of future art history, and why in the end it was rary art collective ManifestAR offers one answer, and at Silicon Valley in San Jose, ca. From these con- of the gallery – the gallery has no longer a need to worth the effort. temporary non-sites, and through the papers included purchase hardware because audiences bring their “Whereas the public square was once the quintes- in this special issue of lea, artists ask you to recon- own hardware: their mobile phones. sential place to air grievances, display solidarity, sider the implications of the simple question wayn Lanfranco Aceti express difference, celebrate similarity, remember, (where are you now?) The materiality of the medium, its technological revo- Editor in Chief, Leonardo Electronic Almanac mourn, and reinforce shared values of right and lutionary value, in the case of early augmented reality Director, Kasa Gallery wrong, it is no longer the only anchor for interac- Richard Rinehart artworks plays a pivotal role in order to understand its tions in the public realm. That geography has been Director, Samek Art Gallery, Bucknell University success. It is ubiquitous, can be replicated everywhere relocated to a novel terrain, one that encourages in the world, can be installed with minimal hassle and exploration of mobile location based monuments, can exist, independently from the audience, institu- tions and governmental permissions. Capital costs for ar installations are minimal, in the order of a few 8 LEONARDOELECTRONICALMANAC VOL 19 NO 1 ISSN 1071-4391 ISBN 978-1-906897-20-8 ISSN 1071-4391 ISBN 978-1-906897-20-8 VOL 19 NO 1 LEONARDOELECTRONICALMANAC 9 C O N T E N T S C O N T E N T S Leonardo Electronic Almanac Volume 19 Issue 1 5 106 EDITORIAL AUGMENTED RESISTANCE: THE Lanfranco Aceti POSSIBILITIES FOR AR AND 9 DATA DRIVEN ART INTRODUCTION Richard Rinehart + Interview, Statement, Artwork Conor McGarrigle 12 THE VARIABLE MUSEUM: OFF-TOPIC ART 122 SITUATED SOUNDSCAPES: REDEFINING + Interview, Statement, Artwork MEDIA ART AND THE URBAN EXPERIENCE John Bell + Interview, Statement, Artwork 20 TRANSLOCATED BOUNDARIES Natasa Paterson & Fionnuala Conway + Interview, Statement, Artwork 140 A NEW RELIC EMERGES: IMAGE AS Jacob Garbe SUBJECT TO OBJECT 44 IN BETWEEN: EXPERIENCING LIMINALITY + Interview, Statement, Artwork + Interview, Statement, Artwork Rebecca Peel Dragoş Gheorghiu & Livia Ştefan 158 RE-VISUALIZING AFGHANISTAN IN 62 HACKING: A NEW POLITICAL AND “WHAT IF IM THE BAD GUY”: USING CULTURAL PRACTICE PALIMPSEST TO CREATE AN AR DOCUMENTARY Christina Grammatikopoulou + Interview, Statement, Artwork 76 CONNECTICITY, AUGMENTED PERCEPTION Aaron A. Reed & Phoenix Toews OF THE CITY + Interview, Statement, Artwork Salvatore Iaconesi & Oriana Persico 10 LEONARDOELECTRONICALMANAC VOL 19 NO 1 ISSN 1071-4391 ISBN 978-1-906897-20-8 ISSN 1071-4391 ISBN 978-1-906897-20-8 VOL 19 NO 1 LEONARDOELECTRONICALMANAC 11 A R T I C L E A R T I C L E A B S T R A C T ConnectiCity, We constantly re-interpret and transform the spaces around us. The ways in which we constantly personalize the spaces which we tra- augmented verse and in which we perform our daily routines communicate information about emotions, knowledge, skills, methodologies, cultures and desires. perception of the city This process takes place in digital realms as well, which start to ubiqui- tously merge with cities. Mobile devices, smartphones, wearables, digital tags, near field communi- by 1. INTRODUCTION cation devices, location based services and mixed/augmented reality have turned the world into an essentially read/write, ubiquitous publishing surface. SALVATORE IACONESI We constantly re-program the spaces around us. 1 The usage of mobile devices and ubiquitous technologies alters the under- As pointed out by Edward Krupat and William Guild, 2 standing of place. & ORIANA PERSICO Jack L. Nasar 3 and Susan L. Scheiberg 4 the ways In our research, we investigated the possibilities to conceptualize, design in which we reinterpret and personalize spaces ef- fectively convey important information about our and implement a series of usage scenarios, moving fluidly across arts, sciences emotional states, working methodologies, knowledge, Salvatore Iaconesi and the practices of city governance and community design. skills, cultural backgrounds, desires and our visions. La Sapienza University of Rome The objective we set forth sees the creation of multiple, stratified nar- It is a pragmatic manifestation of the ways in which ISIA Design Florence we perceive our living environments, a constructiv- ratives onto the city, set in place by citizens, organizations and administrations. Rome University of Fine Arts ist act of world-making: “In the course of time every IED Rome These real-time stories and conversations can be captured and observed, to gain section and quarter of the city takes on something [email protected] of the character and qualities of its inhabitants. Each insights on fundamental issues such as ecology, sustainability, mobility, energy, separate part of the city is inevitably stained with the politics, culture, creativity and participatory innovation processes. Oriana Persico peculiar sentiments of its population.” 5 La Sapienza University of Rome These methodologies for real-time observation of cities help us take part [email protected] On the other side, the forms and essence of urban in a networked structure, shaped as a diffused expert system, capturing dissemi- space directly affect people’s behavior, describing nated intelligence to coagulate it into a framework for the real-time processing of what is possible or impossible, allowed or prohibited, suggested or advised against. 6 urban information. Our experience of the contemporary world is charac- terized by the presence of a ubiquitous digital mem- brane, 7 8 represented and accessed by technologies and networks whose wide availability and accessibility transform our perception of reality and enable us to program and re-interpret the spaces around them allows us to fill space/time with digital information plan and act in novel ways. using digital tools; and allow opportunities for interaction, interrelation » to observe in real-time the digital discussions and communication. 9¯11 A set of objectives has been set forth in the process: which take place in cities, to both transform them » to gain a better understanding of human presence into a component of the ubiquitous information These observations allowed us to embrace a research in contemporary urban spaces; landscape of urban spaces and to understand the process to investigate the ways in which ubiquitous » to understand the ways in which it is possible emotional approaches, themes and issues which technologies and networks alter our sense of place, to understand and visualize the way people re- emerge from human perception of the city. Con- 76 LEONARDOELECTRONICALMANAC VOL 19 NO 1 ISSN 1071-4391 ISBN 978-1-906897-20-8 ISSN 1071-4391 ISBN 978-1-906897-20-8 VOL 19 NO 1 LEONARDOELECTRONICALMANAC 77 A R T I C L E A R T I C L E VersuS, speaking circle, 2011, Salvatore Iaconesi and Oriana Persico, real-time data and information visualization. © Iaconesi and Persico, 2011. City of Rome, February 15th 2012. Each dot around the circle represents a social network user residing in the city of Rome. 1000 users are shown with their mutual interactions from 2pm to 6pm. The 1000 users are chosen from all users in Rome in order to define a “community”: a group of people who intensively exchange information. Courtesy of Art is Open Source. to one which is multiplied according to all the individu- VersuS, flower garden, 2011, Salvatore Iaconesi and Oriana als which experience that specific location. Persico, real-time data and information visualization. © Iaconesi and Persico, 2011. City of Rome, February 15th 2012. However both modalities potentially allow us to imag- Each sphere represents a single social network message ine and design stratified spaces in which multiple points sent from 10am to 10:30am from the territory of the city of of view find their expression, much in the same ways Rome. Each message is color-coded according to the emotion in which Gilles Clément, 17 John Paul Eberhard 18 and expressed in the message. The height of the message Almo Farina 19 describe our landscapes from entirely describes the length of the conversation it generated. different perspectives: space/time as a continuous, Courtesy of Art is Open Source. emergent, fluid, recombinant stratification of analog/ which originally underpinned the inception of the digital information/communication/interaction. World Wide Web. Alowing us to expand our possibili- stantly generating insights on issues related to Devices such as the Sony Walkman allow us to trav- ties for awareness and consciousness through the ecology, mobility, land use, wellness, need for ser- erse urban spaces – with their cognitive, aesthetic The possibility to access these multiplied definitions of wide and ubiquitous availability of multiple sources vices and infrastructures, sense of place, definition and moral significance – and to benefit from the use space alter our own perception of it, opening it to cul- of information, which are hyperlinked to the physical of emergent boundaries and attention groups; of a critical tool in the management of our space and tures, backgrounds and symbolical apparatuses which elements of our reality. » to propose novel forms of mixed-media urban time, in the construction of boundaries around our- have the potential to be entirely different from our own, interstices in which multiple cultures, languages, selves, as well as in the creation of sites of fantasy and using devices which we hold in our pockets. Operating in this direction, it is possible to imagine religions and political orientations can meet and memory.13 and design forms of disseminated intelligence which interact; Derrick De Kerckhove 20 suggested that the augmen- can be coagulated in multiple ways by actors travers- » create methodologies, for all actors involved, to Mobile devices, smartphones, wearables, digital tags, tation of architecture, should include the concepts ing cities and using mobile devices to enact novel transform these possibilities into tools for aware- location based services and mixed/augmented reality ness and consciousness about the expression of have gone much further in this direction, turning the needs and emotions of people, for ethical, sustain- world into an essentially read/write, ubiquitous pub- able, participatory policies, plans, businesses, initia- lishing surface 14 and altering our understanding of tives and processes; the spaces around us. 15 » promote choral initiatives, engaging citizens, orga- nizations and institutions. As David Morley describes: “The mobile phone is often understood (and promoted) as a device for connect- ing us to those who are far away, thus overcoming 2. TRANSFORMING THE SENSE OF PLACE distance – and perhaps geography itself.” 16 Portable devices transform our experience of space/ In this analysis, the possibility to compress space and time. time enables novel opportunities to interconnect and relate to objects, processes, places and people, but For example, the Sony Walkman powerfully intro- also fills “the space of the public sphere with the chat- duced the possibility of being able to be in two places ter of the earth, allowing us to take our homes with us, at once 12 through the personalized sounds playing just as a tortoise stays in its shell wherever it travels.” through our headphones, creating a powerful conjunc- This modality describes a direct, personalized inter- VersuS, crisis, 2011, Salvatore Iaconesi and Oriana Persico, real-time data and information visualization. © Iaconesi and Persico, tion between physical space and the imaginary space vention into the design of space, in both its form and 2011. City of Rome, during a research in winter 2011. The image represents a screen capture of a visualization of the messages of created by the music. function, creating a definite shift in the definition of citizens of the city of Rome using social networks to discuss the financial crisis in Italy. Red colored dots represent expressions (urban) landscape: from a purely administrative one, of particular verbal violence. Courtesy of Art is Open Source. 78 LEONARDOELECTRONICALMANAC VOL 19 NO 1 ISSN 1071-4391 ISBN 978-1-906897-20-8 ISSN 1071-4391 ISBN 978-1-906897-20-8 VOL 19 NO 1 LEONARDOELECTRONICALMANAC 79 A R T I C L E A R T I C L E forms of reading/writing of spaces, symbols and con- figurations, moving fluidly across digital and physical domains. Nicola Green 21 investigated a similar approach, high- lighting the role of mobile devices as spatial/temporal mediators, exposing alternative perceptions and be- haviors in human beings and, thus, proposing different usage grammars for spaces and timeframes. come together, under the form of a peer to peer creation of several prototypes which implement the The process designed and produced for “Rel:Attiva This result can be combined with the ones produced ecosystem in which each subject is an informed, concepts conceived in the investigation phase. Presenza” can be thought of as a practice of archival by Marsha Berry and Margaret Hamilton while ob- aware agent, thus describing novel forms of gover- of the perceptions, experiences and narratives of the serving the usage of mobile devices on trains: “public nance and decision-making processes; 31¯33 Rel:attiva presenza people who live in the territory, and as a research places and spaces are being transformed into hybrid » to reflect on the life and expressions of cities and The first prototype was designed in Mexico City at the into their accessibility. The prototype was exhibited geographies through the introduction of new spatial of their inhabitants, to identify new policies, new Franz Mayer Museum, and it was titled “rel:attiva pre- under the form of an architectural intervention in the infrastructure.” sustainable, ethical business models, urban plan- senza.” The occasion for this project was the presenta- Coyoacan neighborhood, transforming surfaces into ning processes, grass-roots initiatives and opera- tion of the paper “architettura rel:attiva” at the Sev- screens which acted as an accessibility layer for the tive models; enth International Meeting on the Revitalization of the history and emotions of the inhabitants. CONNECTICITY, METHODOLOGY » to use the insights provided by models such as the Historical Centers, focused on the idea of architecture living labs and other user-centric innovation pro- as mediator of the historical and contemporary city. The results of this first experience deeply inspired the ConnectiCity is an arts/science meta-project which cesses 34¯40 in the creation of novel practices following ones. investigates the possibilities offered by the progres- for citizens, organizations and administrations; “Rel:attiva presenza” was designed as a video projection sive availability of real-time, ubiquitous, digital layers of » to reflect on the themes of cognitive accessibil- mapping and of a sound environment in the cloister of The Atlas of Rome information. It is able to design and implement a series ity for this kind of information, analyzing visual the Italian Cultural Institute in Mexico City, in the Coyo- The following prototype created for the ConnectiCity of prototypes which would pursue the following goals: and multi-modal representation and interaction acan neighborhood. The video projection was created project was, in more than one way, a direct extension metaphors that would allow to maximize the effec- by assembling video footage and images from different of the first one. » to create a set of experiences allowing tiveness, ease of use and understanding of these epochs, describing the mutation of the neighborhood » to capture in real-time various forms of city- complex information scenarios; 41¯44 across the years, starting from the beginning of the A 35 meter long architectural projection and relevant user generated content from a variety » to confront with validation models that would century. The images and footage were assembled to- sound environment was created for Rome’s “Festa of sources, including social networks, websites, allow to assess the quality, relevancy and reliability gether with geographical representations of the evolu- dell’Architettura” (Architecture Fair), organized by the mobile applications, of harvested data, affected by information noise, tion of the territory and of the land use in the neigh- City Administration together with the Italian Order of » to interrelate information to the territory 7 8 23 digital-divide related issues (e.g.: not all citizens use borhood. The resulting visual narrative constituted a the Architects in the enormous entrance corridor of using Geo-referencing, Geo-Parsing and Geo- social networks or imagine that they can use them sort of conceptual time-lapse video, in which the life the ex-Mattatoio (ex Slaughterhouse) in the Testaccio Coding techniques; 24¯27 to express opinions about their city)interpretation of the neighborhood was shown in its evolution. The neighborhood of the city. » to analyze and classify information using errors; sound environment was assembled by manually har- Natural Language Processing and Named En- » to imagine and implement strategies for open vesting field-recordings in the neighborhoods streets The Atlas of Rome’s purpose was to portray in real- tity Recognition techniques to identify users’ access of information and services, to reconfigure and markets, collecting dialogues, typical noises, time the evolution of the visions, desires and actions emotional approaches, forms of expression, them as novel forms of freely usable spatial infra- sounds of transit, mobility, transport, commerce, chit- created by architects, institutions, operators and citi- topics of interest, discussion graphs, networks structure; chat, voices in bars and restaurants. zens onto the city of Rome on a series of fundamental of attention and of influence, trending issues, » to reflect on the new models for identity, privacy, themes such as culture, creativity, education, urban evaluations of satisfaction, well-being and hap- ethics and on the new possible emerging defini- The installation was proposed as a novel way to stratify planning, commerce, arts, security and health, classi- piness, and other forms of expression (using tions of public and private space. the neighborhood’s history into an accessible, narrative fied in 16 information domains to describe the overall techniques designed by taking into account the form. By looking at and listening to “rel:attiva presenza,” wellness of the city. many researches of this kind which have been the history of the place could be experienced along performed over these last few years, including RESULTS multiple points of view, in its evolution towards its fundamental contributions which have been present condition. Images and sounds were completely adopted from; 28¯30 The ConnectiCity project is an on-going process start- “user-generated,” as they had been produced by long- » to imagine initiatives through which this informa- ed in 2008. Since then a continuous refinement of time inhabitants of the place, just as the voices and tion allows central administration and individuals to the methodologies and technologies has allowed the sounds collected using the location’s daily life. 80 LEONARDOELECTRONICALMANAC VOL 19 NO 1 ISSN 1071-4391 ISBN 978-1-906897-20-8 ISSN 1071-4391 ISBN 978-1-906897-20-8 VOL 19 NO 1 LEONARDOELECTRONICALMANAC 81 A R T I C L E A R T I C L E A peculiar geographical visualization captured most pop up viewports contextually appeared in front of of the attention of the visitors. Here, color-coded them onto the architectural projection, showing texts, circles represented localized elements of information. videos, images and interactive experiences. A map was not shown under the circles, but their rela- tive geographical positions were correctly calculated. This immediate responsiveness of such a large scale Points were connected by similarity: two points on the projection proved to produce radically positive effects The Atlas of Rome, 2010, Salvatore Iaconesi and Oriana visualization were connected if they were relevant to on visitors. The fact that a large-scale architectural Persico, Urban Screen, 35 meters. the same themes. surface was actually responding in real-time to their © Iaconesi and Persico, 2010. interactions, powerfully combined with the tangible A map formed, composed, not through natural or effect of having the possibility to publish one’s own A complex activity was set-up in the organization of source (for example when the information source administrative boundaries, but through the emotions information onto the projection. The combined effect the project: directly corresponds to a specific place, such as and ideas of citizens. It was constructed continuously of being able to both contribute and interact had a in the case of museums) or extracted, whenever through the activities which take place in real-time in distinct empowering effect on people, who spontane- » an information harvesting scheme was created possible, by Geo-Parsing schemes, which was per- the city of Rome. ously started to discuss possible uses for this kind of to capture real-time information from a variety of formed by using a large database of Named Enti- system in areas such as participatory urban planning, sources: ties with a geographical connotation, including the This kind of emergent geography has proven to be policy making and decision-making at the city level. » institutional and professional information sourc- names of streets, malls, cinemas, museums, land- extremely effective when used as a lens, as a new es such as blogs, websites, news feeds about marks, neighborhoods, common alternative names perception of the city in which it is the behavior of ConnectiCity Neighborhood Edition the city of Rome and relevant to the chosen of places, pubs, bars, shops, stores, gyms, and other people – individually or through their organizations – The system created for the Atlas of Rome was also themes dozens of types of locations for which names could to describe forms, aggregations, coherences and in- implemented in a smaller scale, dedicated to provide » relevant user accounts which were identified on be identified in the text of the harvested content ( consistencies. The analysis of this representation has novel scenarios for the life of neighborhoods. social networks, among those citizens, profes- a multi-modal text-matching engine compared the been of fundamental value in gathering the insights sional operators, members of the institutions, strings in multiple ways for similarity and for the which were used to create the following prototypes of An Urban Screen was designed to capture in real-time museums, art galleries, spaces for creativity and textual context in which the identified words were the ConnectiCity project. the social network conversations which could be entertainment, social aggregation points, active found, to be able to filter out most false-positive identified as originating from within the territory of communities and to continuously benefit from results, and obtaining a correctness of about 97%); Visitors could use their mobile devices and a series the neighborhood. For this purpose, the Twitter, Flickr relevant updates on the chosen themes; » a series of direct input channels were created to of multitouch surfaces to interact with the part of and FourSquare social networks were used, thanks to » harvesting took place using a selection of tech- accept content (text, images and videos) from the projection that they had in front of them. The the accessibility of their geographical features. niques, involving both automatic processes (rss citizens using mobile devices and a series of multi- position of the multitouch terminal, the geographical feed parsing, micro-formats, public api usage, au- touch kiosks which were set-up in various areas of coordinates identified by the smartphone application Harvested information was processed using the same, thorized web-scraping, database connections, im- the city. and a customized wireless network setup allowed the yet evolved, strategies described for the Atlas, and port of structured data in a variety of formats) and system to understand which part of the projection to were shown on the Urban Screen using a simple, mini- manual ones (such as in the case of those organi- Collected information was shown on the 35 meter activate, eventually alerting the user that the area was mal interface in which large, black dots represented zations which sent us press releases to be added wide surface using a processing application. A series currently being used by another visitor, suggesting single contributions, appearing onto the screen and into the system); of different information visualizations were designed to moving slightly to the left or right to obtain a free connected to the edges of the screen. Here, textual » information was parsed using Natural Language to convey information according to different meta- projection area. representations of the content were presented. Also, Analysis, 45 46to classify information according to phors. Somewhere dedicated to aggregating informa- two or more dots were visualized as connected when the selected topics; tion according to themes, time-frames and the types Individuals could navigate detailed versions of the they represented messages dealing with the same » information was then geo-referenced either by of activities. content by touching interface elements, choosing the topic or if they represented direct interactions (e.g.: using the coordinates provided by the information bits of information they wished to experience. Large re-tweets and comments). 82 LEONARDOELECTRONICALMANAC VOL 19 NO 1 ISSN 1071-4391 ISBN 978-1-906897-20-8 ISSN 1071-4391 ISBN 978-1-906897-20-8 VOL 19 NO 1 LEONARDOELECTRONICALMANAC 83