Lecture Notes in Computer Science 7468 CommencedPublicationin1973 FoundingandFormerSeriesEditors: GerhardGoos,JurisHartmanis,andJanvanLeeuwen EditorialBoard DavidHutchison LancasterUniversity,UK TakeoKanade CarnegieMellonUniversity,Pittsburgh,PA,USA JosefKittler UniversityofSurrey,Guildford,UK JonM.Kleinberg CornellUniversity,Ithaca,NY,USA AlfredKobsa UniversityofCalifornia,Irvine,CA,USA FriedemannMattern ETHZurich,Switzerland JohnC.Mitchell StanfordUniversity,CA,USA MoniNaor WeizmannInstituteofScience,Rehovot,Israel OscarNierstrasz UniversityofBern,Switzerland C.PanduRangan IndianInstituteofTechnology,Madras,India BernhardSteffen TUDortmundUniversity,Germany MadhuSudan MicrosoftResearch,Cambridge,MA,USA DemetriTerzopoulos UniversityofCalifornia,LosAngeles,CA,USA DougTygar UniversityofCalifornia,Berkeley,CA,USA GerhardWeikum MaxPlanckInstituteforInformatics,Saarbruecken,Germany Charlotte Magnusson Delphine Szymczak Stephen Brewster (Eds.) Haptic and Audio Interaction Design 7th International Conference, HAID 2012 Lund, Sweden, August 23-24, 2012 Proceedings 1 3 VolumeEditors CharlotteMagnusson DelphineSzymczak LundUniversity,DepartmentofDesignSciences P.O.Box118,22100Lund,Sweden E-mail:{charlotte.magnusson,delphine.szymczak}@certec.lth.se StephenBrewster UniversityofGlasgow,SchoolofComputingScience Glasgow,G128QQ,UK E-mail:[email protected] ISSN0302-9743 e-ISSN1611-3349 ISBN978-3-642-32795-7 e-ISBN978-3-642-32796-4 DOI10.1007/978-3-642-32796-4 SpringerHeidelbergDordrechtLondonNewYork LibraryofCongressControlNumber:2012944601 CRSubjectClassification(1998):H.5.1-2,K.4,H.1.2,H.5.5,K.3.1 LNCSSublibrary:SL3–InformationSystemsandApplication,incl.Internet/Web andHCI ©Springer-VerlagBerlinHeidelberg2012 Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerialis concerned,specificallytherightsoftranslation,reprinting,re-useofillustrations,recitation,broadcasting, reproductiononmicrofilmsorinanyotherway,andstorageindatabanks.Duplicationofthispublication orpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember9,1965, initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Violationsareliable toprosecutionundertheGermanCopyrightLaw. Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublicationdoesnotimply, evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotectivelaws andregulationsandthereforefreeforgeneraluse. Typesetting:Camera-readybyauthor,dataconversionbyScientificPublishingServices,Chennai,India Printedonacid-freepaper SpringerispartofSpringerScience+BusinessMedia(www.springer.com) Preface Thesenseoftouch,togetherwithgesturesandsounds–infactallthe nonvisual interaction channels – are as yet sadly undervalued and underused in most ap- plications today. Yet these modalities are becoming increasingly important for good mobile user experiences. One can no longer just focus on the screen in the mobile use situation; the use is embedded in a context where people and events intheenvironmentmayneedyourattention.Morenonvisualinteractiondesigns willsimplymakeapplicationsanddeviceseasierforeveryonetouse.HAID2012, the 7thInternationalWorkshoponHapticandAudio InteractionDesign,brings togetherefforts in this challengingarea.How do we designeffectively for mobile interaction? How can we design effective haptic, audio, and multimodal inter- faces? In what new application areas can we apply these techniques? Are there design methods that are useful? Or evaluation techniques that are particularly appropriate? Intheseproceedingsyouwillfindpapersonhownavigationcanbesupported by the use of sounds, touch, and gestures, papers on how nonvisual modalities can create rewarding user experiences, work on how interaction, objects, and interfaces should be designed – but also reports from more detailed studies and researchon how to properly evaluate these types of designs. The importance of sounds, gestures, and touch in mobile settings is evident by the fact that HAID 2012 was organized in collaboration with HaptiMap. This EU project on haptic, audio, and visual interfaces for maps and location- based services is funded by the European Commission in its Seventh Frame- work Programme under the Cooperation Programme ICT – Information and Communication Technologies (Challenge 7 – Independent living and inclusion). More information about HaptiMap can be found at the project website: www.haptimap.org. The papers in these proceedings reflect promising progress in the field of haptics and audio interaction, but also show that these types of interactions present challenges – and that much work remains to be done in the field. The day when it is taken for granted that haptics and audio are important and necessarycomponents in allapplications anddevices is still distant– but HAID 2012 has taken us one step further toward achieving this goal. August 2012 Charlotte Magnusson Organization The 7th International Workshop on Haptic and Audio Interaction Design was organized by Lund University (Sweden), Certec, Division of Rehabilitation En- gineeringResearchintheDepartmentofDesignSciences,FacultyofEngineering and the University of Glasgow (UK), Department of Computing Science. Conference Chairs Charlotte Magnusson Lund University, Sweden Certec, Rehabilitation Engineering Research Kirsten Rassmus-Gr¨ohn Lund University, Sweden Certec, Rehabilitation Engineering Research Stephen Brewster University of Glasgow, UK Department of Computing Science Program Committee Helen Petrie University of York, UK David McGookin University of Glasgow, UK Ian Oakley Universidade da Madeira, Portugal Sile O’Modhrain University of Michigan, USA Eva-Lotta Sallna¨s Royal Institute of Technology, Sweden Antti Pirhonen University of Jyva¨skyla¨, Finland Ercan Altinsoy Dresden University of Technology, Germany Wilko Heuten OFFIS, Germany Roope Raisamo University of Tampere, Finland Johan Kildal NOKIA, Finland Federico Fontana University of Udine, Italy Marcelo Wanderley McGill University, Canada Stefania Serafin Aalborg University Copenhagen, Denmark Ravi Kuber UMBC, USA Emanuel Habets University of Erlangen-Nuremberg,Germany Andrew Crossan University of Glasgow, UK Nicolas Castagne ACROE, France Antonio Frisoli Scuola Superiore Sant’Anna, Italy Yon Visell UPMC Universit´e Paris 06, France Table of Contents Haptics and Audio in Navigation Understanding Auditory Navigation to Physical Landmarks ........... 1 David McGookin and Stephen A. Brewster Supporting Sounds: Design and Evaluation of an Audio-Haptic Interface........................................................ 11 Emma Murphy, Camille Moussette, Charles Verron, and Catherine Guastavino A Haptic-Audio Interface for Acquiring Spatial Knowledge about Apartments ............................................... 21 Junlei Yu and Christopher Habel Supporting Experiences and Activities Mobile Haptic Technology Development through Artistic Exploration... 31 David Cuartielles, Andreas Go¨ransson, Tony Olsson, and St˚ale Stenslie Improving Cyclists Training with Tactile Feedback on Feet ............ 41 Dominik Bial, Thorsten Appelmann, Enrico Rukzio, and Albrecht Schmidt HapticPulse – Reveal Your Heart Rate in Physical Activities .......... 51 Janko Timmermann, Benjamin Poppinga, Susanne Boll, and Wilko Heuten Audio-Haptic Simulation of Walking on Virtual Ground Surfaces to Enhance Realism ................................................ 61 Niels C. Nilsson, Rolf Nordahl, Luca Turchet, and Stefania Serafin Object and Interface Interaction InteractingwithDeformableUser Interfaces:EffectofMaterialStiffness and Type of Deformation Gesture.................................. 71 Johan Kildal An Interactive and Multi-sensory Learning Environment for Nano Education....................................................... 81 Karljohan Lundin Palmerius, Gunnar Ho¨st, and Konrad Sch¨onborn VIII Table of Contents Augmenting Media with Thermal Stimulation ....................... 91 Martin Halvey, Michael Henderson, Stephen A. Brewster, Graham Wilson, and Stephen A. Hughes Embodied Interactions with Audio-Tactile Virtual Objects in AHNE ... 101 Koray Tahiro˘glu, Johan Kildal, Teemu Ahmaniemi, Simon Overstall, and Valtteri Wikstro¨m Test and Evaluation Towards an Objective Comparison of Scanning-Based Interaction Techniques...................................................... 111 Benjamin Poppinga, Martin Pielot, Wilko Heuten, and Susanne Boll Knocking Sound as Quality Sign for Household Appliances and the Evaluation of the Audio-Haptic Interaction.......................... 121 M. Ercan Altinsoy Spectral Discrimination Thresholds Comparing Audio and Haptics for Complex Stimuli.............................................. 131 Lorenzo Picinali, Christopher Feakes, Davide A. Mauro, and Brian F.G. Katz How Does Representation Modality Affect User-Experience of Data Artifacts?....................................................... 141 Trevor Hogan and Eva Hornecker Author Index.................................................. 153 Understanding Auditory Navigation to Physical Landmarks David McGookin and Stephen A. Brewster School of Computing Science Universityof Glasgow, Glasgow, G12 8QQ {David.McGookin,Stephen.Brewster}@glasgow.ac.uk http://www.dcs.gla.ac.uk/~mcgookdk Abstract. We present two studies that seek to better understand the role spatialised (3D) audio can play in supporting effective pedestrian navigation. 24 participants attempted to navigate and locate physical landmarks in a local botanical gardens using a gpsTunes [1] based auditorynavigationsystemcoupledwithamap.Participantsweresignif- icantly better at locating prominent than non-prominent physical land- marks. However, nosignificant quantativedifference was found between the use of a map only and map + audio. Qualitative analysis revealed significantissueswhenphysicallandmarksareused,andcommonstrate- gieswhencombiningaudioandmapnavigation.Wehighlighttheimpli- cations of these in relation to existing work, and provide guidelines for future designers to employ. Keywords: 3D Audio, Pedestrian Navigation, Maps, Landmarks. 1 Introduction Thewidescaledeploymentofmodernsmartphoneshasleadtoarenewedinterest inthewayspedestriansnavigatetheirenvironment.Researchhasshownsuchnav- igationtobemuchmorediversethantheturn-by-turnnavigationemployedwhen driving[2].Thishasleadtonewwaystosupportusersnavigationoftheenviron- ment.Whilsthaptic,specificallytactile,feedbackhasbeeninvestigated(e.g.[3]), amuchmoreestablishedtechniqueistoprovidebearinganddistanceinformation toauserviaspatialisedaudiopresentation(3Dsound).Usershear,throughaset ofheadphones,ageo-fixedspatialisedsoundthatemanatesfromalocationinthe environment.Thisauditorycuecanthenbeusedasavirtuallandmarktonavigate towards.Althoughoveradecadeold,withrepeatedstudiesshowingittobeeffec- tive,there arestillmany aspectsofthis technique wedo notunderstand. Inthis paperweoutline someofthese,why they areimportantandpresenttwostudies thatseektobetterunderstandtheirimpactinauditorynavigation. 2 Related Work The use of 3D audio as a means of guiding users in the physical environment has been considered for around a decade. The AudioGPS system [4] aimed to C.Magnusson,D.Szymczak,andS.Brewster(Eds.):HAID2012,LNCS7468,pp.1–10,2012. (cid:2)c Springer-VerlagBerlinHeidelberg2012 2 D. McGookin and S.A. Brewster provide a MAUI (Minimal Attention User Interface), and remove the necessity to look atGPSscreenswhilst navigating.Rather than full 3Daudio via ahead- relatedtransferfunction(HRTF), stereopanningwasusedtoindicatedirection. AGeigercountermetaphor,withcustomdesignedauditorypulsesthatrepeated more rapidly as the user approached a virtual landmark, was used to indicate distance. Holland, Morse and Gedenryd showed, in informal tests, that users could successfully pilot along a route of these virtual auditory landmarks [4]. Landmarks were presented one at a time. When the user came close enough (approximately10m),thelandmarkwasswitchedoffandthenextwaspresented. Otherworkhasextendedthisprinciple,investigatingdifferenttypesofaudioand adding inertial sensors, such as magnetometers (a digital compass), to obtain more accurate heading data. Strachan, Eslambolchilar and Murray-Smith’s [1] gpsTunes used panning of a playing music track, rather than a custom designed auditory cue, to indicate direction. Music volume was increased as the user approached the target. They founduserswereabletosuccessfullypilotalongarouteoffourvirtuallandmarks in an open grassy area. They noted that such navigation might be integrated with a map, and adapted for more exploratory navigation(such as proposed by Allen [2]). However, they carried out no work to investigate this. More recent workhasshownthebasictechniquetoholdwhentheenvironmentis“cluttered” with environmental features users must navigate around [5], rather than being an open grass area, and that it is useful to indicate general areas, such as in navigatingazoo[6].Workhasalsoappliedthetechniquetonavigationforblind and visually impaired pedestrians [7]. However, there are still many gaps in our knowledge. Firstly, the previous workassumesthatusersarenavigatingtowards“virtual”landmarks:thereisno physical manifestation of a landmark or waypoint in the environment. Jones et al. [5] observed that users were often drawn toward physical objects that were close to virtual landmarks, although the objects only happened to be in the environmentandplayedno partintheir study. McGookin, Brewster andPriego [8] and Vazquez-Alvarez, Oakley and Brewster [9] did associate sounds with physical objects, but it was their system, rather than the user, that determined ifthe objectshadbeen reached.This is unlike everydaynavigationwhere auser is trying to uniquely identify a physical landmark [10]. Michon and Denis [11] note the importance of landmarks in both making de- cisions when navigating, and confirming that the correct path is being taken. They note that landmarks that are prominent (stand out from their surround- ings)aremoreusefulthanthosethatblendin.However,thegoalofusersisoften tofindthesenon-prominentlandmarks.Inethnographicstudiesofnavigationby tourists, Brown and Chalmers [10] note that even with a map and guidebook it is often difficult for tourists to find a landmark if it is not prominent (such as a small statue), even when close by. They note the example of a group of tourists tryingtolocateaspecificbuildingalongastreet.Thegrouprequiredaprolonged discussionaroundaguidebookandmap, aswellasinputfromthe investigators, to correctly identify the building. There is no guidance as to the usefulness of