Pamela Forner Henning Müller Roberto Paredes Paolo Rosso Benno Stein (Eds.) Information Access Evaluation 8 3 1 8 S Multilinguality, Multimodality, C N and Visualization L 4th International Conference of the CLEF Initiative, CLEF 2013 Valencia, Spain, September 2013, Proceedings 123 Lecture Notes in Computer Science 8138 CommencedPublicationin1973 FoundingandFormerSeriesEditors: GerhardGoos,JurisHartmanis,andJanvanLeeuwen EditorialBoard DavidHutchison LancasterUniversity,UK TakeoKanade CarnegieMellonUniversity,Pittsburgh,PA,USA JosefKittler UniversityofSurrey,Guildford,UK JonM.Kleinberg CornellUniversity,Ithaca,NY,USA AlfredKobsa UniversityofCalifornia,Irvine,CA,USA FriedemannMattern ETHZurich,Switzerland JohnC.Mitchell StanfordUniversity,CA,USA MoniNaor WeizmannInstituteofScience,Rehovot,Israel OscarNierstrasz UniversityofBern,Switzerland C.PanduRangan IndianInstituteofTechnology,Madras,India BernhardSteffen TUDortmundUniversity,Germany MadhuSudan MicrosoftResearch,Cambridge,MA,USA DemetriTerzopoulos UniversityofCalifornia,LosAngeles,CA,USA DougTygar UniversityofCalifornia,Berkeley,CA,USA GerhardWeikum MaxPlanckInstituteforInformatics,Saarbruecken,Germany Pamela Forner Henning Müller Roberto Paredes Paolo Rosso Benno Stein (Eds.) Information Access Evaluation Multilinguality, Multimodality, and Visualization 4th International Conference of the CLEF Initiative, CLEF 2013 Valencia, Spain, September 23-26, 2013 Proceedings 1 3 VolumeEditors PamelaForner CenterfortheEvaluationofLanguageandCommunicationTechnologies(CELCT) Povo,Italy E-mail:[email protected] HenningMüller UniversityofAppliedSciencesWesternSwitzerland,HES-SOValais Sierre,Switzerland E-mail:[email protected] RobertoParedes PaoloRosso UniversitatPolitècnicadeValència,Dept.deSistemasInformáticosyComputación València,Spain E-mail:{rparedes;prosso}@dsic.upv.es BennoStein Bauhaus-UniversitätWeimar,Germany E-mail:[email protected] ISSN0302-9743 e-ISSN1611-3349 ISBN978-3-642-40801-4 e-ISBN978-3-642-40802-1 DOI10.1007/978-3-642-40802-1 SpringerHeidelbergNewYorkDordrechtLondon LibraryofCongressControlNumber:2013946902 CRSubjectClassification(1998):I.7,I.2.7,H.3.1,H.3.3,H.3.7,H.4.1,H.5.3,H.2.8, I.1.3 LNCSSublibrary:SL3–InformationSystemsandApplication, incl.Internet/WebandHCI ©Springer-VerlagBerlinHeidelberg2013 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation, broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection withreviewsorscholarlyanalysisormaterialsuppliedspecificallyforthepurposeofbeingenteredand executedonacomputersystem,forexclusiveusebythepurchaserofthework.Duplicationofthispublication orpartsthereofispermittedonlyundertheprovisionsoftheCopyrightLawofthePublisher’slocation, initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Permissionsforuse maybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violationsareliabletoprosecution undertherespectiveCopyrightLaw. Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant protectivelawsandregulationsandthereforefreeforgeneraluse. Whiletheadviceandinformationinthisbookarebelievedtobetrueandaccurateatthedateofpublication, neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityforanyerrorsor omissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,withrespecttothe materialcontainedherein. Typesetting:Camera-readybyauthor,dataconversionbyScientificPublishingServices,Chennai,India Printedonacid-freepaper SpringerispartofSpringerScience+BusinessMedia(www.springer.com) Preface Since 2000 the Cross-LanguageEvaluation Forum (CLEF) has played a leading role in stimulating research and innovation in a wide range of key areas in the domain of multimodal and multilingual information access. Through the years, CLEFhaspromotedthe study andimplementationofevaluationmethodologies fordiversetasks,resultinginthecreationofabroad,strongandmultidisciplinary researchcommunity. Until 2010, the outcomes of the experiments carried out under the CLEF umbrellawerepresentedanddiscussedatannualworkshopsinconjunctionwith theEuropeanConferenceforDigitalLibraries.CLEF2010representedaradical change from this“classic”CLEF format. While preserving CLEF’s traditional core goals, namely, benchmarking activities carried out in various tracks, we complemented these activities with a peer-reviewed conference component that aimed at advancing research in the evaluation of complex information access systems in different languages and modalities. The theme of the CLEF conference this year is“InformationAccess Evalua- tion Meets Multilinguality, Multimodality, and Visualization”.Thus, the papers accepted for the conference included research on information access that meets these three key aspects: multilinguality, multimodality, and visualization. Two keynote speakers highlight important issues related to our field. Rada Mihal- cea presents the growing need in Internet for effective solutions for multilingual natural language processing. In her talk, Rada explores the hypothesis that a multilingual representation can enrich the feature space for natural language processingtasks,andhenceentailsignificantimprovementsovertraditionalsolu- tionsthat relyexclusivelyona monolingualrepresentation.EvangelosKanoulas presentsthe increasinginterestincreatingtestcollectionsthatbettermodel the variabilityencounteredinreal-lifesearchscenariosinordertoassesstheinforma- tionretrievaleffectiveness.Thisincludesexperimentingoveravarietyofqueries, corpora and even users and their interactions with the search results. This year the overviews of the different evaluation campaigns are included in the publicationinhand, while the experiments carriedoutby systems during the evaluation campaigns are described in a separate publication, namely, the Working Notes. The success of CLEF 2013 would not have been possible without the in- valuable contributions of the members of the Program Committee, Organizing Committee,students,andvolunteerswhosupportedtheconferenceinitsvarious VI Preface stages. We would also like to express our gratitude to the sponsoring organiza- tions for their significant and timely support. These proceedings were prepared with the assistance of the Center for the Evaluation of Language and Commu- nication Technologies (CELCT), Trento, Italy. July 2013 Pamela Forner Henning Mu¨ller Roberto Paredes Paolo Rosso Benno Stein Organization CLEF 2013 was organized by the Universitat Polit`ecnica de Val`encia, Spain. General Chairs Paolo Rosso Universitat Polit`ecnica de Val`encia, Spain Benno Stein Bauhaus-Universit¨atWeimar, Germany Program Chairs Henning Mu¨ller University of Applied Sciences Western Switzerland, Switzerland Roberto Paredes Universitat Polit`ecnica de Val`encia, Spain Evaluation Labs Chairs Roberto Navigli Sapienza University of Rome, Italy Dan Tufis RACAI, Romania Resource Chair Khalid Choukri Evaluations and Language resources Distribution Agency (ELDA), France Organization Chair Pamela Forner CELCT, Italy Organizing Committee Centro de Formacio´n Permanente, Universitat Polit`ecnica de Val`encia, Spain: Ma Francisca Collado L´opez Ester Srougi Ramon Universitat Polit`ecnica de Val`encia, Spain Enrique Flores Parth Gupta Mauricio Villegas VIII Organization Center for the Evaluation of Language and Communication Technologies (CELCT), Italy: Pamela Forner Giovanni Moretti Program Committee Shlomo Argamon Illinois Institute of Technology, USA Alexandra Balahur Joint Research Centre - JRC - European Commission, Italy Barbara Caputo IDIAP ResearchInstitute, Switzerland Khalid Choukri ELDA, France Walter Daelemans University of Antwerp, Belgium Nicola Ferro University of Padua, Italy Norbert Fuhr University of Duisburg-Essen, Germany Allan Hanbury Vienna University of Technology, Austria Donna Harman NationalInstitute ofStandardandTechnology, USA Eduard Hovy Carnegie Mellon University, USA Antoine Isaac Europeana & VU University Amsterdam, The Netherlands Alexis Joly Inria, France Evangelos Kanoulas Google, Switzerland Birger Larsen Royal School of Library and Information Science, Denmark Mihai Lupu Vienna University of Technology, Austria Walid Magdy Qatar Computing Research Institute, Qatar Thomas Mandl University of Hildesheim, Germany Paul Mcnamee Johns Hopkins University, USA Manuel Montes-Y-G´omez INAOE, Mexico Jian-Yun Nie University of Montreal, Canada Anselmo Pen˜as National Distance Learning University, Spain Carol Peters ISTI-CNR Pisa, Italy Vivien Petras Humboldt University, Germany Florina Piroi Vienna University of Technology, Austria Martin Potthast Bauhaus-Universit¨atWeimar, Germany Alvaro Rodrigo National Distance Learning University, Spain Paolo Rosso Universitat Polit`ecnica de Val`encia, Spain Giuseppe Santucci Sapienza University of Rome, Italy Tobias Schreck University of Konstanz, Germany Efstathios Stamatatos University of the Aegean, Greece Organization IX Benno Stein Bauhaus-Universit¨atWeimar, Germany Bart Thomee Yahoo! Research, Spain Elaine Toms University of Sheffield, UK Theodora Tsikrika University of Applied Sciences Western Switzerland, Switzerland Jose-Luis Vicedo University of Alicante, Spain Christa Womser-Hacker University of Hildesheim, Germany David Zellhoefer Brandenburg Technical University Cottbus, Germany Sponsoring Institutions CLEF 2013 benefited from the support of the following organizations: PROMISE Network of Excellence European Science Foundation (ELIAS project) Universitat Polit`ecnica de Val`encia Escola T`ecnica Superior d’EnginyeriaInform`atica Departamento de Sistemas Inform´aticos y Computacio´n WIQ-EI - Web Information Quality Evaluation Initiative Corex, Building Knowledge Solutions ELIAS Keynotes