Springer Tracts in Advanced Robotics 148 Luigi Villani Ciro Natale Michael Beetz Bruno Siciliano   Editors Robotics for Intralogistics in Supermarkets and Retail Stores Springer Tracts in Advanced Robotics Volume 148 SeriesEditors BrunoSiciliano,DipartimentodiIngegneriaElettricaeTecnologie dell’Informazione,UniversitàdegliStudidiNapoliFedericoII,Napoli,Italy OussamaKhatib,ArtificialIntelligenceLaboratory,DepartmentofComputer Science,StanfordUniversity,Stanford,CA,USA AdvisoryEditors NancyAmato,ComputerScience&Engineering,TexasA&MUniversity,College Station,TX,USA OliverBrock,FakultätIV,TUBerlin,Berlin,Germany HermanBruyninckx,KULeuven,Heverlee,Belgium WolframBurgard,InstituteofComputerScience,UniversityofFreiburg,Freiburg, Baden-Württemberg,Germany RajaChatila,ISIR,Pariscedex05,France FrancoisChaumette,IRISA/INRIA,Rennes,Ardennes,France WanKyunChung,RoboticsLaboratory,MechanicalEngineering,POSTECH, Pohang,Korea(Republicof) PeterCorke,QueenslandUniversityofTechnology,Brisbane,QLD,Australia PaoloDario,LEM,ScuolaSuperioreSant’Anna,Pisa,Italy AlessandroDeLuca,DIAGAR,SapienzaUniversitàdiRoma,Roma,Italy RüdigerDillmann,HumanoidsandIntelligenceSystemsLab,KIT-Karlsruher InstitutfürTechnologie,Karlsruhe,Germany KenGoldberg,UniversityofCalifornia,Berkeley,CA,USA JohnHollerbach,SchoolofComputing,UniversityofUtah,SaltLake,UT,USA LydiaE.Kavraki,DepartmentofComputerScience,RiceUniversity,Houston,TX, USA VijayKumar,SchoolofEngineeringandAppliedMechanics,Universityof Pennsylvania,Philadelphia,PA,USA BradleyJ.Nelson,InstituteofRoboticsandIntelligentSystems,ETHZurich, Zürich,Switzerland FrankChongwooPark,MechanicalEngineeringDepartment,SeoulNational University,Seoul,Korea(Republicof) S.E.Salcudean,TheUniversityofBritishColumbia,Vancouver,BC,Canada RolandSiegwart,LEEJ205,ETHZürich,InstituteofRobotics&Autonomous SystemsLab,Zürich,Switzerland GauravS.Sukhatme,DepartmentofComputerScience,UniversityofSouthern California,LosAngeles,CA,USA TheSpringerTractsinAdvancedRobotics(STAR)publishnewdevelopmentsand advances in the fields of robotics research, rapidly and informally but with a high quality. The intent is to cover all the technical contents, applications, and multi- disciplinaryaspectsofrobotics,embeddedinthefieldsofMechanicalEngineering, Computer Science, Electrical Engineering, Mechatronics, Control, and Life Sciences,aswellasthemethodologiesbehindthem.Withinthescopeoftheseries aremonographs,lecturenotes,selectedcontributionsfromspecializedconferences andworkshops,aswellasselectedPhDtheses. Specialoffer:Forallclientswithaprintstandingorderweofferfreeaccesstothe electronicvolumesoftheSeriespublishedinthecurrentyear. IndexedbySCOPUS,DBLP,EICompendex,zbMATH,SCImago. AllbookspublishedintheseriesaresubmittedforconsiderationinWebofScience. · · · Luigi Villani Ciro Natale Michael Beetz Bruno Siciliano Editors Robotics for Intralogistics in Supermarkets and Retail Stores Editors LuigiVillani CiroNatale DepartmentofElectricalEngineering UniversityofCampania andInformationTechnology Aversa,Italy UniversityofNaplesFedericoII Naples,Italy BrunoSiciliano UniversityofNaples MichaelBeetz Naples,Italy UniversityofBremen Bremen,Germany ISSN 1610-7438 ISSN 1610-742X (electronic) SpringerTractsinAdvancedRobotics ISBN 978-3-031-06077-9 ISBN 978-3-031-06078-6 (eBook) https://doi.org/10.1007/978-3-031-06078-6 ©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNature SwitzerlandAG2022 Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuse ofillustrations,recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,and transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar ordissimilarmethodologynowknownorhereafterdeveloped. Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant protectivelawsandregulationsandthereforefreeforgeneraluse. Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional claimsinpublishedmapsandinstitutionalaffiliations. ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland Toourfamilies Series Editor’s Foreword Atthedawnofthecentury’sthirddecade,roboticsisreachinganelevatedlevelof maturityandcontinuestobenefitfromtheadvancesandinnovationsinitsenabling technologies.Theseallarecontributingtoanunprecedentedefforttobringingrobots to human environment in hospitals and homes, factories, and schools; in the field for robots fighting fires, making goods and products, picking fruits, and watering the farmland, saving time and lives. Robots today hold the promise for making a considerable impact in a wide range of real-world applications from industrial manufacturingtohealthcare,transportation,andexplorationofthedeepspaceand sea.Tomorrow,robotswillbecomepervasiveandtouchuponmanyaspectsofmodern life. TheSpringerTractsinAdvancedRobotics(STAR)isdevotedtobringingtothe research community the latest advances in the robotics field on the basis of their significanceandquality.Throughawideandtimelydisseminationofcriticalresearch developmentsinrobotics,ourobjectivewiththisseriesistopromotemoreexchanges andcollaborationsamongtheresearchersinthecommunityandcontributetofurther advancementsinthisrapidlygrowingfield. Luigi Villani, Ciro Natale, Michael Beetz, and Bruno Siciliano present in this book the results of REFILLS (Robotics Enabling Fully Integrated Logistics Lines forSupermarkets),aEuropeanResearchproject.Thisprojectbroughtastrongteam ofroboticsresearcherswithawiderangeofcomplementaryskillsandcompetencies to work together on the development of robotic systems for the logistics of retail storesandsupermarketsthatarecapabletooperateincollaborationwithhumans. The volume offers an extensive view on REFILLS development of innovative robotictechnologiesforin-storelogisticsincludingmonitoring,transportation,and shelfrefilling,inbothcollaborativeandautonomousmodalities.Theresultsareorga- nizedinsixchapterscoveringprincipalissuesrangingfromtheautomaticacquisition of stocks, pre-sorting, and sequencing of goods to collaborative ergonomic shelf- filling and full autonomous refilling of products on shelves. These areas were all experimentallyvalidatedwithsometestedinstores—aremarkableaccomplishment oftheREFILLSteam. vii viii SeriesEditor’sForeword TheoutcomeisabookthatisconfirmedtobeshininginourSTARseries! Stanford,USA OussamaKhatib April2022 STAREditor Preface REFILLS(RoboticsEnablingFully-IntegratedLogisticsLinesforSupermarkets)is aEuropeanresearchproject,whichstartedinJanuary2017andendedinDecember 2020.Theambitionoftheprojectwasthatofimprovinglogisticsinasupermarket thanks to mobile robotic systems in close and smart collaboration with humans, addressingthemainin-storelogisticsprocessesforretailshops.Thegoalofthisedited volumeistopresenttheprincipalresultsoftheprojecttothescientificcommunity but also to operators of the logistic and retail world, sharing knowledge about the technologicalopportunitiesandthemainchallengesregardingroboticsforlogistics insupermarketandretailstores. Althoughthemarketforonlineorderingisgrowing,expertsagreethattherewill alwaysbedemandforphysicalsupermarketsastheycombinetherelaxingsensory experience of shopping with the extra service of human advice. Retail and logis- ticscompaniesarecommittedtowardsashoppingexperiencemorecomfortableand exciting while, at the same time, using technology to reduce costs and increase efficiency. In particular, there is still a lack of automation in the logistics manage- ment of retail stores. Most of the logistics costs arise from items handling, items transportation,shelvesreplenishment,andbackroommanagement.Thesetasksare time-consuming, repetitive, inefficient, monotonous, and wearing for supermarket clerks but are rather complex to automate. A mobile robotic system in close and smartcollaborationwithhumanscanideallyperformalltheseoperationsprovided somechallengesaresolved. ThesolutionsproposedwithinREFILLScoveranumberofintermediatesteps— withthepotentialforearlyexploitationandstoreandcustomerbenefits—uptothe desiredlevelofautomationtofulfillthelogisticsneedsoftheretailmarketdomain, as illustrated in Chap. 1. In particular, three scenarios are explored and discussed, withdifferentrobotsappliedanddifferentprocessessupported,i.e.: ix x Preface 1. Storemonitoring 2. Collaborativeshelfrefilling 3. Autonomousshelfrefilling. Thefirstscenariodealswiththeacquisitionoffullinformationonlocation,stock and availability of goods within the store, by using a mobile robotic system that autonomouslyacquiresasemanticdigitaltwinmodelofthestore,whichispresented in Chap. 2. This operation is necessary to keep supermarket shelves stocked and minimizeemptyordisarrangedshelves.Inparticular,theautomaticverificationthat the real shelves match the ideal layout is a very challenging task and requires the developmentofrobustandscalabletechniquesfortheidentificationofthousandsof differentproducts,asdiscussedinChap.3. The digital twin data generated by the scanning robot of the first scenario is the foundation for the second scenario, where effective product pre-sorting and sequencing are achieved with robots. The idea is that of developing an automatic depalletizingcellinthebackroom,describedinChap.4,inchargeofputtingthecases withtheproductsontrolleysinasuitableorder.Then,afleetofstore-readyautomated guided vehicle is in charge of transporting the trolleys in proximity of the shelf to berefilled.Atthispoint,theshelf-fillingprocessstarts.Thisoperationisperformed bythestoreemployeessupportedbysuitableroboticunits.Namely,apointingunit sheddinglightonthespotoftheshelfwhereagivenproductshouldbeplaced,anda handlingunitthatservesthecasepackstotheclerkswiththeaimofreducingtherisk ofmusculoskeletaldisordersrelatedtotherefillingoperation.Chapter5presentsthe ergonomicanalysisthatallowsselectingthebestrefillingprocessmodality,forboth humanandrobot,tailoredtotheanthropometriccharacteristicsoftheclerk. Finally,thethirdscenarioisfocusedonthetime-consumingprocessofstacking individualproductsontheshelves,whichisperformedautonomouslybyarobotic arm.Thisisaverychallengingtask,requiringhighlyreliablemanipulationcapability innarrowspacesofproductswithalargevarietyofsize,shape,weight,andfragility, asshowninChap.6. Allthepresentedtechnologieshavebeenvalidatedinrealisticenvironmentsand someofthemhavebeentestedalsoinrealsupermarkets.Furtherinformationabout the project’s achievement can be found at http://www.refills-project.eu, including videos illustrating experiments. They are the result of the coordinated effort of threeacademicpartners(CREATE-UniversityofNaplesFedericoII,Universityof Campania Luigi Vanvitelli, University of Bremen), one robotic company (KUKA RoboterGMBH),onehightechnologycompany(INTELIrelandLTD),onelogistic company (SWISSLOG), with the fundamental contribution of a retail stores chain (dm-drogeriemarkt)asenduser. Wehopereaderswillfindthematerialcontainedinthisvolumeusefultodeepen theknowledgeonthemainlogisticprocessinretailstoresandonthepossiblerobo- tized solutions, and we would like to take this opportunity to express our sincere