Christoforos Kachris · Babak Falsafi  Dimitrios Soudris Editors Hardware Accelerators in Data Centers Hardware Accelerators in Data Centers fi Christoforos Kachris Babak Falsa (cid:129) Dimitrios Soudris Editors Hardware Accelerators in Data Centers 123 Editors Christoforos Kachris Dimitrios Soudris Microprocessors andDigital National Technical SystemsLab University of Athens National Technical University Athens, Greece ofAthens Athens, Greece Babak Falsafi ICIINFCOM PARSA ÉcolePolytechnique Fédérale deLausanne Lausanne,Switzerland ISBN978-3-319-92791-6 ISBN978-3-319-92792-3 (eBook) https://doi.org/10.1007/978-3-319-92792-3 LibraryofCongressControlNumber:2018943381 ©SpringerInternationalPublishingAG,partofSpringerNature2019 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar methodologynowknownorhereafterdeveloped. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations. ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland Preface Emergingcloudapplicationslikemachinelearning,artificialintelligence(AI),deep neural networks, and big data analytics have created the need for more powerful data centers that can process huge amounts of data without consuming excessive amounts of power. To face these challenges, data center operators have to adopt novel architectures with specialized high-performance energy-efficient computing systems, such as hardware accelerators, that can process the increasing amount of datainamoreenergy-efficientway.Tothisend,anewerahasemerged;theeraof heterogeneous distributed computing systems that consist of contemporary general-purpose processors (CPUs), general-purpose graphic processing units (GP-GPUs), and field-programmable gate arrays (FPGAs or ACAP Adaptive Compute Acceleration Platform). The utilization of several heterogeneous archi- tectures poses several challenges in the domain of efficient resource utilization, efficient scheduling, and resource management. Inthisbook,wehavecollectedthemostpromisingandthemostrecentresearch activitiesinthisemergingdomainofheterogeneouscomputingthatisbasedonthe efficient utilization of hardware accelerators (i.e., FPGAs). The book contains 13 system-levelarchitecturesthatshowhowtoefficientlyutilizehardwareaccelerators inthedatacenterstofaceemergingcloudapplications.Theproposedarchitectures tackle several challenges such as the energy efficiency of hardware accelerators in data centers, the resource utilization and management, and the programmability of heterogeneousinfrastructurebasedonthehardwareaccelerators.Wehopethisbook servesasareferencebookforanyresearcher,engineer,andacademicworksonthe exciting new area of heterogeneous computing with accelerators. Athens, Greece Christoforos Kachris April 2018 v Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Christoforos Kachris, Babak Falsafi and Dimitrios Soudris 2 Building the Infrastructure for Deploying FPGAs in the Cloud . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Naif Tarafdar, Thomas Lin, Daniel Ly-Ma, Daniel Rozhko, Alberto Leon-Garcia and Paul Chow 3 dReDBox: A Disaggregated Architectural Perspective for Data Centers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Nikolaos Alachiotis, Andreas Andronikakis, Orion Papadakis, Dimitris Theodoropoulos, Dionisios Pnevmatikatos, Dimitris Syrivelis, Andrea Reale, Kostas Katrinis, George Zervas, Vaibhawa Mishra, Hui Yuan, Ilias Syrigos, Ioannis Igoumenos, Thanasis Korakis, Marti Torrents and Ferad Zyulkyarov 4 The Green Computing Continuum: The OPERA Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 A. Scionti, O. Terzo, P. Ruiu, G. Giordanengo, S. Ciccia, G. Urlini, J. Nider, M. Rapoport, C. Petrie, R. Chamberlain, G. Renaud, D. Tsafrir, I. Yaniv and D. Harryvan 5 Energy-Efficient Acceleration of Spark Machine Learning Applications on FPGAs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Christoforos Kachris, Elias Koromilas, Ioannis Stamelos, Georgios Zervakis, Sotirios Xydis and Dimitrios Soudris vii viii Contents 6 M2DC—A Novel Heterogeneous Hyperscale Microserver Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Ariel Oleksiak, Michal Kierzynka, Wojciech Piatek, Micha vor dem Berge, Wolfgang Christmann, Stefan Krupop, Mario Porrmann, Jens Hagemeyer, René Griessl, Meysam Peykanu, Lennart Tigges, Sven Rosinger, Daniel Schlitt, Christian Pieper, Udo Janssen, Holm Rauchfuss, Giovanni Agosta, Alessandro Barenghi, Carlo Brandolese, William Fornaciari, Gerardo Pelosi, Joao Pita Costa, Mariano Cecowski, Robert Plestenjak, Justin Cinkelj, Loïc Cudennec, Thierry Goubier, Jean-Marc Philippe, Chris Adeniyi-Jones, Javier Setoain and Luca Ceva 7 Towards an Energy-Aware Framework for Application Development and Execution in Heterogeneous Parallel Architectures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 Karim Djemame, Richard Kavanagh, Vasilios Kelefouras, Adrià Aguilà, Jorge Ejarque, Rosa M. Badia, David García Pérez, Clara Pezuela, Jean-Christophe Deprez, Lotfi Guedria, Renaud De Landtsheer and Yiannis Georgiou 8 Enabling Virtualized Programmable Logic Resources at the Edge and the Cloud . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 Kimon Karras, Orthodoxos Kipouridis, Nick Zotos, Evangelos Markakis and George Bogdos 9 Energy-Efficient Servers and Cloud . . . . . . . . . . . . . . . . . . . . . . . . 163 Huanhuan Xiong, Christos Filelis-Papadopoulos, Dapeng Dong, Gabriel G. Castañé, Stefan Meyer and John P. Morrison 10 Developing Low-Power Image Processing Applications with the TULIPP Reference Platform Instance. . . . . . . . . . . . . . . . 181 Tobias Kalb, Lester Kalms, Diana Göhringer, Carlota Pons, Ananya Muddukrishna, Magnus Jahre, Boitumelo Ruf, Tobias Schuchert, Igor Tchouchenkov, Carl Ehrenstråhle, Magnus Peterson, Flemming Christensen, Antonio Paolillo, Ben Rodriguez and Philippe Millet 11 Energy-Efficient Heterogeneous Computing at exaSCALE—ECOSCALE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 Konstantinos Georgopoulos, Iakovos Mavroidis, Luciano Lavagno, Ioannis Papaefstathiou and Konstantin Bakanov Contents ix 12 On Optimizing the Energy Consumption of Urban Data Centers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 Artemis C. Voulkidis, Terpsichori Helen Velivassaki and Theodore Zahariadis 13 Improving the Energy Efficiency by Exceeding the Conservative Operating Limits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 Lev Mukhanov, Konstantinos Tovletoglou, Georgios Karakonstantis, George Papadimitriou, Athanasios Chatzidimitriou, Manolis Kaliorakis, Dimitris Gizopoulos and Shidhartha Das Index .... .... .... .... .... ..... .... .... .... .... .... ..... .... 273 Chapter 1 Introduction ChristoforosKachris,BabakFalsafiandDimitriosSoudris 1.1 Introduction Emerging applications like cloud computing, machine learning, AI and big data analytics require powerful systems that can process large amounts of data without consuminghighpower.Furthermore,theseemergingapplicationsrequirefasttime- to-marketandreduceddevelopmenttimes.Toaddressthelargeprocessingrequire- ments of emerging applications, novel architectures are required to be adopted by thedatacentervendorsandthecloudcomputingproviders. RelyingonMoore’slaw,CPUtechnologieshavescaledinrecentyearsthrough packinganincreasingnumberoftransistorsonchip,leadingtohigherperformance. However,on-chipclockfrequencieswereunabletofollowthisupwardtrenddueto strictpower-budgetconstraints.Thus,afewyearsago,aparadigmshifttomulticore processorswasadoptedasanalternativesolutionforovercomingtheproblem.With multicoreprocessors,wecouldincreaseserverperformancewithoutincreasingtheir clockfrequency.Unfortunately,thissolutionwasalsofoundnottoscalewellinthe longer term. The performance gains achieved by adding more cores inside a CPU comeatthecostofvarious,rapidlyscalingcomplexities:inter-corecommunication, memorycoherencyand,mostimportantly,powerconsumption[1]. In the early technology nodes, going from one node to the next allowed for a nearly doubling of the transistor frequency, and, by reducing the voltage, power density remained nearly constant. With the end of Dennard’s scaling, going from B C.Kachris ( ) InstituteofCommunicationandComputerSystems(ICCS/NTUA), Athens,Greece e-mail:[email protected] B.Falsafi ÉcolePolytechniqueFédéraledeLausanne(EPFL),Lausanne,Switzerland D.Soudris DepartmentofElectricalandComputerEngineering, NationalTechnicalUniversityofAthens,Athens,Greece ©SpringerInternationalPublishingAG,partofSpringerNature2019 1 C.Kachrisetal.(eds.),HardwareAcceleratorsinDataCenters, https://doi.org/10.1007/978-3-319-92792-3_1 2 C.Kachrisetal. one node to the next still increases the density of transistors, but their maximum frequency is roughly the same and the voltage does not decrease accordingly. As a result, the power density increases now with every new technology node. The biggestchallengethereforenowconsistsofreducingpowerconsumptionandenergy dissipationpermm2. Therefore,thefailureofDennard’sscaling,towhichtheshifttomulticorechips ispartiallyaresponse,maysoonlimitmulticorescalingjustassingle-corescaling has been curtailed [2]. This issue has been identified in the literature as the dark siliconerainwhichsomeoftheareasinthechiparekeptpowereddowninorderto complywiththermalconstraints[3].Onewaytoaddressthisproblemisthroughthe utilization of hardware accelerators. Hardware accelerators can be used to offload theprocessor,increasethetotalthroughputandreducetheenergyconsumption. 1.2 TheEraofAcceleratorsintheDataCenters As the requirements for processing power of the data centers continue to increase rapidly,higherperformancecomputingsystemsarerequiredtosustaintheincreased communicationbandwidthdemandwithinthedatacenter.Currentserverprocessors cannotaffordablesatisfytherequiredcomputationaldemandofemergingapplication withoutconsumingexcessivepower.Hardwareacceleratorsprovideaviablesolution offeringhighthroughput,reducedlatencyandhigherenergyefficiencycomparedto currentserversbasedoncommodityprocessors. Thisbookpresentsthemostrecentandpromisingsolutionsthathavebeenpre- sentedintheareaofthecomputinghardwareacceleratorsinheterogeneousdatacen- ters.Hardwareacceleratorfordatacentersisaninterdisciplinarytopicforallthecom- munitiesthatareactiveinthedomainofcomputerarchitectures,high-performance computing,datacenterdesignandcloudcomputing. Hardwareacceleratorsaremainlyusedinembeddedsystemstooffloadthepro- cessors for several tasks like compression, encryption, etc. and to provide higher performanceandlowerenergyconsumption.Currentdatacentersneedtoembrace newtechnologiesinordertofacetheincreasednetworktrafficduetoemergingappli- cationslikecloudcomputing.Hardwareacceleratorsbasedonreconfigurablelogic (i.e.FPGAs)canprovidehigherthroughputandbetterenergyefficiency. Hardwareacceleratorindatacentersisanemergingtopicthathasrecentlygained attentionbymajordatacenterandcloudcomputingvendors.Therecentpurchaseof Altera(amajorvendorofFPGA-basedaccelerators)byIntel,andthecollaboration betweenIBMandXilinxshowsthatsoonthehardwareacceleratorswillpenetrate thehyperscaledatacenterinordertoprovidehigherperformanceandhigherenergy efficiency.Thisbookaimstoprovideanoverviewofthearchitectures,programming frameworksandhardwareacceleratorsfortypicalcloudcomputingapplicationsin datacenters. Thefollowingsectiongivesanoverviewofthemainchaptersthatarepresented inthisbook.