FINANCIAL ASSISTANCE FUNDING OPPORTUNITY ANNOUNCEMENT ADVANCED RESEARCH PROJECTS AGENCY – ENERGY (ARPA-E) U.S. DEPARTMENT OF ENERGY ENERGY-EFFICIENT LIGHT-WAVE INTEGRATED TECHNOLOGY ENABLING NETWORKS THAT ENHANCE DATACENTERS (ENLITENED) Announcement Type: Initial Announcement Funding Opportunity No. DE-FOA-0001566 CFDA Number 81.135 Funding Opportunity Announcement (FOA) Issue Date: Friday, June 10, 2016 First Deadline for Questions to [email protected]: 5 PM ET, Friday July 15, 2016 Submission Deadline for Concept Papers: 5 PM ET, Monday July 25, 2016 Second Deadline for Questions to [email protected]: 5 PM ET,TBD Submission Deadline for Full Applications: 5 PM ET, TBD Submission Deadline for Replies to Reviewer Comments: 5 PM ET, TBD Expected Date for Selection Notifications: TBD Total Amount to Be Awarded Approximately $25 million, subject to the availability of appropriated funds. Anticipated Awards ARPA-E may issue one, multiple, or no awards under this FOA. Awards may vary between $250,000 and $10 million. • For eligibility criteria, see Section III.A of the FOA. • For cost share requirements under this FOA, see Section III.B of the FOA. • To apply to this FOA, Applicants must register with and submit application materials through ARPA-E eXCHANGE (https://arpa-e-foa.energy.gov/Registration.aspx). For detailed guidance on using ARPA-E eXCHANGE, see Section IV.H.1 of the FOA. • Applicants are responsible for meeting each submission deadline. Applicants are strongly encouraged to submit their applications at least 48 hours in advance of the submission deadline. Questions about this FOA? Check the Frequently Asked Questions available at http://arpa-e.energy.gov/faq. For questions that have not already been answer ed, e mail [email protected] (with FOA name and number in s ubject line); see FOA S ec. VII.A. Problems with ARPA-E eXCHANGE? Email [email protected] (with FOA name and number in subject line). AR-311-02.16 • For detailed guidance on compliance and responsiveness criteria, see Sections III.C.1 through III.C.4 of the FOA. Questions about this FOA? Check the Frequently Asked Questions available at http://arpa-e.energy.gov/faq. For questions that have not already been answer ed, e mail [email protected] (with FOA name and number in s ubject line); see FOA S ec. VII.A. Problems with ARPA-E eXCHANGE? Email [email protected] (with FOA name and number in subject line). AR-311-02.16 TABLE OF CONTENTS REQUIRED DOCUMENTS CHECKLIST ................................................................................................................... - 1 - I. FUNDING OPPORTUNITY DESCRIPTION ..................................................................................................... - 2 - A. AGENCY OVERVIEW ........................................................................................................................................... - 2 - B. PROGRAM OVERVIEW........................................................................................................................................ - 3 - 1. SUMMARY ................................................................................................................................................. - 3 - 2. BACKGROUND AND MOTIVATION ................................................................................................................... - 4 - C. PROGRAM OBJECTIVES .................................................................................................................................... - 11 - D. PROGRAM STRUCTURE AND TECHNICAL CATEGORIES OF INTEREST ........................................................................... - 12 - E. TECHNICAL PERFORMANCE TARGETS AND DELIVERABLES ....................................................................................... - 13 - II. AWARD INFORMATION ........................................................................................................................... - 17 - A. AWARD OVERVIEW ......................................................................................................................................... - 17 - B. ARPA-E FUNDING AGREEMENTS ...................................................................................................................... - 18 - 1. COOPERATIVE AGREEMENTS........................................................................................................................ - 18 - 2. FUNDING AGREEMENTS WITH FFRDCS/DOE LABS, GOGOS, AND FEDERAL INSTRUMENTALITIES .......................... - 18 - 3. TECHNOLOGY INVESTMENT AGREEMENTS ...................................................................................................... - 19 - C. STATEMENT OF SUBSTANTIAL INVOLVEMENT ....................................................................................................... - 20 - III. ELIGIBILITY INFORMATION ...................................................................................................................... - 21 - A. ELIGIBLE APPLICANTS ....................................................................................................................................... - 21 - 1. INDIVIDUALS ............................................................................................................................................. - 21 - 2. DOMESTIC ENTITIES ................................................................................................................................... - 21 - 3. FOREIGN ENTITIES ..................................................................................................................................... - 21 - 4. CONSORTIUM ENTITIES ............................................................................................................................... - 22 - B. COST SHARING ............................................................................................................................................... - 22 - 1. BASE COST SHARE REQUIREMENT ................................................................................................................ - 23 - 2. INCREASED COST SHARE REQUIREMENT ........................................................................................................ - 23 - 3. REDUCED COST SHARE REQUIREMENT .......................................................................................................... - 23 - 4. LEGAL RESPONSIBILITY ............................................................................................................................... - 24 - 5. COST SHARE ALLOCATION ........................................................................................................................... - 24 - 6. COST SHARE TYPES AND ALLOWABILITY ........................................................................................................ - 25 - 7. COST SHARE CONTRIBUTIONS BY FFRDCS AND GOGOS .................................................................................. - 26 - 8. COST SHARE VERIFICATION ......................................................................................................................... - 26 - C. OTHER .......................................................................................................................................................... - 26 - 1. COMPLIANT CRITERIA ................................................................................................................................. - 26 - 2. RESPONSIVENESS CRITERIA.......................................................................................................................... - 27 - 3. SUBMISSIONS SPECIFICALLY NOT OF INTEREST ................................................................................................. - 28 - 4. LIMITATION ON NUMBER OF APPLICATIONS ................................................................................................... - 29 - IV. APPLICATION AND SUBMISSION INFORMATION ................................................................................. - 30 - A. APPLICATION PROCESS OVERVIEW ..................................................................................................................... - 30 - 1. REGISTRATION IN ARPA-E eXCHANGE....................................................................................................... - 30 - 2. CONCEPT PAPERS ...................................................................................................................................... - 30 - Questions about this FOA? C hec k th e Frequently Asked Questions available at http://arpa-e.energy.gov/faq. For questions that have not already been answered, email [email protected] (with FOA name and number in subject line); see FOA Sec. VII.A. Problems with ARPA-E eXCHANGE? Email [email protected] (with FOA name and number in subject line). AR-311-02.16 3. FULL APPLICATIONS ................................................................................................................................... - 30 - 4. REPLY TO REVIEWER COMMENTS ................................................................................................................. - 31 - 5. PRE-SELECTION CLARIFICATIONS AND “DOWN-SELECT” PROCESS ...................................................................... - 31 - 6. SELECTION FOR AWARD NEGOTIATIONS ........................................................................................................ - 31 - 7. MANDATORY WEBINAR ............................................................................................................................. - 32 - B. APPLICATION FORMS ....................................................................................................................................... - 32 - C. CONTENT AND FORM OF CONCEPT PAPERS .......................................................................................................... - 32 - 1. CONCEPT PAPER ........................................................................................................................................ - 33 - A. CONCEPT SUMMARY .................................................................................................................................. - 33 - B. INNOVATION AND IMPACT .......................................................................................................................... - 33 - C. PROPOSED WORK ..................................................................................................................................... - 34 - D. TEAM ORGANIZATION AND CAPABILITIES ...................................................................................................... - 34 - D. CONTENT AND FORM OF FULL APPLICATIONS ....................................................................................................... - 35 - E. CONTENT AND FORM OF REPLIES TO REVIEWER COMMENTS .................................................................................. - 35 - F. INTERGOVERNMENTAL REVIEW ......................................................................................................................... - 35 - G. FUNDING RESTRICTIONS ................................................................................................................................... - 35 - H. OTHER SUBMISSION REQUIREMENTS .................................................................................................................. - 35 - 1. USE OF ARPA-E eXCHANGE .................................................................................................................... - 35 - V. APPLICATION REVIEW INFORMATION ..................................................................................................... - 36 - A. CRITERIA ....................................................................................................................................................... - 36 - 1. CRITERIA FOR CONCEPT PAPERS ................................................................................................................... - 37 - 2. CRITERIA FOR FULL APPLICATIONS ................................................................................................................ - 38 - 3. CRITERIA FOR REPLIES TO REVIEWER COMMENTS ............................................................................................ - 38 - B. REVIEW AND SELECTION PROCESS ...................................................................................................................... - 38 - 1. PROGRAM POLICY FACTORS ........................................................................................................................ - 38 - 2. ARPA-E REVIEWERS ................................................................................................................................. - 39 - 3. ARPA-E SUPPORT CONTRACTOR ................................................................................................................. - 39 - C. ANTICIPATED ANNOUNCEMENT AND AWARD DATES ............................................................................................. - 40 - VI. AWARD ADMINISTRATION INFORMATION ......................................................................................... - 40 - A. AWARD NOTICES ............................................................................................................................................ - 40 - 1. REJECTED SUBMISSIONS ............................................................................................................................. - 40 - 2. CONCEPT PAPER NOTIFICATIONS .................................................................................................................. - 40 - 3. FULL APPLICATION NOTIFICATIONS ............................................................................................................... - 40 - B. ADMINISTRATIVE AND NATIONAL POLICY REQUIREMENTS ..................................................................................... - 41 - C. REPORTING .................................................................................................................................................... - 41 - VII. AGENCY CONTACTS ............................................................................................................................. - 41 - A. COMMUNICATIONS WITH ARPA-E .................................................................................................................... - 41 - B. DEBRIEFINGS .................................................................................................................................................. - 42 - VIII. OTHER INFORMATION ......................................................................................................................... - 43 - A. FOAS AND FOA MODIFICATIONS ...................................................................................................................... - 43 - B. OBLIGATION OF PUBLIC FUNDS.......................................................................................................................... - 43 - C. REQUIREMENT FOR FULL AND COMPLETE DISCLOSURE .......................................................................................... - 43 - Questions about this FOA? C hec k th e Frequently Asked Questions available at http://arpa-e.energy.gov/faq. For questions that have not already been answered, email [email protected] (with FOA name and number in subject line); see FOA Sec. VII.A. Problems with ARPA-E eXCHANGE? Email [email protected] (with FOA name and number in subject line). AR-311-02.16 D. RETENTION OF SUBMISSIONS ............................................................................................................................ - 44 - E. MARKING OF CONFIDENTIAL INFORMATION ........................................................................................................ - 44 - F. TITLE TO SUBJECT INVENTIONS .......................................................................................................................... - 44 - G. GOVERNMENT RIGHTS IN SUBJECT INVENTIONS.................................................................................................... - 45 - 1. GOVERNMENT USE LICENSE ........................................................................................................................ - 45 - 2. MARCH-IN RIGHTS .................................................................................................................................... - 45 - 3. U.S. MANUFACTURING REQUIREMENT ......................................................................................................... - 46 - H. RIGHTS IN TECHNICAL DATA .............................................................................................................................. - 46 - I. PROTECTED PERSONALLY IDENTIFIABLE INFORMATION .......................................................................................... - 47 - J. COMPLIANCE AUDIT REQUIREMENT ................................................................................................................... - 47 - IX. REFERENCES ............................................................................................................................................ - 48 - X. GLOSSARY ............................................................................................................................................... - 49 - Questions about this FOA? C hec k th e Frequently Asked Questions available at http://arpa-e.energy.gov/faq. For questions that have not already been answered, email [email protected] (with FOA name and number in subject line); see FOA Sec. VII.A. Problems with ARPA-E eXCHANGE? Email [email protected] (with FOA name and number in subject line). AR-311-02.16 - 1 - REQUIRED DOCUMENTS CHECKLIST For an overview of the application process, see Section IV.A of the FOA. For guidance regarding requisite application forms, see Section IV.B of the FOA. For guidance regarding the content and form of Concept Papers, Full Applications, and Replies to Reviewer Comments, see Sections IV.C, IV.D, and IV.E of the FOA. OPTIONAL/ FOA SUBMISSION COMPONENTS DEADLINE MANDATORY SECTION • Each Applicant must submit a Concept Paper in Adobe PDF format by the stated deadline. The Concept Paper must not exceed 4 pages in length and must include the following: 5 PM ET, Concept Paper Concept Summary Mandatory IV.C Monday July o Innovation and Impact 25, 2016 o Proposed Work o Team Organization and Capabilities o [TO BE INSERTED BY FOA MODIFICATION IN SEPTEMBER Full Application 2016] Mandatory IV.D 5 PM ET, TBD Reply to [TO BE INSERTED BY FOA MODIFICATION IN SEPTEMBER Reviewer 2016] Optional IV.E 5 PM ET, TBD Comments Questions about this FOA? Check the Frequently Asked Questions available at http://arpa-e.energy.gov/faq. For questions that have not already been answered, email [email protected] (with FOA name and number in subject line); see FOA Sec. VII.A. Problems with ARPA-E eXCHANGE? Email [email protected] (with FOA name and number in subject line). AR-311-02.16 - 2 - I. FUNDING OPPORTUNITY DESCRIPTION A. AGENCY OVERVIEW The Advanced Research Projects Agency – Energy (ARPA-E), an organization within the Department of Energy (DOE), is chartered by Congress in the America COMPETES Act of 2007 (P.L. 110-69), as amended by the America COMPETES Reauthorization Act of 2010 (P.L. 111- 358) to: “(A) to enhance the economic and energy security of the United States through the development of energy technologies that result in— (i) reductions of imports of energy from foreign sources; (ii) reductions of energy-related emissions, including greenhouse gases; and (iii) improvement in the energy efficiency of all economic sectors; and (B) to ensure that the United States maintains a technological lead in developing and deploying advanced energy technologies.” ARPA-E issues this Funding Opportunity Announcement (FOA) under the programmatic authorizing statute codified at 42 U.S.C. § 16538. The FOA and any awards made under this FOA are subject to 2 C.F.R. Part 200 as amended by 2 C.F.R. Part 910. ARPA-E funds research on and the development of high-potential, high-impact energy technologies that are too early for private-sector investment. The agency focuses on technologies that can be meaningfully advanced with a modest investment over a defined period of time in order to catalyze the translation from scientific discovery to early-stage technology. For the latest news and information about ARPA-E, its programs and the research projects currently supported, see: http://arpa-e.energy.gov/. ARPA-E funds transformational research. Existing energy technologies generally progress on established “learning curves” where refinements to a technology and the economies of scale that accrue as manufacturing and distribution develop drive down the cost/performance metric in a gradual fashion. This continual improvement of a technology is important to its increased commercial deployment and is appropriately the focus of the private sector or the applied technology offices within DOE. By contrast, ARPA-E supports transformative research that has the potential to create fundamentally new learning curves. ARPA-E technology projects typically start with cost/performance estimates well above the level of an incumbent technology. Given the high risk inherent in these projects, many will fail to progress, but some may succeed in generating a new learning curve with a projected cost/performance metric that is significantly lower than that of the incumbent technology. ARPA-E funds technology with the potential to be disruptive in the marketplace. The mere creation of a new learning curve does not ensure market penetration. Rather, the ultimate value of a technology is determined by the marketplace, and impactful technologies ultimately Questions about this FOA? Check the Frequently Asked Questions available at http://arpa-e.energy.gov/faq. For questions that have not already been answered, email [email protected] (with FOA name and number in subject line); see FOA Sec. VII.A. Problems with ARPA-E eXCHANGE? Email [email protected] (with FOA name and number in subject line). AR-311-02.16 - 3 - become disruptive – that is, they are widely adopted and displace existing technologies from the marketplace or create entirely new markets. ARPA-E understands that definitive proof of market disruption takes time, particularly for energy technologies. Therefore, ARPA-E funds the development of technologies that, if technically successful, have the clear disruptive potential, e.g., by demonstrating capability for manufacturing at competitive cost and deployment at scale. ARPA-E funds applied research and development. The Office of Management and Budget defines “applied research” as “systematic study to gain knowledge or understanding necessary to determine the means by which a recognized and specific need may be met” and defines “development” as the “systematic application of knowledge or understanding, directed toward the production of useful materials, devices, and systems or methods, including design, development, and improvement of prototypes and new processes to meet specific requirements.”1 Applicants interested in receiving financial assistance for basic research should contact the DOE’s Office of Science (http://science.energy.gov/). Office of Science national scientific user facilities (http://science.energy.gov/user-facilities/) are open to all researchers, including ARPA-E applicants and awardees. These facilities provide advanced tools of modern science including accelerators, colliders, supercomputers, light sources and neutron sources, as well as facilities for studying the nanoworld, the environment, and the atmosphere. Projects focused on the improvement of existing technology platforms along defined roadmaps may be appropriate for support through the DOE offices such as: the Office of Energy Efficiency and Renewable Energy (http://www.eere.energy.gov/), the Office of Fossil Energy (http://fossil.energy.gov/), the Office of Nuclear Energy (http://www.energy.gov/ne/office- nuclear-energy), and the Office of Electricity Delivery and Energy Reliability (http://energy.gov/oe/office-electricity-delivery-and-energy-reliability). B. PROGRAM OVERVIEW 1. SUMMARY The growing demand for datacenter services across a range of applications has resulted in significant and sustained growth in electrical energy consumption in the Information Communications Technology (ICT) sector. Currently, datacenters consume more than 2.5 % of US electricity and this percentage is projected to double in about 8 years [1-4, 6, 7, 9]. Efficiency improvements due to more efficient cooling, power delivery, and electronic processor chips via Moore’s law improve overall efficiency, but do not significantly slow the current growth trend; to do so requires a transformative improvement. The overall objective of the ENLITENED (ENergy-efficient Light-wave Integrated Technology Enabling Networks that Enhance Datacenters) program, therefore, is to provide a transformative change - to achieve an overall doubling in datacenter energy efficiency in 10 years through deployment of novel 1 OMB Circular A-11 (http://www.whitehouse.gov/sites/default/files/omb/assets/a11_current_year/a11_2014.pdf), Section 84, p. 8. Questions about this FOA? Check the Frequently Asked Questions available at http://arpa-e.energy.gov/faq. For questions that have not already been answered, email [email protected] (with FOA name and number in subject line); see FOA Sec. VII.A. Problems with ARPA-E eXCHANGE? Email [email protected] (with FOA name and number in subject line). AR-311-02.16 - 4 - network topologies enabled by integrated photonics technologies. ARPA-E estimates that if the technical challenges posed by ENLITENED can be overcome, these alone would reduce projected US energy use by about 1% after 10 years and realize at least twice the number of datacenter transactions with the same amount of energy. Industry projections show that in order to achieve future datacenter performance requirements, metal interconnects must be increasingly replaced by photonic technologies, yet costs for deployment are often prohibitive and energy efficiency is not necessarily the highest priority in the commercial sector [11, 15, 16]. Though a broad industrial consensus pushes toward further photonic integration in switches and other datacenter components [11,16,18], in some cases, large companies will build another 80 MW datacenter to meet demand, rather than increase efficiency with photonics, due to a combination of reliability risks, cost and limited component supply. To overcome metal interconnect limitations on future datacenter energy-efficiency performance, ENLITENED will target the critical packaging and integration challenges needed to exploit the inherent performance advantages of dense photonic interconnects and switching technology at the chip-scale within datacenters. Specifically, ENLITENED will target packaging and integration of novel and efficient photonics-enabled hardware systems that can demonstrate at least a 2-fold increase in energy efficiency at the datacenter level. To validate hardware solutions, ENLITENED will also entail modeling and simulation of the new datacenter architectures and data traffic protocols under realistic workloads, to provide quantifiable measures to validate transformative design strategies for future datacenters and retrofits. 2. BACKGROUND AND MOTIVATION Impact on Datacenters: Energy Consumption and Industrial Trends The energy consumed by datacenters has been the subject of much study [1-5], reaching 91 TWh in 2013, and increasing at a rate that doubles about every 8 years [1]. Currently, datacenters account for about 2.5% of electricity consumption in the US – while the total annual electricity consumption in the US has remained approximately constant at about 3800 TWh [6] from 2005 to 20122. Datacenters in the US can be divided according to their electricity consumption by market segment as: Small and Medium-Sized (49%), Enterprise/Corporate (27%), Multi-Tenant (19%), Hyper-Scale Cloud Computing (4%), and High Performance Computing (1%) [4] – which shows not only a wide diversity in scale and function, but also highlights the importance of finding energy efficiency solutions that address all market segments. Trends indicate that Multi-Tenant and Cloud services are growing, which may improve efficiencies due to the possibility of 2 US electrical power generation is projected (EIA reference case) to be 3.9 trillion kWh in 2016, 4.2 in 2026 and 4.7 in 2040. Annual Energy Outlook 2016, early release: http://www.eia.gov/forecasts/aeo/er/index.cfm Questions about this FOA? Check the Frequently Asked Questions available at http://arpa-e.energy.gov/faq. For questions that have not already been answered, email [email protected] (with FOA name and number in subject line); see FOA Sec. VII.A. Problems with ARPA-E eXCHANGE? Email [email protected] (with FOA name and number in subject line). AR-311-02.16 - 5 - virtualizing smaller datacenter functions within larger server farms that allocate resources more efficiently. Nevertheless, the fundamental technologies used, and solutions envisioned are relevant to all datacenters. With growth in all sectors, the importance for reducing energy consumption across the datacenter application domain continues to grow. Figure 1 depicts the current and projected trends in aggregate power consumption for datacenters in the US (red line). The demands for datacenter services have resulted in an exponential growth rate in the total power requirement. The exponential growth in service demand is actually outpacing the exponential growth in ICT equipment computing efficiency (due in part to Moore’s Law scaling of processor performance). Even more concerning is the trend towards an increasing fraction of datacenter power consumed by the internal datacenter network [17]. This pending upswing in the projected power consumption is attributed to anticipated growth in interconnect energy demands with increasing ICT performance, but with existing network technologies it appears unlikely that network energy efficiency will increase proportionally to server performance. Ultimately, the electrical interconnects to the server chips themselves will not be able to keep up with the increasing computing power of those computing elements [14] – which could potentially add to the energy-efficiency burden of datacenters as depicted in Figure 1 for the years beyond approximately 2022. As depicted with the dashed green line in Figure 1, the ENLITENED Program seeks to suppress this increasing power consumption trend by enabling energy efficient datacenters with packaged integrated photonic interconnects and switches and associated novel networking architectures. Specifically, ARPA-E sees an opportunity to exploit high-density integrated photonic chip-scale I/O at <1 pJ/bit and lower latency network fabrics enabled by integrated photonic links and switching technology. This combination will make possible significant increases in overall system efficiencies and thereby halt the increasing energy demand, and perhaps even lower the aggregate datacenter energy consumption to manageable levels as depicted in Figure 1. Questions about this FOA? Check the Frequently Asked Questions available at http://arpa-e.energy.gov/faq. For questions that have not already been answered, email [email protected] (with FOA name and number in subject line); see FOA Sec. VII.A. Problems with ARPA-E eXCHANGE? Email [email protected] (with FOA name and number in subject line). AR-311-02.16