Cloud Computing Technologies Facilitate Earth Research NASA Technology land data, and the Moderate Resolution Imaging In the late 1990s, NASA made all of its satellite data Spectroradiometer (MODIS), launched on Terra in 1999 freely available to the public. Outside researchers could, T hroughout the years, NASA has done more and Aqua in 2002, offers a fresh perspective of Earth’s and still do, make requests through NASA data centers, than any agency to explore what lies beyond our surface every one to two days. which upload requested information via File Transfer world. It has sent rovers to traverse the Martian By using satellite data, NASA scientists are able to Protocol applications. But over the years, as datasets have landscape and launched spacecraft to monitor the sun. investigate an array of environmental issues, such as grown larger in volume and are covering longer periods of Another satellite, Voyager, is even leaving the solar system deforestation, aerosol accumulation, carbon cycles, time, it has become more time-consuming for data to be for the uncharted territory of interstellar space. But and glacier recession, to name just a few. The data shared that way. equally important are its satellites whose lenses are trained are expansive enough for tracking worldwide carbon According to Rama Nemani, senior Earth scientist back on Earth. Since 1972, Landsat satellites operated by emissions, yet refined enough for monitoring vacant at the Advanced Supercomputing Division at Ames NASA have been providing high-resolution topographical Californian farmland. Research Center, “For somebody to do a large-scale NASA's Goddard Earth Observing System Model, known as the GEOS-5, uses satellite data to simulate worldwide weather at resolutions as fine as 3.5 kilometers. Here it displays a snapshot of global winds with a 10-kilometer resolution. Slower surface winds, shown in white, trace features such as storm systems. Meanwhile, faster-moving upper-level winds are colored red. 118 Information Technology Spinoff 2015 continental or global analysis, things are difficult because expected to change in the next 100 years,” notes Nemani. for custom-developed applications, including large-scale you have to download all this data from the various “That’s how fine the scale is.” supercomputing. Among its noted clients are the televi- centers, which takes weeks to months,” he says, “and While NEX has done much to improve access to sion show and movie provider Netflix, software company then you have to write the code on your own to analyze NASA satellite data and supercomputing services and to Adobe Systems, and Thomson Reuters news service. the data instead of reusing the codes that other scientists promote scientific collaboration, after two years, Nemani AWS’s experience hosting data also extended to already wrote.” says, he and his colleagues concluded that more could be government. For example, earlier that year the company To help researchers—specifically those receiving done to simplify access and encourage innovation. A few collaborated with the National Institutes of Health NASA funding—more quickly access and analyze satel- issues were apparent. First, because outside researchers (NIH) to host the world’s largest set of data on human lite data, in 2010 Nemani led the development of NASA were accessing NASA’s computer network, they were genetic variation, which was collected as part of the Earth Exchange, or NEX. Through NEX, researchers not waiting as long as six to eight months to receive security 1000 Genomes Project. Organizations and university only have access to datasets; they are also able to tap into clearance. Another bottleneck was that the large number researchers interested in the data now have easy access; in Ames’ Pleiades supercomputer—one of the world’s most of requests to use Pleiades often resulted in weeks of wait addition, they can pay to utilize Amazon’s EC2 services, powerful. The platform also facilitates collaboration and time. Lastly, only NASA-funded researchers were allowed giving them the necessary supercomputing power to information sharing between participating scientists. “In access. “Given these constraints, we started looking for yet pursue scientific objectives. this way, it’s like Facebook,” Nemani says. “We share other ways of engaging the community,” Nemani says. The successful NIH-AWS collaboration gave NASA results, algorithms and things like that, and in the forums, At about the same time, in 2012, the Obama admin- the impetus to reach a similar agreement with the com- people suggest new ways of doing things.” istration announced its Open Data Executive Order, the pany because it would improve on NEX in all the ways aim being to make government data more accessible to that Nemani and his team had hoped. For one, NASA- the public. One way the White House was pushing to funded researchers would be able to access the datasets make that happen, says Tsengdar Lee, program manager directly, bypassing the time-consuming security clearance In November 2013, NASA and AWS for High-End Computing at NASA headquarters, was by procedures for accessing the agency’s network. Work [Amazon Web Services] announced the encouraging more public-private partnerships. “That’s would also get done faster because there would be less of availability of climate and Earth science how we got into a conversation with Amazon.” a backlog of requests to use Pleiades. Finally, the greater satellite data to researchers and educational scientific community would also now have access to both Technology Transfer NASA datasets and AWS’s supercomputing services. users through the AWS cloud. Amazon.com Inc., headquartered in Seattle, started as With those goals in mind, following the signing of a retail business in the mid-1990s. As the Internet began a NASA Nonreimbursable Space Act Agreement in to play an increasing role in commerce, the company November 2013, NASA and AWS announced an initial In the few years since its founding, the platform has developed the infrastructure to meet demand. Its data- one-year partnership called OpenNEX, whereby select been utilized in several studies and investigations, includ- centers are now located all over the world and have the ing the NASA Earth Exchange Downscaled Climate capacity to simultaneously handle millions of purchases Projections, or NEX-DCP30. Scientists at Ames, the and also stream multimedia to its global consumer base. Climate Analytics Group in nearby Palo Alto, and As it happens, the technology developed for this intri- California State University, Monterey Bay, collaborated cate computer network would also benefit researchers and on the dataset, which provides a view of future conti- companies that need access to both large amounts of data Background image: Produced with nental U.S. temperature and precipitation patterns based storage and supercomputing capabilities. As a result, in vegetation indices gathered from the Moderate Resolution Imaging on four different greenhouse gas emissions scenarios, 2006, the company began offering those options through Spectroradiometer (MODIS) on NASA's spanning the period from 1950 to 2099. What makes Amazon Web Services (AWS), which operates through a Terra satellite, the map displays a multi- the projections even more intriguing is their scale: While worldwide set of datacenters separate from its retail ones. year average of leaf surface area to ground most climate change forecasts cover large geographic Some of AWS’s tools include Amazon S3, or Simple surface area in the Amazon rainforest. The darker the shade of green, the thicker the areas on the order of 100-250 kilometers, the downscaled Storage Services, for keeping digital objects for website tree canopy. Through NASA's partnership dataset can provide half-mile resolution analysis. “Folks hosting, and EC2, or Amazon Elastic Compute Cloud, with Amazon Web Services, researchers can look at this data and see how their neighborhoods are which hosts database servers and server infrastructure will also have access to MODIS data. Spinoff 2015 Information Technology 119 Scientists at Ames Research Center, the Climate Analytics Group in nearby Palo Alto, and California State University, Monterey Bay, collaborated on the Downscaled Climate Projections (NEX-DCP30), which are now freely available through OpenNEX. Taken from NEX- DCP30, the image on the left shows average maximum temperatures across the continental US in July 2000, while the image on the right displays projected average maximum temperatures over the same region in the year 2100. Data for 1950-2000 is based on climate simulations driven with observed data, and data for 2010-2100 is based on the ensemble average of all climate models under the highest greenhouse gas emissions pathway evaluated. At present, the global rate of greenhouse gas emissions is tracking at or above this highest emissions pathway. agency satellite data and climate change datasets would be activities, including terrestrial ecology, land use and land Global Land Survey information from the 1970s freely available through the AWS cloud. Researchers could cover, carbon cycle science, ecological forecasting, biodi- to 2005, MODIS vegetation indices, and the NEX either download the datasets directly to their computers versity, data mining, climate change impacts, and climate Downscaled Climate Projections. After the one-year and run their own analysis, or they could access AWS’s change mitigation strategies. Besides the datasets in NEX, agreement passes, the agency will review the feedback computing as a service (a pay-as-you-go approach), the researchers have access to NASA Earth science models, it has received from the scientific community. If the which comes with free data storage. In addition, such as TOPS (Terrestrial Observation and Prediction reception is positive, NASA plans on extending the through the OpenNEX platform, tens of thousands of System), the Goddard Earth Observing System Model, agreement with AWS and will make available additional researchers—climatologists and geophysicists, among Version 5 (GEOS-5) and the NASA-Unified Weather satellite datasets as well as provide regular updates of many others—from around the world will have a means Research and Forecasting (NU-WRF) model. those that are currently available. to collaborate and share information in order to address In exchange for freely hosting the data, AWS benefits Benefits the world’s most pressing environmental problems. from users having the option of purchasing the compa- Data and models hosted on NEX and OpenNEX will Through the AWS platform, users can currently ny’s computing services. Jamie Kinney, principal solution aid researchers in their investigations for a plethora of work with portions of three sizeable datasets: Landsat architect at AWS, notes that there are two major benefits 120 Information Technology Spinoff 2015 of using AWS for computing: simplicity of use and cost- is excited to help bring about future discoveries. “We make Data Initiatives, which encourage government agencies effectiveness. investments in public datasets and these types of grants to improve their ability to extract knowledge and insight “Now anybody who has access to the Amazon cloud, so that we can really help develop the next generation from their digital data collections, as well as President which is publicly available, can easily create an account of technologies that will be used by all of our customers Obama’s calls for developing tools to fight climate change. and, within a few minutes, provision a cluster,” says down the road, both public sector and commercial,” he “We already have over 400 scientists signed up to be Kinney. “For a few dollars or maybe 10 or 20 dollars an says. “So it’s a very natural partnership for us.” part of the challenges,” Nemani said in July, “so we’re hour, they can provision a very powerful 25–30-teraflop In its short run, Nemani says reception to OpenNEX pretty excited to see what they’ll come up with. But keep cluster on Amazon and quickly analyze and visualize that has been very positive. In June, the agency instituted a in mind that this is just the beginning; in a few years we data using the same exact software that NASA researchers series of contests on OpenNEX that challenges citizen hope to have a large section of the scientific community are using on internal facilities.” scientists to use the available datasets for developing appli- from around the planet using OpenNEX on a daily basis Moving forward, Kinney says that Amazon is looking cations and algorithms that promote climate resilience, for climate change research. That’s our big goal.” v forward to continuing its partnership with NASA, not or our ability to adapt to climate change. The contest only for increasing business but also because the company falls in line with the White House Big Data and Climate Spinoff 2015 Information Technology 121