EnvironmentalModelling&Software39(2013)103e115 ContentslistsavailableatSciVerseScienceDirect Environmental Modelling & Software journal homepage: www.elsevier.com/locate/envsoft The NASA-Goddard Multi-scale Modeling FrameworkeLand Information System: q Global land/atmosphere interaction with resolved convection Karen I. Mohra,*, Wei-Kuo Taoa, Jiun-Dar Cherna,b, Sujay V. Kumara,c, Christa D. Peters-Lidarda aNASA-GoddardSpaceFlightCenter,Greenbelt,MD20771,USA bGoddardEarthSciencesTechnologyandResearch,MorganStateUniversity,Baltimore,MD21251,USA cScienceApplicationsInternationalCorp.,McLean,VA22102,USA a r t i c l e i n f o a b s t r a c t Articlehistory: Thepresentgenerationofgeneralcirculationmodels(GCM)useparameterizedcumulusschemesand Received5May2011 runathydrostatic gridresolutions. Toimprovethe representationofcloud-scalemoistprocessesand Receivedinrevisedform landeatmosphereinteractions,aglobal,Multi-scaleModelingFramework(MMF)coupledtotheLand 8February2012 InformationSystem(LIS)hasbeendevelopedatNASA-GoddardSpaceFlightCenter.TheMMFeLIShas Accepted28February2012 three components, a finite-volume (fv) GCM (Goddard Earth Observing System Ver. 4, GEOS-4),a 2D Availableonline10May2012 cloud-resolving model (Goddard Cumulus Ensemble, GCE), and the LIS, representing the large-scale atmosphericcirculation,cloudprocesses,andlandsurfaceprocesses,respectively.Thenon-hydrostatic Keywords: Landeatmosphereinteraction GCE model replaces the single-column cumulus parameterization of fvGCM. The model grid is Earthsystemmodeling composed of an array of fvGCM gridcells each with a series of embedded GCE models. A horizontal Globalmodeling couplingstrategy,GCE4fvGCM4Coupler4LIS,offeredsignificantcomputationalefficiency,withthe Atmosphericprediction scalability and I/O capabilities of LIS permitting landeatmosphere interactions at cloud-scale. Global Hydrologicprediction simulationsof2007e2008andcomparisonstoobservationsandreanalysisproductswereconducted. Usingtwodifferentversionsofthesamelandsurfacemodelbutthesameinitialconditions,divergence inregional,synoptic-scalesurfacepressurepatternsemergedwithintwoweeks.Thesensitivityoflarge- scalecirculationstolandsurfacemodelphysicsrevealedsignificantfunctionalvaluetousingascalable, multi-modellandsurfacemodelingsysteminglobalweatherandclimateprediction. PublishedbyElsevierLtd. 1. Introduction andprecipitationmodulatesthelarge-scaleatmosphericdynamics of thelowand mid-latitudes, affectingthe distribution,intensity, Thelandandatmosphereformahighlycoupledsystem.Surface and longevity of waves, jets, and fronts, and thus to future heatandmomentumfluxesarelinkedtothesurfacenetradiation precipitationpatterns.Couplingageneralcirculationmodel(GCM) flux,thevegetationstate,andtheprofilesoftemperatureandwater to a land surface model (LSM) allows for two-way interaction of from below the surface up through the atmospheric boundary atmospheric moist processes with the land surface. By coupling layer.Thefluxesofheat,momentum,andmoistureacrosstheland/ aGCMtoamulti-modelLandInformationSystem(LIS)ratherthan atmosphere interface are influenced by the heterogeneous char- to a single LSM, significant additional physical and functional acterof the land surface layer and vary on spatial scales ranging flexibility is achieved (Kumar et al., 2006; Peters-Lidard et al., from meters to thousands of kilometers. Linking the water and 2007). This paper describes the NASA-Goddard finite-volume energy cycles is precipitation. Feedbacks between the heteroge- Multi-scale Modeling FrameworkeLand Information System neouslandsurfaceandtheboundarylayeraffectthedevelopment (MMFeLIS), a global model framework capable of explicitly of clouds and precipitation (review in Pielke, 2001). The vertical resolving cumulus convection and simulating cloud-scale land/ distributionoflatentheatreleasedthroughtheformationofclouds atmosphereinteractions.TheMMFeLISintegratesanatmospheric GCMwitha2Dcloud-resolvingmodel(CRM)forexplicitsimulation ofcumuluscloudsandcouplestheLIStotheGCM.Wedescribethe q ThematicIssueontheFutureofIntegratedModelingScienceandTechnology. development and operation of the current Goddard MMFeLIS, * Correspondingauthor.MesoscaleAtmosphericProcessesLaboratory,Code612, focusingonthemodelcouplinganditsinitialtesting,particularly NASA-GoddardSpaceFlightCenter,Greenbelt,MD20771,USA.Tel.:þ1301614 6360;fax:þ13016145492. with respect to surface variables. This paper can be viewed as E-mailaddress:[email protected](K.I.Mohr). a third companion to two previous papers on LIS, the first 1364-8152/$eseefrontmatterPublishedbyElsevierLtd. doi:10.1016/j.envsoft.2012.02.023 104 K.I.Mohretal./EnvironmentalModelling&Software39(2013)103e115 description of LIS by Kumar et al. (2006) and the role of LIS with SP-CAM in reproducing the diurnal cycle of convection in coupled mesoscale modeling by Kumar et al. (2008). The (DeMott et al., 2007), orogenic propagating cloud systems MMFeLISenhancesourabilitytoinvestigatetheintegratedimpact (Pritchardetal.,2011),subtropicallowcloudfields(Blosseyetal., of small-scale cloud microphysics and soil and vegetation states 2009), and precipitation anomalies associated with the on regional to global-scale circulations, cloud patterns, and MaddeneJulian oscillation (Benedict and Randall, 2009) and El precipitation. NiñoeSouthernOscillation(ENSO,Khairoutdinovetal.,2008).Tao et al. (2009) compare the SP-CAM and an earlier version of the 2. Background:globalmulti-scalemodeling Goddard MMF. Both MMFs resulted in better representation of globalenergyandwatercycles compared to GCMswith cumulus The current generation of GCMs used in operational global parameterizations but had theirown setof biases fromusing 2D weatherandshort-termclimateforecastingbytheNationalCenters CRMs and prescribed sea surface temperatures. Researchers at for Environmental Prediction (NCEP), the European Centre for PacificNorthwestNationalLaboratory(PNNL)addedtoSP-CAMan Medium-RangeWeatherForecasts(ECMWF),andtheNASAGlobal explicit-cloudparameterized-pollutantapproachthatlinksaerosol Modeling and Assimilation Office have fully interactive land/ and chemical processes on the large-scale grid with statistics of atmosphere coupling using single LSMs, respectively, Noah (Ek cloudpropertiesandprocessesresolvedbytheCRM(Wangetal., et al., 2003), Hydrology-Tiled ECMWF Scheme for Surface 2011a, 2011b). The PNNL MMF can be used to study aerosol ExchangesoverLand(H-TESSEL,Balsamoetal.,2009),andCatch- effectsoncloudmicrophysics(indirecteffect)globally,astudytopic ment (Koster et al., 2000). Although the NCEP Global Forecast typicallyconfinedtoaCRM-sizeddomain. System (GFS) uses the NASA LIS for land data assimilation, only InTaoetal.(2009),thedifferencesbetweentheCSUandGod- Noah is fullyand directlycoupled to the GFS atmospheric model dard MMFs were smaller than their differences with standard component(Sahaetal.,2010).TheseoperationalLSMsusetilesor GCMs.Thesedifferencesmaybecomelargerlessfromthediverging catchmentsub-divisionstoimprovetherepresentationoftheland evolutionoftheirparentGCMsthanfromtheadditionofadditional surfaceheterogeneitywithinGCMgridcells.However,thesurface modelcomponents.TheemphasishereatGoddardonland/atmo- fluxes generated are spatially averaged so that the atmospheric sphere interactions and hydrologic model development has componentcanuseaparameterizedcumulusschemetodetermine producedanMMFinwhich“multi-scale”includesthelandsurface gridcellcloudsandprecipitation.Modelcomparisonprojectsinthe andasignificantrangeofoptionsareavailabletotheuserthrough GlobalEnergyandWaterCycleExperiment(GEWEX)haveshown LIS. Here, we describe the development and operation of the thatsimulationsofvarioustypesofcloudsandcloudsystemsfrom currentGoddardMMFeLIS,focusingonthemodelcouplingandits different geographic locations by cloud-resolving models (CRM) initialtesting,particularlywithrespecttosurfacevariables. agreewithobservationsbetterthanthosefromcumulusparame- terizationsusedbythecurrentgenerationofGCMs(Moncrieffetal., 3. ComponentsofMMFeLIS 1997; Randall et al., 2003b). The lumping of land/atmospheric interactions and the useof cumulus parameterizations forcloud- Fig.1depictstheintegrationandcouplingofthecomponentsof scale moist processes are sources of significant uncertainty in MMFeLIS.Thethreeprincipalcomponentsareafinite-volume(fv) predictionsatlargerscales(Zhangetal.,2005;PauluisandGarner, GCM(GoddardEarthObservingSystemVer.4,GEOS-4),a2DCRM 2006;ShuttsandPalmer,2007). (GoddardCumulusEnsemble,GCE),andtheLIS,representingthe ThegridsizeofGCMsismovingtowardgridssufficientlyfineto large-scale atmospheric circulation, cloud processes, and land explicitlyresolvemanycloudsystems,butthecomputationalcost surfaceprocesses,respectively.Allnumericalanalysisiswrittenin is enormous and, because of the importance of unresolved FORTRAN90. The C language is used to expand object-oriented processesatstillfinerscales,convergenceisbynomeansassured.A features already in FORTRAN90, providing a virtual object- CRMcansimulatecloudsatmeter-tokilometer-scalegridresolu- oriented programming environment managing operations within tions.Computationalinfrastructuretypicallylimitsthesimulation and between components. The MMFeLIS components represent ofcloudsandcloudsystemsbyCRMstoarelativelysmalldomain theworkofseveraldifferentteamsofscientistsandengineersat ((cid:2)103-km (cid:3) 103-km) and short time periods (<1 month). Goddard. The fvGCM was developed in the former NASA Data Grabowski (2001) and Khairoutdinov and Randall (2001) first AssimilationOffice.ThesuccessortotheDataAssimilationOffice, proposed the use of 2D cloud-resolving models as a “super- theGlobalModelingandAssimilationOffice(GMAO),adoptedthe parameterization” to simulate cloud processes within GCM grid- fvGCMastheirfirstoperationalmodel.TheCRMwasdevelopedin cells, replacing cumulus parameterizations. Arakawa (2004) theMesoscaleAtmosphericProcessesLaboratory,andtheLISwas describes this configuration as a multi-scale modeling framework developedbytheHydrologicalSciencesLaboratory.Assistedbythe (MMF).IntheMMF,anon-hydrostatic2DCRMtakestheplaceof Hydrological Sciences Laboratory, the Mesoscale Atmospheric thesingle-columncumulusparameterizationusedinconventional ProcessesLaboratoryperformedtheintegrationandcouplingofthe GCMs(Randalletal.,2003a;Arakawa,2004;Taoetal.,2009). threeMMFeLIScomponents. TherearetwoteamsdevelopingMMFs,anewereffortbyGod- dardandalongerrunningeffortbyColoradoStateUniversity(CSU). 3.1. TheGoddardEarthObservingSystemVer.4(GEOS-4) The CSU MMF combines the Community Atmosphere Model 3.0 (CAM,Collinsetal.,2006),theSystemforAtmosphericModeling The fvGCM of MMFeLIS, the GEOS-4, was constructed by (SAM,KhairoutdinovandRandall,2003),andtheCommunityLand combiningthefinite-volumedynamicalcoredevelopedatGoddard Model(CLM,Daietal.,2003)toformthesuper-parameterizedCAM (Lin, 2004) with the physics package of the NCAR CCM3, which (SP-CAM). The GCMs at the core of the Goddard and CSU MMFs representsawell-balancedsetofprocesseswithalonghistoryof share a common ancestor, the National Center for Atmospheric development and documentation (Kiehl et al.,1998). The unique Research (NCAR) Community Climate Model Ver. 3 (CCM3), but features of the finite-volume dynamical core include an accurate underwentseparateadditionaldevelopmentbyGoddardandNCAR conservativeflux-form semi-Lagrangian transport algorithmwith researchers. amonotonicityconstraintonsub-griddistributionsthatisfreeof Takingonphenomenathathavebeenidentifiedasdifficultfor Gibbs oscillation (Lin and Rood, 1996, 1997), a terrain-following GCMstoreproducewell,CSUresearchershaveshownbetterresults Lagrangian control-volume vertical coordinate (s-coordinate), K.I.Mohretal./EnvironmentalModelling&Software39(2013)103e115 105 microphysicsscheme.TheGCEmodel’sdefaultbulkmicrophysical scheme has been modified to improve graupel concentrations in the stratiform region and cloud ice concentrations in the upper troposphere(Langetal.,2007,2011).Thesechangesbetteraddress saturation issues in these regions and result in more realistic columnicewatercontentsforlonger-termsimulations(Zengetal., 2008,2009). 3.3. TheLandInformationSystem(LIS) TheLISisascalablelanddataassimilationsystemthatintegrates asuiteofadvancedLSMs,highresolutionsatelliteandobservational data,dataassimilationandparameteroptimizationtechniques,and high-performance computing tools. The initial development and principal features of LIS are described in Kumar et al. (2006) and Peters-Lidard et al. (2007). The LIS infrastructure has three layers, 1)acorethatcontrolstheprogramexecutionandI/Oandmanages the user-defined components; 2) a middle abstractions layer con- sisting of generic representations of LSMs, domains, land surface parameters,andrunningmode;3)anextensionoftheabstractions layerfortheuser-selectedLSM,domain,parameterset,andrunning mode.ExecutingLISgeneratesspatiallyandtemporallydistributed estimatesoflandsurfaceprocessesusingeitherobservedormodel- derived meteorology to constrain and force the user-specified LSMs.ThesemodelsincludetheoperationalLSMs:CLM,Noah,and Catchment.InadditiontonewLSMs,multipleversionsofthesame LSM may be added and used within LIS. These features facilitate ensemble modeling studies and the benchmarking and sensitivity testing of new parameterizations and coupled modeling system Fig.1. IntegrationandcouplingofMMFeLIS.Blackarrowsdepicttheflowofforcing configurations. The data assimilation functions of LIS are not inputfromthefvGCMtoGCEandLIS,andgreenarrowsdepicttheflowofoutputfrom currentlyusedinMMFeLIS.Plannedfutureupgradeswilladddata GCEandLIStothefvGCM.Table1liststhespecificinputforcingvariablestoGCEand assimilation to the fvGCM, allowing future users to utilize the LIS.Inthetopandbottomleftexamplemaps,globaldailypressurefields(hPa,black assimilationfunctionsalreadyinLIS. contours)anddailysurfaceprecipitation(mm,purpleshading)aredepicted.Global landcoverisdepictedinthebottomrightmap. ThethreeavailablerunningmodesinLIS,analysis,forecast,and coupled, are described in Kumar et al. (2006, 2008). Initially, the coupled running mode was designed for local- to regional-scale a physically consistent integration of the pressure gradient force modeling. Input land cover datasets masked out areas containing foraterrain-followingcoordinate(LinandRood,1997;Lin,1998), inlandwaterbodies,wetlands,andglaciers,andparameterizations and a mass-, momentum-, and total-energy-conserving vertical for evaporation from these surfaces were by-passed. During remappingalgorithm. MMFeLISdevelopment,theseparameterizationswerere-integrated, Dependingupontheapplication,thenumberoflevelsusedin andaglobalinputdatasetwiththeselandcoverswascreated. GEOS-4 varies between 32 and 64, while the horizontal grid spacing can vary between 2.5(cid:4) and 0.125(cid:4). The GEOS-4 and its 4. CouplingandexecutionofMMFeLIS successor, the GEOS-5, have been applied in climate simulation, dataassimilation,andweatherpredictionmodes.Atlasetal.(2005, 4.1. Theatmosphere,fvGCMandGCE 2007) and Shen et al. (2006a,b, 2010) tested their capability to simulate Atlantic hurricanes, adequately resolving problems like InFig.2,theatmosphereconsistsofanarrayofGCMgridcells erratictrack,abruptre-curvature,intenseextratropicaltransition, each with a series of embedded 2D GCE models, representing multiple landfall and re-intensification, and interaction among aseriesofx-zslicesoftheatmosphere.ThesizeoftheGCMgrid- vortices. cells, the number of embedded GCE models per gridcell, and the numberof GCEinternalgridcellsare user-specified.Initialization, 3.2. TheGoddardCumulusEnsemble(GCE) execution,andfinalizationofGCEoperationsarecontrolledbythe fvGCM.Aninternalstatevariable(ISV)datastructureisusedtopass The GCE model has the longest history of the MMFeLIS forcingandoutputvariablessuchastemperatureandhumidityto components. It has been developed and refined at Goddard over and from the MMFeLIS components. The ISV has 5 degrees of twoandahalfdecadesforsimulatingconvectivecloudsandcloud freedom:2absolutespatialcoordinates,latitudeandlongitude,and systems. The initial development and core features of the GCE threerelativespatialcoordinates.ForGCE,thelatitudeandlongi- model are inTao and Simpson (1993), with reviewsof the appli- tudearethecentralgeographicallocationofeachfvGCMgridcell, cationoftheGCEmodeltounderstandingprecipitationprocesses andthreeCartesiancoordinates(x,y,z)designatetherelativeposi- inTao(2003)andTaoetal.(2003).TheGCEisanon-hydrostatic, tionofGCEinternalgridcells.TheISVmakesitpossibletopreserve anelasticCRMcomposedofprognosticequationsformomentum, bothgridgeographicalandprocessorlayoutseamlessly. potential temperature, and water vapor mixing ratio. It includes The passing of forcing ISVs from fvGCM to GCE initiates GCE solar and infrared radiative transfer processes, a Kessler-type execution. The atmospheric forcing variables passed to GCE such two-category (cloud drops and rain) liquid water scheme, and as temperature, humidity, and ozone are fvGCM-gridcell layer a three-category (cloud ice, snow, and graupel/hail) bulk ice means,e.g.,Tðlon;lat;0;0;sÞ.Acoordinatetransformationfromthe 106 K.I.Mohretal./EnvironmentalModelling&Software39(2013)103e115 andGCElowerboundaryconditions.TheTimeManageraccounts forthetimeelapsedduringLISoperations.BecausetheMMFeLIS coupling is based on the template developed for coupling LIS to themesoscaleWeatherResearchandForecasting(WRF)model,the sequence of interactions between fvGCM, the Coupler, and LIS is equivalent to the sequence for LISeWRF diagrammed and describedinKumaretal.(2008). Thefirstattempttoaddtwo-waylandsurfaceprocessestoMMF involvedcouplingasingleLSM,CLM2.1,tofvGCM,inahorizontal structure,GCE4fvGCM4CLM.ThiswastheMMFconfiguration used in Tao et al. (2009). Although CLM permits up to 7 surface types to be specified within its gridcells to approximate the underlying land surface heterogeneity, its limited I/O capabilities requiredspatialaveragingsuchthatbothinputforcingandoutput Fig.2. TheblacksquaresareanarrayofGCMgridcellsandtheredlinesinsidethem areembedded2Dtime-variantCRMs.Forcomparison,theblackdotsrepresent1D boundaryconditionspasstoandfromfvGCMatfvGCMresolution, time-invariantcumulusparameterizationswithinatraditionalGCM.Figureadapted e.g., Hðlon;lat;0;0;0Þ. The horizontal structure of the older MMF fromTaoetal.(2009). wasadaptedforMMFeLIS,swappingLISandtheESMFCouplerfor thesingleLSM,creatingGCE4fvGCM4Coupler4LIS. Asecondstrategywasconsidered,averticalstructure,fvGCM4 terrain-following fvGCM s-coordinate system to the fixed GCE GCE4Coupler4LIS,inwhichtherewouldbemultipleGCE-LIS Cartesian coordinate system completes the transfer from the couples embedded withineachfvGCMgridcell. The vertical struc- fvGCMtoGCE.Forlandsurfacefluxes,therearetwooptions,1)an ture would avoid the communication overhead of passing ISVs fvGCM-gridell mean that is then randomized across the GCE through fvGCM. However, it would complicate processor distribu- internal grid to approximate surface turbulence, 2) the perturba- tioninparallelcomputingoperationsandincursignificantcompu- tionoccurringateachGCEinternalgridcell,e.g.,H0ðlon;lat;x;0;0Þ. tationalcosttostartandstopmanyinstancesofLISandorganizethe DuringGCEexecution,GCEmeanlayertemperature,humidity, ISVsforthesurfaceboundaryconditions.Incomparison,thehori- andcloudpropertiessuchascloudicemixingratioarecalculated zontalstrategyinvolvesasingleinstanceofLISandamainLISgridat foreachmemberoftheGCEarrayfromitsinternalperturbations. the same resolution as fvGCM. For simulating higher resolution AfterexecutionoftheGCEarray,temporalaveragingtakesplaceto interactionbetweentheatmosphereandthelandsurface,multiple accountforthefvGCMclocktimeelapsedduringexecutionofthe surfacetypes,“tiles”,at1e101-km, canbespecifiedwithintheLIS GCEarray.ThetemperatureandhumiditytendenciesforthefvGCM gridcellsandtheI/OpassedtoandfromtheCouplerattileresolu- gridcell {e.g., ½vTðlon;lat;0;0;sÞ=vt(cid:5) } are calculated from the tion.Thecomputationalcostisseveraltimesgreatertocalculateand moist array of GCE layer means, a Cartesian to s-coordinate trans- pass surface fluxes at cloud-scale (<10-km) resolutions. Because formation, and the elapsed clock time. These results are then most land surface processes are parameterized as 1D phenomena passedbacktothefvGCM.Therearetwooptionsforsubsequently and LIS time steps are on the order of several minutes, the total forcingLIS,1)thefvGCM-gridellmeansatthelowestfvGCMmodel computationalcostofrunningoneinstanceofLIS,evenwithcloud- layer, 2) temporally-averaged GCE internal perturbations at the scaletiles,isnominalcomparedtoexecutingtheGCEarrays. lowestGCEmodellayer. 5. Firstresults:acomparisonofdifferentlandsurfacemodel 4.2. ThelandeatmospherecouplingwithLIS physics ThecouplingofthefvGCMtoLISisenabledbytheNASAEarth 5.1. Physicsdifferences,CLMversion2.0vs.2.1 SystemModelingFramework(ESMF,Hilletal.,2004).TheESMFis open-source software to assist the development of high- AsignificantfunctionaladvantageofcouplingtoLISoverasingle performance,multi-componentnumericalearthsystemmodels.It LSMisthesuiteofabstractionsavailableinLISthatexpandstheuser’s offers a variety of data structures for transferring data between components, tools and utilities to ensure component interopera- bility with consistent component behavior, and libraries for re- gridding, time advancement, and other common modeling func- Table1 tions.The MMFeLIS coupling uses the ESMF Coupler Component ListofforcinginputstoGCE(leftcolumn)andLIS(rightcolumn).TheLISforcing heightisfixedatthelowestfvGCMmodellayer.TheozoneandSSTstatevariables (ESMF_CplComp), import and export state objects (ESMF_State), arederivedfromNOAAweeklyReynoldsOptimumInterpolationSSTAnalysisVer.2 andtheTimeManagerutility(c.f.,Collinsetal.,2005;Balajietal., (Reynoldsetal.,2002)andanozoneproductmergingtheNASAUpperAtmosphere 2011).TheCouplerisused onlyfor transferringdatabetweenthe Research Satellite (UARS) ozone measurements (Ziemke et al., 1998) and the fvGCMandLIS,asESMFcomponentsandutilitiesaredesignedfor AtmosphericModelIntercomparisonProject2(AMIP2)ozonedataset(Kanamitsu etal.,2002),respectively. datatransferandre-gridding,notphysical/dynamicalcomputations. Because the original design of LIS wrapped the ESMF Coupler GCEforcing LISforcing superstructurearoundLIS(Kumaretal.,2006),theCouplerdidnot Localsolartime,t(lat,lon) Localsolartime,t(lat,lon) havetobeadded,onlymodifiedtorecognizetheISVsusedinthe O3(lat,lon,z),imposed Winds,u,v(lat,lon) atmosphericcomponentsasforcing/outputdatacontainervariables SST(lat,lon),imposed Rainandsnowrate,r(lat,lon) thatcouldbepointedtobytheimport/exportESMFstateobjects. THeummpideritayt,uqre(l,aTt,(llaotn,,lozn),z) ASpirecteifimcpheuramtuidriet,yT,aqir((llaatt,,lloonn)) ToinitiateLISexecution,theCouplertakesintheimportstate Winds,u(lat,lon,z),v(lat,lon,z) Sealevelpressure,P(lat,lon) objectpointingtotheISVsofforcingdataforLIS(Table1)andmaps Advection(lat,lon)ofTandq, Shortwaveradiation,SWin(lat,lon) the forcing data tothe LIS grid. After LIS execution, any required adv(T),adv(q) temporal and spatial averaging takes place within LIS. An export Sensibleandlatentheatflux, Longwaveradiation,LWin(lat,lon) H(lat,lon),LE(lat,lon) stateobjectthenpointstotheoutputthatwillbemappedasfvGCM K.I.Mohretal./EnvironmentalModelling&Software39(2013)103e115 107 capabilitytotestcodesandcomparesimulationsusinganensemble MMFeLISagainsttheMMFconfigurationofTaoetal.(2009).We modeling approach with different LSMs and/or versions of those ranandcomparedtwosimulationsofMMFeLIS,oneusingCLM2.0 models.ThetestingofnewmodelsornewmodelphysicsinLISis andtheotherusingCLM2.1,toverifythatMMFeLIScanproduce accomplishedbytheadditionorremovalofcomponentsinthethird physicallyrealisticresultsandassesshowchangestothephysicsof layer(theextensionofabstractionslayer)wherecodesforspecific land surface processes would integrate over space and time to LSMs reside. If new LSMsare added,communication betweenthe affectregionaltoglobalatmosphericcirculations. newmodelandlowerlayersisestablishedusingthetemplatesforI/O containedinthesecondlayer.TheLSMsaddedtoLISarenolonger 5.2. Surfacediagnostics limitedbytheirnativeI/Oandtilingcapabilityasthesefunctionsare determinedandcontrolledbythetop-levelcorelayer. ThecomparisonofthedifferentversionsofCLM,original(2.0) TheoriginalversionofCLMinLISwasversion2.0.AddedtoLIS versus modified (2.1), involved running MMFeLIS for two years, wasthemodifiedversionofCLM,version2.1,thesameversionof 2007e2008, using each version at horizontal resolutions of CLM directly coupled to MMF in the configuration of Tao et al. 2(cid:4) (cid:3)2.5(cid:4) forfvGCMand4-kmforGCE,withtimestepsof30-min (2009). In CLM 2.1, there are several important changes to the (fvGCM),10-s(GCE),and3-min(LIS).Verticalresolutionswere30 model physics involving the computation of atmospheric forcing layers(terrain-following)forfvGCMand32layers(fixed)forGCE. height, vegetation temperature, canopy interception of precipita- WespecifiedthesamesimplesurfaceheterogeneityastheMMFof tion, and the drag coefficient between the underlying soil Taoetal.(2009),7tilesperLISgridcellandoption1forlandsurface (or canopy surface) and the canopy air. All four of these state fluxes(fvGCMgridcellmeans).Becauseof theadditionalphysical variablesareusedindeterminingthesurfaceheatfluxessupplied and computational complexity of cloud-scale heterogeneity, to the atmospheric components. The underlying assumptions, development of the MMFeLIS required that we first benchmark theformulationoftheequationsforeachvariable,andthesolution MMFeLIS against MMF using the same simple heterogeneity, methods vary between versions. For example, the canopy inter- leaving the testing of more complex representations of surface ceptioninCLM2.0iscalculatedassumingauniformprecipitation heterogeneitytofuturework. rateoverthegridcellcomparedtoanassumptionofalogarithmic Fig.3summarizestheannualdailymean2-mairtemperatures probabilitydistributionfunctionforprecipitationratesinCLM2.1. for CLM 2.0 (original, top panels), CLM 2.1 (modified, middle BecauseLIShadneverbeforebeenrungloballyincoupledmode panels), and theirdifference (originalemodified, bottom panels) and new land covers and model physics were introduced, it was for2007(leftpanels)and2008(rightpanels).In2007,mostofthe critical in the model development process to benchmark the differencesbetweentheoriginalandmodifiedversionsofCLMare Fig.3. Mapsoftheannualdailymeananddifference2-mairtemperaturesinKelvinsforCLM2.0andCLM2.1for2007(aec)and2008(def)attheresolutionofthefvGCM (2(cid:4)(cid:3)2.5(cid:4)).Thetoppanels(a,d)aretheoriginal,CLM2.0,results.Themiddlepanels(b,e)arethemodified,CLM2.1,results.Thebottompanels(c,f)aredifferencemapsofCLM 2.0eCLM2.1.Thescalesontherightcolumnapplytothemapsontheleftcolumnaswell. 108 K.I.Mohretal./EnvironmentalModelling&Software39(2013)103e115 positive, implying CLM 2.0 tended to produce warmer air During the daytime over vegetated surfaces, net radiation is temperatures than CLM 2.1. Differences greater than 1 K occur dominatedbytheshortwavedownwardterm. across muchofRussia and Canadaandinsmallerareas of India, The drag coefficient is important in determining how much Australia, the Arabian Peninsula, and Antarctica. Noteworthy SDF penetrates through the canopy and reaches the ground. negative differences occur in Greenland, far-eastern Siberia, Comparing the SDF in 2008 to the ground heat flux (Fig. 5a) southernBrazil,andtheNorthAfricancoast.Thetrendsin2008 indicateshowmucheachversionofCLMpartitionedthisenergy amplify the major trends of 2007, with a larger area of positive into the fraction conducted into the surface versus the fraction differencesinEurasiaandCanadatoAlaskaandexpandedareasof available at the surface for evapotranspiration. The ground heat negativedifferencesinGreenlandandNorthAfricatotheMiddle fluxoftheoriginalCLM2.0tendstobegreaterinthoseareaswith East. Differences in Antarctica between 30(cid:4)E and 150(cid:4)E change positivetemperaturedifferencesinFig.3andlessinthoseareas signfrom2007to2008asthepoolofcoldestair(<220K)inCLM with negative differences. Although on diurnal time scales, 2.1in2007(panelb)issmallerin2008(panele)versusCLM2.0 warmersurfacetemperaturesduetoenhanced groundheatflux (panels a and d, respectively). The positive differences in the feedbacktotheatmospherethroughenhancedsensibleheatflux. northern middle to high latitudes are in vegetated areas, and Theenhancedsensibleheatfluxoccursonlyaslongasittakesthe negative differences in Greenland, North Africa, Antarctica, and surface layer air temperatures to adjust upward. The areas of far-easternSiberiaareinbareoricecoveredareas. positive and negative differences in Figs. 3 and 5 coincide such Fig.4depictstheshortwavedownwardflux(SDF),typicallythe thatthelong-termmeansensibleheatfluxes(notshown)inCLM largestinputtothesurfaceenergybudget.Becausethefirst-order 2.0and2.1aresimilarintheseregions.Becauseoftheheatstorage determinants of SDF are latitude, cloud fraction, and the optical capacityofsoil,groundheatfluxesandthussurfacetemperature thickness of clouds, the maps in Fig. 4 reflect zonal differences (2008, Fig. 5b), both instantaneous and long-term means, may between the two versions in cloud cover and thus the radiation reflectsignificantmemoryofradiationinputs.ForLSMslikeCLM input to the surface energy budget. The areas of SDF less than 2.0thathaveaconstantdragcoefficient,Zengetal.(2005)found 120Wm(cid:6)2overthenorthernhighlatitudesarenoticeablylargerin thattoomuchradiationwaspartitionedintothegroundinareas CLM 2.1 (panels b and e) than in CLM 2.0 (panels a and d). The withsparsecanopies(e.g.,CanadianandSiberiantundras)leading differences betweenthetwomodels(bottompanels)arepropor- towarmsurfacetemperaturebiases.ThedragcoefficientinCLM tionally much larger north and south of 30(cid:4), over 30% in Central 2.1 is allowed to vary with the friction velocity and thus the Siberia, but only 10e15% in Central America in 2007 (panel c). canopythickness,amelioratingthisbias. Fig.4. MapsoftheannualdailymeananddifferenceshortwavedownwardfluxinWm(cid:6)2forCLM2.0andCLM2.1for2007(aec)and2008(def).Theresolutionandlayoutofthis figurearethesameasFig.3. K.I.Mohretal./EnvironmentalModelling&Software39(2013)103e115 109 Fig.5. The2008annualdailymeandifferencemapsfora)groundheatfluxinWm(cid:6)2b)surfacetemperaturesinKelvins.Forthesakeofbrevity,onlymapsfor2008areshown. 5.3. Effectsongeneralcirculation towersinawidevarietyofbiomeshasmadeitpossibletocreate gridded flux products useful for model validation from local to Fig.4cdepictsalargepositivedifferenceinSDFinMexicoin2007. globalscales(Jungetal.,2009,2010;Blythetal.,2010;Schlosser Fig.6containstimeseriesofthegridcellcenteredat26(cid:4)N102.5(cid:4)Wfor andGao,2010).InadditiontocomparingtheMMFeLISresultsto thefirstmonthoftheMMFeLISsimulations.Bothsimulations,CLM 2.0 and CLM 2.1, start with identical initial conditions. Differences betweenthetwoversionsbegintoemergeon2Jan.By3Jan,the difference in the daily maximum SDF is more than 300 W m(cid:6)2 becausethetotalcloudamounttimeseriesareatoppositeendsofthe y-axis. Although the SDF and cloud amount variability track each otherclosely,surfacetemperaturevariabilityinbothversionstracks the variability in SDF/cloud amount closely only for the first two weeks.Aftertwoweeks,thereislessvariabilityinanyofthevariables inFig.6andasmallerlagbetweenchangesinSDF/cloudamountsand changesinsurfacetemperaturesintheoriginalCLM2.0.Differences between the drag coefficients and vegetation temperatures in the modelscontributetotherapidlyevolvingdifferencesbetweenthe twoversionsinthesurfacevariablesofFig.6. The gridcell examined in this section is an important source regionfordry,warmairthat,ifitpenetratestheCentralUS,forms an elevated mixed-layer over humid airmasses transported from theGulfofMexico.Together,thesetwoairstreamshavelongbeen recognized for their impact on the convective weather environ- mentintheCentralUS,particularlyoftheseverevariety(Benjamin, 1986;LakhtakiaandWarner,1987;Laniccietal.,1987).Trieretal. (2011) demonstrate how differences in model predictions of surface conditions in the high terrain of Mexico can result in significantdifferencesinwarmseasonprecipitationintheCentral US.InFig.7,themappanelsshowhowquicklythesimulatedUS regionalcirculationsdivergeafterinitialization.Subtledifferences betweenthetwosimulationsappearbyDay5at200mbandDay 10 for the sea level pressures. By Day 15, there are substantial differences in the locations, orientations, and magnitudes of all cyclones/troughs and anticyclones/ridges in the region.After Day 16 in the critical Mexican gridcell (panels d and h), the sea level pressure time series of the original CLM 2.0 simulation indicates a series of transitory anticyclones versus a persistent longwave trough in the modified CLM 2.1 simulation. The anticyclones contributetoandarestrengthenedbythesurfaceheatingindicated inFig.6bytheCLM2.0simulation.Thedifferencesintheregional pressure patterns due to feedbacks between surface conditions, cloud amounts, and the upper troposphere take approximately 10e14daystoemerge,consistentwiththedeterministic predict- abilitylimitoftwoweeks(reviewsinLewis,2005;Yoden,2007). 6. Comparisontoglobalgriddeddatasets 6.1. Surfacefluxes Fig.6. Timeseriesofthegridcellcenteredat26(cid:4)N102.5(cid:4)W(Mexico)forthefirst monthoftheMMFeLISsimulationsfora)shortwavedownwardradiationflux,b)total TheFLUXNETisaglobalnetworkofmorethan500microme- cloudamount,c)surfacetemperature.Bothsimulations,originalCLM2.0(blackline) teorologicaltowersites(Baldocchietal.,2001).Thesitingofthese andmodifiedCLM2.1(redline),startwithidenticalinitialconditions. 110 K.I.Mohretal./EnvironmentalModelling&Software39(2013)103e115 Fig.7. NorthAmericanregionalmapsof200mbtemperatures(aec)andsealevelpressures(deg)fortheCLM2.0(blackline)andCLM2.1(redline)simulationsafter5days(a,e), 10days(b,f),and15days(c,g).Thebottompanelsarethetimeseriesforthegridcellcenteredat26(cid:4)N102.5(cid:4)W(Mexico)ford)200mbtemperatureandh)sealevelpressure. K.I.Mohretal./EnvironmentalModelling&Software39(2013)103e115 111 aFLUXNETgriddedproduct,wechosetheNASAGMAOreanalysis terrainandinthenortherntundrasaffectedbythewarmbiasin product, the Modern Era Retrospective-analysis for Research and CLM2.0. Applications(MERRA).TheMERRAisbasedontheGoddardEarth ObservingSystemVer.5(GEOS-5,Bosilovichetal.,2006,2011).The 6.2. Precipitation GEOS-5hasthesamedynamicalcoreastheGEOS-4butimproved moistprocessparameterizationsanddataassimilation(Rienecker There are a number of global gridded precipitation datasets et al., 2007). The data assimilation system of GMAO’s interactive available to assess model performance from diurnal time scales land-ocean-atmospheremodeling framework is NCEP’s Gridpoint onward. The CMORPH (Climate Prediction Center MORPHing Statistical Interpolation (Wu et al., 2002). By comparison, the technique)iscomposedofprecipitationestimatesderivedfromlow MMFeLIS results are from free-running simulations, i.e., lacking Earthorbitingsatellitemicrowaveobservationswhosefeaturesare adataassimilationsystem,andpartiallyinteractive,usingobserved transported via spatial propagation information obtained from SST/seaiceratherthanafull-physicsoceanmodel. geostationary satellite IR data (Joyce et al., 2004). Fig. 9 contains Fig. 8 depicts JuneeAugust 2007 mean latent heat flux for map views and latitudinal and longitudinal cross sections of FLUXNET,MERRA,andthetwoMMFeLISsimulations.Compared CMORPH,MERRA,andthetwoMMFeLISsimulationsofthemean totheFLUXNETandtoMMFeLIS,theMERRAmaphassignificantly daily precipitation for JuneeAugust 2007 (50(cid:4)N-50(cid:4)S). There are more latent heat flux over the humid regions of Central Africa, areaswhereallofthemodelsproducetoolittleprecipitation(the CentraltoSouthAmerica,EastAsia,Indonesia,andeasternNorth equatorial Atlantic, northern Europe, Central US), but there is America.Reichleetal.(2011)attributethispositivebiasinareasto generally more precipitation in the model output than in the excessive interception by dense canopies. The MMFeLIS maps CMORPH. In the Indian-Asian monsoon region, all of the models havefewerandsmallerareaswithlatentheatfluxesgreaterthan produceamuchlargerareaofheavy(>10mmday(cid:6)1)precipitation 120Wm(cid:6)2,althoughthereisanotablehotspotintheextensive thaninCMORPH.ThisisparticularlyacuteovertheWesternPacific wetlands around the Ob River in Central Siberia that is not re- warmpoolintheMMFeLISsimulationsversustheIndianmonsoon flectedintheFLUXNETmap.InaridAustraliaandCentralAsia,the regioninMERRA.AroundIndonesiaandinnorthernSouthAmer- MERRA performs better than both MMFeLIS simulations. ica,CentralAsia,andsoutheasternAustraliaMERRAisbothcloseto ComparingthedifferentversionsofCLMinMMFeLIS,theareasof the CMORPH and clearly better than both MMFeLIS simulations. elevatedlatentheatfluxinEurasiaandNorthAmericaaresmaller Both MMFeLIS simulations outperform MERRA in sub-Saharan andweakerinthemodifiedCLM2.1.Theoneareaoflatentheat Africa. Over the Pacific, the modified CLM 2.1 outperforms flux greater than 120 W m(cid:6)2 in the FLUXNET map is in the MERRAeastandnorthofthewesternwarmpool.Inthelatitudinal Central US. It is better represented in the original CLM 2.0 map, cross section, the boreal summer rainfall peak is wider in the althoughthemodifiedCLM2.1performsbetterthantheCLM2.0 MERRA,contributingtoaglobalmeanrainfallrate30%greaterthan in the important dryairmass source region in the Mexican high CMORPHversus23%greaterforMMFeLIS. Fig.8. GlobalmapsofJuneeAugust2007meanlatentheatfluxinWm(cid:6)2fora)FLUXNET,b)CLM2.0,c)MERRA,d)CLM2.1.Thecolorscaleisthesameforallmaps.Gridresolution ofeachmapisindicated. 112 K.I.Mohretal./EnvironmentalModelling&Software39(2013)103e115 Fig.9. Globalmeandailyprecipitationinmmday(cid:6)1forJuneeAugust2007(50(cid:4)N-50(cid:4)S)withmapviewsandlatitudinalandlongitudinalcrosssectionsofa)CMORPH,b)CLM2.0,c) MERRA,d)CLM2.1. There are important differences between the MMFeLIS simu- lowerthanobserved,particularlyinhumidregions(Suietal.,2007; lations.ThemodifiedCLM2.1comparesbettertoCMORPHthanthe Kain et al., 2008; Weisman et al., 2008). Unable to sustain high originalCLM2.0inmostofthePacific,theSouthAtlantic,northeast columnhumidity,theymayrainprematurely,affectingthesimu- Asia,theeasternandCentralUS,andsub-SaharanAfrica.Theareas lation of the diurnal cycle (Mohr et al., 2003; Tompkins and Di affectedbytheMexicanhighterrainsourceregion,theCentralUS Giuseppe, 2003; Bernie et al., 2007) and of convective develop- andMexico,arewetterinthemodifiedCLM2.1map.Theoriginal ment associated with fronts and troughs (Zeng et al., 2007; CLM 2.0 performs better than the modified CLM 2.1 in a couple Weismanetal.,2008).Thetimingandamountofprecipitationin muchsmallerareas,thenorthwesternAtlanticalongtheUScoast, simulationswithresolvedconvectionisalsohighlysensitivetothe whereCLM2.1istoodry,andinsoutheasternAustralia,whereCLM choiceoficemicrophysicalschemes(Lietal.,2009a,b;Satohetal., 2.1 is too wet. In the South Atlantic, the modified CLM 2.1 has 2010)andtotheuseofcycliclateralboundaryconditionsthattrap alowerprecipitationmaximumthanMERRA,butitishigherthan convection within the CRM, producing artificially long lifetimes theoriginalCLM2.0,andinnoneofthemodelsisittangentialto (Taoetal.,2009). theSouthAmericancoastasinCMORPH.Comparingthelongitu- For coupled ocean-atmosphere model configurations, adjust- dinalcrosssections,thepeakat150(cid:4)EislowerinthemodifiedCLM mentbetweenthesurfaceatmosphericandoceaniclayerstendsto 2.1,contributingtothe1%reductioninglobalmeanrainfallversus warm surface air temperatures and reduce total cloud amount theoriginalCLM2.0.TheresultsinFig.9forthemodifiedCLM2.1 (Costaetal.,2001;Biasuttietal.,2005;Räisänenetal.,2008).This arehighlysimilartotheresultsfromtheMMFconfigurationofTao effectoccurseveninslaboceanmodels,althoughanomalouslatent et al. (2009), completing a successful benchmarking of the heatfluxesmaydevelopinslabmodelstocompensateforimposed MMFeLIScodeagainsttheolderconfiguration. oceanheattransport,aneffectnotpresentinfull-physicsmixed- InastudyofprecipitationbiasesoverthetropicalAtlanticby6 layeroceanmodels(SuttonandMathieu,2002).Althoughtheuse differentfree-runningGCMsbyBiasuttietal.(2006),thecumulus of mixed-layer ocean models and data assimilation systems can parameterizations in the models overestimated the correlation reduce precipitation biases, the results in Figs. 8 and 9 and of between convective precipitation and surface humidity and previousstudies(e.g.,Biasuttietal.,2006;ShuttsandPalmer,2007; underestimated the correlation to upper tropospheric humidity. Stanetal.,2010)suggestimprovedmodelphysicsisstillimportant. TheywerethusacutelysensitivetoSSTdistributionandlatentheat Issueswithmodelmacro-andmicrophysics,surfaceobservational flux. This sensitivity is compounded in both parameterized and networks,satelliteradianceretrievalschemes,gridresolution,and resolved convection schemes by several additional factors. These theexpensesassociatedwithcomputingandstorageremainmajor schemesproduceunrealisticallyhighupdraftanddowndraftmass issuesinglobalmodeling(reviewsinZhangetal.,2005;Guilyardi fluxesandprecipitationefficiencies,rainingatcolumnhumidities etal.,2009).