Final Technical Report on Research Agreement DAAD190210452 Title: Improving High Resolution Weather Forecasting Principal Investigator: Edward Ott Co-PI: Eugenia Kalnay 1. Introduction The main goal of the project was to conduct research that can lead to the improvement of high- resolution weather forecasting. Our research plan to achieve this goal called for the development of an ensemble based Kalman filter data assimilation system. Such a system can improve forecasts by providing a highly accurate estimate of the atmospheric state (an analysis) and an ensemble representation of the uncertainty in the analysis. The analysis serves as the initial condition of a deterministic forecast, while the analysis provides initial conditions for the generation of an ensemble of forecasts. This ensemble provides an estimate of the uncertainty in the deterministic forecast and it can also be utilized to derive probabilistic forecast products. We achieved our goal by developing the Local Ensemble Kalman Filter, a model independent ensemble data assimilation system. We implemented this system on the Global Forecast System (GFS) of the National Centers for Environmental Prediction (NCEP), on the Finite- Volume Global Circulation Model (Fv-GCM) of the NASA Goddard Space Flight Center, and on the Regional Spectral Model (RSM) of NCEP. These three forecast systems are state- of- the- art operational forecast models and one of them, the NCEP RSM, can also be integrated at very high spatial resolutions in a non- hydrostatic mode. With these implementations, our major objective was to prove the concept of an accurate, computationally efficient and model independent ensemble Kalman filter system (algorithm and computer code). In this report we summarize the main concept and features of the LEKF (Section 2). We provide a brief summary of the results of our experiments with the global forecast models (Section 3) and the regional forecast model (Section 4). A list of papers (Section 5) and list of talks on our ARO sponsored research are also included. 2. The LEKF data assimilation system A detailed description and mathematical justification of the different components of the LEKF scheme can be found in Ott et al. (2002, 2004). Our current implementation of the computer code is based on Hunt (2005), which is essentially a mathematically optimized version of the concept and algorithm described in Ott et al. (2004). This latest version of the scheme is called Local Ensemble Transform Kalman Filter (LETK). Our implementation also includes a 4- dimensional treatment of the observations described in Hunt et al. (2004). We are currently in the process of implementing another extension of the algorithm to compensate for the effects of model errors (Baek et al. 2005). The most important unique feature of the LEKF is that it obtains the analysis and the estimate of the uncertainty in the analysis independently for each model grid points. The advantage of this approach is twofold. First, the preparation of the analysis for the different grid points can be easily distributed between many processors. Second, the amount of the mathematical calculations and data that each processor has to handle becomes modest if a sufficiently large number of processors is available. Figure 1. Error in the analysis of the zonal component of the wind at the 300 hPa pressure level. Shown is the 45- day (180 analysis cycles) mean of the absolute value of the error. The analysis is based on simulated observations at the real observational locations. Streamlines based on the 45- day mean of the ’true’ zonal wind in is also shown. The analysis errors are significantly lower in the areas where the observational coverage is dense, e.g., in the densly populated part of North America. 3. Implementation on the global circulation models The initial implementation of the LEKF on the NCEP GFS was described in Szunyogh et al. (2005). With the exception of the resolution that we reduced to T62 (about 150 km in the extratropics) in the horizontal and to 28 levels in the vertical, the model was identical to the operational version. In Szunyogh et al. (2005), we evaluated the implementation of the data assimilation scheme by assimilating simulated observations at randomly selected grid- point locations. The results demonstrated that the scheme was both accurate and computationally efficient. Encouraged by the positive results of these experiments, we then assimilated simulated observations at the locations of the real observations operationally assimilated by NCEP (the National Weather Service). We assimilated observations at the location of all operationally assimilated observations except for the satellite radiances. (Currently, the forward observational operators that are needed for the assimilation of satellite radiances are not implemented in the code. We hope to start implementing the forward operators in the near future). But, we assimilated all satellite derived wind observations, all observations collected by commercial airliners and reconnaissance planes, rawinsonde data, etc. As an example, we show the error in the zonal wind analysis for the 300 hPa pressure level. It can be seen that the analysis is highly accurate in and downstream of the regions of high observational density. Figure 2. Analyses of the geopotential height of the 500 hPa pressure level. Shown are the high resolution operational NCEP analysis (black contours) and the lower resolution LEKF analysis (red contours). There is a good agreement between the analyses of the main synoptic features. We note that satellite radiences are assimilated only in the operational cycle. We also found the system to be computationally highly efficient: the assimilation of an operational data set collected for a six- hour period takes between 10 and15 minutes on a cluster of 20 2.8 GHz Xeon processors, a machine which had a market value of less than $200k in 2003. Encouraged by the positive results obtained with simulated observations at realistic observational location, we have started experiments with real observational data. The preliminary results are very encouraging. An example is shown in Figure 2, where the LEKF analysis, obtained at T62 28- level resolution, is compared to the operational analysis of NCEP, which has T256 horizontal resolution (about 50 km in the mid- latitudes) and 54 vertical levels. Also, the operational analysis uses a large number of satellite radiances that are not included in the LEKF analysis. Nevertheless, the LEKF analysis closely resembles the NCEP analysis, which is a very encouraging results in this early phase of tuning for real observations. We are currently working on a verification package that will allow for the comparison of short term forecast errors for the two systems using observational information for verification. Figure 3. Time mean of the RMS error of zonal wind (A) and temperature (B) averaged over Northern Hemisphere (the blue line is the error for PSAS, and the red line is the error for the LETKF) with simulated rawinsonde observation assimilation. As mentioned earlier, the LEKF has also been implemented on the FvGCM model of NASA. The results are summarized in Liu (2006). The results of the validation experiments with this system and simulated grid point observations were in good agreement with those for the NCEP GFS. In addition, we carried out experiments with simulated rawinsonde observations. Since we have had access to the operational PSAS data assimilation system of NASA, we are able to assimilate the same simulated rawinsonde observations with both the LEKF and PSAS. Detailed results of this comparison are presented in Liu (2006). Here, we only show an example for the vertical distribution of the errors in the two systems (Figure 3). It can be seen that the LEKF analysis is considerably more accurate than the PSAS analysis throughout the troposphere. 4. Implementation on the regional model A summary of the results obtained with our initial implementation of the LEKF on the NCEP GFS is provided in Merkova (2005). In this initial implementation of the scheme the regional analysis is not cycled, i.e., the initial conditions of the regional background analysis are those obtained with the global analysis cycle. In Merkova et al. (2005), we show results that were obtained for simulated observations at model grid- points. The large- scale analysis is based on observations collected at randomly selected grid- point of the sparser global model grid, while the regional analysis is based on observations collected at randomly selected grid- points of the high resolution regional grid (the resolution of the grid is 50 km in the experiments). Figure 4. Analyses of the geopotential height of the 500 hPa pressure level. Shown are the true state of regional model NCEP RSM (black contours) and the LEKF analysis of the regional model (red contours). We are currently working with simulated observations taken at the real observational locations. An example for our preliminary results is show in Figure 4. Based on our experience with the global model, we expect that the transition from using simulated observations at real observational locations to real observation location will be relatively easy. We are also working on an implementation in which both the global and the regional data assimilations are cycled. 5. List of papers published by the support of ARO 5.1 Papers in refereed journals 1. Baek, S.-J., B. R. Hunt, I. Szunyogh, and E. Ott: Local ensemble Kalman filtering in the presence of model error. Tellus, (in print). 2. Kuhl, D., I. Szunyogh, E. Kostelich, G. Gyarmati, M. Oczkowski, B. Hunt, E. Kalnay, E. Ott, and J. A. Yorke, 2005: Assessing predictability with a local ensemble Kalman filter. J. Atmos. Sci., (under review) 3. Zimin, A. V., I. Szunyogh, B. R. Hunt, and E. Ott, 2005: Extracting envelopes of non- zonally propagating wave packets. Mon. Wea. Rev., (in print) 4. Szunyogh, I, E. J. Kostelich, G. Gyarmati, D. J. Patil, B. R. Hunt, E. Kalnay, E. Ott, and J. A. Yorke, 2005: Assessing a local ensemble Kalman filter: Perfect model experiments with the NCEP global model. Tellus, 57A. 528- 545. 5. Oczkowski, M., I. Szunyogh, and D. J. Patil, 2005: Mechanisms for the development of locally low dimensional atmospheric dynamics. J. Atmos. Sci., 65, 1135- 1156. 6. Baek, S.-J., B. Hunt, I. Szunyogh, A. Zimin, and E. Ott, 2004: Localized error bursts in estimating the state of spatiotemporal chaos. Chaos 14, 1042- 1049. 7. Ott, E., B. R. Hunt, I. Szunyogh, A. V. Zimin, E. J. Kostelich, M. Corazza, E. Kalnay, D. J. Patil, J. A. Yorke, 2004: A local ensemble Kalman Filter for atmospheric data assimilation.Tellus 56A , 415- 428.. 8. Ott, E., B. R. Hunt, I. Szunyogh, A. V. Zimin, E. J. Kostelich, M. Corazza, E. Kalnay, D. J. Patil, and J. A. Yorke, 2004: Estimating the state of large spatio- temporally chaotic systems. Phys. Lett. A., 330, 365- 370. 9. Hunt, B. R., E. Kalnay, E. J. Kostelich, E. Ott, D. J. Patil, T. Sauer, I. Szunyogh, J. A. Yorke, A.V.Zimin, 2004: Four- dimensional ensemble Kalman filtering. Tellus 56A, 273- 277. 10. Corazza, M., E. Kalnay, D. J. Patil, R. E. Morss, M. Cai, I. Szunyogh, B. R. Hunt, E. Ott, and J. Yorke, 2003: Use of the breeding technique to estimate the structure of the analysis "errors of the day". Nonlinear Processes in Geophysics , 10, 233- 243. 11. Zimin, A. V., I. Szunyogh, D.J. Patil, B. R. Hunt, and E. Ott, 2003: Extracting Envelopes of Rossby Wave Packets. Month. Wea. Rev, 131, 1011- 1017. 5.2 Papers in conference proceedings 1. Merkova, D., I. Szunyogh, E. J. Kostelich, G. Gyarmati, 2004: Local Ensemble Kalman Filtering with Limited Area Model. First Thorpex International Science Symposium, 6- 10 December 2004, Montreal, Canada. 2. Merkova, D., I. Szunyogh, E. J. Kostelich, G. Gyarmati, 2005: A Local Ensemble Kalman Filter for the NCEP regional spectral model. The Fourth WMO International Symposium on Assimilation of Observations in Meteorology and Oceanography, 18- 22 April 2005, Prague, Czech Republic. 6. List of talks on research supported by the ARO 6.1 Invite Talks Istvan Szunyogh 1. Statistical and Applied Mathematical Sciences Institute, Research Triangle Perk, NC, USA, October 5, 2005, Assessing Predictability with a Local Ensemble Kalman Filter. [Towards a Mathematical Advancement of Data Assimilation Workshop] 2. George Mason University , Fairfax, VA, September 29, 2005, The Local Ensemble Transform Kalman Filter of the University of Maryland for Atmospheric Data Assimilation. [School of Computational Sciences Colloqium]. 3. Statistical and Applied Mathematical Sciences Institute, Research Triangle Park, NC, USA, March 18, 2005, On the inherently local nature of atmospheric dynamics: Implications for Data Assimilation. [2004- 05 Data Assimilation for Geophysical Systems program seminar] 4. Institute for Pure and Applied Mathematics, University of California Los Angeles, Los Angeles, CA, USA, February 25, 2005. Assessing a Local Ensemble Kalman Filter. [Mathematical Issues and Challenges in Data Assimilation for Geophysical Systems: Interdisciplinary Perspectives short program] 5. Jackson State University, Jackson, MS, USA, May 25, 2004. The Local Ensemble Kalman Filter of the University of Maryland. [Workshop on Mesoscale and CFD Modeling for Military Applications sponsored by the Army’s High Performance Computing Research Center.] 6. United Kingdom Meteorological Office, Exeter, UK, May 12, 2004. A Local Ensemble Kalman Filter for the NCEP GFS: Perfect Model Experiments. 7. European Centre for Medium- Range Weather Forecasts, Reading, UK, May 10, 2004 . A Local Ensemble Kalman Filter for the NCEP GFS: Perfect Model Experiments. 8. Cooperative Institute for Meterological Satellite Studies, Madison, WI, USA, March 18, 2004 . Implementation of the LEKF on the NCEP GFS: “Perfect Model” Experiments. 9. National Centers for Environmental Prediction, Camp Springs, MD, USA, February 27, 2004 . Implementation of the LEKF on the NCEP GFS: “Perfect Model” Experiments. (Environmental Modeling Center Seminar Series]. 10. IBM, Watson Research Center, Yorktown Heights, NY, USA, January 29, 2004. Estimating the State of the Atmosphere. 11. University of Maryland, College Park, Maryland, USA, September 11, 2003. A Local Ensemble Kalman Filter for the NCEP Global Forecast System. [Seminar Series of the Department of Meteorology]. 12. Forecast Systems Laboratory, Boulder, Colorado, USA, September 8, 2003 . A Local Ensemble Kalman Filter for the NCEP Global Forecast System. [Data Assimilation Seminar Series]. 13. National Institute of Aerospace, Hampton, Virginia, USA, August 18, 2003. Data Assimilation Research at the University of Maryland. Atmospheric Composition Workshop. 14. Arizona State University , Arizona, USA, March 28 2003 . Spatio- Temporal Propagation of Localized Improvements in the Initial Conditions of Nummerical Weather Predictions, [Environmental Fluid Dynamics Seminar Series]. 15. National Centers for Environmental Prediction , Camp Springs, MD, USA, March 18 2003 . Local Ensemble Kalman Filtering: Numerical Experiments, [2nd Workshop on Ensemble Data Assimilation] 16. NASA Goddard Space Flight Center, Greenbelt, Maryland, USA, March 14 2003 [with Edward Ott and Eugenia Kalnay]. Local Ensemble Kalman Filtering, [Special Seminar]. 17. Columbia University , Lamont- Doherty Earth Observatory , New York, USA, March 7 2003 , (with D.J. Patil). Local Low Dimensionality of Atmospheric Dynamics and Applications to Data Assimilation (Physical Oceanography Seminar Series]. 6.2 Contributed Talks Istvan Szunyogh 1. Mechanisms for the Development of Locally Low Dimensional Atmospheric Dynamics. The First THORPEX International ScienceSymposium, Montreal, Canada, December 2. Identifying the origin of forecast errors in the 3 to 14 days range: A challenge. Predictability Workshop, University of Wisconsin, Madison, WI, March 15- 17, 2004. 3. A local ensemble Kalman filter for the NCEP GFS. 20th Conference on Weather Analysis and Forecasting/16th Conference on Numerical Weather Prediction. Seattle, WA, USA, January 11- 15, 2004. 4. Dispersion of the Effects of Targeted Weather Observations. The Norm Phillips Symposium. January 15, 2004. 5. Extracting envelopes of Rossby wave packets. 12th Cyclone Workshop, Far Hills Inn, Canada, September 20- 26, 2003. 6. Implementation of the University of Maryland Ensemble Data Assimilation Scheme on the NCEP GFS. 2nd Workshop on Ensemble Weather Forecasting, The Far Hills Inn, Canada, September 17- 19 , 2003.