RCMES is designed to perform observation-based climate model evaluation, with emphasis on regional analyses. Here we demonstrate how you can perform an end-to-end evaluation of North American Regional Climate Change Assessment Program (NARCCAP) as published in Kim et al. (2013b). If you have not yet completed the Configuration File Training, you might want to do so now.
Quick Navigation
Download
Figure 5 and Figure 6
Figure 7 (summer and winter)
Figure 8 and Figure 9
Figure 10 and Figure 11
Figure 12 (summer and winter)
Figure 14 and Figure 15
Figure 16
Helpful Tips
Evaluation of the Surface Climatology over the Conterminous United States in the North American Regional Climate Change Assessment Program Hindcast Experiment Using a Regional Climate Model Evaluation System
J. Kim, Duane E. Waliser, Chris A. Mattman, Linda O. Mearns, Cameron E. Goodale, Andrew F. Hart, Dan J. Crichton, Seth McGinnis, Huikyo Lee, Paul C. Loikith, and Maziyar Boustani
Journal of Climate
August 2013, Volume 26, pp 5698-5715
DOI: 10.1175/JCLI-D-12-00452.1
All of the images in this tutorial are clickable and will open in a new tab for a zoomed view.
The following plots were produced by this tutorial and are found in your working directory.
Download
Download and Install RCMES
If you have not already done so, please go to the RCMES Download Page and install RCMES using either VM or Easy OCW. Next, download the data for this tutorial.
Download Model and Observation Datasets for Tutorial
This tutorial uses model and observation data that will need to be downloaded and saved to a specific directory within RCMES.
CLICK HERE to download the data for this tutorial.
If you are using VM for this tutorial, place the NARCCAP_data folder within your shared data folder that you set up during the VM Install. If you have restarted your machine since the last tutorial, you will need to remount your data folder.
If you are using Easy OCW for this tutorial, you will need to save your data to an appropriate place. It is recommended that you save your data in the climate/RCMES/data folder.
Configuration Files
The configuration files for this tutorial are found within your Configuration Files in a folder called NARCCAP_examples.
In your terminal, navigate to the NARCCAP_examples folder.
cd ~/RCMES/configuration_files/NARCCAP_examples
Type the following to see all of the example config files in the folder.
ls -lIf you are having difficulty, please refer to the Configuration File Training for help.
The following directions will help you edit the working directory so that your results are saved in one directory when you run the configuration files.
Running Configuration Files
Figure 5 and Figure 6 Configuration File
Using the configuration files within NARCCAP-paper and the skills learned in the Configuration File Training, edit each configuration file to change the Working Directory. In a text editor, change the working directory to where you want your results to be saved. A sample is shown below:
workdir: ./NARCCAP_results
You may also need to change your Target Dataset Path, depending on where you saved your data. If it is in the data folder within RCMES, then you will be able to leave the pre-loaded target dataset path. Otherwise, change the directory to where your data is stored.
Save and close your editor. You can now run the configuration file example. In the Terminal application, navigate to the RCMES folder and execute the program by inputting:
python run_RCMES.py configuration_files/NARCCAP_examples/Fig5_and_Fig6.yaml
RCMES will output the progress of the analysis on the screen. Your results will be saved to your working directory as they are completed. The following plots can be found in your working directory.
 |
| (Fig. 5) The annual-mean surface air temperature (°C) with regard to CRU observation. |
 |
| (Fig. 6) The spatial correlations and standard deviation of the simulated normalized annual-mean surface air temperature (°C) with respect to CRU observation. |
Figure 7 (winter and summer) Configuration Files
Edit Fig7_summer.yaml and Fig7_winter.yaml files to change the working directory. Make sure you are in the RCMES folder and run the following:
python run_RCMES.py configuration_files/NARCCAP_examples/Fig7_summer.yaml
and
python run_RCMES.py configuration_files/NARCCAP_examples/Fig7_winter.yaml
The following results are found within your working directory.
|
Bias |
Standard Deviation | ||
|
WINTER
|
 |
 |
|
|
SUMMER
|
 |
 |
|
| Correlation Coefficient | RMSE | ||
|
WINTER
|
 |
 |
|
|
SUMMER
|
 |
 |
|
| (Fig.7) The bias, standard deviation, correlation coefficient, and RMSE of average simulated surface air temperatures relative to CRU observations. | |||
Figure 8 and Figure 9 Configuration File
Edit Fig8_and_Fig9.yaml file to change the working directory. Make sure you are in the RCMES folder and run the following:
python run_RCMES.py configuration_files/NARCCAP_examples/Fig8_and_Fig9.yaml
The following results are found within your working directory.
 |
| (Fig. 8) The annual-mean precipitation (kg/m2) biases with regard to the CRU observation. |
 |
| (Fig. 9) The spatial correlations and standard deviation of the annual-mean precipitation (kg/m2) with respect to CRU observation. |
Figure 10 and Figure 11 Configuration File (including sub-regions map)
Edit Fig10_and_Fig11.yaml file to change the working directory. Make sure you are in the RCMES folder and run the following:
python run_RCMES.py configuration_files/NARCCAP_examples/Fig10_and_Fig11.yaml
The following results are found within your working directory.
Note: The sub-region map (shown below) is generated whenever sub-regions are set within the configuration file.
 |
| (Sub-regions) The NARCCAP domain and the terrain where the numbered boxes with white boundaries indicate the 21 sub-regions in which the precipitation annual cycle time series are evaluated. |
 |
| (Fig. 10) The simulated and observed precipitation annual cycle (mm/dd) for the sub-regions shown in Subregions Map. |
| Normalized RMSE (% annual mean) | Correlation with the CRU annual cycle |
 |
 |
| Bias | Standard Deviation |
 |
 |
| (Fig. 11) The bias, standard deviation, correlation coefficient, and RMSE between the CRU observation and 14 sub-regions for annual-mean precipiation. | |
Figure 12 (winter and summer) Configuration Files
Edit Fig12_summer.yaml and Fig12_winter.yaml files to change the working directory. Make sure you are in the RCMES folder and run the following:
python run_RCMES.py configuration_files/NARCCAP_examples/Fig12_summer.yaml
and
python run_RCMES.py configuration_files/NARCCAP_examples/Fig12_winter.yaml
The following results are found within your working directory.
Note: The metric automatically creates the figures for the RMSE and Correlation Coefficient in addition to the Bias and Standard Deviation, which are shown here, but not a part of the figures in the paper.
| Bias | Standard Deviation | ||
|
WINTER
|
 |
 |
|
|
SUMMER
|
|
 |
|
| Correlation Coefficient | RMSE | ||
|
WINTER
|
 |
 |
|
|
SUMMER
|
 |
 |
|
| (Fig.12) The bias, standard deviation, correlation coefficient, and RMSE of the simulated seasonal-mean precipitaiton in the 14 sub-regions for summer and winter relative to CRU observations. | |||
Figure 14 and Figure 15 Configuration File
Edit Fig14_and_Fig15.yaml file to change the working directory. Make sure you are in the RCMES folder and run the following:
python run_RCMES.py configuration_files/NARCCAP_examples/Fig14_and_Fig15.yaml
The following results are found within your working directory.
 |
| (Fig. 14) The annual-mean surface insolation over a 20-yr period compared to the SRB data. |
 |
| (Fig. 15) The spatial correlations and standard deviation of the surface insolation with respect to SRB data. |
Figure 16 (summer and winter) Configuration Files
Edit Fig16_summer.yaml and Fig16_winter.yaml files to change the working directory. Make sure you are in the RCMES folder and run the following:
python run_RCMES.py configuration_files/NARCCAP_examples/Fig16_summer.yaml
and
python run_RCMES.py configuration_files/NARCCAP_examples/Fig16_winter.yaml
The following results are found within your working directory.
Note: The metric automatically creates the figures for the RMSE and Correlation Coefficient in addition to the Bias and Standard Deviation, which are shown here, but are not a part of the paper.
| Bias | Standard Deviation | ||
|
WINTER
|
 |
 |
|
|
SUMMER
|
 |
 |
|
| Correlation Coefficient | RMSE | ||
|
WINTER
|
 |
 |
|
|
SUMMER
|
 |
 |
|
| (Fig.16) The bias, standard deviation, correlation coefficient, and RMSE of the surface insolation in the 14 sub-regions for summer and winter relative to SRB data. | |||
Helpful Tips
-
If you are getting errors, double check the target dataset path in your configuration files. Make sure it matches where you haved placed your data.
- Configuration File Workflow
