Quick Navigation
Access to ESGF
Download
Example 1
Example 2
One of the three options for obtaining data that can be used with RCMES is using Earth System Grid Federation (ESGF). ESGF is an international collaboration that supports climate and environmental science by creating a gateway to data collections. The data is stored and managed by multiple sites around the world, including JPL/NASA, NOAA/ESRL, PCMDI/LLNL, and DKRZ. This tutorial will walk you through registering for an ESGF account, enroll in the CMIP5 and Obs4MIPs groups, obtaining an observation and model data set from ESGF, and finally running the evaluation using a configuration file. The first example will host the ESGF data locally on your machine and the second example will use RCMES to directly access the data through ESGF.
Access to ESGF
Obtaining an ESGF Account
There are multiple Gateways that can be registered for - this tutorial will walk you through registering for the NASA-JPL gateway. The different gateways are web sites that allow you to register for access and discover data that is hosted in the Data Nodes. Once you are registered at a Gateway, you gain access to all the Gateways and Data Nodes. If you are already registered for ESGF, you should move to the Download section of this tutorial below.
- Go to the ESGF website.
- In the top right corner, click on the Create Account button.
 |
- This will bring you to a Create User Profile page. Fill out the required fields in bold. Click Submit.
An OpenID account is assigned, which is used to login to all of ESGF. Your registration is effective immediately.
Enroll in Groups
To be able to download data, you must first enroll in a group. From the NASA-JPL ESGF home page, under the Projects Hosted at the NASA/JPL ESGF Node, click on the Obs4MIPs Data Access Registration link and follow the instructions to obtain NASA OBS data access rights.
 |
Download
RCMES
If you have not already done so, please review Getting Started with RCMES and Download RCMES using either VM or Easy OCW.
Example 1: Data downloaded from ESGF and saved locally
Data
Observation Data
This tutorial will be using Sea Surface Height data from the Obs4MIPs NASA/JPL-CNES AVISO dataset. There are multiple ways to search ESGF data including a free text or faceted search on the ESGF website, which then gives you multiple options of downloading data. This section of the tutorial uses a RESTful API that generates a WGET script to download all files matching the given criteria.
Copy and paste the following link into any browser:
http://esgf-node.jpl.nasa.gov/esg-search/wget/?query=*&realm=ocean&project=obs4MIPs&variable=zos&time_frequency=mon
This will generate a wget-#####.sh script that is downloaded. Move this file to a location where you would like to download your data. For example, create an ESGF-data folder in your data directory within RCMES and move the wget file to that location.
Open terminal and navigate to the ESGF-data folder.
cd RCMES/data/ESGF-data
Next you must execute permissions to the script, changing the # to the numbers of the file. Type the code below and press enter.
chmod +x wget-#####.sh
Then you will run the script with the following code:
./wget-#####.sh
The following screen will appear asking for your OpenID or to accept the default (if you were already logged in). Either type in your OpenID or hit Enter.
Next it will ask for your password. Type it in and press Enter. The wget script will run and begin downloading your file. Once it is done, your screen should look as follows:
Check to see that the file is there by typing in:
ls -l
You have successfully downloaded the observation data from ESGF! The data will be downloaded to the folder that your wget script is in. Next, you will download the model data for this tutorial.
Model Data
This tutorial will be using Sea Surface Height model data from MPI-ESM-LR. There are multiple ways to search ESGF data including a free text or faceted search on the ESGF website, which then gives you multiple options of downloading data. This section of the tutorial uses a RESTful API that generates a WGET script to download all files matching the given criteria.
Copy and paste the following link into any browser:
http://esgf-node.jpl.nasa.gov/esg-search/wget/?query=*&realm=ocean&project=CMIP5&variable=zos&time_frequency=mon&experiment=decadal1994&ensemble=r1i1p1&model=MPI-ESM-LR
This will generate a wget-#####.sh script that is downloaded. Move this file to a location where you would like to download your data. For example, create an ESGF-data folder in your data directory within RCMES and move the wget file to that location.
Open terminal and navigate to the ESGF-data folder.
cd RCMES/data/ESGF-data
Next you must execute permissions to the script, changing the # to the numbers of the file.
chmod +x wget-#####.sh
Then you will run the script with the following code:
./wget-#####.sh
The following screen will appear asking for your OpenID or to accept the default (if you were already logged in). Either type in your OpenID or hit Enter.
Next it will ask for your password. Type it in and press Enter. The wget script will run and begin downloading your file. Once it is done, check to see that the file is there by typing in:
ls -l
You have successfully downloaded the model data from ESGF! The data will be downloaded to the folder that your wget script is in.
Open the Configuration File
Once you have your model and observation data downloaded from ESGF, open the ESGF-Example_local.yaml file found in the RCMES/configuration_files/ESGF_example. Change the working directory if you wish and notice that the observation and model data sets are sourced locally.
Save and close the configuration file.
Run Example 1
Now that you have downloaded your data and examined your configuration file, you can run the example.
Change to the RCMES directory and run the ESGF-Example_local.yaml file.
python run_RCMES.py configuration_files/ESGF_example/ESGF-Example_local.yaml
Once it is done running, the following results should be found in your working directory.
 |
| (Fig. 1) The annual-mean sea surface height (m) between 1995 and 2004 with regard to Obs4MIPs NASA/JPL-CNES AVISO observation. |
Example 2: ESGF data accessed through RCMES
Data
Rather than host the data locally, this example will be using both observation and model data sets hosted through ESGF. The observation data set used in this example is Sea Surface Height data from the Obs4MIPs NASA/JPL-CNES AVISO. The model data set used in this example is Sea Surface Height data from MPI-ESM-LR.
Open the Configuration File
Open the ESGF-example.yaml file found in the RCMES/configuration_files/ESGF_example. In the configuration file, the user provides the dataset_id and the variable they wish to use for both the observation and the model data sets. See the image below of how the configuration file should be set up for obtaining the data through ESGF.
Save and close the configuration file.
Run Example 2
Change to the RCMES directory and run the ESGF-Example.yaml file.
python run_RCMES.py configuration_files/ESGF_example/ESGF-Example.yaml
When you run the configuration file using RCMES, you will be asked for your OpenID and password for your ESGF account. The data will be temporarily downloaded and used for your analysis. Once it is done running, the following results should be found in your working directory.
|
| (Fig. 2) The annual-mean sea surface height (m) between 1995 and 2004 with regard to Obs4MIPs NASA/JPL-CNES AVISO observation. |