Building NorESM (CMIP5) on the NSC triolith system

This page describes the building of the CMIP5 configuration of the NorESM (NorESM) model on NSC's triolith system (triolith). Triolith consists of 1200 HP SL230s compute nodes is equipped with two Intel E5-2660 (2.2 GHz Sandybridge) processors with 8 cores each, i.e 16 cores per node. Triolith is equipped with a fast interconnect (Infiniband from Mellanox (FDR IB, 56 Gb/s) in a 2:1 blocking configuration) for high performance for parallel applications.

Triolith is a SNIC-funded system, and computing time on Triolith is exclusively allocated to SNIC projects (Large, Medium and Small). See [1] and [2] for information on how to apply for a SNIC project. Contact: support@nsc.liu.se.

CESM1 unix group on triolith

If you are going to run the NorESM model on triolith, you should join the 'cesm1' unix group on triolith. This is not absolutely necessary but being a member allows access to shared storage space on /nobackup/global/proj/cesm1/NorESM where pre-downloaded initial data and ancillary files are stored. Email support@nsc.liu.se to request membership to the cesm1 unix group. Please note: Files that you copy to the shared /nobackup/global/proj/cesm1/NorESM will be charged to your space allocation on triolith.

NorESM - Getting the source code

NorESM is "free" software however contact with the Norwegian EarthClim consortium is required to use the model. Once an agreement is in place, the source code can be obtained from the shared directory on Triolith: /nobackup/global/proj/cesm1/NorESM.

Ensure the correct intel compile module and wrappers are loaded using:

module load build-environment/nsc-recommended

1. Copy and unzip the noresm-ver1-cmip5.tar.gz in your home directory.

cp /nobackup/global/proj/cesm1/NorESM/noresm-ver1-cmip5.tar.gz /home/${USER}

tar -xvf /home/${USER}/noresm-ver1-cmip5.tar.gz

2. Create a case in the same way cases are created for the CCSM/CESM model.

cd /home/${USER}/noresm-ver1-cmip5/scripts

./create_newcase -case ../cases/triolithTest -mach triolith -res f19_g16 -compset N2000AERL -pecount S

Note: Two pe patterns have been tested, one using 2 nodes (32 pes) and the other using 5 nodes (80 pes): see EC-Earth and CESM1 model timings for timings. the -pecount S keyword will set-up the model using the 2 node/32 pe configuration. For the 5 node/80 pe configuration use -pecount M.

3. Configure and build your model:

cd /home/$USER/noresm-ver1-cmip5/triolithTest

./configure -case

./triolithTest.build

4. Edit the env_run.xml file for the particular simulation you wish to make, then submit your job:

sbatch ./triolith.run

Notes

Initial data files can be found in the /nobackup/global/proj/cesm1/NorESM folder (rest*).

The Oslo aerosol source code can be found in the /home/${USER}/noresm-ver1-cmip5/models/atm/cam/src/physics/cam_oslo folder.