Running the Community Climate Simulation Model CCSM at NERSC

1
Running the Community Climate Simulation Model
(CCSM) at NERSC
CAM T340- Jim Hack
Adrianne Middleton National Center for
Atmospheric ResearchBoulder, Colorado
2
1870 Control Run
TS (Globally averaged surface temperature)
Years (constant-1870-conditions control run)
0
500-1000
3
5 Member 1870-2000 Historical Ensemble
A
C
B
D
E
2000
2000
2000
2000
2000
TS (Globally averaged surface temperature)
1870
1870
1870
1870
1870
Years
360 a
380 b
400 c
420 d
440 e
500
0
4
CCSM IPCC Graph
5
Model Vs Observations
Climate of the last Millennium
Caspar Ammann NCAR/CGD
6
CCSM 3.0 T85 on bassi
Coupler 8 pes
Atmosphere 128 pes
Ocean 40 pes
Sea Ice 16 pes
Land 16 pes

• Small number of processors (208), but long run
  time (16 days)
• Want 5 model years/day
• Queue is generally 3 days long
• Model runs roughly 1 model year/3 hours or max 4
  years/run slot

7
HPC dimensions of Climate Prediction
New Science
Better Science
(new processes/interactions not previously
included)
(parameterization ? explicit model)
Spatial Resolution
Timescale
(Length of simulations)
(simulate finer details, regions transients)

• Data Assimilation

Ensemble size
(quantify statistical properties of simulation)
(decadal prediction/ initial value forecasts)
Lawrence Buja (NCAR) / Tim Palmer (ECMWF)
8
HPC dimensions of Climate Prediction
New Science
Better Science
ESMmultiscale GCRM
Code Rewrite
Earth System Model
Spatial Resolution (xyz)
Climate Model
Timescale (Yearstimestep)
Regular 10000
?
400
1.4 160km
0.2 22km
1Km
100yr 20min
1000yr 3min
1000yr ?
AMR 1000
5
Today Terascale
Cost Multiplier
2010 Petascale
50
500
Data Assimilation
Ensemble size
2015 Exascale
Lawrence Buja (NCAR)
9
Paleoclimate
GlobalGeneral Circulation
IPCC AR3 1998
Continentallarge-scaleflow
BGC/Carbon CycleSpin-ups
IPCC AR4 2004 4TF
Regional
MJO convergence
IPCC AR5 2010 500TF
local
Resolve Hurricanes
CCSM Grand Challenge 2010 1PF
10
Community Climate System Model (CCSM)

• Current Configuration
• Hub and spoke design with 5 executables
• Exchange boundary information through coupler
• Each code quite large 60-200k lines per code
• Need 5 simulated years/day --gt Must run at low
  resolution
• Standard configuration run at scaling sweetspot
  of O(200) processors
• Petascale Configuration
• Single executable
• 5 years wall-clock day
• Targeting 10K - 120K processors per simulation
• CAM _at_ 0.25 (30 km, L66)
• POP _at_ 0.1 Demonstrated 8.5 years/day on
  28K Bluegene
• Sea-Ice _at_ 0.1 Demonstrated 42 years/day on 32K
  Bluegene
• Land _at_ 0.1
• Cpl

11
Data Processing

• Ample disk space (We use up to 600 GB at a time)
• Ample memory (We use 2-4 GB)
• Long serial interactive jobs are fine
• Software we use includes the netCDF Operators
  (NCO), NCAR Graphics Command Language (NCL),
  Ferret 5.81, and more.
• It's dependable, reliable, robust, and fits my
  requirements quite well. -- Gary Strand

12
Thanks! Any Questions?