THE NOAA CLIMATE TEST BED

1
THE NOAA CLIMATE TEST BED
05 Jan 2006
Climate Community
Climate Test Bed
Research Development
NOAA Climate Forecast Operations

Mission to accelerate the transition of research
and development into improved NOAA operational
climate forecasts, products, and applications.
2
THE NOAA CLIMATE TEST BED

• CTB Role
• Motivation
• What does the CTB bring thats new?
• Science Priorities
• SAB Recommendations Following 1st CTB Annual
  Meeting
• Framework Organization
• Accomplishments Milestones

3
MOTIVATION(Scientific)

• NCEP has developed and tested a new fully coupled
  Climate Forecast System (CFS) for use in S/I
  applications that shows considerable promise.
• The CFS is a significant step forward in
  forecasting ENSO related SST variability, having
  achieved at least parity with statistical
  forecasts.
• Faster development of NOAA climate forecast
  systems is needed to improve on these forecasts
  and to provide more useful climate forecast
  products and applications.
• Supporting NOAA climate forecast systems for use
  by climate researchers is an optimal way to
  leverage the expertise of NOAA and the external
  community in order to accelerate the necessary
  improvements.

4
Skill in SST Anomaly Prediction Niño 3.4 (DJF
81/82 AMJ 04)

15-member CFS reforecasts
5
NOAA Seamless Suite of ForecastProducts Spanning
Climate and Weather
Forecast Uncertainty
Years
Seasons
Months
Climate Prediction Products
2 Week
1 Week
Weather Prediction Products
Days
Hours
Minutes
6
MOTIVATION(Programmatic)

• In order to achieve the blend of internal and
  external efforts required to improve NOAA
  operational climate forecast systems, a Climate
  Test Bed (CTB) facility has been organized.
• The CTB will
• provide more traction and visibility to
  intraseasonal-to-decadal climate research by
  accelerating the transition to NOAA climate
  forecast operations
• provide an operational testing environment to
  support projects that result in a direct
  influence on NOAA climate forecast operations
• provide infrastructure and resources for (long
  term, AO-driven, community-wide) projects on
  broader research issues affecting NOAA climate
  forecast operations.

7
CTB ROLE

• What does the CTB bring thats new?
• A Focus on Transition
• Develop guidelines for smooth transitions
• A Means of Engagement
• Research, Operations and Applications communities
• A Mechanism for Distribution of Climate
  Information

8
A FOCUS ON TRANSITION

• In order to accelerate improvements in NOAA
  climate forecasts and applications, the CTB must
  define a process to
• Identify and communicate the scope of
  responsibilities for the
  Research (R), Operations (O), and
  Applications (A) communities.  
• Identify and delineate different types of
  transition
• R2O define where O picks up responsibility, and
  when a "hand-off" from R occurs.
• O2A define where A picks up responsibility, and
  when a hand-off from O occurs.
• R2A define where A picks up responsibility, and
  when a hand-off from R occurs.
• Identify resources for R, O and A to ensure
  smooth transitions.  

9
NCEPs Role in the Model Transition Process
EMC and NCO have critical roles in the transition
from NOAA RD to operations
Other Agencies International
Effort
Service Centers
Field Offices
EMC
NOAA Research
NCO
EMC
Observation System
Life cycle Support
Service Centers
User
OPS
Test Beds JCSDA CTB WRF DTC JHT
RD
Operations
Delivery
Transition from Research to Operations
Launch List Model Implementation Process
Concept of Operations
Requirements
Criteria
Forecast benefits, Efficiency, IT
Compatibility, Sustainability
10
NOAA RESEARCH DEVELOPMENT FUNNEL

• NOAA is working on
• Research to Operations Policy (complete)
• An overarching view of NOAA's research and
  development that provides criteria for research
  location
• A better understanding of how to apportion
  resource allocations
• Operational and life-cycle support for research
  items transitioned to operations is required and
  needs to be addressed

11

1
RD Community
CTB CPC
3
OPERATIONS
EMC CFS
4
O2A
User Community
5
12
RESEARCH TO OPERATIONS GUIDELINES

• The path to implementation of changes in the NCEP
  operational climate model suite consists of the
  following steps
• Model development and refinement
• Preliminary assessment
• Calibration (e.g. hindcasts)
• Interface with operations
• Final skill assessment
• Parallel tests
• Approval
• The roles of the host (research) institution and
  the operational center must be clear for each
  step to ensure smooth transitions.
• Resources are needed for both the research and
  operations communities to ensure smooth
  transitions.

13
CPC OPERATIONS TO APPLICATIONS GUIDELINES

• The path to implementation of operational tools
  in CPCs consolidated seasonal forecasts consists
  of the following steps
• Retroactive runs for each tool (hindcasts)
• Assigning weights to each tool
• Systematic error correction
• Specific output variables (T2m precip for US
  SST Z500 for global)
• Available in real-time
• The roles of the operational center and
  applications community must be clear for each
  step to ensure smooth transitions.
• Resources are needed for both the operations and
  applications communities
• Need to identify and refine steps for other types
  of O2A
• Delivering models and data to the community
  NOMADS Server
• Regional applications (energy, agriculture, H2O
  resources, fire Wx)

14
A MEANS OF ENGAGEMENT

• Collaboration with the climate community is key
• Meet customer demands for climate information.
• Invite broad participation on CTB boards and
  teams.
• Enhance collaborative transition projects (AOs)
  consistent with CTB science priorities and
  community interest.
• Improve coordination between NCPO grants programs
  and operational activities.
• Enhance links to cooperative institutes (e.g.
  CICS, JCSDA), international programs (e.g.
  CLIVAR, GEWEX), and other agencies (e.g. NASA,
  USDA, DOE).
• Expand participation in national meetings and
  regional meetings of topical interest
• Economic valuation of NOAA climate forecasts,
  products and applications.

15
MEETING CUSTOMER NEEDS

• Some Common Customer Demands
• Improve the skill of sub-seasonal and seasonal
  outlooks.
• Make them more usable and more easily understood.
• Expand products beyond averages to extremes.
• Give more temporal and spatial detail.
• Provide access to models and data.

• CTB Action Develop enhanced partnerships with
  intermediaries that bridge research/operations
  and regional applications
• How should we package our products?
• What kind of verification information should we
  provide?
• What new products should we provide?
• How often do customers want products updated?

16
POTENTIAL BENEFICIARIES

• Water Resource Managers
• Energy Production Managers
• Emergency Managers
• Agricultural planners
• Economic planners
• Investors
• Retailers
• Others (tourism, outdoor enthusiasts, general
  public)

17
PROGRAMMATIC PRIORITIES AND SCIENCE PRIORITIES

• The CTB management has been working with NCEP
  management to establish a prioritized short list
  of high level institutional priorities for
  accelerated transition of science into operations
  (Programmatic Priorities).
• The CTB Management is working with its CST and
  Transition Project Team leaders) to identify and
  articulate specific scientific priority areas
  (Science Priorities) needed to
  accomplish the Programmatic Priorities.
• These form the basis of a Science and
  Implementation Plan (distinct from the CTB
  Management Operating Plan).
• The SIP will be vetted with the SAB and the
  broader scientific community over the next
  several months.

18
PROGRAMMATIC AND SCIENCE PRIORITIES

• NOAA Climate Forecast System Improvements (CFS
  n1, n2,)
• Land ocean analyses models resolution
  physics diagnostics
• Multi-Model Ensemble Prediction System
• Consolidation verification of statistical
  1-tier dynamical models
• CFS, GFDL, NASA-GEOS, NCAR-CCSM, other, ensembles
  and (25-yr) hindcasts
• Ongoing Analysis of the Climate System
• Preparation for new reanalysis (data mining
  analysis system for climate)
• Verification, diagnostics, global monitoring
  nowcasting
• Development of operational statistical models
• Regional Applications
• Drought/NIDIS, energy, agriculture, water
  resources, fire weather
• Extreme events
• Attribution climate change

19
NOAA CFS IMPROVEMENTS

• Science Priorities
• The possible components of CFS (n1) are coming
  into focus
• Land Couple Noah LSM to CFS
• Ocean MOM4
• Resolution T126
• Physics Convective parameterization
  Multi-physics ensemble
• CTB Role
• Comprehensive diagnostic evaluation of potential
  future changes to CFS/GFS prior to
  implementation.
• Concerted effort to distribute CFS/GFS data sets
  to the community.
• Caveat Reduce emphasis on evaluation of older
  versions of the CFS/GFS

20
CTB FY06 RANKED EXPERIMENTS
CUnits
Group
TPT Experiment
1
1
Deep data assimilation in GODAS
4.1a
1
1
Balanced data assimilation in GODAS
4.1b
6
1
Hindcasts with the GFDL coupled model
5b
4
2
Sensitivity to convection
1a
1
2
GLDAS/Noah experiment
4.2a
2
2
Alternative wind forcing in GODAS
4.1c
1
3
Sensitivity to PBL formation
1b
4
3
CFS/Noah experiment
4.2b
2
3
Simulation with the GFDL coupled model
5a
1
4
Test of the new land scheme
2a
1
4
Soil moisture impact
6a
1
4
Identify areas sensitive to soil moisture
6b
1
5
Sensitivity to the subgrid-scale orographic
2b
10
5
2-Tier GFS experiments
3a
3
5
Test of the GFS model biases over the
3b
5
5
RCM experiment
4.2c

21
MULTI-MODEL ENSEMBLE PREDICTION SYSTEM

• NOAA/CTB is a natural lead to develop the
  strategy.
• CTB Role
• Coordinate "National" models that contribute to
  accelerated improvements in ISI climate
  prediction and projections, both in real-time
  products and with research that contributes to
  the CCSP and IPCC.
• National models are global ocean-atmosphere
  coupled models
• NCEP/CFS
• GFDL/CM2
• NASA/GEOS 5
• NSF/CCSM
• Coordinate with any of several Multi-Model
  projects around the world (e.g. European Euro-SIP
  and the Asian APCC).
• The roles and responsibilities of the host
  institutions and the operational center(s) must
  be clearly spelled out, commensurate with
  resources, to ensure smooth transitions.  

22
ONGOING ANALYSIS OF THE CLIMATE SYSTEM

• GEOSS will deliver unprecedented oceanic,
  atmospheric and land surface obs.
• Data assimilation systems have been developed for
  Wx modeling and analysis.
• Question How should the enhanced observations
  from GEOSS be analyzed?
  
• Answer Develop an analysis system for
  climate.
• CTB Role
• Promote and coordinate synthesis of ongoing
  reanalysis activities within NOAA, across
  agencies and internationally.
• Benefits leverage CTB infrastructure (e.g. SAB,
  OB, computer)
• Management Project Manager for Interagency
  Coordination Program Manager

23
REGIONAL APPLICATIONS(EXAMPLE HYDROLOGIC
APPLICATIONS)

• Proposal
• Meld key initiatives of the GAPP Core Project
  into the CTB and designate the GAPP Core Project
  and its external RD collaborators as a CPT for
  Land on the CTB
• Science Priorities
• Examine Noah LSM and GLDAS impacts on coupled CFS
  seasonal forecasts (which also addresses issues
  of land memory impacts in CFS in general)
• Provide objective input for operational Drought
  Monitor and Outlooks
• Develop a Drought Early Warning System in support
  of NIDIS
• Provide Regional Climate Model (RCM) seasonal
  forecasts
• Provide Multi-Model Ensembles for Hydrologic
  Forecasts
• Benefits
• Consistent with NWS Integrated Water Science Plan

24
SAB Recommendations(Written Report from 1st CTB
Annual Meeting)

• Science Priorities
• Focus the science strategy as a function of
  resources (, computing).
• Prepare separate Science Implementation, and
  Management Operations plans.
• (2) Community Involvement
• Increase participation in large annual meetings
  topical workshops.
• Use intermediaries (e.g. NWS CSD) to bridge
  research/operations applications.
• Clarify CTB proposal solicitations
• (3) Model Output Access and Distribution
• Develop hardware/software tools and data policy
  for disseminating climate information.
• Work with established data distribution centers.
  
• (4) SAB Structure and Meetings
• Hold a dedicated annual review meeting.
• Expand SAB membership (international and private
  sector).

25
FRAMEWORK

• Infrastructure
• Computing Support (1/3rd NOAA Red Research
  Computer)
• System and Science Support Teams (models, data,
  diagnostics)
• Contractors, Technical Assistants, System
  Administrators
• Management and Administrative Staff
• Director, Deputy, Secretary, Program Assistant
• Transition Projects
• Base (internally) Funded
• Short Term potential for impact on operations
  in 2 years or less
• FTEs reallocated from NCEP and other line
  offices
• Announcements of Opportunity
• Long term potential for impact on operations in
  2 or more years
• Projects are reviewed, competitive, community
  wide, broader science issues
• NOAA or external lead PIs
• Multi-agency (NOAA, NASA, DOE, etc)

26
CURRENT CTB RESOURCES

• NCEP Contributions
• Computing Support (1/3 of the NOAA Red Computer)
• 23 Redirected FTEs
• Management Team 0.5 (CPC) 0.25 (EMC) 0.25
  (CPC)
• System Support Team (data, software) 4 (CPC)
  0.5 (EMC)
• Science Support (Transition Project Teams) 16
  (CPC) 4 (EMC)
• NOAA Climate Program Office
• Augmentation funding for CTB Infrastructure
  (System Support TA in FY05, FY06) Additional
  funding for near term and competitive transition
  projects.
• CDEP/CTB AO-driven competitive science projects
  (FY06)

27
CTB COMPUTER RESOURCE

• NOAA Red Computer 1280 processors
• CTB Allocation 420 processors (1/3rd).
• Assume
• one unit CPU time for a 10 year CFS (CMIP) run
  at T126L64
• one unit 18 days on 64 processors.
• CTB Resource 120 units, or enough time for 120
  of these runs per FY.
• CTB Resources for Hindcasts
• 10 years worth of 15-member 9-month hindcast runs
  per FY (Tier-1)
• 17 years worth of 15-member 9-month hindcast runs
  per FY (Tier-2)
• Therefore, it will take 2 years to complete a 25
  year hindcast suite!

28
CTB ORGANIZATION
Oversight Board CPC CDC EMC GFDL IRI NCPO
Science Advisory Board
Director Higgins
Program Manager Ji
Deputies Pan Gelman
Climate Science Team EMC CPC CDC GFDL NASA
NCAR OST OCCWS COLA IRI EMC GFDL CPC CDC (Focal
Points)
Science / Software Support from Contractors
TAs, SAs
Test Bed Users
29
CHARGES FOR OB, CST, AND SAB

• Oversight Board (OB)
• Makes recommendations to Directors of NOAA
  Climate Program Office and NCEP concerning the
  CTB activities.
• Climate Science Team (CST)
• Guides CTB activities at the working level
  evaluates results of CTB activities makes
  recommendations to the CTB management on
  computing resource usage and access
• Science Advisory Board (SAB)
• Works with the broader climate community to
  provide independent advice on high-priority
  scientific challenges coordinates with other
  programs (e.g., CLIVAR, GEWEX)
• Complete ToR and Responsibilities are on the CTB
  Website http//www.cpc.ncep.noaa.gov/products/ct
  b

30
CTB FY05 ACCOMPLISHMENTS

• Q1 Annual Operating Plan (4 CTB NTOPs)
• Q1 CFS Assessment Meeting (WWB/NCEP, November
  16, 2004)
• Established 7 CTB Transition Project Teams
• Model parameterization,
• Model sensitivity
• Assessments
• Climate data assimilation (ocean and land)
• Multi-model ensembles
• Climate products and applications
• Q2 Initiated CTB Infrastructure (System and
  Science Support Teams)
• Q2 Established OB, CST and SAB
• Q2 Implemented CTB website
• Q2 CTB ToR , White Paper and Transition to
  Operations Plan
• Q3 CDEP / CTB Competitive Transition Projects AO

31
CTB ROSTER

• Director
• W. Higgins (CPC)
• Deputy Directors
• H. Pan (EMC) Transition
• M. Gelman (CPC) - Administration
• Program Manager
• M. Ji (OGP)
• Administrative Assistant
• J. Johnson (RSIS)

• Oversight Board
• R. Dole (CDC)
• F. Einaudi (NASA)
• J. Laver (CPC) Chair
• S. Lord (EMC)
• A. Leetmaa (GFDL)
• K. Mooney (OGP)
• S. Zebiak (IRI)
• Appropriate representatives of other stakeholders

32
CST ROSTER

• Climate Science Team
• K. Mo (CPC) Co-Chair L. Goddard (IRI)
• M. Hoerling (CDC) B. Kirtman (COLA)
• A. Kumar (CPC) K. Mitchell (EMC)
• S. Moorthi (EMC) J. Roads (Scripps)
• S. Saha (EMC) J. Schemm (CPC)
• S. Schubert (GMAO) Co-Chair M. Timofeyeva
  (NWS/OCCWS)
• J. Zhou (OST)
• Current balance reflects near-term CTB
  activities. Membership will naturally evolve to
  include more members from the broader research
  community (e.g. PIs with funded CTB projects) as
  the number of AO-driven activities increases.

33
SAB ROSTER

• Science Advisory Board
• T. Barnston (IRI)
• T. Busalacchi (ESSIC, University of Maryland) -
  Chair
• J. Kinter (COLA)
• M. Harrison (UKMO)
• E. Harrison (PMEL)
• D. Hartmann (University of Washington)
• G. Lau (GFDL)
• D. Lettenmaier (University of Washington)
• K. Redmond (DRI)
• M. Suarez (GMAO)

34
TRANSITION PROJECT TEAMS

• 6 Teams have been organized under the 3 CTB
  NTOPs
• Climate Model Assessments (CTB NTOP 18)
• Model Parameterization Team Assessment Team
  
  
  Model Sensitivity Team
  Data Assimilation Team
• Climate Products Forecast Tools and Analyses
  (CTB NTOP 20)
• Multi Model Ensembles (CTB NTOP 19)
• Each CTB team is asked to establish goals and
  milestones, prepare a list of experiments with
  justification and computational requirements.
• CST helps CTB management prioritize and carry out
  the experiments.
• The above process is carried out each fiscal
  year.
• PIs with funded (AO-driven) CTB projects will
  join one of the teams.
• The review process for CTB competitive transition
  projects and for NCEP computer resources is
  carried out concurrently.

35
CTB SYSTEM SUPPORT TEAM

• The CTB System Support Team includes Technical
  Assistants (TAs), System Administrators (SAs),
  and Information Technology (IT) specialists to
• support external users on NCEPs Climate
  Computer
• support users of CTB software, including the NCEP
  CFS components by providing code, scripts,
  computing information, training and debugging
  services
• request new accounts as required
• work with NCEP Central Operations (NCO), manage
  on-line and archive storage, including file
  organization and quotas, and act as a liaison to
  NCOs system administrators for CTB computing
• manage access to NCEPs climate information,
  including calibration runs, and work with EMC
  system administrators to put CTB output on NOMADS
  servers
• maintain CTB Web page
• CPC and EMC are supporting 4 SST members in FY05.
• OGP is supporting 1 CTB TA in FY05 (to manage CFS
  data sets). There are strong needs for
  additional SST members starting in FY06 to
  support CTB external PIs.

36
CTB SCIENCE SUPPORT TEAM

• The CTB Science Support Team includes Technical
  Assistants (TAs), and Focal Points (FPs) for each
  model and product system supported by the CTB to
• facilitate external collaboration on improvements
  to CTB model components
• coordinate experiments with CTB investigators
• train outside users of CTB-supported software,
  including forecast systems, products and
  applications
• to interface with CTB product developers and with
  forecast operations and applications activities.
• CTB has strong needs for additional SST members
  starting in FY06, especially to support
• CFS / GFS forecast model (1)
• CFS ocean model (and data assimilation) (1)
• CFS land surface (and data assimilation) (1)

37
CTB VISITING SCIENTIST PROGRAM

• Visiting scientists and post-doctoral fellows
  from other organizations such as NSF, NCAR,
  Universities and foreign organizations (e.g.
  international desks) will participate in CTB
  Transition Projects at NCEP.
• These visitors will be supported by their home
  institutions or by grants sponsored by NOAA or
  other agencies.
• Visiting Scientists can include graduate
  students.
• CTB plans to initiate this program in FY06.

38
CTB MILESTONES (FY06)

• CTB draft Science Implementation, and
  Management Operating Plans
• Support external CTB users to the extent
  resources are available
• Set up GFDL S/I hindcast system on NCEP computer
  and begin to test reproducibility of data
  compared to runs on GFDL computer
• Complete availability of twice-daily CFS data on
  publicly accessible server
• Upgrade the seasonal forecast consolidation tool
  to improve its use of skill information
• Contribute to Science and Implementation Plan for
  ongoing analysis of the climate system

39
CTB PROVISIONAL MILESTONES (FY07-FY11)

• Assessment of next version of the NCEP CFS/GFS
• Complete experimental testing of tier-1
  multi-model ensemble forecast system (EMCGFDL)
• Transition NASA model to CTB
• Transition CCSM (NCAR) model to CTB
• Complete experimental testing of tier-1
  multi-model ensemble forecast system
  (EMCGFDLCCSMNASA)
• Drought monitoring and seasonal prediction for
  NIDIS with multi-models via NLDAS

40
CTB ANNUAL REPORTING CALENDAR

• Sep CTB Transition Project Team Annual Reports
  due send to SAB
• CTB Annual Operating Plan and Budget
• Oct CTB Annual Meeting
• SAB/OB Meeting
• Nov SAB Written Summary Report due to CTB
• Dec CTB Written Response due to SAB
• Jan CTB Annual Report due to OB (starting FY07)
• Jan CTB SPTP updated
• CTB priorities shared with participating agencies
  and institutions
• Jul Update CTB provisional milestones

41
The CTB Message

• The NOAA CTB will
• Enhance cooperative partnerships between NOAA
  operational and research centers and the broader
  external research and applications communities by
  providing an operational testing facility that
  facilitates smooth transitions.
• Deliver opportunities for goal directed research
  using the Climate Forecast System, other climate
  models, and a state of the art multi-model
  ensemble approach to improving climate
  prediction.
• Accelerate the transition of research advances
  into enhanced NOAA operational climate forecasts,
  products and applications.
• Increase the range and scope of regional
  applications, and the economic benefit, of
  operational climate forecasts for policy-making
  and decision-making by end users.