Framework for Modernization and Componentization of Fusion Modules FMCFM: Status and Vision DEFG0205

1
Framework for Modernization and Componentization
of Fusion Modules (FMCFM) Status and Vision
DE-FG-02-05ER84192
Tech-X Corporation

• Ovsei Volberg (PI)
• Svetlana Shasharina and Johan Carlsson

2
Acknowledgements

• We would like to express our gratitude for the
  help, advice, comments and encouragement to the
  following people (in alphabetical order)
• Glenn Bateman, John R. Cary, Stephen Jardin, Jon
  Kinsey and Arnold Kritz
• All the people of the NTCC project.
• US DOE for the Phase 1 SBIR award.
• Finally I personally would like to thank all of
  you for inviting me to this seminar and for
  coming to listen to my talk.

3
Project Goal

• Assist Fusion Integrated Modeling projects by
  developing a framework that will contain a set of
  tools, standards, and techniques for
  modernization of established fusion codes with
  subsequent conversion of these modules to
  components with clear defined interfaces in
  accordance with modern software engineering
  practices.
• The project addresses the following problems
  relevant for the FSP
• Modernization of Fortran modules to avoid their
  reimplementation, protect investment in software
  development, and increase performance and safety.
• Conversion of modernized modules components with
  well-defined interfaces to facilitate their
  coupling.
• Solve the language interoperability problem use
  in the simulation of components written in
  different programming languages.
• Gain experience in components coupling.

4
DOE SBIR Program

• Contains 2 development phases
• Phase I  100K, prototype development, projects
  concretization
• Phase II  750, full development
• FMCFM is currently a Phase I project

5
Goals in Phase 1

• Evaluate NTCC standards and procedures.
• Compare them with other community-wide efforts in
  other areas of science.
• Extend them based on this comparison.
• Use the new standards and Fortran 90
  modernization techniques to modernize MMM95 and
  GLF23 transport modules.
• Define simplified interfaces between models and
  solvers
• Investigate the feasibility of using Common
  Component Architecture for converting fusion
  transport modules and solvers into components
• Evaluate CCA CCaffiene framework for component
  coupling and simulation.
• This will result in a prototype framework for
  fusion code modernization and componentization

6
Accomplishments to date

• Modernization
• We produced drafts of the additional (to NTCC
  Standards) development documents to be included
  in the FMCFM Developers Guide
• Fortran Coding Rules
• Naming Convention
• Policy for Interoperability Document that
  incorporates modified NTCC Standards as
  requirements
• Provided a flexible and portable build system
  based on GNU Autotools (de facto standard for
  open source development) for the framework that
  could benefit other community projects as well.
• By using modern software engineering concepts
  available in Fortran 90/95 we created Fortran 90
  wrappers to modernize two NTCC transport modules
• MMM95 - Multi-Mode transport model
• GLF23 - Gyro-Landau-Fluid drift wave model
• Incorporated a new transport equation solver
  implemented in C
• A combination of a relaxation scheme with a
  quasi-Newton scheme, well-suited for
  block-diagonal problems, like transport equations
  
• FMCFM is work in progress, it is not finished yet.

7
Outline for the rest of the talk

• Overview of Phase 1 modernization efforts
• What do we like in the NTCC ?
• Comparison of NTCC with other community wide
  efforts
• New development documentation
• Modernization of MMM95 and GLF23
• Componentization and coupling by using CCA tools
  and techniques
• Simplified simulation
• Driver, Transport and Solver CCA Components
• Development of Ccaffeine coupled simulation
• Conclusions

8
Phase 1 FMCMF is a prototype

• Phase I project is limited in scope
• Centered around the NTCC Modules Library and NTCC
  Standards
• Focused on modernization of just 2 transport
  modules
• MMM95 - Multi-Mode transport model
• GLF23 - Gyro-Landau-Fluid drift wave model
• Uses only CCA tools for componentization
• But the FMCFM will help us to answer the
  following questions important for fusion
  integrated modeling
• Which tools and techniques could be used for
  modernization?
• What are the benefits and difficulties of using
  the CCA tools for componentization?
• How does one make a component out of existing
  code?
• What kind of interfaces are needed for coupling?

9
What do we like about the NTCC project?

• NTCC Standards - a good set of requirements
• Rob Jacob, one of the designers of the CCSM Flux
  Coupler I wish that the Climate community had
  such a set of requirements before we started
• Thanks to the strict acceptance requirements,
  NTCC modules are modified already
• Carries out a specific task (transport, heating
  and etc.)
• Isolated with an interface such as an argument
  list of a procedure
• Contains the driver program for testing, test
  cases, and documentation
• We like how the maintenance issues are solved
• Peer Review Process
• Modern coding practices encouraged (for the new
  modules)
• Web-based access to module download and
  documentation

10
We evaluated several community-wide projects and
their development standards

• Sun-Earth System Modelling and Space Weather
  Prediction
• Space Weather Modelling Framework (SWMF)
  http//csem.engin.umich.edu/SWMF
• Center for Integrated Space Weather Modelling
  (CISM) http//www.bu.edu/cism
• Earth Climate and Weather Modelling and
  Forecasting
• Earth System Modelling Framework (ESMF)
  http//www.esmf.ucar.edu
• European Program for Integrated Earth System
  Model (PRISM) http//prism.enes.org
• Community Climate System Model (CCSM)
  http//www.ccsm.ucar.edu
• Aerodynamics Analysis and Design
• NASA Report TM-2003-212421
• Meteorology
• European Standards for Writing and Documenting
  Exchangeable Fortran 90 Code
• Common Components Architecture
• http//www.cca-forum.org

11
Results Evaluation and comparison with the NTCC

• We elevated several NTCC goals to standards,
  for example
• Avoid using hard-wired I/O unit numbers. Allow
  informational output to be switched on or off.
  Provide a method for rerouting warning or error
  message output to a user specified file or I/O
  unit number
• We enhanced the NTCC Standards by the
  Parallelization Standards to accommodate
  requirements of high-performance computing.
• We developed additional software engineering
  documents for the future FMCFM Developers Guide
• Fortran 90 Coding Rules
• Naming Convention (both for C/C and Fortran)
• Interoperability Specification (the enhanced and
  modified NTCC Standards became a section on
  Requirements in this document)
• The documents have been published on the Tech-X
  website via Confluence software(
  http//grid.txcorp.com and go to Fusion Framework
  Space) We will publish number of evaluated
  documents on this web site as well.
• We use the portable build system based on GNU
  Autotools similar to many community-wide
  projects, for example CCA and ESMF. We
  reinforced the build system by the Tech-X config
  module.

12
Results (cont.) Parallelization Standards

• Standard The parallelization mechanism (if any)
  should employ the MPI standard for inter-module
  communications.
• Standard If the MPI standard is used for
  intra-module communications, the physics module
  should be able to use an arbitrary MPI
  communicator.
• Standard The module needs to be compiled as a
  library in addition to its standard mode when it
  is compiled and linked into a stand-alone
  executable.
• Standard The module should read input from and
  write output to files. No hard-coded unit numbers
  are allowed.
• Goal Module should be able to write its state to
  restart file(s) and continue its run after a stop
  of execution using restart file(s) with bit-to
  bit compatibility on the same platform-compiler
  combination.

13
Our Fortran 90 Modernization Approach
Abstraction, Encapsulation and Modularity

• MODULE modules encapsulate data and routines
  modules are containers for grouping type
  definitions, interface blocks, data and
  procedures. USE gives the access to modules
  content.
• Allocatable arrays/pointers support for dynamic
  data structures.
• TYPE the user defined data types support
  abstraction.
• INTERFACE and MODULE PROCEDURE support method
  overloading the same method name , different
  argument lists.
• PRIVATE and PUBLIC support data hiding and
  public access.
• Explicit interface blocks enforce argument types
  verification in procedure calls.
• Renaming prevent method name clashes between
  modules.
• Array syntax whole array operations

14
Fortran features to be avoided (from our Fortran
Coding Rules document)

• COMMON blocks use the declaration of module
  instead.
• real8 - use kind statement instead
• EQUIVALENCE - use pointer or derived data types
  instead to form data structures.
• Assigned and computed GOTO - use the case
  construct instead.
• Arithmetic IF statements - use the block if,
  else, else if , end if or select case construct
  instead. Labeled DO constructs - use unlabeled
  end do instead.
• PAUSE
• DATA and BLOCK DATA.

15
Modernization of the NTCC Fortran 77 Modules

• Each transport module main program is broken into
  2 Fortran 90 (F90) modules a data definition
  module and a module with procedures that operate
  on this data.
• Each COMMON block is substituted by a F90 module
• No hard-wired I/O unit numbers, all I/O
  operations to and from file these files are
  associated with unit numbers at run-time via
  ModIoUtil F90 module.
• ModKind F90 module is used
• ModTransportInterfaceBlocks that includes
  explicit interface blocks for callglf2d and mmm95
  routines
• Each transport module is built into a library we
  link it with the test program if we need a stand
  alone version or we use it to create a CCA
  component with ports.

16
Example two main F90 modules for each transport
module

• Data F90 module
• module ModMMM95
• use ModUnitNumbers
• implicit none
• save
• ..
• real zdx, zxfact, zepslon
• ! the dimension of arrays below lt kr
• real, dimension(),allocatable zxb , zrminor
  , zrmajor , zelong , zdense
• real, dimension(),allocatable zdensi ,
  zdensh , zdensz , zdensf , zdensfe
• end module ModMMM95

• Procedure F90 module
• module ModWrapMMM95
• use ModMMM95
• use ModTranspInterfaceBlocks
• implicit none
• ! private ! except
• !PUBLIC MEMBER FUNCTIONS
• public init_io
• public set_defaults
• public read_input_namelist
• public verify_data
• public setup
• public compute
• public write_output
• public finalize
• contains
• end ModWrapMMM95

17
Example ModIoUnit

• Checks for I/O units that are free (unused).
• Associates files named input, temp and output
  with unused I/O units and opens them.
• Associates any named file with an unused I/O unit
  number and opens it.
• Closes opened files.
• An additional set of subroutines in a separate
  file allows to interface module procedures from
  Fortran 77 when USE statement cannot be used.

18
Example ModTranspInterfaceBlocks

• ModTranspInterfaceBlocks contains the explicit
  interface blocks for subroutines callglf2d and
  mmm95 that perform calculations for the transport
  modules GLF23 and MMM95. If this module is
  accessed by a USE statement in any program unit
  that makes calls to any of those subroutines,
  then those calls would be checked by compiler for
  errors in the number or type of arguments.
• An explicit interface for a computational routine
  of any new transport module will be added to this
  module

19
Evaluation of the CCA is in progress

• Our current/future use of the CCA tools
• The Ccaffeine execution environment for
  assembling coupled simulation from the models
  converted to components
• A peer-wise communication pattern via ports
  (common interfaces)
• Provide ports that are implemented by a component
• Uses ports, where c component makes calls to
  ports provided by another component
• Scientific Interface Definition Language (SIDL)
  that works with Babel language interoperability
  tool to address the language interoperability
  between Fortran transport components and the
  solver written in C.
• Our current work is concentrated on
• We prototyped the provide/uses ports (CCA
  interfaces) for a Driver Component, Transport
  Component and Solver Component
• At the current step, we have started to work with
  the CCA language interoperability tools
  (SIDL/Babel) to create interoperable components

20
Simplified coupling as a test

• For the initial coupling we consider densities
  and electron temperature and its gradient
  constant
• We assume that only ion temperature Ti and its
  gradient grad Ti constitute a state vector at the
  moment t in the simplified simulation (not to
  produce good physics, but to learn coupling in
  Ccaffeine)
• The initial input parameters for a transport
  component are taken from its test cases
  downloaded from the NTCC web site.
• Set and Get methods for the state vector
• The new evaluate method that includes set methods
  and compute method.

21
Sketch for Ccaffeine Coupling of FMCFM Components
22
We are ready for CCA

• Specifying CCA provide and use ports for the
  driver component, transport component and solver
  component
• Using SIDL/Babel to address the language
  interoperability.
• Using Ccaffeine framework as an environment to
  couple components
• Receiver makes right approach to insure that
  components get data in the form they need it.

23
Conclusions

• We gained valuable experience in NTCC module
  modernization, standards development and
  documentation.
• We created a portable build system that has been
  successfully used in our project.
• We developed Fortran modules and collected tools
  that can assist the community in the
  modernization efforts.
• We incorporated a new solver into the FMCFM
• We are working to formulate a coherent approach
  to how to build CCA components from Fortran
  legacy codes.
• We need to spend more time to determine common
  interfaces for coupling Driver with Transport
  Components (problems input/output
  transformations)

24
Questions to you

• What suggestions do you have for our work with
  standards and development documentation?
• What improvements could you recommend to our
  Fortran code modernization process?
• Suggestions for finishing the Phase 1 project and
  contents of the final report?
• Could you recommend what kind of role we could
  play in modernization and componentization which
  would help existing FSP efforts?
• Other modules?