The Importance of Adequate Verification and Validation Strategies in Risk Management - PowerPoint PPT Presentation

Loading...

PPT – The Importance of Adequate Verification and Validation Strategies in Risk Management PowerPoint presentation | free to download - id: 13fd8e-MDIyM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

The Importance of Adequate Verification and Validation Strategies in Risk Management

Description:

What's new in simulation and Computer Assisted Engineering (CAE) ... due to inaccurate analytical models, which overestimated the system margins. ... – PowerPoint PPT presentation

Number of Views:115
Avg rating:3.0/5.0
Slides: 47
Provided by: charles341
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: The Importance of Adequate Verification and Validation Strategies in Risk Management


1
The Importance of Adequate Verification and
Validation Strategies in Risk Management
  • Prof. Ch. Hirsch
  • Vrije Universiteit Brussel
  • President, NUMECA International

2
Content
  • Introduction
  • Error identification and management
  • Verification requirements
  • Validation requirements
  • Beyond VV
  • The new generation of simulation tools
  • Non-deterministic simulations
  • Robust design methodologies
  • Conclusions

3
Introduction
  • Whats new in simulation and Computer Assisted
    Engineering (CAE)
  • CAE covers all areas of physical simulation,
    particularly
  • Computational Structural Mechanics (CSM)
  • Computational Fluid dynamics (CFD)
  • Computational Electromagnetics (CEM)
  • Growth of computer power leads to unprecedented
    levels of CAE simulations
  • Million of points or degree of freedoms (DoF)
  • Complex geometries
  • Multiphysics (coupling fluid-thermal
    fluid-structure aero-acoustics,…)
  • Leading to Multidisciplinary design and
    optimization software systems (MDO)

4
Objectives
  • This leads to a growing request for quality
    assurance (QA) on the simulation tools
  • In order to respond to this request, joint
    efforts are required to define
  • uncertainty bounds on simulation results
  • limits on models
  • error control methodologies
  • appropriate test cases and QA strategies

5
Validation and verification
  • Validation The process of determining the degree
    to which a model is an accurate representation of
    the real world from the perspective of the
    intended uses of the model.
  • Verification The process of determining that a
    model implementation accurately represents the
    developer's conceptual description of the model
    and the solution to the model.
  • Verification is a prerequisite, upstream of the
    validation process and requires a dedicated
    effort towards adequate methodology

6
VV
7
Identification of Error Sources
  • Model errors and uncertainties
  • Difference between exact solution of the
    equations and real flow
  • Turbulence model uncertainties predominate
  • Application uncertainties
  • Uncertain boundary conditions or geometry
  • Discretization or numerical errors
  • Difference between exact solution and solution on
    a finite grid
  • In industrial CFD, solutions are usually not grid
    independent
  • Iteration or convergence error
  • Difference between converged solution and result
    after n steps
  • User errors
  • Mistakes, blunders, carelessness, optimism, etc.,
  • Programming or code errors (bugs)

8
Verification process
9
Verification process
  • No matter how many tests and empirical
    experiments we perform, we can never prove that a
    software implementation is error free, or has
    zero software defects.
  • Experience suggests that one can never totally
    eliminate bugs from complex CFD software,
    certainly not simply by performing algorithm
    testing alone.
  • An important approach to algorithmic testing in
    verification is the method of manufactured
    solutions

From W.L. Oberkampf, T. G. Trucano, Ch. Hirsch
(2002), Verification, Validation, and Predictive
Capability in Computational Engineering and
Physics. Paper presented at FOUNDATIONS 02,
Foundations for Verification and Validation in
the 21st Century Workshop, October 2002
10
Validation process
11
Validation hierarchy
12
Validation Application domain
  • In case a) we have high confidence that the
    relevant physics is understood and modeled at a
    level that is commensurate with the needs of the
    application.
  • In case b) the boundary of the domain indicates
    that outside this region there is a degradation
    in confidence in the quantitative predictive
    capability of the model.
  • Outside the validation domain the model is
    credible, but its quantitative capability has not
    be demonstrated.

From W.L. Oberkampf, T. G. Trucano, Ch. Hirsch
(2002), Verification, Validation, and Predictive
Capability in Computational Engineering and
Physics. Paper presented at FOUNDATIONS 02,
Foundations for Verification and Validation in
the 21st Century Workshop, October 2002
13
Validation Application domain
  • Figure 5a depicts the prevalent situation in
    engineering which shows the complete overlap of
    the validation domain with the application
    domain. The vast majority of modern engineering
    system design is represented in Fig. 5a.
  • Figure 5b represents the occasional engineering
    situation where there is significant overlap
    between the validation domain and the application
    domain. Some examples are prediction of crash
    response of new automobile structures, entry of
    spacecraft probes into the atmosphere of another
    planet, and the structural response of new
    designs for deep-water offshore oil platforms.
  • Figure 5c depicts the situation where there is no
    overlap between the validation domain and the
    application domain.
  • Predictions for many high-consequence systems are
    in this realm because we are not able to perform
    experiments for closely related conditions.
  • The inference from the validation domain can only
    be made using both physics-based models and
    statistical methods. Model calibration, which
    employs explicit tuning or updating of model
    parameters to achieve some degree of agreement
    with existing validation experiments, does not
    fully assess uncertainty in the predictive use of
    the model.
  • The requirement for predictive code use far from
    the validation database necessitates
    extrapolation beyond the understanding gained
    strictly from experimental validation data.
  • For example, the modeling of specific types of
    interactions of physical processes may not have
    been validated together in the given validation
    database. There is then uncertainty in the
    accuracy of model predictions describing such
    interactions.

14
Dramatic illustration
  • A significant example is to be found in the
    recent report from NASA about the failure of the
    scramjet hypersonic vehicle experiment X-43A.
  • The report, issued recently at http//www.space.co
    m, mentionsSPACE.com has learned that the
    failure of the NASA X-43A hypersonic aircraft in
    June 2001 was the result of inaccuracies in
    computer and wind-tunnel tests that were based on
    insufficient design information about the vehicle
    itself. According to the NASA-MIB documents, the
    X-43A Hyper X launch vehicle "failed because the
    vehicle control system design was deficient for
    the trajectory flown due to inaccurate analytical
    models, which overestimated the system margins."

15
VERIFICATION AND VALIDATION EFFORTS IN EUROPE
16
European activities
  • Various collective efforts in Europe for CFD
    validation
  • ERCOFTAC privately funded action on CFD Quality
    and Trust
  • QNET-CFD Thematic Network on QT
  • FLOWNET Collection of data for validation test
    cases
  • ECORA QT for CFD in the nuclear industry

17
PHASE 1 ERCOFTAC BEST PRACTICE GUIDELINES
18
Objectives
  • Guide to avoid the most common pitfalls
  • Guidelines not specific to individual codes,
    methods or applications
  • Guidelines that have very wide support
  • Not exhaustive 20 of rules to cover 80 of
    aspects
  • Modular form for each topic
  • Discussion section
  • Problem description and discussion, including
    references to important books, articles and
    reviews with relevant examples for the
    user.
  • Short simple statements of advice which provide
    clear guidance, are generally accepted, and are
    easily understandable without elaborate
    mathematics.

19
Follow-up
Expert Knowledge-base Expert knowledge on limits
of physical models in specific flow regimes
Extension Extension of scope of BPG to more
difficult flow regimes
Revision Updates, corrections and improvements
within current scope
Application Procedures Advice on how to best do
CFD and statements on accuracy for relevant
industrial cases
20
PHASE II THE QNET-CFD NETWORK
21
Objectives
  • To assemble, structure and collate existing
    knowledge on the industrial application of CFD
    and to make these available to European industry
  • To improve the quality of the industrial
    application of CFD
  • To improve the level of trust that can be placed
    in industrial CFD calculations by assembling,
    structuring and collating existing know-ledge
    encapsulating the performance of models
    underlying the current generation of CFD codes
  • To establish a shared database of computational
    and experimental results to support industrial
    applications
  • To provide a regular state-of-the-art review on
    quality and trust
  • To promote technology transfer between industries
    through workshops, regular meetings and
    electronic communication

22
Membership
  • 44 participating organisations with
    representatives from
  • 8 European Union member states
  • 1 EFTA member state
  • 2 Pre-accession states
  • The membership is principally built from
    industries, although there is a strong
    representation from universities and research
    organisations
  • A diverse range of industrial sectors is
    represented, including industries as diverse as
    civil construction and textiles
  • Also included representatives from the major
    European CFD code vendors
  • The organisations involved range from large
    industrial concerns to SMEs

23
Research Centres
Industries
Q N E T - C F D
DERA Health and Safety Executive NCSRD The
Meteorological Office CEA CIRA CIMNE Vattenfall
Utveckling AB Czech Academy of Science
  • WS Atkins Science Technology
  • CFS Engineering SA
  • Renault
  • Sulzer Innotec
  • NUMECA Int.
  • Electricite de France
  • SNECMA Moteurs
  • MTU
  • AEA Technology
  • BMW Rolls-Royce
  • MAN Turbo
  • ABB ALSTOM Technology
  • Rolls-Royce Power Engineering
  • Fluent Europe
  • Magnox Electric
  • Mott MacDonald
  • SNPE Propulsion
  • HR Wallingford
  • Arup RD



Universities
University of Brussels (VUB) University of
Karlsruhe University of Surrey University of
Poitiers University of Southampton University of
Rome Martin-Luther-Universitat NTUA FH
Niederrhein Cranfield University University of
Florence University of Czestochowa

24
Work Procedure
  • Development of knowledge base
  • Develop Quality Trust knowledge base that
    consist of a library of application challenges
    (AC) within each of the industry sectors and
    information on a series of well-documented flow
    regimes that underlie these industrial
    applications
  • Identification and documentation of application
    challenges
  • Quality checks of application challenges
  • Identification of underlying flow regimes (UFR)
  • Documenting underlying flow regimes and
    development of best practice advice
  • Quality checks on underlying flow regimes and
    review of best practice advice
  • Development of application challenge best
    practice advice
  • Review of best practice advice
  • Exploitation of knowledge base after end of
    project

25
Quality Control
  • Quality Control is a major element through the
    QNET-CFD operation
  • Managed by a Quality Coordinator (A. Hutton)
    and a Scientific Coordinator (W. Rodi)
  • A Quality procedure document has been generated,
    providing guidelines for acceptance or rejection
    of submitted ACs
  • All ACs have been screened and cross-reviewed by
    2 partners
  • After acceptance, they are recorded in the
    knowledge and data base.

26
Thematic Areas
  • The Network is organised around 6 Thematic Areas
    (TA) aligned with the following industrial
    sectors
  • TA 1 EXTERNAL AERODYNAMICS
  • TA 2 COMBUSTION AND HEAT TRANSFER
  • TA 3 CHEMICAL PROCESS, THERMAL HYDRAULICS AND
    NUCLEAR SAFETY
  • TA 4 CIVIL CONSTRUCTION HVAC
  • TA 5 ENVIRONMENT
  • TA 6 TURBOMACHINERY INTERNAL FLOWS
  • Each Thematic Area has a TA coordinator that
    assumes responsibility for the activities of the
    group

27
Underlying flow regimes UFR -
  • Subdivided in 4 categories
  • UFR 1 -- Free flows
  • UFR 2 -- Flow around bodies
  • UFR 3 -- Semi-confined flows
  • UFR 4 -- Confined flow
  • Each UFR can be attributed to several ACs
  • Contains experimental data and a variety of CFD
    simulations, from which Best Practice Advice
    (BPA) guidelines are derived
  • From the collection of BPAs of all the UFRs
    associated to an AC, a BPA for the AC is to be
    established

28
UFR 1 -- Free flows
  • 1-01 Underexpanded jet Partner-07
  • 1-02 Blade tip and tip clearance vortex flow
    Partner-09
  • 1-03 Buoyant plumes Partner-12
  • 1-04 Annular coaxial jets, flow and mixing
  • Partner-19
  • 1-05 Jet in a cross flow Partner-36

29
UFR 2 -- Flow around bodies
  • 2-01 Blunt base flow (streamwise flow past
    cylinder with cut-off end)
  • 2-02 Flow past cylinder  
  • 2-03 Flow around oscillating airfoil  
  • 2-04 Flow around (airfoils and) blades
    (subsonic) 
  • 2-05 Flow around airfoils (and blades) A-airfoil
    (Ma0.15, Re/m2x106)
  • 2-06 Flow around (airfoils and) blades
    (transonic) 
  • 2-07 3D flow around blades
  • 2-08 Flow around grid bars
  • 2-09 Rotor/stator interaction

30
UFR 3 -- Semi-confined flows
  • 3-01 Boundary layer interacting with wakes
    under adverse pressure gradient - NLR 7301 high
    lift configuration
  • 3-02 Atmospheric boundary layer with rough wall
  • 3-03 2D Boundary layers with pressure gradients
  • 3-04 Laminar-turbulent boundary layer
    transition
  • 3-05 Shock/boundary-layer interaction (on
    airplanes)
  • 3-07 Natural and mixed convection boundary
    layers on vertical heated walls
  • 3-08 3D boundary layers under various pressure
    gradients, including adverse pressure gradient
    causing separation
  • 3-09 Impinging jet
  • 3-10 Plane wall jet
  • 3-11 Plane wall jet in counter current flow
  • 3-12 Stagnation point flow
  • 3-13 Flow over isolated hills/valleys (without
    dispersion)
  • 3-14 Flow over surface-mounted cube/rectangular
    obstacles
  • 3-15 2D flow over backward facing step
  • 3-16 Wave-driven flow in a basin

31
UFR 4 -- Confined flow
  • 4-01 Secondary flow in rotating and non-rotating
    channels
  • 4-02 Confined coaxial swirling jets
  • 4-03 Pipe flow - rotating
  • 4-04 Pipe flow - non rotating
  • 4-05 Curved duct/pipe flow (accelerating)
  • 4-06 Swirling diffuser flow
  • 4-07 Developing channel flow with mass injection
    through wall
  • 4-08 Orifice/deflector flow
  • 4-09 Confined buoyant plume
  • 4-10 Natural convection in simple closed cavity
  • 4-11 Simple room flow
  • 4-12 Flows in chambers with multiple
    inlet/outlets
  • 4-13 Compression of vortex in cavity

32
Best Practice Advices
  • To be derived from the BPA of individual UFR
  • BPA to be strongly supported by the evidence
    examined in the UFR document Section on
    Comparison of CFD Calculations with Experiments
  • Recommendations for Future Work
  • Such as new experiments to be undertaken for
    which the values of key parameters are much
    closer to those encountered in real application
    challenges
  • Visit http//www.qnet-cfd.net

33
THE FUTURE OF VV
34
New approach to VV
  • It is generally recognized that, although actions
    towards the reduction of simulation uncertainties
    are still required, numerous sources of
    uncertainties will always remain.
  • Therefore new methodologies are required in order
    to incorporate the presence of uncertainties in
    the simulation process and in order to introduce
    the existence of these uncertain simulation
    results in the decision process related to
    industrial design.
  • This implies responses to the following questions
  • How to manage uncertainties in simulations, that
    is how to quantify, in a rational way, the impact
    of the different sources of uncertainties on the
    simulation results.
  • Given the existence of uncertain simulation
    results, how are multidisciplinary design
    optimizations (MDO) techniques to be developed
    and applied in order to support the industrial
    design process?

35
Next steps
36
Required steps 1
  • Quantification and management of simulation
    uncertainties 
  • Quantification and management of uncertainties is
    required at the individual discipline level (CFD,
    CSM, CHT,…) as well as at the integrated system
    level.
  • The current practice of CAE simulations is based
    on deterministic parameters such as fixed
    boundary or initial conditions, fixed geometry
    and physical properties, fixed physical model
    parameters, etc.
  • However, these conditions are not generally known
    precisely and are attached with unavoidable error
    levels, as listed above. Therefore one should be
    able to evaluate simulation results by
    incorporating these uncertainties in the
    simulation process, in order to approach a
    rational quantification of uncertainties,
    including the establishment of a confidence
    interval for the simulation-based predictions.

37
Required steps 1a
  • Quantification of uncertainties
  • Methods have to be developed to achieve the
    following objectives
  • model and quantify the parameter uncertainties
    and stochastic inputs (such as initial
    conditions, boundary conditions, geometrical
    uncertainties)
  • model the modeling uncertainties associated
    with physical models, transport properties,
    source terms
  • specification of the nature of the parameter and
    model uncertainties (interval limits, probability
    density distributions, fuzzy logic sets,…)
  • quantification of the parameter (and model
    uncertainties, based on all possible sources of
    information, including experimental data,
    analytical estimates, results from computational
    processes, ….

38
  • Quantification of uncertain variables

39
Required steps 1b
  • Management of uncertainties 
  • Implement probabilistic simulations, for which
    innovative mathematical and algorithmic methods
    have to be developed for the treatment of
    differential equations containing stochastic
    input parameters and model parameters. Several
    methods are to be developed, assessed and
    validated, such as
  • perturbation techniques whereby the variables of
    the problem are expanded in terms of Taylor
    series around their mean value (mainly for
    Gaussian or weakly non-Gaussian processes)
  • Monte Carlo methods for complete statistics, but
    generally excessively expensive
  • Fuzzy logic methods. Fuzzy logic allows to create
    models based on inexact, incomplete, or
    unreliable knowledge or data, and, moreover, to
    infer approximate behavior of the system from
    such models
  • Polynomial Chaos methods, whereby the randomness
    of the solution uncertainties is represented and
    the PDF of the different solution components is
    defined at every point and time.

40
Management of uncertainties 
  • Examples of probabilistic output for a model
    equation, with a relative viscosity variation

41
Probabilistic Design and Risk management
  • Introduction of probabilistic simulations into
    the design and decision process

42
Probabilistic Design approach
  • The objectives are therefore
  • To develop aerodynamic and structural
    optimization algorithms that provides designs,
    which are robust with respect to uncertainties in
    geometry, operating conditions, and code
    simulation uncertainties.
  • To control and reduce risks by providing designs
    with performances insensitive to intrinsically
    uncertain quantities
  • To reduce system risks by enabling uncertainty
    quantification and design strategies at the
    conceptual design stage

43
Probabilistic Design approach
  • The two major classes of uncertainty-based design
    problems are robust design problems and
    reliability-based design problems.
  • A robust design problem seeks a design that is
    relatively insensitive to small changes in the
    uncertain quantities.
  • A reliability-based design seeks a design that
    has a probability of failure that is less than
    some acceptable value.

44
Conclusions
  • Classical verification and validation require a
    large scale effort, which has to continue,
    although we will never eliminate uncertainties.
  • Risk management needs new generation methods to
    allow for
  • The quantitative assessment and management of
    simulation uncertainties by Non-Deterministic
    methodologies
  • Efficient probabilistic simulation outputs to
    quantify reliability bounds of the predictions
    (mean and standard deviations of relevant design
    quantities) in a rational way,
  • The development and application of design
    methodologies incorporating probabilistic based
    simulations

45
Conclusions
  • The non-deterministic approach is critical in
    order to
  • Reduce system risks by enabling uncertainty
    quantification.
  • Increase design confidence and safety by
    measuring and controlling uncertainty in
    performance predictions and to enable estimates
    of probabilities of failure

46
  • Thank you for your attention !

47
Verification procedures
  • From
  • W.L. Oberkampf, T. G. Trucano, Ch. Hirsch (2002),
    Verification, Validation, and Predictive
    Capability in Computational Engineering and
    Physics. Paper presented at FOUNDATIONS 02,
    Foundations for Verification and Validation in
    the 21st Century Workshop, October 2002

48
The KNOWLEDGE BASE
  • Created and managed by Univ. Surrey and Atkins
    company
  • Collects all the data from the ACs, their
    relevant UFRs
  • Covers experimental and the CFD data
  • Highly interactive and user-friendly environment
  • http//www.qnet-cfd.net

49
Required steps 2
  • Applications to subsystems and systems
  • The objective at this step is to provide
    efficient uncertainty quantification and control
    strategies and algorithms for multidisciplinary
    uncertainty quantification.
  • The general methodologies are to be applied,
    validated and tested for the different
    disciplines CFD, CSM, CHT,…for various aerospace
    and other representative applications.
  • In a second phase, the validated subsystem
    methods are to be applied for integrated
    components and systems, to be defined by the
    different application areas (aerospace,
    automotive, environmental,…).
About PowerShow.com