Testing ontological support for multidisciplinary model-based problem-solving for water management

1
Testing ontological support for
multidisciplinary model-based problem-solving
for water management
Huub Scholten and Adrie J.M. Beulens Wageningen
University - Netherlands
2
Index

• Introduction
• Problems to solve
• HarmoniQuA - MoST
• Project
• Ontological KB and Knowledge Editor
• Modelling Support Tool
• Verification - validation
• Mental Modelling Model
• Testing criteria and results
• Conclusions

3
Introduction

• Model-based problem solving is seen as a process
• Integrated problem-solving (with socio-economics
  aspects) ?
• multidisciplinary problems
• higher demands to QA
• QA through
• Process Knowledge Base
• (here for modelling, instantiated for water
  management)
• Process support tool

4
Problems to solve

• Growing interest for QA in modelling with a
  multitude of reasons
• ambiguous terminology
• a lack of mutual understanding between
  key-players,
• malpractice in regard to input data
• inadequate model set-up
• insufficient calibration/validation
• model use outside of its intended scope
• insufficient knowledge on some processes
• miscommunication of the modeller to the end-user
• overselling of model capabilities
• confusion on how to use model results in decision
  making
• a lack of documentation and transparency of
  modelling processes.

5
HarmoniQuA - the project

• Period 1-1-2002 31-12-2005 (4 years)
• 2.57 million Euro (1.65 million Euro financed by
  EC)
• 12 partners in 10 countries with expertise in 7
  domains of model based water management
• www.HarmoniQuA.org

6
HarmoniQuA - partners

• WU, The Netherlands
• GEUS, Denmark
• NTUA, Athens, Greece
• CEH-Wallingford, UK
• WL Delft Hydraulics,The Netherlands
• Cemagref, France
• BfG, Germany
• SMHI, Sweden
• VITUKI Plc, Hungary
• Uni Dortmund, Germany
• LNEC, Portugal
• DHI Hydroinform a.s., Czech Republic

7
HarmoniQuA - aims

• Developing a methodology and support for
  modelling in water management according to the
  Water Framework Directive consisting of
• Knowledge Base (KB)
• Guidance on how to model for water management
• Glossary of terms
• Modelling Support Tool (MoST)
• Providing guidance from the KB
• Monitoring what modelling teams do
• Helping generating fancy reports
• Training website for professionals and students
• Screen-recording movies on MoST
• Presentations, exercises, background information

8
MoST (KB tool) overview
9
Knowledge Base

• Content (what to do and how to do it) in
  increasing specialization
• Knowledge on processes (co-operating in teams)
• Specialised for modelling
• Instantiated for water management
• Implemented in an ontology (e.g. smart database
  Protégé2000, (http//protege.stanford.edu)
• Protégé2000 is for knowledge engineering experts
  and too difficult for domain experts
• HarmoniQuAs Knowledge Base Editor act as
  front-end for Protégé2000 and deals with
  authorisation issues
• Provided to users by MoST (Modelling Support Tool)

10
Modelling Support Tool (MoST)

• Provides guidance from the KB to users
• Can set-up and support the execution of
  multidisciplinary modelling projects
• Monitors the work of all persons involved in a
  modelling project
• Reports to specific audiences

11
Verification validation of KBS

• Verification determining that the KBS is built
  according to its specifications.
• Validation determining that the KBS actually
  fulfils the purpose for which it was intended.
• Testing more generic term, referring to
  verification and validation.

12
Mental Modelling Model

• If problem owner has a problem, he/she can
  organize a multidisciplinary team, consisting of
  team members, who have different disciplinary
  backgrounds and fulfil different roles in that
  team.
• The multidisciplinary team solve the problem at
  hand in a process with best professional
  practices.
• The process is carried out in a modelling
  project, consisting of one or more subprojects,
  belonging to one or more domains/disciplines.
• Subprojects
• are single domain subproject or multidomain
  subproject
• consist of steps.
• Steps consist of tasks.
• Tasks consists of activities.
• Tasks and activities can use methods and tools.
• What teams have to do can be made explicit in a
  knowledge base.

13
Structured testing

• Implemented ontological KBS (1)
• KB structure (11)
• KB content (related to water management) (12)
• Process decomposition (121)
• Content of decomposition elements (122)
• MoST (13)
• Training material (14)
• KBS technology (2)

14
KB structure (11)

• Criteria
• Correctness, completeness, consistency,
  granularity.
• Method(s)
• Discussions with (1) internal experts, (2)
  internal testers and (3) external reviewers.
• Using the structure to fill KB with the
  knowledge.
• Result(s)
• Initially discussions were fruitful, later not.

15
KB content process decomposition (121)

• Criteria
• Correctness, completeness, redundancy,
  consistency, transparency, granularity.
• Method(s)
• Discussions with (1) internal experts, (2)
  internal testers and (3) external reviewers.
• Using the KB in 2x10 test cases.
• Result(s)
• Demands for changing the process decomposition
  came from project partners and from the wider
  modelling society. These were related to (1)
  order of tasks, (2) dependencies of tasks and (3)
  is some piece of work a task or an activity.
• Achieving agreement was difficult, but
  implementing changes was easy and not time
  consuming.

16
KB content decomposition elements (122)

• Criteria
• Correctness, completeness, redundancy,
  consistency, meaningfulness.
• Method(s)
• Reading and using by (1) internal testers (not
  involved in filling KB, (2) professional
  modellers in test cases, (3) external reviewers.
• Result(s)
• Long list of errors, wishes, comments, but all
  appreciated the guidance from the KB.

17
Modelling Support Tool (13)

• Criteria
• Correctness, reliability, according to
  requirements, adequate for professionals,
  appropriate for students.
• Method(s)
• Using MoST by (1) internal testers (not involved
  in filling KB),
• (2) professional modellers in test cases, (3)
  external reviewers.
• Result(s)
• Incremental (prototyping) development of MoST in
  3 phases, with a test after phase 1 and phase 2 ?
  
• long list of bugs
• more support needed for multidisciplinary teams
  more flexibility (working on 2 decomposition
  levels), because activity-level is often too
  detailed.

18
Testing training material (14)

• Criteria
• Correctness, usefulness professionals,
  usefulness students.
• Method(s)
• Using MoST in (internal and external)
  workshops/seminars and university courses.
• Result(s)
• Many small changes, but training website and
  especially screen-recording movies on MoST
  appeared to flatten the (steep) learning curve.

19
Testing re-use of technology (2)

• Criteria
• Is the KB ontological structure reusable and is
  the tool MoST useful for other processes?
• Method(s)
• Applying the presented technology for other
  processes, i.e. within the EC-funded project
  Aquastress (aiming to mitigate water stress
  problems).
• Result(s)
• Applying the ontological KB structure for the
  Aquastress project, did not show any technical
  problem.
• MoST could show and use the new KB without a
  problem.

20
Conclusion

• Testing is a continuous process.
• Peer reviewed scientific output testing.
• Applying our technology for other processes in
  which multidisciplinary teams have to perform a
  complex job, shows that the technology has a wide
  field of application
• Public website www.HarmoniQuA.org
• free software
• training material
• etc.