DWD GISC Node Current Status

1
DWD GISC NodeCurrent Status
2
Agenda

• Welcome
• Current status Overview based on functional
  blocks
• High Level Architecture
• Web Interface
• DAR Component
• Catalogue systems, Metadata and identified issues
• GISC-to-GISC communication
• Experience with Blog solution and AFD
• Schema for GISC backup scenario
• Data Components
• MSS, AFD and subscription management
• Live Demo
• Challenges during the Implementation
• QA

3
High level architecture Functional blocks
Logging, Monitoring and Reporting
Authorisation/Authentication services
Subscription Management
Gray List Container
WMO GISC-GISC Interface

• Delivery tools
• Moving Weather
• AFD
• controller

• Datapool(s)
• controller/manager
• access adapter
• data sources

Input Management
Receiver
External DCPC

• Web functions
• Frontend
• Web services (interfaces)
• Web services (core)
• Business logic

• DAR component
• controller/manager
• catalogue
• editor

DAR-Generator/Importer
4
HLA Web Interface (User/Admin/Customer)
Logging, Monitoring and Reporting
Authorisation/Authentication services
Subscription Management
Gray List Container
WMO GISC-GISC Interface

• Delivery tools
• Moving Weather
• AFD
• controller

• Datapool(s)
• controller/manager
• access adapter
• data sources

Input Management
Receiver
External DCPC
DAR-Generator/Importer
5
HLA Web Interface (User/Admin/Customer)

• WebWerdis
• Redesign of WebWerdis
• Navigation structure / search functions
• Internationalization
• Integrate logging framework (necessary for
  monitoring)
• Code clean-up
• Integration with Geonetwork
• Geonetwork
• Customization of GUI

6
HLA Web Interface (User/Admin/Customer)

• Search by URL ISO 23950
• Timo (WMO Secretariat) did a SRU implementation
  in Geonetwork
• Solution is running in DWD DMZ
• 3 day implementation workshop with Timo about SRU
  and jzkit at DWD
• Direct integration into DWD GISC node

7
High level architecture Metadata Management
Logging, Monitoring and Reporting
Authorisation/Authentication services
WMO GISC-GISC Interface

• Delivery tools
• Moving Weather
• AFD
• controller

Input Management
External DCPC

• Web functions
• Frontend
• Web services (interfaces)
• Web services (core)
• Business logic

8
High level architecture Metadata Management

• DAR Catalogue
• Did extensively Performance Tests (Geonetwork
  2.2, terraCatalog 2.3)
• DWD formulated test cases
• Contracted external partner for tests
• Functional Tests
• DAR Generator / Importer
• Developed DAR metadata generator (template driven
  architecture)
• External review of DAR metadata
• Developed DAR importer
• Issues we identified during our work
• Tested DAR catalogues fulfilled most, but not all
  INSPIRE and WMO requirements
• DAR Metadata
• Paper with Request for Clarification (JMA, CMA
  and DWD)
• Schematron rules for validating WMO profile would
  be very helpful

9
High level architecture Metadata Management

• Metadata Harvesting
• CMA and DWD did several functional tests with
  OAI-PMH in 09/09
• Different implementations of OAI-PMH could not
  harvest each other
• Deleted metadata records different
  implementations
• Standard vanilla implementations modify harvested
  metadata
• Ongoing tests between CMA and DWD
• Started January 2010
• Initial tests showed 1 record 1 second ? space
  for improvement
• Backup strategy
• Exchange of metadata files as products

10
High level architecture Metadata Management
11
High level architecture G2G communication
Logging, Monitoring and Reporting
Authorisation/Authentication services
Subscription Management
Gray List Container
WMO GISC-GISC Interface

• Delivery tools
• Moving Weather
• AFD
• controller

• Datapool(s)
• controller/manager
• access adapter
• data sources

Input Management
Receiver
External DCPC

• Web functions
• Frontend
• Web services (interfaces)
• Web services (core)
• Business logic

• DAR component
• controller/manager
• catalogue
• editor

DAR-Generator/Importer
12
HLA GISC-to-GISC communication

• Collaboration CMA and DWD
• Memorandum of Understanding signed end of last
  year
• Two Workshops held
• GISC-to-GISC communication tests with CMA
• CMA and DWD installed Blog solution and AFD
  solution
• Data sets with appropriated time profile (issue
  warning during model transfer)
• Blog solution
• CMA and DWD did several improvements on
  publishing and polling components
• Usage of tar minimize latency of transfer
• Necessary to include size of metadata
  (AtomFeed) into calculation of transfer volume
• Identified several further improvements like
  prioritization, AtomFeed handling, logging
• JMA has provided a new improved solution in
  10/2009
• AFD solution
• Average transport time between 4-5 sec during
  24hour continuously test, warnings between 3-4
  sec
• In case of unsecured connection sometimes
  corrupted data occur

13
HLA GISC-to-GISC communication backup

• Technical specification requires a bilateral
  agreement from 2 GISC for a backup scenario
• CMA and DWD prepared proposal for priorities in
  DAR metadata
• First common draft of dissemination metadata (XML
  schema)
• Need to prove in practice

14
High level architecture Delivery components
Logging, Monitoring and Reporting
Authorisation/Authentication services
Gray List Container
WMO GISC-GISC Interface
Input Management
Receiver
External DCPC

• Web functions
• Frontend
• Web services (interfaces)
• Web services (core)
• Business logic

• DAR component
• controller/manager
• catalogue
• editor

DAR-Generator/Importer
15
High level architecture Delivery components

• Message Switching System
• REST Web-Service to manage and control Moving
  Weather
• First proposal for generic Interface
• Integration into WebWerdis (still under work)
• Subscription Management
• Controller time and event triggered management
  (still under work)
• Customization according to GISC-to-GISC backup
  necessary

16
Live Demonstration
17
TC 5 - Maintenance of Users, Roles, Authorization
and Authentication

• Search metadata
• Access data
• Show difference between authorized and
  unauthorized User
• User and Customer management
• Subscription
• View historic transactions

18
Metadata Management TC 1

• Show metadata editor
• Fully integration open because of separation
  between Intranet and DMZ
• Authorization required to add/modify/delete
  metadata

19
Metadata Management TC 2, TC 6

• A WIS Centre provides new metadata records
• Show that GISC Geonetwork doesnt contains new
  metadata records
• Show that GISC_PRODUCTS is empty
• Copy new metadata records to remote provider,
  reindex
• GISC harvests new metadata records
• Update of GISC Geonetwork and jOAI Provider
• Show update of Geonetwork with SRU query
• Show update of GISC_PRODUCTS

20
Challenges during the Implementation

• Metadata generation
• See paper Questions on Metadata from CMA, JMA
  and DWD from 11.2009
• WIS Specification is open what is good!

21
QA