Online data access in the SeaDataNet V1 system - PowerPoint PPT Presentation

1 / 46
About This Presentation
Title:

Online data access in the SeaDataNet V1 system

Description:

Online data access in the SeaDataNet V1 system by Dick M.A. Schaap technical coordinator Oostende, June 08 – PowerPoint PPT presentation

Number of Views:179
Avg rating:3.0/5.0
Slides: 47
Provided by: Thier72
Category:

less

Transcript and Presenter's Notes

Title: Online data access in the SeaDataNet V1 system


1
  • Online data access in the SeaDataNet V1 system
  • by
  • Dick M.A. Schaap technical coordinator
  • Oostende, June 08

2
SeaDataNet Infrastructure versions and planning
  • Version 0 Continuation and maintenance of
    existing Sea-Search systems with minor
    modifications
  • Version 1 All data centres involved in
    upgraded metadatabases and transparent data
    access. Prototype ready by March 2008 all TTT
    partners by end 2008 thereafter gradually
    migrating all partners operational by mid 2009.
  • Version 2 All data centres involved in
    Transnational Access and Access upgraded. Plus
    viewing services and further virtualisation.
    Operational by 2010.

3
Discovery services
  • EDMED - Data Sets
  • EDMERP - Research Projects
  • CSR - Cruise Summary Reports
  • EDIOS - Monitoring systems
  • EDMO - Marine Organisations
  • CDI - Common Data Index

4
SeaDataNet Users portal schema V1
Data download
User registration
Status of request
Data request
General request
Metadata request
data
metadata
Project info
Registr.
Standards
Cross search
My transact.
Requestsstatus manager
CDI
Shopping basket
EDIOS
EDMED
Software
AAA
CSR
EDMERP
EDMO
Vocabularies
Organisation data source id
User Register
CSR
EDIOS
CDI
EDMO
BODC Database
EDMED
EDMERP
BSH Database
at BODC
at BSH
at MARIS
Metadata Data catalogues
...
Entry point for access hits
services for metadata
Data downloading services
Download managers in Data centers
Visualization services (WMS) for V2
5
Common Data Index CDI
  • CDI is a fine-grained index to individual data
    measurements (such as a CTD cast or moored
    instrument record). This is in contrast to EDMED,
    which is an inventory of data sets, each of which
    comprises a large number of measurements.
  • CDI paves the way for direct online data access
    through file downloads or online data requests.
  • Present CDI V0 system gt 340.000 CDI entries
    from 36 Data Centres from 29 countries
  • http//seadatanet.maris2.nl/v_cdi_v0/search.asp

6
Common Data Index CDI V0
7
Common Data Index CDI V0
8
Common Data Index CDI V0
9
CDI V0
  • Central and homogeneous metadatabase, but for
    data access the users are confronted with the
    various user interfaces of the SeaDataNet data
    centres (which can provide online search and
    access / ordering facility / e-mail request)

User interface
CDI
url
url
url
Data centre User interface
Data centre User interface
Data centre User interface
data
data
data
10
CDI V1
  • Central and homogeneous metadatabase plus
    transparent access to the distributed data sets
    via a unique user interface at the SeaDataNet
    portal

User interface
CDI
Central User Register
Shopping basket
Request Status Manager
Download Manager
Download Manager
Download Manager
data
data
data
11
CDI V1 System Modules
  • CDI Metadatabase and User Interface
  • Searching and browsing of metadata of data sets,
    managed by SeaDataNet Data Centres
  • SeaDataNet Central User Register with User Roles
    db web service
  • Details of users, their organizations and
    addresses, license period, Id-Passwords,
    SeaDataNet Roles of users
  • SeaDataNet Shopping Basket
  • Preparing a user request of multiple data sets,
    handling validation of users and roles via the
    Central User Register, and routing requests to
    the Request Status Manager
  • SeaDataNet Request Status Manager
  • Processing and administration of all requests
    and data deliveries (downloads), for users in
    communication with data centres. For data
    centres to oversee all transactions.

12
System Modules
  • SeaDataNet Data centre Download Manager
  • Handles communication with SeaDataNet Request
    Status Manager and takes care that requested
    files are made ready for downloading by User (if
    cleared! ) in the agreed formats via a local
    website address.

search shop
Data download
SeaDataNet Portal
Data centre
Data centre
Data centre
Data centre
Metadata transaction data
13
Order flow diagram
Search in CDI directory
CDI Meta database
Register Users Licenses

Results
Requests list
Shopping Basket
User Register
User Roles
Submit Requests (specify filetype? )
Log in
WS
User Check
Info about Conditions, Registration etc
NO
YES User Role Requests
14

CDI V1 Search home page
15

Search Results page
16

Search Detail page (full)
17

Search records selected for Shopping basket
18

Shopping basket
19

Submit Shopping list gt User Authentication
20
Authentication, Authorization and Administration
  • Single Sign On system set up for access to
    distributed system
  • Users authentication information based on
    personal login / password
  • User must register to get one login, thereby
    agreeing with the SeaDataNet User Licence
  • Central User Register, but management is
    decentralized
  • Authorisation based on User Roles

21

User identified and request for motivation and
order confirmation
22

Request confirmed and link to RSM (also by e-mail)
23
YES User Role Requests

Shopping Basket
Submit extended Request Form with motivation
Requests user info in RSM Split by DC
Status Pending
Request Status Manager (RSM)
Confirmation email to User with RSM info
A B C D
U
V
W
Combi logic
Validation of access via business matrix

24
Data access restrictions in CDI V1
CC Collection cost charge
FE Commercial charge
LI Licence
MO Moratorium
NA No access
NC Distribution cost
OG Organisation
RS Restricted
SR Academic
UK Unknown
UN Unrestricted
See Vocabulary L081 for definitions Multiple
restrictions may apply
25
User Roles in AAA Register, of relevance for data
access
SDRN02 Public
SDRN03 Academic
SDRN04 Commercial
SDRN05 National and local government
SDRN06 Pan-National government

See Vocabulary C866 for definitions
Multiple roles may apply
26
Business Matrix
Restriction Role A B C D
U
V
W
TBD
TBD
NO
OK
OK
TBD
TBD
NO
NO
NO
OK
TBD
27
Request Status Manager (RSM)
Validation of access via business matrix
Data Centre
Download Manager

Status OK
Accepted requests in Download Manager
Status TBD
Status NO
E-mail to DC manager
Check in SDN User Register
Status OK
OK
Retrieving files preparing formats giving
URL for ZIP file
Not OK
Status Ready for Download
Validate in RSM
Access Denied
Status Access Denied
28
Request Status Manager (RSM)
Data Centre
Download Manager
Retrieving files preparing formats giving
URL for ZIP file

URL
Downloading data sets directly from Data Centres
Status Downloaded
29
Request Status Manager and Download Manager
  • RSM has Web service to communicate with the
    Download Managers, locally installed
  • RSM keeps IP-addresses, EDMO codes, data manager
    e-mail, trigger addresses and V0/V1 labels of all
    connected Data Centres in config file
  • Download Manager is Java programme, that must be
    locally installed
  • Functions DM
  • Communication with RSM Web service
  • Retrieving and processing order lists after being
    triggered
  • Double checking user via AAA Web service
  • Retrieving data files, related to requested
    CDIs, via local_CDI_ID
  • Preparing data files in agreed and requested
    formats (ODV, MedATLAS, NetCDF)
  • Placing data files in zip format for downloading
  • Informing RSM Web service of order processing
    status and urls
  • Keeping a local log
  • First version DM slim version gt no NetCDF and
    use of pre-processed files
  • Second version DM full version gt includes
    conversion of file formats and support for file
    management and database storage

30

Request Status Manager - Homepage
31

Submit Shopping list gt User Authentication
32

User RSM Overview of request per Data Centre
and Status
33

User RSM Status Ready for Download Download
option
34

User RSM Status All
35
Monitoring Tracking
  • All submitted requests, their users, the data
    sets concerned, and processing of these requests
    is administered in the Request Status Manager
  • SeaDataNet Data centres can also log on to the
    Request Status Manager and oversee all requests,
    handle tbd requests and prepare a number of
    online reports.

36

Data centre RSM overview Standing Orders
37

Data centre RSM overview To be Discussed Orders
38

Data centre RSM overview History Orders
39
SeaDataNet URLs
  • CDI V1 retrievalhttp//seadatanet.maris2.nl/v_cd
    i_v1/search.aspCDI V1 RSMhttp//seadatanet.mar
    is2.nl/request/welcome.html---------------------
    --------------------------------------------------
    ----
  • CDI V1 RSM Web service  TESTERhttp//seadatanet
    .maris2.nl/test/rsmtest.aspxWSDL
    http//seadatanet.maris2.nl/rsm/RequestStatusManag
    er.asmx?wsdl

40
SeaDataNet V1 Implementation planning
  • Prototype is ready
  • It includes a temporary set of CDI records and
    data sets, that are managed by RNODC
  • Majority of CDI test records is unrestricted,
    but some records have other data access
    restrictions
  • First version Download Manager installed gt ODV
    ASCII files and use of pre-processed files
  • See http//seadatanet.maris2.nl/v_cdi_v1/search.as
    p
  • Use your SeaDataNet login - password

41
SeaDataNet V1 Full implementation by TTT
partners in 2008
  • Coming months all TTT partners will start
    installing the Download Manager and configuring
    their infrastructure for being a full V1 data
    centre.
  • Preparing CDI V1 XML records following the
    upgraded CDI XML schema and using the new MIKADO
    software
  • CDI must cover all available data sets
  • Special attention to the data access
    restriction
  • Installing the Download Manager software
  • Filling the coupling table between the
    LOCAL_CDI_ID (in the CDI XML records) and the
    local Data sets
  • Choosing between pre-processed data files OR
    conversion via the Download Manager
  • Note data files can come from a local file
    management system OR from a local database via
    specific queries
  • Must be ready end 2008

42
SeaDataNet interim V1 Implementation by other
partners in 2008
  • Coming months all other partners will also start
    populating the CDI V1 database
  • Preparing CDI V1 XML records following the
    upgraded CDI XML schema and using the new MIKADO
    software
  • CDI must cover all available data sets
  • Special attention to the data access
    restriction and LOCAL_CDI_ID
  • Not yet installation of Download Manager, but
    data centres will be informed by the Request
    Status Manager by e-mail of all requests
  • Data centres will process all requests manually
    using the Request Status Manager and indicating
    the URL where the user can download the data sets
  • Must be ready end 2008
  • Effect is, that users will experience a common
    approach to all data centres. The difference is
    behind the scenes.

43
Request Status Manager (RSM)
Validation of access via business matrix

Status OK
Status TBD
Data Centre
Status NO
E-mail to DC manager
Retrieving files giving URL for ZIP file
Status OK
Validate in RSM
Status Ready for Download
Access Denied
Status Access Denied
interim V1
44
Request Status Manager (RSM)
Data Centre
Retrieving files giving URL for ZIP file

URL
Downloading data sets directly from Data Centres
Check by DC Manager
Status Downloaded
interim V1
45
SeaDataNet full V1 Implementation by other
partners in first half of 2009
  • Early 2009 all other partners will start with
    installation and configuration of the Download
    Manager, using the experience from the TTT
    partners
  • Installing the Download Manager software
  • Filling the coupling table between the
    LOCAL_CDI_ID (in the CDI XML records) and the
    local Data sets
  • Choosing between pre-processed data files OR
    conversion via the Download Manager
  • Note data files can come from a local file
    management system OR from a local database via
    specific queries
  • Must be ready mid 2009

46
SeaDataNet V1 Demonstration
  • See http//seadatanet.maris2.nl/v_cdi_v1/search.as
    p
Write a Comment
User Comments (0)
About PowerShow.com