Online data access in the SeaDataNet V1 system - PowerPoint PPT Presentation

1 / 40
About This Presentation
Title:

Online data access in the SeaDataNet V1 system

Description:

Online data access in the SeaDataNet V1 system by Dick M.A. Schaap technical coordinator Bologna, September 08 – PowerPoint PPT presentation

Number of Views:191
Avg rating:3.0/5.0
Slides: 41
Provided by: Thier60
Category:

less

Transcript and Presenter's Notes

Title: Online data access in the SeaDataNet V1 system


1
  • Online data access in the SeaDataNet V1 system
  • by
  • Dick M.A. Schaap technical coordinator
  • Bologna, September 08

2
Common Data Index CDI
  • CDI is a fine-grained index to individual data
    measurements (such as a CTD cast or moored
    instrument record). This is in contrast to EDMED,
    which is an inventory of data sets, each of which
    comprises a large number of measurements.
  • CDI paves the way for direct online data access
    through file downloads or online data requests.
  • Present CDI V0 system gt 340.000 CDI entries
    from 36 Data Centres from 29 countries
  • http//seadatanet.maris2.nl/v_cdi_v0/search.asp

3
Common Data Index CDI V0
4
Common Data Index CDI V0
5
Common Data Index CDI V0
6
CDI V0
  • Central and homogeneous metadatabase, but for
    data access the users are confronted with the
    various user interfaces of the SeaDataNet data
    centres (which can provide online search and
    access / ordering facility / e-mail request)

User interface
CDI
url
url
url
Data centre User interface
Data centre User interface
Data centre User interface
data
data
data
7
CDI V1
  • Central and homogeneous metadatabase plus
    transparent access to the distributed data sets
    via a unique user interface at the SeaDataNet
    portal

User interface
CDI
Central User Register
Shopping basket
Request Status Manager
Download Manager
Download Manager
Download Manager
data
data
data
8
CDI V1 System Modules
  • CDI Metadatabase and User Interface
  • Searching and browsing of metadata of data sets,
    managed by SeaDataNet Data Centres
  • SeaDataNet Central User Register with User Roles
    db web service
  • Details of users, their organizations and
    addresses, license period, Id-Passwords,
    SeaDataNet Roles of users
  • SeaDataNet Shopping Basket
  • Preparing a user request of multiple data sets,
    handling validation of users and roles via the
    Central User Register, and routing requests to
    the Request Status Manager
  • SeaDataNet Request Status Manager
  • Processing and administration of all requests
    and data deliveries (downloads), for users in
    communication with data centres. For data
    centres to oversee all transactions.

9
System Modules
  • SeaDataNet Data centre Download Manager
  • Handles communication with SeaDataNet Request
    Status Manager and takes care that requested
    files are made ready for downloading by User (if
    cleared! ) in the agreed formats via a local
    website address.

search shop
Data download
SeaDataNet Portal
Data centre
Data centre
Data centre
Data centre
Metadata transaction data
10
Order flow diagram
Search in CDI directory
CDI Meta database
Register Users Licenses

Results
Requests list
Shopping Basket
User Register
User Roles
Submit Requests (specify filetype? )
Log in
WS
User Check
Info about Conditions, Registration etc
NO
YES User Role Requests
11

CDI V1 Search home page
12

Search Results page
13

Search Detail page (full)
14

Search records selected for Shopping basket
15

Shopping basket
16

Submit Shopping list gt User Authentication
17

User identified and request for motivation and
order confirmation
18

Request confirmed and link to RSM (also by e-mail)
19
YES User Role Requests

Shopping Basket
Submit extended Request Form with motivation
Requests user info in RSM Split by DC
Status Pending
Request Status Manager (RSM)
Confirmation email to User with RSM info
A B C D
U
V
W
Combi logic
Validation of access via business matrix

20
Data access restrictions in CDI V1
CC Collection cost charge
FE Commercial charge
LI Licence
MO Moratorium
NA No access
NC Distribution cost
OG Organisation
RS Restricted
SR Academic
UK Unknown
UN Unrestricted
See Vocabulary L081 for definitions Multiple
restrictions may apply
21
User Roles in AAA Register, of relevance for data
access
SDRN02 Public
SDRN03 Academic
SDRN04 Commercial
SDRN05 National and local government
SDRN06 Pan-National government

See Vocabulary C866 for definitions
Multiple roles may apply
22
Business Matrix
Restriction Role A B C D
U
V
W
TBD
TBD
NO
OK
OK
TBD
TBD
NO
NO
NO
OK
TBD
23
Request Status Manager (RSM)
Validation of access via business matrix
Data Centre
Download Manager

Status OK
Accepted requests in Download Manager
Status TBD
Status NO
E-mail to DC manager
Status OK
Retrieving files preparing formats (if
database) placing/updating daily ZIP file in
user Download directory notifying RSM of
ready for download status
Status Ready for Download
Validate in RSM
Access Denied
Status Access Denied
24
Request Status Manager (RSM)
Data Centre
Download Manager
Retrieving files preparing formats (if
database) placing/updating daily ZIP file in
user Download directory notifying RSM of
ready for download status

Download URL
User Check via AAA
Not OK
OK
User Download webpage at Data Centre with option
for downloading daily zip files option for
removing daily zip files
No access
If Removed
Status Downloaded
25
Request Status Manager and Download Manager
  • RSM has Web service to communicate with the
    Download Managers, locally installed
  • RSM keeps IP-addresses, EDMO codes, data manager
    e-mail, trigger addresses, generic download paths
    and V1/interim V1 labels of all connected Data
    Centres in config file
  • Download Manager is Java programme, that must be
    locally installed
  • Download Manager also has a config file with
    IP-address of RSM service, EDMO code, paths to
    support directories with data sets, database
    mapping files, and coupling table

26
Functionality Download Manager V1.1
  • Communication with RSM Web service
  • Retrieving and processing order lists after being
    triggered
  • Retrieving data files, related to requested
    CDIs, via local_CDI_ID and local Coupling Table
  • Support for retrieving data sets from file
    management systems and from database systems,
    following the data set requests and the
    corresponding indicators in the local Coupling
    Table
  • If data sets are stored in database, then
    producing the data files in the agreed ODV format
    via read-method
  • Note Data sets from a file management system
    must be pre-processed in SDN transport formats
    (ODV and optional MedATLAS for ASCII type data,
    or NetCDF for grid type data)
  • Combining data set files in a new daily zip file
    or updating an existing daily zip file with the
    latest data sets and placing the daily zip files
    in a user download directory
  • Generating dynamic user download webpage,
    following SDN template, to give user overview and
    option of downloadable zip files
  • Note the download URL is composed of the generic
    download path of the Data Centre as specified in
    the RSM configuration file (at installation)
    the user login.
  • Protecting the download URL via the AAA login
    service (CAS)
  • Notifying the RSM Web service once a User has
    removed daily zip files (because the user has
    completed the download successfully) by
    indicating the status Downloaded for the
    associated data set requests
  • Keeping a local log

27

Request Status Manager - Homepage
28

Submit Shopping list gt User Authentication
29

User RSM Overview of request per Data Centre
and Status
30

User RSM Status Ready for Download Download
option
31

User personal Download webpage at Data Centre
32

User RSM Status All
33
Monitoring Tracking
  • All submitted requests, their users, the data
    sets concerned, and processing of these requests
    is administered in the Request Status Manager
  • SeaDataNet Data centres can also log on to the
    Request Status Manager and oversee all requests,
    handle tbd requests and prepare a number of
    online reports.

34

Data centre RSM overview Standing Orders
35

Data centre RSM overview To be Discussed Orders
36

Data centre RSM overview History Orders
37
SeaDataNet URLs
  • CDI V1 retrievalhttp//seadatanet.maris2.nl/v_cd
    i_v1/search.aspCDI V1 RSMhttp//seadatanet.mar
    is2.nl/request/welcome.html---------------------
    --------------------------------------------------
    ----
  • CDI V1 RSM Web service  TESTERhttp//seadatanet
    .maris2.nl/test/rsmtest.aspxWSDL
    http//seadatanet.maris2.nl/rsm/RequestStatusManag
    er.asmx?wsdl

38
SeaDataNet V1 Data Access planning
  • CDI V1 interface, database and XML import are
    ready
  • Download Manager V1.1 ready
  • Coming months all TTT partners will install the
    DM V1.1 and configure their infrastructure for
    being a full V1 data centre.
  • Preparing CDI V1 XML records following the
    upgraded CDI XML schema and using the new MIKADO
    software
  • Installing the Download Manager software
  • Choosing between pre-processed data files OR
    conversion via the Download Manager
  • Converting data files to SeaDataNet formats, in
    case of choice for pre-processed mode
  • Operational by end 2008

39
SeaDataNet interim V1 Implementation by other
partners in 2008
  • Coming months all other partners will also start
    populating the CDI V1 database
  • Preparing CDI V1 XML records following the
    upgraded CDI XML schema and using the new MIKADO
    software
  • Not yet installation of Download Manager, but
    data centres will be informed by the Request
    Status Manager by e-mail of all requests
  • Data centres will process all requests manually
    using the Request Status Manager
  • Effect is, that users will experience a common
    approach to all data centres. The difference is
    behind the scenes.
  • In 2009 partners gradually moving to full V1
    status by installing DM and delivering data sets
    in SeaDataNet Formats (conversion via new NEMO
    tool).

40
Thank you
Write a Comment
User Comments (0)
About PowerShow.com