THREDDS

1
THREDDS LEAD

• Overview of LEAD, Data, and THREDDS
• What is THREDDS
• How can THREDDS be used
• THREDDS in Prototype 1-SC

2
ADASAssimilation
UnidataDecoders
WRF
Decoded Data
Data
Analyzed Data
IDD (Raw data)
Analyzed Data
FileSystem
WRF Output
THREDDSCatalog Generator
Station Obs.Surface sounding
Data
IDD (Raw data)
Inventory Catalogs
THREDDS Integrated Server (Catalogs, OPeNDAP,
ADDE )
Data
Enhanced Catalogs
ADAMMining
3
ADASAssimilation
UnidataDecoders
WRF
Decoded Data
Data
Analyzed Data
IDD (Raw data)
Analyzed Data
FileSystem
WRF Output
THREDDSCatalog Generator
Station Obs.Surface sounding
Data
IDD (Raw data)
Inventory Catalogs
THREDDS Integrated Server (Catalogs, OPeNDAP,
ADDE )
Data
Enhanced Catalogs
ADAMMining
4
THREDDS/IDD
Config file
FileSystem
THREDDSCatalog Generator
Inventory Data
IDD (Raw data)
THREDDS Cataloged Data Server (Catalogs,
OPeNDAP, ADDE )
Inventory Catalogs
Station Obs.Surface sounding
THREDDS IDD Catalog
Annotations
Data
Data Sets (URL wrapped w/metadata)
5
What is THREDDS ?

• Thematic Realtime Environmental Distributed Data
  Services
• Designed to provide discovery mechanism for
  existing data repositories
• Enable data providers to describe their data
• Enable data consumers to find data
• XML catalog files
• Services built on top of catalogs

6
What is a THREDDS Catalog?

• Information (i.e. metadata) about a collection of
  data products
• URL to data or service
• Format, spatial/temporal coverage,
• Content, provenance,
• References to other catalogs

7
What is a THREDDS Catalog?(continued)

• XML
• Well defined (yet evolving) schema
• Data URL wrapped with metadata
• Minimal structured metadata - Flexible
• Can include elements from other namespaces
• Supports use of other metadata vocabularies
• e.g. Dublin Core, DIF, FGDC, ADN
• Services to generate other flavors of metadata

8
THREDDS Catalog XML Content
THREDDS catalogs are for communicating
information about a collection of datasets

• Hierarchal structure of datasets
• Dataset access methods
• Metadata

1
0..
0..
0..
0..
9
The Master LEAD Catalog
lt?xml version"1.0" encoding"UTF-8"?gt ltcatalog
xmlns"http//www.unidata.ucar.edu/namespaces/thre
dds/InvCatalog/v1.0"
xmlnsxlink"http//www.w3.org/1999/xlink"
name"Master LEAD THREDDS Catalog"gt
ltdataset name"LEAD Testbed THREDDS Catalogs"gt
ltcatalogRef xlinktitle"Unidata LEAD Testbed"
xlinkhref"http//lead1.unidata.uc
ar.edu8080/thredds/catalog.xml"/gt
ltcatalogRef xlinktitle"Indiana LEAD Testbed"
xlinkhref"http//chinkapin.cs.indi
ana.edu8080/thredds/catalog.xml"/gt
ltcatalogRef xlinktitle"Oklahoma LEAD Testbed"
xlinkhref"http//aquaman.caps.ou.
edu8080/thredds/catalog.xml"/gt ltcatalogRef
xlinktitleUAH LEAD Testbed"
xlinkhref"http//moby.itsc.uah.edu8080/thredds/
catalog.xml"/gt lt/datasetgt lt/cataloggt
10
Sample Unidata Testbed Catalog
lt?xml version"1.0" encoding"UTF-8"?gt ltcatalog
xmlns"http//www.unidata.ucar.edu/namespaces/thre
dds/InvCatalog/v1.0
xmlnsxlink"http//www.w3.org/1999/xlink"
name"LEAD Eta Catalog"gt ltservice name"lead"
serviceType"DODS" base"http//lead1.unidata.ucar
.edu/cgi-bin/nph-dods/test-data/GRIB/" /gt
ltdataset name"LEAD Eta Catalog"gt ltdataset
name"NCEP Eta 80km CONUS model data"
harvest"true" IDlead.unidata.model.eta_211"gt
ltauthoritygtunidata.ucar.edult/authoritygt
ltdocumentation type"summary"gtETA model data is
available at 12Z and 00Z, with analysis
lt/documentationgt ltdocumentation
type"rights"gtFreely Availablelt/documentationgt
ltpublishergt ltname vocabulary"DIF"gtUCAR
/UNIDATAlt/namegt ltcontact
url"http//www.unidata.ucar.edu/"
email"support_at_unidata.ucar.edu" /gt
lt/publishergt ltgeospatialCoveragegt
lt/geospatialCoveragegt lttimeCoveragegt
lt/timeCoveragegt ltvariables
vocabulary"CF-1.0"gt ltvariable name"T"
vocabulary_name"Temperature at isobaric levels"
units"degK" /gt ltvariable name"RH"
vocabulary_name"Relative humidity at isobaric
levels" units"percent" /gt
lt/variablesgt ltdataset name"NCEP Eta 80km
CONUS 2004-10-14 120000 GMT" urlPath"2004101412
_eta_211.nc"gt ltserviceNamegtleadlt/serviceNa
megt lt/datasetgt lt/datasetgt
lt/datasetgt lt/cataloggt
11
Possible Catalog Hierarchy
12
THREDDS Catalog Generation

• Scheduled automated process
• Could be event driven
• Dynamically generated
• New data product
• Query result
• GUI tools
• Java API
• XML file hacking
• Third Party annotation

13
THREDDS Catalog Consumption

• Reusable UI components to browse
• Use in client applications
• Swing, JSF?
• THREDDS enabled clients (e.g. IDV, THREDDS Data
  Viewer) access metadata via Java API

14
THREDDS Java API
THREDDSCatalog Generator
THREDDS JAVA API
Data Sets (URL wrapped w/metadata)
Inventory Catalogs
THREDDS IDD Catalog
15
THREDDS MetadataThreddsMetadata.get methods

• getCreators()
• getContributors()
• getDates()
• getDocumentation()
• getKeywords()
• getMetadata()
• getProjects()
• getProperties()
• getPublishers()
• getVariables()
• getGeospatialCoverage()

• getTimeCoverage( )
• getServiceName()
• getDataType()
• getDataFormatType()
• getAuthority()
• getDocumentation(String type)
• getHistory()
• getProcessing()
• getRights()
• getSummary()
• getDataSize()

16
THREDDS in LEAD Prototype 1-SC
17
LEAD Prototype 1SC Run the Workflow via the
Portal
Users Machine
GWSTB Machine(s)
Browser
Portal
Web Server
Workflow
Launch Workflow
Click
RAW Data (GRIB)
Decoded Data
Decoders
THREDDS Metadata For the Decoded Data
Real-Time ETA Model Data
?
DBMS
LEAD Metadata For the Decoded Data
myLEAD
18
LEAD Prototype 1SC View Result
Users Machine
GWSTB Machine
Browser
Portal
Web Server
WebStart
Launch IDV
Launch
Click
IDV
Decoded Data
THREDDS Metadata For the Decoded Data
19
LEAD Prototype 1SC Search myLEAD
GWSTB Machine
Users Machine
Browser
Portal
Web Server
Search myLEAD
WebStart
Launch/Search myLEAD
DBMS
Launch
myLEAD
Click
IDV
Decoded Data
THREDDS Metadata For the Found Data
LEAD Metadata For the Found Data
?
Metadata Translation Service
20
THREDDS Links

• Home Page
• http//www.unidata.ucar.edu/projects/THREDDS/
• Catalog Specification (version 1.0)
• http//www.unidata.ucar.edu/projects/THREDDS/tech/
  catalog/InvCatalogSpec.html
• Java API
• http//www.unidata.ucar.edu/projects/THREDDS/tech/
  javadoc/v2.0/catalog/
• Other THREDDS Users
• http//my.unidata.ucar.edu/content/projects/THREDD
  S/Overview/Collaborations.html