Installing the THREDDS and Aggregation Servers

1
Installing the THREDDS and Aggregation Servers

• Install and verify the Tomcat servlet engine
• Install and verify the THREDDS servlet (which
  also contains the Aggregation Server)

2
Install Tomcat

• Expand/unpack Jakarta/Tomcat
• gt cd tutorial
• gt tar -xzf /scr/OPeNDAP_Software/Servers/
• Binary/jakarta-tomcat-5.0.28.tar.gz
  
• This will add the jakarta-tomcat-5.0.28 directory.

3
Install THREDDS

• Put the THREDDS server 'war' file into Tomcat
• gt cp /scr/OPeNDAP_Software/Servers/Binary
• /thredds.war jakarta-tomcat-5.0.28/webapps
• Note that the jakarta-tomcat-5.0.28/content is
  empty
• gt ls jakarta-tomcat-5.0.28/content

4
Start Tomcat

• Start Tomcat
• gt jakarta-tomcat-5.0.28/bin/startup.sh
• Starting Tomcat automatically configures it and
  any servlets it contains
• Verify the THREDDS server installation
• gt ls jakarta-tomcat-5.0.28/content
• You should see that a 'thredds' subdirectory and
  'thredds.INIT' file have been created.

5
Verify the Installation
6
Verify the THREDDS Server
7
Why Tomcat?

• There are two dominant server types
• Those based on C/C and
• those based on Java.
• Apache is the most commonly used C/C based
  server.
• Tomcat is the most commonly used Java based
  server.
• If the server that you want to use is Java based,
  then Tomcat is a logical choice.
• The Aggregation Server is a Java based server.

8
Special Servers and Services

• THREDDS catalogs
• The Aggregation server

9
THREDDS Catalogs

• THREDDS is an alternative to the File Server
• Like the File Server, THREDDS catalogs
• Provide a way to group data granules
  independently from their physical location in a
  file system
• May span several computers
• In addition, THREDDS catalogs
• May have several associated access mechanisms
  (e.g., OPeNDAP, FTP, ...)
• Provide a hierarchical catalog directory to
  inventory
• Are stored as XML files

10
Validate the Catalog

• Get a catalog for the pathfinder data
• Go to http//www.opendap.org/support/stennis_tutor
  ial/ in your web browser
• Down load and save the file pathfinder.xml to
  /tutorial/pathfinder_data/

11
Validate the THREDDS catalog
http//localhost8080/thredds/validateForm.html
http//localhost/path/pathfinder.xml
12
THREDDS Validated Response
13
Use the Catalog HTML Service

• THREDDS provides a service which transforms the
  XML catalog into HTML
• The catalog service of the THREDDS server is at
  http//localhost8080/thredds/catalog.html?catalog
  
• Provide your THREDDS cataloghttp//localhost/path
  /pathfinder.xml
• as an argument to the THREDDS catalog
  service i.e., enter the following in your
  browser

http//localhost8080/thredds/catalog.html?catalog
http//localhost/path/pathfinder.xml
14
(No Transcript)
15
Following a Link
16
The Aggregation Server (AS)

• What the AS can do configurating the AS
• Hands-on

17
Aggregation

• The Aggregation Server (AS) creates virtual data
  sets from
• NetCDF file
• OPeNDAP data sources
• The AS is configured using THREDDS catalogs
• It can aggregate local NetCDF files (and serve
  the files individually)
• It can aggregate local and remote data using
  OPeNDAP servers

18
Types of Aggregation

• Limited to Array and Grid variables
• Three types of aggregation
• JoinNew Combine arrays using a new dimension
• JoinExisting Combine using an existing dimension
• Union For the union of existing variables

19
Type Join New
20
Type Join Exist

• A, B and C are aggregated.
• They each have an x, y and z dimension.
• When the AS gets a request, it knows that
  different values of z map to the different files,
  A B or C
• JoinNew and JoinExist are often used to combine
  arrays of time-series data.

21
Type Union

• The data sets A and B contain some common and
  some unique variables.
• The Union aggregation combines different
  parameters from the data sets into one logical
  data set.
• Often this is used to combine tables of data.

22
Configuration

• The data provider
• Defines an aggregation using a THREDDS catalog,
  and
• Controls the performance of the AS via the
  web.xml file for the servlet.
• For example, setting the cache size to
  accommodate all the files in a JoinNew or
  JoinExist aggregation will improve performance.

23
The Aggregation Server (AS)

• What the AS can do configuring the AS
• Hands-on

24
Configure the AS for the Pathfinder Data

• Download/Edit the AS configuration file
• Verify operation of the AS

25
Install the New Catalog

• Go to http//www.opendap.org/support/stennis_tutor
  ial and save the file 'stennisTutorialAS.xml' to
  the directory
• /tutorial/jakarta-tomcat-5.0.28/content/thredds/d
  odsC
• Change to that directory
• gt cd /tutorial/jakarta-tomcat-5.0.28/content/thre
  dds/dodsC
• and look at the files there
• gt ls
• You should see catalogConfig.xml,
  catalogExample.xml and stennisTutorialAS.xml

26
Install the New Catalog (continued)

• Move the stennisTutorialAS.xml file to
  catalogConfig.xml
• gt mv stennisTutorialAS.xml catalogConfig.xml
• The AS configuration file must be called
  cataloConfig.xml!

27
Restart Tomcat

• Restart Tomcat to reinitialize the AS
• gt cd /tutorial/jakarta-tomcat-5.0.28
• gt bin/shutdown.sh
• gt bin/startup.sh
• After any change to the AS configuration file,
  Tomcat must be restarted.

28
Access the AS
29
The DDS Shows the Added Dimension
30
Exercise

• Compare the DDS from the aggregation with one
  from a single pathfinder data file
• In a second browser window (or tab), type
• http//localhost/opendap/nph-dods/path/
• Select one of the files and access it's DDS

31
Access Data

• Use the AS to get the Data Request Form
• In the form, access values for the 'dsp_band_1'
  variable by clicking the variable's checkbox
• Try a constraint like
• time 011 lat 01001023 lon 01001023

32
Data Returned
33
Letting Others Know about Your Server

• The Global Change Master Directory (GCMD)
• The NVODS data set list
• THREDDS catalogs

34
Why Register Your Server?

• The DAP provides data access services only it
  does not provide data location services.
• The DAP was designed to be used in concert with
  other protocols and systems which will provide
  lookup, location and searching capabilities.

35
Where Can I Register My Server?

• The Global Change Master Directory
• The NVODS Dataset List
• THREDDS catalogs
• Your own HTML pages

36
The GCMD

• The Global Change Master Directory (GCMD) is a
  registry of Earth-Science data sources.
• The GCMD
• Has been in existence for more than a decade.
• Stores very complete metadata for each registered
  data source.
• Has a team of domain specialists who work with
  providers to write the metadata records.
• Provides explicit support for OPeNDAP data sources

37
The GCMD (continued)

• To register a data set with the GCMD, visit their
  home page and follow the Add to the GCMD link.

38
The NVODS Dataset List

• The National Virtual Ocean Data System (NVODS)
  maintains a list of data sets as an XML document.
• Registration is easy
• Go to the OPeNDAP home page
• Click on Data Sources in the menu on the left
  hand side of the page.
• Click on Register your server.
• The NVODS list provides almost no metadata
• It is very complete
• This list is accessible on-line at the web page
  through an HTML interface as well as from the ODC.

39
THREDDS

• You can write THREDDS catalogs
• THREDDS catalogs can be linked together
• Unidata maintains a catalog of catalogs

40
HTML and Google

• You write your own web pages with links to your
  datasets
• Let Google crawl your site.
• A Google search on DODS Salinity Data returned
  about 4,000 hits, the first page listed data from
  the Arabian Sea Experiment, Massachusetts, Maine
  and IFREMER (France).

41
Server Registration Conclusions

• There are a number of ways to register your
  server, although the DAP doesn't provide any
  explicit support for this activity.
• We have largely left the data location problem
  untouched since other groups have devoted 100 of
  their effort to solving that problem.
• Existing registration and search options are
  diverse and powerful.