Title: Spatial and non spatial information integration using I2T and web services
1Spatial and non spatial information integration
using I2T and web services
- Presented by
- Ashraf Memon and Kirsten Barber
2Collaborators
- Ashraf Memon (San Diego Supercomputer Center)
- Kirsten Barber (SDSU/NPACI EdCenter)
- Ilya Zaslavsky (San Diego Supercomputer Center)
- Chaitan Baru (San Diego Supercomputer Center)
- Peter Joftis (Inter-University Consortium on
Political and Social Research) - Joshua Tran (San Diego Supercomputer Center)
- Erdem Kurul (San Diego Supercomputer Center)
3Web Services
4Web Service Continued
- SOAP
- Simple Object Access Protocol
- Allows the access of Simple Object over the Web.
- SOAP XML HTTP standards
- WSDL
- Web Services Description Language
- WSDL describes what a web service can do, where
it resides, and how to invoke it. - UDDI
- Universal Description, Discovery and Integration
- UDDI is a registry for connecting producers and
consumers of web services.
5Information Integration Testbed (I2T) Chaitan
Baru
A Mediator Architecture
USER-Query
XML Q/A
INTEGRATED VIEW
MIX MEDIATOR
XML Integrated View Definition
XML Q/A
XML Q/A
Wrapper
Wrapper
Wrapper
Files
Data Sources
6I2T Demo 1
- Integration of non-spatial (SDRTA) and spatial
data information (schools districts and
municipalities) - Web service based model
- Heterogeneous sources
- Mediator (Enosys Xmediator) based approach
7I2T Demo 1
Clients AxioMap, Polexis
Find school districts in San Diego where computer
ownership rates among residents are over 80
Java Servlet
XML Mediator (Enosys)
Spatial Mediator
XML query (XCQL)
XML
WSDL
WSDL
Web Server SOAP
Web Server SOAP
Sociology Workbench SWB
Boundaries of municipalities and school districts
San Diego Digital Divide Survey
Java Servlets
Java Servlets
Oracle DBMS
Oracle DBMS
8SWB Architecture XML related
Client
Function Query
XML Survey Document
XML Output
XML Ouput with Table Format Stylesheet
SWB Interface
Table Querying SQL statements (via JDBC)
XML Survey Document
Table output in XML format
XML Parser
Oracle
XML Tree
SQL statements for table creation and the
insertion of all question information (via JDBC)
XML Processor
9- Developed by the Inter-University Consortium for
Political and Social Research at the University
of Michigan. - Challenges working with the ddi
- All tags in the Variable Description portion of
the DTD are optional. - Flexibility of the ddi DTD allows for extensive
personal judgement in tag usage. - Our Solutions
- Place restrictions on tags by making them
mandatory for our application. For a listing of
SWB mandatory tags and their usage
http//edcenterdev.sdsu.edu/SOURCE/ddi.html - Use XSL Stylesheets to preprocess XML documents
to adhere to our restrictions
10Example of a ddi compliant XML document
lt?xml version"1.0" ?gt - ltcodebookgt-
ltdocDscrgt ltguidegt Sample Data From
SWB lt/guidegt lt/docDscrgt-
ltstdyDscrgt- ltstdyInfogt
ltabstractgtExample Survey lt/abstractgt lt/stdyInfogt
lt/stdyDscrgt- ltdataDscrgt-
ltvar name"SPANKING" format"String" ID"q1"gt-
ltvalrnggt ltrange min"0" max"9" /gt
lt/valrnggt lttxtgtFavor Spanking to
Discipline Childlt/txtgt -
ltcatgrygt ltcatValugt1lt/catValugt lttxtgtAgreelt/txtgt
lt/catgrygt- ltcatgrygt
ltcatValugt2lt/catValugt lttxtgtNeutrallt/txtgt
lt/catgrygt- ltcatgrygt
ltcatValugt3lt/catValugt lttxtgtDisagreelt/txtgt
lt/catgrygt- lt/var lt/dataDscrgt
lt/codebookgt
11XML Output from Oracle
- API - oracle.xml.sql.query.OracleXMLQuery
- In order to produce XML output from Oracle the
SWB team had to create SQL functions to organize
the tables in a way in which we can easily
convert from Oracle tables to an XML document.
This document can either be displayed to the
screen or stylized to create a table. - Output Tags- The tags are determined by the SQL
query, and vary with each type of table.
12Converting SWB to Web Service
- Wrote SOAP wrappers for each SWB function that we
wanted to be a service - Wrapper calls the servlet and parses the
resulting XML. Returns the root node of the XML
document
lt?xml version"1.0" ?gt - ltROWSETgt - ltROW
num"1"gt ltDESCRIPgtMissing Datalt/DESCRIPgt
ltAMOUNTgt979lt/AMOUNTgt ltAMOUNT_PERCENTgt33.71lt/AM
OUNT_PERCENTgt lt/ROWgt lt/ROWSETgt
13Oracle Spatial
- 4 Spatial Services
- Discover the spatial dataset for the
municipalities and schools layers in Oracle - Select the geometries for the selected
municipalities - Intersect these geometries with all the school
district geometries - Convert the intersection results into XML
- Demo
14I2T Demo 2
- Environmental Pollution Agency Web Spatial Data
- Web service based model
- Heterogeneous sources
- Procedural Composite Web Services (using Blue
Titan Studio) Approach
15I2T Demo 2
Web spatial source, EPA data ArcObjects spatial
service
Spatial Mediator
Java Servlet
XML
WSDL
Web Server SOAP
ESRI ArcObjects
XML Wrapper
EPA Envirofacts Website
Coordinate Conversion Service
Local Pollution Data
16I2T Demo 2 continued
- Wraps the EPA web site
- Extract Description of facilities for treatment
of pollutant - Extracts Latitude Longitude
- Co-ordinate Conversion
- Latitude-Longitude (Geographical Coordinate
System) to UTM (Projected Coordinate System) - ESRI ArcObjects based conversion web service
- Demo
17I2T Demo 3
GIS source, WSDL for spatial analysis,
survey data analysis, DBMS query UCR/FBI
data Process flow across Web services
Counties crossed by an interstate
Counties with decrease in homicide rates over
, 1993-99
Counties with decrease in victims of firearms
over , 1993-99
18Datasets
- Datasets provided by the Inter-University
Consortium on Political and Social Research at
the University of Michigan (ICPSR) - 2 datasets, aggregate and non-aggregate
- Aggregate provides one response per county on the
number of incidents, population, and count of
firearm homicides - Non-Aggregate provides on response per incident.
Base more closely on the SHR and details number
of offenders, number of victims, weapon type,
relationship of victims, etc. - Compiled by ICPSR using the following datasets
- Uniform Crime Reports
- Offenses Known and Clearances by Arrest, 1975
1997 (ICPSR 9028) - Offenses Known and Clearances by Arrest, 1999
(ICPSR 3158) - Supplementary Homicide Reports, 1979 1999
(ICPSR 2876) - Law Enforcement Agency Identifiers Crosswalk,
1996 (ICPSR 2876)
19Spatial Services
- Spatial Service
- ESRI ArcIMS/ArcObjects Intersection between
Counties and Interstates - ArcIMS
- Pass the selected counties
- Perform intersection between this selections and
interstates layer - ArcObject
- Read shape files for Counties and Interstate
- Perform intersection of selected geometries
- Return the result in XML
- Wrapped as a web services using Microsoft Soap
3.0 - Procedural Composite Web Services (using Blue
Titan Studio) Approach - Demo
20Acknowledgement
- University of California, Life Sciences
Information Technology - ESRI
- The EdCenter on Computational Science and
Engineering - ICPSR
- California Institute for Telecommunications and
Information Technology (Cal-(IT)2) - Blue Titan
- Enosys Markets