Semantic Sensor Web

1
(No Transcript)
2
Semantic Sensor Web Talk at Open Geospatial
Consortium (OGC) Sensor Web Enablement (SWE)
WG St. Louis, MO, March 26, 2008 Amit Sheth
Kno.e.sis Center Wright State University Semanti
c Sensor Web team Cory Henson, Prateek Jain,
Josh Pschorr, Satya Sahoo
3
Presentation Outline

• Motivating scenario (SAVig)
• Utility of metadata in the sensors domain
• Semantic Sensor Web
• Prototyping the Semantic Sensor Web

4
Motivating Scenario
High-level Sensor
Low-level Sensor

• How do we determine if the three images depict
  
• the same time and same place?
• same entity?
• a serious threat?

4
5
Sensor Aided Vigilance (SAVig)
Stand off Staring
Sensor Aided Vigilance

• GWOT requires ability to operate seamlessly
  across layers to sense and track asymmetric
  threats.
• Puts increased demands on novel concepts for
  establishing and exploiting netted persistence
  and empirical phenomenal data.
• Key role for revolutionary taggant materials and
  advanced data management, all within an
  integrated solutions framework

Close in Staring
(Cupid Fire) ATR-Driven Small UAV on Steroids
Surface Near-Surface Staring (SUAV, Bldg
Sensors, Taggants, UGS)
6
The Challenge
Collection and analysis of information from
heterogeneous multi-layer sensor nodes
7
Why is this a Challenge?

• There is a lack of uniform operations and
  standard representation for sensor data.
• There exists no means for resource reallocation
  and resource sharing.
• Deployment and usage of resources is usually
  tightly coupled with the specific location,
  application, and devices employed.
• Resulting in a lack of interoperability.

8
Interoperability

• INTEROPERABILITY
• The ability of two or more autonomous,
  heterogeneous, distributed digital entities to
  communicate and cooperate among themselves
  despite differences in language, context, format
  or content.
• These entities should be able to interact with
  one another in meaningful ways without special
  effort by the user - the data producer or
  consumer - be it human or machine.

9
Survey
Many diverse sensor data management application
frameworks were compared, such as

• GSN
• Global Sensor Network
• Digital Enterprise Research Institute (DERI)
• http//gsn.sourceforge.net/
• Hourglass
• An Infrastructure for Connecting Sensor Networks
  and Applications
• Harvard
• http//www.eecs.harvard.edu/syrah/hourglass/
• IrisNet
• Internet-Scale Resource-Intensive Sensor Network
  Service
• Intel Carnegie Mellon University
• http//www.intel-iris.net/

However, it soon became obvious that these
application frameworks provided only localized
interoperability and that a standards-based
framework was necessary.
10
OGC Sensor Web Enablement
Sensor and Processing Description Language
Information Model for Observations and Sensing
Observations Measurements (OM)
SensorML (SML)
SWE Commons (SWE)
SWE Common Data Structure And Encodings
Multiplexed, Real Time Streaming Protocol
Sam Bacharach, GML by OGC to AIXM 5 UGM, OGC,
Feb. 27, 2007.
11
Presentation Outline

• Motivating scenario (SAVig)
• Utility of metadata in the sensors domain
• Semantic Sensor Web
• Prototyping the Semantic Sensor Web

12
Data Pyramid
13
Data Pyramid
Sensor Data Pyramid
Knowledge
Ontology Metadata
Expressiveness
Entity Metadata
Information
Feature Metadata
Raw Sensor (Phenomenological) Data
Data
14
Sensor Data Pyramid

• Avalanche of data
• Streaming data
• Multi-modal/level data fusion
• Lack of interoperability

Ontology Metadata
Entity Metadata
Feature Metadata
Raw Sensor Data
(e.g., binary images, streaming video, etc.)
15
Sensor Data Pyramid

• Extract features from data
• Annotate data with features
• Store and query feature metadata

Ontology Metadata
Entity Metadata
Feature Metadata
Raw Sensor Data
(e.g., lines, color, texture, etc.)
16
Sensor Data Pyramid

• Detect objects-events from features
• Annotate data with objects-events
• Store and query objects-events

Ontology Metadata
Entity Metadata
Feature Metadata
Raw Sensor Data
(e.g., objects and events such as cars driving)
17
Sensor Data Pyramid

• Discover and reason over associations
• objects and events
• space and time
• data provenance

Ontology Metadata
Entity Metadata
Feature Metadata
Raw Sensor Data
(e.g., situations such as cars speeding
dangerously)
18
Presentation Outline

• Motivating scenario (SAVig)
• Utility of metadata in the sensors domain
• Semantic Sensor Web
• Prototyping the Semantic Sensor Web

19
Semantic Sensor Web

• What is the Semantic Sensor Web?
• Adding semantic annotations to existing standard
  Sensor Web languages in order to provide semantic
  descriptions and enhanced access to sensor data
• This is accomplished with model-references to
  ontology concepts that provide more expressive
  concept descriptions
• For example,
• using model-references to link OM annotated
  sensor data with concepts within an OWL-Time
  ontology allows one to provide temporal semantics
  of sensor data
• using a model reference to annotate sensor device
  ontology enables uniform/interoperable
  characterization/descriptions of sensor
  parameters regardless of different manufactures
  of the same type of sensor and their respective
  proprietary data representations/formats

20
Semantic Annotation (model reference)

• RDFa
• Used for semantically annotating XML documents. 
  
• Several  important attributes within RDFa
  include
• about describes subject of the RDF triple
• rel describes the predicate of the RDF triple
• resource describes the object of the RDF triple
• instanceof describes the object of the RDF
  triple with the predicate as rdftype
• Other used Model Reference in Semantic
  Annotations
• SAWSDL Defines mechanisms to add semantic
  annotations to WSDL and XML-Schema components
  (W3C Recommendation)
• SA-REST Defines mechanisms to add semantic
  annotations to REST-based Web services.

W3C, RDFa, http//www.w3.org/TR/rdfa-syntax/
21
Semantically Annotated OM
ltswecomponent name"time"gt ltsweTime
definition"urnogcdefphenomenontime"
uom"urnogcdefunitdate-time"gt ltsaswe
rdfaabout"?time" rdfainstanceof"timeInstant"gt
ltsasml rdfaproperty"xsdate-time"/gt lt/sa
swegt lt/sweTimegt lt/swecomponentgt ltswecomponent
name"measured_air_temperature"gt ltsweQuantity
definition"urnogcdefphenomenontemperature
uom"urnogcdefunitfahrenheit"gt
ltsaswe rdfaabout"?measured_air_temperature
rdfainstanceofsensoTemperature
Observation"gt ltsaswe rdfaproperty"weatherfa
hrenheit"/gt ltsaswe rdfarel"sensooccurred_wh
en" resource"?time"/gt ltsaswe
rdfarel"sensoobserved_by" resource"sensobucke
ye_sensor"/gt lt/sasmlgt lt/sweQuantitygt lt/sw
ecomponentgt ltswevalue nameweather-data"gt 200
8-03-08T050000,29.1 lt/swevaluegt
?time rdftype timeInstant ?time xsdate-time
"2008-03-08T050000"
?measured_air_temperature rdftype
sensoTemperatureObservation ?measured_air_tempera
ture weatherfahrenheit "29.1" ?measured_air_tem
perature sensooccurred_when ?time ?measured_air_
temperature sensoobserved_by
sensobuckeye_sensor
22
Semantic Query

• Semantic Temporal Query
• Model-references from SML to OWL-Time ontology
  concepts provides the ability to perform semantic
  temporal queries
• Supported semantic query operators include
• contains user-specified interval falls wholly
  within a sensor reading interval (also called
  inside)
• within sensor reading interval falls wholly
  within the user-specified interval (inverse of
  contains or inside)
• overlaps user-specified interval overlaps the
  sensor reading interval
• Example SPARQL query defining the temporal
  operator within

23
Sensor Data Architecture

• Knowledge
• Object-Event Relations
• Spatiotemporal Associations
• Provenance Pathways

Data Storage (Raw Data, XML, RDF)
Semantic Analysis and Query

• Information
• Entity Metadata
• Feature Metadata

Feature Extraction and Entity Detection
Semantic Annotation

• Data
• Raw Phenomenological Data

Sensor Data Collection
Ontologies

• Space Ontology
• Time Ontology
• Situation Theory Ontology
• Domain Ontology

23
24
Presentation Outline

• Motivating scenario (SAVig)
• Utility of metadata in the sensors domain
• Semantic Sensor Web
• Prototyping the Semantic Sensor Web

25
Prototyping the Semantic Sensor Web

• Application 1 Temporal Semantics for Video
  Sensor Data
• Semantically annotated police cruiser videos
  collected from YouTube with model references to
  an OWL-Time ontology
• Enables time-interval based queries, such as
  contains, within, overlaps

26
Temporal Semantics for Video Sensor Data
Data Collection
Data Source (e.g., YouTube)
Extraction Metadata Creation
Storage
Query
UI
Video Conversion
AVI
SML (XML-DB)
SML Interface
Google Maps
Filtering OCR
Ontology (OWL/RDF-DB)
Ontology Interface
GWT (Java to Ajax)
Time Date information
SML Annotation Generation
OWL-Time Annotation Generation
26
27
Temporal Semantics for Video Sensor Data
Channel Minimal Suppression 1 8-neighbor median
for bad pixels 1 Temporal Minimal Suppression 2
Binarization via adaptive threshold 1
Tesseract OCR engine
Regular Expression parsing SensorML output

• https//research.microsoft.com/xshua/publications
  /pdf/2002_ISCAS_TimeStampOCR.pdf
• http//www.informedia.cs.cmu.edu/documents/vocr_ie
  ee98.pdf

27
28
Temporal Semantics for Video Sensor Data
28
Demo http//knoesis.wright.edu/library/demos/ssw/
prototype.htm
29
Prototyping the Semantic Sensor Web

• Application 2 Semantic Sensor Observation
  Service
• Semantically annotated weather data collected
  from BuckeyeTraffic.org with model references to
  an OWL-Time ontology, geospatial ontology, and
  weather ontology
• Calpable of multi-level weather queries and
  inferences on a network of multi-modal sensors

29
30
SOS-S Architecture
S-SOS Client
BuckeyeTraffic.org
Collect Sensor Data
HTTP-GET Request
OM-S or SML-S Response
Semantic Sensor Observation Service
Oracle SensorDB
Get Observation
Describe Sensor
Get Capabilities

• Ontology Rules
• Weather
• Time
• Space

SWE
Annotated SWE
SA-SML Annotation Service
31
SOS-S Data Collection
BuckeyeTraffic, http//www.buckeyetraffic.org/
32
S-SOS Ontology Concepts
Sensor
Time
observed_by
occurred_when
occurred_where
Observation
Location
described
measured
Weather_Condition
Phenomena

• Key
• Sensor Ontology
• Weather Ontology
• Temporal Ontology
• Geospatial Ontology

subClass
subClass
Temperature
Precipitation

33
S-SOS Ontology Concepts
Weather_Condition
subClass
Wet
Instances of simple weather conditions created
directly from BuckeyeTraffic data
Icy
Clear
Instances of complex weather conditions inferred
through rules
Freezing
Potentially Icy
34
S-SOS Rules for Weather Conditions

• Rules allow inferred knowledge from the sensor
  data
• For example Based on temperature, wind speed,
  precipitation, etc., we can infer the potential
  road condition eventually the type of storm
  being observed

Example Potential_Ice_with_Rain_and_Celcius_Temp
Observation(?obs) measured(?obs, ?precip)
Rain(?precip) measured(?obs, ?temp)
Temperature(?temp) temperature_value(?temp,
?tval) lessThanOrEqual(?tval, 0)
unit_of_measurement(?temp, celcius") ?
described(?obs, Potential_Ice)?

• Clear
• Potential Ice
• Severe Hazard
• etc.

35
SOS-S Client
HTTP-GET Request http//knoesis1.wright.edu/weath
er/weather ?serviceSOS version1.0 requestGetO
bservation offeringWEATHER_DATA formatapplicat
ion/com-xml time2008-03-08T050000Z/2008-03-08T
060000Z interval_typewithin weather_conditio
npotentially_icy
OM-S Response ltsweTime definition"urnogcdef
phenomenontime" uom"urnogcdefunitdate-time"
gt ltsaswe rdfaabout"?timerdfainstanceof"ti
meInstant"gt ltsasml rdfaproperty"xsdate-time
"/gt lt/saswegt lt/sweTimegt ltswevalue
nameweather-data"gt 2008-03-08T050000,29.1 lt/s
wevaluegt
Semantic Sensor Observation Service
Get Observation
Describe Sensor
Get Capabilities
Demo http//knoesis1.wright.edu/weather/SSW.html
36
Conclusion

• Future Work
• Incorporation of spatial ontology in order to
  include spatial analytics and query (perhaps with
  OGC GML Ontology or ontology developed by W3C
  Geospatial Incubator Group - GeoXG)
• Extension of SPARQL with enhanced spatiotemporal
  query and analytics (including semantic
  assoications)
• Integration of framework with emergent
  applications, including video on mobile devices
  running Android OS

Kno.e.sis/Wright State Univ. is a member of W3C
and its research led to the development of SAWSDL
37
References

• Cory Henson, Amit Sheth, Prateek Jain, Josh
  Pschorr, Terry Rapoch, Video on the Semantic
  Sensor Web, W3C Video on the Web Workshop,
  December 12-13, 2007, San Jose, CA, and Brussels,
  Belgium
• Matthew Perry, Amit Sheth, Farshad Hakimpour,
  Prateek Jain. Supporting Complex Thematic,
  Spatial and Temporal Queries over Semantic Web
  Data, Second International Conference on
  Geospatial Semantics (GEOS 07), Mexico City, MX,
  November 29-30, 2007
• Matthew Perry, Farshad Hakimpour, Amit Sheth.
  Analyzing Theme, Space and Time An
  Ontology-based Approach, Fourteenth
  International Symposium on Advances in Geographic
  Information Systems (ACM-GIS 06), Arlington, VA,
  November 10-11, 2006
• Farshad Hakimpour, Boanerges Aleman-Meza, Matthew
  Perry, Amit Sheth. Data Processing in Space,
  Time, and Semantic Dimensions, Terra Cognita
  2006 Directions to Geospatial Semantic Web, in
  conjunction with the Fifth International Semantic
  Web Conference (ISWC 06), Athens, GA, November
  6, 2006
• Amit Sheth et al., SA-Rest Semantically
  Interoperable and Easier-to-Use Services and
  Mashups, IEEE Internet Computing,
  November/December 2007 (Vol.11, No.6) pp.91-94.
  DOI http//doi.ieeecomputersociety.org/10.1109/MI
  C.2007.133
• Open Geospatial Consortium, Sensor Web Enablement
  WG, http//www.opengeospatial.org/projects/groups/
  sensorweb
• W3C, Time Ontology in OWL, http//www.w3.org/TR/ow
  l-time/
• W3C, Geospatial Incubator Group,
  http//www.w3.org/2005/Incubator/geo/
• W3C, Semantic Annotations for WSDL and XML
  Schema, http//www.w3.org/TR/sawsdl/
• W3C, RDFa, http//www.w3.org/TR/rdfa-syntax/
• Google Code, Tesseract, http//code.google.com/p/t
  esseract-ocr/

37