Middleware for Context-aware Ubiquitous Computing (In preparation for Distributed System Course)

1
Middleware for Context-aware Ubiquitous
Computing(In preparation for Distributed System
Course)

• Hung Q. Ngo
• Ubiquitous Computing Group (UCG)
• Real-time and Multimedia LAB
• Kyung Hee University, Korea
• Sept., 2004

2
Sequence of presentation

• Introduction
• Middleware for Context-Awareness
• CAMUS Architecture
• Feature Extraction Agent
• CAMUS Context Model
• Reasoning Mechanisms
• System Prototyping
• Conclusion and Future Work

3
Introduction

• Ubiquitous/Pervasive Computing
• Calm technology embedded, invisible, seamlessly,
  unobtrusive, intelligent.

Image source Friedemann Mattern (ETH Zürich)
4
Introduction (cont.)

• Context-awareness
• An important aspect of the intelligent pervasive
  computing systems
• Systems that can anticipate users needs and act
  in advance by understanding their context.

5
Introduction (cont.)

• Context
• The specific conditions, external to the
  application itself, such as audience, speaker
  (user), situation (place and its surroundings),
  time, environmental and network conditions, etc.,
  which determine the application behavior, will be
  called the context of the application.
• Out of Context Computer Systems That Adapt to,
  and Learn from, Context, H. Lieberman, T.
  Selker, MIT
• A Survey of Context-Aware Mobile Computing
  Research, by G. Chen, D. Kotz, Dartmouth College
• Context-Aware Applications Survey, M.
  Korkea-aho, Helsinki University of Technology

6
Introduction (cont.)

• Context in our viewpoint
• the situational conditions that are associated
  with a user location, surrounding conditions
  (light, temperature, humidity, noise level, etc),
  social activities, user intentions, personal
  information, etc.

7
Introduction (cont.)

• Sensing is a key enabling technology
• Heterogeneity
• e.g. location-sensing techniques triangulation,
  proximity, scene analysis
• (Jeffrey Hightower, Location Systems for
  Ubiquitous Computing, IEEE Computer August
  2001.)

In the environment
Wearable
On devices
8
Introduction (cont.)

• Shortcomings of the previous systems
• Context acquisition and use was often tightly
  integrated into a single application, and could
  not easily be incorporated into other
  applications.
• Individual agents are responsible for managing
  their own context knowledge

9
Introduction (cont.)

• Shortcomings of the previous systems
• Lacking an adequate representation for context
  modeling and reasoning
• Existing solutions for Context information are
• Name-value pairs or entity relation model
• Objects to represent context with methods and
  fields for retrieval of information
• Simple matching mechanisms for context access,
  and developers must perform low-level programming
  for reasoning
• Users often have no control over the information
  that is acquired by the sensors
• Privacy Concerns

10
Middleware for Context-Awareness

• Desired Characteristics
• Support for heterogeneous and distributed sensing
  agents
• Make it easy to incrementally deploy new sensors
  and context-aware services in the environment
• Provide different kinds of context classification
  mechanisms
• Different mechanisms have different power,
  expressiveness and decidability properties
• Rules written in different types of logic (first
  order logic, description logic, temporal/spatial
  logic, fuzzy logic, etc.)
• Machine-learning mechanisms (supervised/unsupervis
  ed classifiers)

11
Middleware for Context-Awareness

• Desired Characteristics (cont.)
• Follow a formal context model using ontology
• To enable syntactic and semantic
  interoperability, and knowledge sharing between
  different domains
• Facilitate for applications to specify different
  behaviors in different contexts easily, as well
  as privacy policy and security mechanism
• Graphical development tool to ease developers in
  writing code.

12
CAMUS Core Architecture
CAMUS ---------------------- Context- Aware Middle
ware for Ubicomp Systems
13
CAMUS Details

• Feature Extraction Agents
• These sensing agents extract the most descriptive
  features for deducing contexts in upper layers
• Feature - Context Mapping performs the mapping
  required to convert a given feature into
  elementary context based on the meta-information
  saved in the ontology repository
• Self-configuring components
• Access wrappers
• Unification interface for acquiring contexts from
  sensors and delivering to consumers.

14
Feature Extraction Agent (cont.)
Table 1. Meta-data for feature encapsulation
Attribute Meaning
Sensor_ID The unique ID of the sensor board, assigned at development stage mapped to corresponding location/devices at deployment stage
Type_ID Sensor type e.g. audio, temperature etc., for further distinction of context information sources, especially in case multiple sensor types on a single board.
FeatureID Refers to feature category of each sensor type. It is actually the signature of corresponding feature extraction function in the library of the sensing device.
Feature Value Symbolic (e.g. light intensity) or absolute numerical value (e.g. temperature absolute value) of the extracted and segmented sensor feature.
Probability The uncertainty or confidence of the context information, could be 0/1 if the feature is segmented using thresholds, or within 0, 1 range if fuzzy sets are applied. This attribute is used to decide which feature values will be sent to backend system (probability gt 0) and useful for probability based context recognition mechanisms e.g. Bayesian networks.
Timestamp The absolute time when the feature is extracted. Used to keep consistency of context information, and sometimes useful for temporal reasoning mechanisms.
15
Feature Extraction Agent (cont.)
Table 2. Examples of features and their semantic
meaning. Sensor_IDs 3,7 are mapped to Bedroom
Sensor ID Sensor Type Feature ID Value Value Time Stamp
Sensor ID Sensor Type Feature ID Numerical Value Segmented Value (Symbolic, Probability) Time Stamp
3 1 (Audio) 1 (Intensity) x (dB) 1 (Silent, 0.9) xxxxxx
3 1 (Audio) 1 (Intensity) x (dB) 2 (Moderate, 0.1) xxxxxx
3 1 (Audio) 1 (Intensity) x (dB) 3 (Loud, 0) xxxxxx
3 1 (Audio) 2 (ACDCRatio) y 1 (Low, 0) xxxxxx
3 1 (Audio) 2 (ACDCRatio) y 2 (Medium, 0.7) xxxxxx
3 1 (Audio) 2 (ACDCRatio) y 3 (High, 0.3) xxxxxx
7 4 (Temperature) 1 (AbsValue) 95 (oF) NA xxxxxx
7 5 (Humidity) 1 (Humidity) z () 1 (Dry, 1) xxxxxx
7 5 (Humidity) 1 (Humidity) z () 2 (Normal, 0) xxxxxx
7 5 (Humidity) 1 (Humidity) z () 3 (Humid, 0) xxxxxx
FT Sesnor_ID, Type_ID, Feature_ID,
Feature_Value, Probability, Timestamp
FT1 3, 1, 1, 1, 0.9, xxxxx Location.Bedroo
m, Environment.Sound.Intensity Silent,
Probability 0.9, TimeStamp xxxxx
16
Formal Context Modeling

• In order to have common understanding among HW/SW
  entities, they must have invariant meanings, i.e.
  the context semantics must be formalized
• Advantages
• Storing context for long-term
• Communicating context universally with other
  systems
• Leads to its testability of being a formalized
  knowledge
• Result
• A growing pool of well-tested context knowledge
  available to different context aware systems

17
Formal Context Modeling (cont.)

• Formal Context Modeling using OWL
• Context entities are concepts in a domain under
  investigation
• Ontologies are used for above purpose, defined as
• Specification of a conceptualization of a domain
• Domain contains the vocabularies of concepts and
  relationships among them
• _at_syntactic level
• They are XML documents
• _at_structure level
• They consist of a set of triples
• _at_semantic level
• They constitute one or more graphs with partially
  pre-defined semantics

18
CAMUS Context Model
Basic Model
19
Context Model in detailsAgent ontology
20
Context Model in details Device ontology

• Based on FIPA device ontology specification
  knowledge reuse

21
Reasoning Mechanisms

• The contextual information provided by the
  environment leads to only elementary contexts
• Some contexts are useful only when they are
  combination of some elementary and/or composite
  contexts, and also need consistency of contextual
  information

22
Reasoning Mechanisms (cont.)

• Our framework supports various pluggable
  reasoning modules and developer of the context
  Aggregator services can exploit any kind of
  reasoning mechanism based on application
  requirements
• These reasoning modules are broadly classified
  into ontology context reasoning mechanisms

23
Reasoning Mechanisms (cont.)

• Inference services in DL (Description Logic)
• Terminological Reasoning
• Subsumption check - checks if one concept is a
  sub-concept of another
• Consistency check - checks for (in)consistency of
  concept definitions
• Taxonomy construction - explicit concept
  hierarchy
• Classification - determines the concepts that
  immediate subsume or are subsumed by a given
  concept

24
Reasoning Mechanisms (cont.)

• Inference services in DL (Description Logic)
• Instance Reasoning
• Consistency check - existence of a model of ?
• Realisation - given a partial description of an
  instance, finds the most specific concepts that
  describe it
• Individual retrieval - finds all instances that
  are described by a given concept

25
Examples for Ontology Reasoning
26
Examples for Ontology Reasoning
27
Reasoning Mechanisms (cont.)
An example of Context Reasoning Using
Bayesian Net to deduce User Activity
28
System Prototyping

• A Smart Home has been set up for system
  prototyping.
• RFID tags, Door Sensors, Bluetooth and WLAN
  access points are used for location and identity
  detection
• Audio/Video based activity recognition

29
System Prototyping

• A Smart Home has been set up for system
  prototyping.
• Jini is used as underlay communication and
  discovery mechanisms
• Jena2 Semantic Web Toolkit is used to implement
  the Context Repository, Context Reasoner, and
  Context Query Engine.
• OSGi Gateway for interconnection with other
  systems

30
Our Research Contributions

• A Middleware Framework for building Context-Aware
  applications
• Unified Sensing Framework for Context acquisition
  from disparate sensors
• Abstractions to relieve the developers of the
  burden of low level interaction with various
  hardware devices
• Formal Context Modeling in terms of ontologies
  using OWL
• To enable syntactic and semantic
  interoperability, and knowledge sharing between
  different domains
• Ontology and Context Reasoning Mechanisms
• Pluggable Modules

31
Conclusion and Future work

• We believe that formalizing domains should be
  seen as emergent phenomenon constructed
  incrementally, leading to the sharing of
  contextual information among heterogeneous
  context-aware systems
• API for application developers to access systems
  functionalities while hiding the complexity of
  underlying context processing
• (gathering contexts from sources, managing
  contexts in knowledge base, handling application
  queries, and reasoning about contexts based on
  rules)
• Privacy policy and security mechanism

32
Selected References

• 1 M. Weiser, The Computer for the 21st
  Century, Scientific America (Sept. 1991) 94-104
  reprinted in IEEE Pervasive Computing. (Jan.-Mar.
  2002) 19-25
• 2 M. Satyanarayanan, Pervasive Computing
  Vision and Challenges, IEEE Personal
  Communications (Aug. 2001) 10-17
• 3 Dey, A.K., et al., A Conceptual Framework
  and a Toolkit for Supporting the Rapid
  Prototyping of Context-Aware Applications,
  Anchor article of a special issue on
  Context-Aware Computing, Human-Computer
  Interaction (HCI) Journal, Vol. 16. (2001)
• 4 S. Jang, W. Woo, Ubi-UCAM A Unified
  Context-Aware Application Model, Context 2003,
  Stanford, CA, USA. (Jun. 2003)
• 5 J. Hong, The Context Fabric,
  http//guir.berkeley.edu/projects/confab/
• 6 Kumar, M. Shirazi, B.A. Das, S.K. Sung,
  B.Y. Levine, D. Singhal, M, PICO a middleware
  framework for pervasive computing, IEEE
  Pervasive Computing, Vol. 2 Issue 3. (July
  Sept, 2003) 72- 79
• 7 Chen Harry, Tim Finin, and Anupam Joshi, An
  Intelligent Broker for Context-Aware Systems,
  Ubicomp 2003, Seattle, Washington. (Oct. 2003)
• 8 Hung Q. Ngo, Anjum Shehzad, S.Y.Lee,
  Developing Context-Aware Ubiquitous Computing
  Systems with a Unified Middleware Framework,
  accepted for publication, The 2004 International
  Conference on Embedded and Ubiquitous Computing
  (EUC04), Aizu-Wakamatsu City, Japan, 25-27
  August, 2004.
• 9 Anjum Shehzad, Hung Q. Ngo, S.Y. Lee, Formal
  Modeling in Context Aware Systems, KI-Workshop
  Modeling and Retrieval of Context (MRC2004),
  German.

33

• For further discussion
• Hung Q. Ngo
• Ubiquitous Computing Group
• RTMM LAB
• Kyung Hee Univ. KOREA
• Email sylee_at_oslab.khu.ac.kr
• www http//ucg.khu.ac.kr/nqhung