SensorGrid: A new Cyberinfrastructure Integrating Sensor Network and Grid Computing for e-Science Applications

1
SensorGrid A new Cyberinfrastructure
Integrating Sensor Network and Grid Computing for
e-Science Applications

• Dr. Rajkumar Buyya

Grid Computing and Distributed Systems (GRIDS)
Lab. Dept. of Computer Science and Software
EngineeringThe University of Melbourne,
Australiawww.buyya.com
2
SensorGrid A new Cyberinfrastructure for
Linking the Physical World with the Digital World

• Dr. Rajkumar Buyya

Fellow of Grid
Computing Grid Computing and Distributed Systems
(GRIDS) Lab. Dept. of Computer Science and
Software EngineeringThe University of Melbourne,
Australiagridbus.org/raj/tut/gridbus.zip
3
GRIDS Lab _at_ Melbourne
Education
R D

• The youngest and one of the largest research labs
  in the CSSE Dept
• 2 PostDocs
• 2 Research Programmers
• 7 RHD (6 PhD) students
• 5 honours/masters projects
• Funding
• National and International organizations
• Australian Research Council
• Many industries (Sun, StorageTek, Microsoft, IBM)
• University-wide collaboration
• Faculties of Science, Engineering, and Medicine
• Many national and international collaborations.
• Academics
• Industries
• Software
• Our Grid middleware technologies are widely in
  academic and industrial users.
• Publication
• My research team produces 20 of our Depts
  research output.

Community Services
4
Books at Glance Co-authored/edited
5
Outline

• Introduction
• Technology Trends in Sensors and Grids
• Sensors and Grids Integration Proposal
• Grid Computing
• What is it?, architecture, and technologies
• NOSA (NICTA Open SensorWeb Architecture)
• Summary and Conclusion

6
(No Transcript)
7
(No Transcript)
8
Internet, Web, and Grid Effect
140
Web Services and Grid Effect
120
100
The 'Network Effect kicks in, and the web goes
critical'
Number of hosts (millions)
Business
80
60
40
20
0
1965
1970
1975
1980
1985
1990
1995
2000
2005
2010
TCP/IP
HTML
Mosaic
XML
PHASE
2. The Internet is
Born
3. The World Wide Web
4.
with
XML
5. The Grid
1. Packet Switching Networks
HTML hypertext system created
1969 4 US Universities linked to form ARPANET
TCP/IP becomes core protocol
CERN launch World Wide Web
1972 First e-mail program created
Domain Name System created IETF created (1986)
1976 Robert Metcalfe develops Ethernet


NCSA launch Mosaic interface
9
Grid Resource Broker
SensorWeb Service
R2
R3
R4
R5
RN
R6
R1
Grid Information Service
The customer can have access to the Grid Resource
Broker, and then through Web Service, requests
can be sent to obtain real sensor network data.
Sensor Network
10
Traditional Devices, Sensors, and their Networks
at Glance

• Traditional Devices (Computers and High End
  Resources) are
• Powerful
• Connected to Power Grid so we dont worry too
  much about it power consumption
• Large Storage Space
• Good for archival and large-scale analysis
• Connected by High Bandwidth/Speed Network
• Sensors
• Less powerful
• Scarcity of power (battery operated, or even
  self-power generated)
• Less Storage
• No good for archival
• Connected by Low Bandwidth/Speed Network
• But they can sense/smell a phenomena in the
  physical world.

11
It will be nice to marry them

• Both of them benefit
• Grids
• Get Eye to see the world (so that it can sense
  and assist the us)
• Sensors
• Off load their processing, storage, archival,
  analysis, etc. requirements to the Grid.
• Sensors Grids SensorGrid

12
Outline

• Introduction
• Technology Trends in Sensors and Grids
• Sensors and Grids Integration Proposal
• Grid Computing
• What is it?, architecture, and technologies
• NOSA (NICTA Open SensorWeb Architecture)
• Summary and Conclusion

13
Grid (computing) ParadigmCyberinfrastructure
for sharing resources

• Inspired by Power Grid!
• A service-oriented/utility computing paradigm
  that enables seamless sharing of geographically
  distributed, autonomous resources.
• This was the original aim of building Internet
  although it ended up in giving birth to email!

14
Grids have Emerged as Scalable Cyberinfrastructure
for e-Science Applications
15
Type of Services Modern Grids Offer

• Computational Services CPU cycles
• SETI_at_Home, NASA IPG, TeraGrid, I-Grid,
• Data Services
• Data replication, management, secure access--LHC
  Grid/Napster
• Application Services
• Access to remote software/libraries and license
  managementNetSolve
• Information Services
• Extraction and presentation of data with meaning
• Knowledge Services
• The way knowledge is acquired and manageddata
  mining.
• Utility Computing Services
• Towards a market-based Grid computing Leasing
  and delivering Grid services as ICT utilities.

Utility Grid
Knowledge Grid
Information Grid
ASP Grid
Data Grid
Computional Grid
16
Grid Capabilities
17
Some Grid Initiatives Worldwide

• Australia
• Nimrod-G
• Gridbus
• DISCWorld
• GrangeNet.
• APACGrid
• ARC eResearch
• Brazil
• OurGrid, EasyGrid
• LNCC-Grid many others
• China
• ChinaGrid Education
• CNGrid - application
• Europe
• UK eScience
• EU Grids..
• and many more...
• India
• I-Grid

• USA
• Globus
• NASA IPG
• AccessGrid
• TeraGrid
• Cyberinfrasture
• and many more...
• Industry Initiatives
• IBM On Demand Computing
• HP Adaptive Computing
• Sun N1
• Microsoft - .NET
• Oracle 10g
• Infosys Business Grid
• StorageTek Grid..
• and many more
• Public Forums
• Global Grid Forum
• Australian Grid Forum

27 million
1.3 billion 3 yrs
2? billion
120million 5 yrs
450million 5 yrs
486million 5 yrs
1.3 billion (Rs)
1 billion 5 yrs
http//www.gridcomputing.com
18
The Gridbus Project _at_ MelbourneEnable Leasing
of ICT Services on Demand
Distributed Data
WWG
Gridbus
World Wide Grid! ?On Demand Utility Computing
www.gridbus.org
19
(No Transcript)
20
Gridbus Architecture Layer
21
On Demand Assembly of Services Putting Them All
Together
Data Source
(Instruments/distributed sources)
Cluster Scheduler
PE
Grid Service Provider (GSP) (e.g., CERN)
22
Alchemi .NET-based Enterprise Grid Platform
Web Services
Alchemi Manager
Web Services
Internet
Alchemi Users
Internet

• SETI_at_Home like Model
• General Purpose
• Dedicated/Non-dedicate workers
• Role-based Security
• .NET and Web Services
• C Implementation
• GridThread and Job Model Programming
• Easy to setup and use
• Widely in use!

Alchemi Worker Agents
23
Some Users of Alchemi
Tier Technologies, USA Large scale document
processing using Alchemi framework
Satyam Computers Applied Research Laboratory,
India Micro-array data processing using Alchemi
framework
CSIRO, Australia Natural Resource Modeling
The University of Sao Paulo, Brazil The Alchemi
Executor as a Windows Service
stochastix GmbH, Germany Asynchronous Excel Tasks
using ManagedXLL and Alchemi .Net Grid Computing
framework.
The Friedrich Miescher Institute (FMI) for
Biomedical Research, Switzerland Patterns of
transcription factors in mammalian genes
Many users in Universities See next for an
example.
24
                           
Students' project gives old computers new life  - 1/25/2005
25
The Gridbus Grid Service Broker for Data Grid
Applications

• Builds on the Nimrod-G Computational Grid Broker
  and Computational Economy Buyya, Abramson,
  Giddy, Monash University, 1999-2001
• And
• Extends its notion for Data and Service Grids

26
Gridbus Broker Architecture
Gridbus Client
Gridbus Client
Gribus Client
(Bag of Tasks Applications)
App, T, , Opt
(Data Grid Scheduler)
Gridbus Farming Engine
Schedule Advisor
Trading Manager
RecordKeeper
Grid Dispatcher
Grid Explorer
Grid Middleware
TM TS

GE GIS, NWS
Grid Info Server
RM TS
G

Data Catalog
Data Node
C

U
G
Unicore enabled node.
Globus enabled node.
L
A
RM Local Resource Manager, TS Trade Server
Alchemi enabled node.
27
Gridbus Services for eScience applications

• Application Development Environment
• XML-based language for composition of task
  farming (legacy) applications as parameter sweep
  applications.
• Task Farming APIs for new applications.
• Web APIs (e.g., Portlets) for Grid portal
  development.
• Threads-based Programming Interface
• Workflow interface and Gridbus-enabled workflow
  engine.
• Resource Allocation and Scheduling
• Dynamic discovery of optional computational and
  data nodes that meet user QoS requirements.
• Hide Low-Level Grid Middleware interfaces
• Globus, Alchemi, Unicore, NorduGrid, XGrid, etc.

28
Click Here for Demo
Figure 3 Logging into the portal.
Drug Design Made Easy!
29
Sample Applications of Gridbus Broker

• Molecular Docking - WEHI
• Drug Discovery
• Brain Activity Analysis Osaka University
• Neuroscience studies
• Natural Language Engineering Melbourne NLP
• Indexing of newswire data
• High Energy Physics School of Physics/Melbourne
• Belle experiment data analysis
• Finance - Portfolio Analysis U. Comp.
  Madrid/Spain
• Investment risk analysis
• Astronomy School of Physics_at_UoM and QUT
  (Queensland University of Technology)
• Australian Virtual Observatory
• Spreadsheet Processing
• Microsoft Excel

30
Outline

• Introduction
• Technology Trends in Sensors and Grids
• Sensors and Grids Integration Proposal
• Grid Computing
• What is it?, architecture, and technologies
• NOSA (NICTA Open SensorWeb Architecture)
• Summary and Conclusion

31
Sun Proposal For Integration of Sensors and High
End Computers using Network
Source Anil Velluri, Sun
32
Our Proposal SensorGrid - Integrated sensor
network and grid architecture
sensor and actuator networks
query
response
grid
command
sense
actions
actuate
33
Sensor Web and NOSA

• Goal To create Web-based sensor networks by
  exploiting web-connected sensors (flood gauges,
  air pollution monitors, mobile heart monitors,
  satellite-borne earth imaging monitors).
• This would make all sensors and repositories of
  sensor data discoverable, accessible,
  process-able and where applicable controllable
  via the WWW.
• OGC (Open Geospatial Consortium) describes five
  important encoding and service standards for a
  Sensor Web Implementation (see next slide).

34
What is NOSA?

• The NICTA Open SensorWeb Architecture (NOSA)
  project is developing a complete standards
  compliant platform and middleware for integration
  of sensor networks with emerging distributed
  computing platforms such as Grids.
• It confirms to Web Services standard defined by
  W3C (World-Wide Web) and SensorML (Sensor Model
  Language) standard defined by OpenGeospatial
  Consortium.

35
What are NOSA Benefits?

• This integration of sensor networks with Grid
  computing brings out dual benefits (i) sensor
  networks can off-load heavy processing activities
  to the Grid and (ii) Grid-based sensor
  applications can provide advance services for
  smart-sensing by deploying scenario-specific
  operators at runtime.
• Fundamental services are provided by lower-level
  layer components whereas components at the
  higher-level layer provide tools for creation of
  applications and management of life-cycle of data
  captured through sensor networks.

36
NICTA Open SensorWeb Architecture (NOSA)
ApplicationsLayer
ApplicationDevelopmentLayer
ApplicationServicesLayer
Sensor FabricSimulationEnvironment
37
OGC-Sensor Web Enablement Standard Specifications

• SensorML XML encoding language for sensors. Used
  to discover, query and control Web-resident
  sensors.
• Observations Measurements The general models
  and an XML encoding for what a sensor observes or
  measures (The value returned by or derived from a
  sensor observation -e.g. quantity, count,
  boolean, category, ordered category, position-).
• Sensor Collection Service A service to fetch
  observations from a sensor or constellation of
  sensors. Provides real time or archived observed
  values. Clients can also obtain information that
  describes the associated sensors and platforms.
• Sensor Planning Service A service by which a
  client can determine collection feasibility for a
  desired set of collection requests for one or
  more mobile sensors/platforms, or the client may
  submit collection requests directly to these
  sensors/platforms. SPS enables sensor tasking,
  acquisition requests, processing and simulation
  requests, and registration for alert
  notification.
• Web Notification Service A service by which a
  client may conduct a dialog with one or more
  other services. Provides a means for Sensor
  Planning Services to alert people, software, or
  other sensor systems of SPS results or alerts
  regarding phenomena of interest.

38
A sensor node includes a sensor and radio board.
Then the sensor node could send data message
through radio to Station which is the gateway
connected with the PC.
Here is the architecture of the interface
connecting sensor network and the real PC.
39
Crossbow Wireless Sensor Network Kit and Testbed
Setup in GRIDS Lab
Sensor MTS 300
Base station MIB510CA
40
SensorWeb Implementation
41
A SensorWeb Collection Client
42
Performance of collecting auto-sending and query
data
43
Performance of collecting auto-sending and query
data
44
Outline

• Introduction
• Technology Trends in Sensors and Grids
• Sensors and Grids Integration Proposal
• Grid Computing
• What is it?, architecture, and technologies
• NOSA (NICTA Open SensorWeb Architecture)
• Summary and Conclusion

45
Summary and Conclusion

• Sensors and Grids two major elements of emerging
  Cyberinfrastructure that support eApplications
  (e-Science, e-Business, e-Health, e-Life).
• They support creation of smart office, house, and
  business environments.
• Current developments in Sensors and Grids is
  heavily driven by applications and both
  compliment and need each other.
• SensorGrid is just emerging and there are many
  opportunities available for creating many
  interesting applications in various domains.

46
References

• Rajkumar Buyya and Srikumar Venugopal, A Gentle
  Introduction to Grid Computing and Technologies,
  CSI Communications, pages 9-19, Computer Society
  of India, Vol.29, No.1, July 2005.
• Chen-Khong Tham and Rajkumar Buyya, SensorGrid
  Integrating Sensor Networks and Grid Computing,
  CSI Communications, pages 24-29, Computer Society
  of India, Vol.29, No.1, July 2005.

47
Any Questions ?
Web - http//www.gridbus.org
http//www.buyya.com