GRID enabled remote instrumentation and sensors with distributed control

1
GRID enabled remote instrumentation and sensors
with distributed control
EU FP6 Project

• Francesco Lelli
• Istituto Nazionale di Fisica Nucleare
  Laboratori di Legnaro
• Legnaro (PD), Italy

2
The Grid Technologies to extend the limit of a
single computer (center)
Storage Element
Computing Element
Grid Gateway
Grid Technologies
Computing Element
User Interface
Computing Element
3
Extending the Grid Concepts
Grid Gateway
Grid Technologies
Satellite views to monitor the volcano
Terrestrial probes to monitor The volcano
activities
To model calculations and disaster predictions
Control and Monitor Room
4
GridCC
Data for Model Calculations
Predictions

Instruments Grid
Computational Grid
GridCC
5
The GRIDCC project Goals Objectives

• The GRIDCC - Grid Enabled Remote Instrumentation
  with Distributed Control and Computation
  project has the main aim to bring Instruments to
  the GRID
• It is a 3-years EU FP6 project started in
  September 2004
• Web site www.gridcc.org
• More in particular the project goals are
• Definition of a Instrument Element allowing a
  standard remote access to any type of
  instrumentation
• Tight integration between instrument grid and
  classical computational grid
• Human interaction with Grids via Virtual Control
  Room (collaborative environment)
• Enactment of complex workflows

6
The GRIDCC partners
7
GridCC The Basic Idea
Instrument Element
Instrument Element
Instrument Element
Virtual Control Room
Virtual Control Room
Computing Element
Computing Element
Computing Element
Storage Elements
Storage Elements
Storage Element
Existing Grid Infrastructures
Execution Service
8
GRIDCC Architecture
gLite GRID
GRIDCC
Virtual Control Room
Information System (BDII)
WMS Work Management System WfMS Work Flow Mng
System AS Agreement Service
Broker
Web Service Interface
9
GridCC PermanentTest Bed
VCR, IE
WfMS
VOMS
IE, CE/CREAM,SE, LFC
AS, KrbServer
WMProxy
VCR, SE, IE, BDII
10
GRIDCC main target areas
Remote process control Accelerator control
(Tele-) Biomedicine Robotics Automotive Electron
ic microscopes
(Large-scale) scientific experiments High energy
particle physics (Radio-) Telescopes
GRIDCC Middleware
Widely Sparse Instrumentation Power Grids
Monitoring of the territory Monitoring of the
sea Geo-hazard prediction Distributed
laboratories Transportation monitoring Sensor
network
11
GRIDCC pilot applications
Power Grid
Particle Accelerator
High Energy Physics CMS
Meteorology
Device Farm
Geohazard Monitoring
12
Instrument Element Requirements
1 Provide a uniform access to the physical
devices

• 2 Allow a standard grid access to the
  instruments

3 Allow the cooperation between different
instruments that belong to different VOs
13
A new 4 channels element to fit the
requirements The Instrument Element (IE)
Custom (JMS, ..... )
Discovery
Data/Info Pubblishing
Im here
Instrument Element
Instrumentation Contorl
Web Service
GRID Access
SRM/GRIDFTP
Instruments Access
Advance Reservation
Custom
QoS
14
Instrument Element Architecture
IMS
The term Instrument Element describes a set of
services that provide the needed interface and
implementation that enables the remote control
and monitoring of physical instruments.
VIGS
IMS Proxy
Control Manager
Data Collector
Control Manager
Event Processor
FSM Engine
Instrument Protocols
Input Manager
Resource Proxy
Real Instruments
15
Virtual Control Room (VCR)

• The VCR is a web portal
• Based on the Gridsphere framework
• (portlet technology) and providing
• general cooperative tools
• event tools
• resource tools
• communicatin tools
• chat
• video conf
• application specific tools
• instrument control
• log book
• log reports
• data monitor

16
VCR examples
17
GRIDCC at Work
18
Pilot applications CMS - I

• The GridCC middleware has been deployed to
  control the run of the CMS (Compact Muon
  Solenoid), one of the four high energy
  experiments in LHC (Large Hadron Collider) at
  CERN laboratory.
• CMS Magnet Test and Cosmic Challenge (MTCC) I and
  II, a milestone in the CMS construction,
  positively carried out.

CMS Detector
CMS Control Structure
User Interface
Top IE
CSC IE
Tracker IE
HCAL IE
DAQ IE
RPC IE
Trigger IE
DT IE
ECAL IE
19
CMS MTCC phase I and phase II carried out in 2006
Scale MTCC versus CMS FEDs 20 out of 600
3 EVB RUs 6 out of 600 1 Filter Farm 14
out of 2000 0.3 Trigger rate 100 Hz out of 100
kHz 0.1 Event size 200 kB out of 1
MB 20 IEs 15 out of 150 10

20
VCR
Main Data Flow (Narval Systems)
Resource Service Log Collector Error Collector ..
IM Function Manager
Services
Web Services
Services
Top IM
Ancillary IM
SlwCtrl IM
Narval Systems
PSA Chf dOr
EVB Chf dOr
Tracking Chf dOr
SlowCtrl Sub- System
Tracking Sub-System
PSA Sub-System
EVB Sub- System
Storage Sub- System
Ancillary Sub- System
21
Pilot applications Power grid

• GRIDCC deployed to monitor
• A 50kW generator
• A 1 kw Photo-Voltaic array

Instrument Manager
Power Grid V.O.
Instrument Element
Solar Panel
...
Gas
22
Pilot applications Remote Operation of an
Accelerator
23
GRIDCC other applications

• Meteorology (Ensemble Limited Area Forecasting)
• Weather forecasting system to detect hazardous
  weather
• Device Farm for the Support of Cooperative
  Distributed Measurements in Telecommunications
  and Networking Laboratories
• The Device Farm consists of a pool of Measurement
  Instruments for Telecommunication Experiments
• Geo-hazards Remote Operation of Geophysical
  Monitoring Network
• The monitoring net will be characterized by
  different levels of activity stand-by,
  pre-alert, alert, plus a control modality
• An event worth to be monitored is for example the
  evolution of a landslip

24
Instrument Element Scalable on embedded systems
1 Gbps Ethernet
IE
Instrument Manager
Custom Logic
Standalone Axis
Montavista www.montavista.com
USER INTERFACE
JamVM http//jamvm.sourceforge.net
25
IE technologies

• Web Service compliant (WS-I)
• Tomcat Axis (and Java) and Axis standalone are
  the main technologies of the IE
• All the services are deployed on a single or
  multiple instances of Tomcat, according to the
  needs of the application
• Message oriented middleware (Pub/Sub) is based
  on the Java Messaging System (JMS). The following
  implementations are used in the project
• Sun
• Narada Brokering
• RMM - JMS (GridCC IBM)
• MySQL and Oracle are used as Data Base for the RS
• SRM interface version 2.2 used

26
Conclusions I

• The GRIDCC technology (based on web services) is
  used to have an homogenous backbone to control
  and monitor geographically widely distributed
  instrumentation.
• Data taken from the instrumentation can be
  directly seen by the european computational and
  storage GRID
• CMS experiment is using for its data taken GRIDCC
  based run control system. The final configuration
  is target to control O(1000) nodes
• AGaTA experiment is building its data taken
  GRIDCC based run control system. The final
  configuration is target to control O(1000) nodes

27
Conclusions II

• GRIDCC is ending this year having reached all the
  mail milestones foreseen and having, in
  particular, a production environment (CMS) daily
  running and serving hundreds of physicists in
  commanding and controlling either their sub
  detectors or the full experiment.
• GRIDCC team is now keen to look for new
  exploitations of their outcomes in different
  contexts adding, possibly, new developments and
  customization of  the middleware according to
  the new needs.
• The control and monitor of  pan European 
  projects like EMSO and KM3NET appears to be very
  attractive in this sense, as it could be
  implemented like a  new e-infrastructure across
  Europe dedicated to the remote control of big
  scientific apparatus. Moreover the online data
  integration with the computational and storage
  GRID provides to fill a gap in the existing
  computational grid and at the same time give a
  strong added value to the proposal of this new
  e-infrastructure.

28
Conclusions III
Lambda Network
http//sadgw.lnl.infn.it2002/MapsMonitor
Grid Technologies
Pervasive Computing
Web 2.0 Tools
29
Discovery Instruments (I)

• The discovery of the instruments or of the IEs is
  an issue when the number of elements is high
• We can have two cases
• Quasi static cases
• The number of IEs are well defined and the single
  IE is quite complex with a good hardware support
• In this case a register based discovery mechanism
  can be used.
• GRIDCC tesbed is using the LCG BDII (Berkely
  Database Information Index ) based on LDAP
• The information collected in the BDII follow a
  GLUE schema and can be used for match making
  querries

Querring Client
Querring Client
Querring Client
BDII
Local BDII
30
Discovering Instruments (II)

• Dynamic cases
• The number of IEs can change very quickly, they
  are very simple devices, often with poor hardware
  support
• The discovery is just use to know which are the
  online IEs
• A new approach has been developed based on Peer
  to Peer (P2P) protocols
• See the demo

31
Question?

• Thx for your time!

More

• The GRIDCC Project. Grid enabled Remote
  Instrumentation with Distribute Control and
  Computation. Official Webpage of FP6 European
  funded project.
• Video On-Line Demo. A sample video that shows the
  GRIDCC service orchestration.
• Live Demo. A Peer to Peer Approach for the
  Geo-Location of a Grid of Instruments
• GridCC in GRIDToday. Gaetano Maron and Francesco
  Lelli presented the GRIDCC project in one of the
  leading online journals.
• GridCC in Wikipedia
• Instrument Element Facade Online documentation
  for Instrument Elements developers.