The GRIDCC Project: providing a real-time GRID for distributed instrumentation

1
The GRIDCC Project providing a real-time GRID
for distributed instrumentation

• F. Asnicar1, L. Del Cano1, G. Maron2,
  R.Pugliese1, C.Scafuri1
• on behalf of the GRIDCC Collaboration
• 1Sincrotrone Trieste ELETTRA, Trieste, Italy,
  2 INFN, Legnaro, Italy
• ICALEPCS'05
• Geneva, Switzerland, 10-14 October 2005

2
Outline

• Introduction
• The GRIDCC project
• Pilot Applications
• The GRIDCC architecture
• The Instrument Element
• The Execution Services
• The Multipurpose Collaborative Environment
• Status of the Art
• Requirements and first design choices
• The MCE _at_ Work

3
Project goals

• ... the GRIDCC project extends the state of the
  art of computing Grid technologies, by
  introducing the handling of real-time constraints
  and interactive response into the existing Grid
  middleware
• build a widely distributed system that is able
  to remotely control and monitor complex
  instrumentation these new applications
  introduce requirements for real-time and highly
  interactive operation of GRID resources.

4
Project goals (cont.)

• verify the feasibility of a Grid-based remote
  control of systems requiring real-time response
  with real applications running on existing Grid
  test beds over both national and international
  network infrastructures (e.g. GEANT).
• integrate a grid of instruments into existing
  Grid infrastructures that provide the
  computational power and storage needed for the
  applications

5
Project in 3 steps

• Development of generic Grid middleware, based on
  existing building blocks (Grid Services) which
  will allow the remote control and monitoring
  instrumentation such as distributed systems.
• Testing of the middleware on challenging
  applications to validate it both in terms of
  functionality and quality of service
• European Power Grid
• Geo-hazards
• Remote Operation of an Accelerator Facility
• High Energy Physics Experiment
• Dissemination of the new software technology to
  encourage a wide range of enterprises to evaluate
  and adopt our Grid-oriented approach to real-time
  control and monitoring of remote instrumentation.

6
GRIDCC project landscape
Use of the Grid technology, as extension of the
Web Service Technologies, to develop a widely
distributed control system with access to grid
enabled computing and data storage facilities
Virtual Ctrl. Room
Instrument 1
Computing Element
Supporting Services
Instrument 2
Diagnostics
Storage Element
Instrument 3
Virtual Ctrl. Room
7
Participants
Participant name Country
Istituto Nazionale di Fisica Nucleare Italy
Institute Of Accelerating Systems and Applications Greece
Brunel University UK
Consorzio Interuniversitario per Telecomunicazioni Italy
Sincrotrone Trieste S.C.P.A (Elettra) Italy
IBM (Haifa Research Lab) Israel
Imperial College of Science, Technology Medicine UK
Istituto di Metodologie per lAnalisi ambientale Consiglio Nazionale delle Ricerche Italy
Universita degli Studi di Udine Italy
Greek Research and Technology Network S.A. Greece
8
Pilot Application Power Grid

• In electrical utility networks (or power grids),
  the introduction of very large numbers of
  embedded power generators often using renewable
  energy sources, creates a severe challenge for
  utility companies.
• GRIDCC technology would allow the generators to
  participate in a Virtual Organization, and
  consequently to be monitored and scheduled in a
  cost-effective manner.
• The test bed (by means of computer simulation and
  emulation) will demonstrate the performance of
  the emulat system under various conditions,
  ranging from light power system loading (where
  energy economics is most important) to power
  system emergency conditions (where overloaded
  power circuits necessitate co-ordinated generator
  control to avoid power black-outs).

9
Pilot application Geo-hazardsRemote Operation
of Geophysical Monitoring Network

• Objective To enable real-time processing of data
  acquired by heterogeneous (multi-parametric,
  distributed, remote or unmanned, etc.)
  geophysical network exploiting GRID
  capabilities.
• Monitoring of fluid and gas migration processes
  in volcanic areas
• Monitoring of diffusion processes of contaminant
  plumes and the study of groundwater circulation
  system in landslide bodies.

10
Pilot ApplicationHigh-Energy Physics control
and monitor of experiments

• The so-called Run Control and Detector Control
  Systems of the experiment are charged with
  supervising the full configuration of the
  detector, but also with monitoring the data read
  out, their analysis and on-line interpretation.
• Monitoring this detector, and potentially
  changing settings as a result of analysis on the
  monitoring data, is a complex task shared by a
  few hundred people distributed in geographically
  distributed laboratories.
• This task requires continuous analysis and
  display of large amounts of data generated by the
  detector and in the past was done in a counting
  room near to the detector. In the context of
  GRIDCC, this application will be made to run in a
  completely distributed fashion, over the Grid.

11
Pilot Application Far Remote Operations of
ELETTRA
12
Pilot Application Far Remote Operations of
ELETTRA

• Far remote operation of an accelerator facility
  (i.e. the ELETTRA Synchrotron Radiation Facility)
  involves
• maintenance of the accelerator and its
  troubleshooting, the repair of delicate
  equipment
• understanding and pushing performance limitations
• performing commissioning and set ups
• routine operations
• All these activities are based on large amounts
  of information, which are at present accessible
  only at the accelerator site.

13
GRIDCC Architecture
14
The role of the Instrument Element
15
The Architecture of the Instrument Element
16
The Execution Service Arhitecture
17
An Example Workflow 1BM
18
Multipurpose Collaborative Environment

• The MCE is a groupware which provides general
  purpose services to control remote
  instrumentation, manage experimental activity.
• The system will be used to implement the VCR for
  the different applications of the GRIDCC project
  through customization and integration with
  application specific services.
• The MCE will be based on a core groupware
  application (e.g., authentication, management of
  the VO users and instruments, monitor of the
  instrument status, job control, etc) and a set of
  plug-ins
• general (e.g., chat, file browser, notebook,
  video conference)
• specific to the particular application (e.g.,
  accelerator control, instrument control)

19
MCE General Requirements

• Support for general distance collaboration and,
  at the same time, access to remote control and
  monitoring of scientific instrumentation
• Support of different classes of users MCE users,
  MCE administrators, MCE grid service providers
• Modularity and flexibility different VCR
  application support
• MCE will mainly rely upon web-based interfaces
  allowing users to access its resources at any
  location through a standard web-browser
• Partial support of PDA and mobile devices

20
What is a Collaboratory?

• A laboratory without walls in particular, the
  core capabilities that constitute a collaboratory
  can be seen as technologies to link
• People to people (e.g., electronic mail, and
  tools for data conferencing, such as VRVS)
• People to information (e.g., the World Wide Web
  and digital libraries)
• People to facilities (e.g., status of remote
  instruments) to enhance utilization by expanding
  access to resources

21
State of the Art HCI perspective

• Collaboration Tools
• Communication Support Systems
• Shared Workspaces and Applications
• Electronic Notebooks
• Meeting and Decision Support Systems
• Remote access and Control Systems

22
State of the Art technological and architectural
perspective

• Collaboration Portals
• GRID Information Portal
• GRID Application Portal
• Portal Framework
• Collaboration Tools
• Emerging Standards (JSR-168, WSRP,)

23
www.lightsources.org
24
First design choices
25
The MCE _at_ Work
26
The MCE _at_ Work
27
The MCE _at_ Work
28
The MCE _at_ Work
29
The MCE _at_ Work
30
More info on www.gridcc.org
31
What is a Collaboratory?

• The term collaboratory was coined by William
  Wulf by merging the words collaboration and
  laboratory, and defined as ... Centre without
  walls, in which researchers can perform their
  research without regard to geographical location
  - interacting with colleagues, accessing
  instrumentation, sharing data and computational
  resource, and accessing information in digital
  libraries.

32
Reporting I
33
Project Timing
1
2
3
Years
34
Human Resources
IASA BRNL 3 CNIT 4 ELETTRA 5 IBM 6 Imperial 7 INFN 1 IMAA 8 UNIUD 9 GRNET 10 TOTAL PARTNERS
Research/innovation activities
WP1 System Architecture 6 6 6 6 6 30
WP2 Real time Web Service 38 36 33 107
WP3 Grid Enabled Instrumentation 106 42 20 154 24 346
WP4 Real-time workload 149 149
WP5 Cooperative Env. 101 48 149
WP6 System integ. deply. 136 50 62 54 44 80 75 30 531
WP7 Dissemination 4 4 8 4 4 4 4 32
Total research/innovation 246 140 90 165 36 203 277 79 48 60 1344
Consortium management activities
WP8 Management 6 6 6 10 6 48 82
Total consortium management 6 6 6 10 6 48 82
35
State of the Art Technical and Architectural
Perspective
PRODUCT TYPE TECHNOLOGY
CHEF Collaboration Portal Jakarta Jetspeed
Global-MMCS Collaboration Portal OpenH323, NIST SIP stack, VIC and RAT, Narada Brokering, Java Media Framework. Jetspeed, Java Applets, ActiveX, Apache Batik
ElettraVirtual Collaboratory (EVC) Collaboration Portal web / tomcat / struts / webmin / vnc / axis
Upper Atmospheric Research Collaboratory (UARC) Collaboration Portal 1st version NeXTStep 2nd version Java Applets
Pervasive Collaborative Computing Environment (PCCE) Collaboration Portal Old Modified IRC server. PCCE server. Java Swing User Interface New Based on Jabber XMPP protocols, BPEL4WS and grid services
Space Physics and Aeronomy Research Collaboratory (SPARC) Collaboration Portal Built as a thin client (e.g. HTTP), migrating toward CHEF-based system (e.g. portlet technology)
Core2000 Collaboration Portal Web technology. Mbone tools
GENIUS GRID Application Portal EnginFrame Apache, HTTPS, Java, XML, rfb (remote frame buffer) GridIce, tightVNC
OGCE GRID Application Portal Jetspeed, CHEF, OGSA/OGSI, Globus CoG, GridSphere. MyProxy, GridFTP, NaradaBroker, Xmessages, Java JMS
HotPage GRID Information Portal GP-IR (XML-centric GridPort Information Repository Database), SOAPLite (Perl)
InfoPortal GRID Information Portal Perl/CGI Web technology. Globus Monitoring and Discovery Service, XML database. PHP-Nuke
36
State of the Art Techical and Architectural
Perspective
PRODUCT TYPE TECHNOLOGY
LCPortal GRID Application Portal GridPort, MyProxy, Globus. GridSphere
GRB GRID Application Portal Perl/CGI Web technology. C API to Globus. GridSphere
GridPort GRID Application Portal Perl/CGI Web technology, GlobusJetSpeed
PPDG GRID Application Portals VDT (Virtual Data Toolkit) Globus toolkit, Condor-G, SRM, SAM, GridFTP, JIM, Public Key Infrastructure (X.509 certificates), MonaLisa
GPDK Portal Framework Java Servlets, JSP
GridSphere Portal Framework Portlets, standards, OGSA. Starts from Websphere, Jetspeed, GPDK, role based access control, Castor JDO JSR-168 standard
JetSpeed Portal Framework Java Applets, Servlets, JSP. XML JSR-168 standard
AccessGrid Collaboration tools Python, SoapPy, OpenSSL, Globus Toolkit
VRVS Collaboration tools Mbone tools (VIC RAT). H.323 Java J2EE XML
PCCE Collaboration tools Jabber Instant Messenger
Collaboratory for Macromolecular Crystallography Collaboration tools NX Industry software
Molecular Interactive Collaborative Environment (MICE) Collaboration tools Standalone application. Java, CORBA, ICE (Interactive Collaborative Environment)