AIST Grid Initiative in Japan and the Asia Pacific Region

1
AIST Grid Initiative in Japan and the Asia
Pacific Region

• Yoshio Tanaka
• Grid Technology Research Center,
• Advanced Industrial Science and Technology, Japan

2
Talk Contents

• Introduction
• myself and AIST
• Research Activities
• Ninf-G GridRPC Programming Middleware
• ApGrid Asia Pacific Grid
• Grid PSE Builder

3
Self Introduction

• 1995
• Received Ph.D from Keio University
• Parallel GC (Garbage Collection)
• 19961999 Real World Computing Partnership
  (RWCP)
• Programming and performance evaluation of SMP
  clusters
• Firewall-compliant Globus Toolkit MPICH-G
• 2000 Electrotechnical Laboratory (ETL)
• Ninf-G, ApGrid
• 2001 AIST

4
What is the AIST ?

• One of the largest Natl Labs in Japan
• Research topics include
• Environment
• Material
• Bio/Life science
• Standards (JIS/OSI)
• Geographical survey
• Semiconductor device
• Computer Science
• etc.
• 3,500 employee 3,000 staff
• roughly 1,400M USD/FY2002

AIST Tsukuba Main Campus
7 other campuses across Japan
Tsukuba
40km
Narita
50km
Tokyo
50km
5
Grid Technology Research Center

• Establishment
• Since Jan. 1, 2002
• 7 years term
• 24th Research Center of AIST
• Location
• Tsukuba Central
• Umezono 1-1, Tsukuba
• Tokyo Office
• Ueno area
• 30 people for software development
• Engaged in developing grid middleware,
  applications and system technologies
• Research approx. 1000M JPY

2002/1H 2002/2H 2003/1H
Researchers Researchers Researchers Researchers Researchers
Full time 14 16 19
Fellowship 1 8 9
Collaborators Collaborators 7 21 32
Sub total Sub total 22 44 60
Staff Staff Staff Staff Staff
Administration 2 1 1
Support 5 7 9
One of the worlds foremost GRID Research Center,
the largest in Japan
6
Grid Tech. Research Center
Deputy Director Mitsuo Yokokawa
Director Satoshi Sekiguchi
Grid Science Application Team (Leader Umpei Nagashima)
RD on Scientific Applications on Grid. QC Grid / Gaussian Portal RD on Scientific Applications on Grid. QC Grid / Gaussian Portal
Grid Diversification Team (H15.4.1- ) (Leader Satoshi Itoh)
RD of Middleware and Applications for Business on Grid. Grid PSE Builder RD of Middleware and Applications for Business on Grid. Grid PSE Builder
Data-Intensive Computing Team (Leader Isao Kojima)
Data Grid / Database and Grid (OGSA-DAIS, etc.) Data Grid / Database and Grid (OGSA-DAIS, etc.)
Grid Infraware Team (Leader Yoshio Tanaka)
Programming Middleware, Testbed Development, Grid Security. Ninf-G, ApGrid Programming Middleware, Testbed Development, Grid Security. Ninf-G, ApGrid
Secure Programming Team (Leader Hiromitsu Takagi)
Security of Grid and Web Security of Grid and Web
Cluster Technology Team (Leader Tomohiro Kudoh)
Interconnection, GFarm Interconnection, GFarm
7
AIST GTRC (Grid) Super Cluster
P32 IBM eServer325 Opteron 2.0GHz, 6GB 2way
x 1074 node Myrinet 2000 8.59TFlops/peak
10,200mm
Myrinet
M64 Intel Tiger 4 Madison 1.3GHz, 16GB 4way
x 131 node Myrinet 2000 2.72TFlops/peak
10,800mm
F32 Linux Networx Xeon 3.06GHz, 2GB 2way x
256 node GbE 3.13TFlops/peak
P32
M64
total 14.5TFlops/peak, 3188 CPUs
8
National Research Grid Initiative (NAREGI)
ProjectOverview

• A new Japanese MEXT National Grid RD project
• (US)17M FY03 (similar until FY07) 45mil
• One of two major Japanese Govt. Grid Projects
• c.f. BusinessGrid ((US)25M FY03-05) METI
• Collaboration of National Labs. Universities and
  Major Computing and Nanotechnology Industries
• - Acquisition of Computer Resources underway
  (FY2003)

MEXTMinistry of Education, Culture,
Sports,Science and Technology
9
NAREGI Software Stack
WP6 Grid-Enabled Apps
WP3 Grid Visualization
WP3 Grid PSE
WP3 Grid Workflow
WP2 Grid Programming-Grid RPC -Grid MPI
WP4 Packaging
WP1 Grid Monitoring Accounting
WP1 SuperScheduler
(Globus,Condor,UNICORE?OGSA)
WP5 Grid PKI
WP1 Grid VM
WP5 High-Performance Grid Networking
10
Ninf-G GridRPC Programming Middleware
11
Layered Programming Model/Method
Easy but inflexible
Portal / PSE GridPort, HotPage, GPDK, Grid
PSE Builder, etc
High-level Grid Middleware MPI (MPICH-G2,
PACX-MPI, ) GridRPC (Ninf-G, NetSolve, )
MPI
Low-level Grid Middleware Globus Toolkit
Primitives Socket, system calls,
Difficult but flexible
12
Some Significant Grid Programming Models/Systems

• Data Parallel
• MPI - MPICH-G2, Stampi, PACX-MPI, MagPie
• Task Parallel
• GridRPC Ninf, Netsolve, Punch
• Distributed Objects
• CORBA, Java/RMI,
• Data Intensive Processing
• DataCutter, Gfarm,
• Peer-To-Peer
• Various Research and Commercial Systems
• UD, Entropia, Parabon, JXTA,
• Others

13
GridRPC RPC based Programming model
Utilization of remote supercomputers
? Notify results
Internet
user
? Call remote procedures
Call remote libraries
Large scale computing utilizing multiple
supercomputers on the Grid
14
GridRPC RPC tailored for the Grid

• Medium to Coarse-grained calls
• Call Duration lt 1 sec to gt week
• Task-Parallel Programming on the Grid
• Asynchronous calls, 1000s of scalable parallel
  calls
• Large Matrix Data File Transfer
• Call-by-reference, shared-memory matrix arguments
• Grid-level Security (e.g., Ninf-G with GSI)
• Simple Client-side Programming Management
• No client-side stub programming or IDL management
• Other features

15
GridRPC (contd)

• v.s. MPI
• Client-server programming is suitable for
  task-parallel applications.
• Does not need co-allocation
• Can use private IP address resources if NAT is
  available (at least when using Ninf-G)
• Better fault tolerancy
• Activities at the GGF GridPRC WG
• Define standard GridRPC API later deal with
  protocol
• Standardize only minimal set of features
  higher-level features can be built on top
• Provide several reference implementations
• Ninf-G, NetSolve,

16
Typical Scenario Optimization Problems and
Parameter Study on Cluster of Clusters
rpc
rpc
rpc
Structural Optimization
Vehicle Routing Problem
Slide by courtesy of Prof. Fujisawa
17
Sample Architecture and Protocol of GridRPC
System Ninf -
Server side
Client side

• Server side setup
• Build Remote Library Executable
• Register it to the Ninf Server

• Call remote library
• Retrieve interface information
• Invoke Remote Library Executable
• It Calls back to the client

IDL file
Numerical Library
Client
IDL Compiler
Ninf Server
18
GridRPC based on Client/Server model

• Server-side setup
• Remote libraries must be installed in advance
• Write IDL files to describe interface to the
  library
• Build remote libraries
• Syntax of IDL depends on GridRPC systems
• e.g. Ninf-G and NetSolve have different IDL
• Client-side setup
• Write a client program using GridRPC API
• Write a client configuration file
• Run the program

19
The GridRPC API

• Provide standardized, portable, and simple
  programming interface for Remote Procedure Call
• Attempt to unify client access to existing grid
  computing systems (such as NetSolve and Ninf-G)
• Working towards standardization through the GGF
  GridRPC WG
• Initially standardize API later deal with
  protocol
• Standardize only minimal set of features
  higher-level features can be built on top
• Provide several reference implementations
• Not attempting to dictate any implementation
  details

20
Rough steps for RPC

• Initialize
• Create a function handle
• Abstraction to a remote library
• RPC
• Call remote procedure

grpc_initialize(config_file)
grpc_function_handle_t handle grpc_function_han
dle_init( handle, host, port, lib_name)
grpc_call(handle, args)
or grpc_call_async(handle, args)
21
Data Parallel Application

• Call parallel libraries (e.g. MPI apps).
• Backend MPI orBackend BLACSshould be
  specifiedin the IDL

Parallel Computer
Parallel Numerical Libraries Parallel Applications
22
Task Parallel Application

• Parallel RPCs using asynchronous call.

23
Task Parallel Application

• Asynchronous Call
• Waiting for reply

Client
ServerA
ServerB
grpc_call_async(...)
grpc_call_async
grpc_call_async
grpc_wait_all
grpc_wait(sessionID) grpc_wait_all() grpc_wait_a
ny(idPtr) grpc_wait_and(idArray,
len) grpc_wait_or(idArray, len,
idPtr) grpc_cancel(sessionID)
Various task parallel programs spanning clusters
are easy to write
24
Ninf Project

• Started in 1994
• Collaborators from various organizations
• AIST
• Satoshi Sekiguchi, Umpei Nagashima, Hidemoto
  Nakada, Hiromitsu Takagi, Osamu Tatebe, Yoshio
  Tanaka,Kazuyuki Shudo , Hirotaka Ogawa
• University of Tsukuba
• Mitsuhisa Sato, Taisuke Boku
• Tokyo Institute of Technology
• Satoshi Matsuoka, Kento Aida
• Tokyo Electronic University
• Katsuki Fujisawa
• Ochanomizu University
• Atsuko Takefusa
• Kyoto University
• Masaaki Shimasaki

25
History of Ninf Project
1994
1997
2000
2003
Ninf-G development
Ninf project launched
Ninf-G Ver.2.0.0 Release
Standard GridRPC API proposed
Release Ninf version 1
GridRPC WG at GGF8 / GGF9
Start collaboration with NetSolve team
Release Ninf-G version 0.9
Release Ninf-G version 1.0
26
What is Ninf-G?

• A software package which supports programming and
  execution of Grid applications using GridRPC.
• Ninf-G includes
• C/C, Java APIs, libraries for software
  development
• IDL compiler for stub generation
• Shell scripts to
• compile client program
• build and publish remote libraries
• sample programs
• manual documents

27
Ninf-G Features At-a-Glance

• Ease-of-use, client-server, Numerical-oriented
  RPC system
• No stub information at the client side
• Users view ordinary software library
• Asymmetric client vs. server
• Built on top of the Globus Toolkit
• Uses GSI, GRAM, MDS, GASS, and Globus-IO
• Supports various platforms
• Ninf-G is available on Globus-enabled platforms
• Client APIs C/C, Java, Fortran

28
Architecture of Ninf-G
Server side
Client side
IDL file
Numerical Library
Client
IDL Compiler
Generate
Globus-IO
Interface Request
Interface Reply
Remote Library Executable
GRAM
GRIS
Interface Information LDIF File
retrieve
29
Demo System of a Climate Simulation

• Integrating 2 Ninf-G programs
• Climate Simulation program
• Visualization program
• Executed in a pipelined fashion
• Accessing through GridLib portal

S-model Program
Reading Data Averaging results
Solving Equations
Solving Equations
Solving Equations
Visualizing Results
30
Replica Exchange Monte-Calro Simulation

• Potential survey of molecules using direct method
  (ab-initio calc.)
• Random walk survey
• enables survey of complicated potential surface
• Ab-initio calculation
• enables precise energy calculation of molecules
• Replica Exchange method
• Enables efficient MC survey

31
Gridifying the program

• Two levels of parallelization
• Coarse grained parallel monte-carlo sampling
• Fine grained parallel ab-initio energy
  calculation
• Dynamic task scheduling, machine reconfiguration
• Task scheduling for balancing load on a
  heterogeneous computing resources
• Machine scheduling for reconfiguring machine sets
  on the fly

Bookkeeper
Reconfiguration request
Dynamic scheduling
Energy calc.
REXMC client
Task allocation
Monitoring Reconfiguration
T1
T2
MC Sampling
Servers
T3
meta-computing test bed 10 institutes/20
Supercomputers
ab initio calculation
32
HPC Challenge in SC2002
Metacomputing Test-bed

• 10 institutions (3 continentals) / 20 parallel
  computer (7 types)
• High Performance Computing Center Stuttgart
  (HLRS),
• Sandia National Laboratories (SNL),
• Pittsburgh Supercomputing Center (PSC),
• Grid Technology Research Center (AIST),
• Manchester Computing Centre (MCC),
• National Center for High Performance Computing
  (NCHC),
• Japan Atomic Energy Research Institute (JAERI),
• Korea Institute of Science and Technology
  Information (KISTI),
• European Center of Parallelism in Barcelona
  (CEPBA/CIRI),
• Finnish IT center for Science (CSC).

33
Current Status

• Ninf-G Ver. 2 alpha is available at
  http//ninf.apgrid.org/
• Ninf-G Ver. 2 will be released by the end of this
  March

34
ApGrid Asia Pacific Partnership for Grid
Computing
35
ApGrid Asia Pacific Partnership for Grid
Computing
North America
Europe

• International Collaboration
• Standardization

Asia
ApGrid Testbed International Grid Testbed over
the Asia Pacific countries

• ApGrid focuses on
• Sharing resources, knowledge, technologies
• Developing Grid technologies
• Helping the use of our technologies in create new
  applications
• Collaboration on each others work

• Possible Applications on the Grid
• Bio Informatics
• (Rice Genome, etc.)
• Earth Science
• (Weather forecast, Fluid prediction, Earthquake
  prediction, etc.)

36
PRAGMA Pacific Rim Application andGrid
Middleware Assembly
http//www.pragma-grid.net
37
History and Future Plan
2000
2001
2002
Demo _at_ HPCAsia Gold Coast, Australia
Kick-off meeting Yokohama, Japan
demo _at_ SC2002 Baltimore, USA (50cpu)
1st ApGrid Workshop Tokyo, Japan
Presentation _at_ GF5 Boston, USA
presentation _at_ SC2001 SC Global Event
demo _at_ iGrid2002 Amsterdam, Netherland
1st Core Meeting Phuket, Thailand
ApGrid PRAGMA Presentation _at_ APAN Shanghai, China
1st PRAGMA Workshop San Diego, USA
2nd PRAGMA Workshop Seoul, Korea
2nd ApGid Workshop/Core Meeting Taipei, Taiwan
38
History and Future Plan (contd)
2003
2004
3rd PRAGMA Workshop Fukuoka, Japan
demo _at_ SC2004 Pittsburgh, USA
presentation _at_ APAN Hawaii, USA
7th PRAGMA Workshop San Diego, USA
demo _at_ CCGrid Tokyo, Japan (100cpu)
6th PRAGMA Workshop Beijing, China
Asia Grid Workshop (HPC Asia) Oomiya, Japan
4th PRAGMA Workshop Melbourne, Australia (200cpu)
demo _at_ SC2003 Joing Demo with TeraGrid Phoenix,
USA (853CPU)
demo ApGrid Informal Meeting _at_ APAC03 Gold
Coast, Australia (250cpu)
5th PRAGMA Workshop Hsinchu, Taiwan (300cpu)
39
ApGrid Branch in Suns Boothat SC2001
40
Sun Grid Engine on the ApGrid Testbed
Ultra Enterprise Cluster Sun Grid Engine
(AIST, Japan)
Sun Demo Station Denver, USA
622Mbps x 2
Job submisssion via Globus
41
Large-scale Sun Grid Engine Grid Testbed in Asia
Pacific
42
ApGrid/PRAGMA Testbed

• Architecture, technology
• Based on GT2
• Allow multiple CAs
• Build MDS Tree
• Grid middleware/tools from Asia Pacific
• Ninf-G (GridRPC programming)
• Nimrod-G (parametric modeling system)
• SCMSWeb (resource monitoring)
• Grid Data Farm (Grid File System), etc.
• Status
• 26 organizations (10 countries)
• 27 clusters (889 CPUs)

43
Lessons Learned

• We have to pay much efforts for initiation
• Problems on installation of GT2/PBS/jobmanger
• Installation/configuration of GT2/PBS/jobmanager
  is still not so easy for application people.
• Most sites needed help for the installation.
• Software requirements depends on the application
  and middleware used by the application.
• In order to run GridRPC (Ninf-G) applications, I
  asked
• Open firewall/TCP Wrapper
• Additionally build Info SDK bundle with gcc32dbg
• Install Ninf-G
• change configuration of xinetd/inetd
• Enable NAT

44
Lessons Learned (contd)

• MDS is not scalable and still unstable
• Terrible performance
• GIIS lookup takes several ten seconds minutes
• Some parameters in grid-info-slapd.conf such as
  sizelimit, timeout, cahcettl, etc., should be set
  to appropriate values depends on your environment
  (number of registered objects, network
  performance between GRISes and GIISes, etc.).
• Well known problem ?
• Firewall, private IP addresses

45
Lessons Learned (contd)

• Difficulties caused by the grass-roots approach.
• It is not easy to keep the GT2 version coherent
  between sites.
• Different requirements for the Globus Toolkit
  between users.
• Middleware developers needs the newest one.
• Application developers satisfy with using the
  stable (older) one.
• It is not easy to catch up frequent version up of
  the Globus Toolkit.
• CoG is a current problem ?

46
Lessons Learned (contd)

• Difficulties caused by the grass-roots approach
  (contd)
• Most resources are not dedicated to the ApGrid
  Testbed. (though this is a common problem for
  Grids)
• There may be busy resources
• Need grid level scheduler, fancy Grid reservation
  system?

47
Lessons Learned (contd)

• Some resources are not stable
• example If I call many RPCs, some of them fails
  (but sometimes all will done)
• not yet resolved
• GT2? Ninf-G? OS? Hardware?
• Other instability
• System maintenance (incl. version up of software)
  without notification
• realized when the application would fail.
• it worked well yesterday, but Im not sure
  whether it works today
• But this is the Grid ?

48
Observations

• Still being a grass roots organization
• Less administrative formality
• cf. PRAGMA, APAN, APEC/TEL, etc.
• Difficulty in establishing collaboration with
  others
• Unclear membership rules
• Join/leave, membership levels
• Rights/Obligations
• Vague mission, but already collected
  (potentially) large computing resources

49
Observations (contd)

• Duplication of efforts on similar activities
• Organization-wise
• APAN - participation by country
• PRAGMA most organizations are overlapped
• Operation-wise
• ApGrid testbed vs PRAGMA-resource
• may cause confusion
• technically, the same approach
• Multi-grid federation
• Network-wise
• Primary APAN TransPAC
• Skillful engineering team

50
Summary of current status

• Difficulties are caused by not technical problems
  but sociological/political problems
• Each site has its own policy
• account management
• firewalls
• trusted CAs
• Differences in interests
• Application, middleware, networking, etc.
• Differences in culture, language, etc.
• Human interaction is very important

51
Summary of current status (contd)

• Activities at the GGF
• Production Grid Management RG
• Draft a Case Study Document (ApGrid Testbed)
• Groups in the Security Area
• Policy Management Authority RG (not yet approved)
• Discuss with representatives from DOE Science
  Grid, NASA IPG, EUDG, etc.
• Federation/publishing of CAs (will kick off)
• Ill be one of co-chairs

52
Summary of current status (contd)

• What has been done?
• Resource sharing between more than 10 sites
  (853cpus are used by Ninf-G application)
• Use GT2 as a common software
• What hasnt?
• Formalize how to use the Grid Testbed
• I could use, but it is difficult for others
• I was given an account at each site by personal
  communication
• Provide documentation
• Keep the testbed stable
• Develop management tools
• Browse information
• CA/Cert. management

53
Future Plan

• Draft Asia Pacific Grid Middleware Deployment
  Guide, which is a recommendation document for
  deployment of Grid middleware
• Minimum requirements
• Configuration
• Draft Instruction of Grid Operation in the Asia
  Pacific Region, which guides how to run Grid
  Operation Center to support management of stable
  Grid testbed.

54
Future Plan (contd)

• Should think about GT3/GT4-based Grid Testbed
• Each CA must provide CP/CPS
• International Collaboration
• TeraGrid, UK eScience, EUDG, etc.
• Run more applications to evaluate feasibility of
  Grid
• large-scale cluster fat link
• many small cluster thin link

55
Grid PSE Builder
56
Overview of Grid ASP

• Grid ASP provides users PSE( Grid services )
• Portal system hides Grid environment from users

GridASP (Grid Service Provider)
User

• PSE components
• (Grid Services)
• Application service
• Storage service
• Computing service
• DB service

Main service is a batch job
57
Activities _at_ GTRC, AIST

• Software toolkit for constructing portal
• Grid PSE Builder (GridLib)
• Grid application portals
• ISV software( Gaussian, Phoenics, ... )
• user programming application
• Experiment of Grid ASP (planning)
• Feasibility study with real business players

58
Overview of the Grid PSE Builder

• Framework for building an application portal on a
  grid environment

Globus Toolkit 2.x (MDS, GRAM, GSI)
59
Single Sign On / Session Manager
Delegation using (JWS/Applet)
client auth.
AIST GridLib Portal
SignOn/SignOff Job Control submission/query /cance
l
globusrun
Job Queuing Manager Signing Server
Accounting DB (Postgress)
accounting information
60
PSE Component information

• Interface of application
• XML-based Web page description language
• Application name, location ...
• Contents
• Arguments (input parameters, ..)
• Options

61
(No Transcript)
62
Climate Simulation on theTeraGrid/ApGrid/PRAGMA
TestbedDemonstration at SC2003
63
Demo Summary

• Application Climate Simulation
• Short- to Middle- term climate simulation
• Barotropic S-Model
• Portal Grid PSE Builder
• Visit AIST Booth 739 for more detail
• Middleware used for the implementation of
  Grid-enabled climate simulation Ninf-G V2
  (alpha)
• GridRPC middleware based on the Globus Toolkit
  which is used for gridifying the original
  (sequential) application
• Resources ApGrid/TeraGrid Testbed (500cpu)
• NCSA (225cpu), AIST (50cpu), TITECH (200cpu),
  KISTI (25cpu)

64
Why the climate simulation?

• Climate simulation is used as a test application
  to evaluate progress of resource sharing between
  ApGrid/PRAGMA institutions
• We can confirm achievements of
• Globus-level resource sharing
• Globus is correctly installed
• Mutual authentication based on GSI
• High-level Middleware (GridRPC) level resource
  sharing
• JobManager works well
• Network configuration of the cluster(note that
  most clusters use private IP addresses)

65
Application Climate Simulation

• Goal
• Short- to Middle- term, global climate simulation
• Winding of Jet-Stream
• Blocking phenomenon of high atmospheric pressure
• Barotropic S-Model
• Climate simulation model proposed by Prof. Tanaka
  (U. of Tsukuba)
• Simple and precise
• Modeling complicated 3D turbulence as a
  horizontal one
• Keep high precision over long periods
• Taking a statistical ensemble mean
• several 100 simulations
• Introducing perturbation at every time step
• Typical parameter survey

66
Ninfy the original (seq.) climate simulation

• Dividing a program into two parts as a
  client-server system
• Client
• Pre-processing reading input data
• Post-processing averaging results of ensembles
• Server
• climate simulation, visualize

S-model Program
Reading data
Solving Equations
Solving Equations
Solving Equations
Averaging results
VIsualize
67

• ApGrid / PRAGMA Testbed
• 10 countries
• 21 organizations
• 22 clusters
• 853 CPUs

68
Behavior of the System
Severs NCSA Cluster (225 CPU)
Ninf-G
Client (AIST)
Severs AIST Cluster (50 CPU) Titech Cluster (200
CPU) KISTI Cluster (25 CPU)
69
Preliminary Evaluation

• Testbed 500 CPU
• TeraGrid 225 CPU (NCSA)
• ApGrid 275 CPU (AIST, TITECH, KISTI)
• Ran 1000 Simulations
• 1 simulation 12 seconds
• 1000 simulation 12000 seconds 3 hours 20
  min(if runs on a single PC)
• Results
• 150 seconds 2.5 min
• Insights
• Confirm application-level resource sharing among
  21 sites
• Ninf-G2 efficiently works on large-scale cluster
  of cluster
• Ninf-G2 provides good performance for fine grain
  task-parallel applications on large-scale Grid.

70
Univ. of Hong Kong OPEN Campus, Oct 18, 2003
71
Other Activities
72
QC Grid / Gaussian Portal

• Web based User-Interface
• Input Analyzer
• CPU time estimation
• Results reference (Quick and Detailed)
• Database and Archives
• Knowledge DB
• Input and Result Archives
• Resource allocation scheduler

73
Goal and feature of Grid Datafarm

• Goal
• Dependable data sharing among multiple
  organizations
• High-speed data access, High-speed data
  processing
• Grid Datafarm
• Grid File System Global dependable virtual file
  system
• Integrates CPU storage
• Global parallel and distributed processing
• Features
• Secured based on Grid Security Infrastructure
• From small scale to world wide scale depending
  the data size and usage scenarios
• Data location transparent data access
• Automatic and transparent replica access for
  fault tolerance
• High-performance data access and processing by
  accessing multiple dispersed storages in parallel

Source Osamu Tatebe
74

• Efficient use around the peak rate in long fat
  networks
• IFG (interframe gap)-based precise pacing (using
  large input
• buffer 16 Mb) with GNET-1 -gt packet loss free
  network
• Stable network flow even with HighSpeed TCP

75
P3 Personal Power Plant

• Middleware for distributed computation
• Traditional goals
• Cycle scavenging
• Harvest compute power of existing PCs.
• Internet-wide distributed computing
• E.g. distributed.net, SETI_at_home
• Challenging goals
• Aggregate PCs and expose them as an integrated
  Grid resource.
• Integrate P3 with Grid middleware ?
• Circulation of computational resources
• Transfer individual resources (C2C, C2B) and also
  aggregated resources (B2B).
• Commercial dealings need a market and a system
  supporting it.

Conventional dist. computing
Transfer and aggregation of individual resources
76
For more Info

• http//www.aist.go.jp/
• http//unit.aist.go.jp/grid/
• http//ninf.apgrid.org/
• http//www.apgrid.org/
• yoshio.tanaka_at_aist.go.jp