Cluster and Grid Computing for RS Based Crop Growth Simulation Model

1
Cluster and Grid Computing for RS Based Crop
Growth Simulation Model
Md. Shamim Akhter Doctorate Student?AIDA
Laboratory, Tokyo Institute of Technology
(TITECH), Japan Email shamim7862002_at_yahoo.com
2
Overview of the Presentation

• Introduction
• Importance of Crop Model Parameters
  Identification
• Swap Model
• Genetic Algorithm (GA)
• SWAP Model Parameter identification Data
  Assimilation
• using RS and GA
• Practical Problem Arises
• Methodologies for Parallel or Distributed
  SWAP-GA
• Parallel SWAP-GA with Distributed Population
• Parallel SWAP-GA with Distributed Pixel
• Parallel SWAP-GA with Hybrid Model
• Results and Discussion Cluster Implementation
• Proposed GRID Implementation
• Conclusions

3
Importance of Crop Model Parameters Identification

• Agriculture Activity Monitoring
• Simulated how things happened
• Generate the pattern like rainfall pattern.
• Sowing date, Cropping intensity, Growth, Water
  stress, Yield, etc.
• Production for Food Security
• Water Management in Irrigation Activity
• Crop growth model ( SWAP )
• Continuous monitoring in various aspects
• Prediction
• Spatial Parameter estimation Calibration using
  RS

4
Importance of Crop Model Parameters
Identification(Contd)

• Data Assimilation Technique
• To estimate parameters which cannot be observed
  by RS
• Necessity of High Performance Computing
• One pixel 2 min to 30 minutes.
• Cluster, Grid Computing
• AIT Cluster Optima, Kasetsart University
  Clusters Magi, Maeka.

5
ETa

• Evapotranspiration is necessary for understanding
  how to make better decision about food security
  in irrigation systems.
• ETa tells, how much water was consumed
  effectively by irrigation system.
• High ETa means that plants are using more water
  to make food.
• ETa is possible to be observed/estimated by RS
  processing
• Large pixel- Modis /AVHRR have pixels 1000m and
  come back at least once per day.
• Small pixel- Aster/ Landsat have pixels lt 100m
  but cannot come back on the same place every day.
• Irrigation managers can have already large pixels
  ETa everyday (about, considering clouds of
  course).

6
ETa contd

• Technical design of satellite makes that small
  pixels cannot be taken everyday (besides cloud
  problems also exist)
• Means that the same satellite design restrictions
  apply to large pixels that can return more than
  once a day.
• it is technically impossible, by satellite
  platform design (and astronautics of course)
• so how to find out ETa at those impossible time
  and space resolutions?
• here it comes to fill up the blanks by running a
  model

7
Remote Sensing (RS) and ETa

• The Original RS data is proceeded into ETa by the
  SEBAL algorithm.
• However, near real time monitoring, RS cannot
  see, like cropping calendar (sowing dates and
  harvesting dates), Ground water depth, Time
  extent of crops.

8
Swap Model

• SWAP (Soil, Water, Atmosphere, and Plant) is
  equipped with crop models and water management
  modules.
• The growth and development of a crop can be
  simulated under different climatic and
  environmental conditions.
• Inside SWAP is WOFOST, simulating the daily
  growth of a specific crop, given the selected
  weather and soil data.

Adopted from Van Dam et al. (1997) Drawn by
Teerayut Horanont (AIT)
9
Swap contd

• SWAP recreates a simplified version of a crop
  growing in the field, with transfer of water and
  heat
• Some inputs are coming from Meteorological data.
• Some others are generic inputs (constants) about
  crops and soil and environmental conditions.
• Some specific inputs are variable?

10
ETa generated by SWAP Model

• Yearly Meteorological data is required to
  generate potential evapotranspiration (ETp) and
  this is calculated by Penman-Monteith equation.
• As soil dries, the model reduces ETp into ETa
  (actual ET).
• Ep component is reduced to Ea by Darcys law
  applied to soil surface and Tp into Ta using a
  water stress reduction factor.

11
SWAP Model Parameter identification - Data
Assimilation using RS and GA -
SWAP Input Parameters sowing date, soil property,
Water management etc.
RS Observation
SWAP Crop Growth Model
LAI, Evapotranspiration
LAI, Evapotranspiration
4
.
00
4
.
00
Fitting
3
.
00
3
.
00
Assimilation by finding Optimized parameters By GA

Eavpotranspiration LAI
Evapotranspiration LAI
2
.
00
2
.
00
1
.
00
1
.
00
0
.
00
0
.
00
0
45
90
135
180
225
270
315
360
0
45
90
135
180
225
270
315
360
Day Of Year
Day Of Year
RS
Model
12
Serial SWAP_GA Model Diagram
Simulated f(sawing date, water, soil property,,,,)
RS Observation
Minimize Cost Function
(

cm/d
Fxyd500 1/ Cxyd500 d/cm
Fitness Function
13
(No Transcript)
14
A Practical Problem Arises
RS image of 100 x 100 square kilometer of 100x100
pixels will take more than 0.5 years (30 min.
100 100)

• A Cluster is a group of individual, stand alone
  computers.
• Several to several thousands of CPU
• Clusters are particularly well suited to meeting
  the needs of high availability, load balancing
  and scientific computing.

15
Clusters Technical Specifications

• 2 Clusters (AIT and Kasetsart universitys
  Clusters) are being used.
• Optima 9 nodes
• Front-end Athlon XP 1800 512
  MB
• RAM 80 GB IDE Disk
• Compute Node Athlon Xp 1800 512 MB
• RAM 40 GB IDE Disk
• Disk Interconnection Fast Ethernet
• http//optima.ait.ac.th/scmsw
  eb/scms_home.html
• Magi 4 nodes
• Athlon XP 2500
  processors
• 512 MB RAM 80 GB Hard
  Disk
• Gigabit Ethernet Interface
• http//magi.cpe.ku.ac.th/scmsweb/scms_home.h
  tml

16
Implementation Schemes

• 3 Implementation Schemes have been proposed
• Method1 Distributed Population for single pixel
• Method2 Distributed Pixel
• Hybrid Model
• Population 1 chromosome1 set of parameter
• for 1 evaluation of SWAP approx 2 sec.
• 1 pixel needs 1 to several hundred thousands
  evaluation 30 minutes by 1 CPU

17
Distributed Population Method
Master equally distributed populations among
slaves and the rest populations (if exists) are
distributed to slaves sequentially through the
rank 1N slaves. Slaves do the evaluation,
generate fitness and send back the populations
(with fitness) to Master. This same procedure
will continue for all pixels.
Idle CPU
Master
Work Loaded CPU
Populations
Result Pixel
Pixel
Result
SWAP_GA Parameters 1 pixel 5 Populations 3 Slaves
Best Fitness Symbol of best Assimilation
Slave 1
Slave 2
Slave 3
18
Distributed Pixel Method
All pixels will be distributed among the
available slaves. First all pixels are equally
distributed among slaves and the rest pixels are
distributed to slaves sequentially through the
rank 1N slaves. Each slave will evaluate
total serial SWAP-GA procedure inside itself for
one pixel at a time and produce a total
assimilation result for that pixel in a file in
their local Hard Disk.
Idle CPU
Work Loaded CPU
Pixel Input
SWAP_GA Parameters 4 pixels 2 Populations 3
slaves
Population
Result File
Slave1
Slave3
Slave2
19
Hybrid (Distributed Population and Distributed
Pixel Together)
Master
Master
6 Available Nodes
Master
Master
8 Available Nodes
No of Masters sqrt ( No of available processors
) No of Slaves(No of available processors-No of
Master)/No of Masters
20
Animation Hybrid Model
Idle CPU
Work Loaded CPU
Pixel Input
Population
Master
Local Master
Result File
Slave3
Local Master
Pixel2
Pixel1
Slave1
Slave4
Slave5
Slave2
Result Pixel1
Result Pixel2
SWAP_GA Parameters 2 pixels 2 Populations 5
slaves
21
Distributed Population Time Curve
1CPU
3 Slave CPU
22
Distributed Population Time Estimation Generic
Equation
23
Distributed Population Compare Simulated Time
and Generic Equation Generated Time
Average error margin is 10
24
Distributed Pixel Time Curve
1CPU
5 Slave CPU
25
Distributed Pixel Time Estimation Generic
Equation
26
Distributed Pixel Compare Simulated Time and
Generic Equation Generated Time
Average error margin is 6.5
27
Hybrid Model Time Estimation Generic Equation
28
Hybrid Model Compare Simulation and Equation
Generate Time
Average error margin is 3
29
Limitations of Cluster Implementations

• More over our Cluster implementation SWAP-GA has
  some limitations (given below).
• Available Clusters is integrated with a few
  amount of PC (atmost 32). However, our RS image
  (363x480) holds 174, 240 pixels. Usually, to
  analyze the image a big distributed computing is
  required.
• A scheduling problem also occurs to distributed
  pixels in available slaves. To evaluate pixels
  high work loaded slaves (busy) may be requested
  again rather than idle slaves (less work loaded).
  
• At the running time of distributed SWAP-GA, any
  crash stop in any slave will fall down the whole
  system model.

30
Methodology for SWAP-GA GRID Implementation
31
Pixel Distribution Sequentially

• Grid Master will distribute all extracted pixels
  to the available Cluster master nodes.
• The number of pixels will be sent to the Cluster
  master depends on the number of available nodes
  in a Cluster.
• Cluster master then distribute the pixels to its
  available slaves for evaluation.
• Each slave will internally run the SWAP-GA,
  create the result information that will be sent
  back to Grid Master.

32
Population Distribution Sequentially

• Grid master will extract and process one pixel at
  a time.
• Distribute the number of populations to the
  available Clusters.
• Cluster Master will distribute each population to
  each slave to run the SWAP-GA and generate the
  fitness value for that particular population.

33
Pixel Distribution with MPI

• Grid Master will work similar as the previous
  one.
• The difference is that rather distributing the
  pixels inside Cluster, at a time one pixel will
  arrive in a given Cluster and the population will
  be distributed by the MPI code.

34
Conclusion

• Remote Sensing and Agriculture modeling programs
  are time consuming and require high computational
  resources.
• Cluster and Grid computing solves the
  requirements through providing computational
  resources required.