Automatically Tracking and Analyzing the Behavior of Live Insect Colonies

1
Automatically Tracking and Analyzing the Behavior
of Live Insect Colonies

• Tucker Balch, Zia Khan, and Manuela Veloso

2
Introduction

• Behavior of social insects like ants is a source
  of inspiration for computer scientists
• relates to multi-agent systems and robotics
• Ant Algorithms used in network routing systems,
  robot navigation, and scheduling
• Most of this research extends computer science to
  the study of biology

3
Goals

• Full automation of
• simultaneous tracking of multiple ants
• recognition of individual and colony behaviors
• acquisition of single and multi-agent behavior
  models
• application of models to software and robotics

4
Goals

• This research contributes
• New observing and tracking algorithms for
  developing multi-agent modeling tools
• Advanced knowledge of social insect behavior
• Progress towards the goals
• setting up ant colonies for automated observation
• machine vision algorithms for simultaneous
  tracking of multiple moving animals
• new methods of analyzing spatial behavior in the
  colony

5
Ant Society

• At least one queen, and act cooperatively but
  there is no leader
• Aggregate behavior stems from chemical cues,
  physical contact between ants and environmental
  pressures
• Queen is fertilized before leaving home and
  establishes a new colony
• can live for 20 years as regular ants live for 1

6
Ant Society

• Workers have different jobs based on age and
  morphology (Figure 1)
• Minor much nursing, little nest work, little
  foraging
• Media little nursing and nest work, much
  foraging
• Major little nursing, much defense
• Myrmecologists developed a method called an
  ethogram to model ant behavior
• similar to a Markov Process in csci

7
Barriers to Research

• Field
• 1 or more observers staring at 1 or more colonies
  with a pencil and papers
• must not miss anything of importance
• might be at site from dusk till dawn for weeks
• Solution
• laboratory research with videotapes
• still requires a person to operate

8
Research

• Uses image processing techniques to track ants
• Setup (Figures 4 and 5)
• 10 total colonies of 2 species of ants
• each nest housed in open test tubes mounted in a
  petri dish that is in a10 gallon aquarium
• food located towards the left
• Video camera captures entire tank
• 640 x 480 pixels at 30 Hz processed in real time

9
Finding the Ants

• Problems
• ants are quite small
• dark color of ants conflicts with food, waste,
  shadows etc.
• dark areas in image are noisy
• Solution
• classify by color, then movement
• color identification is fast and highlights areas
  that require further study (slower)

10
Color Tracking

• Pixels that match predetermined colors are
  grouped into bounding boxes for study
• CMVision algorithm used for color classification
  (fast and reliable)
• Great for marked objects, but quite
  difficult/dangerous to mark ants
• Best to use this for colors and something else
  for motion

11
Movement Tracking

• Pixels in current image are compared with the
  same location in the previous image
• If the difference is greater than some value, the
  pixel is said to have moved
• called frame differencing
• Drawbacks
• if object is uniform in color, only edges will
  seem to have moved
• objects must be continually moving, why?

12
Adaptive Background Subtraction

• Background (scene without motion) is found
• Bij (a 1)Bij aIij
• B is the pixel at (i, j) a running average is
  made
• I is the pixel in the current image
• a is the leaning rate when it is low, new
  objects only become part of the scene if they
  remain for a long time
• 0.0005 used here

13
ABS (cont.)

• Finally, the values of the pixels matching the
  given ant color are subtracted from the
  background
• If value is greater than a given threshold, (35
  here) the pixel is labeled moving ant color
• Connected pixels of moving ant color are grouped
  into regions
• if the region is large enough, it is said to
  contain an ant

14
Individual Motion Tracking

• Easier to do with robots than living things
• Solutions
• compute minimum distance under the assumption
  that the ant cannot move past a given circle in
  consecutive video frames
• algorithm is greedy and can cause errors
• Second method
• algorithm to generate all possible sets of
  observed matching points between frames
• calculates the total fitness of each set with an
  equation
• still faulty, but better

15
Spatial Activity

• 2-d arena is divided into bins
• bin value is incremented when ant enters and 3-d
  model is created (height of visits)
• Experiments (with and w/o food)
• without arena fairly even, high around dish and
  arena edge
• ants may use walls for navigation references
• with spike at food, and at nest entrance, not
  around nest
• less interested in exploring when known food
  location is available
• ants interact at entrance after gathering food

16
Multiple ants

• The system was successful in tracking multiple
  (4-6) ants at once as shown in Figure 11

17
Accuracy and Efficiency

• To test accuracy, a human observer counted the
  number of ants at a given time and compared that
  to the system
• usually 10-11 present, system off by 1.2
• (11 error)
• Efficiency
• 42ms to capture and process an image into a log
  file
• at 10 frames a second, uses only 35 of CPU
• 3MB/hr, 20 days of data can be stored on 1 CD-ROM
• scales well, works for even 100 animals
• color searching predominates and ants are very
  small

18
Limitations

• Occlusion
• by the walls of the petri dish
• solved by using memory of tracked objects
• Clumping
• ants may be on top of one another and
  indistinguishable
• Splitting
• perhaps reflections may cause bounding boxes to
  split and more ants than normal are counted
• solved by evaluating the size of the bounding
  boxes
• Ant Behavior
• if ants remain motionless (can take 15 minutes
  depending on a) they become part of the
  background
• if ants move dark objects, those may also be seen
  as ants

19
Conclusion

• Successful application of computer science to
  biology
• Using a new computer vision algorithm
• reliable and accurate
• Learn more about multi-agent models