Machine Based Visual Fish Identification or One fish, two fish, red fish, blue fish

1
Machine Based Visual Fish IdentificationorOne
fish, two fish, red fish, blue fish

• By Eric S. Davis
• Committee
• Dr. Roth
• Dr. Nance
• Dr. Bueler

2
This talk

• The history leading to the Digital Observer
• An overview of the Digital Observer
• My role in the project
• Current results
• Future deadlines and objectives

3
The history

• Sometimes difficult to obtain accurate
  catch/bycatch statistics from fishermen.
• To obtain more objective data, independent
  observers were included to do sampling.
• The two estimations often do not match.
• Observers are in short supply, expensive, take up
  room, and give a real warm Big Brother feel to
  things.

4
The idea

• Provide a computerized system for counting and
  classifying fish harvests.
• Not in the way of workers.
• Cheaper than human observers.
• Easier to reproduce than human observers.
• Secure from elements and tampering of data.
• Records only fishing data and can do that 24
  hours a day.

5
Prior Art

• Canadian system from the mast that gave periodic
  images from mast stamped with GPS and time data.
• Another European system was able to identify fish
  species under heavily controlled lighting and
  environment conditions.

6
New Constraints

• Needed to be closer to the action to record
  incoming fish.
• Cannot cause excessive interference with
  harvesting operations
• Little lighting control(outside)
• Little environment control(maximum control is in
  the chute).

7
Apparatus Conditions
8
Digital Observer Overview
Fish Outline
Image Capture
Segmentation
Measure Fish
Identify Species
Fish Metrics
Fish Image
Fish Metrics
Fish Image
Fish Species
Data Storage
9
Image capture

• Currently done from standard video stream by a
  camera over the chute and a frame grabber.
• Trying to get multiple images of each fish.

10
Segmentation

• Trying to separate the image of the fish from the
  background, line, etc.
• The main part of the project that I will be
  working on.
• One of three segmentation approaches being
  tested.

11
Metrics

• Calculate fish dimensions to send to the neural
  network for identification.

12
Neural Network identification

• Uses fish metrics to determine fish species.
• Yep, thats what it does.
• Sure enough.

13
Current Group Status

• A prototype form of each component has already
  been constructed.
• Works at a little more than 80 accuracy for
  limited in-the-chute test data.

14
New worries

• Variable lighting- Needs to work reliably in
  variable conditions- cloudy, sunny, even at
  night.
• Over the side data(Loss of all environment
  control).
• Both worries have to do with segmentation.

15
Extra conditions-The fish whacker.
16
Over the side data

• Improves counts of fish actually caught in the
  lines instead of those hauled and kept on the
  boat.
• Lose any control of picture elements- no choice
  of background along with little light control. A
  whole extra level of difficulty.
• Need a breakthrough in segmentation to make this
  possible(and possibly improve current software).

17
My task

• Evaluate and implement a segmentation algorithm
  for the final product.
• Necessary documentation(segmentation and
  project).
• Evaluation of LEGION

18
LEGION

• An oscillator based neural network that has shown
  some amazing results in difficult segmentation
  problems.
• Consists of a smoothing algorithm followed by the
  oscillator network algorithm.
• Must evaluate its feasibility for this particular
  project, and fully implement/integrate the final
  segmentation algorithm to meet the product
  requirements.

19
LEGION uses

• Currently implementing against the chute data to
  gain familiarity with the project and see basic
  feasibility of LEGION.
• If successful, will try LEGION to approach the
  over-the-side problem.

20
Legion- current results.

• Very recently been able to run and interpret some
  of the results from the LEGION algorithm.
• Currently too slow for the Observer requirements,
  but will be looking at optimizations that could
  be performed to improve the speed of the code.

21
LEGION results
Original image
Segmented image
22
Segment This!!!Next task
23
LEGION evaluation success?

• LEGION must produce segmentation that is at least
  as good as current segmentation algorithm.
• Must do it in a reasonable time(lt15 sec per frame
  not optimized. lt 3 seconds per frame optimized. )
• Segmentation must be at least as stable as
  current algorithm in varying conditions.

24
Approximate Dates

• Firmer dates in 2 weeks.
• Have LEGION evaluation completed in under 4
  weeks.
• Finish with whole project by the beginning of
  September.

25
Questions