Infrared Thermal Camera-Based Real-Time Identification and Tracking of Large Animals to Prevent Animal-Vehicle Crashes(AVCs) On Roadways

1
Infrared Thermal Camera-Based Real-Time
Identification and Tracking of Large Animals to
Prevent Animal-Vehicle Crashes(AVCs) On Roadways

• Debao Zhou
• Department of MIE, UMD
• November 8, 2020

2
Objective

• To build a system on roadside to automatically
  detect the presence of large animals, such as
  deer, and track their motions for the safe
  driving on roadways
• This system can also be installed on cars

3
The Problem Where Are Animals
Animals off road
Cars at low/high speed
Animals off road
Animals on road
4
The Problem Road Conditions

• Tree and/or grass,
• Straight or curved
• Hill or ditch

Road
Road
Tree
Tree
Tree
Tree
Road
Grass
Grass
Hills
or ditches
Case 1
Case 2
Case 4
5
Thermal Camera Capability

• Temperature sensitivity up to 14 mK
• Optical resolution up to 640 by 480 TV Lines
• Activity detection of adult human size objects up
  to 8 Km with motorized focal length adjustment
• Active IR thermal camera and passive IR thermal
  camera

6
Example

• Thermal-Eye 3620AS camera core
• FOV 17?
• Range to detect human activity 1000ft (305m)
• Real-time 30fps
• -20C to 85C
• 3500

7
Technical Approach

• Through the processing of infrared thermal images
  to detect and track big animals
• Speed method will be used to distinguish big
  animals from their environment and high speed
  cars
• Pattern matching method will be used to identify
  big animals from low speed cars
• Centroid method will be used to track the motion
  of big animals
• A mechatronic system with two degrees of freedom
  will be developed to realize the scanning and
  tracking by making the camera point to a specific
  animal

8
Advantages

• Advantages
• Automatic detection and tracking
• All-time day and night
• Coverage area much bigger than break-the-beam
  method
• Usage
• On-vehicle
• Roadside for infrastructure

9
Accomplishments To Date

• Infrared thermal images have been acquired and
  analyzed. The results shows that these images can
  be used to identify the presence of big animals
  and cars.
• Pattern matching method has been applied to
  identify the presence of cars and deer. The
  algorithm is capable to tell the object is a deer
  or a car.
• Tracking has been realized using two degrees of
  freedom system.
• A prototype has been built and the scanning and
  tracking have been realized.

10
System Prototype

• Camera and image grabbing
• Stepper motors
• Enclosure Contain power supply and motion
  controller
• Computer

11
Identification and Tracking Realization
Image acquisition

• Three main phases
• Image acquisition
• Image processing
• Motion control

Image processing
?
Image center

Pixel to step adjustment
Motion control
Motion generation
Position measurement
12
Motion Control and Realization

• Motion realization

13
Object Identification and Tracking

• Through Centroid and its changing locations

14
Pattern Matching Car and Big-animals

• Original Patterns

• Original Images

15
Object Identification

• Matched pattern

• Processed Images

16
Object Identification and Tracking in Light
In light
Three cameras are used 1 Thermal camera to
take the thermal image and put on computer
screen 2 Normal digital camera to take video
of the whole scenario (video on left) 3
Normal digital camera to take video of the
computer screen (video on right)
17
Object Identification and Tracking in Dark
In dark, no lights
Three cameras are used 1 Thermal camera to
take the thermal image and put on computer
screen 2 Normal digital camera to take video
of the whole scenario (video on left) 3
Normal digital camera to take video of the
computer screen (video on right)
18
Proposed System Setup
Animals off road
Cars at low/high speed
Animals off road
Animals on road
19
Proposed Study

• Speed measurement
• 3D distance measurement and distance change with
  time
• Off-road big-animal identification from
  stand-still objects
• On-road high-speed vehicles identification
• Development of pattern matching algorithms
• On-road low speed vehicles identification - tell
  cars from any big animals when car moves in slow
  speed
• Multiple objects tracking
• Building robust system electrical, mechanical,
  suitable for roadside application, low-high
  temperature, etc.

20

• Thanks!

21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
Review On-vehicle / Roadside

• Natural habitat prevention
• Roadside vegetation control
• Electronic mats
• Population reduction
• Fencing
• Deer guards and crosswalks
• Underpasses and overpasses
• Roadside reflectors
• Animal reflectors
• Flash light sensing
• IR LASER

• The Driver
• Whistles
• TH-HID Lighting System
• Forward-collision Sensors
• Ultrasonic Sensors
• Radar
• Camera
• Auto-identification of humans
• in the environment,

25
Animal Detection System Installed Worldwide
Area-cover systems and Break-the-beam systems
Location Distance Cost Date(s)
7 locations Switzerland 50-200m 11,500 1993
Box Uusimaa Finland 220 m 100,000 1996
Mikkeli Finland 90 m 70,000 1999
2 locations in Germany Unknown 20,000 1998
2 locations Netherlands 200-250 m / 50,000 1999
Rosvik Sweden 100 m / 30,000 1999
Colville WA USA 402 m 12,000 2000-2002
Nugget Canyon WY USA 92 m 200,000 Dec 2000 - May 2001
Sequim WA USA 400 m / 73,000 Apr 2000 - present
Marshall MN USA 200 m / 57,000 Jun 2001- Nov 2001
Kootenay NP BC CAN 1000 m Unknown Jun 2002- Oct 2003
Indiana Toll Road IN USA 1609 m 1,300,000 Apr 2002
Wenatchee WA USA 213 m 40,000 Oct 2002- spring 2004
Yellowstone NP MT USA 1609 m 409,000 Oct/Nov 2002
Los Alamos NM USA 30 m 2,500 Nov 2002 - Feb 2003
Thompson-town PA USA 91 m / 220,000 May 2004 - Jan 2005
Herbertville Quebec CAN 10 m / 8,200 Fall 2004
26
IR System and Warning Signs in Netherlands
27
System in Calville, WA, USA Break-the-beam
System

• 20 June 2000
• Hwy 395 north of Spokane, south of Colville
• Two lasers, one placed on each side of the road
• Two standard deer warning signs, two smaller
  rectangular signs that read When Flashing, and
  two solar powered red flashing beacons
• Lasers operated on batteries
• Problems
• Clear line-of-sight in the right-of-way
• Heat in the box

28
FLASH System in Nugget Canyon WY USA, IR Sensor
(17m-19m)

• Signs were placed 300 m (327 yd) before the
  crossing area
• Sing read

• Flashing Light Animal Sensing Host (FLASH)
• Series of infrared sensors placed at 17-19 m
  (18.5-20.7 yd) intervals, HW 30 spanned 92m
• Body heat of large animals

29
System in Yellowstone National Park, Montana USA
(Break-the-beam)

• Oct./Nov. 2002
• Along a 1,609 m (1 mi) long road section of US
  Hwy 191
• Transmitter microwave RF signal (35.5 GHz) to
  its intended Receiver
• Once actived, master station then sends the
  beacon-on command to the nearest beacons

30
System in Yellowstone National Park, Montana USA
(Break-the-beam)
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34

• Cannot detect the distance/speed of a moving
  object
• Cannot distinguish moving animals from moving
  cars
• Cannot distinguish sun-heated objects from
  big-animals
• Cannot be able to tell that a fast running animal
  will enter the road.
• Intrusion detection rate is not acceptable
• Camrix A.L.E.R.T and Xtralis ADPRO

35
(No Transcript)
36
Infrared Cameras for Monitoring Animal Activity

• Thermacam B1 FLIR (forward-looking infrared)
  Systems Inc Boston, Massachusetts
• Trailmaster 1500 Active Infrared Trail Monitors
  By Goodson and Associates, Inc. Lenexa, Kansas
• FLIR Nightsight Palm IR 250 Digital Camera
  Raytheon, Waltham, MA
• TM-2500 Trailmonitor Grypon Engineering, Inc.
  Richmond, Michigan
• Su320ktx-1.7rt Goodrich Corporation Princeton,
  New Jersey
• Hunting Electronics / Military Usage

37
Capability of Infrared Cameras

• Distance, from up to 8 Km for the identification
  of adult human size objects
• Temperature -40F to 100F
• Resolution 0.1F

38
Component Integration
39
Object Identification

• Processed image

• Original image