SPAWAR Systems Center San Diego Unmanned Systems Branch Code 2371

1
SPAWAR Systems CenterSan DiegoUnmanned Systems
BranchCode 2371
2
Advances in Autonomous Obstacle Avoidance for
Unmanned Surface Vehicles

• SPAWAR Systems Center San Diego
• Unmanned Systems Branch
• Mike Bruch
• Jacoby Larson

3
OA in Marine Environment
4
USV OA Software Architecture
Global Pose
MOCU (Operator interface at host ship)
Contacts and radar image
Chart Legend
Wireless
Heading and vel. commands
Navigator, Path Planner, Chart Server, Radar
Server, Reactive OA, MOCU
Tele-op commands
Wired
Initial route
Modified route
Planned
Path Planner
Navigator (Global Waypoint Driver)
Global Vector Driver
Off-board
Route handshaking
On-board
Logical switch
Contacts
Big Charts
Is Reactive Component on and running?
0.33 Hz
Low rate
Chart Server
Stereo Vision
Radar Server
Planned Sensors
Vel and turn rate commands
10 Hz
Obstacle map
2 Hz
10 Hz
Small Charts
Small image
Monocular vision
Reactive OA
Driver
MMW radar
Actuator cmds
Tele-op cmds
LADAR
5
World Model

• 2-D occupancy grid obstacle map
• Two levels of abstraction
• Deliberative Map (far-field)
• Digital nautical charts (DNC)
• Automated radar plotting aid (ARPA) contacts
• Automatic identification system (AIS) contacts
• Future Bathymetry data
• Future Host ship or shore side contacts
• Reactive Map (near-field)
• Radar
• Stereo vision
• Monocular vision
• DNC

6
USV OA Requirements

• Plans around stationary and moving obstacles
• Minimal changes to the original route
• Fast (real-time)
• Operator has view and control of route at all
  times
• Follows navigation rules of the road

7
SSC SD Previous Work

• Far-field path planning to avoid stationary and
  moving obstacles
• Nautical chart data
• ARPA contacts
• Initial development to follow rules of the road
• Reactive OA for a ground vehicle platform

8
Deliberative OA Path Planning

• A search basis
• Cost-focused exploration of grid space
• Costs include path distance and proximity to
  obstacles
• Obstacle-proximity cost variable provides means
  for setting a safety barrier around obstacles
  different obstacles may have different safety
  barrier distances
• Extendable to other costs (direction, shipping
  lanes, soft obstacles, route ETE, etc.)
• Path planning example northeast of Stockholm
  Sweden
• 30m resolution grid

9
DNC - Data Conversion

• NGA Digital Nautical Charts
• Embedded chart extraction and conversion on USV
• Vector Product Format (VPF) data converted to an
  occupancy grid map
• Coded according to type of obstacle
• Data parsed to extract only true obstacles

10
DNC - Map Resolution

• VPF allows maps of varying resolutions

50x50
30x30
10x10
5x5
1x1
11
DNC - Feature Validation

• Parse through feature attributes and remove
  features that pose no serious obstacle threat

No feature validation
Feature validation
12
DNC - Data Overlap

• Merge overlapping and inconsistent harbor and
  approach data

Harbor and approach data overlap
No overlap
13
Maintain User-defined Route

• Maintain user-defined route unless obstructed
• Path of route is generally important
• Revert back to users route if obstruction no
  longer exists
• Contact may slow or reverse direction or not even
  exist

14
Rules of the Road COLREGS

• Navigation rules as defined in 1972 International
  Regulations for Preventing Collisions at Sea (72
  COLREGS)
• Overtaking the passing vessel shall pass on the
  port side of other vessel
• Meeting both vessels shall alter course to
  starboard so that each shall pass on the port
  side of the other
• Crossing the vessel that has the other on her
  starboard side shall keep out of the way and
  avoid crossing in front of the other vessel
• Rules are vague for angles and ranges for which
  they apply

Overtaking Meeting Crossing
15
Rules of the Road Detection and Avoidance

• Detecting COLREGS violations point and time of
  closest approach to each radar contact
• Compute heading for USV and contact
• Solve for time of closest approach
• Compare headings and bearings at closest approach
  to determine if any rules are violated
• Algorithms used to follow COLREGS
• A
• Markov Chains Monte Carlo (MCMC)
• Uses a statistical model pseudo random sampling
  rule that keeps states better than the last and
  avoids local minima

16
Rules of the Road Projected Obstacle Areas

• Increase or decrease the projected obstacle area
  of a moving obstacle to bias the A planner to
  mimic rules of the road

Average POA
Increase Port Angle
Increase Starboard Angle
Increase Ahead Distance
Overtaking Meeting Crossing
Increase Astern Distance
17
Rules of the Road A

• Pass port-to-port when meeting head-to-head
• Follow direction of traffic flow
• Give right-of-way to other vehicles

18
Rules of the Road MCMC
19
Target Tracking

• Uses deliberative path planner to chart course
  and velocity to track the target while avoid
  obstacles
• Future add behavior to reactive OA to complete
  final action
• Pull up on the port or starboard side of vessel
• Cut-off and stop target
• Trail behind

20
Radar ARPA Contacts

• Networked marine radar
• Xenex controller with a stock Furuno antenna
• New Xenex controller on order-2008
• Provides ARPA contacts
• Position, speed and course of up to 100 contacts
• Issues
• Data corrupted and contacts lost when USV turns
  at moderate rate
• Time to acquire small boats
• False contacts from shore
• Contact Acquire time

21
Radar Eliminate False Contacts From Shoreline
Contacts allowed outside zone
Contacts excluded from this zone
Zone boundary
22
Radar Contact Acquire Time
23
Real World Path Planning Example
24
Reactive OA

• Real-time trajectory modification
• Modifies throttle and steering commands at the
  same rate as the navigation system
• Common occupancy grid map
• Sensor data is fused into a common data space
• Behavior based
• Path following, OA, target tracking, etc.
• Very easy to add new sensors and new behaviors
• Operates in arc space
• Selects best arc each cycle
• Arcs are defined by a speed and turn-rate
• Given the desired arc and speed a required
  turn-rate is calculated

25
Reactive OA

• Loosely based on CMU Morphin algorithm and the
  Distributed Architecture for Mobile Navigation
  (DAMN)
• Distributed behavior based system
• Multiple behaviors vote on desired actions
• Votes scaled from 1 to -1
• Obstacle avoidance behaviors vote for or against
  a fixed set of arcs
• Arcs translate to vehicle speed and turn rate

26
Reactive OA

• OA behavior votes against (0 to -1) arcs that are
  blocked
• Vote is determined by the distance the vehicle
  could travel along that arc before it encountered
  an obstacle
• Path following behavior votes for arcs (0 to 1)
  that are nearest the arc calculated by the
  waypoint navigation routine

27
Reactive OA

• OA behaviors and arbiter
• Arbiter combines weighted votes from all
  behaviors
• Arc with highest vote is selected and used to set
  the velocity and turn rate for that iteration

28
Reactive OA Sensors

• DNC data
• Raw radar
• Stereovision
• Monocular vision
• LADAR

29
Reactive OA SensorsRaw Radar

• Raw radar data from the USV radar server
• 10Hz update rate
• Small section of the radar image oriented heading
  up for the USV
• Dead reckoned between radar updates
• Converted from polar grid to Cartesian grid
• Image processing performed to eliminate noise and
  extract useful data from center disk

30
Reactive OA SensorsRaw Radar Conversion to
Obstacle Map
Raw Radar image
Obstacle map centered on USV
31
Reactive OA SensorsRaw Radar Center Disk
After Filtering
Before Filtering
Buoy inside center disk
32
Reactive OA SensorsStereo Vision

• Leveraging our work with the NASA Jet Propulsion
  Laboratory on stereo vision for our UGVs
• Collected stereo data on our USV at two different
  baselines
• Initial results look promising

Provided by the NASA Jet Propulsion Laboratory
Provided by the NASA Jet Propulsion Laboratory
33
Reactive OA SensorsStereo Vision other uses

• Stereo data allows calculation of the plane of
  the water surface
• This allows us to find the infinite horizon line
  in the image
• The horizon line allows us to estimate range to
  obstacles detected from RGB data only
• It also enables image stabilization

Provided by the NASA Jet Propulsion Laboratory
34
Reactive OA SensorsMonocular Vision

• Detect obstacle on the water with a single camera
• Color and texture segmentation, optical flow,
  etc.
• Investigating both color and IR cameras
• Detect horizon line
• Can obtain a rough estimate of range by
  determining how far below the horizon an obstacle
  appears in an image

35
Reactive OA SensorsMonocular Vision

• Early results
• 10 weeks of effort
• Horizon detection in presence of landmass
• Optical flow used to segment objects on water

36
Reactive OA SensorsMonocular Vision

• Recent work includes range and bearing
  calculations

37
Reactive OA World Model

• Limited field of view of sensors loses obstacles
  as vehicle rotates
• World Model stores obstacles with a value based
  on the length of time it is seen by the sensors
• Also allows for increased filtering only show
  obstacle if it has been seen 3 times, etc.

Path following without world model
Path following with world model
38
Conclusions

• Deliberative and Reactive techniques provide a
  robust OA solution
• Tested in a real-world environment
• Sensor systems still need to be refined
• Working with radar manufacture to improve
  performance
• Developing more robust vision based obstacle
  detection techniques for small obstacles

39

• Questions?
• bruch_at_spawar.navy.mil
• www.spawar.navy.mil/robots/

40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
Toolbar Release
Return to Monitor Mode
USV Status Summary
Overview chart
49
(No Transcript)
50
(No Transcript)
51

52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
(No Transcript)
56
(No Transcript)
57
(No Transcript)
58

• Elevation Map

• Left Image

• Range Map

59
(No Transcript)
60
more
61
(No Transcript)