Rover Technology Infusion incl CLARAty

1
Rover Technology Infusion (incl CLARAty)

• Issa A.D. Nesnas
• Jet Propulsion Laboratory
• NASA Ames Research Center
• Carnegie Mellon
• University of Minnesota

2
Rover Technology Test beds with CLARAty

• Objectives
• Facilitate infusion of performance-enhancing
  navigation and manipulation technologies into MSL
  flight system
• Provide a flexible framework for integrating and
  comparing competing technologies on all research
  rovers Rocky8, FIDO, Rocky7, K9, and FIDO5

CLARAty Coupled Layer Architecture for Robotic
Autonomy
Funding Profile (K)
Task Manager Issa A. D. Nesnas (818)
354-9709 nesnas_at_jpl.nasa.gov Participating
Organizations JPL, Ames Research Center,
Carnegie Mellon, U. of Minnesota, RMSA
Participating Universities Facilities Rocky 8,
Rocky 7, K9, FIDO, FIDO 5, ATRVs, CLARAty test
bed, ROAMS, WITS, JPL Mars Yard
FY03-FY05 Milestones FY03 mobility and
navigation for long traverse FY04 pose
estimation, tracking, and manipulation for
instrument placement FY05 complete simulations,
onboard science and health monitoring
3
Problem Statement

• Problem
• Non-integrated robotic infrastructure for
  validation and infusion into flight
• Non-reusable and redundant robotic infrastructure
  for each rover
• Non-unified framework requiring technology
  integration for each robot
• Non-common infrastructure for infusion of
  technologies from universities
• Non-interchangeable software for Rocky 8, FIDO,
  Rocky 7, K9, ATRVs
• Absence of repository that captures domain
  expertise (JPL and other centers)
• Key Challenges
• Collaborative software development with multiple
  institutions
• Software accessibility and intellectual property
• Large legacy code bases
• Continuous working software for all platforms
• Different hardware architecture for each rover
• Large customer base users, contributors, and
  collaborators
• Rover access to remote developer
• Flexible framework for advanced research

4
Mission Relevance and State-of-the-Art

• Mission Relevance
• Enables transfer of technology to flight from a
  single integrated source
• Enables integration and validation of competing
  technologies for selection by missions
• Enables usage of several robotic platforms
• Captures competed technologies for future flight
  missions
• Makes research rovers viable test bed for flight
  algorithms
• Adapts easily to future rovers with different
  hardware architectures
• Relevant to MSL, future rover missions, and
  future robotic research
• State-of-the-art
• Separate disparate robotic software systems
• Interchangeable robotic software limited to
  high-level encapsulation
• Several efforts attempted to build common
  infrastructure DARPA (Jaus), Intel (Robotics
  Engineering Task Force)

5
Technical Approach

• Capture legacy algorithms and infrastructure
• Decompose robotics domain into subsystems
• Use domain knowledge to guide design of each
  subsystem
• Identify common infrastructure needs across and
  within subsystems
• Make all implicit assumptions explicit
• Define proper interfaces for each component
• Develop generic framework to support various
  implementations
• Adapt multiple legacy implementations to validate
  framework
• Encapsulate large components where re-factoring
  is not yet possible
• Test on multiple robotic platforms and study
  limitations
• Enable learned experience from users to feedback
  into the design
• Project potential advances to these components
• Review and update implementation
• Modify/extend/redesign to address limitations
• Provide building blocks for new development
  efforts
• After several iterations - your generic base is
  reusable

6
Statement of Work FY00-FY02

• Study various robotic architectures including
  legacy JPL/ARC/CMU
• Develop an architecture to address robotic domain
  problems
• Integrate model-based programming (Decision
  Layer) and procedural-based programming
  (Functional Layer)
• Design and implement first prototype of
  architecture
• Develop generic framework for I/O control,
  motion control, communication, locomotion,
  manipulation, and rover control
• Demonstrate encapsulation of local rover
  navigation on Rocky 8
• Demonstrate on Rocky 8 and Rocky 7 locomotion,
  navigation, pose estimation, and science
  scenarios (visiting multiple science targets)
• Develop an adaptation to FIDO rover and
  demonstrate locomotion
• Develop generic framework for vision,
  estimation, navigation, path planning
• Demonstrate long-range traverse using the
  integration of global path planning and local
  navigation on Rocky 8
• Demonstrate CLARAty locomotion with ROAMS
  simulation
• Build a CLARAty Test bed for all research rovers

7
Statement of Work FY03

• Procurements
• ATRV Jr for efficient university collaborations
• Rocky 8 Bench top II x86, K9 bench top
• CLARAty Deliverables

8
Statement of Work FY04

• Capture Technologies from competed RMSA tasks
• Integrate functionality and encapsulate with
  proper CLARAty APIs
• Deliver mature technologies to validation tasks
• Long Range Traverse
• Instrument Placement
• Autonomous Science
• Support upcoming NRA competed tasks

9
Statement of Work FY04

• Improve tools and processes to allow rapid but
  stable development
• Review and cleanup all CLARAty APIs and module
  distribution
• Provide an integrated end-to-end
  WITS/CLARAty/ROAMS for developers
• Reduce learning curve currently necessary for get
  people up to speed
• Develop procedure for 3rd parties to integrate
  software into CLARAty
• Properly refactor frequently used CLARAty modules

10
Statement of Work FY05

• Deliver integrated competed technologies based on
  MSL needs
• Continue integration of newly selected competed
  NRA tasks

11
Milestones

• FY03
• Delivery of several technologies to validation
  tasks and to MDS/MSL
• Theme I demonstrate multiple algorithms on
  multiple rovers
• Extended Kalman Filter (FIDO algorithm) on (1)
  Rocky 8, (2) FIDO, and (3) ROAMS
• Replace with pure odometry on (1) Rocky 8, (2)
  FIDO, and (3) ROAMS
• Replace with visual odometry on (1) Rocky 8 and
  (2) FIDO
• Theme II integrate multiple complex algorithms
• Morphin Navigator local D wheel odometry on
  (1) Rocky 8 and (2) FIDO
• Morphin Navigator local D visual odometry on
  (1) Rocky 8 and (2) FIDO
• Demonstrate integrated capabilities (WITS,
  CLARAty, ROAMS)
• FY02
• Demonstrate 60 m traverse using path planning and
  navigation in rough terrain
• Demonstrate locomotion onto FIDO rover platform
• Demonstrate mission-like scenario using improved
  position estimation
• Demonstrate higher fidelity interface to ROAMS
• FY01
• Demonstrate visiting multiple targets using both
  Decision and Functional Layers on Rocky 7, Rocky
  8 (in Mars Yard) and on ROAMS.
• Demonstrate reusability by running software on
  Rocky 8, K9, and Rocky 7
• FY00

12
FY03 Accomplishments
13
CLARAty Team

• NASA Ames Research Center
• Maria Bualat
• Sal Desiano
• Clay Kunz (Data Structure Lead)
• Eric Park
• Randy Sargent
• Anne Wright (Cog-E Core lead)
• Carnegie Mellon University
• David Apelfaum
• Reid Simmons (Navigation lead)
• Chris Urmson
• David Wettergreen
• University of Minnesota
• Stergios Roumeliotis
• Yukikazu Hidaka
• MIT
• Brian Williams

• Jet Propulsion Laboratory
• Max Bajracharya (34) (Cog-E Vision lead)
• Edward Barlow (34)
• Antonio Diaz Calderon (34)
• Caroline Chouinard (36)
• Gene Chalfant (34)
• Tara Estlin (36) (Deputy Manager Decision Layer
  lead)
• Erann Gat (36)
• Dan Gaines (36) (Estimation Lead)
• Mehran Gangianpour (34)
• Won Soo Kim (34) (Motion lead)
• Michael Mossey (31)
• Issa A.D. Nesnas (34) (Task Manager)
• Richard Petras (34) (Adaptation lead)
• Marsette Vona (34)
• Barry Werger (34)
• OphirTech
• Hari Das Nayar

14
A Two-Layered Architecture
THE DECISION LAYER Declarative model-based
Mission and system constraintsGlobal planning
INTERFACE Access to various levelsCommanding
and updates
THE FUNCTIONAL LAYER Object-oriented
abstractionsAutonomous behaviorBasic system
functionality
Adaptation to a system
15
Supported Platforms
K9
Linux
x86
Rocky 8
Rocky 7
Ames
VxWorks
x86
VxWorks
ppc
JPL
JPL
FIDO
FIDO
ROAMS
ATRV
x86
VxWorks
Linux
Linux
x86
JPL
CMU
JPL
16
Collaborations
RTD
CICT
MTP
MDS Lai
MIT Williams
ALERT Matthies
Adv. Avionics Bolotin
CMU Glymour/Ramsey
CLEaR Estlin/Fisher
Instr. Placement Pedersen
New NRA Competed Tasks
Craters Cheng
CLARAty
OASIS Castano, Judd
RMSA Competed Tasks
Validation Tasks
U. Washington Olson
Inst. Placement Kim
CMU Stentz
Jet Propulsion Lab
Long Traverse Huntsberger
U. Michigan Borenstein
NASA ARC
JPL Nesnas
Auto. Science Gat
CMU
U. Survey Tunstel
JPL Matthies
U. Minnesota
ARC Roush
WITS Backes
ROAMS Jain
ARC Dearden
Science Sim M. Lee
MIT Dubowsky
Ohio State U. Li
17
General Highlights

• Two-day CLARAty Workshop Jan 2003
• Attended by 36 individuals from 4 institutions
• First face-to-face meeting for entire team
• Included management and technical presentations,
  discussion topics, users feedback, and tutorials
• ITAR
• CLARAty was cleared by State Department for ITAR
  and was classified by Commerce Department
  enabling most foreign countries access to CLARAty
• Distribution
• Worked with General Counsels and International
  Affairs office to setup a process to enable
  access to most foreign nationals
• IT
• Working with JPL institutional IT to
• Simplify process to allow collaborator access to
  CLARAty
• Improve JPL infrastructure to no longer require
  local JPL accounts

18
General Highlights

• Robotic Engineering Task Force (RETF) and Intel
• Participated in Intels Workshop on Robotics
• Hosted Intel for a day of interactions on RETF
  collaboration
• Invited CLARAty paper to IROS
• ASTEP (Astrobiology Science and Technology for
  Exploring Planets)
• CMU plans to use CLARAty now that ITAR has
  cleared
• ARC currently using CLARAty for ASTEP work
• Information and Processes
• New CLARAty site with comprehensive information
• Overview objectives, publications,
  presentations, movies
• Project team, working groups, schedule,
  milestones, meetings, access process,
  intellectual property,
• Software - FAQ, procedures, software modules,
  documentation, conventions, development tools,
• Hardware concepts, technology, data sheets,
  upgrades
• Test bed test targets, build targets, hosts,
  vxWorks, FAQ and sign up

19
Is CLARAty Paying Off?

• Some Data Points
• After integrating and tuning EKF into new
  estimation framework on Rocky 8 (3 months),
  integrating algorithm for FIDO took 1 day
• After getting locomotion working on FIDO, moving
  to K9 took 4 days
• After testing CLARAty/Morphin navigator on Rocky
  8
• Integrating and testing on FIDO took 2 days
• Integrating on K9 took 4 days and testing 2
  weeks
• Integrating on ROAMS took 1 day
• After getting mast software working on Dexter
• Integrating on Rocky 8 took 2 weeks
• Integrating on FIDO took 4 days
• Adaptation of entire locomotion, motion control,
  I/O control and communication onto completely new
  avionics FPGA hardware with PPC405 for
  controlling Rocky 8 took 2 weeks
• N.B. These results are from professional CLARAty
  developers and should not be used to assess
  future development costs for new users
• (do not try this at home)

20
We Dont Always Get it Right

• The price for high reusability
• CLARAty core is becoming harder to change because
  many tasks at JPL, ARC, CMU, U. Washington,
  depend on it. Making changes, building and
  testing is complex and tedious.
• Process needs improvement very high on FY04
  priority list
• Other Data Points
• Adaptation to ROAMS after Rocky 8, FIDO, Rocky 7
  and K9 expected to take 5 weeks took 3 months
• Sun sensor delivery expected to take 2 months has
  taken 3 months and further debugging is needed
  subtle integration bug
• Several software bugs experienced and reported by
  validation tasks. Some are easy to fix, others
  are harder to reproduce
• Turn around debug and redeliver cycle requires
  some improvements
• Observation
• Code developed using CLARAty structures requires
  much shorter debugging cycle and shorter time to
  port to different platforms (significant cost
  savings)

21
Package Development Highlights

• Infrastructure
• New improved tools in place to simplify
  collaborative development
• Estimation
• New framework with EKF adapted to Rocky 8, FIDO,
  and ROAMS
• Visual odometry integrated on Rocky 7, Rocky 8
  and FIDO
• Manipulation
• Mast code tested on Dexter, Rocky 8, and FIDO,
  and arm code on Dexter
• Navigation
• Navigation integrated with VO and D. Test on
  Rocky 8, FIDO, and ROAMS
• Vision
• Vision adaptations to all rovers and addition of
  many image operations
• Sensing
• Developed infrastructure and integrated sun
  sensing capabilities
• Decision Layer
• Tested on multiple, randomly generated scenarios
  and support for onboard science
• I/O and Motion Control
• Progress towards merging two major branches
• Data Structure
• Flexible data marshalling for CLARAty structures

22
Levels of Integration

• Level I - Deposited
• Code in CLARAty repository - all Intellectual
  Property items cleared
• Compiles as a standalone application - no
  dependencies to other modules
• Have test programs and user documentation for
  getting started
• Level II - Encapsulated
• Integrated with other CLARAty modules
• Uses CLARAty components to interact with rover
• Does not support a CLARAty API
• Runs on at least one robot platform
• Level III - Integrated
• Conforms to a generic CLARAty API (or parent
  class)
• Has no unsupported 3rd party dependencies
• Runs on all applicable rover platforms
• Level IV - Refactored
• Uses all applicable CLARAty classes
• Internally conforms to CLARAty conventions and
  coding standard
• Level V - Reviewed
• Software reviewed by committee to ensure
  internal/external consistency

• Reused
• Re-used by other modules in CLARAty - dependent
  module
• Provides access to all internal data products

23
Technology Algorithms in CLARAty
24
Serving the Customer

• Closer interaction with MDS on
• Interfaces and behavior of robotic specific
  components
• Planning configuration for rovers
• Test bed support and access to CLARAty software
• Formal Delivery Documentation
• Executive summary
• Algorithm description
• How to check out and use
• References
• CLARAty Documentation
• API and behavior definitions
• Web-based access to most information

25
Software Development Process
AFS Backbone
Authentication
...
CMU
JPL
UW
ARC
U. Minnesota
Repository
Repository
CLARAty
VxWorks
K9
ATRV
3rd Party
Releases
Web
Repository
Rocky 8
FIDO
Rocky 7
Bench tops
Bench tops
Bench tops
26
Some CLARAty Statistics

• About 300 modules in Repository goal is to
  limit modules
• About 500,000 lines of C code revise and
  reduce
• Five adaptations to the following rovers
• Rocky 8, FIDO, Rocky 7
• ATRV
• K9
• Most technology modules are at Level II and Level
  III
• None are at Level IV or Level V (fully compliant,
  documented, and formally reviewed)

27
CLARAty Test Bed
Rocky 8 Bench top
Dexter ManipulatorsRocky 7 Bench top
FIDO Benchtop
28
CLARAty Test Bed

• Used by
• MDS/MSL
• RMSA university tasks
• CLARAty Developers
• JPL
• ARC
• CMU
• Manipulation task
• Validation tasks
• Remote Access
• Web camera
• Remote power cycle

29
FY03 Accomplishments

• Publications
• C. Urmson, R. Simmons, "Approaches for
  Heuristically Biasing RRT Growth," Proceedings
  IROS 2003, October, 2003
• I.A. Nesnas, A. Wright, M. Bajracharya, R.
  Simmons, T. Estlin, Won Soo Kim, "CLARAty An
  Architecture for Reusable Robotic Software," SPIE
  Aerosense Conference, Orlando, Florida, April
  2003. (730 KB)
• I.A. Nesnas, A. Wright, M. Bajracharya, R.
  Simmons, T. Estlin, "CLARAty and Challenges of
  Developing Interoperable Robotic Software,"
  invited to International Conference on
  Intelligent Robots and Systems (IROS), Nevada,
  October 2003. (410 KB)
• C. Urmson, R. Simmons, I. Nesnas, "A Generic
  Framework for Robotic Navigation," Proceedings of
  the IEEE Aerospace Conference, Montana, March
  2003. (8 pages, 730KB)
• C. M. Chouinard, F. Fisher, D. M. Gaines, T.A.
  Estlin, S.R. Schaffer, "An Approach to Autonomous
  Operations for Remote Mobile Robotic
  Exploration," Proceedings of the IEEE Aerospace
  Conference, Montana, March 2003 (277 KB)
• T. Estlin, F. Fisher, D. Gaines, C. Chouinard, S.
  Schaffer, I. Nesnas, "Continuous Planning and
  Execution for an Autonomous Rover," Proceedings
  of the Third International NASA Workshop on
  Planning and Scheduling for Space, Houston, TX,
  Oct 2002. (168 KB)

30
Significant Events
31
Integration of Multiple Algorithms on Multiple
Rovers

• Provides multiple inter-changeable algorithms to
  compare performance against mission requirements
• Demonstrated multiple pose estimation algorithms
  on multiple rovers
• Pose Estimation Algorithms
• FIDO EKF 3D pose estimation
• Wheel odometry
• Visual odometry
• Platforms demonstrated on
• Rocky 8
• FIDO
• ROAMS (without visual odometry)

32
Integration of Multiple Complex Algorithms

• Provides an end-to-end capability enabling
  evaluation of multiple integrated technologies.
• Important for evaluation against mission
  requirements
• Demonstrated the integration
• CLARAty Morphin navigator
• D path planner
• Wheel odometry pose estimator
• Visual odometry pose estimator
• Platforms demonstrated on
• Rocky 8 (all)
• FIDO (all)
• ROAMS
• All except visual odometry
• With real scanned Mars Yard terrain
• Stereo on synthetic images
• Very preliminary tests results

33
1st CLARAty Rocky 8 Field Trial

• Demonstrated a field-ready development and test
  environment using CLARAty and Rocky 8 rover
• 1st Field Trial for CLARAty and Rocky 8
• Johnson Valley Mojave Desert
• Sept 22 25, 2003
• In collaboration with Slope Navigation task
• Demonstrated CLARAty off-line
• Ability to rebuild software in-field
• Ability to run tests and collect data
• Collected data to support
• Wide Baseline Stereo task
• Visual Tracking task
• Slope Navigation task

34
Year-End Review Action Item Response

• MTP Annual Review
• A16 The task manager and the MTP are strongly
  encouraged to research other successful NASA
  software distribution activities to understand
  what customer support and funding arrangements
  (or even licensing) arrangements have been made
  to sustain these ongoing developments, in
  anticipation of a continually growing demand for
  CLARAty support
• Response
• MTP created a University Robotics Survey task
• Close CLARAty interactions with above task
• University Survey task investigated software
  distribution for IGOAL at JSC
• Joint development between JSC and Muniz
  Engineering initial distribution through
  Cosmic.
• No IP restrictions (civil servants) IP
  transferred to Muniz
• Only other example was RCS from NIST
• NIST developing and supporting product
• Adopted at some universities (e.g. Ohio State
  University)

35
Issues/Concerns

• Task encountered setbacks due to inability to
  share software with foreign national personnel on
  the task before ITAR clearance
• IP concerns with respect to sharing technology
  algorithms with the community at large
• Due to limited resources and increased level of
  collaborations and activities, some
  infrastructure work has not received the proper
  attention

36
Measuring Success or Failure

• We succeed IF we
• Significantly reduce integration time of new
  technology software onto real robotic systems
• Support multiple platforms with different
  hardware architectures
• Provide a service that is enabling for
  technologists
• Simplify the development/integrate/debug/test
  cycle for current and next generation NASA rovers
• Have people other than the developers using and
  liking the system

37
Video of Morphin, VO, D on FIDO and Rocky 8