Using Hierarchical Reinforcement Learning to Solve a Problem with Multiple Conflicting Sub-problems - PowerPoint PPT Presentation

Loading...

PPT – Using Hierarchical Reinforcement Learning to Solve a Problem with Multiple Conflicting Sub-problems PowerPoint presentation | free to download - id: 78dc37-MDc3M



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Using Hierarchical Reinforcement Learning to Solve a Problem with Multiple Conflicting Sub-problems

Description:

Using Hierarchical Reinforcement Learning to Solve a Problem ... Explore the terrain Collect soil samples Return to base for refuelling My Project Apply ... – PowerPoint PPT presentation

Number of Views:31
Avg rating:3.0/5.0
Slides: 10
Provided by: Stephe775
Learn more at: http://www.cs.ru.ac.za
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Using Hierarchical Reinforcement Learning to Solve a Problem with Multiple Conflicting Sub-problems


1
Using Hierarchical Reinforcement Learning to
Solve a Problem with Multiple Conflicting
Sub-problems
2
Reinforcement Learning
  • Involves an agent interacting with an environment
  • The agent can be in one of various states in the
    environment
  • The agent is not told which action is correct,
    but is given a measure of an action for a given
    state
  • After a while the agent develops a policy

3
The curse
  • As complexity of the environment grows, state
    space increases exponentially
  • We can try to cleverly reduce state space
  • Hierarchical reinforcement learning

4
Hierarchical Reinforcement Learning
  • A complex problem can often be broken up into
    multiple conflicting sub-problems
  • Hierarchical reinforcement learning can handle
    this
  • Deals with each sub-problem separately using
    reinforcement learning
  • Decides which sub-problem to attempt next using
    reinforcement learning

5
A Practical Example The Mars Lander
  • Perform Various Conflicting Tasks
  • Explore the terrain
  • Collect soil samples
  • Return to base for refuelling

6
My Project
  • Apply hierarchical reinforcement learning to a
    complex problem
  • Consist of an agent existing in an environment
    where it will have to achieve an overall goal
  • Agent will be a primitive creature trying to
    survive in the wilderness

7
My Project
  • The overall goal will be for the creature to
    remain happy or comfortable in the wilderness
  • Overall goal can be divided into sub-goals
  • These sub-goals will be
  • Eating food
  • Drinking water
  • Resting under a Shelter
  • Repairing Shelter
  • Avoiding hazards

8
The Gridworld
9
Motivation for this approach
  • X pos Y pos Hunger Thirst Fatigue
    Shelter Condition
  • 13 x 13 x 10 x 10 x 10 x 10
  • 1690000 Possible states
  • Sub-goals separated out
  • (Xpos, Ypos, hunger) , (Xpos, Ypos, Thirst)
  • (Xpos, Ypos, Fatigue), (Xpos, Ypos, Shelter
    Condition)
  • (13 x 13 x 10) x 4
  • 1690 x 4 6760 Possible states
About PowerShow.com