# Using Hierarchical Reinforcement Learning to Solve a Problem with Multiple Conflicting Sub-problems - PowerPoint PPT Presentation

PPT – Using Hierarchical Reinforcement Learning to Solve a Problem with Multiple Conflicting Sub-problems PowerPoint presentation | free to download - id: 78dc37-MDc3M

The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
Title:

## Using Hierarchical Reinforcement Learning to Solve a Problem with Multiple Conflicting Sub-problems

Description:

### Using Hierarchical Reinforcement Learning to Solve a Problem ... Explore the terrain Collect soil samples Return to base for refuelling My Project Apply ... – PowerPoint PPT presentation

Number of Views:31
Avg rating:3.0/5.0
Slides: 10
Provided by: Stephe775
Category:
Tags:
Transcript and Presenter's Notes

Title: Using Hierarchical Reinforcement Learning to Solve a Problem with Multiple Conflicting Sub-problems

1
Using Hierarchical Reinforcement Learning to
Solve a Problem with Multiple Conflicting
Sub-problems
2
Reinforcement Learning
• Involves an agent interacting with an environment
• The agent can be in one of various states in the
environment
• The agent is not told which action is correct,
but is given a measure of an action for a given
state
• After a while the agent develops a policy

3
The curse
• As complexity of the environment grows, state
space increases exponentially
• We can try to cleverly reduce state space
• Hierarchical reinforcement learning

4
Hierarchical Reinforcement Learning
• A complex problem can often be broken up into
multiple conflicting sub-problems
• Hierarchical reinforcement learning can handle
this
• Deals with each sub-problem separately using
reinforcement learning
• Decides which sub-problem to attempt next using
reinforcement learning

5
A Practical Example The Mars Lander
• Explore the terrain
• Collect soil samples

6
My Project
• Apply hierarchical reinforcement learning to a
complex problem
• Consist of an agent existing in an environment
where it will have to achieve an overall goal
• Agent will be a primitive creature trying to
survive in the wilderness

7
My Project
• The overall goal will be for the creature to
remain happy or comfortable in the wilderness
• Overall goal can be divided into sub-goals
• These sub-goals will be
• Eating food
• Drinking water
• Resting under a Shelter
• Repairing Shelter
• Avoiding hazards

8
The Gridworld
9
Motivation for this approach
• X pos Y pos Hunger Thirst Fatigue
Shelter Condition
• 13 x 13 x 10 x 10 x 10 x 10
• 1690000 Possible states
• Sub-goals separated out
• (Xpos, Ypos, hunger) , (Xpos, Ypos, Thirst)
• (Xpos, Ypos, Fatigue), (Xpos, Ypos, Shelter
Condition)
• (13 x 13 x 10) x 4
• 1690 x 4 6760 Possible states