Towards Robot Theatre - PowerPoint PPT Presentation

About This Presentation
Title:

Towards Robot Theatre

Description:

Professor Perky. 1 dollar latex skin from China ... Professor Perky with automated speech recognition (ASR) and text-to-speech (TTS) capabilities ... – PowerPoint PPT presentation

Number of Views:511
Avg rating:3.0/5.0
Slides: 151
Provided by: ee516
Learn more at: http://web.cecs.pdx.edu
Category:
Tags: perky | robot | theatre | towards

less

Transcript and Presenter's Notes

Title: Towards Robot Theatre


1
Towards Robot Theatre
Marek Perkowski Department of Electrical and
Computer Engineering, Portland State University,
Portland, Oregon, 97207-0751
2
  • Humanoid Robots and Robot Toys

3
Talking Robots
  • Many talking robots exist, but they are still
    very primitive
  • Work with elderly and disabled
  • Actors for robot theatre, agents for
    advertisement, education and entertainment.
  • Designing inexpensive natural size humanoid
    caricature and realistic robot heads

Dog.com from Japan
We concentrate on Machine Learning techniques
used to teach robots behaviors, natural language
dialogs and facial gestures.
Work in progress
4
Robot with a Personality?
  • Future robots will interact closely with
    non-sophisticated users, children and elderly, so
    the question arises, how they should look like?
  • If human face for a robot, then what kind of a
    face?
  • Handsome or average, realistic or simplified,
    normal size or enlarged?
  • The famous example of a robot head
  • is Kismet from MIT.
  • Why is Kismet so successful?
  • We believe that a robot that will interact with
    humans should have some kind of personality and
    Kismet so far is the only robot with
    personality.

5
Robot face should be friendly and funny
  • The Muppets of Jim Henson are hard to match
    examples of puppet artistry and animation
    perfection.
  • We are interested in robots personality as
    expressed by its
  • behavior,
  • facial gestures,
  • emotions,
  • learned speech patterns.

6
Behavior, Dialog and Learning
Words communicate only about 35 of the
information transmitted from a sender to a
receiver in a human-to-human communication. The
remaining information is included in
para-language. Emotions, thoughts, decision and
intentions of a speaker can be recognized earlier
than they are verbalized. NASA
  • Robot activity as a mapping of the sensed
    environment and internal states to behaviors and
    new internal states (emotions, energy levels,
    etc).
  • Our goal is to uniformly integrate verbal and
    non-verbal robot behaviors.

7
(No Transcript)
8
Moritas Theory
9
  • Our Base Model and Designs

10
Neck and upper body movement generation
11
Robot Head Construction, 1999
High school summer camps, hobby roboticists,
undergraduates
Furby head with new control
Jonas
We built and animated various kinds of humanoid
heads with from 4 to 20 DOF, looking for comical
and entertaining values.
12
  • Mister Butcher

Latex skin from Hollywood
4 degree of freedom neck
13
Robot Head Construction, 2000
Skeleton
Alien
We use inexpensive servos from Hitec and Futaba,
plastic, playwood and aluminum. The robots are
either PC-interfaced, use simple
micro-controllers such as Basic Stamp, or are
radio controlled from a PC or by the user.
14
Technical Construction, 2001 Details
Marvin the Crazy Robot
Adam
15
  • Virginia Woolf

2001
heads equipped with microphones, USB cameras,
sonars and CDS light sensors
16
2002
Max
  • BUG (Big Ugly Robot)

Image processing and pattern recognition uses
software developed at PSU, CMU and Intel (public
domain software available on WWW). Software is
in Visual C, Visual Basic, Lisp and Prolog.
17
Visual Feedback and Learning based on
Constructive Induction
2002
Uland Wong, 17 years old
18
Professor Perky
2002, Japan
Professor Perky with automated speech recognition
(ASR) and text-to-speech (TTS) capabilities
  • We compared several commercial speech systems
    from Microsoft, Sensory and Fonix.
  • Based on experiences in highly noisy environments
    and with a variety of speakers, we selected Fonix
    for both ASR and TTS for Professor Perky and
    Maria robots.
  • We use microphone array from Andrea
    Electronics.

1 dollar latex skin from China
19
Maria, 2002/2003
20 DOF
20
Construction details of Maria
location of head servos
skull
location of controlling rods
location of remote servos
Custom designed skin
21
Animation of eyes and eyelids
22
Cynthia, 2004, June
23
Currently the hands are not moveable.
We have a separate hand design project.
24
Software/Hardware Architecture
  • Network- 10 processors, ultimately 100
    processors.
  • Robotics Processors. ACS 16
  • Speech cards on Intel grant
  • More cameras
  • Tracking in all robots.
  • Robotic languages Alice and Cyc-like
    technologies.

25
Face detection localizes the person and is the
first step for feature and face recognition.
Acquiring information about the human face
detection and recognition, speech recognition and
sensors.
26
Face features recognition and visualization.
27
Use of Multiple-Valued (five-valued) variables
Smile, Mouth_Open and Eye_Brow_Raise for facial
feature and face recognition.
28
HAHOE KAIST ROBOT THEATRE, KOREA, SUMMER 2004
Czy znacie dobra sztuke dla teatru robotow?
Sonbi, the Confucian Scholar
Paekchong, the bad butcher
29
Editing movements
30
Yangban the Aristocrat and Pune his concubine
The Narrator
31
The Narrator
32
(No Transcript)
33
We base all our robots on inexpensive
radio-controlled servo technology.
34
We are familiar with latex and polyester
technologies for faces
Martin Lukac and Jeff Allen wait for your help,
whether you want to program, design behaviors,
add muscles, improve vision, etc.
35
New Silicone Skins
36
A simplified diagram of software explaining the
principle of using machine learning based on
constructive induction to create new interaction
modes of a human and a robot.
37
  • Probabilistic and Finite State Machines

38
Probabilistic State Machines to describe emotions
you are beautiful / Thanks for a compliment
P1
Happy state
you are blonde! / I am not an idiot
P0.3
you are blonde! / Do you suggest I am an
idiot?
Unhappy state
P0.7
Ironic state
39
Facial Behaviors of Maria
Do I look like younger than twenty three?
Maria asks
Response
  • no
  • no
  • yes

0.7
0.3
Maria smiles
Maria frowns
40
Probabilistic Grammars for performances
Speak Professor Perky, blinks eyes twice
P0.1
Speak Professor Perky
Where?
P0.3
Who?
P0.5
P0.5
P0.5
Speak in some location, smiles broadly
Speak In the classroom, shakes head
Speak Doctor Lee
What?
P0.1
P0.1
P0.1
Speak Was singing and dancing
P0.1
Speak Was drinking wine
.
41
Human-controlled modes of dialog/interaction
Human teaches
Thanks, I have a lesson
Hello Maria
Lesson finished
Robot performs
Robot asks
Question
Stop performance
Questioning finished
Command finished
Thanks, I have a question
Thanks, I have a command
Human commands
Human asks
42
  • Dialog and Robots Knowledge

43
Robot-Receptionist Initiated Conversation
Human
Robot
What can I do for you?
Robot asks
This represents operation mode
44
Robot-Receptionist Initiated Conversation
Human
Robot
What can I do for you?
I would like to order a table for two
Robot asks
45
Robot-Receptionist Initiated Conversation
Human
Robot
Smoking or non-smoking?
Robot asks
46
Robot-Receptionist Initiated Conversation
Human
Robot
Smoking or non-smoking?
I do not understand
Robot asks
47
Robot-Receptionist Initiated Conversation
Human
Robot
Do you want a table in a smoking or non-smoking
section of the restaurant? Non-smoking section
is near the terrace.
Robot asks
48
Robot-Receptionist Initiated Conversation
Human
Robot
Do you want a table in a smoking or non-smoking
section of the restaurant? Non-smoking section
is near the terrace.
A table near the terrace, please
Robot asks
49
Human-Initiated Conversation
Human
Robot
Hello Maria
initialization
Robot asks
50
Human-Initiated Conversation
Human
Robot
Hello Maria
What can I do for you?
Robot asks
51
Human-Asking
Human
Robot
Question
Question
Robot asks
Human asks
52
Human-Asking
Human
Robot
Question
Yes, you ask a question.
Human asks
53
Human-Asking
Human
Robot
What book wrote Lee?
Yes, you ask a question.
Human asks
54
Human-Asking
Human
Robot
What book wrote Lee?
I have no sure information.
Human asks
55
Human-Asking
Human
Robot
Try to guess.
I have no sure information.
Human asks
56
Human-Asking
Human
Robot
Try to guess.
Lee wrote book Flowers.
Human asks
57
Human-Asking
Human
Robot
This is not true.
Lee wrote book Flowers.
Human asks
58
Human-Teaching
Human ends questioning
Human
Robot
Questioning finished
Human asks
Human teaches
Questioning finished
Robot asks
Thanks, I have a lesson
59
Human-Teaching
Robot enters asking mode
Human
Robot
Questioning finished
What can I do for you?
Human asks
Human teaches
Questioning finished
Robot asks
Thanks, I have a lesson
60
Human-Teaching
Human starts teaching
Human
Robot
Thanks, I have a lesson
What can I do for you?
Human asks
Human teaches
Questioning finished
Robot asks
Thanks, I have a lesson
61
Human-Teaching
Human
Robot
Thanks, I have a lesson
Yes
Human teaches
62
Human-Teaching
Human
Robot
I give you question-answer pattern
Yes
Human teaches
63
Human-Teaching
Human
Robot
Question pattern What book Smith wrote?
Yes
Human teaches
64
Human-Teaching
Human
Robot
Answer pattern Smith wrote book Automata
Theory
Yes
Human teaches
65
Human-Teaching
Human
Robot
Checking question What book wrote Smith?
Yes
Human teaches
66
Human-Teaching
Human
Robot
Checking question What book wrote Smith?
Smith wrote book Automata Theory
Human teaches
67
Human-Teaching
Human
Robot
I give you question-answer pattern
Yes
Human teaches
68
Human-Teaching
Human
Robot
Question pattern Where is room of Lee?
Yes
Human teaches
69
Human-Teaching
Human
Robot
Answer pattern Lee is in room 332
Yes
Human teaches
70
Human-Checking what robot learned
Human
Robot
Lesson finished
Human teaches
Lesson finished
Question
Robot asks
Human asks
71
Human-Checking what robot learned
Human
Robot
Lesson finished
What can I do for you?
Human teaches
Lesson finished
Question
Robot asks
Human asks
72
Human-Checking what robot learned
Human
Robot
Question
What can I do for you?
Human teaches
Lesson finished
Question
Robot asks
Human asks
73
Human-Asking
Human
Robot
Question
Yes, you ask a question.
Human teaches
Lesson finished
Question
Robot asks
Human asks
74
Human-Asking
Human
Robot
What book wrote Lee?
Yes, you ask a question.
Human asks
75
Human-Asking
Human
Robot
What book wrote Lee?
I have no sure information.
Human asks
76
Human-Asking
Human
Robot
Try to guess.
I have no sure information.
Human asks
77
Human-Asking
Human
Robot
Try to guess.
Lee wrote book Automata Theory
Observe that robot found similarity between Smith
and Lee and generalized (incorrectly)
Human asks
78
Behavior, Dialog and Learning
  • The dialog/behavior has the following components
  • (1) Eliza-like natural language dialogs based on
    pattern matching and limited parsing.
  • Commercial products like Memoni, Dog.Com, Heart,
    Alice, and Doctor all use this technology, very
    successfully for instance Alice program won the
    2001 Turing competition.
  • This is a conversational part of the robot
    brain, based on pattern-matching, parsing and
    black-board principles.
  • It is also a kind of operating system of the
    robot, which supervises other subroutines.

79
Behavior, Dialog and Learning
  • (2) Subroutines with logical data base and
    natural language parsing (CHAT).
  • This is the logical part of the brain used to
    find connections between places, timings and all
    kind of logical and relational reasonings, such
    as answering questions about Japanese geography.
  • (3) Use of generalization and analogy in dialog
    on many levels.
  • Random and intentional linking of spoken
    language, sound effects and facial gestures.
  • Use of Constructive Induction approach to help
    generalization, analogy reasoning and
    probabilistic generations in verbal and
    non-verbal dialog, like learning when to smile or
    turn the head off the partner.

80
Behavior, Dialog and Learning
  • (4) Model of the robot, model of the user,
    scenario of the situation, history of the dialog,
    all used in the conversation.
  • (5) Use of word spotting in speech recognition
    rather than single word or continuous speech
    recognition.
  • (6) Continuous speech recognition (Microsoft)
  • (7) Avoidance of I do not know, I do not
    understand answers from the robot.
  • Our robot will have always something to say, in
    the worst case, over-generalized, with not valid
    analogies or even nonsensical and random.

81
  • Constructive Induction

82
(No Transcript)
83
Example Age Recognition
Name (examples) Age (output) d Smile Height Hair Color Hair Color
Joan Kid (0) a(3) b(0) c(0) c(0)
Mike Teenager (1) a(2) b(1) c(1) c(1)
Peter Mid-age (2) a(1) b(2) c(2)  
Frank Old (3) a(0) b(3) c(3) c(3)
Examples of data for learning, four people, given
to the system
84
Example Age Recognition
Smile - a Very often often moderately rarely
Values 3 2 1 0
Height - b Very Tall Tall Middle Short
Values 3 2 1 0
Color - c Grey Black Brown Blonde
Values 3 2 1 0
Encoding of features, values of multiple-valued
variables
85
Multi-valued Map for Data
Groups show a simple induction from the Data
ab\ c 0 1 2 3
00 - - - -
01 - - - 3
02 - - - -
03 - - - -
10 - - - -
11 - - - -
12 - - 2 -
13 - - - -
20 - - - -
21 - 1 - -
22 - - - -
23 - - - -
30 0 - - -
31 - - - -
32 - - - -
33 - - - -
ab\ c 0 1 2 3
00 - - - -
01 - - - 3
02 - - - -
03 - - - -
10 - - - -
11 - - - -
12 - - 2 -
13 - - - -
20 - - - -
21 - 1 - -
22 - - - -
23 - - - -
30 0 - - -
31 - - - -
32 - - - -
33 - - - -
d F( a, b, c )
86
Old people smile rarely
Groups show a simple induction from the Data
blonde hair
Grey hair
ab\ c 0 1 2 3
00 - - - -
01 - - - 3
02 - - - -
03 - - - -
10 - - - -
11 - - - -
12 - - 2 -
13 - - - -
20 - - - -
21 - 1 - -
22 - - - -
23 - - - -
30 0 - - -
31 - - - -
32 - - - -
33 - - - -
Middle-age people smile moderately
Teenagers smile often
Children smile very often
87
Another example teaching movements
Input variables
Output variables
88
Generalization of the Ashenhurst-Curtis
decomposition model
89
This kind of tables known from Rough Sets,
Decision Trees, etc Data Mining
90
Original table
Second variant
First variant of decomposition
At every step many decompositions exist
Decomposition is hierarchical
Which decomposition is better?
91
Constructive Induction Technical Details
  • U. Wong and M. Perkowski, A New Approach to
    Robots Imitation of Behaviors by Decomposition
    of Multiple-Valued Relations, Proc. 5th Intern.
    Workshop on Boolean Problems, Freiberg, Germany,
    Sept. 19-20, 2002, pp. 265-270.
  • A. Mishchenko, B. Steinbach and M. Perkowski, An
    Algorithm for Bi-Decomposition of Logic
    Functions, Proc. DAC 2001, June 18-22, Las Vegas,
    pp. 103-108.
  • A. Mishchenko, B. Steinbach and M. Perkowski,
    Bi-Decomposition of Multi-Valued Relations, Proc.
    10th IWLS, pp. 35-40, Granlibakken, CA, June
    12-15, 2001. IEEE Computer Society and ACM SIGDA.

92
Constructive Induction
  • Decision Trees, Ashenhurst/Curtis hierarchical
    decomposition and Bi-Decomposition algorithms are
    used in our software
  • These methods create our subset of MVSIS system
    developed under Prof. Robert Brayton at
    University of California at Berkeley 2.
  • The entire MVSIS system can be also used.
  • The system generates robots behaviors (C program
    codes) from examples given by the users.
  • This method is used for embedded system design,
    but we use it specifically for robot interaction.

93
Ashenhurst Functional Decomposition
Evaluates the data function and attempts to
decompose into simpler functions.
F(X) H( G(B), A ), X A ? B
X
B - bound set
if A ? B ?, it is disjoint decomposition if A ?
B ? ?, it is non-disjoint decomposition
94
A Standard Map of function z
Explain the concept of generalized dont cares
Bound Set
a b \ c
Columns 0 and 1 and columns 0 and 2 are
compatible column compatibility 2
Free Set
z
95
NEW Decomposition of Multi-Valued Relations
F(X) H( G(B), A ), X A ? B
A
X
Relation
Relation
B
if A ? B ?, it is disjoint decomposition if A ?
B ? ?, it is non-disjoint decomposition
96
Forming a CCG from a K-Map
Columns 0 and 1 and columns 0 and 2 are
compatible column compatibility index 2
Column Compatibility Graph
z
97
Forming a CIG from a K-Map
Columns 1 and 2 are incompatible chromatic number
2
Column Incompatibility Graph
98
Constructive Induction
  • A unified internal language is used to describe
    behaviors in which text generation and facial
    gestures are unified.
  • This language is for learned behaviors.
  • Expressions (programs) in this language are
    either created by humans or induced automatically
    from examples given by trainers.

99
  • Braitenberg Vehicles and Quantum Automata Robots

100
Another Example Braitenberg Vehicles and Quantum
BV
101
Braitenberg Vehicles
102
Quantum Circuits
Toffoli gate Universal, uses controlled square
root of NOT
0? 1? x?
0? 1? Vx?
0? 1?
0? 1? x?
0? 1?
0? 1? x?
?
  • Example 1 Simulation

103
Quantum Portland Faces
104
Conclusion. What did we learn
  • (1) the more degrees of freedom the better the
    animation realism. Art and interesting behavior
    above certain threshold of complexity.
  • (2) synchronization of spoken text and head
    (especially jaw) movements are important but
    difficult. Each robot is very different.
  • (3) gestures and speech intonation of the head
    should be slightly exaggerated superrealism,
    not realism.

105
Conclusion. What did we learn(cont)
  • (4) Noise of servos
  • the sound should be laud to cover noises coming
    from motors and gears and for a better
    theatrical effect.
  • noise of servos can be also reduced by
    appropriate animation and synchronization.
  • (5) TTS should be enhanced with some new
    sound-generating system. What?
  • (6) best available ATR and TTS packages should be
    applied.
  • (7) OpenCV from Intel is excellent.
  • (8) use puppet theatre experiences. We need
    artists. The weakness of technology can become
    the strength of the art in hands of an artist.

106
Conclusion. What did we learn(cont)
  • (9) because of a too slow learning, improved
    parameterized learning methods should be
    developed, but also based on constructive
    induction.
  • (10) open question funny versus beautiful.
  • (11) either high quality voice recognition from
    headset or low quality in noisy room. YOU CANNOT
    HAVE BOTH WITH CURRENT ATR TOOLS.
  • (12) low reliability of the latex skins and this
    entire technology is an issue.

107
We won an award in PDXBOT 2004. We showed our
robots to several audiences
Robot shows are exciting
Our Goal is to build toys for 21-st Century and
in this process, change the way how engineers are
educated.
International Intel Science Talent Competition
and PDXBOT 2004, 2005
108
  • Commercial Value of Robot Toys and Theatres

109
Robot Toy Market - Robosapiens
toy, poses in front of
toy, poses in front of
toy, poses in front of
110
Globalization
  • Globalization implies that images, technologies
    and messages are everywhere, but at the same time
    disconnected from a particular social structure
    or context. (Alain Touraine)
  • The need of a constantly expanding market for its
    products chases the bourgoise over the whole
    surface of the globe. It must nestle everywhere,
    settle everywhere, establish connections
    everywhere. (Marx Engels, 1848)

111
India and China - whats different?
  • They started at the same level of wealth and
    exports in 1980
  • China today exports 184 Bn vs 34 Bn for India
  • Chinas export industry employs today over 50
    million people (vs 2 m s/w in 2008, and 20 m in
    the entire organized sector in India today!)
  • Chinas export industry consists of toys (gt 60
    of the world market), bicycles (10 m to the US
    alone last year), and textiles (a vision of
    having a share of gt 50 of the world market by
    2008)

112
Learning from Korea and Singapore
  • The importance of Learning
  • To manufacture efficiently
  • To open the door to foreign technology and
    investment
  • To have sufficient pride in ones own ability to
    open the door and go out and build ones own
    proprietary identity
  • To invest in fundamentals like Education
  • to have the right cultural prerequisites for
    catching up
  • To have pragmatism rule, not ideology

113
Samsung
  • 1979 Started making microwaves
  • 1980 First export order (foreign brand)
  • 1983 OEM contracts with General Electric
  • 1985 All GE microwaves made by Samsung
  • 1987 All GE microwaves designed by Samsung
  • 1990 The worlds largest microwave
    manufacturer - without its own brand
  • 1990 Launch own brand outside Korea
  • Samsung microwaves 1 worldwide, twelve
    factories in twelve countries (including India,
    China and the US)
  • 2003 the largest electronics company in the
    world

114
How did Samsung do it?
  • By learning from GE and other buyers
  • By working very hard - 70 hour weeks, 10 days
    holiday
  • By being very productive - 9 microwaves per
    person per day vs 4 at GE
  • By meeting every delivery on time, even if it
    meant working 7-day weeks for six months
  • By developing new models so well that it got GE
    to stop developing their own

115
  • Fundamental question for humanoid robot builders

116
Should we build humanoid robots?
  • Mans design versus robots design
  • The humanoid robot is versatile and adaptive, it
    takes its form from a human, a design
    well-verified by Nature.
  • Complete isomorphism of a humanoid robot with a
    human is very difficult to achieve (walking) and
    not even not entirely desired.
  • All what we need is to adapt the robot maximally
    to the needs of humans elderly, disabled,
    children, entertainment.
  • Replicating human motor or sensor functionality
    are based on mechanistic methodologies,
  • but adaptations and upgrades are possible for
    instance brain wave control or wheels
  • Is it immoral?

117
Is it worthy to build humanoid robots?
  • Can building a mechanistic digital synthetic
    version of man be anything less than a cheat when
    man is not mechanistic, digital nor synthetic? 
  • If reference for the ultimate robot is man,
    then there is little confusion about ones aim to
    replace man with a machine.

118
Man Machine
  • Main reason to build machines in our likeness is
    to facilitate their integration in our social
    space
  • SOCIAL ROBOTICS
  • Robot should do many things that we do, like
    climbing stairs, but not necessarily in the way
    we do it airplane and bird analogy.
  • Humanoid robots/social robots should make our
    life easier.

119
The Social Robot
  • developing a brain
  • Cognitive abilities as developed from classical
    AI to modern cognitive ideas (neural networks,
    multi-agent systems, genetic algorithms)
  • giving the brain a body
  • Physical embodiment, as indicated by Brooks
    Bro86, Steels Ste94, etc.
  • a world of bodies
  • Social embodiment
  • A Social Robot is
  • A physical entity embodied in a complex, dynamic,
    and social environment sufficiently empowered to
    behave in a manner conducive to its own goals and
    those of its community.

120
Anthropomorphism
  • Social interaction involves an adaptation on both
    sides to rationalise each others actions, and the
    interpretation of the others actions based on
    ones references
  • Projective Intelligence the observer ascribes a
    degree of intelligence to the system through
    their rationalisation of its actions

121
Anthropomorphism The Social Robot
  • Objectives
  • Augment human-robot sociality
  • Understand and rationalize robot behavior
  • Embrace anthropomorphism
  • BUT - How does the robot not become trapped by
    behavioral expectations?
  • REQUIRED A balance between anthropomorphic
    features and behaviors leading to the robots own
    identity

122
Finding the Balance
  • Movement
  • Behavior (afraid of the light)
  • Facial Action Coding System
  • Form
  • Physical construction
  • Degrees of freedom
  • Interaction
  • Communication (robot-like vs. human voice)
  • Social cues/timing
  • Autonomy
  • Function role
  • machine vs. human capabilities

123
Humanoid Robots Experiments and Research Tasks
  • Autonomous mobile robots
  • Emotion through motion
  • Projective emotion
  • Anthropomorphism
  • Social behaviors
  • Qualitative and quantitative analysis to a wide
    audience through online web-based experiments

124
The perception learning tasks
  • Robot Vision
  • Where is a face? (Face detection)
  • Who is this person (Face recognition, learning
    with supervisor, persons name is given in the
    process.
  • Age and gender of the person.
  • Hand gestures.
  • Emotions expressed as facial gestures (smile, eye
    movements, etc)
  • Objects hold by the person
  • Lips reading for speech recognition.
  • Body language.

125
The perception learning tasks
  • Speech recognition
  • Who is this person (voice based speaker
    recognition, learning with supervisor, persons
    name is given in the process.)
  • Isolated words recognition for word spotting.
  • Sentence recognition.
  • Sensors.
  • Temperature
  • Touch
  • movement

126
The behavior learning tasks
  • Facial and upper body gestures
  • Face/neck gesticulation for interactive dialog.
  • Face/neck gesticulation for theatre plays.
  • Face/neck gesticulation for singing/dancing.
  • Hand gestures and manipulation.
  • Hand gesticulation for interactive dialog.
  • Hand gesticulation for theatre plays.
  • Hand gesticulation for singing/dancing.

127
Learning the perception/behavior mappings
  • Tracking the human.
  • Full gesticulation as a response to human
    behavior in dialogs and dancing/singing.
  • Modification of semi-autonomous behaviors such as
    breathing, eye blinking, mechanical hand
    withdrawals, speech acts as response to persons
    behaviors.
  • Playing games with humans.
  • Body contact with human such as safe
    gesticulation close to human and hand shaking.

128
What to emphasize in future work?
  • We want to develop a general methodology for
    prototyping software/hardware systems for
    interactive robots that work in human
    environment.
  • Image processing, voice recognition, speech
    synthesis, expressing emotions, recognizing human
    emotions.
  • Machine Learning technologies.
  • Safety, not hitting humans.

129
Can we build the first complete robot theatre in
the world?
Yes, if we will have more students who really
want to learn practical skills and not only to
take classes for grades.
Robotics I, Robotics II, individual projects,
RAS, high school students.
130
Where are we going?
  • This is an adventure, we do not know where our
    research will lead us.
  • This is truly interdisciplinary project. We need
    artists and psychologists.
  • If this takes the social functions of a theatre,
    it is a theatre.
  • Lessons from CAD and computer chess knowledge
    and search rather than super-intelligent logic
    mechanism.
  • Initial complexity of knowledge.

131
  • Lessons
  • degeneration of robot soccer.
  • OMSI project and security
  • Laws about future robots, can he sue me?
  • Our goal build a working environment for
  • Education
  • Entertainment
  • Verification of theories (bacteria foraging,
    social dynamics, Freud, immunological robots)
  • Verification of technologies (FPGA, clusters, net
    in chip technologies and AMBRIC).
  • Many researchers will be able to base their own
    research on our environment. We provide the
    technical background for more advanced or
    artistic work.
  • When there will be
  • the first commercially successful robot theatre?
  • the first humanoid social robot?

132
Humanoid robots
  • 1. Teachers and helpers
  • Language teachers
  • Teaching children
  • Teaching disabled children
  • Helpers for disabled adults
  • Helpers for old people
  • Helpers and companions for mentally disabled

133
Humanoid robots
  • 2. Toys
  • Conversational toys for lonely girls and young
    woman
  • Human-like robots as pets.
  • Animal-like robots as pets.
  • Interactive theatres of little robots sold
    separately and collected to families.

134
Humanoid robots
  • 3. Robot Theatres
  • Battle Bots (already commercial)
  • Robot theatres for children, next generation of
    Chucky Cheese Pizza Theatres and Disney Worlds.
  • Avangarda theatres for Adults (Umatilla, sex,
    violence, special effects like head separation,
    interaction, battle bots of new generation, and
    large size robot theatres in the prerries).
  • Artistic robot theatres (none exist see
    Japanese Bunraku and Noh single robots, Kissmet,
    aquarium and new robots of Cynthia Breazeal from
    MIT).

135
Humanoid robots
  • 4. Kiosks and receptionists
  • Toy-like and simplified (commercial products).
  • Realistic in view and size.
  • Mobile museum robots (commercial).
  • Wheeled humanoid robot of child-like size to be
    rented for exhibitions.

136
Humanoid robots
  • 5. Top research robots
  • Kissmet
  • Honda
  • Sony
  • Fujitsu
  • Hubo and KAIST
  • Samsung
  • Many Japanese
  • 6. Commercial Robot kits.
  • Mobile robots
  • Walking robots
  • Heads
  • Humanoids small
  • Humanoids childlike and expensive. Pino.

137
Existing technologies for robot theatre
  • Mobile robots (battlebots, Los Angeles group,
    Carnegie Mellon Group)
  • Walking animals
  • Walking big humans with robotic featuresJapanese
    robots like trump playing Sony)
  • Walking big humans with human-like features (head
    only - Albert Hubo, Small humans.
  • Body on wheels.
  • Head only
  • Head with neck and shoulders.
  • Upper body
  • Head on wheels

138
New Robots
  • 2005

139
  • "Nothing serious. Just stunts. There are dogs,
    dolls, faces that contort and are supposed to
    express emotion on a robot," he said.
  • Mr Engelberger, an American, founded the world's
    first company making industrial robots in 1961
    and became a specialist manufacturer of robots
    for hospitals.
  • It was pointless, expensive and unnecessary for
    Japan, which today makes three-quarters of the
    world's robots, to tinker with trivial inventions
    like robotic house sitters that rang to say there
    was a burglary going on, he said.
  • It made more sense to use the formidable amount
    of research that it had already done on personal
    robot technology to apply it to machines made for
    tasks that actually needed doing.
  • Such as robots that could be told by elderly or
    infirm people to fetch a book from a shelf or
    find the television remote or get a beer from the
    fridge.

140
  • "I've talked to visiting nurses who say that
    older people have to go to the bathroom more
    often and are embarrassed to say to somebody in
    the house, 'Please take me to the bathroom
    again'. But who cares how many times you ask a
    robot to take you to the bathroom?"
  • The future market for robots installed in the
    homes of elderly people was bigger than the
    luxury car market, he said, predicting that they
    would be leased out for US500 (673) per month.
  • Human care-givers cost 10 times that, Mr
    Engelberger said, and nursing homes were higher
    still.
  • "I know that there are things that a robot can't
    do. It's not going to bathe you and it's not
    going to dress you but it can be made to find the
    milk in the fridge," he said.
  • What the US8 billion robotics industry needs is
    for engineers to design practical robots for
    personal care. So why isn't more work being done?
    Mainly, Mr Engelberger thinks, it's because
    everyone is immersed in needless research and
    companies are distracted by the uneconomic quest
    for the humanoid, which he derides as toy making.
  • "I say, stop it all go for the whole damn
    schmeer I've recently become an octogenarian
    and I'd ask you, please, hurry up."

141
Albert Hubo
  • At an IT exhibition on the sidelines of the
    Asia-Pacific Economic Cooperation (APEC) summit
    in Busan, a participant shakes hands with a
    humanoid robot named Albert Hubo which has the
    face of Albert Einstein on Monday.
  • The robot can walk and speak and expresses
    emotions by moving facial muscles

142
Albert Hubo meets President Bush
Help me robo-Einstein, youre my only hope
143
Fujitsus Enon is getting a job at the grocery
store
  • Enon will be helping Aeon customers with
    everything from packing shopping bags and picking
    up groceries to find their way around the store.

144
  • This is the new HAL-5, or more specifically the
    Hybrid Assistive Limb. Bionic Suit.
  • Its developed as a walking aid for those who
    could use a bit of extra power, such as the
    disabled or apparently farmers who must add bags
    of sodium to their basement water softeners.

145
Walking Actors, Japan
  • 1000 iXs Research Corp. robots at Tokyos
    International Robot Exhibition.

146
The Bandai BN-17 Swiffer bot
  • Bandai BN-17 robot anthropomorphized robot for
    cleaning.
  • It can also handle your email and act as a
    security system

147
SORA, a receptionist robot
The little bot sports a camera, microphone, and
speaker for one way video conferencing with
visitors, who can interface with an included
touchscreen for information, and even scan a
business card to show their identity to whoever
is subjecting them to this robotic greeter. Once
theyre all approved, the robot can wave its
arms at them and point out the directions to the
office being displayed on the screen.
148
The WowWee Robosapien v2
  • The 230 second generation Robosapien v2 with
    remote from WowWee.
  • The Robosapien v2 can see, hear, touch, and
    interact with you and his surroundings with a
    full range of fluid movement.
  • Seething with attitude, his full-functioning arms
    with grippers allow him to pick-up and throw
    objects and then kung-fu your azz if you sass
    him.
  • He features 100s of functions including a
    low-level gastro-intestinal condition resulting
    in the occasional air-biscuit or belch to your
    childrens (and yours, admit it) amusement.
  • Hes also fully programmable which means youll
    find hacks-a-plenty in the open-source community
    allowing you to extend his functionality.

149
Questions to students
  1. Learn about new robot toys and other toys that
    can be used in our theatre or converted to useful
    robots or their components.
  2. Explain the concept of mapping architecture for a
    robot. Mapping being a combinational functions
    and mapping based on Finite State Machines.
  3. Explain the concept of Probabilistic Finite State
    Machine and how it can be used to control
    movements of a robot.
  4. How to use finite state machines and
    probabilistic machines for dialog and speech
    generation

150
Questions to students
  1. Explain Morita Theory and think if it can be
    generalized.
  2. Our robots have speech recognition and vision.
    Some have also sonar, infrared, touch and other
    sensors. What kind of sensors you would like to
    add and how you would like to program them for
    your applications.
  3. What is your concept of interactive robot toy
    that would extend the ideas of our Theatre.
  4. Write a script-scenario of conversation with
    robot that can be in 3 emotional states. The
    robot is a receptionist in Electrical Engineering
    Department at PSU.
Write a Comment
User Comments (0)
About PowerShow.com