Behavioral Animation

1
Behavioral Animation

• ??????
• ??????????
• 3D??
• ??? Edward Shen
• May 21, 2005

2
Course Information

• Date 5/19, 5/21, 5/26, 5/28 (2005)
• Lecturer Edward Yu-Te Shen ???
• Course Website http//graphics.csie.ntu.edu.tw/e
  dwards/dokuwiki/doku.php?idlecture

3
Lecturer

• ???, Edward Shen
• PhD Candidate (1st year)
• Graphics group, Dept. of CSIE, National Taiwan
  University
• http//graphics.csie.ntu.edu.tw/edwards/
• edwards_at_cmlab.csie.ntu.edu.tw

4
Course Overview

• This course course introduces several topics
  related to behavioral animation, including
  real-time crowd animation, collision detection
  and bounding boxes, self-generating behaviors
  through various techniques such as reinforcement
  learning, genetic algorithms, and so on.
• I hope that the class helps the students to gain
  general knowledge in the related areas as well as
  to apply such techniques in computer games and
  animated films development. The four classes will
  address more about design methodologies than
  practical programming issues.

5
Schedule

• Artificial Lives Fishes and Evolving Creatures
  (Saturday, May 21)
• Introduction to Artificial Lives
• Artificial Fishes and Reinforcement Learned
  Behavior
• Evolving Creatures and Genetic Algorithm
• Simulating Human Crowds (Thursday, May 26)
• Collision Detection and Bounding Boxes (Saturday,
  May 28)

6
Artificial Lives Fishes and Evolving Creatures

• Introduction to Artificial Lives
• Artificial Fishes and Reinforcement Learned
  Behavior
• Evolving Creatures and Genetic Algorithm

7
Applications
Source http//ligwww.epfl.ch/research/research_in
dex.html http//www.imdb.com/gallery/ss/0266543/F
NC-131.jpg http//www.lordoftherings.net/legend/g
allery/
8
Overview of Artificial Life / Behavioral
Animation

• Artificial life modeling and the computer graphic
  modeling hierarchy

Demetri Terzopoulos, Artificial life for computer
graphics, Communications of the ACM, v.42 n.8,
p.32-42
9
Overview of Artificial Life / Behavioral
Animation

• Combine geometric models and kinematic models to
  simplify key-framing
• Mid 1980s Dynamic Simulation
• particles
• rigid bodies
• deformable solids
• fluids

10
Overview of Artificial Life / Behavioral
Animation

• Biomechanical Modeling
• Biological tissue
• Internal muscles actuators
• Behavior Modeling
• Self-animating characters/ objects that react to
  environment stimuli
• Cognitive Modeling
• Artificial intelligence
• Knowledge representation
• Reasoning
• Planning

11
Examples of Behavioral Animation in the Movies
Demetri Terzopoulos, Artificial life for computer
graphics, Communications of the ACM, v.42 n.8,
p.32-42
12
Examples of Behavioral Animation in the Video
Games

• Creatures (http//www.creaturs.co.uk)
• The Sims (http// thesims.ea.com)
• Demosaurus Rex (an experimental interactive game
  environment)
• etc

13
Artificial Life in Computer Graphics
Artificial plants
Conference proceedings cover image, SIGGRAPH '94
R. Mech, P. Prusinkiewicz, B. Wyvill, and A.
Glassner
14
Artificial Evolution
Hopping
Sims

• Sims Sims 91,94
• simulate evolution from genotypes to phenotypes
• occasional mutations

15
Steering Behaviors
Action Selection strategy, goals, planning

• Wandering
• Seek / Flee
• Pursuit / Evade
• Path Following
• Cohesion
• Alignment
• Separation
• Leader Following

Steering path determination
Locomotion animation, articulation
16
Artificial Animals

• Terzopoulos Tu and Terzopoulos 94
• Basic behaviors
• Simulation of fish movements using a simple
  muscle system

17
Artificial Humans
HUMANOID-2

• Thalmann Thalmann 98
• Facial animation
• Simulation of sensory perception
• Interaction / communication between real humans
  and virtual ones

18
Behavioral Animation

• Goal Model animated characters / objects that
  interact autonomously within a virtual world
• Behavioral Animation shifts the traditional role
  of the animator

• We still probably want external control and
  direction on an autonomous agent
• Behavioral animation can be used as a vehicle for
  setting up simulations of real world events

19
Behavioral Animation v.s. Key-framing

• Key-framing
• Animator Specifies the first and last frames of
  the animation sequence
• Computer Generates the in-between frames
• Behavioral Animation
• Animator Determines a set of rules for the
  behavior of a character
• Computer Animates the scene based on the rules
  specified

20
Components of a Behavior Animation System

• Framework of a generic Behavioral Animation
  system

Millar et al. 99
21
Hanna et al. 2001
22
Perception Techniques

• The perception techniques are the means by which
  the animated character perceives its environment
• Can be grouped roughly in 3 classifications
• Zonal approach
• Sensory approach
• Synthetic vision approach

23
Zonal Approach

• Objects in the zone are sensed by the character
  
• Usually, perception zones are circular or
  spherical but it can be otherwise

Perception Zone
24
Sensory Approach

• There are areas where perception of an object is
  vague and where it is precise
• There might be many perception regions depending
  on the senses involved (e.g. smell, touch, sight,
  etc.)
• Each object in the environment has a transmissive
  area
• Perception occurs when characters sensory area
  intersects with the objects transmissive area

25
Behavioral Model

• Decision making model of the system
• Behavior can be
• reactive
• intelligent response
• Form of response is various
• movement vector
• change in internal attributes
• etc.
• Different approaches used to implement the
  behavioral techniques
• Rules
• Network
• Artificial intelligence
• Mathematical

26
Rule-Based Approach

• Involves giving each animated character a set of
  rules defining how they react to their
  environment
• () Can produce reasonable behaviors in a dynamic
  environment
• () Relatively easy to modify the rules to
  produce different behaviors
• (-) Reduce the chance of modeling behavior which
  cannot be easily predicted
• (-) Number of rules can grow very large in
  complex environments
• (-) Specific to a particular environment

27
Artificial Intelligence Approach

• Artificial Intelligence Approach
• Uses AI techniques such as reasoning engines and
  neural networks
• () Provides the ability to generate behaviors
  which have not been explicitly programmed
• (-) Requires a level of understanding by the
  animator of the techniques being used

28
Mathematical Approach

• The behavior of the character is defined in
  mathematical terms
• Ex Model of objects nature Model of
  observers liking
• Nature (N1, N2)
• Liking (2, -3.7)
• Model of function Attractiveness
• Attractiveness N . L 2N1 3.7N2
• () Provides a means of specifying behavioral
  responses in a precise manner
• (-) Not very intuitive for animators

29
Methods of Motion Generation

• Traditional Principles (Keyframing)
• Performance Capture (Motion Capture)
• Modeling/Simulation (Physics, Behaviors)
• Automatic Discovery (High-Level Control)

30
Keyframing (I)

• Specify the key positions for the objects to be
  animated.
• Interpolate to determines the position of
  in-between frames.

31
Keyframing (II)

• Advantages
• Relatively easy to use
• Providing low-level control
• Problems
• Tedious and slow
• Requiring the animator to understand the intimate
  details about the animated objects and the
  creativity to express their behavior in key-frames

32
Motion Interpolation

• Interpolate using mathematical functions
• Linear
• Hermite
• Bezier
• and many others (see Richard Parents online
  book)
• Forward inverse kinematics for articulation
• Specifying representing deformation

33
Basic Terminologies

• Kinematics study of motion independent of
  underlying forces
• Degrees of freedom (DoF) the number of
  independent position variables needed to specify
  motions
• State Vector vector space of all possible
  configurations of an articulated figure. In
  general, the dimensions of state vector is equal
  to the DoF of the articulated figure.

34
Forward vs. Inverse Kinematics

• Forward kinematics motion of all joints is
  explicitly specified
• Inverse kinematics given the position of the
  end effector, find the position and orientation
  of all joints in a hierarchy of linkages also
  called goal-directed motion.

35
Forward Kinematics

• As DoF increases, there are more transformation
  to control and thus become more complicated to
  control the motion.
• Motion capture can simplify the process for
  well-defined motions and pre-determined tasks.

36
Inverse Kinematics

• As DoF increases, the solution to the problem may
  become undefined and the system is said to be
  redundant. By adding more constraints reduces
  the dimensions of the solution.
• Its simple to use, when it works. But, it gives
  less control.
• Some common problems
• Existence of solutions
• Multiple solutions
• Methods used

37
Motion Capture (I)

• Use special sensors (trackers) to record the
  motion of a performer
• Recorded data is then used to generate motion for
  an animated character (figure)

38
Motion Capture (II)

• Advantages
• Ease of generating realistic motions
• Problems
• Not easy to accurately measure motions
• Difficult to scale or adjust the recorded
  motions to fit the size of the animated
  characters
• Limited capturing technology devices
• Sensor noise due to magnetic/metal trackers
• Restricted motion due to wires cables
• Limited working volume

39
Basic Behavioral Component Loop
after Hanna et al. 2001

• Execute all the mental triggers for each entity
• Ex. Increase hunger level
• Execute all the rules for each entity
• Enter associated goals in entitys goal queue
• Ex. Go to Wander Feed Mate
• Select highest priority goal from queue and
  execute it
• Delete all expired goals
• Display state of the animation

40
Coffee Break
41
Artificial Lives Fishes and Evolving Creatures

• Introduction to Artificial Lives
• Artificial Fishes and Reinforcement Learned
  Behavior
• Introduction
• Physics-Based Fish Model Locomotion
• Sensory Perception
• Behavioral Modeling Animation
• Intention Generator
• Artificial Fish Types
• Evolving Creatures and Genetic Algorithm

42
Go Fish!
43
Overview
Artificial Fish
Habits
Intention Generator
intention
Behavior Routines

F O C U S S E R
Filtered sensory data

Sensory data
Sensors

Control parameters
Behavior
Perception
Motor Controllers

Actuators(Muscles) Underlying Physical Model
Motor (Action)
3D Virtual World
44
Spring-Mass Dynamic Fish Model
45
Mechanics

• node i have mass mi, position xi(t) xi(t),
  yi(t), zi(t), velocity vi(t) dxi/dt,
  acceleration ai(t) d2xi/dt2.
• spring Sij connect node i to node j.
• Denote springs constant as cij natural, rest
  length as lij, eij(t) rij -lij, where rij
  xj(t) xi(t)
• The Lagrange equations of motion of the dynamics
  fish

i 0, , 22,
where wi(t) ? j?Ni
46
Muscles Hydrodynamics

• The artificial fish moves as a real fish does, by
  contracting its muscles.
• Characteristic of swinging tail contracting one
  side, relaxing the other side.

• When the fishs tail swings, it sets in motion a
  volume of water. The inertia of the displaced
  water produces a reaction force normal to the
  fishs body and proportional to the volume of
  water displaced per unit time, which propels the
  fish forward.

• With tail swinging towards positive X axis,
  reaction force at point ni acts along the
  inward normal. Component resists the
  lateral movement, while is forward thrust.
  Aggregate thrust propels fish towards positive Y
  axis.

47
Motor Controllers

• A artificial fish has three MCs, swim-MC,
  left-turn-MC right-turn-MC.
• The MCs prescribe muscle contractions to the
  mechanical model. The swim-MC controls the
  swimming segment muscle, while the turning MCs
  control the turning segment muscles.
• swim-MC(speed) r1, s1, r2, s2
• Turn-MC(angle) r1, s1, r2, s2

48
Pectoral Fins

• The pectoral fins control pitching and yawing.
• The pectoral fins work by applying reaction force
  to nodes in the midsection, i.e. nodes 1 ? i ?
  12.
• The pectoral fins are analogous to the wings of
  an airplane. Pitch, yaw, and roll control stems
  from changing their orientations relative to the
  body i. e., the angle p/4 ? ? ? p.
• A fin has an area A, surface n and the fish in
  velocity v relative to the water, the fin force

49
Learning How to Swim

• Eu Guiding the muscle actuator control function
  u(t) by discouraging large, rapid flutuation of u
• Ev Guiding good trajectory of the artificial
  fish (final distance from a target location,
  closeness to a specified speed, etc)

50
Learning Autonomously
51
Sensory Perception

• The artificial fish has a vision sensor and a
  temperature sensor.
• The temperature sensor samples the surrounding
  water temperature at the center of the artificial
  fishs body.
• The vision sensor has access to the geometry,
  material property, and illumination information.
  It can also interrogate the object database to
  identify nearby objects and interrogate the
  physical simulation to obtain information.

52
Behavioral Modeling Animation

• At each time step, the intention generator issues
  an intention based on the fishs habits, mental
  state, and incoming sensory.
• Habits a set of parameters that determine fish
  habits.
• Mental State function
• H(t) min1 ne(t)R(?tH)/? , 1
• L(t) mins(?tL)(1-H(t)), 1
• F(t) min?Fi, 1, where Fi minD0/di(t), 1
• H(t) hunger function
• L(t) libido function
• F(t) fear function
• t time

53
Mental State Function (Hunger)

• H(t) min1 ne(t)R(?tH)/? , 1
• ne(t) amount of food consumed as measured
  by the number of food particles or prey
  fishes eaten.
• R(x) R(x) 1-p0x with constant p0 is the
  digestion rate
• ?tH the time since the last meal
• ? a constant that dictates the appetite of
  the fish

54
Mental State Function (Libido)

• L(t) mins(?tL)(1-H(t)), 1
• s(x) s(x) p1x with constant p1 is the libido
  function
• ?tL the time since the last mating

55
Mental State Function (Fear)

• F(t) min?Fi, 1,
• where Fi minD0/di(t), 1
• D0 constant, D0 100
• Fi the fear to sighted predator i
• di the distance to sighted predator i

56
Intention Generator
Collision detection Danger of collision? Yes No

It avoid
Predator detection F gt f0 ?
If It-1 ? avoid Push the memory
No
Yes
Fm lt f1 ? And likes schooling? No Yes
Pop the memory Empty?
Yes
No
No
It escape
It school
IS eat or mate?
Yes
Generate new intention It by checking the mental
state and the habit string
Go to the focuser
Go to the next layer
57
Behavior Routines

• Avoiding-static-obstacle
• Avoiding-fish
• Eating-food
• Mating
• Leaving
• Wandering
• Escaping
• Schooling

58
Schooling Behavior Routine
gt 2 neighbours within one body length to the
front?
No
yes
find the closest schoolmate in front and speed up
towards it
gt 2 neighbours within one body length to the
sides?
No
yes
find the closest schoolmate to the sides and turn
to it
swim in generally the same direction as the
neighbours?
No
yes
make appropriate turns to match the general
orientation of the neighbours
Swim at the standard speed
59
Predators
Collision detection Danger of collision? Yes No

It avoid
Pop the memory Empty?
If It-1 ? avoid Push the memory
Yes
No
Hungry? Yes No
No
IS eat?
Yes
It eat
It wander
Go to the focuser
Go to the next layer
60
Predators

• Chasing cost function of a prey k
• Ck dk(1 ß1Sk ß2Ek /p)
• Ck chasing cost
• dk the distance between the mouth of the
  predator and the center of prey ks body
• Sk Sk1 if prey k is in a school of fishes,
  otherwise Sk0
• Ek the turning cost
• ß parameters that tune the contributions of Sk
  and Ek

61
Prey

• The intention generator of a prey fish is a
  specialization of the generic intention generator
  as follows

Hungry?
Yes
No
It eat
likes schooling? or Is school?
Generate new intention It by checking the mental
state and the habit string
No
Yes
It wander
It school

• Schooling and evading predators are two distinct
  behaviors of prey.

62
Pacifists

• The intention generator of a pacifist included
  activated mate intention, and deactivated escape
  and school intention

63
Plate 1 Predator stalking eating prey
64
Plate 2 Schooling Behavior
65
Coffee Break
66
Artificial Lives Fishes and Evolving Creatures

• Introduction to Artificial Lives
• Artificial Fishes and Reinforcement Learned
  Behavior
• Evolving Creatures and Genetic Algorithm

67
Virtual Creatures

• Complexity vs. Control
• Genetic Algorithms
• Darwin (fitness)
• Differs from previous work

68
Creature Representation

• Phenotype

• Genotype

69
Creature Representation

• Directed Graph
• Nodes
• Information
• Dimensions
• Joint-type
• Joint-limits
• Recursive-limit
• Neurons
• Connections
• Child Node
• Position
• Orientation

70
Creature Control

• Brain
• A directed graph of neurons
• Effectors
• Applied at Joints as Forces or Torques
• Muscle Pairs

71
Creature Control

• Sensors
• Joint Angle Sensors
• Contact Sensors
• Multiple Sensors per Unit
• Self vs. Environment
• Photosensors
• Provides relative coordinates of light source

72
Creature Control

• Neurons
• Provide different functions
• Sum, product, abs, max, sin, cos, oscillators,
  etc
• Output vs. Input
• Number of inputs dependant on function
• Output dependant on input and maybe previous state

73
Creature Control

• Effectors
• Connected to sensor or neuron
• Control a single DOF of a joint
• Scaled by Maximum-strength

74
Combining Control and Representation

• Nodes
• Contain unit information
• Contain neuron information
• Also a global set of neurons

75
Combining Control and Representation
76
Physical Simulation

• Collision Detection
• Bounding Box Pair Specific
• Collision Response
• Impulses penalty springs
• Friction
• Viscosity
• For simulating underwater
• Bugs will be found and exploited!

77
Behavior

• Evolution for a specific behavior
• Swimming
• Walking/Limping
• Jumping
• Following (Land/Water)
• Fitness function evaluated at each step
• Weights for more preferred methods

78
Evolution

• Recipe for a successful evolution
• Create initial genotypes
• From scratch
• From pre-evolved creatures
• From the user
• Calculate survival ratio
• Evaluate fitness and kill off the weaklings
• Reproduce the Alpha-males
• Reproductive ability
• Evolve, and proceed to step 3.

79
Evolution

• Directed Graph Mutation
• Node internal parameters
• Adding random node
• Parameters of each connection
• Adding/removing random connections
• Removing unconnected elements
• Mutation frequency scales inversely with graph
  size

80
Evolution

• Mating Directed Graphs

• Reproductive Method
• 40 Asexual
• 30 Crossover
• 30 Grafting

81
Evolution

• User Control
• Choose genotypes to survive
• Design initial aesthetic genotype
• Evolve only the aesthetic phenotype, then add a
  brain.

82
Performance

• CM-5 with 32 processors 3 Hours
• Population of 300
• 100 Generations

83
Results

• Homogeneity
• Swimmers
• Paddlers
• Tail-waggers
• Walkers
• Lizard-like
• Pushers/Pullers
• Hoppers
• Followers
• Steering Fins
• Paddlers

84
Virtual Creatures

• They are the one-armed, three-legged, limping,
  falling and drowning creatures of the virtual
  world.

85
Thats for today