Toward a Unified Scripting Language : Lessons Learned from Developing CML and AML

1
Toward a Unified Scripting Language Lessons
Learned from Developing CML and AML

• Soft computing Laboratory
• Yonsei University
• October 1, 2004

2
1. Introduction

• Two approaches to specifying scripting languages
  for character animation
• Character Markup Language(CML)
• Top-down approach by defining high-level
  attributes for character personality, emotion and
  behavior that are integrated to form the
  specification of synchronized animation script
• Avatar Markup Language(AML)
• Bottom-up approach in which the language provides
  a generic mechanism for the selection and
  synchronized merging of animations
• Need for powerful yet generic scripting languages
  to bridge the gap between behavior generation and
  animation tools

3
2. Scripting with the Character Markup Language

• Visual Behavior Definition
• A third factor that governs character behavior
• The actions an agent needs to perform in a
  session to achieve given tasks
• Personality and current mental state
• The role the agent is given
• The behaviors are defined as XML tags

CML Processor
High-level behavior tags
Animation Script
(map appropriate action point parameters)
4
2. Scripting with the Character Markup Language

• Classification of Motion
• Derived from Blumberg and Russells architecture
  (three-layer structure geometry, motor,
  behavior system)
• The initial set of CML base motions is classified
  by the goal of the motion as follows
• Movement defines motion that require the rotation
  or movement of a character from one position to
  another. Positions are defined by exact
  coordinates, an object position, or a character
  position. (move-to, turn-to)
• Pointing defines a pointing gesture toward a
  coordinate, object, or character. (point-to)
• Grasping defines motions that require the
  character to hold, throw, or come in contact with
  an object or another character (grasp, throw,
  touch)
• Gaze defines the movements related to the head
  and eyes. (Gaze are gaze, track, blink, look-to,
  and look-at. )
• Gesture includes motions that represent known
  gestures like hand movement to convey an
  acknowledgment, a wave, etc. (gesture-at)

5
2. Scripting with the Character Markup Language

• CML Specification
• Animated character behavior is expressed through
  the interpretation of XML Schema structures
• The language contains
• Low-level tags (specific character gesture
  representations defining movements, intensities,
  and explicit expression)
• High-level tags (commonly used combinations of
  these low-level tags
• Synchronization between the audio and visual
  modalities is achieved through the use of SMIL (
  ltseqgt, ltpargt)

6
2. Scripting with the Character Markup Language

• CML Representation Language
• Head Gesture Taxonomy
• Symbolic gestures, Iconic gestures, Deictic
  gesture
• Hand Gestures Taxonomy
• Posture, Motion, Orientation, Gestlets, Fingers
• Body Gestures Taxonomy
• Natural, Relax, Tense, Iconic, Incline
• Emotions (based on the OCC theory of emotion)
• Class, Valence, Subject, Target, Intensity,
  Time-stamp, Origin

7
2. Scripting with the Character Markup Language

• CML Scripting Language
• Face Animation Scripting
• Head Movement
• tilt movement in a slant with often subtle or
  superficial neck movement
• turn require more profound movement of the
  neck

ltturn-togt ltorder 0 to n/before/after
/gt ltpriority 0 to n/gt ltbegin ssmmm/before/aft
er/object /gt ltend ssmmm/before/after/object
/gt ltspeed 0.n to n.n (unit)
/defualt/slow/fast /gt lttarget
x,y,z/object/character /gt ltdirection
(rightside, leftside, frontside, backside)
/gt ltdegree (n) /gt ltrepeat 0 to n/dur
/gt ltinterrupt yes/no /gt lttransAnimat head
groups /gt lttransPos x,y,z/object/character
/gt lttransSpeed default/slow/ intermediate/fast
/gt lt/turn-togt
8
2. Scripting with the Character Markup Language

• Head Gesture
• Deictic
• Symbolic
• Face Movement and Gesture
• Define the elements and behaviors for specific
  parts of the face including the Brow, Gaze, and
  Mouse

ltdisagreegt lttilt-to-rightgt lt/tilt-to-rightgt
lttilt-to-leftgt lt/tilt-to-leftgt lt/disagreegt
9
2. Scripting with the Character Markup Language

• Body Animation Scripting
• A set of defined body elements which are low
  level tags based on MPEG-4 BAPs
• A set of high-level tags representing body parts
  which are grouped from a set of respective
  low-level tags
• Base elements are..
• Movement (moving, bending, turning)
• Gesture defines body postures that include
  motions representing common Iconic, Symbolic, or
  Deictic body gestures
• Posture (expression) defines a set of high-level
  tags representing general body gesture
• Natural ,Relax ,Tense, Incline

10
2. Scripting with the Character Markup Language
BDB

• Generating Script

PDTD
FDB
CML Generation
Animation Generation
EDTD
State and Context
CML Processor
CML Decoder
FAP Audio/TTS
Face
Action Composition
Input
FAP BAP Audio/TTS
Emotion Signal
Utterance Composition
CML Script
Face Body
Behavior/Action
Synchronisation
Speech Text
11
3. Avatar Markup Language

• Objective to design and develop a full
  end-to-end MPEG-4 multimedia framework to
  support, amongst other features, 3D avatar-based
  multi-user chat rooms and autonomous synthetic
  characters
• Three component
• A database of basic facial and body animation
  units(can extended by third party)
• A rendering system capable of merging multiple
  face and body animation units and text to speech
  input in realtime
• A high-level scripting language designed to allow
  animators to specify which animations to use
  together with timing, priority, and decay
  information

12
3. Avatar Markup Language

• AML Specification
• Flexiblity as many Expression Tracks as
  required, each containing as many Expressions as
  required

ltAFMLgt ltSettingsgt lt/Settingsgt ltExpressions
Track nameTrack namegt ltExpressiongt ltStartT
imegtmmssmmmlt/StartTimegt ltExNamegtnamelt/ExNam
egt ltEnvelopegt lt/Envelopegt lt/Expressiongt lt
/ExpressionTrackgt lt/AFMLgt
ltAFML syntaxgt
13
3. Avatar Markup Language
ltBodyAnimationTrack namechar, name of
trackgt ltUserAnimation typebaptrkwrl
filenamechargt ltStartTimegt mmssmmmautosync
autoafterlt/StartTimegt ltSpeedgtnormalslowfastlt/S
peedgt ltIntensitygtfloat, 0 to fn
lt/Intensitygt ltPrioritygtinteger, 0 to
nlt/Prioritygt lt/UserAnimationgt ltEmotiongtlt/Emotio
ngt ltStandardGesturegt lt/StandardGesturegt lt/BodyAn
imationTrackgt
ltABMLs Body AnimationTrack Syntaxgt
ltBodyAnimationTrack namechar, name of
trackgt ltFacingAction bodypartbodyheadonlygt
ltXCoorgtfloat, targets X coordinate in
meterslt/XCoorgt ltYCoorgtfloat, targets Y
coordinate in meterslt/YCoorgt ltZCoorgtfloat,
targets Z coordinate in meterslt/ZCoorgt lt/FacingA
ctiongt ltPointingAction handconfigonefingeropen
gt lt/ PointingActiongt ltWalkingAction
modedefaultrungtlt/WalkingActiongt ltResettingAc
tiongtlt/ResettingActiongt lt/BodyAnimationTrackgt
ltABMLs Body AnimationTrack Behaviorgt
14
4. CML and AML Applied

• CML
• Sample CML Happy Move and Point Script

ltcmlgt ltcharacter nameJames
personalityextravert rolepsa genderm
base-animation-filebutler.livgt lthappy
intensity0.3 decay0.5 targetgoal
priority1gt ltmove-to order0 priority0
speeddefault objectproduct1 /gt ltsync
typepar order1 priority0gt
ltpoint-to objectproduct1 /gt
ltutterancegt lt/syncgt lt/happygt
lt/charactergt lt/cmlgt
15
4. CML and AML Applied

• AML
• Sample AML Walk and Point

ltAML face_idx body_idy root_pathc\
namepoint to phonegt ltFA start_time0000000
gt ltTTS output_fapspeech.fap
output_wavspeech.wavgt ltTextgtLet me show
you another phone over herelt/Textgt lt/TTSgt
ltAFMLgt ltSettingsgt ltFpsgt25lt/Fpsgt
ltDurationgt0006000lt/Durationgt
ltFADBPathgt.\Expressions\lt/FADBPathgt
ltSpeechPathgt.\Speech\lt/SpeechPathgt
lt/Settingsgt ltExpressionsTrack namesmilegt
ltExpresiongt ltStartTimegt 0000800
lt/StartTimegt ltExNamegt smile.ex lt/ExNamegt
ltEnvelopegt ltPointgt lt/Pointgt
ltPointgtlt/Pointgt
ltPointgtlt/Pointgt lt/Envelopegt
lt/Expressiongt lt/ExpressionTrackgt
ltSpeechTrackgt lt/SpeechTrackgt lt/AFMLgt lt/FAgt
ltBAgt ltABMLgt.lt/ABMLgt lt/BAgt lt/AMLgt
16
5. Discussion and Lessons Learned

• A Comparison

17
5. Discussion and Lessons Learned

• Towards a Unified Language
• Objectives of a Unified Language
• Define a framework to decouple embodied agent
  animation tools and the underlying affect and
  planning engines
• Establish a formal specification for unified
  /consistent interpretation
• Provide for modular development
• Create a markup language based on XML (providing
  semantic and scripting annotations)
• Language Requirements
• High Level
• Usability
• Extensibility
• Parameterized
• Synchronization Support
• Consistency
• Domain

18
6. Conclusion

• There is a trade-off to be made between higher
  levels of control (high granularity) and the
  complexity of the resulting language
• Striving toward a unified scripting and
  representation language may be the catalyst for
  much needed agreement