LING124 Finite state transducers, weighted automata

1
LING124 Finite state transducers, weighted
automata

• September 30, 2008

2
Class outline

• Finite state transducers
• Definition
• Operations on FST
• Deterministic vs. nondeterministic FST
• Weighted finite state machines
• Preliminaries
• Definition of weighted automata
• Example applications of FSM
• References

3
Finite state automata (FSA)

• An FSA is a quintuple ltS, so, F, S, dgt
• S Set of states
• so Initial state
• F Set of final states
• S Alphabet (a set of symbols)
• d Transition relation
• dS x S -gt S
• dS x S U e -gt S, where e is a null symbol, for
  (nondeterministic) FSA with epsilon transition

4
Finite state transducer (FST)

• An FST is a septuple ltS, so, F, S, ?, d, sgt
• S Set of states
• so Initial state
• F Set of final states
• S Input alphabet (a set of input symbols)
• ? Output alphabet (a set of output symbols)
• d Transition relation (dSxS -gtS)
• s Output relation (sSxS -gt ?)

5
FST as a state diagram
FSA
FST
6
Functions of FST

• Recognizer
• Accepts input-output string pair if there is a
  path from the initial state to a final state
• Generator
• Generates a string pair the FST encounters along
  its path from the initial state to a final state
• Translator
• Reads input string and prints out the
  corresponding output string
• Set relator
• Computes relation between sets

7
Inversion

• Switches the input and output labels
• T maps from the input alphabet I to the output
  alphabet O
• T-1 maps from the output alphabet O to the input
  alphabet I

8
Composition

• T1 maps from I1 to O1
• T2 maps from O1 to O2
• T1?T2 maps from I1 to O2

9
Projection

• Extract one side of the relation (e.g. input) and
  produce an FSA that accepts a string if and only
  if the original FST translates the string (e.g.
  input string) to another string (e.g. output
  string)
• First projection (p1) Extracts the input (left)
  side
• Second projection (p2) Extracts the output
  (right) side

10
First projection (p1)
11
Second projection (p2)
12
Deterministic vs. nondeterministic

• Nondeterministic FST
• At a given state, each input may be translated to
  one or more output symbols or symbol sequences
• Deterministic FST
• At a given state, each input is translated to a
  unique output symbol or output symbol sequence
  (sSxS -gt ? is a function)
• A state-input pair is mapped to a unique state
  (dSxS -gtS is a function)
• Unlike FSA, there is not always a deterministic
  FST equivalent to a nondeterministic FST

13
Preliminaries (1)

• Operand
• Input to a mathematical operator
• e.g. 3 6
• Binary operation
• Calculation involving two operands
• e.g. Addition

14
Preliminaries (2)

• Monoid
• A pair (M,)
• M is a set
• is a binary operation on M, satisfying
• Closure If a, b are in M, ab is also in M
• Associativity For a, b, c in M, (ab)c
  a(bc)
• Identity For a in M, there exists an element e
  in M such that ae ea a
• e is called the identity element

15
Preliminaries (3)

• Semiring
• A set R together with two binary operators and
  
• (R,) is a commutative monoid with identity
  element 0
• a00aa
• ab ba
• (R, ) is a monoid with identity element 1
• a11aa
• Multiplication () distributes over addition
• a(bc)abac
• (ab)cacbc
• a00a0

16
Weighted automata (1)

• AltS, so, F, S, d, ?, s, ?gt
• S, so , F, S, d the same as in the definition
  of FSA
• ? Initial output function that assigns weight
  for entering FSA (? so -gt K)
• s Output function that assigns a weight to
  paths between two states (sSxSxS -gt K)
• ? Final output function that assigns weight for
  leaving FSA (? F -gt K)

17
Weighted automata (2)

• Function f S-gt (K,, ) associated with A
• Assuming the operator means addition
• For example, if K were a set of real numbers
  between 0 and 1, and and meant addition
  (plus) and multiplication (times), f would be
  interpreted as finding the probability of a
  string w assigned by the weighted automaton A

18
Example WFST
Figure from Mohri et al. (2002)
19
Example applications

• FST implementation of phonological rules
• JM 11.1 and 11.2
• Kaplan and Kay (1994)
• Koskenniemi (1983)
• Mohri and Sproat (1996)
• Weighted finite state transducers in speech
  recognition
• Mohri et al. (2002)
• And a LOT more