91'304 Foundations of Theoretical Computer Science

1
91.304 Foundations of (Theoretical) Computer
Science

• David Martin
• dm_at_cs.uml.edu

This work is licensed under the Creative Commons
Attribution-ShareAlike License. To view a copy of
this license, visit http//creativecommons.org/lic
enses/by-sa/2.0/ or send a letter to Creative
Commons, 559 Nathan Abbott Way, Stanford,
California 94305, USA.
2
Measuring DFA complexity

• Suppose
• you have a DFA with states named 00000000 ..
  11111111 (28 256 unique states)
• an LCD attached to the thing showing the current
  state name
• ? c (for clock pulse)
• ?(q, c) (q 1) 0xFF
• This is a simple counter machine feed it clocks
  and it counts upwards

3
Measuring DFA complexity

• Time complexity
• A DFA always takes one transition per input
  character
• So time complexity is not useful here
• Program complexity
• A DFAs program is (mostly) its ?
• The model specifies no particular programming
  language for ? its just a table mapping
  (state, input) pairs to (state) outputs
• Though it can sometimes be specified concisely,
  as in ?(q, c) (q 1) 0xFF
• Reprogram the clock for any permutation of 0,18
  and ?s table remains just as big

4
Measuring DFA complexity

• Space complexity the amount of memory used
• But a DFA has no extra memory it only remembers
  what state it is in
• Cant look back or forward
• So a DFA always uses the same amount of memory,
  namely the amount of memory required to remember
  what state its in
• Needs to remember current element of Q
• Can write down that number in log2 Q bits

5
DFAs as real computers

• Consider a 256 MB computer that takes a finite
  input and produces a finite output
• Inputs clock pulses, interrupts, hard drive,
  keyboard, mouse, network, etc.
• Outputs video, hard drive, network, etc.
• Can code everything in binary
• But DFA only accepts or rejects input

6
Recognition model for functions

• Can still sort of be modeled by a DFA
• PC x y x,y 20,1 and the input x
  produces the output y
• Note character is just a separator
• DFA plays the role of equipment verifier
• Verifying correctness seems easier than computing
  the output, but at least its related

7
Are DFAs reasonable?

• One issue is that the programs dont seem to
  reflect much about the problem being solved
• If you can figure out how many bits of memory are
  needed for the solution, then you can always
  build a DFA based on that knowledge could be
  tedious and really large
• No difference in program complexity between same
  amount of memory means DFAs dont help us see the
  difference between programs very easily
• Neural nets??

8
Are DFAs reasonable?

• Similarly An 8-bit counter is structurally very
  different than a 9-bit counter
• More memory needed ) totally different ? program
  needed
• Not very modular!

9
Are DFAs reasonable?

• Another issue is that DFAs prefer the beginning
  of their inputs to the end of their inputs
• L5 x20,1 first 5 digits of x are 0
• L6 x20,1 last 5 digits of x are 0
• DFAs know where the input begins but not where it
  ends

10
Is REG reasonable?

• We should be able to combine computations as
  subroutines in simple ways
• logical OR (A B)
• logical AND (A Å B)
• concatenation (A B) and star (A)
• hard to prove!! motivation for NFA
• compl?ment (Ac)
• reversal (AR)
• All above are easy to do as logic circuits
• Will discuss further as closure under language
  operations

11
Nondeterministic Finite Automata

• Will relax two of these DFA rules
• Each (state, char) input must produce exactly one
  (state) output
• Must consume one character in order to advance
  state
• Example L6 ?bob?
• See M6
• The NFA accepts the input if there exists any way
  of reading the input that winds up in an
  accepting state at the end of the string
• Otherwise it rejects the input

12
NFAs

• Thus the NFA rejects the input if there doesnt
  exist any way of reading the input that winds up
  in an accepting state at the end of the string
• In other words every way of reading the input
  leads to a nonaccepting state
• Example M7
• L7 ?

a
b
c
?
?
1
2
3
13
Ways to think of NFAs

• NFAs want to accept inputs and will always take
  the most advantageous alternative(s)
• Because they will accept if there exists any way
  to get to an accepting state at the end of the
  string
• The quickest way there may be just one of many
  ways, but it doesnt matter

14
Ways to think of NFAs
a
a
a

• fork() model
• Input string is in a variable
• fork() at every nondeterministic choice point
• subprocess 1 (parent) follows first transition
• subprocess 2 (child) follows second
• subprocess 3 (child) follows third (if any), etc.
• A process that cant follow any transition calls
  exit() -- and gives up its ability to accept
• A process that makes it through the whole string
  and is in an accepting state prints out ACCEPT
• A single ACCEPT is enough

15
Syntax of DFA (repeat)

• A deterministic finite automaton (DFA) is a
  5-tuple (Q,?,delta,q0,F) such that
• Q is a finite set of states
• ? is an alphabet
• ?Q ? ! Q is the transition
  function
• q02 Q is the start state
• F µ Q is the set of accepting states
• Usually these names are used, but others are
  possible as long as the role is clear

16
Syntax of NFA

• A nondeterministic finite automaton (NFA) is a
  5-tuple (Q,?,delta,q0,F) such that
• Q is a finite set of states
• ? is an alphabet
• ?Q(? ?)!P(Q) is the transition function
• q02 Q is the start state
• F µ Q is the set of accepting states
• Usually these names are used, but others are
  possible as long as the role is clear

17
Syntax of NFA

• Definition ?? ? ?
• Well use this frequently enough
• Differences on state-transition diagram
• ?(1,a) 1 (not ?(1,a) 1)
• ?(1,?) 1, 2
• ?(3, c) 2, 3
• ?(2,a)
• ?(3,?) 3

a
b
c
?
?
1
2
3
c
Example M8
18
NFA computation

• This definition is different from but equivalent
  to the one in the text
• Books definition may be easier to understand at
  first, but that makes its version of Theorem 1.19
  (subset construction) harder
• Goal a function ?Q?! P(Q) where ?(q,x) is
  the set of all states reachable in the machine
  after starting in state q and reading the entire
  string x
• Then for an NFA M, we will define L(M)
  x2? ?(q0,x) contains some accepting
  state

19
NFA computation

• Let M(Q,?,?,q0,F) be an NFA. We define some
  auxiliary functions
• E Q ! P(Q) by ("?-closure")
• E(q) p2 Q p is reachable from q by
  following a chain of 0 or more ?
  transitions
• Although E takes elements of Q as input, we'll
  also use it as a function that takes subsets of Q
  as input (that is, elements of P(Q)). SoE P(Q)
  ! P(Q) by

In other words, given a set as input, just
process each element independently...
20
NFA computation

• Thus E(q) is the set of all states you can get to
  from q without reading any input
• In M8, E(3) ? E(2,1) ?
• We define a simple extension of ? that takes a
  set of states as input
• ? Q ??! P(Q) (this comes with the NFA)
• ?P(Q)?? ! P(Q) defined by

Again, given a set as input, just process each
element independently...
21
NFA computation

• We have a function E() that follows ?-transitions
  and a function ? that behaves like ? but takes
  sets as input
• ?Q?! P(Q) is defined inductively For all q2
  Q, ?(q,?) E( q )
• If w2? and c2?, let
• ?(q,wc) E(?(?(q,w),c))

22
NFA computation

• Finally, we defineL(M) x2? ?(q0,x)
  contains some accepting state
  x2?
• ?(1,ac) E(?(?(1,a),c))
• ?(1,a)E(?(?(1,?),a))
• ?(1,?) ?
• ?(1,ac) ?

?(q0,x) Å F ?
23
Question

• "How do I know when to follow ? transitions and
  when not to?"
• If you're talking about ?, then don't--it's the
  program itself. ? can express that "there is an
  ? transition here" but you never go any further
  than that one hop.
• If you're talking about ?, then do--because it
  includes E() as part of its definition, which is
  there precisely in order to follow ? transitions

24
NFAs are good at union (or)

• L2x20,1 the binary number x is a multiple
  of 2
• L3x20,1 the binary number x is a multiple
  of 3
• Let A L2 L3
• NFA for A using guess-and-verify strategy
• Preview of Theorem 1.22

25
The Subset Construction

• Theorem 1.19 For every NFA M1 there exists a DFA
  M2 such that L(M1) L(M2)
• Proof idea Well, how does fork() work on a
  uniprocessor machine?

26
The Subset Construction

• Proof Let M1(Q1,?,?1,init1,F1) be the NFA and
  define the DFA M2(Q2,?,?2,init2,F2) as follows
• Q2 P(Q1).
• Each state of the DFA records the set of states
  that the NFA can simultaneously be in
• Can compare DFA states for equality but also look
  "inside" the state name to find a set of NFA
  state names
• Define ?2 Q2 ? ! Q2 ?2 P(Q1)? !
  P(Q1) by
• ?2(S,a) E1(?1(S,a)) Go to whatever states
  are reachable from the states in S and reading
  the character a

Remember in an NFA,?1 Q1 ?? ! P(Q1) from
def ?1P(Q1)?? ! P(Q1) extend to sets E1P(Q1)
!P(Q1) ?-closure
27
The Subset Construction

• init2 E(init1)
• F2q 2 Q2 q Å F1? , in other wordsF2S µ
  Q1 S Å F1?
• The effect is that the DFA knows all states that
  are reachable in the NFA after reading the string
  so far. If any one of them is accepting, then
  the current DFA state is accepting too, otherwise
  it's not.
• If you believe this then that's all it takes to
  see that the construction is correct. So,
  convince yourself with an example. QED

28
Subset construction example

• Q2 ,1,2,3,1,2,1,3,2,3,1,2,3
• (On board)
• init21,2,3
• F23,1,3,2,3,1,2,3

a
b
c
?
?
3
1
2
c
Example M8 (think of this as M1 in the
construction)
29
Be methodical

• Need to compute ?2(1,2,3,c)
  E1(?1(1,2,3,c))
• By definition, ?1(1,2,3,c) ?1(1,c) ?1(2,c)
  ?1(3,c)
• 
  2,3
• Then take E1( 2,3 ) 2,3
• Save intermediate results for reuse
• It's OK to eliminate unreachable states in
  practice, even though that's not what the
  construction really does

30
Subset construction conclusion

• Adding nondeterminism makes programs shorter but
  not able to do new things
• Remember regular languages are defined to be
  those "recognized by a DFA"
• We now have a result that says that every
  language that is recognized by an NFA is regular
  too
• So if you are asked to show that a language is
  regular, you can exhibit a DFA or NFA for it and
  rely on the subset construction theorem
• Sometimes questions are specifically about DFAs
  or NFAs, though... pay attention to the precise
  wording

31
More NFA examples

• Write an NFA for ab,abc with 3 states
• NFA and DFA for ? over ?0,1
• Rule ? 2 L(M) , ?
• NFA and DFA for over ?0,1