CSC312

1
CSC312 Automata Theory Lecture 1 Introduction
2
Administrative Stuff

• Instructor Dr. Mudasser Naseer
• mnaseer_at_ciitahore.edu.pk Cabin 1, Faculty
  Room C7
• Lectures Sec-B Wed 1630, Thu 1800 hrs.
• Sec-C Wed 1800, Thu 1630 hrs.
• Office Hrs Tue Thu 1400 1600 hrs
• (or by appointment)
• Prerequisite CSC102 - Discrete Structures

3
Course Organization
Text Book i) J. E. Hopcroft, R. Motwani, J.
D. UllmanIntroduction to Automata
Theory,Languages, and Computation, Third
Edition,Pearson, 2008. ii) Denial I. A. Cohen
Introduction to Computer Theory, First Edition,
John Wiley Sons, 1986. Instruments There will
be 23 assignments, 45 quizzes,
Weights Assignments 10 Quizzes 15 S-I
10 S-II 15 Final Exam 50
4
Course Outline
Study and mathematically model various abstract
computing machines that serve as models for
computations and examine the relationship between
these automata and formal languages. Regular
expressions, NFAs. Core concepts of Regular
Languages and Finite Automata Decidability for
Regular Languages Non-regular Languages
Context-free Languages and Pushdown Automata
Decidability for Context-free Languages
Non-context-free Languages Turing Machines and
Their Languages are important part of the course.
Transducers (automata with output).
5
Schedule of Lectures
Lect. Topics/Contents
1 Introduction to Automata theory, Its background, Mathematical Preliminaries, Sets, Functions, Relations, Graphs, Proof Techniques
2 Formal Languages, Introduction to defining languages, alphabet, language, word, null string, length of a string, reverse of a string, Palindrome, Kleene closure.
3 Defining languages recursively, Recursive definitions of languages like EVEN, INTEGERS, POSITIVE, POLYNOMIALS, ARITHMETIC EXPRESSIONS
4 Theorems relating to recursive definitions and their proofs.
5 Formal definition of Regular Expressions, Defining languages with regular expressions, Languages associated with regular expressions.
6 Equality of Regular Expressions, Introducing the language EVEN-EVEN.
7 More examples related to regular expressions.
8 Introducing Finite Automata., Defining languages using Finite Automata. Constructing Finite Automata for different languages.
9 Recognizing the language defined by the given Finite Automata.
10 More examples related to Finite Automata.
11 Sessional I
6
Schedule of Lectures (Cont)
Lect. Topics/Contents
12 Transition Graphs with examples, Generalized Transition Graphs, Non-determinism in case of Transition Graphs.
13 Non-deterministic FAs. Differences between FA, TG and NFA.
14 Finite Automata with output, machines and Mealy machines with examples. 1s Complement machine, Increment machine.
15 Transducers as models of sequential circuits, Theorems for Converting Moore machines into Mealy machines and vice versa.
16 Regular Languages, Closure properties (i.e. , Concatenation and Kleene closure) of Regular Languages with examples. Complements and Intersections of Regular Languages, Theorems relating to regular languages and the related examples.
17 Decidability, decision procedure, Blue-paint method, Effective decision procedure to prove whether two given REs or FAs are equivalent.
18 Non-Regular Languages, The pumping Lemma, Examples relating to Pumping Lemma.
19 Myhill-Nerode theorem, Related Examples, Quotient Languages.
20 Context-Free Grammars, CFGs for Regular Languages with examples.
21 CFGs for non-regular languages, CFGs of PALINDROME, EQUAL and EVEN-EVEN languages, Backus-Naur Form.
22 Parse Trees, Examples relating to Parse Trees, Lukasiewicz notation, Prefix and Postfix notations and their evaluation.
23 Sessional II
7
Schedule of Lectures (Cont)
Lect. Topics/Contents
24 Ambiguous and Unambiguous CFGs, Total language tree. Regular Grammars, Semi-word, Word, Working String, Converting FAs into CFGs.
25 Regular Grammars, Constructing Transition Graphs from Regular Grammars. Killing null productions.
26 Killing unit productions, Chomsky Normal form with examples, Left most derivations.
27 Pushdown Automata, Constructing PDAs for FAs, Pushdown stack. Nondeterministic PDA.
28 Examples related with PDA, PDA for Odd Palindrome, Even Palindrome, PalindromeX.
29 Proving CFG PDA with examples. Constructing PDA from CFG in CNF and making trace table for any word.
30 Context Free Languages, their closure properties, , Concatenation and Kleene Closures using CFGs and PDAs.
31 Top down parsing, Bottom up parsing with examples, Turing machines, Examples of Turing Machines with trace tables, Converting FAs into Turing machines.
32 The subprogram Insert, The subprogram Delete, Constructing Turing machine for the language EQUAL using Insert subprogram.
8
Some basics

• Automaton A self-operating machine or mechanism
  (Dictionary definition), plural is Automata.
• Automata abstract computing devices
• Automata theory the study of abstract machines
  (or more appropriately, abstract 'mathematical'
  machines or systems, and the computational
  problems that can be solved using these machines.
• Mathematical models of computation
• Finite automata
• Push-down automata
• Turing machines

9
History

• 1930s Alan Turing defined machines more
  powerful than any in existence, or even any that
  we could imagine Goal was to establish the
  boundary between what was and was not computable.
• 1940s/150s In an attempt to model Brain
  function researchers defined finite state
  machines.
• Late 1950s Linguist Noam Chomsky began the
  study of Formal Grammars.
• 1960s A convergence of all this into a formal
  theory of computer science, with very deep
  philosophical implications as well as practical
  applications (compilers, web searching, hardware,
  A.I., algorithm design, software engineering,)

10
Computation
memory
CPU
11
temporary memory
input memory
CPU
output memory
Program memory
12
Example
temporary memory
input memory
CPU
output memory
Program memory
compute
compute
13
temporary memory
input memory
CPU
output memory
Program memory
compute
compute
14
temporary memory
input memory
CPU
output memory
Program memory
compute
compute
15
temporary memory
input memory
CPU
Program memory
output memory
compute
compute
16
Automaton
temporary memory
Automaton
input memory
CPU
output memory
Program memory
17
Different Kinds of Automata

• Automata are distinguished by the temporary
  memory
• Finite Automata no temporary memory
• Pushdown Automata stack
• Turing Machines random access memory

18
Finite Automaton
temporary memory
input memory
Finite Automaton
output memory
Example Vending Machines
(small computing power)
19
Pushdown Automaton
Stack
Push, Pop
input memory
Pushdown Automaton
output memory
Example Compilers for Programming Languages
(medium computing power)
20
Turing Machine
Random Access Memory
input memory
Turing Machine
output memory
Examples Any Algorithm
(highest computing power)
21
Power of Automata
Finite Automata
Pushdown Automata
Turing Machine
Less power
More power
Solve more computational problems
22

• Mathematical Preliminaries
• Sets
• Functions
• Relations
• Graphs
• Proof Techniques

23
SETS

A set is a collection of elements
We write
24
Set Representations C a, b, c, d, e, f, g,
h, i, j, k C a, b, , k S 2, 4, 6,
S j j gt 0, and j 2k for some kgt0
S j j is nonnegative and even
finite set
infinite set
25
A 1, 2, 3, 4, 5
Universal Set all possible elements
U
1 , , 10
26

• Set Operations
• A 1, 2, 3 B 2, 3, 4, 5
• Union
• A U B 1, 2, 3, 4, 5
• Intersection
• A B 2, 3
• Difference
• A - B 1
• B - A 4, 5

2
4
1
3
5
U
2
3
1
Venn diagrams
27

• Complement
• Universal set 1, , 7
• A 1, 2, 3 A 4, 5, 6, 7

4
A
A
6
3
1
2
5
7
A A
28
even integers odd integers
Integers
1
odd
0
5
even
6
2
4
3
7
29
DeMorgans Laws
A U B A B
U
A B A U B
U
30
Empty, Null Set

S U S S S - S
- S
U
Universal Set
31
Subset
A 1, 2, 3 B 1, 2, 3, 4,
5
Proper Subset
B
A
32
Disjoint Sets
A 1, 2, 3 B 5, 6
A
B
33
Set Cardinality

• For finite sets

A 2, 5, 7 A 3
(set size)
34
Powersets
A powerset is a set of sets
S a, b, c
Powerset of S the set of all the subsets of S
2S , a, b, c, a, b, a, c, b,
c, a, b, c
Observation 2S 2S ( 8 23 )
35
Cartesian Product
A 2, 4 B 2, 3, 5 A
X B (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3),
(4, 5) A X B A B Generalizes to more
than two sets A X B X X Z
36
FUNCTIONS
domain
range
B
A
4
f(1) a
a
1
2
b
c
3
5
f A -gt B
If A domain then f is a total function
otherwise f is a partial function
37
RELATIONS
Let A B be sets. A binary relation R from A
to B R (x1, y1), (x2, y2), (x3, y3),
Where and R ? A x B xi R
yi to denote e. g. if R gt 2 gt 1,
3 gt 2, 3 gt 1
38
Equivalence Relations

• Reflexive x R x
• Symmetric x R y y R x
• Transitive x R y and y R z
  x R z
• Example R
• x x
• x y y x
• x y and y z x z

39
Equivalence Classes
Given an equivalence relation R on a set A and an
element , the equivalence class of
is the set Example R
(1, 1), (2, 2), (1, 2), (2, 1),
(3, 3), (4, 4), (3, 4), (4, 3) Equivalence
class of 1 1, 2 Equivalence class of 3 3,
4
40
GRAPHS
A directed graph
e
b
node
d
a
edge
c

• Nodes (Vertices)
• V a, b, c, d, e
• Edges
• E (a,b), (b,c), (b,e),(c,a), (c,e),
  (d,c), (e,b), (e,d)

41
Labeled Graph
2
6
e
2
b
1
3
d
a
6
5
c
42
Walk
Walk is a sequence of adjacent edges
(e, d), (d, c), (c, a)
43
Path
Path is a walk where no edge is repeated Simple
path no node is repeated
44
Cycle
e
base
b
3
1
d
a
2
c
Cycle a walk from a node (base) to
itself Simple cycle only the base node is
repeated
45
Euler Tour
8
base
e
7
1
b
4
6
5
d
a
2
3
c
A cycle that contains each edge once
46
Hamiltonian Cycle
5
base
e
1
b
4
d
a
2
3
c
A simple cycle that contains all nodes
47
Finding All Simple Paths
e
b
d
a
c
origin
48
Step 1
e
b
d
a
c
origin
(c, a) (c, e)
49
Step 2
e
b
d
a
(c, a) (c, a), (a, b) (c, e) (c, e), (e, b) (c,
e), (e, d)
c
origin
50
Step 3
e
b
d
a
c
(c, a) (c, a), (a, b) (c, a), (a, b), (b, e) (c,
e) (c, e), (e, b) (c, e), (e, d)
origin
51
Step 4
e
b
d
a
(c, a) (c, a), (a, b) (c, a), (a, b), (b, e) (c,
a), (a, b), (b, e), (e,d) (c, e) (c, e), (e,
b) (c, e), (e, d)
c
origin
52
Trees
root
parent
leaf
child
Trees have no cycles
53
root
Level 0
Level 1
Height 3
leaf
Level 2
Level 3
54
Binary Trees
55
PROOF TECHNIQUES

• Proof by induction
• Proof by contradiction

56
Induction
We have statements P1, P2, P3,

• If we know
• for some b that P1, P2, , Pb are true
• for any k gt b that
• P1, P2, , Pk imply Pk1
• Then
• Every Pi is true

57
Proof by Induction

• Inductive basis
• Find P1, P2, , Pb which are true
• Inductive hypothesis
• Lets assume P1, P2, , Pk are true,
• for any k gt b
• Inductive step
• Show that Pk1 is true

58
Example
Theorem A binary tree of height n
has at most 2n leaves.
Proof by induction let L(i) be the
maximum number of leaves of any
subtree at height i
59

• We want to show L(i) lt 2i
• Inductive basis
• L(0) 1 (the root node)
• Inductive hypothesis
• Lets assume L(i) lt 2i for all i 0, 1, , k
• Induction step
• we need to show that L(k 1) lt 2k1

60
Induction Step
height
k
k1
From Inductive hypothesis L(k) lt 2k
61
Induction Step
height
L(k) lt 2k
k
k1
L(k1) lt 2 L(k) lt 2 2k 2k1
(we add at most two nodes for every leaf of level
k)
62
Remark
Recursion is another thing Example of recursive
function f(n) f(n-1) f(n-2) f(0) 1,
f(1) 1
63
Proof by Contradiction

• We want to prove that a statement P is true
• we assume that P is false
• then we arrive at an incorrect conclusion
• therefore, statement P must be true

64
Example
Theorem is not rational Proof Ass
ume by contradiction that it is rational
n/m n and m have no common
factors We will show that this is impossible
65
n/m 2 m2 n2
n is even n 2 k
Therefore, n2 is even
m is even m 2 p
2 m2 4k2
m2 2k2
Thus, m and n have common factor 2
Contradiction!