Formal Languages

1
Formal Languages Models of Computation Slides
based on RPI CSCI 2400 Thanks to Petros Drineas
2
Why study formal languages?

• 1. Formal languages can be used to formalize the
  computational power of computing devices.
• We can answer questions like
• How long does it take to compute X?
• How much storage do I need to compute X?
• Can X be computed at all?
• Is it as hard to compute X as it is to compute Y?

3
Why study formal languages?

• 2. Formal languages can be used to formalize
  natural languages.
• This is the main interest in computational
  linguistics.
• Typical approach
• Show that a certain phenomenon occurs in natural
  language
• e.g., long-distance dependencies
• Infer which type of formal language is best
  suited for natural language
• Use this type for developing parsers etc.

4
Computation
memory
CPU
5
temporary memory
input memory
CPU
output memory
Program memory
6
Example
temporary memory
input memory
CPU
output memory
Program memory
compute
compute
7
temporary memory
input memory
CPU
output memory
Program memory
compute
compute
8
temporary memory
input memory
CPU
output memory
Program memory
compute
compute
9
temporary memory
input memory
CPU
Program memory
output memory
compute
compute
10
Automaton
temporary memory
Automaton
input memory
CPU
output memory
Program memory
11
Different Kinds of Automata

• Automata are distinguished by the temporary
  memory
• Finite Automata no temporary memory
• Pushdown Automata stack
• Turing Machines random access memory

12
Finite Automaton
temporary memory
input memory
Finite Automaton
output memory
Example Vending Machines
(small computing power)
13
Pushdown Automaton
Stack
Push, Pop
input memory
Pushdown Automaton
output memory
Example Compilers for Programming Languages
(medium computing power)
14
Turing Machine
Random Access Memory
input memory
Turing Machine
output memory
Examples Any Algorithm
(highest computing power)
15
Power of Automata
Finite Automata
Pushdown Automata
Turing Machine
Less power
More power
Solve more computational problems
16
Languages
17

• A language is a set of strings
• String A sequence of letters
• Examples cat, dog, house,
• Defined over an alphabet

18
Alphabets and Strings

• We will use small alphabets
• Strings

19
String Operations
Concatenation
20
Reverse
21
String Length

• Length
• Examples

22
Length of Concatenation

• Example

23
Empty String

• A string with no letters
• Observations

24
Substring

• Substring of string
• a subsequence of consecutive characters
• String
  Substring

25
Prefix and Suffix

• Prefixes Suffixes

prefix
suffix
26
Another Operation

• Example
• Definition

n
?
w
ww
w
n times
27
The Operation

• the set of all possible strings from
• alphabet

28
(No Transcript)
29
Languages

• A language is any subset of
• Example
• Languages

30

• German?

31
Note that
Sets
Set size
Set size
String length
32
Another Example

• An infinite language

33
Operations on Languages

• The usual set operations
• Complement

?
?
?
?
aaaa
bb
ab
a
ab
bb
aaaa
ab
a

,
,
,

,
,
,
?
?
?
?
?
ab
ab
bb
aaaa
ab
a


,
,
,
?
?
?
?
?
?
?
aaaa
a
ab
bb
aaaa
ab
a
,
,
,
,
?
?
34
Reverse

• Definition
• Examples

n
n

0


n
b
a
L
?
?
n
n
R
0


n
a
b
L
?
?
35
Reverse

• Definition
• Examples
• Reverse?

n
n

0


n
b
a
L
?
?
36
Concatenation

• Definition
• Example

37
Another Operation
n

• Definition
• Special case

L
LL
L
?
38
More Examples

39
Star-Closure (Kleene )

• Definition
• Example

2
1
0

L
L
L
L
?
40
Positive Closure

• Definition

2
1
?
L
L
L
?
?
?

L
?
?
?
41
Formal Languages Mathematical Preliminaries Slides
based on RPI CSCI 2400 Thanks to Petros Drineas
42

• Mathematical Preliminaries
• Sets
• Functions
• Relations
• Graphs
• Proof Techniques

43
SETS

A set is a collection of elements
We write
44
Set Representations C a, b, c, d, e, f, g,
h, i, j, k C a, b, , k S 2, 4, 6,
S j j gt 0, and j 2k for some kgt0
S j j is nonnegative and even
finite set
infinite set
45
A 1, 2, 3, 4, 5
Universal Set all possible elements
U
1 , , 10
46

• Set Operations
• A 1, 2, 3 B 2, 3, 4, 5
• Union
• A U B 1, 2, 3, 4, 5
• Intersection
• A B 2, 3
• Difference
• A - B 1
• B - A 4, 5

2
4
1
3
5
U
2
3
1
Venn diagrams
47

• Complement
• Universal set 1, , 7
• A 1, 2, 3 A 4, 5, 6, 7

4
A
A
6
3
1
2
5
7
A A
48
even integers odd integers
Integers
1
odd
5
even
6
2
4
3
7
49
DeMorgans Laws
U
A U B A B
A B A U B
U
50
Empty, Null Set

S U S S S - S
- S
U
Universal Set
51
Subset
A 1, 2, 3 B 1, 2, 3, 4,
5
Proper Subset
B
A
52
Disjoint Sets
A 1, 2, 3 B 5, 6
A B
U
A
B
53
Set Cardinality

• For finite sets

A 2, 5, 7 A 3
(set size)
54
Powersets
A powerset is a set of sets
S a, b, c
Powerset of S the set of all the subsets of S
2S , a, b, c, a, b, a, c,
b, c, a, b, c
Observation 2S 2S ( 8 23 )
55
Cartesian Product
A 2, 4 B 2, 3, 5 A
X B (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3),
(4, 5) A X B A B Generalizes to more
than two sets A X B X X Z
56
FUNCTIONS
domain
range
B
A
4
f(1) a
a
1
2
b
c
3
5
f A -gt B
If A domain then f is a total function
otherwise f is a partial function
57
RELATIONS
R (x1, y1), (x2, y2), (x3, y3),
xi R yi e. g. if R gt 2 gt 1,
3 gt 2, 3 gt 1
58
Equivalence Relations

• Reflexive x R x
• Symmetric x R y y R x
• Transitive x R y and y R z
  x R z
• Example R
• x x
• x y y x
• x y and y z x z

59
Equivalence Classes
For equivalence relation R equivalence
class of x y x R y Example
R (1, 1), (2, 2), (1, 2), (2, 1),
(3, 3), (4, 4), (3, 4), (4, 3)
Equivalence class of 1 1, 2 Equivalence
class of 3 3, 4
60
GRAPHS
A directed graph
e
b
node
d
a
edge
c

• Nodes (Vertices)
• V a, b, c, d, e
• Edges
• E (a,b), (b,c), (b,e),(c,a), (c,e),
  (d,c), (e,b), (e,d)

61
Labeled Graph
2
6
e
2
b
1
3
d
a
6
5
c
62
Walk
Walk is a sequence of adjacent edges
(e, d), (d, c), (c, a)
63
Path
Path is a walk where no edge is repeated Simple
path no node is repeated
64
Cycle
e
base
b
3
1
d
a
2
c
Cycle a walk from a node (base) to
itself Simple cycle only the base node is
repeated
65
Euler Tour
8
base
e
7
1
b
4
6
5
d
a
2
3
c
A cycle that contains each edge once
66
Hamiltonian Cycle
5
base
e
1
b
4
d
a
2
3
c
A simple cycle that contains all nodes
67
Finding All Simple Paths
e
b
d
a
c
origin
68
Step 1
e
b
d
a
c
origin
(c, a) (c, e)
69
Step 2
e
b
d
a
(c, a) (c, a), (a, b) (c, e) (c, e), (e, b) (c,
e), (e, d)
c
origin
70
Step 3
e
b
d
a
c
(c, a) (c, a), (a, b) (c, a), (a, b), (b, e) (c,
e) (c, e), (e, b) (c, e), (e, d)
origin
71
Step 4
e
b
d
a
(c, a) (c, a), (a, b) (c, a), (a, b), (b, e) (c,
a), (a, b), (b, e), (e,d) (c, e) (c, e), (e,
b) (c, e), (e, d)
c
origin
72
Trees
root
parent
leaf
child
Trees have no cycles
73
root
Level 0
Level 1
Height 3
leaf
Level 2
Level 3
74
Binary Trees
75
PROOF TECHNIQUES

• Proof by induction
• Proof by contradiction

76
Induction
We have statements P1, P2, P3,

• If we know
• for some b that P1, P2, , Pb are true
• for any k gt b that
• P1, P2, , Pk imply Pk1
• Then
• Every Pi is true

77
Proof by Induction

• Inductive basis
• Find P1, P2, , Pb which are true
• Inductive hypothesis
• Lets assume P1, P2, , Pk are true,
• for any k gt b
• Inductive step
• Show that Pk1 is true

78
Example
Theorem A binary tree of height n
has at most 2n leaves.
Proof by induction let L(i) be the
maximum number of leaves of any
subtree at height i
79

• We want to show L(i) lt 2i
• Inductive basis
• L(0) 1 (the root node)
• Inductive hypothesis
• Lets assume L(i) lt 2i for all i 0, 1, , k
• Induction step
• we need to show that L(k 1) lt 2k1

80
Induction Step
height
k
k1
From Inductive hypothesis L(k) lt 2k
81
Induction Step
height
L(k) lt 2k
k
k1
L(k1) lt 2 L(k) lt 2 2k 2k1
(we add at most two nodes for every leaf of level
k)
82
Remark
Recursion is another thing Example of recursive
function f(n) f(n-1) f(n-2) f(0) 1,
f(1) 1
83
Proof by Contradiction

• We want to prove that a statement P is true
• we assume that P is false
• then we arrive at an incorrect conclusion
• therefore, statement P must be true

84
Example
Theorem is not rational Proof Ass
ume by contradiction that it is rational
n/m n and m have no common
factors We will show that this is impossible
85
n/m 2 m2 n2
n is even n 2 k
Therefore, n2 is even
m is even m 2 p
2 m2 4k2
m2 2k2
Thus, m and n have common factor 2
Contradiction!