Summary of Previous Class

1
Summary of Previous Class

• Undecidability of the following langauges by
  reductions to LAP, Lhalt, or to each other
• Equality of two languages
• Inequality of two languages
• Empty language
• Nonempty language
• Language of all strings in the alphabet
• Etc.

2
Decidability and Complement Languages

• Language L over alphabet ?
• Its complement ? ? L
• Theorem. A language L is decidable if and only if
  both
• L is recognizable
• its complement is recognizable.

3
Exercise Decidability and Complement Languages

• Suppose languages L1, L2, ,Lk over an alphabet
  ? form a partition of ? (their union is ? and
  any two are disjoint). Suppose also that each Li
  is recognizable. Is it true that each Li is
  decidable?
• 5 min

4
Exercise Complement Languages (solution)

• Suppose languages L1, L2, ,Lk over an alphabet
  ? form a partition of ? Suppose also that each
  Li is recognizable. Is it true that each Li is
  decidable?
• Yes. Let be the machine that recognizes
  the language Li. Let TM Mi that decides Li works
  as follows
• Do
• For j1,k,i
• Run the next step of . If it ends in
  accept break
• Until (the previous step resulted in accept)
• If the ji then return accept otherwise return
  reject.

5
Computable functions

• Let f0,1?0,1. We say that a TM M computes
  a function f if when M is initialized to start on
  the tape containing an input x? 0,1 it halts
  with the output f(x) written on the tape (i.e.
  M(x)f(x)).

6
Reducibility

• A language A is mapping reducible to a language B
  (written A?mB) if there is a computable function
  f??? such that for every x?A?f(x)?B.

7
Simple Facts about reducibility

• If A ?m B and B is decidable then A is decidable.
• If A ?m B and A is undecidable then B is
  undecidable.
• If A ?m B and B is Turing-recognizable then A is
  Turing-recognizable.

8
Exercise reducibility

• Is it possible to reduce a language L to its
  complement?L?
• 5 min

9
Exercise reducibility

• Is it possible to reduce a language L to its
  complement?L?
• Assume it is possible. Let L be recognizable but
  not decidable. Then ?L is recognizable (reducible
  to L). But then L is decidable. Contradiction.

10
Computational Complexity

• Complexity of Algorithms
• Complexity of Problems

11
Why Do We Need to Know That?

• Even if we have an algorithm for our problem,
  this algorithm may require infeasible amount of
  time or memory.
• Can we estimate the time and memory required by
  an algorithm?
• Can we estimate the time and memory required to
  solve a problem?
• How can we cope with a problem that requires
  enormous amount of time or space?

12
Runtime of a computation

• Let f0,1?0,1 and TN?N be some functions.
  We say that a TM M computes a function if when
  initialized to start on the tape containing input
  x? 0,1 it halts with the output f(x) written
  on the tape (i.e. M(x)f(x)).
• We say that M computes f in T(n)-time if for all
  n and all inputs x of size n the running time of
  M on x is at most T(n).

13
Examples of computable functions

• Decision Problems modify deciding TM M to
  machine M that on input x upon entering
  accepting state continues to erase tape, until
  when it meets . Then it prints 1 and halts. If
  it enters rejecting state, it continues to erase
  tape, until when it meets . Then it prints 1 and
  halts.
• Search problems
• Optimization problems

14
Asymptotic Notation

• To simplify the measuring of time and memory
  required by an algorithm, we use asymptotic
  notation
• Big-O notation f(n) O(g(n))
• Small-O notation f(n) o(g(n))

15
Big-O Notation

• We write
• f(n) O(g(n))
• if there exist numbers c and n0 such that f(n)
  c g(n) holds for every integer n n0.

16
Small-o Notation

• We write
• f(n) o(g(n))
• if f(n)/g(n) ? 0 as n ? ?.
• Equivalently
• f(n) o(g(n))
• if for any positive number ?, there exists a
  number n0 such that f(n) ? g(n) holds for every
  integer n n0.

17
Examples

• Does 2n2n4 belong to O(n)? O(n2)? O(n3)?
• Does it belong to o(n)? o(n2)? o(n3)?
• 3 min

18
Examples

• 2n2n4?O(n). For any c there exist n0c such
  that for all ngtn0 holds 2n2n4?n2gtcn
• 2n2n4?O(n2). For c4 and n02 holds 4n2?
  2n2n4. Obviously, as well holds 2n2n4?O(n3).
• 2n2n4?o(n). Since limn?? (2n2n4)/n limn??
  (n 14/n) ?
• 2n2n4?o(n2). Since limn?? (2n2n4)/(n2)
  limn?? (2 1/n 4/n) 2
• 2n2n4?o(n3). Since limn?? (2n2n4)/(n2)
  limn?? (2/n1/(n2) 4/(n3)0

19
Examples

• Order functions, i.e. establish the relations of
  the type g(n)O(f(n)) or g(n)o(f(n))
• n2/3, 2n, 2log n, 3n2n-logn, 2n, n!
• 5 min for all

20
Examples

• n2/3o(2n)
• 2nO(2log n), 2log nO(2n) ? 2n 2log n
• 2no(2n)
• 3n2n-lognO(2n), 2nO(3n2n-logn) ?
• 2n 3n2n-logn
• 2n o(n!)

21
Computing f(n) in time O(g(n)) - Example

• Problem Given an input string over 0,1, test
  whether the string has the form 0k1k.
• Equivalently
• Give a decider for language 0k1k k 0

22
Algorithm 1 decider for 0k1k k 0

• Scan across the tape and reject if there is a 0
  to the right of a 1.
• Repeat the following while both 0s and 1s remain
  on the tape
• Scan across the tape, crossing off a single 0 and
  a single 1.
• Reject if only 0s remain or only 1s remain on the
  tape. Otherwise (if neither 0s nor 1s remain)
  accept.
• This algorithm runs in time O(n2) where n is the
  length of the input string.

23

Time Complexity Class
Each function f (from positive integers to
positive integers) determines the following class
of languages Definition TIME(f) L L is a
language decided by a Turing machine in time
O(f(n)) Example Language 0k1k k 0
belongs to class TIME(n2).

24

Algorithm 2 decider for0k1k k 0

• Scan across the tape and reject if there is a 0
  to the right of a 1.
• Repeat the following while both 0s and 1s remain
  on the tape
• Scan across the tape, checking whether the total
  number of 0s and 1s (remaining on the tape) is
  even or odd. Reject if it is odd.
• Scan again across the tape, crossing off every
  other 0 starting with the first 0, and then
  crossing off every other 1 starting with the
  first 1.
• Accept if no 0s and no 1s remain. Otherwise
  reject.
• This algorithm runs in time O(n log2 n).

25

Time Complexity Classes

• We have shown that
• Language 0k1k k 0 belongs to class TIME(n2)
  Algorithm 1.
• Language 0k1k k 0 belongs to class TIME(n
  log n) Algorithm 2.
• Second is faster!!!
• But, does the complexity depend on the model of
  computation?

26

Algorithm 3 (for Two-Tape Turing Machines)

• Scan across the tape and reject if there is a 0
  to the right of a 1.
• Scan across Tape 1 until the first 1. At the same
  time, copy the 0s onto Tape 2.
• Scan across the 1s on Tape 1 until the end of the
  input. For each 1 read on Tape 1, cross off a 0
  on Tape 2. Reject if all 0s are crossed off
  before all 1s are read.
• Accept if all 0s have now been crossed off.
  Reject if any 0s remain.
• This algorithm runs in time O(n).

27
Complexity Relationships Among Models

• Theorem.
• Let f(n) be a function such that f(n) n. Then
  every multitape Turing machine running in time
  O(f(n)) has an equivalent single-tape Turing
  machine running in time O( (f(n))2 ).
• Fact.
• All reasonable computational models are
  polynomially equivalent.