The Audacity of Computational Complexity Theory

1
The Audacity of Computational Complexity Theory
2
Todays Message

• Computational complexity theory is successful
  because of some audacious notions.

3
The Case against Complexity Theory

• Exhibit 1 The P vs NP question.
• If youve heard anything at all about
  computational complexity theory, this is what
  youve heard about.

4
Three Absurd Notions in Complexity Theory

• Polynomial time
• Time n100 is not feasible.
• Turing Machines
• Clearly an unrealistic model, even for
  deterministic machines.
• Asymptotic Analysis
• An example will illustrate an important point.

5
An example the Game of Go

• Computing strategies for Go requires exponential
  time.
• More precisely, given an n-by-n Go board with
  tiles on it, no program can compute an optimal
  next move in fewer than c2n d steps, for some
  constants c and d.
• Thus any program solving this problem must run
  very slowly on large inputs. This is the essence
  of asymptotic analysis.

6
An example the Game of Go

• Computing strategies for Go requires exponential
  time.
• More precisely, given an n-by-n Go board with
  tiles on it, no program can compute an optimal
  next move in fewer than c2n d steps, for some
  constants c and d.
• This is a much stronger statement about
  complexity than we are able to prove for most
  problems.

7
An example the Game of Go

• Computing strategies for Go requires exponential
  time.
• More precisely, given an n-by-n Go board with
  tiles on it, no program can compute an optimal
  next move in fewer than c2n d steps, for some
  constants c and d.
• Conceivably, there is a linear-time program that
  computes optimal moves, even for Go boards of
  size 1000-by-1000!

8
Conclusions

• Complexity Theory is a failure.
• For most problems of interest, no proofs of
  infeasibility are known.
• Even for problems where we can prove that
  exponential time is required (such as Go), we
  cant conclude anything about input sizes that
  are actually of practical interest.
• Computational complexity theory is a huge waste
  of time, and we should all go home now.

9
Wrong Conclusions

• Complexity Theory is a failure.
• For most problems of interest, no proofs of
  infeasibility are known.
• Even for problems where we can prove that
  exponential time is required (such as Go), we
  cant conclude anything about input sizes that
  are actually of practical interest.
• Computational complexity theory is a huge waste
  of time, and we should all go home now.

10
An Example of the Type of Theorem that We Need

• Theorem Any circuit that takes as input a
  logical formula (in WS1S) of length 616 and
  produces as output a correct answer, saying if
  the formula is valid or not, has at least 10123
  gates. (Stockmeyer, 1974)
• Corollary No program can run on any computer
  existing today, and solve this problem with a
  running time of less than 1000 years.

11
Possible Objections

• Theorem Any circuit that takes as input a
  logical formula (in WS1S) of length 616 and
  produces as output a correct answer, saying if
  the formula is valid or not, has at least 10123
  gates. (Stockmeyer, 1974)
• Is this a problem anyone would ever want to
  compute?

Yes! Its used in hardware verification.
12
Possible Objections

• Theorem Any circuit that takes as input a
  logical formula (in WS1S) of length 616 and
  produces as output a correct answer, saying if
  the formula is valid or not, has at least 10123
  gates. (Stockmeyer, 1974)
• Isnt this technology-dependent?

Slightly. Its based on AND and OR gates.
13
Possible Objections

• Theorem Any circuit that takes as input a
  logical formula (in WS1S) of length 616 and
  produces as output a correct answer, saying if
  the formula is valid or not, has at least 10123
  gates. (Stockmeyer, 1974)
• Aha!! What about quantum computers?

Essentially the same result holds. Some of the
constants will be slightly different.
14
No Possible Objections!

• Theorem Any circuit that takes as input a
  logical formula (in WS1S) of length 616 and
  produces as output a correct answer, saying if
  the formula is valid or not, has at least 10123
  gates. (Stockmeyer, 1974)
• Although this theorem is about finite functions,
  the proof is in terms of functions on infinite
  domains. (That is, it uses asymptotic analysis.)

15
Two Parts to the Proof

• Lemma There is a problem A computable in space
  2n that requires circuits of size nearly 2n.
• Diagonalization is good for creating monsters.
• Small circuits for the validity problem yield
  small circuits for A.
• Reducibility

An easy-to-compute function f such that x is in
A if and only if f(x) is a valid formula
16
Two Parts to the Proof

• Lemma There is a problem A computable in space
  2n that requires circuits of size nearly 2n.
• Diagonalization is good for creating monsters.
• Small circuits for the validity problem yield
  small circuits for A.
• Every problem computable in space 2n is reducible
  to the validity problem.

17
Reductions A Reason to Celebrate

• Reducibility is a shockingly effective tool for
  understanding the complexity of real-world
  computational problems.
• One reason that we are here today, is to
  celebrate the discovery of this fact.
• Lets learn more about reductions.

18
Desirable Properties for Reductions

• Reductions should be easy functions.
• If f and g are easy to compute, then it should
  also be easy to compute f(g(x)).
• If f is computable in time n2, then f is easy.

But this implies that there are easy functions
that time n100,000 to compute!
19
Desirable Properties for Reductions

• Reductions should be easy functions.
• If f and g are easy to compute, then it should
  also be easy to compute f(g(x)).
• If f is computable in time n2, then f is easy.

• We have two choices
• Forget about our properties above, or
• Embrace them, and claim that they make sense.

20
A Justification for Polynomial Time

• Complexity theoreticians arent trying to show
  that things are easy theyre trying to show that
  things are hard.
• If a problem cannot be solved in polynomial time,
  then this is a strong indication that it is not
  feasible to compute.
• A Karp reduction is a function f computable in
  polynomial time.

21
Karp Reductions

• Recall that if f reduces A to B, then A is no
  harder than B.
• Say that A and B are equivalent if A is
  reducible to B and vice-versa.
• Equivalent problems have roughly the same
  complexity. They are two different ways of
  looking at the same problem.
• Hundreds of computational problems have been
  studied. Surprisingly, they fall into a very
  small number of equivalence classes.

22
Checkers Go

• Checkers and Go are equivalent.
• Fact Optimal strategies can be computed in time
  approximately 2n.
• Fact Every problem computable in time 2poly(n)
  is reducible to Go.
• Hence this is a very special equivalence class.
  It consists of all of the hardest problems in
  EXP (the class of problems computable in
  exponential time).

23
Checkers Go

• Checkers and Go are equivalent.
• Fact Optimal strategies can be computed in time
  approximately 2n.
• Fact Every problem computable in time 2poly(n)
  is reducible to Go.
• We say these are complete for EXP.
• Since some problems in EXP require time 2n, we
  know that the complete problems also require
  (nearly) this much time.

24
Completeness

• Fact Another big equivalence class is complete
  for PSPACE (the class of problems that can be
  computed using a polynomial amount of memory).
• If any problem in PSPACE requires exponential
  time to compute, then all of these
  PSPACE-complete problems do.

25
Completeness

• Fact Another big equivalence class is complete
  for PSPACE (the class of problems that can be
  computed using a polynomial amount of memory).
• A quick program solving any PSPACE-complete
  problem gives a general technique to speed up any
  memory-limited computation.
• Thus, we guess that all PSPACE-complete problems
  require (nearly) exponential time.

26
Completeness, continued

• Some equivalence classes correspond to time and
  space complexity.
• but most dont!
• and they are the classes with the problems we
  really care the most about.

27
Nondeterminism

• The solution Consider weird machines.
• Given an input x, a nondeterministic machine is
  allowed to search for free through an
  exponentially-large set of strings, looking for a
  proof that x should be accepted.
• You may object This is absurd. No machine like
  this can be built!
• Thats the point!

28
NP

• NP is the class of problems solvable in
  polynomial time on nondeterministic machines.
• A huge number of important computational problems
  are NP-complete
• The traveling salesman problem
• The 3-colorability problem
• Partition
• Knapsack
• Nondeterminism is forced on us.

29
coNP

• What is the relationship between the following
  two problems?
• 3-colorability
• Non-3-colorability
• They have the same (deterministic) time
  complexity, but they seem not to be in the same
  Karp-equivalence class.
• There are short proofs that a graph can be
  three-colored.
• Simply present the coloring!
• Is there always a short proof that a graph cannot
  be three-colored?

30
coNP

• What is the relationship between the following
  two problems?
• 3-colorability
• Non-3-colorability
• They have the same (deterministic) time
  complexity, but they seem not to be in the same
  Karp-equivalence class.
• coNP consists of the complements of problems in
  NP.
• Nondeterminism highlights differences among
  problems of equivalent deterministic complexity.

31
Other Equivalence Classes

• Many other equivalence classes of important
  problems are captured in terms of time- and
  space-bounds on variants of nondeterministic
  machines.
• Any two problems in P are in the same
  Karp-equivalence class.
• But using more restricted reductions, a similar
  picture emerges inside P.
• Amazingly, even with restricted reductions, the
  classes of complete sets for big complexity
  classes (EXP, NP, ) are essentially unchanged.

32
Before Reducibility

• There was almost nothing that we could say about
  the computational complexity of real-world
  computational problems.
• For evidence of intractability, one could only
  point to the fact that other people had tried,
  and failed, to find an efficient algorithm.

33
With Reducibility

• Order is imposed on the chaos of problems.
• Natural computational problems fall into a
  handful of equivalence classes.
• We have a reason to believe that these classes
  are, in fact, distinct.
• Time- and Space-bounded complexity classes (on
  different types of machines)
• A theory that explains perceived differences in
  computational difficulty.

34
Factoring

• A good illustration of how far weve come.
• Complexity theory has little to say about the
  factorization problem
• Some cryptographic problems are reducible to
  factoring.
• There is no machine model or complexity class for
  which factoring is complete.
• The only real reason to think that factoring is
  hard, is because smart people have failed to find
  fast algorithms.

35
Factoring

• A good illustration of how far weve come.
• Complexity theory has little to say about the
  factorization problem
• RSA is reducible to factoring.
• There is no machine model or complexity class for
  which factoring is complete.
• This is the situation we had for almost all
  computational problems, before the 70s.

36
Factoring

• A good illustration of how far weve come.
• Complexity theory has little to say about the
  factorization problem
• RSA is reducible to factoring.
• There is no machine model or complexity class for
  which factoring is complete.
• Now, the reverse holds. For almost all problems
  that we suspect to be difficult to compute, we
  can explain why they are hard.

37
but what about the problem of asymptotic
analysis???

• Recall that, although Go and other EXP-complete
  problems require exponential time, we couldnt
  say anything about fixed input lengths.
• In contrast, we could say something about the
  validity problem for inputs of length 616.

38
Two Parts to the Proof that Validity is Hard for
Inputs of size 616

• Lemma There is a problem A computable in space
  2n that requires circuits of size nearly 2n.
• Diagonalization is good for creating monsters.
• Everything computable in space 2n is reducible to
  the validity problem.

Can we mimic this, to show a similar result for
Go?
39
Two Parts to the Proof that Validity is Hard for
Inputs of size 616

• Lemma There is a problem A computable in space
  2n that requires circuits of size nearly 2n.
• Diagonalization is good for creating monsters.
• Everything computable in space 2n is reducible to
  the validity problem.

Can we mimic this, to show a similar result for
Go?
40
An Attempt to Prove a Similar Infeasibility
Theorem for Go.

• Lemma There is a problem A computable in sitime
  2n that requires circuits of size nearly 2n. ..
• We dont know how to prove this! is good for
  creating monsters.
• Everything computable in sitime 2n is reducible
  to the problem of finding optimal moves in Go.
• This part goes through with no problem!

Can we mimic this, to show a similar result for
Go?
41
A Central Question in Complexity Theory

• Does every problem in EXP have small circuits?
• Is EXP contained in P/poly?
• It would be very strange if this were true!
• If NP requires large circuits, then it should be
  possible to prove infeasibility results for fixed
  input lengths for NP-complete problems.

42
A Central Question in Complexity Theory

• Does every problem in EXP have small circuits?
• Is EXP contained in P/poly?
• It would be very strange if this were true!
• In the next talk, Avi Wigderson will present more
  reasons why the circuit complexity of EXP is of
  such importance.

43
Conclusions

• Complexity theory does provide convincing proofs
  that certain transformations from input to output
  cannot be computed.
• In order to prove that finite functions are hard
  to compute, we use a theoretical framework based
  on infinite functions.
• This theoretical framework can be used to provide
  evidence of infeasibility, even when we cannot
  prove that functions are hard to compute.

44
Conclusions

• Reducibility is unreasonably effective in
  characterizing the complexity of problems.
• With surprisingly few exceptions, most problems
  are complete for one of a handful of complexity
  classes.
• A lot of exciting progress is being made. The
  next two talks will discuss some more recent
  developments.