Title: Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds
1Circuit Complexity, Kolmogorov Complexity, and
Prospects for Lower Bounds
2Todays Goal
- To raise awareness of the tight connection
between circuit complexity and Kolmogorov
complexity. - And to show that this is useful.
- To plant seeds of optimism, regarding the
prospects of proving lower bounds in circuit
complexity.
3Kolmogorov Complexity
- C(x) mind U(d) x
- Important property
- Invariance The choice of the universal Turing
machine U is unimportant. - x is random if C(x) x.
- CA(x) mind UA(d) x
4Circuit Complexity
- Let D be a circuit of AND and OR gates (with
negations at the inputs). Size(D) of wires
in D. - Size(f) minSize(D) D computes f
- We may allow oracle gates for a set A, along with
AND and OR gates. - SizeA(f) minSize(D) DA computes f
5K-complexity Circuit Complexity
- There are some obvious similarities in the
definitions. What are some differences? - A minor difference Size gives a measure of the
complexity of functions, C gives a measure of the
complexity of strings. - Given any string x, let fx be the function whose
truth table is the string of length 2logx,
padded out with 0s, and define Size(x) to be
Size(fx).
6K-complexity Circuit Complexity
- There are some obvious similarities in the
definitions. What are some differences? - A minor difference Size gives a measure of the
complexity of functions, C gives a measure of the
complexity of strings. - A more fundamental difference
- C(x) is not computable Size(x) is.
- The Minimum Circuit Size Problem (MCSP) (x,i)
Size(x) i.
7MCSP
- MCSP is in NP, but is not known to be
NP-complete. - MCSP is not believed to be in P.
- Factoring is in BPPMCSP.
- Every cryptographically-secure one-way function
can be inverted in PMCSP/poly.
8So how can K-complexity and Circuit complexity be
the same?
- C(x) SizeH(x), where H is the halting problem.
- For one direction, let U(d) x. We need a small
circuit (with oracle gates for H) for fx, where
fx(i) is the i-th bit of x. This is easy, since
(d,i,b) U(d) outputs a string whose i-th bit
is b is computably-enumerable. - For the other direction, let SizeH(fx) m. No
oracle gate has more than m wires coming into it.
Given a description of D (size not much bigger
than m) and the m-bit number giving the size of
y in H y m, U can simulate DH and produce
fx
9So how can K-complexity and Circuit complexity be
the same?
- C(x) SizeH(x), where H is the halting problem.
- So there is a connection between C(x) and Size(x)
- but is it useful?
- First, lets look at decidable versions of
Kolmogorov complexity.
10Time-Bounded Kolmogorov Complexity
- The usual definition
- Ct(x) mind U(d) x in time t(d).
- Problems with this definition
- No invariance! If U and U are different
universal Turing machines, CtU and CtU have no
clear relationship. - (One can bound CtU by CtU for t slightly
larger than t but nothing can be done for
tt.) - No nice connection to circuit complexity!
11Time-Bounded Kolmogorov Complexity
- Levins definition
- Kt(x) mindlog t U(d) x in time t(d).
- Invariance holds! If U and U are different
universal Turing machines, KtU(x) and KtU(x) are
within log x of each other. - Let A be complete for E Dtime(2O(n)). Then
Kt(x) SizeA(x).
12Time-Bounded Kolmogorov Complexity
- Levins definition
- Kt(x) mindlog t U(d) x in time t(d).
- Why log t?
- This gives an optimal search order for NP search
problems. - Adding t instead of log t would give every string
complexity x. - So lets look at how to make the run-time be
much smaller.
13Revised Kolmogorov Complexity
- C(x) mind for all i x 1, U(d,i,b)
1 iff b is the i-th bit of x (where bit i1 of
x is ). - This is identical to the original definition.
- Kt(x) mindlog t for all i x 1,
U(d,i,b) 1 iff b is the i-th bit of x, in time
t(d). - The new and old definitions are within O(log x)
of each other. - Define KT(x) mindt for all i x 1,
U(d,i,b) 1 iff b is the i-th bit of x, in time
t(d).
14Kolmogorov Complexity is Circuit Complexity
- C(x) SizeH(x).
- Kt(x) SizeE(x).
- KT(x) Size(x).
- Other measures of complexity can be captured in
this way, too - Branching Program Size KB(x) mind2s
for all I x 1, U(d,i,b) 1 iff b is the
i-th bit of x, in space s(d).
15Kolmogorov Complexity is Circuit Complexity
- C(x) SizeH(x).
- Kt(x) SizeE(x).
- KT(x) Size(x).
- Other measures of complexity can be captured in
this way, too - Formula Size KF(x)
mind2t for all I x 1, U(d,i,b) 1
iff b is the i-th bit of x, in time t(d), for
an alternating Turing machine U.
16but is this interesting?
- The result that Factoring is in BPPMCSP was first
proved by observing that, in PMCSP, one can
accept a large set of strings having large KT
complexity (and by making use of many important
results in the theory of pseudorandom generators
and derandomization). - (Basic Idea) There is a pseudorandom generator
based on factoring, such that factoring is in
BPPT for any test T that distinguishes truly
random strings from pseudorandom strings. MCSP
is such a test.
17This idea has many variants.
- Consider RKT, RKt, and RC.
- RKT is in coNP, and not known to be coNP hard.
- RC is not hard for NP under poly-time many-one
reductions, unless PNP. - How about more powerful reductions?
- Is there anything interesting that we could
compute quickly if C were computable, that we
cant already compute quickly? - Proof uses PRGs, Interactive Proofs, and the fact
that an element of RC of length n can be found in
- But RC is undecidable! Perhaps H is in P
relative to RC?
18This idea has many variants.
- Consider RKT, RKt, and RC.
- RKT is in coNP, and not known to be coNP hard.
- RC, is not hard for NP under poly-time many-one
reductions, unless PNP. - How about more powerful reductions?
- PSPACE is in P relative to RC.
- NEXP is in NP relative to RC.
- Proof uses PRGs, Interactive Proofs, and the fact
that an element of RC of length n can be found in
poly time, relative to RC BFNV. - But RC is undecidable! Perhaps H is in P
relative to RC?
19Relationship between H and RC
- Perhaps H is in P relative to RC?
- This is still open. It is known that there is a
computable time bound t such that H is in
DTime(t) relative to RC Kummer. - but the bound t depends on the choice of U in
the definition of C(x). - We also know that H is in P/poly relative to RC.
20This idea has many variants.
- Consider RKT, RKt, and RC.
- What about RKt?
- RKt, is not hard for NP under poly-time many-one
reductions, unless ENE. - How about more powerful reductions?
- EXP NP(RKt).
- RKt is complete for EXP under P/poly reductions.
- Open if RKt is in P!
21Kolmogorov Complexity is Circuit Complexity
- C(x) SizeH(x).
- Kt(x) SizeE(x).
- KT(x) Size(x).
- Other measures of complexity can be captured in
this way, too - Formula Size KF(x)
mind2t for all I x 1, U(d,i,b) 1
iff b is the i-th bit of x, in time t(d), for
an alternating Turing machine U.
22Kolmogorov Complexity is Circuit Complexity
- C(x) SizeH(x).
- Kt(x) SizeE(x).
- KT(x) Size(x).
- Other measures of complexity can be captured in
this way, too - A similar definition captures depth k threshold
circuit size. - This is the clever transition to start the
discussion of circuit lower bounds
23Big Complexity Classes
- NP
- P
- .
- .
- NC
- L (Deterministic Logspace)
24The Main Objects of InterestSmall Complexity
Classes
- TC0 O(1)-Depth Circuits of MAJ gates
- AC0 6
- NC1 Log-Depth Circuits
- AC0 cant compute Mod 2 FSS,A
- AC0 O(1)-Depth Circuits of AND/OR gates
25The Main Objects of InterestSmall Complexity
Classes
- TC0 O(1)-Depth Circuits of MAJ gates
- AC0 6
- NC1 Log-Depth Circuits
- AC0 cant compute Mod 2 FSS,A
- AC0 O(1)-Depth Circuits of AND/OR gates
26The Main Objects of InterestSmall Complexity
Classes
- TC0 O(1)-Depth Circuits of MAJ gates
- NC1 Log-Depth Circuits
- AC0 2 cant compute Mod 3 R,S
- AC0 2
- AC0 O(1)-Depth Circuits of AND/OR gates
27The Main Objects of InterestSmall Complexity
Classes
- NC1 Log-Depth Circuits
- TC0 O(1)-Depth Circuits of MAJ gates
- AC0 6
- AC0 2
- AC0 O(1)-Depth Circuits of AND/OR gates
28The Main Objects of InterestSmall Complexity
Classes
- NC1 poly-size formulae
- TC0 O(1)-Depth Circuits of MAJ gates
- AC0 6
- AC0 2
- AC0 O(1)-Depth Circuits of AND/OR gates
29Complete Problems
- NP has complete sets (under polynomial time
reducibility P) - These small classes have complete sets, too
(under AC)
30Reductions
- A AC B means that there is a constant-depth
circuit computing A that has the usual AND and OR
gates, and also has oracle gates for B.
B
31Complete Problems
- sorting, multiplication, division
- Naor,Reingold Pseudorandom Generator
32Complete Problems
- BFE Balanced Boolean Formula Evaluation
(AND,OR,XOR) - Word problem over S5
33The Word Problem Over S5
- A regular set complete for NC1
34Complexity Classes are not Invented Theyre
Discovered
- NP (SAT, Clique, TSP,)
- P (Linear Programming, CVP, )
- NL (Connectivity, Shortest Paths, 2SAT, )
- L (Undirected Connectivity, Acyclicity, )
- NC1 (BFE, Regular Sets)
- TC0 (Sorting, Multiplication, Division)
Were interested in NC1 (for instance) not
because we want to build formulae for these
functions
35Complexity Classes are not Invented Theyre
Discovered
- NP (SAT, Clique, TSP,)
- P (Linear Programming, CVP, )
- NL (Connectivity, Shortest Paths, 2SAT, )
- L (Undirected Connectivity, Acyclicity, )
- NC1 (BFE, Regular Sets)
- TC0 (Sorting, Multiplication, Division)
but because we want to know if the blocks of
this partition are distinct.
36Complexity Classes are not Invented Theyre
Discovered
- NP (SAT, Clique, TSP,)
- P (Linear Programming, CVP, )
- NL (Connectivity, Shortest Paths, 2SAT, )
- L (Undirected Connectivity, Acyclicity, )
- NC1 (BFE, Regular Sets)
- TC0 (Sorting, Multiplication, Division)
These classes are real. Theyre important.
37Longstanding Open Problems
- Is P NP?
- Is AC06 NP?
- Is depth 3 AC06 NP?
Well focus on questions such as Is BFE in
TC0? Is BFE in AC06?
38How Close Are We to Proving Circuit Lower Bounds?
- Conventional Wisdom Not Close At All!
- No new superpolynomial size lower bounds in over
two decades. - Razborov and Rudich Any natural argument
proving a lower bound against a circuit class C
yields a proof that C cant compute a
pseudorandom function generator. - Since the Naor, Reingold generator is
computable in TC0, this is bad news.
39More Modest Goals
- Problems requiring formulae of size n3 Håstad
- Problems requiring branching programs of size
nearly n loglog n Beame, Saks, Sun, Vee - Problems requiring depth d TC0 circuits of size
n1c Impagliazzo, Paturi, Saks - Time-Space Tradeoffs Fortnow, Lipton, Van
Melkebeek, Viglas - There is little feeling that these results bring
us any closer to separating complexity classes.
40How Close Are We to Proving Circuit Lower Bounds?
- How close are the following two statements?
- TC0 Circuits for BFE must be of size n1O(1)
- For some cgt0, TC0 Circuits for BFE must be of
size n1c.
41How Close Are We to Proving Circuit Lower Bounds?
- How close are the following two statements?
- TC0 Circuits for BFE must be of size n1O(1)
- For some cgt0, FTC0 Circuits for BFE must be of
size n1c
This is known IPS97
This implies TC0 ? NC1 A, Koucky
42Self-Reducibility
- A set B is said to be self-reducible if B P B
43Self-Reducibility
- A set B is said to be self-reducible if B P B
via a reduction that, on input x, does not ask
about whether x is in B. - Very well-studied notion.
- For example, f is in SAT if and only if
(f0 is in SAT) or (f1 is in SAT)
44Self-Reducibility
- Many of the important problems in (or near) NC1
have a special self-reducibility property
45Self-Reducibility
- Many of the important problems in (or near) NC1
have a special self-reducibility property
Instances of length n are AC0-Turing (or
TC0-Turing) reducible to instances of length n½
via reductions of linear size. - Examples
- BFE
- the word problem over S5
- MAJORITY
- Iterated Product of 3-by-3 Integer Matrices
46Self Reducibility
A subformula near the root
Subformulae near inputs
47Self Reducibility
48Self Reducibility
- The self-reduction of S5, on inputs of size n,
uses (n½ 1) oracle gates of size n½. - Thus if S5 has TC0 circuits of size nk, it also
has circuits of size (n½ 1)nk/2 O(n(k1)/2). - Similar arguments hold for other classes (such as
AC06 and NC1). - More complicated self-reductions can be presented
for MAJORITY and Iterated Product of 3-by-3
matrices.
49A Corollary
- If BFE has TC0 or AC06 circuits, then it has
such circuits of nearly linear size. - If S5 has TC0 or AC06 circuits, then it has
such circuits of nearly linear size. - If MAJ has AC06 circuits, then it has such
circuits of nearly linear size. (Etc.) - Thus, e.g., to separate NC1 from TC0, it suffices
to show that BFE requires TC0 circuits of size
n1.0000001.
50A Corollary
- If BFE has TC0 or AC06 circuits, then it has
such circuits of nearly linear size. - If S5 has TC0 or AC06 circuits, then it has
such circuits of nearly linear size. - If MAJ has AC06 circuits, then it has such
circuits of nearly linear size. (Etc.) - How widespread is this phenomenon? Is it true
for SAT? (I.e., can we show NP ? TC0 by proving
that SAT requires TC0 circuits of size
n1.0000001?)
51Limitations of Self-Reducibility
- Any problem for which instances of length n are
TC0-Turing reducible to instances of length n½
via poly-size reductions lies in NC. - Thus there is no obvious way to apply these
techniques to SAT or to problems complete for P. - but perhaps, rather than showing directly that
SAT has this strong form of self-reducibility,
one can argue that if SAT is in TC0 then it has
TC0 circuits of nearly-linear size.
52Limitations of Self-Reducibility
- Any problem for which instances of length n are
TC0-Turing reducible to instances of length n½
via poly-size reductions lies in NC.
53Limitations of Self-Reducibility
- Any problem for which instances of length n are
TC0-Turing reducible to instances of length n½
via poly-size reductions lies in NC.
d levels of oracle gates
54Limitations of Self-Reducibility
- Any problem for which instances of length n are
TC0-Turing reducible to instances of length n½
via poly-size reductions lies in NC.
d2 levels of oracle gates
55Limitations of Self-Reducibility
- Any problem for which instances of length n are
TC0-Turing reducible to instances of length n½
via poly-size reductions lies in NC.
After log log rounds, the depth is logO(1)n
d3 levels of oracle gates
56Prospects for Progress
- We have seen that existing techniques prove
bounds that are nearly good enough to separate
NC1 and TC0. Some of these proofs are natural. - Dont the results of Razborov Rudich indicate
that further progress will require very different
approaches? - Not necessarily!
57Prospects for Progress
- The Razborov Rudich framework of natural
proofs assumes that a natural proof of a lower
bound will make use of a combinatorial property
that (among other things) is shared by a large
fraction of the functions on n bits. - In contrast, we are making use of a
self-reducibility property that allows us to
boost a n1e lower bound to a superpolynomial
lower bound. This self-reducibility property
holds for only a vanishingly small fraction of
all functions.
58Prospects for Progress
- These observations are simple, but
- they have forever changed the way that we look at
quadratic (and smaller) lower bounds. - We are not claiming to have found a way around
the obstacles identified by Razborov Rudich.
(Such a claim will have to wait until someone
proves that NC1 ? TC0.) But we do believe that
this avenue deserves further exploration.
59Conclusion
- There are good reasons to develop and explore the
connections between Kolmogorov complexity and
circuit complexity. - There are many open problems in this area that I
will be delighted to discuss with you in more
detail. - There are two bad typos in the proceedings
version of the paper. (P should be NP.) A
corrected version is available at my home page.
60Speculation
Connections between Kolmogorov Complexity and
Circuit Complexity might be relevant to the
question of whether NEXP is contained in
(non-uniform) TC0 (depth 3).
61Speculation
- IKW showed that NEXP is in P/poly iff
NEXP MA
iff MA cannot be derandomized - The proof shows that NEXP is in P/poly iff
every set in P contains strings of KT-complexity
O(log n) iff
NEXP IPP/poly.
62Speculation
- Similar techniques show
- NEXP is in nonuniform NC1
iff every set in P contains strings of
KF-complexity O(log n)
iff NEXP MIPNC1
iff MIPNC1 cannot be derandomized. - NEXP is in nonuniform TC0
iff every set in P contains strings of small
complexity
iff NEXP MIPTC0
iff MIPTC0
cannot be derandomized.
63Speculation
- What else happens in such a collapse?
- If NP uniform TC0, then P is not contained in
non-uniform TC0 (so NEXP is not in non-uniform
TC0). - So lets consider NEXP MIPTC0 and
NP ? uniform TC0. If this hardness assumption
were sufficient to derandomize MIPTC0 then this
would give the desired lower bound on NEXP - Fortnow, Klivans, van Melkebeek, Santhanam