Course 9 NP Theory?? An Introduction to the Theory of NP

1
Course 9NP Theory??An Introduction to the
Theory of NP
2
Outlines

• ????
• Polynomial Time
• Intractability
• Optimization Problems vs. Decision Problems
• The Theory of NP
• P problem
• NP problem
• NP-complete problem
• NP-hard problem
• ?????????NP-complete??

3
Polynomial Time (?????)

• ??? ????? (Polynomial Time)?

Polynomial Time
Non-Polynomial Time
4

• Def
• ?????????????(Polynomial-time Algorithm)
  ???????????? (input size)?,?????????(Worst-case)?
  ??????????????
• ??,?n?input size,????????? p (n) ,?
• Polynomial-time computable
• ??? f(x) ?polynomial-time computable,??????????,??
  ?????? x,???Polynomial Time??? f?

5
Intractability (????)

• ?????????????,?????????????
• ?? 1 ?????????????????????,????????????????????
  ?
• ?? 2 ????????????????
• ???????????????,??????????????????
• ?????????,??????????????????? (tractable) ?????
  (intractable)?

6

• ????????,????????(Worst-case)?,?????????????????,?
  ???????? (Intractable)???
• ???????,????????????????????
• ??,????????????(Worst-case)?,????????Polynomial-Ti
  me Algorithm??,?????????????Polynomial-Time
  Algorithm??????,?????????Intractable.
• For example
• ??,?? brute-force algorithm (?????)
  ?????????(Chained Matrix Multiplication problem)
  ,???????non-polynomial time.
• ??,?? dynamic programming algorithm (Algorithm
  3.6) ??,????????T(n3).

7

• ??????,????????????
• Problems for which polynomial-time algorithms
  have been found
• ????????MST????????????
• Problems that have been proven to be intractable
• ??????????????(??)??????????
• ??????????????????????? (Halting
  Problem)???????(Equivalence Problem)
• Problems that have not been proven to be
  intractable, but for which polynomial-time
  algorithms have never been found
• ?????????????,????????
• ??????,?????????????? (Traveling Salesman Problem)

8
The Traveling Salesman Problem TSP
9

• TSP??
• ????????????????n????(A salesman spends his time
  visiting n cities cyclically. )
• ???????,????????????,??????????????????????????(In
  one tour he visits each city exactly once, and
  finishes up where he started.)
• ??????????????????????(??)???(In what order
  should he visit the cities to minimize the
  distance traveled?)

10

• ????????TSP??,????????????????????????
  (Exponentially) ???!!
• 3 cities ? 1 solution.
• 10 cities ? 181,440 possible tours
• n cities ? (n-1)!/2 possible tours

11

• ? n26,?? 25! /2?????
• 25!15511210043330985984000000?1.55 x
  1025????????,??????
• ????????? 106 ??????,??? 3.15 x 107?, ?????? 3.15
  x 1013???,???????????
• ???????? n26,???????,???????????

12
Optimization Problem vs. Decision Problem

• ????????,???????????????? (Optimization Problem)
  ??,??????????? ???? (Decision Problem)
• Decision Problem ?????????????,?? yes ? no
  ?????

13

• The partition problem (????)
• ??????????Sa1, a2, , an,?
  ??????????????S1?S2,???????????????
• Ex Let S 13, 2, 17, 20, 8. The answer to
  this problem instance is "yes" because we can
  partition S into S1 13, 17 and S2 2, 20,
  8.
• The Sum of Subset Problem (????????)
• ??????????Sa1, a2, , an?????c,?
  ??S??????????S,????S??????c?
• Ex Let S 12, 9, 33, 42, 7, 10, 5 and c 24.
  
• The answer of this problem instance is "yes" as
  there exists S 9, 10, 5 and the sum of the
  elements in S is equal to 24.
• If c is 6, the answer will be "no".

14

• The Satisfiability Problem (???? SAT)
• ???????E,?????????E??????????True?False,????????Tr
  ue?
• Ex Let E (-x1 ? x2 ?- x3)?(x1 ? -x2 ) ?(x2 ?
  x3). Then the following assignment will make E
  true and the answer will be yes.
• x1 ?F, x2 ?F, x3 ?T
• If E is -x1 ? x1 , there will be no assignment
  which can make E true and the answer will be
  no.
• ?????????????NP-Complete??? (by S. A. Cook,
  1971).
• The Minimal Spanning Tree Problem (???????)
• Given a graph G, find a spanning tree T of G with
  the minimum length.

15

• The Traveling Salesperson Problem (???????)
• ?????G (V,E) ,?????????????????cycle,?cycle?????
  ??????????????,?????????
• Ex Consider following graph. There are two
  cycles satisfying our condition. They are C1
  a?b?e?d?c?f?a and C2 a?c?b?e ?d?f?a. C1 is
  shorter and is the solution of this problem
  instance.

16

• Some problems
• The Partition Problem
• The Sum of Subset Problem
• The Satisfiability Problem
• The Minimal Spanning Tree Problem
• The Traveling Salesperson Problem
• ????,?????(Optimization Problems)?????(Decision
  Problems)??????!!

(Decision Problems)
(Optimization Problems)
17

• ?????????????????????
• For example (MST Problem)
• Given a graph G and a constant c.
• The total length of the spanning tree of the
  graph G is a
• If a lt c, then the answer is yes,
• otherwise, its answer is no.
• ??????????????,?????????????(The minimal spanning
  tree decision problem)
• ????????????????(Decision version of the
  optimization problem)??????,?????????????????

18
The Theory of NP

• ??????????????,???????????????????????????
• ??,???????????,??????,???????

???????,??????!
19

• ??,??????????????,??????????????????,????????

???????,???????????!
20

• ??????????,????????????????????
• ??,???????????,?????!!??????????????????,?????????
  ???????
• ???????

21

• ??,?????????????????,????????????
• ?????????????????????(TSP)???????????????,????????
  ?????????????,????????????????TSP??????,??????????
  ?????????

22

• ????????!! ??,????????????????????????????????????
  ???????? (???????????),?????????????????????????
  (???????????)
• ??????????????????????????

23

• ?????,??????????????

???????,???????????!
24

• ??
• Computer Science????????????????????
• ???????(??)?????????????????
• ???????????????
• ?????????????,??????,???????????

25
Deterministic v.s. Non-deterministic

• ???????????????????,??????????,????? ???? ?
  ???? ??????????????????????????????????

???? Yes/No
????????
?????? (Deterministic Algo.)
??????? (Non-deterministic Algo.)
??????
26

• Deterministic Algorithm (??????)
• Def ???????????,?????????????????(Permitting at
  most one next move at any step in a computation)
• ??????????????????????,?????????????
• ?????????????,???????? (Deterministic
  Machine)??????????????
• ?????????????????????,????????,??,??????????,?????
  ?

27

• Non-deterministic Algorithm (???????)
• Def ???????????,??????????????????????
  (Permitting more than one choice of next move at
  some step in a computation)
• ????????????????????
• ??????????????,????????? (Non-deterministic
  Machine)?
• ?????????????,??????????????,?????????????????????
  ???????,?????????????????????????
• ??,????????????

28

• Non-deterministic Algorithm???????????
• ????(Guess)
• ??????????????????????,??????Non-deterministic
• ?????,????????
• ????????????,???????????????????,???????????,????
  ????????
• ???????????????????,??????(????????????) ?
• ????(Verification)
• ???????????????????? (True)

29
Nondeterministic SAT

• ?????n????????E????? (SAT) ??
• ??????????????,???????O(2n)
• ???????????????,???????O(n)
• / Guess /
• for i 1 to n do
• xi ? choice(true, false)
• / Verification /
• if E(x1, x2, ,xn) is true then
• success
• else failure

30
Polynomial-Time Reducible (????????)

• Def ??????Q1?Q2,???????L1?L2
• ??Q1??????????Q2 (?L1 pL2 ? Q1 pQ2),?
• ?????? f(x) /??????????????,?????????????/
• ??? f(x) ?polynomial-time computable
• ??? f(x) ?????x??,x?L1 ???? f(x)?L2 (x?L1 ?
  f(x)?L2)
• ??Reduce????? ???Q1?Q2????????????

f(x)
???L1????,?f(x)?????L2??????L1????,?f(x)???????L2
?
31

• ????Reduce?
• Q1 reduce ?Q2,??Q1???????Q2??????????

Reduce
?Q2?Algo.
Q1?Input
Q2?Input
?f(x)
Reduce
Q1?Output
Q2?Output
? x?L1 ? f(x)?L2
32

• ??????????
• Q1???????Q2???????????Q2????????????,??????????Q2?
  ????,??Q1??????????????????
• ??,?? f(x) ???? polynomial-time computable?
• ?????Q1???????? ??f(x)????? ?Q2???????????
  ????
• ???????,??Q2???????????,???????Q1??????
• ???? L1 ? L2 ? Q1 ? Q2

33

• Q1 ? Q2???
• Q2???Q1??? (????????????)
• ?Q2??,?Q1???
• ????Q1?Q2???,????Q1 pQ2 ?Q2 pQ1
• ?Q1 ? Q2?Q2 ? Q3,?Q1 ? Q3 (???)
• ?????????????
• Q1 ??????4??,???????????????
• Q2???4???????G(V,E),????????????
• ??Q1 ? Q2?

34

• ?
• ?????f,??
• ?i ? ??vi
• ?i ? ?j ???? ? (vi, vj)?E
• ?i ? ?j ???? ? (vi, vj)?E
• ?????f ?polynomial time computable
• ???????G(V,E),????????,??????? V2?????????????
  O(V2)?
• ???f?Polynomial time computable
• ?????f(x)?????x??,x?L1 ? f(x)?L2
• ??????? ? ?????????G?
• (???)?I ? ?j? ?k???? ? vi, vj?vk??????? vi,
  vj?vk?????
• (???)(????),?I ? ?j? ?k????
• Q1 ? Q2??

????????????
35
P, NP, NP hard, NP complete

• P
• ???Decision Problem???,????????Deterministic
  Algorithm?Worst Case????,?Polynomial
  Time?????????
• NP
• ?? NP ??Non-deterministic?Polynomial???????
• ???????????,???Polynomial Time?????????? (Verify)
  ??????????
• ?????????????????????,??????????????????,P???NP???
  ???
• P?NP (?)
• PNP (?)

36

• NP-hard
• ???Decision Problem????? Q ? NP-hard
  ????????NP?Decision Problem Q (?Q?NP)
  ??????????Q (Q?Q)
• ?????????????????????????,???????????????????????
• NP-complete
• ???Decision Problem???? ??????Q??NP-complete,?????
  ????
• Q??NP
• Q??NP-hard

37

• ????,?????????????????
• NP-hard?NP-complete???
• ??NP-complete????NP-hard?? (????????)??NP-hard??
  ????NP-complete?? (???????,???NP??)
• NP-complete ? NP-hard
• ?????NP-hard???????????????????,???NP-complete????
  ????????????????

NP
NP-hard
NP- complete
P
38
Summary

• Nearly all of the decision problems are NP
  problems.
• In NP problems, there are some problems which
  have polynomial algorithms. They are called P
  problems.
• Every P problem must be an NP problem.
• There are a large set of problems which, up to
  now, have no polynomial algorithms.

39

• Some important properties of NP-complete
  problems
• Up to now, no NP-complete problem has any worst
  case polynomial algorithm.
• If any NP-complete problem can be solved in
  polynomial time, NP P.
• If the decision version of an optimization
  problem is NP-complete, this optimization problem
  is called NP-hard.
• We can conclude that all NP-complete and NP-hard
  problems must be difficult problems because
• They do not have polynomial algorithms at
  present.
• It is quite unlikely that they can have
  polynomial algorithms in the future.

40
?????????NP-complete??

• ??????NPC???
• ???NP-complete?????????????,????NP-complete???????
  ???????????????,??????NP-complete?????????????????
  ???
• ???????Q?NPC,????????
• ??Q?NP (can guess an answer, and check it in
  polynomial time)
• ??????NPC?? Q ,??Q?Q
• ??????f(x),
• ???f(x)?polynomial-time computable,
• ???f(x)?????x??,x?L1 ???? f(x)?L2 ?
• (L1?Q??????L2?Q??????)

(??)
(??)
41
???????????????Q?NPC

• ??
• ?? 1 ?????Q???NP??
• ?? 2 ?????Q???NP-hard??
• ??NP-complete??????NP???????NP-hard???? Q
  ??????NP-complete??,?? Q ????NP??,???NP-hard???
• ?? Q ?NP-hard?????,???????????NP?? ? Q?
• ?? ???NP?? ? Q ?????????Q?Q,?????,???????NP??
  ? Q (???NP?? ? Q ? Q)??? Q ???NP-hard???
• Q??????NP??,????NP-hard??,?Q???NP-complete???

42

• ???? (Cooks Theorem)
• SAT??NP-Complete
• (SAT?NPC)
• ????????????????NPC??
• ??????????NPC??,??????????????
• ??NPC???????????? (??????)?

43

• ??????
• 3-SAT???NP-Complete
• Clique (??) ???NP-Complete
• Vertex-Cover (????) ???NP-Complete
• Dominating Set (???) ???NP-Complete

44
????3-SAT???NP-Complete

• ??3-SAT??
• ?SAT???????????SAT??,??????????????3???,??????????
  E??????????True?False,????????True?
• Ex Let E (-x1 ? x2 ?- x3)?(x1 ? -x2 ?- x4) ?
  (-x5 ? x2 ? x3). Then the following assignment
  will make E true and the answer will be yes.
• x1 ?T, x2 ?T, x3 ?T, x4 ?F, x5 ?F
• ?????????Q?NPC,?
• ??Q?NP (can guess an answer, and check it in
  polynomial time)
• ??????NPC?? Q,??Q?Q
• ??????f(x),
• ???f(x)?polynomial-time computable,
• ???f(x)?????x??,x?L1 ???? f(x)?L2 ?

45
??Q?NP

• Can guess an answer, and check it in polynomial
  time
• ???????,???????????????3????
• ????????????????????????O(n),???????????O(1)?????P
  olynomial Time?????????????????,???????NP???
• / Guess /
• for i 1 to n do
• xi ? choice(true, false)
• / Verification /
• if E(x1, x2, ,xn) is true then
• success
• else failure.

46
??Q?Q

• SAT????????NPC problem (by ????)???,?????Q,?????3
  -SAT???Q?
• ??Q?Q??????????
• ????????f(x),???Q????????Q?????
• ???f(x)???polynomial-time computable,
• ???f(x)?????x??,x?L1 ? f(x)?L2
• (L1?Q??????L2?Q??????)

47

• ????? 1?????????f(x),???Q????????Q??????
• ????SAT???????E,?????? E ?????????????????????????
  E,????? E ??????? E ????????????,??? Q ? Q
  ?????????? f(x)?
• ?????,Q?Q??????????f(x)

???? ?? E ????? ?????? E ????
1 (x1) (x1 ? y1 ? y2) ? (x1 ? ?1 ? y2) ? (x1 ? y1 ? ?2) ? (x1 ? ?1 ? ?2)
2 (x1 ? x2) (x1 ? x2 ? y1) ? (x1 ? x2 ? ?1)
3 (x1 ? x2 ? x3) ?????
??3 (x1 ? x2 ? ? xk) (x1 ? x2 ? y1) ? (?1 ? x3 ? y2) ? (?2 ? x4 ? y3) ? ? (?k-4 ? xk-2 ? yk-3) ? (?k-3 ? xk-1 ? xk)
48

• ????? 2????f(x)???polynomial-time computable?
• ???? SAT ??????? E ? n ???,????????? k ???,??? E
  ??? 3-SAT ??????? E ?????????????? O(nk)?
• ?SAT??????? E ?,?????????
• ???????????,????????????? (?????????)?
• ???????,????????,????? 0 ?
• ???????? (?k gt 3) ?,?????????? k-2,??????O(k)?
• ?????n???,???????????????????? O(nk)
• ?????,??????f(x)?polynomial-time computable?

49

• ????? 3????f(x)?????x??,x?L1 ???? f(x)?L2 ?
• 
  (?L1?Q??????L2?Q??????)
• ?????,??? ??????SAT??????? E ?True ? ??????
  3-SAT ??????? E ?True
• (E is satisfiable ? E? is satisfiable )
• ??? E is satisfiable ? E? is satisfiable?
• ?????SAT??????? E ?True,?????????True?
• ????????True,??????????????True?
• ?
• ???????SAT??????? E ?True??,???????3-SAT???????
  E ?True,?????? E ???????????True?
• ????SAT????True? xi ????????? yi
  ????False???SAT????False?????????? yi
  ????True,?????? E?True

E is True
E is True
50

• ??? E is satisfiable ? E? is satisfiable?
• ?????3-SAT??????? E ?True,?????????True?
• ????????True,???? yi ????,??????????xi???True?
• ?
• ???????3-SAT??????? E ?True??,????SAT??????? E
  ?True?
• ? ? ? ? ???,???? ??????SAT??????? E ?? ? ??????
  3-SAT ??????? E ??
• ????????,?????? SAT ? 3-SAT (?Q ? Q)?
• ?????,?????? 3-SAT?? ? NP-Complete Problem

E is True
E is True
51
????

• The instance E? in 3-SAT
• x1 ? x2 ? y1
• x1 ? x2 ? -y1
• -x3 ? y2 ? y3
• -x3 ? -y2 ? y3
• -x3 ? y2 ? -y3
• -x3 ? -y2 ? -y3
• x1 ? -x2 ? y4
• -y4 ? x3 ? y5
• -y5 ? -x4 ? y6
• -y6 ? x5 ? x6

• An instance E in SAT
• (x1 ? x2)?(-x3) ?(x1 ? -x2 ? x3 ? -x4 ? x5 ?
  x6)
• x1 v x2
• -x3
• x1 v -x2 v x3 v -x4 v x5 v x6

f(x)
SAT 3-SAT E
E?
????
52
????? (Approximation Algorithm)

• ????Q?????????,?????NP-complete??,????????????????
  ?????(????Polynomial Time???) ?
• ??,????NP-complete?????????????!!?????????,???????
  ????????,????????????????????Approximation
  Algorithm????
• ?????????????Issue
• ?????????????? (????Polynomial Time)
• ??????????????????????
• ???????,??????????????????????

53
Approximation Ratio

• Approximation Ratio???????????? ??? ???
• ????????,??????x????,??????Opt(x),????????A??????A
  (x)????????A?e-approximation,???
• ??,e????????Approximation Ratio?
• ???????,????????????? (???) e?

54

• ???????????????????????
• ???????,? ?? ?,??
  ?????Approximation ratio?
• ??TSP??????NP-complete?????????????????????
• Algo. 1 ??????????????? (??9.6)minapprox lt 2
  mindist
• Algo. 2 ??????????????? (??9.7)minapprox2 lt
  1.5 mindist
• ???????,?????? ?? ?,??
  ?????Approximation ratio??