Randomized Algorithms (2-SAT - PowerPoint PPT Presentation


PPT – Randomized Algorithms (2-SAT PowerPoint presentation | free to download - id: 6d551a-MWI3Y


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Randomized Algorithms (2-SAT


Randomized Algorithms (2-SAT & MAX-3-SAT) (14/02/2008) Sandhya S. Pillai(2007MCS3120) Sunita Sharma(2007MCS2927) Advanced Algorithms Course (CSL 758) – PowerPoint PPT presentation

Number of Views:13
Avg rating:3.0/5.0
Date added: 28 April 2020
Slides: 43
Provided by: erne151


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Randomized Algorithms (2-SAT

Randomized Algorithms (2-SAT MAX-3-SAT) (14/02/2
  • Sandhya S. Pillai(2007MCS3120)
  • Sunita Sharma(2007MCS2927)
  • Advanced Algorithms Course (CSL 758)
  • Prof. Kavitha Telikepalli Prof. Naveen Garg

  • Introduction
  • 2-SAT Problem
  • Monte Carlo vs. Las Vegas methods
  • Analysis of 2-SAT
  • Random Walks and Markov inequality
  • 3-SAT Problem
  • Why 3-SAT is NP-Hard?
  • Max 3-SAT Problem randomization
  • References

Randomized Algorithm
  • In addition to input, algorithm takes a source of
    random numbers and makes random choices during
    execution. Behavior can vary even on a fixed

  • Satisfiability (SAT) is the problem of deciding
    whether a boolean formula in propositional logic
    has an assignment that evaluates to true. SAT
    occurs as a problem and is a tool in applications
    (e.g. Artificial Intelligence and circuit design)
    and it is considered a fundamental problem in
    theory, since many problems can be naturally
    reduced to it and it is the 'mother' of
    NP-complete problems.

2-SAT Problem
  • f (x1 Vx2) (x1 V x3) (x1 V x3) (x1 V
  • The k-SAT problem is the variant of SAT, in which
    each clause consists of exactly k distinct
    literals. For k gt 3, k-SAT is NP hard, but for k
    1 and 2, there are polynomial time solutions.
  • For k 1, this solution is trivial
  • But for k 2, it is slightly tricky. There is a
    much easier random algorithm.
  • For a system with literals Xi, 0 lt i lt m, and
    clauses Cj, 0 lt j lt n, it goes as follows
  • SATGiven a boolean formula ? in
    CNF,determine if ? is satisfiable or not.

  • Given a Boolean formula f in CNF, determine,
    whether f is satisfiable or not.!
  • Ex AND of clauses
  • (A1 V A2 V A3 V A4)? (A1 V A3 V A4) ? (A1 V
    A2 V A3 )
  • Each clause has at least one literal set to true.
  • If A1 true, A2 true then
  • Clause 1 and 2 are satisfiable but the Clause 3
    requires A3 literal set to be false for it to be
    a satisfiable one.

  • Input A Boolean formula f in conjunctive normal
    form with exactly two distinct literals in every
    clause. E.g., f (x1 Vx2) (x1 V x3) (x1
    V x3) (x1 V x2)
  • 1)/ Start with an arbitrary initial assignment
    to the literals/
  • for (i 0 i lt m i) xi true Let us
    call this assignment A
  • 2)/ here function number_satisfied() returns the
    number of satisfied clauses for the current
    assignment /
  • for (t 0 t lt T number_satisfied(x, n) lt n
  • select an arbitrary non-satisfied clause CJ
  • randomly and uniformly pick one of the literals
    xi in CJ
  • xi (xi 1) mod 2
  • 3) If (number_satisfied(x, n) n) report
    that the set of clauses is satisfiable
  • else report that the set of clauses is not

  • Function number_satisfied() can be computed in
    linear time, some savings might be achieved by
    only keeping track of the changes between rounds.
    However, computation time is not so much the
    issue here, the main point is answering the
    question how large T must be taken to be
    reasonable sure that this Monte-Carlo algorithm
    gives the correct answer for satisfiable systems
  • There is a chance of error and we need to bound

Monte Carlo vs. Las Vegas methods
  • Definition A Las Vegas algorithm is a randomized
    algorithms that always return the correct result.
    The only variant is that its running time might
    change between executions.
  • An example for a Las Vegas algorithm is the
    QuickSort algorithm.
  • Definition A Monte Carlo algorithm is a
    randomized algorithm that might output an
    incorrect result. However, the probability of
    error can be diminished by repeated executions of
    the algorithm.
  • The MinCut algorithm is an example of a Monte
    Carlo algorithm.

Bounding the Error in Algorithm
  • This algorithm will always give correct
    non-satisfiable instances of ?
  • But if ? is satisfiable Then we need to fix value
    of T such that this algorithm says not
    satisfiable with a probability lt1/4
  • we will assume in the following that there is a
    satisfiable assignment to the xi Let call it S
    There may even be several such assignments, but
    we concentrate on a single one, which we will
    refer to as the correct assignment.
  • The above process can be modeled as a random
    walk. The graph is the line graph with n nodes
    node y is connected to node y - 1 and node y 1,
    as far as these indices are at least 1 and at
    most n.

Random Walk
  • Let G (V,E) be a connected, undirected graph. A
    random walk on G, starting from
  • vertex s ? V , is the random process defined as
  • u s
  • repeat for T steps
  • choose a neighbor v of u uniformly at random.
  • u v
  • Given G, this is obviously trivial to implement
    on a computer.

Consider a particle moving in an one-dimensional
line. At each point in time, the particle will
move either1 step to the right with probability p
or 1 step to the left with probability 1 -
p. Analysis let A 2 0,1n be any satisfying
assignment With probability at least ½ distance
to A is reduced With probability at most ½
distance to A is increased
Randomized 2-SAT Analysis
  • Distance can never be larger than n
  • if it starts at some 0 lt i lt n
  • Dominated by a walk where
  • With probability exactly ½ distance to A is
  • With probability exactly ½ distance to A is

Analysis contd..
  • Let us define a random variable Xi
  • Xi of steps to reach state n starting from
    state i
  • Xi1 of steps to reach state n starting from
    state i1 with probably ½ (Xi1)
  • Xi1 of steps to reach state n starting from
    state i-1 with probably ½ ( Xi-1)
  • this is a memory less property There for start
    from the current state is a fresh start.

Analysis contd..
  • EXi1/2 E1Xi11/2E1 Xi-1
  • 1(E Xi-1 EXi1)/2
  • Let SiE Xi Then
  • Si1( Si-1Si1)/2
  • Sn-11( Sn-2 Sn)/2

Analysis contd..
  • S0S11 ............(1)
  • 2S1S0S22 .............................(2)
  • .
  • .
  • 2Sn-1Sn-22..............................(n)
  • adding equations 1 to n we get
  • Sn-12(n-1)12n-1
  • Sn-2(2n-1)(2n-3)
  • .
  • S0(2n-1)(2n-3)..............31n2

Analysis contd..
  • Let us define another random variable Y
  • Y of variables whose truth value is same in
    current assignment(A) satisfiable assignment(S)
  • Movement of Y is similar to
  • i-1 ? i ? i1
  • Now we have to fix T

Markov Inequality
Markov's inequality gives an upper bound for the
probability that a non-negative function of a
random variable is greater than or equal to some
positive constant Proposition 1For any
non-negative random variable Y and any
real number k gt0 we have Pr Ygtk lt E Y /
k As an example let k 2E Y . Then the above
says Pr Ygt 2 E Y lt 1/2. Namely, if
you move out to twice the expectation, you can
have only half the area under the curve to your
right. This is quite intuitive.

Proof for Markov Inequality
EY? PrYyy ? PrYyy? PrYyy
yltk ygtk gt 0 k Pr Ygtk Pr Ygtk
ltE Y / k Therefore T4n2
  • Prf is satisfiable but algorithm returns
    unsatisfiable Prour algorithm does not reach
    state n in 4n2 steps
  • Pr Zgt4EZ lt ¼
  • Z of steps to reach state n from our start
  • EZlt n2

Practical Example of Satisfiability
  • Circuit satisfiability is a good example of
    this problem that we don't know how to solve in
    polynomial time. In this problem, the input is a
    boolean circuit a collection of and, or, and not
    gates connected by wires. We will assume that
    there are no loops in the circuit (so no delay
    lines or ip-ops). The input to the circuit is a
    set of m boolean (true/false) values x1
    xm. The output is a single boolean value. Given
    specific input values, we can calculate the
    output in polynomial (actually, linear) time
    using depth-first-search and evaluating the
    output of each gate in constant time.

Why NP hard ?
  • The circuit satisfiability problem asks, given a
    circuit, whether there is an input that makes the
    circuit output True, or conversely, whether the
    circuit always outputs False. Nobody knows how to
    solve this problem faster than just trying all 2m
    possible inputs to the circuit, but this requires
    exponential time. On the other hand, nobody has
    ever proved that this is the best we can do
    maybe there's a clever algorithm that nobody has
    discovered yet!
  • Hence this comes under NP hard problem.

  • A special case of SAT that is incredibly useful
    in proving NP-hardness results is 3SAT (or
  • A boolean formula is in conjunctive normal form
    (CNF) if it is a conjunction (and) of several
    clauses, each of which is the disjunction (or) of
    several literals, each of which is either a
    variable or its negation.

For example
  • (A1 V A2 V A3 )? (A1 V A3 V A4) ? (A1 V A2
    V A3 )
  • Given such a boolean formula, can we come up with
    an algorithm, that is polynomial in time ? The
    answer to this question is NO!!! Hence this is NP
    hard problem.

  • We could prove that 3SAT is NP-hard by a
    reduction from the more general SAT problem,but
    it's easier just to start over from scratch, with
    a boolean circuit. We perform the reduction in
    several stages.
  • 1. Make sure every and and or gate has only two
    inputs. If any gate has k gt 2 inputs, replace it
    with a binary tree of k-1 two-input gates.
  • 2. Write down the circuit as a formula, with
    one clause per gate. This is just the previous

  • 3. Change every gate clause into a CNF formula.
    There are only three types of clauses, one for
    each type of gate
  • A1 A2 ?A3 -gt (A1 V A2 V A3 ) ?(A1 V A2 )
    ? (A1 V A1)
  • A1 A2 V A3 -gt ( A1 V A2 V A3) ? (A1 V
    A2 ) ? ( A1 V A3 )
  • A1 A2 -gt (A1 VA2 ) ? ( A1 V A2 )
  • 4. Make sure every clause has exactly three
    literals. Introduce new variables into each one-
    and two-literal clause, and expand it into two
    clauses as follows
  • A1-gt ( A1 V e V u) ? (A1V e V u) ? (A1 V e
    V u) ? (A1 V e V u )
  • A1 V A2 -gt (A1V A2 V e) ? (A1 V A2V e )

If we start with the above example formula , we
obtain the following 3CNF formula.
  • Although the 3CNF formula is complicated than the
    original one at first glance, it's actually only
    a constant factor larger. Even if the formula
    were larger than the circuit by a polynomial,
    like n373, we would have a valid reduction.
  • The formula is satisfiable if and only if the
    original circuit is satisfiable. As with the more
    general SAT problem, the formula is only a
    constant factor larger than any reasonable
    description of the original circuit, and the
    reduction can be carried out in polynomial time.
    Thus, we have a polynomial-time reduction from
    circuit satisfiability to 3SAT

  • TCSAT(n) lt O(n) T3 SAT(O(n)) gt T3 SAT(n) gt
    TCSAT(( ? (n)) - O(n)
  • So 3SAT is NP-hard. Finally, since 3SAT is a
    special case of SAT, it is also in NP, so 3SAT is

Max 3-CNF
  • This problem is to find an assignment which
    maximizes the number of satisfiable clauses.
  • Let us take for example
  • (A1 V A2 V A3) (A1 V A2 VA4) ( A2 V A3 V

Randomized Algorithm for Max 3-CNF
  • Set each variable to true with probability 1/2
    independently. (i.e. for instance, toss the coin
    and if head, set the variable to true and if
    tail, set the variable to false).
  • If a clause is not satisfied, this means all the
    3 variables in the clause are false.
  • Prob. that a clause is not satisfied 1/2 1/2
  • (since each variable is independently set to
  • Prob. that a clause is satisfied 1 - Prob. that
    a clause is not satisfied
  • 1 - 1/8 7/8

  • There may be dependent clauses too.
  • Eg. (A1 V A2 V A3) ? ( A1 V A2 V A3)
  • In the above two clauses the first clause is true
    and hence the second clause is bound to be true
    if x2 true or x3 true or both are true.
  • Each clause is satisfied with prob. 7/8
  • Let m be the no. of clauses.
  • Expected number of satisfiable clauses can be
    found as follows.
  • Let Xi be a random variable.
  • Xi 1 if ith clause is satisfied.
  • 0 otherwise
  • X ?i Xi
  • Expected value of X
  • EX EX1 X2 ... Xm

  • Linearity of expectation Let r be any real
    number and let X and X1 , . . . , Xn be random
    variables on a discrete probability space ? such
    that their expectations all exist. Then the
    expectation of rX and of X1 . . .Xm exists and
    we have
  • E rX rE X and E X1 . . .
    Xn E X1 . . . E Xn .
  • Since linearity of expectations always holds,
  • EX EX1 EX2......... EXm
  • 7/8 7/8 ............ 7/8
  • 7/8 m

Approximation Algorithm
  • Approximation algorithms are algorithms used to
    find approximate solutions to optimization
    problems. Approximation algorithms are often
    associated with NP-hard problems since it is
    unlikely that there can ever be efficient
    polynomial time exact algorithms solving NP-hard
    problems, one settles for polynomial time
    sub-optimal solutions.
  • This is termed as 7/8th approximation algorithm
    because, if the MAX-3SAT is satisfiable, then the
    expected weight of the assignment found is at
    least 7/8 of optimal clauses.

  • De randomization First devise a randomized
    algorithm then argue that it can be derandomized
    to yield a deterministic algorithm
  • Consider the following formula.
  • (A1 V A2 V A3) ? ( A2 V A3 V A1) ?( A3 V A2
    V A1 )
  • The assignment for the literals in the above
    clauses can be drawn in the form of a binary tree.

Assignment of the Variables
  • At each leaf node, the number of clauses
    satisfied by the assignment of the variables is
    shown on selection of the particular path towards
    that leaf. If n variables, then the tree will
    consist of n levels.
  • If n levels are there, 2n leaves are there. Thus
    2n assignments are possible. The average of the
    numbers on the leaves 7/8 m
  • In 7/8 of the leaves, the clause will be
    contributing for appearing in the leaf.There are
    2n leaves. Each clause contributes 1 to 7/8 2n
  • Sum of the numbers on leaves 7/8 2n m
    21 (Since m 3 in the example here)
  • Avg. of the numbers on leaves (7/8 2n m)/
    2n 21/82.65

  • Here, starting from the root, first compute the
    average at each node. Then pick the path having
    greater average.
  • The process for computing the average is
  • Consider the root assignment A10, A1 1
  • if A1 1 , compute the satisfiable clauses
  • i.e. (A1 V A2 V A3) ... satisfied.
  • ( A2 V A3 V A1) ... satisfied
  • ( A3 V A2 V A1 ) ... not
    satisfied. -gt
  • By discarding A1 from the clause we obtain (
    A2 V A3 ).
  • The probability that this clause is not satisfied
    is 1/4. So the probabilitythat this clause is
    satisfied is ¾ . Thus the total probability turns
    out to be (1 1 ¾) 11/4. This is the
    average when A1 1
  • Similarly compute the average for A1 0. which
    comes out to be 10/4. Choose the greater average
    and move towards that path. So we move towards A1
    1 path.

  • Now compute the average by taking the assignment
    A20 and A21 and move towards the path with
    greater average.
  • Thus at each level i, we have to compute the
    average, with assignment true, and with false.
  • Thus, on reaching the leaf, we would have the
    average number of clauses which will be
    satisfiable with the assignment on the chosen

  • In the above algorithm greedy approach is been
    followed, at each level we check which sub tree
    will give the best average we take decision
    according to the current maximum. Thus algorithm
    may result in an sub-optimal result.

  • Randomized Algorithmes R.Motwani
  • Chapters 1,5,6,10
  • Introducton to Algorithms CLRS
  • Chapter 5
  • A compendium of NP webpage http//www.nada.kth.se/

Thank You
  • -Sandhya S. Pillai (2007MCS3120)
  • -Sunita Sharma (2007MCS2927)
About PowerShow.com