Constructive Algorithms for Discrepancy Minimization - PowerPoint PPT Presentation

About This Presentation
Title:

Constructive Algorithms for Discrepancy Minimization

Description:

Constructive Algorithms for Discrepancy Minimization Nikhil Bansal (IBM) Discrepancy: What is it? Study of gaps in approximating the continuous by the discrete. – PowerPoint PPT presentation

Number of Views:59
Avg rating:3.0/5.0
Slides: 44
Provided by: tcsTifrR5
Category:

less

Transcript and Presenter's Notes

Title: Constructive Algorithms for Discrepancy Minimization


1
Constructive Algorithms for Discrepancy
Minimization
  • Nikhil Bansal (IBM)

2
Discrepancy What is it?
  • Study of gaps in approximating the continuous by
    the discrete.
  • Problem How uniformly can you distribute points
    in a grid.
  • Uniform For every axis-parallel
    rectangle R
  • ( points in R) - (Area of R)
    should be low.

Discrepancy Max over rectangles R ( points
in R) (Area of R)
n1/2
n1/2
3
Distributing points in a grid
  • Problem How uniformly can you distribute points
    in a grid.
  • Uniform For every axis-parallel
    rectangle R
  • ( points in R) - (Area of R)
    should be low.

n 64 points
Van der Corput Set
Uniform
Random
n1/2 discrepancy
n1/2 (loglog n)1/2
O(log n) discrepancy!
4
Discrepancy Example 2
  • Input n points placed arbitrarily in a grid.
  • Color them red/blue such that each rectangle is
    colored as evenly as possible
  • Discrepancy max over rect. R ( red in R -
    blue in R )

Continuous Color each element 1/2 red and 1/2
blue (0 discrepancy) Discrete Random has about
O(n1/2 log1/2 n) Can achieve O(log2.5 n)
5
Applications
CS Computational Geometry, Comb. Optimization,
Monte-Carlo simulation, Machine learning,
Complexity, Pseudo-Randomness, Math Dynamical
Systems, Combinatorics, Mathematical Finance,
Number Theory, Ramsey Theory, Algebra, Measure
Theory,
6
Combinatorial Discrepancy
  • Universe U 1,,n
  • Subsets S1,S2,,Sm
  • Color elements red/blue so each
  • set is colored as evenly as possible.
  • Find ? n ! -1,1 to
  • Minimize ?(S)1 maxS ?i 2 S ?(i)
  • For simplicity consider mn henceforth.

7
Best Known Algorithm
  • Random Color each element i independently as
  • x(i) 1 or -1 with probability ½ each.
  • Thm Discrepancy O (n log n)1/2
  • Pf For each set, expect O(n1/2) discrepancy
  • Standard tail bounds Pr ?i 2 S x(i) c
    n1/2 ¼ e-c2
  • Union bound Choose c ¼ (log n)1/2
  • Analysis tight Random actually incurs ?((n log
    n)1/2).

8
Better Colorings Exist!
  • Spencer 85 (Six standard deviations suffice)
  • Always exists coloring with discrepancy
    6n1/2
  • (In general for arbitrary m, discrepancy
    O(n1/2 log(m/n)1/2)
  • Tight For mn, cannot beat 0.5 n1/2
    (Hadamard Matrix, orthogonal sets)
  • Inherently non-constructive proof
  • (pigeonhole principle on exponentially large
    universe)
  • Challenge Can we find it algorithmically ?
  • Certain algorithms do not work Spencer
  • Conjecture Alon-Spencer May not be possible.

9
Beck Fiala Thm
  • U 1,,n Sets S1,S2,,Sm
  • Suppose each element lies in at most t sets (t
    ltlt n).
  • Beck Fiala 81 Discrepancy 2t -1.
  • (elegant linear algebraic argument, algorithmic
    result)
  • Beck Fiala Conjecture O(t1/2) discrepancy
    possible
  • Other results O( t1/2 log t log n ) Beck
  • O( t1/2 log n )
    Srinivasan
  • O( t1/2 log1/2 n )
    Banaszczyk

Non-constructive
10
Approximating Discrepancy
  • Question If a set system has low discrepancy
    (say ltlt n1/2)
  • Can we find a good discrepancy coloring ?
  • Charikar, Newman, Nikolov 11
  • Even 0 vs. O (n1/2) is NP-Hard
  • (Matousek) What if system has low Hereditary
    discrepancy?
  • herdisc (U,S) maxU ½ U disc
    (U, SU)
  • Robust measure of discrepancy (often same as
    discrepancy)
  • Widely used TU set systems, Geomety,

11
Our Results
  • Thm 1 Can get Spencers bound constructively.
  • That is, O(n1/2) discrepancy for mn sets.
  • Thm 2 If each element lies in at most t sets,
    get bound of O(t1/2 log n) constructively
    (Srinivasans bound)
  • Thm 3 For any set system, can find
  • Discrepancy O(log (mn)) Hereditary discrepancy.

Other Problems Constructive bounds (matching
current best) k-permutation problem Spencer,
Srinivasan,Tetali Geometric problems ,
12
Relaxations LPs and SDPs
  • Not clear how to use.
  • Linear Program is useless. Can color each element
    ½ red and ½ blue. Discrepancy of each set 0!
  • SDPs (LP on vi vj, cannot control dimension
    of vs)
  • ?i 2 S vi 2 n 8 S
  • vi2 1
  • Intended solution vi (1,0,,0) or
    (-1,0,,0).
  • Trivially feasible vi ei (all vis
    orthogonal)

Yet, SDPs will be a major tool.
13
Punch line
  • SDP very helpful if tighter bounds needed for
    some sets.
  • ?i 2 S vi 2 2 n
  • ?i 2 S vi2 n/log n
  • vi2 1
  • Not apriori clear why one can do this.
  • Entropy Method.
  • Algorithm will construct coloring over time and
  • use several SDPs in the process.

Tighter bound for S
14
Talk Outline
  • Introduction
  • The Method
  • Low Hereditary discrepancy -gt Good coloring
  • Additional Ideas
  • Spencers O(n1/2) bound

15
Our Approach
16
Algorithm (at high level)
Each dimension An Element Each vertex A
Coloring
Cube -1,1n
Algorithm Sticky random walk Each
step generated by rounding a suitable SDP
Move in various dimensions correlated, e.g. ?t1
?t2 ¼ 0
Analysis Few steps to reach a vertex (walk has
high variance) Disc(Si) does a
random walk (with low variance)
17
An SDP
  • Hereditary disc. ? ) the following SDP is
    feasible

SDP Low discrepancy ?i 2 Sj vi 2
?2
vi2 1
Obtain vi 2 Rn
Rounding Pick random Gaussian g
(g1,g2,,gn) each coordinate gi is iid
N(0,1) For each i, consider ?i g vi
18
Properties of Rounding
  • Lemma If g 2 Rn is random Gaussian. For any v 2
    Rn,
  • g v is distributed as N(0, v2)
  • Pf N(0,a2) N(0,b2) N(0,a2b2)
    g v ?i v(i) gi N(0, ?i v(i)2)

Recall ?i g vi
  • Each ?i N(0,1)
  • For each set S,
  • ?i 2 S ?i g (?i2 S vi) N(0, ?2)
  • (std deviation ?)

SDP vi2 1 ?i2 S vi2 ?2
?s mimics a low discrepancy coloring (but is not
-1,1)
19
Algorithm Overview
  • Construct coloring iteratively.
  • Initially Start with coloring x0 (0,0,0, ,0)
    at t 0.
  • At Time t Update coloring as xt xt-1 ?
    (?t1,,?tn)
  • (? tiny 1/n suffices)

xt(i) ? (?1i ?2i ?ti) Color of
element i Does random walk over time with step
size ¼ ? N(0,1)

x(i)
Fixed if reaches -1 or 1.
Set S xt(S) ?i 2 S xt(i) does a random
walk w/ step ? N(0, ?2)
20
Analysis
  • Consider time T O(1/?2)
  • Claim 1 With prob. ½, at least n/2 elements
    reach -1 or 1.
  • Pf Each element doing random walk with size ¼
    ?.
  • Recall Random walk with step 1, is ¼
    O(t1/2) away in t steps.
  • A Trouble Various element updates are correlated
  • Consider basic walk x(t1) x(t) 1 with
    prob ½
  • Define Energy ?(t) x(t)2
  • E?(t1) ½ (x(t)1)2 ½ (x(t)-1)2 x(t)2
    1 ?(t)1
  • Expected energy n at t n.
  • Claim 2 Each set has O(?) discrepancy in
    expectation.
  • Pf For each S, xt(S) doing random walk with step
    size ¼ ? ?

21
Analysis
  • Consider time T O(1/?2)
  • Claim 1 With prob. ½, at least n/2 variables
    reach -1 or 1.
  • ) Everything colored in O(log n) rounds.
  • Claim 2 Each set has O(?) discrepancy in
    expectation per round.
  • ) Expected discrepancy of a set at end O(?
    log n)
  • Thm Obtain a coloring with discrepancy O(? log
    (mn))
  • Pf By Chernoff, Prob. that disc(S) gt 2
    Expectation O(? log m)

  • O(? log (mn))
  • is tiny (poly(1/m)).

22
Recap
  • At each step of walk, formulate SDP on unfixed
    variables.
  • Use some (existential) property to argue SDP
    is feasible
  • Rounding SDP solution -gt Step of walk
  • Properties of walk
  • High Variance -gt Quick convergence
  • Low variance for discrepancy on sets -gt Low
    discrepancy

23
Refinements
  • Spencers six std deviations result
  • Goal Obtain O(n1/2) discrepancy for any
    set system on m O(n) sets.
  • Random coloring has n1/2 (log n)1/2
    discrepancy
  • Previous approach seems useless
  • Expected discrepancy for a set O(n1/2),
  • but some random walks will deviate by up
    to (log n)1/2 factor

Need an additional idea to prevent this.
24
Spencers O(n1/2) result
  • Partial Coloring Lemma For any system with m
    sets, there exists a coloring on n/2 elements
    with discrepancy O(n1/2 log1/2 (2m/n))
  • For mn, disc O(n1/2)
  • Algorithm for total coloring
  • Repeatedly apply partial coloring lemma
  • Total discrepancy
  • O( n1/2 log1/2 2 ) Phase 1
  • O( (n/2)1/2 log1/2 4 ) Phase 2
  • O((n/4)1/2 log1/2 8 ) Phase 3
  • O(n1/2)

25
Proving Partial Coloring Lemma
  • Beautiful Counting argument (entropy method
    pigeonhole)
  • Idea Too many colorings (2n), but few
    discrepancy profiles
  • Key Lemma There exist k24n/5 colorings X1,,Xk
    such that
  • every two Xi, Xj are similar for every set
    S1,,Sn.
  • Some X1,X2 differ on n/2 positions
  • Consider X (X1 X2)/2
  • Pf X(S) (X1(S) X2(S))/2 2 -10 n1/2 ,
    10 n1/2

26
A useful generalization
  • There exists a partial coloring with non-uniform
    discrepancy bound ?S for set S
  • Even if ?S ?( n1/2) in some average sense

27
An SDP
  • Suppose there exists partial coloring X
  • 1. On n/2 elements
  • 2. Each set S has X(S) ?S

SDP Low discrepancy ?i 2 Sj vi 2
?S2 Many colors ?i vi2 n/2
vi2 1
Pick random Gaussian g (g1,g2,,gn) each
coordinate gi is iid N(0,1) For each i,
consider ?i g vi
Obtain vi 2 Rn
28
Algorithm
  • Initially write SDP with ?S c n1/2
  • Each set S does random walk and expects to reach
  • discrepancy of O(DS) O(n1/2)
  • Some sets will become problematic.
  • Reduce their ?S on the fly.
  • Not many problematic sets, and entropy penalty
    low.

Danger 3
Danger 1
Danger 2

35n1/2
0
30n1/2
20n1/2
29
Concluding Remarks
  • Construct coloring over time by solving sequence
    of SDPs (guided by existence results)
  • Works quite generally
  • Can be derandomized Bansal-Spencer
  • (use entropy method itself for derandomizing
    usual tech.)
  • E.g. Deterministic six standard deviations can be
    viewed as a way to derandomize something stronger
    than Chernoff bounds.

30
Thank You!
31
(No Transcript)
32
(No Transcript)
33
Rest of the talk
  • How to generate ?i with required properties.
  • How to update ?S over time.
  • Show n1/2 (log log log n)1/2 bound.

34
Why so few algorithms?
  • Often algorithms rely on continuous relaxations.
  • Linear Program is useless. Can color each element
    ½ red and ½ blue.
  • Improved results of Spencer, Beck, Srinivasan,
    based on clever counting (entropy method).
  • Pigeonhole Principle on exponentially large
    systems (seems inherently non-constructive)

35
Partial Coloring Lemma
  • Suppose we have discrepancy bound ?S for set S.
  • Consider 2n possible colorings
  • Signature of a coloring X (b(S1), b(S2),,
    b(Sm))
  • Want partial coloring with signature (0,0,0,,0)

36
Progress Condition
  • Energy increases at each step
  • E(t) \sum_i x_i(t)2
  • Initially energy 0, can be at most n.
  • Expected value of E(t) E(t-1) \sum_i
    \gamma_i(t)2
  • Markovs inequality.

37
Missing Steps
  1. How to generate the \eta_i
  2. How to update \Delta_S over time

38
Partial Coloring
  • If exist two colorings X1,X2
  • 1. Same signature (b1,b2,,bm)
  • 2. Differ in at least n/2 positions.
  • Consider X (X1 X2)/2
  • -1 or 1 on at least n/2 positions, i.e. partial
    coloring
  • Has signature (0,0,0,,0)
  • X(S) (X1(S) X2(S)) / 2, so X(S) ?S
    for all S.
  • Can show that there are 24n/5 colorings with
    same signature.
  • So, some two will differ on gt n/2 positions.
    (Pigeon Hole)

X1 (1,-1, 1 , , 1,-1,-1) X2 (-1,-1,-1, ,
1,1, 1)
39
(No Transcript)
40
Spencers O(n1/2) result
  • Partial Coloring Lemma For any system with m
    sets,
  • there exists a coloring on n/2 elements
    with discrepancy O(n1/2 log1/2 (2m/n))
  • For mn, disc O(n1/2)
  • Algorithm for total coloring
  • Repeatedly apply partial coloring lemma
  • Total discrepancy
  • O( n1/2 log1/2 2 ) Phase 1
  • O( (n/2)1/2 log1/2 4 ) Phase 2
  • O((n/4)1/2 log1/2 8 ) Phase 3
  • O(n1/2)

Let us prove the lemma for m n
41
Proving Partial Coloring Lemma
-10 n1/2
-30 n1/2
10 n1/2
30 n1/2
0
2
1
-1
-2
  • Pf Associate with coloring X, signature
    (b1,b2,,bn)
  • (bi bucket in which X(Si) lies )
  • Wish to show There exist 24n/5 colorings with
    same signature
  • Choose X randomly Induces distribution ? on
    signatures.
  • Entropy (?) n/5 implies some signature has
    prob. 2-n/5.
  • Entropy (? ) ?i Entropy( bi)
    Subadditivity of Entropy
  • bi 0 w.p. ¼ 1- 2 e-50,
  • 1 w.p. ¼ e-50
  • 2 w.p. ¼ e-450
  • .

42
A useful generalization
  • Partial coloring with non-uniform discrepancy ?S
    for set S

Suffices to have ?s Ent (bs) n/5 Or, if ?S
?s n1/2 , then ?s g(?s) n/5 g(?)
¼ e-?2/2 ? gt 1
¼ ln(1/?) ? lt 1
43
Recap
  • Partial Coloring ?S ¼ 10 n1/2 gives low
    entropy
  • ) 24n/5 colorings exist with same
    signature.
  • ) some X1,X2 with large hamming
    distance.
  • (X1 X2) /2 gives the desired
    partial coloring.
  • Trouble 24n/5/2n is an exponentially small
    fraction.

Only if we could find the partial coloring
efficiently
Write a Comment
User Comments (0)
About PowerShow.com