Loading...

PPT – Smoothed Analysis of Algorithms PowerPoint presentation | free to download - id: 6c53e0-N2NiN

The Adobe Flash plugin is needed to view this content

Smoothed Analysis of Algorithms

- Shang-Hua Teng
- Boston University
- Akamai Technologies Inc

Joint work with Daniel Spielman (MIT)

Outline

- Part I Introduction to Algorithms
- Part II Smoothed Analysis of Algorithms
- Part III Geometric Perturbation

Part I Introduction to Algorithms

- Type of Problems
- Complexity of Algorithms
- Randomized Algorithms
- Approximation
- Worst-case analysis
- Average-case analysis

Algorithmic Problems

- Decision Problem
- Can we 3-color a given graph G?
- Search Problem
- Given a matrix A and a vector b, find an x
- s.t., A x b
- Optimization Problem
- Given a matrix A and a vector b, and an

objective vector c, find an x that - maximize cT x
- s.t. A x b

The size and family of a problem

- Instance of a problem
- Input
- for example, a graph, a matrix, a set of points
- desired output
- yes,no, coloring, solution-vector, convex hull
- Input and output size
- Amount of memory needed to store the input and

output - For example number of vertices in a graph,

dimensions of a matrix, the cardinality of a

point set - A problem is a family of instances.

An Example

- Median of a set of numbers
- Input a set of numbers a1,, a2,, an
- Output ai that maximizes

1 2 3 4 5 6 7 8 9 10

Quick Selection

- Quick Selection (a1,, a2,, an, k )
- Choose a1 in a1,, a2,, an
- Divide the set into
- Cases
- If k L 1, return a1
- If k lt L, recursively apply Quick_Selection(L,k)

- If k gt L1, recursively apply

Quick_Selection(L,n-k-1)

Worse-Case Time Complexity

- Let T(a1,, a2,, an ) be the number of basic

steps needed in Quick-Selection for input a1,,

a2,, an. - We classify inputs by their size
- Let An be the set of all input of size n.
- T(n) n2 - n

Better Algorithms from Worst-case View point

- Divide and Conquer
- Linear-time algorithm
- Blum-Floyd-Pratt-Rivest-Tarjan

Average-Case Time Complexity

- Let An be the set of all input of size n.
- Choose a1,, a2,, an uniformly at random
- E(T(n)) O(n)

Randomized Algorithms

- Quick Selection (a1,, a2,, an, k )
- Choose a random element s in a1,, a2,, an
- Divide the set into
- Cases
- If k L 1, return s
- If k lt L, recursively apply Quick_Selection(L,k)

- If k gt L1, recursively apply

Quick_Selection(L,n-k-1)

Expected Worse-Case Complexity of Randomized

Algorithms

E(T(n)) O(n)

Approximation Algorithms

- Sampling-Selection (a1,, a2, , an)
- Choose a random element ai from a1,, a2,, an
- Return ai.
- ai is a dmedian if
- Probai is a (1/4)median 0.5

Approximation Algorithms

- Sampling-Selection (a1,, a2, , an)
- Choose a set S of random k elements from

a1,,, an - Return the median ai of S.
- Complexity O(k)

When k 3

Iterative Middle-of-3 (Miller-Teng)

Randomly assign elements from a1,, a2, , an

to the leaves

Summary

- Algorithms and their complexity
- Worst-case complexity
- Average-cast complexity
- Design better worse-case algorithm
- Design better algorithm with randomization
- Design faster algorithm with approximation

Sad and Exciting Reality

- Most interesting optimization problems are hard
- P vs NP
- NP-complete problems
- Coloring, maximum independent set, graph

partitioning - Scheduling, optimal VLSI layout, optimal

web-traffic assignment, data mining and

clustering, optimal DNS and TCPIP protocols,

integer programming - Some are unknown
- Graph isomorphism
- factorization

Good News

- Some fundamental problems are solvable in

polynomial time - Sorting, selection, lower dimensional

computational geometry - Matrix problems
- Eigenvalue problem
- Linear systems
- Linear programming (interior point method)
- Mathematical programming

Better News I

- Randomization helps
- Testing of primes (essential to RSA)
- VC-dimension and sampling for computational

geometry and machine learning - Random walks various statistical problems
- Quicksort
- Random routing on parallel network
- Hashing

Better News II

- Approximation algorithms
- On-line scheduling
- Lattice basis reduction (e.g. in cryptanalysis)
- Approximate Euclidean TSP and Steiner trees
- Graph partitioning
- Data clustering
- Divide-and-conquer method for VLSI layout

Real Stories

- Practical algorithms and heuristics
- Great-performance empirically
- Used daily by millions and millions of people
- Worked routinely from chip design to airline

scheduling - Applications
- Internet routing and searching
- Scientific simulation
- Optimization

PART II Smoothed Analysis of Algorithms

- Introduction of Smoothed Analysis
- Why smoothed analysis?
- Smoothed analysis of the Simplex Method for

Linear Programming

Smoothed Analysis of Algorithms Why The Simplex

Method Usually Takes Polynomial Time

Gaussian Perturbation with variance s2

- Daniel A. Spielman (MIT)
- Shang-Hua Teng (Boston University)

Remarkable Algorithms and Heuristics

- Work well in practice, but
- Worst case bad,
- exponential,
- contrived.
- Average case good,
- polynomial,
- meaningful?

Random is not typical

Smoothed Analysis of Algorithms

worst case maxx T(x) average case avgr

T(r) smoothed complexity maxx avgr T(xer)

Smoothed Analysis of Algorithms

- Interpolate between Worst case and Average Case.
- Consider neighborhood of every input instance
- If low, have to be unlucky to find bad input

instance

Smoothed Complexity

Classical Example Simplex Method for Linear

Programming

max zT x s.t. A x y

- Worst-Case exponential
- Average-Case polynomial
- Widely used in practice

The Diet Problem

Carbs Protein Fat Iron Cost

1 slice bread 30 5 1.5 10 30

1 cup yogurt 10 9 2.5 0 80

2tsp Peanut Butter 6 8 18 6 20

US RDA Minimum 300 50 70 100

Minimize 30 x1 80 x2 20 x3 s.t. 30x1

10 x2 6 x3 ? 300 5x1

9x2 8x3 ? 50 1.5x1 2.5 x2

18 x3 ? 70 10x1

6 x3 ? 100

x1, x2, x3 ? 0

Linear Programming

max zT x s.t. A x y

Max x1 x2 s.t x1 1

x2 1 -x1 - 2x2 1

Smoothed Analysis of Simplex Method

G is Gaussian

Smoothed Analysis of Simplex Method

- Worst-Case exponential
- Average-Case polynomial
- Smoothed Complexity polynomial

max zT x s.t. aiT x 1, ai 1

max zT x s.t. (aisgi )T x 1

Perturbation yields Approximation

- For polytope of good aspect ratio

But, combinatorially

The Simplex Method

History of Linear Programming

- Simplex Method (Dantzig, 47)
- Exponential Worst-Case (Klee-Minty 72)
- Avg-Case Analysis (Borgwardt 77, Smale 82,

Haimovich, Adler, Megiddo, Shamir, Karp, Todd) - Ellipsoid Method (Khaciyan, 79)
- Interior-Point Method (Karmarkar, 84)
- Randomized Simplex Method (mO(?d) )
- (Kalai 92, Matousek-Sharir-Welzl

92)

Shadow Vertices

Another shadow

Shadow vertex pivot rule

start

z

objective

Theorem For every plane, the expected size of

the shadow of the perturbed tope is poly(m,d,1/s )

Theorem For every z, two-Phase Algorithm runs in

expected time poly(m,d,1/s )

z

A Local condition for optimality

z

a2

z

0

Vertex on a1,,ad maximizes z iff z Î

cone(a1,,ad )

a1

Primal a1T x ? 1 a2T x ? 1 amT x ?

1

Polar ConvexHull(a1, a2, am)

(No Transcript)

Polar Linear Program

z

max ? ?z Î ConvexHull(a1, a2, ..., am)

Initial Simplex

Opt Simplex

Shadow vertex pivot rule

(No Transcript)

Count facets by discretizing to N directions, N ??

Count pairs in different facets

So, expect c Facets

Expect cone of large angle

Angle

Distance

Isolate on one Simplex

Integral Formulation

Example For a and b Gaussian

distributed points, given that ab

intersects x-axis Prob? lt

e O(e2)

a

?

b

a

?

b

(No Transcript)

(No Transcript)

a

(No Transcript)

Change of variables

u

a

z

?

v

b

da db (uv)sin(q) du dv dz dq

Analysis For e lt e0, Pe lt e2

Slight change in q has little effect on ni

for all but very rare u,v,z

Distance Gaussian distributed corners

Idea fix by perturbation

Trickier in 3d

Future Research Simplex Method

- Smoothed analysis of other pivot rules
- Analysis under relative perturbations.
- Trace solutions as un-perturb.
- Strongly polynomial algorithm for linear

programming?

A Theory Closer to Practice

- Optimization algorithms and heuristics, such as

Newtons Method, Conjugate Gradient, Simulated

Annealing, Differential Evolution, etc. - Computational Geometry, Scientific Computing and

Numerical Analysis - Heuristics solving instances of NP-Hard

problems. - Discrete problems?
- Shrink intuition gap between theory and practice.

Part III Geometric Perturbation

- Three Dimensional Mesh Generation

Delaunay Triangulations for Well-Shaped 3D Mesh

Generation

- Shang-Hua Teng
- Boston University
- Akamai Technologies Inc.

Collaborators Siu-Wing Cheng, Tamal Dey,

Herbert Edelsbrunner, Micheal Facello Damrong

Guoy Gary Miller, Dafna Talmor, Noel

Walkington Xiang-Yang Li and Alper Üngör

3D Unstructured Meshes

Surface and 2D Unstructured Meshes

courtesy NASA

courtesy N. Amenta, UT Austin

courtesy Ghattas, CMU

Numerical Methods

Formulation MathEngineering

Domain, Boundary, and PDEs

ad hoc

Point Set Triangulation

Mesh Generation geometric structures

octree

Delaunay

element

difference

Approximation Numerical Analysis

Finite

volume

Linear System algorithm data structures

Axb

multigrid

direct method

iterative method

Outline

- Mesh Generation in 2D
- Mesh Qualities
- Meshing Methods
- Meshes and Circle Packings
- Mesh Generation in 3D
- Slivers
- Numerical Solution Control Volume Method
- Geometric Solution Sliver Removal by Weighted

Delaunay Triangulations - Smoothed Solution Sliver Removal by Perturbation

Badly Shaped Triangles

Aspect Ratio (R/r)

Meshing Methods

The goal of a meshing algorithm is to generate a

well-shaped mesh that is as small as possible.

- Advancing Front
- Quadtree and Octree Refinement
- Delaunay Based
- Delaunay Refinement
- Sphere Packing
- Weighted Delaunay Triangulation
- Smoothing by Perturbation

Balanced Quadtree Refinements (Bern-Eppstein-Gilb

ert)

Quadtree Mesh

Delaunay Triangulations

Why Delaunay?

- Maximizes the smallest angle in 2D.
- Has efficient algorithms and data structures.
- Delaunay refinement
- In 2D, it generates optimal size, natural looking

meshes with 20.7o (Jim Ruppert)

Delaunay Refinement (Jim Ruppert)

2D insertion

1D insertion

Delaunay Mesh

Local Feature Spacing (f)

The radius of the smallest sphere centered at a

point that intersects or contains two

non-incident input features

Well-Shaped Meshes and f

f is 1-Lipschitz and Optimal

Sphere-Packing

b -Packing a Function f

f(p)/2

p

q

- No large empty gap the radius of the largest

empty sphere passing q is at most b f(q).

The Packing Lemma (2D) (Miller-Talmor-Teng-Walkin

gton)

- The Delaunay triangulation of a b -packing is a

well-shaped mesh of optimal size. - Every well-shaped mesh defines a b -packing.

Part I Meshes to Packings

Part II Packings to Meshes

3D Challenges

- Delaunay failed on aspect ratio
- Quadtree becomes octree (Mitchell-Vavasis)
- Meshes become much larger
- Research is more Challenging!!!

Badly Shaped Tetrahedra

Slivers

Radius-Edge Ratio (Miller-Talmor-Teng-Walkington)

R/L

The Packing Lemma (3D) (Miller-Talmor-Teng-Walkin

gton)

- The Delaunay Triangulation of a b -packing is a

well-shaped mesh (using radius-edge ratio) of

optimal size. - Every well-shaped (aspect-ratio or radius-edge

ratio) mesh defines a b -packing.

Uniform Ball Packing

- In any dimension, if P is a maximal packing of

unit balls, then the Delaunay triangulation of P

has radius-edge at most 1.

e is at least 2

r

r is at most 2

Constant Degree Lemma (3D) (Miller-Talmor-Teng-Wa

lkington)

- The vertex degree of the Delaunay triangulation

with a constant radius-edge ratio is bounded by a

constant.

Delaunay Refinement in 3D Shewchuck

Slivers

Sliver the geo-roach

Coping with Slivers Control-Volume-Method (Miller

-Talmor-Teng-Walkington)

Sliver Removal by Weighted Delaunay

(Cheng-Dey-Edelsbrunner-Facello-Teng)

Weighted Points and Distance

z

p

Orthogonal Circles and Spheres

Weighted Bisectors

Weighted Delaunay

Weighted Delaunay and Convex Hull

Parametrizing Slivers

D

L

Y

Interval Lemma

- Constant Degree The union of all weighted

Delaunay triangulations with Property r and

Property 1/3 has a constant vertex degree

Pumping Lemma (Cheng-Dey-Edelsbrunner-Facello-Ten

g)

z

P

p

H

D

q

Y

r

s

Sliver Removal by Flipping

- One by one in an arbitrary ordering
- fix the weight of each point
- Implementation flip and keep the best

configuration.

Experiments (Damrong Guoy, UIUC)

Initial tetrahedral mesh 12,838 vertices, all

vertices are on the boundary surface

Dihedral angle lt 5 degree 13,471

Slivers after Delaunay refinement 881

Slivers after sliver-exudation 12

1183 slivers

15,503 slivers

142 slivers, less elements, better distribution

563 slivers

5636 slivers

Triceratops

18 slivers, less elements, better distribution

Heart

4532 slivers

173 slivers

1 sliver, less elements, better distribution

Smoothing and Perturbation

- Perturb mesh vertices
- Re-compute the Delaunay triangulation

Well-shaped Delaunay Refinement (Li and Teng)

- Add a point near the circumcenter of bad element
- Avoids creating small slivers
- Well-shaped meshes