Theory of Algorithms: Brute Force

1
Theory of AlgorithmsBrute Force

• James Gain and Edwin Blake
• jgain edwin _at_cs.uct.ac.za
• Department of Computer Science
• University of Cape Town
• August - October 2004

2
Objectives

• To introduce the brute force mind set
• To show a variety of brute-force solutions
• Brute-Force String Matching
• Polynomial Evaluation
• Closest-Pair and Convex-Hull by Brute Force
• Exhaustive Search
• To discuss the strengths and weaknesses of a
  brute force strategy

3
Brute Force

• A straightforward approach usually directly based
  on problem statement and definitions
• Motto Just do it!
• Crude but often effective
• Examples already encountered
• Computing an (a gt 0, n a nonnegative integer)
  by multiplying a together n times
• Computation of n! using recursion
• Multiplying two n by n matrices
• Selection sort

4
Brute Force String Matching

• Problem Find a substring in some text that
  matches a pattern
• Pattern a string of m characters to search for
• Text a (long) string of n characters to search
  in
• Algorithm
• Align pattern at beginning of text
• Moving left to right, compare each character of
  pattern to the corresponding character in text
  UNTIL
• All characters are found to match (successful
  search) or
• A mismatch is detected
• WHILE pattern is not found and the text is not
  yet exhausted, realign pattern one position to
  the right and repeat step 2.

5
Understanding String Matching

• Example
• Pattern AKA
• Text ABRAKADABRA
• Trace AKA
• AKA
• AKA
• AKA
• Number of Comparisons
• In the worst case, m comparisons before
  shifting, for each of n-m1 tries
• Efficiency ?(nm)

6
Brute Force Polynomial Evaluation

• Problem Find the value of polynomial
  p(x) anxn
  an-1xn-1 a1x1 a0 at a point x x0
• Algorithm
• Efficiency?

p ? 0.0 for i ? n down to 0 do power ? 1
for j ? 1 to i do power ? power
x p ? p ai power return p
7
Brute Force Closest Pair

• Problem
• Find the two points that are closest together in
  a set of n 2-D points P1 (x1, y1), , Pn (xn,
  yn)
• Using Cartesian coordinates and Euclidean
  distance
• Algorithm
• Efficiency ?(n2)

dmin ? 8 for i ? 1 to n-1 do for j ? i1 to n
do d ? sqrt((xi - xj)2 (yi - yj)2) if
d lt dmin dmin ? d index1 ? i index2 ?
j return index1, index2
8
The Convex Hull Problem

• Problem Find the convex hull enclosing n 2-D
  points
• Convex Hull If S is a set of points then the
  Convex Hull of S is the smallest convex set
  containing S
• Convex Set A set of points in the plane is
  convex if for any two points P and Q, the line
  segment joining P and Q belongs to the set

Convex
Non-Convex
9
Brute Force Convex Hull
P4
P9
P2
P8
P7
P3
P5
P1
P6

• Algorithm
• For each pair of points p1 and p2
• Determine whether all other points lie to the
  same side of the straight line through p1 and p2
• Efficiency
• for n(n-1)/2 point pairs, check sidedness of
  (n-2) others
• O(n3)

10
Pros and Cons of Brute Force

• Strengths
• Wide applicability
• Simplicity
• Yields reasonable algorithms for some important
  problems and standard algorithms for simple
  computational tasks
• A good yardstick for better algorithms
• Sometimes doing better is not worth the bother
• Weaknesses
• Rarely produces efficient algorithms
• Some brute force algorithms infeasibly slow
• Not as creative as some other design techniques

11
Exhaustive Search

• Definition
• A brute force solution to the search for an
  element with a special property,
• Usually among combinatorial objects such a
  permutations, combinations, or subsets of a set
• Method
• Construct a way of listing all potential
  solutions to the problem in a systematic manner
• All solutions are eventually listed
• No solution is repeated
• Evaluate solutions one by one (disqualifying
  infeasible ones) keeping track of the best one
  found so far
• When search ends, announce the winner

12
Travelling Salesman Problem

• Problem
• Given n cities with known distances between each
  pair
• Find the shortest tour that passes through all
  the cities exactly once before returning to the
  starting city
• Alternatively
• Find shortest Hamiltonian Circuit in a weighted
  connected graph
• Example

2
a
b
5
3
4
8
c
d
7
13
Travelling Salesman by Exhaustive Search

• Tour
  Cost . a?b?c?d?a
  2375 17
• a?b?d?c?a 2478 21
• a?c?b?d?a 8345 20
• a?c?d?b?a 8742 21
• a?d?b?c?a 5438 20
• a?d?c?b?a 5732 17
• Improvements
• Start and end at one particular city
• Remove tours that differ only in direction
• Efficiency (n-1)!/2 ?(n!)

14
Knapsack Problem

• Problem Given n items
• weights w1 w2 wn
• values v1 v2 vn
• a knapsack of capacity W
• Find the most valuable subset of the items that
  fit into the knapsack
• Example (W 16)

Item Weight Value
1 2kg R200
2 5kg R300
3 10kg R500
4 5kg R100
15
Knapsack by Exhaustive Search

• Efficiency ?(2n)

Subset Total Wgt Total Value
1 2 kg R200
2 5 kg R300
3 10 kg R500
4 5 kg R100
1,2 7 kg R500
1,3 12 kg R700
1,4 7 kg R300
2,3 15 kg R800
Subset Total Wgt Total Value
2,4 10 kg R400
3,4 15 kg R600
1,2,3 17 kg n/a
1,2,4 12 kg R600
1,3,4 17 kg n/a
2,3,4 20 kg n/a
1,2,3,4 22 kg n/a
16
Generating Combinatorial Objects Subsets

• Combinatorics uses Decrease (by one) and Conquer
  Algorithms
• Subsets generate all 2n subsets of A a1, ,
  an
• Divide into subsets of a1, , an-1 that contain
  an and those that dont
• Sneaky Solution establish a correspondence
  between bit strings and subsets. Bit n denotes
  presence (1) or absence (0) of element an
• Generate numbers from 0 to 2n-1 ? convert to bit
  strings ? interpret as subsets
• Examples 000 Ø , 010 a2 , 110 a1, a2

17
Generating Combinatorial Objects Permutations

• Permutations generate all n! reorderings of 1,
  , n
• Generate all (n-1)! permutations of 1, , n-1
• Insert n into each possible position (starting
  from the right or left, alternately)
• Implemented by the Johnson-Trotter algorithm
• Satisfies Minimal-Change requirement
• Next permutation obtained by swapping two
  elements of previous
• Useful for updating style algorithms
• Example
• Start 1
• Insert 2 12 21
• Insert 3 123 132 312 321 231 213

18
Final Comments on Exhaustive Search

• Exhaustive search algorithms run in a realistic
  amount of time only on very small instances
• In many cases there are much better alternatives!
  
• Euler circuits
• Shortest paths
• Minimum spanning tree
• Assignment problem
• In some cases exhaustive search (or
  variation) is the only known solution