Bisection Algorithm

1
Bisection Algorithm
ECOR 1606 - Problem Solving and Computers
Carleton University Department of Systems and
Computer Engineering
2
Root-Finding Algorithms

• Mathematical problems of the form "find x such
  that f(x) 0" belong to a category of problems
  known as root-finding problems.
• a root of any function, f(x), is any value, x,
  such that f(x) 0.
• A more general problem is "find x such that f(x)
  c", for some number, c. That is, find x f
  -1(c).
• Example For f(x) x2, one solution to f(x) c
  is x .
• In this case, the sqrt( ) function gives us the
  answer.
• Other cases are not so easy, such as f(x) xe-x.
• For many engineering problems, it is not possible
  to find exact expressions for the roots.
• In such cases it is common to use numerical
  methods to determine the roots.
• The bisection algorithm is one numerical
  root-finding method.

3
Finding the Square Root

• To illustrate numerical root-finding techniques,
  suppose that the sqrt( ) function did not exist,
  but we needed to calculate .
• That is, we have to find the solution to f(x) c
  wheref(x) x2 and c 3.
• To illustrate the accuracy of the techniques
  discussed in this section, note that the correct
  answer is ? 1.73205.
• We will also consider the more general case of
  finding for any c gt 1.
• Then we will consider the case for finding x f
  -1(c) for any c and any function f(x).

4
Graphical Method

• We could make a plot of f(x) vs. x, and use the
  plot to find the value of x such that f(x) c.
• We first need to find limits on the range of
  possible values of x, so we know where to start
  and end our plot.
• In general, from analysis of either the
  underlying physical problem or of f(x), we can
  find upper and lower limits for x.
• E.g., for the square root problem, we know that 1
  ? ? c for c ? 1.
• Therefore, to find we need to plot f(x) vs.
  x for x ? 1,3.

5
Plot of f(x) vs. x
2
c
If c 3 then ? 1.75
6
Graphical Method

• This approach is not very efficient, because it
  often requires a lot of work to produce a graph.
• To plot in increments of 0.1 for x in the range
  1,3 requires that the function f(x) be
  evaluated 21 times.
• This may not be a big deal if evaluating f(x)
  only takes a few microseconds, but what if it
  takes a day of computer calculations to calculate
  f(x)?
• Making and printing graph is a real hassle.
• The accuracy of the result isn't very good.
• By visually inspecting this graph, it is possible
  to estimate the square root of c to within ?
  0.05, at best.

7
Random Guessing

• Alternative approach keep making guesses for
  the value of .
• Let be our guess. Calculate .
• If is equal to c, then is equal to .
  So our guess is correct and we're done.
• Otherwise, make another guess.
• Since there are an infinite number of possible
  values within the range 1,3, it is unlikely
  that we'd ever guess the exact value.
• We could stop guessing once we make a guess that
  gives close to c (within some specified
  tolerance).
• But if the tolerance is sufficiently small to
  give accurate results, it could still take an
  extremely long time to guess the answer, and we
  are not guaranteed to ever make a close guess.

8
Exhaustive Search

• A slightly better approach is to guess in a
  sequential manner.
• For example, make the first guess as 1.000,
  the second guess as 1.001, the third guess
  as 1.002, and so on.
• We would stop guessing when we find a value for
  that gives close to c.
• This approach is guaranteed to require only a
  finite number of guesses, since we can stop when
  we either find the correct answer or get up to
  3.

9
Exhaustive Search

• This approach is guaranteed to only require a
  finite number of guesses, but
• It may never find a value of that is close
  enough to c (no convergence). This may occur if
  f(x) is very steep.
• It may find a value of that is very close to
  c, but is still a long way from (false
  convergence). This may occur if f(x) is very
  flat.
• Even if this approach does work, it requires a
  huge amount of work.
• For example, with a step size of 0.001, the
  algorithm would require that be
  evaluated 733 times for our problem.

10
Bisection Algorithm

• The bisection algorithm is a much more efficient
  algorithm for finding roots of functions.
• It is based on organizing your guesses in a
  controlled fashion.
• It works by maintaining a window (a range of
  values), that always contains the root.
• For our example, since f(1) lt c and f(3) gt c, we
  know that there is a value for x?1,3 such that
  f(x) c.
• Therefore 1 lt lt 3.
• Our first guess is the midpoint between 1 and 3.
• We evaluate f(x) at the midpoint (i.e., at x
  2).
• Since c ( 3) is less than 22, we now know that
  is less that 2.
• Now we know that
• 1 lt lt 2

11
Bisection Search

• Now try the midpoint between 1 and 2
• f(1.5) 2.25, which is less than c
• Now we know that
• 1.5 lt lt 2
• Try the midpoint between 1.5 and 2
• f(1.75) 3.0625, which is greater than c
• Now we know that
• 1.5 lt lt 1.75
• We repeat this procedure several times, always
  dividing the range of values in two.
• Eventually, we get a very narrow range.
• The correct value of is always somewhere
  in the range.

12
Basic Algorithm

• use two variables, low and high, to store the
  lower and upper limits on what the value of
  could be.
• start with low 1 and high c
• loop
• guess (low high) / 2
• calculate f(guess)
• if f(guess) is less than c
• guess is too low, so set low guess
• else
• guess is too high, so set high guess
• repeat until low and high come close enough
  together.

13
Loop Termination Condition

• The correct value of is always between low
  and high.
• As the program runs, low and high come closer
  together.
• the distance between low and high decreases by a
  half with each step.
• Eventually, the difference between low and high
  becomes negligible (less than some specified
  tolerance).
• If (high - low) lt 2?, then the correct value of
  is within ?? of the midpoint between low and
  high.
• When this happens, you can safely terminate the
  loop, and return the value of (lowhigh) / 2 as
  an accurate estimate of the correct value of
  .

14
Loop Termination Condition

• The bisection algorithm is guaranteed to converge
  to the correct value, within a specified
  tolerance.
• The algorithm is much more efficient than
  exhaustive searching.
• For our example, estimating with a
  tolerance of ? 10-6 requires only 20
  evaluations of the function f(x).
• compare this with the graphical method, with gave
  an answer accurate to within 0.05 with 21
  evaluations,
• and the exhaustive search, which gave an answer
  accurate to within 0.001 with 733 evaluation.

15
Example Square Root

• Function to find the square root of a number gt 1

double findSquareRoot(double c) const
double TOLERANCE 1e-6 double low, high,
x low 1 // sqrt(c) gt 1 for c gt 1
high c // sqrt(c) lt c for c gt 1 while
(true) x (low high) / 2.0
if (high-low lt 2.0TOLERANCE)
break if (f(x) c)
break else if (f(x) lt c)
low x else
high x return x
set upper and lower limits
make guess
check tolerance
test guess
update limits
return answer
16
Example Square Root

• We also need to provide the function we want to
  invert
• And provide a suitable main program to test the
  algorithm.

double f(double x) return xx
(see sqrt1.cpp on the web page)
17
Example Square Root

• The algorithm as written only works for finding
  the square root of numbers greater than or equal
  to 1.
• It's easy to modify the algorithm to work with
  numbers between 0 and 1 as well.
• All we need to change are the starting limits.
• Since c lt lt 1 for 0 ? c lt 1, we would set
  the limits as follows

if (c lt 1) low c // sqrt(c) gt c
for 0 lt c lt 1 high 1.0 // sqrt(c) lt 1
for 0 lt c lt 1 else low 1.0 //
sqrt(c) gt 1 for c gt 1 high c //
sqrt(c) lt c for c gt 1
(see sqrt2.cpp on the web page)
18
General Problem

• In general, to solve many engineering problems it
  is often necessary to find the roots of a
  mathematical function. For example
• suppose f(x) is some function, and you want to
  find the value of x which minimizes f(x).
• this can be achieved by finding x such that f
  '(x) 0, where f '(x) is the first derivative of
  f(x).
• The bisection algorithm can be used to finding
  the roots of any arbitrary mathematical function
  provided we know in advance which interval (low,
  high) contains the root.
• This is the same as finding x so that f(x) c,
  for some c, for any arbitrary function f(x).
  That is, we can find f -1(c).

19
General Algorithm

• To find x f -1(c), the inverse of f(x)

double f_inverse(double c) const double
TOLERANCE 1e-6 double low, high, x
low LOW_LIMIT high HIGH_LIMIT
while (true) trial_x (low high) /
2.0 if (high-low lt 2TOLERANCE)
break trial_c f(trial_x)
if (trial_c c) break
else if (trial_c lt c)
low trial_x else
high trial_x return
trial_x
specify tolerance
depend on f(x) and physical problem
specify function to invert
Change these three elements to adapt algorithm to
any problem.
20
General Problem

• This algorithm will work for any "increasing"
  function, f(x), provided that we can deduce in
  advance an upper and lower bound on the solution.
• Specify the tolerance.
• the answer will be accurate to within TOLERANCE.
• Set low and high to the lower and upper bounds.
• you will have to deduce these from the physical
  problem.
• replace f( ) with the name of the C function
  that you wish to invert.
• sometimes this function will require additional
  parameters. You will have to modify the header
  for f_inverse( ) to accept these parameters, and
  pass them on to f( ).

21
General Algorithm

• After setting the limits, it is usually a good
  idea to verify that there is a root in the
  interval low, high.
• if f(low) gt c then the root is before low.
• if f(high) lt c then the root is after high.
• in either case, the function should return some
  special number to indicate that an error
  occurred.
• alternatively, the function could return low to
  indicate that the root is less than or equal to
  low, and return high to indicate that the root is
  greater than or equal to high.
• i.e., insert this code fragment after the limits
  are set

if (f(low) gt c) return low if (f(high) lt
c) return high
(see bisect.cpp)
22
General Algorithm

• Make these changes to handle "decreasing"
  functions instead

double f_inverse(double c) const double
TOLERANCE 1e-6 double low, high, trial_x,
trial_c low LOW_LIMIT high
HIGH_LIMIT if (f(low) gt c)
return low if (f(high) lt c)
return high // continued on next slide
change this to if (f(low) lt c) return
low if (f(high) gt c) return high
23
General Algorithm
// continued from previous slide while
(true)) trial_x (low high) /
2.0 if (high-low lt 2TOLERANCE)
break trial_c
f(trial_x) if (trial_c c)
break else if (trial_c lt c)
low trial_x else
high trial_x
return trial_x
change this to if (trial_c gt c)

• It also isn't too hard to modify the function to
  work with either increasing or decreasing
  functions. To determine if a function is
  decreasing, check to see if f(low) gt f(high).

24
Other Root-Finding Methods

• Secant Method
• instead of dividing the interval in two equal
  portions (bisecting), divide the interval by
  "guessing" the location of the root.
• is able to find the root much more quickly than
  the bisection algorithm
• Newton's Method
• use the derivative of the function to help speed
  up the process.
• much faster convergence than Bisection or Secant
  method.
• but requires knowledge of the derivative of the
  function.