A simple mathematical model

1
A simple mathematical model
Consider the forces acting on a turtle thrown
out from a plane. According to Newtons law
(1.1)
where m is the mass of the turtle, v velocity
of the turtle, t time
2
For a turtle in free fall, nett
forcegravitational weight of turtle drag force
on turtle due to air
mg - cv
where c is some coefficient of friction. Use
this information and substitute into Eq. (1.1)
will give
(1.2)
Solving Eq. (1.2) will give you the velocity of
the turtle at various times, t. It turns out
that Eq. (1.2) is simple enough to solve
analyticall (by hand). Lets now find the
solution. Divide Eq. (1.2) by m gives
(1.3)
3
Exercise (1.1) Solve Eq. (1.3) and show that
(1.4)
if v0 at t0
The function v(t) is shown on the left
4
Problem (1.1). Given that the velocity of the
turtle is governed by Eq. (1.4), find the value
of the drag coefficient, c, such that a turtle
of mass, m5, can attain a prescribed velocity,
v10, at a set period of time, t9. Use g10.
In the context of the problem mentioned above, m,
v, and t are constant (parameters) of the
problem. The only variable that we can adjust is
c.
To solve this problem, rewrite Eq. (1.4) as
5
(1.5)
So our task of solving the problem reduces to
merely finding the value of c, such that f( c )
0, for the given values of v, m and t.
6
The graph on the left shows how f( c ) (Eq.
(1.5)) looks like
We need to find a value of c such that f( c )0.
In numerical analysis this problem is called
finding the roots of equation.
The rest of the lectures in this section will
introduce different methods to obtain the roots
of equation. I will illustrate each method by
finding the root of Eq. (1.5).
7
Finding roots of equation.
Graphical method
One crude way of finding roots for an equation is
to just plot the graph and visually inspect where
the function evaluates to zero. For example, plot
f( c ) and find the value of c where f( c )0.
In the example on the left the root at f( c )
occur when c is approximately 5 or 4.99382 to be
more precise.
8

• As you can probably imagine, this method is not
  very practical
• because-
• the accuracy is poor
• difficult to repeat for a large number of
  different parameter values

• The main advantage of this method is that-
• you can see exactly how the function behaves.
• how many roots there are for the equation.
• the graphical method sometimes serve as a guide
  to finding the first
• guess for the root of the equation.

9
Numerical algorithms for finding roots of
equations
Method of false position
One point iteration
Bisecton method
Newton-Raphson
Secant method
10
Bracketing Methods - At least two guesses are
required - Require that the guesses bracket the
root of an equation - More robust that open
methods
Open Methods - Most of the time, only one
initial guess is required - Do that require that
the guesses bracket the root of the equation -
More computationally efficient than bracketing
methods but they do not always
work..may blow up !!
11
Bracketing Methods

• Bisection method
• Method of False position

• These methods are known as bracketing methods
• because they rely on having two initial
  guesses.
• - xl - lower bound and
• - xu - upper bound.
• The guesses must bracket (be either side of) the
  root. WHY ?

12

• If f(xu) and f(xl) are of different
• sign, then there must be at least
• one root, xr, in between xu and xl.
• I.e. xl lt xr lt xu.

• Or there might be an odd number of
• roots.

13

• If f(xu) and f(xl) are of the same sign,
• then there might not be any root
• in between xu and xl.

xl
xu

• Or there might be an even number
• of roots.

14
There are exceptions to the rules
When the function is tangential to the x-axis,
multiple roots occur
Functions with discontinuities do not obey the
rules above
15
The Bisection Method

• STEP 1
• Obtain two guesses for the root, xl and xu,
  that brackets
• the real root, xr. In general, if f(xu) f(xl)
  lt 0 then you can be
• reasonably sure that you have at least one
  root in between
• xl and xu. To be 100 sure, and if you have
  the time, plot
• f(x).
• STEP 2
• Estimate the root of f(x) to be

16

• STEP 3
• Make a choice-
• -If f(xl) f(xr) lt 0, then the root is in
  the lower subinterval.
• Therefore set xu xr and go back to
  STEP 2.
• -If f(xu) f(xr) lt 0, then the root is in the
  upper subinterval.
• Therefore set xl xr and go back
  to STEP 2
• -If f(xl) f(xr) 0, then the root is xr.

Usually, you cannot get exactly f(xl) f(xr) 0.
In practice, if f(xl) f(xr) lt e, where e is a
small value (the stopping criterion), you would
take xr to be the root.
17
Example 1.1
Problem statement Solve Problem 1.1 by using
the Bisection method to obtain the root of
Solution

• We need to start with two guesses. By looking
  at the figure
• above, lets try the lower guess to be cl3 and
  the upper guess
• cu9.

18

• Estimate the root to be

The first estimate of the root is 6. Remember
that the exact solution is 4.99382.

• Calculate f(cr)

Since f(cu) f(cr) 6.59 x (-1.67) lt 0, we let
cl6 and repeat the process.
19

• Estimate the root to be

Note that this is much closer to the exact
solution of 4.99382 than the previous estimate
of 6.

• Calculate f(cr)

Since f(cl) f(cr) -1.67 x 1.11 lt 0, we let cu
4.5 and repeat the process. You can do this
until the result is accurate enough for you.
20
Use the Bisection Method to find the root of the
equation. This is illustrated in the table
below to get the solution.
One disadvantage of the Bisection method is that
it does not take into account the magnitudes of
f(xl) and f(xu).
21
The Method of False Position
The method of false position is based on the
following observation. If f(xl) is closer to
zero than f(xu), then the root is likely to be
closer to xl than xu. See the figure below.
22
Using similar triangles, we can obtain the
following expression
Exercise Show that the above expression reduces
to
or
(1.6)
23
So, to use the method of False Position to find
roots, use Steps 1-3 of the Bisection method but
replace Step 2 with the formula given in Eq.
(1.5).
Example 1.2
Problem statement Solve Problem 1.1 by using
the method of False position to obtain the root
of
24
Solution

• We need to start with two guesses. By looking
  at the figure
• above, lets try the lower guess to be cl3 and
  the upper guess
• cu9.

• Calculate f(cl) and f(cu)

25

• Estimate the root using Eq. (1.6) to be

The first estimate of the root is 6.58. Remember
that the exact solution is 4.99382.

• Calculate f(cr)

Since f(cl) f(cr) 6.59 x (-2.41) lt 0, we let
cu 6.58 and repeat the process.
26

• Estimate the root to be

Note that this is much closer to the exact
solution of 4.99382 than the previous estimate
of 6.58.

• Calculate f(cr)

Since f(cl) f(cr) 6.59 x (-1.11) lt 0, we let
cu 5.63 and repeat the process. You can do this
until the result is accurate enough for you.
27
Comparison between Bisection method and method of
False Position

• Note that
• there is an monotonic
• convergence to the exact
• solution using the method
• of False position.
• The Bisection method
• oscillates towards the
• exact answer.
• Method of False position
• converges slightly faster to
• the exact answer.

28
Open Methods

• Fixed-point iteration
• Newton-Raphson method
• Secant method

• These methods require either only one initial
  guess or two
• initial guesses which do not have to bracket
  the actual root.
• Unlike bracketing methods which always converge
  (i.e. the
• root for a particular equation is found), the
  open methods of root
• finding may not converge. However, if they do
  converge, they
• do so much more rapidly than bracketing methods.

29
Fixed point iteration
To find the root of f(x)0, arrange the
function so that x is on the left hand side
(LHS) of the equation. i.e. xg(x). We will
rewrite the above equation as xi1g(xi). Eq.
(1.8) above provides us with a iterative formula
to find the roots of Eq. (1.7). Note that the
functional form of Eq. (1.7) is not changed.
Eq. (1.8) is just a rearrangement of Eq. (1.7).
(1.7)
(1.8)
Lets say we want to find the root to f(x)
e-x-x0, we will rearrange it to get
xe-x I.e. g(x) e-x and h(x)x
(1.9)
30
Old guess of root to Eq. (1.9)
New guess of root to Eq. (1.9)
Example 1.3
Problem statement Solve Problem 1.1 by using
fixed point iteration to obtain the root of
31
Solution
We have to set f( c )0 and re-arrange it so that
we get
So we let the iterative methodology of finding
the roots to f( c ) be
(1.10)
If we compare this equation with Eq. (1.8), then
This is shown in the figure below
32
(No Transcript)
33
To find the root of f( c ) using fixed point
iteration, we will use Eq. (1.10). Let our
first guess be c11.
You can repeat however many times you like.
Observe that at the third iteration, you are
already very close to the exact value of 4.99382
! This method is very fast.what is the catch ???
34
The convergence of fixed point iteration
The iterative equation for fixed-point iteration
is
Let the true solution for f(x)0 be
Subtracting Eq. (1.12) from (1.11) gives
(1.13)
We can manipulate Eq. (1.13) and write it as
35
(1.14)
The derivative mean value theorem states that
where x is somewhere in between a and b
36
In plain English, derivative mean value theorem
say that there is a point (xx) of a function
(g) in between a and b such the derivative of
that function is given by g(b)-g(a)/(b-a)
If we apply the derivative mean value theorem in
the context of the current discussion, we get
(1.14)
where x is somewhere in between xr and xi. If we
substitute Eq. (1.14) into Eq. (1.13), we get
(1.15)
Difference between new guess, xi1 , and actual
root, xr
Difference between old guess, xi , and actual
root, xr
37
Eq. (1.15) says that the newer guess will be
closer to the actual root than the older guess
providing that dg(x)/dx lt 1 . So this is
one condition for convergence. Note that it does
not mean that if you have dg (x)/dx gt 1, the
solution process will blow up. Use this
condition only as a good guide. Sometimes you
can still get a converged solution if you start
have dg (x)/dx gt 1. It really depends on the
problem you are trying to solve.
Example 1.4
Problem statement Find the root for the
following equation
by using the fixed point iteration. Try to find
the solution with g(x)ex/10 and x01.0 and
x04.0 as your initial guesses. Which on of your
solutions blow up ? Why ? Can you get a
solution with x03.5 ?
38
Partial solution f(x) looks like
There are actually two roots to f(x). They are
x0.1118 and x3.5772.
To use fixed point iteration, we let f(x)0 to
find the iterative formula
so g(x)ex/10.
39
If we start off the solution with x01.0, we will
converge to the root at x0.1118 since
dg/dx(x0) lt 1.0. If we start the solution with
x04.0, will NOT converge to the root at x3.5772
and the solution will blow up since dg/dx(x0) gt
1.0!!
40
Exercise Let
a) Find the two roots for f(x)
analytically. b) Show that f(x) can be
rearranged into the following forms
41
c) With these three expressions for gi(x), use
the fixed point iteration method to find
the approximate root for f(x). Start with
the initial guess of xo4. Compare your
solutions to the answer you get in part
a). d) Sketch all the gi(x) and show what is
happening graphically.
42
Newton-Raphson Method

• Most widely used root finding method.
• Converges quickly.
• Only need one initial guess.

From the graph on the left, it is easy to express
xi in terms of f(xi), f(xi) and xi1.
1
x
(1.16)
43
So to find a root using Newton-Raphson method,
do the following 1) Let the initial guess be
xi 2) Find xi1 by using Eq. (1.16) 3) Let xi
xi1 repeat steps 2 and 3 until you feel your
answer is accurate enough.
Example Solve Problem 1.1 by using Newtons
method
44
To find the root of f( c ) using Newton Raphson
Method, use Eq. (1.16) with f( c ) and f( c )
given above. Start with c13.
iter1
iter2
iter3
iter4
45
One of the reasons why Newton Raphsons method is
so popular is that it converges very fast.
Proof-
The Taylor series expansion can be represented as
(1.17)
Where x lies somewhere in the interval from xi to
xi1. If we let xi1 xr, then we will have
f(xi1)f(xr)0. Put this information into Eq.
(1.17) to get
(1.18)
We can also rearrange Eq. (1.16) to get
(1.19)
46
Subtract Eq. (1.19) from Eq. (1.18) to get
Rearrange the above equation to obtain
(1.20)
Error between the actual root and the previous
guess
Error between the actual root and the current
guess
According to Eq. (1.20), the error in Newton
Raphson method is roughly proportional to the
square of the previous error. Therefore we have
quadratic convergence i.e. the number of correct
decimal places approximately doubles with each
iteration.
47

• Several pitfalls of Newton Raphson method
• Requires the evaluation of the derivative of a
  function.
• Sometimes, this is difficult or inconvenient
  to do.
• Have lots of problems in the vicinity of local
  maxima
• or minima because f(xi)0 if xi is at a local
  maxima or
• minima.

• This method also diverges in the situation below

f(x)
x2
xo
x1
48
Secant Method
In the secant method, the derivative of a
function is approximated by a backward finite
divided difference
(1.21)
Substitute Eq. (1.21) into Eq. (1.16) to obtain
the following iterative formula for the Secant
Method
(1.22)
Notice that the above formula requires two
initial estimates of x, i.e. xi and xi-1.
However, because the function f(x) is not require
to change signs between the estimates, it is not
classified as a bracketing method.
49
What is the difference between the Secant method
and the method of False Position ?
False position
Secant
f(x)
xi-1
xi
xr
f(x)
f(x)
xr
xl
xu
xi
xi-1
xr
50

• Critical difference is how one of the initial
  values is replaced
• by a new estimate.
• In the method of False Position, the latest
  estimate of the
• root replaces whichever of the original
  estimate such that
• the two guesses will always bracket the root.
  Therefore
• it is always stable.
• In the secant method, the values are replaced in
  a sequential
• fashion, xi1 replaces xi, and xi replaces
  xi-1. For certain cases, this
• could lead to divergence.

Exercise Solve Problem 1.1 using the Secant
method.
51
Systems of nonlinear equations
So far, we have only talked about methods
of finding roots for 1 equation. What if we need
to simultaneously find roots for 2 or more
equations ? For example, what if you to find a
set of x and y values so that they satisfy the
following two equations.
(1.23)
(1.24)
Point that satisfies both Eqs. (1.23) and (1.24)
52
In general, what we would like to do is to find
all the xis so that we satisfy all of the
following equations.
For the purpose of this course, lets just
confine ourselves to n2. So we wish to find x1
and x2 so that the following equations are
satisfied
53
The multivariable Taylor series for u and v can
be written as
For simplicity, I will write
So the above two equations can be re-written as
(1.25)
(1.26)
54
Eqs. (1.25) and (1.26) can be re-written as
(1.27)
(1.28)
if we set ui1 vi10. Equations (1.27) and
(1.28) are a set of two linear equations. They
can easily be put into matrix form and solved.
55
Exercise Solve Eqs. (1.27) and (1.28) for xi1
and yi1 and show that
(1.29)
(1.30)
56
Exercise Solve
(1.31)
(1.32)
Using the two-equation Newton-Raphson method with
the iterative formula given by Eqs. (1.29) and
(1.30)