Module

1
Module 5Algorithms

• Rosen 5th ed., 2.1
• 31 slides, 1 lecture

2
Chapter 2 More Fundamentals

• 2.1 Algorithms (Formal procedures)
• 2.2 Complexity of algorithms
• Analysis using order-of-growth notation.
• 2.3 The Integers Division
• Some basic number theory.
• 2.6 Matrices
• Some basic linear algebra.

3
2.1 Algorithms

• The foundation of computer programming.
• Most generally, an algorithm just means a
  definite procedure for performing some sort of
  task.
• A computer program is simply a description of an
  algorithm in a language precise enough for a
  computer to understand, requiring only operations
  the computer already knows how to do.
• We say that a program implements (or is an
  implementation of) its algorithm.

4
Algorithms You Already Know

• Grade school arithmetic algorithms
• How to add any two natural numbers written in
  decimal on paper using carries.
• Similar Subtraction using borrowing.
• Multiplication long division.
• Your favorite cooking recipe.
• How to register for classes at UF.

5
Programming Languages

• Some common programming languages
• Newer Java, C, C, Visual Basic, JavaScript,
  Perl, Tcl, Pascal
• Older Fortran, Cobol, Lisp, Basic
• Assembly languages, for low-level coding.
• In this class we will use an informal,
  Pascal-like pseudo-code language.
• You should know at least 1 real language!

6
Algorithm Example (English)

• Task Given a sequence aia1,,an, ai?N, say
  what its largest element is.
• Set the value of a temporary variable v (largest
  element seen so far) to a1s value.
• Look at the next element ai in the sequence.
• If aigtv, then re-assign v to the number ai.
• Repeat previous 2 steps until there are no more
  elements in the sequence, return v.

7
Executing an Algorithm

• When you start up a piece of software, we say the
  program or its algorithm are being run or
  executed by the computer.
• Given a description of an algorithm, you can also
  execute it by hand, by working through all of its
  steps on paper.
• Before WWII, computer meant a person whose job
  was to run algorithms!

8
Executing the Max algorithm

• Let ai7,12,3,15,8. Find its maximum
• Set v a1 7.
• Look at next element a2 12.
• Is a2gtv? Yes, so change v to 12.
• Look at next element a2 3.
• Is 3gt12? No, leave v alone.
• Is 15gt12? Yes, v15

9
Algorithm Characteristics

• Some important features of algorithms
• Input. Information or data that comes in.
• Output. Information or data that goes out.
• Definiteness. Precisely defined.
• Correctness. Outputs correctly relate to inputs.
• Finiteness. Wont take forever to describe or
  run.
• Effectiveness. Individual steps are all do-able.
• Generality. Works for many possible inputs.
• Efficiency. Takes little time memory to run.

10
Our Pseudocode Language A2

• procedurename(argument type)
• variable expression
• informal statement
• begin statements end
• comment
• if condition then statement else statement

• for variable initial value to final value
  statement
• while condition statement
• procname(arguments)
• Not defined in book
• return expression

11
procedure procname(arg type)

• Declares that the following text defines a
  procedure named procname that takes inputs
  (arguments) named arg which are data objects of
  the type type.
• Exampleprocedure maximum(L list of
  integers) statements defining maximum

12
variable expression

• An assignment statement evaluates the expression
  expression, then reassigns the variable variable
  to the value that results.
• Examplev 3x7 (If x is 2, changes v
  to 13.)
• In pseudocode (but not real code), the expression
  might be informal
• x the largest integer in the list L

13
Informal statement

• Sometimes we may write a statement as an informal
  English imperative, if the meaning is still clear
  and precise swap x and y
• Keep in mind that real programming languages
  never allow this.
• When we ask for an algorithm to do so-and-so,
  writing Do so-and-so isnt enough!
• Break down algorithm into detailed steps.

14
begin statements end

• Groups a sequence of statements togetherbegin
  statement 1 statement 2 statement
  n end

• Allows sequence to be used like a single
  statement.
• Might be used
• After a procedure declaration.
• In an if statement after then or else.
• In the body of a for or while loop.

15
comment

• Not executed (does nothing).
• Natural-language text explaining some aspect of
  the procedure to human readers.
• Also called a remark in some real programming
  languages.
• Example
• Note that v is the largest integer seen so far.

16
if condition then statement

• Evaluate the propositional expression condition.
• If the resulting truth value is true, then
  execute the statement statement otherwise, just
  skip on ahead to the next statement.
• Variant if cond then stmt1 else stmt2Like
  before, but iff truth value is false, executes
  stmt2.

17
while condition statement

• Evaluate the propositional expression condition.
• If the resulting value is true, then execute
  statement.
• Continue repeating the above two actions over and
  over until finally the condition evaluates to
  false then go on to the next statement.

18
while condition statement

• Also equivalent to infinite nested ifs, like so
  if condition begin statement
  if condition begin
  statement (continue
  infinite nested ifs) end end

19
for var initial to final stmt

• Initial is an integer expression.
• Final is another integer expression.
• Repeatedly execute stmt, first with variable var
  initial, then with var initial1, then with
  var initial2, etc., then finally with var
  final.
• What happens if stmt changes the value that
  initial or final evaluates to?

20
for var initial to final stmt

• For can be exactly defined in terms of while,
  like so

begin var initial while var ? final
begin stmt
var var 1 endend
21
procedure(argument)

• A procedure call statement invokes the named
  procedure, giving it as its input the value of
  the argument expression.
• Various real programming languages refer to
  procedures as functions (since the procedure call
  notation works similarly to function application
  f(x)), or as subroutines, subprograms, or methods.

22
Max procedure in pseudocode

• procedure max(a1, a2, , an integers)
• v a1 largest element so far
• for i 2 to n go thru rest of elems
• if ai gt v then v ai found
  bigger?
• at this point vs value is the same as
  the largest integer in the list
• return v

23
Another example task

• Problem of searching an ordered list.
• Given a list L of n elements that are sorted into
  a definite order (e.g., numeric, alphabetical),
• And given a particular element x,
• Determine whether x appears in the list,
• and if so, return its index (position) in the
  list.
• Problem occurs often in many contexts.
• Lets find an efficient algorithm!

24
Search alg. 1 Linear Search

• procedure linear search(x integer, a1, a2, ,
  an distinct integers)i 1while (i ? n ? x ?
  ai) i i 1if i ? n then location ielse
  location 0return location index or 0 if not
  found

25
Search alg. 2 Binary Search

• Basic idea On each step, look at the middle
  element of the remaining list to eliminate half
  of it, and quickly zero in on the desired element.

ltx
gtx
ltx
ltx
26
Search alg. 2 Binary Search

• procedure binary search(xinteger, a1, a2, ,
  an distinct integers) i 1 left endpoint of
  search intervalj n right endpoint of
  search intervalwhile iltj begin while
  interval has gt1 item m ?(ij)/2?
  midpoint if xgtam then i m1 else j
  mendif x ai then location i else location
  0return location

27
Module 6Orders of Growth

• Rosen 5th ed., 2.2
• 22 slides, 1 lecture

28
Orders of Growth (1.8)

• For functions over numbers, we often need to know
  a rough measure of how fast a function grows.
• If f(x) is faster growing than g(x), then f(x)
  always eventually becomes larger than g(x) in the
  limit (for large enough values of x).
• Useful in engineering for showing that one design
  scales better or worse than another.

29
Orders of Growth - Motivation

• Suppose you are designing a web site to process
  user data (e.g., financial records).
• Suppose database program A takes fA(n)30n8
  microseconds to process any n records, while
  program B takes fB(n)n21 microseconds to
  process the n records.
• Which program do you choose, knowing youll want
  to support millions of users?

A
30
Visualizing Orders of Growth

• On a graph, asyou go to theright, a
  fastergrowingfunctioneventuallybecomeslarger.
  ..

fA(n)30n8
Value of function ?
fB(n)n21
Increasing n ?
31
Concept of order of growth

• We say fA(n)30n8 is order n, or O(n). It is,
  at most, roughly proportional to n.
• fB(n)n21 is order n2, or O(n2). It is roughly
  proportional to n2.
• Any O(n2) function is faster-growing than any
  O(n) function.
• For large numbers of user records, the O(n2)
  function will always take more time.

32
Definition O(g), at most order g

• Let g be any function R?R.
• Define at most order g, written O(g), to be
  fR?R ?c,k ?xgtk f(x) ? cg(x).
• Beyond some point k, function f is at most a
  constant c times g (i.e., proportional to g).
• f is at most order g, or f is O(g), or
  fO(g) all just mean that f?O(g).
• Sometimes the phrase at most is omitted.

33
Points about the definition

• Note that f is O(g) so long as any values of c
  and k exist that satisfy the definition.
• But The particular c, k, values that make the
  statement true are not unique Any larger value
  of c and/or k will also work.
• You are not required to find the smallest c and k
  values that work. (Indeed, in some cases, there
  may be no smallest values!)

However, you should prove that the values you
choose do work.
34
Example 1
35
(No Transcript)
36
Big-O Proof Examples

• Show that 30n8 is O(n).
• Show ?c,k ?ngtk 30n8 ? cn.
• Let c31, k8. Assume ngtk8. Thencn 31n
  30n n gt 30n8, so 30n8 lt cn.
• Show that n21 is O(n2).
• Show ?c,k ?ngtk n21 ? cn2.
• Let c2, k1. Assume ngt1. Then cn2 2n2
  n2n2 gt n21, or n21lt cn2.

37
Big-O example, graphically

• Note 30n8 isntless than nanywhere (ngt0).
• It isnt evenless than 31neverywhere.
• But it is less than31n everywhere tothe right
  of n8.

30n8
30n8?O(n)
Value of function ?
n
Increasing n ?
38
(No Transcript)
39
(No Transcript)
40
Example 2
41
Example 3
42
Theorem 1
43
Example 4
44
Example 5
45
Example 6
46
(No Transcript)
47
Useful Facts about Big O

• Big O, as a relation, is transitive f?O(g) ?
  g?O(h) ? f?O(h)
• O with constant multiples, roots, and logs...? f
  (in ?(1)) constants a,b?R, with b?0, af, f
  1-b, and (logb f)a are all O(f).
• Sums of functionsIf g?O(f) and h?O(f), then
  gh?O(f).

48
More Big-O facts

• ?cgt0, O(cf)O(fc)O(f?c)O(f)
• f1?O(g1) ? f2?O(g2) ?
• f1 f2 ?O(g1g2)
• f1f2 ?O(g1g2) O(max(g1,g2))
  O(g1) if g2?O(g1) (Very useful!)

49
(No Transcript)
50
Theorem 3
51
Example 7
52
Example 8
53
Orders of Growth (1.8) - So Far

• For any gR?R, at most order g,O(g) ? fR?R
  ?c,k ?xgtk f(x) ? cg(x).
• Often, one deals only with positive functions and
  can ignore absolute value symbols.
• f?O(g) often written f is O(g)or fO(g).
• The latter form is an instance of a more general
  convention...

54
Order-of-Growth Expressions

• O(f) when used as a term in an arithmetic
  expression means some function f such that
  f?O(f).
• E.g. x2O(x) means x2 plus some function
  that is O(x).
• Formally, you can think of any such expression as
  denoting a set of functions x2O(x) ? g
  ?f?O(x) g(x) x2f(x)

55
Order of Growth Equations

• Suppose E1 and E2 are order-of-growth expressions
  corresponding to the sets of functions S and T,
  respectively.
• Then the equation E1E2 really means
  ?f?S, ?g?T fgor simply S?T.
• Example x2 O(x) O(x2) means ?f?O(x)
  ?g?O(x2) x2f(x)g(x)

56
Useful Facts about Big O

• ? f,g constants a,b?R, with b?0,
• af O(f) (e.g. 3x2 O(x2))
• fO(f) O(f) (e.g. x2x O(x2))
• Also, if f?(1) (at least order 1), then
• f1-b O(f) (e.g. x?1 O(x))
• (logb f)a O(f). (e.g. log x O(x))
• gO(fg) (e.g. x O(x log x))
• fg ? O(g) (e.g. x log x ? O(x))
• aO(f) (e.g. 3 O(x))

57
Definition 2

• ?(g) f g?O(f)The functions that are at
  least order g.

58
Definition ?(g), exactly order g

• If f?O(g) and g?O(f) then we say g and f are of
  the same order or f is (exactly) order g and
  write f??(g).
• Another equivalent definition?(g) ? fR?R
  ?c1c2k ?xgtk c1g(x)?f(x)?c2g(x)
• Everywhere beyond some point k, f(x) lies in
  between two multiples of g(x).

59
Example 10
60
Example 11
61
Theorem 4
62
Rules for ?

• Mostly like rules for O( ), except
• ? f,ggt0 constants a,b?R, with bgt0, af ? ?(f),
  but ? Same as with O. f ? ?(fg)
  unless g?(1) ? Unlike O.f 1-b ? ?(f), and
  ? Unlike with O. (logb f)c ? ?(f).
  ? Unlike with O.
• The functions in the latter two cases we say are
  strictly of lower order than ?(f).

63
? example

• Determine whether
• Quick solution

64
Other Order-of-Growth Relations

• ?(g) f g?O(f)The functions that are at
  least order g.
• o(g) f ?cgt0 ?k ?xgtk f(x) lt cg(x)The
  functions that are strictly lower order than g.
  o(g) ? O(g) ? ?(g).
• ?(g) f ?cgt0 ?k ?xgtk cg(x) lt f(x)The
  functions that are strictly higher order than g.
  ?(g) ? ?(g) ? ?(g).

65
Relations Between the Relations

• Subset relations between order-of-growth sets.

R?R
?( f )
O( f )
f
?( f )
?( f )
o( f )
66
Why o(f)?O(x)??(x)

• A function that is O(x), but neither o(x) nor
  ?(x)

67
Strict Ordering of Functions

• Temporarily lets write f?g to mean f?o(g),
  fg to mean
  f??(g)
• Note that
• Let kgt1. Then the following are true1 ? log
  log n ? log n logk n ? logk n ? n1/k ? n ? n
  log n ? nk ? kn ? n! ? nn

68
Review Orders of Growth (1.8)

• Definitions of order-of-growth sets, ?gR?R
• O(g) ? f ? cgt0 ?k ?xgtk f(x) lt cg(x)
• o(g) ? f ?cgt0 ?k ?xgtk f(x) lt cg(x)
• ?(g) ? f g?O(f)
• ?(g) ? f g?o(f)
• ?(g) ? O(g) ? ?(g)