CSE 202 - Algorithms

1
CSE 202 - Algorithms

• Spring 2002
• Instructor Larry Carter
• carter_at_cs.ucsd.edu
• Office hours (4101 APM)
• Tu Th 300 400
• (or by appointment
• or whenever Im in)

2
What well study

• General Techniques
• Divide and Conquer, Dynamic Programming, Hashing,
  Greedy Algorithms, Reduction to other problems,
  ...
• Specific Problems
• Sorting, sorting, shortest paths, max flow,
  sorting, ...
• Various Paradigms
• Probabilistic algorithms
• Alternate models of computation
• NP Completeness

3
Sounds like my undergrad course ...

• Going over same material twice is good!
• Well probably go deeper
• mathematical formalisms
• modified assumptions
• assorted topics every computer scientist should
  know.

4
Logistics

• Textbook Introduction to Algorithms, 2nd edition
• Cormen, Leiserson, Rivest Stein
• First edition probably OK (new chapter K old
  chapter (K1).)
• Homework tests
• Individual homeworks
• Group projects
• In-class Midterm
• Oral Final
• Grades
• A Demonstrate mastery of material.
• B (or B or B-) Typical grade. Understand most
  of material, solve most routine problems and some
  hard ones.
• C Really dont get it.
• D or F Gave up.

5
Logistics

• Classes will include lecture (with overheads),
  discussion (at board), and exercises
  (parcipitory).
• Id prefer you pay attention, think, ask
  questions, and participate in discussions rather
  than taking notes.
• Website //www.cs.ucsd.edu/classes/sp03/cse202
• .pdf of lectures.
• Homeworks, announcements, etc. too
• You are responsible for checking website
• Be sure to register for the class email list
• Ill use it to send out urgent messages, like HW
  corrections
• Directions on class webpage

6
Formal Mathematics

• Most objects we encounter will be one of
• A number (what type of number?)
• A character (from what alphabet?)
• A set (of what type elements?)
• A sequence (of what type elements?)
• A function (what the type of domain range?)
• A tuple, i.e. an ordered pair, triple, ...
• (what are the types of each element of the
  tuple?)
• Just like declarations in a program.
• You should always know the type of whatever we
  are talking about.

Note I underline terms that you should know.
7
Formal Analysis of Algorithms

• A problem is a (infinite) set of instances.
• An instance is a triple ltinput, output, sizegt
• Less formally, we treat input as an instance,
  and output as a function of input.
• Size is related to the number of symbols needed
  to represent the instance, but the exact
  convention is problem-specific. (E.g., nxn matrix
  may be size n.)
• A decision problem is a problem in which every
  instances output is yes or no.
• Example Sorting is a problem.
• lt5, 2, 17, 6gt is an instance of sorting, with
  output lt2, 5, 6, 17gt and size 4.

8
Formal Analysis of Algorithms

• Algorithm a step-by-step method of producing
  output of instances given input
• The steps are instructions for some model of
  computation.
• Standard model is the Random Access Machine
  (RAM)

One needs to be careful about what operations are
allowed - E.g. operations on very long numbers
arent one step. - For this course
anything reasonable is OK.
9
Formal Analysis of Algorithms

• The complexity of an algorithm is a function from
  ? (the instance size) to ? (the running time on
  instances of that size).
• But what happens if different instances of a
  given size have different running times??
• Worst-case complexity maximum of the running
  times over all instances of a given size.
• Average complexity average of the running times.
• Probabilistic complexity well study later can
  be used when a single instance has different
  running times.
• Best-case complexity not a very useful concept

? 0, 1, 2, ... (the natural numbers)
10
Mathematical notation

• P.T. Barnum asserted, You can fool all of the
  people some of the time.
• Suppose on Monday, I fooled all the men, and on
  Tuesday, I fooled all the women (but not vice
  versa).
• Is this an example of Barnums assertion?
• Mathematics is (or should be) precise
• Let P set of all the people, T
  Monday,Tuesday,
• Let F(p,t) mean I fooled person p at time t
• Is ?p ? P ?t ? T F(p,t) true?
• Is ?t ? T ?p ? P F(p,t) true?

• means for all
• ? means there exists

11
Classifying (complexity) functions

• We focus on functions from ? to ?, though
  definitions are general.
• Given function g, O(g) (pronounced Oh of g or
  Big Oh of g) is the set of functions (from ? to
  ?) for which g is an asymptotic upper bound. This
  means
• O(g) f ?n0?? ?cgt0 ?ngtn0 0 ? f(n) ? c g(n)
  
• Example 3n log n 20 ? O(n2)

Note Since O(n2) is a set, using ? is
technically correct. But people often use .
12
Classifying (complexity) functions

• ?(g) (pronounced Omega or Big Omega) is the
  set of functions for which g is an asymptotic
  lower bound. Formally
• ?(g) f ?n0?? ?cgt0 ?ngtn0 0 ? c g(n) ? f(n)
  
• ?(g) (Theta) is the set of functions for which
  g is an asymptotic tight bound. This means
• ?(g) f ?n0?? ?c1,c2gt0 ?ngtn0 0 ? c1g(n) ?
  f(n) ? c2g(n)
• Theorem ?(g) ?(g) ? O(g).
• Note we wont use little-o or little-omega
  notation.

13
Review

• Consider the decision problem, Is x prime?
• How should we define size?
• Consider the algorithm
• if(ilt2) return no
• for i 2 to sqrt(x)
• if (xi 0) return no
• return yes
• Note when x is even, algorithm takes constant
  time.
• What is the asymptotic complexity?
• You should be asking, average or worst-case?
• Lets say worst case (average complexity is
  harder)

14
What is a proof?

• Informal Proof (Hand waving)
• Anything that should convince an alert reader
• Formal Proof (First few classes)
• A sequence of statements, each being
• a hypothesis of the theorem
• a definition, axiom, or known theorem
• a statement that follows from previous statements
  via a rule of inference
• Proof (Rest of course)
• A subset or summary of a formal proof that
  convinces a literate but sleepy reader that the
  writer could write a formal proof if forced to.

15
Notes about proofs

• Use complete sentences.
• Sentences have a subject (you may be
  understood), a verb, a period at the end.
• If sentence is too long, introduce notation or
  definitions.
• Each statement should indicate whether its an
  assumption, a definition, a known fact, ...
• Dont just write, x ? A. Instead write
• Let x ? A. if youre introducing x and want it
  to be in A.
• Suppose x ? A. if x has already been
  introduced, and youre seeing what would happen
  if it were in A.
• Thus, x ? A. if it follows from earlier
  statements.
• Make sure each variable is properly introduced
• Like declaring variables in a program.

16
Some proof schemas

• Literate readers even sleepy ones know that
• To show A? B, can write Let x ? A. (blah ...).
  Thus, x ? B.
• To show A B (for sets A and B), show A? B and
  B? A.
• To show P implies Q, write Assume P. (blah ...).
  Thus Q.
• To show ?x P, write Let x(whatever). (blah
  ...) Thus P.
• To show ?x P, write Given any x, (blah ...).
  Thus P.
• To show an algorithm has complexity O(f), you
  will probably construct a positive constant c and
  an integer n0 and then show that if ngtn0 and I is
  an instance of size n, then the algorithm
  requires time at most c f(n) on I.
• These schemas can be mostly implicit. Thus, after
  writing Let x ? A. (blah blah). Thus, x ? B.
  you often dont need to write This shows that A?
  B.

17
Example formal proof

• Thm Suppose f?O(g) and g?O(h). Then f?O(h).
• Proof
• Since f? O(g), ?n0?? ?c0gt0 ?ngtn0 0 ? f(n) ? c0
  g(n).
• Similarly, ?n1?? ?c1gt0 ?ngtn1 0 ? g(n) ? c1
  h(n).
• Let n2 max(n0, n1) and c2 c0 c1. Note that
  c2 is positive since both c0 and c1 are.
• Suppose ngtn2. Then ngtn0 and so 0 ? f(n) ? c0
  g(n).
• But we also have ngtn1 and so g(n) ? c1 h(n).
• Thus, 0 ? f(n) ? c0 c1 h(n) c2 h(n).
• Q.E.D.

Just the definition of O(g), but introduces c0
and n0.
Setup for proving ?n2?? ?c2gt0 ...
Setup for proving ?ngtn2 ...
Means, Weve proved what we intended to.
18
Your turn ...

• Thm If f?O(g) then g??(f).

19
Mathematical Induction

• Suppose for ? i??, Pi is a statement.
• Suppose also that we can prove P0, and we can
  prove ? i??, Pi implies Pi1
• Then the Principle of Mathematical Induction
  allows us to conclude ? i?? Pi.

...
P4
P3
P2
P1
P0
20
Mathematical Induction

• Alternatively, suppose we can prove P0, and we
  can prove ? i??, (P0 P1 ... Pi) implies
  Pi1
• Again, the Principle of Mathematical Induction
  allows us to conclude ? i?? Pi.

P6
P4
...
P2
P7
P5
P3
P1
P0
21
Induction Example

• Definitions from CLRS pg. 1088
• A binary tree is a structure on a finite number
  of nodes that either contains no nodes or is
  composed of three disjoint subsets a root node,
  a binary tree called the left subtree and a
  binary tree called the right subtree.
• The depth of a node is the length of the path
  from the root to the node.
• The height of a non-empty tree is the maximum
  depth of its nodes.
• Thm If T is a non-empty binary tree of height h,
  then T has fewer than 2h1 nodes.

22
Induction Example

• Let Ph be the statement, If T is a binary tree
  of height h, then T has at most 2h11 nodes.
• We will prove ? h?? Ph by induction.
• Base Case (h0) T is non-empty, so it has a root
  node r. Let s be any node of T. Since the height
  of T is 0, the depth of s must be 0, so s r.
  Thus, T has only one node (which is ? 201-1 1).

23
Induction Example

• Let Ph be the statement, If T is a binary tree
  of height h, then T has at most 2h11 nodes.
• We will prove ? h?? Ph by induction.
• Base Case (h0) T is non-empty, so it has a root
  node r. Let s be any node of T. Since the height
  of T is 0, the depth of s must be 0, so s r.
  Thus, T has only one node (which is ?? 201-1).
• Obviously, the only binary tree of height 0 is
  the tree of one node, so P0 is true.

24
Induction Example

• Induction step Assume that Ph is true.
• Let T be a tree of height h1. Then the left
  subtree L is a binary tree.
• If L is empty, it has 0 nodes.
• Otherwise, each node in L is has depth one less
  than its depth in T. Thus, L is a non-empty
  binary tree of depth at most h. By assumption
• OOOPS!
• Ive only assumed Ph. But L may have smaller
  height.

25
Induction Example

• Induction step Assume that Pi is true ? i?h.
• Let T be a tree of height h1. Then the left
  subtree L is a binary tree.
• If L is empty, it has 0 nodes.
• Otherwise, each node in L is has depth one less
  than its depth in T. Thus, L is a non-empty
  binary tree of depth at most h. By assumption, L
  has at most 2h11 nodes.
• Similarly, the right subtree R has at most 2h11
  nodes.
• Thus, T has at most 1 2h11 2h11 2
  2h11
• 
  2(h1)11 nodes.
• This shows that Ph1 is true, and completes our
  proof.

for L
for the root
for R