Metrics - PowerPoint PPT Presentation


PPT – Metrics PowerPoint presentation | free to view - id: aae21-ZjVhN


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation



Outline the Goal, Question, Metric approach. Justify Asymptotic Complexity ... Is a single user's rating of usability reliable? Can be manipulated ... – PowerPoint PPT presentation

Number of Views:272
Avg rating:3.0/5.0
Slides: 21
Provided by: julie57
Tags: metrics


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Metrics

Metrics Efficiency
  • Explain why metrics are used
  • Outline the Goal, Question, Metric approach
  • Justify Asymptotic Complexity
  • Assess the complexity of simple algorithms

Metrics What and Why?
  • Measure of an interesting attribute
  • An attribute belongs to an entity
  • A metric attaches a value to an attribute
  • Examples

Why do we want metrics?
  • To compare features of different items
  • Is x faster or more reliable or cheaper than y
  • To assess the value of an individual item
  • How fast is z?
  • Other features quality, reliability, cost
  • To make predictions
  • If w has a complexity score of c, how long will
    it take to develop?
  • Is the design for module m too complex

What can we measure?
  • We can measure attributes of
  • Products
  • E.g. a program efficiency, quality
  • Processes
  • The methods used cost effectiveness
  • Resources
  • Inputs to a process
  • E.g. people productivity
  • E.g. software tools - usability

Desirable features of a metric
  • Valid
  • actually measures the attribute
  • Does counting if statements measure complexity?
  • Reliable
  • consistent and repeatable
  • Is a single users rating of usability reliable?
  • Can be manipulated
  • Combine measures, take averages,
  • This is affected by scale type of metric

Metric Scale Type
  • Nominal (classify search, sort, calculation)
  • Value is a name
  • Ordinal (first-1, second-2, third-3 )
  • Difference between values no meaning
  • Interval (A date)
  • Differences between values are meaningful
  • Ratio (Execution time)
  • There is a fixed 0 point
  • Absolute
  • A count

Scale Type Affects Manipulation
  • Nominal (classify search, sort, calculation)
  • Cant calculate average or compare values
  • Ordinal (first-1, second-2, third-3 )
  • Can put into order, but cant calculate mean
  • Interval (A date)
  • Can average compare
  • Cant say one is twice another
  • Ratio of differences is meaningful
  • Ratio (Execution time) Absolute
  • Can average say one is n-times another

Deciding what to measure
Would be similar for network performance
  • Goal, Question, Metric
  • What is the purpose/goal of the investigation?
  • E.g. to select the the best algorithm
  • What questions are relevant?
  • E.g. how fast? How much memory used?
  • What measures can answer the questions?
  • Experimentally measured speed
  • Algorithmic complexity (see later)
  • Other issues
  • Reliability, ease of measurement
  • Asking the right questions

Another Example Goal Effectiveness of web
server Question How fast How long do
users wait Metric Throughput (average?)
Waiting time (average, worst)
What Is Complexity?
  • Complexity has two meanings
  • Algorithmic complexity
  • Performance efficiency
  • Measures of time space demands
  • Standard approach asymptotic complexity
  • Module complexity
  • Measure of difficulty / understandability
  • No standard approach

Measuring Algorithm Efficiency
  • How long will the algorithm take?
  • Why not just measure the programs time?
  • What factors affect Efficiency/Complexity?
  • Desirable features of a complexity metric
  • Independent of programming language
  • Independent of hardware implementation
  • Allows comparison of algorithms
  • Robust
  • Allows determination of time on any PC
  • Asymptotic complexity doesnt do this

Asymptotic Complexity
  • Identify the key operation
  • Usually depends on the problem domain
  • Sorting comparison
  • Numerical analysis multiplication/division
  • Virtual memory memory access
  • The expensive operation
  • Count operations for given input size
  • Results in a formula
  • Investigate formula for large input size
  • Essentially keep only the big terms
  • Asymptotic means as a value gets large

  • Joining all the strings in an array
  • Key operation
  • Joining a string
  • Analysis
  • Let n the number of strings
  • There will be n-1 joins
  • The complexity is about n for large arrays
  • Can ignore the 1

Big-O Notation
  • A way of comparing function size
  • Classification Examples
  • Can you give a rule for identifying O(f(n))

The effect of the main term
n Log n
Log n
Why Asymptotic Complexity?
  • Advantages
  • A measure of the algorithm
  • Independent of
  • implementation details
  • Hardware
  • Robust
  • Straightforward to calculate
  • Disadvantages
  • May not be true for small amount of input
  • Ignores slight differences in algorithms

Example 1 Sorting an Array
  • Bubblesort
  • for (int i0 i lt data.length-1 i)
  • for (int ji j lt data.length j)
  • if (dataj gt dataj1)
  • swap(dataj, dataj1)
  • Complexity
  • Approximately n times round outer loop
  • Inner loop approximately n comparisons
  • Overall complexity O(n2)
  • Rough guide nested loop ? n2

Sorting an array Quicksort
  • Quicksort
  • quicksort(start, end, data)
  • int mid partition(start, end, data)
  • quicksort(start,mid)
  • quicksort(mid,end)
  • Complexity
  • Complexity of partition is O(n)
  • quicksort will be called O(log n) times, doing a
    partition each time
  • Overall quicksort is O(n log n)

Example 2 Accessing a Tree
Are 0, 13 in the tree?
  • Ordering
  • Values in a left (right) descendent of a node are
    less (greater) than the value in a node
  • Depth of a tree
  • A 3-layer tree can hold 7 items (formula?)
  • A tree with log2 N layers can hold N items
  • How long would it take to check for an item?

Trees that arent compact
  • Trees are often built as values arrive
  • If values arrive at random, the tree is usually
    reasonably compact lots of branching
  • If we are unlucky, a linear tree can result
  • How many comparisons are needed in this worst

Yes values to the right are bigger than the
value at a node. All left branches are empty.
Is this a tree?
Are 0, 13 in the tree?
  • A metric is a measurement of an attribute
  • Goals, Questions, Metrics
  • Systematic way of identifying metrics
  • Asymptotic complexity
  • Comparing algorithmic efficiency
  • Identify key operation
  • Formula counting operations for input size
  • Only retain most significant terms
  • Robust implementation independent