Metrics - PowerPoint PPT Presentation

Loading...

PPT – Metrics PowerPoint presentation | free to view - id: aae21-ZjVhN



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Metrics

Description:

Outline the Goal, Question, Metric approach. Justify Asymptotic Complexity ... Is a single user's rating of usability reliable? Can be manipulated ... – PowerPoint PPT presentation

Number of Views:272
Avg rating:3.0/5.0
Slides: 21
Provided by: julie57
Category:
Tags: metrics

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Metrics


1
Metrics Efficiency
  • Explain why metrics are used
  • Outline the Goal, Question, Metric approach
  • Justify Asymptotic Complexity
  • Assess the complexity of simple algorithms

2
Metrics What and Why?
  • Measure of an interesting attribute
  • An attribute belongs to an entity
  • A metric attaches a value to an attribute
  • Examples

3
Why do we want metrics?
  • To compare features of different items
  • Is x faster or more reliable or cheaper than y
  • To assess the value of an individual item
  • How fast is z?
  • Other features quality, reliability, cost
  • To make predictions
  • If w has a complexity score of c, how long will
    it take to develop?
  • Is the design for module m too complex

4
What can we measure?
  • We can measure attributes of
  • Products
  • E.g. a program efficiency, quality
  • Processes
  • The methods used cost effectiveness
  • Resources
  • Inputs to a process
  • E.g. people productivity
  • E.g. software tools - usability

5
Desirable features of a metric
  • Valid
  • actually measures the attribute
  • Does counting if statements measure complexity?
  • Reliable
  • consistent and repeatable
  • Is a single users rating of usability reliable?
  • Can be manipulated
  • Combine measures, take averages,
  • This is affected by scale type of metric

6
Metric Scale Type
  • Nominal (classify search, sort, calculation)
  • Value is a name
  • Ordinal (first-1, second-2, third-3 )
  • Difference between values no meaning
  • Interval (A date)
  • Differences between values are meaningful
  • Ratio (Execution time)
  • There is a fixed 0 point
  • Absolute
  • A count

7
Scale Type Affects Manipulation
  • Nominal (classify search, sort, calculation)
  • Cant calculate average or compare values
  • Ordinal (first-1, second-2, third-3 )
  • Can put into order, but cant calculate mean
  • Interval (A date)
  • Can average compare
  • Cant say one is twice another
  • Ratio of differences is meaningful
  • Ratio (Execution time) Absolute
  • Can average say one is n-times another

8
Deciding what to measure
Would be similar for network performance
  • Goal, Question, Metric
  • What is the purpose/goal of the investigation?
  • E.g. to select the the best algorithm
  • What questions are relevant?
  • E.g. how fast? How much memory used?
  • What measures can answer the questions?
  • Experimentally measured speed
  • Algorithmic complexity (see later)
  • Other issues
  • Reliability, ease of measurement
  • Asking the right questions

Another Example Goal Effectiveness of web
server Question How fast How long do
users wait Metric Throughput (average?)
Waiting time (average, worst)
9
What Is Complexity?
  • Complexity has two meanings
  • Algorithmic complexity
  • Performance efficiency
  • Measures of time space demands
  • Standard approach asymptotic complexity
  • Module complexity
  • Measure of difficulty / understandability
  • No standard approach

10
Measuring Algorithm Efficiency
  • How long will the algorithm take?
  • Why not just measure the programs time?
  • What factors affect Efficiency/Complexity?
  • Desirable features of a complexity metric
  • Independent of programming language
  • Independent of hardware implementation
  • Allows comparison of algorithms
  • Robust
  • Allows determination of time on any PC
  • Asymptotic complexity doesnt do this

11
Asymptotic Complexity
  • Identify the key operation
  • Usually depends on the problem domain
  • Sorting comparison
  • Numerical analysis multiplication/division
  • Virtual memory memory access
  • The expensive operation
  • Count operations for given input size
  • Results in a formula
  • Investigate formula for large input size
  • Essentially keep only the big terms
  • Asymptotic means as a value gets large

12
Example
  • Joining all the strings in an array
  • Key operation
  • Joining a string
  • Analysis
  • Let n the number of strings
  • There will be n-1 joins
  • The complexity is about n for large arrays
  • Can ignore the 1

13
Big-O Notation
  • A way of comparing function size
  • Classification Examples
  • Can you give a rule for identifying O(f(n))

14
The effect of the main term
Complexity
n2
n Log n
n
Log n
Size
15
Why Asymptotic Complexity?
  • Advantages
  • A measure of the algorithm
  • Independent of
  • implementation details
  • Hardware
  • Robust
  • Straightforward to calculate
  • Disadvantages
  • May not be true for small amount of input
  • Ignores slight differences in algorithms

16
Example 1 Sorting an Array
  • Bubblesort
  • for (int i0 i lt data.length-1 i)
  • for (int ji j lt data.length j)
  • if (dataj gt dataj1)
  • swap(dataj, dataj1)
  • Complexity
  • Approximately n times round outer loop
  • Inner loop approximately n comparisons
  • Overall complexity O(n2)
  • Rough guide nested loop ? n2

17
Sorting an array Quicksort
  • Quicksort
  • quicksort(start, end, data)
  • int mid partition(start, end, data)
  • quicksort(start,mid)
  • quicksort(mid,end)
  • Complexity
  • Complexity of partition is O(n)
  • quicksort will be called O(log n) times, doing a
    partition each time
  • Overall quicksort is O(n log n)

18
Example 2 Accessing a Tree
Are 0, 13 in the tree?
  • Ordering
  • Values in a left (right) descendent of a node are
    less (greater) than the value in a node
  • Depth of a tree
  • A 3-layer tree can hold 7 items (formula?)
  • A tree with log2 N layers can hold N items
  • How long would it take to check for an item?

19
Trees that arent compact
  • Trees are often built as values arrive
  • If values arrive at random, the tree is usually
    reasonably compact lots of branching
  • If we are unlucky, a linear tree can result
  • How many comparisons are needed in this worst
    case?

Yes values to the right are bigger than the
value at a node. All left branches are empty.
0
3
7
8
Is this a tree?
11
Are 0, 13 in the tree?
19
70
20
Summary
  • A metric is a measurement of an attribute
  • Goals, Questions, Metrics
  • Systematic way of identifying metrics
  • Asymptotic complexity
  • Comparing algorithmic efficiency
  • Identify key operation
  • Formula counting operations for input size
  • Only retain most significant terms
  • Robust implementation independent
About PowerShow.com