Framework%20for%20Metric%20Composition%20 %20Spatial%20Composition%20of%20Metrics

1
Framework for Metric Composition Spatial
Composition of Metrics

• Al Morton many others
• December 3, 2007

2
draft-ietf-ippm-framework-compagg-05.txt
draft-ietf-ippm-spatial-composition-04
and maybe others
3
Types of Composition

• Complete/Sub-Path (or Concatenation in Space)

• Aggregation in Time (12x5min stats -gt1hr)
• Aggregat. in Space

4
Framework 05 Comments from Alan Clark - make the
draft a little more generic and apply beyond IPPM

• Add that metrics can be composed from dissimilar
  metrics (see new sentence in 3.4 and Spatial
  draft Scope of Work 3.1)
• Add new definition of Index, e.g., R-factor (see
  new 3.6)
• Add a new requirement in Section 5, to state the
  (statistical) model of the process being measured
  (and the validity of the assumption) if
  applicable (RFC 2330 Sec 12 guides this to some
  degree).
• Describe an Intermediate Model in Section 4
  capable of representing some key aspect of the
  performance of a class of applications using
  metrics on the system of interest. gtgt This seems
  different from our composition/aggregation scope.
  It might take one or more complete path metrics
  and estimates some reaction at higher layers ltlt

5
Recent Comments on 05

• Reza Fardid
• Section 3.6 R-factor is a valid example of an
  index as defined here, however, it is composed of
  both IP performance metrics and analog metrics.
  On this basis, is it within scope of this draft?
• Agree some refinement is needed here
• There is a packet-performance subset of the
  E-model, where all the analog metrics are fixed
• Lokis point Scope is Loss, Delay, and Delay
  Variation
• Another relevant index composed of IP performance
  metrics is MDI, per rfc4445
• Lets just say NO to referencing REF Editor
  submissions

6
Recent Comments on 05

• Reza Fardid
• Section 4.1 Is the comment referring to
  forecasting of one metric from another, whose
  temporally aggregated measurements are available?
• Al believes what was meant here is like weather
  forecasting prediction of future
• If so, then I fail to see how this is feasible
  given that the primary metrics have been defined
  to be orthogonal.
• Agreed

7
Recent Comments on 05

• Reza Fardid
• Section 4.5 Higher-order Composition
• Another example will be helpful here.
• In particular, if a composed metric is derived
  from dissimilar metrics, e.g., R-factor or MDI,
  then can one make a general statement or provide
  guidance with respect to the order of
  composition to minimize measurement error?
• For instance, is it preferred to compose sub-path
  R-factor or MDI metrics (assuming their
  composition functions are defined) derived from
  primary metrics, or compose the sub-path primary
  metrics first, followed by R-factor or MDI
  composition?
• Is there statistical or empirical evidence to
  support either order in general?

8
Recent Comments on 05

• Loki Jorgenson
• Section 3.2 - there is a reference to "true
  path" with no entry under Terminology to define
  it. Is it related to "ground truth" (which is
  defined)? If it is not intended to be a unique
  term but only a descriptor, I would suggest
  changing it to something more like "actual (as
  used in "ground truth").
• Section 4.1 - with regard to weighting function
  and transient vs. intransient measures (see
  comments further down).
• Section 4.5 - second sentence has "would be a the
  delay"
• Section 5. - As I was dealing with IP issues
  today and aspects of viral licensing, is it
  appropriate to include in any composition metric
  description IP references to embedded proprietary
  metrics and/or use of standards?
• Otherwise, I note, most of the requirements in
  section 5. are technical.
• (a longer discussion on time varying and
  steady-state categories for ground truth that I
  havent had time to summarize)

9
Status of Framework 05

• Next Steps
• Address the recent comments
• Consider this text sufficiently stable?
• If so, Convince more folks to Read and Comment on

10
Status Spatial Composition 05

• Revisions to Outline to keep in step with
  Framework Modifications up 05
• Statement of Conjecture and Assumptions
• Added Assumptions in each of the re-titled
  sections
• NEXT STEPS
• Need Volunteers to read and comment