Metrics

1
Metrics

• The Dark Side of Management Part 1

Victor W. Lowe, Jr. Fellow, American Society for
Quality ASQ Ft Worth Section Meeting 6
September 2007
2
Purpose
Thought is the father to the deed Proverb
"A beginning is the time for taking the most
delicate care that the balances are
correct." Frank Herbert, Dune, 1965
Knowing There is a Trap is the First Step in
Evading It Duke Leto Atreides, ibid
Purpose Identify common management traps
associated with the way key decision makers often
think about metrics
3

• Whats Ahead
• Metaphor
• Parable
• Managers create problems for themselves by not
  understanding how even simple metrics behave.
  Examples include the metrics used for
• goal setting
• comparisons / rankings
• performance evaluations
• learning
• politics
• policy

• Managing Knowledge
• An informal, irreverent handbook of the most
  important knowledge managers need to know (in
  manuscript)
• Thesis management is/should be primarily a
  thinking/knowledge-based activity
• What should management think about?
• To make their organizations better, managers must
  learn to think better (Demings thesis)
• Einstein The problems we face today cannot be
  solved at the same level of thought we were at
  when we created them.

• Demings System of Profound Knowledge
• Theory of Knowledge
• Theory of Variation
• Theory of a System
• Theory of Psychology

4
Key Ideas to Keep in Mind about the Examples

• Based on arithmetic so simple that it can be done
  without a calculator or pencil and paper
• Illuminate profound problems with some management
  thought processes concerning metrics
• Generalize across a full-spectrum of
  misapplication of metrics

5
Metaphor Fuel GaugeWhat performance
characteristics must a fuel gauge (metric)
possess before it is useful?

• Least important Cost, ease of use, stability,
  reliability, etc.?
• Most important metric and measured reality must
  vary together

6
The Parable of Company F Percent Defectives
Company F
8.3 40/480
Last Year
Division A
Division B
Division C
Division D
8.3 10/120
8.3 10/120
8.3 10/120
8.3 10/120
Last Year
Plant 1
Plant 2
Plant 3
Plant 4
Plant 5
Plant 6
Plant 7
Plant 8
25 5/20
5 5/100
25 5/20
5 5/100
25 5/20
5 5/100
25 5/20
5 5/100
Last Year
7
Management Cascades a New Quality Strategy
8
We will ignore gauging problems such as
variation, sampling bias, etc.
Evaluation
Evaluation
Last Year
This Year
Evaluation
9
The Parable of Company F Revisited The Learning
Experience
Company F
The same problem reject rate too high The same
objectives cascaded
Division Managers mandate that the plants use the
managers favorite improvement tool
8.3
Last Year
Division A
Division B
Division C
Division D
8.3
8.3
8.3
8.3
Last Year
Plant 1
Plant 2
Plant 3
Plant 4
Plant 5
Plant 6
Plant 7
Plant 8
25
5
25
5
25
5
25
5
Last Year
10
Company F
What lessons will the Division Managers take away
for SPC, Lean, ISO, Six Sigma?
What lessons should the Division Managers take
away for SPC, Lean, ISO, Six Sigma?
8.3
Evaluation
Division A
Division B
Division C
Division D
8.3
8.3
8.3
8.3
Evaluation
Plant 1
Plant 2
Plant 3
Plant 4
Plant 5
Plant 6
Plant 7
Plant 8
25
5
25
5
25
5
25
5
Last Year
This Year
11
Fuel Gauge Metaphor Revisited
The usual method of cascading objectives is
equivalent to (unintentionally) designing and
using a faulty gauge
12
Take-Aways from the Parable

• About Systems
• Systems (even simple systems) can exhibit counter
  intuitive behavior
• The whole / aggregated system is not the simple
  sum of its parts
• Making the parts better does not necessarily make
  the whole better
• Making the parts worse does not necessarily make
  the whole worse.
• Failure to understand system behavior can lead to
  disastrous results
• May need new terms / metrics for thinking about
  systems (Hint ponder connection between
  functionals and optical illusions, ala certain
  works of Escher)

• About Other Topics
• Six-Sigma, ISO, Lean, etc. do not generally
  address the problem of bad management direction
• By failing to address the general class of
  problems (of which this is but one simple
  example), management creates many of its own
  problems

13
Federal Income Tax Policy

• Questions to Think About
• Is it more accurate to say the tax rate went up
  or went down?
• How might politicians in our two major political
  parties characterize the change in the tax rate
  for political purposes?
• When would such political statements concerning
  tax rates become spin and when would they
  become outright deceptions?
• As a taxpayer, exactly what information would you
  need to be well informed

14
Comparing Death Rates in Oncology Wards of Two
Hospitals

• Questions to Think About
• Which hospital has the better record?
• If you had Cancer 2, would you rather be treated
  in Hospital A or B?
• If you knew you had cancer but did not know which
  type of cancer it was, which hospital would you
  rather go to?
• Suppose a patients rights group wanted to
  provide the public with a ranking of hospitals.
  How might they fairly rank these two hospitals so
  that the public would know which hospital was the
  better of the two?

15
Company Fs Electronic Division Needed to Build
5 New Greenfield Plants
16
Company Fs Canadian Plant Before the Edict
Outputs
17
Same Plant After Edict Went Into Effect
Outputs
18
The SolutionAdd a Stand-by Machine
Outputs
Goldratts Second Principle Overall plant
throughput is important component throughput is
not
19
Take-Aways Company Fs Electronic Division
Example

• Policy of Company F, as articulated by its
  Finance Manual, specifically Prohibited the step
  that doubled the plant throughput)
• The Finance Manual dictated that all equipment
  must have a certain minimum level of utilization.
• Company President, deliberately and knowingly,
  created two sets of accounting records
• The official set with false data that satisfied
  the Finance Manual requirements
• The real data for what was going on
• He created an invisible company outside of
  upper managements control
• He cheerfully admitted everything after the
  experiment turned out to be a success
• Major implication Value of a component to the
  system is not determined by its utilization rate
• Grading and ranking of components / team members
  / employees

20
Summary and Other Examples

• Summary
• Behavior of the system is not necessarily the
  same as the behavior of the components
• Improving components may degrade system
  degrading components may improve system
• Well-managed components do not necessarily add up
  to a well managed system
• Failure to understand the behavior of a system
  can lead to catastrophic consequences (i.e. a
  company policy that reduces plant throughput by
  half)

• Other Examples
• Setting targets for reducing cost and weight (a
  la Ford 2000), defects, head count, warranty
  (repairs per 1000), ROI, etc.
• Strategy of wanting every worker fully employed
  (no one available for walk-in customers,
  problems, etc.)
• Using best practices everywhere in the
  organization
• Force Effectiveness calculations
• Possible to win every engagement, yet lose the
  war, and conversely

21
Knowing there is a trap is the first step in
evading it Duke Leto Atreides Dune
Managers must learn to think and plan how to use
metrics better
There is no substitute for knowledge W. Edwards
Deming
22

• Return To Presentation Summary