Trade Studies

1
Trade Studies

• Systems Engineering Fundamentals
• PMBOK Guide
• This presentation also uses material from a T.
  Bahill presentation Tradeoff Studies

2
Smart Technical Decisions

• Use quantitative information
• Are justifiable and reproducible
• Respect requirements

3
There are many tools

• Multicriterion decision making is a big field
  with lots of papers, books and tools. We barely
  scratch the surface here.

4
Basics of trade studies

• Define the problem limit scope as much as
  possible
• Define Degrees of Freedom DOF
• Define Figures of Merit FOM
• Define method of combining FOMs for scoring
• Develop relationship between FOM and DOFs
• Choose solution with the best score
• Look hard at this solution. Dont just believe
  the numbers

5
(No Transcript)
6
Some tips for trade studies

• Seek input from anybody who has a stake in the
  outcome
• Approach things open minded(however, experience
  is very important)
• Document your workTwo years from now in a
  product review, somebody is going to ask, What
  Bozo decided did that? If your work is not
  documented, your decision may get overruled by
  somebody else who may not take everything into
  consideration

7
Some details

• Figures of Merit
• Combining functions
• (from Bahill)

8
FoMs should be objective (observer independent)

• Being Pretty or Nice should not be a FoM for
  selecting crewmembers.
• In sports, Most Valuable Player selections are
  often controversial.
• Deriving a consensus for the Best Football Player
  of the Century would be impossible.

9
FoMs should be quantitative

• Each FoM should have a scoring function

10
FoMs should be worded so that more is better

• Instead of using Total Life Cycle Cost, use the
  negative or the reciprocal.
• When using scoring functions make sure more
  output is better.

11
FoMs should be independent

• Human Sex and IQ are independent
• Human Height and Weight are dependent

12
FoMs should show compensation

• From the Systems Engineering literature,
  preference requirements show compensation

13
Perfect compensation

• Astronauts growing food on a trip to Mars
• Two FoMs Amount of Rice Grown and Amount of
  Beans Grown
• Goal maximize total amount of food
• A lot of rice and a few beans is just as good as
  lots of beans and little rice
• We can tradeoff beans for rice

14
No compensation

• A system that produces oxygen and water for our
  astronauts
• A system that produced a huge amount of water,
  but no oxygen might get the highest score, but,
  clearly, it would not support life for long.
• From the Systems Engineering literature,
  mandatory requirements show no compensation

15
Choosing todays lunch

• Candidate meals pizza, hamburger, fish and
  chips, chicken sandwich, beer, tacos, bread and
  water
• FoMs Cost, Preparation Time, Tastiness, Novelty,
  Low Fat, Contains the Four Basic Food Groups,
  Complements Merlot Wine
• These FoMs are independent and also show
  compensation
• FoMs are usually nouns, noun phrases or verb
  phrases

16
Sometimes it is hard to get both independence and
compensation

• If two FoMs are independent, they might not show
  compensation.
• If they show compensation, they might not be
  independent.
• Independence is more important for mandatory
  requirements.
• Compensation is more important for preference
  requirements.

17
FoM hierarchy

• The FoM tree should be hierarchical
• The top level often contains
• Performance
• Cost
• Schedule
• Risk
• Dependent entries are grouped into subcategories
• The FOM set should cover the domain evenly
• Relate to requirements

18
(No Transcript)
19
Nomenclature

• Real-world data will not fall neatly onto lines
  such as this circle. But often they may be
  bounded by such functions. In the operations
  research literature such data sets are called
  convex, although the function bounding them is
  called concave (Kuhn and Tucker, 1951).

20
(No Transcript)
21
(No Transcript)
22
Candidate combining functions

• Sum Combining Function x y
• Used most often by engineers
• Product Combining Function x ??y
• Cost to benefit ratio
• Risk analyses
• Sum Minus Product x y - xy
• Probability theory
• Fuzzy logic systems
• Expert system certainty factors
• Compromise

23
The American public accepts the Sum Combining
Function

• It is used to rate NFL quarterbacks.
• It is used to select the best college football
  teams.

24
NFL quarterback passer ratings

• FoM1(Completed Passes) / (Pass Attempts)
• FoM2(Passing Yards) / (Pass Attempts)
• FoM3(Touchdown Passes) / (Pass Attempts)
• FoM4Interceptions / (Pass Attempts)
• Rating5(FoM1-0.3) 0.25(FoM2-3) 20(FoM3)
  25(-FoM40.095)100/6

25
When should p be 1, 2 or ??

• Use p 1 if the figures of merit show perfect
  compensation.
• Use p 2 if you want Euclidean distance.
• Use p ? if you are selecting a hero and there
  is no compensation.
• Compromise

26
Use p? when choosing

• a water treatment plant to reduce the amount of
  mercury, lead and arsenic in the water.
• Trace amounts are not of concern.
• First, find the poison with the maximum
  concentration, then choose the alternative with
  the minimum amount of that poison.
• Hence the term minimax.

27
Design of a baseball bat

• The ball goes the farthest, if it hits the sweet
  spot of the bat.
• Error sweet spot - hit point
• Loss number of feet short of 500
• For an amateur use minimax minimize the Loss, if
  the Error is maximum.
• For Mark McGuire use minimin.

28
Distance the ball travels depends on where the
ball hits the bat
29
Scoring functions

• FoMs should always have scoring functions so that
  the preferred alternatives do not depend on the
  units used.

30
Scoring function for Cost
31
Scoring function for Quantity
32
Using weights

• For the Sum Combining Function
• For the Product Combining Function the weights
  can be put in the exponent.

33
Deriving weights

• Decision maker assigns numbers between 1 and 10
  to criteria
• Decision maker rank orders the criteria
• Decision maker makes pair-wise comparisons of
  criteria (AHP)

34
The do nothing alternatives

• (1) the status quo
• (2) do absolutely nothing
• Replacing a Datsun 240Z
• Status quo, keep the 240Z
• Do without a car, i.e., walk or take the bus

35
If Do Nothing wins,

• you have too many Cost or Risk FoMs and not
  enough Performance FoMs.
• You should have a similar number of Performance,
  Cost, Schedule and Risk FoMs.

36
If a do nothing alternative wins

• Just as you should not add apples and oranges,
  you should not combine Performance, Cost,
  Schedule and Risk FoMs with each other.
• combine the Performance FoMs (with the weights
  normalized so that they add up to one),
• combine the Cost FoMs,
• combine the Schedule FoMs, and
• combine the Risk FoMs.
• Then the Performance, Cost, Schedule and Risk
  combinations can be combined with clearly stated
  weights, 1/4, 1/4, 1/4 and 1/4 could be the
  default.

37
When you get the wrong answer you could change

• importance weights,
• scores of the FoMs,
• parameters of the scoring functions,
• parameters of the combining function,
• the combining function itself, or
• the tradeoff method.
• Have you captured all of the requirements?