Statistics for Product Engineering Part 2

1
Statistics for Product EngineeringPart 2

• Elida de Obaldia, PhD
• Texas Instruments
• March 8, 2008

2
About the SpeakerDr. Elida de Obaldia
BS in Physics with minor in Education - LSU
1982 Master of Art in Teaching (MAT)
- BU 1987 Master of Art in Physics (MA)
- BU 1989 PhD in Applied Physics
- BU 1994
Product Engineer Manager at Texas instruments.
1997 - Present Work load has include product
Engineering, Test Engineering, Characterization
Manager, Design for Test. ? Holds 4 patents in
Self Build RF test. Bell Labs (ATT) Post Doc
position in Research. 1994 1997 ?
Application of superconductivity in the RF
field. Physics Instructor (Panama). 1983 -
1985
3
Agenda Part 1 - Mike

• Introduction
• Role of the product Eng.
• What is a Engineer Spec
• How do we guarantee a spec
• Direct measurements
• By characterization
• By design
• Why do we need Statistics and Data Analysis
• Fundamental Statistical
• Population vs. Sample
• Histogram
• Measures of central tendency
• Measures of spread
• Normal distribution and empirical rule
• Z-score
• Central limit theorem

4
Agenda Part 1(cont) - Elida

• Process Capability, Yield and Limits
• Process capability definition
• Yield definition
• Outlier definition
• CP CPK
• Relation between CPK and Yield.
• Spec compliance Matrix
• Gauge Repeatability and Reproducibility
• Total variation vs. measurement variation
• Accuracy
• Repeatability and reproducibility
• RR calculation method
• Test limits vs. Spec limits
• Put it all together
• Spec limits vs temperature
• Other statistic representations Box plot,
  Pareto
• Eng Report

5
Agenda Part II - Tools

• Excel
• R Statistical process
• Tools use in TI
• SAS
• Data power
• Spot fire

6
Why companies spend a lot of on test.

• Test cost ? 2 to 10 of each device revenue.
• Data Sheet is a contract between the Supplier and
  the Customer. Click here.
• This is not just a nice thing to do but a
  company requirement for IS09001.
• The task of a product Engineer is to determine if
  a particular device meets the device
  specification (Data Sheet), can be shipped to
  customers and it will meet the same
  specifications at the customers measuring system.
• To achieve this, we measure a set of parameters
  before shipping to customers.
• For some manufactures this could be a few
  devices a day to millions of devices a day.
  My team averages about 100K units per day.
• How good is my measuring system? Is it accurate?
  How do we secure that we take into account the
  measuring system variations?. Gauge
  Reproducibility and Repeatability helps answer
  some of these question.

7
ISO9001, Paragraph 4.11Regarding Measurement
Uncertainty

• ISO9001, Para. 4.11
• Equipment shall be used in a manner which
  ensures that
• measurement uncertainty is known and is
  consistent with the
• required measurement capability.
• This requires two things regarding the
  measurement uncertainty
• 1. It Is Known.
• 2. It Is Consistent With The Required
  Measurement Capability.

8
ISO9001 Measurement Requirement 1 Measurement
Uncertainty Is Known

• There are two main types of analyses that can be
  performed to determine the measurement
  uncertainty
• 1. A Propagation-of-Errors (POE) or Worst Case
  (WC) Error Analysis on The Test Circuit Design.
• A Test Capability (TC) or Gage RR Study on The
  Test System. (Gage RR Gage Repeatability
  Reproducibility )
• In this lecture we will only concentrate on GRR

9
ISO9001 Measurement Requirement 2Measurement
Uncertainty Is ConsistentWith The Required
Measurement Capability

• The required measurement capability requires that
  the product specification be met.
• Test limits will need to be set narrower than the
  product tolerance according to the amount of
  measurement uncertainty.
• To minimize yield loss due to the narrower test
  limits, make the measurement uncertainty as small
  as possible.

10
Introduction Purpose

• Increase understanding of
• Basic statistics as applied to test engineering
• Test capability studies to estimate test
  uncertainty
• Test guardbands to ensure product quality
• Monitoring controlling test quality
• Methods of improving test capability test
  designs

11
Introduction Motivation

• Tighter tolerances require less measurement
  uncertainty
• Consistent yield and reduced test cost
• Improve outgoing quality.
• Huge opportunity for reducing manufacturing
  costs
• Consistent test capability studies and test
  guardbands
• Proper guardbands are critical for correlation
  lockout.

Continued
12
Inaccurate Test System

• An Inaccurate Test System Can
• Pass Bad Product Which Should Be Rejected,
• Reject Good Product Which Should Be Accepted
• Negatively Impact Worker Productivity
• Physically Cause The Product To Be Worse

Continued
13
Accuracy and Precision Definitions
Good accuracy is associated with readings being
centered on the true value (case B or D). Also
associated with low systematic errors. Good
precision is associated with readings being
grouped closely together (case A or D). Also
associated with low repeatability errors.
14
Inaccurate Test System
Continued
15
Inaccurate Test System
Continued
16
Inaccurate Test System
17
Accuracy

• Suppose we have a 1cm Golden Standard and we
  want to measure it. We do this by taken 20
  measurements of the standard which resulted on x
  1.00403 cm and s0.00024 cm.
• Gauge Accuracy
• Accuracy True value Observe average
• Accuracy Accuracy X 100.
• True Standard

Accuracy comes into play when we are
calibrating our measuring system. In RF this
is one of the most important steps. Lets
concentrate in the measurement variation.
18
How do achieve accuracy

• Test system calibration
• ISO9001 requires that all the equipment use in
  manufacturing (which includes all the testing
  equipment) be calibrated at least once a year.
• TI performs weekly routine calibrations on all
  Test Equipment. (does not include the load
  board)
• Offset Generations
• Based on a set of golden units - units which
  true value are known Test Engineers develop a
  set of parametric offset after measuring a set of
  parameters and compare them to the true value
• Site to Site Compensation
• For multiple site test, after the system is
  calibrated, each site is then re-calibrated to
  the best site. This is typical done for each
  new lot.

19

User Developed Interfaces Requires MoreThan Just
Calibration of Core Instruments

• Mere calibration of the core instruments of a
  measurement system may not be sufficient to
  ensure the complete calibration of the
  measurement system. The core instruments may not
  be the only elements that can affect
  measurements.
• User developed interfaces (family boards, DUT
  adapters, gain setting resistors, etc.) can
  significantly affect the measurement uncertainty
  of the measurement process.

20
Accuracy and Precision Error Terms
Continued
21
Total Measurement Variation
Measuring system 1
Measuring system 2
s total2 sdev 2 s mea 2
smeas
smeas
Measurement Uncertainty
22
Measurement Uncertainty Definition

• Assuming the systematic error is independent of
  the repeatability error, then Measurement
  Uncertainty is defined as,

Measurement Uncertainty
where
Total Measurement Error Standard Deviation,
Systematic Error Standard Deviation, and
Repeatability Error Standard Deviation.

Continued
23
Measurement System

• C Rule-of-Thumb The uncertainty of a
  measurement system should
• be no more than 10 of the device's
  tolerance (spec range).

24
Measurement Uncertainty Definition

• Assuming a normal distribution (and a sampling
  over several test systems) there will be a 99.7
  chance that a measured value will take on a
  value in the interval "True Value Uncertainty".
  That is,
• Measured Value True Value
  Uncertainty
• The uncertainty is defined as the 3? value of
  the measurement error which involves both the
  variance of the systematic error and the variance
  of the repeatability error.

Continued
25
Gauge Repeatability and Reproducibility

• Gauge RR measures the amount of variability
  induced in measurements that comes from the
  measurement system itself and compares it to the
  total variability observed to determine the
  viability of the measurement system.
• There are five components affecting a measurement
  system
• Measurement devices (machine), the gauge itself
  and all mounting blocks, supports, fixtures, load
  cells etc. The machine ease of use, sloppiness
  among mating parts, "zero" blocks are examples of
  sources of variation in the measurement system
  
• Operators (people), the ability and/or discipline
  of a person to follow the written or verbal
  instructions.
• Measurement instructions (method), how to setup
  your devices, how to mount your parts, how to
  record the data, etc.
• Specification, the measurement is reported
  against a specification or a reference value. The
  range of the specification does not affect the
  measurement, but is an important factor affecting
  the viability of the measurement system.
• Parts (what is being measured), some parts are
  easier to measure than others. A measurement
  system may be good for measuring block length but
  not for measuring rubber pieces.

26
Repeatability
Repeatiability
Gauge Repeatability, the ability of the device to
provide consistent results. It is a measure of
the variation of a measurement sytem obtained by
repeating measurements on the sample back-to-back
using the same measurement system. Same
operator takes measures the same device multiple
times following the same procedure every time
27
Repeatability
If the same device is measured three times, on
the same test system, then the measurements would
be modeled as follows

• T does not change because the same device is
  being measured.
• ES does not change because the same test system
  is being used.
• The repeatability error ER, does change for each
  measurement.

28
Repeatability Error

• Repeatability Error, ER
• Does not have any consistent effects across the
  entire sample.
• The repeatability errors sum to 0 there would
  be as many negative errors as positive ones
  (relative to the average measurement of the group
  of data taken).
• Adds variability to the data but does not affect
  the average performance of the group.

29
Reproducibility
Gauge Reproducibility is the variation among the
averages of measurements made at different
measurements conditions, i.e. different
operators, testers, sites, load boards, etc.
30
Error Distributions
31
Reproducibility Error
Can Systematic Error/ Reproducibility be
Subtracted Out In A Test System?

• Yes and No. Systematic error can be due to
  recalibration, a different test socket, different
  family or interface boards, etc.
• Each time a test system is set up the systematic
  error will be different.
• Trying to subtract out the real systematic error
  is often impractical. However for RF test
  system the attempt is made by generating offsets
  with a set of golden units per test system.
• The best way to reduce the systematic error is to
  improve the test design.

32
Measurement Uncertainty Illustration
33
Setting up a measurement system study

• Samples should be selected to cover the full
  operating range to assess the measurement
  variation across the whole range of processes.
• The measurements conditions should be selected to
  cover variations across the whole operating
  range.
• Common industrial practices is to select 2 to 5
  readings per samples.
• Randomize among samples and measurements
  conditions to minimized systematic errors.
• Needless to say that careful considerations
  should be given to the system test plan What are
  the factor that most affect the measurements.

34
Measurement System RR AnalysisThe Average and
Range Method

• The average and range method is mathematical
  method that uses range (R) to estimate both
  repeatability and reproducibility for a
  measurement system.
• Note The number of samples times the numbers of
  conditions must be at least 15 for this method.
• Note Another similar method is the Average and
  Standard deviation method.

35
GRR Example 1
36
GRR Example 1 (Cont)
Few observations - The device Specifications
play an important role. Its range determines
the of RR. - Since the goal is to ship
good parts then I must consider the part
variations as part of my system.
37
Measurement Cp

• Measurement CP is defined as
• Where
• USL and LSL are the upper and lower
  specification limits
• EV is the standard deviation for repeatability
• SR is the standard deviation for reproducibility
• The capability index, Cp, is calculated as the
  ratio of the spec tolerance to six sigmas of the
  process been evaluated. Cp of the measurement
  should be much higher than the Cp for the whole
  process

38
Precision to Tolerance Ratio
The average and range method calculates GRR.
GRR represents the percentage of the spec that
is taken by measurement system variation
(uncertainty) and it is the reciprocal of Cp
(measurement)
Measurement System Ratings
39
Test Capability StudyMeasurement Error ( of
Spec Range)

• Measurement Error As Percent of
  Specification Range

For clear reporting it is helpful to express the
Measurement Error as a percent of the
specification range. Notice that, as defined
here, the measurement error is the region
consumed by the measurement uncertainty relative
to the specification range.
40
Put it all together

• GRR
• Temperature Guard-banding

41
Characterization Review
Single parameter in DS that needs to be guarantee
on temperature.
42
Characterization (cont)
43
Input Documents
Assumptions

• Characterization Review
• Current CPK per temperature
• Temp distribution
• Data Sheet
• Determine parameters that are guarantee in the DS
• GRR data
• 2 boards 2 testers
• Data Log of current program

• Data Sheet parameters spec must be guard banded
  inside the spec limits, regardless of
  distribution. These parameters are guard banded
  based on CZ (temp) and GRR.
• Non-Data Sheet parameters are guard banded based
  on CZ results and GRR results so that the limits
  are set to catch outliers outside CZ
  distribution.

44
Data Sheet Parameters
Temp guard band ? 0.3dB GRR ? 0.2dB Limit Spec
TGB GRR
45
Other Parameters

• Check distribution
• Set limits to get gt 1.3 CPK ? LL and UP
• Production limits are
• LL GRR
• UP GRR
• Exceptions
• Parameters that are not in the data sheet but
  affect the RF performance
• VREF ? Spec limits determine by design and CZ
• IREF ? Spec limits determine by design and CZ

46
Tools
47
Excel
Excel has an statistical package
48
Excel

• Pivot Tables

49
R Statistics

• http//www.r-project.org/

50
R - Examples
file///C/Program20Files/R/R-2.4.0/doc/manual/R-
intro.htmlA-sample-session
An Introduction to R R (GNU S) is a language
and environment for statistical computing and
graphics. R is similar to the award-winning S
system, which was developed at Bell Laboratories
by John Chambers et al. It provides a wide
variety of statistical and graphical techniques
(linear and nonlinear modelling, statistical
tests, time series analysis, classification,
clustering, ...).
Its free an very versatile
51
Data power/ Spot Fire/ SAS

• Requires license
• Training classes at TI
• Upload data from manufacture sites
• Test floors in Taiwan, Philippines, Dallas Eng,
  Dallas production, etc
• Specific functions (scripts) develop for Tiers to
  make life simpler
• Example GRR calculations in Data Power
• Process Sensitivity Analysis
  in Spotfire
• Spec Compliance Matrix auto
  generation in SAS Flow
• Data merge per TI DIE ID in SAS flow.
• etc..

52
Data Power
53
Back Up
54
Example of Data Sheet

• Integrated BT device for Texas Instruments.
• DS consists of 76 pages
• Shows Pin out, how to connect each pin.
• Overall device architecture
• Specific input requirements
• IO level
• Reference Clock specs
• Interface Protocol
• Overall operation range
• Specifications and Electrical Characteristics.
• There are over 100 parameters in the data
  sheet.

55
Example of Specifications

• Back

Disclaimer
56
Sample Variance Standard Deviation
Sample Variance Sample Standard Deviation
(Unbiased) For theoretical reasons the sample
variance and sample standard deviation are
defined as
and
Dividing the summation by N-1 instead of N causes
the variance estimate to have the property of
being unbiased. In statistical terms for s2
to be unbiased it must have the property that
E(s2) ? 2, where s2 sample variance and ? 2
(theoretical) population variance. The E( )
operation is equivalent to a theoretical
averaging operation.
57
Some Properties of The Variance
Definitions properties of the Variance and
Standard Deviation
Variance Sample Variance Standard
Deviation Sample Std Dev
58
Std. Error of the Mean Illustration
59
Std. Error of the Mean Illustration