Experimental designs and Analysis of Variance Discriminant Analysis

1
Experimental designsandAnalysis of
VarianceDiscriminant Analysis
2
Overview

• Multivariate designs
• Significance in k-group MANOVA
• Discriminant Analysis
• Follow-up for MANOVA
• Discriminant functions
• Classification

3
Discriminant Analysis

• DA classification of sampling units into k
  groups
• Two types of analyses (two purposes)
• 1. Descriptive analysis (DDA)
• Describing major differences between k groups
• ? Follow-up of MANOVA
• 2. Predictive analysis (PDA)
• Predicting group membership, i.e., classifying
  cases into k groups
• cf. logistic regression

4
Discriminant Analysis

• Descriptive Discriminant Analysis
• Technically MANOVA and DDA are the same
• MANOVA compare groups (means) based on several
  (dependent) variables
• DA combine several variables to describe
  differences between groups (DDA) or predict group
  membership (PDA)
• The major question in MANOVA is whether group
  membership is associated with statistically
  significant mean differences in combined
  dependent variable scores

5
MANOVA

• Multivariate techniques best honor the reality
  of social science because they assume that human
  behavior has multiple causes and multiple effects
  and that these causes and effects exist
  simultaneously, not mutually exclusively from
  each other (Sherry, 2006)
• MANOVA (one-way, k groups)
• Assume that the dependent variables are
  multivariate normally distributed with (vectors
  of) means µ1, , µk and covariance matrix S
  (constant over all groups)

6
MANOVA

• MANOVA (one-way, k groups)
• Test
• Partitioning of total variance T B W
• Test statistic Wilks lambda
• Approximate F distribution

7
Example Stevens

• Two dependent variables y1 and y2
• Two groups (n1 5 and n2 5)

8
Example Stevens
9
Example Stevens
univariate test for y2
multivariate test
univariate test for y1
10
Discriminant Analysis

• MANOVA
• p dependent variables, k groups
• Test differences between groups on all p
  variables
• Discriminant Analysis
• p dependent variables used to distinguish k
  groups
• Are used as independent variables
• Find the major dimensions on which the groups
  differ
• Using so-called discriminant functions V
• Linear combinations of p variables which best
  distinguish
• the groups

11
Example functions
2 groups 2 variables
12
Discriminant functions

• Discriminant function V
• Total number of functions r min(k 1), p
• Compare k 5 groups on p 6 variables, the
  number of possible discriminant functions equals
  r 4

Example
13
Discriminant functions

• Discriminant functions distinguish the groups
• How do you find them?
• By maximizing the ratio of between and within
  variance
• between covariance matrix B
• within covariance matrix W
• If ratio BW1 is large, the between groups
  variance (explained) is large relative to the
  within groups variance (unexplained)
• Procedure find coefficients a of the
  discriminant functions by maximizing BW1 for
  that function
• Function gives maximum discrimination among
  groups with respect to variances

14
Discriminant functions

• Procedure find coefficients a by maximizing BW1
• It finds that linear combination of the dependent
  variables that maximizes between-group variance
  to within-group variance
• Corresponds to finding eigenvalues of BW1
• Redistribute the variances and covariances in
  matrix BW1, consolidating it into a few
  composite variates rather than many individual
  variables
• First eigenvalue accounts for the largest amount
  of discriminating power (largest ratio explained
  to unexplained variance)
• Second eigenvalue for the second largest amount,
  etc.
• Coefficients of the discriminant functions are
  given by the eigenvectors that correspond to the
  eigenvalues

15
Example functions
3 groups 2 variables
3 groups
smallest amount of discriminating power Worst
function
largest amount of discriminating power Best
function
16
Example Stevens
17
Discriminant functions

• The functions found in this way are uncorrelated
• The functions are independent or orthogonal,
  that is, their contributions to the
  discrimination between groups will not overlap
• Each function accounts for a percentage of the
  between/within variance ratio
• The separation of the groups or discriminating
  power
• This means that the functions break down the
  total (co)variance into additive pieces

18
Discriminant functions

• To summarize DA is used to break down the total
  (co)variance into additive pieces
• Uncorrelated discriminant functions that provide
  for maximum separation on the groups
• By calculating the eigenvalues of BW1 the
  coefficients of the discriminant functions are
  found
• Determine which functions are important by
  comparing the eigenvalues give the percentage of
  between/within variance accounted for by the
  functions
• Significance of the discriminant functions can
  be tested by using Wilks ?

19
Testing discriminant functions

• Peel-off test
• Test procedure with Bartletts Chi-square test
• 1. Test significance of all functions overall ?
  (MANOVA)
• 2. If significant, remove V1, test V2 to Vr
• 3. If significant, remove V2, test V3 to Vr, if
  not conclude only V1 is significant
• 4. If significant, remove V3, test V4 to Vr, if
  not conclude V1 and V2 are significant
• etc

20
Example Stevens
21
Discriminant Analysis

• Result
• Linear combinations of dependent variables that
  separate groups discriminant functions
• How to interpret these functions?
• Use standardized coefficients
• Importance of each variable y in V
• Use structure matrix (correlations between V and
  ys)
• Importance of each variable y in separation of
  groups
• Indication of the nature of the discriminant
  functions

22
Example Lab class nutrition

• Impact of group nutrition education fish diet
• Five dependent variables self-efficacy,
  attitude, social norm, intention and consumption
• Three treatment groups control (132), group
  education (42), group education plus individual
  information (29)

Univariate tests show differences for attitude,
intention and consumption
Multivariate effects? DA ? 2 functions
23
Example Lab class nutrition
24
Example Lab class nutrition
25
Example Lab class nutrition
26
Example Sherry
27
Discriminant Analysis

• Discriminant analysis is MANOVA turned around
• MANOVA given group membership, test differences
  in means of combination of variables y
• DA use combinations of variables y to separate
  groups and predict group membership
• First type of DA description (MANOVA follow-up)
• Second type of DA classification
• Use discriminant functions to classify cases in
  one of the k groups
• Because MANOVA and DA are mathematically the
  same, the same assumptions underlie both
  techniques

28
Classification

• Classifying subjects into one of several groups
  they most resemble based on the set of variables
• Closest in distance to the group centroid
  (Mahalanobis distance Di2) use a decision rule
• Calculate probabilities of group membership
• cf. logistic regression (predicted probabilities,
  classification table)

29
Example Lab class nutrition
30
Example Lab class nutrition