Sample analysis using the ICCS data An application of HLM

1
Sample analysis using the ICCS data An
application of HLM

• Daniel Caro
• November 25

2
Purpose

• Illustrate the use of hierarchical linear models
  (HLM) with ICCS 2009 data through the evaluation
  of specific hypotheses

3
Table of contents

• HLM theory
• Applied research example
• HLM data importing/estimation settings
• Hypothesis testing

4
Data structure

• Often participants of studies are nested within
  specific contexts
• Patients treated in hospitals
• Firms operate within countries
• Families live in neighborhoods
• Students learn in classes within schools
• Data stemming from such research designs have a
  multilevel or hierarchical structure

5
Implications of research design

• Observations are not independent within
  classes/schools
• Students within schools tend to share similar
  characteristics (e.g., socioeconomic background
  and instructional setting)
• Traditional linear regression (OLS) assumes
• Correlation (ei,ej)0, i.e., the ? between
  observed and predicted Y are uncorrelated
• Ignoring dependence of observations may lead to
  wrong conclusions

6
Intra-class correlation coefficient

• The intra-class correlation coefficient (ICC)
  measures the degree of data dependence
• It is equal to the proportion of the variance
  between schools, i.e., ICC s2 b / (s2 b s2 w)
  
• where s2 b is the variance between schools and s2
  w the variance within schools or between students
• If ICC 0, responses of students within schools
  are uncorrelated
• Si ICC 1, responses within schools are identical

7
Effective sample size

• A higher ICC value indicates greater dependence
  among observations within schools
• Effective sample size is smaller than observed
  sample size
• Effective n mk / (1 ICC(m-1))
• where nsample size, m number of students per
  schools and k number of schools
• If ICC1, effective n is equal to the of
  schools (k)
• If ICC0, effective n is equal to the observed n
  (i.e., mk)
• In general, effective n lies between k and mk

8
Limitations of OLS

• OLS neglects ICC and considers standard errors
  based on observed n
• But effective n is smaller than observed n when
  observations are correlated
• Standard error is inversely proportional to n
• Thus, OLS tends to underestimate the standard
  error
• Underestimated standard errors can lead to
  incorrect significance tests and inferences
• The JRR method produces correct standard errors
  under a multilevel research design

9
Hierarchical linear models

• Additionally, hierarchical linear models
  distinguish effects between and within
  clusters/schools
• For example, they enable evaluating
• The effect of SES on student achievement within
  schools and between schools
• The effect of school location (urban/rural) on
  the average achievement between schools

10
Hierarchical linear models

• Account explicitly for the multilevel nature of
  the data with the introduction of random effects
• Consider ICC for calculation of standard errors,
  tests, and p-values
• Decompose variance within and between schools
• Student level variables explain variance within
  schools or between students
• School level variables explain variance between
  schools
• A single R-squared cannot be reported
• Instead, there is one for each level

11
Hierarchical linear models

• Estimate regressions within schools
• Provide estimates of the intercept and
  coefficients (e.g., gender gap, SES effect) for
  each school
• Level 1 (students) coefficients may depend on
  level 2 (schools) characteristics as if they were
  dependent variables
• For example, the gender gap at the student level
  (i.e., gender coefficient) may vary between
  classes for the gender of the class teacher at
  level 2

12
Table of contents

• HLM theory
• Applied research example
• HLM data importing/estimation settings
• Hypothesis testing

13
Research goal

• Evaluate 10 hypotheses related to the attitudes
  of students towards equal rights for immigrants
• The literature underscores the importance of
• Family SES, participation in diverse networks,
  intergroup discussion about civic issues, gender,
  social dominance orientation, civic knowledge,
  religion beliefs, the school location
  (urban/rural), the school climate
• References in C\ICCS2009\HLM training\References
  .pdf
• For each hypothesis
• Theory and independent variables

14
Related data and variables

• Selected country
• England
• The analysis is restricted to international
  scales/variables
• A description of the dependent and independent
  variables, their type, coding scheme, and source
  is in
• C\ICCS2009\HLM training\List of variables.pdf
• The student (england1.sav) and class level
  (england2.sav) datasets are in
• C\ICCS2009\HLM training\Data

15
Data structure

• Students (level 1 units) are nested in classes
  (level 2 units)
• The ICCS sample design yields an optimal sample
  of students within classes, and not optimal
  sample of students within schools
• Usually one class was selected within each
  school, rather than students across different
  grades

16
NOTE

• This is a didactic example only. You will not be
  able to readily repeat this analysis during the
  presentation

17
Table of contents

• HLM theory
• Applied research example
• HLM data importing/estimation settings
• Hypothesis testing

18
HLM software

• HLM estimates different type of hierarchical
  linear models
• The applied example is for two-level models
  (student nested in classes)
• Several steps are required to estimate a model
• Creating data specifications file (.mdmt)
• Importing data to HLM (.mdm)
• Deciding on settings (e.g., weights, plausible
  values)
• Specifying model (.hlm)
• Estimating model

19
Beginning with HLM
20
Data specifications (.mdmt)
21
Selecting student level data
22
Missing data

• HLM accepts multiply imputed datasets
• Multiple imputation (MI) procedure is performed
  in another software
• Consult NORM, PAN, MICE in Stata and R, for
  example
• Since missing data are normally not completely at
  random, it is recommended to conduct MI before
  model estimation
• But for this example we will use available data,
  only
• HLM offers two options at level 1
• Listwise deletion (making mdm) Sample is the
  same for all models
• Pairwise deletion (running analysis) Sample
  depends on included variables
• Missings at level 2 reduce substantially the
  sample size

23
Selecting class level data
24
Save data specifications (.mdmt)
25
Create data file (.mdm)
26
Check stats
27
Add dependent variable
28
Declare weights
29
Save null model
30
Run null model
31
View output
32
Interpret and save
Folder C\ICCS2009\HLM training\Models\model0.tx
t
Class variance12.14 Student variance103.99 ICC
12.14/(12.14103.99)0.11 11 of differences
occur between classes
33
Table of contents

• HLM theory
• Applied research example
• HLM data importing/estimation settings
• Hypothesis testing

34
Hypotheses

1. The SES Hypothesis
2. The Contact Hypothesis
3. The Intergroup Discussion Hypothesis
4. The Gender Hypothesis
5. The Social Dominance Orientation Hypothesis
6. The Learning Hypothesis
7. The Religion Belief Hypothesis
8. The National Identity Hypothesis
9. The Urban/Rural Differences Hypothesis
10. The School Climate Hypothesis

35
The SES Hypothesis

• The SES hypothesis predicts more positive views
  of minorities among students of higher SES
  families than among students of lower SES
  families
• Competition among low SESs
• High SESs travel and confront culturally diverse
  realities
• Independent variables
• Parental education (HISCED)
• Parental occupational status (HISEI)

36
The SES Hypothesis
37
Centering of Xs

• The intercept is the expected value of Y when Xs
  are zero
• E(Y(Xs0))E(ß0j)ß1j0 ß2j0 ßkj0 E(rij)
• Since E(rij) and E(uoj) are zero gt g00Y(Xs0)
• But sometimes zero is not in the range of Xs
• If X is age, achievement score, etc.
• Here, the intercept is not interpretable
• By centering the Xs, the intercept can be
  interpreted as the expected value of Y at the
  centering value(s) of Xs

38
Centering of Xs

• Two options at level 1
• Grand and group (class) mean centering
• The type of centering depends on the research
  interest (Enders Tofighi, 2007 Raudenbush
  Bryk, 2002)
• Group mean centering is appropriate for
  unadjusted or pure within and between school
  effects
• Grand mean centering yields school effects
  adjusted for student characteristics and is
  preferable for contextual effects

39
The SES Hypothesis
40
The SES Hypothesis

• The hypothesis is supported by the parental
  education data
• Effect size? (see stats and model estimates)
• For a 1 SD increment in HISCED, IMMRGHT increases
  in 0.67 (1.040.64), that is, about 6 percent
  (0.67/10.75) of a SD in IMMRGHT

41
The Contact Hypothesis

• The contact hypothesis anticipates greater
  tolerance among students participating in
  diversified and extended social networks
  (Allport, 1954 Cote Erikson, 2009)
• Independent variables
• Students' civic participation in the wider
  community (PARTCOM)
• Students' civic participation at school
  (PARTSCHL)
• Control for SES
• Higher SES have more diversified social networks
  (Erickson, 2004) and are more active in voluntary
  associations (Curtis Grabb, 1992)

42
The Contact Hypothesis
43
The Contact Hypothesis

• The hypothesis holds in England
• Both students' civic participation in the wider
  community (PARTCOM) and students' civic
  participation at school (PARTSCHL) are positively
  related to the attitudes toward immigrants
• For a 1 SD increment in the independent
  variables, the associated positive change in
  IMMRGHT amounts to
• 7 percent of SD in IMMRGHT for PARTCOM
• 11 percent of SD in IMMRGHT for PARTSCHL

44
The Intergroup Discussion Hypothesis

• The intergroup discussion hypothesis posits that
  more positive attitudes toward minorities develop
  from dialogue on social and civic issues inside
  and outside the school (Dessel, 2010a)
• Independent variables
• Students' discussion of political and social
  issues outside of school (POLDISC)
• Student perceptions of openness in classroom
  discussions (OPDISC)
• Control variables
• Parental education (HISCED)

45
The Intergroup Discussion Hypothesis
46
The Intergroup Discussion Hypothesis

• The hypothesis is validated by the data
• Both students' discussion of political and social
  issues outside of school (POLDISC) and student
  perceptions of openness in classroom discussions
  (OPDISC) are positively related to IMMRGHT
• For a 1 SD increment in the independent
  variables, the associated positive change in
  IMMRGHT amounts to
• 9 percent of SD in IMMRGHT for POLDISC
• 18 percent of SD in IMMRGHT for OPDISC

47
The Gender Hypothesis

• The gender hypothesis predicts greater tolerance
  among girls than boys. Women tend to be more
  liberal, nurturing and social than men and are
  also expected to be more tolerant (Cote
  Erikson, 2009 Gidengil, Blais, Nadeau,
  Nevitte, 2003)
• Independent variable
• The students sex (GIRL)

48
The Gender Hypothesis
49
The Gender Hypothesis

• The gender hypothesis holds in England
• Differences between girls and boys amount to 2.24
  score points in the IMMRGHT scale, that is, 21
  percent of a SD in IMMRGHT

50
The Social Dominance Orientation Hypothesis

• The social dominance orientation (SDO) hypothesis
  states that gender differences are partly
  explained by a differences in support for social
  inequality (Mata, Ghavami, Wittig, 2010).
• Independent variables
• Female (GIRL)
• Students' support for democratic values (DEMVAL)
• Students' attitudes towards gender equality
  (GENEQL)
• Students' attitudes towards equal rights for all
  ethnic/racial groups (ETHRGHT)

51
The Social Dominance Orientation Hypothesis
52
The Social Dominance Orientation Hypothesis

• The hypothesis is supported by the data
• When proxies for social dominance orientation are
  included, gender differences are no longer
  significant

53
The Learning Hypothesis

• The learning hypothesis predicts greater
  tolerance when individuals know more about
  minorities and civic issues in general (Cote
  Erikson, 2009)
• Independent variables
• Civic knowledge (PV1CIV)
• Control for participation (Curtis Grabb, 1992
  Erickson, 2004)
• Students' civic participation in the wider
  community (PARTCOM)
• Students' civic participation at school
  (PARTSCHL)

54
The Learning Hypothesis
55
The Learning Hypothesis

• The learning hypothesis holds in England
• Students showing higher knowledge in civic issues
  also have more positive attitudes toward
  immigrants even when civic participation is
  controlled
• A 1 SD increment in PV1CIV is associated with a
  positive increase in IMMRGHT of about 22 percent
  of a SD

56
The Religion Belief Hypothesis

• The religion belief hypothesis anticipates an
  association between holding religious beliefs and
  tolerance toward minorities (Hall, Matz, Wood,
  2010 Schwartz Huismans, 1995). The direction
  of the association is not clear
• Negative for values of social conformity,
  tradition, conventionalism, and an authoritarian
  belief system
• Positive for humanitarianism, values of
  benevolence toward others, and a search for
  spiritual meaning
• Independent variables
• Students' belonging to a religion (RELIG),
• Students' attitudes towards the influence of
  religion on society (RELINF)
• Control variables
• Parental education (HISCED)

57
The Religion Belief Hypothesis
58
The Religion Belief Hypothesis

• The hypothesis is not supported by the data
• The RELIG coefficient is non-significant
• The RELINF coefficient is positive and
  significant, suggesting that students attaching a
  greater value to the influence of religion in
  society also share more positive attitudes toward
  immigrants. But the association with RELINF alone
  does not evaluate the hypothesis

59
The National Identity Hypothesis

• The National Identity Hypothesis maintains that
  individuals are less tolerant of immigrants when
  they have a greater sense of national identity
• Independent variables
• Students' attitudes towards their country
  (ATTCNT)
• Control variables
• Parental education (HISCED)

60
The National Identity Hypothesis
61
The National Identity Hypothesis

• The hypothesis is not supported by the data

62
The Urban/Rural Differences Hypothesis

• The urban/rural hypothesis anticipates more
  positive views of minorities in urban areas than
  in rural areas (Côté Erickson, 2009) due to
  greater opportunities to meet socially and
  culturally diverse people in cities (Erickson,
  2004)
• Independent variable
• School location (RURAL)
• Control variables
• School level SES
• School mean parental education (MHISCED)
• School mean parental occupational status (MHISEI)
• Availability of resources in local community
  (RESCOM)

63
The Urban/Rural Differences Hypothesis
64
The Urban/Rural Differences Hypothesis

• The hypothesis is not supported by the data
• The RURAL coefficient is non-significant

65
The School Climate Hypothesis

• The school climate hypothesis states that a safe
  and positive school climate favors more positive
  attitudes toward minorities. Such climate
  contributes to reduce the anxiety and threat
  underlying anti-minority attitudes (Comerford,
  2003 Dessel, 2010b Moradi et al., 2006)
• Independent variables
• Teachers' perceptions of classroom climate
  (TCLCLIM)
• Teachers' perceptions of social problems at
  school (TSCPROB)
• Controls
• School average parental education (HISCED)
• Availability of resources in local community
  (RESCOM)

66
The School Climate Hypothesis
67
The School Climate Hypothesis

• The hypothesis cannot be supported by the data

68
Questions?

• THANK YOU FOR YOUR ATTENTION
• ?
• Daniel.Caro_at_iea-dpc.de