Outcomes from a QuasiExperimental Study of the Effectiveness of SchoolBased Social Skills Training t

1
Outcomes from a Quasi-Experimental Study of the
Effectiveness of School-Based Social Skills
Training to Prevent Conduct Problems in Childhood
Mark W. Fraser Maeda J. Galinsky Paul R.
Smokowski Steven H. Day Roderick A. Rose Mary A.
Terzian Shenyang Guo Jungsook Lee
University of North Carolina Chapel Hill, NC
27599 Contact mfraser_at_email.unc.edu
Sponsored grant R21 DA013874 from the National
Institute on Drug Abuse
2
Presentation of a published report

• Fraser, M. W., Galinsky, M. J., Smokowski, P. R.,
  Day, S. H., Terzian, M. A., Rose, R. A., Guo,
  S. (2005). Social information-processing skills
  training to promote social competence and prevent
  aggressive behavior in third grade. Journal of
  Consulting and Clinical Psychology, 73(6),
  1045-1055.

3
A Classroom Intervention
4
Making Choices ProgramsTwo Versions

• Making Choices (MC)
• Guided classroom process, Social learning model
  18.4 hours
• 7 Units
• Teaching skills
• Supervised practice and application
• Making Choices Plus (MC Plus)
• Guided classroom process, plus
• Infusion packet 4-week application to infuse
  content across other classroom activities
• Classroom behavior management Good Behavior
  Game or similar strategy
• Parent involvement program five Family
  Nights and Making Choices Newsletter

Making Choices
5
Hypotheses
6
Working Hypotheses

• Classrooms receiving MC and MC Plus will
  demonstrate more prosocial behavior than those
  receiving a routine health curriculum only
• A broader pattern of effects will be observed for
  MC Plus relative to MC only
• No significant differences will be observed by
  sex or race/ethnicity

One-tailed tests for intervention effects at
classroom level
7
Methods
8
Design

• Setting Two rural/suburban elementary schools
• Classrooms 29 (3rd Grade)
• Teachers 14
• Cohort Design (jclassrooms)
• Year 1 Routine Services (j9)
• Year 2 Making Choices Only (j9)
• Year 3 Making Choices Plus (j11)

Note. Complicated nesting structure of students
within classrooms within teachers (raters) within
schools.
9
Sociodemographic Characteristics by School
10
Measures Carolina Child Checklist and CBCL
Aggression

• Social Competence
  (?.92)
• Social Contact
• (?.84)
• Cognitive Concentration
• (?.96)

• Social Aggression (? .91)
• Authority Acceptance (? .89)
• CBCL Aggression (? .94)

Data Source Pretests for Cohorts 1, 2, and 3
(N548)
For a review of CCC instrument, see Macgowan,
M. J., Nash, J. K., Fraser, M. W. (2002). The
Carolina Child Checklist of risk and protective
factors for aggression. Research on Social Work
Practice, 12(2), 253-276.
11
Skill Level Assessment of Social Information
Processing Skills

• Encoding (?.78)
• Hostile Attribution(?.52)
• Goal Formulation(?.76)

• Response Decision(?.80)
• All SIP skills(?.71)

Data Source Posttests for Cohorts 1, 2, and 3
(N548)
Note. Students listen to a series of six short
stories in which a peer interaction of ambiguous
intent occurs. They are asked to put themselves
in the place of the main character and answer
paper-and-pencil questions according to how they
would respond in the given situation. The Skill
Level Assessment is an adaptation of Dodges Home
Interview for attributional bias (Dodge, 1980
Dahlberg, Toal, Behrens, 1998).
12
Analytic Strategy

• Multilevel (random effects) ANCOVA for
  differences in pretest across cohorts
• 2-level (student and teacher) hierarchical linear
  models, controlling for pretest, gender,
  race/ethnicity, and school
• 3-level (student, classroom, and teacher)
  hierarchical linear models, controlling for
  pretest, gender, race/ethnicity, and school

Note. The effects of MC and MC are estimated as
one-tailed tests at the classroom level. For a
comparable example, see Raudenbush and Bryk
(2002, pp. 112-113).
13
2-Level Hierarchical Linear Model
Conditional posttest classroom mean

• Level 1
• POSTij ß0j ß1j Preij ß2j AFRij ß3j LATij
  ß4j MALEij rij
• Level 2 (random intercept model)
• ß0j p00 p01MCj p02MCPj p03 SCHj u0j
• Andfor models with random intercepts and slopes
• Cognitive concentration
• ß2j p20 u2j (random slope for
  race/ethnicity)
• ß1j p10 p11MCj p12MCPj (interaction
  with pretest)
• Authority acceptance, social contact, CBCL
  aggression
• ß1j p10 u1j (random slope for pretest)

Adjusted posttest classroom score, conditioned on
MC by classroom, MCP by classroom, School by
classroom and a classroom error term
Average AA effect across classrooms plus (random)
error by classroom
14
3-Level Hierarchical Linear Model

• Level 1
• POSTijk ß0jk ß1jk Preijk ß2jk AFRijk ß3jk
  LATijk ß4jk MALEijk rijk
• Level 2
• ß0jk p00k p01kMCjk p02kMCPjk u0jk
• ß1jk p10k u1jk
• Level 3
• p00k ?000 e00k

Note. The effects of MC and MC are estimated as
one-tailed tests at the classroom level. For a
comparable example, see Raudenbush and Bryk
(2002, pp. 112-113).
15
Findings
16
Pretest Differences across the Three
Conditions/Cohorts

• Sex none
• Race/ethnicity none
• Cognitive concentration none
• Social competence none
• Social contact none
• Social aggression none
• Authority acceptance none
• CBCL aggression none

Multilevel ANCOVA with random effect covariate
for classrooms
17
Fitted 2-Level HLMs for Behavioral Outcomes
18
Fitted 3-Level HLM for Social Aggression
19
Fitted 3-Level HLM for Overt Aggression
Level Effect
Estimate SE Student
Conditional Mean (Intercept) 0.18
.03 Pretest
0.83 .09
African American
0.03 .03
Latino
-0.04 .02
Gender (male) 0.03
.02 Classroom Making Choices
-0.08 .03
Making Choices Plus -0.08
.03 Teacher School
0.05
.06 Student x Classroom Interaction
Gender x Making Choices -0.10
.05 Gender x Making
Choices Plus -0.06 .05
20
Fitted 2-Level HLM Posttest Social Information
Processing Differences by Group
21
Positive effects for

• Making Choices Only
• Social Competence
• Social Aggression
• Overt Aggression
• Social Contact
• SIP encoding, goal formulation
• Making Choices Plus
• Social Competence
• Social Aggression
• Overt Aggression
• Cognitive Concentration
• SIP encoding, hostile attribution, goal
  formulation, and response decision

22
Effect Sizes
Note. Effect size d ß/(t2 s2)1/2 where t2
is variance in conditional mean and s2 is error
variance
23
Are outcomes mediated

• by social information processing skills?
• by social competence?

Mary A. Terzian Jungsook Lee
24
What is Mediation?
Intro
Reduced Model Y X e
Bc Total effect of X on Y
Full Model Y X M e
BcDirect effect of X on Y BaDirect effect of X
on M BbDirect effect of M on Y BaBbIndirect
effect of X on Y Bc-Bc Indirect effect of X on Y
25
Theoretical Mediators Outcomes
Intro

• Outcomes
• Teacher-Rated
• Overt Aggression
• Social Aggression
• Social Contact
• Cognitive Concentration

• Mediators
• Teacher-Rated
• Social Competence
• Child Report
• Encoding
• Hostile Attribution
• Goal Formulation
• Enactment

Mediators and outcomes were measured at Time 2.
26
Two Ways to Test Mediation
Method
Method 1 Conduct deviance test (indirect
effectBc-Bc)

• Run Reduced (M0) and Full Models (M1) using ML.
• Obtain Deviances (-2LL) of Full and Reduced
  Models (D0 and D1). Chi-Squared Value D0-D1.
  Test the significance by running a Chi-Squared
  Test on this value with df1 (D0-D1, 1). Obtain
  p-value (no SE for this estimate).

Method 2 Test sig. of indirect effect (BaBb)

• Run Reduced (M0) and Full Models (M1). Also run a
  model regressing M on X. All models estimated in
  ML.
• Obtain estimates and SEs for Ba and Bb. Calculate
  SE for indirect effect using formula v(sb2Ba2
  sa2Bb2).Calculate t-value and find p-value.

27
Tests of Mediation
Results
28
Tests of Mediation
Results
29
Direct Effects
Cognitive Concentration Authority
Acceptance Social Competence Social Contact CBCL
Aggression Social Aggression Overt Aggression
Making Choices
Making Choices Plus
Response Decision Encoding Goal
Formulation Hostile Attribution All Social Info.
Processing Skills
30
Direct Effects
Mediating Effects
Cognitive Concentration Authority
Acceptance Social Contact CBCL Aggression Social
Aggression Overt Aggression
Interventions Theoretical Mediators Outcomes
Social Competence
Making Choices
Making Choices Plus
Response Decision Encoding Goal
Formulation Hostile Attribution All Social Info.
Processing Skills
31
Mediating Effects
Interventions Theoretical Mediators Outcomes

• Social Competence
• Goal Formulation
• Response Decision

Social Contact Cognitive Concentration Social
Aggression
MC MCP
Results of Method 1 only
(Dashed line) Results of Methods 1 2
(Straight line)
32
Implications
33
Implications for Intervention

• Skills training (alone) appears to affect social
  competence, social aggression, and overt
  aggression
• To produce a somewhat broader pattern of effects,
  add
• Classroom behavior management
• Classroom MC infusion
• Modest family involvement

34
A developmental perspective...

• From early life experiences, some children
  develop cognitions (e.g., attributions and
  scripts) and skills that negatively influence
  conduct
• In the absence of social changes to promote more
  positive early life experiences, elementary
  school intervention provides important
  opportunities to change cognitive processes and
  skills related to conduct problems.

35
Limitations

• Sample not urban
• History is confounded with cohort
• Experimental contamination of Years 2 and 3
  teachers in Year 3 had benefit of Year 2
  intervention
• Selection on unmeasured factors

36
Conclusions

• Hypotheses are largely supported
• The programs appear to reduce social and overt
  aggression and to increase social competence
• MC Plus has a stronger pattern of classroom and
  SIP effects

• Next Steps
• Estimate effects of differential exposure to
  MC/MC (dose)
• Estimate effects at 6-month follow-up when
  children are in 4th grade

37
Thank You
38
Selected References

• Crick, N. R., Dodge, K. A. (1994). A review and
  reformulation of social information-processing
  mechanisms in children's social adjustment.
  Psychological Bulletin, 115, 74-101.
• Dahlberg, L. L., Toal, S. B., Behrens, C. B.
  (1998). Measuring violence-related attitudes,
  beliefs, and behaviors among youths A compendium
  of assessment tools (pp. 66-76). Atlanta, GA
  Centers for Disease Control and Prevention,
  National Center for Injury Prevention and
  Control.
• Dodge, K. A. (1980). Social cognition and
  childrens aggressive behavior. Child
  Development, 51, 162-170.
• Dodge, K. A., Lansford, J. E., Burks, V. S.,
  Bates, J. E., Pettit, G. S., Fontaine, R., et al.
  (2003). Peer rejection and social
  information-processing factors in the development
  of aggressive behavior problems in children.
  Child Development, 74, 374-393.
• Dodge, K. A., Pettit, G. S. (2003). A
  biopsychosocial model of the development of
  chronic conduct problems in adolescence.
  Developmental Psychology, 39, 349-371.
• Fraser, M. W., Day, S. H., Galinsky, M. J.,
  Hodges, V. G., Smokowski, P. R. (2004). Conduct
  problems and peer rejection in childhood A
  randomized trial of the Making Choices and Strong
  Families programs. Research on Social Work
  Practice, 14, 313-324.
• Fraser, M. W., Nash, J. K., Galinsky, M. J.,
  Darwin, K. M. (2000). Making choices Social
  problem-solving skills for children. Washington,
  DC NASW Press.
• Macgowan, M. J., Nash, J. K., Fraser, M. W.
  (2002). The Carolina Child Checklist of risk and
  protective factors for aggression. Research on
  Social Work Practice, 12, 253-276.
• Raudenbush, S. W., Bryk, A. S. (2002).
  Hierarchical linear models Applications and data
  analysis methods (2nd ed.). Newbury Park, CA
  Sage.
• Smokowski, P. R., Fraser, M. W., Day, S. H.,
  Galinsky, M. J., Bacallao, M. L. (2004).
  School-based skills training to prevent childhood
  aggression Using the Making Choices program as a
  universal prevention initiative. Journal of
  Primary Prevention. 25(2), 233-251.
• Snijders, T. A. B, Bosker, R. J. (1999).
  Multilevel analysis An introduction to basic and
  advanced multilevel modeling. Thousand Oaks, CA
  Sage.