Making math work: An evidencebased approach to improving math skills of high school students

1
Making math work An evidence-based approach to
improving math skills of high school students

• James R. Stone III
• Director

2
The work reported herein was supported under the
National Dissemination for Career and Technical
Education, PR/Award (No. VO51A990004) and /or
under the National Research Center for Career and
Technical Education, PR/Award (No. VO51A990006)
as administered by the Office of Vocational and
Adult Education, U. S. Department of
Education.However, the contents do not
necessarily represent the positions or policies
of the Office of Vocational and Adult Education
or the U. S. Department of Education, and you
should not assume endorsement by the Federal
Government.
Disclaimer
3
Background What do we know about CTE?

• CTE does not necessarily limit postsecondary
  education (NAVE, 2004 Stone Aliaga, 2004 but
  see Deluca et al., forthcoming)
• There is evidence that math and science course
  taking by CTE students is increasing amount and
  complexity (NAVE, 2004 Stone Aliaga, 2004)
• CTE as a function of the HS experience reduces
  the probability of dropping out of school (Plank,
  2001, 2005 Castellano et al 2006)
• CTE is an economic value to the individual and
  the community (ROI) (Bishop Mane, 2004 NAVE,
  2004Hollenbeck, 2001)
• It is possible to major in CTE and Academics
  (NAVE, 2004)

4
What else do we know? CTE enrolls many students
who are the focus of closing the achievement
gap
Levesque, K. (2003). Public High School Graduates
Who Participated in Vocational/Technical Education
5
Major Issues of HS Reform

• Engagement attending school and completing
  (graduating) high school
• Achievement academic (and technical) course
  taking grades, test scores
• Transition to postsecondary education without
  the need for remediation and to the workplace

6
National Graduation Rates 1998 and 2001 The
problem of engagement
7
of 9th Graders who complete High School
68
Source One-Third of a Nation (ETS, 2005)
8
Carnegie Grows!
9
CTE and School Engagement
Source Plank, 2001
10
CTE and Drop Out Reduction The Debate Continues

• NAVE, 2004 No effect (NELS88 - Class of 1992
  data)
• Plank, 2001 Significant effect, especially for
  low ability youth (NELS 88 - Class of 1992 data)

Plank, forthcoming. NLSY97 Transcript
data
11
When do they leave?
9th grade 10th grade
11th grade 12th grade 5th year

From Plank, forthcoming
12
Graduation School and CTE Effects




From Castellano, Stringfield Stone, Forthcoming
13
CTE Structures and Pedagogies and Dropping Out

• Students in or Career Majors are 16 more likely
  to graduate from high school.
• Students in Tech Prep are 30 more likely to
  complete high school.
• Students who participated in specific STW
  activities are 18 more likely to complete high
  school.

Stone Aliaga, in press
14
Do CTSOs Add Value to CTE?
Comparisons
General Student Population Class (same school)
CTE Class with CTSO
General Student Population Class (same school)
CTE Class-No CTSO
15
Preliminary Findings (NRCCTE, forthcoming)

• Compared to the general population of students
• No direct effect of CTSO on grades (student
  self-report)
• Being in a CTSO positively affects academic
  motivation and engagement
• General student population and CTSO students have
  different educational aspirations (4-year vs.
  2-year) more realistic?

16

• Within CTSOs
• Experience of SCANs type skills in the classroom
  positively affects career self-efficacy and
  grades
• Degree of participation in the CTSO positively
  affects grades, aspirations, career
  self-efficacy, academic engagement, self-esteem,
  and civic engagement the more participation,
  the better
• Effect is strongest for competitive events

17
Academic Achievement of Youth in CTE Articulated
Programs

18
Development of Skills
19
Transition to college The Challenge
31 Leave with 0 Credits
68 Graduate HS in 4 Years
18 GraduateCollege in 4 Years
100 Start 9th Grade
40 Start College
27 Start Sophomore Year
31
Source Education Weekly March 2005
20
College for All The Reality
Percentages by Race and Ethnicity

• By age 29
• 34 of white
• 18 of African Americans
• 10 of Hispanic
• Have bachelors degrees

Hoffman, N. (2003)
Venezia, A., M. W. Kirst, et al. (2003)
21
College Degree At What Cost?
According to the Public Interest Research Group's
Higher Education Project, 39 percent of new
graduates with loans carry an "unmanageable debt,"
22
CTE What do we know?

• CTE keeps kids in school
• CTE helps kids focus their PS education plans
• CTE is an economic benefit to participants and to
  states
• CTE-based structures (e.g.,dual enrollment,
  career academies) can affect achievement and
  transition of youth to college and work.
• But what more value can CTE provide as part of
  the high school experience?

23
Math-in-CTE An evidenced based approach to
improving academic performance of CTE students
24
The Problem Math PerformanceOf American Youth
NAEP Scores for 17 Year olds
25
More ProblemsScience Performance 17 Year Olds
A Nation At Risk
26
The number of 17-year-old students taking
advanced math classes has also increased -- with
17 percent studying calculus and 53 percent
studying second-year algebra --  it is unclear
why that trend has not resulted in higher average
math scores over all.

•  

http//nces.ed.gov/nationsreportcard/ltt/results20
04/
27
Math-in-CTE

• A study to test the possibility that enhancing
  the embedded mathematics in Technical Education
  coursework will build skills in this critical
  academic area without reducing technical skill
  development.

1. What we found 2. What we learned
28
Key Questions of the Study

• Does enhancing the CTE curriculum with math
  increase math skills of CTE students?
• Can we infuse enough math into CTE curricula to
  meaningfully enhance the academic skills of CTE
  participants (Perkins III Core Indicator)
• Without reducing technical skill development
• What works?

29
Study Design Key Features

• Random assignment of teachers to experimental or
  control condition
• CTE teachers were the teachers of math in study
• Five simultaneous study replications
• Three measures of math skills (applied,
  traditional, college placement)
• Focus of the experimental intervention was
  naturally occurring math (embedded in curriculum)
• A model of Curriculum Integration
• Monitoring Fidelity of Treatment

30
Study Design 04-05 School Year
Sample 2004-05 69 Experimental CTE/Math teams
and 80 Control CTE Teachers Total sample
3,000 students
31
Math-in-CTE Experimental Treatment What we
tested

• Professional Development
• Curriculum mapping
• Development of math-enhanced lessons
• Scope Sequence
• On-going math support

32
What we tested The Seven Elements

• Introduce the CTE lesson
• Assess students math awareness
• Work through the embedded example
• Work through related, contextual examples
• Work through traditional math examples
• Students demonstrate understanding
• Formal assessment

33
What we found Map of Math Concepts Addressed by
Enhanced Lessons in each SLMP
34
What we found (results) All CTEx vs All
CTEcPost test correct controlling for pre-test
p.08
p.03
p.02
Results follow one year of implementation in one
CTE class
35
What we found Site level analyses
Only Significant effects shown
36
Magnitude of Treatment Effect Effect Size
Measure
Effect Size Cohens d .80
the average percentile standing of the average
treated (or experimental) participant relative to
the average untreated (or control) participant
50thpercentile
X Group
C Group
79thpercentile
0
50th
100th
Example of an effect size of .80
37
What we found Magnitude of effect
Effect size (Cohens d) All Classes Terra
Nova (d.34) Accuplacer
(d.17) By Site Site A WorkKeys
(d2.8) Site B- TerraNova (d.69)
Site C Accuplacer (d.85) Site E- Terra
Nova (d.64) Site F AccuPlacer (d.39)

• Percentile shift
• From 50th to
• 62nd
• 56th
• 99th
• 76th
• 81st
• 74th
• 66th

Comparison Carnegie Learning Corporation
Cognitive Tutor
Algebra I d.22
38
Does Enhancing Math in CTE

• Affect Technical Skill Development?

39

No difference in four sites experimental
students scored significantly higher in one site
plt.10
40
What we found Time invested in Math Enhancements

• Average of 18.55 hours across all sites devoted
  to math enhanced lessons (not just math but math
  in the context of CTE)
• Assume a 180 days in a school year one hour per
  class per day
• Average CTE class time investment 10.3 in one
  CTE class

41
What we learned

• When We Began (assumptions)
• A box of curriculum
• Individual teacher training
• Replicable by individual teachers

• After the Study, we know
• A curriculum development is a process
• Replicable by teams of committed teachers working
  together over time
• Core Principles

42
Replicating the Math-in-CTE ModelCore
Principles

• Develop and sustain a community of practice
• Begin with the CTE curriculum and not with the
  math curriculum
• Understand math as essential workplace skill
• Maximize the math in CTE curricula
• CTE teachers are teachers of math-in-CTE NOT
  math teachers

43
What we are and are not A contextual continuum

• Traditional academic class (e.g. Algebra 1)
• CTE Academic teachers coordinate around themes
  (e.g. health)
• Occupation is the context for delivery of
  traditional academics
• (Related or applied math)
• Academics emerge from occupational content

• Disconnected
• Coordinated
• Context Based
• Contextual

• Algebra 1
• Academies
• Integrated math
• NRC Model

44
Final thoughts Math-in-CTE

• A powerful, evidence based strategy for improving
  math skills of students
• A way but not THE way to help high school
  students master math
• Not a substitute for traditional math courses
• Lab for mastering what many students learn but
  dont understand

45
For more information

• James R. Stone III
• stone003_at_umn.edu

The University of Minnesota 612-624-1795