Effect Size Lens on Local SAT and State Math Test Results NC Accountability Conference February 2008 - PowerPoint PPT Presentation

Loading...

PPT – Effect Size Lens on Local SAT and State Math Test Results NC Accountability Conference February 2008 PowerPoint presentation | free to download - id: 177835-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Effect Size Lens on Local SAT and State Math Test Results NC Accountability Conference February 2008

Description:

Presentation by Johnny Wilson, Charles Lanier, and Mike ... Entice folks to exploit the usefulness of such comparisons. PUBLIC SCHOOLS OF NORTH CAROLINA ... – PowerPoint PPT presentation

Number of Views:26
Avg rating:3.0/5.0
Slides: 57
Provided by: dpi91
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Effect Size Lens on Local SAT and State Math Test Results NC Accountability Conference February 2008


1
Effect Size Lens on Local SAT and State Math
Test Results NC Accountability
ConferenceFebruary 2008
  • Presentation by Johnny Wilson, Charles Lanier,
    and Mike Gallagher, all from the Accountability
    Services Dept., NCDPI

2
Purposes Overall Session
  • Provide background and access paths to important
    data. (important helpful)
  • Show how LEAs and schools can access and compare
    test results across EOG, EOC and SAT.
  • Entice folks to exploit the usefulness of such
    comparisons.

3
Whos on First?
  • Johnny Wilson, Reporting Section, SAT Data
    Report. jwilson_at_dpi.state.nc.us
  • Charles Lanier, Test Development Section, State
    Testing Data in the Green Book.
    clanier_at_dpi.state.nc.us
  • Mike Gallagher, Test Development, Effect Size
    Look at SAT and State Test Results.
    mgallagher_at_dpi.state.nc.us

4
Purposes Part 1 SAT
  • Get acquainted with the major trends in SAT
    results for North Carolina.
  • Look at school level variability of SAT results
    from year-to-year.
  • Show the access path for information at the
    school and LEA levels.

5
SAT Math Score Trends
  • Between 1997 and 2007, the Math gap between NC
    and the Nation narrowed from 23 pts. to 6 pts.
  • NCs Math scores fell in 2007 after more than a
    decade of gains.

6
SAT Math Score Trends
7
SAT Verbal Score Trends
  • By comparison, NCs Verbal scores have been up
    and down over the same period.
  • Between 1997 and 2007, the Math gap between NC
    and the Nation has narrowed from 15 pts. to 7 pts.

8
SAT Verbal Score Trends
9
SAT Total Score Trends for NC and the Nation
  • After 10 consecutive years of gains,
  • NCs mean total SAT scores fell in
  • 2006 and 2007.
  • Gap between NCs mean total score
  • and the nations score has narrowed
  • from 53 in 1990 to 13 points in
  • 2007.

10
Total Score Trends for NC and the Nation
11
NCs Schools with Declining SAT Scores
  • At the school level, math scores also declined in
    2006 2007, compared with the previous year.
  • Notably, fewer schools increased math scores in
    2007 than in 2006.

12
Percent of NCs Schools with Declining SAT Scores
13
Changes in SAT Math Scores
  • Most changes in SAT scores from year to year are
    not unusual.
  • Low-Volume Schools tend to have larger changes.
  • The larger the test-taking population, the
    smaller changes in SAT scores tend to be.

14
Changes in SAT Math Scores by Test-taking Volume
(US)
15
Changes in SAT Math Scores by Test-taking Volume
(NC)
16
Changes in SAT Math Scores All Schools (US)
17
Changes in SAT Math Scores All Schools (NC)
18
Factors that Might Influence Score Fluctuations
  • Fatigue Factor
  • SAT fee increase in 2005 resulted in decline in
    repeat test-takers.
  • Difficulty of the test

19
NC SAT Math Scores by Race
  • NCs Asian students had the highest SAT score
    among racial/ethnic groups, followed by White
    students.
  • Only Asian and White students scored above the
    state average.

20
NC SAT Math Scores by Race
Mean SAT Score
21
SAT Mathematics Scores By Race/Gender
Mean SAT Score
22
For More Information . . .
  • SAT Reports from 1995 to 2007 include information
    and data
  • for
  • the Nation
  • the State
  • 117 Public School Systems
  • Charter Schools
  • North Carolina School of the Arts
  • North Carolina School of Science and Mathematics
  • http//www.ncpublicschools.org/accountability/repo
    rting/sat/

23
Sample SAT Report Page
  • Some questions to explore
  • Are districts closer to the state average in Math
    than in Critical Reading?
  • How about variability among schools within
    districts?
  • How about participation rates?

24
Question of the Day
  • SAT puts an official question of the day on its
    web site.
  • Excellent way to get a better understanding of
    what students will be facing. Why not have 8th
    graders try them?
  • www.collegeboard.com

25
Purposes Part 2 Green Book
  • Find out more about getting data from the Green
    Book,
  • both in its standard form
  • and its interactive, electronic form.

26
Electronic GreenBookA Web-tool for Data
27
Electronic GreenBookA Web-tool for Data
Your
28
Electronic GreenBook
29
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
  • Equally accurate
  • Equally reliable
  • More timely
  • More accessible

30
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
31
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
32
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
33
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
34
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
35
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
1. participation
2. performance
36
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
37
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
So far
  • Questions?
  • Answers?
  • 5 / 95 Rule

38
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
39
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
40
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
41
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
42
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
43
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
44
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
45
Electronic GreenBookhttp//report.ncsu.edu/ncpubl
icschools
46
Classic Green Book
  • SAMPLE PAGES FOLLOWING ..
  • Finding average scale scores for LEAs.
  • Finding standard deviations for EOC and EOG scale
    scores.
  • .. Which leads in to PART 3

47
Purposes Part 3 THE LENS
  • Discuss effect size, a useful tool for a fuller
    understanding of LEA or school performance across
    grade and time.
  • Compute some math effect sizes and highlight
    where the information is obtained.
  • Show the relationship between LEA math
    performance on SAT, EOC, and EOG.

48
The Importance of Where.
  • A school (or LEA) is data driven wise if
    everyone knows where they are and where they are
    going.
  • Donna Snodgrass, et al.

49
Where is this LEA?
  • Math ABC Results for 2006-2007
  • (Percent at or above grade level)
  • EOG Grade 5 .. 59.8 (state 66.7)
  • EOG Grade 8 .. 48.9 (state 65.1)
  • EOC Geometry 45.0 (state 63.0)
  • Pass rates and gaps in pass rates leave out a big
    part of the picture.

50
The Average is above Average
  • The percent above a cut score counts a score the
    same whether it is just above (below) the cut or
    far above (below) the cut. BUT the average
    includes all the score information.
  • Year-to-year changes in percent above are
    affected by where on the distribution the cut is
    located. BUT the average follows the center of
    scores.

51
Then Where is this LEA?
  • Math Score Averages for 2007
  • EOG Grade 5 . 353.0 (State 354.7)
  • EOG Grade 8 . 357.1 (State 360.5)
  • EOC Geom. 147.1 (State 151.3)
  • SAT 493 (State 509)
  • SAT Tested 72.8 (State 71.0)
  • Thats still difficult to answer with different
    tests on different scales.

52
Effect Size takes care of Differing Scales
  • Example Calculate the effect size for math SAT
  • Subtract the overall average from the local
    average 493-509 -16
  • Divide by the Standard Deviation
  • -16 / 110 -0.1 (rounded to 10ths)

53
So Where is this LEA?
  • Math Effect Sizes for 2007
  • EOG Grade 5 .. -0.2
  • EOG Grade 8 .. -0.4
  • EOC Geometry -0.4
  • Average SAT . . . . -0.1
  • SAT is very close to the state. Grade 5 is low
    but still close. Geometry and Grade 8 are much
    lower.

54
Questions to pose ..
  • In your LEA (or school) which grades, courses,
    SAT results are stronger?
  • In your LEA how do schools compare on average
    EOG, EOC, SAT?
  • How does your LEA (school) compare with similar
    LEAs (schools)?
  • What is the effect of what youve been trying to
    improve?

55
NC maths great progress
  • NAEP results from 1992 to 2005
  • Compared with the U.S. average and with other
    states, North Carolina went from back of the pack
    to ahead of the pack. NC was somewhat sluggish
    2003 to 2005.
  • For the NAEP data in the following slides, go to
  • http//nces.ed.gov/nationsreportcard/mathematics/

56
NAEP Scale Score Averages A Decade of Great
Progress
Grade 8
U.S. Average
North Carolina
Grade 4
U.S. Average
North Carolina
About PowerShow.com