Making MAP More Meaningful - PowerPoint PPT Presentation

Loading...

PPT – Making MAP More Meaningful PowerPoint presentation | free to download - id: 74e9b4-NjgxY



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Making MAP More Meaningful

Description:

Making MAP More Meaningful David Dreher, Project Coordinator Dr. Kathryn Sprigg, Assistant Director Office of Accountability, Highline Public Schools – PowerPoint PPT presentation

Number of Views:77
Avg rating:3.0/5.0
Slides: 37
Provided by: dreh151
Learn more at: http://www.wera-web.org
Category:
Tags: map | making | meaningful | more

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Making MAP More Meaningful


1
Making MAP More Meaningful
  • David Dreher, Project Coordinator
  • Dr. Kathryn Sprigg, Assistant Director
  • Office of Accountability, Highline Public Schools
  • Dr. Sandra L. Hunt , Literacy Coach
  • Beverly Park Elementary, Highline Public Schools

2
Overview
  • The needs of the data users
  • The objectives of the data producers
  • The products
  • The process
  • The implementation
  • The results
  • The future

3
What is MAP
  • Measures of Academic Progress
  • Developed by the Northwest Evaluation Association
  • Norm-referenced assessment
  • Computerized and adaptive
  • Performance is reported as a RIT score
  • The RIT Scale
  • Uses individual item difficulty values to
    estimate student achievement
  • A RIT score has the same meaning regardless of
    grade level
  • Equal interval scale
  • Highline Public Schools
  • Three testing windows per year (Fall, Winter,
    Spring)
  • Test students in the areas of math and reading
  • Test students in grades 3-10

4
The Needs of the Data User
  • Building staff were saying things like . . .
  • How can we use MAP data to help us make
    decisions?
  • How do MAP and WASL performance compare?
  • I want to know what a students history is with
    MAP.
  • What is a RIT score?
  • Giving me a RIT score is like telling me the
    temperature in Celsius!

5
The Objectives for Us
  • Include more historical data in reports.
  • Make the data more accessible.
  • Put MAP scores in context with WASL scores.
  • Provide indication of a students likelihood of
    meeting standard.

6
Some General Challenges
  • Fear of Numbers
  • The products generated had to be fairly simple to
    explain and understand.
  • Availability of Time
  • Because it had to be there yesterday it has to be
    fairly simple for us to produce.

7
The Products
  • Fall Predictions
  • Our best guess about each students performance
    on the upcoming WASL.
  • Used for
  • Identifying level of risk for not meeting
    standard
  • School- and District- level WASL forecasts
  • Benchmark, Strategic, Intensive (BSI) Updates
  • Status update produced after each testing
    window.
  • Coarse filter based only on MAP.
  • Cut Score Document
  • A quick reference table that could be used to
    help put a MAP score in context.

8
Making The Predictions
  • Snooped and found the best indicators of WASL
    success
  • Applied linear regression models to generate WASL
    scores for each student
  • Examined the predicted WASL scores

9
Snooping (Reading) R-Values
WASL 2007 Reading Scale vs. Grade Grade Grade Grade Grade Grade Grade
WASL 2007 Reading Scale vs. 3 4 5 6 7 8 10
WASL Reading 2006   0.730 0.769 0.764 0.745 0.755  
WASL Reading 2005       0.727      
WASL Reading 2004             0.723
MAP-R Spring 2007 0.792 0.778 0.799 0.774 0.744 0.726 0.755
MAP-R Winter 2007 0.782 0.782 0.796 0.767 0.780 0.711 0.786
MAP-R Fall 2006 0.769 0.750 0.793 0.768 0.768 0.755 0.806
High MAP-Read (F06, W07, S07) 0.804 0.779 0.816 0.784 0.791 0.746 0.804
High MAP-Read High MAP-Math 0.814            
High MAP-Read WASL06 Read scale   0.803 0.835 0.824 0.816  0.780  
High MAP-Read WASL 04_Rscale             0.787
10
Snooping (Math) R-values
WASL 2007 Math vs. Grade Grade Grade Grade Grade Grade Grade
WASL 2007 Math vs. 3 4 5 6 7 8 10
WASL Math 2006   0.801 0.840 0.875 0.873 0.890  
WASL Math 2005       0.603      
WASL Math 2004             0.856
MAP-Math Spring 2007 0.817 0.862 0.862 0.865 0.888 0.896 0.828
MAP-Math Winter 2007 0.809 0.851 0.860 0.863 0.883 0.906 0.856
MAP-Math Fall 2006 0.794 0.817 0.848 0.833 0.879 0.907 0.865
High MAP-Math (F06, W07, S07) 0.832 0.877 0.879 0.878 0.902 0.915 0.879
High MAP-Math High MAP-Read 0.846            
High MAP-Math WASL06 Math scale   0.892 0.896 0.910 0.921 0.930  
Highest MAP-M WASL04 Math scale             0.908
11
What we learned by snooping. . .
  • Correlations were generally good.
  • Reading R-value range 0.711 - 0.835
  • Math R-value range 0.603 - 0.921
  • Correlations in math were stronger than in
    reading.
  • Highest MAP consistently correlated better than
    any single MAP score.
  • Correlations were generally strongest when
    Highest MAP and WASL 2006 factors were combined.

12
Regression Models
  • For students with both MAP and 2006 WASL scores
    (95)
  • WASL 2007 b0 b1Highest MAP b2WASL 2006
  • For students that only had MAP score(s) (3)
  • WASL 2007 b0 b1Highest MAP
  • For students that only had WASL 2006 score
    (2)
  • WASL 2007 b0 b1WASL 2006
  • Where
  • Highest MAP The students highest score on MAP
    from the Fall 2006, Winter 2007, or Spring
    2007 windows.
  • Typically Spring 2007.
  • WASL 2006 The students raw score from the
    2006 WASL Spring testing.

13
Prediction Models
  • For students with both MAP and 2007 WASL scores
  • WASL 2008 b0 b1Projected MAP b2WASL 2007
  • For students with only MAP score(s)
  • WASL 2008 b0 b1Projected MAP
  • For students with only WASL 2007 score
  • WASL 2008 b0 b1 WASL 2007
  • Where
  • Projected MAP Projected Spring 2008 MAP score
    based on the students highest score on MAP
    from the Winter 2007, Spring 2007 or Fall
    2008 windows.
  • WASL 2007 The students raw score from the
    2007 WASL Spring testing.

14
Projecting MAP to Spring
  • For the models with Projected MAP as one of the
    factors individual student performance on MAP in
    the Spring of 2008 was projected.
  • The amount of expected growth added to a
    students Highest MAP score came from NWEAs
    Growth Study

15
Example of Projection and Prediction 7th Grade
Student in Reading
MAP Scores MAP Scores MAP Scores
Winter 2006 Spring 2007 Fall 2007 Highest MAP
210 212 216 216
Expected MAP Growth
3
WASL 2007 Projected MAP Spring 2008
392 219
Predicted Predicted
Prediction Model 2008 WASL Score 2008 WASL Range
MAP and WASL 399 392 - 406
MAP Only 402 395 - 409
WASL Only 397 390 - 404
16
WASL Prediction Range
  • Constructed using the SEM values reported in the
    2001 WASL Technical Reports.
  • Predicted Range Predicted WASL Score /- SEM

Grade Level SEM Reading SEM Math
3, 4, 5, 6 7 12
7,8 7 16
10 10 12
17
Examining the Predictions
  • What are the predictions saying about how we
    might do in 2008?
  • Forward look
  • How would we have done if we had predicted 2007
    WASL scores in the fall of 2006?
  • Backward look

18
What are the predictions saying about how we
might do in 2008?
19
What are the predictions saying about how we
might do in 2008?
20
What are the predictions saying about how we
might do in 2008?
21
What are the predictions saying about how we
might do in 2008?
22
What are the predictions saying about how we
might do in 2008?
23
What are the predictions saying about how we
might do in 2008?
24
Looking Backwards How would we have done
predicting the 2007 WASL?
  • Successful prediction
  • Accurately predicting whether a student would or
    would not meet standard on the WASL
  • Unsuccessful prediction
  • Predicted to meet standard and did not
  • false positive (the kind we dont want)
  • Predicted not to meet standard but did
  • false negative (the kind we are okay with)

25
Looking Backwards - Math
Grade Successful Prediction () Unsuccessful Prediction Unsuccessful Prediction
Grade Successful Prediction () False Positives () False Negatives ()
3 80.8 7.2 12.0
4 87.3 6.3 6.4
5 86.8 5.2 8.0
6 86.8 4.8 8.4
7 89.8 4.5 5.7
8 89.0 5.3 5.7
10 86.9 5.5 7.6
District 86.8 5.5 7.7
26
Looking Backwards - Math
  Actual Met Predicted to Meet Actual Met Predicted to Meet
Grade Number of Students Percent of Total
3 -55 -4.8
4 -1 -0.1
5 -33 -2.8
6 -45 -3.6
7 -13 -1.2
8 -5 -0.4
10 -19 -2.1
District -171 -2.2
27
Looking Backwards - Reading
Grade Successful Prediction () Unsuccessful Prediction Unsuccessful Prediction
Grade Successful Prediction () False Positives () False Negatives ()
3 80.5 9.1 10.4
4 83.1 9.0 7.9
5 81.8 10.3 8.0
6 83.1 7.8 9.0
7 82.5 7.3 10.2
8 81.0 8.3 10.7
10 87.7 8.2 4.2
District 82.7 8.6 8.8
28
Looking Backwards - Reading
  Actual Met - Predicted to Meet Actual Met - Predicted to Meet
Grade Number of Students Percent of Total
3 -15 -1.3
4 14 1.2
5 27 2.3
6 -15 -1.2
7 -32 -2.9
8 -27 -2.5
10 35 4.0
District -13 -0.2
29
The Implementation
  • Fall Predictions
  • Rolled out at Fall Math Summit
  • Cut Scores
  • Released in November 2007
  • BSI Status Updates
  • Delivered in February 2008
  • Use of the information was determined within each
    building by principals, coaches, and teachers.

30
The Results
  • How good were the predictions?
  • We wont know how good they are until after we
    get our WASL results.
  • Come see our Fall WERA presentation!
  • Did the products work for the end user.
  • Feedback has been fairly limited
  • Most feedback has been positive
  • Some feedback says more work is still needed.

31
The Future
  • Check the predictions after WASL results are
    released
  • Continue to refine the products to make them
    work for the end user
  • job security

32
Work in Progress Cut Scores Document
  • Predictions Roll Out
  • Cut Score Table
  • Augmented with BSI graph
  • NWEAs recently released Cut Scores document

33
Email to Principals
  • If the prediction range is
  • Entirely below 400 (ex. 380-396) student has
    less than a 20 chance on the WASL this spring
    unless we accelerate their learning.
  • Straddles 400 (ex. 396-410) student has
    basically a coin-flip chance on the WASL, even if
    their prediction is above 400.
  • Entirely above 400 (ex. 408-424) student has
    more than an 80 chance on the WASL in the
    spring, IF they continue to progress.

34
(No Transcript)
35
(No Transcript)
36
Contact Information
  • David Dreher, Project Coordinator
  • Dr. Kathryn Sprigg, Assistant Director
  • Office of Accountability, Highline Public Schools
  • www.hsd401.org
  • 206-433-2334
About PowerShow.com