Increasing Student Performance on NJASK 5-7 Through Knowledge of Test Design and Scoring and Student Expectations - PowerPoint PPT Presentation

Loading...

PPT – Increasing Student Performance on NJASK 5-7 Through Knowledge of Test Design and Scoring and Student Expectations PowerPoint presentation | free to view - id: 7bafe6-OTYzO



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Increasing Student Performance on NJASK 5-7 Through Knowledge of Test Design and Scoring and Student Expectations

Description:

Mathematical Thinking for NJASK Presenter, Barbara VanDenBerg Mathletics Training Workshops – PowerPoint PPT presentation

Number of Views:117
Avg rating:3.0/5.0
Slides: 38
Provided by: mlu70
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Increasing Student Performance on NJASK 5-7 Through Knowledge of Test Design and Scoring and Student Expectations


1
Mathematical Thinking for NJASK Presenter,
Barbara VanDenBerg Mathletics Training Workshops
2
Mathematical Thinking for NJASK Presenter,
Barbara VanDenBerg AMTNJ
3
Increasing Student Performance on NJASK Through
Knowledge of Test Design, Scoring,and Student
Expectations
  • NJASK 2008

4
  • These are not long-term tests.
  • This is the start of a 5 year cycle.

5
Topics
  • Concerns From the Field
  • Directives From Policy Makers at DOE/Assessment
    Advisory Committee (AAC)
  • Learnia
  • New Test Design 2008-?
  • Online Resources/Sample Items
  • Standards Clarification Project
  • Teacher Concerns/Testing Notes
  • Opportunities for Educator Involvement
  • Miscellaneous Items
  • Sample Items
  • Suggestions

6
Concerns From the Field
  • could questions be grouped so that at least 2
    easy items are first
  • Consider placement of questions within the test
    and the difficulty level of each.
  • Yes, its difficult but could questions be
    grouped so that at least 2 easy items are first
    (just to reduce brain freeze).

7
  • Estimation questions are not effectively tested
    when a calculator is available. This is too
    important a skill to be underrepresented this
    way. (Same with number sense. Non-calculator
    section of this test would have real value).
  • Readability in many problems is the more
    influential factor in setting level of
    difficulty. Concepts and application can vary so
    much, why not look for better ways to determine
    mathematical knowledge?

8
  • There is not a short section of straight
    calculations without a calculator on the test
    (5-7).
  • A 5 minute silent break during 80 minutes of
    testing is unfair to students of this age.
  • Mathematics testing before LAL.
  • Concerns about separate test booklet and answer
    folder.

9
Assessment Advisory Committee (AAC)
  • Representatives from constituent and stakeholder
    groups (2005-2007)
  • Formative Assessment
  • Bifurcated Test Design
  • a. January- Performance Assessments
    (NJPAA)
  • b. May- Objective items

10
Directives From Policy Makers at DOE
  • More open ended content (NJPAA)
  • More raw score points
  • More items, more than one or two more
  • Mathematics testing over two days

11
Learnia- the formative assessment component
  • Introduced by August 24, 2007 letter from
    Commissioner Davy to CSAs
  • http//newjersey.learnia.net
  • Class Views/Class Links
  • Janet McMillan
  • 1-800-228-0752 x 8453
  • janet_mcmillan_at_harcourt.com
  • Scott Herr 1-800-228-0752 x 5680

12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
Math AssessmentOverview contd
  • The NJ ASK is not a diagnostic test but a test to
    measure how well children are achieving the NJ
    CCCS
  • Common misconception is that the NJDOE expects to
    have ten months of material taught in just over 6
    or 7 months. This would be true if answering 27
    out of 30 questions correctly equaled an A and
    so on, but that isnt the case.

18
Math AssessmentOverview Grade 3
  • 27 multiple choice items, which include 6
    non-calculator multiple choice items
  • Each multiple choice item is each worth 1 point,
    the non-calculator items are each worth ½ point
  • Total points for multiple choice items is 24
  • NJ ASK 3 contains a total of 30 items

19
Math AssessmentOverview contd
  • NJ ASK 4 contains a total of 37 items
  • 32 multiple choice items, which include 8
    non-calculator multiple choice items
  • Each multiple choice item is each worth 1 point,
    the non-calculator items are each worth ½ point
  • Total points for multiple choice items is 28

20
Math AssessmentOverview contd
  • NJ ASK 4 contains 5 open-ended items worth 3
    points each for a total of 15 points
  • Total points on the NJ ASK 4 is 43

21
New Test Design 2008-?
  • Grades 5 and 6 will have a calculator inactive
    section (not a computation section)
  • Types of items short constructed response (SCR),
    multiple choice (MC), and extended constructed
    response (ECR)
  • Mathematics testing is over two days with five
    sections (Grade 8 remains one day)

22
Test Design Grades 5 and 6
  • Day 1
  • Section 1, 8 SCR, 16 minutes,
  • calculator inactive
  • Section 2, 12 MC, 2 ECR, 35 minutes,
  • calculator active
  • Total Day 1 51 minutes total testing time

23
  • Day 2
  • Section 3, 10 MC, 1 ECR, 23 minutes
  • Section 4, 10 MC, 1 ECR, 23 minutes
  • Section 5, 10 MC, 1 ECR, 23 minutes
  • Total Day 2 69 minutes total testing
  • time, all sections on day 2 are
  • calculator active

24
Test Design Grades 7 and 8
  • Two changes from Grades 5 and 6
  • Section 1 becomes calculator active
  • and number of SCR items is increased to
  • 10, 55 minutes total testing time
  • on day 1
  • Grade 8 only- all five sections will be
  • administered in one day- total testing time
  • 133 minutes

25
ITEM INFORMATION
  • MC- multiple choice, 1 raw score point, 1.25
  • minutes per item (1.5 _at_ grade 8)
  • SCR- short constructed response, 1 raw score
    point, 2 minutes per item
  • ECR- extended constructed response, 3 raw score
    points max., 10 minutes per item

26
Grade 5 Grade 6 Grade 7 Grade 8
Item Count by Type MC 42 42 42 42
Item Count by Type SCR 8(non- calculator) 8(non-calculator) 10 10
Item Count by Type ECR 5 5 5 5
of sections of sections 5 5 5 5
Total raw score points possible Total raw score points possible 50 50 52 52
Approximate total testing time Approximate total testing time 120 min. 120 min. 124 min. 133 min.
27
New Features
  • Raw score points 2005/06 39 vs. 50 (52) in 2008,
    test reliability
  • More opened content 2005/06 9 raw score points
    vs. 18(20) in 2008
  • More items, more than one or two more 2005/06 33
    vs. 42(44) in 2008
  • Calculator inactive sections for grades 5 and 6

28
New Features
  • Two day testing limits total testing time in one
    day
  • SCR items to assess computation, estimation,
    structure, knowledge items, algebraic
    transformations, etc.
  • SCR items limit amount of reading required of a
    child

29
New Features (SCR Items)
  • Each child starts with a section of Short
    Constructed Response Items (8/10)
  • NOT strictly computational items.
  • 2 minutes per item.
  • Answer is written in a 2 answer slot in answer
    folder
  • Human scored (0 or 1- no partial credit for
    work). There is flexibility in scoring- answer
    options, units, etc.

30
New Features
  • Remaining four sections are a mix of MC and
    Extended Constructed Response (ECR) Items
  • MC Items (1.25/1.5 minutes)
  • ECR Items- 3 point maximum, human/rubric scored,
    10 minutes
  • Increase in constructed response items and points
    (10 and 18/12 and 20)

31
New Features
  • Grades 56 Have a Non-calculator Active Section
  • More Raw Score Points (50/52)
  • Each Content Cluster Receives an Extended
    Response Item (12 or 13/13)
  • Grades 5-7 Two Days of Mathematics Testing

32
New Features (SCR Items)
  • Limit reading required of a child
  • Assess estimation, structure, knowledge items,
    algebraic transformations (reduces plug and
    chug), computational algorithms, etc.

33
Hard Core Mathematical Facts
  • All ECR items are at the end of a section.
  • Generic answer slot for all ECR items, parts are
    gone and replaced with bullets. Answer does not
    have to be narrative.
  • Release items 1 ECR item per grade with scoring
    rubric and exemplar papers 1 SCR item per grade
    per year and, 4 MC items per year with related
    statistics.
  • Measurements may be used to make conclusions.

34
Hard Core Mathematical Facts
  • Tick marks will be used to show congruence as
    early as grade 5.
  • Range of data is a value. The range of 5, 9,
    10, 4, 2, and 6 is 8 not 2 to 10.
  • Probability- student may assume equally likely
    events.
  • 3x36 is acceptable in grade 5.

35
New Developments at DOE
  • Standard Clarification Project (SCP)
  • Conducted by Dr. Grant Wiggins (UbD) and Dr.
    Robert Riehs
  • Areas of Focus (Bold) Grade 6 page 8 with Sample
    Performance Task
  • Sample Items MC Grade 7 page 2, SCR Grade 6
    page 9, ECR Items

36
New Developments at DOE
  • Document is embedded with Big Ideas, Essential
    Questions, Enduring Understandings, Comments and
    Examples, and Instructional Foci.
  • Mile Wide and Inch Deep Grade 7 page 5. We DO
    NOT assess explore CPIs. This is NOT a Power
    Standards document.
  • Show Sample Unit Framed by Big Ideas, Essential
    Questions, and Enduring Understandings.
  • http//www.nj.gov/education/aps/njscp/

37
New Developments at DOE
  • Formative Assessment- Learnia, free to districts-
    no plans otherwise
  • http//newjersey.learnia.net
  • Scott Herr 1-800-228-0752x5680 or Janet McMillan
    1-800-228-0752x8453 janet_mcmillan_at_harcourt.com
  • 150 new districts for 2008/09
  • Four assessment literacy workshops are planned
    for March 2008.
  • Class Links/Class Views/Distractor Analysis
  • 5 items per CPI by end of 2007/08 school year in
    Class Links Item Bank.

38
Cut Scores and Percents
Gradegt Year v 5 6 7
2006 Partial 18/39 54.24 Adv. 30/39 27.52 Partial 17/39 53.49 Adv. 31/39 17.26 Partial 13/39 49.85 Adv. 26/39 14.44
2007 Partial 18/39 56 Adv. 29/39 22.6 Partial 16/39 53 Adv. 30/39 13.4 Partial 14/39 47.6 Adv. 27/39 11.8
39
Miscellaneous
  • MC Items- 1.25 minutes (grade 8 1.5 minutes), 1
    point each, machined scored
  • SCR Items- 2 minutes, 1 point each, no partial
    credit, hand scored
  • ECR Items- 10 minutes, 3 points max, hand scored
    using rubric
  • All ECR will be at the end of a section
  • Generic answer slot for all ECR items, Parts are
    gone and replaced with bullets

40
Online Resources/Sample Items
  • Grade specific reference sheets
  • http//www.nj.gov/education/assessment/ms/5-7/
    gr5_NJASK06_math_ref.pdf
  • Manipulatives Sheet
  • http//www.nj.gov/education/assessment/ms/5-7/2
    006-07ManipulativesSheet.pdf

41
  • Learnia Formative Assessment Product
  • Test samplers
  • http//www.nj.gov/education/assessment/ms/5-7/
  • Calculator Policy
  • http//www.nj.gov/education/assessment/ms/5-7/t
    cm.pdf

42
  • Performance Level Descriptors
    http//www.nj.gov/education/assessment/descriptors
    /
  • Standards Clarification Project
  • http//www.nj.gov/education/aps/cccs/math/
  • Grades 5-8 document which highlights Areas of
    Focus and will contain MC, SCR, and ECR sample
    items aligned to NJCCCS CPIs

43
Sample SCR Items Grades 5 -7
  • Grade 5
  • Standard to be assessed 4.1.5 B.3 Use an
    efficient and accurate paper-and pencil procedure
    for division of a 3-digit number by a 2-digit
    number.
  • Item A gallon contains 128 ounces. Paul wants to
    divide 3 gallons of apple cider equally among the
    2 dozen friends at his party. How many ounces of
    apple cider will each friend receive? (answer
    16)
  • This standard cannot be assessed with a
    calculator active item. Nor could this standard
    be assessed with a multiple choice item as
    students could multiply the answer choices by 24
    until they get 384.

44
  • Grade 6
  • Standard to be assessed 4.4.6 B.2 Determine
    probability using intuitive, experimental, and
    theoretical methods (e.g., using model of picking
    items of different colors from a bag).
  • Item A bag contains 5 red marbles, 8 blue
    marbles, and 7 green marbles. Arturo reaches into
    the bag and removes one marble. What is the
    probability that marble is red? (correct answers
    .25, ¼, 5/20, or 25)
  • Note SCR item allows a child to express the
    probability in the form they are most familiar
    with, not having to select from a form they may
    not be familiar with.)

45
  • Grade 7
  • Standard to be assessed 4.3.7 D.2 Solve simple
    linear equations informally and graphically.
    Multi-step, integer coefficients only (although
    answers may not be integers). Using
    paper-and-pencil, calculators, graphing
    calculators, spreadsheets, and other technology.
  • Sample Item What is the value of x, if 3x-513.
    (correct answer 6)
  • Note Correct solving procedures cannot be
    assessed using a multiple choice format as the
    test taker can merely plug the answer choice into
    the given equation until they create a correct
    mathematical sentence.

46
Other possible uses of SCR items Standard 4.2.5
E.1 and 4.2.6 E.1 Use a protractor to measure
angles. The SCR format will allow scorers to
accept an interval of answers (/- 5) to account
for differing motor skills, etc. Other sample
areas to be assessed with SCR Items estimation,
linear vs. quadratic structure, knowledge items,
algebraic transformations, etc.
47
Standards Clarification Project
  • Under direction of Dr. Grant Wiggins
  • Robert.riehs_at_doe.state.nj.us
  • Process produced
  • Essential Questions
  • Enduring Understandings
  • Area of Focus
  • Explanatory Bullets, Examples, and Comments
  • Areas pf Focus Across Grades

48
Teacher Concerns/Testing Notes
  • Measurements may be used to make geometric
    conclusions
  • Starting to use tick marks to show congruence
    at grade 5
  • Range of data is a value. The range of 10, 12,
    13, 26 is 16.
  • Probability- a student may assume equally likely
    events unless told otherwise. Items dont use
    draw or select.

49
  • Multiplication Symbols
  • 5th grade 3x36, (_)(_), x
  • 6th and 7th grades 3x36, (_)(_), x, dot
  • CPIs are exhaustive
  • CPIs across grades may be assessed at the lowest
    grade in which they occur

50
Opportunities for Educator Involvement
  • Item Review Committee
  • PLD Development Committee
  • Standard Setting Committee
  • The p-scores do not take into account the
    fact that the test is given three-quarters of the
    way through the year. Every teacher addresses
    standards in different sequence. This factor must
    be separate from p-scores for individual items.

51
Sample Items
  • New test design will allow for the release of 1
    ECR item with scoring rubric and exemplar papers,
    4 MC items with related statistics, and 1 SCR
    item per year

52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
Suggestions
  • Take a card. Email or call with any questions,
    concerns, or comments.
  • Thank You-
  • Michael Luke
  • michael.luke_at_doe.state.nj.us
  • 609-984-9637

56
(No Transcript)
57
(No Transcript)
58
(No Transcript)
59
(No Transcript)
60
(No Transcript)
61
Observations
  • Standard 1 Estimation ex 36 x 7 40 x 10.
    Students are rounding single digit numbers.
  • Standard 1 Make sure students understand the
    difference between 3 groups of 4 and 4 groups of
    3.
  • Standard 1 Extend the use of fractions beyond
    pies, cakes and pizza slices (ie containers
    being ½ full, ¼ full, etc.)

62
Observations
  • Standard 2 Make sure students are familiar with
    vocabulary of geometric objects (vertex
    vertices).
  • Standard 2 Students understanding of line(s) of
    symmetry
  • Standard 2 Students need to understand the
    difference between perimeter, area, and volume.

63
Observations
  • Standard 3 Understand the properties of the
    arithmetic operations (commutative, identity)
  • Standard 3 Concept of less than, greater than
  • Standard 3 (grade 4) Understand, NAME, and
    APPLY properties of arithmetic operations
    (commutative, associative, identity)

64
Observations
  • Standard 4 (Graphing) students are forgetting
    to give their graphs a label and title when
    prompted to do so.
  • Standard 4 (Graphing) a very frequent error
    that students make is numbering the boxes instead
    of the lines (axes) on the graph.
  • Standard 4 (Graphing) students need to be
    familiar with the different types of graphs (bar
    graphs, pictographs), reading tables.
  • Standard 4 (Graphing) (4th grade) line plots,
    line graphs

65
Observations
  • Standard 4 probability vocabulary- more likely,
    equally likely, less likely, 1 out of 5,
    fractions (4th grade)
  • Standard 4 simple counting (combination)
    situations involving charts, trees, diagrams
  • Standard 4 Coloring a map
  • Standard 4 Average (mean), most frequent
    (mode), middle term (median)

66
Observations
  • Standard 3 Patterns and sequences (grades 5-8)
  • Standard 4 simple counting (combination)
    situations involving charts, trees, diagrams
  • Standard 4 Coloring a map
  • Standard 4 Vertex edge graphs

67
SUMMARY
  • Students should answer every item
  • Have students use multiple approaches on multiple
    choice items
  • Students should attempt all parts of constructed
    response items- later parts may not be dependent
    on earlier parts.
  • Proficiency is not Perfection
About PowerShow.com