OSPI Science Assessment Update - PowerPoint PPT Presentation

Loading...

PPT – OSPI Science Assessment Update PowerPoint presentation | free to view - id: 6f76a4-OTYwN



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

OSPI Science Assessment Update

Description:

The State of Material on 2011 Science MSP OSPI Science Assessment Update WERA, December 2010 – PowerPoint PPT presentation

Number of Views:51
Avg rating:3.0/5.0
Slides: 49
Provided by: cind1167
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: OSPI Science Assessment Update


1
The State of Material on 2011 Science MSP
  • OSPI Science Assessment Update
  • WERA, December 2010

2
Science Assessment Office
  • Elaine WoodSecondary Specialist
  • elaine.wood_at_k12.wa.us
  • Kara MonroeElementary Specialist
  • kara.monroe_at_k12.wa.us
  • Troi WilliamsAdministrative Assistant
  • troi.williams_at_k12.wa.us

3
Agenda
  • 2011 Measurements of Student Progress
  • Standard Setting
  • Resources available
  • Lessons learned from scoring

4
2011 Measurements of Student Progress
  • Grades 5 and 8
  • First assessment of the K-12 Science Learning
    Standards (adopted June 2009)

5
Big Ideas in 2009 Science Standards
EALRs 1-3Crosscutting Concepts and Abilities EALR 4 Domains of Science
EALR 1 Systems Physical Science
is a way of thinking that makes it possible to analyze and understand complex phenomena. Force and Motion Matter Properties and Change Energy Transfer, Transformation and Conservation
EALR 2 Inquiry Earth and Space Science
is a process of asking and answering questions about the natural world that forms the bedrock of science. Earth and Space Earth Systems, Structures and Processes Earth History
EALR 3 Application Life Science
is about the interaction between science and technology, and how both can help solve real-world problems. Structures Functions of Living Systems Ecosystems Biological Evolution
6
Science Assessment Development Cycle
Scenario Item Writing Committee
Develop Test and Item Specifications

Content Review Bias/Sensitivity Committees
Scenario Items reviewed for content by
OSPI/contractor

Pilot Range Finding Committee
Scenario Item Pilot Testing
Items available for Operational Testing
Data Review Committee
Pilot Scoring
Operational Scoring
Operational Range Finding Committee
7
Scenarios and Items
  • Scenarios provide context for sets of items.
  • Systems, Inquiry, and Application
  • Physical, Earth/Space, and Life Science
  • The items associated with a scenario cover a
    range of standards range of cognitive
    complexity.
  • Items not connected to a scenario are called
    Stand Alone items, which is clearly marked for
    students.

8
From WASL to Measurements of Student Progress
Grade 5 2009 WASL 2010 MSP 2011 MSP
Multiple Choice 21 26 22-25
Completion NA pilot only 1-4
Short Answer 8 4 4
Extended Response 0 0 0
Points from MC/CP 57 76 76
Embedded pilot items makes the total number of
items 35.
9
From WASL to Measurements of Student Progress
Grade 8 2009 WASL 2010 MSP 2011 MSP
Multiple Choice 28 30 26-29
Completion NA pilot only 1-4
Short Answer 7 5 5
Extended Response 2 0 0
Points from MC/CP 56 75 75
Embedded pilot items makes the total number of
items 40.
10
Test Specifications Grade 5 8
  • The test map designates the following
    proportions 
  • At least 20 Systems (with connected EALR 4
    content)
  • 30 Inquiry
  • 20 Application (which will apply EALR 4 content)
  • At most 30 EALR 4 domain knowledge that is not
    associated with a cross-cutting concepts or
    abilities
  • EALR 4 content may be assessed in questions
    that only assess EALR 4 or in questions that cut
    across both EALR 4 and a cross-cutting concept or
    ability (Systems, Inquiry, Application).

11
More on Science MSP
Test Length Grade 5 Single session 75-90 minutes Grade 8 Single session 90-110 minutes additional time is needed for passing out materials and reading directions
New Item Type Completion worth 1 score point
Test Window May 2 to May 19 for paper/pencil tests May 2 to June 3 for online tests
Online Testing Voluntary in grades 5 and 8 in 2011 Goal 25
12
Standard Setting Summer 2011
  • Following the process used by Mathematics in
    grades 3-8 last year.

13
Science Standard Setting Plan
Date Event
June 2009 New Science standards approved
May 2010 Spring 2010 MSP New items piloted
Aug 2010 Test Map meeting
Sept 2010 Performance Level Descriptors (PLDs) developed
Oct 2010 Spring 2011 tests built








14
Science Standard Setting Plan
Date Event
June 2009 New Science standards approved
May 2010 Spring 2010 MSP New items piloted
Aug 2010 Test Map meeting
Sept 2010 Performance Level Descriptors (PLDs) developed
Oct 2010 Spring 2011 tests built
Feb/Mar 2011 Teachers from across state trained on how to use PLDs to participate in Contrasting Groups Study (CGS)
Mar/Apr 2011 NTAC and SBE formally approve standard setting plan
April 2011 Teachers participate in CGS by providing ratings of students
May 2011 Spring 2011 MSP administered




15
Science Standard Setting Plan
Date Event
June 2009 New Science standards approved
May 2010 Spring 2010 MSP New items piloted
Aug 2010 Test Map meeting
Sept 2010 Performance Level Descriptors (PLDs) developed
Oct 2010 Spring 2011 tests built
Feb/Mar 2011 Teachers from across state trained on how to use PLDs to participate in Contrasting Groups Study (CGS)
Mar/Apr 2011 NTAC and SBE formally approve standard setting plan
April 2011 Teachers participate in CGS by providing ratings of students
May 2011 Spring 2011 MSP administered
Aug 2011 Standard Setting events --Grade level panels (use Bookmark process, start with teacher ratings)
--Policy panel recommendations
--NTAC certifies the planned process was followed
Aug 2011 SBE Sets Achievement Standards (Reviews all recommendations)
16
Contrasting Groups method affords broader
involvement in standard setting
  • Everyone in the state who teaches science in
    grades 5 and 8 is invited to help set the
    performance standards.
  • Performance Level Descriptors (PLDs) are
    developed for Basic and Proficient
  • Web-based training on PLDs is provided to
    participating teachers
  • In April, participating teachers rate each of
    their students as being Basic or below or At
    or above Proficient
  • Two distributions of scores are produced Scores
    obtained by students rated as Basic, and scores
    for Proficient
  • Where the two distributions merge represents the
    separation point between Basic and Proficient.

17
Students rated as Basic or below using criteria
in PLD for Basic
Number of students
Raw score
points
18
Students rated as Proficient or above using
criteria in PLD for Proficient
Number of students
Raw score
points
19
Intersection indicates a region for where
Basic separates from Proficient
Number of students
Raw score
points
20
PLD training
  • Available via Moodle, February 1, 2011
  • Learn to use evidence to evaluate 5th or 8th
    grade students proficiency (other grade levels
    can participate)
  • Using Moodle website
  • Short videos of presentations
  • All documents available for download
  • Can accomplish all at once, or in multiple
    sittings
  • Free clock hours (3)
  • Encourage groups of teachers to participate
    together
  • To sign up, go to link and create an account
  • http//moodle.ospi.k12.wa.us/course/view.php?id19

21
Contrasting Groups Study
  • Available via EDS, April 11-29, 2011
  • After completing the PLD Training, 5th or 8th
    grade teachers can predict their students
    performance on the Science MSP
  • 5th and 8th grade science teachers can gain
    access to the EDS application by contacting their
    District Assessment Coordinator

22
Resources Available
  • On the OSPI website
  • www.k12.wa.us/Science/

23
OSPI Science Assessment Webpage
www.k12.wa.us
www.k12.wa.us/science
www.k12.wa.us/Science/Assessments.aspx
Select Science Assessment from Assessment
Drop-down
Select Assessments
24
Science Assessment Resources
  • Test and Item Specificationsmore information
    about
  • Types of scenarios
  • Test map
  • Item specifications
  • Vocabulary lists
  • Cognitive complexity

25
Test and Item Specs
www.k12.wa.us/Science/Assessments.aspx
www.k12.wa.us/Science/TestItemSpec.aspx
Select Test and Item Specifications
Select a Grade
26
Test and Item Specs
www.k12.wa.us/Science/TestItemSpec.aspx
Pages 5-14
Page 22
27
Science Assessment Resources
  • Updates for 2011each grade level contains
  • General information on the 2011 test
  • Sample items for classroom use
  • Scoring information, including rubrics and sample
    student responses

28
Updates for 2011
www.k12.wa.us/Science/Assessments.aspx
http//www.k12.wa.us/Science/EducatorResources.asp
x
Select Educator Resources
Select a Grade
29
Science Assessment Resources
  • Online Testing Website http//wa-online.caltestin
    g.org/
  • Demo
  • Flash player
  • Tutorial
  • CAL testing program must be installed
  • Practice items, same as Update for 2011 documents
  • Both tools help students become familiar with
    testing platform, and tests the computer system

30
CAL online testing
http//wa-online.caltesting.org
Stand-Alone item from 8th grade tutorial
Select Downloads or Demos
31
CAL online testing
Short answer item from 5th grade tutorial
32
CAL online testing
Short answer item from 5th grade tutorial
Same item scrolled down to show symbols button
and symbols tool pop-up.
33
Science Assessment Resources
  • Teacher Resource tooltests based on previous
    standards
  • Types of scenarios
  • Generic questions
  • Item specifications (previous standards)
  • Cognitive complexity
  • State-wide statistics

34
Teacher Resource Tool
www.k12.wa.us/Science/Assessments.aspx
www.k12.wa.us/TeacherResourceTool2009-10/default.a
spx
Select Teacher Resource Tool
Select Science
Select a Grade and a Year
35
Teacher Resource Tool
Type of scenario
Generic question, rolls over to item specification
Percent correct or mean score
36
Science Assessment Resources
  • Powerful Classroom Assessments (PCAs)currently
    linked to WASL released scenarios and items
  • Sample items
  • Scoring information, including rubrics
  • New PCAs will be developed from future released
    items

37
Powerful Classroom Assessments
www.k12.wa.us/Science/Assessments.aspx
http//www.k12.wa.us/Science/EducatorResources.asp
x
Select Educator Resources
http//www.k12.wa.us/Science/PCAs/default.aspx
Select PCA
Select grade level
38
Lessons Learned from Scoring
39
Transition to new standards
  • Grade 5
  • Grade 8
  • Terms
  • system and subsystem
  • inputs and outputs
  • Human needs leading to inventions
  • Weight as a measurement of the force of gravity
  • Terms
  • closed and open systems
  • balanced and unbalanced forces
  • Roles of engineers and scientists in
    technological advances

40
Common misconceptions (gr. 5)
  • When the results dont match the prediction,
    students struggle with how to report honestly
  • Students think the mass of an substance changes
    when the substance breaks or melts
  • Students think that plants receive food from soil
  • Students have misconceptions about what makes
    something a producer in an ecosystem.
  • Students think genetic characteristics are
    acquired from the environment

41
Common misconceptions (gr. 8)
  • Students have trouble comparing a model to the
    real-world system
  • Students think the mass of an substance changes
    when the substance changes state or temperature
  • Students have trouble calculating speed
  • Students dont understand photosynthesis
  • Students struggle to see the how natural
    selection leads to changes in a population over
    time

42
Going well (gr. 5)
  • Variables questions
  • 65 to 90 of the time
  • Students are including data in conclusion items
  • 28-37 get full points
  • Between 30 and 40 give each supporting data
    point
  • Students are attempting energy transfer and new
    procedure items
  • 3.1 to 5.0 blanks

43
Going well (gr. 8)
  • Variables questions
  • 66 to 84 of the time
  • Students are including data in conclusion items
  • 54 get full points
  • Between 55 and 58 give each supporting data
    point
  • Students are attempting new procedure items
  • 7.9 - 10 blanks

44
Constructed responses (both)
  • Students confuse different forces (gravity, push,
    pull, magnetic) with forms of energy (5th light,
    heat, sound, motion, electricity 8th light,
    thermal (heat), chemical, sound, kinetic,
    electrical)
  • Students need more help writing their own
    procedures
  • Changing the manipulated variable indicated in
    the question
  • Measuring the responding variable indicated in
    the question
  • Students need to be more detailed when answering
    application items more specific than it will
    work or it must be broken, so fix it.

45
Upcoming Events
  • Performance Level Descriptors training
  • Via Moodle website (February-March 2011)
  • Scenario Writing Workshops
  • 5th and 8th grade (February 2011)
  • High School (February 2011)
  • Contrasting Groups Study (April 2011)

46
Upcoming Events
  • Operational Rangefinding Committees
  • High School (Olympia, April 2011)
  • 5th and 8th grade (Cincinnati, Ohio, May 2011)
  • Pilot Rangefinding Committees (July 2011)
  • Standard Setting (August 2011)

47
Opportunities for Involvement
  • PEPPER email list
  • PreSALTers Enthusiastically Promoting Powerful
    Educational Resources
  • Get updates about science happenings
  • Sign up today with your e-mail address, or e-mail
    us and ask to be a PEPPER
  • SALT list
  • Science Assessment Leadership Team
  • Anyone can apply
  • Respond to emails to sit on Item Development
    committees
  • Committees are chosen to balance state
    representation and content expertise

48
Science Assessment Office
  • Elaine WoodSecondary Specialist
  • elaine.wood_at_k12.wa.us
  • Kara MonroeElementary Specialist
  • kara.monroe_at_k12.wa.us
  • Troi WilliamsAdministrative Assistant
  • troi.williams_at_k12.wa.us
About PowerShow.com