Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments

1
Developing Monitoring and Pre-Scoring Plans for
Alternate/Alternative Assessments

• Virginia Department of Education
• Division of Student Assessment
• and School Improvement
• August 2010

2
Purpose of Web Conference

• To assist school divisions in developing plans to
  monitor and pre-score Collections of Evidence
  (COE) for the Virginia Alternate Assessment
  Program (VAAP) and the Virginia Grade Level
  Alternative (VGLA) and Course Work Compilations
  (CWC) for the Virginia Substitute Evaluation
  Program (VSEP).

3
Developing Monitoring and Pre-Scoring Plans for
Alternate/Alternative Assessments

• What are monitoring and pre-scoring?
• Why should divisions develop monitoring and
  pre-scoring plans?
• Who should be involved in developing the
  monitoring and pre-scoring plans?
• What components should the monitoring plan
  include?
• What components should the pre-scoring plan
  include?

4
What are monitoring and pre-scoring?
5
Monitoring

• A periodic and systematic review of COEs and
  CWCs while they are in the development process
  with opportunities for feedback and intervention.

6
What Monitoring is Not

• Checking in with the teacher
• Flipping through COEs or CWCs
• Reviewing COEs and CWCs without providing
  feedback

7
Pre-Scoring

• A detailed review of COEs and CWCs to address
  technical errors (i. e., grading, SEI tags) prior
  to scoring with opportunities for feedback and
  corrections.

8
What Pre-Scoring is Not

• Assigning a rubric score to the evidence
• Altering student evidence or student responses
• Judging the instructional quality of the evidence

9
Why should divisions develop monitoring and
pre-scoring plans?
10
Monitoring

• Keeps division staff abreast of the status of
  each COE and/or CWC
• Ensures that teachers are providing instruction
  based on the Standards of Learning (SOL)
• Ensures that instruction is on target with the
  divisions pacing guide

11
Monitoring

• Ensures that technical issues (i.e., grading, SEI
  tags) are appropriately addressed
• Provides the opportunity to address issues with
  ample time for correction and/or intervention

12
Pre-scoring

• Provides a final opportunity to correct technical
  errors and omissions that may result in the loss
  of credit to students prior scoring
• Results in the creation of better COEs and CWCs

13
Who should be involved in developing the
monitoring and pre-scoring plans?
14
Who should be involved in developing the plans?

• Depends on...
• Number of COEs and CWCs
• Available central office resources
• Available building level resources

15
Who should be involved in developing the plans?

• Central Office Staff
• Division Director of Testing (DDOT) or designee
• Special Education Director or designee
• Director of Instruction or designee
• Title III Coordinator or designee

• Building Level Staff
• Principal or designee
• School Test Coordinator
• Special Education Chairperson

16
Questions

• Questions may be submitted by typing your
  question into the chat window. Make sure that
  your questions are sent to Host and Presenter.

17

• What components should the monitoring plan
  include?

18
Critical Planning Information

• Develop a Master Participation List for each
  alternate/alternative assessment program that
  includes...
• Name of student
• Grade level of student
• School
• Content area being assessed by alternate/alternati
  ve assessment
• Teacher(s) responsible for instruction in the
  content area assessed

19
Monitoring Plan Decisions

• Who will monitor?
• How will monitors be trained?
• When will monitoring occur?
• Who will receive feedback?
• How will feedback be given?
• What options are available if corrections are
  needed?

20
Remember!

• Your monitors will provide training and/or
  technical guidance to teachers submitting COEs
  and are not allowed to score these COEs.

21
Who will monitor?

• Staff with content knowledge and
  alternate/alternative assessment knowledge
• Hired consultants
• Special education or testing specialists
• Instructional specialists
• LEP specialists
• School test coordinators
• Title I mathematics and reading specialists
• Experienced teachers
• Building administrators

22
How will monitors be trained?

• Provide access to implementation manuals and
  other resources (i.e. testing memos, VGLA
  worksheets, SEI tags, Curriculum Frameworks)
• Conduct local training for monitors to include
  review and discussion of scoring rules, SOL or
  ASOL information, issues generated from the
  previous years scoring, etc.

23
When will monitoring occur?

• Monthly
• End of each grading period
• At the same time as benchmark testing
• Every other month
• Other

24
Who will receive feedback?

• Teachers
• Principals
• Alternate/Alternative Assessment Leaders
• Central Office Staff
• Trainers/Technical Assistance
• Providers

25
How will feedback be given?

• Post It notes on COEs and CWCs
• Locally created feedback forms given to teachers,
  principals, and/or others
• Meetings with teachers, principals, and/or others
  about monitoring results
• Other

26
What options are available if corrections are
needed?

• Individualized consultation and support
• Peer or small group training sessions
• Division-level professional development
• Access to state and local SOL and/or ASOL
  assessment resources
• Other

27
A Sample Monitoring Planfor 500 COEs

• School-based Review Teams will be created in each
  school.
• DDOT and Special Education Director will train
  School-based Review Teams using implementation
  manuals, VDOE power points, division pacing
  charts, and other resources.
• Teachers submit COEs or CWCs to the School-based
  Review Team at the end of each nine-weeks.

28
A Sample Monitoring Plan

• School-based Review Teams will report their
  finding on each COE or CWC using a locally
  created form. A copy of the form will be
  distributed to the teachers, principals and
  central office staff.
• Central office will dispatch instructional
  specialist and assessment specialist to schools
  in need of additional support and training.

29
A Sample Monitoring Plan
2 School-based Review Teams check COEs every 9
weeks according to Pacing Chart and report to
principals and central office staff
1 Select And Train School-based Review Teams
3 Instructional and assessment specialists dispatc
hed to schools based on 9 Week Reports
30
A Sample School-Based Review Team 9-Week
Monitoring Form Grade 4 Reading
Student 100 - 90 of SOL Evidenced 89 - 70 of SOL Evidenced 69- 50 of SOL Evidenced Less than 50 of SOL Evidenced
Ted Smith X
Kay Blue X
Joe Davis X
Ann Jones X
31
A Sample Teacher Review Form

• VGLA 9 Week Review Sheet
• School-Based Teams
• Teacher ____________________
• Student ___________________________ Grade
  Level______
• Content Area_____________________________________
  ____
• REVIEW for ____ 1st Nine Weeks ____
  3rd Nine Weeks
• ____ 2nd Nine Weeks
  ____ 4th Nine Weeks
• Reviewed By____________________________
  Date________
• Collection Status Address the following
  questions (1) Is there evidence for all of the
  standards for the nine weeks according to the
  Pacing Chart? (2) Does the work submitted align
  with the standards satisfactorily? (2) Does the
  evidence demonstrate student mastery? (3) Has the
  student work been graded accurately? (4) Other?
• Recommendation(s) Be as specific as possible
• Follow-up review needed for implementation of
  recommendations
• ___Yes ____No
• Follow up Review Date ________________

32
Monitoring Questions

• Questions may be submitted by typing your
  question into the chat window. Make sure that
  your questions are sent to Host and Presenter.

33
Pre-Scoring Decisions

• Who will pre-score?
• How will pre-scorers be trained?
• When will pre-scoring occur in relation to
  submission date?
• Who will receive feedback?
• How will feedback be given?
• What options are available if corrections are
  needed?

34
Who will pre-score?

• Persons selected for pre-scoring can not serve as
  scorers.
• Pre-scorers should be knowledgeable about all
  scoring rules and the SOL and/or ASOL content
• Pre-scorers should be detail oriented.

35
Remember!

• Your pre-scorers will provide training and/or
  technical guidance to teachers submitting COEs
  and are not allowed to score these COEs.

36
How will pre-scorers be trained?

• Training should include a list of specific issues
  to look for in each COE or CWS
• grading
• correctly completed SEI tags
• detailed anecdotal records
• captioned photographs

37
Other helpful questions to ask in Pre-scoring

• Is the evidence organized according to the
  scoring worksheet?
• Are all required state and local forms included
  and completed?

38
When will pre-scoring occur in relation to local
due date?

• Select a time to pre-score
• that will give ample time for COEs or CWCs to be
  complete or near completion.
• that will give opportunity to return COEs or CWCs
  to teachers for corrections and returned by local
  due date.

39
Who will receive feedback?

• Teacher(s) submitting COE or CWC.
• Building level principals
• Others

40
How will feedback be given?

• Post-it Notes on COEs and CWCs
• Local created forms given to teachers,
  principals, and/or others
• Meetings with teachers, principals, and/or others
• Other

41
What options are available if corrections are
needed?

• Return COEs to teachers for corrections.
• Assign staff to assist teacher in corrections
  such as grading evidence, organizing evidence,
  completing SEI tags.
• Options may be limited or non-existent due to
  time constraints
• Other

42
Sample Pre-scoring Plan

• Pre-scoring will be conducted by teachers,
  building level staff and central office staff in
  three phases
• Teacher Level
• Building Level
• Central Office Level
• Scoring rules will be reviewed with all persons
  involved in pre-scoring. Checklists will be
  provided.

43
Sample Pre-Scoring Plan

• PHASE 1 Teacher Review
• Conducted April 12-16
• Submitting Teacher reviews COEs or CWCs using
  Teacher Checklists in Implementation Manuals

• PHASE 2 School-based Review
• Conducted April 19 26
• COEs or CWCs using notes and local forms gathered
  
• Building Administrator reviews COEs or CWCs using
  Administrator Checklist in Implementation Manuals

44
Sample Pre-Scoring Plan

• Phase 3 Central Office pre-scoring Team
  Review
• Conducted April 27 May 7
• Central Office team reviews COEs and CWC against
  scoring rules. Findings noted on Post-It notes
  placed on evidence.
• Central Office teams return COEs or CWCs to
  schools to correct errors and omissions.

• COEs or CWCs are corrected by teachers.
• Teachers sign affidavits
• COEs or CWCs submitted to principals or designee.
  Principal or designee signs affidavits.
• COEs or CWC submitted for scoring.

45
Monitoring Pre-Scoring Results

• Reduces or Eliminates COEs and CWCs with
• Missing Evidence or SEI tags
• Unacceptable Evidence (textbooks, homework)
• Inaccurate or Ungraded Evidence

46
Monitoring Pre-Scoring Results

• Eliminates Surprises
• COEs or CWCs not done for students educated
  outside of the division
• Incomplete COEs or CWCs in the division
• COEs or CWCs not driven by IEP, 504 or LEP Plans

47
Questions

• Questions may be submitted by typing your
  question into the chat window. Make sure that
  your questions are sent to Host and Presenter.

48
Thank you for participating in this web
conference.

• You will receive
• an evaluation form from Pearson
• regarding this presentation.
• Please complete and submit.
• We appreciate your feedback.