Dos for Assessment Process

1
Dos for Assessment Process

• Margaret Biggerstaff
• Minnesota Department of Education
• margaret.biggerstaff_at_state.mn.us
• 651-582-8511

2
Objectives

• Overview of Calendar
• What Changes Need Attention
• What is the AYP/SI Coordinators Role

3
Calendar
4
Calendar
Posttest Edit Window
MCA-II, MTELL MTAS Testing Ends
Finalize Assessment Data
Assessment Summary Data Available at PEM
Public Release of AYP Results
Minnesota State Fair Begins
AYP Results to Districts
Media Receives Embargoed Data
AYP Appeals Window
School Report Card Data Entry Window
Public Release of Assessment Data
Media Receives Embargoed AYP Results
5
Change 1 Must Edit Data to Appeal AYP Status
In order for a district to be eligible to appeal
its AYP status, it must review its student data
in the Posttest Edit Window, make any corrections
indicated on the discrepancy reports, and
finalize the data. Appeals from districts that do
not take these steps will not be accepted.
6
Change 2 Denominator Different for AYP

• Based on students enrolled during testing window
• Student will only count 1 time in the state
• Can not be counted in multiple districts
• Can not be counted in multiple schools in a
  district
• Can not be counted multiple times in a school
• Can not be matched to an enrollment record in
  MARSS then test record will not be used
• If in MARSS and no test record, will be in
  denominator for participation

7
Change 3 Two Goals for Data Reporting

• In electronic data files, each student receives
  exactly
• 1 mathematics record (grades 3 8 11) and
• 1 reading record (grades 3 8 10)
• Each student in the state receives 1 ISR, either
  MCA-II/MTELL or MTAS, no matter how many
  districts and/or schools they test in.

8
How Does 1 Record in State Look?

• Student centered correction system
• Same demographics for all tests taken by student
• All tests for a student appear together in
  Posttest Edit Window and have same vendor ID
• Duplicate records for students identified across
  the state
• Records will be merged within schools
• Records will be moved and merged across districts

9
What Happens Multiple Records within School

• Same subject with multiple tests
• All tests shown in Posttest Edit Window
• Scoring/reporting rules applied after close of
  Posttest Edit Window
• VS on MTAS trumps MTELL MCA-II
• VS on MTELL trumps MCA-II
• Same subject and same test multiple times
• Record with larger Score 1 Characteristic
  reported in Posttest Edit Window
• If Score 1 Characteristic equal, first scanned
  document

10
What Happens Multiple Records within District

• Both records appear in Posttest Edit Window
  listed as a discrepancy
• District expected to move both records to same
  school
• School where student was first enrolled or
• School listed for primary services for student
• District has option to merge files based on
  school where first enrolled
• MDE will assist in process

11
What Happens Multiple Records Across Districts

• Both records appear in Posttest Edit Window
• MDE will investigate to determine
• District where student first enrolled and
• District which tested first
• MDE will move records to district and school
• District/School where student was first enrolled
  or
• District/School providing primary services
• MDE will merge files

12
Change 4 Progress Score Reported for First Time

• Two years of data available for students
• Match scores to 2006 Test Results Data File when
  possible
• Must have big 5 (MARSS, Last Name, First Name,
  DOB, Gender) matching exactly to receive prior
  score
• Expect to have less than 2000 students who could
  not be matched
• Progress Score calculated using an IRT model with
  vertical linking items for grade 3 - 8
• Range expected to be about 2300 to 5200

13
Change 5 Communication Document (to be released
soon)

• District Assessment Accountability Work Plan
  Summer 2007
• Two reasons for plan are changes in process
• Correcting student data is happening in first two
  weeks of June
• ONLY opportunity to correct data
• Denominator for AYP participation calculations
  will be based on the number of students enrolled
  at the time of testing (April 16 through April
  27, 2007) based on the July 11, 2007 submission
  of MARSS.
• Expected to be larger denominator for most schools

14
Testing Window vs AYP Calculations Window

• Testing Window
• MCA-II/MTELL April 16 to May 4
• MTAS April 2 to May 4
• Students included in AYP Calculations
• Students enrolled in district between April 16
  and April 27

15
Why test a student arriving after April 27?

• Test score this year will provide a baseline
  score for Progress Score in 2008
• May be helping out other Minnesota districts/
  schools for participation
• Student left another district/school before
  completing all the assessments required
  (mathematics or reading)
• If tested in new school, results transferred to
  previous school and merged with other record

16
Communication Plan (cont)

• Work plan begins May 7
• Activities for different groups
• Test Coordinators
• School Assessment Coordinator
• District Assessment Coordinator
• MARSS Coordinator
• Superintendent
• Encourages collaboration of multiple people

17
What about before May 4?

• Test all students enrolled in grades 3 8 with
  only 1 test per subject
• Mathematics MCA-II, MTELL or MTAS
• Reading MCA-II or MTAS
• Test all students in grade 10 with only 1 reading
  test
• Test all student in grade 11 with only 1
  mathematics test

18
What about before May 4? (cont)

• Do NOT double test a student within a school, the
  district or the state

19
What about before May 4? (cont)

• Use precoded documents whenever possible to test
  students on MCA-II
• Use pre-populated students whenever possible for
  all online tests, MTELL MTAS
• Do NOT double test a student within a school, the
  district or the state

20
What about before May 4? (cont)

• If errors in student data exist on precoded
  documents or for students pre-populated for
  online tests
• Do NOT hand-bubble another answer document or
  enter a new student
• Check with MARSS coordinator to see if different
  than what was submitted
• Make a list of all corrections needed for either
• MARSS next submission or
• Test data in Posttest Edit Window

21
Why not new document?

• Think about the chances of making another error
  when bubbling in the following information for a
  student to correct a 2 digits reversal in MARSS
  
• MARSS 0001000000346 (13 chances)
• Name Mary Blue (8 chances)
• Date of Birth 12/23/1999 (8 chances)
• Gender Female (1 chance)
• Total 30 chances
• Each one requires bubbling in the correct space
• Only touch the 2 numbers in the Posttest Edit
  Window

22
What about before May 4? (cont)

• Accurately complete the big 5 on all
  hand-bubbled answer documents or students added
  online
• MARSS
• Last Name
• First Name
• Date of Birth
• Gender
• Do NOT allow students to hand-bubble answer
  documents

23
What does complete accurately mean?

• Two sets of directions released at different
  times and exist today
• Exactly what is submitted in MARSS
• Assures accurate match of test record with
  student for AYP calculations
• Exactly what appeared in 2006 MCA-II data file
• Assures link to prior score for Progress Score
• Which is the best way at this time?
• Exactly what is submitted in MARSS

24
What about before May 4? (cont)

• Check all School/Grade ID sheets for the correct
  district and school information, including
  district number and type and school number.
• If you are not sure about the numbers for your
  district and school, check with district
  personnel or look up the information at
  http//app.education.state.mn.us/Directories/repor
  t_c6.jsp.
• Note Every building usually has a different
  school number.

25
What about before May 4? (cont)

• Make sure every student either
• Has an answer document for every subject
• Has had a finalized test online for MTELL
• Has a finalized test entry for MTAS
• Note Every district/school received a list of
  students expected to test on MTELL and MTAS
• Make sure all students either took the designated
  tests or has an MCA-II answer document
• Nothing can be done after testing window closes

26
What about before May 4? (cont)

• Make sure every grade 9 or 10 student who had ALT
  test code bubbled in on answer document for BST
  has an alternate writing assessment entered
• Same location as the MTAS
• Nothing can be done after testing window closes

27
May 7 to 11 Returning Materials

• Confirm bubbling of hand-bubbled tests
• Confirm additional test codes are correctly
  bubbled in for every student
• Home Schooled
• New-to-Country
• Significant Gap
• Foreign Exchange Student
• Review test codes and accommodation codes on
  every answer document

28
May 7 to 11 Returning Materials

• Check every group of tests with a header to make
  sure it is on top of correct tests
• Return all answer documents as directed
• If 2 test documents exist for 1 student, place
  the one to be scored under the correct header
  (preferrably the precoded one) and place the
  other in an envelope.

29
May 14 to 29 Prepare for Posttest Edit Window

• Work with MARSS coordinator to assure every
  student tested is enrolled in MARSS for June
  MARSS submission
• May want to have the MARSS coordinator print off
  a list of students being submitted
• May want list to include big 5 pieces of
  student information plus the following flags and
  codes
• LEP, Language Code, SPE, Special Ed Code, FRP,
  Oct1 School, Oct 1 District, Ethnicity
• Test data needs to be same as MARSS

30
May 14 to 29 Prepare for Posttest Edit Window

• If DAC
• Attend DAC Meeting on May 16 either in person or
  by Webex
• OR
• Plan to participate in one of 5 Webex to be
  offered on the Posttest Edit Window
• If MARSS Coordinator
• May want to participate in one of 5 Webex to be
  offered on the Posttest Edit Window with DAC
• Make sure superintendent has given authorization
  for necessary staff to access AYP results

31
May 30 to June 2 Posttest Edit Window First
Steps

• Log into Posttest Edit Window
• Notices sent if never logged on to Posttest Edit
  Window

32
May 30 to June 2 Posttest Edit Window First
Steps

• Obtain available reports
• Unknown Status of Students Report
• Enrolled in December
• No test documents found
• May need to identify as NE if left district prior
  to April 16
• New Students Report
• Not enrolled in December
• Check they are enrolled in MARSS
• Daily Discrepancy Report
• Run daily on at least 30 potential discrepancies
• Needs to be dwindled down to nothing
• Electronic downloadable file of students???
• Format being determined at this time not UFF
• Finalized by May 15

33
June 3 to 11 Posttest Edit Window First Steps

• Check Daily Discrepancy Report until it has
  dwindled down to nothing
• Make edits to students as needed
• Sorting features allow to easily find groups of
  students so take advantage of it
• Collaborate with MARSS coordinator to make sure
  June MARSS submission matches edits being made
• Notices sent if never logged on to Posttest Edit
  Window

34
June 3 to 11 (cont) Posttest Edit Window First
Steps

• Make edits of errors in big 5 from answer
  documents or students pre-populated for online
  tests
• Edit known discrepancies between MARSS and
  information in Posttest Edit Window
• Merge students if multiple records in school or
  district
• Notices sent if never logged on to Posttest Edit
  Window

35
June 12 to 15 Posttest Edit Window First Steps

• Complete all edits
• Make copy of report on changes made in Posttest
  Edit Window
• Finalize data
• Indicates you are finished with edits
• Ready to send to scoring reporting phase
• If change of mind, can return to do edits and
  will need to finalize again when finished with
  edits
• Last chance to impact assessment results
• If not finalized by June 13, notices will be sent

36
June 18 to July 11 Cleaning up MARSS

• Review report of June MARSS submission available
  to MARSS coordinators to make sure all students
  are included in MARSS
• Work with MARSS coordinator to make sure MARSS
  July 11 data submission
• Has same big 5 as test data
• Has correct start and end dates for all students
• Has same demographic information as test data in
  Posttest Edit Window
• Last chance to impact AYP calculations

37
July 16 17 Review Assessment Data

• Visit SchoolHouse at 9 am to obtain
• district and school summary reports and
• District Student Results (DSR) file
• Work with other district staff to prepare for
  media questions
• Media have file at 1 pm on July 16
• Make sure authorized staff have registered to
  access AYP results

38
July 18 Public Release of Assessment Data

• Enjoy the outstanding assessment results with the
  rest of the community

39
July 19 August 17 Review AYP Results

• Go to Educator Portal to access AYP results
• Follow steps in work plan for each week until the
  window closes
• Go to School Report Card Data Entry to enter the
  school and district self report information
  requested for the printable School Report Card
  and other reports online at the School Report Card

40
August 22 Media Receives AYP Results

• Districts have had results since July 19 so they
  should be well prepared to respond to the media
  inquiries about their successes on the 2007 AYP
  results

41
August 23 AYP Results Released to Public

• Enjoy the day at the Great Minnesota Get Together!