DAC Academy

1
DAC Academy August 19, 2010 Colorado
Department of Education Unit of Student Assessment
2
DAC Academy 2010
Todays Agenda

• Assessment 101
• Essential Information for 2010-2011
• Lunch
• Standard Accommodations
• Break-out Sessions
• Revised Standards and Assessment System
• School Performance Frameworks
• Non-Standard Accommodations

3
DAC Academy 2010
DAC Roles Responsibilities

• In regards to the administration of the state
  assessments, DACs ensure
• Quality
• Accuracy
• Security
• Documentation
• Communication
• Training

4
DAC Roles Responsibilities

• What is a DAC?
• District Assessment Coordinator
• Appointed by the district superintendent
• Responsibilities and powers may vary
• Need to have a specific and specialized skill
  set/level of expertise
• Educational or psychometric background

5
DAC Roles Responsibilities

• Training/Communication
• Train SACs and Test Proctors (documentation)
• Consistent and reliable distribution of critical
  information to superintendents, teachers, and
  core assessment team members
• Manage Materials
• Maintain security of all materials
• Ensure accurate score reports
• Pre-coded label use and student data grid

6
DAC Roles Responsibilities

• Oversee Administration
• Scheduling
• Ethical practices
• Misadministrations
• Distribution, collection, packing and shipping
• Accommodations are provided and documented
• Document the chain of custody
• Keep the communication lines open

7
DAC Roles Responsibilities

• Failure to implement the responsibilities of the
  DAC role will probably result in some
  catastrophic disaster in your district.
• Whenever you are gone, you need to have a backup.

8
DAC Roles Responsibilities

• You Have a Vital Leadership Role!
• Official point of contact
• - One DAC (One official district email address)
• ONLY district staff member officially designated
  to communicate with the school district, USA/CDE,
  and the assessment vendor CTB
• The first question we ask those who call us
• Are you the DAC?

9
(No Transcript)
10
DAC EMAILS

• The DAC must forward the important information to
  the appropriate people in the district.
• When responding to a DAC email
• DO NOT click Reply
• Send an email directly to the contact person
• Forward the message to the contact person OR
  click on the contact persons email address

11
DAC Academy 2010
Colorado Student Assessment System
Unit of Student Assessment
Achievement(Colorado Model Content Standards)
College Entrance
National and State Trends
EnglishLanguage Acquisition(Colorado ELD
Standards)
Colorado Student Assessment Program
Colorado ACT
National Assessment of Educational Progress
Colorado English Language Acquisition Program
CSAP
COACT
NAEP
CELApro
CELAplace LCE
CSAPA
On December 18, 2006 Colorado received FULL
Approval from the USDoE for our Standards and
Assessment System
12
Assessment Purposes
DAC Academy 2010
Program Evaluation
USA Unit of Student Assessment
Achievement Tests
CSAP CSAPA
Body of Evidence (Program / Placement)
Language Acquisition Test
CELAplace CELApro
13
DAC Academy 2010
Unit of Student Assessment Management,
Administration and Data Reporting
Where the data go
AYP (Title I)
SchoolView
Colorado Longitudinal Growth Model
Accreditation
AMAOs (Title III)
14
DAC Academy 2010
Accountability Contacts
Where the data go
Longitudinal Growth
Accreditation
AYP
AMAOs
SchoolView
AlyssaPearson Donna Morganstern
Bill Bonk
John Condie John Penn
Morgan Cox Genevieve Hale
Bill Bonk
15
DAC Academy 2010
Colorado Student Assessment Program CSAP

• PURPOSE To measure student achievement relative
  to the Colorado Model Content Standards.
• The assessment has been in place since 1997
  (pre-NCLB).
• 98 percent participation rate across grades 3-10
• 31 tests
• Science (Grades 5, 8, 10)
• Reading(Grades 3-10)
• Writing (Grades 3-10)
• Mathematics (Grades 3-10)
• Lectura/Escritura (Grades 3 4)

16
DAC Academy 2010
Colorado Student Assessment Program CSAP
2011 Early Testing Windows 2011 Early Testing Windows
Grade 3 Reading/ Lectura Feb 7 18        
Grade 310 all other tests Feb 28 Apr 1
Those approved for the early testing window have an equal, just earlier, testing window. Those approved for the early testing window have an equal, just earlier, testing window.
2011 Regular Testing Windows 2011 Regular Testing Windows
Grade 3 Reading/Lectura Feb 22 Mar 4
Grade 310 all other tests Mar 14 - Apr 15
17
DAC Academy 2010
Colorado Student Assessment Program Alternate
CSAPA

• PURPOSE Standards based achievement test for
  students with a significant cognitive disability.
  
• Eligibility
• Determined through a students IEP team.
• Less than 1 percent of all students.
• 27 tests (all CSAP grades and content areas)
• Mandatory Training prior to
  testing

2011 Testing Windows 2011 Testing Windows
All grades and content areas Feb 2 Mar 25
18
DAC Academy 2010
Colorado English Language Acquisition Assessment
CELApro

• PURPOSE English Language Acquisition
• NOT to be confused with an achievement test
• Measures four domains of language development
• Speaking, Listening, Reading, Writing
• Academic Language Social, Basic Interpersonal
  Communication
• Given to all NEP and LEP K-12 students
• As defined per LCE Unit definitions
• Mandatory Training prior to
  testing

2011 Testing Windows 2011 Testing Windows
Grades K-12 All four domains Jan 3 Jan 31
19
DAC Academy 2010
Colorado ACT

• College Entrance Exam
• Accepted by U.S. Colleges, Universities, Military
  Academies and NCAA
• Colorado ACT (COACT)
• State test date April 27, 2011
• Make-up test date May 11, 2011
• Accommodations Testing Dates April 27 May 11,
  2011
• All 11th grade students by law
• 11th Grade Alternate for students eligible to
  take CSAPA
• Managed by Exceptional Student Leadership Unit
• 2011 dates TBD
• Contact Dena Coggins, ESLU 303-866-6863
  Coggins_d_at_cde.state.co.us

20
Colorado ACTCOACT
DAC Academy 2010
Test Supervisor (TS)
DACs may need to develop a communication process
with Test Supervisors for Accountability.
CONTRACT
DAC
COMMUNICATION
Test Supervisors must develop a communication
plan with Back-up Test Supervisors and Test
Accommodations Coordinators.
21
CO ACT UPDATES
DAC Academy 2010

• ONLINE Schools
• New changes online schools Two national tests
  date options to online students ONLY
• February, 12, 2011 April 9, 2011
• Student can choose either OPTION
• Student can pay 20 to take April 9, 2011 if they
  registered and miss Feb. 12, 2011 test option

22
DAC Academy 2010
National Assessment of Educational Progress NAEP

• 2011 State and National sample of grades 4, 8 and
  12 (about 937K students nationwide)
• -CO has shortened NAEP window Jan 31- Feb 25,
  2011

Type Grade Subjects
Paper / Pencil 4 8 Reading, Math Reading, Math, Science, NAEP TIMMS
Computer -based 8 12 Writing or Math Writing
TIMSS only- national and state 8 Math, Science
TIMSS/ PIRLS 4 Math, Reading, Science
23
National Assessment of Educational Progress NAEP
DAC Academy 2009

• Participating schools selected by National
• NAEP Statisticians
• Most schools identified last May- Over 300
  selected G4 122, G8 181, G12 8
• Most participating NAEP districts have received
  initial notification from NAEP State Coordinator
• Will receive national and state results for grade
  4 in reading and math and grade 8 in reading,
  math and science (by variables and background
  survey responses.) Do not receive disaggregated
  results for districts or individual schools- not
  designed for this.

24
NAEP-TIMSS, Trends in International Mathematics
Science Study
DAC Academy 2010

• TIMSS window Apr 4 - May 27, 2011

• Grade 8 only in TIMSS
• Two groups national TIMSS (10 schools) state
  TIMSS (50 schools)
• CO one of only 8 states selected for state TIMSS
• CO will get state results compared to over 60
  nations
• Participating schools will get a School Report
• More info to come to districts/schools in these
• assessments

25
National Assessment of Educational Progress NAEP
Roles
DAC Academy 2010
National NAEP Office
NAEP State Coordinator NSC
School Community
Contracted Assessment Team
26
National Assessment of Educational ProgressNAEP-
Relationship between NAEP School Community
DAC Academy 2010
NAEP State Coordinator

• Confirms the assessment date
• Provides schools with info for parental
  notification
• Responds to questions
• Works with district/school personnel to ensure a
  smooth process

District Contact

• Assists NSC in School
• communications
• Works with NSC and the Supervisor to
• Oversee the process

• NAEP School Coordinator

School Community
27
NAEP-The Nations Report Card public web site
DAC Academy 2010

• http//nces.ed.gov/nationsreportcard/

28
NAEP-The Questions Tool http//nces.ed.gov/nation
sreportcard/itmrls/
DAC Academy 2010
29
NAEP-Thousands of Released Items
DAC Academy 2010
Sample Item Math Grade 8 MC, Medium Difficulty
  What is the intersection of rays PQ and QP in
the figure above?    A) Segment PQ B) Line
PQ C) Point P D) Point Q E) The empty set 
30
DATA OPERATIONSYou Live and Die by Your Data!If
you miss any element, you are doomed!
DAC Academy 2010
31

School List (August 31st)
CSAP/CSAPA CELApro (CTB)
Student October
Online Enrollment (Oct/Nov)
Pre-coded Labels CSAP/CSAPA, COACT (January)
OR
CO ACT (ACT)
Final Label Files CELApro, CSAP/CSAPA, COACT
NAEP Other Tests
USDOE
CDEs Data Warehouse
SBD
SchoolView AYP AMAO CEDAR EDEN
Schools/Parents/Students
Press Releases
Growth/ Accreditation
32
Data Operations5 Essentials for DACs
DAC Academy 2010

• Online Enrollments on CTB Navigator this fall
• CELApro
• CSAPA and CSAP special populations
• ADE Respondents
• Pre-Coded Labels
• SBD CELApro, CSAP/CSAPA, COACT
• Logistics Training in November online
• SBD Training in February online
• New for 2010-11 Race/Ethnicity codes

33
DAC Academy 2010
Assessment Data Life Cycle CSAP Example

• Reporting

District Validates their own data prior to final
reports
August/July
Online Enrollments
May/June
Student Biographical Data Review and NCount
After
Oct/Nov
Nov
During
Dec
DAC Training
Pre-coded Labels
DACs Train SACs
34
CTB.com and Navigator
DAC Academy 2010

• Secured Site for DACs
• Online training
• Online enrollments and special populations
• Access oral scripts, translated oral scripts,
  teacher read directions for CSAP
• Non-standard Accommodations Forms
• Short/Add Forms
• Archived Emails
• Data NCounts, GRT files, official data retrieval

35
CTB Navigator (CSAP, CSAPA, and CELApro)
DAC Academy 2010
36
DAC Academy 2010
Core Assessment Team

• Who is it?
• District Assessment Coordinator
• ELA Director/Coordinator
• SPED Director/Coordinator
• CBLA Contact
• ADE Respondent(s)
• Title I and Title III (AYP, AMAO)
• Others

37
DAC Academy 2010
Core Assessment Team

• Their Role
• Are students receiving the services they need?
• Instructional
• Programmatic
• Assessment
• Monitoring
• Collaboration and Communication are key!
• This ensures alignment between instruction and
  assessment.
• Required trainings

38
DAC Academy 2010
Core Assessment Team

• Critical functions include, but are not limited
  to
• Online Enrollments/Early Test Windows
• On CTB Navigator
• Automated Data Exchange Collections
• Through CDE (Student October, Student
  Biographical Data, etc.)
• Accommodations in place THINK EARLY!
• Attendance at November trainings
• CSAP, CSAPA, CELApro (face to face)
• CSAP, CELApro, CSAPA logistics (online only)
• Online administration trainings for CSAP, CSAPA,
  and CELApro will still be available.

39
DAC Academy 2010
Core Assessment Team District Assessment
Coordinator (DAC)

• Oversees and manages all aspects of state
  assessment administration
• Ensures standardized ethical test administrations
• Training
• Manage data collection processes

40
DAC Academy 2010
Core Assessment Team ELA Director

• Appropriate accommodations
• Colorado Accommodations Manual for ELs
• Accommodations monitoring
• Assessing language proficiency
• Designation and re-designations of ELs (NEP, LEP
  and FEP) for assessment purposes
• Meeting the needs of ELs with disabilities
  (dually identified)
• Distinguishing between language difficulty and
  learning disability
• Language Culture and Equity Unit
• Joanna Bruno (303) 866 - 6870

41
DAC Academy 2010
Core Assessment Team Special Education Director

• Meeting the needs of students with disabilities
• Colorado Accommodations Manual
• Matching a specific need by including specific
  accommodations in plans
• IEP (Includes eligibility for CSAPA)
• 504
• RtI
• Instructional Accommodations monitoring
• Work with EL team to meet needs of dually
  identified students
• Exceptional Student Leadership Unit
• Dena Coggins (303) 866 - 6863

42
DAC Academy 2010
Core Assessment Team CBLA Contact

• Body of evidence from assessments
• Programmatic interventions/ILPs
• Accommodations aligned
• Literacy Grants Initiatives
• John Love [phone]

43
DAC Academy 2010
Core Assessment Team District Trainers

• Proctors, staff, faculty and administrators must
  be trained through an interactive format on a
  yearly basis!
• This is true for EVERYONE
• If you are an educational leader, you must step
  up.
• How do you as the DAC know they have been
  trained?
• Training Schedule
• Attendance List
• Copy of Materials
• (November training will be discussed in future
  slides.)

44
DAC Academy 2010
Core Assessment Team Expansion

• Districts are unique
• Other staff may need to be included in the work
• ADE respondents
• Superintendents/other District Leadership
• SASID/RITS respondents
• Lead Teachers, etc.
• Communication is key

45
DAC Academy 2010
Core Assessment Team Automated Data Exchange
(ADE) Respondents

• Who are they?
• Responsible for submitting data for the various
  data collections throughout the year.
• These include
• Student October (November)
• Precoded Labels (January)
• CELApro Student Biographical Data (March)
• CSAP/CSAPA Student Biographical Data (June)
• COACT Student Biographical Data (July)

46
DAC Academy 2010
Reporting Results

• Requirements
• CRS 22-7-409-1.9
• The results of the assessmentsshall be part of
  the students permanent academic record.
• CSAP labels on high school transcripts
• Contact CTB for these
• Parents receive reports in a timely manner
• All schools in each district receive reports in a
  timely manner

47
DAC Academy 2010
CDE/USA Support and Service

• We recognize that you have other duties as
  assigned.
• How we support districts
• Resources
• Comprehensive manuals and guides
• Trainings
• November
• Face-to-face (January)
• Web ex (CTB Navigator)
• -24/7 availability
• E-mail and Phone calls

48
DAC Academy 2010
CDE/USA Support and Service

• Unit of Student Assessment e-mails to DACs
• Try to limit these
• E-mails archived on CTB Navigator
• Scoop
• Weekly newsletter sent out to districts
• Subscribe by contacting the CDE Communications
  Unit at CDE_Communications_Office_at_cde.state.co.us
  
• Includes important deadlines, announcements and
  reminders
• Compiles multi-unit info from CDE for easy access
  to
• Superintendents, DACs, etc. (anyone who
  subscribes)

49
DAC Academy 2010
Assessment Resources

• 2010-2011 Procedures Manual
• 2010-2011 Accommodations Manuals
• 2011 SAC/DAC Manuals
• Test Proctors/Test Examiners Manuals
• SBD Manual
• Technical Reports
• Data Interpretation Guidelines
• Guides to Test Interpretation
• Parent Brochures (CSAP, CSAPA, Accommodations)

50
Questions about Items

• CSAP items are developed and then reviewed by CTB
  Content Editors and Content Leads
• Items are reviewed by the CDE CSAP Consultant
• Colorado Teachers and Educators review the items
  at CVAR (CSAP Item Review)
• CTB and CDE have a Post-CVAR review
• CTB does their internal review of the items
• CDE does a final review of the items in the test
  book
• Colorado Teachers and Educators review how
  constructed response items will be scored at
  Anchor Review
• All test items are the property of the state of
  Colorado

51
DAC Academy 2010
Questions about Items

• Proctors should not be studying items
• Items only accessed during test administration
• If a test proctor or examiner has a question
  about an item
• Contact the SAC, who will contact the DAC
• The DAC contacts the appropriate person at CDE in
  the USA by phone
• We are glad to address important questions about
  the items
• Include in proctor training instructions
• Do Not copy the item
• Do Not discuss the item (with other teachers,
  etc.)
• Do Not email the content of the item

52
DAC Academy 2010
Questions about Scores

• If school/district personnel or parents have a
  question about a score
• First, the DAC should review
• GRT file
• Past results
• District assessment results
• Irresolvable questions are forwarded to CDE by
  the DAC
• USA works with CTB to examine the data
• May possibly call you in to review material with
  us
• Parents cannot view assessments

53
DAC Academy 2010
How to Handle Advice from Sources other than USA

• When you have questions
• Contact Unit of Student Assessment
• Record the name of the staff member you speak
  with
• We cannot verify the accuracy of information
  given by other districts or outside sources
• Know your lifelines

54
DAC Academy 2010

• BREAK
• (15 minutes)

55
Quotes to Inspire You

• The kids are veteran test-takers and experts in
  what it means to be a standardized assessment.
• Train early and train often.
• Have faith, but verify.

56
Standardized Assessment

• All students will have the same test content,
  resources, directions, testing conditions, and
  scoring procedures.
• One score obtained by one student in one part of
  Colorado, will mean the same as the same score
  obtained by another student in a different part
  of Colorado.

57
Media

• The media are not allowed to have access to the
  tests, or take pictures or video of the testing
  materials (including the front or back covers of
  test booklets and packaged boxes of testing
  materials for shipping), during any part of the
  assessment process. Not only is media presence in
  the school disruptive, it creates a non-standard
  assessment environment and can lead to
  misadministrations.

58
Important Procedures Manual

• Please Note Due to the complex nature of a
  standardized assessment process, any practice not
  specifically permitted should be presumed
  inappropriate until and unless specifically
  authorized by the program sponsor (in this case,
  CDE).

59
Whats New for 2010-2011

• 2010-2011 Procedure Manual
• Posted in September
• Accommodations Manual
• Eligibility flowchart for CSAP vs. CSAPA
• Updates with Accommodations
• These will be explained later
• Race/Ethnicity Codes
• The Assessment System is the same as in previous
  year.
• Our goal is to do the same thing this year, only
  better.

60
2010 Ethnicity and Race Reporting
DAC Academy 2010
61
DAC Academy 2010

• Ethnicity and race data will be collected through
  two data fields on the CELApro, CSAP, and CSAPA
  BIO Grids.
• CELApro, CSAP, and CSAPA BIO Grid
• Ethnicity will be indicated by choosing one
  bubble for
• Hispanic/Latino
• Not Hispanic/Latino.
• Race will be indicated by marking all bubbles
  that apply for
• American Indian or Alaska Native
• Asian
• Black or African American
• White
• Native Hawaiian or Other Pacific Islander

62
DAC Academy 2010
63
DAC Academy 2010
64
DAC Academy 2010
Reminders

• Training
• Housecleaning
• Chain of Custody
• Schedules

65
DAC Academy 2010
Peer Review Requirements

• Training
• Inadvertent misadministrations do not occur
• Comprehensive, interactive, ongoing trainings
  must include
• Test Security
• Standardized Environment
• Test Proctor Requirements
• Teacher vs. Test Proctor
• Ethics
• Required documentation
• Individual documentation (DACs keep)
• One form submitted to Unit of Student Assessment
• School administrations must be involved

66
DAC Academy 2010
Peer Review Requirements

• Housekeeping
• USA recommends signed statements on file in each
  district
• Every principal signs ensuring that there are no
  assessments or proprietary materials related to
  the state assessments stored anywhere on the
  school premises (e.g. test booklets from previous
  years)
• Ensure that any teacher-created materials
  (whether distributed or not) are free of actual
  assessment items
• (Does not apply to released items) DACs need a
  methodology to document that this has been taken
  care of at every school.

67
DAC Academy 2010
Peer Review Requirements

• Chain of Custody
• Documented before, during, and after testing
• Large print, Braille, Teacher Read Directions,
  Oral Scripts, Translated Oral Scripts
• Materials should not be delivered to schools more
  than 1 week in advance
• Distribute only the content area being assessed
• Returned to a designated secure location
• Not stored in classrooms

68
Peer Review Requirements

• CSAP Testing Administration Schedules
• The DAC must ensure that all testing follow the
  requirements stated in the Procedures Manual and
  Proctors Manual.
• Allow time for the proctors to read the
  directions from the Proctors Manual and
  distribute materials before and to collect and
  secure materials after the administration of each
  assessment.
• The sessions for each content area must be
  administered in chronological order.

69
Peer Review Requirements

• Multiple grades and multiple content areas must
  not be tested in the same room.
• CDE strongly recommends that the SAC keep on file
  the schedules of exactly when, where and to whom
  each assessment is being administered.
• CDE strongly recommends that each proctor be
  given a clear schedule of when to administer each
  test by content area and session.
• Make sure that accommodation considerations are
  included in the development of the schedule.

70
DAC Academy 2010
Assessment Timeline 2010-2011

• CSAP, CSAPA, CELApro and CELAplace
• Training
• September ELA, Accommodations, and Procedures
  Manuals
• November Required training for CSAP, CSAPA,
  CELApro, and logistics (i.e. SAC/DAC manuals)
• Ordering and Delivery of Materials
• October Online Enrollments for CELApro
• October Online Enrollments for CSAPA
• November Online Enrollments for CSAP special
  populations
• November Finalize shipping information for CTB
• May Online enrollments for CELAplace 2010-2011
• Test Administration
• October-December Submit Nonstandard
  Accommodation forms for approval
• November Early testing window request completed
• January CELApro Administration window
• February-April CSAP and CSAPA Administration
  window
• Data
• September Student October collection opens
  (CELApro Precoded labels)
• December Precoded Labels collection opens (CSAP,
  CSAPA, and COACT
• March Grade 3 CSAP Reading Ncount and CELApro
  Ncount and SBD

71
DAC Academy 2010
Assessment Timeline 2010-2011

• COACT
• Test Supervisor Checklist of Dates
• Late September 2010
• ACT mails initial letter from CDE and School
  renew form to principals with Checklist of Dates
  and Standard Testing Requirements.
• October 2010
• Receipt Deadline for principals to return School
  Renewal Forms.
• Receipt Deadline for TS and TACs to return
  renewal forms to ACT
• November 2010
• TACs receive mailing which includes procedures
  for requesting accommodations, quantities of
  individual request forms, and an order form for
  practice test.
• Online schools receive a special mailing which
  includes National Test Date Voucher Eligibility
  Request and Test arrangements.
• Mid November 2010
• Schools are reminded to submit off-site testing
  requests, if appropriate.
• Late November 2010
• Announcement of training workshop locations and
  dates mailed to TS, BTS, TACs, and DACs.
• December 2010
• Receipt deadline for schools to submit completed
  proposals for off-site testing arrangements to
  ACT
• Early January 2011
• Receipt deadline for CDE Assessment to receive
  National Test Date Voucher Eligibility Request
  for ONLINE students

72
DAC Academy 2010
November Training Schedule

• CSAPA and CELA Training
• 6 Face-to-Face Trainings around CO (Nov 1-9)
• Webinar (Nov 15)
• Every DAC must attend one training
• New DACs must attend a face-to-face one
• We recommend bringing 2 other guests who are
  responsible for CSAPA/CELA training in your
  district.

73
Office of Standards and Assessments

• Maile Rieple Program Assistant [phone]
• Standards
• Jo OBrien Assistant Commissioner
• Angela Norlander Standards
• Anna Huffman Standards
• Student Assessment
• Beth Celva Director
• Andrina Aragon Assessment Policy
• Glen Sirakavit CSAP
• Margaret Lake Data Operations
• Mira Monroe CSAPA
• Liliana Graham CELApro
• Pam A. Sandoval NAEP
• Christine Deines COACT
• Jolene Fukuhara Data Operations