DAC Academy

1
DAC Academy

• September 11, 2008
• Colorado Department of Education
• Unit of Student Assessment

2
District Assessment Coordinator

• You are it!

3
What is a DAC?

• DACs ensure quality, accuracy, communication and
  training for assessment administration in every
  district in Colorado.

4
What is a DAC?

• Appointed by district superintendent
• Responsibilities and powers may vary
• CDEs requirements of the DAC
• Educational or psychometric background
• DAC official point of contact
• Management of materials
• Ethical administration of the assessment
• Account for any and all information to USA/CDE
  and USDoE
• Responsible for training licensed educational
  professionals
• Determines misadministrations
• Personnel Issues

5
DAC Responsibilities and Powers

• Vital leadership role
• Communication Cycle
• Secure Information
• Secure Data
• One DAC (One official district email address)
• Required Training
• The first question we ask those who call us
  Have you talked to your DAC?

6
Big Ideas - New for 2008-2009

• Misadministrations
• More information forthcoming
• Preventing Misadministrations
• Procedures Manual
• Test Examiners / Proctors Manuals
• Training
• Security and Chain of Custody
• Training Documentation

7
Big Ideas - New for 2008-2009

• Assessment Accommodations
• Revised manuals
• Research Based
• Video
• General Accommodations
• Spelling of Words
• New rules about the use of a scribe
• Updated Nonstandard Accommodation Request Form
• ELL Accommodations
• Accommodations Monitoring

8
Big Ideas - New for 2008-2009

• New Icons Grade 9 and 10 mathematics

9
Big Ideas- New for 2008-2009

• CSAPA
• 3rd Grade CSAPA
• CSAPA Expanded Accommodations

10
Colorados Assessment System
Unit of Student Assessment
Achievement(Colorado Model Content Standards)
College Entrance
National and State Trends
EnglishLanguage Acquisition(Colorado ELD
Standards)
Colorado Student Assessment Program
Colorado English Language Acquisition Program
Colorado ACT
National Assessment of Educational Progress
CSAP
COACT
NAEP
CELApro
CELAplace LCEU
CSAPA
Colorado has received FULL APPROVAL on its state
standards and assessment system from the US
Department of Education (USDoE) December 18, 2006
11
Colorado Student Assessment ProgramCSAP

• Purpose Measure student achievement relative to
  the Colorado Model Content Standards
• In place since 1997 (pre-NCLB)
• 98 student participation across all grade levels
  3-10
• 31 Tests measure achievement of Colorado
  Standards
• Reading Grades 3-10
• Writing Grades 3-10
• Mathematics Grades 3-10
• Science Grades 5, 8 and 10
• Lectura/ Escritura Grades 3, 4
• Annual Calendar
• February Grade 3 Reading (Law re CBLA)
• March/April All other CSAP tests

12
CSAP AlternateCSAPA

• Purpose Standards based achievement test for
  students with a significant
  cognitive disability
• Eligibility
• Determined through a students IEP Team
• Less than 1 of students
• How many tests
• 27 in grades 3-10 (all subjects tested by CSAP)
• Spanish version available for R/W in grades 3 and
  4
• 2009 Calendar
• February 4 through March 27
• Expanded Accommodations replaces the term
  Adaptations

13
Colorado English Language Acquisition
AssessmentCELApro

• Purpose English Language Acquisition
• NOT to be confused with an achievement test
• Measures four domains of language development
• Speaking, Listening, Reading, Writing
• Academic Language Social, Basic Interpersonal
  Communication
• ALL NEP and LEP K-12
• As defined per LCE Unit definitions
• 2009 Calendar
• January 2nd through February 3rd

14
ACT for Colorado COACT

• College Entrance Exam
• Accepted by U.S. Colleges, Universities, Military
  Academies, and NCAA
• ALL students enrolled in the 11th grade
• Colorado COACT is given
• State test date 04/22/09
• Make-up test date 05/06/09
• Vouchers
• Only for online schools
• 11th Grade Alternate for students eligible to
  take CSAPA.
• Managed by ESLU
• 2009 dates TBD

15
ACT for Colorado COACT
Test Supervisor (TS)
DACs may need to develop a communication process
with Test Supervisors for Accountability.
CONTRACT
DAC
Test Supervisors must develop a communication
plan with Back-up Test Supervisors and Test
Accommodations Coordinators.
16
National Assessment of Educational ProgressNAEP

• 2009 National (1 million students) and State
  Sampling of
• grades 4 and 8 (CO 280 schools)
• Only National Sampling of grade 12 in CO (15
  schools)
• Operational Reading, Math and Science two
  cognitive sessions of 25 minutes each, background
  questionnaire - 10 to 15 minutes
• Pilot/Probe in Civics, U.S. History, Geography,
  Reading and Math same timing as Operational
• Pilot/Probe in Sci. Hands on Tasks (HOTs) or Sci.
  Interactive Computer Tasks (SICTs) HOTs - two 40
  minutes- hands on science kit tasks and
  background questionnaire SICTs - on
  NAEP-provided laptops in disconnected mode, two
  40 minute blocks and background questionnaire.

17
National Assessment of Educational ProgressNAEP

• Participating schools selected by National
• NAEP
• All schools identified last May- close to 300
  selected
• All participating schools have received initial
  notification from NAEP State Coordinator
• Receive state results (grades 4 8), national
  (grade 12) by variables and background
  questionnaire responses. Do not receive
  disaggregated results for districts or individual
  schools- not designed for this.
• NCLB requirement (relative to Title I-A)

18
National Assessment of Educational ProgressNAEP

• Infrastructure
• NAEP State Coordinator (NSC)
• Works with schools to confirm the assessment
  date
• Provides schools with information about notifying
  parents of participating students
• Responds to questions from the school community
• throughout the assessment period
• Works with the Contracted Assessment Team and
  district/school personnel to ensure a smooth
  process and
• Provides schools with a copy of the NAEP state
  report as
• soon as it becomes available.

19
National Assessment of Educational ProgressNAEP

• Infrastructure continued
• District Contact
• Assists NSC in school communications in order to
  ensure the success of the 2009 NAEP
  administration and to ensure a smooth and
  comfortable assessment for the school(s)/district.
  
• Will receive communications from NSC relevant to
  NAEP in the district and ccs of communications
  with the schools.
• NAEP Contracted Supervisor 
• In charge of all the administration, proctoring,
  handling of the assessments at each school works
  with School Coordinator.
• NAEP School Coordinator
• Works with NSC and the Supervisor to oversee the
  process leading up to and including the actual
  assessment date and tasks associated soon after.

20
The Nations Report Card public web site

• http//nces.ed.gov/nationsreportcard/

21
The Questions Tool
22
Thousands of Released Items

• Sample Item Math Grade 8 MC, medium
• Difficulty
•  
• What is the intersection of rays PQ and QP in
  the figure above?   
• A) Segment PQ
• B) Line PQ
• C) Point P
• D) Point Q
• E) The empty set 

23
Assessment Purposes
Program Evaluation
USA Unit of Student Assessment
Achievement Tests
CSAP CSAPA
Body of Evidence (Program / Placement)
Language Acquisition Test
CELAplace CELApro
24
Unit of Student Assessment Management,
Administration and Data Reporting
Where the data go
AYP (Title I)
State SAR (School Accountability Report)
Colorado Longitudinal Growth Model
Accreditation
AMAOs (Title III)
25

School List (August 31st)
CTB CSAP/CSAPA/CELApro
Student (October)
Online Enrollment (November)
Updated Pre-coded Labels (January)
CO ACT (Sept)
Final Label File
Other Test Companies CAP4K
USDOE
CDEs Data Warehouse
SBD
SAR AYP AMAO CEDAR EDEN
Schools/Parents/Students
Growth/ Accreditation
Press Releases
26
Data Operations - Top 5!

• The lifecycle of assessment data
• School Master, Data grids, SBD, reporting,
  accountability
• Student October and ADE collections
• ADE respondent?
• Logistics Training in November online
• Online Enrollments on Navigator in the fall
• New for 2010-11 Race/Ethnicity codes

27
Assessment Life CycleCSAP Example
August/July
Student Biographical Data Review
May/June
Online Enrollments
After
Before
Oct/Nov
Nov
During
Dec
DAC Training
DACs Train SACs
Precoded Labels
28
Colorado Growth Model

• Annual view of progress
• Test scores follow the child

29
Peer Review

• Respond to requirements of USDoE
• Higher Stakes
• Greater Scrutiny
• Accommodations
• Research Based
• Monitoring
• Annual Training
• Security of Materials

30
Critical Dates and Timelines

• Planning for accommodations
• Navigator for CTB
• Early testing window request
• Online enrollment
• Materials requests
• COACT Submission Deadlines

31
Assessment Timeline 2008-09CSAP, CSAPA, and
CELAplace/pro

• Training
• September Online Pre-assessment training
• September ELA, Accommodations, and Procedures
  Manuals
• November Required training for CSAP, CSAPA,
  CELApro, and logistics (i.e. SAC/DAC manuals)
• Ordering and Delivery of Materials
• October Online Enrollments for CELApro
• November Online Enrollments for CSAPA and CSAP
  special populations (no overage for CSAPA this
  year)
• November finalize shipping information for CTB
• May Online enrollments for CELAplace 2008-09
• Test Administration
• October-December Approved Nonstandard
  Accommodation form
• November Early testing window request completed
• January CELApro Administration window
• February-April CSAP and CSAPA Administration
  window
• Data
• September Student October collection opens
  (CELApro Precoded labels)
• December Precoded Labels collection opens (CSAP,
  CSAPA, and COACT
• March Grade 3 CSAP Reading Ncount and CELApro
  Ncount and SBD
• April Final CELApro 2008 Data Release

32
Assessment Timeline 2008-09COACT

• Test Supervisor Checklist of Dates
• Late Sept. 08
• ACT mails initial letter from CDE and School
  renew form to principals with Checklist of Dates
  and Standard Testing Requirements.
• October 08
• Receipt Deadline for principals to return School
  Renewal Forms.
• Receipt Deadline for TS and TACs to return
  renewal forms to ACT
• Early November 08
• TACs receive mailing which includes procedures
  for requesting accommodations, quantities of
  individual request forms, and an order form for
  practice test.
• Online schools receive a special mailing which
  includes National Test Date Voucher Eligibility
  Request and Test arrangements to be made for May
  makeup.
• Mid November 08
• Schools receive quantity of Preparing for the Act
  student test preparation booklets for
  distribution to students who will be testing.
  Quantity based on grade 11 enrollment provided by
  school
• Schools are reminded to submit off-site testing
  requests, if appropriate.
• Late November 2008
• Announcement of training workshop locations and
  dates mailed to TS, BTS, TACs, and DACs.
• December 2008
• Receipt deadline for schools to submit completed
  proposals for off-site testing arrangements to ACT

33
Assessment Timeline 2008-09COACT Cont.

• Test Accommodations Coordinator Checklist of
  Dates
• Late Sept. 08
• Act mails renewal forms to last years appointed
  state testing TACs to verify or update
  information.
• October 08
• Receipt Deadline for TACs to return renewal forms
  to ACT
• November 08
• TACs receive mailing that includes procedures for
  requesting accommodations, quantities of
  individual request forms, and an order form for
  practice test.
• Late November 08
• Announcement of training workshop locations and
  dates mailed to TACs

34
CDE USA support and service

• How we support districts
• Resources
• Comprehensive manuals and guides
• Trainings
• Face to face
• Web ex
• 24/7 availability
• Email and Phone calls
• DAC as single point person

35
Resources

• Procedures Manual (Oct/Nov)
• Policies and procedures relative to
  administration of the states assessments
• Accommodations Manuals
• DAC/SAC Manuals
• Test Proctors/Test Examiners Manuals
• SBD Manual
• Technical Reports
• Data Interpretation Guide

36
USA Procedures Manual

• Includes but is not limited to information
  related to
• Contacts at CDE
• Acronyms
• Assessment fundamentals
• Security requirements
• Ethical administration requirements
• Special circumstances
• Data Grids/Precoded Labels
• CSAPA rating form for each grade level

37
Data Interpretation Guidelines

• Resource for all stakeholders
• Superintendents to parents
• Growth model guidance
• What data from each assessment can be used for
  what questions it can answer
• Dynamic
• First edition
• Please provide us with feedback

38
How to handle advice from sources other than USA

• When you have questions
• Contact Unit of Student Assessment
• Record the name of the staff member you speak
  with
• We cannot verify the accuracy of information
  given by other districts
• Know your lifelines

39
Acronyms

• IMS Information Management Services
• LCEU Language, Culture, and Equity Unit
• LEP Limited English Proficient
• MDB Microsoft Access Database File
• NAVIGATOR CTB.com host system for forms,
  registrations, file sharing, and materials
• tracking.
• NCLB No Child Left Behind
• NEP Non-English Proficient
• PCL Precoded Labels
• .PRN Microsoft Excel Formatted Text (space
  delimited). This is the file format used by ADE
• for all SBD file submissions.
• RITS Record Integration Tracking System. This
  system maintains SASIDs.
• SAR School Accountability Reports
• SASID State Assigned Student Identification
  number
• SBD Student Biographical Data review
• SGL School Group List
• SIMU Student Information Management System. This
  unit manages RITS and SASIDs.
• SOA School of Accountability (or Dist Use B
  removed for 2008).
• SPSS Statistical Package for the Social
  Sciences, a useful file management/analysis
  program.

• ACCOM 1 Accommodation Field for Reading, Math,
  and Science
• ACCOM 2 Accommodation Field for Writing
• ADE Automated Data Exchange
• AMAO Annual Measurable Achievement Objectives
• AYP Adequate Yearly Progress
• BLOCK T CDE is required to collect specific
  student information for COACT. Block T
• refers to a section of the scannable COACT Answer
  Folder.
• CBLA Colorado Basic Literacy Act
• CDE Colorado Department of Education
• CELAplace Colorado English Language Acquisition
  Placement Test
• CELApro Colorado English Language Acquisition
  Proficiency Assessment
• COACT Colorado ACT
• CSAP Colorado Student Assessment Program
• CSAPA Colorado Student Assessment Program
  Alternate
• .CSV Comma Separated Value. This is a common
  data file format.
• CTB California Test of Basic Skills. Better
  known today as the test vendor for CSAP, CSAPA,
• and CELA CTB/McGraw-Hill
• DIST USE A Expelled status
• DIST USE C Free/Reduced Meal status

40
Communications

• USA Emails to DACs
• Emails archived on Navigator
• Scoop - Weekly newsletter sent out to districts
• Includes important deadlines, announcements and
  reminders
• Compiles multi-unit information into one weekly
  communications
• Superintendents
• DACs
• Etc.
• Subscribe on the CDE Communications Unit webpage

41
Questions about items

• If a test proctor or examiner has a question
  about an item
• Contact the SAC who will then contact the DAC
• USA works with the DAC
• We are glad to address important questions about
  the items
• Must include in training instructions to
  proctors
• Do Not copy the item
• Do Not discuss the item (other teachers, etc.)
• Do Not email the content of the item

42
Questions about scores

• If school/district personnel or parents have a
  question about a score
• DAC reviews
• GRT file
• Past CSAP results
• District assessment results
• Irresolvable questions are forwarded to CDE by
  the DAC
• USA works with CTB to examine the data
• Parents cannot view assessments

43
CDEs new look
44
CDEs new look cont
45
CDEs new look cont
46
CDEs new look contURL http//www.cde.state.co.
us/index_stnd-access.htm
47
CTB.com and Navigator

• Secured Site for DACs
• Online training
• Online enrollments and special populations
• Access oral scripts, translated oral scripts,
  teacher read directions for CSAP
• Archived Emails
• Data NCounts, GRT files, official data retrieval

48
CTB Navigator (CSAP, CSAPA, and CELApro)
49
Chain of Custody

• Managing materials and Security
• USDoE peer review requirements

50
Training

• Comprehensive and ongoing training must include
• Test Security
• Test Proctor Requirements
• Teacher versus Test Proctor
• Ethics
• Required documentation
• More information forthcoming

51
Who comprises the core assessment team?

• District Assessment Coordinator
• ELA Director/Coordinator
• SPED Director/Coordinator
• CBLA Contact
• Others

52
DAC Core Assessment TeamELL, SPED CBLA
contacts

• All students eligible to take the various state
  assessments in the assigned grades must have the
  opportunity to be assessed.
• The DAC and the core team will attend required
  trainings, and will communicate about their
  mutual, assessment related work, to ensure
  ethical assessment opportunities for their
  students.

53
The Role of the Core Assessment Team

• Meeting the assessment needs of the students
• Instructional
• Programmatic
• Assessment
• Communication and alignment between instruction
  and assessment
• Best practices

54
Critical Functions of the Core Assessment Team

• These include but are not limited to
• Online Enrollments/Early Test Windows
• (On CTB Navigator)
• Automated Data Exchange Collections
• (CDE Student October, Student Biographical
  Data, etc.)
• Accommodations in place
• Ready for November trainings
• CSAPA and CELApro (face to face)
• CSAP Administration (online only)
• CSAP, CELApro, CSAPA logistics (online only)

55
District Assessment Coordinator

• Oversees and manages all aspects of state
  assessment administration
• Ensures standardized ethical test administrations
• Responsible for ensuring everyone is trained on a
  yearly basis
• Manage data collection processes
• Submitted for assessment purposes
• Particularly important for CELApro and CSAPA

56
ELA Director

• Accommodations appropriate for ELLs (New Colorado
  Accommodations Manual for ELLs)
• Accommodations Monitoring
• Assessing language proficiency
• Designation and Re-designations of ELLs (NEP, LEP
  and FEP) for assessment purposes
• Meeting the needs of ELLs with disabilities
  (Dually identified)
• Distinguishing between language difficulty and
  learning disability
• Language Culture and Equity Unit
• Joanna Bruno (303) 866 - 6870

57
Special Education Director

• Meeting the assessment needs of students with
  disabilities
• Colorado Accommodations Manual
• Matching specific need with specific
  accommodations in plans
• IEP (Includes eligibility for CSAPA)
• 504
• RtI
• Accommodations Monitoring
• Non-standard Accommodations Requests
• Exceptional Student Leadership Unit (303)
  866-6732

58
CBLA Contact

• New Core Assessment Team Resource
• Body of Evidence
• ILPs
• Accommodations
• Literacy Grants and Initiatives [phone]

59
District Trainers

• Yearly required training
• District responsibility
• New Training Materials
• Accommodations Video

60
Expanding your Core Assessment Team

• Districts are unique
• Other staff may need to be included in Core Team
  assessment work
• ADE respondents
• Superintendents/other District Leadership
• SASID/RITS respondents
• Lead Teachers, etc.

61
What is an ADE respondent?

• District staff responsible for submitting
  accurate data for various data collections
  conducted through the ADE system
• Assessment-related collections
• Student October (November)
• Precoded Labels (January)
• CELApro Student Biographical Data (March)
• CSAP/CSAPA Student Biographical Data (June)
• COACT Student Biographical Data (July)

62
Reporting requirements-

• Labels in permanent record
• transcripts at HS level
• CRS 22-7-409-1.9
• The results of the assessmentsshall be part of
  the students permanent academic record.
• Parents receive reports in a timely manner
• All schools in your district receive of reports
  in a timely manner
• Charters and Online schools included

63
Housecleaning

• USA recommends signed statements on file in
  district
• Every principal signs ensuring that there are no
  assessments or proprietary materials related to
  the state assessments stored anywhere on the
  school premises
• Ensure that any teacher created materials
  (whether distributed or not) are free of actual
  assessment items
• (Does not apply to CSAP released items)
• DACs need a methodology to document this has been
  taken care of.

64
CELApro Standard Setting

• October 14th through October 16th, 2008
• Participants Needed!
• Application available at http//www.cde.state.co.u
  s/cdeassess/documents/csap/assess_events.htm
• Purpose set cut scores on the test scale for
  each proficiency level due to handful of Colorado
  items.
• Content experts provide input about the content
  that students at each proficiency level should
  know

65
Standards Review

• The Colorado State Board and the Department are
  conducting a third party analysis to refresh the
  standards as well as to bridge pre-school and
  post-secondary expectations with the K-12th grade
  program.
• Common definitions of 21st century skills,
  college readiness, adequate career and workforce
  skills.
• An analysis of the other 49 states and several
  nations grade span architecture
• identify at each grade (or grade span) what
  students should know beginning at Pre-school
  through the beginning of any post-secondary
  experience.
• Analyze the current standards for rigor, clarity,
  balance and relevance.
• A gap analysis by a nationally recognized third
  party which will inform Coloradoans of the
  differences between and among the best states and
  international content standards.
• A trend study will be done of such recent reports
  as the Colorado Alignment Council, Education
  Trust, Achieve, Fordham Institute, etc. which
  will indicate common and repeated recommendations
  for all of our content areas.

66
Standards Review

• The review will modernize the core content,
  articulate new skills and attempt to tighten the
  focus of a viable scope of expectations.
• http//www.cde.state.co.us/cdeassess/documents/OLR
  /stand_rev.html

67
Office of Standards and Assessments
[phone]

• Standards
• Jo OBrien Assistant Commissioner
• Angela Norlander Standards
• Anna Huffman - Standards
• Student Assessment
• Beth Celva Director
• Holly Baker CSAP
• Dena Coggins CSAPA
• Marisol Enriquez CELApro
• Jim McIntosh Data Operations
• Margaret Lake Data Operations
• Pam Sandoval NAEP
• Fabian Maes - COACT
• Research and Evaluation
• Dianne Lefly Director
• Bill Bonk Longitudinal Growth