FCAT Processes and Procedures: How is the FCAT Developed, Proofread and Transcribed?

1
FCAT Processes and Procedures How is the FCAT
Developed, Proofread and Transcribed?

• Weekend With the Experts, October 2006, St.
  Pete Beach
• Suzanne A. Dalton, Supervisor
• Florida Instructional Materials Center for the
  Visually Impaired (FIMC-VI)
• 4210 W. Bay Villa Avenue
• Tampa, FL 33611
• [phone] [phone] [phone]
  fax
• sdalton_at_fimcvi.org

2
Student success in testing situations is based
upon working together!!!
3
The FCAT from my view

• Test items written according to specifications
  which assess benchmarks in Sunshine State
  Standards (SSS).
• Some SSS subject areas are currently under
  revision Check DOE website for latest versions.

4

• Language Arts SSS Revision to State Board of
  Education in October, 2006
• New Standards will impact
• FCAT Reading
• FCAT Writing
• Mathematics SSS Revision to State Board of
  Education in February, 2007
• Science SSS Revision to State Board of Education
  in February, 2008

5
A Related Legislation

• Sunshine State Standards Revision(Section
  1000.041, F.S.)
• Reading K-12
• Proficiency Descriptors Start 6/05 Complete
  9/06
• Basic/Advanced Descriptors Start 6/06 Complete
  12/06
• Language Arts
• Proficiency Descriptors Start 6/05 Complete
  9/06
• Basic/Advanced Descriptors Start 6/06 Complete
  12/06
• Math
• Proficiency Descriptors Start 6/06 Complete
  2/07
• Basic/Advanced Descriptors Start 2/07 Complete
  8/07
• Science
• Proficiency Descriptors Start 6/07 Complete
  2/08
• Basic/Advanced Descriptors Start 2/08 Complete
  8/08

6
FCAT Committees

• Advisory Committee, Sensitivity Committee, Bias
  Committee, Reading Committee, Mathematics
  Committee, Writing Committee and Science
  Committee
• All items are field-tested through their
  inclusion within existing FCATs

7
Meetings with Educators 2005-06

• Meetings held during 38 weeks of the year
• More than 600 educators and other citizens
• 76 different meetings
• Annual report and nomination form coming in next
  month

• Coordinated by the Test Development Center

8
One Year of FCAT Participants
2005 2006

• Advisory - 47 Members
• Interpretive Products
• Assessment Accountability
• Technical
• NRT
• FCAT Reports

• Sensitivity - 35 Members
• Reading
• Mathematics
• Science
• Writing

• Bias - 43 Members
• Reading Mathematics
• Science
• Writing

• Science - 133 Members
• Content Advisory
• Expert Review
• Item Content Review
• Rangefinder
• Rangefinder Review
• Standard Setting

• Reading - 158 Members
• Content Advisory
• Item Content Review
• Passage Review
• Rangefinder
• Rangefinder Review

• Writing - 140 Members
• Content Advisory
• Prompt Review
• Item Content Review
• Rangefinder
• Rangefinder Review

• Mathematics -121 Members
• Content Advisory
• Gridded Adjudication
• Item Content Review
• Rangefinder
• Rangefinder Review

677 participants, 38 weeks of meetings, and 76
different meetings
9
One Year of FCAT Participants
2005 2006

• County Type
• Rural 22
• Urban 38
• Suburban 39

• Ethnicity
• Asian lt 1
• African Am 20
• Caucasian 66
• Hispanic 8
• Other 3

• Region
• Panhandle 23
• East Central 20
• Northeast 18
• South 21
• West Central 16

Gender Female 77 Male 21

• County Size
• Small 24
• Medium 37
• Large 38

10
Braille Version of FCAT

• A team which includes the braille vendor, a blind
  mathematician and two certified teachers of the
  visually impaired, evaluate the individual test
  items which are to be included in each subject
  area at every grade level for the current school
  year.

11
Braille FCAT, cont.

• Potential problematic graphics, items which
  require slight alterations that do not
  significantly change the intent of the skills
  assessed, test items which require picture
  descriptions, etc., are reported to DOE.
• Subject area specialists at the Test Development
  Center review/approve all suggested changes.

12
Braille FCAT, cont.

• A braille vendor from Texas produces the FCAT
  using only Library of Congress-certified braille
  transcribers/proofreaders. DOE requires that all
  tactile graphics are hand-drawn. The braille
  vendor then conducts its own proofreading of the
  FCAT.
• In addition, DOE requires an independent
  third-party proofreading of the FCAT of the
  contracted and uncontracted versions of all FCAT
  grade levels/subject areas.

13
Braille FCAT, cont.

• Finally, blind professionals, with the assistance
  of individuals hired to hand-write answers,
  transcribe the students individual responses
  from braille into print for later scoring by the
  FCAT Test Contractors (Harcourt Assessment
• NRT, CTB-McGraw-Hill SSS).
• This ensures that test scorers have no knowledge
  as to which students used
• braille, which students used large print or
  which students used regular print.

14
What TVIs Need to Know

• Get to know your School Test Coordinator and your
  District Test Coordinator and invite yourself to
  their FCAT Training.
• Schedule a quiet place to test your student.
• Note that our students may need more rest breaks
  (follow directions in Scripts).
• Spend time teaching students test-taking tricks
  such as folding pages so they can quickly
  identify places within the FCAT where they can
  return to review answers to difficult questions.
  This encourages student to continue progressing
  through the FCAT rather than spending too much
  time on one item.

15
What TVIs Need to Know

• Test Administration Manual
• Carefully read the section titled Large Print and
  Braille Materials, Test Administrator
  Responsibilities Before and During Testing
• Ensure that a supply of FCAT calculators, FCAT
  large key/large display calculators and FCAT
  talking calculators is available PRIOR to the
  FCAT administration as these are not packed in
  the box with the tests.

16
What TVIs Need to Know

• Regular FCAT Calculators are needed because
• Some of the FCAT Talking Calculators do not have
  a key for finding the square root of a number AND
  do not have a Change-Sign (/-) Key.
• Prior to the beginning of testing, inform all
  students using the talking calculators that they
  can request the use of a regular FCAT calculator
  for finding the square root of a number if their
  FCAT talking calculator does not have the square
  root key or they need to perform an operation
  involving the Change-Sign Key (/-) involving
  negative numbers. The student must tell TVIs what
  keys to press and then you may read the display
  to the student. TVIs may only perform actions
  authorized by the student.

17
What TVIs Need to Know

• Graphic Aid for Mathematics/Geoboard at all
  grade levels but especially for performance items
  in Grades 5, 8, and 10.
• Abacus at all grade levels.
• Coins BL/VI students are allowed to use real
  coins during math portion of FCAT as determined
  by the IEP Team. Coins are NOT to be used as
  manipulatives for counting. They are ONLY to be
  used to access the print representation of the
  coin such as feeling the ridges of dime/quarter
  or to compare size of nickel vs. penny.
• LP/Braille Ruler included with tests at all
  grade levels (NRT only).

18
What TVIs Need to Know

• FCAT Calculator, FCAT Talking Calculator, FCAT
  Large Key/Large Display Calculator Grades 7-10
  (Math) Grades 8 11 (Science)
• FCAT Mathematics Reference Sheets Grades 6-8
  and Grades 9-10 with SSS Grades 7-10 with NRT
• FCAT Science Reference Sheets Grades 8 and 11
• FCAT Periodic Table Grades 8 11

19
What TVIs Need to Know

• Preparing the regular print answer document
• If the student has a preidentified regular print
  answer document, it may be used after its
  accuracy is verified.
• If the student has a label, it may be affixed to
  an appropriate non-preidentified regular print
  answer document.
• If the student does not have a label or a
  preidentified regular print answer document, the
  student grid sheet of an appropriate regular
  print answer document must be gridded by hand.
  Complete all applicable categories especially the
  Primary Exceptionality (VI), and ESE/504
  Accommodation Types.

20
What TVIs Need to Know

• Multiple Forms of the FCAT are printed at each
  grade level for all subject areas.
• Only the first answer document in each series
  (Answer Book AA, Answer Book A or Form 1) should
  be used when recording student answers on the
  SSS. Sample Answer Book covers in packet. (The
  next slide will distinguish between recording and
  transcribing.)
• Check for accuracy because the Student PreID
  label may have been printed/placed on an Answer
  Book that is not identified as AA, A or Form 1.

21
What TVIs Need to Know

• Large Print Scripts
• Before administering the FCAT to students using
  large print materials, modify the Scripts that
  are included within the Test Administration
  Manual.
• Modifications may change year-to-year, but may
  include adjustments such as removing all
  references to students breaking seals on their
  test materials replacing test book or answer
  book with test and answer book ignoring the
  Directions for Completing the Response Grid and
  omitting directions for removing the perforated
  reference sheets. See your 2007 Test
  Administration Manual for specifics!

22
What TVIs Need to Know

• Braille Scripts
• The Braille Scripts AND the Braille Notes are
  provided separately with the test materials. It
  is advantageous to the student if the TVI can
  read through the Braille Scripts and the Braille
  Notes prior to administering the FCAT. This will
  allow the TVI to use the information contained in
  them during the test administration, as
  appropriate.

23
(No Transcript)
24
What TVIs Need to Know

• Transcription of Student Responses
• TVIs MAY NOT TRANSCRIBE student responses
  however, TVIs may RECORD responses that a student
  prepares using a Graphic Aid for Mathematics or
  real coins.
• Please remember that RECORDING responses is not
  the same as TRANSCRIBING responses.

25
What TVIs Need to Know

• LP and BRL documents containing the original
  student responses should be sent back in the
  large Return Envelopes.
• Complete the front of the Return Envelope
  according to the instructions on the envelope.
• Do not place more than one students test
  materials in the same envelope.
• Return all materials to the school coordinator.

26
LARGE PRINT BRAILLE
27
Process Improvements

• Documents sent for transcription within 24 hours
  of receipt
• Enhanced document tracking
• Integration improvements across multiple
  processing sites
• Established cut off date for inclusion in census
  reporting

28
Handscoring Quality Controls for all students

• Florida educators set scoring criteria and select
  material to train scorers
• Scorers receive high-quality training under the
  supervision of DOE
• Scorers must qualify to score by demonstrating
  accuracy and consistency in matching scores set
  by educators

29
Handscoring Quality Controls for all students

• Scoring is monitored by DOE staff (onsite for
  first training and first weeks of scoring)
• Scorers that drift are retrained and/or
  dismissed. Scores assigned by them are deleted
  from the system
• All student responses are scored independently by
  two scorers. Discrepant scores are resolved
  with a third scoring by expert scorer

30
Handscoring Quality for all students Assurance

• More Statistics
• 2,500 Handscorers and/or Team Leaders
• 40 Handscoring Managers Scoring Directors
• 400 CTB support staff Research, Technical,
  Publishing, and Security
• 7 Handscoring sites with 500,000 square feet
• Over 2,000 computer scoring stations
• 47 scanners and over 150 other types of hardware
  dedicated to FCAT

31
Guidelines for FCAT Accommodations Rule
6A1.0943, FAC

• Accommodations are identified and recorded on
  students IEP or Section
• 504 plan
• Students are given ample opportunity to receive
  instruction in testing procedures and practice in
  test-taking strategies
• Accommodations provided to the student on the
  FCAT should not be new to the student

32
Unique Accommodations... (Rule 6A-1.0943(4),
FAC)

• Usually involves alterations to existing test
  materials
• Recorded on IEP or 504 plan
• Used in classroom instruction
• Does not invalidate the purpose of the test
• Approved on a case-by-case basis by the
  Commissioner of Education

33
Possible NEW Accommodation for 2007

• DOE is seriously considering the use of text
  Reading software for items other than the reading
  items for the 2007 FCAT (Unique Accommodation)

34
Unique Accommodations Request Process

• District Assessment Coordinators and ESE
  directors will receive a memo and unique
  accommodations request form via email
• Nov. 17, 2006 - Deadline for unique accommodation
  request. If there is an emergency situation after
  the deadline, it will be handled on a
  case-by-case basis.
• The unique accommodation request MUST be signed
  by District Assessment Coordinator AND the ESE
  Director and sent to the Florida Department of
  Education.

35
Resources

• New Resource - Question and Answer Document on
  FCAT Accommodations for Students with
  Disabilities (copy in packet)
• Web site http//www.firn.edu/doe/bin00014/fcata
  sd.htm

This site contains the many publications that
have been disseminated related to accommodations
for students with disabilities.
36
A Assessment(Section 1008.22)

• FCAT Waiver
• Accommodations that are not allowable on the FCAT
  may be used in the classroom if included on the
  students IEP
• Students using instructional accommodations in
  the classroom that are not allowable
  accommodations on the FCAT may have the 10th
  Grade FCAT requirement waived (student must meet
  other waiver requirements)

37
FCAT Waiver

• Must complete form Waiver of FCAT Graduation
  Requirement for Students with Disabilities during
  students senior year.
• Student must have demonstrated mastery of Grade
  10 SSS and must have taken the Grade 10 FCAT at
  least twice.
• Copy of Waiver Form is in your packet.

38
FCAT Waiver Use

• In 2003-04, FCAT waiver forms were submitted for
  2,593 students from 65 school districts/FSDB
• As of February 17, 2006, 62 districts and FSDB
  reported the completion of 2,744 waiver forms for
  the 2004-05 school year, representing a growth of
  6 in the number of waiver forms
• Figure 1 More students receive waivers in
  reading than in math

39
Waiver vs. Special Exemption

• Rule 6A-1.09431, FAC
• District School Superintendent makes a request to
  the Commissioner for an exemption under
  extraordinary circumstances that would cause the
  test to reflect the students impaired sensory,
  manual, or speaking skills rather than the
  students achievement.
• Checklist included in packet.

40
FCAT Communication ChainTin cans as a
communication method are NOT recommended!
41
FCAT Problems Who Are You Going to Call?TVIs
should contact their School Test Coordinator
Who will call District Test Coordinator Who
will call DOE (as appropriate)
One Ringy Dingy...
Two Ringy Dingy...

More

More

More...
42
Whats New for 2006-07

• Online Comment Forms
• All comment forms will be completed online at
  www.ctb.com/fcat
• Packaging Video
• DOE and CTB are designing a packaging video for
  training purposes
• Test Group Codes
• Reading, Math, Science, and Writing
• Grades 5 Adult

43
Spring 2006 Recap TEST SECURITY/INVESTIGATIONS

• Thank you
• Congratulations
• Hats off
• Guard up

44
Test Security looking forward

• Test Group Codes
• Enhanced Check Proctor Analysis
• Permanent Revocation
• Increased District Monitoring

45
FCAT Parent Network

• A pioneering effort
• Only available for those Pre-ID
• Only available for Reading and Math
• Considerations for 2007 include adding Writing
  and Science
• NRT will not be available
• Recent survey to gather input into the
  implementation process

46
FCAT Parent Network
Available for approximately 1.5 million students
47
Theres nothing in the middle of the road but
yellow stripes and dead armadillos. --Jim
Hightower