Qualified Trainer Training For AK AA Mentors

1
Qualified Trainer TrainingFor AK AA Mentors

• October 31, 2006
• BP Energy Center
• Anchorage, AK

Aran Felix, Alaska EEDJerry Tindal and Pat
Almond, Dillard Research Associates
2
Todays Schedule
3
Introductions and Overview of Mentors Qualified
Trainer Role

• Aran Felix
• 9 915

4
GoalsParticipate in 1 day training and meet the
following objectives

• Qualified Trainer Learn about the role that
  mentors will play as Qualified Trainers and the
  conditions for becoming qualified to train and
  certify assessors as qualified.
• Orientation Obtain orientation materials and
  talking points to use in introducing Alaskas new
  alternate assessment to educators.
• Test Administration Obtain training materials in
  the following areas
• Administration Directions
• Conditions of Administration and Decisions
• Decision Paths and Stopping an Assessment Early
  including who takes the Assistive Technology
  Survey
• Guidelines for Student Materials, Adaptations,
  ELOS
• Evaluating Scoring Protocols from the Practice
  Assessment
• Strategies Materials and approaches to coach and
  support protégés as they become qualified to
  administer and score an individual students
  performance on the ExRWM.

5
What is the role of a qualified trainer?

• Local point person
• Provide training and coaching
• Assist in problem solving
• Award assessor certificates

6
A Qualified Trainer

• Develops Awareness
• Provides Knowledge
• Encourages Practice
• Supports Experience

7
Develops Awareness

• Walk through web resources
• Explain conditions of administration
• Describe extended assessments and provide examples

8
Provides Knowledge

• Makes resources available
• Demonstrates how to use them
• Answers questions
• Provides encouragement

9
Encourages Practice
Assessor administers a practice assessment to a
student.
Opportunity to work through online training and
proficiency
10
Supports Experience

• Answer questions about administration and scoring
• Help solve technical problems
• Facilitate using the Internet and obtaining
  needed materials

11
And Award Assessor Certificates
12
QUALIFIED ASSESSOR
13
A Qualified Assessor

• Administers assessment in a standardized manner,
• Scores student responses fairly and reliably,
• Delivers scores to the online data system, and

14
Reviews the Individual Student Report (ISR)

• and interprets results for the student, family,
  and educational team.

15
2006-2007 Mentor Training Sequence

• Obtain a Qualified Assessor certificate
• Online Proficiency PLUS
• Proficiency on the Practice Test
• Obtain a Qualified Trainer certificate
• Support a protégé to Qualified Assessor status
• Submit protégés qualifying scoring protocol with
  your feedback for review
• Attend Qualified Assessor Trainer sessions
  (9/10-11 or 9/11-12 AND 10/31/2006)

16
Certification Checklist for Qualified Assessors

• Complete Qualified Assessor online training
• Attain proficiency in the Reading/Writing/Math/Adm
  inistration online modules
• Administer each Practice Test (reading, writing,
  and mathematics) to a student
• Submit Practice Test Scoring Protocols for review
  and approval by a Qualified Trainer (in this case
  DRA)
• Obtain a Qualified Assessor certificate
• Refresh proficiency annually to maintain access
  to online system

17
Certification Checklist for Qualified
Mentor/Trainers

• Hold a Qualified Assessor certificate
• Complete Qualified Trainer training
• Train a protégé to become a qualified assessor
  by
• Providing orientation to assessments and to
  online training program, give ongoing support,
• Reviewing and providing feedback to protégés on
  practice test after they achieve proficiency on
  the online training,
• Submit protégés scoring protocols with trainer
  feedback for evaluation
• Award QA certificate after protégé has produced
  corrected scoring protocols to the qualifying
  level.

18
Debrief of Online Training Practice
TestsScoring Protocol Feedback and Distribution
of Qualified Assessor Certificates

• Jerry Tindal
• 915-945

19
Qualified Assessor from Pilot

• Macklin, Karen
• Ramponi, Jill L.
• Robbins, Terri
• Street, Stacey L.
• Concilus, Rebecca
• Feliciano, Regina

• Galbraith, Rachel
• Robertson, Linda
• Koentopp, Nan
• Harvey, Sandra

20
Qualified Assessor from Fall Training

• McKay, Scott
• Putnam, Mary
• Silvernale, Sabrina
• Treece, Debbie
• Walker, Lesa
• Waxman, Robin

• Brunsvold, Penny
• Byers, Kelly
• Hansen, Bernadette
• Johnson, Barry
• McDonald, Amy

21
Orientation PowerPoint with Script and Handouts

• Aran Felix
• 945-1015

22
Discussion and Assignment for Conducting
Orientation

• Pat Almond
• 1015-1030

23
Decide Forum for Orientation
24
Small Group Discussion

• Turn to your neighbor and the couple behind you
• Spend 10 minutes sharing your ideas for
  conducting an orientation in your district
• Jot down your current thinking about
• Audience
• Content
• Forum

25
Make a Plan for Yourself

• Identify your audiencespecial education
  teachers, school principal, district test
  coordinator
• Determine what they need to knowassessor
  qualifications, format of new alternate
  assessment, testing window, online components
  (training, materials download, data entry,
  report)
• Decide forum for your orientationPowerPoint
  presentation, conversation in the teachers
  lounge, orientation in the computer lab

26
Break

• 1030 1045

27
Implementation Plan

• Aran Felix
• 1045 1115

28
Working with Protégés Orientation, On-line
Training, Administering Practice Assessments

• Jerry Tindal
• 1105 1130

29
Scoring the Practice Test Scoring Protocols

• Pat Almond
• 1130 - 12

30
Review Protégés Scoring Protocols

• Review Criteria
• Cover Page
• Task Administration
• Administration Condition
• Scoring
• Clear Markings
• Prerequisite Skills
• Stop Testing

31
Provide Feedback to Protege

• Make written comments on review sheet
• Give comments and feedback on scoring protocols
• Point out any areas that may need fixing
• Obtain corrected protocols for submission to DRA

32
Upcoming Mentor Due Dates
33
Lunch

• 12 noon 100

34
StandardTest Administration

• Jerry Tindal
• 100 130

35
Standard with or without Accommodations

• Standard administration conditions refer to a
  student taking the test in a manner consistent
  with the test directions and listed
  accommodations. A score obtained under standard
  administration conditions is comparable to other
  scores obtained under the same conditions. Items
  may be presented in an order most beneficial to
  the student.

36
ExRWM Administration with Flexibility

• The Extended Reading, Writing, and Mathematics
  (ExRWM) assessments, are designed to
• provide standardization in the administration of
  tasks and items in order to increase
  comparability of scores and the inferences that
  can be made and
• provide systematic options for flexibility in
  administration by allowing extensive supports as
  needed for individual students and, when
  necessary, allowing changes in the content being
  assessed.

37
Putting it All Together
1. Prepare
2. Administer
3. Enter Data
4. View Report
38
Administer

• Administer the assessment in format familiar to
  the student
• Ensure student has an opportunity to show what he
  or she knows and can do

39
Example Scripting directions for Reading Words
Task

• Directions and script provide guidance for
  administering the task under the most
  standardized conditions.
• For flexibility
• One for reading and
• One for pointing.
• Scoring for reading words
• ReadingCorrect, partially correct, and
  incorrect.
• PointingOnly correct and incorrect
• A number of tasks within the ExRWM assessments
  suggest flexible formats.

40
(No Transcript)
41
Scoring Protocol Student Materials

• Points
• Digits Correct 2
• Digits Partially Correct 1
• Digits Incorrect 0

42
Test Administration Flexibility and
Standardization

• Pat Almond130

43
Two main areas of Flexibility

• Conditions of Administration
• Decision Paths and Stopping an Assessment Early

44
Putting it All Together
1. Prepare
2. Administer
3. Enter Data
4. View Report
45
Conditions of Administration Pat Almond 130
230

• Standard Administration without or with
  accommodations (STD)
• Expanded Levels of Support formerly Expanded
  Levels of Support (ELOS)
• Not AdministeredInappropriate (NA-I)
• Not AdministeredProficient (NA-P)
• NOTE Applies to each task separately.

ELOS formerly Modified
46
August 22nd Decision to Expand Levels of Support

• Considerations
• Implement both the ExRWM and continue with the
  portfolio
• OR
• Administer ExRWM to students with students with
  the most significant cognitive disabilities

• Problems
• Input from pilot participants
• May TAC feedback
• Guidance from the NCLB facilitator
• EED Director of Assessment

47
Begin by assuming that students with significant
cognitive disabilities can demonstrate academic
achievement if assessed with the appropriate
supports and provided with effective
instruction.(Browder Spooner, 2006)
48
Standard with or without Accommodations

• Standard administration conditions refer to a
  student taking the test in a manner consistent
  with the test directions and listed
  accommodations. A score obtained under standard
  administration conditions is comparable to other
  scores obtained under the same conditions. Items
  may be presented in an order most beneficial to
  the student.

49
Expanded Levels of Support(formerly Modified)

• Expanded levels of support are alterations in
  test administration that limit the
  interpretations that can be made from the
  assessment results. The alterations substantially
  change the level, content, or performance
  criteria.
• Be sure to describe the expanded levels of
  support on the scoring protocol.

50
Expanded Levels of Support cont.

• Changes in level affect difficulty changes in
  content affect interpretation and finally,
  changes in performance criteria affect scoring
  systems resulting in interpretations that are
  stipulated.

Complete and Return Assistive Technology Survey
51
Not Administered Inappropriate Expanded Levels
of Support not Attempted

• An assessor may be unable to design an expanded
  level of support that adequately adapts a task
  for a student. The task, in its standard form,
  may be inappropriate based on the nature of the
  students disability.

(a) Complete Assistive Technology Survey (b)
Return Survey Scoring Protocol with explanation
52
Not Administered InappropriateELOS not
Attempted cont.

• Do not use NA-I for tasks the assessor
  considers too difficult for the student.
• Administer the task under standard conditions or
  with expanded levels of support and follow the
  decision points and error rules
• Record the students responses.

53
Not Administered ProficientDiscouraged for 2007

• On tasks that are too easy for an individual
  student, an assessor may decide that a student is
  proficient based on evidence from the students
  classroom work. When this occurs, mark NA-P as
  the administration code. The student will receive
  75 accuracy. To receive more than 75 accuracy,
  the student must perform the task and earn the
  points.

54
Special Data Collection
55
Standard Preserves Construct with or without
Accommodations

• The student responds to the task independently.
• Accommodations are adaptations provided to
  mitigate the effects of disability.
• In a selected response task, student chooses
  response from 4 options (the correct response 3
  distractors).

56
Extended Reading Task 8Identify Beginning
Reading Words

• Present the cards in the order shown . . . and
  say, Read each word as I show you the card.
  Continue presenting words. Prompt the student
  after a delay with no response.

• . . . Randomly place all of the words face up on
  the table and say, Point to the word after I say
  it. Continue saying words in the order listed in
  the table below. Prompt student after a delay
  with no response.

57
Standard Administration with Accommodation
Identify Beginning Words

• Discussion
• Describe a student
• Identify another accommodation

• Place 4 of the words face up on the table and
  say, Point to the word after I say it.
• Rearrange cards after each item substituting
  alternate distractors.
• Prompt student after a delay with no response.

58
Standard Administration with Accommodation
Note This is not an exhaustive list. The listing
is intended to provide examples of accommodations
that will not change the content or complexity of
the task items.
59
Ways to adapt this taskTask 1 Identifying
Pictures
60
Adapting Identify Pictures

• Principles of Adaptation
• (a) if possible, maintain the construct
• (b) adjust the content to something related, and
• (c) focus on access and opportunity

• Options
• A Dog
• A Photo of Dog
• A Picture of a Dog
• A Dog Rebus
• The word dog

61
Dog Picture
62
What is being assessed?

• Reproductions of coins and bills
• vs
• Real moneyactual coins and bills

• Drawing representing a dog
• vs
• A dog 3-D figure
• vs
• Photo of a dog

63
More on . . . Conditions of Administration STD,
ELOS, NA-I, NA-Pw/ Examples Accommodations and
Expanded Levels of Support
64
Levels of Independence
65
Expanded Levels of SupportChange the Content
being Assessed

• Examples
• Reduce the number of options in a selected
  response
• Provide physical assistance or prompting to cue
  the correct response
• Give the student the correct answer
• Substitute items the student knows for the items
  on the assessment

66
ELOSExample 1

• In the reading words task the assessor made up
  four word cards and substituted the four words
  for the words included in the task. This change
  provides Anna with access to reading words but
  alters the conclusion that can be made about the
  results.

67
ELOSExample 2

• Copy Numbers. For this task the construct is the
  ability to copy numbers. The assessor presents
  Jon with the student material and a felt tip
  marker and directs Jon to trace the numbers.
  Under standard administration conditions it is
  scored for correct orientation and sequence.
  Tracing changes what is being tested.

68
ELOS For students with the most significant
cognitive disabilities

• Administer at least three tasks in each subject
  (STD or ELOS).
• Administer at least three items in each task.
• UNACCEPTABLE administer assessment with 3
  consecutive NA-I tasks ONLY

69
Expanded Levels of SupportChanges the Content
being Assessed

• Minimal Support
• Responds with clarification or fewer options3
  response options
• The student performs the task accurately when the
  teacher clarifies or highlights important
  information.

70
Expanded Levels of SupportChanges the Content
being Assessed

• Moderate Support
• Responds with specific prompts
• The student performs the task accurately with
  specific prompts.
• 2 response options

71
Expanded Levels of SupportChanges the Content
being Assessed

• Extensive Support
• Responds given exact responsee.g. assessor
  points to the correct response as model for
  student
• The student performs the task when the teacher
  provides extensive assistance or exact response.
  
• Only one response option available

72
Still more about accommodations and expanded
level of supports
73
Q. Expanded Levels of Support?

• Mathematics. The student materials for the
  manipulate mathematics concepts provide stimuli
  that can be presented to students. In some cases,
  enlarging the items for example geometric shapes
  and cutting them out of cardboard or foam may
  allow the student to work with the concepts.

• A. No. This is an example of an accommodation.

74
Q. Expanded Levels of Support?

• Writing. Some students with significant
  disabilities do not write their name with a
  pencil, pen, or word processor. A student uses a
  name stamp when asked to write his name.

• A. Yes. The task calls for the student to
  reproduce letters of the first and last name in
  sequence with capitals.

75
Q. Expanded Levels of Support?

• Reading. In the reading words task, substitute
  words that the student knows or in the naming
  letters task, substitute letters the student
  knows, or use all caps.

• A. Yes. In each of these tasks, the assessment
  calls for a random set of words and letters.

76
Frequency of Administration Conditions

• Standard (STD)
• Expanded Levels of Support (ELOS)
• Not AdministeredInappropriate (NA-I)
• Not AdministeredProficient (NA-P)

77
A special case Sarah
78
Sarah gives no response

• Sarah is a student in grade 5 with a significant
  cognitive disability. Her vision is somewhat
  impaired and she has limited movement and control
  of her arms and hands. In school, she has been
  learning to touch the object named by the
  teacher.

79
Reading Task 1Identify Pictures

• She is making progress but the teacher needs to
  provide partial physical guidance to assist Sarah
  in responding. Her teacher, Helen Jones, adapted
  the materials for the first three tasks so that
  Sarah would have an opportunity to respond.

80

• Ms. Jones used objects instead of pictures and
  placed them on the table in front of Sarah. She
  said Show me the horse and waited for Sarah to
  respond. Sarah did not respond. Ms. Jones
  repeated the direction Show me the horse and
  physically guided Sarahs hand to touch the
  horse. Without physical guidance Sarah did not
  respond.

81

• Ms. Jones repeated the task with a chair and a
  hammer. In all three cases, Sarah did not respond
  without physical guidance. Ms. Jones continued
  with Task 2 and Task 3, Read Signs and Symbols
  and Identify Letters by first enlarging and
  embossing the images and then assisting Sarah to
  view the images by helping her trace the signs
  and letters.

82

• In both of these tasks, Sarah made no independent
  response on three consecutive items in each task.
  Ms. Jones marked Sarahs scoring protocol as ELOS
  to indicate Expanded Levels of Support for each
  task and then marked 0 to represent No
  Independent Response.

83
Special Case of Scoring

• 0 Incorrect or No Response
• 1 Partially Correct
• 2 Fully Correct

84
STD, ELOS, NA-I, NA-P?

• Robin The teacher made a florescent orange disc
  (4 X 4) with a picture of a helping hand as a
  symbol for help. Typically when Robin came
  across a problem he would begin humming at first,
  escalate to high-pitched yells and biting his own
  fist. To help him communicate more effectively,
  Ms. Lopez would prompt him to give her the orange
  disc as soon as he started humming. After she
  received the disc, she would help him resolve the
  problem. Ms. Lopez planned to move away
  bit-by-bit, gradually increasing the distance
  between herself and Robin so that he would reach
  out and ultimately move forward to give her the
  disc.

85
STD, ELOS, NA-I, NA-P?

• Saleems vision was impaired and he needed
  adapted materials accented by stark contrasts.
  He did not have the cognitive skills to produce a
  written sentence and could not mark with a pencil
  because of physical limitations. He did respond
  verbally. His goal was to use his skill in
  scanning visually. Saleem used rubber stamps to
  indicate a yes or no response to questions on
  activity forms and to mark the selected picture
  and he used the computer to respond. He needed
  substantive physical assistance to do both.
  Either a teacher or another student assisted him.

86
STD, ELOS, NA-I, NA-P?

• Steve comes to class on a medical gurney. When
  Steve arrives there is no discernable interaction
  with his environment. When his teacher greets
  Steve, he is not sure Steve recognizes him.
  During a 6-hour school day, Steve appears alert
  about 20 minutes a day. Steve sleeps the
  remainder of the day. Steve is tube fed
  breakfast and lunch and the teacher empties
  Steves toileting bag periodically. The teacher
  doesnt feel successful involving Steve in any
  educational activity- functional or academic.
  Steve has very poor tone and is unaware and
  uninvolved in physical therapy activities.

87
STD, ELOS, NA-I, NA-P?

• Ivan attends his local public school and has an
  itinerate specialist in deaf and hard-of-hearing.
  His mother is an instructional assistant in the
  school half-time and is fluent in sign-language.
  In addition to being deaf, Ivan also has mental
  retardation. Because Ivans mother has been very
  involved in his education, Ivan can read, write,
  and do a number of math activities. Ivans
  reading is mostly based on sight words that he
  has learned through practice and supplemental
  instruction in the evenings at home. He has three
  favorite books at home about bears and wildlife.
  He can read them without help. He has never
  learned about phonetic principles of sounding out
  words. (After discussing the topic with his
  mother, Ivans teacher decided not to administer
  the reading tasks for letter sounds and blend
  sounds.)

88
STD, ELOS, NA-I, NA-P?

• Kosumi is in the 10th grade. He can read text
  with a dificulty level at grade 3. He uses a
  calculater for computations involving money and
  can use a scale and a tape to measure salmon he
  catches with his father. He uses a word processor
  for classroom asignments and to write letters to
  his cousin, Telutci, in Oregon. (Kosunis
  teacher decided not to administer the first four
  reading tasks identify pictures, signs, letter
  names, and letter sounds.)

89
Decision Paths for Stopping a Test Early

• Pat Almond
• 230 3

90
Test Administration Flexibility and
Standardization

• Decision Paths and Stopping an Assessment Early
  including who takes the Assistive Technology
  Survey
• Decision Points
• 3 Consecutive Errors/3 Consecutive Tasks
• Minimum 3 Task Rule for No Response

91
Ways to stop testing prior to completing all of
the tasks
92
Error Rule

• 3 Consecutive Error Rule
• Stop a task if student makes substantive errors
  and struggles significantly on three consecutive
  items.
• When the task is stopped early, record the
  responses and move to next task.
• Administer at least three items in a task before
  moving on.
• When this occurs on 3 tasks in a sequence, stop
  the assessment.

93
Graphic View 3 Error Rule
94
No Response

• Inconclusive
• There is not enough information to score, because
  the student does not respond even with specific
  prompts.

95
No Response Criteria 3 No Responses on 3 Tasks

• Even with extensive supports under a Expanded
  Levels of Support
• A student provides no response
• Under these conditions an assessor will terminate
  testing after
• no response on 3 consecutive items
• 3 consecutive tasks with
• Complete the Student Assistive Technology Needs
  and Communication Systems

96
Decision Points
97
Points Earned
98
Section A

• Did the student earn at least 19 points (50 of
  Total Adapted Possible points) in Section A?
• Yes
• No

99
Decision PointReading Example

• 1. Does the student read some words at school?
• Yes/No
• 2. Did the student correctly respond to at least
  half of the items in Section A?
• Yes/No
• If the answer to BOTH Questions 1 and 2 is "No"
• STOP TESTING AFTER TASK 6
• If the answer to ONE or BOTH Questions 1 and 2 is
  "Yes"
• CONTINUE TESTING Proceed to Section B and
  administer Tasks 7 -10

100
Administration Judgments

• Administration Conditions with Levels of
  Independence
• Error Rules (3 items, 3 tasks)
• Non Response Criteria
• Decision Points (reading math)

101
Break

• 3 315

102
Strategies, to coach and support protégésin
using ELOS Decisions

• Aran, Jerry, Pat
• 315 - 4

103
Your Task20 minutes

• Identify ways to administer the assessments with
  accommodations rather than ELOS
• Identify a challenging student
• Select a subject and a task or tasks
• Identify an accommodation for that student to
  administer under standard conditions
• Identify assistive technology and/or augmentative
  or alternative communication system
• If no accommodation, adapt the task for ELOS
  administration

104
Report Out15 minutes

• Description of Student
• Subject and task description
• Accommodation for task
• AT and/or AAC
• ELOS adaptations
• Rationale

105
Evaluation Implementation PlanClosing Discussion

• Aran, Jerry, Pat
• 4 430

106
AdjournHampton Shuttle Arrives

• 430 p.m.