The UALPA The Utah Academic Language Proficiency Assessment

1
The UALPAThe Utah Academic Language Proficiency
Assessment

• Owned By The Utah State Office of Education
• Presented by
• USOE
• Judy Park, Assessment Accountability Director
• Deborah Swensen, Development Coordinator
• Rita Brock, ELL Specialist

October 2006
2
Training Objectives

• Attendees will receive the following
• Purpose, need and federal requirements for an
  academic language proficiency assessment.
• Coordinators Manual
• Samples of all testing materials
• Instruction and practice on administering all
  four modalities
• Instruction and practice on scoring the speaking
  modality

3
Utah Academic Language Proficiency Assessment
Purpose

• To establish a new assessment of academic
  language proficiency for compliance under the
  federal NCLB Act
• - Title III
• To provide a consistent statewide assessment
• To be given to English Language Learners (ELLS)
• A, B, C proficiency levels
• To identify language proficiency

4
Utah Academic Language Proficiency Assessment
Purpose

• Address the Goals of NCLB
• Learners achieve English proficiency
• Provides a total proficiency score
• Assesses Academic English acquisition
• Vocabulary, syntax, discourse, and function to
  assess English language acquisition
• Knowledge and skills of language through the
  modes of communication
• Listening, Speaking, Reading Writing
• Comprehension (Listening Reading)
• Composite (Total Score)

5
The Need For Two Language Assessments
6
Language Acquisition

• Two aspects in language acquisition
• BICS (takes 2 years to achieve)
• CALP (takes 5-7 years to achieve)

7
BICS (Basic Interpersonal Communication Skills

• Social Language (Everyday conversational
  language)
• Playground language
• Relates to the here and now.

8
CALP (Cognitive Academic Language Proficiency)

• Specialized language of learning
• Unfamiliar concepts are encountered, concepts are
  not shared
• Less visible pragmatic and semantic meanings
• Content language, linguistic and cognitive
  challenges

9
Administration Timeline

• October 30
• UALPA materials available
• October 30 February 28
• 50 of students tested on UALPA and answer
  documents returned to USOE
• March April 30 (complete testing)
• Remaining 50 of students tested on UALPA and
  answer documents returned to USOE

10
Administration Timeline

• April 30
• IPT proficiency level and demographic data on
  new students reported to current contractor
• May June
• Testing on UALPA for new students entering
  districts during May and June
• Scanning and scoring by current contractor on
  UALPA Answer documents submitted by April 30th.
• June July
• Scanning and scoring on UALPA assessments by
  current contractor for students tested in May
  and June

11
New Students

• All new students will be assessed on both of the
  following instruments
• IPT (for placement)
• UALPA

12
Who is tested?

• Students to be tested on the UALPA
• All A students
• All B students
• All C students
• Students not to be tested on the UALPA
• No D students
• No E students
• No Native Speakers tested

13
Proficiency Levels

• 2006 - 2007 school year Students will continue
  to be identified using the A, B, C, D, and E
  proficiency levels.
• 2007 2008 school year As students complete
  UALPA testing, the scores will identify them as
  proficiency level of P, E, I, A, or F.

14
UALPA Information

• The UALPA is divided into five grade spans
• K Kindergarten
• 1-2 Grade one through two
• 3-6 Grade three through six
• 7-8 Grade seven through eight
• 9-12 Grade nine through twelve

15
Test Design

• Five Modalities
• Reading
• Writing
• Listening
• Speaking
• Comprehension
• Reading and Listening
• Composite
• Total Score

16
Testing Materials

• Examiners Manual
• Student Testing Booklet
• Answer Document
• The answer document may be in the form of a
  consumable booklet (1-2)
• Answer documents may be referenced in TAMs as
  Answer sheets
• All Answer documents/sheets are booklets
• Flipchart for Speaking for all grades
• Elementary 1-2 3-6. Speaking only
• Secondary 7-8 9-12. Speaking only
• Flipchart for Kindergarten for all modalities

17
The testing materials are organized as follows

• .

18
UALPA Answer Documents

• 2006-07 school year A UALPA pre-print file is
  required from each LEA prior to UALPA testing.
• Bar coded labels will be provided which must be
  applied to the UALPA multiple page answer booklet
• Answer documents, for new students, must be
  hand-bubbled as they are not identified as LEP
  students in the LEA SIS
• Students who are new to the state will need to be
  given an SSID number before the UALPA is
  administered to them
• Back page of Answer document will still need to
  hand bubble as these students do not have LEP
  identifying code in system
• SSID number will allow for identification of
  district submitted demographic data to scored
  answer document

19
Reporting New Students - IPT

• Each District submits to Current Contractor
  demographic records and IPT scores for new
  students who are placed as LEP students in the
  06-07 school year
• The information to be submitted must include
• student SSID, LEA student ID, student last
  name, student first name, student middle
  name/middle initial, LEP code (A, B, C, D, E),
  grade, primary language, parents primary
  language, gender, ethnicity and instruction type.
  
• Instructions on how to submit this information to
  Current Contractor will be provided in the
  completed Coordinators manual.

20
UALPA Answer Document

• Certain unique codes will have to be completed
  when applicable
• Special codes (non participation)
• Special codes (non standard participation)
• Frustration level/refused to test

21
UALPA Answer DocumentSpecial Codes

• Marking the Special Codes Box
• For not participating mark
• Officially withdrawn from school if the student
  is no longer enrolled in the school.
• Home school if the student is educated in a
  home school setting.
• RT if the student refused to take the test.
• This should be marked in the "Office Use Only"
  section for each modality for which the student
  refused to respond.
• Unknown student if the answer sheet was
  pre-printed for a student who cannot be
  identified as ever having been enrolled in the
  school.

22
UALPA Answer DocumentSpecial Codes

• Since All ELL students (A, B, C) must participate
  in the UALPA do not mark the following
  non-participation codes
• ELL 1st year before 4/15 if the student is
  an ELL student and enrolled before April 15 of
  the current school year.
• ELL 1st year on or after 4/15 if the student
  is an ELL student and enrolled on or after April
  15 of the current school year.

23
UALPA Answer DocumentSpecial Codes

• Marking the Special Codes Box
• For non-standard participation mark
• Accommodated (ELL, IEP, 504 Plan)
• Modified (IEP Team)
• UAA, Utah Alternate Assessment (IEP Team)
• OTHER
• ELL First Year in U.S. Before April 15
• If Accommodated is marked please identify ALL
  specific accommodations for that student in the
  new Accommodations box located on the back of the
  answer document in the upper right hand corner .

24
Accommodations

• The 2006-2007 approved Accommodations manual can
  be found at the following url
• www.schools.utah.gov/eval/DOCUMENTS/Special_Needs_
  Participation_Requirements.doc
• If an ELL student is also identified under an IEP
  or 504 Plan, the designated school team must
  determine in advance what accommodations are
  appropriate.
• For the UALPA (Utah Academic Language Proficiency
  Assessment) the following accommodations are not
  allowed
• Bilingual Word List
• Translated formulas
• Directions (oral translation)
• Questions (oral translation)
• The application of the above accommodations would
  invalidate test results.

25
UALPA Answer Document

• Test administrator/proctor will mark the
  following circles for each modality where
  applicable if
• student/parent refuse testing (mark RT)
• student reaches frustration level
  (mark FL
  

26
Testing Schedule

• UALPA should be given over a minimum of two days
  to ensure best test conditions.

27
Pre - testing instructions

• Notify students and parents about testing.
• Become thoroughly familiar with this Test
  Administration Manual.
• Organize materials including
• a supply of sharpened 2 pencils with erasers
  (students may not use pens, colored pencils,
  crayons, or markers).
• some scratch paper for student use.
• all necessary materials.
• Secure and place a TESTINGDO NOT DISTURB sign
  on the door.

28
Pre - testing instructions

• Have an extra test booklet and answer document
  available for demonstration purposes.
• Have a roster of the students who are taking the
  test under your supervision. The roster must
  include district student identification numbers.
• Obtain the district/school code from the testing
  coordinator.
• Make sure students have previously been
  instructed to bring a book or something quiet to
  do when they have completed the test.
• Plan for sufficient time to administer each
  modality. Although each modality is designed to
  take approximately 30 minutes, students should be
  given enough time to attempt to answer each
  question in each modality.

29
Pre - testing instructions

• Avoid interruptions during the test.
  Circumstances over which the test
  administrator/proctor has no control may occur
  (e.g., power failures, fire alarms) may interrupt
  testing. If such an interruption occurs,
  instruct students to insert their answer document
  in their test booklet and to close their
  booklets. When normal conditions are restored,
  resume testing.
• Interruptions should not reduce the total amount
  of time students are given to complete a modality.

30
Testing Instructions

• Prepare the room
• Put the "TESTINGDO NOT DISTURB sign on the door
  to the room.
• Arrange seating so that the students are
  encouraged to work independently.
• Students may not use dictionaries or other
  standard reference materials or have them on
  their desks. Normal instructional materials such
  as word walls and posters displayed in a
  classroom are not considered standard reference
  materials and may remain in place.

31
Testing Instructions

• Ensure that students receive the answer documents
  with their specific label on it when distributing
  answer documents.
• Move around the room to be sure that all students
  are working on the test when the test session has
  begun.
• Each student should work alone. There should be
  no group discussion.
• Be certain that students understand how to fill
  in the appropriate spaces on their answer
  documents. Demonstrate the appropriate process
  before beginning the test

32
Testing Instructions

• Supply the student with a new pencil if a student
  breaks his/her pencil.
• Keep communication with the students to a
  minimum.
• Create a positive, non-threatening environment in
  which students feel as comfortable as possible.
  Let students know that it is okay if they do not
  know a specific answer.
• Maintain and oversee the security of all test
  materials.

33
Administering the UALPA

• For modalities that may be administered in
  groups, students should be grouped according to
  their grade spans.
• Groups should be small
• Recommended 10 students
• Recommended not to exceed 20 students
• Students that may be a behavior concern can be
  tested individually.

34
Administering the UALPA

• Some students may not know how to complete an
  answer document. To assist students in correctly
  filling in their answer documents, demonstrate
  the process on the board or on an overhead.
• In some grade spans and modalities that are
  administered in a group, there are practice
  items. For the practice items, please circulate
  among students to make sure they have identified
  the correct answer. If not, then explain the
  correct answer to the group.
• In the Reading and Writing modalities, students
  may be instructed to read a passage. It may be
  necessary to remind students to read these
  passages quietly to themselves, rather than out
  loud.

35
Administering the UALPA

• Instructional cues
• Bolded words
• are instructions for the administrator/proctor
• Special instructions to assist the students
  through the test are identified by the word
  Note
• or
• If they are in an italicized statement they are
  to be emphasized when read
• Italicized words should be read aloud to the
  student
• Occasionally, these words will be instructions
  that notify the student they are now working on
  their own. So that students clearly understand,
  do not hurry through these instructions.

36
Individual administration of the UALPA

• Kindergarten students
• Speaking modality for all grade spans
• Students who have reached frustration level
• frustration level is defined as scoring a 0 on
  five consecutive questions.

37
Administering the UALPA Grade Span K
38
Administering the UALPA Grade Span K

• Materials
• Flip chart
• Kindergarten Answer Document per student
• Any student without a pre-printed Answer Document
  will need to have his/her information recorded on
  a blank Answer Document. (Refer to the section
  Instructions for Coding Blank Answer Sheets.)
• Testing administrator/proctor will need to read
  the general instructions at the beginning of the
  flipchart before starting to test the student.
  These pages are not set up in flipchart format.
• Test administrator/proctor will begin the
  flipchart format with the speaking modality and
  continue through all modalities.

39
Administering the UALPA Grade Span K

• Testing should proceed unless students reach
  frustration level
• Frustration level is defined as the student
  making five consecutive scores of 0 (incorrect
  or no response)
• On the flip chart, the choices for options are
  marked as A, B, or C on the students side. Make
  sure you position the flipchart so you can see
  which option they have chosen.
• A writing checklist must be completed for each
  student.

40
Administering the UALPA Grade Span 1-2
41
Grade Span 1-2

• Materials Needed
• Form 1-A Elementary Speaking Flipchart, Section
  Grade span 1-2
• Grade Span 1-2 Student Consumable Booklet
• Grade Span 1-2 Test Administration Manual
• Any student without a pre-printed Student
  Consumable Booklet will need to have his/her
  information recorded on a blank Student
  Consumable Booklet. (Refer to the section
  Instructions for Coding Blank Answer Sheets.)
• Testing should proceed unless students reach
  frustration level
• Frustration level is defined as the student
  making five consecutive scores of 0 (incorrect
  or no response)

42
Grade Span 1-2

• On the speaking flip chart, the choices for
  options are marked as A, B, or C on the students
  side. Make sure you position the flipchart so you
  can see which option they have chosen.
• The administrator/proctor in the Test
  Administration Manual (TAMs) is directed to read
  some of the answer choices with an identifying A,
  B, or C. However, the students do not have the
  same identifying letters under their choices. If
  this is confusing, the administrator/proctor can
  just read the answer choices in a series.

43
Administering the UALPA Grade Span 3-6, 7-8,
9-12
44
Grade Span 3-6

• Materials Needed
• Form 1-A Elementary Speaking Flipchart, Section
  Grade span 3-6
• Student Booklet Grade Span 3-6
• Grade Span 3-6 Test Administration Manual
• Student Answer Document
• Any student without a pre-printed answer document
  will need to have his/her information recorded on
  a blank answer document. (Refer to the section
  Instructions for Coding Blank Answer Sheets.)
• Testing should proceed unless students reach
  frustration level
• Frustration level is defined as the student
  making five consecutive scores of 0 (incorrect
  or no response)

45
Grade Span 3-6

• On the speaking flip chart, the choices for
  options are marked as A, B, or C on the students
  side. Make sure you position the flipchart so you
  can see which option they have chosen.
• The speaking modality is administered
  individually using the flipchart, the examiner
  scores the student response.
• The listening, reading, and writing tests can be
  given in a group setting.
• 10 students per group is recommended.

46
Grade Span 7-8 or 9-12

• Materials Needed
• Grade Span 7-8 or 9-12 Test Administration Manual
• Form 1-B Secondary Speaking Flipchart, Section
  Grade span 7-8 or 9-12
• Student Booklet Grade Span 7-8 or 9-12
• Student Answer document Grade Span 7-8 or 9-12
• Any student without a pre-printed answer document
  will need to have his/her information recorded on
  a blank answer document. (Refer to the section
  Instructions for Coding Blank Answer Sheets.)
• Testing should proceed unless students reach
  frustration level
• Frustration level is defined as the student
  making five consecutive scores of 0 (incorrect
  or no response)

47
Grade Span 7-8 or 9-12

• On the speaking flip chart, the choices for
  options are marked as A, B, or C on the students
  side. Make sure you position the flipchart so you
  can see which option they have chosen.
• The speaking modality is administered
  individually using the flipchart, the examiner
  scores the student response.
• The listening, reading, and writing tests can be
  given in a group setting.
• 10 students per group is recommended.

48
After Testing Instructions

• Arrange make-up sessions for students who miss
  part or all of the test.
• Check to see that the student answer documents
  are complete and appropriately marked (e.g.
  pencil was used instead of pen)
• Separate test materials as described below
• Test booklets (used and unused)
• The Test Administration Manual
• Student answer documents (used and unused)
• A list of students who did not complete the test
• Complete the special codes box on the answer
  document for students who meet the specified
  criteria. Refer to the section Marking the
  Special Codes Box" for specific instructions.
• Organize and return all used and unused secure
  test materials to your testing coordinator.

49
Walk Through

• Perusing Test Materials

50
UALPA Training Materials

• Nondisclosure Agreement (One per person)
• Power Point (One per person)
• UALPA Training Materials Binders (One per
  district)
• Coordinators Manual (Two per binder)
• Flipcharts (K all modalities, 1-12 Speaking
  modality only)
• TAMs (inside binder Grade Spans 1-12)
• Modality specific instructions (Listening,
  Reading, and Writing)
• Answer Documents (All Grade Spans)
• Scoring Guide Conversion Chart (2 per binder)

51
Suggested Testing Aides

• Review the Checklist in the TAMs
• Make sure you know how to fill in the student
  number on their answer document
• Use an overhead to complete the example items and
  to demonstrate how to fill in the answer document

52
General Instructions

• The information to be read aloud to students is
  printed in italics.
• Instructions to the Test Administrator/proctor
  are printed in bold.
• All other information is presented in regular
  type.
• When a letter is enclosed by two parallel lines
  (/d/) you say the sound of the letter.
• Demonstration (D) and practice (P) questions may
  be included in the different modalities and will
  be noted by D or P.

53
Before testing

• Specific step-by-step instructions for the
  administration of each modality of the test can
  be found just prior to that section in the TAM.
• Please read through all instructions before
  administering the test.

54
During Testing

• You must follow the script exactly as it is
  written in the TAM.
• This is an English language proficiency
  assessment, students must respond in English in
  order for responses to be considered correct.

55
Prompting Test Information

• Prompting is not allowed in this test.
• Prompting means providing additional information,
  such as
• Elaborating on questions
• Clarifying information provided in reading
  selections or any test question
• Providing cues
• Suggesting strategies that a student may use to
  arrive at a correct answer

56
Prompting Test Information

• You may give the following general prompts where
  partial or unclear responses are given.
• I dont understand what you said.
• Can you tell me more?
• You may emphasize the word in the directions if
  it is written all in capital letters.

57
Prompting Test Information

• You should allow approximately 15 seconds of wait
  time for a student to begin a response to a
  question.
• If the student has not responded after 15
  seconds, you should move on to the next item or
  task and score the item as a 0 (no response).

58
Speaking

• Modality

59
Speaking

• This test is to be administered individually.
• Read all selections and questions aloud that are
  italicized.
• Should a student ask to have the directions, or
  question read again, you may do so only once,
  before the student begins to respond.
• stories will be read only once.

60
Speaking

• You must read with appropriate pronunciation,
  inflection, pauses, and general clarity.
• Point to each item that has a picture on the
  Flipchart.
• All responses should be given verbally and must
  be in English to be considered correct.

61
Scoring Speaking Modality

• The test administrator/proctor will score the
  speaking modality for all grade spans
• mark answers on the appropriate section of the
  student answer document/consumable booklet
• Below the numerical scoring box in the TAM,
  scoring guides are provided to guide you on
  possible appropriate responses
• Review the scoring guides before administering
  the test
• The score is numerical but is recorded as a
  letter on the student answer booklet.
• the test administrator/proctor will use the
  following equivalencies for marking numerically
  scored response items.
• A 0 B 1 C 2 D 3
  E 4

62
Scoring Guide

• Scoring guide includes the correct answer and
  variations
• Examples provided of answers for partial credit
• Lists are intended to be guides not a
  comprehensive list of appropriate responses
• Students possibly will give an answer not
  included in the guides
• Administrator/proctor should use their best
  judgment to score answers based on scoring guide
  information

63
Details of Scoring

• Some examples of responses earning minimum credit
  (1) are as follow
• Student answer that is partially correct
• A student gives a nonverbal response, such as
  nodding or pointing.
• If a student does not respond, give a score of
  0 (no response).
• Student answers in a language other than English
• Students response is so poorly articulated it
  cannot be understood
• Student makes no response

64
Listening

• Modality

65
Listening

• Read all selections and questions that are
  italicized.
• Should a student ask to have the directions,
  questions, or options read again, you may do so
  only once, before the student begins to respond.

66
Listening

• The same guidelines for instructions apply
• Provide paper so that students may take notes as
  they listen.

67
Reading

• Modality

68
Reading

• The student will read the options, directions, or
  stories to themselves. Do not read any of the
  options with letter choices to the students.
• Allow time for students to read the text in the
  box before you read the questions aloud.

69
Writing

• Modality

70
Writing

• You will read the direction, stories and
  questions to the students that are italicized.

71
Test Practice Time Mock practice the test in
pairs

• Speaking -15/20 minutes
• Writing - 10/15 minutes
• Reading - 20/30 minutes
• Listening - 30/35 minutes
• Rating Scoring Rubrics - 40 minutes

72
Production Up-date of UALPA
73
Production Up-date of UALPA
74
Closing

• Questions and Comments