Avoiding the False Negative: Placing Students Into Mathematics Courses According to Their Abilities

1
Avoiding the False Negative Placing Students
Into Mathematics Courses According to Their
Abilities

• Margaret J. Hager, EdD
• AMATYC 32nd Annual Conference
• Cincinnati, Ohio
• November 2, 2006

2
My Motivation

• Many stakeholders believed the advising and
  placement systems needed improvement
• It was important to find out the perceptions of
  the institution so that recommendations made
  could be successful
• Webbs Chapter focuses on theory building
  through a critical evaluation of current efforts
  in mathematical assessment (p. 661).

3
Relevant Research

• Many studies were quantitative in nature
• Reliability of placement tests Bridgeman
  Wendler (1989), Johnson (1984), Latterell Regal
  (2003), Sawyer (1996), Sworder (1986)
• Cut-off scores Abraham (1986), Hughes Nelson
  (1990) Morante (1987)
• Use of Multiple Measures Armstrong (2000), Askt
  Hirsch (1991),Bridgeman Wendler (1989) Hoyt
  (1999), Jenkins (1989), Jones (1997) Jue (1993)
  Morante (1987)

4
Relevant Research (Continued)

• Revak, Frickenstein, Cribb (2000)
• A small amount of studying prior to taking the
  placement test resulted in significantly higher
  placement scores.

5
Relevant Research (Continued)

• An assessment involves a situation, responses to
  that situation, analyses of the responses,
  interpretation of the results, and interaction
  among all of these. . . . (Webb, 1992, p.
  680).

6
Problem Statement

• Why do so many students place into Preparatory
  mathematics courses even after multiple years of
  high school mathematics preparation?

7
Problem Statement (Continued)

• Are there any recommendations that might be made
  to the mathematics placement process to help
  ensure proper placement of students into
  mathematics courses?

8
Problem Statement (Continued)

• Guiding Questions
• 1. Why do students not practice before taking the
  mathematics placement test?
• 2. Why do students who place into a lower level
  mathematics course often choose to take a
  higher-level mathematics course?
• 3. What other measures of student preparedness
  for mathematics courses are possible in addition
  to the one placement test score?
• 4. Should mathematics placement be mandatory?

9
Method

• Student Participants (15)
• Semi-structured focus groups and individual
  interviews
• Current students
• Students at Orientations
• Students appealing their mathematics placement

10
Method (Continued)

• Advisors/Administrators (A/A) (20)
• Semi-structured individual interviews
• Faculty and Staff Advisors
• Placement Test Coordinator
• Assistant Dean of Student Services
• Placement Test Proctor
• Division Chairs

11
Method (Continued)

• Further Data Collection
• Unobtrusive Placement Testing Observations
• Document Collection
• Compass/ESL Manual Practice Test packet
• Orientation Placement Test Proctor Scripts
• Various Reports/memos on placement test issues

12
Method (Continued)

• Data Analysis
• Individual/focus Group answers coded
• Triangulated with field notes and documents

13
Results

• Compass/ESL Process
• Computer-adaptive
• Students instructed to use only the on-line
  limited function calculator
• Average of 7 questions per testing domain
• All students began in Algebra domain
• Students not able to check their work
• Average time spent was approx. 16 minutes

14
Results (Continued)

• Accuracy of Mathematics Placement Test
• Majority of the students and A/A believed the
  results were inaccurate

15
Results (Continued)

• Misconceptions Regarding the Placement Process
• A common belief of A/A was that students did not
  need to study before taking the test

16
Results (Continued)

• Mathematics Placement Appeal Process
• 46 in pilot, 27 judged, 81.5 successful
• However, there was no formal process that
  required students to appeal

17
Results (Continued)

• Six-month retake policy with intervention
• Most A/A did not realize the six-month rule was
  in place
• Most A/A believed it was too long of a time-frame
• Students were not informed of the need for
  intervention

18
Results (Continued)

• Practicing Before Test
• Most students did not practice (many wished they
  had)
• Web site with arithmetic problems
• http//mathematics.clc.uc.edu/hager/index.html
• Need of practice packet

19
Results (Continued)

• Calculator Use
• Instructed to use limited function on-line
  version
• Students wanted their own hand-held calculator
• Compass has an approved list of calculators

20
Results (Continued)

• Multiple Measures
• Implementation was a concern
• Time and effort in gathering data
• Current testing was easy to administer
• Results were almost immediate

21
Results (Continued)

• Mandatory Placement
• Many A/A believed this would make their advising
  easier
• Must keep the placement appeal process
• Will be pursued if web system can block students
  from skipping any pre-requisite courses

22
Discussion

• Guiding questions
• 1. Why do students not practice before taking the
  mathematics placement test?
• Students had no information on what to study
• They were not made aware of the importance of the
  test and therefore, did not take mathematics
  placement test seriously

23
Discussion (Continued)

• 2. Why do students who place into a lower level
  mathematics course often choose to take a
  higher-level mathematics course?
• They did not believe the results of the
  mathematics placement test to be reflective of
  their ability

24
Discussion (Continued)

• 3. What other measures of student preparedness
  for mathematics courses are possible in addition
  to the one placement test score?
• ACT/SAT scores HS math grades
• Motivation to do well HS GPA
• Use multiple measures for students near cut-off
  scores or requesting appeal

25
Discussion (Continued)

• 4. Should mathematics placement be mandatory?
• If implemented, it would make A/A job easier.
• A/A inherently trusted the mathematics placement
  recommendation
• Web-based registration system unable to handle it
• Appeal process should continue

26
Recommendations

• Changes in mathematics placement process
• Increase number of questions per testing domain
  to the maximum number (average of 13)
• Have students choose the domain appropriate to
  their level of preparedness

27
Recommendations (Continued)

• Allow students to use a calculator listed on the
  Compass/ESL approved list of calculators
• Allow students to retake the test per the
  guidelines of Compass/ESL Manual
• Allow students an additional or optional
  paper/pencil, machine-scored test as an
  additional measure of their ability

28
Recommendations (Continued)

• Five-week co-requisite course
• Allows students an option to spending 10 weeks in
  a course when they only need some review
• An additional one-credit hour course co-requisite
  with the Introductory Algebra I course

29
Recommendations (Continued)

• Information Sheets
• Describe computer-adaptability
• Strong suggestion of reviewing prior to testing
• Where to get practice information
• List of approved calculators
• How to choose which domain of questions they
  should begin in
• How to interpret results of test
• Description of the retake and appeal policies

30
Recommendations (Continued)

• Further recommendations
• Pursue mandatory placement implementation
• Follow students
• What course they tested into, what they took, how
  well they did
• Add Algebra section/questions to practice web
  site

31
Limitations

• Limitations
• Generalizability Cannot generalize to other
  colleges
• Low number of placement testing observations
• Not all students tested were interviewed

32
Future Research

• Future Research
• Follow-up with student placements
• Follow-up with effectiveness of recommendations
• Research why students skip pre-requisite courses

33
Thank you