The Effectiveness of Pullout Programs: What Does Research Show?

1
The Effectiveness of Pullout Programs What Does
Research Show?

• Cheryl Temple
• EDUC 872

2

• Are pullout programs an effective strategy to
  help struggling students OR does it stretch them
  too much?

3
What are Pullout Programs?

• Remedial education programs
• Special education programs
• Gifted education programs
• Band, orchestra, and chorus
• Speech and Language
• Adaptive physical education, or OT/PT
• Counseling
• Etc.

4
ProPullout programs

• Allows for the individualization of instruction
  many students need in order to be successful in
  school
• Allows staff with specializations in various
  areas to provide intense instruction with
  students in a setting with a small number of
  students

5
ConPullout Program

• Classroom teacher is held accountable for
  learning but many children are missing valuable
  instructional time because of pullout programs
• Fragmentation of students instructional day

6

• Disruptive to whole-class teaching
• Missed instructional time
• Create teacher frustration and friction between
  staff members
• Students are believed to suffer academically

7
What does the research show about the value of
pullout programs?
8
Lets Look Back

• Affleck, J., Madge, S., Adams, A., Lowenraun,
  S. (1988). Integrated classroom versus resource
  model academic viability and effectiveness.
  Exceptional Children, 54, 339-349.

9
Summary

• University of Washington and the Issaquah,
  Washington School District developed a service
  delivery model for educating students with mild
  disabilities in integrated classrooms
• Administered jointly by regular and special
  education personnel

10
Summary

• Conducted 2 studies over the course of 3 years
  Academic Achievement of Special Education
  Students and Academic Achievement of Regular
  Education Students

11
Definition of Integrated Classroom

• Regular curriculum and materials are used
• Students with disabilities (LD, MR, ED) are
  educated in regular classrooms
• Teachers have all had successful prior experience
• Aides are assigned for 1 ½ - 3 hours per day.

12
Method for Spec. Ed.

• 3 year study
• All students with LD, ED, or MR were assigned to
  the Integrated Classroom Model (ICM) classes
• Only LD included in this study because of
  insufficient numbers of students with other types
  of disabilities

13
Method

• Contrast group composed of special education
  groups in the same district who were enrolled in
  a resource room
• Students pulled out from 30 to 150 minutes daily
• Same instructional materials and methods for
  basic skill instruction are used in resource room
  as ICM classrooms

14
Method

• Nonequivalent control group design was used
• Age percentile scores of three subtests were
  converted to normal curve equivalent scores to
  allow a more appropriate statistical analysis
• ANCOVA was applied using the pretest scores as
  the covariate for the posttest scores
• Trait-treatment-interactional analysis was used
  to determine further effects of treatment

15
Sample for Spec. Ed

• All students in both groups were Caucasian
• Both groups had equal socioeconomic status as
  determined by reduced school lunch data
• Teachers in ICM and resource room model had
  similar experience and background
• Teachers from both models participated in staff
  development activities during the 3 years

16
Instruments for Spec. Ed.

• Reading, math, and language subtests of
  Woodcock-Johnson Psycho-Educational Battery was
  administered in October and May of each year

17
Results for Spec. Ed.

• No significant differences between groups during
  all 3 years in reading or language
• One significant difference in math during Year 1,
  in which the adjusted mean for the integrated
  students was significantly higher than for
  resource students
• Trait Treatment Analysis shows more gains than
  losses in each subject in each program by at
  least 21.
• No significant difference in gains/losses between
  the two programs

18
Method for Reg. Ed.

• Subjects were placed in ICM during Year 1 of the
  study and a regular classroom the following year.
• Contrast group was randomly selected from each
  corresponding grade level at the same building

19
Sample for Reg. Ed.

• 39 regular education students in grades 3 and 4
  from one building, and grade 5 from another
  building.

20
Instrument for Reg. Ed.

• California Achievement Test Battery was
  group-administered in the fall of Year 1
  (pretest) and Year 2 (posttest)
• Battery percentile scores were converted to NCE
  scores
• ANOVA was used on pre, post, and gain scores.

21
Results for Reg. Ed.

• No significant differences between the two groups

22
Discussion

• Cost-benefit
• Savings of 13,500 in ICM classroom compared to
  resource
• Savings of 41,250 for regular education in a
  school that converts from resource model to ICM
  because special education funds part of teachers
  salaries.

23
Conclusion for Study 1

• Results of Study 1 support ICM as an alternative
  service delivery model for students with LD.
• ICM is not a more favorable program, just
  comparable

24
Conclusion for Study 2

• Support ICM as an effective program for regular
  education students as there were no
  distinguishable differences in achievement
  between students in ICM classroom and in a
  classroom with students with no disabilities

25
Additional Limitations

• Only addresses students with LD what about when
  classrooms that have students with other types of
  disabilities
• Cant generalize to urban, rural, or culturally
  diverse settings (all Caucasian students in the
  study)
• Working in a school system that was already using
  integrative instruction
• Study conducted by University of Washington.
  They have been working with this school system on
  this model. Perhaps biased because they want to
  prove this model is effective in order to
  continue their work

26
Benefits

• No pullout
• Possibly reduces stigma
• Eliminates scheduling problems
• Eliminates coordination of curriculum in two
  settings
• Support LRE
• ICM is co-funded
• Less space needed

27
A Comparison Study of the Ohio Proficiency Test
Results between Fourth-Grade String Pullout
Students and Those of Matched Ability

• Wallick,M. Journal of Research in Music
  Education, Summer 1998, Vol. 46, No. 2, pp.
  239-247.

28
Summary

• Examine the effects of a pullout string program
  on student achievement in the writing, reading,
  math, and citizenship sections of the Ohio
  Proficiency Test

29
Method

• Two-group static-group comparison design
• A two-sample independent t-test analysis was used
  to determine if there was a significant
  difference between the achievement scores of the
  string students who were excused from class twice
  a week for 30 minutes and the matched group of
  nonstring students who remained in class.

30
Method cont.

• T-test, mean, SD, and p value of each matched
  group were calculated and compared
• .05 was the level of significance used

31
Sample

• 148 fourth-grade string students and 148
  fourth-grade nonstring students from a
  southwestern Ohio city school district
• Ability-matched according to their performance on
  the verbal section of the Cognitive Abilities test

32
Sample cont.

• Hamilton, Ohio school system serves 9,900
  students from diverse cultural and socioeconomic
  backgrounds
• 296 students in this study were drawn from 12 of
  the 13 elementary buildings (one building doesnt
  have strings in the 4th grade)

33
Instruments

• Cognitive Abilities Test to match students
• Ohio Proficiency Test to measure student
  achievement

34
Results

• Significant difference in favor of the string
  students achievement in reading and citizenship
• No significant difference between the two matched
  groups in the writing and math sections of the
  Ohio Proficiency Test

35
Proficiency section Mean SD t Prob.

Writing
SS 5.05 1.43
NS 4.85 1.29 1.270 .20512
Reading
SS 229.5 23.73
NS 223.2 26.22 2.127 .03429
Mathematics
SS 214.8 24.83
NS 211.8 27.34 0.998 .31907
Citizenship
SS 231.3 23.71
NS 224.8 31.59 2.003 .04604
SS string students who were excused from class
30 minutes twice a week. NS matched-ability
nonstring students who remained in the classroom.
OPT Ohio Proficiency Test
36
Discussion

• One explanation for string students scoring
  significantly higher in reading and citizenship
  could be that both sections require good reading
  comprehension and critical thinking skills
• Reading music involves interpreting abstract
  symbols and translating them similar to reading
  text.

37
Discussion cont.

• Authors suggest that string students arent
  losing instruction because they are learning new
  concepts like understanding fractions (1/4 note,
  ½ note) and manipulating rhythm

38
Conclusion

• There were no negative effects to the pullout
  program

39
Limitations

• Study was conducted by music teachers perhaps
  biased
• Inability to assess the impact of different
  classroom teachers and school atmospheres on
  string students learning (in schools where
  strings participation is encouraged students
  may have less difficulty because of cooperating
  teachers)

40
Outcomes for students with learning disabilities
in inclusive and pullout programs

• Rea, P., McLaughlin, V. Walther-Thomas, C.
  (2002). Exceptional Children, 68, 203-223.

41
Summary

• Investigated the relationship between placement
  in inclusive and pullout special education
  programs and academic and behavior outcomes for
  students with learning disabilities (LD)

42
Method

• Descriptive investigation which used quantitative
  and qualitative methods
• Explored relationship between placement of
  students with LD and achievement, behavior, and
  attendance
• Archival data was used
• Significance was established at .05

43
Method

• Objective data relating to number of
  accommodations and amount of special education
  services were collected from a review of
  students IEP.
• Doctoral students analyzed the data and
  interrater reliability was established.

44
Sample

• Students in the 8th grade in two middle schools
  in a suburban school district in the southeast
• Students with LD who had not been enrolled in
  their assigned school program for at least 2
  years were removed from the sample
• Sample size was 58 students
• Majority of students were Caucasian
• 12.1 percent received free/reduced lunch

45
Instruments

• Final course grades in language arts, math,
  science, and social studies
• Iowa Test of Basic Skills standard scores on
  reading, math, science and social studies
• State Literacy Passport Test
• Student attendance records

46
Results

• The two programs differed significantly
• Students in inclusive classrooms
• Earned higher grades
• Achieved higher or comparable scores on
  standardized tests
• Committed no more behavioral infractions
• Attended more days of school

47
Results cont.

• IEPs for students in inclusive settings had 3.22
  number of goals compared to 2.50 number of goals
  for students in pullout program
• Goals inclusive settings focused on general ed.
  curriculum
• Goals pullout programs focused on deficit areas
  and remedial skills

48
Results cont.

• Statistically different patterns of
  implementation for classroom accommodations
  (instruction, assessment, behavior)
• Inclusion avg. of 14.8 accommodations
• Pullout avg. of 5.6 accommodations
• Time
• Inclusion avg. of 740 minutes per week
• Pullout avg. of 252 minutes per week

49
Results cont.

• Students in inclusive classrooms earned
  significantly higher grades in all four areas of
  academic instruction
• Statistical analysis of data on performance on
  the reading, math, and writing subtest of the
  state proficiency test revealed no significant
  differences between students who received
  inclusion vs. pullout services

50
Results cont.

• Statistical analyses of data from Iowa Test of
  Basic Skills showed students with LD receiving
  inclusive services achieved higher standard
  scores on language and math than students
  receiving pullout services
• Both groups had similar mean scores on reading
  comprehension, science, and social studies
  subtests

51
(No Transcript)
52
Results cont.

• Statistical analyses of data indicated no
  significant differences between the two groups
  relative to behaviors that warranted in-school or
  out-of-school suspensions.
• Attendance data showed that students in inclusive
  classrooms attended significantly more days of
  school than students in pullout programs.

53
Discussion

• Five key findings
• 1. Students in inclusive settings received
  higher course grades suggesting that programs
  that provide a strong focus on the standard
  school curriculum is beneficial for students with
  LD

54
Discussion

• 2. Since students with LD in inclusive settings
  scored higher on language and math subtests of
  ITBS, the assumption that small group instruction
  results in improved scores on standardized tests
  should be questioned

55
Discussion

• 3. Students in both settings had comparable
  scores on the state proficiency test. Authors
  state that the standard curriculum focus and
  accommodations for LD are factors in positive
  school outcomes.

56
Discussion

• 4. Since students in inclusive classrooms did
  not have more in-school or out-of-school
  suspension, the increased demands of full-time
  general education placement did not appear to
  result in more acting-out behavior.

57
Discussion

• 5. Since students in the inclusive program
  attended more days of school, this may indicate
  greater student satisfaction with inclusive
  services. Also increased opportunities for
  quality instruction and social experiences may
  have positive influence.

58
Conclusion

• Results suggest that with adequate adaptations,
  individualized programs, and sufficient support,
  students with disabilities can achieve academic
  and social success in general education
  classrooms.

59
Limitations

• Authors are pro inclusion models so the study may
  be biased
• Authors state that the standard curriculum focus
  and accommodations for LD are factors in positive
  school outcomes. This statement doesnt match
  their previous comments that students in pullout
  programs are mainly receiving remedial instruction

60
Limitations

• Sample size
• Archival data from 1994-1996 was used but study
  was written in 2002
• Only students with LD were studied so cant
  generalize to any other disability group
• Convenience sample not random sample
• One author references herself 5 times in this
  article so her beliefs are well-known

61
Are pullout programs sabotaging classroom
community in our elementary schools?

• Brandts, L. (1999). Primary Voices, 7, 9-16.

62
Summary

• Teacher Researcher collects quantitative and
  qualitative data to support her opinion that
  pullout programs may have a negative effect on
  the community of learners that may affect
  learning.

63
Method

• Kept anecdotal records regarding
• Loss of time-travel
• Sense of disengagement
• Lack of connection
• Not understanding what was going on when they
  reentered the classroom
• What they missed while out of the room

64
Method cont.

• Halfway through the year shifted to no-pullout
  classroom.
• Reading specialist worked each day with the
  children in the classroom
• Kept field notes, videotapes, and interviews

65
Sample

• 2 students in 2nd grade class who left to attend
  reading pullout program

66
Instruments

• Observation/Anecdotal records
• Student questionnaire
• Woodcock Reading Mastery
• San Diego Quick Informal Word I.D.
• Stanford Diagnostic

67
Results

• By end of year, one student had gained two years
  in reading and was recommended to be dropped from
  the program
• Other student was making steady progress

68
Results cont.

• Receiving support did not impede their growth as
  a reader and enhanced self-confidence (opinion)

69
(No Transcript)
70
Discussion

• Teacher-researcher with the Santa Barbara
  Classroom Discourse Group caused her to focus on
  what was happening to the pullout segment of her
  classroom.
• Encourages school districts to review the way
  specialized instruction is delivered.

71
Conclusion

• Feels that pullout programs may retard social
  interaction
• Supports learner remaining in the classroom

72
Limitations

• Extremely small sample size
• Biased because person collecting the research was
  frustrated with pullout
• Made generalizations that werent based on data

73
Parents attitudes to inclusion of their children
with special needs

• Elkins, J., van Kraayenoord, C., Jobling, A.
  (2003). Journal of Research in Special
  Educational Needs, 3, 122-129.

74
Summary

• Investigated the attitudes of 354 Australian
  parents who have a child with a disability and
  who attends a state school in Queensland.
  Students were receiving a continuum of services.
  Many parents favored inclusion and a small group
  favored special placement.

75
Method

• Coding schemes were devised to record the
  responses
• Data was analyzed to generate the frequencies,
  means and standard deviations.
• Responses to open-ended questions were
  transcribed.

76
Sample

• 100 preschools, 150 primary schools, and 150
  secondary schools were randomly selected from
  lists in the database of Education Queensland
  schools.
• 2 parents of students with disabilities were
  selected by the principal to complete the Parent
  Survey
• Tried to get equal number of parents from each
  disability group

77
Instrument

• Survey of Parents Attitudes and Opinions About
  their Children with Special Needs and their
  Support was adapted by the researchers from the
  Survey of Teacher Attitudes and Opinions about
  Students with Special Needs and the Types of
  Support for Integration/Inclusion used in the
  collaborative project with the Korea Institute
  for Special Education

78
Results

• 354 parents responded
• Almost all parents were strongly or moderately
  supportive of the benefits of inclusion for
  children with special needs in general.
• 50 percent favored special classes when it came
  to their own child

79
Discussion

• Most salient benefits of inclusion were
• Social interaction
• Greater independence
• Greater understanding and tolerance by peers
• Friendship with non-disabled peers
• Imitating behaviors of peers

80
Conclusion

• 70 percent of parents regarded their child as
  requiring more patient teachers, extensive change
  in regular classroom procedures, and substantial
  additional training for regular teachers but
  regarded special class placement as causing
  slower social and emotional development.
• Catch 22 do you want academic or social growth?

81
Limitations

• Principal selected participants
• Validity and reliability of instrument is
  questionable since researchers changed the
  instrument

82
Implications for Policy Change in Public Schools

• Does the school staff support pullout programs?
  If not, other options need to be explored.
• Examine scheduling carefully. What do students
  who attend pullout programs miss in the
  classroom?
• Is it feasible for the pullout teacher to work
  with the classroom teacher in the classroom?

83
Policy concerns - schools

• Heterogeneous classes are those classes that are
  structured to include students from some or all
  of the following different ethnic and cultural
  groups, high and low achievers, students with
  challenging behaviors, students identified as
  gifted and talented, students with special needs,
  English as a Second Language learners, and
  students with social/emotional difficulties.
  While teachers do the best they can, how are they
  supposed to meet the needs of all of these
  students? School administrators and teachers
  will have to carefully examine current policy and
  determine if they are adequately meeting the
  needs of the heterogeneous class.

84
Policy concerns schools

• If general education classrooms are going to
  support heterogeneous groups, then
  differentiation of instruction will be
  imperative. Do all teachers know how to provide
  differentiated instruction?

85
Implications for Policy Changes in Higher
Education

• Changing roles for classroom teachers and
  specialists need to be addressed
• Preservice teachers need to develop effective
  instructional and interpersonal skills to work
  with colleagues
• Classroom management skills are key for differing
  service delivery models
• Today's teachers must deal, as never before, with
  heterogeneity in their classrooms

86
Is pullout an effective way to help students who
are weak in particular subjects?

• The research articles found do not support
  pullout as being any more effective than leaving
  students in classrooms.
• Music students were leaving for a different
  reason and being out of the classroom did not
  negatively impact their scores on statewide tests.

87
Is there sufficient evidence to reach consensus
on this matter?

• Most of the evidence suggests that students do
  just as well or better when they are not pulled
  out.
• However, in the area of students with
  disabilities, this evidence is for students with
  mild disabilities (usually LD)
• In the area of pullout for music students, they
  actually performed better on statewide tests in
  certain subjects and the same in others

88
What evidence is missing and what research might
be done to fill the gaps?

• Evidence for students with ED, Autism, Mild and
  Moderate Retardation is missing. Research with
  other disability types is needed.
• Do students in MS and HS benefit as much from a
  more inclusive approach because the gap often
  becomes wider. When team teaching, can
  individual skills really be addressed? Research
  with older students would be beneficial.

89

• Impact of pullout programs is a very hard thing
  to measure because there are too many other
  variables.