With fiscal pressures to produce many students can educational leadership programs achieve quality

1
With fiscal pressures to produce many students
can educational leadership programs achieve
quality?

• Mark J. Fenster
• fenster_21stcentury_at_hotmail.com
• Valdosta State University

2
Fiscal pressures paper

• Paper presented at the annual meeting of the
  Southeastern Evaluation Association
• Tallahassee FL
• February 2, 2006
• HaPpY gRoUnDhOgDaY!

3
Introduction

• Arthur Levine wrote and disseminated a critique
  of administrator preparation in the United
  States, Educating School Leaders. The study
  was widely distributed in the popular press and
  received national and international attention.

4
Introduction

• The American Educational Research Association
  (AERA) Division A put out a special newsletter in
  the spring 2005 to address some concerns of the
  Levine (2005) study.
• Young, Grow, Orr, and Ogawa (2005) point out in
  their rebuttal of the Levine study (p. 4), for
  example, he Levine claims that programs
  educational leadership programs have low
  admissions standards but provides as evidence the
  mean scores of test takers, not those who were
  admitted.

5
Introduction

• If students desiring entry into educational
  leadership programs have low test scores, but
  those matriculating into these same leadership
  programs have test scores similar to other
  graduate programs, then the Levine criticism of
  low admission standards can be dismissed.

6
Introduction

• However, if students matriculating into
  educational leadership programs have lower test
  scores when compared to other graduate programs,
  then the Levine criticism of low admission
  standards can be supported.

7
Introduction

• Using administrative data from one institution,
  called XYZ, we can test the Levine contention
  that students matriculating in educational
  leadership programs have lower test scores when
  compared to other graduate programs run by the
  same institution.

8
Leadership preparation

• Different paths to certification by prospective
  educational leaders
• university training program to receive an
  advanced degree and certification.
• Alternative certification programs for people
  with managerial experience from business have
  sprung up in some states.
• Additionally, some very large school districts
  (like New York City) run their own leadership
  preparation program (Gootman, 2004).

9
Leadership preparation

• The New York City leadership program is
  particularly interesting because the goal in the
  program is to teach people how to become
  principals.

10
Leadership preparation

• As Levine (2005) notes, nobody knows if
  university sponsored educational leadership
  programs are any better than a district based
  program, or one of the alternative programs in
  training effective educational leaders. Without
  a base of knowledge to say that one type of
  leadership preparation program is better than
  another, a mix of potential avenues to achieve
  leadership certification can be expected to
  continue to exist in the next few years.

11
What to do with the Ed.D. degree?

• What to do with the Ed.D. degree in educational
  administration
• Levine (2005) also focused attention on the
  doctorate in educational administration,
  especially the doctorate of education degree
  (Ed.D). Levine argued to do away with the Ed. D.
  degree, arguing the professoriate was cited for
  poorly preparing their students as researchers,
  and being inexperienced in or incapable of
  carrying out or supervising quality research
  themselves (p. 44).

12
What to do with the Ed.D. degree?

• Hawley (1988) noted that Few persons teaching in
  Ed. Adm. doctoral programs are now or ever have
  been involved in research and are not qualified
  to supervise research. Thus, very little good
  research is being conducted by faculty and
  students (p. 85).
• If educational administration professors do
  little research, is it surprising that their work
  and their students work add little to theory and
  to the educational administration knowledge base
  (Achilles, 1990, 1991 Achilles Finn, 2002)?

13
Methods

• Data Sources
• The primary data source was the institutions
  administrative database, including grade point
  averages and Scholastic Aptitude Test (SAT)
  scores (total, verbal and math) on all college of
  education graduate students.
• Additional data came from questionnaires
  distributed to students by the instructor on
  their first day of class. This questionnaire
  asked whether the student had any interest in
  becoming an educational leader and the reason for
  enrolling in the program.

14
Methods

• Variables
• Dependent Variables.
• There were three dependent variables analyzed in
  this study
• (1) SAT scores of graduate students matriculating
  at XYZ,
• (2) grade point averages of graduate students
  enrolled at XYZ, and
• (3) a combined z-score of overall academic
  performance when enrolled at XYZ.

15
Methods

• Independent Variables.
• There were three independent variables analyzed
  in this study.
• One independent variable was the program a
  student matriculated into (psychology,
  educational leadership, early childhood, middle
  grades, or music education, to mention some
  pertinent examples).

16
Methods

• A second independent variable was the students
  response to a question whether they had an
  interest in becoming an educational leader. Data
  on this variable was collected though a
  questionnaire handed out to students on the first
  day of class.
• A third independent variable was whether a
  student became an educational leader. Data on
  this variable was collected through contacts with
  former students, and with these contacts we were
  able to ascertain the exact number of students
  who became leaders.

17
Results

• Analyzing data from XYZ, we found that
  educational leadership students had the second
  lowest SAT total scores across 10 graduate
  programs.

18
Results

• Educational leadership students scored, on
  average, about 84 points below the overall
  graduate mean on their total SAT scores.
• Educational leadership students scored 54 points
  below the average SAT verbal score, and 30 points
  below the average SAT math score.

19
Table 1 SAT scores by graduate program
20
(No Transcript)
21
(No Transcript)
22
Results

• Educational Leadership students had one of the
  lowest average SAT scores, but that did not
  prevent them from getting virtually all As in
  coursework, as shown in Table 2.

23
Results

• Educational leadership students had nearly the
  highest grade point averages of the ten graduate
  programs.
• The median grade point average of educational
  leadership graduate students was 3.94, second
  only to Music Educations 4.00.
• However, students in Music Education had an
  average SAT score 183 points higher than the
  educational leadership students.

24
Results

• The only program admitting students with lower
  SAT scores than Educational Leadership was
  Physical Education. In contrast, students in the
  Physical Education program had the lowest median
  grade point average of the graduate programs at
  XYZ.

25
Table 2 Analysis of graduate student grade point
averages by degree program, as of March 1, 2005.
26
Results

• Interest in becoming an educational leader
• The questionnaire allowed us to more closely
  analyze those students desiring to become
  educational leaders compared to those students
  who reported to us that they had no interest in
  becoming leaders. At XYZ, about 85 of enrolled
  students in educational leadership programs
  reported on a questionnaire that they had no
  interest in becoming an educational leader.

27
Results

• Analysis of our data revealed that those students
  reporting no interest in becoming an educational
  leader performed slightly better in the program
  than students reporting that they wanted to
  become educational leaders, as shown in Figure 1.
  

28
Figure 1. Differences in performance between
students interested and not interested in
becoming educational leaders.
29
Results

• Eighty-six percent reported they had no interest
  in becoming educational leaders. These students
  averaged about 0.09 standard deviations better
  than the typical student in academic work.
• The students reporting that they desired to
  become educational leaders (14) averaged 0.06
  standard deviations worse than the typical
  student.

30
Results

• The difference in performance between students
  with no interest in educational leadership and
  those who were interested in educational
  leadership, about 0.15 standard deviation units,
  was statistically insignificant (t0.30, df17,
  ns). The effect size, 0.16, would be considered
  below small by Cohens (1988) criterion.

31
Results

• In Figure 2 we disaggregate students desiring to
  become educational leaders into two groups those
  who were interested in becoming educational
  leaders but have not yet secured a leadership
  position, and those who were interested in
  becoming educational leaders and actually became
  educational leaders.

32
Results

• Those interested in becoming educational leaders,
  but did not yet become a leader performed well in
  the program.
• These students averaged 0.64 standard deviations
  better than the typical student.

33
Results

• Those who were interested in becoming educational
  leaders and actually became educational leaders
  performed poorly in the program. These students
  averaged 0.75 standard deviations worse than the
  typical student.

34
Results

• These two groups were compared to those students
  reporting they had no interest in becoming
  educational leaders.
• From Figure 1 we know that students with no
  interest in becoming educational leaders averaged
  a performance nearly 0.09 standard deviations
  better than the typical student

35
Results

• The differences presented in Table 4 were
  statistically insignificant (f1.22, df2, ns),
  with a small effect size (.08) calculated per
  Cohen (1988, pp. 407-415).

36
Figure 2. Differences in academic performance
between those with no interest in educational
leadership, those who want to become an
educational leader but have not yet secured a
leadership position, and those who want to become
an educational leader and actually became an
educational leader.
37
Discussion

• Overall, this study supports one contention of
  Levines (2005) study and refutes one claim by
  Young et al. (2005).
• Specifically, the educational leadership programs
  at XYZ took in students with nearly the lowest
  test scores.

38
Discussion

• Paradoxically, once in the program, virtually all
  educational leadership students received the
  grade of A in every course.

39
Discussion

• Educational leadership programs have to be able
  to play the role of gatekeeper if they are going
  to be a positive force in producing the next
  generation of effective educational leaders. The
  practice of letting in weak students, giving all
  students the grade of A, and passing all
  students through the system meant that all
  students, both weak and strong were certified to
  become leaders.

40
Discussion

• Superintendents and other personnel in these
  districts had a plethora of choices when it time
  to fill leadership vacancies, and given the
  option of choosing academically weak or strong
  students, local school districts chose the
  weakest students to become educational leaders.

41
Discussion

• Local school districts may not have known (or
  cared) that their selected candidates to become
  leaders from XYZ were from the bottom 15 of
  their respective classes.
• However, if the educational leadership program at
  XYZ had served a better role as gate-keeper, with
  more rigor in course grading resulting in fewer
  As, weaker students may have been filtered out
  and not able to secure certifications as leaders.

42
Discussion

• Leadership preparation programs like XYZ make it
  easier for local school districts to hire
  academically weak, marginally qualified people as
  principals.
• These weak students selected as leaders are
  minimally qualified by the standards of the State
  and the educational leadership program.
• More academically highly qualified people who
  want to become principals wait for leadership
  appointments as weak students are chosen first.

43
Discussion

• Differences in incentives between university and
  district based leadership programs
• University based educational leadership programs
  may have greater problems playing the gatekeeper
  role than large district based programs because
  university programs generate revenue by producing
  students.
• The incentives in university based educational
  leadership programs are weighted to succumbing to
  the all too easy practice of letting in weak
  students, and then giving virtually every student
  the grade of A.

44
Discussion

• The result of this process gives leadership
  programs accolades within the institution from
  high ranking administrators due to a programs
  high retention and low attrition.
• The retention rate of educational leadership
  students at XYZ routinely exceeded 95.

45
Discussion

• In large district programs, like the New York
  City example, there would seem to be the greater
  potential for appropriate gate-keeping, since the
  programs reputation would be harmed if graduates
  consistently performed poorly on the job.
• The retention rate of educational leadership
  students in the first graduating class at the New
  York City program was 85. The 15 dropout rate
  was one piece of evidence cited for the programs
  rigor (Gootman, 2004).

46
Discussion

• Additionally, the motivation for enrolling in an
  educational leadership program would be quite
  different in a district based leadership program,
  compared to a university based leadership
  program.
• In a district leadership program like the New
  York City example, 100 of attendees become
  principals (Gootman, 2004).
• However, becoming a principal is not the primary
  motivation of the vast majority of students
  entering a university based educational
  leadership program (Davis et al., 2005 Levine,
  2005).

47
Discussion

• The primary motivation for students enrolling in
  a masters degree in educational leadership is
  receiving a salary increase (Davis et al.
  Levine). The State of Georgia pays students
  roughly 6400 for people employed in the k-12
  environment receiving a masters degree,
  irrespective of whether the masters degree is in
  or out of field.

48
Discussion

• XYZs educational leadership programs enroll many
  of these 6400 desiring salary increasing
  students because the program is convenient for
  them to attend, and perhaps, because it is an
  easy program to complete.
• Low admission standards and all A grading
  distributions did not negatively impact the
  departments reputation externally because so few
  of its graduates ever become educational leaders.

49
Discussion

• With few graduates as leaders, there are very few
  disaster stories of alumni performing poorly on
  the job. Stated another way, the 14 who desire
  educational leadership positions at XYZ were
  dispersed across many school districts. Some of
  these 14 may go on to become educational
  leaders.

50
Discussion

• However, the small numbers of people who actually
  become educational leaders, combined with the
  relatively large number of school districts in
  the area (44 in XYZs service area), make it
  unlikely that any one school district would
  experience multiple job performance failures from
  graduates of XYZ.

51
Discussion

• All the fiscal incentives at universities push
  educational leadership programs in the direction
  of letting in virtually all students who apply,
  then giving the grade of A to nearly all
  students in all courses.
• Levine (2005) called this process the race to the
  bottom.
• To take any other path, university based
  educational leadership programs take the risk of
  declining enrollments and downsized departments.

52
Recommendation

• The State of Georgia should consider reducing the
  pay incentive for the doctoral degree. The
  current pay incentive, 6400, is one of the
  largest in the country.
• Florida pays less than 3200 for a doctoral
  degree.

53
Recommendation

• Reducing the pay incentive for doctorates would
  reduce the demand in Georgia for a research
  degree that really does not fit what educational
  administrators do.