The ABET Process and the ECE community

1
The ABET Process and the ECE community

• Comments by Moshe Kam
• ECE Department, Drexel University
• 21 March 2009

2
First, a Few Numbers

• which may indicate that the ABET accreditation
  process has been increasingly harder for
  programs to manage

3
EAC ACTIONS 1997-2007
NGR IR IV SC NA 1997 80
10 7 2 1 1998 77 12 9 3 1
1999 78 11 8 3 1 2000 66 22
11 1 1 2001 72 13 14 1 1
2002 68 21 11 1 0 2003 77 17
5 1 0 2004 71 20 7 1 1 2005
72 22 5 1 0 2006 65 26 9 0
0 2007 65 30 5 0 0
http//abet.org/Linked20Documents-UPDATE/Stats/07
-AR20Stats.pdf
4
EAC ACTIONS 1997-2007
http//abet.org/Linked20Documents-UPDATE/Stats/07
-AR20Stats.pdf
5
EAC ACTIONS 1997-2007
http//abet.org/Linked20Documents-UPDATE/Stats/07
-AR20Stats.pdf
Collectively, we are not getting better at this
6
Now a Few Theses
7
Main Theses (1)

• (1) The focus of the ABET review process has
  shifted from Outcomes to the Assessment Process
• It is questionable whether this provides our
  students with better education
• (2) The value of accreditation is questioned
• By industry
• as demonstrated by industry's actions
• By programs that worry about Return on Investment
  

8
Main Theses (2)

• (3) The complexity of the ABET evaluation process
  encourages wholesale adoption of Program Outcomes
  a-k
• variations and additions make life harder
• the result is less diversity
• exactly the opposite of EC2000s intent
• (4) The fact that ABET is a monopoly is a problem

9
Main Theses (3)

• (5) There are significant opportunities to
  simplify and automate the review process
• make it continuous and transparent using
  information technology
• (6) Opportunities of the ECE academic community
  to be heard by ABET go unused

10
Thesis 1 The focushas shifted from Outcomes to
the Assessment Process

• Criterion 2. Program Educational Objectives
• Each program for which an institution seeks
  accreditation or reaccreditation must have in
  place
• (a) published educational objectives that are
  consistent with the mission of the institution
  and these criteria
• (b) a process that periodically documents and
  demonstrates that the objectives are based on the
  needs of the program's various constituencies
• (c) an assessment and evaluation process that
  periodically documents and demonstrates the
  degree to which these objectives are attained.

This criterion is touchy feely, i.e., more
qualitative than the others ECEDHA March 2008
ABET Workshop
11
Thesis 1 The focushas shifted from Outcomes to
the Assessment Process

• Most identifies weaknesses are due to Criterion 2
• Reports to the ABET BoD in November 2007 and
  November 2008
• Increasingly, schools are hiring assessment
  consultants or adding assessment staff in
  Colleges of Engineering
• Was the emergence of this industry envisioned?

12
Letter of the Consortium of Associate Deans of
the Big Ten Plus Consortium of Engineering
Schools (June 2007)

• the use of assessment tools is a choice left to
  the institutionsbut an evaluator will penalize a
  program if they have used a method that the
  evaluator thinks is inappropriate.
• This experience comes across as, "We won't tell
  you the right way to do it, but we'll be the
  first to tell you if you're doing it wrongly.

13
June 2007 Big 10 Letter (2)

• Neither faculty nor evaluators commonly are
  experts in learning assessment, so this situation
  runs the risk of putting programs at the mercy of
  chance pairings with evaluators.
• This stifles innovation, as faculty may then
  stick only with "tried and true" methods with
  which some previous evaluator had declared
  compliance.

14
June 2007 Big 10 Letter (2)

• For solutions to the moving target problem, we
  urge the EAC to be deliberate in the pace at
  which they make changes to the Criteria.
• We also urge the EAC to articulate more clearly
  how to interpret Criteria wording and its
  expectations of good practice in complying with
  them.

15
Some related thoughts

• If criterion 2 disappeared, who will miss it?
• If criterion 2 disappeared, will it harm
  engineering education?
• How about returning the focus to OUTCOMES and not
  to ASSESSMENT PROCESSES??

16
Thesis 2 The value of accreditation is
questioned

• Industry is voting with its feet
• Little support for program evaluators
• Absent at the table of ABETs BoD
• Industry does not participate in sustaining ABET
• Industry's influence on ABETs policies is
  indirect
• The role of licensing in ECE is diminished and
  diminishing
• Meantime in Academia
• Engagement in the process is often limited to few
  faculty and members of the staff
• Persistent questions on ROI

17
Some related thoughts

• If ABET disappeared, who will miss it?
• If ABET disappeared, what alternative quality
  control and evaluation is our community likely to
  adopt?
• If we were to start a new accrediting body for
  Engineering now, will it look like ABET?
• The answers to these questions may help us
  understand where ABET should move

18
Thesis 3 Theprocess encourages wholesale
adoption of Program Outcomes a-k

• We have adopted ABET a-k as our program
  outcomes
• Iowa State University ECPE www.eng.iastate.edu/hl
  c/ecpeoutcomes.asp
• The ABET a-k outcomes, as adopted by the
  Department of Industrial and Manufacturing
  Engineering, are highlighted
• North Dakota State University Survey
  http//www.ndsu.edu/fileadmin/ime/Assessment_files
  /IE_M-GraduationSeniorsExitInterviewSurvey.pdf
• We have adopted the ABET a-koutcomes as our
  Materials Science and Engineering Program
  Outcomes
• Self Study Report OSU MSE Department
  http//www.matsceng.ohio-state.edu/ug/abet/05_ABET
  _MSE.pdf
• Purdue https//engineering.purdue.edu/ECE/Academi
  cs/Undergraduates/ProgramObjectivesandOutcomes
• Drexel http//www.ece.drexel.edu/Mission.html

19
Some related thoughts

• We seem to have achieved the exact opposite of
  what EC2000 tried to achieve
• Uniformity rather than Diversity
• As long as demonstrating compliance with outcomes
  is so expensive, it will not be possible to
  address this issue
• Back to Criterion 2, I think

20
Thesis 4 ABET is a Monopoly

• One wonders if competition would
• Improve responsiveness to constituents
• Increase the circle of consistencies
• Currently major stake holders are professional
  associations and the reviewed institutions
• Where are industry, government and students?
• Provide plurality of methods and approaches
• Encourage processes that are more efficient
• Reduce costs

21
Thesis 5 There are significant opportunities to
simplify and automate the review process

• There are very few issues that still require a
  site visit and face-to-face interviews
• Pertinent data can be made available on-line and
  be updatet automatically
• There are opportunities to transform the process
  from a discrete dramatic event every few years to
  a truly continuous process

22
Thesis 6 Opportunities of the ECE academic
community to be heard by ABET go unused

• IEEE has three representatives on the ABET Board
  of Directors
• They are recommended by and work closely with the
  IEEE Educational Activities Board
• The IEEE Educational Activities Board operates
  the Accreditation Policy Council
• Recommends policies and actions to EAB and IEEE
  BoD
• IEEE has an official policy about accreditation

23
Do you know these fellows?
IEEE Representative Directors to the Board of
Directors of ABET
Michael Lightner
Moshe Kam
Bruce Eisenstein
24
Questions or Comments