ALIGNING THE COMPUTER ENGINEERING DEPARTMENT WITH ABET EC 2K - PowerPoint PPT Presentation

About This Presentation
Title:

ALIGNING THE COMPUTER ENGINEERING DEPARTMENT WITH ABET EC 2K

Description:

Title: PowerPoint Presentation Author: ECE Publications Last modified by: mayez Created Date: 11/8/2001 3:05:58 PM Document presentation format: On-screen Show – PowerPoint PPT presentation

Number of Views:4173
Avg rating:3.0/5.0
Slides: 120
Provided by: ECEPubli9
Category:

less

Transcript and Presenter's Notes

Title: ALIGNING THE COMPUTER ENGINEERING DEPARTMENT WITH ABET EC 2K


1
ALIGNING THE COMPUTER ENGINEERING DEPARTMENT
WITH ABET EC 2K
  • Computer Engineering Department
  • King Fahd University of Petroleum and Minerals

2
CONTENTS
  • ABET EC 2K Motivation
  • ABET EC 2K Program Outcomes and Outcomes
    Assessment (2004/5)
  • Aligning the Computer Engineering Department with
    ABET EC 2K
  • Assessment results for the COE
  • Rubrics Assessment Process
  • Course Outcomes Assessment
  • Indirect Assessment
  • Conclusion

3
ABET EC 2K Motivation
4
ABET EC 2K
  • Cost of education (mid 90s) More than 15 of
    median family income
  • Industry mutation Engineers should be educated
    in a way to facilitate their adaptation to
    important industry mutation (Bio, Nano, Health,
    Comp. Technology, change in products, etc)
  • (not to be tight to one original area of
    formation)
  • Demographic issues, population aging, etc.

5
ABET EC 2K
  • Shift in Engineering Education
  • Teacher-Centered Focus
  • Traditional classroom
  • Distance learning using synchronous
    video-conferencing
  • Learner-Centered Focus (outcome-based)
  • Interactive multimedia
  • Educational solutions designed by multi-skilled
    academic teams
  • Asynchronous learning (any time and anywhere)
  • Since 1995 a shift in ABET criterion A
    meaningful, major engineering design experience
  • Pilote ABET visits in 96-98, required to meet EC
    2K by 2001

6
Where did EC2K Come From? Early 1990s
  • NSF Study
  • Deans Council (American Society for Engineering
    Education)
  • National Research Council
  • Much industrial involvement
  • Led directly to EC2000
  • EC 2K New Paradigm
  • Demands will be for the solution of problems
    involving human values, attitudes, behavior
  • And interrelationships and dynamics of social,
    political, environmental, and economic systems
  • And on a global basis

7
New Paradigm
  • Emphasis on inquiry-based learning
  • Preparation for lifelong learning
  • Stress integrative, systems thinking
  • Coping with change
  • Communications skills, including listening
  • Group skills, identification through finish
  • Ecoefficient designfocus on design issues
    involving life-cycle economics, environmental
    impact, sustainable development, ethics,
    timeliness, quality, health safety,
    manufacturability, social, legal, standards, and
    ad hoc concerns.
  • Continuous Process Improvement (as in Baldridge,
    ISO 9000, etc)

8
Some Results
  • More emphasis on soft skillswith no decrease
    in technical content
  • Communication
  • Lifelong Learning
  • Multidisciplinary
  • Teamwork
  • Ethics and Professionalism
  • Emphasis on learning, not teaching
  • And design..Program Outcomes and Assessment
    (Criterion 4)
  • major design experience based on the knowledge
    and skills acquired in earlier course work and
    incorporating engineering standards and realistic
    constraints that include most of the following
    considerations economic environmental
    sustainability manufacturability ethical
    health and safety social and political.

9
Program Outcomes and Assessment (2004/5)
  • Some Outcomes are Easily Understood and
    Implemented
  • a Science, Mathematics, Engineering
  • b Experiments
  • c Design
  • e Solve engineering problems
  • k Modern tools (software)
  • Others Present New Challenges
  • d Multidisciplinary
  • f Professional, ethical
  • g Communicate
  • h Broad education, societal, global
  • i Lifelong learning
  • j Contemporary issues

10
ABET EC 2K PROGRAM OUTCOMES AND OUTCOMES
ASSESSMENT (2004/5)
11
Criterion 1 Students
  • The quality and performance of the students are
    important in the evaluation of an engineering
    program.
  • The institution must
  • evaluate, advise, and monitor students to
    determine its success in meeting program
    objectives.
  • have and enforce policies for the acceptance of
    transfer students and for the validation of
    courses taken for credit elsewhere.
  • enforce procedures to assure that all students
    meet all program requirements.

12
Criterion 2. Program Educational Objectives
  • The program educational objectives describe the
    expected accomplishments of graduates during the
    first several years following graduation from the
    program. (NEW)
  • An engineering program must have in place
  • Published educational objectives that are
    consistent with the mission
  • a process based on the needs of the program's
    various constituencies in which the objectives
    are determined and periodically evaluated

13
Criterion 2. Program Educational Objectives
(Contd)
  • A curriculum and processes that prepare students
    for the achievement of these objectives (REVISED)
  • A system of ongoing evaluation that demonstrates
    achievement of these objectives and uses the
    results to improve the effectiveness of the
    program.

14
Criterion 3. Program Outcomes and Assessment
  • The program outcomes are intended to be
    statements that describe what students are
    expected to know or be able to do by the time of
    graduation from the program. (NEW)
  • Engineering programs must demonstrate that their
    graduates have
  • (a) an ability to apply knowledge of mathematics,
    science, and engineering
  • (b) an ability to design and conduct experiments,
    as well as to analyze and interpret data

15
Criterion 3. Program Outcomes and Assessment
  • (c) an ability to design a system, component, or
    process to meet desired needs
  • (d) an ability to function on multi-disciplinary
    teams
  • (e) an ability to identify, formulate, and solve
    engineering problems
  • (f) an understanding of professional and ethical
    responsibility

16
Criterion 3. Program Outcomes and Assessment
  • (g) an ability to communicate effectively
  • (h) the broad education necessary to understand
    the impact of engineering solutions in a global
    and societal context
  • (i) a recognition of the need for, and an ability
    to engage in life-long learning
  • (j) a knowledge of contemporary issues
  • (k) an ability to use the techniques, skills, and
    modern engineering tools necessary for
    engineering practice.

17
Criterion 4. Professional Component (Curriculum)
  • Specify subject areas appropriate to engineering
    but do (not courses)
  • The curriculum devotes adequate attention and
    time to each component, consistent with the
    objectives of the program and institution.
  • Students must be prepared for engineering
    practice
  • a curriculum culminating in a major design
    experience based on the knowledge and skills from
    earlier course work
  • incorporating engineering standards economic
    environmental sustainability manufacturability
    ethical health and safety social and
    political.

18
Criterion 4. Professional Component (Curriculum)
  • The professional component must include
  • (a) one year of a combination of college level
    mathematics and basic sciences
  • (b) one and one-half years of engineering topics,
    consisting of engineering sciences and
    engineering design
  • (c) a general education component that
    complements the technical content of the
    curriculum and is consistent with the program and
    institution objectives.

19
Criterion 4. Professional Component (Curriculum)
  • ABET 2004-2005 criteria added just before a-k
  • Each program must formulate program outcomes that
    foster attainment of the program objectives.
  • Design requirements strengthened again
  • There must be processes to produce these outcomes
    and an assessment process, with documented
    results, that demonstrates that these program
    outcomes are being measured and indicates the
    degree to which the outcomes are achieved.
  • There must be evidence that the results of this
    assessment process are applied to the further
    development of the program.

20
Proposed Changes 2005-2006
  • There must be processes to produce these outcomes
    and an assessment process, with documented
    results, that demonstrates that these program
    outcomes are being measured and indicates the
    degree to which the outcomes are achieved. There
    must be evidence that the results of this
    assessment process are applied to the further
    development of the program.

21
Criterion 5. Faculty
  • The faculty must be of sufficient number and
  • Must have the competencies to cover all of the
    curricular areas of the program.
  • There must be sufficient faculty to accommodate
  • adequate levels of student-faculty interaction,
  • student advising and counseling,
  • university service activities,
  • professional development, and
  • interactions with industrial and professional
    practitioners, as well as employers of students.

22
Criterion 5. Faculty (Contd)
  • The program faculty must have appropriate
    qualifications and demonstrate sufficient
    authority to
  • Ensure the proper guidance of the program,
  • Develop and implement processes for the
    evaluation, assessment, and continuing
    improvement of the program, its educational
    objectives and outcomes.
  • The factors determining faculty competence
  • Education and diversity of backgrounds,
  • Engineering experience and teaching experience,
  • Ability to communicate, enthusiasm for developing
    more effective programs,
  • level of scholarship, participation in
    professional societies, and registration as
    Professional Engineers.

23
Criterion 6. Facilities
  • Classrooms, laboratories, and associated
    equipment must be adequate to accomplish the
    program objectives and provide an atmosphere
    conducive to learning.
  • Appropriate facilities must be available to
    foster faculty-student interaction and to create
    a climate that encourages professional
    development and professional activities.
  • Programs must provide opportunities for students
    to learn the use of modern engineering tools.
  • Computing and information infrastructures must be
    in place to support the scholarly activities of
    the students and faculty and the educational
    objectives of the institution.

24
Criterion 7. Institutional Support and Financial
Resources
  • Institutional support, financial resources, and
    constructive leadership must be adequate to
    assure the quality and continuity of the
    engineering program.
  • Resources must be sufficient to attract, retain,
    and provide for the continued professional
    development of a well-qualified faculty.
  • Resources also must be sufficient to acquire,
    maintain, and operate facilities and equipment
    appropriate for the engineering program.
  • In addition, support personnel and institutional
    services must be adequate to meet program needs.

25
Criterion 8. Program Criteria
  • Distinguish branches of engineering
  • In EE and Computer Engineering
  • Breadth and depth
  • Ability to analyze and design complex devices,
    software, and systems
  • In Electrical Engineering
  • Knowledge of Advanced Mathematics
  • In Computer Engineering
  • Knowledge of Discrete Mathematics

26
Criterion 8. Program Criteria
  • These exist for all branches of engineering, e.g.
    ME, IE, ChemE, Civil, etc.
  • These deal with faculty qualifications and with
    curricular matters

27
ALIGNING THE COMPUTER ENGINEERING DEPARTMENT
WITH ABET EC 2K
28
INSTITUTION VISION
  • KFUPM vision
  • To be a vibrant multicultural University of
    international repute focuse on quality education
    and innovative research that prepares
    professionals and entrepreneurs to lead social,
    economic and technical development in the region.
  • CCSE main objectives
  • To provide the skilled manpower needed for the
    fulfillment of the country's development plans.
    In particular information computer scientists,
    computer engineers, and systems engineers.
  • To prepare students for graduate work and
    research in their field of specialization.
  • To provide a link through which computer
    technologies and their applications could be
    transferred to the country.
  • To provide the country, through research and
    graduate studies, with skills, ideas, and
    innovations in certain areas of advanced
    technologies.

29
EDUCATIONAL OBJECTIVES
  • The objectives of the COE program
  • To produce computer engineering
    graduates prepared to
  • Objective 1 Practice their profession with
    confidence and global competitiveness and make
    intellectual contributions to it
  • Objective 2 Pursue a life-long career of
    personal and professional growth with superior
    work ethics and character and
  • Objective 3 Pursue advanced study and research
    at the graduate level.

30
Program Outcomes
  • (a) an ability to apply knowledge of mathematics,
    science, and engineering
  • (b) an ability to design and conduct experiments,
    as well as to analyze and interpret data
  • (c) an ability to design a system, component, or
    process to meet desired needs
  • (d) an ability to function on multi-disciplinary
    teams
  • Our interpretation of multidisciplinary
    teams includes teams of individuals with similar
    educational backgrounds focusing on different
    aspects of a project as well as teams of
    individuals with different educational
    backgrounds.
  • (e) an ability to identify, formulate, and solve
    engineering problems

31
Program Outcomes (Cont)
  • (f) an understanding of professional and ethical
    responsibility
  • (g) an ability to communicate effectively
  • (h) the broad education necessary to understand
    the impact of engineering solutions in a global
    and societal context
  • (i) a recognition of the need for, and an ability
    to engage in life-long learning
  • Our interpretation of this includes teaching
    students that the underlying theory is important
    because the technology will change, coupled with
    enhancing their self-learning ability.

32
Program Outcomes (Cont)
  • (j) knowledge of contemporary issues
  • Our interpretation of this includes
    presenting students with issues such as the
    impact of globalization, the outsourcing of both
    engineering and other support jobs as practiced
    by modern international companies.
  • (k) an ability to use the techniques, skills, and
    modern engineering tools necessary for
    engineering practice.
  • (l) Knowledge of Probability and Statistics and
    their applications in Computer Engineering
  • (m) Knowledge of Discrete Mathematics
  • (n) The ability to design a system that involves
    the integration of hardware and software
    components

33
Relation of Program Outcomes and Educational
Objectives
Program Educational Objectives Program Outcomes
1. Practice profession with confidence and global competitiveness and make intellectual contributions to it a, b, c, d, e, g, k, l, m , n
2. Pursue a life-long career of personal and professional growth with superior work ethics and character f, i, h, j
3. Pursue advanced study and research at the graduate level a, b, e, g, i, k
34
(No Transcript)
35
Program Outcomes Coverage in Curriculum
  • Program outcomes are injected and well covered by
    core courses in the program curriculum.
  • Each program outcome is addressed by a set of
    core courses in the program.
  • Learning outcomes of core courses mapped to
    program outcomes with a level of emphasis being
    either low (L), medium (M), or High (H).
  • Level of emphasis for an outcome is determined
    based on the weight as follows
  • Course outcome weight lt 10, given a Low rank
    (L).
  • Course outcome weight between 10 and 20 given
    a Medium rank (M).
  • Course outcome weight ? 20 given a High rank
    (H).

36
Program Outcomes Coverage in Curriculum
A B C D E F G H I J K L M N
COE 202 H H L
COE 203 M H L L H
COE 205 H L L L
COE 305 M L H L H L L
COE 308 H H L L L
COE 341 M H H L L
COE 344 M L H L L
COE 360 L L H L L M
COE 390 M H L M M
COE 400 M M M L M L L H
COE 485 L M H M L L M L L L M
COE 351 H M M H M M
COE 399 M M H M H
STAT 319 H
ICS 252 H
IAS 211 H
ENGL 214 H
37
Program Educational Objectives Assessment Process
38
Program Outcomes Assessment
39
Assessment Method
  • Indirect Assessement
  • Industrial Advisory Board
  • Employer Survey
  • Alumni Survey
  • Graduate Exit Survey
  • Student Survey (course)
  • Direct Assessement
  • Computer Engineering Exit Exam
  • Rubrics
  • Microcomputer System Design (COE 305) lab
    (outcome b)
  • Computer Networks lab (COE 344) lab (outcome b)
  • Cooperative Work (COE 350/351) (outcomes a, c, d,
    e, g, h, i, j, k, and n)
  • Seminar (COE 390) (outcome f)
  • Summer Training (COE 399) (outcome g)
  • System Design Laboratory (COE 400) (outcomes (a,
    b, c, d, e, g, h, i, j, k, n)
  • Senior Design Project (COE 485) (outcomes a, c,
    d, e, g, h, i, j, k, and n)
  • Course Assessemnt
  • All COE core courses

40
What is a Rubric
  • A scoring tool that
  • lists the criteria for a piece of work, or what
    counts
  • For example, purpose, organization, details,
    voice, and mechanics are often what count in a
    piece of writing
  • describes levels of quality for each criterion
  • For example, qualitative scores (e.g. Excellent,
    Good, Satisfactory, Needs Improvement), or as
    numerical scores (e.g., 4, 3, 2, 1)

Heidi Goodrich Andrade, a rubrics expert
(http//learnweb.harvard.edu/alps/thinking/docs/ru
bricar.htm)
41
Why use Rubrics
  • Rubrics help students and teachers define
    quality
  • When students use rubrics regularly to judge
    their own work, they begin to accept more
    responsibility for the end product
  • Rubrics reduce the time teachers spend grading
    student work and makes it easier for teachers to
    explain to students why they got the grade they
    did and what they can do to improve

Heidi Goodrich Andrade, a rubrics expert
(http//learnweb.harvard.edu/alps/thinking/docs/ru
bricar.htm)
42
How Rubrics are used in COE
  • As presented earlier, for each COE program
    outcome, an assessment and evaluation plan is
    developed that contains the following elements
  • Assessment and Evaluation Methods
  • Performance Criteria
  • Logistics
  • Rubrics are one of the direct assessment
    methods used to evaluate the COE program outcomes
  • A Rubric exists for each of the following COE
    program outcomes
  • a, b, c, d, e, f, g, h, i, j, k, and n

43
Program Outcomes Assessment Process
  • Two committees to conduct assessment process,
    assessment committee and undergraduate committee.
  • Assessment committee responsible of
  • design and control of the direct and indirect
    assessment processes,
  • data collection and presentation,
  • data delivery to undergraduate committee.
  • Undergraduate Committee responsible of
  • Carrying out analysis of direct and indirect
    assessment data provided by the Assessment
    Committee and the Faculty based on course
    assessment results.
  • identify potential problems and suggest
    recommendations for making improvements.
  • Implementing approved recommendations.

44
Course Outcomes Assessment Process
  • Each COE course has a Course Learning Outcomes
    Table that includes the following for each
    outcome
  • Outcome indicators and details this describes
    the main course topics that will be focused on to
    achieve the outcome.
  • Suggested assessment methods and metrics.
  • Outcome minimum weight this indicates the
    importance of the outcome in the course. It is
    the minimum weight from the total course score
    (out of 100) that must be used for assessing the
    outcome or covering the outcome in the course.
  • A mapping between the course learning outcome and
    ABET program outcomes.
  • Each outcome is given a rank as Low, High, Medium
    that correlates with the weight used for
    assessing the outcome.

45
Course Outcomes Assessment Process
  • Course outcomes are assessed by course
    instructors both directly and indirectly.
  • Suggested direct assessment of course learning
    outcomes based on using Course Learning Outcomes
    Evaluation Table includes the following for each
    outcome
  • Outcome minimum weight.
  • Outcome weight this is to be filled by the
    instructor indicating how much weight was used by
    the instructor for assessing the outcome.
  • Assessment Method this describes what methods
    were used to asses the outcome, the weight of
    each method, and the evidence of assessment.
  • Class Average indicates the students average
    performance in the outcome.

46
Course Learning Outcomes Table Example of COE 205
Course Learning Outcomes Outcome Indicators and Details Assessment Methods and Metrics Min. Weight ABET 2000 Criteria
1. Ability to analyze, design, implement, and test assembly language programs. Instruction Set Architecture Number (unsigned and signed) and character representation Addressing modes Syntax, semantics, and effect on flags of Pentium instructions. Input/output. Arithmetic and logic operations. Flow-control structures. Procedures. Macros. String manipulation. Interrupt mechanism. Implementation of Pseudo code algorithms in assembly language. Assignment Quizzes Exams Project 55 C(H)
47
Course Learning Outcomes Table Example of COE 205
Course Learning Outcomes Outcome Indicators and Details Assessment Methods and Metrics Min. Weight ABET 2000 Criteria
2. Ability to use tools and skills in analyzing and debugging assembly language programs. Assembly language vs. machine language. Assembling and linking assembly programs (including use of multiple files). Use of debugger to analyze and debug programs. Use of libraries. Lab work 4 K(L)
3. Ability to design the datapath and control unit of a simple CPU. Fetch-execute cycle Data, address and control busses Register transfer Data path design 1-bus, 2-bus and 3-bus CPU. Derivation of control steps for assembly instructions. Hardwired Control unit design Microprogrammed control unit design. Fixed vs. variable instruction format. Assignments Quizzes Exams 15 C(M)
48
Course Learning Outcomes Table Example of COE 205
Course Learning Outcomes Outcome Indicators and Details Assessment Methods and Metrics Min. Weight ABET 2000 Criteria
4. Ability to demonstrate self-learning capability. Ability to learn a course topic alone (e.g. Macros) Course Project may involve topics not studied in the course Assignment Quizzes 2 I(L)
5. Ability to work in a team. Project is divided into separate parts that will be integrated for project completion. Project 2 D(L)
49
Course Learning Outcomes Evaluation Table Example
(COE 205)
Outcome Outcome Min. Weight Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method
Outcome Outcome Min. Weight Assign. Quizz Exam I Exam II Exam III Final Exam Lab Work Proj. Total
O1 55 15 8 15 20 5 8 71
O1 Average 12.1 5.3 9.5 12.1 4.1 7 50.1(70.6)
O1 Evidence 1-4 1-4, 6 Q1-5 Q1-5 1-13 Rep.
O2 4 5 5
O2 Average 4.1 4.1 (82)
O2 Evidence 1-13
O3 15 20 20
O3 Average 11.8 11.8 (59)
O3 Evidence Q1-5
50
Course Learning Outcomes Evaluation Table Example
of COE 205
Outcome Outcome Min. Weight Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method
Outcome Outcome Min. Weight Assign. Quizz Exam I Exam II Exam III Final Exam Lab Work Proj. Total
O4 2 2 2
O4 Avg. 1.3 1.3 (65)
O4 Eviden. 5
O5 2 2 2
O5 Avg. 1 1 (50)
O5 Eviden. Rep.
Weight Weight 15 10 15 20 20 10 10 100
Average Average 12.1 6.6 9.5 12.1 11.8 8.2 8 68.3
51
Course Learning Outcomes Indirect Assessment
Example of COE 205
Criteria Student Evaluation Student Evaluation Student Evaluation Student Evaluation Student Evaluation Student Evaluation
Criteria E (4) G (3) A (2) P (1) NA (0) Composite
1. As a result of this course, my ability to analyze, design, implement, and test assembly language programs can be described as, 10  5  3  2    3.15
2. As a result of this course, my ability to use tools and skills in analyzing and debugging assembly language programs can be described as,           0
3. As a result of this course, my ability to design the datapath and control unit of a simple CPU can be described as,           0
4. As a result of this course, my ability to demonstrate self-learning capability can be described as,           0
5. As a result of this course, my ability to work in a team can be described as,           0
Number of Responses 20 Number of Responses 20 Number of Responses 20 Number of Responses 20 Number of Responses 20 Number of Responses 20
52
Course Assessment Results Example of COE 205
Section Source of Outcome Data Outcome1 Outcome2 Outcome3 Outcome4 Outcome5
I Instructor Evaluation 64.8 48.2 72.3 62.8 61.7
I Student Survey 75 65 68.2 73.3 75
II Instructor Evaluation 77.8 80.7 70.1 75.2 75.2
II Student Survey 88.25 88.25 69 75 86.8
III Instructor Evaluation 77.4 86 77.5 78 78.8
III Student Survey 87 79 77.8 81.5 80.3
Overall Recommend. Acceptable Needs Improvem. Needs Improvem. Acceptable Acceptable
53
Course Assessment Results Example of COE 205
  • Observations
  • Outcome 2 and Outcome 5 have not been assessed
    directly in Section II and Section III and the
    lab mark has been used.
  • Based on the overall assessment and instructors
    feedback, it seems that Outcome 2 and Outcome 3
    need improvement. All Other outcomes are
    considered well-achieved.
  • Recommendations
  • Outcome 2 needs more emphasis in the lab and
    should be directly assessed by lab instructors.
  • Outcome 3 can be improved by increasing the
    number of assignments on this part from one to
    two.

54
Program Outcomes Assessment
  • For each program outcome, an assessment and
    evaluation plan is developed that contains the
    following elements
  • Assessment and Evaluation Methods This describes
    what assessment methods are used to collect data
    and how will the data be evaluated and
    interpreted.
  • Performance Criteria This determines the
    criteria used to indicate that an outcome has
    been achieved with satisfactory levels or needs
    improvement.
  • Logistics This indicates when the data will be
    collected and who will collect it, interpret it,
    and report the results.

55
Industry Advisory Board
  • Attend Advisory Board regular meeting
  • Review of department area of concentrations and
    recommend some enhancement to undergraduate
    program in some specialized areas
  • Provide inputs on the undergraduate student
    qualifications and skills based on new trends and
    local industry needs
  • Provide feedback on the Educational Objectives
    and Program Outcomes and their implications on
    the program curriculum.
  • Consideration of the level of support provided to
    faculty to further their professional
    development, research, and teaching goals,
  • Survey the board members regarding their views on
    the importance of each aspect of the program
    departmental Educational Objectives.

56
Program Outcomes Assessment
Program Outcome Assessment Evaluation Methods Performance Criteria Logistics
an ability to apply knowledge of mathematics, science, and engineering Samples of COE 400, COE 485 and COE 351 reports Math 101, Math 102, Math 201, Math 260, Phys. 101, Phys. 102, chem.. 101 Exit exam Graduate Exit Survey Coop Employer Survey A score ? 2.5 out of 4 Average GPA ? 2.5 out of 4 A score ? 60 A score ? 3 out of 5 A score ? 3 out of 5 Assessments will be conducted every semester. However, grades of Math, Phys. Chem. Courses will be collected and analyzed once a year.
(b) an ability to design and conduct experiments, as well as to analyze and interpret data Samples of COE 400, COE 344 and COE 305 lab reports Graduate Exit Survey Coop Employer Survey A score ? 2.5 out of 4 A score ? 3 out of 5 A score ? 3 out of 5 Assessments will be conducted every semester.
57
Program Outcomes Assessment
Program Outcome Assessment Evaluation Methods Performance Criteria Logistics
(c) an ability to design a system, component, or process to meet desired needs Samples of COE 400, COE 485 and COE 351 reports Graduate Exit Survey Coop Employer Survey A score ? 2.5 out of 4 A score ? 3 out of 5 A score ? 3 out of 5 Assessments will be conducted every semester.
(d) an ability to function on multi-disciplinary teams Samples of COE 400, COE 485 and COE 351 reports Peer instructor evaluations in COE 400 Graduate Exit Survey Coop Employer Survey A score ? 2.5 out of 4 A score ? 2.5 out of 4 A score ? 3 out of 5 A score ? 3 out of 5 Assessments will be conducted every semester.
58
Summary
  • The COE department defined its mission, vision,
    educational objectives, and educational outcomes.
  • Two departmental committees will be in charge of
    the process of periodically assessing the
    program, provide input data on how to tune the
    COE program, and carry out correction action at
    all levels.
  • The COE department is determined to improve its
    program both the technical and behavioral
    components to meet EC 2K
  • New instruction techniques for outcome-based
    education will be gradually introduced at all
    levels to improve the quality of the Computer
    Engineer to some international standard.
  • The Industrial Advisory Board is one important
    channel to provide the department with feedback
    on the achievement of long term educational
    objectives as experienced by our COE alumni

59
COE RUBRICS ASSESSMENT PROCESS
60
Outline
  • What is a Rubric
  • Why use Rubrics
  • How Rubrics are used in COE
  • Process
  • Samples
  • Currently available Rubrics results
  • Summary

61
How Rubrics are used in COE
  • The following COE courses are used to assess the
    COE program outcomes, and accordingly have an
    associated set of Rubrics
  • Microcomputer System Design (COE 305) lab
    (outcome b)
  • Computer Networks lab (COE 344) lab (outcome b)
  • Cooperative Work (COE 350/351) (outcomes a, c, d,
    e, g, h, i, j, k, and n)
  • Seminar (COE 390) (outcome f)
  • Summer Training (COE 399) (outcome g)
  • System Design Laboratory (COE 400) (outcomes (a,
    b, c, d, e, g, h, i, j, k, n)
  • Senior Design Project (COE 485) (outcomes a, c,
    d, e, g, h, i, j, k, and n)
  • An equivalent summary is provided in the
    following slide

62
How Rubrics are used in COE
  • Note the following
  • COE 399 is used for COE program assessment
    through Rubrics only once a year (i.e. in the
    1st term)
  • All other courses are used for COE program
    assessment through Rubrics twice a year (i.e.
    in the 1st and the 2nd terms)

63
How Rubrics are used in COEThe Process
  • The instructor of each of the COE 305 lab, COE
    344 lab, COE 399, COE 400, and COE 485 as well as
    each member of the examining committee of COE 399
    and COE 350/351 is responsible for performing the
    following
  • Select a sample of students from their respective
    course
  • Conduct the respective Rubrics using the
    selected sample of the students from the previous
    step
  • Forward the results of the conducted Rubrics to
    the COE Assessment Committee (AC)
  • The AC collects the various results of Rubrics
    associated with each COE program outcome, and
    accordingly calculates a weighted average, out of
    4.0, for each COE program outcome
  • Forward Rubrics summary results to the
    Undergraduate Committee (UC) to consider
    curriculum correction actions, if any
  • From Rubrics point of view, a COE program
    outcome is declared to be achieved if the
    corresponding weighted average is ? 2.5 out of 4.0

64
How Rubrics are used in COE
  • The following slides provide a sample of the
    Oral Presentation Rubrics used to assess COE
    program outcome (g)

65
Oral Presentation Rubrics (Sample)
ORAL PRESENTATION ASSESSMENT Presenters Name
________________________ Presenter ID
___________ Presentation Title
________________________________________________ E
valuators Name ___________________________
Date ________________
Outcome Novice (1) Apprentice (2) Proficient (3) Exemplary (4) Score Comments
Audience awareness (interacts with audience e.g. stepping toward audience and speaking to them, not at them), looking at them, making eye contact Does not interact with audience at all Does not look at the audience Look at PC, screen, or elsewhere Little interaction with audience Most of the time looks elsewhere Some interaction with audience Interacts with audience throughout presentation
Focus goal, evidence, conclusion (gives audience a roadmap and follows it) Does not give audience an adequate road map of goal, evidence and conclusion Gives audience some road map of goal, evidence and conclusion Gives audience an adequate road map of goal, evidence and conclusion Gives audience very clear road map of goal, evidence and conclusion
Transitions (phrases smoothly link one part to next) Abruptly transitions from one phase to the next No linking Some transition is provided though not smooth Transitions are generally smooth Very smooth Transitions
Use of visual aids (any non-plain text methods such as graphs, charts, flow diagrams etc.) to tell the story and enhance the quality of the presentation Either does not use visual aids at all or too much dependency on visual aids There is some use visual aids effectively to tell the story Overall, uses visual aids effectively to tell the story visual aids add to presentation Uses visual aids very effectively to tell the story visual aids enhance presentation
66
Oral Presentation Rubrics (Sample)
Mechanics Novice (1) Apprentice (2) Proficient (3) Exemplary (4) Score Comments
Body position (e.g., facing audience or screen) Body position (faces screen or board all the time) Body position (faces audience some of the time) Body position (faces audience most of the time) Body position (always facing audience)
Eye contact (e.g., scanning entire audience) No eye contact Some eye contact (not enough, looking down a lot) Eye contact (some scanning of audience, looking at people) Eye contact (excellent scanning of audience, looking at people)
Visual aids (e.g., clear, not too busy, readable size font) Visual Aids (too busy, blurry) Visual Aids (a little bit busy, sometimes not clear) Visual Aids (can read clearly, usually not too much material) Visual Aids (clear, right amount on each slide)
Delivery (e.g., fluency, pace, voice projection, ums, uhs) Delivery (too fast, too many ums, not projecting voice, lack of enthusiasm) Delivery (a little bit fast, sometimes ums, little projecting voice, little enthusiasm) Delivery (good pace, usually projects voice, some enthusiasm) Delivery (excellent pace, projects voice, great enthusiasm)
Questions Novice (1) Apprentice (2) Proficient (3) Exemplary (4) Score Comments
Asks audience for questions Does not ask for questions rarely ask for questions Asks for questions Effectively opens (Id be happy to answer questions)
Answers questions effectively and smoothly Does not answer questions adequately rarely answer questions adequately Answers questions adequately Answers questions effectively and smoothly
67
Currently Available Rubrics Results
  • The COE program outcomes Rubrics have been
    conducted for both T062 and T071
  • The summary results for both T062 and T071 are
    provided next

68
T062 Rubrics Results
69
T071 Rubrics Results
70
Currently Available Rubrics Results
  • To explain the weighted average Rubrics score
    for an outcome, consider outcome (c) in the T071
    results
  • Rubrics for outcome (c) were conducted using
    COE 350/351, COE 400, and COE 485 with students
    sample of 3, 5, and 3, respectively, and with an
    average result of 2.13, 2.36, and 2.57,
    respectively
  • Weighted average score for outcome (c) (3?2.13
    5?2.36 3?2.57)/(353) 2.35

71
Currently available Rubrics results
  • By comparing the rubrics results for both T062
    and T071, the following can be concluded from a
    Rubrics point of view
  • COE program outcomes (b), (d), (e), (f), (g),
    (i), (k), and (n) were achieved
  • COE program outcomes (a), (c), (h), and (j) were
    not achieved

72
Summary
  • Rubrics are used as one method to directly assess
    the COE program outcomes
  • A process is defined to explain how to conduct
    the Rubrics every term
  • The Rubrics were conducted for T062 and T071
  • Rubrics results for T062 and T071 reflect that
    the COE program outcomes (b), (d), (e), (f), (g),
    (i), (k), and (n) were achieved while the COE
    program outcomes (a), (c), (h), and (j) were not
    achieved

73
Course Outcomes Assessment
74
Outline
  • Course Outcomes Assessment Process
  • Course Learning Outcomes Table
  • Course Learning Outcomes Evaluation Table
  • Course Learning Outcomes Indirect Assessment
  • Course Assessment Results Example
  • Term 061 Course Outcomes Assessment
  • Term 062 Course Outcomes Assessment
  • Term 071 Course Outcomes Assessment
  • Course Assessment Summary

75
Course Outcomes Assessment Process
  • Each COE course has a Course Learning Outcomes
    Table that includes the following for each
    outcome
  • Outcome indicators and details this describes
    the main course topics that will be focused on to
    achieve the outcome.
  • Suggested assessment methods and metrics.
  • Outcome minimum weight this indicates the
    importance of the outcome in the course. It is
    the minimum weight from the total course score
    (out of 100) that must be used for assessing the
    outcome or covering the outcome in the course.
  • A mapping between the course learning outcome and
    ABET program outcomes.
  • Each outcome is given a rank as Low, High, Medium
    that correlates with the weight used for
    assessing the outcome.

76
Course Outcomes Assessment Process
  • Course outcomes are assessed by course
    instructors both directly and indirectly.
  • Suggested direct assessment of course learning
    outcomes based on using Course Learning Outcomes
    Evaluation Table includes the following for each
    outcome
  • Outcome minimum weight.
  • Outcome weight this is to be filled by the
    instructor indicating how much weight was used by
    the instructor for assessing the outcome.
  • Assessment Method this describes what methods
    were used to asses the outcome, the weight of
    each method, and the evidence of assessment.
  • Class Average indicates the students average
    performance in the outcome.

77
Course Learning Outcomes Table Example (COE 205)
Course Learning Outcomes Outcome Indicators and Details Assessment Methods and Metrics Min. Weight ABET 2000 Criteria
1. Ability to analyze, design, implement, and test assembly language programs. Instruction Set Architecture Number (unsigned and signed) and character representation Addressing modes Syntax, semantics, and effect on flags of Pentium instructions. Input/output. Arithmetic and logic operations. Flow-control structures. Procedures. Macros. String manipulation. Interrupt mechanism. Implementation of Pseudo code algorithms in assembly language. Assignment Quizzes Exams Project 55 C(H)
78
Course Learning Outcomes Table Example (COE 205)
Course Learning Outcomes Outcome Indicators and Details Assessment Methods and Metrics Min. Weight ABET 2000 Criteria
2. Ability to use tools and skills in analyzing and debugging assembly language programs. Assembly language vs. machine language. Assembling and linking assembly programs (including use of multiple files). Use of debugger to analyze and debug programs. Use of libraries. Lab work 4 K(L)
3. Ability to design the datapath and control unit of a simple CPU. Fetch-execute cycle Data, address and control busses Register transfer Data path design 1-bus, 2-bus and 3-bus CPU. Derivation of control steps for assembly instructions. Hardwired Control unit design Microprogrammed control unit design. Fixed vs. variable instruction format. Assignments Quizzes Exams 15 C(M)
79
Course Learning Outcomes Table Example (COE 205)
Course Learning Outcomes Outcome Indicators and Details Assessment Methods and Metrics Min. Weight ABET 2000 Criteria
4. Ability to demonstrate self-learning capability. Ability to learn a course topic alone (e.g. Macros) Course Project may involve topics not studied in the course Assignment Quizzes 2 I(L)
5. Ability to work in a team. Project is divided into separate parts that will be integrated for project completion. Project 2 D(L)
80
Course Learning Outcomes Evaluation Table Example
(COE 205)
Outcome Outcome Min. Weight Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method
Outcome Outcome Min. Weight Assign. Quizz Exam I Exam II Exam III Final Exam Lab Work Proj. Total
O1 55 15 8 15 20 10 68
O1 Average 10.9 5.2 10.4 14.4 8.3 49.2 (72.4)
O1 Evidence 1-4 1-5 Q1-4 Q1-5 Lab Exp .
O2 4 2 2
O2 Average 1.4 1.4 (70)
O2 Evidence Lab Exp .
O3 15 20 20
O3 Average 13.1 13.1 (65.5)
O3 Evidence Q1-4
81
Course Learning Outcomes Evaluation Table Example
(COE 205)
Outcome Outcome Min. Weight Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method Assessment Method
Outcome Outcome Min. Weight Assign. Quizz Exam I Exam II Exam III Final Exam Lab Work Proj. Total
O4 2 2 2
O4 Avg. 1.56 1.56 (78)
O4 Eviden. 6
O5 2 8 8
O5 Avg. 5.8 5.8 (72.5)
O5 Eviden. Rep.
Weight Weight 15 10 15 20 20 12 8 100
Average Average 10.9 6.76 10.4 14.4 13.1 9.7 5.8 71.1
82
Course Learning Outcomes Indirect Assessment
Example (COE 205)
Criteria Student Evaluation Student Evaluation Student Evaluation Student Evaluation Student Evaluation Student Evaluation
Criteria E (4) G (3) A (2) P (1) NA (0) Composite
1. As a result of this course, my ability to analyze, design, implement, and test assembly language programs can be described as, 9  7  3    3.32
2. As a result of this course, my ability to use tools and skills in analyzing and debugging assembly language programs can be described as,  2  8 8     1 2.53
3. As a result of this course, my ability to design the datapath and control unit of a simple CPU can be described as,  2 9  6  6  1  2.53
4. As a result of this course, my ability to demonstrate self-learning capability can be described as,  6 8  2  3    2.89
5. As a result of this course, my ability to work in a team can be described as, 7  9  2  1    3.16
Number of Responses 19 Number of Responses 19 Number of Responses 19 Number of Responses 19 Number of Responses 19 Number of Responses 19
83
Course Assessment Results Example (COE 205)
Section Source of Outcome Data Outcome1 Outcome2 Outcome3 Outcome4 Outcome5
I Instructor Evaluation 64.8 48.2 72.3 62.8 61.7
I Student Survey 75 65 68.2 73.3 75
II Instructor Evaluation 77.8 80.7 70.1 75.2 75.2
II Student Survey 88.25 88.25 69 75 86.8
III Instructor Evaluation 77.4 86 77.5 78 78.8
III Student Survey 87 79 77.8 81.5 80.3
Overall Recommend. Achieved Needs Improv. Needs Improv. Achieved Achieved
84
Course Assessment Results Example(COE 205)
  • Observations
  • Outcome 2 and Outcome 5 have not been assessed
    directly in Section II and Section III and the
    lab mark has been used.
  • Based on the overall assessment and instructors
    feedback, it seems that Outcome 2 and Outcome 3
    need improvement. All Other outcomes are
    considered well-achieved.
  • Recommendations
  • Outcome 2 needs more emphasis in the lab and
    should be directly assessed by lab instructors.
  • Outcome 3 can be improved by increasing the
    number of assignments on this part from one to
    two.

85
Term 061 Course Outcomes Assessment
O1 O2 O3 O4 O5 O6 O7 O8 O9 O10 O11
COE 202 A NI NA
COE 203
COE 205 A NI NI A A
COE 305 A NI NI A A A A
COE 308 NI A NI NI A
COE 341 NI A A NI A
COE 344 NA A NI A A
COE 360 NI A A NA NA NI
COE 390
COE 400
COE 485
A Achieved NI Needs Improvement
NA Not Achieved
86
Term 062 Course Outcomes Assessment
O1 O2 O3 O4 O5 O6 O7 O8 O9 O10 O11
COE 202 A NI NI
COE 203 A A NI A A
COE 205 A A A A A
COE 305 A A A A NI A NI
COE 308 A A NI A A
COE 341 NI A NI A A
COE 344 A A A NI A
COE 360 A A NI NI A A
COE 390 A A A A A
COE 400 A A A NI NI A A A
COE 485 A A A A A A NI A A A NI
A Achieved NI Needs Improvement
NA Not Achieved
87
Term 071 Course Outcomes Assessment
O1 O2 O3 O4 O5 O6 O7 O8 O9 O10 O11
COE 202 A NI NI
COE 203 A A A A A
COE 205 A NI NI A A
COE 305 A A A A A A A
COE 308 A A A NI A
COE 341 NI A A A A
COE 344 NA A NI A A
COE 360 A A NI A A A
COE 390 A A A A A
COE 400 A A A A NI A A A
COE 485 A A A A A A NI A A A NI
A Achieved NI Needs Improvement
NA Not Achieved
88
Course Assessment Summary
  • While some of the course outcomes were considered
    not achieved in Term 061, all outcomes in Term
    062 were either Achieved or Need Improvement.
  • Only one course outcome not achieved in Term 071
    (COE 344).
  • More outcomes are achieved in Term 062 than Term
    061 and more outcomes are achieved in Term 071
    than Term 062.
  • Results of course assessment over the three
    semesters indicate corrective action resulting in
    improving course outcomes achievement.

89
Indirect Assessment
90
Outline
  • Summary of Industrial Advisory Committee Feedback
  • Summary of Exit Survey
  • Summary of Alumni Survey

91
Summary of COE Industrial Advisory Committee
(IAC) Feedback
92
COE Industrial Advisory Committee
  • Since 2006, the COE IAC has eight members
  • Six from the local Industry
  • Two from the COE department
  • IAC Goal Provide feedback to assist the COE
    Department in achieving its mission and
    objectives
  • First meeting on April 25, 2007
  • Attended by all COE faculty and by students
    representatives
  • CCSE Dean and chairmen of SE and ICS departments
    were invited
  • The main issue discussed is how to improve the
    Relationship between COE-KFUPM and Industry

93
COE IAC Feedback
  • The IAC members
  • Expressed their satisfaction about the BS Program
    in Computer Engineering
  • Discussed
  • Partnership with industry
  • Stronger coop and summer training programs
  • Quality of computer engineering graduates

94
COE IAC Feedback BS Program
  • COE graduates lack wider exposure to
    High-Performance Computing and Software areas,
    while COE department has a stronger VLSI area
  • Need to concentrate on system design projects,
    providing more emphasis on system integration
    using available components
  • Need to develop top most quality and capability
    access to massive data - data storage, retrieval,
    analysis, management and security

95
COE IAC Feedback - Lab
  • IAC satisfied regarding the lab facilities and
    experiments conducted
  • IAC offered their support for further lab
    development
  • IAC stated that the kind of system applications
    available in the System Design Lab is exactly
    what they are looking for future collaboration of
    COE-KFUPM and Industry

96
COE IAC Feedback - Quality of Computer
Engineering Graduates
  • Lack of communication and presentation skills in
    COE-KFUPM graduates, in general
  • As a result, the industrial requirements are not
    fulfilled
  • Lack of exposure of COE Graduates to current
    developments in IT that is used in the industry

97
COE IAC Feedback - COE Department-Industry
Interaction
  • There is an urgent need to improve COE
    faculty-industry relations through
  • Faculty Exposure to Industry
  • Bringing case studies related to course subjects
    from local industrial sections
  • Research Interaction with Industry
  • Strengthening the involvement of faculty in
    solving local industrial problems
  • May require looking at the rules and regulations
    for the university faculty to work with local
    industry

98
COE IAC Feedback - Effectiveness of Coop and
Summer Training Programs
  • Work plan should be identified ahead of time for
    students going to industry/company for coop or
    summer training program
  • Coop students from KFUPM are focusing on mere
    reporting to the department rather than
    presenting their learning experience
  • Low level of work presentation

99
IAC Survey
  • IAC Survey for seeking the input and assessment
    of IAC members on
  • Program Educational Objectives
  • Program Outcomes
  • Program Assessment System
  • Quality of COE-KFUPM Graduates at the workplace,
    and Abilities, Attributes and Skills
  • Five members responded to the survey

100
IAC Inputs on the Educational Objectives
  • The nature of our business is to have engineers
    with both business and engineering backgrounds
  • Good objectives, however, there is a need to
    focus on few areas to shine in and gain regional
    and worldwide reputation
  • There is breadth but not much depth in the
    overall objectives
  • Provide graduate engineers with the ability to
    support the industry in the areas of
    communication network design and embedded system
    design

101
IAC Inputs on the Program Outcomes
  • KFUPM graduates are very good especially as to
    work ethics
  • Stress more practical disciplines
  • COE graduates lack the specialization
  • They have basic knowledge in designing networks
    and developing software codes for embedded
    systems, but have no in depth focus
  • They should go in depth in specific disciplines
    that will make them productive faster
  • Dedicate at least the last 2 semesters as a
    minimum are for a specialization
  • Need some improvements, since we are now
    competing at the international level
  • Companies now look for skilled employees
    regardless of their physical locations

102
IAC Inputs on the Program Assessment Method
  • Reasonable
  • Very good

103
COE IAC Feedback - COE Program Improvement Areas
  • The COE program needs to improve the listed below
    graduate abilities, attributes, and skills which
    are estimated to be important for the Industry
  • Function on multi-disciplinary or
    cross-functional teams (all)
  • Understand the contemporary issues surrounding
    him (all)
  • Use computing technology in communications (all)
  • Communicate orally informal and prepared talks
    (2)
  • Communicate in writing letters, technical
    reports, etc. (2)
  • Ability to use state of the art techniques, and
    tools (2)
  • Design a system, component to meet a desired need
    (1)
  • Analyze and interpret data from experiments (1)
  • Recognize professional and ethical responsibility
    (1)
  • Use computing technology in engineering
    analysis/design (1)

104
COE IAC Feedback - Most Useful Knowledge, Skills,
and Tools Needed Prior to COOP or ST
  • Technical writing skills
  • Learning how to search the Internet for
    information
  • In depth technical knowledge in practical areas
    relevant to need of local and multinational
    companies operating in the region
  • KFUPM need to assess the need of the industry and
    focus more in the area that is more relevant,
    e.g., HPC is needed locally
  • Infrastructure design using Microsoft and Sun
    environments
  • Networking and storage designs
  • Troubleshooting skills and analysis
  • Programming in C and C
  • Automated Office skills (Word/Excel, etc.)
  • Study of industrial standard C51 microcontroller,
    and the use of the microcontroller development
    tools such as compiler, simulator, and
    in-circuit emulator

105
COE IAC Feedback - How to Improve the COE
Education?
  • More interaction with the local industry
  • Capitalize on local and related technologies to
    address country challenges
  • Increase the exchange of skills and knowledge
    between world class universities in related
    fields
  • Introduce the distance learning / video
    conferencing in the related fields
  • More up-to-date dynamic curriculum to reflect the
    ever changing technology, e.g., web 2.0
    programming and HPC
  • Enhance the laboratory skills
  • Study software quality
  • Study embedded systems operation system

106
COE IAC Feedback - How to Improve the
Professional Partnership Between COE and the
Industry?
  • Professors and educators need to spend time in
    the industry
  • To match KFUPM programs with the industry
    requirement
  • Conduct knowledge transfer sessions where
    industry and academia exchange their experience
    and skills
  • Introduce industry challenges to the education
    sector to address these challenges through senior
    projects or special studies
  • KFUPM needs to visit the companies and survey
    them to understand where the needs are
  • Based on this, align KFUPM direction and change
    its strategy
  • At the end, the industry and students are KFUPM
    end customers
  • More frequent visits of students and Faculty to
    Industry
  • Getting advisors from Industry
  • Focus on coop programs
  • Encourage graduate students to do their projects
    on topics that have direct impact on the local
    needs

107
Summary of COE Graduating Student Survey (Exit
Survey)
108
Exit Survey - COE Program Outcomes (Abilities and
Skills)
  • The COE graduating students gave high rating (gt
    80) to
  • Ability to apply general principles of
    mathematics, science, and engineering to analyze
    and solve computer engineering problems
  • Quality and variety of COE design projects
    helpful in developing engineering design skills
  • Oral and written communication skills
  • Understanding of the impact of computer
    engineering solutions in my society and in the
    world
  • Understanding the contemporary social, political,
    and technical issues that surround our society
  • Ability to integrate different hardware and
Write a Comment
User Comments (0)
About PowerShow.com