FacultyIndustry Collaborations for Assessing Student Learning - PowerPoint PPT Presentation

Loading...

PPT – FacultyIndustry Collaborations for Assessing Student Learning PowerPoint presentation | free to download - id: 962f5-ZTVmZ



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

FacultyIndustry Collaborations for Assessing Student Learning

Description:

NJIT/ NCE Industrial Advisory Board. Gateway Engineering Education Coalition ... Industrial partners add value to assessment process, especially in providing a ... – PowerPoint PPT presentation

Number of Views:97
Avg rating:3.0/5.0
Slides: 45
Provided by: synthesis6
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: FacultyIndustry Collaborations for Assessing Student Learning


1
Faculty-Industry Collaborations for Assessing
Student Learning
  • Flora McMartin
  • University of California - Berkeley
  • Jack McGourty
  • Columbia University

2
Why Involve Industry?
  • Insures what is assessed is valued
  • Strengthens accountability
  • Addresses accreditation mandate

3
Campus-Industry Relationship
ACE Campus Trends Survey - 1996
4
Synthesis Coalition
Cornell
Iowa State
Berkeley
Hampton
Stanford
Northern Arizona
Cal Poly
Tuskegee
Southern
  • Synthesis Coalition Goals
  • Institutionalize reforms
  • Assess disseminate reforms
  • Develop NEEDS
  • Institutionalize K-12 outreach

5
Gateway Coalition
- Drexel - Cooper - USC - Ohio State - Academic
Associates - Polytechnic - NJIT - Columbia
  • Major Gateway Goals
  • Curriculum
  • Assessment
  • Underrepresented Populations
  • Instructional Technology
  • Professional Development
  • Linking Sharing

6
Assessment Components
  • Stakeholder participation (faculty, students,
    industry)
  • Faculty assessment training program
  • Campus (local) control of assessment process
  • Flexible, valid, and reliable tools

7
Assessment Goals
  • A. Involve stakeholders to measure effects of
    coalition program reforms
  • B. Develop/promote valid reliable performance
    assessment tools
  • C. Promote the institutionalization of
    assessment at participating campuses
  • D. Satisfy needs of individual faculty,
    colleges, coalition program evaluation
  • E. Provide a foundation for ABET accreditation

8
ABET 2000 Learning Outcomes
  • 1. math, science engineering knowledge
  • 2. experiment, analyze interpret data
  • 3. design a system, component or process
  • 4. function on multi-disciplinary teams
  • 5. identify, formulate solve engineering
    problems
  • 6. understand professional ethics
  • 7. communicate effectively
  • 8. understand global/societal impact of
    engineering
  • 9. engage in life-long learning
  • 10. knowledge of contemporary issues
  • 11. use modern engineering techniques, tools
    skills

9
Synthesis Case Study
10
Student Learning Outcomes
  • Open Ended problem solving
  • Teamwork
  • Multi-disciplinary design
  • Hands-on facility with hardware
  • Communication skills - (oral, written,
    interpersonal)

11
Goals for Involving Industry
  • Determine common learning outcomes valued by
    industry engineering education
  • Articulate student learning outcomes in a common
    language
  • Bridge industry engineering educational needs
  • Energize the assessment process through active
    participation of stakeholders

12
Mechatronics Industrial Board
  • Berkeley Process Control Motorola
  • Maytag Rockwell International
  • Xerox PARC Ford
  • Raychem Hewlett-Packard

13
The MIB, continued
  • Membership
  • primarily project engineers
  • observed needs vs. self reports
  • Role
  • define what industry values in educational
    outcomes
  • try out review assessment tools
  • try out review assessment rubrics measures
  • evaluate assessment results

14
Planning Process Track 1
ALL stakeholders define articulate Student
learning outcomes (SLOs) classroom activities
related to Synthessis goals
Identifying Learning Outcomes
SLOs reduced refined
SLOs reviewed by ALL stakeholders - draft of
Synthesis Assessment Framework (SAF)
Draft SAF compared to scenario activities
Overlap between SLOs activities MOST VALUED
OUTCOMES Final version of SAF
15
Track 1 Learning Outcomes
  • Industry

Engineering Ed.
Most valued student learning outcomes
Synthesis Assessment Framework
16
Planning Process Track 2
MIB writes scenarios
Validating the Outcomes
Scenarios edited, reviewed by MIB for accuracy
Scenarios analyzed to ID activities related to
SLOs
Draft SAF compared to scenario activities
Overlap between SLOs activities MOST VALUED
OUTCOMES Final version of SAF
17
Track 2 Validating with Scenarios
  • Scenario
  • a description of common or critical situations
    faced by an engineer
  • actual past, present, or future situation
    experienced by the MIB member
  • includes the context and actions of the engineer

18
Validation
  • approximately 80 overlap
  • Differences
  • ability to estimate resources (costs, time)
  • ability to supervise others
  • ability to take risks
  • ability to deal with ambiguity
  • ability to decide if project is worth pursuing

19
Assessment Planning Process
Track 1 Identification
Track 2 Validation
ALL stakeholders define articulate Student
learning outcomes (SLOs) classroom activities
related to Synthesis goals
MIB writes scenarios
Scenarios edited, reviewed by MIB for accuracy
SLOs reduced refined
Scenarios analyzed to ID activities related to
SLOs
SLOs reviewed by ALL stakeholders - draft of
Synthesis Assessment Framework (SAF)
Draft SAF compared to scenario activities
Overlap between SLOs activities MOST VALUED
OUTCOMES Final version of SAF
20
Assessment Tools
21
Scenario-Based Assessment
  • Assignment developed from MIB scenarios
  • Scoring rubric based on SAF
  • Assignment rubric tested by MIB
  • MIB responses established baseline expert score
  • MIB analysis of rubric used to refine measurement
    criteria

22
Scenario Assessment Process
Faculty administer pre post test
  • Syn. HQ assess
  • holistic
  • analytic
  • Faculty evaluate
  • diagnostic
  • course grade

faculty revise course
dept/col - revise curriculum Syn. - longitudinal
study
23
Synthesis - Lessons Learned
  • Collaboration created a bridge between industry
    faculty regarding engineering education
  • Clarified what is/is not possible to teach
    learn
  • validated measurable outcomes
  • Identifying articulating learning outcomes
    valued by ALL stakeholders built commitment to
    the assessment process results.
  • Vitalized assessment process for all stakeholders

24
Gateway Case Study
25
NJIT/NCE Assessment Statement
Institute an assessment plan and process that
moves from the current anecdotally based
assessment to a formal, rigorous, valid, and
useful assessment and continuous improvement
process.
26
NJIT/ NCE Industrial Advisory Board
Lucent Technologies
PSEG
Foster Wheeler
PrimeMedia
SIAC
Raytheon
US Army
Curtiss Wright
CompUSA
Becton Dickinson
27
Industrial Sub-Committees
NJIT/NCE Advisory Board
Strategic Planning
Curriculum Review
Assessment
Gateway National Visiting Committee
28
Industrial Assessment Committee
  • Charter
  • To support the development of a comprehensive
    qualitative and quantitative curriculum
    assessment process that incorporates inputs and
    feedback from the employer and takes into account
    the fast pace of technological change.

29
Industrial Assessment Committee
  • Short-term objectives
  • Identify/prioritize the core competencies
    required for entry-level engineers
  • Work with faculty to identify educational
    requirements based on core competencies
  • Assess current curricula as to its effectiveness
    in providing students with the required core
    competencies
  • Establish employer-level metrics to measure
    graduates skill proficiency and provide a
    baseline for continuous improvement

30
Industry Partner Role
  • Participating member of faculty assessment team
  • Bring in assessment best practices from industry
    such as planning, TQM processes, metrics,
    multi-source feedback, assessment centers, etc
  • Form relationships between school and
    engineering/human resources organizations
  • Act as liaison with Gateway Coalition National
    Visiting Committee

31
Assessment Design Steps
Step 1 Define Objectives, Strategies,
Outcomes
Step 2 Identify Assessment Methods
Step 5 Apply Results
Continuous Improvement
Step 3 Develop/Pilot Assessment Processes
Step 4 Implement/Expand Assessment Processes
32
Focus Group Objectives
  • Identify outcomes, objectives performance
    criteria for each course and program represented
  • Discuss existing assessment tools
  • Review potential use for continuous improvement

Design Step 1
33
Participating Programs
  • Curriculum Reform
  • FE/FED - Freshman course in design
    manufacturing
  • General University Requirements
  • Learning Tools
  • Computer-aided learning
  • Industrial Interaction
  • Intern program
  • Advanced Manufacturing Laboratory (AML)
  • Seminars Series - What Do Engineers Do?
  • Advance Design Engineering in Product Teams
    (ADEPT)

34
People Involved
  • 35-45 faculty members involved
  • Multidisciplinary - 3 of 4 colleges involved
  • Across all levels with enthusiastic involvement
    from tenured faculty
  • Covers courses programs in all four years
  • Deans and department chairs
  • Members of NJIT/NCE Advisory Board

35
Learning Outcomes
What critical knowledge, skills, and behaviors
must students acquire in this course?
Question
Student will demonstrate an ability to actively
participate, listen and collaborate with other
team members while working on a technical design
project.
Example
36
Results 9 Core Learning Outcomes
  • Analytical Thinking
  • Communication Skills
  • Creative Problem Solving
  • Project Management
  • Research Skills
  • Self-Learning
  • Systems Thinking
  • Teamwork
  • Technical Competence

37
Learning Outcome Definitions
  • Analytical Skills
  • Applies logic in solving problems and analyzes
    problems from different points of views.
    Translates academic theory into practical
    applications and recognizes interrelationships
    among problems and issues.

38
Learning Outcome Survey
NCE Learning Outcomes Instructor/Coordinator
The purpose of this survey is to gather your
ratings on the importance of each of the nine
broad learning outcomes and their specific
objectives as identified by you during the focus
group sessions conducted recently by the NCE
Assessment Team. Please review each statement
carefully and rate it using the scale provided.
In addition, you are provided with extra space to
add and rate objectives within each broad
learning outcome. Technical Competence Not at
all Minor Moderate Considerable
Critical How important are the following
student important importance importance
importance importance learning objectives to
the satisfactory completion of your course or
program Demonstrates a basic knowledge of
1 2 3 4
5 fundamental engineering principles in the
specific disciplines focused on in
this course/program Integrates basic knowledge
of other 1 2 3
4 5 engineering disciplines
within the scope of the courses project
39
Ratings by Faculty and Industry Partners
  • Research Skills (3.82, 3.54)
  • Systems Thinking (3.84, 4.65)
  • Self-Learning (3.82, 3.22)
  • Teamwork (4.23, 4.67)
  • Technical Competence (3.61, 3.54)
  • Analytical Thinking (3.83, 3.52)
  • Communication Skills (3.99, 4.62)
  • Problem Solving (3.62, 4.20)
  • Project Management (4.10, 4.44)

Other analyses show that learning outcome
emphasis changes as one moves to upper division
curricula.
40
Advisory Board - Post Review
  • Discuss/answer the following questions
  • Think about an engineer that has recently entered
    your organization and is regarded as highly
    effective (up to speed quickly, little
    retraining, etc.)
  • Does he or she demonstrate the selected
    competencies? Give examples of how they
    demonstrate each selected competency. Would you
    add specific statements to the definition of a
    selected competency?
  • What performance criteria would you use to
    evaluate successful demonstration of the selected
    competencies in your organization?
  • Would you add any other competencies that new
    engineers should have based on your experience
    with the successful engineer above?

41
Other Outcomes
  • Creation of several competency-based assessment
    processes
  • Revised portfolio process for freshman design
    experience
  • Team Developer version for engineering learning
    outcomes
  • Senior Exit, Alumni, and Employer surveys
  • Development of new assessment planning process
    diffused to all Coalition Schools and beyond

42
Gateway - Lessons Learned
  • Industrial partners add value to assessment
    process, especially in providing a real-word
    perspective on educational objectives and
    learning outcomes
  • Process fostering comparison helps to promote new
    thinking
  • Important for sub-group of industrial partners to
    work closely with faculty throughout assessment
    cycle
  • Must keep all participants actively involved in
    planning and decisions

43
Conclusions
  • INPUT
  • Fresh perspectives real world insights to the
    assessment enterprise.
  • New ways of thinking about education
  • Commitment to improving education
  • OUTCOME
  • Vitalizes planning process boosts faculty
    commitment/involvement to assessment
  • Increased campus support for innovative teaching
    assessment practices
  • Industry partnerships focused on education in
    both sectors

44
The Future of Collaboration
  • Current research shows
  • In most cases, industry kept to the periphery of
    assessment
  • Faculty campus administrators fear industry
    partners will make unreasonable demands because
    they dont understand the challenges facing
    higher education
  • Colleges do not know how to capitalize on the
    expertise of industrial partners
  • Assessment is still in fledgling stages
  • The conclusions from case studies confirmed by
    Colleges who integrated industry into their
    assessment processes
About PowerShow.com