C16 Tools for Assessing Learning in Engineering Teri ReedRhoads Purdue University School of Engineer

1
C16 Tools for Assessing Learning in
EngineeringTeri Reed-RhoadsPurdue
UniversitySchool of Engineering Education

• Inventions and Impact 2 Building Excellence in
  Undergraduate Science, Technology, Engineering,
  and Mathematics (STEM) Education
• 13-15 August 2008
• Washington, DC

2
Engineering Education Grant Writing Workshop

• ASEE Section Meeting
• April 3, 2008
• Teri Reed-Rhoads
• Purdue University
• trhoads_at_purdue.edu

3
Assessment

• Formative checking along the way
• Summative final analysis of achieving research
  goals
• Consider an outside evaluator unbiased,
  measurable feedback
• Consider an external advisory board
• Give details, actual tools, timelines, etc.

4
Assessment

• Course
• Curriculum
• Program
• Project

5
Types of Assessment

• Cognitive Domain
• Content knowledge (Concepts)
• Process knowledge, e.g., teamwork (Skills,
  Abilities)
• Student interest, perceptions, and attitudes
  (Affective)
• Rate change (Achievement)

6
PDs Response Goals on Cognitive Changes

• Goals on cognitive changes
• Increase understanding of concepts
• Ability to solve statics problems
• Ability to construct free-body diagrams
• Ability to describe verbally the effect of
  external forces on a solid object
• Increase processing skills
• Ability to solve out-of-context problems
• Ability to visualize 3-D problems
• Ability to communicate technical problems

Connie Della-Piana, Russ Pimmel, Bev Watford
Project Evaluation, Workshop for Faculty from
Minority Serving Institutions, Feb. 8 10, 2006.
7
Concept Inventories https//engineering.purdue.edu
/SCI/workshop/tools.html

• Engineering Related
• 20 instruments
• Science and Mathematics Related
• 8 instruments
• Physics and Astronomy Specific
• 12 instruments

8
Engineering Related CI
9
Science and Mathematics CI
10
Physics and Astronomy CI
Rebecca S. Lindell, Elizabeth Peak and Thomas M.
Foster, Are They All Created Equal? A
Comparison of Different Concept Inventory
Development Methodologies, 2006 Physics
Education Research Conference Proceedings,
American Institute of Physics.
11
Goals on Process Knowledge

• Teamwork
• Team Effectiveness (Imbrie)
• Team Developer (McGourty)
• Communications
• 3-D Visualization
• Etc.
• Leadership

12
PDs Response Goals on Affective Changes

• Goals on affective changes
• Improve students attitude about
• Profession
• Curriculum
• Department
• Improve students confidence
• Improve students intellectual development

Connie Della-Piana, Russ Pimmel, Bev Watford
Project Evaluation, Workshop for Faculty from
Minority Serving Institutions, Feb. 8 10, 2006.
13
Affective Measures

• Pittsburgh Freshman Engineering Survey
  (Besterfield-Sacre)
• Attitudes Toward Statistics (Wise)
• Computer Science Attitude Survey (Moskal)
• Structured Interviews

14
(No Transcript)
15
Consider a Graphic
16
PDs Response Goals on Achievement Rate Changes

• Goals on achievement rate changes
• Improve
• Recruitment rates
• Retention or persistence rates
• Graduation rates

Connie Della-Piana, Russ Pimmel, Bev Watford
Project Evaluation, Workshop for Faculty from
Minority Serving Institutions, Feb. 8 10, 2006.
17
Know the Motivation or Background
18

• URMs 30 of population, few in labor force
• Women 50 of population, 10 in labor force

Chubin and Babco (2003), Walking the Talk in
Retention to Graduation
19
Engineering Disciplines
Chubin and Babco (2003), Walking the Talk in
Retention to Graduation
National Science Foundation. Women, Minorities,
and Person with Disabilities in Science and
Engineering 2000. Arlington, VA, 2000 (NSF
00-327)
20
(No Transcript)
21
Authentic Teaching Alliance Assessment Cycle
22
Group Questions

• What are the known instruments/methods in each of
  the three areas? (Cognitive, Affective, and Rate
  Change)
• Where have these been used successfully/unsuccessf
  ully?
• Where are the gaps in each area?

23
Process

• One note taker
• One reporter
• Everyone discusses
• We will report out with 15 minutes left

24

• There is no failure. Only feedback.
• -Anonymous
• Happy grant writing!
• trhoads_at_purdue.edu
• https//engineering.purdue.edu/SCI/Workshop

25
Affective

• National academy of engineering website with book
  of surveys

26
Rate Change

• Longitudinal measurements, courses, rate changes
  of minorities, retention, enrollment, GPA for
  performance measures
• Use Associate Deans offices as resources for
  stats, changes, etc.

27
Wish

• Critical Thinking one that is free
• Definition of engineering changes for students as
  they go through the curriculum
• Digital Systems within computer engineering
• Teaming attitude by individual
• Measuring how well the student can deal with an
  out of box situation.
• Clearinghouse that explains what already exists,
  how long is it, how it was used in the past,
  cliff notes version

28
Wish

• ABET could sponsor (or someone) or post tools
  that are being used
• More CI https//engineering.purdue.edu/SCI/worksho
  p/tools.html
• Tools for writing and communication
• Life-long learning tools (continuous learning)
• Procedural knowledge on engineering design
• FREE Critical Thinking

29
Wish

• Meta-cognition
• Societal context of knowledge, global
  competencies, service learning, research skills,
  discourse and collaborative dialogue, oral
  communication, problem solving, leadership (some
  instruments from Business schools), ability to
  integrate knowledge, longitudinal data,
  qualitative data analysis
• Evaluation of case studies

30
Cognitive

• Math placement test
• English Placement test
• Concept Inventories https//engineering.purdue.edu
  /SCI/workshop/tools.html
• FE, GRE standardized tests
• Critical thinking
• Criticalthinking.org has tools, and rubric
  developed by Gary Brown at Washington State,
  Watson Glazier (also not free), U of South
  Carolina EFFECTS (web-based)
• http//www.ce.sc.edu/effects

31

• California Critical Thinking
• Collegiate Learning Assessment
• CAAP from ACT
• SALG Student Assessment of Learning Gains
• IDEA teaching assessments
• MSLQ Motivated Strategies for Learning
  Questionnaire
• NSF User Friendly Guide to Assessment

32

• Rogers and Sando, Rose-Hulman
• Learning Styles Instruments (Kolb, Felder)
• Career Instruments (Holland)
• FMRI, Caps measuring brain activity
• Experience sampling method student wears a
  bracelet and when it goes off the student writes
  down what they are doing (engagement measures)
• Calibrated Peer review
• Teamwork effectiveness (Imbrie, Smith, Layton)
• National Center for Case Studies (Buffalo)