Portraying Engineering Design Expertise: Empirical Insights

1
Portraying Engineering Design Expertise
Empirical Insights

• Cindy Atman
• Director
• Center for Engineering Learning and Teaching
• University of Washington
• First International Computing Education Research
  Workshop
• October 1, 2005

Acknowledgements This work has been supported
by grants from the National Science Foundation
RED-9358516, DGE-9714459, EEC-9872498,
REC-012554, ESI-0227558, SBE-0354453, the GE
Fund, the Ford Motor Company Fund and the Boeing
Company.
2
Research Program

• University of Pittsburgh (1993-1998) Mary
  Besterfield-Sacre, Karen Bursic, Justin Chimka
  and, Stefanie Lozito Carrie Mullins, Heather
  Nachtmann
• undergraduates Jamari Atkinson, John Clune, Rona
  Colassimo, Georgette Diab, Morris Fields, Brandie
  Hill, Anthony Horton, Jennifer Kreke, Stefanie
  Lozito, Terra Mitchell, Pamela Moore, Jill Nagel,
  Eugene Parker, Sharon Petrichko, Elizabeth
  Rodriguez, Jason Saleem, Jessica Shuman, Laura
  Simpson, Gwen Staples, Nick Tettey, Chris Yarsky,
  LisaYounger, Maryland Vick and Kathryn Zoffel,
• University of Washington, Center for Engineering
  Learning Teaching (1998-2005) Robin Adams
  (Purdue, 6/05), Theresa Barker, Laura Bogusch,
  Monica Cardella, Susan Mosborg, Eddie Rhone, Roy
  Sunarso, Jennifer Turns, Ken Yasuhara
• undergraduates Jacob Burghardt, Diana Baral,
  Yimin Chen, Louise Cheung, Jennifer Chin, Julie
  Christianson, Athena Epilepsia, Meera Kumar,
  Rebecca Kim, Angela King, Ashley Lam, Ian Louie,
  Jana Littleton, Cameron Lowen, Michael Music,
  Jennifer Rees, Alison Schwerzler, Cathie Scott,
  Robert Tai, Jennifer Temple, Tarrah Wells,
  Shannon Westphal and Bettina Vuong

3
Research Program

• Lots of conversations
• Engineering Education Community
• Computer Science Education Community
• Design Thinking Research Community
• Learning Sciences Community
• LIFE The Center for Learning in Informal and
  Formal Environments (John Bransford, Dan
  Schwartz, Nancy Vye)

4
Todays Agenda

• Engineering and engineering education
• Research program on engineering design knowing
• Portraits of engineering design expertise
• Students
• Faculty
• Experts

5
What is engineering?

• Engineering (NAE)
• applied science
• design under constraint
• Engineering method (Koen, 2003, p. 7)
• The strategy for causing the best change in a
  poorly understood situation within the available
  resources

6
What do engineers need to know?
ethics
global context
management skills
math
engineering analysis
problem solving
leadership
professionalism
design
contemporary issues
teamwork
analyze data
conduct experiments
life-long learning
societal context
science
communication
business knowledge
engineering tools
creativity
7
A focus on design knowing
ethics
global context
management skills
math
engineering analysis
problem solving
leadership
design
professionalism
contemporary issues
teamwork
analyze data
conduct experiments
life-long learning
societal context
science
communication
business knowledge
engineering tools
creativity
8
Research Program on Engineering Design Knowing

• Look at different populations of engineers
• Students
• Faculty
• Experts
• Descriptive studies
• Comparative studies

9
Connecting to Computer Science Education

• As you listen, think about this
• What would analogous work look like in the
  computer science education community?
• What might you do differently?
• Different model of design?
• Different experimental design?
• What kind of task would you use?

10
Todays Agenda

• Engineering and engineering education
• Research program on engineering design knowing
• Portraits of engineering design expertise
• Students
• Faculty
• Experts

11
Portraits of Design Expertise Students
Lenses Information Gathering Design Process
Iteration Broad Thinking



Engineering Design Process Skills



Freshmen
Seniors
12
One Dimension Broad Thinking
Lenses Information Gathering Design Process
Iteration Broad Thinking



Engineering Design Process Skills



Freshmen
Seniors
13
Broad Thinking Current Research at Multiple
Institutions
14
Portraits of Design Expertise Faculty
Lenses Information Gathering Design Process
Iteration Broad Thinking


?


?
Engineering Design Process Skills

?

?
Freshmen
Seniors
Faculty
15
Portraits of Design Expertise Experts
Lenses Information Gathering Design Process
Iteration Broad Thinking


?


?
Engineering Design Process Skills

?

?
Experts
Freshmen
Seniors
16
One Model of the Engineering Design
Process(derived from analysis of 7 engineering
texts)
Design Activities
Design Stages
(Identification of a Need) Problem
Definition Information Gathering Generation of
Ideas Modeling Feasibility of analysis Evaluation
Decision Communication (Implementation)
Problem Scoping Developing Alternative
Solutions Project Realization
17
Experimental Design

• Participant groups Freshmen, Seniors, Faculty
  (engineering educators) and Experts (practicing
  professionals)
• Solved Playground Problem thinking out loud
• Asked experiment administrator for information
  while solving the problem
• Took from 2 to 3 hours

18
(No Transcript)
19
Playground Problem Statement

• You live in a mid-size city. A local resident
  has recently donated a corner lot for a
  playground. Since you are an engineer who lives
  in the neighborhood, you have been asked by the
  city to design a playground.
• You estimate that most of the children who will
  use the playground will range from 1 to 10 years
  of age. Twelve children should be kept busy at
  any one time. There should be at least three
  different types of activities for the children.
  Any equipment you design must be safe for the
  children, remain outside all year long, not cost
  too much, and comply with the Americans with
  Disabilities Act.
• The neighborhood does not have the time or money
  to buy ready made pieces of equipment. Your
  design should use materials that are available at
  any hardware or lumber store. The playground
  must be ready for use in 2 months.

See Handout
20
Todays Agenda

• Engineering and engineering education
• Research program on engineering design knowing
• Portraits of engineering design expertise
• Students
• Faculty
• Experts

21
Think About It

• 1. Review the handout
• 2. Think about this question What differences
  might you see between the freshmen and seniors in
  terms of how they distribute their time across
  design activities?

then discuss with a neighbor
22
Mean Cumulative Time per Design StageFreshmen
vs. Seniors
minutes
design stage
23
Student Timelines
Successful Graduating Student (Quality Score
0.63)
Canonical Entering Student (Quality Score 0.37)
24
Student Timelines
Freshmen
Seniors
25
Results Students

• 26 Freshmen 24 Seniors (8 CE, 8 ME, 8 IE)
• Seniors
• had higher quality designs (whew!!)
• gathered more information, covering more
  categories
• made more transitions among design steps
• spent more time iterating and iterate more
  effectively
• progressed farther in the design process

26
Todays Agenda

• Engineering and engineering education
• Research program on engineering design knowing
• Portraits of engineering design expertise
• Students
• Faculty
• Experts

27
Think About It

• What differences might you see between the
  students and the faculty members in terms of how
  they distribute their time across design
  activities?

then discuss with a neighbor
28
Results Faculty

• 4 faculty members (2 mechanical engineering, 1
  industrial engineering, 1 engineering physics)
• 2 main approachesimplementers idea
  generators
• Implementers looked similar to seniors, with lots
  of iteration, information gathering, and
  precision.
• Idea generators worked at higher level of
  abstraction and thought beyond initial problem
  requirements.

29
Faculty Timelines
30
Faculty Cumulative Time Charts
31
Faculty Some Details

• Ann, Implementer Approach
• Transition from gathering information to modeling
  ideas
• User-centered (children, parents)
• Precise (e.g., swings at 93.71 each)
• Carl, Idea Generator Approach
• Made assumptions instead of gathering information
• Added own goals/constraints
• Children learning about participating in society
• Cognitive (not just physical) challenges

32
Todays Agenda

• Engineering and engineering education
• Research program on engineering design knowing
• Portraits of engineering design expertise
• Students
• Faculty
• Experts

33
Think About It

• What differences might you see between the
  students and the experts (or between the faculty
  members and the experts) in terms of how they
  distribute their time across design activities?

then discuss with a neighbor
34
Early, Emerging, New Results
19 Experts practicing engineers with an average
of 19 years of experience

• 2 Civil Engineers
• 3 Electrical Engineers
• 2 Industrial Engineers
• 2 Materials Science Engineers
• 8 Mechanical Engineers
• 2 Systems Engineers

35
Mean Total TimeFreshmen vs. Seniors vs. Experts
minutes
36
Mean Cumulative Time per Design StageFreshmen
vs. Seniors vs. Experts
minutes
design stage
37
Mean number of information requests Freshmen vs.
Seniors vs. Experts
38
Current focus Inductive analysis

• Experts efficiency framing vs. innovation
  framing

Susan Mosborg
39
Two Dimensions of Adaptive Expertise
Confused Novice
Adaptive Expert
Optimal Adaptivity Corridor
Routine Expert
Novice
Schwartz, D. L. Bransford, J. D.
40
Eric
High info seeker Gets neighborhood survey
equipment specs

• Swings
• Slides, with climbing towers
• Monkey bars
• Seesaw

41
Oscar

• Rocket ship to climb into, with slide
• Hip Martian sandbox--crater
• Clubhouse--space station
• Water geyser with flying saucer
• Lunar lander--geodesic dome of pipe

42
Oscar starts with high level goals

• So the key at this point is to think about what
  would be cool things for kids to do who are
  between the ages of 1 and 10 . . . . What do
  kids between 1 and 5 like? . . . They like to
  climb on stuff, they like to crawl through stuff,
  they like to hide, they like to run around, they
  definitely like to swing.

43
Pursues novelty with audience in mind

• What would kids think is cool?
• That hasnt been overdone
• What would kids like?

Im not super excited about this, just because
it seems really typical, really common in a
playground, but kids like it.
44
Pursues novelty with audience in mind

• What would kids think is cool?
• That hasnt been overdone
• What would kids like?

45
Oscar
2/3 way in before he starts rough detailing
Materials and how to construct
Tunnel mountain Make believe play set
(movie sets) Rope bridge Seesaw Swing Climber Ma
ze Fort Skate park Play hill Rope spider
web Tires Fire truck Basketball
hoop Horeshoes Ball pit Climbing wall

• Martian sandbox
• Water geyser
• Balance beam area
• Open grass play area
• Asphalt path around

46
Experts Timelines
Eric, Efficiency Framing
PD GATH GEN MOD FEAS EVAL DEC COM
Oscar, Innovation Framing
PD GATH GEN MOD FEAS EVAL DEC COM
47
Experts Cumulative Time Charts
48
Eric

• What I did was not very inventive, if you
  will, or creative, but it was expeditious. . . .
  On the other hand, it may be more real world in
  that sense.

49

• . . . . Frankly, in the real world, clients
  never pay us enough to really optimize. I think
  there is a clients have an expectation that
  theyre getting the best solution. Well, nobody
  typically pays enough money to get the best. You
  find a solution thats plausible, you might try
  to fine tune it from there. . . .
• My design philosophy is generally find a
  solution that works, and then if you can make it
  longer lasting, cheaper, easier to build, more
  durable, better looking, all those kinds of
  criteria, easier to maintainthats a big
  onethen you go from there off the plausible
  solution.

50

• An example of something that wouldnt be like
  that, if somebody said, here, build me a machine
  that will fly a person 200 feet off the ground
  using, I dont know, solar energy or something.
  Well, its something that we havent seen before,
  and so the design process might have to be
  different.
• However, in this case, if you said design me an
  office layout for a computer and a printer and a
  window looking out at the view, thats something
  people have seen before, they can pull together
  fairly quickly. This exercise with the
  playground I think allows somebody who has been
  around playgrounds and has been through the
  design process to draw on established solutions.
  

51
Eric and Oscar A Discussion

• Would Eric display more innovation framing if
  the task required it (i.e., if the task was more
  novel)?
• What counts as a novel problem?
• Does Eric need to be more innovative?
• Does Oscar need to be more efficient?
• Who is the more adaptive expert?

52
ReviewPortraits of engineering design expertise

• Students canonical freshman, successful senior,
  etc.
• Faculty Implementer (Ann), Idea Generator
  (Carl)
• Experts Efficiency Framing (Eric), Innovation
  framing (Oscar)

53
Emerging data...emerging patterns?Patterns of
patterns?
  Faculty "Implementer Approach Expert
"Efficiency Framing Student "Successful
Graduate"
?
Faculty "Idea Generator Expert "Innovative
Framing         
54
(No Transcript)
55
Portraits of Design Expertise
Lenses Information Gathering Design Process
Iteration Broad Thinking


? ?


Engineering Design Process Skills

?
?

?
?
Faculty
Experts
Freshmen
Seniors
56
Connecting to Computer Science Education

• As you listened, what kinds of thoughts
• What would analogous work look like in the
  computer science education community?
• What might you do differently?
• Different model of design?
• Different experimental design?
• What kind of task would you use?

57
More Information

• Contact atman_at_engr.washington.edu
• Read
• Adams, Robin S. Cognitive Processes in Iterative
  Design Behavior, Ph.D. Dissertation, University
  of Washington, 2001.
• Adams, Robin S., Jennifer Turns and Cynthia J.
  Atman. Educating Effective Engineering
  Designers The Role of Reflective Practice,
  Design Studies, Special Issue on Designing in
  Context, vol. 24, no. 3, 2003. pp. 275-294.
• Adams, Robin S., Jennifer Turns and Cynthia J.
  Atman. What Could Design Learning Look Like?,
  Design Thinking Research Symposium VI
  Proceedings, Sydney, Australia, November 2003.
• Atman, Cynthia J., Monica Cardella, Jennifer
  Turns, Robin S. Adams, Comparing Freshmen and
  Senior Engineering Design Processes An In Depth
  Follow-up Study, Design Studies, vol 26, no. 4,
  2005. pp.325-357
• Atman, Cynthia J., Justin R. Chimka, Karen M.
  Bursic, and H. L. Nachtmann, A Comparison of
  Freshman and Senior Engineering Design
  Processes, Design Studies, vol. 20, no. 2, pp.
  131-152, March 1999.
• Atman, Cynthia J., Jennifer Turns, Monica
  Cardella, and Robin S. Adams. The Design
  Processes of Engineering Educators Thick
  Descriptions and Potential Implications, Design
  Thinking Research Symposium VI Proceedings,
  Sydney, Australia, November 2003.
• Bogusch, Laura L., Jennifer Turns, and Cynthia J.
  Atman, Engineering Design Factors What Do
  Students Think is Important, Frontiers in
  Education Conference Proceedings, Kansas City,
  MO, November 2000.
• Bursic, Karen M. and Cynthia J. Atman.
  "Information Gathering A Critical Step for
  Quality in the Design Process." Quality
  Management Journal, 4(4), 1997, 60-75.
• Cardella, Monica E., Cynthia J. Atman, Jennifer
  Turns and Robin S. Adams Which Engineering
  Students Design Processes Change? Four Cases In
  progress, to be submitted to Journal of
  Engineering Education.
• Moore, Pamela L, Cynthia J. Atman, Karen M.
  Bursic, Larry J. Shuman, and Byron S Gottfried
  Do Freshmen Design Texts Adequately Define the
  Engineering Design Process? Proceedings of the
  American Society for Engineering Education Annual
  Conference, Anaheim, CA, 1995.
• Mosborg, Susan, Robin Adams, Rebecca Kim, Cynthia
  J. Atman, Jennifer Turns, and Monica Cardella
  Conceptions of the Engineering Design Process
  An Expert Study of Advanced Practicing
  Professionals, Proceedings of the American
  Society for Engineering Education Annual
  Conference, Portland, Oregon, 2005.
• Turns, J., Robin S. Adams, Angela Linse and
  Cynthia J. Atman. Bridging from Research to
  Practice in Undergraduate Engineering Design
  Education, International Journal of Engineering
  Education, vol. 20, no. 3, pp. 379-390, 2004.