Empirical Studies of Design Ideation: - PowerPoint PPT Presentation

About This Presentation
Title:

Empirical Studies of Design Ideation:

Description:

Department of Industrial and Manufacturing Systems Engineering ... 'Analysis of the Engineer Design Process in an Industrial Context', Grant Hill ... – PowerPoint PPT presentation

Number of Views:90
Avg rating:3.0/5.0
Slides: 26
Provided by: sant261
Learn more at: http://people.tamu.edu
Category:

less

Transcript and Presenter's Notes

Title: Empirical Studies of Design Ideation:


1
Empirical Studies of Design Ideation Alignment
of Design Experiments with Lab Experiments
JamiJ. Shah Noe Vargas-Hernandez Mechanical and
Aerospace Engineering Arizona State University,
Tempe, AZ
Steve M. Smith David R. Gerkens Department of
Psychology Texas AM University, College Station,
TX
Muqi Wulan Department of Industrial and
Manufacturing Systems Engineering Beijing
University of Aeronautics and Astronautics,
Beijing, China
NSF Grant Number DMI-0115447
ASME 2003 International Conference on Design
Theory and Methodology September 5, 2003, Chicago
Il
2
INTRODUCTION
MOTIVATION
There are many methods for design synthesis How
useful are these? Which ones are better? There
isnt much empirical data on specific
effectiveness of Idea Generation (IG) methods
3
INTRODUCTION
OBJECTIVE
If our experiments at different levels have the
same results, we can establish a connection and
we can run more of the simpler Lab
Experiments More Experiments means more empirical
data on specific IG methods Our alignment
approach is still WIP and is part of a bigger
project Our ultimate objective is to develop a
theoretical model of design ideation Such a
model would help us better understand IG methods
4
INTRODUCTION
DESIGN IDEA GENERATION METHODS
Figure 1. Classification of Idea Generation
Methods (Shah et al., 2000)
5
PRELIMINARY RESEARCH
PAST EXPERIMENTS
For several years we have been doing experimental
studies to define the effectiveness of ideation
  • We used two distinct approaches to conducting
    experiments
  • Direct Method IG methods are studied as a whole
  • Indirect Method Ideation Components are studied
  • And two distinct approaches to assessing the
    effectiveness
  • Process Assess the ideation process (e.g.
    protocol studies)
  • Outcome Assess the ideas produced (e.g.
    sketches)

6
PRELIMINARY RESEARCH
IDEATION COMPONENTS
Ideation Components are mechanisms that are
believed to intrinsically promote IG or to help
designers overcome mental blocks.
  • Examples of Ideation Components
  • Provocative Stimuli
  • Deferred Judgment
  • Flexible Representation
  • Frame of Reference Shifting
  • Incubation
  • Example Exposure
  • Examples of Blocks
  • Being Judgmental
  • Emphasis on Quality
  • Lack of Motivation
  • Having a tight grip on problem specs.
  • Rigid Problem Representation
  • Design Fixation
  • Imposing Fictitious Constraints

These are common Known in Engineering Design
Research and acknowledged by Cognitive Psychology
7
PRELIMINARY RESEARCH
OUTCOME EFFECTIVENESS
Four measures were defined in our previous
projects (Shah, Kulkarni, and Vargas-Hernandez,
2000).
Table 1. Effectiveness Measures for Idea
Generation Outcome.
8
RESEARCH APPROACH
FUNDAMENTAL ISSUES
How can we compare results from experiments at
different levels?
  • LAB EXPERIMENTS
  • (Done AM Psychologists)
  • Focus on atomic cognitive processes
  • Little similarity between the condition of these
    experiments and design concept generation in the
    real world.
  • DESIGN EXPERIMENTS
  • (Done by ASU Engineers)
  • Simulate real world better
  • Incorporate more and complex variables
  • Require prohibitive number of experiments
  • Unable to explain the performance of methods
    under different conditions.

How can we align these two?
9
RESEARCH APPROACH
RESEARCH APPROACH
How can we compare results from experiments at
different levels?
Figure 2. Research Strategy
10
RESEARCH APPROACH
IDEATION COMPONENTS
The number of components identified is more than
a dozen. Because of limited resources and the
prohibitive number of experiments required to
study all possible interactions, only the most
relevant were selected.
Table 2. Selected Components
11
RESEARCH APPROACH
DESIGN OF EXPERIMENTS (DOE)
Two levels were considered for each of the
selected components. Although more levels could
be defined, it is recommended to run experiments
initially with few levels.
Full Factorial Experiments A Frame of Reference
Shifting B Incubation C Example Exposure
  • Simple Comparative Experiments
  • Provocative Stimuli
  • Suspend Judgment
  • Flexible Representation

12
RESULTS
DESIGN EXPERIMENTS DONE BY ASU ENGINEERS
Experiment Variables Subjects Undergraduate
Engineering students. Comparable
expertise/knowledge between subjects is assumed.
Task One design problem was used for all
experiments. The objective was to design a device
for throwing a ping-pong ball the farthest
distance. A list of allowed materials was given
this to improve the quality of sketches. Idea
Recording Subjects were asked to generate ideas
individually using sketches Nuisance
Variables Similar environmental settings procured
for each run (classroom, noise, light, etc.)
13
RESULTS
DESIGN EXPERIMENTS DONE BY ASU ENGINEERS
Figure 3. Sample Sketches from Design Experiments
14
RESULTS
LAB EXPERIMENTS DONE BY TAMU PSYCHOLOGISTS
  • Experiment Variables
  • Subjects
  • Undergraduate Psychology students. Comparable
    expertise/knowledge between subjects is assumed.
  • Task
  • Listing members of large taxonomic categories,
    sense impression categories, and ad-hoc
    categories.
  • Divergent thinking, unusual uses of common
    objects.
  • Idea Recording
  • Subjects were asked to generate ideas
    individually using text for member listing tasks
    and sketches for divergent thinking tasks
  • Nuisance Variables
  • Similar environmental settings procured for each
    run (classroom, noise, light, etc.)

15
RESULTS
LAB EXPERIMENTS DONE BY TAMU PSYCHOLOGISTS
Figure 4. Sample Sketches from Lab Experiments
16
RESULTS
IDEATION COMPONENT INCUBATION
Table 5. Lab Experiments Done by TAMU
Psychologists Mean Ideation Effectiveness Scores
Table 6. Design Experiments Done by ASU
Engineers Mean Ideation Effectiveness Scores
17
RESULTS
IDEATION COMPONENT INCUBATION
Table 7. Two Sample t-test
18
RESULTS
IDEATION COMPONENT INCUBATION
Table 8. Correlation Between Lab and Design
Experiments
19
CONCLUSIONS
IDEATION COMPONENT INCUBATION
  • Based on the results from (TAMU Psychologists)
    Lab and (ASU Engineers) Design Experiments,
    Incubation increases the effectiveness of ideas
    generated. Results correlate at both levels and
    show a satisfactory confidence level.
  • Incubations positive impact on Design Ideation
    is substantiated by concrete Engineering evidence
    (from ASU Design Experiment results) and has a
    theoretical basis (from TAMU Lab Experiment
    results).

20
CONCLUSIONS
OVERALL
  • More experiments needed to prove connection
  • The alignment procedure provides a framework for
    comparison between both levels. Results for
    Incubation exemplify how the alignment works.

Connection Proven
Experiments on other components have been
completed at ASU and TAMU.
21
FURTHER CONCLUSIONS
  • According to our results we found Frame of
    Reference Shifting (FORS), Incubation (I), and
    Example Exposure (EE) to have similar main
    effects.
  • Interaction effects weren't that clear, probably
    because some components are much alike (specially
    FORS and EE) and hence aren't independent.
  • This generates a question Maybe these Ideation
    Components have the same effect on ideation ?
  • Two or more Ideation Components sharing the same
    effect could belong to the same higher level
    Ideation Principle

22
FUTURE WORK
IDEATION PRINCIPLES
Table 9. Comparison of Ideation Principles with
Cognitive Components
23
FUTURE WORK
IDEATION PRINCIPLES
  • Refine Ideation Principles and its
    Implementations (Ideation Components)
  • Run more exercises, collect and analyze more data
    to prove/disprove our theory about Ideation
    Principles
  • We still continue experimenting on Ideation
    Components, but with a better understanding of
    Ideation Principles, experiments can be better
    targeted (e.g. less redundant) and more efficient.

24
REFERENCES
  • Altshuller, G., 1984, Creativity as an Exact
    Science, Gordon and Breach, New York.
  • Dennehy, E.B., Bulow, P., Wong, F., Smith, S.M.,
    and Aronoff, J.B. (April, 1992). A test of
    cognitive fixation in brainstorming groups. Paper
    presented at the meeting of the Eastern Psych.
    Association, Boston, MA.
  • Dodds, R.A., and Smith, S.M., 1999, Fixation. In
    M.A. Runco Pritzker (Eds.) Encyclopedia of
    Creativity, San Diego, CA Academic Press.
  • Ericsson, K., and Simon, H., 1984- Protocol
    Analysis - verbal reports as data, MIT Press.
  • Finke, R.A., Ward, T.B., and Smith, S.M., 1992,
    Creative Cognition Theory, Research, and
    Applications, Cambridge, MA MIT Press.
  • Hale, C., Analysis of the Engineer Design
    Process in an Industrial Context, Grant Hill
    Pubs, Cambridge, 1987.
  • Jansson, D. G., and Smith, S. M., 1991, Design
    Fixation, Design Studies, Vol. 12, pp. 3-11.
  • Koestler, A., 1964, The art of Creation,
    Hutchinson and Co., London.
  • Langley, P., and Jones, R., 1988, Computational
    model of scientific insight, in R. J. Sternberg,
    ed., The nature of creativity contemporary
    psychological perspectives, Cambridge University
    Press, NY.
  • McKoy, F., 2000, Experimental Evaluation of
    Engineering Design Representations for Idea
    Generation, MS Thesis, Arizona State University,
    Tempe, AZ.
  • Mckoy, F., Vargas-Hernandez, N., Summers, J. D.,
    Shah, J., 2001, Experimental Evaluation of
    Engineering Design Representation on
    Effectiveness of Idea Generation, Proceedings,
    ASME Design Theory and Methodology Conference,
    Pittsburgh, PA.
  • Koestler, A., 1964, The art of Creation,
    Hutchinson and Co., London.
  • Martindale, C., 1995, Creativity and
    Connectionism, in S. M. Smith et al., eds., The
    creative cognition approach, MIT Press,
    Cambridge, MA.
  • Schön, D., 1991,Teaching and learning as a
    design transaction, in Research in Design
    Thinking Delft Press, 1991.

25
REFERENCES
  • Schwartz, B.L. and Smith, S.M., 1997, The
    retrieval of related information influences
    tip-of-the-tongue states. Journal of Memory
    Language, 36, 68-86.
  • Shah, J., 1998, Experimental Investigation of
    Collaborative Techniques for Progressive Idea
    Generation, Proceedings, ASME Design Theory and
    Methodology Conference, Atlanta, GA.
  • Shah, J., Kulkarni, S., Vargas-Hernandez, N.,
    2000, Guidelines for Experimental Evaluation of
    Idea Generation Methods in Conceptual Design,
    Journal of Mechanical Design, vol. 122, no. 4,
    pp. 377-384.
  • Shah, J. J., Vargas-Hernandez N., Summers, J. D.,
    Kulkarni, S., 2001, Evaluation of Collaborative
    Sketching as an Idea Generation Technique for
    Engineering Design, Journal of Creative
    Behavior, 353, pp.1-31.
  • Smith, D. K., Paradice, D. B., and Smith, S. M.
    (2000). Prepare your mind for creativity.
    Communications of the Association for Computing
    Machinery , 43, 110-116.
  • Smith, S. M., 1995, Creative Cognition
    Demystifying Creativity, in C. N. Hedley et al.,
    eds., Thinking and literacy the mind at work,
    Lawrence Erlbaum Associates, Hillsdale, NJ.
  • Smith, S. M. and Blankenship, S. E., 1991,
    Incubation and the persistence of fixation in
    problem solving, American Journal of Psychology,
    104, 61-87.
  • Smith, S. M., Carr, J. A., and Tindell, D. R.,
    1993, April, Fixation and incubation in word
    fragment completion. Paper presented at the
    meeting of the Midwestern Psychological
    Association, Chicago, IL.
  • Smith, S. M., and Vela, E., 1991. Incubated
    reminiscence effects. Memory Cognition, 19 (2),
    168-176.
  • Smith, S. M., Sifonis, C. M., and Tindell, D. R.,
    1998, Hints do not evoke solutions via passive
    spreading activation. Proceedings of the
    Twentieth Annual Meeting of the Cognitive Science
    Society, Madison, WI.
  • Vargas-Hernandez, N. and Shah, J. J., 2002,
    Inventory of Creativity Exercises 1995-2002,
    Tech. Report ASU/DAL/IG/02-1, Arizona State
    University.
  • Wallas, G., 1926, The Art of Thought, Harcourt,
    New York.
Write a Comment
User Comments (0)
About PowerShow.com