Conceptual testing in engineering courses: Testing students conceptual understanding of thermodynami

1
Conceptual testing in engineering
coursesTesting students conceptual
understanding of thermodynamics and fluid
mechanics in a course in Energy Systems
Michael May, LearningLab DTU Technical University
of Denmark mma_at_dtv.dk
2
A course in Energy Systems at DTU

• Energy Systems An introductury (1. semester)
  course in
• Thermodynamics
• Fluid mechanics
• Case studies of Energy Systems (ex.
  Refrigerator)

3
Different uses of conceptual tests

• When?
• Test of students understanding relative to the
  requirements for a course
• Test of students understanding of core elements
  during a course
• Test of students understanding at the end of the
  course

• How? (-gt A typology of conceptual tests)
• Representational forms used in the test
  questions
• Level of learning addressed by the type of
  questioning

• Why? (-gt Fuctional roles of testing)
• Test used to promote student reflection (on own
  learning)
• Test used as a form of measurement of learning
  (information to teachers)
• Test used as portfolio documentation

There seems to be a kind of progression at the
institutional level in the discovery and
acknowledgement of functional roles of testing
4
Testing a conceptual requirement for the course
Example of a test borrowed from the
Thermodynamics Concept Inventory reused here
as a test of a conceptual requirement (regarding
the concept of mechanical energy which should be
well established in high school physics) for the
course in Energy Systems
Consider a marble that is held at the lip of a
bowl. The marble is released, with no force
added, and begins to roll down into the bowl.
Assuming the rolling process is frictionless,
for which of the figures below does the ball
have maximum energy?
5
Distribution of answers (beginning of the course)
(43 answers)
Could there be something in the test that
promotes wrong answers?Is it valid as a measure
of understanding the concept of mechanical
energy?
6
Repeating the question with variation

• At the end of the course we wanted to test
  understanding of the same concept
• The dilemma of testing for the same concept we
  cannot use the same test, because some will
  remember the correct answer (even without
  understanding)
• We can however tested again with a similar
  question in this case we introduced specific
  questions about potential and potential engergy
• This way we also avoided the problem of students
  being confused about the unspecific question
  about energy (for which of the figures below
  does the ball have maximum energy?)

7
The revised test

• Consider a marble that is held at the lip of a
  bowl. The marble is released,
• with no force added, and begins to roll down into
  the bowl.
• Assuming the rolling process is frictionless, for
  which of the figures below
• does the ball have
• maximum potential energy?
• maximum kinetic energy?
• maximum mechanical energy?

8
Distribution of answers (end of the course)
Distribution for subquestion (b) on maximum
kinetic energy
(38 answers)
9
Promoting student reflection through testing

• This is another example from the same test given
  in the beginning of the course on Energy Systems
  (also copied from the Thermodynamics Concept
  Inventory) this time testing understanding of
  the Ideal Gas law

• The figure depicts a gas heated and expanding in
  a sealed, frictionless, piston-and cylinder
  arrangement, where the piston mass and the
• atmospheric pressure above the piston remain
  constant.
• The pressure of the gas will
• Increase
• Remain the same
• Decrease
• Insufficient information..

10
Distribution of answers
11
Student in-class reflection during test feedback

• In giving feedback to students on the test a
  discussion arose,
• which revealed important aspects of students
  reasoning
• Some of the students did perhaps not recall and
  fully understand theIdeal Gas law as taught in
  high school physics (again a requirement forthe
  course), but more importantly
• Many students defended their wrong answer with
  a form of (highly relevant) practical
  resoning(in practice the work done in moving
  the piston will correspond to a fluctuation in
  the pressure, meaning that it will momentarily be
  increased untill the volume is expanded and an
  equilibrium is established with the atmospheric
  pressure)
• The nature of ideal models and the conditions
  under which they are true as opposed to more
  concrete physical models are not understood by
  students in introductory engineering courses and
  this understanding is in itself an important
  objective for engineering education
• The in-class discussion of the test was an
  important learning experiencefor students

12
Features of students conceptual understanding

• There will (allmost) always be some
  preconception of core content - based on
  intuitive anchoring - based on experience -
  base on prior education

• Prior understanding will generally include
  misconceptions
• - but it will also be partially true / valid
  - this should be used actively as a bridge to
  scientific understanding - conflicts (experience
  versus intuition or conflicting conceptualization
  
• among students) can be utilized to promote
  conceptual change - there will generally be a
  small set of prototypical misconceptions of core
  content (i.e. misconceptions are not arbitrary
  and chaotic but motivated and systematic) (the
  phenomenographic approach)

• The misconceptions and intuitions of prior
  understanding will persist in parallel with
  scientific understanding (if it is not
  explicitely addressed) !

• Scientific understanding is not organized as an
  axiomatic systems(it can be inconsistent,
  incomplete and full of holes) - specifically
  students can have advanced knowledge of core
  elements even if some of the conceptual
  requirements are not fully established

13
Testing understanding of the Bernouilli equation
Consider a horizontal pipe with a contraction as
shown in the figure. The fluid flow from plane 1
to plane 2 takes place without loss. The
pressuere at plane 1 and plane 2 is called p1 og
p2 respectively. Indicate whether you expect (and
give your reasons for this belief) (a) p1 gt
p2 (b) p1 lt p2 (c) p1 p2
14
Intuitive preconceptions of pressure
There is an intuitive preconception of
pressure that is sustained byexperience in
the sense of being under pressure, have less
space, as When the human body is under
pressure from all sides during anescape through
an emergencyexit (think of a fire in a building).
The problem of cause is that this experience /
intuition of pressure is verydifferent from
the concept of pressure in fluid mechanics, but
never the less it persists as a naïve
conception of pressure in spite of physics
teaching andit can interfere with students
reasoning (cf. diagram illustrating an
interview fragment).
15
Invalid qualitative reasoning

• Many students attempt to apply a form of
  qualitative reasoning to the question of fluid
  flow i a pipe with contraction
• but fail to give a correct answer because they
  are mislead by the static nature of their
  conceptualization
• They are perhaps applying reasoning implied by
  the Ideal Gas law (and not Bernouillis equation)

16
Testing understanding of the Bernouilli equation
The Bernoulli equation can be written Describe
the physical meaning of the terms of the
equation.
The intended answer implies an understanding the
equation as a particular expression of the
general principle of conservation of
energy pressure kinetic energy potential
energy constant A considerable part of the
students on a more advanced course could only
describe (some of) the individual parameters (not
the physical meaning of the terms)
17
Understanding across forms and scenarios

• Deep understanding will be characterized by the
  ability to
• Apply a general law schema to different specific
  situations
• Use examples from one domain as analogies in
  another domain
• Reformulate conceptual understanding (not bound
  to one specific text book formulation), i.e. it
  can be stated in the students own words
• Translate conceptual understanding between
  (moving across) different forms of
  representation (in language, in diagrams, in
  equations, in graphs etc.)

Didactic consequence Variations in forms of
representation and in scenarios of application
are important for improvements in conceptual
understanding
18
Graph-based sketching Testing Fouriers law
19
Prototypical answers Phenomenography

• There will be a small set of distinct
  prototypical answers (rather than a large
  diffuse set)
• The answers can be classified in a
  meaningfull way
• Wrong answers can be interpreted as a sign
  of different types of conceptual difficulties

The 3 types of answers found among students of
chemical engineering tothe Fourier question a
is the correct answer, b is the answer given by
thosestudents who remember the linearity implied
in Fouriers law (but forgets to takearea into
consideration, and c is the answer given on the
basis of recollection ofthe shape of the graph
as a pure image (with no understanding of
model).
20
A classification scheme for conceptual tests

• The following classification scheme is based on
  the assumptions
• that different representational forms makes a
  difference for the level and type of
  understanding that can be tested and
• that different types of problems can be used to
  probe different levels of learning and
  understanding.

21
Types of conceptual tests
Representational form / Problem type
(level)
Recognize(Know)
Compute (Apply)
Sketch(Understand)
Diagnose (Analyze)
Construct (Synthesize)

Construct theproblem fromthe description
Find errors inthe followingtext
Explain inyour ownwords
Text Numerical/ Algebraic Diagram Graph
Image
Indicate the unitsof the entities
Construct a model of thefollowing ..
Compute theresult of ...
Sketch asolution to ..
Indicate physicalmeaning of the terms of the
equation
Find errorsin the equation..
How will the circuit representedin this
diagramwork?
Sketch adiagram for ..
What does thediagram express ..
Sketch thegraph youexpect for ..
Find the graph for ..
Give a physical interpretation ofthe graph ..
Identify situationfrom image ormeasurement ..
A few exemples are indicated here (but all
combinations in the matrix can be explored!)