Using Electronic Interviews to Explore Student Understanding

1
Using Electronic Interviews to Explore Student
Understanding DJ Wagner1,2, JJ Rivera1, Fran
Mateycik1, and Sybillyn Jennings3 1Rensselaer
Polytechnic Institute, 2Grove City College,
3Russell Sage College
Context
Study
Abstract
The Science of Information Technology (ScIT)
introduces students of all majors to the
physical principles behind the operation of
information systems. NSF is supporting current
efforts to make ScIT materials useful to a large
audience. As part of that effort, we are
designing diagnostic questions addressing the
topics covered in ScIT. Diagnostic questions
are most useful when they address student
preconceptions 1. Clinical Interviews have
long been regarded as an effective means of
eliciting students conceptual frameworks 1-3.
Piaget 3 developed the method of critical
exploration to see what thoughts lay behind a
childs initial answer to a question, and the PER
community has used this method extensively 1-2.
We used interviews to study students
understanding of total internal reflection (TIR)
and how TIR relates to the operation of optical
fibers.
This poster reports on methods used to probe
student understandings of optical fibers and
total internal reflection (TIR). The study was
conducted as part of the expansion and
improvement of web-based materials for an
innovative introductory physics course.
Development of these materials includes the
refining of multiple-choice diagnostic questions
by examining preconceptions and misconceptions
commonly held by students. Initially, we
conducted face-to-face Piaget-style interviews
with a convenience sample. Our next step was to
interview students taking the course at
Rensselaer. Physical limitations necessitated
that this be done from a distance, so we
conducted e-interviews using a Chat Room. In
this paper we focus on the e-interview
experience, discussing similarities to and
differences from the traditional face-to-face
approach. In the process, we address how each
method informs us about students activation of
prior experiences in making sense of unfamiliar
phenomena (e.g., transfer of learning).

• What is an optical fiber and what does it do?
• Follow-up questions included How does the fiber
  keep the light from escaping? and What can you
  tell me about refraction?

2 Physics Faculty
5 Physics Students (REU)

• Group A Faculty, REU students, and in-service
  (non-physics) teachers were interviewed in a
  traditional (face-to-face) form in the Summer of
  2002.
• Group B ScIT students (both pre- and
  post-instruction) were interviewed via WebCTs
  Chat Room in the Spring of 2003.

5 In-Service Teachers
22 ScIT Pre-Instruction
8 ScIT Post-Instruction
Electronic Interviews
Face-to-Face Interviews
Features
Demographics of Interviewees

• Convenience Can be conducted from wherever,
  whenever, and however is convenient (at home,
  listening to music, brushing teeth, in jammies,
  etc.).
• S313 Yea, I must agree that this is very
  convenient for me too.

Physics Background of Interviewees. Physics Background of Interviewees. Physics Background of Interviewees.
Highest Physics Course Group A Group B
High School Physics 1 9
College Physics I 0 10
College Physics II 4 1
Upper-Level Coll. Phys. 5 1
Physics Faculty 2 0

• Can involve participants in different parts of
  the country, as was the case for our study.

Physics Information Also see our other poster ?

• Typed thoughts often come across more coherently
  than spoken thoughts no place holders.

Features
Students in both formats Activated Memories

• Minimal transcription needed!!! (WebCT records
  conversations.)

I When you hear optical fiber (or fiber optic)
what do you envision?S310 Those wands that they
would sell in Disney Land I OK. Tell me about
those wands.S310 Well, they were handheld and
looked similar to a flashlight with a battery
inside and a switch to turn them on or off --
when you turned them on, there were all of these
wires coming out of the top of the wand, and the
ends would glow different colors

• The well-established standard lots of studies,
  literature

S5 Well I'm kind of thinking like those things
you get at like Disney World or something with a
characters head and then it's got like those
little strings that come out of the top. And
there's like a light that you see at the end of
the strings, but the strings themselves don't
look lit up. Like in those I think they're
plastic because theyre soft and you can kind of
brush your hand through them. So I would imagine
that it's some kind of non-metallic material.

• Interviewer does not need to worry about her
  gestures giving away her thoughts about the
  students answers.

• Can utilize multiple communication channels.
  (words, gestures, tone of voice, etc)

• Apparent anonymity (due to lack of face-to-face
  contact) makes some students more comfortable in
  expressing their lack of knowledge.

• Can supply props or equipment.

• Lack of distractions 1 interview at a time in
  controlled environment

Students in both formats Tried On Different
Models as they tried to describe fibers.

• Multitasking Can hold 2 or 3 overlapping
  interviews in different Rooms, typing questions
  for student 3 while students 1 and 2 are typing
  their responses. (4 at a time is a bit much.)

For complete S314 transcript, see papers
below S314 starts with a visualization involving
water I think of a long hollow cable, made of
a light weight material, it should be waterproof
Possibly glass or plastic -- Light-weight in
the sense that it should be flexible -- I think
it has these characteristics to ensure that the
data will reach or has the potential to reach its
final destination. Provides more detail upon
request light flows Inside the hollowed out
area -- Like water flowing through a pipe. The
light doesn't escape because the water
surrounding the cable on the outside doesnt sic
allow the light to ecape sic from the inside of
the cable. I think the water has a higher
density surrounding the cable, allowing the light
not able to escape the cable. The light never
touches the water, it only flows in the inside
(hollowed part ) of the cable -- The water
surrounds the outside of the cable. Upon
further questioning, she starts to doubt her
model and qualifies her statements Now I think
I am confused -- Let me re-phrase my answer We
have a hollow cable, made of either glass or
plastic This cable is felxible sic Data
flows through this cable, i think this data is
made of bundles of light I am not sure if this
cable should be waterproof or not I think as
long as the data (bundles of light) are able to
bounce around inside the cable, this data should
be able to travel from one port to another.
For complete S7 transcript, see papers below S7
starts with a reflective model its almost a
series of reflections I dont want to say
mirrors, but its got to be mirror-like, a
mirror-like substance. Then thinks about a
containment model I guess if, if you did just
enclose light in, cause I know the, the tubing
you know the light cant get out of it. Maybe
it, maybe it wouldnt need to reflect if it, uh,
if it, you cant escape the, the insulator,
right? maybe it can just, shwooo, travel right
through. Maybe it doesnt need to
reflect. Then goes back to reflection Well,
from there the edge of the fiber, from there,
you would have to, would have to reflect off
cause you gotta keep going. So it would be kind
of like, uh, draws zig-zag. You know?
And ends in a quandary But then it also
couldnt slow down at the same time Ive seen,
it almost looks like ... its a plastic
substance, I know, cause they use it for
computers and things, and it almost looks like
its a plastic, consistent construction, but
its gotta be reflecting somehow. I dont know.

• Can draw figures and write eqns on paper (see
  samples below).

• Ubiquity of Instant Messaging makes e-interviews
  familiar and fun.
• S301 i must say this is pretty neat

• Interviewees do not need to be familiar with
  technology

Constraints

• E-interviews are a good fit for development of
  web-based materials.

Constraints

• Substantial transcription time for interviewer.

• Larger time commitment for interviewee. Our
  median e-interview time was 6 times our median
  f2f time.

• Hmms, sentence fragments, and other place holders
  can make transcript choppy (see full transcript
  below).

• Not a controlled environment.
• S304_post sorry about the pause, i'm in my
  apartment and the UPS guy came and I live on the
  third floor..

• Subject may convey information using gestures
  and/or tone of voice, which are not easily
  transcribed (see video, compare to transcript).

• No ability to write or draw free-hand. Some
  work-arounds were developed, e.g., using
  keyboard symbols / \ - to depict lights path

• Can run into conflicts if one interview runs over
  and has to be awkwardly curtailed when next
  interviewee shows up.

• Interviewer cannot glean affective information
  from unintended reactions or unconscious
  responses of interviewee

1 Lillian C. McDermott, Bridging the Gap
Between Teaching and Learning The Role of
Research, The Changing Role of Physics
Departments in Modern Universities Proceedings
of ICUPE, ed. by E. F. Redish and J.S.
Rigden. 2 E. Hunt and J. Minstrell, A
Cognitive Approach to the Teaching of Physics,
Classroom Lessons Integrating Cognitive Theory
and Classroom Practice, ed. by McGilly (MIT
Press, 1994). 3 J. Piaget, The childs
conception of the world, trans. by J. A.
Tomlinson. (Littlefield, NJ, 1926/1972).
RPI work supported in part by NSF CCLI Program
under grant DUE-0089399. Thanks to Leo Schowalter
for access to ScIT students, and to the rest of
the ScIT advisory committee Karen Cummings,
Toh-Ming Lu, Saroj Nayak, Jim Napolitano, Peter
Persans, and Wayne Roberge.