Closing the Communication Gap Between Undergraduates and Mathematics Professors

1
Closing the Communication Gap Between
Undergraduates and Mathematics Professors

• Daniel Villarreal
• November 24, 2009

2
Presentation Format

• Basics about the project
• Background Linguistics info
• Mini literature review
• Methodology
• Results/Discussion
• Homework!

3
Project Basics
4
Project Basics

• Conducted in support of an Honors thesis in
  Linguistics for the 2009-2010 academic year
  (Committee Charity Hudley, Taylor, Li)
• Supported by a Dintersmith Fellowship
• Social-science-grounded project
• The inspiration

5
James Villarreal
6
Background Linguistics Info
7
Terminology

• Syntax
• Semantics
• Phonetics/Phonology
• Prosody
• Pronunciation
• Includes phonetics, phonology, and prosody

8
Terminology

• Accent
• The way a person pronounces the words in a
  language
• Dialect
• A variety of the language spoken by some
  pre-defined group (regional, social, ethnic)
• Ideolect
• The variety of the language particular to a
  single person

9
Languages Species

• A good way to conceptualize linguistic variation
  is to consider biodiversity
• Example Dogs
• Just as theres no one canonical dog, theres
  no canonical English
• It is possible, however, to define a Standard

10
Linguistic Principles

• Universality of Accentedness
• Rule-based Accents
• William Labov 1969 The Logic of Nonstandard
  English
• In other words, no accent is inherently inferior
  to any other
• But some accents may be harder for a speaker of
  another dialect to understand
• When that different dialect is Standard
  American, that accent is incomprehensible

11
Mini Literature Review
12
Backstory

• Beginning in the 1980s, the majority of
  American-born STEM (Science, Technology,
  Engineering, and Mathematics) graduates began to
  choose jobs in industry rather than academia
  (Mooney, 1989)
• By 1989, more than half of STEM degree recipients
  were foreign-born (ibid.)

13
The Foreign TA Problem

• In 1984, the linguist Kathleen Bailey identified
  what she called the Foreign TA Problem the
  communicative difficulties engendered by the
  interaction between non-native speaking teaching
  assistants and their students (Bailey, 1984, p.
  3)
• In a 1980 study of University of Minnesota
  undergraduates, almost half reported that having
  a NNS TA had hurt the quality of a course they
  had taken, whereas only 9 believed that an NNS
  TA had helped (cited in Bailey, 1984)

14
The Communication Gap

• A 1989 study examined the effects of instructor
  gender, student SAT score, class term, age,
  international TA, and textbook on undergraduates
  test scores in a macroeconomics survey course.
• Of these, no variable was responsible for a
  greater drop in scores than was the presence of
  an international TA (Watts Lynch, 1989)
• Anecdotal problems abound

15
The Communication Gap, Contd

• So its just a matter of the professor speaking
  in a way that the students cant understand,
  right?
• WRONG
• Rubin, 1999
• Prof. Lis California story
• Thus, in my formulation, the communication gap
  consists not only of actual misunderstanding, but
  also of bias

16
English Language Learners

• The older a second language learner is at the
  time of learning a language, the more difficult
  it becomes to make ones accent resemble a native
  accent (Gass Selinker, 2001)
• English language learners can have great
  difficulties pinpointing the source of
  accent-related communication breakdowns (Derwing,
  2003)

17
Professor Training

• Lots of universities, especially state
  universities, mandate that new hires (especially
  TAs) whose first language is not English pass
  TSE, TOEFL, etc. (Cassell, 2007 Plakans, 1997
  Davies, Tyler, and Koran, 1989)
• Several researchers have doubts as to the
  efficacy of these tests in evaluating classroom
  readiness (Tyler, 1992 Young, 1989 Halleck and
  Moder, 1995)

18
Student-Centered Research

• Donald Rubin Support for ITAs (and also
  continuing support for non-native English
  speaking faculty members) is key, and much
  progress has been made in many fine programs on
  that score. But also key is attention to
  undergraduates' listening abilities. Very few--if
  any--programs exist to support undergraduates as
  listeners of World Englishes. (qtd. in Gravois,
  2005)

19
Math is Different

• Byrd and Constantinides, 1992 so many of our
  early assumptions about teaching (based on
  teaching styles preferred in ESL) do not hold for
  the teaching of mathematics. (p. 166)
• Topic familiarity influences comprehension of
  non-native speakers to a greater degree than even
  accent familiarity (Gass and Varonis, 1984)
• Math anxiety

20
The Need for This Research

• In summary, there are several factors that make
  this research unique and necessary
• Focuses on interactions with professors, not TAs
• Shifts some of the burden of communication to
  students
• Focuses on the mathematics classroom
• Attempts to address both components of the
  communication gap bias and genuine
  misunderstanding

21
Methodology
22
Methodology

• Null hypothesis Training program not effective
  in aiding students comprehension
• Alternative hypothesis Training program IS
  effective
• Population of interest William and Mary
  undergraduates
• Sampling method Randomish
• In social science research, recruitment is often
  the hardest part!

23
Recruitment

• Hoped to get students to participate out of the
  goodness of their hearts
• Help advance research that improves classroom
  interactions between professor and student
• Help a fellow student out with research
• But in the end
• Fortunately, the Charles Center agreed to
  reimburse Cheese Shop-related expenses

24
Testing Sessions

• Six testing sessions in Dulin Learning Center (in
  Swem) from October 3-6
• Five parts
• Consent Form/Assignment
• Lesson 1 Assessment 1
• Inter-lesson Module
• Lesson 2 Assessment 2
• Linguistic Profile Questionnaire

25
Testing Sessions

• Six testing sessions in Dulin Learning Center (in
  Swem) from October 3-6
• Five parts
• Consent Form/Assignment
• Lesson 1 Assessment 1
• Inter-lesson Module
• Lesson 2 Assessment 2
• Linguistic Profile Questionnaire

• Three different inter-lesson modules the accent
  training program, a Control group, and another
  variable group
• Thus, the variable of interest was the difference
  in scores
• A2 A1

26
Experimental Design Check

• But what if Assessment 2 were significantly
  harder (or easier) than Assessment 1?
• We need to randomize the order of presentation to
  control for difficulty effects
• I ended up creating three lesson videos for the
  project, so there were ?? different orders for
  presenting two videos
• What do I mean by creating lesson videos?

6
27
Lesson Videos

• In my original design, I would record professors
  teaching mini-lessons (audio only)
• Confidentiality
• Bias effects (Rubin 1999)
• I would then make Flash video animations of these
  lessons, simulating professors writing on a
  chalkboard
• But recruiting professors turned out to be a lot
  harder than I thought

28
Webcasts

• On the advice of a friend, I looked around to see
  if I could find any webcasts or open courseware
  to fit my needs
• At least moderate accent
• Not too high-level, not too low-level
• Not too long
• Derivative works permitted
• This turned out to eliminate just about
  everything on the Web!

29
Lesson Videos

• Using audio from the Discrete Structures course,
  I chose three lessons
• Permutations and Combinations
• The Proof That the Square Root of Two is
  Irrational
• Relations and the Cartesian Product
• I then created three videos on PowerPoint, with
  text appearing on the screen as Prof. Kamala
  spoke it (chalkboard simulation)

30
Inter-Lesson Modules

• Three groups, as defined by the inter-lesson
  module they viewed Control, Bias, and Training
• Training module instructed students on accent
  ideology, then specific features of Prof.
  Kamalas accent
• Control module consisted of inert materials
• Bias module contained an article, blog post, and
  table of professor ratings meant to simulate
  campus conversation by presenting professors in a
  less-than-favorable light

31
Testing Groups

• Each participant, then, had two different
  treatment factors their inter-lesson module and
  the videos they viewed (and in what order)
• 3 inter-lesson modules
• 6 permutations of 2 videos
• These were independently randomized
• This meant that there were 6 x 3 18 testing
  groups

32
Testing Groups

• Examples of testing groups
• Square Root of 2 gt Bias gt Perm-Comb
• Relations gt Control gt Square Root of 2
• Relations gt Training gt Square Root of 2
• These would have been TERRIBLE names
• Actual names for the above groups
• Iceberg
• Sunflower
• Fuchsia

33
Web-Based Content

• Key parts of the project were hosted on the
  Internet at my William and Mary webspace
  http//djvill.people.wm.edu
• This helped avoid the need to download lesson
  videos onto Learning Center computers
• In addition, this makes it easy to turn the
  experiment into a fully web-based one

34
Testing Sessions

• After students handed back consent forms, I used
  a random number generator to determine their
  testing group
• Students first loaded the URL http//djvill.people
  .wm.edu/_lesson.html
• This redirects the student to the page containing
  the first lesson video for that group
• At the end of the video, the student receives an
  assessment from a moderator

35
Testing Sessions, Contd

• Once the student completes the assessment, they
  are instructed to load http//djvill.people.wm.edu
  /_mid.html
• This redirects the student to their groups
  inter-lesson module
• Once that is complete, the student is instructed
  to load http//djvill.people.wm.edu/_lesson2.ht
  ml
• This redirects the student to their lesson 2

36
Testing Sessions, Contd

• Again, the student receives an assessment
  corresponding to lesson 2
• Once the student is done with the second
  assessment, they are instructed to load
  http//djvill.people.wm.edu/LPQ.html
• This page is a Linguistic Profile Questionnaire
  (on Google Docs)
• How many Math classes taken in college
• Where parents are from
• Childhood exposure to accents

37
Discussion Sessions

• In addition to testing sessions (collecting
  mostly quantitative data), I held discussion
  sessions to collect qualitative data
• Nine discussion sessions between October 17-20,
  60-90 minutes apiece
• Between 4 and 11 participants in each session
• Segregated by testing group
• Sandwiches!

38
Discussion Sessions, Contd

• Three parts to session
• Reviewed ground rules and human subjects
  protections
• Actual discussion itself
• Nuts and bolts of project
• Discussion questions
• For starters, how many classes, if any, have you
  taken with a foreign-born professor? Have any
  been Math classes?
• Did his or her accent ever hurt your
  understanding of the material?
• Do you talk about professors accents a lot with
  your friends?

39
Discussion Questions

• More discussion questions
• Have you ever dropped a class or even changed
  your academic plans because the professor had a
  foreign accent? What about other students you
  know?
• What do you think impedes communication between
  students and professors the most, regardless of
  accent?
• What do you wish professors (or even the College)
  would do to deal with the issue of the
  communication gap between undergraduates and
  mathematics professors?
• Do you feel that you gained anything from this
  process?
• Do you think that you are now at least somewhat
  better equipped to deal with issues of accent in
  your instructors?

40
Discussion Session Ending

• In discussing methodology, I finished by walking
  the groups through the Training module
• This led to the final question
• Do you think that programs such as these would be
  effective in dealing with the communication gap?

41
Results/Discussion
42
Quantitative Results

• I havent yet been able to do a full statistical
  analysis of quantitative data
• (cough, cough, Complex Analysis)
• However, preliminary analyses are not encouraging

Variable Macro-group N Mean StDev
Minimum Median Maximum IQR Improvement Bias
23 -1.09 6.99 -13.00 -2.00
10.00 12.00 Control 28 0.79
6.48 -15.00 0.50 12.00 9.50
Training 29 -0.79 6.24 -12.00 0.00
12.00 6.00
43
(No Transcript)
44
T-Tests

• Remember, our null is that µC µB µT (But we
  want µB lt µC lt µT)

Two-sample T for Improvement Macro-group N
Mean StDev SE Mean Bias 23 -1.09
6.99 1.5 Training 29 -0.79 6.24
1.2 Difference mu (Bias) - mu
(Training) Estimate for difference -0.29 95 CI
for difference (-4.05, 3.46) T-Test of
difference 0 (vs not ) T-Value -0.16
P-Value 0.875 DF 44
Two-sample T for Improvement Macro-group N
Mean StDev SE Mean Bias 23 -1.09
6.99 1.5 Control 28 0.79 6.48
1.2 Difference mu (Bias) - mu
(Control) Estimate for difference -1.87 95 CI
for difference (-5.71, 1.96) T-Test of
difference 0 (vs not ) T-Value -0.98
P-Value 0.330 DF 45
Two-sample T for Improvement Macro-group N
Mean StDev SE Mean Control 28 0.79
6.48 1.2 Training 29 -0.79 6.24
1.2 Difference mu (Control) - mu
(Training) Estimate for difference 1.58 95 CI
for difference (-1.80, 4.96) T-Test of
difference 0 (vs not ) T-Value 0.94
P-Value 0.353 DF 54
Fail to reject null hypothesis
45
ANOVAs
Source DF SS MS F
P Macro-group 2 54.3 27.1 0.63
0.534 Error 77 3299.3 42.8 Total
79 3353.5 S 6.546 R-Sq 1.62 R-Sq(adj)
0.00 Individual
95 CIs For Mean Based on
Pooled StDev Level N Mean StDev
------------------------------------ Bias
23 -1.087 6.986 (-------------------------) C
ontrol 28 0.786 6.483
(----------------------) Training 29 -0.793
6.241 (----------------------)
--------------------------------
---- -2.0
0.0 2.0 4.0 Pooled StDev 6.546
46
ANOVAs
Source DF SS MS F
P Lesson Order 5 1995.4 399.1 21.74
0.000 Error 74 1358.1 18.4 Total
79 3353.6 S 4.284 R-Sq 59.50
R-Sq(adj) 56.76
Individual 95 CIs For Mean Based on
Pooled StDev Level N Mean
StDev ------------------------------------ PC
-R2 12 -5.000 3.464 (--------) PC-Rel
16 1.563 4.647
(------) R2-PC 14 6.000 4.224
(--------) R2-Rel 10 5.200
3.736 (---------) Rel-PC
12 -0.667 4.755
(--------) Rel-R2 16 -7.438 4.442 (------)
----------------------
--------------
-5.0 0.0 5.0 10.0 Pooled StDev
4.284
47
Why these results?

• Major discrepancy in difficulty of tests
• Not enough participants for the number of testing
  groups
• Training module needs improvement

Individual 95 CIs For Mean Based on
Pooled StDev Level N
Mean StDev -------------------------------
---- PermComb 28 12.714 3.184
(--------) Relations 28 14.857 2.663
(--------) Root2 24
8.375 4.009 (---------)
-----------------------------------
7.5 10.0 12.5 15.0
48
In the end

• Thats just how it goes with research in the
  social sciences, especially with research on
  humans
• A project like this is often the first in a
  series of many
• Plenty of qualitative data gathered from
  Linguistic Profile Questionnaire and discussion
  sessions (about 6 ½ hours worth of participants
  responses)

49
Questions?

• Any questions or comments?

50
Homework

• As part of your homework, you will simulate being
  a participant in a testing session
• http//djvill.people.wm.edu/ltgroupname-nocapsgt_les
  son.html and follow the instructions from there
• When youre done Assessment 1, go to
  http//djvill.people.wm.edu/ltgroupname-nocapsgt_mid
  .html
• Finally, theres a discussion question for you to
  answer on the BlackBoard discussion board (not
  for extra credit)

51
Works Cited (in order of appearance)

• Mooney, C. J. (1989, January 25). Uncertainty is
  Rampant as Colleges Begin to Brace for Faculty
  Shortage Expected to Begin in 1990s. Chronicle
  of Higher Education, A14-A17.
• Bailey, K. M. (1984). The Foreign TA Problem.
  In K. Bailey, F. Pialorsi, J. Zukowski/Faust
  (Eds.), Foreign Teaching Assistants in U.S.
  Universities (3-15).
• Watts, M., Lynch, G. (1989). The Principles
  Course Revisited. The American Economic Review,
  79, 236-241. Retrieved from http//www.jstor.org/s
  table/1827763.
• Gass, S. M., Selinker, L. (2001). Second
  Language Acquisition An Introductory Course (2nd
  ed.). Mahwah, New Jersey Lawrence Erlbaum
  Associates.
• Derwing, T. (2003). What Do ESL Students Say
  About Their Accents? The Canadian Modern Language
  Review, 59, 547-566.

52
Works Cited (in order of appearance)

• Cassell, Edith Camilla (2007).  Understanding
  community linguistic diversity An ecological
  approach to examining language use patterns of
  international graduate students. Ph.D.
  dissertation, Purdue University, United States --
  Indiana. Retrieved June 1, 2009, from
  Dissertations Theses Full Text database.
  (Publication No. AAT 3287301).
• Plakans, B. (1997). Undergraduates' Experiences
  with and Attitudes toward International Teaching
  Assistants. TESOL Quarterly, 31. Retrieved April
  27, 2009, from http//www.jstor.org/stable/3587976
  .
• Davies, C. E., Tyler, A., Koran, J. J., Jr.
  (1989). Face-to-Face with English Speakers An
  Advanced Training Class for International
  Teaching Assistants. English for Specific
  Purposes, 8, 139-153.

53
Works Cited (in order of appearance)

• Tyler, A. (1992). Discourse Structure and the
  Perception of Incoherence in International
  Teaching Assistants' Spoken Discourse. TESOL
  Quarterly, 26, 713-726. Retrieved from
  http//www.jstor.org/stable/3586870.
• Young, R. (1989). Introduction. English for
  Specific Purposes, 8, 101-107.
• Halleck, G. B., Moder, C. L. (1995). Testing
  Language and Teaching Skills of International
  Teaching Assistants The Limits of Compensatory
  Strategies. TESOL Quarterly, 29, 733-758.
• Gravois, J. (2005, April 8). Teach Impediment.
  Chronicle of Higher Education, 51, A10. Retrieved
  from http//chronicle.com/free/v51/i31/31a01001.ht
  m.
• Byrd, P., Constantinides, J. (1992). The
  Language of Teaching Mathematics Implications
  for Training ITAs. TESOL Quarterly, 26, 163-167.
  Retrieved from http//www.jstor.org/stable/3587384
  .
• Gass, S., Varonis, E. (1984). The effect of
  familiarity on the comprehensibility of nonnative
  speech. Language learning, 34, 65-89.