Brain signatures of game play and language processing - PowerPoint PPT Presentation

Loading...

PPT – Brain signatures of game play and language processing PowerPoint presentation | free to download - id: 6bb27a-YTFjO



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Brain signatures of game play and language processing

Description:

GAME PLAY AND LANGUAGE PROCESSING Robert V. Reichle rreichle_at_niu.edu Department of Foreign Languages Center for the Interdisciplinary Study of Language and Literacy – PowerPoint PPT presentation

Number of Views:37
Avg rating:3.0/5.0
Slides: 48
Provided by: uicEdudept5
Learn more at: http://www.uic.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Brain signatures of game play and language processing


1
Brain signatures of game play and language
processing
  • Robert V. Reichle
  • rreichle_at_niu.edu
  • Department of Foreign Languages
  • Center for the Interdisciplinary Study of
    Language and Literacy
  • Northern Illinois University

2
L2 PROCESSING
  • For late L2 learners, L2 processing is difficult
  • Corresponding features or constructions in the
    native language do not ensure success in the L2
  • Variability is typical
  • Native-like attainment is not typical
  • Previously thought to relate to putative critical
    period for L2 acquisition

3
L2 PROCESSING
  • Among other explanations for divergence in L1/L2
    processing, Ullmans (2001, 2004)
    Declarative/Procedural model
  • Two domain-general memory systems
  • Declarative facts, associations between form and
    meaning, non-compositional words
  • Procedural rule-governed processes,
    morphosyntactic composition, cognitive-motor
    skills
  • Some L2 learners become more nativelike as they
    come to rely more on procedural memory in the L2

4
EVENT-RELATED POTENTIALS (ERPs)
Lexical/semantic processing N400
Morphosyntactic/grammatical processing LAN
Differential ERPs for L1 syntax/semantics
interpreted as evidence for DP model (Ullman,
2004)
  • (e.g., graphs from Osterhout et al., 2006, p. 204
    and Neville et al., 1991 Friederici, Steinhauer,
    Frisch, 1999 Friederici, 2002 Hahne
    Friederici, 1999, 2001 Hagoort Brown, 1999
    Kaan et al., 2000 Kutas Hillyard, 1980
    Osterhout, Bersick, McLaughlin, 1997 Osterhout
    Holcomb, 1992 Osterhout Mobley, 1995
    Steinhauer, Halter, Friederici, 1999
    Friederici, Pfeifer, Hahne, 1993 Kluender
    Kutas, 1993 Münte, Heinze, Matzke, Wieringa,
    Johannes, 1998)

5
EVENT-RELATED POTENTIALS (ERPs)
  • Proficiency effects on sensitivity to
    inflectional morphology and syntax
  • Artificial language learning with language
    training, N400 ??LAN/P600
  • (e.g., Friederici, Steinhauer, Pfeifer, 2002
    Morgan-Short, Sanz, Steinhauer, Ullman, 2010
    Morgan-Short, Steinhauer, Sanz, Ullman, 2012
    Morgan-Short, Finger, Grey, Ullman, 2012)
  • Real language learning with increasing
    proficiency, N400 ??LAN/P600
  • (e.g., Bowden et al., 2007 Hahne et al., 2006
    Osterhout et al., 2006 Rossi et al., 2006
    Steinhauer et al., 2006 Gillon Dowens, Vergara,
    Barber, Carreiras, 2010 McLaughlin, Osterhout,
    Kim, 2004 McLaughlin, Tanner, Pitkänen,
    Frenck-Mestre, Inoue, Valentine, Osterhout,
    2010 Tanner, Osterhout, Herschensohn, 2009
    Ojima, Nakata, Kakigi, 2005)

6
EVENT-RELATED POTENTIALS (ERPs)
  • Proficiency effects on sensitivity to
    inflectional morphology and syntax
  • Artificial language learning with language
    training, N400 ??LAN/P600
  • (e.g., Friederici, Steinhauer, Pfeifer, 2002
    Morgan-Short, Sanz, Steinhauer, Ullman, 2010
    Morgan-Short, Steinhauer, Sanz, Ullman, 2012
    Morgan-Short, Finger, Grey, Ullman, 2012)
  • Real language learning with increasing
    proficiency, N400 ??LAN/P600
  • (e.g., Bowden et al., 2007 Hahne et al., 2006
    Osterhout et al., 2006 Rossi et al., 2006
    Steinhauer et al., 2006 Gillon Dowens, Vergara,
    Barber, Carreiras, 2010 McLaughlin, Osterhout,
    Kim, 2004 McLaughlin, Tanner, Pitkänen,
    Frenck-Mestre, Inoue, Valentine, Osterhout,
    2010 Tanner, Osterhout, Herschensohn, 2009
    Ojima, Nakata, Kakigi, 2005)

7
brocanto training game
  • Artificial language (BROCANTO)
  • Fulfills syntactic requirements of natural
    language
  • Contains morphosyntactic features that differ
    from L1 of participants
  • Participants trained in lexicon of artificial
    language
  • One group trained to proficiency in BROCANTO via
    a two-player computer board game
  • Played against each other in pairs
  • Orally expressed all game moves in BROCANTO
  • Received feedback on valid game moves and
    grammatical BROCANTO utterances
  • The structure of the language not dependent on
    the structure of the game
  • (Friederici, Steinhauer, Pfeifer, 2002 for
    similar BROCANTO2 design, see also Morgan-Short,
    Sanz, Steinhauer, Ullman, 2010 Morgan-Short,
    Steinhauer, Sanz, Ullman, 2012 Morgan-Short,
    Finger, Grey, Ullman, 2012)

8
brocanto training game
aaf trul prez nöri aak füne plox the trul-piece
captures horizontally the round plox-piece aaf
trul prez nöri rix füne ploxthe trul-piece
captures horizontally buy round plox-piece
  • ERPs
  • ERP data collection using syntactically
    felicitous and anomalous items took place after
    participants in the training group reached a 95
    accuracy threshold.
  • Training group had biphasic response to
    violations that resembled LAN/P600 of L1
    processing

9
brocanto training game
  • Why was this a successful training paradigm?

10
brocanto training game
  • Why was this a successful training paradigm?
  • Gee (2003) active, critical learning
  • cooperation
  • competition
  • a requirement of successful learning in order to
    progress
  • feedback
  • meta-awareness of gameplay mechanics

11
brocanto training game
  • Why was this a successful training paradigm?
  • Declarative/Procedural model
  • nativelike syntactic processing should be handled
    by procedural memory
  • learning a cognitive-motor skill (like the rules
    of the training game) should also be handled by
    this system
  • Did these two types of learning engage similar
    cognitive processes? Does words and rules have
    an equivalent sprites and rules (Reichle,
    2012)?
  • P600 not necessarily fully language-specific
    also observed for structural revision in music
    (e.g. Patel et al., 1998)

12
Research questions
  • Does the execution of gameplay skills engage the
    same processes as syntactic processing?
  • Specifically, examining violations of syntax and
    violations of expected gameplay behavior.
  • If the same processes, then could aid learning,
    but could also present confound to interpretation
    of results.
  • Does processing linguistic stimuli in the context
    of a game lead to a task effect compared to
    linguistic stimuli in isolation?

13
Hypotheses
  • Violations of expected gameplay behavior will
    elicit LAN and P600 effects comparable to those
    seen for violations of morphosyntax.
  • Processing violations of morphosyntax in the
    context of a game causes a task effect compared
    to violations of morphosyntax in isolation.

14
ERP Experiment 1
  • 23 participants (5 rejected due to artifacts)
  • native speakers of English
  • pre-screened for knowledge of rules of movement
    in chess
  • EEG data were recorded from nine sites (F3, Fz,
    F4, C3, Cz, C4, P3, Pz, P4)Four conditions
  • A Control. Legal moves followed by grammatical
    sentences
  • B Morphosyntax violation. Legal moves followed
    by sentences with a morphosyntax violation
  • C Gameplay mismatch. A chess move that is legal
    for a piece other than the one shown in the
    image, followed by a grammatical sentence
  • D Gameplay violation. A chess move that is never
    legal for any piece, followed by a grammatical
    sentence

15




(No Transcript)
16




(No Transcript)
17




A The white queen takes the black pawn.
18




B The white queen taking the black pawn.
19




(No Transcript)
20




(No Transcript)
21




C The white queen takes the black pawn.
22




(No Transcript)
23




(No Transcript)
24




D The white queen takes the black pawn.
25
ERP Experiment 1
  • Stimulus presentation
  • each image onscreen for 1 sec
  • 600ms pause
  • word-by-word presentation 290 ms plus 30 ms per
    letter, up to a maximum of 590 ms, with a 150 ms
    interval between words
  • Analysis
  • At fourth word (takes/taking)
  • Mean amplitude data in 200-500ms and 500-800ms
    windows
  • Repeated-measures ANOVAs 4 (Condition) x 3
    (Anteriority) x 3 (Laterality)

26
F3
Fz
F4
C3
Cz
C4
P3
Pz
P4
27
(No Transcript)
28
F3
Fz
F4
Omnibus 200-500ms Cond Lat, p lt .05
C3
Cz
C4
P3
Pz
P4
29
F3
Fz
F4
C3
Cz
C4
P3
Pz
P4
30
F3
Fz
F4
C3
Cz
C4
P3
Pz
P4
31
F3
Fz
F4
C3
Cz
C4
P3
Pz
P4
32
Results
  • P600 effect for morphosyntax violations
  • No effect for game rule mismatch
  • Game rule violation
  • Negativity effect, strongest in left and mid
    central and frontal sites
  • Interpreted as possible variation of LAN
  • Extended into later time window than typical
    biphasic LAN
  • But, not biphasic no P600 effect followed

33
Discussion
  • Hypothesis Violations of expected gameplay
    behavior will elicit LAN and P600 effects
    comparable to those seen for violations of
    morphosyntax.
  • Not fully supported no P600. Consistent with
    generally language-specific nature of P600.
  • Partially supported by presence of LAN. But work
    still remains on reproducibility and functional
    significance of LANs (e.g. Steinhauer, Drury).

34
Discussion
  • Implications
  • Possibility remains that processing gameplay
    rules engages similar cognitive processes as
    processing morphosyntax (LAN)
  • No evidence that gameplay elicits P600
  • Doubtful that P600s seen in BROCANTO studies were
    indices of gameplay-related processing

35
Hypotheses
  • Violations of expected gameplay behavior will
    elicit LAN and P600 effects comparable to those
    seen for violations of morphosyntax.
  • Processing violations of morphosyntax in the
    context of a game causes a task effect compared
    to violations of morphosyntax in isolation.

36
ERP Experiment 2
  • Show morphosyntactic anomalies within and without
    game context
  • 2 groups
  • Image group Saw chess move after reading
    sentence
  • NoImage group Did not see chess moves
  • 12 participants (6 per group)
  • native speakers of English
  • pre-screened for knowledge of rules of movement
    in chess
  • EEG data were recorded from nine sites (F3, Fz,
    F4, C3, Cz, C4, P3, Pz, P4)

37
ERP Experiment 2
  • 2 Conditions
  • Control The white queen takes the black pawn.
  • Morphosyntax violation The white queen taking
    the black pawn.
  • 2 fillers
  • Progressive filler The queen is taking the black
    pawn.
  • Game rule violation The black queen takes the
    white pawn.

38
ERP Experiment 2
  • Stimulus presentation
  • word-by-word presentation 290 ms plus 30 ms per
    letter, up to a maximum of 590 ms, with a 150 ms
    interval between words
  • for Image group, presentation of game images 1
    sec after final word
  • Analysis
  • At fourth word (takes/taking)
  • Mean amplitude data in 200-500ms and 500-800ms
    windows
  • Repeated-measures ANOVAs 2 (Condition) x 3
    (Anteriority) x 3 (Laterality)

39
NoImage group
40
Image group
41
Image group
42
Image group
43
Results
  • Trend toward between-group difference (task
    effect)
  • Trend toward P600 for Image group
  • Data collection still in progress more
    participants needed

44
Discussion
  • Hypothesis Processing violations of morphosyntax
    in the context of a game causes a task effect
    compared to violations of morphosyntax in
    isolation.
  • Preliminary results trend toward support of
    hypothesis.
  • Attributable to motivation in the broadest
    sense. Additional studies needed to separate out
    Gees factors (competition, meta-awareness, etc.)
    from simpler explanations (i.e. more attentive
    due to presence of images)

45
CONCLUSIONS
  • Evidence of an LAN for the processing of gameplay
    violations suggests gameplay may engage cognitive
    processes also used in language processing.
  • Lack of P600 for gameplay processing indicates
    that L2 studies using game training paradigms are
    not likely to confound indices of gameplay with
    indices of morphosyntactic processing.
  • Preliminary data suggest trend toward
    game-related task effect, suggesting increased
    motivation or attention within game context.
  • Additional studies needed to further tease apart
    variables.

46
Acknowledgments
  • This research has been supported by the NIU
    Division of Research and Graduate Studies, the
    NIU Center for the Interdisciplinary Study of
    Language and Literacy, and the NIU Center for the
    Study of Family Violence and Sexual Assault.
  • The author thanks Cody Happ and Regina Hiraoka
    for their assistance with materials creation and
    data collection, and all participants for their
    cooperation.

47
Thank you
About PowerShow.com