The Temporal Dynamics of Learning: The UCSD Dynamic Learning Center

1
(No Transcript)
2
The Temporal Dynamics of Learning Center
(TDLC)Better Learning by Advancing the Science
of Time
Garrison W. Cottrell, PI
3
At each point in TIME, the brain changes to best
meet the changing demands of the environment.
4
Time A crucial variable in learning

• Understanding how the brain learns is important
  for understanding how students learn in the
  classroom.
• And how the brain learns depends crucially on
  time and timing.
• Hence, our purpose

5
Our Purpose

• No less than to develop a new Science of the
  Temporal Dynamics of Learning
• Change educational practice based on sound
  science.
• To do this by creating a new collaborative
  research structure, the network of networks, to
  transform the practice of science

6
A science of the temporal dynamics of learning?
What does that mean?

• We believe it means achieving an integrated
  understanding of the role of time and timing in
  learning, across multiple
• temporal and spatial scales,
• brain systems, and
• social systems.

7
What are the components of a science of the
temporal dynamics of learning?

• A complete science of temporal dynamics of
  learning would include how we learn
• To adapt to the input (the temporal dynamics of
  the world)
• To adapt our outputs (the temporal dynamics of
  action)
• And how the brain in between these accomplishes
  this (the temporal dynamics of the brain)
• But a science requires formalism (the theory of
  temporal learning)
• These, mutatis mutandis, are our four research
  initiatives.

8
The Four TDLC Research Initiatives

• 1. TEMPORAL DYNAMICS OF THE WORLD How is
  temporal information about the world learned and
  how do the temporal dynamics of the world
  influence learning?
• 2. TEMPORAL DYNAMICS OF THE BRAIN What are the
  temporal dynamics of brain cells, brain systems,
  and behavior? How do these dynamics change with
  learning, and how do they influence learning?
• 3. TEMPORAL DYNAMICS OF MOVEMENT AND EXPLORATION
  What are the temporal structures for body
  movements and sampling the environment and how
  are they learned?
• 4. TEMPORAL DYNAMICS OF LEARNING What mechanisms
  determine the time course of learning itself and
  what general principles explain the dynamics of
  learning across multiple scales and domains?

9
TDLC Initiatives
10
Strands capture important subcomponentsof
initiatives
Strand 1.1 Learning of Temporal Patterns. How
do organisms recognize, learn, and remember
temporal patterns of sensory stimuli, including
sequences? How are these represented in the
brain, and how do they guide perception and
behavior?
Projects represent tractable research questions
within strands
1.1.1 Cross-modal comparison of learning simple
sensory patterns and sequences
1.1.2 Neural encoding of sequences
as compressed units during learning
1.1.3 Learning to recognize actions from
sequences of visual events
SMN, IMS
IMS, SMN
PEN
Feldman, Chiba, Harris, de Sa, Sereno
Chiba, Buszaki, Harris, OReilly, Tallal, Bell.
Curran, Sheinberg,Tarr
11
Why does time matter?

• Time matters for processing (input dynamics)
• Rapid Auditory Processing (RAP) thresholds
  predict later language impairments
• Time matters for learning (brain dynamics)
• The spacing of study episodes predicts later test
  scores
• Precise spike timing is necessary for LTP
• Time matters for remembering (brain dynamics)
• Consolidation during sleep is necessary for
  storage
• Time matters for teaching (output dynamics)
• Positive feedback that comes too late is not
  effective

12
Time matters for teaching

• RUBIs more realistic head movement timing
  appears to have induced pointing behavior.

13
Time matters for remembering

• Gamma band power increases in a particular layer
  of hippocampus - CA1 stratum radiatum - when the
  rat is at a choice point in this spatial
  alternation task, showing the synchrony that
  co-occurs with recall of the previous choice.
  (video by Sean Montgomery, Buzsaki Lab)

14
Time matters for learning the dynamics of the
world

• This rat has learned a sequence and rapidly pokes
  the right ports in sequence - then he shows why
  he is a PhD of rat-dom - he can poke in sequence
  when all lights are on, meaning he has internally
  generated the sequence (Chiba lab pilot data).

15
Time matters for processing

• We need to direct our attention towards
  interesting things - in this video, a
  probabilistic model of salience is applied to
  video - the interesting points (hot colors) are
  the ones that are most surprising over time - the
  ones with the highest self-information.

16
Time matters for processing
These wave forms are identical except for the
artificially inserted gap! Yet we all hear a /t/
inserted
17
Time matters for learning
This baby has learned that exciting things go on
to her left when the two tones are different --
at a 70ms boop-beep interval
18
Time matters for learning

• The ability that baby showed - being able to
  detect differences in tones 70 ms apart - is
  crucial for learning language.
• Rapid auditory processing (RAP) ability at 6
  months of age accurately predicts language
  impairment (91 accuracy) at 3 years of age.
  Benasich Tallal (2002) Behav Brain Res.
• Remarkably, training children to make these
  distinctions with the FastForWord program can
  improve reading, and measurably changes their
  fMRI profile.
• This training is grounded in basic research on
  neural plasticity from animal models.
• The FastForWord experience is an inspiration for
  our Center - science translated to education.

19
Time matters for learningat this short time
scale

• Abnormalities in temporal processing at the 20-50
  ms time scale lead to poor phonemic perception
• Which leads to a cascade of detrimental learning
  effects on ALL longer time scales Poor language
  skills.
• By concentrating on remediating the basic
  temporal processing deficit, we can improve
  higher-level performance.

20
And at Many Time Scales
0 100 msec 1 sec 10 sec minutes hours da
ys months

• 5-20 ms Synaptic integration, spike-timing
  dependent plasticity (STDP), speech
• 50-200 ms Event and motor sequence
  perception/production
• 200-500 ms Fixation duration, attentional
  blinks, saccades, attention shifts, timing of
  social interactions
• 0.5-2 s Perception, motor rhythms, emotional
  expressions
• 2-50 s repeated exposure and training effects,
  reward signals
• minutes, hours, days, months, years Spacing
  effects, memory consolidation, analytic/holistic
  changes in expertise
• 10 years The time span of a Center!

21
Meeting This Challenge Requires

• Well-organized collaborations between scientists
  from multiple disciplines.
• Large-scale data infrastructure that makes data
  persistent and accessible to many scientists.
• Theoretical models capable of spanning time
  scales.

22
Seems like a tall order.How will we do it?
A new kind of Center The Network-of-Networks
research structure
23
The Network-of-Networks Solution
Mathematics Physics Machine Learning Robotics Comp
uter Science Computational Neuroscience Neuroscien
ce Cognitive Science Linguistics Neuropsychology C
ognitive Psychology Developmental
Psychology Learning Theory Education
UC San Diego Rutgers University Vanderbilt
University UC Berkeley University of Colorado The
Salk Institute Queensland University Victoria
University Brown University Carnegie-Melon
University Yale University San Diego State
University
SensoriMotor Network
Social Interaction Network
Bridge Members
Perceptual Expertise Network
Interacting Memory Systems
24
Why these four?

• Sensorimotor Learning mediates our interactions
  with the world - input and output
• Interacting Memory Systems maintain continuity
  across time - brain dynamics
• Social Interaction is necessary for teaching -
  acting and interacting
• Perceptual Expertise represents exquisite skills
  we use for making fine-grained discriminations,
  and is acquired through years of experience - the
  time scale of learning to read, or learning a
  trade

25
The Network of Networks a new collaborative
research structure

• Research networks focus on common issues (memory,
  sensation and action, etc.)
• Researchers can more easily communicate
• The manageable size allows the formation of
  scientific communities.
• They become the research engines of the
  initiatives
• The network-of-network organization encourages
  bridges between levels.

26
A proof of concept The Perceptual Expertise
Network (PEN)
27
The Perceptual Expertise Network (PEN)
A group of 10 researchers from Psychology,
Neuroscience, Neuropsychology and Computer
Science started in 2001 The goal of our network
was to understand the learning mechanisms and
representations underlying visual expertise Our
first meetings were used to establish a common
vocabulary, and to synchronize our research
around a set of common questions, rather than
focus on our techniques
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
Initiative 8 Spreading the news

• A purpose of our Center is to demonstrate that
  this research structure - the network of networks
  - is an effective means of organizing a large
  group of people around common goals.
• We will have demonstrated this if we can
  generalize the PEN model to a larger scale with
  PEN-size components
• I will show that this is working!
• But first, an introduction to the other
  networks..

32
Sensorimotor Network
Network Goal To understand the temporal
dynamics of sensory perceptual learning and motor
learning, from biological learning rules at
synapses to human behavior.
Precise timing is a critical feature of natural
sensory stimuli, and of movements, and is
actively learned.
We will study the synaptic mechanisms, neural
systems, and computational strategies used by the
brain to implement temporal features of sensory
perceptual and motor learning.
33
Interacting Memory Systems Network
Network Goals How does a learner abstract the
temporal structure of the environment ? How
does the brain arbitrate control across the
multiple memory systems involved in
learning? How does the timing of training and
testing regimes affect the rate of learning and
long-term retention?
We will study brain synaptic mechanisms,
electrophysiological properties, computational
strategies, and neuroimaging correlates, in
order to explain the temporal features that
underlie efficient behavioral performance.
34
Perceptual Expertise Network
Network Goals To relate the short-term dynamics
of perception and perceptual decisions to the
long-term dynamics of expert learning, and to
exploit that understanding to significantly
improve learning.
We will take a multidisciplinary approach to
studying how perceptual expertise changes as a
function of time and across experience.
35
Social Interaction Network
Network Goals
What are the effects of timing on social
interactions and social learning? How does the
brain organize behavior in real time? What are
the neural mechanisms engaged during learning in
a social context?
We will study active learning and teaching as
well as real-time social interaction.
36
The Network-of-Networks Solution
Mathematics Physics Machine Learning Robotics Comp
uter Science Computational Neuroscience Neuroscien
ce Cognitive Science Linguistics Neuropsychology C
ognitive Psychology Developmental
Psychology Learning Theory Education
UC San Diego Rutgers University Vanderbilt
University UC Berkeley University of Colorado The
Salk Institute Queensland University Victoria
University Brown University Carnegie-Melon
University Yale University San Diego State
University
SensoriMotor Network
Social Interaction Network
Bridge Members
Perceptual Expertise Network
Interacting Memory Systems
37
Interweaving Initiatives Networks

• Each research network has a coherent focus the
  initiatives weave them together
• The goal of network meetings is to synchronize
  research within networks
• Initiative meetings - here, at the All-Hands
  Meeting - will synchronize research between
  networks.
• This creates the coherence - the synergy - the je
  ne sais quoi - that makes the whole greater than
  the sum of its parts

38
Networks and Initiatives The weft and the warp
of the Center
39
How networks work Preamble

• At the first PEN meeting, Jim Tanaka presented
  his work on how perceptual expertise could be
  trained
• Bob Schultz presented his work showing how
  children with autism had face processing deficits
• They began to collaborate on a project to train
  children with autism to process faces
• The Lets Face It! project was born

40
How networks work Preamble

• Bob and Jim had never worked together before
• In fact, there was no plan for them to be working
  together in our proposal
• But a synergy developed, resulting in a system
  that has now been shown to improve face
  processing skills in Autism

41
Synergy

• This demonstrates the power of a network - new
  interactions that wouldnt have happened without
  the network.
• I will now tell two stories that demonstrate how
  this experience has been already been replicated
  in our Center
• 1. RUBI meets Lets Face It!
• 2. The Neurogenesis Project

42
Story 1 RUBI meets LFI!

• Javier Movellan and Marni Bartlett have been
  working for 15 years on recognizing facial
  expressions
• Now they have a system that works in real time
  with video
• Bob, Jim, Javier and Marni attended our first
  All-Hands meeting in January

43
RUBI Meets LFI!

• Jim presented Lets Face It!
• Marni presented CERT - their Computer Expression
  Recognition Toolbbox
• Javier highlighted RUBIs interactive potential
• Light bulbs went off in a few heads!
• Could the real-time face recognition capabilities
  of CERT, RUBI the social robots interactive
  capability, and Lets Face It! be combined to
  create a new, exciting version of the program?

44
RUBI Meets LFI!

• Imagine a Lets Face It! that
• Displays facial expressions dynamically
• Responds to facial expressions
• Trains facial expressions by giving real-time
  feedback

• This project is now partially funded from our
  Year 2 budget.
• This project would not have happened without the
  Center!

45
Story 2 Neurogenesis spreads through the IMS
network

• Pre-TDLC, Janet Wiles, Brad Aimone, Jeff Elman
  and Rusty Gage were developing a computational
  model of the Dentate Gyrus (DG) in the
  hippocampus
• DG is the site of daily neurogenesis in the
  Hippocampus
• Prior models assumed these new neurons were there
  to orthogonalize new memories - to encode the
  contexts of those memories
• But the models indicated that the sparse
  connectivity was sufficient to orthogonalize
  memories - so what is neurogenesis for??

46
Story 2 Neurogenesis spreads through the IMS
network

• New neurons are born every day, but they mature
  slowly.
• Hypothesis Perhaps they are there to make
  temporal linkages between experiences.
• Who could they turn to to help with training rats
  on tasks requiring temporal associations?
• From being in the same network, they realized
  that Andreas methods for developing animal
  models could answer the question.

47
Story 2 Neurogenesis spreads through the IMS
network

• New neuroscience student Lara Rangel became a
  bridge student between the Gage and Chiba labs
  focused on learning the techniques required from
  both labs to do the project.
• The project became an un-funded Center project of
  the IMS network.
• They are now funded by the McDonnell Foundation.
• This would not have happened without the Center.

48
Where will we be in Five Years?Deliverables

• A Science of the Temporal Dynamics of Learning.
• We need to understand the temporal dynamics of
  learning in order to manipulate it towards better
  learning outcomes.
• We will provide scientifically sound principles
  for educational practice.

49
DeliverablesNew Tools for Education

• With our partner Scientific Learning Corporation,
  we have the expertise to move our research into
  the classroom, as they have done with the
  FastForWord platform.
• We are developing the science needed to decide
  how to time review classes for maximum retention.
• With our partner Jensen Learning Corporation, we
  are providing free continuing education to
  teachers from our partner schools.
• We are developing Lets Face It!, a program for
  teaching kids with social deficits to read faces.
• We are advancing research on a low-cost, fully
  autonomous version of RUBI, our educational robot
  suitable for preschools.

50
DeliverablesNew Tools for Research

• The network of networks structure
• Scalability
• Collaboration
• Shared facilities
• Bridge postdocs
• Shared philosophy

51
DeliverablesNew Tools for Research

• A Data Sharing Facility to provide support for
  management of joint research data, and a
  state-of-the-art Data GRID at the NSF-sponsored
  San Diego Supercomputer Center to allow data
  mining, temporal alignment of imaging, spike data
  and behavioral data from multiple sites.
• This will become a resource for all Centers

52
DeliverablesNew Tools for Research

• A Motion capture Facility to provide
  state-of-the-art equipment and software for
  simultaneous capture and analysis of multiple
  fine-scale temporal actions and interactions.

Hand movement
Facial Expression
Body movement
53
DeliverablesNew Tools for Research

• Due to funding constraints, the TDLC Motion
  Capture Facility has morphed into the Motion
  Capture/Brain Dynamics Facility which now
  provides
• state-of-the-art simultaneous real-time EEG and
  motion capture
• analysis tools for aligning movements and brain
  waves during reaching, choice tasks, and
  exploration tasks

54
If you build it, they will comeThere are now 12
MoCap collaborative projects!
55
DeliverablesNew Tools for Outreach

• The Science Network Web TV Channel to broadcast
  Town Meetings between scientists, policy
  makers, parents, teachers and the public and help
  promote a better understanding of science in the
  US and the world.
• Our first Town Hall Meeting on Education
  Neuroscience will be held Monday, March 3rd, with
  state school superintendents, policymakers,
  experts, teachers, and parents in attendance.

56
DeliverablesMore Diverse Scholars

• UCSDs Preuss School 100 school lunch program
  kids, 96 go to college These students intern in
  Center Labs to see real science at work (my
  student went to Amherst!)
• Reach for Tomorrow Inner city kids come to
  campus and are inspired to attend college
• Faculty Partners Program Building pipelines to
  minority institutions

57
Summary

• The time is right to study time!
• We have an excellent team of world-class
  researchers.
• We have a well-designed and flexible
  organizational structure.
• We have a unity of vision.

58
Summary

• We are prepared to train a diverse group of
  scientific leaders for the 21st century.
• We are committed to making our research relevant
  to the classroom.

59

• Life passes in milliseconds,
• but what we learn
• in those milliseconds
• changes us for life.

60

• happy new year
• May 2008 bring you more -- TIME.
• Obviously not jail time!
• Prime time.
• Im alive time.
• Im just fine thank you time.
• Im really here time.
• Im dear (even to me) time.
• More groovin time.
• Less provin time.
• More aah thats yummy time.
• Less oy my tummy time.
• More swayin and movin time.
• Less sittin in one and the same place time.

More croonin time. Less off key and out of
sorts time. More swoonin time. Less behoovin
time. Occasional moonin time. Less wheres my
time gone to time. More Im in charge of my own
time time. Less serving time and more deserving
time. More rhyme time or non-rhyme time Just more
time time. And, if not any of these types of
time then at least the perception of more time
time. -Marta Kutas