Dynamic Assessment in L2 Development: Bringing the Past into the Future James P. Lantolf, Greer Professor in Language Acquisition

1
Dynamic Assessment in L2 Development Bringing
the Past into the FutureJames P. Lantolf, Greer
Professor in Language Acquisition Applied
Linguistics The Pennsylvania State University

• Research funded in part by a grant from the
  United States Department of Education Grant (CFDA
  84.229, P229A020010-03). However, the contents do
  not necessarily represent the policy of the
  Department of Education, and one should not
  assume endorsement by the Federal Government. It
  was also partly funded by a Gil Watz Fellowship
  from the Center for Language Acquisition at The
  Pennsylvania State University. Co-author on
  larger project MATT POEHNER

2
Dialectics of Human Consciousness

• Dialectical logic contradictory unity of
  opposites
• Mind - body NOT two different and originally
  contrary objects, but only one single object and
  this is the thinking body of living real human
• Possible to consider this being from two
  different or even opposite points of view
• Thinking body does not consist of two Cartesian
  halves.
• Thought is a property, a mode of existence, of
  the body -- ie.its spatial configuration and
  position among other bodies

3
Spinoza

• There are not two different and originally
  contrary objects of investigationbody and
  thoughtbut only one single object, which is the
  thinking body of living, real man sic only
  considered from two different and even opposing
  aspects or points of view.

4
Tool Culture Mediation

Human Entity

• Person tool

Vygotskys insight !
Person artifacts
Material World
Culture
5
Mediated Mind

• What is first social becomes psychological

Concepts, Artifacts, Activities
Person
World
6
Mediation is a Functional System

• Mediational Means do not operate independently of
  each other but as organic functional systems.
• Formal Educationa leading activity of many
  cultures integrates symbolic and physical
  artifacts (books, paper, pencil, numbers, charts,
  language) aimed at development of conceptual
  understanding of the world.

7
Person-Environment Relationship

• Major impediment to the theoretical and practical
  study of development
• environment not outside the child
• understanding of environment that developed in
  biology as in evolution of animal species must
  not be transferred to child development.
• Environment is not a factor in development -- it
  is the source of development NB as distinct
  from biological maturation

8
Vygotsky and the Mind - Body

• Downward Reduction to brain -- vulgar
  materialism innately specified knowledge
• Upward Reduction to world -- behaviorism or
  social constructionism
• Vygotskys solution -- find a third concept
  MEDIATION to (re)unify and synthesize mind-body

9
Communication Cognition

• Communicative Conception
• Language thought are independent
• Language serves to transmit thought
• Cognitive Conception
• Humans think IN natural language natural
  language sentences are vehicles of thought
• Supra-Communicative Conception
• Public language (social communication) is
  available as a tool for mediating thinking

10
Traditional Assessment (TA)

• The examiner presents items, one at a time or all
  at once, and each examinee is asked to respond to
  these items successively, without feedback or
  intervention until the test is over, usually in
  the form of a score or set of scores.

11
Dynamic Assessment

• Takes into account results of intervention. The
  examiner teaches the examinee how to perform
  better on individual items or on the test as a
  whole.
• Final score may be a learning score representing
  the difference between pretest (before learning)
  and posttest (after learning) scores, or it may
  be the score on the posttest considered alone.
• Sternberg, R. J. and E. L. Grigorenko. (2002).
  Dynamic Testing. The Nature and Measurement of
  Learning Potential. Cambridge CUPress.

12
DA the ZPD

• determining the actual level of development not
  only does not cover the whole picture of
  development, but very frequently encompasses only
  an insignificant italics added part of it.
• Vygotsky, L. S. (1998). The Problem of Age. In
  The Collected Works of L. S. Vygotsky. Vol. 5.
  Child Psychology. R. W. Rieber (Ed.). New York
  Plenum.

13
Importance of Assistance

• responsiveness to assistance indispensable for
  understanding cognitive ability
• provides insight into the persons future
  development
• what individual is able to do one day with
  assistance, s/he is able to do tomorrow alone
• potential development varies independently of
  actual development
• latter cannot be used to predict former

14
ZPD Pull-to-Sit Consensual Frame

• Mother pulls infant with sufficient force that
  child cannot resist or provide compensatory
  effort
• No sense of self--absorbed in background
• Cant detect relationship between self action
• Mother infant jointing exert dynamic forces to
  create action
• Two-person action system that is
  co- regulated
• (Fogel 1993)

15
Early Formulation of DA

• Imagine that we have examined two children and
  have determined that the mental age of both is
  seven years. This means that both children solve
  tasks accessible to seven-year-olds. However,
  when we attempt to push these children further in
  carrying out the tests, there turns out to be an
  essential difference between them. With the help
  of leading questions, examples, and
  demonstrations, one of them easily solves test
  items taken from two years above the childs
  level of actual development. The other solves
  test items that are only a half-year above, his
  or her level of actual development.

16
Early DA (continued)

• From the point of view of their independent
  activity they are equivalent, but from the point
  of view of their immediate potential development
  they are sharply different. That which the child
  turns out to be able to do with the help of an
  adult points us toward the zone of the childs
  proximal development. This means that with the
  help of this method, we can take stock not only
  of todays completed process of development, not
  only the cycles that are already concluded and
  done, not only the processes of maturation that
  are completed we can also take stock of
  processes that are now in the state of coming
  into being, that are only ripening, or only
  developing.

17
DA TA Understanding the Future

• Vygotskys theorizing in the ZPD is predicated
  upon a radically different understanding of the
  future from that which informs SA

18
Three Models of the Future (Valsiner 2001)

• Atemporal humans do not develop but mature
• genetics (e.g. innatism)
• environment (e.g., behaviorism)
• Past-to-present history of organism leads to
  present state
• Development sequence of stages on way to final
  stage
• stages cannot be skipped Piaget, Krashen,
  Pienemann
• future predicted post factum already has become
  present
• Present-to-future (the future-in-the-making)
  emergence of novelty
• chart development while it is emerging
• Others participate actively in developmental
  process
• Actual development brings past into contact with
  future

19
DAThe Future in the Making

• TA follows performance to the point of
  failure in independent functioning, whereas DA
  leads to the point of achievement of success in
  joint or shared activity
• Ability not stable trait of individuals but
  malleable feature dependent on activities in
  which individuals participate
• Test performance not complete without persons
  response to assistance
• Persons potential to develop (i.e., his/her
  future) depends on ZPD.

20
Methodological TA vs. DA

• FOCUS
• TA product of past development
• DA foregrounds future development
• EXAMINER-EXAMINEE RELATIONSHIP
• TA Examiner is neutral disinterested
  (minimize measurement error)
• DA atmosphere of teaching and helping
• FEEDBACK
• TA none during assessment
• DA mediated assistance
• (Sternberg Grigorenko 2002)

21
Types of DA

• Interventionist
• Sandwich pretest-mediation-posttest
• Individual
• Group
• Layer Cake intervention during assessment
• Hints provided from a pre-established menu
• Individual
• Group
• Interactionist mediation is emergent in
  interaction between examiner examinee

22
Interventionist DALayer Cake

• Lerntest or Leipzig Learning Test (LLT)
• Jurgen Guthke colleagues
• Graduated Prompts
• Ann Brown colleagues

23
LLT L2 Aptitude (Guthke, Heinrich Caruso 1986)
? blo
_ ski
_ ? ski gadu la
? _ ski gadu vep
? _ ?
24
Graduated Hints in LLT

• Thats not correct. Please, think about it once
  again.
• Thats not correct. Think about which rows are
  most relevant to the one you are trying to
  complete.
• Thats not correct. Lets look at rows three
  and four.
• Thats not correct. Lets look at rows three
  and four and focus on the differences in both the
  positions of the objects and the words.
• Thats not correct. The correct pattern is gadu
  ski la because we see that gadu represents the
  triangle, ski represents the square, and la,
  which indicates the objects relative horizontal
  positioning, should be the final element in the
  clause, as can be seen in rows three and four.

25
Computerize LLT (Guthke Beckmann 2000)

• Administered in Group Format
• Adaptive examinees can skip around in programs
  until they either cannot produce the correct
  answer or require assistance to do so, at which
  point they are detoured to earlier items that
  were skipped. Detour sensitive to
• Type of help required
• Source of the problem resulting in the incorrect
  solution.
• If response shows understanding of some aspects
  of language (e.g., word order) but not others
  (e.g., agreement morphology), learner is led into
  a detour that focuses on the problematic
  dimension and then is led back into the more
  complex problems integrating syntax and
  morphology.

26
Interactionist DA

• Follows Vygotskys preference for qualitative
  assessment of psychological processes and
  dynamics of their development (Minick 1987 119)
• Vygotsky (1998 204) insisted that we must not
  measure the child, we must interpret the child
  and this can only be achieved through interaction
  and cooperation with the child.

27
Vygotsky on Subjectivity in Science

• Task of Research Methodology not just to
  measure, but to see, think associate.
• Fear of interpenetration of subjective factors in
  assessment research unfounded
• Research results cannot be fully achieved through
  purely mechanical arithemetic methods
• Without subjective reevaluation (without thought
  interpretation) deciphering results
  evaluation of data is not scientific research

28
Reuven Feuerstein

• Traditional examiner/examinee roles abandoned gt
  Teacher-student relationship
• Both work toward ultimate success of student.
• It is through this shift in roles that we find
  both the examiner and the examinee bowed over the
  same task, engaged in a common quest for mastery
  of the material

29
Two Interactionist Studies

• Karpov, Y. V. and B. Gindis. (2000). Dynamic
  assessment of the level of internalization of
  elementary school childrens problem-solving
  activity. In Dynamic Assessment Prevailing
  Models and Applications. C. Lidz and J. G.
  Elliott (Eds.). Amsterdam Elsevier. Spatial IQ
• Peña, E. D. and R. B. Gillam. (2000). Dynamic
  assessment of children referred for speech and
  language evaluations. In Dynamic Assessment
  Prevailing Models and Applications. C. Lidz and
  J. G. Elliott (Eds.). Amsterdam Elsevier.
  Bilingual Children with potential learning
  problems

30
Interactionist L2 DAAnton 2003

• DA used for placement purposes in university
  level advanced Spanish program
• Integrated into OPI-like procedure
• Students asked to renarrate a film story of a
  family traveling through Spain

31
Example 1 (Anton 2003)

• (E)xaminer You started the story in the past and
  then, half way you switched
• (S)tudent Yes, yes
• E To the present.
• S Yes, yes. I heard
• E Do you want to try again using the past ? And
  you can ask me. If there is a verb you do not
  remember its OK.
• S Yes, yes, from the beginning ?
• E Perhaps from the middle ?
• S In the past, yes, yes.
• E Did you realize that you made the switch ?
• S Yes, yes, I heard.

32
Example 2 (Anton 2003)

• S She arrived at the wall pared of the bus
  and waited with her friends at the wall
• E Wall or stop ? Pared o parada ?
• S Stop Parada
• E Do you know what pared is ?
• S wall.
• E Its a very similar word, isnt it ?

33
Example 2 (continued)

• They then returned to the narrative. The student
  began continued to have problems with the past
  tense
• S Juegué al tenis I played tennis
• the correct form for the third person is jugó
• E Jugué o jugó I played or she played ?
• S Jugó She played

34
Example 2 (continued)

• A bit latter a similar problem arose when the
  student was attempting to narrate the fact that
  one of the characters returned home to eat lunch.
• E . Very good. And here you said, what did she
  do ?
• S Comí I ate
• E Comí o comió? I ate or she ate ?
• S Comió She ate
• E Comió She ate

35
Interactionist DA in L2 ClassroomPart I

• Interactionist DA in L2 ClassroomPart I
• S elle est enceinte elle est oh daccord,
  Julianne Moore elle est enceinte de la bébé
  (laughs) de la bébé de Hugh Grant mais Hugh Grant
  ne croit pas pour
• M but in the past
• S na croit pas, na croyé pas
• M yeah um (...)
• S uh joublie
• M right because it was more a description of
  him right?
• S oui alors il est imparfait
• M voilà voilà so you would say?
• S je sais je sais mais je nai pas le used
  imparfait pour beaucoup de fois alors (...)
• M il ne croyait pas
• S il ne croyait pas et uh um il fait laccident
  de son voiture

36
DA in the L2 ClassroomPart II

• A les gens qui voudraient les enfants (...) ils
  ont besoin dêtre préparé? pour leur
  responsabilité davoir les enfants et, on a
  lidée que il na voulu pas uh na pas voulu la
  responsabilité pour les enfants maintenant mais
  pendant il
• M yeah uh right he so remember youve got the
  two past tenses right? Okay
• A pendant il a parlé Rebecca a dit quelle
  quelle a enceinté et uh

37
(Continued)

• M Im just going to kind of interrupt you there
  for a minute and ask you to go back and renarrate
  it again and this time keeping in mind for
  example the difference between the two major past
  tenses in French the passé composé and the
  imparfait
• A Rebecca et Samuel conduisaient à la maison de
  leur ami Sean et pendant le voyage Samuel a dit
  que les gens qui qui avaient les enfants doit
  être prepare préparé pour leur responsabilité

38
Learner Reciprocity Co-Regulation

• Co-regulation a social process by which
  individuals dynamically alter their actions with
  respect to the ongoing and anticipated actions
  physical or communicative of their partners.
  Fogel (1993, p. 34)
• Each participants behavior is emergent from
  their expectations, the actions of the partner,
  the context (as co-weaving) and the constraints
  of their bodies and communicative abilities. The
  consequence of co-regulation is a consensual
  social pattern that is created and elaborated
  over time (ibid.).
• In co-regulation nothing is exchanged or
  negotiated instead information is created
  (Fogel, 1993, p. 55) by the communicative flow
  arising from the generation of verbal and
  non-verbal linguistic signs.

39
Learner Reciprocity Poehner

• Negotiation of Mediation learner realizes that
  mediation offered is not adequate must be
  supplemented
• Mediator as Resource request for specific type
  of support enables M to better attune mediation
  to learner needs. L realizes they do not have
  access to needed resources.
• Create Opportunities to Develop M may offer a
  particular type of support but it may trigger an
  unanticipated response from L
• Seek Mediation Approval L has resources but is
  not completely sure of performance. Is able to
  plan and execute but needs confirmation from
  other that performance is appropriate
• Non-acceptance of Mediation L attempts to show
  autonomy, even when not fully able to perform
  without support. Self-generate effort to push
  their ability forward.

40
Example of Reciprocity

• Seek Approval
• 1. J Est-ce que ça marche? does that work?
• 2. M uh elle lui a demandé? she asked him?
• 3. J elle lui a demandé sil peut wait sil
  pourrait pouvait être plus positif she asked
  him if he can wait if he would be able could
  be more positive
• 4. pouvait être? Uh (...) could be? Uh
• 5. M okay?
• 6. J okay, um en réponse il in response he

41
DA Formative Assessment

• Definition information which will inform
  teachers and students about the degree of success
  of their respective efforts in the classroom. It
  allows teachers to diagnose students strengths
  and weaknesses in relation to specific curricular
  objectives and thus guides them in organizing and
  structuring instructional material (dAnglejan,
  Harley Shapson 1990 107).

42
Types of Formative Assessment

• Planned or Formal Classroom curriculum driven
  achievement test
• May be administered as
• Interventionist DA LLT
• Interactionist DA Antons Placement Test (maybe
  Summative)
• Practice-Based or Incidental Curriculum driven
  integrated into the instructional process.
• External teachers and students reflect on
  student performance while it unfolds or just
  after it unfolds
• Internal occurs through teacher questioning,
  probing and on-line feedback (Ellis 2003)
• Interactionist DA

43
Functions of Incidental FARea-Dickens Gardner
(2000)

• Offers teachers information
• to meet learners needs
• develop new instructional plans
• identify kinds of support required by individual
  learners
• Provides evidence of student learning
• Record of progress toward meeting institutional,
  local and national goals and standards
• Evaluate teaching, including appropriateness of
  methods, sequencing, and content.

44
Rea-Dickens Gardners Conclusions

• Informal and unsystematic nature of FA could
  result in
• overestimates of ability
• underestimates of progress
• Possible implication
• individual child or group of children lose out on
  the appropriate kind or level of instruction
• Decisions based on FAs are high-stakes
• Summative assessment not the only high stakes
  procedure.

45
FA Promotes Learning

• Potential Learning Function of FA
• Assessment embedded in instruction
• May effectively scaffold learning
• Formal assessment measures learning
• Rea-Dickens 2001. Mirror, mirror on the wall
  identifying processes of classroom assessment.
  Language Testing. 18 429-462.

46
Incidental-External FA gt Interactionist
DATorrance and Pryor (1998)

• Second-grade teacher providing feedback on a
  spelling test
• T OK shall we look at those then difficult
  you nearly got right there should be an ell
  there
• Tim cut
• T yes youve got difficut with an ell it goes
  cult you see

47
Incidental-External FA gt Interactionist
DATorrance and Pryor (1998)

• T OK shall we look at those then difficult
  you nearly got right
• Do you think you can spell it correctly ?

48
Torrance and Pryor (1998)

• T OK and s\ night was fine f\family you had
  one go and crossed it out tried again and gave
  up yes
• Tim no its just I didnt get enough time to do
  it
• T oh dear never mind yes we were a bit rushed
  yesterday werent we - fam/i/ly
• T good all right so you tried your hardest
  thats all I want you to do

49
Incidental-External FA gt Interactionist
DATorrance and Pryor (1998)

• T OK and s\ night was fine f\family you had
  one go and crossed it out tried again and gave
  up yes
• T OK, so do you think you can spell it correctly
  ?
• Can you find where you made the mistake ?

50
Incidental-External FA gt Interactionist
DATorrance and Pryor (1998)

• Tim yeah I was going to do that but I couldnt -
  gt()lt
• Timmy points to where the T is writing as he says
  this. He then withdraws his hand again.
• T gtohlt were you oh well never mind because
• T looks up at Timmy who this time meets his gaze.
• T it was possibly my fault for not giving you
  as much time as we had last week but and
  surprise
• T writes in book again.
• T we need to just that was one of the hardest
  wasnt it surprise OK and friends a little
  aye do you think do you have a good practice
  of these words did you?
• Tim yes
• T good all right so you tried your hardest
  thats all I want you to do

51
Emotional vs. Developmental Support(Torrence
Pryor 1998)

• Teacher relies on intuition
• commitment to child-centered gentleness and
  extrinsic rewards
• manage interaction rather than intervening in the
  developmental process (p. 91).
• Unlikely to impact on learning, intended or not
  (ibid.).
• Teacher isnt likely to appreciate or even see
  what is happening because he does not have an
  understanding of the relationship of assessment
  to learning (ibid.).

52
Incidental FA in a DA FormatGibbons (2003)

• Content-Based ESL class focus on scientific
  language
• Whole class interaction following small group
  work.
• Teacher Tell us what happened
• Beatrice Em we put three magnets together / it
  still wouldnt hold the gold nail.
• Teacher Can you explain that again ?
• Beatrice We / we tried to put three magnets
  together .. to hold the gold nail .. even
  though we had three magnets .. It wouldnt
  stick.

53
Gibbons (2003)

• Teacher Tell us what you found out
• Michelle We found out that the south and the
  south dont like to stick together
• Teacher Now lets / lets start using our
  scientific language Michelle
• Michelle The north and the south repelled each
  other and the south and the south also ..
  repelled each other but when we put the when we
  put the two magnets in a different way they /
  they attracted each other

54
Comparison of DA IFA

• IFA may scaffold learning as assessment
• It distinguishes itself from formal testing where
  assistance is not provided
• IFA doesnt seem to attend to the type of
  mediation required to promote learning
• Not theory based
• DA promotes development
• Instructional Setting
• Formal Testing
• Type of mediation central for understanding
  future development
• Negotiated
• Pre-planed
• Theory based

55
DA and Psychometrics

• Generalizability Reliability
• Future Performance Pedagogy
• From Sample to Group Research
• Assertive Logic (Polkinghorne 1996) Link to
  theory
• Validity
• Systemic
• if test brings about or induces, an improvement
  in the tested skills after a test has been in the
  educational system for a period of time.
  (Shohamy 2001 142)
• Predictive
• Consequential

56
Measurement

• Snow (1990, p. 1135) objects to DA on the premise
  that without linking assessment in some way to
  measurement, fundamental in all science, the
  term is meaningless.
• Bachman (1990, p. 18.) defines measurement as
  the process of quantifying the characteristics
  physical as well as mental of persons according
  to explicit procedures and rules.
• Büchel and Scharnhorst (1993, p. 101.) suggest
  that DA
• researchers can link assessment and measurement
  through standardization of the examinersubject
  interaction, a characteristic of interventionist
  approaches to DA, but not of interactionist
  approaches.

57
Psychometrics Validity Reliability

• Built on a foundation that privileges the
  autonomous individual as the site from which
  performance and development emerge.
• DA, on the other hand, is built on a foundation
  which privileges the social individual, or as
  Wertsch (1998) puts it person-acting-with-mediati
  onal-means.

58
Vygotsky on Measurement

• Vygotsky measuring a childs performance
  provides little more than a purely empirical
  establishment of what is obvious to persons who
  just observe the child and adds nothing new to
  what is already known through direct observation
  (Vygotsky, 1998, p. 205).
• The task of the psychologist is not to measure
  but to interpret the individual (Vygotsky, 1998,
  p. 204).

59
Vygotsky on Subjectivity in Science (vol. 2,
collected works)

• Task of Research Methodology not just to
  measure, but to see, think associate.
• Fear of interpenetration of subjective factors in
  assessment research unfounded
• Research results cannot be fully achieved through
  purely mechanical arithmetic methods
• Without subjective reevaluation (without thought
  interpretation) deciphering results
  evaluation of data is not scientific research

60
DA Reliability

• From the ontology of the social individual and
  the clinical perspective on assessment, the
  examiners participation in the process is
  essential and therefore cast in a positive light.
  
• As Lidz (1991, p. 18) the word dynamic
  implies change and not stability. Items on
  traditional measures are deliberately selected to
  maximize stability, not necessarily to provide an
  accurate reflection of stability or change in the
  real world.
• Because DA integrates teaching and assessment
  change is necessarily an artifact of the test.

61
Predictive Validity

• the ZPD, which is the foundation of DA, is itself
  an empirically grounded prediction of learner
  developmentwhat is at one time carried out
  interpersonally will eventually be carried out
  intrapersonally.
• Specifically, this means that during the course
  of an assessment or from one assessment to
  another, mediation is expected to become less
  frequent and less explicit as learners display
  greater control or self-regulation over the
  construct under consideration
• Transcendence learners are further expected to
  extend their abilities to increasingly complex
  activities once they have internalized the
  mediation (see the discussion of Poehners
  research above).

62
DA Consequential Validity

• From the perspective of the social individual,
  there are serious
• ethical problems using the outcomes of
  assessments based exclusively on solo performance
  to make decisions that impact the lives of
  individuals and the institutions in which they
  function
• how appropriate is it to place students into
  the same language course on the basis of their
  solo performance knowing that their relative
  mediated performance could vary significantly
  and that therefore the individuals in question
  would benefit from different forms of
  instruction?
• how ethical is it to knowingly miss an
  opportunity to help someone develop during an
  assessment for the sake of maintaining
  psychometric principles?

63
Dynamic Assessment Equal Access to Everyone

• A. N. Leontev American researchers are
  constantly seeking to discover how the child came
  to be what he is we in the USSR are striving to
  discover not how the child came to be what he is,
  but how he can become what he not yet is.
  Bronnfenbrenner (1977 528)
• Ratner (2006) The object should be not to beat
  the odds but to change the odds.

64
References

• Lantolf Poehner. 2007. DA Dynamic Assessment
  A Teachers Guide. CALPER Publication. Available
  on DVD through CALPER.
• Lantolf Poehner. 2004. Dynamic assessment of
  L2 development bringing the
• past into the future. Journal of Applied
  Linguistics 1 49-74.
• Lantolf, J. P. M. Poehner (to appear). Dynamic
  assessment. In N. Hornberger (Ed.) The
  Encyclopedia of Language and Education, vol. 7
  Language Testing and Assessment. Cambridge
  Cambridge University Press.
• Poehner, M. (2005). Dynamic assessment in the
  foreign language classroom. Ph.D. dissertation.
  Penn State University. University Park, PA.
• Poehner. M. (forthcoming a). Both Sides of the
  Conversation The Interplay between Mediation
  and Learner Reciprocity in Dynamic Assessment. In
  Lantolf, J. P. M. E. Poehner (Eds.).
  Sociocultural theory and the teaching of second
  languages. London Equinox.
• Poehner, M. (forthcoming b). Dynamic assessment
  in L2 Learning. Berlin Springer Verlag.
• Poehner Lantolf. 2005. Dynamic Assessment in
  the language classroom. Language Teaching
  Research. 9 233-265.
• Pod-Cast on Dynamic Assessment available on
  CALPER website calper.la.psu.edu