Using Technology to Restructure Spanish Curricula: Texas Techs R2R Project - PowerPoint PPT Presentation


PPT – Using Technology to Restructure Spanish Curricula: Texas Techs R2R Project PowerPoint presentation | free to view - id: 12134-NTY5N


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Using Technology to Restructure Spanish Curricula: Texas Techs R2R Project


Using Technology to Restructure Spanish Curricula: Texas Techs R2R Project – PowerPoint PPT presentation

Number of Views:80
Avg rating:3.0/5.0
Slides: 92
Provided by: julian4


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Using Technology to Restructure Spanish Curricula: Texas Techs R2R Project

Using Technology to Restructure Spanish
Curricula Texas Techs R2R Project
  • Dr. Frederick Suppe, PI
  • Tracy A. Rutledge
  • Dr. Stephen Corbett
  • Dr. Comfort Pratt
  • Silvia Peart
  • Dr. Greta Gorsuch
  • Claudia Diaz
  • René Ibarra
  • Hilda Salazar

Special thanks to Judy Collazo, Valerie Huggard,
Natasha Ojeda, Terry Martin, Phade Vader, and the
instructors and students who participated in the
Overview of Texas Tech Beginning Spanish Course
Redesign Project
  • Development of a comprehensive instructional
    system for lower-level language instruction.
  • System is text-book and language independent.
  • System aggressively combines technology with
    highly communicative approaches.
  • Individual components can be used rather than the
    whole system.
  • Extensive assessment and evaluation component to

Overview of Texas Tech Redesigned Beginning
Spanish Course
  • Hybrid 5 credit course with 3 contact hours
    in-class and 2 hours on-line.
  • In-class portion devoted to communicative
    exercises and limited to 20 students.
  • Workbooks replaced by on-line interactive drill
    with targeted diagnostic feedback and automatic
  • Semi-automated grading of on-line compositions
  • Bi-weekly Language Lab assignments.

Overview of Texas Tech Redesigned Beginning
Spanish Course
  • Includes adaptive re-use of stripped Destinos
    episodes for in-class communicative exercises.
  • Development and validation of ACTFL-based
    assessment instruments for evaluating the 4
  • Uses Puntos 7 as textbook, but approach can be
    used with any textbook or language

  • Demonstrations of each component of the system.
  • Presentation of theory underlying the system
    components and how to information.
  • Extensive handouts covering each component.
  • Guidance for adapting our approach or its
    components to your own courses.
  • Evidence of the effectiveness of the approach

Our redesign is part of the Roadmap to
Redesign (R2R) Project
  • A project of the National Center for Academic
    Transformation (NCAT)
  • Successor to the PEW-funded Program in Course
    Redesign (PCR) project that collaborated with 30
    institutions to demonstrate how colleges and
    universities can redesign their instructional
    approaches using technology to achieve cost
    savings as well as quality enhancements.
  • PCR resulted in 5 Redesign Models and 5
    Principles for Successful Course Redesign.

R2R Overview
  • Three-year, FIPSE-funded project to test results
    of PCR and establish an efficient means for
    spreading the PCR ideas and practices to
    additional institutions.
  • Focuses on large-enrollment, introductory courses
    in four disciplines pre-calculus mathematics,
    psychology, Spanish and statistics.

R2R Overview
  • National competition to select participating
    colleges and universities.
  • 40 finalist institutions invited to submit
  • 20 selected for the project, 4 in Spanish
  • Montclair State University
  • Texas Tech University
  • Towson University
  • University of Alabama

The Texas Tech R2R Spanish Redesign Project
The Old Spanish 1507 Course for students who have
had at least 2 years of high school Spanish
  • 630 students/semester
  • 19 instructors
  • 12 sections of 32 students
  • 5 hours in class per week Language Lab
  • 1 section/instructor
  • 32 students, 5 contact hours, 5 preparations
  • Hand graded workbook exercises compositions
  • 110 student Lecture/discussion version
  • 40 students in Destinos on-line version

The Redesigned Course
  • 720 students/semester (vs. 630)
  • 18 instructors (vs. 19)
  • 36 sections of 20 students/section ADFL
    recommended maximum (vs. 32)
  • 3 hours in class per week (vs. 5), 2 hours
    on-line (vs. 0) Language Lab
  • 2 sections/instructor (vs. 1)
  • 40 students, 6 contact hours, 3 preparations (vs.
  • automatic grading of on-line drills,
    semi-automated grading of compositions--3-5
    minutes per student per composition (vs. 35
    minutes grading/student/week)
  • 45 students on-line version, 60

  • Reduce Class Size to ADFL Guidelines
  • Reduce workload of instructors
  • Increase capacity of course by 19
  • Use existing classroom space
  • No increase in instructional costs
  • No Pedagogical Harm to students
  • SPECIFICALLY No significant decline in language
    proficiency under redesign.
  • Develop improved testing materials

The On-line components were originally developed
for a broadcast TV/web-based distance-education
Beginning Spanish course using Destinos. The
components were tested against conventional
courses in controlled experiments.
The On-Line Destinos CourseDeveloped by Tracy
Rutledge and Dr. Frederick Suppe
  • Students watch 52 episodes of Destinos telenovela
  • Students do on-line interactive drills and submit
    on-line compositions
  • Drills cover workbook material with automated
    grading that includes targeted diagnostic
    feedback on wrong answers.
  • Compositions are semi-automatically graded with
    targeted diagnostic feedback.
  • Class meets physically only for examinations.

The Destinos On-line Components
  • Uses Web-CT as course management system, but
    approach should work with other systems.
  • Requires work-arounds to get Web-CT to do things
    it wasnt designed to do.
  • Main on-line components
  • On-line compositions with semi-automated
  • grading with targeted diagnostic feedback.
  • 3-5 minutes grading per student per week with
    detailed diagnostic feedback.

The Destinos On-line Components
  • Workbooks replaced by parallel
  • automatically graded drill exercises.
  • Students never given correct answers.
  • Targeted diagnostic feedback given for wrong
  • Students repeat drills as many times as they
    want, with a new drill each time.
  • Goal is mastery of vocabulary and grammar in a
    timely fashion.

On-line Destinos Effectiveness
  • Automated grading of drills is effective and
  • Students learn more than in traditional courses
  • Reading Comprehension Significantly higher
  • (2 SDs)
  • Oral comprehension Very significantly
  • (3 SDs)
  • Writing No significant difference
  • Speaking No significant difference
  • Grammar and vocabulary Significantly
    higher (2 SDs)

  • Our On-line interactive Drill is more effective
    than traditional class-room use of workbooks.
  • On-line submission and semi-automated grading of
    compositions are as effective as traditional
    compositions and far more efficient to grade.
  • Automated grading of drills reduces instructor
    grading time.
  • These findings form the technological basis for
    our R2R redesign of SPAN 1507.

Spring 2005 Pilot Experiment to test the Hybrid
  • Full Human Subjects Approval, signed informed
  • 8 R2R, 8 Control sections
  • 32 students per Control, 20 per R2R
  • 160 R2R, 256 Control students
  • 4 clusters of 2 R2R and 2 Control section pairs
    on basis of differential teaching ability
    (Exceptional, good, average, weak)
  • One pair taught by same instructor
  • One pair taught by different instructors

Main Pilot Findings
  • No Harm Result No statistically significant
    difference in performance between R2R and
  • R2R students generally did slightly better.
  • Some course improvements made and tested in
    Summer 2005.
  • Full Implementation of Redesign in Fall 2005
    with 780 students, 21 instructors, 39 sections, 6
    person coordination R2R research team.

We now present the various course components,
their underlying design, and offer guidance for
adaptation to your own courses
  • On-line interactive Drills
  • On-line semi-automatically graded compositions
  • Multiple adaptive repurposing of Destinos
  • Standardized in-class communicative exercises
  • Mentoring and training of instructors
  • ACTFL-derived testing and assessments

On-Line Components Interactive DrillsDeveloped
by Claudia Diaz, Silvia Peart, Tracy Rutledge and
Dr. Frederick Suppe coordinated by René
On-Line Interactive Drills
  • Workbook exercises are converted into different
    but parallel on-line exercises.
  • Targeted diagnostic feedback given for wrong
  • Correct answers revealed only if student gets
    them correct.
  • Can repeat drills as many times as one wants
    within window of opportunity.
  • Competency-based progression en-echelon Students
    must spend 2 hours a week or get at least 80 on
    one attempt or else their grade is zeroed-out.

Students are presented with aseries of questions
When they submit their drill attempt they get
targeted diagnostic feedback
Multiple Choice Questions
Multiple Choice Feedback
Matching Question Feedback
Short-answer Question
Short-Answer Feedback
Web-CT limitations do not allow targeted separate
comments for each short-answer option so we have
to provide a generic feedback that covers all
Designing On-Line Drills for Targeted Diagnostic
  • Design of the answer space Conventional workbook
    exercises have to be reconceptualized to meet the
    following criteria
  • There must be a fixed answer space with a
    finite set of discrete non-overlapping possible
    answers (cells).
  • Each possible incorrect answer must evidence a
    specific error pattern.
  • Each cell must have a unique diagnostic feedback
    message associated with it.
  • The simplest way to achieve this is to use
    True-False, Multiple-choice, or Matching test

Designing Short-Answer Questions
  • Short Answer questions must be
    reconceptualized to meet the fixed discrete
    answer space requirement
  • Strategies
  • Restricting vocabulary
  • Requiring standardized syntactical structures for
    the answers
  • where anything not meeting the restrictions
    can receive a default response indicating the
    restrictions were violated.
  • In effect this involves making the question a
    disguised multiple-choice question where the
    options are not revealed. This may require as
    many as 40 cells in the answer space.
  • Some classroom management systems (e.g., Web
    CT) compromise your ability to give fully
    targeted diagnostic feedback because they do not
    allow you to attach separate feedback to each
    hidden answer option.

Typical workbook items can be reconceptualized to
meet the above answer space design requirements.
  • Analyze the workbook exercise to determine what
    it is attempting to test or teach.
  • If the exercise is testing or teaching more than
    one thing split into separate exercises for each
    learning/testing objective.
  • Using the same vocabulary and grammar, rework the
    exercise as a replacement exercise meeting the
    answer-space design requirements.
  • Pick your possible answers so as to allow
    response-specific diagnostic feedback.

Strategies for converting typical workbook
exercises for on-linewith diagnostic feedback.
On-Line Components Written Assignments with
Semi-Automated Grading Developed by Tracy
Rutledge, and Dr. Frederick Suppe, coordinated by
them and René Ibarra
Semi-automated grading of on-line compositions
  • Basic Empirical Finding If one restricts
    diagnostic feedback to just items related to a
    given lessons learning objectives, then there
    will be 8-12 comments that cover more than 95 of
    relevant comments that need to be made on student

Exploitive strategy Archive authoritative
comments targeted to each exercise and supply
them to graders. Example
Student down-loads a Word template giving the
assignment and a model response
Student adds composition to Word and submits file
via Web-CT assignment (not quiz) modality.
Instructor downloads and grades using prefabbed
  • Targeted Comments
  • 1. No always goes in front of the verb.
  • 2. When saying someones age you must use tener
  • 3.Agreement error.
  • 4. This verb does not agree with your subject.
  • 5. Following a conjugated verb you should use
    the infinitive.
  • 6. The verb gustar has a special conjugation.
    It uses an indirect object such as me gusta.
  • 7. In Spanish we have letter ñ
  • 8. Missing verb.
  • 9. Omit
  • 10. Word order error
  • 11. Vocabulary error
  • Generic Comments
  • The red underlined words are either proper names
    or misspelled words (a missing accent / tilde( ?
    ) is considered a misspelled word). The green
    are grammar errors. Right click on the word and
    you should get suggestions.
  • ?Excelente!?
  • ?Muy Bien!?

. resulting in an annotated composition to
upload to the student
Before uploading, the instructor uses an
ACTFL-derived holistic standard (see your handout
later for details) to assign a composition grade.
Repurposing Destinosmaterials
Coordinators and developers Dr. Stephen
Corbett, Dr. Comfort Pratt, Dr. Frederick Suppe,
and Tracy Rutledge
Integration into First-Year Spanish
  • Students watch 1 episode of the Destinos
    telenovela a week
  • Broadcast simultaneously via closed circuit TV
    into high-definition projector- equipped
    classrooms from central control room.
  • Didactic material stripped automatically to
    produce a 15 minute story segment.
  • Viewing surrounded by in-class communicative

Structure of a Destinos Session
  • 10 Minute preparation period
  • Review previous episode

Structure of a Destinos Session
  • 10 Minute preparation period
  • Review previous episode
  • Students read composition assignments about
    previous episode.

Structure of a Destinos Session
  • 10 Minute preparation period
  • 15 minute viewing session

  • 15 minute viewing session
  • Other uses of Destinos videos
  • In-class communicative activities.
  • On-line tele-courses
  • Clips from not yet viewed episodes are used in
    listening comprehension tests.

Structure of a Destinos Session
  • 10 Minute preparation period
  • 15 minute viewing session
  • 10 minute post-viewing discussion
  • Summary of episode done by students
  • Instructor asks specific questions to test

A variety of activities role-playing relating
events in episode to other events discussing
topics raised in relation to other matters
speculating on what will happen in the next
episode.Essay assignment Summarize episode,
answer specific questions.
Each session has a fixed lesson plan (in your
handout packet)
Destinos Integration into Second-Year Spanish
  • Viewed during fifty minute period of class
  •      Pre-viewing activities (teacher comments
    and questions, new vocabulary introduced)
  • Viewing with questions (true-false, fill-in the
    blank and short-answer)
  • Follow-up in small groups and with whole class to
    answer questions and share opinions
    or expectations
  • Evaluation of skills and knowledge via....
    Listening comprehension quizzes
  • (multiple-choice format) Written exams
    (short-answer format)
  • Student feedback used to revise implementation.

Use of Destinos materials in Study- Abroad
intensive immersion courses
  • Students do 11 credit hours of lower-level
    Spanish in Seville, living with monolingual
  • Destinos workbook and a street Spanish book used
    as only textbooks.
  • (Texts like Puntos have a Latin
    American cultural emphasis that is inappropriate
    for teaching in Spain.)
  • One hour a day online Destinos drills using
    servers in Lubbock replace homework.
  • Destinos videos not used.
  • Culture component provided by excursions and in
    situ class events (in the mercado, etc.).

Other in-class Communicative Activities
SPAN 1507 Coordinator Dr. Comfort Pratt
Communicative Activities
  • 2 principal components
  • Instructor training
  • Pre-semester Orientation
  • Semester-long Training Sessions
  • 2 classroom observations per semester
  • In-class communicative activities

Pre-semester Orientation 4-day training session
for instructors
  • Instructors are given an overview of
    communicative language teaching (CLT).
  • They are taught how to implement CLT in the
  • They learn how to prepare lesson plans.
  • They are taught how to use the on-line
  • They have practice and calibrations sessions on
    using grading protocols.

Semester-long Training Sessions
  • Once a week instructors meet with coordination
    team for 1 hour and 20 minutes.
  • Every other week are CLT training sessions that
    focus on Communicative Activities.
  • Alternate weeks focus on remaining components of
    the course.

Biweekly Communicative Language Teaching Sessions
  • Instructors
  • receive
  • communicative
  • lesson plans
  • for the
  • following two
  • weeks and
  • accompanying
  • materials.
  • (Example in handout package)

Biweekly Communicative Language Teaching Sessions
  • Instructors are taught how to carry out the
    activities in the classroom by means of a
    hands-on training process

Classroom Observations
  • The coordinator observes each instructor in the
    classroom twice every semester and evaluates
    their performance
  • The instructors meet with the coordinator after
    the visit to discuss the evaluation and

In-Class Communicative Activities
  • Class meets physically for 3 hours a week
  • Class time is entirely dedicated to communicative
  • Instructors follow lesson plans designed by
    coordinator (examples in handout)
  • Structure of lesson plan
  • Communicative goals
  • Communicative activities
  • Assignments

Insert here slides and video of training and then
Teachers practice the activities they will use in
And have their execution of the activities
Practicing a Total Physical Response activity
Practicing a rooms of the house.
and daily routines TPR activity.
Testing and AssessmentCoordination and design
team Dr. Greta Gorsuch, Silvia Peart, Hilda
Salazar, Dr. Frederick Suppe
R2R Assessment and Evaluation
  • Pilot study was done as a controlled experiment
    with full Human Subjects approval. More than
    60,000 data points were collected over 411
  • These data included performance data and 5
  • The same sorts of data were collected during the
    summer 2005 trial implementation and are being
    collected for the full Fall 2005 implementation
    with 780 students.

R2R data collected
  • Surveys
  • Demographic information
  • Academic achievement and motivation survey.
  • Technology survey regarding prior experience with
    on-line courses or components.
  • Mid-semester and end-of-semester attitudinal
    surveys towards the course.
  • Academic performance data.
  • Key-stroke and time-on-task records for all
    on-line drill attempts.
  • Classroom observation reports using a
    standardized reporting schedule.
  • Copies of all exams and graded materials

Basic No Harm Pilot Result
We expect better performance in the
implementation phase!
The most interesting pilot finding is that
standard ways of testing student performance are
unreliable, of unproven validity, or just invalid!
  • We base this conclusion on the basis of standard
    evaluations of tests generated by
    publisher-supplied test banks.
  • We did so using sophisticated test validity
    analyses of our pilot data sets

Problems with standard testing include
  • too few items (causing unreliability).
  • test items not having item-discrimination
  • no construct validity for grading
    protocolsespecially for writing and speaking.
  • factor-analysis indicates that virtually all test
    items load on a single undifferentiated
    language factor precluding differentiation of
    performance into differentiated performance on
    each of the four skills.
  • in particular such testing methods preclude
    separate evaluation for each of the four skills.

Our Texas Tech R2R response to this disturbing
findingDevelop, test, and validate separate
ACTFL-derived evaluations methods for each of the
four skills
Example 1 Listening Comprehension(Sample given
in the handouts.)
  • Students watch a clip from a Destinos episode
    they have not seen.
  • They are given a few minutes to read the
    objective listening comprehension questions
    before they watch the clip.
  • After viewing they have time to answer 25
    objective questions.
  • In order to differentiate listening from other
    skills, questions are in English and of provable

Example 2 Writing Evaluation
  • 10 of each sections on-line compositions are
    randomly selected and randomly sent for double
    blind regrading.
  • A sophisticated computer program compares grading
    and regrading performance against each other and
    against all 36 sections, producing weekly
    diagnostic reports suggesting grading problems
    accompanied by copies of original and regraded
  • ACTFL-derived assessments, and distribution of
    prefabbed comments, and flagged diagnosis of
    suspicious disparities, using a strict standard,
    in application of standards are sent in the
    weekly reports.

Example 3 Speaking(Mini-SOPI example given in
  • The Center for Applied Linguistics (CAL)
    Simulated Oral Evaluation (SOPI) approach has
    been modified to develop Mini-SOPIs that take
    about 13 minutes to administer, producing 5-7
    minutes of taped responses that are evaluated.
  • Unlike SOPIs or ACTFL OPIs, this approach can be
    used twice per semester with about 800 students
    without over-burdening instructors.

  • Are based on the SOPI design principles of CAL,
    hence inherit much of their validations.
  • Unlike OPIs and SOPIs which attempt to discern
    where a speaker is on the entire ACTFL spectrum,
    our Mini-SPOIs only attempt to place students on
    an empirically-based portion of the ACTFL
    spectrum predicted on the basis of proficiency
  • This allows us to reduce time from 50-25 minutes
    to 7 minutes of evaluation time by reducing tasks
    to a range appropriate the students target
  • Recordings use near-professional recording of
    professional or semi-professional actors to
    record the tapes.
  • Training sessions in application of ACTFL-derived
    standards use CAL SOPI training tapes.

Mini-SOPI Evaluations
  • 10 randomized re-evaluation of Mini-SOPIs each
  • Comparisons and reports made similar to those
    for writing evaluations.
  • Inter-rater reliability studies to establish
  • Concurrent validity studies compared with
    official ACTFL OPIs and CAL SOPI evaluations
    will be used to establish construct validity of
    the protocol.

Reading, Grammar, Vocabulary
  • Primary problem with standardized reading
    comprehension testing is insufficient number of
    test items 10 is minimum for reliability (unlike
    publisher test banks that generate 4).
  • Grammar and vocabulary tests generated from test
    banks are ok, but often contain items with poor
    discriminate validity.
  • Need to assess all test items and replace by
    ones having suitable reliability and discriminate

Evaluation of Texas Techs R2R project
  • Is a comprehensive teaching system.
  • Sufficiently structured as to make teaching
    personnel interchangeable parts that can be
    deployed to produce a fairly uniform product.
  • Does not minimize the added value of great
    teaching but insulates the student from damage by
    poor instructors.
  • Marries proven on-line approaches pioneered in
    the TTU Destinos on-line project with
    cutting-edge communicative in-class activities.
  • Involves extensive on-going mentoring and
    training 30-48 hours per semester.
  • Triages using on-line resources where they are
    more effective, communicative classroom
    activities where they are more effective.
  • Exploits technology in a sophisticated manner
    in-class and on-line.

Evaluation of Texas Techs R2R project
  • No Harm result! IT WORKS!!!!
  • Reduced class size without cost/student increase.
  • Has a very extensive and ambitious evaluation
  • All curricular decisions made on the basis of
    empirical data and target goal performance
  • Massive data sets collected that will be the
    basis for research studies and articles for years
    to come and for dissertations.

Continuation of our R2R project
  • Pilot and implementation of SPAN 1507
    conversion is so successful that we plan to
    extend the R2R redesign to second-year (3 credit)
    Spanish courses
  • Hybrid course, 2 hours inclass, 1 hour on-line.
  • Adapt and extend the communicative in-class
    activities including use of Destinos stripped
    episodes in class.
  • Extend the ACTFL-derived assessment/grading
    protocols to second year and validate them.
  • Collect comparable survey and performance data
    for second-year students

Timetable for Second-Year Implementation
  • Pilot conversion of SPAN 2301 (first semester of
    second year)
  • Spring 2006 with 9 R2R and 9 control
    sections in controlled experiment.
  • Depending on outcome of this pilot, implement
    2301 redesign in Fall 2006.
  • Determine whether separate SPAN 2302 (second
    semester of second year) pilot is required and
    then either pilot or implement R2R redesign.

Longer-term objectives
  • As our redesign approach and assessment
    instruments are validated we hope
  • To exploit the fact that our design goals have
    been to develop a teaching system and assessment
    instruments that is language and textbook
  • To port the approach over to other languages in
    a manner where components can be selectively

Costs of the project
  • The piloting and implementation of the SPAN 1507
    redesign cost about 136,000 spread over 18
  • This does not include salary allotments for the
    three professors involved.
  • 30,000 was provided by an external donor the
    remainder came from departmental discretionary
    funds. No special funds were sought from the
  • Continuation of the project for similar redesign
    of second-year Spanish will raise the special
    costs to about 250,000.

Arghh! That is unaffordable!
  • For most institutions you are correct.
  • However, in the spirit of the R2R project, we
    have tried to provide you through this
    presentation and the extensive handouts with
    enough guidance to
  • Decide which elements of our teaching system
    might be adaptable to your local circumstances.
  • Experiment with efficient development and
    adaptation of selected components of our system
    to your local needs and purposes
  • As with the overarching national R2R project, our
    goal today has been to help you redesign your
    lower-level language courses without having to do
    so from scratch and to share the wisdom we have
    gained in our own experiments and from our
    predecessor PEW PCR partners.

  • We hope we have provided you through this
    presentation and the extensive handouts a basis
    whereby you can consider adapting selected
    components of our Texas Tech foreign language
    teaching system to your own local needs and
    institutional resources.

(No Transcript)