Thinking about thinking: Methods for the study of online reading comprehension - PowerPoint PPT Presentation

Loading...

PPT – Thinking about thinking: Methods for the study of online reading comprehension PowerPoint presentation | free to download - id: be7fd-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Thinking about thinking: Methods for the study of online reading comprehension

Description:

... scores that determine students' proficiency level in online reading comprehension ... Are there new proficiencies over and above general reading comprehension and ... – PowerPoint PPT presentation

Number of Views:61
Avg rating:3.0/5.0
Slides: 50
Provided by: newlitera
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Thinking about thinking: Methods for the study of online reading comprehension


1
Thinking about thinking Methods for the study of
online reading comprehension
2
Dr. Donald J. Leu
  • Part I
  • Introduction Theoretical Frame

3
More than a billion people are reading online
today
4
The RAND Reading Study Group concluded
  • accessing the Internet makes large demands on
    individuals literacy skills in some cases, this
    new technology requires readers to have novel
    literacy skills, and little is known about how to
    analyze or teach those skills. (p. 4)

5
Our Strategy
  • A continuously, iterative method within a
    collective team approach.
  • Based on extent research in multiple areas of
    inquiry as well as our own surveys, interviews,
    observations of online readers, and other data.
  • Each of us, bringing our own questions, methods,
    and results, continuously challenge our working
    theoretical definition as we seek to refine and
    expand it.

6
Current theoretical definition of online reading
comprehension
  • The new literacies of online reading
    comprehension include the skills, strategies, and
    dispositions necessary to use the Internet and
    other ICT to (1) identify important questions,
    (2) locate information, (3) analyze the
    usefulness of that information, (4) synthesize
    information to answer those questions, and then
    (5) communicate the answers to others.

7
You will see a team approach to inquiry in this
area, where all members of our research team
are valued as equal colleagues in the research
enterprise.
8
The important next question is
  • How might we build an even more powerful,
    collective team approach to research in order to
    better understand the nature of online reading
    comprehension?

9
Jill Castek
  • Part II
  • The Changing Nature of Reading Comprehension
    Examining the Acquisition of New Literacies in a
    7th Grade Science Classroom

10
The Issue
  • How can we examine the acquisition of new
    literacies in a 7th grade science classroom where
    the Internet has been integrated into instruction
    for a school semester?

11
Create
  • Online problem-solving tasks requiring students
    to demonstrate their ability to
  • 1) locate information
  • 2) critically evaluate information
  • 3) synthesize information from multiple sources,
  • 4) communicate the results efficiently to others
    (Leu, Kinzer, Coiro, Cammack, 2004).

12
Post
  • Tasks on a secure, educationally structured blog
  • Typepad (http//www.typepad.com/)
  • Edublogs (http//edublogs.org/)
  • Blog interface prompts authentic communication
    http//newliteracies.typepad.com/science_exchange/
    )

13
Observe Record
  • As students individually complete tasks
  • List observations on a specialized rubric
    designed to evaluate the tasks.
  • Observations translate into scores that determine
    students proficiency level in online reading
    comprehension

14
Archive Data
  • A real time screen capture program called
    Camtasia is useful for capturing process data.
  • This program records audio as well as students
    screen movements as they complete tasks.
  • Video data can be examined multiple times to
    achieve inter-rater reliability between
    researchers.
  • Responses and processes can be analyzed further
    to identify patterns in strategy use.
  • Link to
    Camtasia Video

15
(No Transcript)
16
Video Examples and Score Sheets
Snapshots of Three Online Readers
http//www.newliteracies.uconn.edu/reading.html
17
Discussion
  • How can the collection of data in these forms
    enhance our understanding of online reading
    comprehension?

18
Julie Coiro
  • Part III
  • Measuring the Nature of Online Reading
    Comprehension with ORCA-Scenario I and II

19
The Issue
  • How can we scale up to reliably measure
    performance in two parallel online reading
    comprehension scenarios with larger groups of
    students (n120) in an online quiz interface?
  • Purpose (mixed-methods) What factors contribute
    to performance on a scenario-based measure of
    online reading comprehension?
  • Are there new proficiencies over and above
    general reading comprehension and domain-specific
    prior knowledge?
  • If so, what patterns emerge among skilled and
    less-skilled readers?

20
Methods Designing Test Items
  • Adapt the ORCA-Blog to design two parallel
    measures of online reading comprehension
    (ORCA-Scenario I and ORCA-Scenario II) as part of
    an Internet Treasure Hunt created by fictitious
    7th graders requesting online resources about the
    respiratory system and carbon monoxide poisoning
  • Designing parallel items that independently
    measure each online reading proficiency (see
    handout)

21
Online Reading Comprehension Assessment (ORCA)
Scenario I
22
Online Reading Comprehension Assessment (ORCA)
Scenario II
23
(No Transcript)
24
Methods Designing Protocols
  • Develop a standardized protocol for moving
    multiple levels of students through the online
    quiz interface and recording observations.
  • 12 students at a time (in 3 cycles every 20
    minutes)
  • Time limits for completing each task
    (15/20/20/10)
  • Reviewing directions individually and adjusting
    prompts for diverse range of students
  • Use field notes to track the time, what each
    student completed, actions (e.g., writing on
    paper), and difficulties (e.g., questions)
  • Start, stop, and reset Camtasia while clearing
    browser settings for each new user

25
Methods Designing Scoring Rubrics
  • Develop a scoring rubric that integrated both
    process and product
  • Rubric item scores are totaled to reflect process
    and product in a way that can be used in
    quantitative analyses (e.g., comparing means,
    correlations, factor analysis, predicting the
    role of isolated variables, etc.)

26
Methods Designing Scoring Rubrics
  • Develop a scoring rubric that integrated both
    process and product
  • Actions noted on score sheet and integrated with
    Camtasia process recordings and strategy
    interview transcripts in a way that can be used
    for qualitative analyses of patterns
  • within readers
  • across multiple tasks or sessions
  • across multiple readers

27
Methods Capturing Reading Actions
28
Scoring locating efficiency as part of online
reading comprehension
29
Scoring how deeply readers read for accuracy or
how efficiently they communicate information to
others
30
Methods Integrating and Analyzing Data Sources
  • Develop a procedure for integrating multiple
    types of data into one transcript
  • Student name (pseudonym)
  • Item and task number (similar items across tasks)
  • Video action (from testing session)
  • Researcher prompts (from follow-up stimulated
    recall interview)
  • Student think-aloud (from follow-up stimulated
    recall interview)
  • Field notes and beginning interpretations

31
(No Transcript)
32
Methods Accomplishments
  • Successfully scaled up to measure more students
    with one researcher in online quiz interface
  • Developed a second valid and reliable measure of
    online reading comprehension (parallel forms -
    Scenario I and II)
  • Developed a standardized protocol for
    administration and follow-up strategy interview
  • Designed a preliminary method of combining
    multiple data sources of process and product data
    for multiple analyses
  • Next questionWhat do we lose (or gain) by
    measuring only the products of online reading
    comprehension as compared to both product and
    process?

33
Laurie A. Henry
  • Part IV
  • Streamlining the Assessment Process for Measuring
    Online Reading Comprehension

34
The Issue
  • How can we streamline the assessment of online
    reading comprehension skills and strategies and
    develop a measurement tool that is highly
    correlated to the previously utilized
    performance-based assessments?

35
Develop
  • A Survey of Internet Use and Online Reading

36
Administer
  • Seventh grade students in partner schools at
    research sites in Connecticut and South Carolina
    (n 1,025)

37
Analyze
  • Analysis of student responses to determine
    skill-level for Internet reading comprehension.

38
Scoring
  • Rubrics developed to score open-ended skill items
  • Inter-rater reliability established between four
    scorers (2 at each research location) with ?
    .87

39
Sample Student Responses
  • (0) I picked this site because it seems to tell
    the most information.
  • (1) i picked my answer because the website says i
    will learn about history, language, and culture
    of ancient Egypt
  • (2) you would choose the ancient egypt site
    because one is a site that has other peoples
    reports, another one is about travling to egypt,
    and the last one does not give you a lot of info
  • Less than 5 of the students reported that they
    check the information they read online for
    accuracy

40
Important Next Question
  • Can we develop a survey instrument that is a
    good measure of online reading comprehension and
    is highly correlated with the ORCA, yet requires
    minimal effort for administration and scoring?

41
Lisa Zawilinski
  • Part V
  • Measuring Pre-service Teacher Knowledge of the
    New Literacies

42
Create
  • Survey of Online Teaching Learning
    http//www.newliteracies.uconn.edu/carnegie/Teache
    rEd.html
  • Course Internet Survey-Written Responses
    (scenarios) http//www.newliteracies.uconn.edu/car
    negie/TeacherEd.html

43
Administer
  • Fifty-two pre-service teachers (38 seniors, 14
    graduate students) in an integrated
    bachelors/masters program at a public university
    in the Northeast participated in the study
  • Survey scenarios administered separately, at
    the start then again at the end of the methods
    course

44
Archive Analyze
45
Important Next Question
  • How can we effectively integrate new literacies
    instruction within teacher preparation programs?

46
Dr. Douglas Hartman
  • Part VI
  • Open Conversation What are the most important
    next questions and how might they be studied?

47
Questions
  • You know our important next questions.
  • What are yours?
  • Are there other questions that we need to
    consider as well?

48
(No Transcript)
49
Visit us on the webhttp//newliteracies.uconn.ed
u
About PowerShow.com