Jason Turner

1
Jason Turner Manager, Professional
Services MetaMetrics, Inc.
2
Writing

• Problem
• Student writing is not professional writing
  Students do not adhere to standard conventions in
  first draft writing
• Solution
• Develop a convention independent analyzer Need
  new proxies for the semantic and syntactic
  variables

3
Six Key Questions

• Do we have measurement of something in the human
  rating case? In the machine rating case?
• Are human ratings and machine ratings measuring
  the same construct?
• Is the unit of measurement the same for human
  ratings and machine ratings?
• Is the linking function constant over
  subpopulations?
• What is the something that is in common between
  human ratings and machine ratings of student
  essays?
• How much uncertainty exists in measuring writing
  ability?

4
Corinth Design

• Data Collection
• 589 students
• Grades 4, 6, 8, 10, 12
• Each student writes to 6 prompts
• Each essay is read by 4 raters (NAEP Rubrics)
• Two reading measures for each student

5
What is the something that is in common
between human ratings and machine ratings of
student essays?

• The machine equation (i.e. Lexile writing
  analyzer) is comprised of a set of proxies for 1)
  semantic store 2) syntactic store, and 3)
  convention knowledge. Because key variables in
  the equation are proxies rather than direct
  causes of student writing performance we have
  followed industry practice and black boxed the
  scoring equation.
• Direct testing of convention knowledge is done
  outside of essay writing in an instructional
  activity.

6
Prompt Describe a favorite object (4AI)
Day 2
My Picture Book My picture book is speaical of
it's shape. It is round. My picture book is pink
and blue. It is big. My family is in there. I
got it in Wal-Mart. I would not lose it, because
it is in my heart. My family picture is in
there. I love it. It is my favite book to me. It
have my Uncle Bobby, Uncle Sam, my grandmother,
and my sister Claudette. I got my dad and my
anat. Me and my mom is in there. My sister and
brother. My grandmother is nice my grandfather
is nice. My picture book is my favite book it is
so, so, big. I love everyone in the world. My
picture book is the speacil thing. My picture
book is in my heart it is beatyful if you see it
you look at it for a longtime if you pick it up.
I love it so, so, so much.
Autograph
Transcription
Writing Analyzer
Machine Measure 345 W
Human Ratings
FACETS Measure 320W
7
The Reading Writing Connection
Cross-sectional Reading and Writing Lexile Means
1400
1200
1000
800
Lexile
600
400
Reading/Writing Correlations Disattennuated
for Measurement Error
r .78
200
r .866 r2 .75
0
4
6
8
10
12
Grade
8
In Summary

• Do we have measurement of something in the human
  rating case? Yes In the machine rating case? Yes
• Are human ratings and machine ratings measuring
  the same construct? Yes
• Is the unit of measurement the same for human
  ratings and machine ratings? It Is Now!
• Is the linking function constant over
  subpopulations? Yes.
• What is the something that is in common between
  human ratings and machine ratings of student
  essays? Semantic and Syntactic Store
• How much uncertainty exists in measuring writing
  ability? About the same as for reading

9
My Writing Web

• Bringing the Lexile Framework for Writing to Life
  in the Classroom

10
Using the Lexile Framework for Writing to
Characterize a Writers Ability

• An estimate of the semantic and syntactic store a
  student can produce using a device
• Convention ability (capitalization, grammar,
  punctuation, spelling)
• Device Fluency

11

• MyWritingWeb

12
Your Turn

• Log in to http//demo.mywritingweb.com
• Log in as a student
• student1/student
• student2/student
• student3/student
• student4/student
• student5/student
• School Lincoln Middle School

13
Your Turn

• Select Essay Writing.
• Explore the essay prompts
• Select a Paragraph Edit and complete the edit.
• Select Keyboarding and take the Typing Test.
• Play the Typing Game.

14
Your Turn

• Select Portfolio.
• Explore the various components of the student
  Portfolio.

15
Your Turn

• Log in as a teacher
• teacher1/teacher (mbell, period 003)
• teacher2/teacher (scannon, period 001)
• teacher3/teacher (astafford, period 001)
• teacher4/teacher (lgentry, period 001)
• teacher5/teacher (jmcmahon, period 001)
• School Lincoln Middle School

16
Your Turn

• Select Class Detail
• Select a student and browse the Essays, Paragraph
  Edits, and Keyboarding activities.
• Write a comment and complete a rubric for a
  student essay.
• Remember that multiple classes may be selected
  from the drop-down menu.

17
Your Turn

• Select Scoring Assistant
• Flagged Essays select a date.
• Does that date range have flagged essays to
  approve or disapprove?
• Are there essays to score?
• Select Rostering
• This is where you can add students.
• This is where you change passwords.

18
Your Turn

• Select Reporting
• View Student Login report export data
• View Usage Report change date range, click go
• View Lexile Writer Measure Growth report change
  date range
• View Lexile Writer Measure Growth by Student
  report change date range

19
Your Turn

• Select Reporting (cont)
• Select Convention Change by Class report change
  date range
• Select Keyboarding Change by Class report
• Select Grouping Report by Writer Measure change
  date range, resort by Lexile Writer Measure,
  resort by student
• Select Grouping Report by Convention Scores

20
Whats Next?

• Implementation
• Each student should write a minimum of two (2)
  essays
• per month within MyWritingWeb.
• Each participating teacher should rate a minimum
  of one
• (1) essay per month.
• Each participating student should complete a
  minimum
• of six (6) paragraph edits per month.
• Each student should complete three (3) typing
  tests for
• the 09-10 school year (one in the beginning, one
  in the
• middle, and one near the end ).

21
Whats Next

• The Site Administrator will handle the
• following
• Adding classes
• Adding teachers
• Adding students
• Updating rosters
• Providing teachers with passwords
• Conferring with technical contact at each site

22
Whats Next?

• Teachers will handle the following
• Student training
• Providing forgotten passwords to students

23
Whats Next?

• Logistical Discussion
• Which students will use MyWritingWeb?
• Where will students use MyWritingWeb?
• How will students use MyWritingWeb?

24
MyWritingWeb User Guide

• Consult the User Guide for
• Training questions
• Student side
• Teacher side
• Installation and technical requirements
• Best practices
• Sample implementation plans
• Using the reports
• Using MyWritingWeb to guide instruction

25
Support

• Site Administrator
• Helen Isbell
• hisbell_at_alcorn.k12.ms.us
• Customer Support
• support_at_mywritingweb.com

26
Questions?
27
MyWritingWeb Administrator

• Administrative Process
• CSV file for each school (teacher/student)
  import into MyWritingWeb
• Distribution of teacher passwords
• Distribution of User Guide
• Confer with IT contact at each site to verify
  proper installation and requirements
• Develop local rostering process (weekly?
  monthly?)
• Usage?