Using Multiple Sources of Data to Measure Success

1
Using Multiple Sources of Datato Measure
Success
December 15, 2009
School Improvement Webinar Series
www.acteonline.org/multimedia.aspx
2
Your Moderator, Host and Presenter

• Diana
• Rogers
• Regional Coordinator
• HSTW NE Ohio Region

• Catherine
• Imperatore
• Electronic Media Manager
• ACTE

• Mike
• Ross
• HSTW/MMGW School Improvement Coach
• HSTW SW Ohio Region

3
Sound Check

• Can everyone hear me?
• If not, please check the volume on your computer

4
Questions

• To ask about the content type a question in the
  QA panel and send to All Panelists.
• Questions will be
• addressed at the end of
• the presentation
• For technical problems or any other questions,
  type in the Chat panel and send to the Host.

5
Replay or Register for Webinars

• Replay/register www.acteonline.org/multimedia.aspx
  
• Invite your colleagues to register
• Complete webinar survey
• Graduate credit available

6
Why do we need multiple sources?

• Single sources of data dont provide us with the
  complete picture!
• Reliance on a single data source is incomplete!
• We need multiple sources of data to more
  accurately identify causes of the problem and to
  find more appropriate solutions.

7
Why do we sometimes have tunnel vision when
looking at data?

• Emphasis on paper and pencil testing for
  accountability
• Reaching the standard, making the cut-off
  score, etc. may blur our focus
• Absence of a data cultureand resources and time
  for creating one!

8

• Data are critical for all parts of the
  improvement process from initial needs
  assessments to monitoring progress and evaluating
  outcomes/results

9
A Data Continuum Where on this continuum is
your school?
10
Delving into Data

• School culture is everything!
• Data are accessible and usable.
• Data are viewed as an invaluable resource for
  improvement.
• Data serve as a basis for inquiry, reflective
  dialogue, problem-solving, and decision-making.
• Discrepant data provide the teachable moment in
  the school improvement process.

11
Bernhardts Suggested Use of Data

• Replace hunches with facts concerning what
  changes are needed
• Identify the root causes of problems so we can
  then solve the problems
• Assess needs to target our services on important
  issues
• Know if goals are being accomplished
• Determine if we are walking our talk

From the work of Victoria L. Bernhardt, Data
Analysis for Comprehensive School Improvement,
1998, Eye on Education.
12
Quantitative versus Qualitative

• Is one form of data better than the other?
• What is the purpose for using the data?
• Both forms can be effectively utilized!

13
Categories of Data
Measures of student learning help us
understand how students are performing and what
students know as a result of instruction.
Student Learning
From the work of Victoria L. Bernhardt, Data
Analysis for Comprehensive School Improvement,
1998, Eye on Education.
14
Categories of Data
School Processes
programs, practices, and instructional
strategiesthat produce school and classroom
results.
From the work of Victoria L. Bernhardt, Data
Analysis for Comprehensive School Improvement,
1998, Eye on Education.
15
Categories of Data
Perceptions
A particular view, judgment, or appraisal formed
in the Mind about a particular matter...a belief
stronger than impression And less strong than
positive knowledge.
From the work of Victoria L. Bernhardt, Data
Analysis for Comprehensive School Improvement,
1998, Eye on Education.
16
Categories of Data
Statistical characteristics of human
populationsbuilds the context of the schoolfor
which change is planned and takes place.
Demographics
From the work of Victoria L. Bernhardt, Data
Analysis for Comprehensive School Improvement,
1998, Eye on Education.
17
Categorizing Data

• D Demographic
• P Perception
• SP School Process
• SL Student Learning

From the work of Victoria L. Bernhardt, Data
Analysis for Comprehensive School Improvement,
1998, Eye on Education.
18
What category?

• An additional hour of extra help is available
  after school this year.
• Only 78 of boys scored at the proficient level
  in reading as compared to 92 of girls.
• For two years, making inferences using
  non-fiction passages has been our lowest
  performing area.

19
Poll Activity

• How would you categorize the following data
• D Demographic, P Perception, SP School
  Process, SL Student Learning
• Our math scores have declined during the past
  three years.
• Students classified as low socioeconomic status
  score less
• well on constructed response test items.
• Last week, problem-based learning was observed
  in about
• 35 of the classrooms.
• None of the students in the focus group
  discussion
• mentioned project work as a quality
  learning experience.
• Over 90 of parents said they were satisfied
  with the new
• grading scale.

20
How may more than one category interact for a
better data analysis?

• How do students who regularly use the writing lab
  perform on their senior projects compared to
  those who dont?
• Does the reading performance of males increase
  with the number of hours intervention they have
  experienced?
• Do the parents of underperforming students have
  confidence in the schools enhanced intervention
  programs?

21
Categories of Data - Interactions
From the work of Victoria L. Bernhardt, Data
Analysis for Comprehensive School Improvement,
1998, Eye on Education.
22
How Data Can Be Used

• To guide improvement efforts!
• Provide students with feedback on their
  performance
• Gain common understanding of what quality
  performance is and how close we are to achieving
  it
• Measure program effectiveness
• To understand if what we are doing is making a
  difference

23
More Ways Data Can Be Used

• Make sure students dont fall through the
  cracks
• Know which programs are getting the results we
  want
• To get to the real causes of problems
• To guide curriculum development
• Promote accountability
• Meet state and federal requirements
• (all from V. Bernhardt)

From the work of Victoria L. Bernhardt, Data
Analysis for Comprehensive School Improvement,
1998, Eye on Education.
24
An Overload of Data???

• OGT score rostersfrequency distributionsextra-he
  lp options and sign-in sheetsquarterly failure
  reports IPDP ICP IRNdisaggregated OGT percent
  passingsubscale analysistest item
  analysisStudent AssessmentTeacher SurveyAnnual
  Progress ReportOhio School report card student
  grades scheduling demand similar district
  comparison course enrollmentcurriculum mapping
  attendance by dateattendance by day of week
  tardies disciplinefree and reduced
  lunchdisability profilecategorical reportIEP
  achievement...parent income leveldrop-out
  rategraduation ratecompleter statusnational
  certification health recordsmobilityfamilies
  on public assistancebooks readcredit
  deficienciesteacher licensure dataCCIPprogram
  evaluation reports student survey
  parent/community survey open house
  attendanceparent conference attendancestudent
  participation in extra-curricular activitiessafe
  school surveyequity survey harassment
  complaints ACTSAT Board of Regents remediation
  reports suspensions expulsionsgifted and
  talentedadvanced placementpost-graduate
  follow-up surveypercent students entering
  post-secondary trainingalternative
  programreasons for drop-outlevel of technology

25
Sources of Data from Measuring Performance and
Practices

• State Assessments
• Teacher Assessments
• Course Failure (ninth-grade)
• ACT/SAT Results
• Attendance Rates
• Graduation Rates
• Certification Exam Results
• Post-Secondary Readiness
• Assessing Readiness Practice

26
Sources of Data from Measuring Performance and
Practices

• Instructional Review
• Staff Experience Chart
• Remedial Studies Reports
• Follow-up studies
• Drop-out exit reports
• Master Schedule
• Focus Group Interviews
• Graduate Feedback
• Assessing Practice

27
Compiling Data Sets

• Collect related data appropriate to a particular
  goal, objective or strategy
• Organize data for review
• Develop thoughtful questions for making meaning
• Draw conclusions

28
Example of Data Sets

• Overriding Goal To close achievement gaps, meet
  state and federal accountability requirementsto
  maintain high expectations and extra help.
• What data sets are appropriate for addressing
  this goal?

29
(No Transcript)
30
Elements of Data Sets

• Organize data for review
• How should we best organize the data?

31
(No Transcript)
32
Elements of Data Sets

• Overriding Goal To close achievement gaps, meet
  state and federal accountability requirementsto
  maintain high expectations and extra help.
• What questions will help us to make the data
  meaningful?

33
Elements of Data Sets

• Overriding Goal To close achievement gaps, meet
  state and federal accountability requirementsto
  maintain high expectations and extra help.
• What conclusions may be draw from analyzing our
  data?

34
on being the best professional

• Over everything else, all the tools they have in
  their toolboxes, the difference between
  ineffective and effective teachers is that
  effective teachers reflect on their teaching in
  meaningful ways.
• Ineffective teachers may work hard, but they
  dont move along the continuum of self-reflection
  from unaware, to consciousness, to action, to
  refinement.
• - Pete Hall
• ASCD 2005
• Outstanding Young Educator Awardee

35
Data Helps Us to Reflect

• Effective teachers consciously use data as a
  source of guidance and reflection.
• The use of multiple sources of data provides a
  more accurate and complete picture of performance
  and effectiveness.
• Make time for using data and reflecting on your
  practice.

36
Recommended Resources

• Book
• Victoria L. Bernhardt, Data Analysis for
• Comprehensive School Improvement, 1998,
• Eye on Education.
• Website
• Common Core of Data, National Center for
• Educational Statistics http//nces.ed.gov/ccd/

37
Questions

• To ask about the content type a question in the
  QA panel and send to All Panelists.
• Questions will be
• addressed at this time
• Or an email response will be sent to you after
  the webinar.

38
Question

• Do you have examples of data sets used by schools
  to measure student success?

39
Question

• What professional development is available to
  assist school teams in learning more about using
  multiple sources of data?

40
More Q A

• Questions and responses

41
Contact Information

• If you have questions or would like to learn more
  
• about motivating students to successfully
• complete assessments please contact
• Mike Ross, School Improvement Consultant
• michaelross_at_embarqmail.com

42
Next Webinar in the Series

• Developing a School-wide Literacy Plan
• Paulette Dewey, HSTW/MMGW Coach
• January 19, 2010
• from 1130 1230 ET

43
Thank you for participating!

• Reminders
• Register for future webinars or to view archived
  webinars at www.acteonline.org/multimedia.aspx or
  www.hstwohioregions.org
• After leaving todays webinar
• Please complete the webinar survey.
• If you are interested in graduate credit,
  remember to print a copy of the survey.
• Please click the X to exit the webinar. Have a
  great day!