Evaluating Your Technology Initiative How do you know it

1
Evaluating Your Technology InitiativeHow do you
know its working?

• The MassCUE Evaluators Program

2

• Sun Associates
• Program evaluators
• State and local initiatives
• www.sun-associates.com/masscue
• Our take on evaluation
• Links vision to goals to actions to measures
• Is a part of a strategic approach to project
  planning and implementation

3
MassCUE Evaluators

• Professional development that helps districts
  integrate evaluation within projects to...
• Clearly articulate goals and outcomes
• Plan and coordinate activities strategically
• Develop performance measures
• Develop a baseline informed by data -- for
  on-going project implementation and planning

4
How Districts Have Used MCE

• Conducting a district technology audit
• Westford, Foxborough, Winchester, etc.
• Provides the initial work for strategic
  technology planning
• Helps the district clarify existing goals,
  develop new goals, and to properly position new
  initiatives
• Creates a compelling, data-based, case for
  technologys role in teaching and learning

5

• Conducting a program evaluation of a new
  instructional and technology initiative
• Bedford, Reading, Hudson, Danvers, etc.
• Helps the project develop clear project goals and
  performance benchmarks
• Frames the project work within a
  research/evaluation framework that clarifies the
  project impact and outcomes
• Creates the documentation necessary to ensure and
  support the continuation of an initiative

6

• Bedford
• Burlington
• Concord
• Needham
• Reading
• Wayland

• Danvers
• Foxborough
• Hopkinton
• Hudson
• Winchester
• Westford

7
Format

• Hands-on professional development with a
  strategic team of stakeholders
• Curriculum and Instruction
• DTC/Digital Learning Director
• Administrators
• Teachers
• Board/Community/Parents

• Day 1 Framing and Indicators
• Day 2 Developing Data Collection Questions and
  Tools
• Day 3 Data Collection Methodology
• Day 4 Data Analysis, Recommendations, Reporting

8
MCE Helps Articulate and Support Return on
Investment

• Define Return

9
A Strategic Approach to Articulating
Outcomes/Returns

• MCE starts with your vision
• How will this initiative positively impact
  teaching and learning?
• What outcome do you anticipate from this project?
• Project could mean your entire technology plan,
  if you are conducting a district technology
  audit.

10

• The main point of evaluation and hence MCE --
  is to assess the degree to which the performance
  of your initiative is reaching your
  vision/intended outcomes
• Current status documenting the situation before
  your implementation
• Formative assessment documenting the impact
  your project is having

11
Indicators for Success
12
Indicators Drive Data Collection

• The next task is to develop data collection tools
  and to collect data based on the indicators
• Teachers
• Students
• Parents/Community
• Technical

• Observations
• Surveys
• Focus Groups
• Relevant Technical and Cost Information
• Multiple data sources allow for triangulation to
  find answers

13
Analysis

• The collected data is analyzed against the
  indicators
• resulting in evaluation findings.
• Findings respond to the points described in the
  indicators.
• Recommendations usually follow the findings.
• How can you improve performance to come closer to
  meeting the vision/claim?

14
Reporting

• The whole point of the evaluation is to
  communicate the progress formative and
  summative of your initiative/project
• Reporting should be to your school or district
  communitythereby providing a statement about
  value to your stakeholders.

15
What Weve Found

• I wish we could meet like this back in the
  district!
• MCE provides the time, and creates a protected
  space, to really focus on your initiative, its
  outcomes, and and what it takes to create
  success.

16

• The focus groups were an eye-opener for me. It
  gave me much more insight into how people were
  thinking and feeling.
• MCE teams learn data collection techniques, and
  create a substantial collection of data for their
  projects.

17

• I hope to approach other projects using these
  guidelines!
• MCE-developed skills are transferrable and build
  district capacity for evaluation.

18
How to Join

• http//tinyurl.com/masscueval
• www.masscue.org
• www.sun-associates.com/masscue