Management's Nightmare or a Dream Come True

1
Management's Nightmare or a Dream Come True

• Dr. Roger P. Quane
• March 09, 2004

2
Introduction what is this and who am I

• Objective Delve into the world of management,
  evaluation and the interaction of each
• Roger P. Quane
• Ph.D. in Education / Evaluation
• MBA Johns Hopkins Univ. (soon)
• 30 years in Management

3
What are we talking about?

• What is evaluation
• Role of Managers in Evaluation activity
• Your Project
• What to Evaluate type of project
• How to evaluate Gut or Paper/Pencil
• Results
• Manager falls on sword! (messy but necessary)
• Sustainability of Evaluation Activity
• Internal vs. External personnel
• Cost of activity
• Where do you stand or are you on your sword?

4
What is Evaluation ?

• Evaluation
• Measuring changes in behavior that result from
  training
• Measuring final results not just changes
• Types of evaluation
• Reaction (Level 1)
• Learning (Level 2)
• Behavior (Level 3)
• Results (Level 4)

5
Manager Evaluation

• Manager is the Leader and responsible for Program
  Success?
• Manager waits for someone else to do evaluation?
• Need to justify your activity
• Take responsibility
• Make calls on type of evaluation
• Honesty versus Survivability

6
Your Project - ?

• Lets Pick a project you are doing
• Must be new or relatively new
• Must be something that is requirements driven
• Must be over 100,000.00 - WOW
• Must have organizational visibility
• Must be needed for success
• You have to have a desire!

7
Evaluate projects of interest

• Not everything should be evaluate
• Pyramid approach to the Levels 1-4 Building not
  stumbling blocks
• Gut Feeling may be all that is needed
  sometimes not!

8
Results

• So the report is finished so what
• Speaks for itself
• Need to sell solution
• Need to sell meaning of solution
• Need to sell .
• Get the point
• If you dont you are DONE!

9
Managers and the Sword

• Isnt this a little too over the edge
• Managers have to believe in the process
• Managers must implement results
• If they dont so what?
• If they do who will want to listen?

10
Sustainability

• Big word does it have application
• Government never cuts things just reorganize
• If it goes away will it ever come back

11
100
75
50
Capabilities
25
0
November
1997
2007
1999
2001
2003
2005
2009
Time
12
Personnel doing work

• Government Personnel
• Biased / Not Current / lack of expertise
• Contractor Personnel
• Biased / Not Current / lack of expertise
• Mix the best of the best
• Get those who are interested/care/want to make a
  difference

13
Cost of evaluation

• Expensive
• Need to really conduct only the things that need
  to be evaluated
• Cost of not doing something

14
Examples

• Classroom evaluations
• Public Information campaigns
• Program evaluations

15
Questions

• Just to get you started
• What do I need to sell and why?
• What is this Sword thing?
• Does it cost a lot to evaluate?
• Does anyone care?
• Who should do evaluation contractor or
  personnel?
• What is your project? Will you do it?