Measurement, Evaluation & ROI

1
Measurement, Evaluation ROI
2006/05/08 ????? ??????? ? ? ??
2
Contents
Overview
1
Evaluating Training Programs 4 Levels
2
Aligning Kirkpatricks 4 Levels with BSC
3
Summary
4
3
Overview

• Measurement, Evaluation, ROI
• Performance Improvement
• Career Planning Talent Management

4
Overview M.E.R.

• SU415 Implementing Training Scorecards Prove
  That Training Pays Lynn Schmidt
• M102 Evaluating Training Programs The Four
  Levels Donald Kirkpatrick
• M105 Justify, Manage, Improve a Learning
  Enterprise With Four Evaluation Events Diederick
  Stoel
• M116 Making Evaluation Work Patti Phillips

5
Overview M.E.R. (contd)

• M204 Making the Business Case Theresa Seagraves
• M216 Demonstrate Training Value Aligning
  Kirkpatrick's Four Levels With Balanced
  Scorecards James Kirkpatrick
• M303 Tomorrow's Evaluation Opportunities
  Diederick Stoel, Jack Phillips, Donald
  Kirkpatrick, Jac Fitz-enz, Robert Brinkerhoff

6
Overview M.E.R. (contd)

• TU104 Using Needs Assessment to Drive Business
  Results, Vanda Vieira
• TU115 Training Impact Evaluations That Senior
  Managers Believe and Use, Robert Brinkerhoff
• TU118 See the Organizational Big Picture An
  Action-Oriented Evaluation Case Study, Deborah
  Stone
• TU204 Evaluation System Design, Rodrigo Lara
  Fernandez

7
Overview M.E.R. (contd)

• TU203 Meaningful Metrics to Measure the Value of
  Knowledge Management Bruce Aaron
• TU214 How to Evaluate "Soft" and "Hard" Data
  Judith Hale
• TU215 Measuring Human Capital Dennis Kravetz
• TU303 A Practitioners Roadmap for Data-Driven
  ROI W. Byerly

8
Overview M.E.R. (contd)

• TU315 Create the Annual Report for Training and
  Development Rachelle Verkamp
• W101 Evaluating Training Programs The Four
  Levels Donald Kirkpatrick
• W114 Take Charge Design Training for Results and
  ROI Ron Stone
• W204 Global Integration of the Learning Function
  ASTD/Raytheon Professional Services Study Jeff
  Lucas

9
Overview M.E.R. (contd)

• W205 Demonstrate Training Value Aligning
  Kirkpatrick's Four Levels With Balanced
  Scorecards James Kirkpatrick
• W310 Leveraging Technology for Practical and
  Scaleable Learning Analytics Jeffrey Berk

10
Evaluating Training Programs 4 Levels
4?? Final Results
Results ??
3?? Change in B.V.
Behavior ??
2?? KSA
Learning ??
1?? CS
Reaction ??
11
Evaluating Training Programs contd
???? ????? ??

• Needs
• Objectives
• Time
• Place
• People
• Leader
• Techniques
• Objectives
• Satisfied
• Evaluated

12
Evaluating Training Programs Why?
????? ?? ??
Performance Improvement
????? ?? ??
HRD Pro. ?? ??
13
Evaluating Training Programs 1??
1 ?? ?? ?? ??

• ??? ????? ??? ????
• ??? ??? ? ??? ????
• ??? ??? ????
• 100 ???? ??? ???
• ??????, ??? ??? ???
• ??? ? ?? ??? ???
• ??? ???? ?? ??? ????

14
Evaluating Training Programs 1?? ?? ?? ??
15
contd

• SA Strongly Agree
• A Agree
• N Neutral
• DA DisAgree
• SD Strongly Disagree

16
Evaluating Training Programs1?? ?? ?? ??
(5x104x53x32x11x1) / (105311)
17
Evaluating Training Programs 2??
2 ?? ?? ?? ??

• ?? ??? ??, ??, ?? ?? ????
• ??? ??? ?? ?? ??? ??
• ??? ?? ?? ??? ??
• 100 ??? ????
• ??????, ????? ????

18
Evaluating Training Programs Scoring Form
19
Evaluating Training Programs Scoring Form
20
Evaluating Training Programs 3??
3 ?? ?? ?? ??

• ?? ??? ??, ??, ?? ?? ????
• ????? ????? 3-6??? ??? ??? ??
• ????? ??? ??, ????, ??? ??? ?? ???? ???? ?????
• 100 ???? ?????
• ??? ??? ????
• ????? ????
• ?? ?? ??? ????

21
Evaluating Training Programs 3??

• ?? ??? ??? ??

• ???? ?? ??
• ??? ??? ??
• ??? ?? ??
• ??? ??
• ??? ?? ??

• Job Climate
• Preventing Discouraging Neutral
  Encouraging - Requiring

22
Evaluating Training Programs 3??
?? ?? ?? 3-6?? ?
23
Evaluating Training Programs 4??
4 ?? ?? ?? ??

• ?? ??? ????
• ??? ??? ?? 6-12??? ??? ??? ??
• ??? ??? ????
• ????? ????
• ?? ?? ??? ????

24
Evaluating Training Programs 4??
4 ?? ?? ?? ??
25
Aligning Kirkpatricks 4 Levels with BSC
26
Aligning Kirkpatricks 4 Levels (contd)
27
Aligning Kirkpatricks 4 Levels (contd)
28
Aligning Kirkpatricks 4 Levels (contd)
29
Aligning Kirkpatricks 4 Levels (contd)
30
Evaluating Training Programs 4 Levels

• ?? ???? ??? ?? ??
• 1. ?? ????? ??? ??
• ??? ?? ??
• ?? ??
• ?? ??? ?? ?? ??
• ???? ??
• 2. ??? ???? ????
• ??
• ??
• ?? (ROI??)
• 3. ??? ????
• ??? ?? ??
• ?? ?????? ??
• ???? ??
• ?? ??
• ??? ??? ??

31

• Thank You !

• chanlee_at_snu.ac.kr

32
Performance Improvement

• 2006/5/10
• ????? ? ? ??

33
Contents
1
Introduction
2
How Top Companies Invest In H.C. (TU110)
3
HPI the Balance Scorecard (TU Performance RM)
4
Q A
34
Click to edit title style

• How do I incorporate my logo to a slide that will
  apply to all the other slides?
• On the View menu, point to Master, and then
  click Slide Master or Notes Master. Change
  images to the one you like, then it will apply to
  all the other slides.

35
HOW TOP COMPANIES INVEST IN HUMAN
CAPITAL Three New Models ?? ??? ?? ??? ????
?? ??? ??? ?? Jac Fitz-enz, Ph.D. Founder
CEO Human Capital Source
36
- AGENDA - Its
All About Human Capital Investment ???? ??? ?? ??
? Working from a HC Planning Model ???? ???????
?? Linking Human Capital Investments To
Operations, Sales and Service ??????? ??,???,
????? ?? Knowing Where to Invest to Solve HC
Problems ?? ?? ??? ?? ??? ???? ???
37
BUSINESS IS PEOPLE
U.S. Companies lose ?? ???? ?? ? ½ their
investors ??? in 1 year ½ their employees ?? in 4
years ½ their customers ?? in 5 years
5 Gain in Customer Retention Can Add Up to 25
Growth in Earnings ???? 5 ???? ?? ?? 25 ??
?? Fred Reichheld, The Loyalty Effect
38
CFOS CATCHING ON
AHA!
92 Believe HC Impacts Customer Service
82 Believe HC Impacts Profitability ???
72 Believe HC Impacts Innovation ??
39

CURRENT STATE
FUNCTIONAL UNITS Design
Production Sales
Customer Development
Distribution Marketing
Service
?? ?? ?? ?? ??
??? ?????
RESOURCE MANAGERS
Human Finance
Information Security Resources
Accounting Technology
Maintenance
?? ?? ??
???? ?? ??
40
HIRING (Blue) NOT COORDINATED WITH BOOKINGS
(Green) MISALIGNMENT COST 1.500,000,000


41
21st CENTURY INTELLIGENCE
If everybody concerned is absolutely clear
about the goals and directives and far purposes
of the organization practically all other
questions then become simple technical questions
of fitting means to ends
Abraham Maslow
Eupsychian Management

42
3 HUMAN CAPITAL MODELS 3 ???? ?? Mental ?? Talen
t ?? Structural ??
43
1. NONLINEAR UN-ALIGNING ??? ???
Relational Capital Customer -
Competitors Vendors - Government
Structural Capital Facilities - Equipment
Processes - Material
Human Capital Skills - Attitudes Compensation -
Culture
Workforce Intelligence Analytics Assessment
Benchmarks
44
CONVERGENCE ???
W O R K F O R C E
?? ??
P L A N N I N G
Leadership Development
Recruitment Management
Retention Performance
?? ??
??? ??
?? ??
45
2. TALENT MODEL ?? ??
C-Level has Balanced Scorecard
Finance has Accounting
Production has Quality
S - M - CS have CRM
Now Human Resources has TMI
TMI
46
HR INVESTMENT RESEARCH HR?? ??
Saratoga Institute HR Metrics
Benchmarks Huselid Becker HR Practices
Productivity/Turnover Watson Wyatt HCM
Consistent Shareholder Value Gallup Managerial
Behavior Employee Performance
Employee Customer Profit
5 2
1
Sears
47
WHAT IS TMI?
Business Intelligence Applied To Human
Resources Management of Human Capital
Staffing
Leadership Development
Workforce Planning
Performance Management
Training Education
Retention
48
NEXT GENERATION ???
2003 Intelligence - TMI
1985-2002 Knowledge - Benchmarks
1980 1985 Information - Metrics
Pre-1980 Data - Feelings
49
TMI FRAMEWORK
Staffing
Retention
Leadership
Operations Sales Service
Planning
Development
Performance
50
HR MANAGEMENT TALENT MANAGEMENT INVESTMENT
HR
HR Internal Efficiency ?? ?? TM
External Effectiveness ?? ??
51
Efficiency vs. Effectiveness
Efficiency Does one approach achieve the
outcomes faster, and is the cost less than the
value of the HR outcomes? Effectiveness Does
one approach result in superior work outcomes,
and is the cost less than the value of the work
outcomes?
52
HR EFFICIENCY MEASURES
Factor Measure Cost Time
Quantity Quality Reaction
Staffing Pay Ben Training
Retention Admin
Develop.
53
TM EFFECTS ON BUSINESS
Factor Service Hiring
Paying Supporting Developing
Retaining
Operations Sales Service
54
VALUE REPORTING CHAIN
???? ?? ???? ??
?? ??
TM Action ? Immediate Result ? Business Effects
Hire . . . . . . Fit Stay Perform . . lt
People Cost gt Margins Pay . . . . . .
Incent Innovate . . . New Product gt Market
Share Support . . Focus Productive . . gt
Output gt Rev Develop . . Skilled
Quality . . . . Retain Cust. lt Marketing
Retain . . . Commit Stay . . . . . . lt T
cost gt Income
55
3. STRUCTURAL MODEL
3.
PREVIEW 3 Capitals as Viewed
1.
2.
External Anticipated Market Changes
Associated Costs
Internal Vision Brand Culture Strategy Plan
Structural Facilities Technology Processes Patent
s
Relational Patients Families Regulators Suppliers
Competitors
Human Executives Managers Professionals Technicia
ns
Key Position Requirements By Year Analysis of
Sources and Methods Executives Managers
Professionals - Technicians
4.
FUTURECAST Future Sources and Methods
Executives Managers Professionals -
Technicians Recruitment Management
Leadership Development
5.
6.
PERFORMANCE MEASUREMENT Measures of Business
Effects
56
4. KEY POSITION REQUIREMENTS
Analysis of Recruitment Sources and Methods ??
??? ?? ??
Workforce Performance Executives Managers Profess
ionals Technicians
Sourcing Media Advertising Agencies Referrals Eve
nts
57
5. FUTURECAST Future Sources and Methods ?? ???
?? Executives Managers Professionals -
Technicians
Recruitment Management Source Analysis
Selection Assessment - Fit - Capability
- Tenure - Lost Time Vacancy Projections

Leadership Development Requirements
Analysis - Job Assignments -
Projects - Training - Mentoring
- Education Readiness Projections
58
TM LINKAGE
HC INVESTMENTS Cost to Manage Contingent
Percent HR spend per Employee LABOR COSTS
Labor cost as of revenue Labor cost per
employee Benefits cost as of revenue
Benefits cost per employee TM INVESTMENTS
Staffing Leadership Development
Performance Management Workforce Planning
Training Education Retention
COMPANY RESULTS FINANCIALS Revenue Gross
Profit Operating Income Earnings per
Share Return on Assets MARKET POSITION Market
Share Share Growth Rate Customer
Retention OPERATIONS IT Spend RD
Spend Marketing Spend
59
TM LINKAGE

HC INVESTMENTS Cost to Manage Contingent
Percent HR spend per Employee LABOR COSTS
Labor cost as of revenue Labor cost per
employee Benefits cost as of revenue
Benefits cost per employee TM INVESTMENTS
Staffing Leadership Development
Performance Management Workforce Planning
Training Education Retention
COMPANY RESULTS FINANCIALS
Revenue Gross Profit Operating Income
Earnings per Share Return on Assets MARKET
POSITION Market Share Share Growth Rate
Customer Retention OPERATIONS IT Spend RD
Spend Marketing Spend
60
TMI ANALYSIS
RESULTS COMPANY FINANCIALS
Revenue Gross Profit SGA Expense Operating
Income Net Income Earnings per Share Return
on Assets PreTax Profit Margin MARKET
POSITION Market Share Growth Rate Customer
Retention Customer Net Gain OPERATIONS IT Spend
as of Revenue RD Spend as of
Revenue Marketing Spend as of Rev
HC INVESTMENTS Cost to Manage Contingent
Percent HR spend per Employee LABOR COSTS
Labor cost as of revenue Labor cost per
employee Benefits cost as of revenue
Benefits cost per employee TM INVESTMENTS
Staffing Leadership Development
Performance Management Workforce Planning
Training Education Retention
61
EARLY RESULTS
PLANNING LEADERSHIP DEVELOPMENT Big Pharma
Found How to Link P D for Accelerated Leader
Development, Improved Manager Retention STAFFING
Assessment Pays Off in Retail and
Telephone Sales (25 time to standard and 6)
in hourly sales) and Retention (40) PERFORMANCE
MEASUREMENT Logistics Service Uses
Restructuring, Pay Redesign and Employee Survey
to Support 20 Gain in Profits
62
EARLY RESULTS
Training Development Medical Center Linked
Training to Competency Models for All Staff
and Measured Results Against Financials and
Patient Care Retention International Supply
Chain Manager Studied Turnover in Warehouses and
Found People Join, Stay and Leave for Different
Reasons Dropped Turnover 50 Reducing Operating
Expense Productivity Distribution Center
Pickers Turnover 3x Norm - Cost 2.25M.
Applied Training Comp Saving 1.85M in
Productivity Loss due to Turnover
63
HPI the Balance Scorecard (TU Performance RM)
1/4
64
HPI the Balance Scorecard 2/4

• Line of Sight problem, gaps, root causes,
  solutions, and evaluation
• Problem
• (who) sales associates in the
• (where) central region are
• (what) 360,000 below Product Y sales goals in
  the
• (when) 1st quarter potentially impacting ABC
  Companys
• (worth) net profitability.

65
HPI the Balance Scorecard 3/4

• Gaps Individual Team__________
• Quantitative
• Expected 3.0 sales/person/day3K 300
  sales assoc900K
• Actual 1.7 1.8K 360K
• Gap 1.3 1.2K 540K
• Qualitative
• Expected mystery shop9.0/10.0 Custom
  er Sat 4.7/5.0
• Actual 6.5 3.2
• Gap 2.5 1.5

66
HPI the Balance Scorecard 4/4

• Root Causes
• Product push for higher net value of sale rather
  than based on customer needs
• Workload and staffing number of customer
  accounts assigned to fewer sales associates
• Lack of skills and knowledge on needs based sales
  and service
• Recommend Solutions
• Implement sales and service process aligned with
  selection and performance management
• Workload, workflow, structure, and staffing
  models analysis and revision
• Targeted needs based sales and service skills
  training and job aids followed by OJT coaching

67
Q A Dr. Chan Lee
chanlee_at_snu.ac.kr Tel. (02) 880 - 4838
68
Career Planning Talent Management

• 2006/5/11 ????? ? ? ??

69
Career Planning Talent Management
70
??
71
BZ Case for CD

• Attract a world-class team
• Retain the stars
• Match workforce skills to bz strategy
• Communicate your brand
• Provide employee voice power
• Acknowledge managerial legacy

• ?? ??? ?? ??
• ??? ??
• ?? ?? ?? ?? ??
• ???? ??
• ??? ?? ?? ??
• ?? ??? ??

72
Bldg a Dev. Culture

• The manager provides the Support
• The Employee provides the Spark
• The Organization provides the System

• ???? ??? ??
• ??? ???? ??
• ??? ???? ??

73
Bldg a Dev. Culture

• The manager provides the Support
• The Employee provides the Spark
• The Organization provides the System

• ???? ??? ??
• ??? ???? ??
• ??? ???? ??

74
Must-Dos for the Dev.-Minded Leader

• Listen pay attention to abilities and talents
• Level Provide feedback Plus
• Look Ahead Talk realistically about the future
• Leverage Discuss options
• Link Suggest learning opportunities

• ????
• ?????
• ?? ??
• ????
• ????

75
Listen

• Skills / Competence
• Interests / Enjoyment
• Values / Commitment

• ?? / ??
• ?? / ??
• ?? / ??

76
Level

• Observation
• Judgment
• Label
• Reputation

• ??
• ??
• ??
• ??

77
Look Ahead

• Your Job
• Your Org.
• Vs.
• Your Profession
• Your Industry

• ??
• ??
• Vs.
• ??
• ??

Global Economy
78
Leverage

• Right Person
• Right Place
• Right Time

• ?? (??)
• ?? (??)
• ?? (??)

79
Link

• An Observation
• An Assignment
• A Course
• A Mentor/Coach
• A Peer Group

• ??
• ??
• ??
• ??
• ??

80
Are Managers Prepared to provide Support?

• To listen hard so they understand employee
  capabilities?
• To level about ways employee can increase their
  effectiveness?
• To jointly look ahead to see how the direction of
  the company and their converge?
• To help them leverage the choices available to
  them?
• To link them to the appropriate resources?

81
Are Employees Prepared to provide the Spark?

• To Say these are my talents
• To Ask are we in agreement about my
  capabilities?
• To Proposeheres where I fit in this
  organizations future
• To Offerthese career options excite me
• To Recommendheres what I want to learn

82
Is your Organization Prepared to Provide the
System?

• Competency profiling?
• Performance Management?
• Strategic staffing?
• Succession planning?
• Training and development?

83
The Retention PunchlineDevelopment Me or Im
History

• Will you utilize the full rage of my talents?
• Will you provide on-going performance feedback?
• Will you clue me into the future direction of
  this organization?
• Will you explore multiple avenues for my career
  growth?
• Will you assure my continued learning?

84
Q A
85
ASTD 2006 Now, Lets Think Bigger!

• ?? ?? ??????
• ?? ?? ??????

• ?? ??