Human Resources Outcome Indicators: Approaches and Prospects - PowerPoint PPT Presentation


PPT – Human Resources Outcome Indicators: Approaches and Prospects PowerPoint presentation | free to download - id: 93b0b-MGJlY


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Human Resources Outcome Indicators: Approaches and Prospects


... of people 'reached,' in some fashion, by NSF funds or, in the ... For individuals, education and career histories. Human Resources Outcome Indicators ... – PowerPoint PPT presentation

Number of Views:66
Avg rating:3.0/5.0
Slides: 18
Provided by: lisa295


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Human Resources Outcome Indicators: Approaches and Prospects

Human Resources Outcome Indicators Approaches
and Prospects
  • Colloquium on Measuring
  • the Impacts of Science
  • June 17-18, 2004
  • Montreal, Canada

Rolf Lehming ltrlehming_at_nsf.govgt Division of
Science Resources Statistics, National Science
Foundation NOTE The views
expressed are those of the author and do not
represent the position of the National Science
Foundation. No endorsement by the National
Science Foundation should be inferred.
Two types of research output
  • New knowledge in the form of research results,
    methodological advances, conceptual breakthroughs
  • People who have gained substantive, conceptual,
    methodological and tacit knowledge along with a
    perspective on science and research
  • Both can theoretically be traced and analyzed
  • Bibliometrics as knowledge output proxy
  • Graduate enrollment, advanced degrees, and
    postdocs as proxies

Human Resources Outcome Indicators
Longer-term research outcomes
  • Individuals research experiences and encounters
    with science may yield outcomes like
  • An appreciation of the scientific perspective
  • An openness to empirical inquiry
  • The acquisition of specific skills
  • The sum of these individual outcomes may yield
  • A technically sophisticated population and
  • An educated citizenry able to deal with the
    science and technology components of public
    policy issues

Human Resources Outcome Indicators
NSF Strategic Goals
  • People as one of NSFs four Strategic
    Performance Goals
  • (along with Ideas, Tools and Administration
    and Management)
  • The strategic People outcome goal is to develop
    a workforce of scientists and engineers that is
  • Diverse, internationally competitive, and
    globally engaged
  • And to develop well-prepared scientifically
    literate? citizens
  • The plan includes annual performance goals and
    prospective and retrospective reporting foci

Human Resources Outcome Indicators
NSF Administrative Data
Number of people involved in NSFs activities

FY2002 FY2003
FY2004 _____________________Actual
Estimated Estimated Senior researchers
28,960 29,820
30,590 Other professionals 12,060
12,180 12,640 Postdoctoral associates
5,740 6,060
6,170 Graduate students
26,170 27,440
28,690 Undergraduate students 34,250
32,710 36,350 K-12 students
11,460 13,640
14,640 K-12 teachers
84,710 85,460
86,830 Total number of people 203,350
207,310 215,910
Human Resources Outcome Indicators
NSF Administrative Data
  • These figures are compiled from proposal budgets
    estimating number and types of people involved
  • Limited personal information is captured about
    the principal investigator(s) only
  • NSF asks principal investigators to fill in, as
    part of project closeout, a matrix of project
    participants by type of position and demographic
  • Submission is voluntary and compliance variable
  • Figures do not include the almost 200,000
    reviewers of the 30,000-plus proposals NSF
    receives annually

Human Resources Outcome Indicators
NSF Administrative Data
  • The data provide approximate estimates of the
    number of people reached, in some fashion, by
    NSF funds or, in the case of peer reviewers, by
    NSF programs
  • These initial project estimates
  • are not exact
  • cannot be aggregated over time (the same persons
    may be counted in 2, 3, or more adjacent years)
  • in their current form invite no further
    analytical use

Human Resources Outcome Indicators
NSFs Strategic People Goal
  • NSF discussion about requiring investigators to
    supply demographic and contact information for
    project participants to allow use for
  • Tracking of careers
  • Studies of (scientific) productivity
  • Program assessments

Human Resources Outcome Indicators
NSFs Strategic People Goal
  • NSF declined for reasons including.(???)
  • Concern about PI response burden
  • Feasibility and legality (informed consent,
    university role)
  • Rules for establishing and maintaining Federal
    Systems of Records
  • Confidentiality and privacy concerns

Human Resources Outcome Indicators
NSFs Strategic People Goal
  • NSF emphasizes HR output and outcomes in proposal
  • Proposals must describe … the broader impacts
    resulting from the proposed activities …
  • Integration of research and education
  • Participation of underrepresented groups
  • Infrastructure enhancement such as facilities,
    instrumentation, networks, partnerships
  • Enhancement of scientific and technological
  • Potential benefits of the activity to society at

Human Resources Outcome Indicators
NSFs Strategic People Goal
  • NSF also emphasizes the importance of HR output
    and outcomes in reviewer instructions
  • Reviewers are instructed to address both merit
    review criteria
  • Intellectual merit of the proposed activity
  • Broader impacts i.e., as described before
  • Response to the broader-impacts criterion by
    proposal submitters and reviewers has been uneven
  • NSF has begun returning some proposals
  • NSF has repeatedly urged reviewers to address the
    second criterion

Human Resources Outcome Indicators
NSF Surveys
  • Three biennial surveys of individuals who have at
    least a bachelors degree in science,
    engineering, or math or work in an SE occupation
  • National Survey of College Graduates samples
    people with a bachelors or higher degree drawn
    from decennial U.S. Census, then followed up
  • National Survey of Recent College Graduates
    samples those with newly awarded SE degrees
  • Survey of Doctorate Recipients draws a sample
    from a census of recipients of U.S. SE
  • Integrated into the Scientists and Engineers
    Statistical Data System (SESTAT)

Human Resources Outcome Indicators
NSF Surveys
  • Extensive demographic, educational, work-related,
    and personal information
  • Permits some outcome-related analyses including
  • Expansion of SE occupations in the labor force
    (4 times faster growth than total LF over 2
  • Tracking of SE degree holders into occupations
    not classified as science or engineering
  • High degree of relatedness of job, training
  • May indicate growing technical requirements of
    the economy
  • For individuals, education and career histories

Human Resources Outcome Indicators
NSF Surveys
  • Outcome type variables but only inferred link to
    input, output items
  • Other limitations include
  • Expanded NAICS coding but aggregating up in
    analyses and barring of firm-level reporting
  • Limited ability to do productivity studies
    because of the paucity of output indicators
  • Matching with data from external sources faces
    serious difficulties
  • No ability to address citizenship competency
  • Separate survey on public attitudes

Human Resources Outcome Indicators
Other NSF Survey
  • Survey of Industrial Research and Development
  • 25,000 firms, 2 personnel items
  • total payroll employment
  • Estimated FTE number of scientists and engineers
    working on RD
  • Excludes social sciences and psychology
  • Analyses of the distribution of FTE researchers
    by firm sector and characteristics including RD
  • In principle, trends in use of researchers across
  • No human resources detail available

Human Resources Outcome Indicators
Non-NSF Surveys
  • The Bureau of Labor Statistics Occupational
    Employment Survey
  • Tracking of trends in SE occupations at industry
    level in the framework of total employment
  • Potential to develop comparable state-level
    estimates (technical issues to be resolved)
  • Potential outcome indicators on changing size,
    composition, location of high-technology segments
    of the economy
  • BLS defines high-tech industries by SE jobs
    percentage at least twice that of the average for
    all industries.

Human Resources Outcome Indicators
Bottom Line and a Note
  • These surveys provide some means of estimating
    the size and some composition of the SE
  • All surveys but Industrial RD Survey have only
    implied link of outcomes to inputs and outputs
    (except education inputs)
  • Limited ability to conduct in-depth panel
    analyses of career patterns in the context of the
    total labor market
  • Limited ability to conduct productivity studies 
  • A note on intergenerational social mobility
  • Valued mark of an open society
  • Education traditionally a strong factor
  • Role of science?

Human Resources Outcome Indicators