The Productive Postdoc: Do Working Conditions Affect Outcomes?

1
The Productive PostdocDo Working Conditions
Affect Outcomes?

• Geoff Davis
• Visiting Scholar and Survey Principal
  Investigator
• Sigma Xi, The Scientific Research Society
• gdavis_at_sigmaxi.org

2
Improving the Postdoctoral Experience

• Many calls for changes to the postdoc
• National Academies, AAU, NPA, etc
• Big question What, if anything, works?

3
What Works?

• Changes have costs (money, time)
• Do benefits justify investments?
• What should priorities be?
• What gives the biggest bang for the buck?
• These are empirical questions

4
Our Experiment

• Postdoc administration takes place largely at the
  level of the PI
• Tremendous variability in conditions from lab to
  lab
• Recent, limited introduction of new practices
• Natural experiment
• Ask postdocs about their working conditions
• Ask about how well they are doing
• Find conditions associated with positive outcomes

5
Sigma Xi Postdoc Survey

• Ran a big web survey
• Contacted 22,400 postdocs at 47 institutions
• 40 of all postdocs in US
• Overall response rate 38
• (See tech report for details)

6
Our Sponsor

• The Alfred P. Sloan Foundation

Alfred P. Sloan
Michael Teitelbaum
7
Additional Support

• Werthheim Fellowship, Harvard University

8
Partner Organizations

• National Postdoc Association
• Sciences Next Wave
• NBER/Sloan Scientific Workforce Group

9
Sketch of Our Analysis

• Create measures of inputs (working conditions,
  demographics, etc) and outcomes
• Build linear models to test hypothesis that
  inputs have an impact, gauge magnitude of impact
  (if any)

10
How Do We Determine Success?

• Ideal track people down in 10 years, see what
  they are doing / have done
• Problems
• Very expensive
• Takes 10 years to learn anything
• Driving via the rear view mirror
• Instead, look at immediate proxies for
  longitudinal data

11
Outcomes

• What makes for a good experience?
• No single best measure
• Different people want different things
• Create collection of outcome measures
• Look at impact of inputs on each

12
Subjective Outcome Measures

• Subjective success measure
• Overall satisfaction, preparation for independent
  research, quality of training in research /
  teaching / management
• Advisor relations measure
• How is your advisor doing? Is s/he a mentor?
  How would s/he say you are doing?
• Generate numerical scores by summing Likert
  scored answers

13
Objective Outcome Measures

• Absence of Conflict/Misconduct
• Has postdoc had a conflict with advisor? Has
  s/he seen misconduct in the lab?
• Productivity
• Rate at which papers submitted to peer reviewed
  journals

14
Outcome Measure Distributions
15
Outcome Measure Details

• Correlations all fairly low
• Subjective success and advisor relations 0.45
• Other pairwise correlations all lt 0.2

16
Our Explanatory Variables

• Model outcomes as function of explanatory
  variables
• Field of research
• Institution
• Basic demographic variables
• Sex
• Citizenship
• Minority/Majority Status
• Type of degree (MD vs PhD)
• Total time as a postdoc
• Working Conditions

17
Working Conditions

• How do we measure working conditions?
• Inspiration comes from various calls for changes
• Look at rate of implementation

18
Recommended Changes

• 5 broad classes of recommended changes
• Pay people more
• Fellowships rather than assistantships
• Better benefits
• More structured oversight
• Transferable skills training

19
Measures of Working Conditions

• Salary measure
• log(annual salary), full-time people only
• Independent Funding measure
• Dummy variable, 1 if fellowship, 0 otherwise
• Benefits measure
• Count of different benefits received (health
  insurance, retirement plan, etc)

20
Structured Oversight

• Structured Oversight measure
• Count of administrative measures in place
• Individual development plans
• Formal reviews
• Policies (authorship / misconduct / IP / etc)
• Letters of appointment
• High values lots of structure, low little

21
Training

• Transferable Skills Training measure
• Count of areas in which postdoc reports receiving
  training
• Grant writing, project/lab management, exposure
  to non-academic careers, negotiation, conflict
  resolution, English language, etc
• High values training in lots of areas
• Low values no training in lots of areas

22
Working Conditions Distributions
23
Working Conditions Details

• Again, correlations all fairly low
• Structured oversight and skills training 0.30
• Other pairwise correlations all lt 0.15

24
What Has Biggest Impact?

• Who is most satisfied, most productive, etc?
• People with
• Independent funding?
• High salaries?
• Lots of benefits?
• Lots of structured oversight?
• Lots of types of transferable training?

25
Simple Analysis

• Crude analysis compare satisfaction,
  productivity, etc for people in appointments with
  
• Fellowships / other funding
• High / low salaries
• High / low benefits
• High / low structure
• High / low training

26
Independent Funding
Fellowship Other
satisfied 74 70
Advisor grade (0F, 4A) 3.0 3.1
reporting conflicts 14 14
Papers submitted / year 1.1 1.2
27
Salary
Highest 25 Lowest 25
satisfied 71 68
Advisor grade (0F, 4A) 3.0 3.1
reporting conflicts 16 13
Papers submitted / year 1.2 1.2
28
Benefits
Highest 25 Lowest 25
satisfied 76 62
Advisor grade (0F, 4A) 3.2 2.9
reporting conflicts 11 18
Papers submitted / year 1.3 1.2
29
Structured Oversight
High structure Low structure
satisfied 80 60
Advisor grade (0F, 4A) 3.4 2.7
reporting conflicts 9 21
Papers submitted / year 1.4 1.0
30
Transferable Skills Training
High training Low training
satisfied 83 56
Advisor grade (0F, 4A) 3.4 2.7
reporting conflicts 10 17
Papers submitted / year 1.3 1.1
31
Regression Coefficients
32
Take Home Message 1

• Structured oversight and transferable skills
  training make a big difference

33
Causality?

• We have correlation. Is there causation?
• Psych literature gives reasons to believe in
  causation
• Alternative explanations
• Structure and training attract people who are
  intrinsically more satisfied / productive /
  successful
• Structure / training correlate with some other
  unobserved factor
• Advisors are effective managers / have more
  resources
• Postdocs take more initiative / are better
  organized / etc

34
Causality?

• 2 classes of explanation
• Structure/training attract intrinsically more
  productive people
• Structure/training directly cause productivity or
  are indicators for some causal mechanism
• (Some combination of 1 2 also possible)
• Should be able to differentiate between 1 2 by
  looking at people with multiple appointments

35
Intrinsic vs. Time-Localized
36
Causality?

• Add in terms that allow for change in slope of
  papers(t) curve starting at beginning of most
  recent postdoc
• Equivalent to adding interactions with ratio
  (months in current postdoc / total months as
  postdoc) to regression model
• Training appears to have a time-localized effect
• Other inputs ambiguous

37
Dont Pay Postdocs?

• Not saying postdocs shouldnt be paid!
• Hard to attract US students to science if you
  dont pay them
• Maslows hierarchy of needs
• Must meet basic physical security needs first
• Living wage, basic benefits
• More nuanced interpretation of data beyond a
  certain threshold, structure and training matter
  more than compensation
• Institutional postdoc tax to support service
  provision?

38
More Details

• Look at individual components of structure and
  training measure
• What specific measures have the greatest impact?

39
Impact

• One measure appears to have significant impact
  all 4 outcomes
• Research / career plans
• Written plans
• Plans that spell out what both postdoc and PI
  will do
• Advocated by FASEB, National Academies

40
Plans

• Compare those with such a plan to those without
• Much less likely (40) to be dissatisfied
• Much less likely (30) to have conflicts
• After controlling for field, institution,
  demographics
• Submitted 14 more papers for publication

41
Why?

• Plans
• Expectation setting device
• Postdocs without plans were much more likely to
  report PI had not lived up to expectations
• Contract
• Research shows that people are more likely to
  live up to explicit (esp. written) commitments
• Forces postdocs to take responsibility for their
  careers early
• More time to take advantage of training
  opportunities
• Time management device
• Mechanism for focusing effort

42
Take Home Message 2

• Individual development plans make a big difference

43
Additional Measures

• Several other measures show concrete benefits
• Teaching experience
• Exposure to non-academic careers
• Training in proposal writing
• Training in project management
• Training in ethics

44
Policy Implications

• For postdocs, more effective to invest additional
  dollars in management than in salaries
• Management at all levels
• Infrastructure for institutional oversight /
  training
• Management training for PIs
• Management training for postdocs

45
Further information

• More information at
• http//postdoc.sigmaxi.org
• Workshop (with NPA) in January 2006
• Contacts
• Geoff Davis, PI, gdavis_at_sigmaxi.org
• Jenny Zilaro, Project Manager,
  jzilaro_at_sigmaxi.org

46
Extra Material
47
End Products

• Sigma Xi
• Highlights in May/June issue of American
  Scientist
• Tech reports (2 out now, more to come)
• Scholarly paper this fall
• NPA Analyses of various topics
• NBER SEWP
• Workshop in January 2006

48
Aside Postdoc Definition

• Half a dozen different definitions
• AAMC, AAU, FASEB, NAS, NSF
• BUT if you read and compare them, they all say
  the same thing
• Only substantive difference is that FASEB
  includes narrow subset of clinical fellows
• (We excluded them from this analysis)
• Most people dont fully satisfy definition anyway

49
Postdoc Definition

• The appointee has a PhD or equivalent degree,
• the degree was received recently,
• the appointment is temporary,
• the purpose of the appointment is training for a
  research career,
• the appointment involves substantially full-time
  research or scholarship,
• the appointee is expected to publish the results
  of his or her research, and
• the appointee works under the supervision of a
  senior scholar or a department in a university or
  research institution.

50
Survey Non-Response

• 30-second summary of non-response analysis
• Non-citizens and African Americans appear to be
  slightly under-represented
• No evidence of bias based on level of
  satisfaction (respondents not overly disgruntled)

51
Survey Non-Response

• Survey respondents atypical in one important way
• Participating institutions all had PDO, PDA, or
  administrator interested in postdoc affairs
• Participating institutions probably better off
  than average

52
Salaries

• Median salary 38,000
• Up from 28,000 in 1995

53
Inflation

• A 10 increase above inflation since 1995
• (28,000 in 1995 34,700 in 2004)
• NIH budget doubled over the same period(in
  inflation-adjusted dollars)

54
Experience

• Salaries increase at about 2.9 per year of
  experience

55
Field

• Overall average 39,300
• Average salary in most common fields ranges from
  37,500 to 40,000
• Higher
• Electrical engineering (45,000)
• Physics (42,600)
• Oncology (41,400)
• Materials science (41,200)
• Lower
• Ecology (35,600)

56
Institution Type

• Govt labs pay 20 more than average
• Public universities pay 9 less than average

57
Taxes

• Tax loophole some postdocs dont have to pay
  FICA (7.65 of income)
• 23 benefit
• New IRS rules affect this
• Tax penalty some postdocs pay extra
  self-employment tax (also 7.65 of income)
• 12 pay
• Independent contractor status carries hidden tax
  penalty!
• Potential 6,000 impact on salary

58
Part-time

• 3 report part-time status
• Average hours worked previous week 45

59
Hours

• 51 hours/week median
• Postdoc hourly wage 14.90

60
Hours

• 51 hours/week median
• Postdoc hourly wages 14.90/hour
• Harvard janitors 14.00/hour

61
Foreign Postdocs

• International Men and Womenof Mystery

62
Basic Demographics

• Citizenship
• Citizens 40
• Permanent residents 6
• Temporary visa holders 54
• PhD
• US PhD 53
• Non-US 47

63
Non-US PhDs

• Where PhD earned
• Almost 80 of postdocs on temporary visas earned
  their PhDs outside the US
• Non-US PhDs invisible in NSF stats

All US citizens (41) Permanent residents (6) Temporary (53)
US 53 97 51 21
Elsewhere 47 3 49 79
64
Non-US PhDs

• Where non-US PhDs were earned
• Country of citizenship 86
• Different country, same continent 7
• Different continent 7

65
Temporary Visa Holders
Citizenship
China 24
India 11
Germany 6
South Korea 6
Japan 6
Canada 5
France 5
United Kingdom 4
Spain 3
Italy 3

Top 10 73
Source of PhD
China 18
India 10
Japan 8
UK 8
Germany 8
France 6
Canada 5
South Korea 4
Israel 3
Spain 3

Top 10 73
66
Non-US Postdocs and PhDs

• China and India dominate
• Market share of postdocs comparable to share of
  doctorates (China 23, India 10)
• Next largest LDC is Argentina, 16 for both
  citizenship and PhDs, with 1 of each

67
Temporary Visa Holders by Field
Electrical engineering 72
Physics 67
Chemistry 61
Molecular biology 58
Biochemistry 57
Cell biology 57
Earth sciences 52
Ecology 36
Psychology 21
68
Broad Field
Temporary visas Non-US PhDs
Life/health sciences 52 47
Physical sciences / engineering 63 44
Social sciences 23 18
69
Other Characteristics

• US postdocs
• 49 men/51 women
• 69 married
• 33 have children
• Median age 33

• International postdocs
• 65 men/35 women
• 69 married
• 35 have children
• Median age 33

70
Other Characteristics

• One notable difference for married postdocs
• US postdocs 15 have non-working spouse
• Non-citizen postdocs 44 have non-working spouse
• Some visas (e.g. H) dont have provision for
  spouse to work

71
Domestic vs International Papers

• International postdocs publish more
• Average peer-reviewed publications as a postdoc
• Citizens/PR 2.6
• Temporary 3.3 (27 more)
• Difference is smaller (.1 papers/year) after we
  control for time as a postdoc, field,
  institution, sex, but statistically significant

72
Domestic vs International Hours

• Non-citizens work longer hours
• Average weekly hours worked
• Citizens/PR 50
• Temporary 52 (4 more)
• Difference is smaller (1.3 hours/week) after we
  control for time as a postdoc, field,
  institution, sex, but still statistically
  significant

73
Domestic vs International Salary

• BUT non-citizens are paid substantially less
• Median annual salary
• Domestic 40,000
• International 37,000 (8 less)
• Domestic postdocs earn 2,200/year more than
  international postdocs after controlling for
  field, institution, sex, time as a postdoc, and
  funding mechanism

74
Domestic vs International Grants

• Citizens write more grant proposals (results
  suggest mostly fellowship applications)
• Grant proposals written while a postdoc
• Citizens 1.6
• Non-citizens 1.1 (31 fewer)
• International postdocs write fewer grant
  proposals even after controlling for field,
  institution, sex

75
Domestic vs International Satisfaction

• Non-citizens report slightly lower levels of
  satisfaction with the postdoc experience
• Average satisfaction (-2 dissatisfied / 2
  satisfied)
• Citizens/PR 0.8
• Temporary 0.6
• Difference disappears when one controls for
  salary, discipline, institution, sex, and time as
  a postdoc

76
Security Problems

• To what extent have US national security
  regulations affected your ability to do the
  following( responding Some or A lot)
• Conduct your research in the US 30
• Travel outside the US to conduct your
  research 40
• Visit your country of citizenship 55
• Re-enter the US after leaving the country 57
• Bring your immediate family members to the
  US 36
• Free-text comments express considerable
  frustration

77
More information

• More information at
• http//postdoc.sigmaxi.org
• Contacts
• Geoff Davis, PI, gdavis_at_sigmaxi.org
• Jenny Zilaro, Project Manager,
  jzilaro_at_sigmaxi.org

78
Survey Responders

• Difficult to obtain ground truth for assessing
  results
• Plan compare results of pilot survey to known
  values for one institution with good records
• Reality survey revealed that the institution in
  question was missing lots of postdocs (10 of
  the local population)

79
Survey Responders

• Fortunately we found an alternative with better
  records
• Differences in response rates consistent with
  levels of variation in a random sample for
• Sex
• Citizenship
• Minority status
• No strong evidence of non-response bias

80
Further Non-response Analysis

• Survey literature propensity to respond is a
  continuous variable
• Early responders high propensity
• Late responders lower
• Non-responders lowest
• Idea is that non-responders are more similar to
  late responders than early responders
• Compare early and late responders. Differences
  suggest potential non-response bias.

81
Non-response Bias?

• Who are missing 66 of postdocs?
• No significant difference between early and late
  responders by
• Sex
• Overall satisfaction
• Significant but small difference by citizenship
  (p 0.04)
• Early responders 49 citizens
• Late responders 45 citizens
• Non-citizen postdocs are probably slightly
  underrepresented

82
Domestic vs International Satisfaction

• Non-citizens report slightly lower levels of
  satisfaction with the postdoc experience
• Average satisfaction (-2 dissatisfied / 2
  satisfied)
• Citizens/PR 0.8
• Temporary 0.6
• Difference disappears when one controls for
  salary, discipline, institution, sex, and time as
  a postdoc

83
Settlement Interests

• Level of interest (0None, 2High) in settling in
  various regions (ignoring visa issues)

US Europe Asia
US citizens 2.0 0.8 0.2
European citizens 1.4 1.8 0.3
Asian citizens 1.6 1.2 1.3
84
Settlement Interests

• Level of interest (0None, 2High) in settling in
  various regions (ignoring visa issues)

US Europe Asia
US citizen, US PhD 1.97 0.75 0.20
US citizen, non-US PhD 1.67 1.50 0.25
European citizen, US PhD 1.64 1.43 0.21
European citizen, non-US PhD 1.35 1.86 0.28
Asian citizen, US PhD 1.73 1.04 1.33
Asian citizen, non-US PhD 1.58 1.20 1.26