Peer Review Michael A. Steinmetz, PhD Center for Scientific Review

1
Peer ReviewMichael A. Steinmetz, PhDCenter for
Scientific Review
2
Overview

• NIH Organization
• Dual Review Process
• Grant Receipt and Referral
• The Study Section
• The Review Process
• Grant Writing Tips

3
NIH Organization

• NIH consists of 27 independent institutes and
  centers (ICs)
• Director and Advisory Council
• 24 ICs award grants
• National Cancer Institute (NCI)
• National Institute of Allergy and Infections
  Diseases (NIAID)
• 3 ICs provide services
• Clinical Center
• Center for Scientific Review (CSR)

4
Grant Review is a Dual Process

• Study Section Evaluation
• Study Sections evaluate scientific merit
• CSR reviews 70 of all proposals
• Review Division of ICs review the rest
• Advisory Councils at ICs
• Concur and recommend funding
• May conduct independent review

5
Receipt and Referral

• CSR receives all NIH grant applications
• Most proposals are assigned to a CSR Study
  Section for review
• Each proposal is also assigned to one or more ICs
  for funding consideration

6
Receipt and Referral

• Referral Officers use guidelines to make study
  section and IC assignments
• Study section assignments are reviewed by IRG
  chiefs and SRAs
• Many applications receive dual IC assignments
• ICs can reject or trade proposals with other ICs
• Cover letter can help to direct the process
  toward appropriate assignments

7
CSR Study Sections

• 230 study sections designed to have some degree
  of overlap
• Study sections are clustered into Initial Review
  Groups (IRGs) led by a chief
• Each section is run by a Scientific Review
  Administrator (SRA)
• Each section has about 20 regular members
• CSR web pages give descriptions and rosters

8
The Role of the SRA

• Organizes study section meetings with help of
  Extramural Support Assistants (ESAs)
• Acts as government official to assure fairness in
  review process
• Mediates conflicts of interest between reviewers
  and applicants
• Represents study section review at IC council
  meetings

9
The Role of the SRA

• Appoints all regular members
• Appoints chairman to moderate discussions
• Appoints ad hoc members to insure sufficient
  expertise and adequate manpower
• Assigns reviewers to each application
• Writes a resume of any discussion
• Generates summary statements

10
The Study Section

• Regular members are appointed to 4-year terms
• Chair serves 2-year term
• Review 60-100 grants in each of 3 rounds
• Each member is assigned to 8-10 proposals
• Must have requisite expertise
• Gender, Minority, and Geographical balance
• Members must have peer-reviewed support

11
Study Section Membership

• Recruited and appointed by SRA
• Ideal Candidate
• Associate Professor
• Competitive renewal of grant
• Review experience (12 year limit)
• Annual Slates
• 25 of members replaced each October
• 6 levels of approval

12
Becoming a Member

• Why?
• Chance to have significant influence on the field
• Will drastically change the way you write grants
• How?
• Get to know SRA
• Talk to Program Officer at IC
• Be responsive and committed (e.g. SEPs)
• Be critical but fair in reviews
• Be on-time

13
Pre-meeting Review

• Administrative review
• Reviewer assignments by SRA
• Conflicts of interest
• Same institution
• Mentor/Mentee
• Recent Co-author
• Financial gain
• Long-standing scientific disagreement
• Confidentiality

14
The Review Meeting

• Internet Assisted Review
• Attendance
• Regular and ad-hoc members
• Program officials
• Streamlining
• Lower half of scores
• Unanimous consent required
• No discussion of grant

15
Grant Discussions

• Chair moderates discussion
• Preliminary scores from assigned reviewers
• 1.0 5.0 range
• Reviewer description and comments
• Discussion
• Final scores
• Voting outside the range
• Deferral by SRA

16
Review Criteria

• Significance
• Approach
• Innovation
• Investigator
• Environment
• Overall evaluation

17
Significance

• Does this study address an important problem? If
  the aims of the application are achieved, how
  will scientific knowledge or clinical practice be
  advanced? What will be the effect of these
  studies on the concepts, methods, technologies,
  treatments, services, or preventive
  interventions that drive this field?

18
Approach

• Are the conceptual or clinical framework, design,
  methods, and analyses adequately developed, well
  integrated, well reasoned, and appropriate to the
  aims of the project? Does the applicant
  acknowledge potential problem areas and consider
  alternative tactics?

19
Innovation

• Is the project original and innovative? For
  example Does the project challenge existing
  paradigms or clinical practice address an
  innovative hypothesis or critical barrier to
  progress in the field? Does the project develop
  or employ novel concepts, approaches,
  methodologies, tools, or technologies for this
  area?

20
Investigators

• Are the investigators appropriately trained and
  well suited to carry out this work? Is the work
  proposed appropriate to the experience level of
  the principal investigator and other researchers?
  Does the investigative team bring complementary
  and integrated expertise to the project (if
  applicable)?

21
Environment

• Does the scientific environment in which the work
  will be done contribute to the probability of
  success? Do the proposed studies benefit from
  unique features of the scientific environment, or
  subject populations, or employ useful
  collaborative arrangements? Is there evidence of
  institutional support?

22
Other Criteria

• Response to prior critique
• Preliminary data
• Progress
• Human subjects should be factored in score
• Safety issues
• Participation of women, minorities, and children
• Must have specific inclusion/recruitment plan
• NIH children under 21
• Animal welfare

23
Budget

• Discussed after the grant is scored
• Cut years if duration is not appropriate for work
  proposed
• Modular budgets need some justification
• Specific items can be recommended for deletion
  from non-modular budgets
• Study section can recommend a target amount
• Budget recommendations are to be negotiated with
  program officials at ICs

24
Post-meeting

• Scores from all reviewers are averaged
• Scores are percentiled using previous two rounds
• Summary statements
• Resume of discussion by SRA
• Written reviews from assigned reviewers
• Scores and reviews sent to ICs for funding
  decisions

25
Grant Writing Tips

• An overarching umbrella
• Ties together specific aims
• Specific and realistic
• Critical test of competing theories
• Discover missing element in a working hypothesis
• Specific Aims
• Focus on 3-5 related aims
• Innovative not incremental
• No weak aims

26
Grant Writing Tips