Evaluation and Evolution of CSR Study Sections and Integrated Review Groups: Principles and Philosop

1
Evaluation and Evolution of CSR Study
Sections and Integrated Review Groups Principles
and PhilosophyDon Schneider, PhDDiscussants
Drs. Pugh, Leinwand, Sassaman, RammPeer Review
Advisory CommitteeMay 22, 2006National
Institutes of HealthDepartment of Health and
Human Services
2
Study Section Organization

• Core Values of Peer Review
• Fair, expert, timely
• Free from influence/conflict
• Organizational Approach
• Integrated Review Groups (IRGs)
• Right size work unit
• Cluster related science

3
Scientific Peer Review at DRG/CSR

• 1945-1998 Case-by-case changes
• Study sections formed in response to increases in
  workloads on a case-by-case basis
• 1998/9 Broad changes
• With scientific community participation, merger
  of Alcohol, Drug Abuse and Mental Health
  Administration (ADAMHA) with NIH
• AIDS IRG formed
• Panel on Scientific Boundaries for Review (PSBR)
  meetings to develop plan for study section
  organization with involvement of scientific
  community

4
CSR IRG Organization2006

• IRGs with focus on organ system or disease serve
  NIH health mission (17 IRGs)
• IRGs with focus on basic scientific discovery
  ensure foundation for future discoveries (3 IRGs
  BCMB, CB, GGG)
• IRGs with focus on development of methods and
  crosscutting science are needed (3 IRGs BDA,
  BST, SBIB)

5
Study Section EvaluationPhilosophical views

• Assurance of Core Values of peer review
• Continuous monitoring of peer review processes
  and organization
• Prompt evolution of organization as scientific
  opportunities change
• Promotion of open communications and transparency
  with the applicant community and NIH staff

6
Procedures for Systematic Evaluation

• Identifying issues, concerns, and problems
• Collecting and analyzing data
• Involving applicants, reviewers, relevant
  advocacy groups and scientific societies, and NIH
  staff

7
Five Year AssessmentsWorking Groups

• Full cycle of assessments completed 2001
• Second cycle started with Neuroscience IRGs
• SBIR study sections also underway
• Other IRGs will follow in order of
  reorganization, AIDS and Behavioral/Social
  Science IRGs, then HEME (2003) and ending with
  BCMB and CB (2005)
• Working Groups meet by email, telephone/video
  conference, asynchronous electronic discussion,
  or face-to-face
• Results are presented to PRAC (a FACA requirement)

8
Additional Working Group Activities

• May address trans-IRG issues resulting from
  excessive overlap or emergence of new areas
• May evaluate broader principles and effectiveness
  of CSR structure by involving senior members of
  broad spectrum of areas similar to Bruce
  Alberts-led PSBR (eight years ago)

9
Continuous Monitoring

• Study section workloads are analyzed every cycle,
  e.g., to assess staffing needs
• CSR supervisors, IRG Chiefs and senior
  management, including Dr. Scarpa, attend study
  sections
• CSR reviews one IRG a month, completing all 23
  every two years

10
Monthly IRG Review

• Initiated February 2006
• Review sequence determined by
• Known/emerging issues
• Scientifically related areas
• For local matter, such as overload in one study
  section, CSR develops plan of action
• For substantial matter, e.g., formation of new
  study section, a Working Group is formed, and the
  issue is presented and discussed at PRAC

11
Cont. Continuous Monitoring

• Contact retiring Study Section Chairs
• Attend scientific society meetings and present
  peer review talks
• Host visiting officials from scientific societies
• Invite society representatives of broad
  scientific areas to CSR for discussion
• About 30 societies per broad area
• See Dr. Scarpas 900 AM Remarks

12
Summary

• Adhere to core values of peer review, in support
  of NIH health mission
• Allow scientific developments, with input from
  senior advisors, to influence/shape design
  organization philosophy
• Monitor review units every 5 years, every 2
  years, and continuously
• Promote communication and transparency with
  community, staff, and PRAC

13
DISCUSSANTS

• Dr. Edward Pugh
• Dr. Leslie Leinwand
• Dr. Anne Sassaman
• Dr. Louise Ramm
• PRAC