Assessing the Quality of VA Human Research Protection Programs

1
Assessing the Quality of VA Human Research
Protection Programs

• Robert Brooks, MD PhD MBA
• Min-Fu Tsan, MD, PhD
• Office of Research Oversight

2
Quality Indicators for VA HRPPs (1)

• In recent years, ORO staff have developed a set
  of 16 Quality Indicators, with an emphasis on
  assessing outcomes rather than process or
  compliance.
• Quality Indicators were targeted at
• Whether there is an adequate system in place to
  protect research subjects
• Results that may indicate that research subjects
  have been harmed or their rights have been
  violated
• Results or factors that are likely to or may
  lead to harm to human subjects

3
Quality Indicators for VA HRPPs (2)

• These 16 QIs covered
• HRPP accreditation
• Initial and continuing review and approval by IRB
  and RDC
• Informed consent requirements
• HIPAA authorization requirements
• Investigator compliance, qualification, and
  training
• Serious adverse events
• Research involving vulnerable subjects
• International research
• Training of IRB and RDC members
• Subject education
• (Tsan MF, Smith K, and Gao B. IRB Ethics and
  Human Research. July-August 2010)

4
VA HRPP QIs Preliminary Data

• We incorporated some HRPP QIs into RCOs Informed
  Consent Document (ICD) audits as well as
  regulatory audits of human research protocols
  during the period between June 1, 2009 through
  May 31, 2010.
• Data were collected as part of OROs Facility
  Directors Certification of Compliance in July,
  2010
• Data was also collected regarding the numbers and
  types of open VA research protocols, and the
  numbers and types of audits performed
• Data were available for analysis from 107 VA
  Medical Centers which represent 100 of VA
  research facilities.

5
VA HRPP QIs Informed Consent Documents
Total number of protocols audited 14,944
Number of protocols with ICDs 3,563 (23.8)
Total number of ICDs audited (Average number of ICDs per protocol 24) 89,216
Incorrect ICDs used 2,143 (2.4)
Not signed and dated by subjects 197 (0.22)
Without HIPAA Authorization 480
Total Number of HIPAA Documents Required is
Unknown
6
VA HRPP QIs IRB and RDC Approval
Total number of protocols audited 2,102
Conducted and completed without IRB approval 1 (0.03)
Conducted and completed without RDC approval 3 (0.14)
Initiated prior to IRB approval 2 (0.10)
Initiated prior to RDC approval 9 (0.43)
7
VA HRPP QIs For-Cause Suspension/Termination
Total number of protocols (human, animal, and safety) audited 2,978
Suspended/terminated 83 (2.79)
Due to human subject concerns 25 (0.83)
Due to investigator-related concerns 2 (0.07)
8
VA HRPP QIs Local SAEs and Unanticipated
Problems
Total number of protocols audited 2,102
Local AEs determined to be serious, unanticipated, and related to research 24
Resulted in hospitalization 17
Resulted in death 1
9
VA HRPP QIs Lapse in Continuing/Annual Reviews
Total number of protocols requiring continuing reviews 1,585
Lapsed in IRB continuing reviews 115 (7.26)
Continued research activities during lapse 5 (0.32)
10
VA HRPP QIs Subject Case History Review
Total number of case histories reviewed 11,311
Informed consent not obtained prior to initiation of study procedure 249 (2.20)
No documentation verifying inclusion/exclusion criteria 271 (2.40)
11
VA HRPP QIs Research Personnel (Scope of
Practice)
Total number of research personnel in protocols (human, animal, and safety) audited 6,787
Without Scope of Practice (SOP) 672 (9.90)
Working outside SOP 10 (0.15)
12
VA HRPP QIs Research Personnel (Training)
Total number of research personnel in protocols (human, animal, and safety) audited 6,787
Required training not current 398 (5.86)
Without initial training 103 (1.52)
Lapse in continuing training 303 (4.46)
13
VA HRPP QIs International Research
Total number of human research protocol audited 2,102
Number of international research protocols 4
Without CRADO approval 2 (50)
14
VISN Quality Indicator Data (1)
VISN LEVEL DATA
15
VISN Quality Indicator Data (2)
VISN LEVEL DATA
16
VISN Quality Indicator Data (3)
VISN LEVEL DATA
17
VISN Quality Indicator Data (4)
VISN LEVEL DATA
18
VISN Quality Indicator Data (5)
VISN LEVEL DATA
19
Assessing the Quality of VA HRPPsSummary

• We developed 16 QIs for assessing the quality of
  VA HRPPs.
• Assessing the quality of VA HRPPs using these QIs
  appeared feasible.
• Preliminary data obtained during the period
  between June 1, 2009, through May 31, 2010,
  provide some baseline data for future comparison
  as well as some useful information to guide
  future improvement of the QIs and HRPPs.