M 6 Quality Assurance A Necessary Evil or Quality Experience

1
M 6 Quality AssuranceA Necessary Evil
or Quality Experience ?
Quality Assurance

• Pam DeWeese, MAT, CCRP
• Administrative Director
• Clinical Trials Program
• Indiana University
• SRA 2005

2
Somebody is going to look at my work !
3
NO NEED TO PANIC !

• Wouldnt you rather find your own mistakes and
  fix them before some outsider
• finds them ?

4
Whats the Difference ?

• Quality Control (QC)
• Quality Assurance (QA)

5
Quality control

• Operational activities that fulfill requirements
• Provides checks of various required tasks
• Usually 100 review in real time
• For example QC of Informed Consent
• Consent form for EACH subject reviewed for
• (a) subject signed, (b) dated, (c) before any
  procedures initiated,
• (d) correct version, (e) initialed all pages,
  (f) signed by person obtaining consent (e) dated
  - within 24 hours of consenting

6
Quality Assurance

• A systematic, planned activity to verify that an
  organization is fulfilling requirements for
  quality and/or meeting quality objectives
• Typically a sample-based, retrospective review of
  key pre-identified aspects related to compliance
• For example Consent forms for 10 of subjects
  enrolled in the last quarter will be reviewed for
  the pre-identified criteria (a-e)

7

• The representative sample is believed to be
  reflective of overall performance
• Indicator of Acceptability
• or
• Indicator of possible problem requiring more in
  depth review of larger sample or all records

8

• Quality should Improve
• as the result of
• Quality Assurance Activities
• Therefore,
• QA QI

9
3 Essential Elements for Successful QA

• Managements commitment to quality
• Standards for performance (e.g. SOPs)
• Ongoing QA program
• 
  
  Research Practitioner, Vol. 4, No. 6, 2003

10
How do you begin?

• TRAINING
• Staff must first be knowledgeable about best
  practices and requirements.
• Only then is it acceptable to check for
  compliance.

11
Besides being essential for patient protection
and regulatory compliance,

• QA is a key component of your staff education and
  development program and vice versa.
• Offered from an educational (self help)
  perspective, it may also be viewed as less
  threatening.

12
Step One

• Create an Education and Training Program

13
Options

• Large and small group instruction
• Technology
• One-on-One

14
Large and Small Group Instruction

• Tap into existing programs, if available
• Survey staff needs/interests/weaknesses
• Identify applicable regulations and SOPs
• Create new programs (voluntary and mandatory) to
  meet the identified needs
• Utilize volunteer staff experts or sponsors as
  presenters, especially if money is an issue
• Advertise internally
• Provide handouts/reference materials
• Evaluate the program

15
In 2004, the IU Clinical Trials Program

• Offered 23 educational programs
• Attracted 1190 total attendees
• 98 rated content as good or excellent
• Most programs were optional
• A few were mandatory
• HIPAA Privacy Security
• 3-day Research Coordinators Education Program
• for new coordinators with lt 2 yrs research
  experience

16
Other Training Options/Methods

• CD-ROM
• Investigator 101 (OHRP)
• Hazardous Materials Handling
• Offers certification
• Very long (gt2hrs) flexibility of own work
  station
• Med library willing to purchase and handle
  check-out
• Web-based training
• IRB course and test
• Movie
• embedded in power point presentation
• Electronic game software
• Research Jeopardy
• One-on-One

17
One-on-One

• Approach problems with an educational, not
  punitive, mentality
• Act as Consultants
• Goal to teach them how to handle the situation
  themselves
• (We are not a fee for service provider.)
• Recommend corrective actions preventative
  maintenance for the future
• Assistance topic is individualized and
  client-driven
• No sticks
• No charge
• Trust
• Positive previous experience
• Maintain their confidentiality

18

• Dont over react
• Little shocks us anymore
• Believe that most people want to do things right,
  but they may
• have had limited training
• be overwhelmed, and/or
• need someone to help them calm down and
  methodically move forward
• (we think out loud for them)
• No issue is too small or too big
• e.g. How to organize a regulatory binder ? How
  to prepare for an FDA audit.
• Have a great working relationships with
  compliance offices on campus
• Work behind the scenes to help them look better

19
DATA
20

• Training is a central objective for all good
  research organizations
• On-site training is much less costly than outside
  training
• There is no substitute for staff equipped with
  the tools to conduct quality research and to
  understand the policies and regulations under
  which they must operate

21

• People generally remember
• 10 of what they read
• 20 of what they hear
• 30 of what they see
• 50 of what they hear and see (lecture)
• 70 of what they see and write (note taking)
• 90 of what they say and do (role play/actual
  practice)
• SoCRA Source May 2005

22
90 of what they say and do

• Which makes being actively engaged in quality
  assurance activities all the more meaningful

23
Step Two

• Establish an internal QA process
• to verify that
• actual practice is consistent with best
  practice

24
Objectives of Internal QA

• To evaluate an organizations own activities
• To determine if they conform with expected
  standards
• To provide an opportunity for improvement

25
QA should be a structured process

• Identify key processes
• Select key indicators
• Determine sample size and frequency of
  measurement
• Set priorities based upon assessment of
  vulnerability and/or risk

26

• Assign responsibility
• Do the work
• 7. Analyze the findings
• 8. Document and report findings
• 9. Identify corrective action
• 10. Evaluate the effectiveness of the plan
• 
  
• Research Nurse Sept/Oct 1999

27
1. Identify key processes to be evaluated

• Regulatory Process Examples
• 1572s up-to-date and accurate
• Protocol amendments filed within 1 week of
  receipt
• Continuing reviews submitted on time no lapses
• Subject-Specific Process Examples
• Consent process and forms
• Eligibility criteria
• Study visits / schedule of events compliance
• SAE reporting

28
Select key indicators to measure

• Examples
• Consent Did patient sign and date proper
  version of consent before any study procedures
  were initiated?
• Eligibility Were all pre-enrollment tests done
  and appropriate results obtained before
  enrollment?
• SAEs Were all occurrences accurately assessed,
  documented and properly reported?

29
3. Determine sample size and frequency

• This is dependent upon a variety of factors, most
  importantly, complexity and how critical the
  factor
• Consider both the number of patient records and
  protocols

30
What do you mean?

• Eligibility and informed consent
• Critical factors that should probably be assessed
  for 100 of enrolled subjects
• Missed study visits
• While important, a sampling (e.g. 10 of
  enrolled subjects) could provide an adequate
  indication of performance and compliance

31
As a general rule,

• When sampling, no fewer than 3 subject records
  should be reviewed to differentiate isolated
  incidences from trends
• All regulatory/ IRB records should receive full
  review (100)

32
Frequency

• Frequently enough to be useful (e.g. quarterly)
  but not so frequent as to be impossible to
  achieve (e.g. weekly) or a nuisance
• Consider incorporating QA into other routine
  activities to make it less intrusive
• e.g. if you have weekly research meetings,
  eliminate one meeting per quarter and devote the
  time to QA instead

33
4. Set priorities based upon assessment of
vulnerability and/or risk

• Consider
• Commercially-sponsored projects are typically
  monitored by the sponsor 4-6 weeks.
• Cooperative group studies may be monitored
  annually.
• Institutionally supported, investigator-initiated
  studies may have no external mechanism for
  review.

34

• Thus investigator-initiated studies may present
  the most vulnerability for non-compliance within
  your research group.
• Therefore, if faced with limited resources, you
  may want to concentrate internal QA program
  activities on these studies first.

35
5. Assign responsibility

• One person should be delegated to have oversight
  and management of the process
• However, this person should not be the
  evaluator

36
PEER REVIEW

• can be a very mutually beneficial method
• Less punitive
• Staff training model (we all make mistakes)
• No bad guy
• Best practices shared
• Provides performance standards for the supervisor
  
• Learn from one another teamwork improved group
  performance were all in this together

37
6. Do the Work

• Reserve adequate time to perform the work
• Obtain support from management
• Develop objective, criteria-based tools/forms
  that are used by everyone for accuracy,
  consistency, and efficiency

38
Sample QA Forms

• See attachments

39
Attachment A

• Regulatory Compliance Complete for EACH
  protocol to be reviewed
• YES NO NA
• 1. ? ? ? Study Approval Date
  -Patients did not receive treatment prior to
  date of study approval.
• 2. ? ? ? Summary Safeguard
  Statement- Information is accurate.
• 3. ? ? ? Informed Consent
  with IRB Approval All versions dated,
• stamped, and filed in reverse chronological
  order
• 4. ? ? ? Authorization for
  the Release of Health Information approved
• 5. Study Protocol
• ? ? ? a. Appropriate version of the
  protocol submitted to the IRB.
• ? ? ? b. All prior versions of the
  protocol retained filed appropriately
• 6. ? ? ? Amendments Approved,
  dated, filed
• 7. ? ? ? Continuing Review
  Dates Approvals without lapses
• 8. ? ? ? Adverse Events (including
  SAEs) Submitted to the IRB in a timely
  manner and per IRB and sponsor requirements.
• 9. ? ? ? CV/Medical
  License/Financial disclosure of P.I. and
  Sub- Investigators filed
• See attachment for complete form

40
Attachments

• Attachment B Clinical Research Study Checklist
• Attachment C Quality Assurance Review Form
  (source docs case report forms)
• Attachment D QA Worksheet for IRB Files
• Attachment E QA Checklist (with citations)

41
7. Analyze the findings

• An important and often overlooked step
• Total the results and aggregate the findings
• e.g. of the 20 patient records reviewed,
• 30 signed the wrong version of the consent

42

• Look for trends
• Dont jump to conclusions
• What are the causes or mitigating circumstances?
• Most problems are not the result of bad
  employees, but rather
• ineffective internal systems
• inadequate employee training
• poorly designed forms
• poorly defined or overly stressed processes

43
8. Document and report findings

• To be helpful, staff must learn from the
  process/results
• Results should be anonymously summarized in a
  consistent report format
• Quantify the results
• Compare the results over time (to evaluate
  effectiveness of corrective actions)
• Involve staff in QA discussions and for input
  regarding corrective action
• Share the good as well as the bad, e.g. best
  practice

44
9. Identify corrective action

• If a problem is identified, and probable cause
  identified, staff should be involved
• ( as appropriate) in designing and/or approving
  corrective actions
• - if the problem is consistently one
  employee, the supervisor has the responsibility
  to handle without penalizing the group

45

• Corrective action for the group could include
• changing a form
• reworking a process
• adding training programs
• Implement the change
• Agree upon a timeframe for improvement
• Shorter for critical problems, longer for less
  critical ones

46
10. Evaluate the effectiveness of the plan

• The QA indicator should be re-evaluated and the
  findings reported to the group
• If the corrective action is successful, no
  further action is required
• If not, a new corrective action is necessary
• Consider whether the root cause was properly
  evaluated
• e.g. perhaps wrong versions of ICSs were used
  because of improper filing vs lack of version
  control numbers on forms

47
Once a year

• Devote a meeting to evaluation of the QA Plan
• Has it been effective?
• Has there been positive improvement?
• Is the time commitment reasonable ?
• Have the priorities changed?
• Share results with others
• Revise as necessary
• Positive feedback creates positive reinforcement
  and motivation to continue

48
(No Transcript)
49
In Summary

• Finding time to incorporate QA into busy
  schedules can be challenging
• Once the forms and process are established, the
  activities generally flow easily
• Most people prefer to find their own mistakes
  before someone external to the group finds them
• Using results for internal improvement (vs
  punitive action) allows participants to feel
  safe and to save face
• This hands-on method of learning and
  self-improvement is more likely to result in
  changes in actual practice (Remember 90 of
  what you see and do)

50

• Provide a mechanism for
• staff to correct their own mistakes
• the group to benefit from a discussion of
  findings
• sharing or featuring a positive result or best
  practice
• Sharing the new idea or practice is an
  educational
• opportunity for the group.
• Recognizing exemplary work is far more motivating
  than
• chastising poor work, and benefits the individual
  and
• the group

51
Necessary Evil or Quality Experience?

• Research involving human subjects is a highly
  regulated activity requiring
• integrity
• sound data and project management processes
• minimization of risk to subjects

52
Necessary Evil or Quality Experience?

• Quality Assurance is not only Necessary,
• but it can be a Quality Experience
• when approached as a
• well-organized educational experience.

53

• Thats all Folks.

Questions ???
54
Pam DeWeese, MAT, CCRP Administrative
Director Clinical Trials Program Indiana
University 1001 W. 10th Street, OPW
526 Indianapolis, IN 46202 317-278-2865 pdeweese_at_
iupui.edu