External Evaluation of Performance of Serology of Infectious Diseases

1
External Evaluation of Performance of Serology of
Infectious Diseases

• Amadeo Sáez-Alquézar

2
Procedures for Quality Control of Infectious
Disease Screening

• QUALITY ASSURANCE
• QUALITY CONTROL (INTERNAL)
• QUALITY ASSESSMENT
• EXTERNAL QUALITY CONTROL
• A PROCEDURE TO VERIFY THE QUALITY OF RESULTS
  GENERATED BY A LABORATORY
• CORRESPONDS TO AN EXTERNAL EVALUATION OF THE
  LABORATORYS PERFORMANCE
• EXTERNAL QUALITY CONTROL PROGRAMS (EQCPs)

3
External Quality Assessment

• External Quality Assessment (EQA).
• or External Quality Control Programs.
• (EQCPs) should be considered a challenge to
  check the efficacy of the Quality Assurance and
  the Internal Quality Control, in each laboratory.
• Multipanels a necessary tool.

4
MULTIPANEL

• In the specific case of the serological trial of
  blood bank, the main interest is to evaluate the
  performance of the laboratory responsible for the
  screening tests
• For this purpose, there is a panel with positive
  samples for all tests normally applied in the
  screening and some negative ones
• This sort of serum panel is called MULTIPANEL

5
BRASIL MANDATORY SEROLOGICAL SCREENING IN BLOOD
DONORS

• Anti-HIV 1/2 (2)
• Anti-HTLV 1/2 (1)
• Anti-HCV (1)
• HBsAg (1)
• Anti-HBc (1)
• Syphilis (1)
• Chagas (2)
• ALT (1)

• Anti-HIV 1/2 (2)
• Anti-HTLV 1/2 (1)
• Anti-HCV (1)
• HBsAg (1)
• Anti-HBc (1)
• Syphilis (1)
• Chagas (1)
• ELISA / Chem

1376 Nov / 93
RDC 343 Dec / 2002
6
Multipanel for EQCPsBlind Panel

• 12 or 24 serum samples
• Positive samples of
• HIV
• HTLV
• HBsAg , Anti-HBc
• T. cruzi
• Syphilis
• HCV
• Negative samples

7
PRODUCTION CaCl2 Centrif. Filtrat Dialysis
Centrif. Preservative Bronidox-L5
Plasma Units
Serum Units
Testing
Samples Selection According To The
Purpose MULTIPANELS SPECIFIC PANELS BORDERLINE
PANELS
Storage
8
of different tests used to caracterize a
multipanel (nov/2002)
() Anti-HIV12, anti-HIV12O and anti-HIVAg
HIV
() Anti-HTLVI/II
9
Borderline panel (MASTER) for Internal Quality
Control Procedures
10
External Quality Control Programs (EQCPs)
11
Organizing Center
Participanting Laboratories
Shipment
Samples Processing
Multipanels Manufacturing
Results
Disclosure of the correct results
Acknowledging the correct results
Final Report Evaluation
Analysis of the results
Self Evaluation
12
Evaluation of the LaboratoriesCriteria

• A Correct Results
• Without FPR or FNR
• B1 False Positive Results
• lt 5 of the total assays
• B2 False Positive Results
• gt 5 of the total assays
• C False Negative Results

13
(No Transcript)
14
(No Transcript)
15
EQCPs - Final Report Contents

• Number, type and geographic distribution of the
  Participant Laboratories
• Charaterization of the Multipanel
• Strategies (type of methodologies) used by each
  Participant Laboratory
• Confirmatory tests used

16
EQCPs - Final Report Contents

• False Positive and False Negative Results
• Number, and distribution by methodology, by
  disease and by trademark of the used kits
• Summary of the discrepancies observed during the
  development of the program
• Relevant information

17
General Comments - 1

• The Programs intend to assess the performance of
  the PLs. Are developed with the intent of
  advising and cooperating ensuring confidential
  individual results.
• Possible failures in the internal procedures of
  the PLs may give rise to False Positive Results
  and / or False Negative Results.

18
General Comments - 2

• The revision of internal procedures and analysis
  of products and equipment utilized will clarify
  the causes of the problems observed.

19
General Comments - 3

• We emphasize that FPR and FNR observed should not
  be attributed exclusively to the origin and
  characteristics of the kits utilized by each PL.
  Results must serve as a warning of the need for a
  revision of all the items related to use of
  equipment in general including technical staff
  involved and internal controls utilized.

20
Types of EQAP

• PARTICIPATION
• Voluntary / Mandatory
• COST FOR THE PLs
• Free / Fee
• LOCATION OF THE PLs
• National / International
• ACTIVITY
• External QC / Internal QC

21
Types of EQCP according participation

• Voluntary.
• System based in professional responsibility.
• Participants are interested in their performance
  and read final reports.
• Only concerned Labs are assessed.

• Mandatory.
• More reliable data.
• Does not guarantee improvement.
• Some participants does not use the final reports.
• The assessment comprises all Labs.

22
Types of EQAS according activity

• External Quality Control Only.
• High cost to get IQC materials.
• Slow quality improvement.
• Combination of External Quality Control and
  Internal Quality Control.
• Integration of EQC and IQC data.
• All needs of PLs covered.
• More work for the Organizer.

23
Whats the cost of an EQCP?

• Calculate
• The number of PLs of Multipanels.
  (frequency?)
• Multipanels price.
• Location of the PLs. (shipment costs)
• International More expensive
• Documents and software / hardware
• Staff involved

24
(No Transcript)
25
Transport of infectious substances from São
Paulo to Latin America Countries / Refrigerated
material
26
Transport of infectious substances from São Paulo
to Latin America Countries / Refrigerated
material (door to door World Courier)

• 16 Countries 22 Laboratories
• Total, without packaging and refrigerated gel
  (previous slide) US 10,282
• Additional costs
• ANVISA Brazilian Health Dept Inspection fee US
  30,00 per group of samples up 50 vials (US 660)
• Special boxes and refrigerating Gel
• US 1,900 (approximate)
• Broker and clearances costs at the destinations,
  which varies from US 50 to 250 per shipment

27
Shipments to Latin America and Caribbean Region
(Nov / 2002)

• Latin America. Total cost of shipment
• 22 Laboratories in 16 countries
• US 17,022
• Caribbean Region. Total cost of shipment
• 23 Laboratories but only one shipment
• 6 boxes for CAREC (Port of Spain)
• US 1,800

28
Development of External Quality Control Programs
in Serology
BRASIL 1994 - 1995
MINISTRY OF HEALTH / FPS PUBLIC BLOOD BANKS 1st
32 Labs 2nd 58 Labs 3th 60 Labs 4th
61 Labs
29
EQCPs - BRASIL 1994 - 1995

• The first EQCPs for Blood Banks developed in the
  country
• PLs showed great interest to participate
• Main problems observed
• High Nr of FNR for anti-T.cruzi (IHA), Syphilis
  (VDRL), anti-HIV and HTLV.
• Inadequate strategies used in the serologica
  screening.

30
Development of External Quality Control Programs
in Serology
BRASIL 1995 - 2003
PANEL / SBHH PUBLIC AND PRIVATE BLOOD BANKS 22
PROGRAMS 125 LABS Options External QC and
Internal QC
Public services 45 Private services 56
() SBHH 1995-2001
31
EQCPs PANEL/SBHH BRASILPLs Evaluation (2001-2002)
32
EQCPs PANEL/SBHH BRASILPLs Evaluation (2001-2002)
33
EQCPs PANEL/SBHH BRASILPLs Evaluation (2001-2002)
34
Tests Used by Brazilian Blood Banks in the
Serological Screening for T.cruzi (1999-2001)
35
EQCPs PANEL/SBHH BRASILPLs Evaluation (1999-2001)
Anti-T.cruzi screening
ACorrect Results without False Positive Results
or False Negative Results. B Presence of False
Positive results. CPresence of False Negative
Results
36
FPR and FNR observed during the development of
eight EQCPs - PANEL/SBHH BRASIL (1999-2001)
Anti-T.cruzi screening tests
FPRFalse Positive Results FNRFalse Negative
Results.
37
False Negative Results (), by Methodology,
During The Development of Eight EQCPs in Brasil
(1999-2001)
Anti-T.cruzi screening tests
IIFIndirect immunofluorescence IHAIndirect
hemmaglutination
38
Development of External Quality Control Programs
in Serology
1995 / 96
1996
1997 / 2000
Latin America 5 Programs 16 countries (21 PLs)
Central America and Colombia (3 Programs)
One Program
São Paulo
PAHO /WHO/ FPS
39
EQCPs - 1997 / 2000 PAHO /WHO/ FPS

• Workshops developed in many countries.
• Different strategies of serological screening.
• Incomplete coverage of serological screening for
  HCV, HTLV, Chagas and Core.
• High number of FNR.
• Problems using in house and RIA tests.
• Performance improved since 1997.

40
False Negative and False Positive Results ()
Observed During The Development of Five EQCPs in
Latin America (1997-2000)
PAHO / WHO / FPS
41
(No Transcript)
42
Serological Screening coverage in Latin America
EQCPs (1997 / 2000)
43
Workshops promoted by PAHOSerological screenig
and Quality Control

• Uruguay 1993
• Chile 1993
• Brasil 1993
• El Salvador 1994
• Peru 1994
• Brasil 1994
• Honduras 1995
• Mexico 1995

• Colombia 1996
• Brasil 1996
• Argentina 1996
• Paraguay 1997
• Brasil 1998
• Colombia 1999
• Guatemala 1999
• Mexico 2001

44
Multiply Effect Generated by the Development of
the EQCPs in Latin America

• 1993
• 4 countries (limited controls)
• 2001
• 15 countries (21 laboratories) participating of
  PAHO/EQCPs
• 10 countries performing EQCPs Nationally (after
  instruction)

45
Development of the EQCPs in Countries of Latin
America
46
Development of External Quality Control Programs
in Serology 2001 - 2003
CARIBBEAN 21 Countries 24 Labs 2 Programs (2)
LATIN AMERICA 15 Countries 20 Labs 2 Programs
(2)
BRASIL 125 Labs 6 Programs (3)
PAHO / PANEL
PAHO/CAREC/PANEL
PANEL ( )
47
EQCPs PAHO/WHO Latin America Participating
Countries (2001-2003)

• Argentina (B. Aires 2)
• Bolivia (Cochabamba)
• Bolivia (S.C de la Sierra)
• Chile
• Colombia (Bogotá)
• Colombia (Medellin)
• Ecuador
• El Salvador (2)

• Guatemala
• Honduras
• Nicaragua
• Panamá
• Paraguay (2)
• Peru
• R. Dominicana
• Uruguay
• Venezuela

48
Performance per Parameter Evaluation of the
Participant Laboratories CAREC / PAHO1101(N 20)
49
EQCPs PAHO/WHO Caribbean Region. Participating
Countries (2001-2003)

• Anguilla
• Antigua
• Aruba
• Bahamas (Nassau)
• Bahamas (Freeport)
• Barbados
• Belize
• Bermuda
• British Virgin Islands
• Cayman Islands
• Curaçao

• Dominica
• Grenada
• Guyana
• Jamaica
• Montserrat
• St. Kitts
• Nevis
• Saint Lucia
• St. Vicent Grenadines
• Trinidad Tobago
• Trinidad

50
Performance per Parameter Evaluation of the
Participant Laboratories Latin America /
PAHO1101(N 20)
51
Serological Screening coverage in The Caribbean
Region EQCPs (2002)
52
Serological Screening coverage in Latin America
EQCPs (2002)
53
Serological Screening coverage.Comparison
between LA and Caribbean
54
Main Problems Detected During the Development of
the EQCPs

• As a consequence of
• Contamination (FPR).
• Transcription of the results (FNR and FPR).
• Features and limitations of the used
  methodologies (FNR and FPR).
• Rapid tests in house tests RIA
• The equipment (manual and automatic).
• Inadequate IQC procedures.

55
SEROLOGICAL
SCREENING
HIV
56
SEROLOGICAL
SCREENING
HBs Ag
57
PAHO/PANEL 2001
LATIN AMERICA
EQCPs
PAHO/WHO
PAHO/PANEL 2002
58
PAHO/PANEL 2001
CARIBBEAN
EQCPs
PAHO/WHO
PAHO/PANEL 2002
59
The support from PAHO/WHO for the development of
the Programs for External Quality Control in
Serology in Latin America has greatly contributed
to the improvement of the quality of Blood used
in all countries in this region.
At the same time, the organizing of workshops and
training programs has generated a multiply
effect, allowing each country to produce their
own sera panels and internally develop their
own Quality Control Programs.
60
It is very important that these actions from
PAHO/WHO continue, and also be intensified, so
that the countries of Latin America and the
Caribbean region can achieve the same quality
level of Blood and Blood Components as the rest
of the Americas.
61
Thanks to

• Marcia Mitiko Otani
• Dalton Fisher Chamone
• Nanci Alves Salles
• Ester C. Sabino
• Márcia Murta
• Waldelania P. Marques
• José Ramiro Cruz
• Gabriel Schmunis
• Lisa Boisson

• PAHO/WHO
• FPS/HSP
• SBHH
• PANEL