Validation of screening methods (2002/657/EC) - PowerPoint PPT Presentation

1 / 45
About This Presentation
Title:

Validation of screening methods (2002/657/EC)

Description:

Validation of screening methods (2002/657/EC) N. Van Wouwe IPH AFSCA-FAVV Definition (2002/657/EC) Screening method : used to detect the presence of a substance or ... – PowerPoint PPT presentation

Number of Views:266
Avg rating:3.0/5.0
Slides: 46
Provided by: NVa9
Category:

less

Transcript and Presenter's Notes

Title: Validation of screening methods (2002/657/EC)


1
Validation of screening methods (2002/657/EC)
  • N. Van Wouwe

AFSCA-FAVV
2
Definition (2002/657/EC)
  • Screening method
  • used to detect the presence of a substance or
    class of substances at the level of interest.
  • have the capability for a high sample throughput
  • gt are used to sift large numbers of samples for
    potential non-compliant results.

Exemple ELISA, plate test, biosensor, receptor
test,
3
Definition (2002/657/EC)
  • Minimum criteria to use an analytical method as
    screening method
  • must be validated (traceability)
  • must have a false compliant rate of lt5 (ß-error)
    at the level of interest

4
Performance characteristics for method validation
(screening)
Qualitative method identifies a substance on
basis of its chemical, biological or physical
propriety (binary response /-,
absence/presence) Quantitative method determines
the amount or mass fraction of a substance
(response numerical value of appropriate unit)
determination is mandatory
5
Validation of screening test
  • Definition of the scope of the method
  • Analyte of group of analytes
  • Range of concentration
  • List of matrices
  • Initial validation with the most often used
    matrice in national monitoring program
  • Detection capacity (CCß)
  • Selectivity/Specificity
  • Applicability/ Ruggedness/Stability
  • Precision (only for semi-quantitative method)

If possible different sources of blank material,
different technicians, different days on the same
spiked sample
6
Validation of screening test
  • Targeted test for 1 compound
  • validation for this compound
  • Targeted test for a family of compounds
  • validation for 1 representative molecule of the
    family (antibody)
  • Wide range test for more than 50 different
    molecules
  • Validation for at least a list of representative
    compounds
  • Common pattern of activity on a specific
    bacteria?
  • Common way of action (acting target)?
  • Published reference data on validation available?

7
Proposition of the CRL for antimicrobials (in
milk)
Representative Compound Antimicrobial class
Cefalonium/Cephapirin/Cefquinome CEPHALOSPORINS
Penicillin G/Cloxacillin PENICILLINS
Tetracycline/Doxcycline TETRACYCLINS
Gentamicin/Streptomycin/Spectinomycin AMINOGLYCOSIDES
Enrofloxacin/Flumequine QUINOLONES
Sulfathiazole/sulfaguanidine/Sulfamerazine SULFONAMIDES
Erythromycin/Tylosin MACROLIDES
Lincomycin LINCOSAMIDES
Thiamphenicol PHENICOLATED
Trimethoprim/colistine MISCELLANEOUS
8
Performance characteristics
  • Detection capacity
  • Selectivity/Specificity
  • Applicability/ Ruggedness/Stability
  • Precision (only for semi-quantitative method)

9
Detection capability (CCß)
  • The smallest content of the substance that may be
    detected, identified and/or quantified in a
    sample with an error probability of ß
  • In case of MRPL, CCß lowest concentration at
    which the method is able to detect truly
    contaminated sample with a statistical certainty
    of 1-ß
  • In case of MRL, CCß concentration at which the
    method is able to detect the MRL concentrations
    with a statistical certainty of 1-ß

10
Detection capability (CCß)
  • No permitted limit
  • Analyse 20 blank materials gt CCa 3x
    signal/noise
  • Analyse 20 blank materials fortified at CCa
  • gt CCß CCa 1.64 x SDRW
  • Calibration curve procedure (ISO 11843)
  • Analyse of blank material fortified at 0 MRLP,
    0.5 MRLP, 1 MRLP, 1.5 MRLP and 2 MRLP
  • Plot analytical results (y-axis) vs
    concentration(x-axis)
  • CCa y-intercept (blank) 2.33 x SDRW
  • CCß CCa 1.64 x SDRW

11
Detection capability (CCß)
  • No permitted limit
  • If no quantitative results
  • Analyse fortified blank samples at and above CCa
  • (n 20 / concentration level)
  • CCß concentration level where only 5 false
    compliant results remain

12
Detection capability (CCß)
CCa
CCb
Blank
1.64xSDRW
2.33xSDblank
a1
ß5
Signal orConcentration
13
Detection capability (CCß)
  • Permitted limit (MRL)
  • Analyse 20 blank materials fortified at MRL
  • gt CCa MRL 1.64 x SDRW
  • Analyse 20 blank materials fortified at CCa
  • gt CCß CCa 1.64 x SDRW
  • Calibration curve procedure (ISO 11843)
  • Analyse of blank materials fortified at 0.5 MRL,
    1 MRL, 1.5 MRL and 2 MRL
  • Plot analytical results (y-axis) vs
    concentration(x-axis)
  • CCa MRL 1.64 x SDRW
  • CCß CCa 1.64 x SDRW

14
Detection capability (CCß)
CCa
CCb
MRL
1.64xSDMRL
1.64xSDRW
ß5
a5
Signal orConcentration
15
Performance characteristics
  • Detection capacity
  • Selectivity/Specificity
  • Applicability/ Ruggedness/Stability
  • Precision (only for semi-quantitative method)

16
Selectivity/specificity
  • Specificity ability of a method to distinguish
    between analyte being measured and other
    substances
  • problem of interference?
  • F(measuring technique, class of compounds,
    matrices,)

17
Selectivity/specificity
  • How to test specificity for qualitative screening
    method?
  • Analyse 20 different blank samples and 20
    positive samples (blind study, same or different
    days/technicians)

Specificity 100 NA/N- Other parameters Accurac
y 100 (PANA)/(N- N) Sensitivity 100
PA/N False positive 100 FP/(N- N) False
negative 100 FN/(N- N)
True positive (N) True negative (N-)
test result positive Positive agreement (PA) False positive (FP)
test result negative False negative (FN) Negative agreement (NA)
18
Selectivity/specificity
  • How to test specificity for semi-quantitative
    screening methods?
  • Select potentially interfering substances
    (metabolites, derivatives,)
  • Analyse relevant blank samples (n 20)
  • Analyse fortified blank samples with interfering
    substances at a relevant concentration
  • Estimate the effect of the interferences
  • False identification?
  • Influence in quantification?
  • Identification of the target analyte is hindered?

19
Performance characteristics
  • Detection capacity
  • Selectivity/Specificity
  • Applicability/ Ruggedness/Stability
  • Precision (only for semi-quantitative method)

20
Applicability
  • Scope of the method must be define in term of
  • Matrix (solid/liquid matrix, type of tissue)
  • Animal species
  • To introduce a new matrix
  • Analyse at least 10 different blank material
    fortified at level of interest for the new matrix
    (CCß) test of interferences
  • If 10 positive results gt method applicable for
    the new matrix
  • If 1 negative result gt 10 additional analyses
  • If 1 negative resultgt CCß must be recalculated
    for the new matrix

21
Ruggedness
  • Ruggedness the susceptibility of an analytical
    method to changes in experimental conditions
  • sample material
  • analytes
  • storage condition
  • environmental condition
  • sample preparation condition

22
Ruggedness
  • How to test ruggedness? (during development)
  • Identify possible factor that could influence the
    results (the analyst, solvents, pH, T, rate of
    heating,)
  • Vary each factor slightly
  • If one factor is found to influence results of
    the representative molecule, conduct further
    experiments
  • gt acceptability limits for this factor
  • (in the method protocol)

Recommendation of CRL analyses of 10 blank and
10 spiked samples at the same concentration and
with minor change of factor to detect influence
on results
23
Stability
  • Test are not necessary if stability data already
    exist (from other lab or from publication)
  • To include in the validation report
  • Stability test
  • the analyte in solution
  • the analyte in matrix
  • Aliquots of a fresh solution or sample stored
    under different conditions (T and/or storing
    time)

24
Performance characteristics
  • Detection capacity
  • Selectivity/Specificity
  • Applicability/ Ruggedness/Stability
  • Precision (only for semi-quantitative method)

25
Precision (for quantitative screening)
  • Precision the closseness of agreement between
    independent test results obtained under
    predetermined conditions
  • Expressed in terms of imprecision / standard
    deviation of test results

26
Precision (for quantitative screening)
  • How to test precision?
  • Repeatability test
  • within-laboratory reproducibility test(or
    intermediate precision)
  • Reproducibility test (between laboratories
    interlaboratory studies)
  • determination of RSD () lt Precision criteria

27
Precision (for quantitative screening)
  • Repeatability
  • 3 concentrations
  • 1x 1,5x 2x MRPL
  • 0,5 1x 1,5x MRL
  • 6 replicates/level
  • 3 times
  • same conditions
  • Within-laboratory Reproducibility
  • 3 concentrations
  • 1x 1,5x 2x MRPL
  • 0,5 1x 1,5x MRL
  • 6 replicates/level
  • 3 times
  • different conditions (analyst, env. condition,)

28
Precision (for quantitative screening)
  • ANOVA treatment of data gt RSDr RSDRW
  • Comparison with precision criteria
  • Horwitz equation RSDR() 2(1-0.5logC)
  • Criteria for repeatability RSDr 1/2 to 2/3
    RSDR
  • Criteria for within-lab reproducibility RSDRW
    2/3 to 1 RSDR

!
For concentration lt 100 µg/kg, RSDR becomes too
high!
29
Other recommendations
  • False negative rate lt5 Analyses of 20 negative
    and 20 positive samples in order to test the
    screening method (see selectivity).
  • One QC sample must be added in routine and
    results must be added to the validation file
  • Method transfer/Commercial test
  • Bibliographical survey to compil the evaluation
    of performance of the test
  • Collection of data from supplier on validation
    study
  • Experimental plan to test skillness of technician
    to perform the test
  • Use of QC sample
  • Participation to proficiency test

30
Exemple analyse of PCDD/F by CALUX bioassay
  • PCDD/F 17 toxic congeners to analyse in various
    matrices (TCDDmost toxic dioxin)
  • Results expressed in TEQ (Sum (CCixTEFi)i1-17)
  • MRL for each matrix (milk, meat, egg, fish oil,)
  • MRL expressed in pg TEQ/g fat or ng TEQ/ kg
  • Reference method GC-HRMS
  • Screening method immunoassay, bioassay,

31
Exemple analyse of PCDD/F by CALUX bioassay
CALUX bioassay genetically modified cell-based
bioassay (luciferase) Amount of light produced is
proportional to the toxicity (TEQ) of extracts
All substances fixing the Ah receptor
32
Analyse of PCDD/F by CALUX bioassay
  • Advantage
  • Rapid
  • Cheaper than GC-HRMS
  • Time for analyses
  • Disadvantage
  • Various compounds can fix the Ah receptor (PAH,
    PCB, PHDD/F,)
  • specificity!!!!

33
Analyse of PCDD/F by CALUX bioassay protocol
Extraction of fat
Clean-up on silica acid carbon columns
Fraction with PCBs
Fraction with PCDD/F
Fraction with interfering compounds
Evaporation
Reading plate
Dosing plate
34
Analyse of PCDD/F by CALUX bioassay validation
  • Selectivity/specificity
  • Ruggedness/Stability
  • Precision
  • Detection capability

35
Analyse of PCDD/F by CALUX bioassay selectivity
  • Possible interfering compounds?
  • PAH mostly in environmental sample
  • PCB fractionation during clean-up
  • Other compounds? (PHDD/F) dependant of the
    matrix? (matrix effect?)
  • Results of the selectivity test
  • No interferences for feedstuff, milk, egg, fat
  • Interferences for fish oil
  • CALUX results 2 x GC-HRMS results

36
Analyse of PCDD/F by CALUX bioassay selectivity
  • Matrix effect for fish oil

37
Analyse of PCDD/F by CALUX bioassay ruggedness
  • What are the critical point in the protocol?
  • Carbon column (interferences)
  • Solvent (interferences)
  • Curve (results)
  • Evaporation time (recovery)
  • Age of CALUX cell line (RSD)

38
Analyse of PCDD/F by CALUX bioassay ruggedness
  • Carbon column amount of carbon used

(Rdt PCDD/F 60)
(Rdt PCDD/F 80)
DX fraction
PCB fraction
Not collected fraction
(Rdt PCDD/F 80)
39
Analyse of PCDD/F by CALUX bioassay ruggedness
  • Evaporation time

40
Analyse of PCDD/F by CALUX bioassay ruggedness
  • Solvent tested before use on a TCDD solution
    (antagonist/agonist effect)
  • Curve tested with an independant TCDD solution
  • Age of CALUX cells new cell every 2 months

41
Analyse of PCDD/F by CALUX bioassay precision
  • Validation protocol

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6
Blank solvent 1 1 1 1 1 1
Blank sample 1 1 1 1
Sample at MRL/2 3 3 2
Sample at MRL 6 2 3 3
Sample at 2MRL 3 2 3
Quality sample 1 1 1 1 10 10
42
Analyse of PCDD/F by CALUX bioassay precision
  • ANOVA results for the TEQ determination of PCDD/F
    in feedstuff by CALUX bioassay
  • At MRL (0.75ng TEQ/kg) XMRL 0.751 ng TEQ/kg
  • Sr 0.063 gt RSDr 8.4
  • SRW0.073 gtRSDRW 9.7
  • At MRL/2 (0.376ng TEQ/kg) XMRL/2 0.464 ng
    TEQ/kg
  • Sr 0.051 gt RSDr 11
  • SRW0.051 gtRSDRW 11
  • At 2MRL (1.5ng TEQ/kg) X2MRL 1.571 ng TEQ/kg
  • Sr 0.107 gt RSDr 6.8
  • SRW0.115 gtRSDRW 7.3

RSD lt 30 (2002/70/EC)
43
Analyse of PCDD/F by CALUX bioassay detection
capacity
  • CCß for the TEQ determination of PCDD/F in
    feedstuff by CALUX bioassay
  • CCa MRL 1.64 x SRW
  • CCa 0.75 1.64 x 0.073 0.87 ng TEQ/kg
  • CCß CCa 2.33 x SRW
  • CCß 0.87 2.33 x 0.073 1.04 ng TEQ/kg

2002/70/EC false negative rate lt 1 !
gt At a concentration of 1.04ng TEQ/kg, we are
sure that the sample is a positive sample with
99 certainty
44
Analyse of PCDD/F by CALUX bioassay confirmatory
range
?
NON COMPLIANT
COMPLIANT
SUSPICIOUS
MRL
CCa
CCb
CC
-2.33sMRL
1.64sMRL
2.33ssample
Signal orConcentration
1
a5
ß5
45
Analyse of PCDD/F by CALUX bioassay confirmatory
range
  • Lower limit of the confirmatory range for the TEQ
    determination of PCDD/F in feedstuff by CALUX
    bioassay
  • CC MRL-2.33 x SDRW
  • CC 0.75- 2.33 x 0.073 0.58 ng TEQ/kg
  • Conclusion
  • Sample lower than 0.58 ng TEQ/kg are negative
    with 99 certainty (false negative rate lt 1)
  • Sample above 0.58 ng TEQ/kg must be confirmed by
    GC-HRMS
Write a Comment
User Comments (0)
About PowerShow.com