WHY PROFICIENCY TESTING

1
WHY PROFICIENCY TESTING?

• Dr Robert Collins

2
Proficiency Testing Many Forms
3
Proficiency Testing - Introduction

• All analytical laboratories or similar
  organisations, should operate a quality assurance
  system which includes
• internal quality control procedures,
• participation in proficiency testing schemes,
• use of reference standards and
• certification/accreditation to a recognised
  standard ( eg. ISO 9001, ISO 17025 or ISO 15189).
  
• Of these, only proficiency testing schemes or the
  use of certified reference materials can
  establish the bias and accuracy of the results.

4
What is Proficiency Testing ? (a)

• Is the use of comparisons of test results or
  measurements to determine the performance of
  individual laboratories and to monitor a
  laboratorys performance.
• A means of assessing the ability of laboratories
  to competently perform specific tests and/or
  measurements.
• Provides laboratories with an objective means of
  assessing and demonstrating the reliability of
  the data they are producing
• Supplements laboratories own internal quality
  control procedures by providing an additional
  external audit of their testing capability

5
What is Proficiency Testing ? (b)

• Gives analysts confidence that the service will
  withstand scrutiny.
• Provides independent reassurance to laboratory
  and clients that procedures, methods and other
  laboratory operations are under control.
• Provides laboratories with a sound basis for
  continuous improvement.
• Proficiency testing is used by accreditation
  bodies to assess the ability of laboratories to
  competently perform tests or calibrations.

6
Proficiency Testing - How

• Participating laboratories are provided with
  uniform testing objects for analysis under
  specified conditions.
• Laboratory analyses the samples, preferably as
  part of their normal routine, and reports the
  results to the scheme organisers.
• Laboratory receive a report showing how closely
  their results agree with accepted value and,
  where necessary, can take action to improve
  performance.
• Confidentiality is essential with test results
  of all participants shown under a specific code
  number.

7
Proficiency Testing Value/ Benefits

• Good value for money compared with the
  consequences of producing inaccurate results
  which might put workers' health at risk, damage a
  company's reputation or contravene national
  regulations.
• Prospective customers can ask to see results from
  a laboratory's participation in proficiency
  testing schemes and satisfy themselves that the
  laboratory is able to obtain a consistent level
  of satisfactory performance over a period of time.

8
Limitations of Proficiency Testing

• It is important to bear in mind the limitation of
  proficiency testing
• It addresses only the analytic process, not the
  pre- or post-analytic activities of the
  laboratory. 
• Proficiency testing results are affected by
  variables not related to patient testing,
  including preparation of the proficiency sample,
  clerical function, selection of statistical
  methods of evaluation, and peer group
  definition. 
• Accordingly, it is not appropriate to use
  proficiency testing as the sole means for
  evaluating the quality of a laboratory.

9
ACCREDITATION PERSPECTIVE - On some uses of PT
schemes

• Especially useful when assessor is not able to
  give a positive judgement on the performance of a
  testing laboratory based only on on-site
  assessment.
• PT schemes are very useful if the following are
  not completely fulfilled
• Competence of staff appears not sufficient for
  the specific test
• Technical assessor decides that the experiences
  of the laboratory does not meet requirements of a
  new test methods/ new materials
• Documentation of in-house procedures seems to be
  insufficient
• Results of PT schemes in other related fields of
  testing have not been fully satisfactory.
• Documentation of corrective actions after
  achieving unsatisfactory performance is
  insufficient
• Systematic errors are detected in the laboratory

10
ACCREDITATION PERSPECTIVE On limitations of PT
schemes

• Bodies assessing technical competence of
  laboratories normally expect satisfactory
  participation in PT as evidence of a laboratory's
  ability to produce reliable test results.
• However a major distinction exists between
• Evaluation of laboratory competence by the
  assessment of its total operation and
• Examination of results of a laboratory's
  participation in PT - which only gives
  information about technical competence of the
  testing laboratory at a single point of time
  under the specific conditions of the test
  involved in a particular proficiency testing
  scheme.

11
Tips for Better PT Results (a)

• Monitor PT shipping schedules notify provider if
  any delay receiving samples.  
• Open PT shipment as soon as received. Check for
  broken and leaking vials or if samples are warm
  frozen/liquid materials may be compromised.
  Consult PT provider may need to be replaced.
• Staff up-to-date on written policies of
  laboratory as regards calibration, linearity
  checks test method choices.
• Staff should be trained in the proper handing of
  PT samples, results and reports.

12
Tips for Better PT Results (b)

• Analysers should be maintained and functioning
  properly.
• Follow the manufacturers instructions carefully
  in regards to reconstitution
• Correct assay vial
• Correct diluent and amount of diluent
• If water is the diluent, use reagent grade water
• Use calibrated pipettes to deliver diluent
• Allow sufficient time for equilibration and use
  correct equilibration conditions (dont use water
  baths, shakers, rockers etc. unless specified in
  the providers instructions)
• For liquid PT samples, allow material to come to
  room temperature naturally or as specified by the
  provider.

13
Tips for Better PT Results (c)

• Keep calibration and linearity checks current. 
  Recalibrate as necessary.
• Accuracy and precision should be established.
• Ensure QC is always current, checked properly
  maintaining your system.
• Avoid waiting to the last minute to analyse
  samples.

14
Tips for Better PT Results (d)

• Clerical errors are common in PT reporting. If
  possible, get another tech to check data entry. 
• As PT may fall outside the stream of normal LIS
  data transfer, be especially careful with
  clerical and transcription functions including
• Reporting correct sample
• Rounding
• Units
• Decimal places
• Conversion factor
• Return results to PT program provider before
  cut-off time / date. Note time zone differences.

15
Other Benefits from PT Participation

• May identify potential problems related to
  imprecision, systematic error and / or human
  error.
• Can carry out investigation systematically to
  identify root cause of any unexpected results.
• Can look for trends over time. Are all PT samples
  showing a negative or positive bias? Is there a
  bias at certain analyte concentrations? These
  events should trigger internal investigations
  looking for warning shifts and trends in the test
  method.
• Allows check on reconstitution, expiration
  reconstituted stability on QC, reagents,
  calibrators, stains and solutions.
• Allows review of test method protocol and
  procedures.
• Ensure instrumentation / methodology is current
  with updated software, revised procedures or
  updated control range.

16
Dos Donts of Proficiency Testing

• Do handle PT samples in the same manner as
  patient samples 
• Tested along with regular (patient) workload
• Tested by personnel who routinely perform the
  analysis
• Rotated among the staff who perform the tests
• Tested the same number of times as routine
  patient samples
• Labile PT samples are tested as soon as possible
  after receipt
• PT data is checked for clerical errors prior to
  submitting
• PT results are submitted without consultation
  with other facilities
• PT result forms indicate if a similar patient
  sample would be routinely referred out (e.g. bone
  marrows)
• All aspects of PT sample handling, including
  preparation, examination, testing and reporting
  are documented  

17
Dos Donts of Proficiency Testing

• Dont give PT samples preferential treatment 
• Not analysed by a special person
• Not analysed in replicate if patient samples are
  not run in replicate
• Not analysed on more than one instrument for the
  purpose of achieving the most acceptable result
  for submission. 
• Not analysed on 2 or more separate days
• Controls are not analysed just prior to running
  PT sample
• PT samples are not bracketed with QC
• When PT samples are analysed, QC is not run more
  frequently
• PT testing is not delayed until the instrument is
  working better
• Results not discussed/compared with other
  facility before submission
• PT samples are not sent to another facility for
  analysis
• PT samples are not split for testing at another
  facility to achieve the most acceptable result
  for submission

18
Accreditation of Proficiency Testing Provider

• Why Accredit Proficiency Testing Scheme
  Providers?
• Accreditation of proficiency testing scheme
  providers enables organisations that are
  interested in planning, providing and evaluating
  proficiency testing schemes to achieve and
  demonstrate a level of competence.
• What Are The Requirements For Accreditation Of
  Proficiency Testing Scheme Providers?
• ILAC G132000 Guidelines for the Requirements for
  the Competency of Providers of Proficiency
  Testing Schemes which is based on ISO Guide 43-1
  and on the relevant elements of ISO/IEC 17025.

19
HKAS Accreditation Programme for Proficiency
Testing Providers launched January 2008

• Accreditation criteria based on International
  Laboratory Accreditation Cooperation (ILAC)
  Guidelines for the Requirements for the
  Competence of Providers of Proficiency Testing
  Schemes (ILAC-G13 2007) and the ISO Guide 43
  Part 1.
• In addition to this essential document, other
  application documents related to this programme
  include
• HOKLAS Supplementary Criteria No. 34
• HOKLAS 017 Technical Criteria for Accrediting
  Proficiency Testing Providers
• HOKLAS 018 Assessment/Reassessment Questionnaire
  for Proficiency Testing Provider including a
  checklist
• http//www.itc.gov.hk/en/quality/hkas/publicatio
  n.htmpt_providers

20
Proficiency Testing - Summary

• Regular participation in an (accredited)
  proficiency-testing scheme
• Provides independent verification of the
  analytical competence of your laboratory
• Shows a commitment to the maintenance and
  improvement of performance.
• Demonstrates to the public, customers,
  accreditation bodies, regulators, and management
  that the analytical procedures are under control
• Gives confidence that the service which you
  provide will withstand scrutiny.
• Provides an educational opportunity
• THANK YOU

21
AAB Working Party on PT Providers

• Chairman Prof. KWAN KH Albert
• Members Prof. CHUNG CY Jonathan
• Dr. Robert J COLLINS
• Mr. TAM Chun-kwong Rick
• Mr. TSE Chun-cheong
• Dr. Terence WAN
• Dr. YAM Wing Cheong
• Prof. Benny Chung-ying ZEE