ILAC G13:2007 Proficiency Testing Requirements - PowerPoint PPT Presentation


PPT – ILAC G13:2007 Proficiency Testing Requirements PowerPoint presentation | free to download - id: 3ba657-ZTQwM


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

ILAC G13:2007 Proficiency Testing Requirements


ILAC G13:2007 Proficiency Testing Requirements A2LA PT Training 2008 Conclave April 11, 2008 Dan Tholen ILAC G13 Training Introduction to A2LA PT Provider ... – PowerPoint PPT presentation

Number of Views:357
Avg rating:3.0/5.0
Slides: 59
Provided by: a2laOrgco
Learn more at:


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: ILAC G13:2007 Proficiency Testing Requirements

ILAC G132007 Proficiency Testing Requirements
  • A2LA PT Training
  • 2008 Conclave
  • April 11, 2008
  • Dan Tholen

(No Transcript)
ILAC G13 Training
  • Introduction to A2LA PT Provider Accreditation
  • How assessing PT providers differs from assessing
  • Management system requirements
  • Technical requirements
  • TNI PT Provider Program (430-600)
  • Unresolved issues, questions

A2LA Accredited PT Providers
  • 16 Providers
  • 8 TNI Environmental Providers
  • Chemistry, Microbiology, Radiochemistry
  • 4 Calibration/metrology
  • Dimensional, electrical, hardness, etc.
  • 1 Food (chemistry and microbiology)
  • 1 Radiochemistry (water, soil, anatomic)
  • 1 Canadian, environmental
  • 1 Australian, multi-specialty

PT Program Details
  • Follows general A2LA policies, except
    accreditation cycle
  • Team members Lead, Technical, Statistical
  • Four-year cycle of accreditation (TNI-2yr)
  • First year, 1-day surveillance visit for newly
    accredited PT providers (new practice)
  • Annual review of reports, internal audits,
    management reviews (8 hours or less)
  • Scope?
  • Assessment of subcontractors?

Varieties of Proficiency Testing (from CD17043)
  • Sequential measurement
  • Simultaneous (bulk) testing
  • Single time programs
  • Continuous programs
  • Measurable (quantitative) properties
  • Nominal and ordinal properties
  • Data transformation or interpretation
  • Inspection and sampling

Subcontracted Services
  • Manufacture of Proficiency Testing items
  • If not fully verified by PT provider
  • Reference results for assigned value
  • Tests for homogeneity and stability
  • Handling and shipping (not courier)
  • Data entry
  • Statistical analysis

Not permitted Subcontracting
  • G13 is silent on services that can be
    subcontracted (all or any or none)
  • ISO/IEC CD17043 (and ISO Guide 34) has
    prohibition of subcontracting
  • Design and planning of PT scheme
  • Evaluation of performance
  • Issuing of reports
  • Technical advisors / professional society may
    have contract relationship w/PTP

PT Laboratory Requirements
  • Applies to PT-related testing/calibration
  • Verification of PT item composition
  • Homogeneity and Stability
  • Reference Values
  • Lab (in-house or subcontractor) should be 17025
    accredited for appropriate Scope
  • If not accreditedshould we assess?

PT Assessment Reports
  • In the past A2LA has offered a choice of reports
    (Deficiency only or Narrative), but not really
  • We have provided the Deficiency Report
  • Non-conformances
  • Observations
  • We have provided the Assessors Report
  • Brief narrative introduction
  • Brief summary of the assessment, including
    positive comments
  • What does A2LA want?

Assessing PT Providers vs. Laboratories -
  • Management system requirements (mostly the same)
  • Technical competence of staff
  • Adequate staff, equipment, facility
  • Laboratory testing competence
  • Need to include program-specific requirements

Assessing PT Providers vs. Assessing Laboratories
  • Customer is different
  • Participant laboratory is main customer
  • Also lab accreditor, regulator, client
  • Product is different
  • PT result evaluation
  • PT scheme summary report
  • PT test item
  • Subcontractors are different
  • Dont share areas of competence w/PTP

PT Providers vs. Laboratories
  • Processes are different
  • Scheme planning and design
  • Use of advisory experts
  • PT item production
  • PT item validation
  • Determination of Assigned Value
  • Determination of evaluation criteria
  • Data processing
  • Writing reports

PT Providers vs. Laboratories
  • Technical assessments are different
  • Plan needs to be complete and appropriate
  • Items need to be similar to client samples
  • Manufacture process and reference values need to
    be adequate
  • Instructions need to be appropriate and
  • Reports need to be clear and informative

ILAC G132007
  • Management system from 170252005
  • Technical requirements revised slightly from
  • Eliminate redundant requirements
  • Clarify wording
  • Add considerations for medical EQA
  • No controversial changes from version 1 to
    version 2.

Management System Requirements
  • Currently follow ISO 170252005 and ISO
  • Quality Management System
  • Organization and Management
  • Document Control
  • Contract Review
  • Use of Subcontractors
  • Purchasing Services and Supplies
  • Service to Customer

Management System Requirements
  • continued
  • Complaints
  • Non-conforming Activities
  • Improvements
  • Corrective Action
  • Preventive Action
  • Records
  • Internal Audits
  • Management Reviews

Management System Highlights
  • Organization Named positions
  • Quality Manager
  • Coordinator
  • Quality Policy Statement includes G13 and any
    relevant program requirements
  • Document Control
  • Software for analysis
  • External documents

MS requirements Review of requests, tenders,
  • Review all contracts and requests
  • New schemes, additions to current scheme
  • Assure Provider has capability
  • Technically appropriate for participants
  • Review includes work by subcontractors
  • Inform participants of changes

Subcontracting Services
  • Assure competence of subcontractors
  • Comply with relevant parts of G13
  • Notify participants of subcontracted services
  • Provider accepts responsibility for subcontracted
  • Maintain register of all subcontractors and
    competence assessments

Service to the Customer
  • Many types of customers need to service all
  • Participant laboratories
  • Accreditation bodies/regulators
  • Laboratory customers
  • Actively solicit feedback
  • No need for customer satisfaction survey
  • Request for comment on response form is adequate

Management System Highlights
  • Complaints lots of opportunities
  • Nonconforming work includes undiscovered
    inhomogeneity, instability, incorrect statistical
    analysis and reports
  • Records may have retention times specified by
    regulation, accreditation body
  • Technical records include results of homogeneity
    and stability testing, and all reports

Technical Requirements
  • General
  • Personnel
  • Accommodation and environment
  • Organisation and design
  • Choice of method
  • Conduct of Schemes
  • Data Analysis and Interpretation
  • Communication with Participants
  • Confidentiality
  • Collusion

  • Provider must have experience with ILC and the
    particular type of tests items
  • Provider or subcontractor must have competence to
    make necessary measurements (e.g., homogeneity)
  • This may not be possible in new areas
  • ISO/IEC 17025 or ISO 15189 can be used to
    demonstrate competence in testing

  • Define minimum levels of competence needed by key
  • Ensure technical management has necessary
    competence and experience
  • Ensure sufficient staff and experience
  • Assure staff receives sufficient training, as
    needed. Objectively assess competence
  • Maintain training records

Accommodation and Environment
  • Ensure appropriate facilities for design and
    operation of schemes
  • Sample reception and handling
  • Sample manufacturing, storage, dispatch
  • Data communications and records
  • Appropriate separation of facilities to prevent
  • Safe decontamination and disposal of potentially
    toxic or hazardous materials

Accommodation and Environment
  • 3.2.2 The provider shall ensure that all
    facilities provided are safe, and meet
    appropriate health and safety requirements. Where
    appropriate, written procedures for the
    decontamination of items of equipment and working
    space shall be available.
  • Note Facilities should meet all safety
    (including biosafety) requirements relevant to
    personnel, the environment and the community,
    although it may be beyond the jurisdiction of the
    accreditation body to inspect for these
  • National or international requirements shall
    apply as appropriate.

Organization and Design
  • Identify and plan all processes that affect the
    quality of the scheme
  • Assure plan is carried out according to the
    defined procedures ( a-p)
  • Types of labs to participate
  • Number of participants
  • Source and nature of test items
  • Outline the statistical analysis
  • Basis of performance evaluation

Organization and Design
  • Shall ensure access to technical expertise and
    detailed experience (including statistics),
    perhaps with advisory group
  • a-p
  • Provide procedures and resources for procurement
    or manufacture of test items, following legal and
    ethical requirements
  • Have procedures and resources for test item
    storage and delivery
  • Should produce extra items

Non-analytic Sources of Error
  • NOTE PT can test other aspects of test quality
  • How does a laboratory respond to inadequate
  • What does a laboratory do when there is
    insufficient information with a sample?
  • Can they interpret the result?
  • Does the laboratory understand the method?
  • (Limit this to situations that really happen)

Preparation - Test Item Integrity
  • Same matrix as routine samples, where possible
  • Assure items are sufficiently homogeneous
  • ISO 13528 assure that inhomogeneity does not
    impact evaluation of performance
  • Inhomogeneous material allowed, if accounted for
  • Assure items are sufficiently stable, with no
    significant change throughout the conduct of the
  • Q can Guide 34/35 demonstrate competence, if ref
    material same as PT item?

Test item Homogeneity
  • Tested on items in final packaged form
  • Select a statistically random, representative
    sample of test items.
  • Documented procedure and accepted statistical
  • ANOVA for statistical significance
  • Between-Sample SD relative to evaluation interval
    (IUPAC 1993 and ISO 13528)
  • Expanded interval per IUPAC (2006)

Test item Homogeneity
  • There are many different ways to test
    homogeneity, no matter the statistics
  • Choice of statistical procedure
  • Number of samples
  • Selection method
  • Test for trend
  • Assessors need to understand the PT and the
    requirement (what is sufficient)
  • Goal is to limit the effect on evaluation

Test item Stability
  • Tested periodically over a range of storage
    conditions prior to distribution (for held
    samples, or remedial PT).
  • Ensure no significant change over conduct of
    proficiency test.
  • Demonstration of stability does not necessarily
    require testing

Homogeneity and Stability
  • Required for all measurands?
  • (TNI-Yes, A2LA maybe not)
  • Can homogeneity be demonstrated with participant
  • (TNI-No, A2LA Yes, for Nominal properties and
    with other conditions)
  • Can stability be demonstrated with participant
  • (TNI A2LA Yes, with other conditions)

Statistical Design
  • Statistical model and analysis techniques must be
    documented, with reasons for selection.
  • Considerations
  • Trueness and precision required for test
  • Minimum number of participants
  • Replicate measurements on each sample
  • Method of determining assigned value
  • Procedure to accommodate outliers
  • Homogeneity and stability
  • Conduct pilot scheme if necessary

Statistical Design
  • a-h
  • Note 2 In cases where a proficiency test scheme
    involve an insufficient number of participating
    laboratories to allow for a statistically
    meaningful analysis of results, the scheme
    provider must document, and provide to
    participants, details of the alternative
    approaches used to analyze the scheme data.

Choice of Method or Procedure
  • Participants normally allowed to use method of
    choice, but special methods may be required by PT
  • PT provider must request details of method used,
    to allow analysis and comment
  • Often (TNI) require a statement in instructions
    that labs must use routine method

Conduct of PT - Instructions
  • Participants must know when PT will be conducted
  • Detailed instructions include details on factors
    that can influence results
  • Specific instructions on storage, reporting
    units, significant figures, end date for receipt
    of results
  • Participant required to treat PT as usual samples
  • No disclosure of assigned values until done (may
    give ranges, as appropriate)
  • Give preliminary results, where appropriate

Conduct of PT Sample Handling
  • Avoid contamination during preparation and
    storage, by segregation of materials
  • Avoid damage or deterioration by proper packaging
    and storage
  • Secure storage areas, authorized dispatch
  • Assess materials periodically to detect

Conduct of PT Sample Packaging
  • Conform with appropriate packaging and labeling
    regulations (e.g., Customs)
  • Consider sample integrity
  • May require detailed instructions to participants
    on subsequent shipping of items
  • Include instructions on transport to other
    participants, when that is part of the scheme
  • Assure labels are securely attached and remain

Conduct of PT Sample Packaging
  • Have a procedure to confirm delivery of
    PT items.
  • Not meant to require recognized courier
  • Allow statement such as an advance notice to
    notify PT provider if shipment does not arrive
    before a specified date.

Data Analysis Equipment
  • Adequate data processing equipment
  • Designated person responsible for DP
  • Documented procedures for maintenance and
    validation of all DP equipment
  • Include back-up and system recovery plan
  • Documented data entry procedures
  • Include verification of data entry
  • Include data transfer
  • Include statistical analysis
  • Records retained for specified period

Data Analysis Statistical Analysis
  • Appropriate summary statistics
  • Control influence of extreme results
  • Outlier removal
  • Robust methods (preferred)
  • Procedures to dealing with results that cannot be
    evaluated (gross errors)
  • Procedures to detect test items (data sets) that
    cannot be evaluated

Data Analysis Evaluation
  • Procedure for appropriate evaluation, including
    the basis of evaluation
  • Advisory Group should provide commentary on
  • Compare prior expectations
  • Compare previous schemes
  • Compare methods or procedures
  • Discuss sources of error
  • Other suggestions

Data Analysis Reports
  • Include information on individual and group
    performance, including distribution of results
    from all participants
  • Can be in summarized form
  • Reports available within specified timetables
  • Interim reports may be needed, if scheme takes
    long time to complete

Data Analysis Reports
  • Detailed report content ( a-q)
  • Report should be stand-alone document, but for
    routine programs, some items can be available in
    other areas.
  • Minimum have lab reported result, assigned
    value, performance statistic, acceptable range
    for all measurands

Communications with Participants
  • Appropriate information for prospective
    participants on details of program
  • Advise participants (in writing) of changes
  • Procedures to appeal evaluations
  • Records of all communications

  • Identity of participants protected, known to
    minimum number of people
  • All information and communication is confidential
  • Confidentiality can be waived
  • If AB gets data from PTP, inform participants
    before they participate
  • If regulator gets PT data from PTP, inform
    participants in writing

  • Design PT to discourage collusion
  • Participants are responsible to ensure integrity
  • Signed attestation statement of non-collusion by
    laboratory director or technician may be

Annex A on Data Analysis
  • General concepts and references
  • ISO 13528
  • IUPAC Harmonized Protocol (2006)
  • ISO Guide 35
  • Topics referenced
  • Demonstration of homogeneity and stability
  • Determination of assigned value
  • Robust statistical techniques
  • Performance statistics
  • Evaluation of performance

  • Program requirements
  • First round assessments, 2006
  • Second round reassessments, 2008
  • Oversight activities and database

Program Requirements
  • A2LA requirements
  • PT program requirements
  • G132007
  • NELAC requirements
  • NELAC 2003 Chapter 2
  • Appendix B, D
  • Relevant parts of ISO 9000,17025, Guide 34, and
    NELAC Chapter 5 and appendices
  • TNI Vol. 3 will be used in Round 3 (maybe)

NELAC Environmental PTPs
  • Eight accredited providers
  • Chemistry (Soil, potable and non-potable water)
    7 providers (1 Micro only)
  • Microbiology qualitative and quantitative
  • Radiochemistry 1 provider
  • Future expansion?
  • WETT
  • Air

NELAC PTPA 1st Round
  • Initial assessments, January-June 2006
  • 2 providers were already A2LA accredited
  • 1 applicant did not receive accreditation
  • First class announcement July 1, 2006
  • Teams had same Statistician and Microbiologist
    for all providers, 4 different Chemists
  • Only one threat of litigation
  • No interim oversight

NELAC PTPA 2nd Round
  • Class anniversary July 2008 for all previously
    accredited providers
  • Assessments April-June (?)
  • Some applicants are likely to request full
    assessment as RMP (water, soil, pharmaceutical
  • Use varied team members for all technical areas
    (chem, micro, stats)

Oversight of PTPs
  • Database being developed (Neptune, Inc.)
  • Extensive oversight
  • Administrative changes
  • Changed assigned values
  • All analytes, ranges of levels
  • Current NELAC criteria
  • Consistent challenge by all providers
  • Unacceptable rates
  • Unresolved Complaints gt90 days
  • Complaints about PTP

The End
  • Thank you!