Surveillance - PowerPoint PPT Presentation

1 / 25
About This Presentation



Surveillance Findings/Analysis/CAPs ATAC 2010 – PowerPoint PPT presentation

Number of Views:146
Avg rating:3.0/5.0
Slides: 26
Provided by: Claude181


Transcript and Presenter's Notes

Title: Surveillance

Surveillance Findings/Analysis/CAPs ATAC 2010
Transport Canada Civil AviationPrairie Northern
  • David White
  • SMS Coordinator PNR
  • Oversight Coordination Division
  • Based in Edmonton, Alberta
  • (780) 495-5216

  • Define surveillance activities
  • Discuss surveillance procedures (SI-SUR-001)
  • Refer to areas of review, SMS, QAP,Ops Control
  • Analysing Issues identified during surveillance
  • Causal analysis (root cause)
  • Generating acceptable Corrective Action Plans
  • Effective communication during the process

What is an Assessment?
  • Assessment means the surveillance activity
    conducted to evaluate effectiveness and level of
    compliance with the CARs.
  • Staff Instruction (SI) SUR-001
  • Sometimes referred to as 17 PVIs.

What is a PVI?
  • A systematic review of one or more SMS components
    of an organization OR other regulated areas of
    a certificate holder.
  • Used to determine if regulatory requirements are
    documented, implemented, in use and effective
    within certified organizations.
  • It is a means to focus on the effectiveness of
    management systems make judgments on
  • Quality Assurance (QA) and the Ops control
    system are examples of management systems

Additional distinction between terms
  • Assessment Evaluation of entire SMS
  • Limited to companies that meet SMS requirements
  • PVI Evaluation of regulated area of companys
    control system or specific SMS component
  • Applicable to all companies
  • Process Inspection In-depth review of steps
    to obtain specific output
  • Our focus will be on Assessments/ PVIs here today.

Surveillance Activity Overview
  • The activity consists of
  • Documentation review (preparation)
  • On-site review (including interviews sampling)
  • Analysis
  • Reporting

Surveillance Procedures in SUR-001
  • Notification
  • Team selection
  • PVI plan
  • Preliminary doc review
  • Documentation review
  • Team meeting
  • Sampling procedures
  • On-site procedures
  • PVI team on-site briefing
  • On-site review
  • Interviewing
  • Analysis
  • PVI worksheets
  • Findings
  • Exit meeting
  • PVI report
  • PVI CAP procedures
  • PVI follow-up closure

Program Validation Inspection (PVI)
  • PVI will review the following areas for
    certificate holder without Quality Assurance (QA)
  • Part III - Airport operators obligations
  • Part IV - Company operational control
  • Part VII - Subpart 3 or 4 - Company operational
  • Part VII - Subpart 5 - Safety Oversight component
  • Part VIII - Operating certificate requirements

Quality Assurance Program
  • Regulations for certificate holders to have QA
  • Will be implemented in phased manner
  • PVIs for Parts III, IV, VII VIII without fully
    implemented QA Program use customized PVI
  • SUR-001 Appendix A contains expanded list of all
    PVI worksheets organized by operating rule

Quality Assurance Program
  • Certificate holders that have QAP
  • Inspectors use worksheet SUR-001 Appendix A Table
    5.1 Quality Assurance
  • This includes certificate pursuant to
  • CAR 561
  • STD 566
  • CAR 573
  • Maintenance requirements for all Part IV VII

Some Examples of Worksheets referenced in
SI-SUR-001     (k) TABLE 3.1 Reactive
Processes   (l) TABLE 3.2 Proactive
Processes   (m) TABLE 3.3 Investigation
Analysis (n) TABLE 3.4 Risk Management
(p) TABLE 5.1 Quality Assurance (r)
Program Validation Inspection Worksheet CAR 302
  (s) Program Validation Inspection Worksheet
CAR 406 (t) Program Validation Inspection
Worksheet CAR 703/704   (u) Program Validation
Inspection Worksheet CAR 801 (ee) Program
Validation Inspection Worksheet EASA NOTE
These Worksheets are internal documents only  
Surveillance Activity Responsibilities
  • Convening Authority
  • Identifies - Who
  • Defines - Scope depth
  • PVI/Assessment Manager
  • Plans
  • Manages
  • Team Member(s)
  • Conduct

Documentation Review
  • Doc review 4-6 weeks before on-site review
  • Doc review 1-2 weeks before on-site review PVI
  • Includes review of all files docs relevant to
    certificate holder and scope of the surveillance
  • Ensure all reference manuals/docs are available
    include latest amendments
  • Confirm revision status of any manual subject to
  • Review approved manuals for conformance to
  • Identify areas that require further review during
  • Develop specific questions for certificate holder
    and appropriate management based on intelligence
    gathered during review.

  • A general policy statement is not usually
    enough to satisfy requirements. Document review
    team members must verify that there is a policy
    in place and that it is support by an
    appropriately documented procedure or process as
  • This varies dependent on the area being reviewed
    and the type of certificate issued but is
    basically the Write what you do and do what you
    write principle.
  • TC may request additional documentation if the
    initial submission does not meet the teams

Onsite Activities
  • Entrance Meeting
  • Interviews ( Management and selected employees)
  • Sampling
  • Based on size, scope and complexity of the
  • Determined through process in SI-SUR-001
  • Used in conjunction with interviews to determine
    regulatory compliance
  • Non-compliance issues Identified
  • Purpose of the activity is to determine
    regulatory compliance.
  • Company briefing(s)
  • Exit meeting

Surveillance Findings
  • We will concentrate on PVI examples today.
  • Ineffective Quality Assurance
  • Loss of Operational Control
  • Examples of Non-compliance with regulatory
  • Examples of Non-compliance with Company
    Policy/Procedures and processes.

Surveillance Reporting
  • Report generated with findings.
  • Required actions determined through review with
  • Issues are typically handled through the
    Corrective Action Plan (CAP) process.
  • Required additional actions are based on the
    severity of non-compliance issues identified.
  • Additional actions may involve Certificate Action
    and/or Enhanced Monitoring.

Review of Findings
  • Recognizing the System Failure
  • Addressing the immediate concerns
  • Causal Analysis
  • Planning the short-term corrections
  • Planning the long term corrections
  • Informing TC

Causal Analysis
  • SI-SUR-001, Section 10.4 CAP Evaluation
  • A Root Cause Analysis of the Non-conformance
  • The certificate holders analysis shall include a
    summary of the root cause as well as any causal
    factors that may have contributed to the
    non-conformance. Even though the certificate
    holder may use a causal analysis method that is
    not familiar to the CAP reviewer (e.g. MEDA
    process, 5 Whys, etc.), the certificate holder
    must be able to demonstrate how they arrived at
    the root cause and it should be clear what caused
    the non-conformance to occur.

Causal Analysis
  • Causal Analysis seeks to identify and understand
    the reasons why things are as they are and hence
    enabling focus of change activity.
  • Some Examples
  • The 5 Whys
  • Meda
  • Fishbone
  • The key is just to keep asking until you get to a
    real 'aha' that you can treat.

CAP Submission
  • TC Inspector Expectations
  • Causal Analysis
  • Appropriate Timelines
  • Documentation
  • Communication
  • SI SUR-001 Section 10.4 (2) states
  • The role of the principal inspector
    responsible for the acceptance of the CAP shall
    be limited to the assessment of the process used
    by the certificate holder in reaching conclusions
    regarding the findings. It is not the inspectors
    role to second-guess the CAP solutions, but
    rather to evaluate the process used in developing
    those solutions.

CAP Acceptance and Follow-up
  • Remember it is your Plan
  • Monitor the implementation to ensure it is
  • Communicate Issues if implementation not as per
  • Document the issues and resolution with the
  • Be prepared for the Activity follow-upConfirming
    your systems are working not just that the
    examples identified during the inspection have
    been addressed

Surveillance Activity Closure
  • Effective systems are a continuing process.
  • Your responsibility for day to day compliance
  • Systems will mature and improve as we continue to
    use them.

Write a Comment
User Comments (0)