Managing IT Outsourcing Risks Auditing Vendor Management

1
Managing IT Outsourcing Risks Auditing Vendor
Management ISACA SD Meeting August 27,
2009 Ben Kotnik, CISA Benjamin.kotnik_at_am.sony.com
or bkotnik_at_yahoo.com
2
Why is Vendor Management Important?

• Money is lost
• Ongoing management and maintenance activities may
  be overlooked
• From Protivitis Managing Contract Risks Third
  Party Contract Audits whitepaper

3
Agenda

• IT Outsourcing Background and Discussion
• Risks
• Vendor Management (VM)
• Our Approach to Auditing VM

4
Other Helpful Information

• Any other success factors from the audience?
• We can try and include as much relevant
  information as possible
• Please ask questions and add your own opinion to
  the discussion
• Thank you for attending, I hope you get something
  out of the presentation.

5
IT Outsourcing Background and Discussion

• Business Drivers
• Types of Services
• Types of Delivery Models

6
IT Outsourcing Business Drivers

• Core Competencies Analysis
• Its difficult to do everything well. What
  distinguishes our organization from our
  competitors? Generally, companies will not
  outsource these differentiating factors.
• Many companies consider this element.
• Cost Savings
• Access to near-shore, off-shore labor markets are
  appealing due to decreased costs
• Potential competition in the marketplace to
  provide such services

7
IT Outsourcing Business Drivers

• Risk Mitigation
• Some processes present risks a company is not
  willing to manage alone. HR functions for
  example.
• Skills Assessment
• Need a particular skillset but may not want to
  hire it into the organization, or might require a
  supporting infrastructure that is costly or
  cumbersome.
• Duration of need
• A temporary need or will this be something we
  need to keep doing?
• Set it and forget it!
• Watch out for too much of this mentality

8
IT Outsourcing Types of Services

• Hosting Web, Application, Infrastructure
• Development
• Data Services
• Consulting
• Support
• Products Software, Hardware

9
IT Outsourcing Delivery Models

• Staff Augmentation (aka Personnel)
• Contractors report to a Company Manager as part
  of a project or ongoing staffing.
• Out-Tasking
• Specific tasks are left to outsourced provider,
  such as QA testing.
• Project-Based
• Entire project is delivered by the provider.
• Managed Service (aka Functional)
• A larger version of Out-Tasking. Entire IT
  function, such as production support is the
  responsibility of the provider.
• Build-Operate-Transfer (BOT)
• The provider builds a business, factory, or other
  good or service, and after a startup period is
  purchased by the client.

10
IT Outsourcing Delivery Models
11
Agenda

• IT Outsourcing Background and Discussion
• Risks
• Vendor Management (VM)
• Our Approach to Auditing VM

12
Risk Factors

• Risks can vary depending on the types of services
  provided and the type of delivery model being
  used
• Additional factors include
• Extent of outsourcing being performed by each
  vendor (over-reliance)
• Whether work is done remotely or onsite
• The existing control environment at the client
  and service company, especially managements
  level of involvement and scrutiny of vendor
  performance

13
Risks to Consider

• Confidentiality
• Integrity
• Availability
• Data Loss proprietary designs, personal and /
  or credit card data, material non-public
  information
• Fraud kickbacks, subcontractors, overstated
  billings, initiating and approving transactions
  (and possibly cover them up)
• Loss of tribal knowledge business processes,
  systems, data flows
• Financial Will it be possible to quantify
  whether the expected ROI is realized? Will there
  be a reduction in managements effort?
• Other process-specific risks

14
Discussion Risks by Service

• Hosting Web, Application, Infrastructure
• Development
• Data Services
• Consulting
• Support
• Products Software, Hardware

15
Discussion Risks by Delivery Model

• Staff Augmentation (aka Personnel)
• Contractors report to a Company Manager as part
  of a project or ongoing staffing.
• Out-Tasking
• Specific tasks are left to outsourced provider,
  such as QA testing.
• Project-Based
• Entire project is delivered by the provider.
• Managed Service (aka Functional)
• A larger version of Out-Tasking. Entire IT
  function, such as production support is the
  responsibility of the provider.
• Build-Operate-Transfer (BOT)
• The provider builds a business, factory, or other
  good or service, and after a startup period is
  purchased by the client.
• Discussion Legal recourse afforded by most
  contracts vs. In-house control

16
Risk Ranking

• A risk ranking is helpful for different reasons
• Prioritizes needs for monitoring and oversight
  between multiple outsourced activities
• Helps establish a schedule for reviewing
  Agreement documents
• Company strategy may change
• Vendors may become insolvent
• Contracts may expand beyond their original scope
• Keeps all stakeholders up to date and aware
• Suggestion consider Process and Vendor risks

17
Risk Ranking - Dimensions

• Process risks
• Volume of transactions
• Materiality for your company
• Number of vendors that offer such services
• Others TBD based on your risk assessment
• Vendor-specific risks
• Quantity of work performed and Quality of
  deliverables
• Delivery Model
• Off-site operations
• Track record in industry
• Ethics and potential for fraud
• Strength of Agreements
• Visibility into vendor operations

18
Risk Ranking A Graphic Approach
19
Risk Ranking - Outputs

• Periodic reviews of Agreements
• Higher risk reviewed more frequently
• Audit schedule
• Develop a rotational schedule to provide coverage
  of key processes and vendors
• Validate contract compliance
• Allocate vendor management resources appropriately

20
Agenda

• IT Outsourcing Background and Discussion
• Risks
• Vendor Management (VM)
• Our Approach to Auditing VM

21
Organizational Context

• Vendor Management is not an isolated activity,
  its closely linked with
• Project Management
• Contract Management
• Resource Management
• Procurement
• Accounts Payable

22
Vendor Management Assessing Maturity

• If the answers to the following questions are
  unclear, then the vendor management function may
  not be mature.

23
Vendor Management Example Maturity Questions
24
Vendor Management Example Maturity Questions
25
Vendor Management Typical Lifecycle

• Contracting Process
• Many companies do this well
• Day to Day Monitoring
• SLA performance
• Deliverables Quality and Timeliness
• Overall Performance Monitoring
• Compliance with contract terms and conditions
• Fulfillment of cost and quality objectives
• Scorecards or other tools to reward or penalize
  vendors
• Award new work based on performance

26
Agenda

• IT Outsourcing Background and Discussion
• Risks
• Vendor Management (VM)
• Our Approach to Auditing VM

27
Approaching a VM Audit

• Typical process-based audit might include
  procedures for specific contracts or service
  types, such as
• Contracting process RFP, competitive bids,
  Legal review, CSA, etc.
• Monitoring Performance metrics, forecasting,
  trends
• Billings and Payments

28
Approaching a VM Audit

• A Governance audit of Vendor Management as its
  own business function or process might be
  justified depending on the risks and magnitude of
  the outsourcing in place.
• Do you usually scope-out activities or processes
  performed by vendors in your audits?

29
Approaching a VM Audit

• How to handle affiliate / parent company service
  providers?
• Same as 3rd party vendor in many respects
• SLAs and defined expectations, roles and
  responsibilities, etc. are still nice to have
• Are other contract elements such as
  Confidentiality important?

30
Our Approach to a VM Audit - Scope

• The governance framework for contracting and
  managing vendors and service providers
• Performance metrics and service level agreements
• Performance monitoring activities
• Billing and payments
• We reviewed MSAs, supporting schedules, SOWs and
  other Amendments for the largest vendors and
  service providers.  We also interviewed
  approximately 40 employees from the IT group,
  Legal, Privacy, PCI, Procurement, and the
  Controllers group. 

31
Our Approach to a VM Audit
32
Vendor Management Audit Approach - Agreements

• Agreements
• Are all stakeholders involved in defining
  requirements or approving new MSAs and SOWs?
• Where are these documents stored?
• Are they accessible to all stakeholders,
  including process owners?
• Is there a process to review these agreements
  periodically?
• Is the purpose of each type of Agreement document
  defined?
• MSA, SOW, Extension, Change Order, Appendices,
  etc.

33
Vendor Management Audit Approach - Agreements

• Agreements (cont.)
• What services are allowed? Are they clearly
  defined in operational terms?
• Are compliance laws and regulations included
  (e.g. SOX, PCI)
• Adherence to policies over time. Which policies?
• Are the SLAs appropriate? Can they be
  manipulated?
• Help Desk tickets as an example

34
Vendor Management Audit Approach - Communications

• Communications
• Are all stakeholders aware of their roles in
  defining requirements, approving new MSAs and
  SOWs, performance monitoring, and assessing
  overall vendor performance?
• How are clarifications to the Agreements
  communicated?
• Is there transparency in what is included in the
  agreements?
• Are the required metrics or milestones known by
  all relevant personnel?
• What are the escalation protocols for day-to-day
  disagreements and for possible breaches of
  contract?

35
Vendor Management Audit Approach - Monitoring

• Monitoring
• Are all SLA metrics enforced?
• Who prepares performance data? Is it validated?
• How are other contract requirements enforced?
• Confidentiality of client data (including such
  things as PCI)
• Background checks of contractors
• Customer Satisfaction
• External factors and vendor health
• Monitoring activities should be designed to
  provide a direct input into future decisions

36
Assessing The SAS 70

• Statement on Auditing Standards No. 70 (SAS 70)
• Is the SAS 70 for the service(s) provided to the
  company?
• Are the dates appropriate?
• Are the control objectives appropriate?
• Do the controls tested support the full breadth
  of each control objective?
• Is it reasonably clear what test procedures were
  performed for each control?
• Does it cover all geographic locations?
• Are any processes or controls carved out?
• Consider AS 5 guidance for reliance
• Inherent bias in some SAS 70 reports
• http//www.isaca.org/Template.cfm?SectionHomeCON
  TENTID48425TEMPLATE/ContentManagement/ContentDi
  splay.cfm

37
Vendor Management Audit Approach - Maintenance

• Maintenance
• Are company objectives maximized?
• What is the process for renewing, expanding, or
  terminating? What information is available and
  considered?
• Is the company active or passive in enforcing
  contract terms and conditions?
• What criteria exists to help determine when a
  relationship should be terminated?
• Is there sufficient visibility and transparency
  into these processes?

38
Audit Clauses

• Specific Items to Include in a Right-to-Audit
  Clause
• In his book, Outsourcing, Downsizing, and
  Reengineering Internal Control Implications
  Albert Marcella Jr. recommends ten specific items
  to be included in a right to audit clause for a
  construction contract. These can be easily
  modified for a non-construction type contract
• References to specific records, such as original
  estimate files, change order estimate files, and
  detailed worksheets, subcontract, and supplier
  proposals for both successful and unsuccessful
  bidders, all project-related correspondence,
  subcontractor and supplier change order files
  (including detailed documentation covering
  negotiated settlements) back-charge logs and
  supporting documentation any records detailing
  cash, trade, or volume discounts earned and
  insurance proceeds, rebates, or dividends
  received.
• A specific requirement for the contractor to
  provide the owner with copies of records in
  computer readable format as well as hard copy.
• A general reference providing the right to audit
  any other supporting evidence necessary to
  substantiate charges related to the contract
  (both direct and indirect costs, including
  overhead allocations as they may apply to costs
  associated with the contract).
• A general reference providing the right to audit
  any records necessary to permit evaluation and
  verification of (a) contractor compliance with
  contract requirements, (b) compliance with the
  owner's business ethics policies, and (c)
  compliance with the provisions for pricing change
  orders, payments, or claims submitted by the
  contractor or any of his payees.

39
Audit Clauses

• Specific Items to Include in a Right-to-Audit
  Clause
• A general description of the length of time the
  contractor's records shall be subject to audit,
  such as "throughout the term of his contract and
  for a period of three years after final payment,
  or longer if required by law."
• A specific "flow-down right-of-audit provision"
  that requires the contractor to include the right
  to audit provisions in the contracts (including
  those of a lump sum nature) of all
  subcontractors, insurance agents, material
  suppliers, or any other business entity providing
  goods and services (specifically providing the
  right of the owner's representatives to examine
  their records).
• A specific provision that allows the owner to
  interview any of the contractor's current and
  former employees during the audit.
• A specific provision that the contractor will
  provide the owner with adequate and appropriate
  workspace, with access to photocopy machines.
• (Optional) A specific proviso that the owner will
  recoup the cost of the audit if the audit detects
  over-charges that reach or exceed a certain
  percentage of the total contract billings (for
  example, overcharges greater than .5 percent).
• (Optional) A specific proviso that the contractor
  will not only repay the owner within a specific
  period of time, but will also pay an additional
  percentage of the overcharges (for example, 1.5
  times the amount of overcharge) to the owner as
  liquidated damages.

40
Audit Clauses Best Practices

• Verbiage, such as "We warrant no gifts or
  gratuities were given or received, either
  directly or indirectly, to obtain this contract.
• Right to audit should be long enough to be
  consistent with regulations. Alternatively a
  general phrase like or longer as required by
  law could be included.
• Right to audit should extend to subcontractors,
  if they are used.
• A provision allowing the client to interview any
  of the vendor's current and former employees
  during the audit
• A provision within the right to audit clause
  allowing for the client, with probable cause, to
  examine the vendors books to identify
  inappropriate payments.
• Right to conduct an independent Audit, such as
  SAS 70

41
Role of Audit, Information Security, or
Compliance Groups

• Assess risks and address them through audits or
  reviews
• Work with the existing governance framework(s) to
  promote efficiencies and risk mitigation
• Engage decision-makers in discussions on the
  criteria used in determining what is outsourced.
• Would legal recourse truly make up for a controls
  breakdown? (e.g., data breach)

42
And on that note
43
Supplemental Information
44
Dilbert says
45
Suggested Reading

• For Government
• You Don't Always Get What You Pay for The
  Economics of Privatization by Elliott D. Sclar