Continuous Monitoring of Business Process Controls: A Pilot Implementation of a Continuous Auditing System at

1
Continuous Monitoring of Business Process
Controls A Pilot Implementation of a Continuous
Auditing System at

• Michael G. Alles
• Gerard Brennan
• Alexander Kogan
• Miklos A. Vasarhelyi

2
IT-enabled Business Processes (BPs)

• A business process is a set of logically related
  tasks performed to achieve a defined business
  outcome, Davenport and Short (1990).
• Modern information technology makes it possible
  to measure and monitor business processes at the
  unprecedented level of detail (disaggregation) on
  the real-time basis.
• Continuous auditing (CA) methodology can utilize
  the IT capability to capture BP data at the
  source and in the disaggregate and unfiltered
  form to achieve more efficient, effective and
  timely audit.

3
Two Approaches Towards Continuous Auditing of BPs

• The environment of highly automated and
  tightly-coupled BPs (e.g., an integrated
  enterprise system) enables CA based on continuous
  monitoring of process control settings.
• Unfortunately, there are numerous enterprise
  environments where process controls are either
  not automated or their settings are not readily
  accessible.
• In environments such as loosely-coupled legacy
  data processing systems, automated audit
  procedures of CA have to be data-oriented (tests
  of details and analytical procedures).

4
Ways of Verifying Existence, Correctness and
Functioning of BP Controls

• Verifying that the observations of a BP agree
  with the existence, correctness and functioning
  of a control benefit - can be applied even if
  controls are not directly accessible by the
  auditor problem - the observed behavior of the
  BP may not completely cover the whole range of
  control functions.
• Verifying by executing a prohibited BP behavior
  that it either cannot happen or is detected and
  compensated for problem - auditor has read-only
  access to the production system and cannot run
  penetration testing.
• Verifying that retrieved control settings stored
  in the enterprise system match the benchmark
  problem - relies on the assumption that the
  programming code of the control in the production
  system is correct (customization).

5
Conceptual View of Continuous Monitoring of BP
Control Settings

• The environment of highly automated and
  tightly-coupled BPs enables CA based on
  continuous monitoring of business process control
  settings.
• Online access to automated BP control settings
  (available in ERP systems) from the continuous
  assurance system.
• Enterprise-dependent benchmarks of acceptable
  control settings.
• Frequent (e.g., daily, hourly) comparison of
  actual settings with the benchmarks.
• Automatic generation of alarms in case of
  critical deviations, such as individual accounts
  without passwords, aggregation of weaknesses in
  certain control areas (e.g., segregation of
  duties).

6
Pilot Implementation of CMBPC Systems by Siemens
IT Internal Audit

• CA/R/Lab approached by Siemens IT internal audit
  to explore use of CA methodology to streamline
  audit of SAP system controls at Siemens USA.
• Siemens is uniquely SAP enabled with 60
  applications in the USA alone.
• IT internal audit examines sites on a 2 year
  cycle labor intensive, costly.
• Internal audit needed to find resources to take
  on 404 implementation without increase in
  headcount.

7
CA Value Proposition as Seen by Siemens
8
Siemens Pilot Leverages Audit Action Sheets of
External Auditor

• Key success enabler for pilot is the ability to
  use AASs provided by KPMG (Germany).
• Provides template for CA analytics 1. No need to
  reinvent the wheel. 2. Immediate buy in from
  relevant parties. 3. Satisfies external auditor.
• 300 AASs need to determine which ones can be
  CA-enabled.
• AASs included a mixture of tasks some of which
  could only be accomplished by a human, such as
• interviewing the client about their
  reconciliation procedures
• while some others involved well-scripted
  interactions with the clients enterprise system
  such as
• execute a certain SAP R/3 transaction and/or
  report and verify that its result is as
  specified.

9
Audit Action Sheet Example
10
AAS Procedure and Rating
11
AAS Formalization
12
Existing IT Audit Procedures at Siemens
13
Steps of Piloting a CA Implementation

• Determining the best mode for the continuous
  monitoring of the chosen BP controls.
• Developing a system architecture for this task
  (either a monitoring and control layer or an
  embedded audit module).
• Designing the interaction and integration between
  the CA mechanism and the ERP system.
• Developing guidelines for the formalization of
  the AAPs into a computer executable form,
  including the determination of those AAPs which
  are automatable and those requiring
  reengineering.
• Creating processes for managing the alarms
  generated by the automated CA system and putting
  in place the required set of audit trails.
• Formulating a change management plan to move the
  project from the pilot stage to industrial
  strength software.

14
System Architecture for Continuous Monitoring of
BP Control Settings

• Continuous assurance system Embedded audit
  modules (EAMs) vs. independent control and
  monitoring layer (CML).
• EAMs are tightly coupled with the enterprise
  system ?
• can be triggered by suspicious events, but
• are intrinsically more vulnerable to
  manipulation.
• CML relies on remote (read-only) access to the
  enterprise system ?
• its code and environment are well-protected, but
• cant query the enterprise system too often, and
  therefore may miss suspicious enterprise events.
• CML implementation is less reliant on the
  cooperation of the enterprise personnel.

15
Interaction of CA System with ERP System

• Modern enterprise systems have a 3-tier
  architecture (presentation, application,
  database).
• CML can query the system through the application
  tier using its APIs (e.g., BAPIs in SAP R/3) or
  EAM can be implemented as a sub-module of the
  application (e.g., Coded in ABAP in SAP R/3).
• CML can query the enterprise database directly
  (using SQL through ODBC) or EAM can be
  implemented as a trigger (written in SQL) stored
  in the database.
• The latter approach is strongly resented by
  enterprise personnel and (in the case of EAM) de
  facto prohibited by enterprise software vendors.

16
Architecture of Generic CMBPC System
Enterprise System
CMBPC System
Database of System Settings and Audit Trail
Presentation Tier
Internet
Application Tier
Program Logic and User Interface
Database Tier
17
Pilot CMBPC System at Siemens
18
Siemens Audit Leverage Tool SALT

• The SALT pilot was prototyped using the formal
  Siemens SAP audit process in the basis area (the
  application layer operating system for SAP)
  covering the application level controls
  applicable to any SAP system.
• 12 AASs were chosen as representative of
  challenges in formalizing and reengineering.
• SAP E-audit download provided sample data for the
  model.
• The pilot was developed in Visual Basic to serve
  as a test environment for evaluating technical
  research questions regarding continuous auditing
  / assurance.

19
Screenshots from SALT
20
Reengineering and Formalization in CA
Implementation

• Implementation of CMBPC starting with a clean
  slate is the most logical and efficient solution,
  BUT impractical clearing it with external
  auditor presents tremendous complications and
  delays.
• Automation requires formalization of AASs some of
  which are formalizable while others are not (such
  as those that require human observation of BPs
  and interviewing enterprise personnel).
• Unavoidability of reengineering stems from the
  necessity for the formalizable and
  non-formalizable parts of the audit program to be
  identified and handled separately.
• Formalizable procedures are to be separated out,
  automated (by two experienced auditors!) and
  executed with high frequency.
• Formalization creates uniformity and assures that
  every implemented procedure is state of the art.

21
Control and Alarm Hierarchy and Its Management

• Organization of controls identify risk areas,
  break them down into sub-areas, and develop
  controls to eliminate or mitigate these risks ?
  enterprise controls form a top-down hierarchy.
• Active component of CMBPC automatically
  triggered audit alarms critical exception
  conditions to trigger alarms.
• Problem associated with the automatic generation
  of alarms in CMBPC the alarm flood either
  initial flood or overly conservative benchmarks.
• The alarms form a hierarchy corresponding to and
  derived from the control hierarchy ? hierarchical
  alarm management.
• Enabled/disabled alarm flag disabled in a
  node overrides the settings in all the children
  nodes down the hierarchy tree.
• The set of alarm recipients specified in a node
  applies by recursion to all the children nodes
  down the hierarchy tree.

22
Developers of Continuous Monitoring Software

• Three main categories enterprise software
  vendors, large public accounting firms, and
  established audit software vendors.
• Enterprise software vendors traditionally
  provided very limited CM capabilities within
  their systems (e.g., SAPs Audit Information
  System) often quoted reason lack of demand
  (assurance does not contribute directly to the
  bottom line!).
• Large public accounting firms have been
  experimenting with CM software (e.g., KPMGs
  KOLA), but remain ambivalent (and question the
  value proposition, ROI, ).

23
Developers of Continuous Monitoring Software - II

• Established audit software (CAAT) vendors have
  domain knowledge and well-developed libraries of
  audit tests, and see an opportunity to leverage
  this intellectual property in the emerging field
  of CA (e.g., ACL Continuous Controls Monitoring
  solution for purchase-to-payment cycle).
• SarbOx has created a window of opportunity to
  sell CA software as a Section 404 compliance
  tool.
• Large internal audit shops have been implementing
  ad-hoc CA-type procedures to mitigate business
  risks in certain high impact areas, and to
  achieve labor savings due to automation of audit
  tasks.

24
Continuing Work

• Developing a methodology for formalizing audit
  procedures.
• Extending CA higher up the audit value
  chaindealing with auditor judgment.
• Managing audit alarms and preventing alarm
  floods.
• Establishing ROI from the pilot.
• Scaling up to the production level.
• Technology is available, laws and standards are
  (mostly) in place, time is now.
• How to make it worthwhile (trade-off between
  cost, effectiveness, efficiency and timeliness)?