Managing the Business Risk of Fraud using Sampling and Data Mining - PowerPoint PPT Presentation

Loading...

PPT – Managing the Business Risk of Fraud using Sampling and Data Mining PowerPoint presentation | free to download - id: 3bb937-MWVmM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Managing the Business Risk of Fraud using Sampling and Data Mining

Description:

CPE presentation 12/3-4,/2009 Raleigh, NC. Managing the Business Risk of Fraud using Sampling and Data Mining – PowerPoint PPT presentation

Number of Views:220
Avg rating:3.0/5.0
Slides: 257
Provided by: ezrstatsC
Learn more at: http://ezrstats.com
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Managing the Business Risk of Fraud using Sampling and Data Mining


1
Managing the Business Risk of Fraud using
Sampling and Data Mining
Mike Blakley
Presented to
Fall 2009
2
PWC Global Survey Nov, 2009 Economic crime in
a downturn
  • Sharp rise in accounting fraud over the past 12
    months
  • Accounting fraud had grown to 38 percent of the
    economic crimes in 2009
  • Employees face increased pressures to
  • meet performance targets
  • keep their jobs
  • keep access to funding

3
Survey findings
  • Greater risk of fraud due to increased incentives
    or pressures
  • More opportunities to commit fraud, partially due
    to reductions in internal finance staff
  • While companies are expecting more fraud, they
    have not done much
  • People who look for fraud are more likely to find
    it

4
Session objectives
  • Understand the framework for managing the
    business risk of fraud
  • Plan, perform and explain statistical sampling in
    audits
  • Reduce audit costs using data mining, sequential
    sampling and other sampling techniques
  • Apply SAS 56, the new SAS suite and the revised
    (2007) Yellow Book.
  • Run, hands-on, the most productive analytic
    technique (regression analysis).
  • Use data mining to introduce greater efficiency
    into the audit process, without losing
    effectiveness.

5
Session agenda - 1
  • Introduction and the Process for Managing the
    Business Risk of Fraud
  • Introductions All Around
  • Course Objectives
  • Framework of risk management for fraud
  • Fundamentals of data mining
  • Data mining The Engine That Drives analysis
  • Analytics and Regression
  • Sources of Analytics Data
  • Basic and Intermediate ARTs
  • SAS 56
  • IIA Practice Advisory 2320
  • The Yellow Book (2007 revision)
  • The Guide Managing the Business Risk of Fraud

6
Session Agenda (contd) Sampling refresher
  • Sampling
  • The sampling process
  • Sampling methods
  • RAT-STATS
  • Random Numbers
  • Determining Sample Size
  • Case Study
  • Attribute sampling
  • Variable Sampling
  • Case study
  • Stratified Sampling
  • Obtaining and Interpreting the results
  • Other Sampling Approaches
  • DCAA Audit Package
  • Sequential Sampling
  • Overview of the process
  • Attribute Sampling
  • Variable Sampling

7
Session Agenda (contd) Linear regression as
an audit tool
  • Regression Analysis
  • Overview
  • Terms
  • Statistical basis
  • Charting Regression Seeing Is Believing
  • Plotting Data
  • Inserting a Trend line
  • Statistical Intervals
  • Confidence Intervals
  • Prediction Intervals
  • Calculation of Statistical Confidence Bounds
  • Case Study - Wake County Schools Bus Maintenance

8
Session Agenda (contd) Data mining, or How to
test 100
  • Overview
  • Statistical Basis
  • Data Conversion and Extraction
  • Data mining objectives
  • Classification
  • Trends
  • Identification of extremes
  • Major types of data analysis
  • Numeric
  • Date
  • Text

9
Session Agenda (contd) Excel as an Analytics
tool
  • Macros
  • Tools Data Analysis
  • The Macro facility
  • Adding a little class to your audit
  • VBA friend or foe

10
Handout (CD)
  • CD with articles and software
  • PowerPoint presentation
  • More info at www.ezrstats.com

11
Cockroach theory of auditing
  • If you spot one roach.

12
Cockroach theory of auditing
  • There are probably 30 more that you dont see

13
Statistics based roach hunting
Many frauds coulda/woulda/shoulda been detected
with analytics
14
Overview
  • Fraud patterns detectable with digital analysis
  • Basis for digital analysis approach
  • Usage examples
  • Continuous monitoring
  • Business analytics

15
The Why and How
Objective 1
  • Three brief examples
  • ACFE/IIA/AICPA Guidance Paper
  • Practice Advisory 2320-1
  • Auditors Top 10
  • Process Overview
  • Who, What, Why, When Where

16
Example 1 Wake County Transportation Fraud
Objective 1a
  • Supplier Kickback School Bus parts
  • 5 million
  • Jail sentences
  • Period of years

17
Too little too late
Objective 1a
  • Understaffed internal audit
  • Software not used
  • Data on multiple platforms
  • Transaction volumes large

18
Preventable
Objective 1a
  • Need structured, objective approach
  • Let the data talk to you
  • Need efficient and effective approach

19
Regression Analysis
Objective 1
  • Stepwise to find relationships
  • Forwards
  • Backwards
  • Intervals
  • Confidence
  • Prediction

20
Data outliers
Objective 1
  • Sometimes an out and out Liar
  • But how do you detect it?

21
Data Outliers
Objective 1
  • Plot transportation costs vs. number of buses
  • Drill down on costs
  • Preventive maintenance
  • Fuel
  • Inspection

22
Scatter plot with prediction and confidence
intervals
23
Cost of six types of AIDS drugs
Example 2
Objective 1a
24
Medicare HIV Infusion Costs
Objective 1
  • CMS Report for 2005
  • South Florida - 2.2 Billion
  • Rest of the country combined - .1 Billion

25
Pareto Chart
Objective 1
26
Typical Prescription Patterns
Objective 1a
Example 2
27
Prescriptions by Dr. X
Example 2
Objective 1a
28
Off-label use
Example 2
Objective 1a
  • Serostim
  • Treat wasting syndrome, side effect of AIDS, OR
  • Used by body builders for recreational purposes
  • One physician prescribed 11.5 million worth (12
    of the entire state)

29
Revenue trends
Example 3
Objective 1a
30
Dental Billings
Example 3
Objective 1a
31
Guidance Paper
Objective 1b
  • A proposed implementation approach
  • Managing the Business Risk of Fraud A Practical
    Guide http//tinyurl.com/3ldfza
  • Five Principles
  • Fraud Detection
  • Coordinated Investigation Approach

32
Managing the Business Risk of Fraud A Practical
Guide
Objective 1b
  • ACFE, IIA and AICPA Exposure draft issued
    11/2007, final 5/2008
  • Section 4 Fraud Detection

33
Guidance Paper
  • Five Sections
  • Fraud Risk Governance
  • Fraud Risk Assessment
  • Fraud Prevention
  • Fraud Detection
  • Fraud Investigation and corrective action

34
Risk Governance
  • Fraud risk management program
  • Written policy managements expectations
    regarding managing fraud risk

35
Risk Assessment
  • Periodic review and assessment of potential
    schemes and events
  • Need to mitigate risk

36
Fraud Prevention
  • Establish prevention techniques
  • Mitigate possible impact on the organization

37
Fraud Detection
  • Establish detection techniques for fraud
  • Back stop where preventive measures fail, or
  • Unmitigated risks are realized

38
Fraud Investigation and Corrective Action
  • Reporting process to solicit input on fraud
  • Coordinated approach to investigation
  • Use of corrective action

39
60 Minutes World of Trouble
  • 2/15/09 Scott Pelley
  • Fraud Risk Governance one grand wink-wink,
    nod-nod
  • Fraud Risk Assessment - categorically false
  • Fraud Prevention my husband passed away
  • Fraud Detection - We didn't know? Never saw one.
  • Fraud Investigation and corrective action -
    Pick-A-Payment losses 36 billion

40
Section 4 Fraud Detection
Objective 1b
  • Detective Controls
  • Process Controls
  • Anonymous Reporting
  • Internal Auditing
  • Proactive Fraud Detection

41
Proactive Fraud Detection
Objective 1b
  • Data Analysis to identify
  • Anomalies
  • Trends
  • Risk indicators

42
Fraud Detective Controls
  • Operate in the background
  • Not evident in everyday business environment
  • These techniques usually
  • Occur in ordinary course of business
  • Corroboration using external information
  • Automatically communicate deficiencies
  • Use results to enhance other controls

43
Examples of detective controls
  • Whistleblower hot-lines (DHHS and OSA have them)
  • Process controls (Medicaid audits and edits)
  • Proactive fraud detection procedures
  • Data analysis
  • Continuous monitoring
  • Benfords Law

44
Specific Examples Cited
Objective 1b
  • Journal entries suspicious transactions
  • Identification of relationships
  • Benfords Law
  • Continuous monitoring

45
Data Analysis enhances ability to detect fraud
Objective 1b
  • Identify hidden relationships
  • Identify suspicious transactions
  • Assess effectiveness of internal controls
  • Monitor fraud threats
  • Analyze millions of transactions

46
Continuous Monitoring of Fraud Detection
  • Organization should develop ongoing monitoring
    and measurements
  • Establish measurement criteria (and communicate
    to Board)
  • Measurable criteria include

47
Measurable Criteria number of
  • fraud allegations
  • fraud investigations resolved
  • Employees attending annual ethics course
  • Whistle blower allegations
  • Messages supporting ethical behavior delivered by
    executives
  • Vendors signing ethical behavior standards

48
Management ownership of each technique implemented
  • Each process owner should
  • Evaluate effectiveness of technique regularly
  • Adjust technique as required
  • Document adjustments
  • Report modifications needed for techniques which
    become less effective

49
Practice Advisory 2320-1 Analysis and Evaluation
  • International standards for the professional
    practice of Internal Auditing
  • Analytical audit procedures
  • Efficient and effective
  • Useful in detecting
  • Differences that are not expected
  • Potential errors
  • Potential irregularities

50
Analytical Audit Procedures
  • May include
  • Study of relationships
  • Comparison of amounts with similar information in
    the organization
  • Comparison of amounts with similar information in
    the industry

51
Analytical audit procedures
  • Performed using monetary amounts, physical
    quantities, ratios or percentages
  • Ratio, trend and regression analysis
  • Period to period comparisons
  • Auditors should use analytical audit procedures
    in planning the engagement

52
Factors to consider
  • Significance of the area being audited
  • Assessment of risk
  • Adequacy of system of internal control
  • Availability and reliability of information
  • Extent to which procedures provide support for
    engagement results

53
Peeling the Onion
Objective 1c
54
Fraud Pattern Detection
Objective 1d
55
Digital Analysis (5W)
Objective 1e
A little about the basics of digital analysis.
  • Who
  • What
  • Why
  • Where
  • When

56
Who Uses Digital Analysis
Objective 1e
  • Traditionally, IT specialists
  • With appropriate tools, audit generalists (CAATs)
  • Growing trend of business analytics
  • Essential component of continuous monitoring

57
What - Digital Analysis
Objective 1e
  • Using software to
  • Classify
  • Quantify
  • Compare
  • Both numeric and non-numeric data

58
How - Assessing fraud risk
Objective 1e
  • Basis is quantification
  • Software can do the leg work
  • Statistical measures of difference
  • Chi square
  • Kolmogorov-Smirnov
  • D-statistic
  • Specific approaches

59
Why - Advantages
Objective 1e
  • Automated process
  • Handle large data populations
  • Objective, quantifiable metrics
  • Can be part of continuous monitoring
  • Can produce useful business analytics
  • 100 testing is possible
  • Quantify risk
  • Repeatable process

60
Why - Disadvantages
Objective 1e
  • Costly (time and software costs)
  • Learning curve
  • Requires specialized knowledge

61
When to Use Digital Analysis
Objective 1e
  • Traditional intermittent (one off)
  • Trend is to use it as often as possible
  • Continuous monitoring
  • Scheduled processing

62
Where Is It Applicable?
Objective 1e
  • Any organization with data in digital format, and
    especially if
  • Volumes are large
  • Data structures are complex
  • Potential for fraud exists

63
Disadvantages of digital analysis
  • Cost
  • Software
  • Training
  • Skills not widely available
  • Time consuming
  • Development costs
  • Testing resources

64
Objective 1 Summarized
Objective 1
  • Three brief examples
  • CFE Guidance Paper
  • Top 10 Metrics
  • Process Overview
  • Who, What, Why, When Where

65
Objective 1 - Summarized
  • Understand the framework for managing the
    business risk of fraud
  • Plan, perform and explain statistical sampling in
    audits
  • Reduce audit costs using data mining, sequential
    sampling and other sampling techniques
  • Apply SAS 56, the new SAS suite and the revised
    (2007) Yellow Book.
  • Run, hands-on, the most productive analytic
    technique (regression analysis).
  • Use data mining to introduce greater efficiency
    into the audit process, without losing
    effectiveness.

Next is plan, perform
66
Statistical Sampling
  • Brief History / Timeline
  • Overview
  • Attribute Sampling Compliance
  • Variable Sampling Numeric Estimates

67
History of Sampling
  • Basis is two laws/theorems of probability
  • Law of Large Numbers
  • Central Limit Theorem

68
Law of large numbers
69
Time Line - LLN
  • Indian mathematician Bramagupta 600 AD
  • Italian mathematician Cardon 1500s
  • Statement without proof that empirical statistics
    improve with more trials

70
Time line LLN (continued)
  • Jacob Bernoulli first to prove in 1713
  • Foundation for central limit theorem

71
Central limit theorem
Classic measure
Mean of a sufficiently large number of random
samples will be approximately normally
distributed.
72
The traditional explanation
73
Central Limit Theorem
  • See it in action today
  • Any population
  • Large number of samples
  • Average is normally distributed

74
History of Central Limit Theorem
  • French mathematician Abraham de Moivre
  • 1733 approximate distribution from tossing coin
    (heads/tails)
  • Ho hum reaction
  • French Mathematician LaPlace expanded it
  • Ho hum reaction

75
History of CLT (contd)
  • Russian mathematician Lyapunov
  • Proof in 1901
  • Same reaction

76
Industrial revolution
Manufacturing
Engineering
Excitement!
77
Students T
William Gosset - 1908
Guinness Brewery
78
SAS 39
  • Effective June, 1983
  • Exposure draft for revision in 2009

79
Attribute sampling
  • Buonaccorsi (1987)
  • Refined calculations
  • Few software packages use it

80
Overview
  • Sample size calculations
  • Attribute sampling
  • Variable sampling
  • Random number generators

81
Sample size calculation
  • Its a guess
  • Every package different answer
  • Need to know the population
  • But thats why youre taking a sample!

82
Attribute Sampling Using RAT-STATS
  • Unrestricted populations

83
Session Objectives
  • Understand what is attribute sampling and when to
    use it
  • Understand unrestricted populations
  • Overview of the process using RAT-STATS
  • Understand the formula behind the computations

84
Attribute sampling
  • Attribute
  • Compliance testing
  • Signatures on approval documents, attachment of
    supporting documentation, etc.

85
Statistical approach
  • Recommended
  • Economical
  • Efficient
  • Requires determination of a sample size

86
Overview of process
  • Determine the sampling objective
  • Confidence
  • Precision
  • Determine required sample size
  • Identify samples to be selected based upon random
    numbers
  • Pull the sample and test
  • Compute the sampling results (i.e. estimate of
    range)

87
How this is done in RAT-STATS
  • The sampling parameters are first developed by
    the auditor
  • RAT-STATS is used to compute sample size
  • RAT-STATS used to generate random numbers
  • Pull the sample and test
  • Enter results in RAT-STATS to compute estimates

88
Step 1 Develop sampling parameters
  • Size of population
  • Expected error rate
  • Required confidence
  • Required precision

89
Step 2 Obtain the random numbers
  • Done by entering info into RAT-STATS
  • Output can be a variety of sources
  • Text File
  • Excel
  • Microsoft Access
  • Print File

90
Step 3 Pull the sample
  • Each random number selected corresponds with an
    item
  • Put the selected item on a separate schedule

91
Step 4 - Test each selected item
  • Generally requires reviewing documents

92
Step 5 Compute the results
  • Enter summary information into RAT-STATS
  • Output can be in a variety of formats
  • Excel
  • Microsoft Access
  • Text File
  • Print File
  • Printer

93
Thats It!
  • Now well see an actual demo using the RAT-STATS
    software
  • Excel population of 5,000 invoices
  • Results of test of attributes stored in the
    worksheet

94
Variable Sampling Using RAT-STATS
  • Unrestricted populations

95
Session Objectives
  • Understand what variable sampling is and when to
    use it
  • Understand unrestricted populations
  • Overview of the process using RAT-STATS
  • Understand the formula behind the computations

96
Variable sampling
  • Variable
  • Estimating account balances
  • Estimating transaction totals

97
Statistical approach
  • Recommended
  • Economical
  • Efficient
  • Requires determination of a sample size

98
Overview of process
  • Determine the sampling objective
  • Confidence
  • Precision
  • Determine required sample size
  • Identify samples to be selected based upon random
    numbers
  • Pull the sample and test
  • Compute the sampling results (i.e. estimate of
    range)

99
How this is done in RAT-STATS
  • The sampling parameters are first developed by
    the auditor
  • RAT-STATS is used to compute sample size
  • RAT-STATS used to generate random numbers
  • Pull the sample and test
  • Enter results in RAT-STATS to compute estimates

100
Step 1 Develop sampling parameters
  • Probe sample
  • Statistical measure
  • Excel formula

101
Step 1 Develop sampling parameters
  • Size of population
  • Average value
  • Standard deviation

102
Step 2 Obtain the random numbers
  • Done by entering info into RAT-STATS
  • Output can be a variety of sources
  • Text File
  • Excel
  • Microsoft Access
  • Print File

103
Step 3 Pull the sample
  • Each random number selected corresponds with an
    item
  • Put the selected item on a separate schedule

104
Step 4 - Test each selected item
  • Generally requires reviewing documents
  • Example data contains both examined and
    audited value.

105
Step 5 Compute the results
  • Enter summary information into RAT-STATS
  • Output can be in a variety of formats
  • Excel
  • Microsoft Access
  • Text File
  • Print File
  • Printer

106
Thats It!
  • Now well see an actual demo using the RAT-STATS
    software
  • Excel population of 5,000 invoices
  • Audited values stored in the worksheet

107
Attribute Sampling Using RAT-STATS
  • Stratified populations

108
Session Objectives
  • Understand what is stratification and when to use
    it
  • Overview of the process using RAT-STATS

109
Stratified sampling
  • Strata
  • Homogenous
  • More efficient in some instances

110
Overview of process
  • Separation into strata
  • Determine the sampling objective
  • Confidence
  • Precision
  • Determine required sample size
  • Identify samples to be selected based upon random
    numbers
  • Pull the sample and test
  • Compute the sampling results (i.e. estimate of
    range)

111
How this is done in RAT-STATS
  • The sampling parameters are first developed by
    the auditor
  • RAT-STATS is used to compute sample size
  • RAT-STATS used to generate random numbers
  • Pull the sample and test
  • Enter results in RAT-STATS to compute estimates

112
Step 1 Develop sampling parameters
  • Size of population
  • Expected error rate
  • Required confidence
  • Required precision

113
Step 2 Obtain the random numbers
  • Done by entering info into RAT-STATS
  • Output can be a variety of sources
  • Text File
  • Excel
  • Microsoft Access
  • Print File

114
Step 3 Pull the sample
  • Each random number selected corresponds with an
    item
  • Put the selected item on a separate schedule

115
Step 4 - Test each selected item
  • Generally requires reviewing documents

116
Step 5 Compute the results
  • Enter summary information into RAT-STATS
  • Output can be in a variety of formats
  • Excel
  • Microsoft Access
  • Text File
  • Print File
  • Printer

117
Thats It!
  • Now well see an actual demo using the RAT-STATS
    software
  • Excel population of 5,000 invoices
  • Results of test of attributes stored in the
    worksheet

118
Variable Sampling Using RAT-STATS
  • Stratified populations

119
Session Objectives
  • Understand what stratified sampling is and when
    to use it
  • Populations benefiting from stratified sampling
  • Overview of the process using RAT-STATS
  • Understand the formula behind the computations

120
Stratified variable sampling
  • Stratified
  • Variable
  • Estimating amounts
  • Narrower standard deviation

121
Overview of process
  • Determine the sampling objective
  • Confidence
  • Precision
  • Determine required sample size
  • Identify samples to be selected based upon random
    numbers
  • Pull the sample and test
  • Compute the sampling results (i.e. estimate of
    range)

122
How this is done in RAT-STATS
  • The sampling parameters are first developed by
    the auditor
  • RAT-STATS is used to compute sample size
  • RAT-STATS used to generate random numbers
  • Pull the sample and test
  • Enter results in RAT-STATS to compute estimates

123
Step 1 Develop sampling parameters
  • Probe sample
  • Statistical measure
  • Excel formula

124
Step 1 Develop sampling parameters
  • Number of strata
  • Size of population
  • Average value
  • Standard deviation

125
Step 2 Obtain the random numbers
  • Done by entering info into RAT-STATS
  • Multi-stage random numbers
  • Output can be a variety of sources
  • Text File
  • Excel
  • Microsoft Access
  • Print File

126
Step 3 Pull the sample
  • Each random number selected corresponds with an
    item in a strata
  • Put the selected item on a separate schedule

127
Step 4 - Test each selected item
  • Generally requires reviewing documents
  • Example data contains both examined and
    audited value.

128
Step 5 Compute the results
  • Enter summary information into RAT-STATS
  • Output can be in a variety of formats
  • Excel
  • Microsoft Access
  • Text File
  • Print File
  • Printer

129
Thats It!
  • Now well see an actual demo using the RAT-STATS
    software
  • Excel population of 5,000 invoices
  • Divided into three strata
  • Audited values stored in the worksheet

130
Objective 2 - Summarized
  • Understand the framework for managing the
    business risk of fraud
  • Plan, perform and explain statistical sampling in
    audits
  • Reduce audit costs using data mining, sequential
    sampling and other sampling techniques
  • Apply SAS 56, the new SAS suite and the revised
    (2007) Yellow Book.
  • Run, hands-on, the most productive analytic
    technique (regression analysis).
  • Use data mining to introduce greater efficiency
    into the audit process, without losing
    effectiveness.

Next is cost reduction
131
Techniques for cost reduction
  • Optimize sample size (most bang for the buck)
  • Skip sampling review 100 of transactions using
    computer assisted audit techniques (CAATs)

132
Sample optimization
  • Sequential sampling

133
University of Hawaii
  • Banana aphids

134
Sequential sampling
  • Banana aphids

135
100 test using CAATs
  • Provides complete coverage
  • Best practice
  • Basis for continuous monitoring
  • Repeatable process

136
Objective 3 - Summarized
  • Understand the framework for managing the
    business risk of fraud
  • Plan, perform and explain statistical sampling in
    audits
  • Reduce audit costs using data mining, sequential
    sampling and other sampling techniques
  • Apply SAS 56, the new SAS suite and the revised
    (2007) Yellow Book.
  • Run, hands-on, the most productive analytic
    technique (regression analysis).
  • Use data mining to introduce greater efficiency
    into the audit process, without losing
    effectiveness.

Next is Yellow Book and SAS 56
137
Yellow book standards
Standards regarding statistical sampling and IT
138
General standards
  • 3.43 Technical Knowledge and competence
  • The staff assigned to conduct an audit or
    attestation engagement under GAGAS must
    collectively possess the technical knowledge,
    skills, and experience necessary to be competent
    for the type of work being performed before
    beginning work on that assignment.
  • The staff assigned to a GAGAS audit or
    attestation
  • engagement should collectively possess

139
Stat sampling and IT
  • Skills appropriate for the work being performed.
    For example, staff or specialist skills in
  • (1) statistical sampling if the work involves use
    of statistical sampling
  • (2) information technology

140
SAS 56 Analytical procedures
  • Requires use of analytic review procedures for
  • Audit planning
  • Overall review stages

141
SAS 56 Analytical Review
  • Encourages use of analytical review
  • Provides guidance

A wide variety of analytical procedures may be
useful for this purpose.
142
Objective 4 - Summarized
  • Understand the framework for managing the
    business risk of fraud
  • Plan, perform and explain statistical sampling in
    audits
  • Reduce audit costs using data mining, sequential
    sampling and other sampling techniques
  • Apply SAS 56, the new SAS suite and the revised
    (2007) Yellow Book.
  • Run, hands-on, the most productive analytic
    technique (regression analysis).
  • Use data mining to introduce greater efficiency
    into the audit process, without losing
    effectiveness.

Next is linear regression
143
Next Metric
  • Outliers
  • Stratification
  • Day of Week
  • Round Numbers
  • Made Up Numbers
  • Market basket
  • Trends
  • Gaps
  • Duplicates
  • Dates

144
Trend Busters
7 - Trends
Does the pattern make sense?
145
Trend Busters
7 Trends
  • Linear regression
  • Sales are up, but cost of goods sold is down
  • Spikes

146
Purpose / Type of Errors
7 Trends
  • Identify trend lines, slopes, etc.
  • Correlate trends
  • Identify anomalies
  • Key punch errors where amount is order of
    magnitude

147
Linear Regression
7 Trends
  • Test relationships (e.g. invoice amount and sales
    tax)
  • Perform multi-variable analysis

148
How is it done?
7 Trends
  • Estimate linear trends using best fit
  • Measure variability (standard errors)
  • Measure slope
  • Sort descending by slope, variability, etc.

149
Trend Lines by Account - Example Results
7 Trends
Generally the trend is gently sloping up, but two
accounts (43870 and 54630) are different.
150
Scatter plot with prediction and confidence
intervals
151
Objective 5 - Summarized
  • Understand the framework for managing the
    business risk of fraud
  • Plan, perform and explain statistical sampling in
    audits
  • Reduce audit costs using data mining, sequential
    sampling and other sampling techniques
  • Apply SAS 56, the new SAS suite and the revised
    (2007) Yellow Book.
  • Run, hands-on, the most productive analytic
    technique (regression analysis).
  • Use data mining to introduce greater efficiency
    into the audit process, without losing
    effectiveness.

Next is data mining
152
Basis for Pattern Detection
Objective 6
  • Analytical review
  • Isolate the significant few
  • Detection of errors
  • Quantified approach

153
Understanding the Basis
Objective 2
  • Quantified Approach
  • Population vs. Groups
  • Measuring the Difference
  • Stat 101 Counts, Totals, Chi Square and K-S
  • The metrics used

154
Quantified Approach
Objective 2a
  • Based on measureable differences
  • Population vs. Group
  • Shotgun technique

155
Detection of Fraud Characteristics
Objective 2a
  • Something is different than expected

156
Fraud patterns
Objective 2b
  • Common theme something is different
  • Groups
  • Group pattern is different than overall population

157
Measurement Basis
Objective 2c
  • Transaction counts
  • Transaction amounts

158
A few words about statistics (the s word)
Objective 2d
  • Detailed knowledge of statistics not necessary
  • Software packages do the number-crunching
  • Statistics used only to highlight potential
    errors/frauds
  • Not used for quantification

159
How is digital analysis done?
Objective 2d
  • Comparison of group with population as a whole
  • Can be based on either counts or amounts
  • Difference is measured
  • Groups can then be ranked using a selected
    measure
  • High difference possible error/fraud

160
Demo in Excel of the process
  • Based roughly on the Wake County Transportation
    fraud
  • Illustrates how the process works, using Excel

161
Histograms
Objective 2d
  • Attributes tallied and categorized into bins
  • Counts or sums of amounts

162
Two histograms obtained
Objective 2d
  • Population and group

163
Compute Cumulative Amount for each
Objective 2d
164
Are the histograms different?
Objective 2d
  • Two statistical measures of difference
  • Chi Squared (counts)
  • K-S (distribution)
  • Both yield a difference metric

165
Chi Squared
Objective 2d
  • Classic test on data in a table
  • Answers the question are the rows/columns
    different
  • Some limitations on when it can be applied

166
Chi Squared
Objective 2d
  • Table of Counts
  • Degrees of Freedom
  • Chi Squared Value
  • P-statistic
  • Computationally intensive

167
Kolmogorov-Smirnov
Objective 2d
  • Two Russian mathematicians
  • Comparison of distributions
  • Metric is the d-statistic

168
How is K-S test done?
Objective 2d
  • Four step process
  • For each cluster element determine percentage
  • Then calculate cumulative percentage
  • Compare the differences in cumulative percentages
  • Identify the largest difference

169
Kolmogorov-Smirnov
Objective 2d - KS
170
Classification by metrics
Objective 2e
  • Stratification
  • Day of week
  • Happens on holiday
  • Round numbers
  • Variability
  • Benfords Law
  • Trend lines
  • Relationships (market basket)
  • Gaps
  • Duplicates

171
Auditors Top 10 Metrics
Objective e
  • Outliers / Variability
  • Stratification
  • Day of Week
  • Round Numbers
  • Made Up Numbers
  • Market basket
  • Trends
  • Gaps
  • Duplicates
  • Dates

172
Understanding the Basis
Objective 2
  • Quantified Approach
  • Population vs. Groups
  • Measuring the Difference
  • Stat 101 Counts, Totals, Chi Square and K-S
  • The metrics used

173
Objective 2 - Summarized
  • Understand why and how
  • Understand statistical basis for quantifying
    differences
  • Identify ten general tools and techniques
  • Understand examples done using Excel
  • How pattern detection fits in

Next are the metrics
174
Its that time!
Session Break!
175
The Top 10 Metrics
Objective 3
  • Overview
  • Explain Each Metric
  • Examples of what it can detect
  • How to assess results

176
Trapping anomalies
Objective 3
177
Fraud Pattern Detection
Objective 3
178
Outliers / Variability
1 - Outliers
Outliers are amounts which are significantly
different from the rest of the population
179
Outliers / Variability
1 - Outliers
  • Charting (visual)
  • Software to analyze z-scores
  • Top and Bottom 10, 20 etc.
  • High and low variability (coefficient of
    variation)

180
Drill down to the group level
1 - Outliers
  • Basic statistics
  • Minimum, maximum and average
  • Variability
  • Sort by statistic of interest
  • Variability (coefficient of variation)
  • Maximum, etc.

181
Example Results
1 - Outliers
Two providers (3478421 and 2356721) had
significantly more variability in the amounts of
their claims than all the rest.
182
Next Metric
  • Outliers
  • Stratification
  • Day of Week
  • Round Numbers
  • Made Up Numbers
  • Market basket
  • Trends
  • Gaps
  • Duplicates
  • Dates

183
Unusual stratification patterns
2 - Stratification
Do you know how your data looks?
184
Stratification - How
2 - Stratification
  • Charting (visual)
  • Chi Squared
  • Kolmogorov-Smirnov
  • By groups

185
Purpose / types of errors
2 Stratification
  • Transactions out of the ordinary
  • Up-coding insurance claims
  • Skewed groupings
  • Based on either count or amount

186
The process?
2 Stratification
  • Stratify the entire population into bins
    specified by auditor
  • Same stratification on each group (e.g. vendor)
  • Compare the group tested to the population
  • Obtain measure of difference for each group
  • Sort descending on difference measure

187
Units of Service Stratified - Example Results
2 Stratification
Two providers (2735211 and 4562134) are shown to
be much different from the overall population (as
measured by Chi Square).
188
Next Metric
  • Outliers
  • Stratification
  • Day of Week
  • Round Numbers
  • Made Up Numbers
  • Market basket
  • Trends
  • Gaps
  • Duplicates
  • Dates

189
Day of Week
3 Day of Week
  • Activity on weekdays
  • Activity on weekends
  • Peak activity mid to late week

190
Purpose / Type of Errors
3 Day of Week
  • Identify unusually high/low activity on one or
    more days of week
  • Dentist who only handled Medicaid on Tuesday
  • Office is empty on Friday

191
How it is done?
  • Programmatically check entire population
  • Obtain counts and sums by day of week (1-7)
  • Prepare histogram
  • For each group do the same procedure
  • Compare the two histograms
  • Sort descending by metric (chi square/d-stat)

192
Day of Week - Example Results
3 Day of Week
Provider 2735211 only provided service for
Medicaid on Tuesdays. Provider 4562134 was
closed on Thursdays and Fridays.
193
Next Metric
  • Outliers
  • Stratification
  • Day of Week
  • Round Numbers
  • Made Up Numbers
  • Market basket
  • Trends
  • Gaps
  • Duplicates
  • Dates

194
Round Numbers
4 Round Numbers
Its about.
Estimates!
195
Purpose / Type of Errors
4 Round Numbers
  • Isolate estimates
  • Highlight account numbers in journal entries with
    round numbers
  • Split purchases (under the radar)
  • Which groups have the most estimates

196
Round numbers
4 Round Numbers
  • Classify population amounts
  • 1,375.23 is not round
  • 5,000 is a round number type 3 (3 zeros)
  • 10,200 is a round number type 2 (2 zeros)
  • Quantify expected vs. actual (d-statistic)
  • Generally represents an estimate
  • Journal entries

197
Round Numbers in Journal Entries - Example Results
4 Round Numbers
Two accounts, 2735211 and 4562134 have
significantly more round number postings than any
other posting account in the journal entries.
198
Next Metric
  • Outliers
  • Stratification
  • Day of Week
  • Round Numbers
  • Made Up Numbers
  • Market basket
  • Trends
  • Gaps
  • Duplicates
  • Dates

199
Made up Numbers
5 Made up numbers
Curb stoning
Imaginary numbers
Benfords Law
200
What can be detected
5 Made Up Numbers
  • Made up numbers e.g. falsified inventory
    counts, tax return schedules

201
Benfords Law using Excel
5 Made Up Numbers
  • Basic formula is log(1(1/N))
  • Workbook with formulae available at
    http//tinyurl.com/4vmcfs
  • Obtain leading digits using Left function, e.g.
    left(Cell,1)

202
Made up numbers
5 Made Up Numbers
  • Benfords Law
  • Check Chi Square and d-statistic
  • First 1,2,3 digits
  • Last 1,2 digits
  • Second digit
  • Sources for more info

203
How is it done?
5 Made Up Numbers
  • Decide type of test (first 1-3 digits, last 1-2
    digit etc)
  • For each group, count number of observations for
    each digit pattern
  • Prepare histogram
  • Based on total count, compute expected values
  • For the group, compute Chi Square and d-stat
  • Sort descending by metric (chi square/d-stat)

204
Invoice Amounts tested with Benfords law -
Example Results
5 Made Up Numbers
During tests of invoices by store, two stores,
324 and 563 have significantly more differences
than any other store as measured by Benfords
Law.
205
Next Metric
  • Outliers
  • Stratification
  • Day of Week
  • Round Numbers
  • Made Up Numbers
  • Market basket
  • Trends
  • Gaps
  • Duplicates
  • Dates

206
Market Basket
6 Market Basket
  • Medical Ping ponging
  • Pattern associations
  • Apriori program
  • References at end of slides
  • Apriori Latin a (from) priori (former)
  • Deduction from the known

207
Purpose / Type of Errors
6 Market basket
  • Unexpected patterns and associations
  • Based on market basket concept
  • Unusual combinations of diagnosis code on medical
    insurance claim

208
Market Basket
6 Market basket
  • JE Accounts
  • JE Approvals
  • Credit card fraud in Japan taxi and ATM

209
How is it done?
6 Market basket
  • First, identify groups, e.g. all medical
    providers for a patient
  • Next, for each provider, assign a unique integer
    value
  • Create a text file containing the values
  • Run apriori analysis

210
Apriori outputs
6 Market basket
  • For each unique value, probability of other
    values
  • If you see Dr. Jones, you will also see Dr. Smith
    (80 probability)
  • If you see a JE to account ABC, there will also
    an entry to account XYZ (30)

211
Next Metric
  • Outliers
  • Stratification
  • Day of Week
  • Round Numbers
  • Made Up Numbers
  • Market basket
  • Trends
  • Gaps
  • Duplicates
  • Dates

212
Numeric Sequence Gaps
8 - Gaps
Whats there is interesting, whats not there is
critical
213
Purpose / Type of Errors
8 Gaps
  • Missing documents (sales, cash, etc.)
  • Inventory losses (missing receiving reports)
  • Items that walked off

214
How is it done?
8 Gaps
  • Check any sequence of numbers supposed to be
    complete, e.g.
  • Cash receipts
  • Sales slips
  • Purchase orders

215
Gaps Using Excel
8 Gaps
  • Excel sort and check
  • Excel formula
  • Sequential numbers and dates

216
Gap Testing - Example Results
8 Gaps
Four check numbers are missing.
217
Next Metric
  • Outliers
  • Stratification
  • Day of Week
  • Round Numbers
  • Made Up Numbers
  • Market basket
  • Trends
  • Gaps
  • Duplicates
  • Dates

218
Duplicates
9 - Duplicates
Why is there more than one?
Same, Same, Same, and
Same, Same, Different
219
Two types of (related) tests
9 Duplicates
  • Same items same vendor, same invoice number,
    same invoice date, same amount
  • Different items same employee name, same city,
    different social security number

220
Duplicate Payments
9 - Duplicates
  • High payback area
  • Fuzzy logic
  • Overriding software controls

221
Fuzzy matching with software
9 - Duplicates
  • Levenshtein distance
  • Soundex
  • Like clause in SQL
  • Regular expression testing in SQL
  • Vendor/employee situations

Russian physicist
222
How is it done?
9 - Duplicates
  • First, sort file in sequence for testing
  • Compare items in consecutive rows
  • Extract exceptions for follow-up

223
Possible Duplicates - Example Results
9 - Duplicates
Five invoices may be duplicates.
224
Next Metric
  • Outliers
  • Stratification
  • Day of Week
  • Round Numbers
  • Made Up Numbers
  • Market basket
  • Trends
  • Gaps
  • Duplicates
  • Dates

225
Date Checking
10 - Dates
If were closed, why is there
Adjusting journal entry?
Receiving report? Payment issued?
226
Holiday Date Testing
10 Dates
  • Red Flag indicator

227
Date Testing challenges
10 Dates
  • Difficult to determine
  • Floating holidays Friday, Saturday, Sunday,
    Monday

228
Typical audit areas
10 Dates
  • Journal entries
  • Employee expense reports
  • Business telephone calls
  • Invoices
  • Receiving reports
  • Purchase orders

229
Determination of Dates
10 Dates
  • Transactions when business is closed
  • Federal Office of Budget Management
  • An excellent fraud indicator in some cases

230
Holiday Date Testing
10 Dates
  • Identifying holiday dates
  • Error prone
  • Tedious
  • U.S. only

231
Federal Holidays
10 Dates
  • Established by Law
  • Ten dates
  • Specific date (unless weekend), OR
  • Floating holiday

232
Federal Holiday Schedule
10 Dates
  • Office of Personnel Management
  • Example of specific date Independence Day, July
    4th (unless weekend)
  • Example of floating date Martin Luther Kings
    birthday (3rd Monday in January)
  • Floating Thanksgiving 4th Thursday in November

233
How it is done?
10 Dates
  • Programmatically count holidays for entire
    population
  • For each group, count holidays
  • Compare the two histograms (group and population)
  • Sort descending by metric (chi square/d-stat)

234
Holiday Counts - Example Results
10 Dates
Two employees (10245 and 32325) were off the
chart in terms of expense amounts incurred on a
Federal Holiday.
235
The Top 10 Metrics
Objective 3
  • Overview
  • Explain Each Metric
  • Examples of what it can detect
  • How to assess results

236
Objective 3 - Summarized
  • Understand why and how
  • Understand statistical basis for quantifying
    differences
  • Identify ten general tools and techniques
  • Understand examples done using Excel
  • How pattern detection fits in

Next using Excel
237
Use of Excel
Objective 4
  • Built-in functions
  • Add-ins
  • Macros
  • Database access

238
Excel templates
Objective 4
  • Variety of tests
  • Round numbers
  • Benfords Law
  • Outliers
  • Etc.

239
Excel Univariate statistics
Objective 4
  • Work with Ranges
  • sum, average, stdevp
  • largest(Range,1), smallest(Range,1)
  • min, max, count
  • Tools Data Analysis Descriptive Statistics

240
Excel Histograms
Objective 4
  • Tools Data Analysis Histogram
  • Bin Range
  • Data Range

241
Excel Gaps testing
Objective 4
  • Sort by sequential value
  • if(thiscell-lastcell ltgt 1,thiscell-lastcell,0)
  • Copy/paste special
  • Sort

242
Detecting duplicates with Excel
Objective 4
  • Sort by sort values
  • if testing
  • if(and(thiscelllastcell, etc.))

243
Performing audit tests with macros
Objective 4
  • Repeatable process
  • Audit standardization
  • Learning curve
  • Streamlining of tests
  • More efficient and effective
  • Examples - http//ezrstats.com/Macros/home.html

244
Using database audit software
Objective 4
  • Many built-in functions right off the shelf
    with SQL
  • Control totals
  • Exception identification
  • Drill down
  • Quantification
  • June 2008 article in the EDP Audit Control
    Journal (EDPACS) SQL as an audit tool
  • http//ezrstats.com/doc/SQL_As_An_Audit_Tool.pdf

245
Use of Excel
Objective 4
  • Built-in functions
  • Add-ins
  • Macros
  • Database access

246
Objective 4 - Summarized
  • Understand why and how
  • Understand statistical basis for quantifying
    differences
  • Identify ten general tools and techniques
  • Understand examples done using Excel
  • How Pattern Detection fits in

Next Fit
247
How Pattern Detection Fits In
Objective 5
  • Business Analytics
  • Fraud Pattern Detection
  • Continuous monitoring

248
Where does Fraud Pattern Detection fit in?
Objective 5
Right in the middle
  • Business Analytics
  • Fraud Pattern Detection
  • Continuous fraud pattern detection
  • Continuous Monitoring

249
Business Analytics
Objective 5
  • Fraud analytics -gt business analytics
  • Business analytics -gt fraud analytics

250
Role in Continuous Monitoring (CM)
Objective 5
  • Fraud analytics can feed (CM)
  • Continuous fraud pattern detection
  • Use output from CM to tune fraud pattern detection

251
Objective 6 - Summarized
  • Understand the framework for managing the
    business risk of fraud
  • Plan, perform and explain statistical sampling in
    audits
  • Reduce audit costs using data mining, sequential
    sampling and other sampling techniques
  • Apply SAS 56, the new SAS suite and the revised
    (2007) Yellow Book.
  • Run, hands-on, the most productive analytic
    technique (regression analysis).
  • Use data mining to introduce greater efficiency
    into the audit process, without losing
    effectiveness.

252
Links for more information
  • Kolmogorov-Smirnov
  • http//tinyurl.com/y49sec
  • Benfords Law http//tinyurl.com/3qapzu
  • Chi Square tests http//tinyurl.com/43nkdh
  • Continuous monitoring http//tinyurl.com/3pltdl

253
Market Basket
  • Apriori testing for ping ponging
  • Temple University http//tinyurl.com/5vax7r
  • Apriori program (open source)
    http//tinyurl.com/5qehd5
  • Article Medical ping ponging
    http//tinyurl.com/5pzbh4

254
Excel macros used in auditing
  • Excel as an audit software http//tinyurl.com/6h3y
    e7
  • Selected macros - http//ezrstats.com/Macros/home.
    html
  • Spreadsheets forever - http//tinyurl.com/5ppl7t

255
Questions?
256
Contact info
  • Phone (919)-219-1622
  • E-mail Mike.Blakley_at_ezrstats.com
  • Blog http//blog.ezrstats.com
About PowerShow.com