Reconsidering the Risk-Based Formulas and Standards Approach

1
Reconsidering the Risk-Based Formulas and
Standards Approach To Improving Security

• Mike Ahmadi VP Operations, GraniteKey LLC
• Gib Sorebo Chief Cybersecurity Technologist,
  SAIC
• Dr. Fred Cohen CEO, Fred Cohen Associates

2
Why Do We Like Mathematical Formulas ?

• Objective Pertains to a known quantity
• Repeatable Formulas used in the same manner
  yield the same results
• Incontrovertible Numbers are considered the
  final arbiter of discussions Numbers Dont Lie

3
When Are Mathematical Formulas Most Useful ?

• Numbers You need numerical information to
  perform mathematical functions.
• Verifiable Numerical results should yield
  information that is supportive of subjective
  observations and/or beliefs.
• Empirical Provable by experience

4
Good Example - Banking

• Numbers Banks have lots of numbers to work
  with.
• Verifiable If a bank assumes more customers and
  more assets leads to more potential losses (even
  before doing the math), this is supported by the
  math.
• Empirical Banks have a lot of experience
  backing the mathematical formulas they rely on.

5
Typical Quantitative Risk Formula

• AV Asset Value
• EF Exposure Factor (Keep it simple and assume
  100)
• SLE Single Loss Expectancy (AV x EF)
• ARO Annualized Rate Of Occurrence
• ALE Annualized Loss Expectancy(AV x EF) x ARO
  ALEExample (1,000,000 x 1) x 2
  2,000,000A company should not spend 5,000,000
  annually to protect 2,000,000.

6
Due Diligence and Due Care

• Due Diligence Calculate the formula(1,000,000
  x 1) x 2 2,000,000We have determined a
  potential loss of 2 million annually from an
  exposure.
• Due Care Do something about itWe are
  investing 1 million per year to address security
  concerns.

Is That Sufficient ?
7
What Happens When The Threat Is Theoretical?

• Due Diligence Calculate the formula(1,000,000
  x 1) x 0 0When the attack has not happened,
  the Annualize Rate of Occurrence (ARO) is zero.
• Due Care Do something about itWe will
  continue to monitor for suspicious activity.

Is That Sufficient ?
8
Black Swan Events

• Nassim Nicholas Taleb developed the Black Swan
  Theory
• The disproportionate role of high-impact, hard to
  predict, and rare events that are beyond the
  realm of normal expectations in history, science,
  finance and technology
• The non-computability of the probability of the
  consequential rare events using scientific
  methods (owing to the very nature of small
  probabilities)
• The psychological biases that make people
  individually and collectively blind to
  uncertainty and unaware of the massive role of
  the rare event in historical affairs

9
How Do We Identify Such Events?

• Based on Talebs Criteria
• The event is a surprise (to the observer).
• The event has a major impact.
• After its first recording, the event is
  rationalized by hindsight, as if it could have
  been expected (e.g., the relevant data were
  available but not accounted for).
• Dr. Fred Cohen There Are No Black Swans
• No surprises
• Bad decisions (sometimes based on risk formulas)
• http//all.net/Talks/2009-07-30-Catalyst.pdf

10
What's a Real Black Swan Example?

• Stuxnet? No way... There is nothing new here
• A computer virus (1984)
• Spread by USB (see floppy disks circa 1987 Brain)
• Exploiting known vulnerability classes (1970s in
  combo since the 1980s)
• With deception at interface (late 1990s)
• By sophisticated (insider?) attackers (since
  1940s today)
• Targeted to a particular mechanism (1980s)
• To do physical harm to ICSs (1980s)
• Black Swans as abused today
• Risk managers excuse risk acceptance or fail to
  do a thorough job of risk identification
• Following existing standards, Stuxnet fails

11
The Crux of the Problem

• Who could have ever known? You!!!
• The IEEE Code of Ethics
• 6. to maintain and improve our technical
  competence and to undertake technological tasks
  for others only if qualified by training or
  experience, or after full disclosure of pertinent
  limitations
• If you are going to do this work you better spend
  your time studying it.

• http//datalossdb.org/
• http//all.net/ -gt Database (click go)
• COSO (or what it's supposed to be)
• Hire a real consultant to do a thorough review
• Look things up on the Internet
• Read the local paper
• Watch the news
• Think!!!

12
If You Must Use Numbers

• When you lack empirical data, try viewing the
  system in comparison to other systems.
• Systems that go from low tech to high tech
  generally go through a similar security cycle
  based on some criteria (this is not an exhaustive
  list)
• How juicy is the target?
• Financial Gain
• Notoriety
• Impact
• Fun Factor
• How aware are those in charge?
• Security brain trust
• Plug in numbers from systems that have taken the
  hit, such as banking.
• Dont lie with numbers.

13
Consider The True Goals

• Are you trying to secure your system, or are you
  trying to avoid a PR nightmare (or a regulatory
  fine)?
• Does the ultimate decision maker potentially
  represent a risk not shown in your assessment?
• Is the formula used to prove how intelligent
  the decisions are?

What Is Really Going On?
14
The Regulation Standards Game

• Current surveys show that the majority of
  security spending is driven by compliance
  concerns, not reducing risk
• This is even more true in heavily regulated areas
  such as government, financial services, health
  care, and electricity delivery
• Evidence also suggest that requiring
  organizations to adhere to security regulations
  and standards does improve security, but only up
  to a point and only for the poor performers

15
Why Do We Have Standards Regulations?

• Externalities
• Third parties (and even second parties) harmed
  often have difficulty successfully suing for
  damages
• Mandatory requirements can help to prevent that
  harm that operators do not have sufficient
  financial incentives to protect on their own
• Level Playing Field
• Removes competitive pressures from the decision
  of whether to implement security control
• Advancement of the Industry
• Industry groups and government regulators may
  want to protect their reputation and not be
  viewed as irresponsible when a harm occurs (e.g.,
  nuclear meltdowns in Japan)
• Payment Card Industry (PCI) was specifically
  designed to prevent costs caused by one entitys
  negligence from affecting others in the network

16
Why Dont Regulations Standards Work?

• Focus on the Minimum
• Normally requirements are high level and entities
  can provide plausible explanations for why their
  controls are sufficient when theyre not
• They Dont Evolve
• Standards usually require industry consensus and
  regulations require comments and political
  negotiation that all take time
• Updates occur slowly, while threats can evolve in
  minutes
• Not Performance Based
• Many organizations are more concerned that they
  can show they met the requirements than that they
  prevented a breach
• Tremendous pressure to provide safe harbors to
  protect against lawsuits and reputational damage
• Inadequate Enforcement
• Auditors often not sufficiently technical to
  focus on technical implementations
• Audits become check the box exercises
• Funding for audits is limited in many industries
  (often driven by complaints and security
  incidents)

17
Past Attempts to Solve

• Leave It to the Trial Lawyers
• Rely on threat of lawsuits to implement
  appropriate security
• So far, this has not worked damages and
  causation are difficult to prove
• Organizations may conclude that insurance and
  self-insurance are cheaper as breaches may be
  viewed as inevitable
• Require More Specific Requirements
• Would provide guidance similar to federal
  government where specific configuration settings
  are mandated for certain platforms
• Process is expensive to maintain and can
  potentially stifle innovation
• Could also lead to a security mono-culture giving
  hackers a roadmap
• Step Up Enforcement
• Hire more technical auditors and deploy them more
  frequently
• Unrealistic from a funding perspective
• Still would lead to check the box approach as
  that is inevitable with most audits

18
An Alternative

• A Hybrid Approach
• Use audits and enforcement to maintain a floor
  but use tiered approach that rewards
  organizations that have more mature security
  programs with fewer audits and lower fees
• Socialize security best practices discovered
  during audits to foster positive competitive for
  better security approaches
• Provide immunity from fines and public disclosure
  for prompt reporting of breaches (obligations to
  report to data subjects may still exist) and the
  implementation of a robust remediation plan
• Require greater automation of compliance and
  continuous monitoring using industry-agreed
  security metrics to reduce compliance reporting
  costs and improve security
• Improve mechanisms for confidentially sharing
  incident information and known attacks (may
  require third party aggregator)

19
Questions?
Thank You. Gib Sorebo SAIC Assistant Vice
President / Chief Cybersecurity Technologist tel
703-676-2605 email sorebog_at_saic.com Mike
Ahmadi Vice President of Operations GraniteKey
LLC tel [phone] email
mike.ahmadi_at_granitekey.com Fred Cohen CEO Fred
Cohen Associates tel [phone] email
fc_at_fredcohen.net