Is Statistical Machine Learning Safe in an Adversarial Environment

1
Is Statistical Machine Learning Safe in an
Adversarial Environment

• Blaine Nelson, RAD Lab
• June 2008

1
2
Motivation for SecML

• Many security-sensitive applications use adaptive
  learning techniques
• Using learning techniques in these systems
  introduces new security vulnerabilities
• Learning techniques can be misled by malicious
  data
• How much of a threat is this new adversary?
• How hard is an attack for the adversary?
• Are there defenses against these threats?

3
RAD Lab Overview
Low level spec
Com- piler
High level spec
Instrumentation Backplane
New apps, equipment, global policies (eg SLA)
Offered load, resource utilization, etc.
Director
Policy-awareswitching
Training data
performance cost models
Log Mining
3
4
RAD Lab Security Concerns

• Misleading Performance Modeling
• How does malicious data affect models?
• Can it cause misclassifications?
• What is the effect of tainted data on the models
  (PCA)?
• Causing Poor Performance
• Can the adversary cause poor decisions?
• Can adversarial data cause misallocation?

5
Outline

• Project 1 Attacking Network Monitors
• Traffic shaping
• Multi-Week traffic shaping
• Project 2 Attacking Spam Filters
• Causing DoS by spamming
• Blocking targeted messages by spamming
• Conclusions

6
Network-Wide Traffic Anomaly Detection

• Ling Huang Anthony D. Joseph
• Shing-hon Lau Blaine Nelson
• Benjamin Rubinstein Nina Taft J.D. Tygar

7
Traffic Anomaly Detection

• Detecting Volume Anomalies Lakhina et al. 2004
• OD flow from origin (O) to destination (D)
• Link volume Yt , i?j traffic i?j at time time t

• PCA Anomaly Detection
• Find low-dim subspace that captures most of link
  traffic
• Detect Anomalous OD flows by large residuals

Yt , a?b
Yt , b?c
Yt , a?c
Yt , c?d
Yt , d?f
Yt , d?e
Yt , c?b
Yt , e?f
7
8
Realistic Threat Model

• Attacker threat model
• Goal source-to-sink DoS attack is undetected
• Control compromised router sends traffic
• Send high-variance chaff to sink PoP
• Risk compromised node could be discovered

8
9
Chaff Methods

• Attacker
• Target attack flow f
• Poison f in training
• Launch DoS on fin test week
• Attack metrics
• FN rate
• Average increase to the links in f

10
Multi-Week Attacks

• Increase week ts traffic by rt, for growth rate
  r
• PCA can reject suspect samples before training

11
Attacking the SpamBayes Filter

• Marco Barreno Fuching Jack Chi Anthony D.
  Joseph Shing-hon Lau Blaine Nelson Benjamin
  Rubinstein
• Udam Saini Charles Sutton Anthony Tran
• J.D. Tygar Kai Xia

12
Attacking a Spam Filter

• Goals
• Novel attacks against statistical learners
• SpamBayes spam filter
• Denial-of-service attacks on filters
• Focused/Dictionary attacks
• Potential defenses
• Shape training set filter according to
  performance.

13
Poisoning the Training Set
Attacker
Attack Corpus
Contamination
Attackers Information
Spam
Ham
Email Distribution
Filter
INBOX
Spam Folder
14
SpamBayes

• SpamBayes statistical spam filter
• Unigram word frequencies
• Token scores are independent spam test
• Build message score from token scores
• Threshold ham, unsure, or spam

token score
15
Outline of our Attacks

• Training on attack msg. changes scores
• Design attacks to increase scores of ham
• Message score increases w/ token scores

16
Dictionary Attack

• Make spam filter unusable
• misclassify ham as spam

Spammer
17
Dictionary Attack

• Initial Inbox 10K messages
• Attacks
• Black Optimal
• Red English dictionary
• Blue 90K most common words in Usenet

18
Focused Attack

• Misclassify specific target message

Rolex Breitling Cartier Porsche Dior Gucci Cheap
quality watches now!!! absent aware dear from I
make sincerely sir school Skinner son that to
today wanted was you your
Dear Sir, I wanted to make you aware that
your son was absent from school
today. Sincerely, S. Skinner
Dear Sir, I wanted to make you aware that
your son was absent from school
today. Sincerely, S. Skinner
19
Focused Attack

• Initial Inbox 10K messages (50 spam)
• 200 targeted attacks
• 50 guessing rate

• Initial Inbox 10K messages (50 spam)
• 200 targeted attacks
• 200 msgs. per attack.

20
DefensesReject on Negative Impact (RONI)

• Method
• Assess impact of query message on training
• Exclude messages with large negative impact
• Preliminary Results
• Perfectly identifies dictionary attacks
• Unable to differentiate focused attacks

SpamBayes Filter
SpamBayes Learner
?
21
Conclusion Future Work

• Novel attacks in different domains
• Successfully caused general DoS attack
• Successfully targeted specific ham message
• Successfully mislead anomaly detectors
• Defenses
• Promising initial ideas
• Ongoing studies on the success of defenses

22
Questions?
23
(No Transcript)
24
Extra Slides
25
Is Statistical Machine Learning Safe in an
Adversarial Environment

• Blaine Nelson Marco Barreno Fuching Jack
  Chi
• Ling Huang Anthony D. Joseph Shing-hon Lau
• Benjamin Rubinstein Udam Saini Charles Sutton
  
• Nina Taft Anthony Tran J.D. Tygar Kai Xia

26
Attack Taxonomy
attacks are all causative unless stated
otherwise.
27
Traffic Anomaly Detection

• Anomography
• Detect anomalous origin-destination (OD) flows
• Use only link traffic
• Principal Components Analysis (PCA)Lakhina et
  al. 2004
• Tier-1 backbone network
• Link space link volumes at time t is a point
• 4 components capture most variance
• Predict anomaly if residual too large

27/32
28
Sensitivity of PCA to Outliers
28
29
PCA Detection Threat Model

• Classify time t link volumes in normal, anomaly
  
• Threat model for attacker
• Goal source-to-sink DoS attack evades detector
• Control compromised router sends traffic
• Information real-time local traffic monitoring
  OR none
• Integrity Attacks (FNs)
• Send high-variance chaff to sink PoP
• In/dependent chaff

30
Network-Wide Anomalies Further Work

• Availability attacks
• Adversary wants FPs to shutdown PCA
• Swing subspace away from normal data
• Local vs. global information
• Robust PCA counter-measures
• Experiments with Marrona 2005
• Temporal vs. Spatial PCA
• Design robust methods for noise model

30/32
31
Attackers Knowledge

• No prior knowledge
• Knowledge of some words (partial)
• Knowledge of exact words in email

Subj Bid Our contract bid is 10,000 ACME
Shipping Las Vegas, Nevada