A Fault Model and Mutation Testing of Access Control Policies

1
A Fault Model and Mutation Testingof Access
Control Policies

• Evan Martin and Tao Xie
• Department of Computer Science
• North Carolina State University

2
Outline

• Motivation
• XACML
• Policy Testing Techniques
• Coverage Criteria
• Request Generation
• Request Selection
• Fault Model Mutation Testing
• Experimental Results

3
Motivation

• Digital information is
• Easy to access
• Easy to search
• Sensitive information requires access control
  mechanisms
• A growing trend is to specify access control
  policies in a specification language such as XACML

4
Problem

• How to ensure the correct specification of access
  control policies?
• What you specify is what you get, but not
  necessarily what you want
• Systematic testing of access control policies
• Complements policy verification, which requires
  properties and may not support full policy
  features
• Just like software testing software
  verification

5
Software Testing
Policy Testing
Software Testing
Expected Responses
Responses
Requests
Policy
6
XACML Policy Structure

• eXtensible Access Control Markup Language
• OASIS standard XML syntax for specifying
  policies, requests, and responses
• A flexible and expressive language but complex
  and verbose
• Key concepts
• A Policy Set holds other policies or policy sets.
• A Policy is expressed as a set of rules.
• A Rule have targets and a set of conditions that
  determine if the rule applies to a given request.
  
• Both rule and policy Combining Algorithms exist
  to reconcile conflicts.

7
XACML Example
lt?xml version"1.0" encoding"UTF-8"?gt ltPolicySet
xmlns"urnoasisnamestcxacml1.0policy"
PolicySetId"college" PolicyCombiningAlgId"urn
oasisnamestcxacml1.0policy-combining-algorith
mpermit-overrides"gt ltDescriptiongtA College
Policy on Gradeslt/Descriptiongt ltTargetgt
ltSubjectsgt ltAnySubject /gt lt/Subjectsgt
ltResourcesgt ltAnyResource /gt lt/Resourcesgt
ltActionsgt ltAnyAction /gt lt/Actionsgt lt/Targetgt
ltPolicy PolicyId"fac" RuleCombiningAlgId"urn
oasisnamestcxacml1.0rule-combining-algorithm
permit-overrides"gt ltDescriptiongtFaculty
Policylt/Descriptiongt ltTargetgt
ltSubjectsgt ltSubjectgt
ltSubjectMatch MatchId"urnoasisnames
tcxacml1.0functionstring-equal"gt
ltAttributeValue DataType"http//www
.w3.org/2001/XMLSchemastring"gt
Faculty lt/AttributeValuegt
ltSubjectAttributeDesignator AttributeId"role"
DataType"http//www.w3.org/2001/XMLSch
emastring" /gt lt/SubjectMatchgt
lt/Subjectgt lt/Subjectsgt ltResourcesgt
ltAnyResource /gt lt/Resourcesgt
ltActionsgt ltAnyAction /gt lt/Actionsgt
lt/Targetgt
Policy Set
Target
Policy
Target
8
XACML Example
ltRule RuleId"fac-assign-view-grades"
Effect"Permit"gt ltTargetgt
ltSubjectsgt ltAnySubject /gt lt/Subjectsgt
ltResourcesgt ltResourcegt
ltResourceMatch MatchId"urnoasisnamestcxacml1
.0functionstring-equal"gt
ltAttributeValue DataType"http//www.w3.org/2001/X
MLSchemastring"gtExternalGradeslt/AttributeValuegt
ltResourceAttributeDesignator
AttributeId"resource-class"
DataType"http//www.w3.org/2001/XMLSchemastring"
/gt lt/ResourceMatchgt
lt/Resourcegt ltResourcegt
ltResourceMatch MatchId"urnoasisnamestcxacml1
.0functionstring-equal"gt
ltAttributeValue DataType"http//www.w3.org/2001/X
MLSchemastring"gtInternalGradeslt/AttributeValuegt
ltResourceAttributeDesignator
AttributeId"resource-class"
DataType"http//www.w3.org/2001/XMLSchemastring"
/gt lt/ResourceMatchgt
lt/Resourcegt lt/Resourcesgt
ltActionsgt ltActiongt
ltActionMatch MatchId"urnoasisnamestcxacml1.0
functionstring-equal"gt
ltAttributeValue DataType"http//www.w3.org/2001/X
MLSchemastring"gtAssignlt/AttributeValuegt
ltActionAttributeDesignator
AttributeId"command"
DataType"http//www.w3.org/2001/XMLSchemastring"
/gt lt/ActionMatchgt
lt/Actiongt ltActiongt
ltActionMatch MatchId"urnoasisnamestcxacml1.0
functionstring-equal"gt
ltAttributeValue DataType"http//www.w3.org/2001/X
MLSchemastring"gtReceivelt/AttributeValuegt
ltActionAttributeDesignator
AttributeId"command"
DataType"http//www.w3.org/2001/XMLSchemastring"
/gt lt/ActionMatchgt
lt/Actiongt lt/Actionsgt lt/Targetgt
lt/Rulegt lt/Policygt
Rule
Target
9
XACML Example
ltPolicy PolicyId"stu" RuleCombiningAlgId"urnoas
isnamestcxacml1.0rule-combining-algorithmper
mit-overrides"gt ltDescriptiongtStudent
Policylt/Descriptiongt ltTargetgt
ltSubjectsgt ltSubjectgt
ltSubjectMatch MatchId"urnoasisnamestcxacml1.
0functionstring-equal"gt
ltAttributeValue DataType"http//www.w3.org/2001/X
MLSchemastring"gtStudentlt/AttributeValuegt
ltSubjectAttributeDesignator
AttributeId"role" DataType"http//www.w3.org/200
1/XMLSchemastring" /gt lt/SubjectMatchgt
lt/Subjectgt lt/Subjectsgt
ltResourcesgtltAnyResource /gtlt/Resourcesgt
ltActionsgtltAnyAction /gtlt/Actionsgt lt/Targetgt
ltRule RuleId"stu-recieve-extgrades"
Effect"Permit"gt ltTargetgt
ltSubjectsgtltAnySubject /gtlt/Subjectsgt
ltResourcesgt ltResourcegt
ltResourceMatch MatchId"urnoasisnamestcxacml1
.0functionstring-equal"gt
ltAttributeValue DataType"http//www.w3.org/2001/X
MLSchemastring"gtExternalGradeslt/AttributeValuegt
ltResourceAttributeDesignator
AttributeId"resource-class"
DataType"http//www.w3.org/2001/XMLSchemastring"
/gt lt/ResourceMatchgt
lt/Resourcegt lt/Resourcesgt
ltActionsgt ltActiongt
ltActionMatch MatchId"urnoasisnamestcxacml1.0
functionstring-equal"gt
ltAttributeValue DataType"http//www.w3.org/2001/X
MLSchemastring"gtReceivelt/AttributeValuegt
ltActionAttributeDesignator
AttributeId"command"
DataType"http//www.w3.org/2001/XMLSchemastring"
/gt lt/ActionMatchgt
lt/Actiongt lt/Actionsgt lt/Targetgt
lt/Rulegt lt/Policygt lt/PolicySetgt
Rules can have a more complicated condition tag
here
10
Coverage Definition
ICICS 06

• Rationale like in program testing, when the
  policy part containing a fault is not covered,
  the error is often not exposed.
• Given a request q and a policy P, we say q covers
  a rule m in P if m contributes to the decision of
  q.
• The rule m contributes to the decision of q if
  all Cond are satisfied by q.
• The sequence of rules and combining algorithms
  may also affect coverage.
• Rule coverage of P by requests Q
• rules covered by at least one request in
  Q rules in P
• Similarly define policy coverage/condition
  coverage

11
Random Request Generation
ICICS 06

• The example policy
• Subjects Student, Faculty
• Actions Assign, Receive
• Objects External Grades, Internal Grades
• Model the set of attribute values as a vector of
  bits and randomize the bits

Student Faculty Assign Receive ExtGrades
IntGrades
12
Random Request Generation
ICICS 06

• The example policy
• Subjects Student, Faculty
• Actions Assign, Receive
• Objects External Grades, Internal Grades
• Model the set of attribute values as a vector of
  bits and randomize the bits

Student Faculty Assign Receive ExtGrades
IntGrades
13
Request Generation via Change-Impact Analysis
(Cirg)
SESS 07
Access Control Policy
Policy Versions
1. Version Synthesis
Requests
2. Change-impact analysis

• Synthesize versions (all-to-negate-one) for
  Margrave Kisler et al. 05

Counter examples
3. Request generation
14
Greedy Algorithm for Request Selection
ICICS 06
Requests
Policy
Reduced Request Set
Increased Coverage?
Yes
No
Discard
15
Fault Model and Mutation Testing

• Fault model used to model things that could go
  wrong when constructing a policy
• Fault model is the underlying foundation of
  mutation testing DeMillo et al. 78 (similar to
  fault injection)
• Policy is iteratively mutated to produce numerous
  mutants each containing one fault
• Usages
• Measure fault-detection effectiveness of test
  generation or test selection
• Select tests

16
Policy Mutation Testing
Responses
Requests
Policy
Mutation Operators
Mutator
Differ?
Mutant Killed!
Mutant Responses
Mutant Policy
17
Mutation Operators

• Each operator mutates a different policy element
  policy set, policy, rule, condition, and/or their
  associated targets and effects.

18
Change Rule Effect (CRE) Example

• IF (faculty AND assign AND grades)
• ELSE IF (student AND receive AND grades)
• Permit
• ELSE
• Deny
• The CRE mutation operator is performed on each
  rule and changes the decision effect (Permit ?
  Deny)

Permit
19
Change Rule Effect (CRE) Example

• IF (faculty AND assign AND grades)
• ELSE IF (student AND receive AND grades)
• Permit
• ELSE
• Deny
• The CRE mutation operator is performed on each
  rule and changes the decision effect (Permit ?
  Deny)

Deny
20
Experiment

• How strong is the correlation between the basic
  coverage criteria and fault-detection capability?
• Does test selection based on the coverage
  criteria produce reduced request sets with low
  loss of fault-detection capability?
• What are the individual characteristics of each
  mutation operator?

21
Metrics

• Policy, rule, and condition coverage
• Test count, t
• Reduced test count, tred
• Mutant-killing ratio, m
• Reduced mutant-killing ratio, mred

22
Policies used in the experiment
23
Basic Coverage Results

• Cirg performs at least as well as the random set
  for rule coverage except for the mod-fedora
  policy because of a policy error.
• Average of selected random tests is smaller
  than Cirg
• Random achieves 0 coverage on the conference
  policy

24
Mutant-killing Results

• Cirg outperforms the random technique in terms of
  fault-detection capability with a far fewer
  number of requests

25
Mutant-killing ratios by subjects
0 coverage gt 0 mutant-kill
Coverage criteria not bad for selection
not great either
26
Mutant-killing ratios by operators
Likely equivalent mutants
27
Conclusions

• Policy testing complements policy verification in
  assuring policy correctness.
• Just like software testing software
  verification
• We have developed coverage criteria, test
  selection, and test generation.
• This paper presents fault model and mutation
  testing for policies
• Used to measure fault-detection effectiveness of
  test generation or test selection
• Used to select tests

28
Questions?
29
Hypothesis

• We can achieve a significant reduction in
  request-set size for large randomly generated
  request sets while maintaining equivalent policy,
  rule, and condition coverage.
• Reducing a request set based on coverage will not
  proportionately decrease its fault detection
  capability.
• Request generation via Change-impact analysis
  (Cirg) will have a higher fault-detection
  capability.

30
XACML Coverage Criteria
ICICS 06

• Policy coverage A policy is covered by a
  request if the policy is applicable to the
  request and the policy is encountered before the
  PDP has fully resolved the decision for the
  request.
• Rule coverage A rule is covered by a request if
  the rule is applicable to the request and the
  rule is encountered before the PDP has fully
  resolved the decision for the request.
• Condition coverage A condition must be
  encountered and evaluate to true and false to be
  fully covered.