An Empirical User Study of a SmartphoneBased AccessControl System

1
An Empirical User Study of a Smartphone-Based
Access-Control System

• Kami Vaniea

Joint work with Lujo Bauer, Lorrie Cranor, Mike
Reiter and Rob Reeder
2
Physical access control
2
3
Limitations

• Must delegate all access tokens in advance
• Necessary to hide an access token for emergency
  situations
• Problems getting access tokens back
• Once given out key can be copied
• Requires users to carry additional objects

3
4
Smartphones

• What about using smartphones for access control?
• Smartphone capabilities
• User interface
• Computing ability
• Communication
• Smartphones are increasing in popularity
• Computational power of mobile phones also
  increasing

5
Research questions

• What are the usability challenges in building a
  smartphone-based access-control system?
• How well does a deployed smartphone-based
  access-control system match users needs?

6
Outline

• Introduction
• Grey Overview Deployment
• Study 1 System Acceptance
• Study 2 Policy Creation
• Related Work
• Conclusion

7
Grey

• Smartphone based access-control system
• Used to open doors in the CIC building
• Allows users to grant access to their doors from
  anywhere at any time

8
Grey example
Kami
Lorries Office
9
Grey advantages

• Can easily delegate authority
• In advance of the access
• At the time of the access
• Guarantee access is no longer allowed after
  specified time

9
10
Field trial environment

• 30 doors
• Perimeter doors to a large research area
• Offices
• Storage closets
• Conference room
• A lab
• A machine room

11
Users

• Chose participants who work together
• Wanted groups of users who share resources
• 29 users
• 9 faculty
• 11 graduate students
• 7 technical staff
• 2 administrative assistants

12
Interview procedure

• Interviewed participants
• Security practices
• Types of resources managed and needed
• Gave participants a smartphone with Grey
  pre-installed and brief instruction on use
• Interviewed one month later
• Changes in security practices
• General reactions to Grey
• Periodically conducted follow-up interviews at
  approximately one month intervals

13
Data

• Recorded approximately 30 hours of interviews
• System was actively used
• Logged 19,500 Grey accesses for 29 users
• Active users averaged 12 accesses a week
• Five users accessed their office almost
  exclusively with Grey
• Users interacted with an average of 7.4 different
  doors during the study
• Study lasted a year

13
14
Outline

• Introduction
• Grey Overview Deployment
• Study 1 System Acceptance
• Study 2 Policy Creation
• Related Work
• Conclusion

15
Research question

• What are the usability challenges in building a
  smartphone-based access-control system?

16
Design issues

• Analyzed interview data and identified five
  different design issues
• Speed
• Failures
• Complex features
• Non-Grey users
• New uses

17
Issue 1 Perceived speed

• Users quickly began to complain about speed and
  convenience of unlocking doors
• We knew Grey and keys required similar amounts of
  time to open a door
• Videotaped a highly trafficked door to better
  understand how doors are opened differently with
  Grey and keys

18
Issue 1 Videotaping

• Videotaped participants accessing kitchenette
  door
• Videotaped two hours daily after 6pm for two
  weeks
• 18 users taped
• 5 Grey participants
• 13 additional participants were solicited as they
  passed through the door

19
Issue 1 Average access times
Keys
Total 14.7 sec
3.6 sec
5.4 sec
s 3.1
s 3.1
Door Closed
Getting keys
Door opened
Stop in front of door
s 5.6
Grey
Total 15.1 sec
8.4 sec
2.9 sec
3.8 sec
s 2.8
s 1.5
s 1.1
Door Closed
Getting phone
Door opened
Stop in front of door
s 3.9
20
Issue 2 Failure

• Cost of failure is potentially high
• Rebooting a phone or door was considered very
  inconvenient
• Several users stopped using Grey actively after a
  single inopportune failure

21
Issue 2 Delays interpreted as failures

• Delays can be interpreted as failures even when
  the system is functioning perfectly
• Humans can be slow or unresponsive
• Providing feedback on the status of the request
  is very important
• Did it arrive?
• Is a human currently responding?

22
Issue 3 Confusing features

• Users would rather choose a suboptimal solution
  that they understand than one with an uncertain
  outcome
• Initially tried for concise interface (top)
• Adopted wizard solution (bottom)

23
Issue 4 Non-Grey users

• Grey is a service that becomes more valuable as
  more people use it
• Our participants were selected so that their work
  network included others with Grey
• Still had many people who would have benefited if
  Grey participant could have given access

24
Issue 4 Alices colleagues
Have Grey
25
Issue 5 Unanticipated uses

• Unlocking door from inside the office without
  having to stand
• Unlocking nearby door for someone else without
  leaving office

26
Study 1 summary

• Perceived speed and convenience are critical to
  user acceptance
• A single failure can strongly discourage adoption
• Users wont use features they dont understand
• Important to consider occasional users of the
  system
• Unanticipated uses can improve acceptance

27
Outline

• Introduction
• Grey Overview Deployment
• Study 1 System Acceptance
• Study 2 Policy Creation
• Related Work
• Conclusion

28
Research question

• How well does a deployed smartphone-based
  access-control system match users needs?
• Do users make more or less secure access-control
  decisions when using Grey than when using
  physical keys?

29
Policies

• A policy is a collection of rules
• A rule is a tuple containing a user, resource and
  condition (Bob, Alices office, true)

Alices Office
Bob
True
30
Methodology overview

• Examined access-control policies created by 8
  resource owners
• 8 offices
• 1 machine room
• Using interviews we created ideal, key and Grey
  policies for each of 9 resources
• Compared ideal and implemented rules

31
Ideal policies

• Ideal Policy Policy the user would enact if not
  restricted by technology
• Based on interview data
• Looked at not only what was enacted but
  endeavored to determine why

31
32
Policy synthesis
Garry
Frank
Rick
Larry
Mary
Joan
. . .
. . .
Lab owner is notified
Logged
True
Logged
Logged
False
Charlies Lab
32
33
Ideal conditions

• True (can access anytime)
• Logged
• Owner notified
• Owner gives real-time approval
• Owner gives real-time approval and witness
  present
• Trusted person gives real time approval and is
  present
• False (no access)

33
34
Policy analysis

• We compared each of the 244 ideal access rules,
  with the key and Grey rules and marked them as
• False Accept User not required to fulfill all
  conditions required by the ideal policy
• False Reject User must fulfill conditions not
  required by the ideal policy
• Faithfully Implemented Matched the ideal policy

34
35
Policy analysis example
Charlies Lab
Faithfully implemented
False Accept
False Reject
Alice
Sue
Bob
35
36
Keys vs. ideal
Alice
Bob
User 29
Sue
User 28
User 4
User 27
User 5
User 26
20 Faithful Implementations (Green) 4 False
Accepts (Red) 5 False Rejects (Yellow)
User 6
User 25
User 7
User 24
Charlies Lab
User 23
User 8
User 22
User 9
User 21
User 10
User 20
User 11
User 19
User 12
User 18
User 13
User 17
User 14
User 16
User 15
37
Conditions
Ideal
Keys

• True (can access anytime)
• Logged
• Owner notified
• Owner gives real-time approval
• Owner gives real-time approval and witness
  present
• Trusted person gives real time approval and is
  present
• False (no access)

• True (has a key)
• Ask trusted person with key access
• Know location of hidden key
• Ask owner who contacts witness
• False (no access)

?
37
38
Key implementation accuracy
Rules
Ideal Conditions
38
39
Conditions
Ideal
Grey

• True (can access anytime)
• Logged
• Owner notified
• Owner gives real-time approval
• Owner gives real-time approval and witness
  present
• Trusted person gives real time approval and is
  present
• False (no access)

• True (has a delegation)
• Ask trusted person with Grey access
• Ask owner via Grey
• Ask owner who contacts witness
• False (no access)

39
40
Implementation accuracy
Rules
Ideal Conditions
40
41
Study 2 Contributions

• Documented the collection of ideal policy data
• Developed a metric and methodology for
  quantitatively comparing accuracy of implemented
  policies
• Showed that a smarphone access-control system
  outperformed keys in overall security and
  effectiveness

42
Outline

• Introduction
• Grey Overview Deployment
• Study 1 System Acceptance
• Study 2 Policy Creation
• Related Work
• Conclusion

43
Related work

• Several Grey-like systems have been proposed but
  not implemented
• Digital Key system Beaufour and Bonnet
• The Master Key Zhu, Mutka and Ni
• Access-control tokens are not very easy to use
  and those that are tend to be less secure Braz
  and Robert Piazzalunga et. al.

44
Related work

• Usability of access control for file systems
• Manipulating access-control lists is difficult
  for users to do accurately Cao and Iverson
• Users have difficulty understanding how rules
  interact to form the effective policy Maxion and
  Reeder
• Studies of users access-control needs
• Identified several different approaches to access
  control management Ferraiolo et al.
• Users have dynamic access-control needs that very
  by task Whalen et al.

45
Summary

• Study 1
• Users have low tolerance for failure and treat
  Grey like an appliance
• Study 2
• Policies made using Grey were less permissive
  than key policies and better matched the ideal
  policies
• Related work
• Unlike previous work we study an actual working
  system and examine gathered empirical data

46
Future work

• Explore the tasks policy authors engage in
• Explore the use of a Grey like system in large
  organizations
• Develop technologies that assist in the authoring
  of policies

47
CMU Usable Privacy and Security
Laboratoryhttp//cups.cs.cmu.edu/
48
Bibliography

• X. Cao and L. Iverson. Intentional access
  management Making access control usable for
  end-users. In Symposium On Usable Privacy and
  Security, 2006.
• A. Beaufour and P. Bonnet. Personal servers as
  digital keys. In 2nd IEEE International
  Conference of Pervasive Computing and
  Communications, 2004.
• C. Braz and J. Robert. Security and usability
  The case of the user authentication methods. In
  IHM 06, p 199-203, 2006.
• D. F. Ferraiolo, D. M. Gilbert and N. Lynch. An
  examination of federal and commercial access
  control policy needs. In 16th National computer
  Security Conference, p 107-116, 1993.

49
Bibliography

• R. A. Maxion and R. W. Reeder. Improving
  user-interface dependability through mitigation
  of human error. International Journal of
  Human-Computer Studies, 63(1-2), 2005.
• U. Piazzalunga, P. Salveneschi, and P. Confetti.
  The usability of security devices. In L. F.
  Cranor and S. Garfinkel, editors, Security and
  Usability Designing Secure Systems that People
  Can Use, p 221-241. OReilly, 2005.
• T. Whalen, D. Smetters, and E. F. Churchill. User
  experiences with sharing and access control. In
  CHI 06 extended abstracts on Human factors in
  computing systems, p 1517-1522, 2006.
• F. Zhu, M. W. Mutka, and L. M. Ni. The master
  key A private authentication approach for
  pervasive computing environments. In 4th IEEE
  Interantional Conference on Pervasive Computering
  and Communications, p 212-221, 2006.

50
Grey accesses per week
Number of Accesses
Week