A Vision for the Testing of Election Systems in a HAVA World - PowerPoint PPT Presentation

1 / 26
About This Presentation
Title:

A Vision for the Testing of Election Systems in a HAVA World

Description:

1983 Turing Award Lecture Ken Thompson showed that conventional ... Security Testing / Penetration Testing. Spike testing (Variation of Load/Stress Testing) ... – PowerPoint PPT presentation

Number of Views:67
Avg rating:3.0/5.0
Slides: 27
Provided by: sos5
Category:

less

Transcript and Presenter's Notes

Title: A Vision for the Testing of Election Systems in a HAVA World


1
A Vision for the Testing of Election Systems in a
HAVA World
  • Eric Lazarus
  • EricLLazarus_at_yahoo.com

2
How Rate a Testing Capability?
  • Transparent
  • Identification
  • Recommendation
  • Cost effective
  • Broad coverage
  • Reliability
  • Accessibility
  • Usability
  • Security
  • Encourage high-value innovation
  • Pick correct structure given success

3
This is a tough problem
  • 1983 Turing Award Lecture Ken Thompson showed
    that conventional methods will fail
  • A Trojan Horse can live in a compiler, linker,
    loader, interpreter, micro code, BIOS, hardware
  • Testing is hard and limited

4
Types of Testing
  • Acceptance/Qualification Testing
  • Code inspections/code walk through
  • Concurrency testing
  • Data table testing
  • Disability Access Testing (Variation of usability
    testing)
  • Installation testing
  • Integration Testing
  • Legal Validation/Verification (Validate legal
    requirements then verify legal requirements met)
  • Load/Stress Testing
  • Performance testing (test response times)
  • Recovery testing
  • Regression testing
  • Reliability Testing
  • Scalability testing (variation of load/stress
    testing)
  • Security Testing / Penetration Testing
  • Spike testing (Variation of Load/Stress Testing)
  • Uninstallation testing (variation of installation
    testing)
  • Unit Testing
  • Upgrade/Patch testing (variation of installation
    testing)
  • Usability testing

5
Applied Common Sense
  • Vision is not hard to come by
  • Create a vision
  • What are the questions?
  • What are common-sense answers?
  • Bring together smart people to think about the
    obvious vision

6
Q States Testing Independently?
  • go it alone, or
  • or Voluntary Consortium of States?

7
Q States Testing Independently?
  • Voluntary Consortium of States
  • Hire more and/or better people
  • Save on duplicated effort
  • Better shared knowledge gained in
  • Product evaluation
  • Use

8
Q Who should pay for it?
  • Not vendor funded as with ITA system
  • Interest clash
  • Barrier to new entries
  • Pooled state election money
  • What about others including
  • Political parties
  • Good government groups
  • Civil rights groups
  • Academic institutions

9
Q Big-Bang or Continuous?
10
Q Big-Bang or Continuous?
  • Like getting regular checkups
  • Nevada gaming control board takes machines out of
    service

11
Q White Box or Black Box?
  • Why handicap our testers by not giving them
    source?
  • We want to find bugs source code review is good
    for this
  • Every branch much be run too many to
    realistically be done in voting system software

12
Q Partisans Included?
  • Brennan Center for Justice projects worked both
    ways
  • Working with people on both sides of debates has
    brought out insights
  • Smart and knowledgably is important such people
    often have opinions

13
Q Team must have
  • Understand election processes
  • Understand computer security techniques
  • Testing in other domains
  • Background from other industries including gaming
  • International perspective
  • Heterogeneous team how do find problems

14
Q Product Roadmap
  • Can election officials impact product direction
    via a consortium?

15
Q Consortium Services?
  • What can they offer?

16
Q Develop Risk Models
  • Testing should be driven by clear view of the
    risks testing is attempting to address
  • We might buy a machine that is not as accessible
    as we are told.
  • not as secure.
  • not as reliable.
  • not as easy to administer.
  • Good to develop and maintain these jointly

17
Q Shared Repository of Knowledge?
  • What was learned under testing?
  • What was learned in use?
  • What procedures work well with this technology?
  • Model Information Sharing and Analysis Centers
    (ISACs) e.g., Financial Services Information
    Sharing and Analysis Center www.fsisac.com/about.
    htm

18
Q Evaluating Election Procedures?
  • Could this same team evaluate procedure manuals?
  • Should be able to evaluate procedures against
    best practices

19
Q Testing When?
  • Product Evaluation
  • Certification
  • Acceptance
  • Logic Accuracy
  • Continuous

20
Q Other services?
  • Negotiate joint purchasing agreements (like GSA
    Schedule)
  • Products
  • Services
  • Transparency Arrange for purchasable by
    responsible organizations
  • Encourage innovation by
  • Adhering to open standards

21
Q Make policy?
  • Should such consortia of states do testing and
    provide testing information or should they take
    on policy making role?

22
Q Make policy?
  • Should such consortia of states do testing and
    provide testing information or should they take
    on policy making role?
  • Ive been assuming that these staff would make no
    policy but only provide the results of their
    tests. They would not, for example, certify or
    decertify machines but would report on results of
    testing.

23
So one vision emerges
  • Multiple states group into a consortium (or two)
  • Has own staff and/or consultants, small
    contractors, academics
  • Performs Testing for
  • Usability
  • Security
  • Evaluates
  • Procedures
  • New technology
  • Cost

24
Does this make sense?
  • Very interested in collaborating around a
    proposal to create a consortium
  • How can we improve this vision?
  • Please contact me if you want to work on this

25
Testing is not an end in itself
GOAL Improved Elections
Authority
Commitment
Skills
Current State
Resources
Testing
26
Illustration Gaming Whats Different?
  • Ladder of trust with signed firmware at bottom
  • Multiple people with different keys
  • Field trails as part of certification
  • Hash compare in the field randomly every two
    years
  • Auditing the auditors
  • Certification done by government employees
    willing to share/discuss their methods
  • Post-employment restrictions on working for
    vendors
  • Penalties for messing up
  • Assumption of cheating
Write a Comment
User Comments (0)
About PowerShow.com