Building and Maintaining a Successful Vulnerability and Patch Management Program

1
Building and Maintaining a Successful
Vulnerability and Patch Management Program

• Presented to Western Regional Educause
• April 2, 2008

2
Naval Postgraduate School Established in
Annapolis in 1909Moved to Monterey in 1951
3
Academic Programs

• Academic Schools
• Graduate School of Business and Public Policy
• Graduate School of Engineering and Applied
  Sciences
• Graduate School of Operations and Information
  Sciences
• School of International Graduate Studies
• Research Institutes
• Cebrowski Institute for Information Innovation
  and Superiority
• Wayne E. Meyer Institute of Systems Engineering
• Modeling Virtual Environments and Simulation
  Institute

4
NPS Population

• 1700 resident students
• Less than 50 are Navy
• 30 international officers
• Remainder Air Force, Army, US Marine Corps, US
  Coast Guard, and civilians
• 1700 faculty and staff
• 880 distance learning students

5
Accreditation

• Regional
• Western Association of Schools and Colleges
  (WASC)
• Programmatic
• ABET (some programs)
• AACSB
• NASPAA

6
Research-related Organizations

• CSU-Monterey Bay
• Monterey Peninsula College
• Monterey Institute of International Studies
• Hopkins Marine Station Stanford University
• Monterey Bay Education Science and Technology
  Center, University of California
• National Undersea Research Program (NOAA)
• Moss Landing Marine Lab (CSU)
• University of California Sea Grant Extension
• National Weather Service

• Monterey Bay National Marine Sanctuary, NOAA
  Pacific Fisheries Environmental Lab, NOAA
• Defense Language Institute
• Fleet Numerical Meteorology and Oceanography
  Center
• Monterey Bay Aquarium Research Institute
• Naval Research Laboratory
• Defense Manpower and Data Center
• Naval Postgraduate School
• Monterey College of Law
• Chapman College
• Golden Gate University

7
(No Transcript)
8
NPS Systems

• Multiple Networks 7 and growing
• Web services
• Extranet 326MB per day 55M hits per day
• Intranet 786MB per day 88M hits per day
• 5425 systems (computer, printer, scanner)
  connected to NPS network
• 6500 campus computer accounts
• 800 software applications
• 3150 active phone lines

9
Definition Information Assurance

• Measures that protect and defend information and
  information systems by ensuring their
  confidentiality, integrity, availability,
  non-repudiation and authentication. This
  includes providing for restoration of information
  systems by incorporating protection, detection
  and reaction capabilities. (DoDD 8500.1)

10
IA Attributes

• Confidentiality Disclosure of Information
• Integrity- Unauthorized modification of data
• Availability- timely, reliable access to data
• Non-repudiation- proof of delivery and identity
• Authentication proof of identity

11
IA Tools

• Computer Network Defense (CND)
• Vulnerability Management
• Alerts, Bulletins, Tech. Advisories Navys IAVM
  program
• Network Vulnerability Detection Tools- Retina/REM
• Patch Management Tools
• LANDesk
• WSUS
• Antivirus Tools
• Centrally managed Symantec Antivirus
• Barracuda Spam Filter
• Network Access Control
• Bradford Network Appliance
• Intrusion Detection
• Snort
• StealthWatch

12
Motivators

• Why did NPS create a Vulnerability and Patch
  Management Program?
• Attacks
• Welchia and Blaster 2003
• Other attacks have followed and continue to pose
  a significant threat.
• Mandates
• DoD/Navy - Information Assurance Vulnerability
  Management (IAVM) Process CJCSM 6510.01
• Best practices

13
Vulnerabilities over the last 10 years
Reference http//nvd.nist.gov/statistics.cfm
14
Scoping the Problem

• Our EDU network poses the biggest challenge
• Largest network at NPS
• Transient systems
• Many locally administered systems
• New vulnerabilities emerge daily.
• A strategy is needed that protects not only
  servers and network services, but workstations as
  well.

15
How to manage?

• Effectively managing this problem requires NPS
  to
• Maintain awareness of our vulnerability posture.
• Scan regularly to ensure compliance
• Obtain Local access to all NPS assets
• Update vulnerability audits.
• communicate vulnerabilities/remediation to system
  owners.
• Close the loop (feedback and documentation).

16
Where we were May 2007

• In-house system bridged gap between Foundstone
  and Remedy
• One remedy ticket One vulnerability on one
  system
• Vulnerability Technician did not work directly
  with system owners.
• Feedback system was almost non-existant
  (duplicate tickets, false positives)
• Vulnerability scanner was not properly
  configured.
• We reached a critical decision point
• Foundstone License was soon to expire.
• Given the expense to continue with Foundstone, we
  needed a more cost effective solution.

17
Leveraging our unique position

• Is NPS a University or a Naval Command?
• We are both!
• As a Navy Command, Retina/REM were available to
  us for free!
• Rather than face a coverage gap, we began
  learning Retina/REM.

18
Configuration

• One dedicated system allows us to scan for
  vulnerabilities.
• 1 Dell PE1950 (Windows 2003)
• eEye Retina Vulnerability Scanner
• eEye Retina Enterprise Management (REM) Console
• SQL Server 2000

19
Developing a scan schedule

• We recommend a more frequent scan schedule than
  once a month.
• Maintain better awareness of your vulnerability
  posture.
• NPS scans our class B address space weekly.
• Avoid scanning a large IP space in one session.
• Find a scan schedule that promotes easy
  troubleshooting.
• We scan by building, and scan 2-3 buildings per
  day.
• Scan DMZ when load is lowest.

20
Developing a Scan Schedule (cont)

• Maximize your coverage
• Do your users powerdown at night?
• Many of ours do.
• We scan DHCP zones during the day and static IP
  ranges at night.

21
Host-based Vulnerability Scans

• Requires local access to the machine
• For windows (local or domain admin)
• Other OSes (SSH account)
• Vulnerability audits are usually dependent upon
  examination of registry settings, file version,
  or package.
• At NPS the majority of our systems are
  Windows-based and belong to our Windows domain
  to those machines access is easy but

22
Host-based Vulnerability Scans (cont)

• Access to Researchers systems often presents a
  challenge.
• We have overcome these challenges by
• Establishing collegial relationships with our
  researchers we try not to be the man behind
  the curtain.
• Establishing a configuration management process
  that requires systems be rid of medium and high
  risk vulnerabilities.

23
Knowing who to contact

• Scans are of little value if the results are not
  shared with system owners for remediation.
• Determining a system owner for every system can
  be challenging and difficult to keep updated.
• We use SQL triggers to automate the
  discovery/assignment of enterprise workstations.
• Other systems are matched to an owner once a
  quarter.

24
Communication/Remediation

• We choose different remediation Strategy
  dependent upon asset type
• Enterprise Servers
• Enterprise Workstations
• Researchers / non-Enterprise administered systems

25
Enterprise Servers

• Administered by Server Management and Business
  Solutions Group
• Server management applies OS patches
• BSG applies application specific patches
• Patching is performed as a part of maintenance
  coincides with Patch Tuesdays.

26
Enterprise Workstations

• Several tools exist which aide patch deployment
  to our workstations
• Group Policy
• LANDesk
• Patch Management
• Remote Control
• Inventory Scanner

27
Enterprise Workstations (cont)

• WSUS
• Solves problem where LANDesk MS updates caused
  auto-reboot.
• Remote Desktop to the machines
• Some failed pushes are easy to fix but cannot
  be deployed to all systems.
• When all else fails re-image the machine.

28
Non-enterprise administered systems

• It is the responsibility of the administrator to
  patch their system.
• But, what about people who just wont patch?
• We deny their operational requests until our
  security requests have been met (most common
  request type is firewall related).
• We avoid threatening system disconnect unless
  absolutely necessary.

29
Closing the loop

• Document the feedback you receive.
• Sometimes recommended fixes fail.
• Occasionally false positives are reported.
• Documenting this information provides you with a
  clearer picture of what your actual vulnerability
  posture is.
• We have created a separate database which
  contains NPS specific fix information and false
  positives.

30
Where are we now? April 2008

• For the time being, remedy is not a part of the
  vulnerability management process all
  vulnerabilities are tracked within Retina/REM.
• Local access has been obtained on approximately
  90 of all network devices.
• Retina/REM appears to be a good fit for NPS and
  we have no licensing worries for the foreseeable
  future.
• Much closer relationships between Network
  Security and the rest of campus.

31
Summary

• Significant milestones in our IA program
• Deploying a vulnerability scanner (2003)
• Brought visibility to the gap between patched
  systems an those at risk
• Deploying LanDesk push (2004)
• Shortened timeline between vulnerability and
  patch
• Reduced sysadmin time accomplishing patching
• Immediately saw a drop in virus infections.
• Adding the WSUS or pull component to patching
• Reduced the patch time again.

32
Future Work

• Next step
• Bradford network appliance compliance appliance
• Challenges
• Scanning is limited to the access rights to each
  system.
• Research networks protected by a Firewall or
  behind a separate gateway are no visible without
  admin rights.

33
Questions?

• Contact Information
• Terri Brutzman
• tbrutzman_at_nps.edu
• Jason Cullum
• jcullum_at_nps.edu