North Carolina Digital Government Summit Continuity of Operations: Technology Trends and Solutions S

1
North Carolina Digital Government
SummitContinuity of Operations Technology
Trends and SolutionsSeptember 2, 20091045am
1200pm Ann Garrett, State CISOOffice of
Information Technology Services
2
Continuity of Operations Technology Trends and
SolutionsOutline
3
Continuity of Operations Technology Trends and
SolutionsHouse Bill 2436, Section 6.10 (2007
2008 Session) IMPROVE DISASTER RECOVERY AND
BUSINESS CONTINUITY

• SECTION 6.10.(a) The State Chief Information
  Officer (CIO) shall utilize the business and
  disaster recovery plans submitted under G.S.
  147-33.89 and any other information at the CIO's
  disposal to determine whether State agencies have
  made adequate preparations for backing up
  critical applications.
• SECTION 6.10.(b) In cases where backup is not
  sufficient to minimize any disruptions in
  critical State services caused by natural or
  man-made disasters, the State CIO, in conjunction
  with the agencies and the Office of State Budget
  and Management, shall develop plans to utilize
  the Western Data Center for providing backup.
• SECTION 6.10.(c) By December 1, 2008, the State
  CIO shall report to the Joint Legislative
  Oversight Committee on Information Technology on
  the number of critical State applications without
  adequate backup, the State agencies utilizing the
  applications, and the plans for providing
  adequate backup.
• SECTION 6.10.(d) This section does not apply to
  the General Assembly, to the Judicial Department,
  or to The University of North Carolina and its
  constituent institutions.

4
Continuity of Operations Technology Trends and
SolutionsRecommendations To General Assembly
December 2008

• 1. Improve, standardize and centrally manage
  business continuity planning tools.
• 2. Use the data from enterprise business
  continuity planning tools to help identify common
  gaps in BCPs for the recovery of critical
  applications.
• 3. Identify critical dependencies and plan for
  adequate support infrastructure.
• 4. Work with Dept. of Homeland Security and the
  Division of Emergency Management on security
  measures at the Eastern Data Center and Western
  Data Center.
• 5. Provide agencies with the opportunity to
  conduct both informal and formal business
  continuity tests and maintain records of agency
  critical application BCP tests.
• http//www.scio.state.nc.us/News_Files/Disaster_R
  ecovery_and_Business_Continuity.pdf

5
Continuity of Operations Technology Trends and
SolutionsAccomplishments / Successes

• 1. Improve, standardize and centrally manage
  business continuity planning tools.
• One Business Continuity Planning Tool for BCP and
  COOP
• - Living Disaster Recovery Planning System
  (LDRPS)
• With NCID front-end for user-id and password
  management
• Web based application in a virtual Windows
  environment
• Graphical dependency mappings
• 2. Use the data from enterprise business
  continuity planning tools to help identify common
  gaps in BCPs for the recovery of critical
  applications.
• Standardized plans using Crystal reports and
  Oracle database
• All BCPs follow the same table of contents
• All have the same look and feel
• Easier to locate critical plan elements
• Application Portfolio Mgt tool captures critical
  application data
• Produced the IT Scorecard for the Governor

6
Continuity of Operations Technology Trends and
SolutionsAccomplishments and Successes

• 3. Identify critical dependencies and plan for
  adequate support infrastructure.
• Example

7
Continuity of Operations Technology Trends and
SolutionsAccomplishments and Successes

• 4. Work with Dept. of Homeland Security and the
  Division of Emergency Management on security
  measures at the Eastern Data Center and Western
  Data Center.
• Completed
• 5. Provide agencies with the opportunity to
  conduct both informal and formal business
  continuity tests and maintain records of agency
  critical application BCP tests.
• Semi-annual formal test cycles in place
• Informal testing can be considered
• Tracking of BCP testing is now more robust

8
Continuity of Operations Technology Trends and
Solutions Western NC Data Center

• The need for a second data center for NC
• Study in 2004 identified need for fully
  redundant, round-the-clock disaster recovery
  capability in N.C.
• Growth causing strain on outside vendors
  capacity at single recovery site
• Disaster recovery testing time consuming and
  expensive
• Escalating costs
• Finite floor space and power capacity at main ITS
  site in Raleigh

9
Continuity of Operations Technology Trends and
Solutions Western NC Data Center

• Additional factors
• Growing recognition in agencies and General
  Assembly of disaster recovery needs for critical
  business programs
• Future capacity needs because of growth and State
  CIO initiatives, such as consolidation of IT
  infrastructure
• Administration desire to create jobs in state

10
Continuity of Operations Technology Trends and
Solutions Western NC Data Center

• Our approach
• Built case with administration, then N.C. General
  Assembly
• Identified site
• Study recommended Western NC
• Physical separation from Raleigh in case of
  hurricane, ice storm, earthquake, homeland
  security incident
• Looked at 20 counties, 20 sites
• Worked with private sector and state and federal
  government agencies

11
Continuity of Operations Technology Trends and
Solutions Western NC Data Center
53,000 square feet 15,000 square feet raised
floor space Full redundancy in power, cooling and
fiber 24/7/365 operation
12
Continuity of Operations Technology Trends and
Solutions Western NC Data Center

• Outcome
• Completed on time and within budget
• Three full-scale disaster recovery exercises
  completed since June 2008
• 15 agencies participating in DR exercises a
  record number
• ITS ended contract with out of state vendor in
  June 2008, and began backing up all core services
  at the Western Data Center

13
Continuity of Operations Technology Trends and
Solutions State Data Centers

• Technologies
• Modernized gear to provide automatic switching
  from line, battery and generator power
  sources at EDC
• MPLS enabled, fully redundant, 10G ring
  architecture providing connectivity between the
  data centers
• VMWare with virtual tape at WDC
• SAN at EDC
• VOIP

14
Continuity of Operations Technology Trends and
Solutions Vision

• State of the Art Data Center going light green
• Redundancy
• Synchronization between EDC WDC
• XRC Extended Remote Copy for Mainframe
• Teleworking expanded
• Enabling more staff to communicate and work
  remotely
• No single points of failure
• Standardization and blending of the BCP and COOP

15
Continuity of Operations Technology Trends and
Solutions

• Questions?