Title: WORLD TRADE CENTER And Related Recoveries Len Guddemi Vice President Northeast Operations
1 WORLD TRADE CENTER And Related
Recoveries Len Guddemi Vice
President Northeast Operations
Comdisco Confidential
2SunGard Worldwide Disaster Support Experience
Recoveries per Year
3SunGard Worldwide Disaster Support Experience
Disruption Cause Number/Percentage
4SunGard Worldwide Disaster Support Experience
Recovery Platforms Used Quantity/Percentage
5SunGard Worldwide Disaster Support Experience
Days of Occupancy Days/Percentage
6Declaration Information
- Number of Events
- Overall - 93 Schedules, 47 Customers
- Simultaneous High Point - 76 Declarations, 16
Facilities, 14 Platforms
Reason Facilities Unavailable Loss of Power,
telecommunications, denied access to buildings,
concerns over building structural safety
Comdisco Confidential
7SunGard Sites Platforms
North Bergen Voice/Work Area, Main-Frame,
ARS Networking Phoenix Work Area Queens
Work Area, Trading Floor, Dedicated, Trading
PIT Rosemont Work Area San Ramon Work Area,
Trading Floor Wood Dale DEC/VAX, DEC/Alpha,
VAX3100, VAX7620, Filenet, AS400, Tandem
Wood Cliff Lake Intel Servers, Work Area Jersey
City, workgroup, AS/400 Philadelphia Work area,
RS6000, AS/400,Mainframe, Voice Herndon Work
Area Warminster Work Area,Unisys
- Atlanta Work Area Seats, Remote Sun, AS400,
RS6000 - Carlstadt Voice/Work Area, Business
- Recovery/Quick Ship, Trading Floor,
- Dedicated Seats, SUN, AS400,
- DEC/VAX, RS6000, HP9000, Intel
- Based Servers Dedicated HP9000,
- RS6000, NT Servers
- Boston SUN, Work Area
- Cypress Work Area, Remote Tandem
- Mobile Recovery 13 Units/5,000sq.ft., 10
Units/3,000sq.ft. for Work Area - Minnetonka Work Area
Comdisco Confidential
8Declaration Information - Continued
Items of Note Implement Your Plan Associated
Vendor Services with Urgency 14 Declarations in
30 minutes 24 within 1 hour 19 in 2nd hour You
will be in competition with other business for
services manufacturers Quickly establish damage
assessment need for services, and implement
- Under-subscription or additional needs assessment
remains an issue - Many businesses requested additional needs ATOD
- Responded to over 500 special requests for
service. - Built out with best efforts
- This puts recovery business continuity at risk
due to extended recovery timeline. - Personnel Intellectual Property Keys to
Recovery - Methodologies, ideas and documented plans
utilized. - Assigned backup roles and succession planning
implemented. - Intellectual capital and personnel in alternate
locations. - Use of professional organizations to augment
staff.
Comdisco Confidential
9Declaration Information - Continued
Items of Note - Continued Reluctance to Travel
End Users generally wont travel far from home.
Ensure adequate end user facilities available
nearby (about 50 miles)
- Air Travel Unavailable
- Delayed delivery of offsite stored media to
recovery location - Identify alternate means of transportation
- Assess need for electronic journaling of data to
alternate location - Expert staff unable to travel
- Identify options for local access by staff to
remote alternate systems (and possibly home
systems) - Assess value of turnkey type services with
vendors - Offsite Storage Critical
- Evaluate for frequency, completeness
- Paper records and transactions may be totally
lost - Evaluate if your offsite storage is far enough
away - Evaluate need/value of electronic journaling
Comdisco Confidential
10Declaration Information - Continued
Items of Note - Continued High Availability
Solutions Extremely Successful Electronic
Journaling, Data Base Mirroring, SRDF, etc. 45
Terabytes ready very quickly Data requiring
physical transport delayed
- Active Testers
- More successful at meeting RTO/RPO
- More realistic understanding of application
performance - Fewer problems
- Very High Demand for Voice Recovery
- Realistically evaluate needs line trunking,
extensions, call recording, voice mail, call
forwarding - Avoid Fast Busy
Comdisco Confidential
11(No Transcript)
12(No Transcript)
13(No Transcript)
14(No Transcript)
15(No Transcript)
16(No Transcript)
17(No Transcript)
18(No Transcript)
19(No Transcript)
20(No Transcript)
21(No Transcript)
22SunGard Challenges Tactics
- Disaster Hotline Notification
- Increased Staff handling incoming Disaster
calls - Security
- Immediate increase in security staff on all
shifts. - Contacted local authorities to assist in access
control - Additional Staff
- Request for additional support staff
- Mobil Recovery Team
- Additional Resource
- Contacted third party vendors
- Arranged additional parking with neighbors
23SunGard Challenges Tactics
SunGard Facilities and Staff not directly
affected
- Implemented Swat Support Teams
- Established 24/7 Command Center at Corporate HQ
with Open Bridge - SunGard Canadian employees, Illinois,
Minneapolis, and Texas employees drove to
Carlstadt North Bergen to help - SunGards Professional Services Consultants,
Sales, Sales Support, Administration and
SunGards Operations staff worked side by side. - Dispatched relief support from other SunGard
facilities once air travel restored - Activated facilities, equipment procurement,
shipping and solutions teams to respond to
special requests - Received offers of help from customers
- Gratefully declined
- Network Alternate Routing Critical
- Significant damage due to the event
- Fiber cut after event by construction/demolition
crews - SunGard alternate routings worked without issue
- Vendors terrific with T1s, DS3 installations in
days
Comdisco Confidential
24Customer Challenges Tactics
Inaccurate Needs Assessment Special requests
for PCs, Servers IP addressing Storage Tape
Drives Electricity Processing power LAN
bridging Televisions Printers Network
capability, Voice services Routers Faxes
Software and more
Comdisco Confidential
25Customer Challenges Tactics
- Logistics
- Travel Restriction an Impact - used ground travel
for many shipments - Shipped over 4,200 PCs along with much other
equipment - Customer parking, bussing, and catering
Getting Restoration Media to Alternate Site
Transportation challenge Complete set available
and labeled Missing or unavailable
documentation Start-up scripts Run instructions
Comdisco Confidential
26Learned / Re-Affirmed
Value in Pre-Planning Staff Placement within
Rooms End User priorities usage End User
equipment requirements avoids setting up more
than once
- Important to Have a Team Leader for Each Shift
- Integration of recovery requirements into change
control - Number of PCs, Positions
- Network Capabilities
- Processors, disk, tape capabilities
- Printers, fax machines
- PBX features functionality
- Platform Technology upgrades, changes,
maintenance - Importance of Quick Assessment Implementation
of Plan when appropriate
Comdisco Confidential
27Learned / Re-Affirmed - Continued
- Value of Solid Backup Records Program
- Tape, electronic, paper records foundation of
recovery - Evaluate right balance for your company
- Documented labeled so vendor(s) can assist
- Have scripts shipped with restore media or stored
at alternate location - Frequency of backups
- Offsite location transportation dependencies
- Examine benefit of electronic transmission of
offsite storage data - Can significantly improve recovery time and
reduce data loss
Evaluate Strategy of Shared versus Dedicated
Resources Guarantees resource access for some
number of most critical requirements Complement
with shared resources
PC Image Restoration Tool Effective Ghost, or
similar
Comdisco Confidential
28Learned / Re-Affirmed - Continued
- Basics are Fundamental
- Establish personnel roles assign backup
- Document methodologies, ideas, business plans
recovery plans - Review, update follow established procedures
- Test the way you would recover, recover the way
you test - Worthwhile to review how home procedures apply in
alternate site production mode - Offsite Storage, Disk mirroring, Miscellaneous
other
Revisit your Plan Recovery Requirements Often
Technology Business In Times of Need, Help
Comes From Many Sources
Comdisco Confidential
29Customer Comments
Although operating in our primary location is
preferred, the equipment, professional and
courteous staff, and outstanding overall
accommodations provided for an effective and
successful business recovery. I am glad that we
have contracted with your organization!
I have in the past worked with a number of
outstanding individuals, but never before have I
worked with such a highly functional, customer
service oriented team. Individually each team
member I have had the pleasure to deal with was
exceptional as a team there is none better.
Comdisco Confidential
30Customer Comments
I would like you to know of the "more than
wonderful" service we are receiving at the
SunGard site. ... have gone out of their way to
assist us with some special needs. ... have
answered numerous questions and assisted --she is
great. ... worked with ENS last night, assisted
me with set up so I can stay on line (due to the
situation in New York and the weather closing
down our site) and has been a great help to the
IT staff.
I would like you to know it is people like this
that make SunGard a vendor we choose to depend on
in the event of an emergency. Please pass this
message along to their managers, it would be
greatly appreciated.
- Thanks for all the support and assistance.
Were glad you are here.
Comdisco Confidential
31Questions and Answer
???
Comdisco Confidential