Systems Management Pack formerly SAM Pack Release 2.0

1
Systems Management Pack(formerly SAM Pack)
Release 2.0

• Scalability Review

2
Disclaimer

• NSM scalability is outside of the scope of this
  presentation.
• Prior to reviewing Systems Management Pack
  scalability, you must first ensure NSM meets
  your scalability requirements. If it does not,
  Systems Management Pack will not help.
• If Systems Management Pack is configured
  correctly, it should meet your scalability
  requirements - provided NSM ALSO meets your
  scalability requirements

3
Important Notes

• Systems Management Pack was formerly known as
  Service Availability Management Pack or SAM
  Pack
• Name change is being gradually implemented and
  some supporting documents and dialogs may still
  reflect the previous name. These are still valid

4
Objectives

• This presentation reviews Systems Management Pack
  configuration options for large deployments. This
  is particularly important when Systems Management
  Pack is configured to connect to a Repository
  with large number of WorldView Objects
• This review applies to Repositories with more
  than 50,000 objects.

5
Repository Size

• Tests performed in lab environment with 560K WV
  objects. In addition, system was loaded by
  running refreshservercache every 5 mins

The tests were performed in a lab environment -
using a Repository of this size is NOT recommended
6
NSM Scalability Considerations

• Review the following considerations for NSM
  scalability
• How many objects are in the Worldview?
• Will NSM applications scale with large number of
  WorldView objects?
• How many WorldView status updates are likely to
  be generated as a result of large number of
  objects?
• How many events are generated?
• How many Alert Management System alerts are
  expected?
• Will MCC scale?
• Will DSM scale?
• And others..

7
Systems Management Pack Maintenance

• This review is based on Systems Management Pack
  2.0, with the following maintenance applied.
• TSAMP004
• Addresses scalability when associating faults to
  IT Services
• TSAMP007
• Significantly improves scalability when
  refreshing and generating Systems Management Pack
  cache
• If Systems Management Pack is not at the correct
  level, you may achieve the desired results
• Systems Management Pack 2.01 includes the above
  fixes

8
Systems Management Pack Scalability Considerations

• Key considerations when Systems Management Pack
  is expected to work with large number of objects
• Systems Management Pack maintains two caches
  which are generated from Repository.
• SAMP Classification Cache
• IT Services Caches
• If the Repository has a large number of objects,
  cache will be impacted if not configured
  correctly.

9
SAMP Classification Cache

• This cache contains information such as-
• The importance level of classified objects
• Maintenance flag
• Details of the Service Calendar associated with
  the object
• Cache file resides in CA_TND_HOME\EvtAds
• File Names HostClass.csv and StaticClass.csv
• Please review Getting Started Guide for
  additional information about StaticClass.csv

10
SAMP Classification Cache

• When should this cache be refreshed?
• Maintenance flag is set or unset
• Importance level of a server is changed
• New server is discovered and needs to be
  classified
• Service Calendar added or modified and Service
  Calendar is used to suspend Monitoring

11
SAMP Classification Cache Configuration Option

• Interval value determines how frequently to
  refresh cache. If using service calendar it is
  recommended to set interval in multiples of 15
  minutes

12
SAMP Classification CacheRefresh Interval

• Frequency of interval can depend how often the
  elements that impact refresh cache are changed
• With TSAMP007 applied, it takes less than 60
  seconds to refresh roughly 390,000 objects. The
  time it takes to refresh may vary depending
  available server resources
• If new servers are discovered frequently, a
  refresh interval around 30 - 120 minutes should
  be fine
• If the server does not have adequate resources,
  the refresh interval can be roughly 1440 minutes
  - once per day

13
SAMP Classification Cache Automatic Refresh

• If automatic refresh is not required, set
  Refreshcache option to No
• Enables you to refresh cache on demand only
  (e.g., when elements change)

14
SAMP Classification Cache Refresh Methods

• Different methods to generate and refresh SAMP
  Classification Cache
• Execute RefreshServerCache.cmd from command
  prompt
• Select Refresh SAMP Cache option from MCC IT
  Services Action Menu
• Start Up Menu
• Automatic Refresh
• Refresh individual Device Entry
• This is TSAMP007 feature

15
SAMP Classification Cache RefreshServerCache

• Here you see SAMP Server cache refreshed using
  Refreshservercache command (560K objects in
  Repository)
• It took 18 seconds to build cache of 390K
  objects. It will also require approx 10 seconds
  to refresh cache once it is built. There were
  560K objects in the repository

16
SAMP Refresh CacheMCC Actions

• Here you see SAMP Server Actions options to
  Refresh Cache from MCC

17
Component MonitoringEvent Nodes

• Set AutoSelectEventNodes to No
• With large number of WV objects, it can take
  significant time to build list Events Nodes to
  monitor
• Manually create list of Event Nodes to monitor
• Or run it once with AutoSelectEventNodes Yes
  and then use the list generated to manually
  maintain it.

18
IT Services Cache

• With automatic SAMP Classification Cache refresh,
  it will request IT Services cache to be rebuilt
  as well
• With large number of objects, this should be
  disabled
• Set RefreshAECCache to No

19
What is IT Service Cache?

• Contains list of objects mapped to IT Services
• Depthlevel denotes how deep to search each
  container object to select the list of objects
  pertaining to that IT Service
• It takes a significant time to recursively walk
  through each child object
• If the IT Service cache is not refreshed, it may
  associate the fault to the Unknown IT Service

20
IT Services CacheDepthLevel

• The time it takes to generate IT Service cache
  will depend on the DepthLevel setting
• If set to max value of 6, it can take a
  significant time
• If direct children are pasted, then this
  significantly reduces the time it takes to
  generate cache

21
IT Service Cache Configuration Options

• Change CI Association Refresh Period to 0. This
  will disable automatic refresh of IT Service
  cache.

22
How to refresh IT Services Cache on Demand

• To build and refresh IT Services Cache, execute
• cawto SAMPDLL1 CRefreshAll
• Build process can take significant time but, once
  built, it only takes a few seconds (lt10 seconds)
  to refresh cache
• With depthlevel6, it processes approx. 1000
  objects per minute. This time is significantly
  reduced if depthlevel1.
• To restore cache from previously built IT
  Services cache, execute following
• cawto SAMPDLL1 CRestoreCache
• For 390K IT services, it took lt10 seconds to
  refresh it.

23
IT Service Cache

• IT Service cache build process does not impact
  Systems Management Pack Alert processing
• During the build process, cache is generated in
  staging area. Once build process is complete,
  cache promoted to be used by Systems Management
  Pack.

24
Multiple Systems Management Pack Servers
Architecture
25
Shared MDB Architecture

• If you deploy multiple Systems Management Pack
  Servers which share the same MDB, it is likely
  the repository will be a very large containing
  over 50,000 objects!
• If this is the case, you might consider pushing
  the cache files to each of the distributed
  Systems Management Pack servers that share the
  same MDB.
• If you have not deployed multiple Systems
  Management Pack Server sharing the same MDB, skip
  this section!

26
Multiple Server Shared MDB Architecture
Create AMS Alert on the Enterprise Server with
the shared MDB
Forward Normalized Processed Events


Local Sys Mgt Pack Server A
Local Sys Mgt Pack Server n
DSM
UNS
DSM
UNS
EVT
EVT
AMS
AMS
27
Shared MDB with Large Number of Objects

• In shared MDB environment - no requirement to
  generate cache from each Systems Management Pack
  server
• In this architecture, both cache types can be
  generated from a local Systems Management Pack
  server
• The local server usually has both Systems
  Management Pack and the local MDB installed. If
  there are no Systems Management Pack Servers with
  local MDB, then select the one with smallest
  network latency.
• Refresh caches on designated local server and
  push cache files to distributed Systems
  Management Pack Servers.

28
Shared MDB OptionsDisable Automatic Refresh

• Disable automatic refresh of cache on all
  distributed servers but NOT on the designated
  local server
• Set RefreshCache option to No

29
Shared MDB OptionsDisable IT Service Cache
Refresh

• Change CI Association Refresh Period to 0. This
  will disable automatic refresh of IT Service
  cache.

30
IT Service Cache Option

• Update samp.dat to set AutoRefresh_Message option
  to Yes

31
IT Services Refresh Message

• Create MRA for 511 messages to copy IT Services
  cache file to the distributed servers.
• MRA must set evalnode to designated Systems
  Management Pack Server to prevent MRA from being
  executed on any other distributed servers

32
Restore Cache Files

• IT Service cache file ITServices_Cache.csv
  resides in the SAMP\logs directory
• Define MRA for 511 message-
• Copy cache files on each distributed servers
• Trigger Restore Cache requests on each
  distributed server as well
• To trigger restore cache for the IT Service cache
  file, execute
• cawto ltdistributed samp Server namegt SAMPDLL1
  COption CRestoreCache
• The above request will restore cache - not
  generate it. This process only takes few seconds
  (normally lt 10 seconds)

33
Restore Cache

• Server classification cache file name is
  Hostclass.csv and can be found in the
  WVEM\EvtAds directory.
• File copy and cache restore can be triggered at
  the same time as 511 message.
• To trigger a restore of the server classification
  cache file on the distributed server, execute
• cawto ltdistributed samp Server namegt EVTADS
  CHostCache
• Similar to IT Services cache file, the copy and
  restore request must be sent to each distributed
  server

34
Scalability Tests
35
Event Storm Test

• Event storm of 16,100 faults was generated.
• Out of these, 16000 expected to map to 5 IT
  Services
• 100 were unclassified
• Notification method was SendKeep without Alert
• First 11,500 Alerts were processed without any
  latency. The last event had latency of around 3
  minutes.
• To load the system, refreshServercache was
  executed every 5 minutes

36
Event Storm Latency

• Here it shows 11500th Event processed without any
  latency

37
Event Storm Test

• The test was performed with
• 390,000 Classified objects
• 390,000 IT Service Association

38
Notification Method

• In the test, the notification method did not
  create AMS alert.
• Notification of service desk incident or AMS
  alert may take little bit longer.

39
Summary

• Configuration options for Systems Management Pack
  w/ Large Repository
• AutoSelectEventNodesNo
• Do not automatically select Event Nodes
• RefreshAECCacheNo
• Disable automatic refresh of IT Services Cache
• Depthlevel
• If direct children are pasted, set Depthlevel1
  (default)
• CI Association Refresh Period
• Set it 0 to disable automatic refresh of IT
  Service

40
Conclusion

• Refresh of SAMP Classification Cache is
  significantly fast and will not be impacted by
  large number of WorldView objects
• IT Services Cache refresh can require significant
  time depending on Depthlevel selected
• If using multiple Systems Management Pack
  servers, which share a single MDB, review push
  and restore request concept

41
Questions and Answers