Before%20we%20start

1
(No Transcript)
2
A New Star Is Born

• Before we start
• Introducing Maklee Engineering Solutions
• High quality services in the area of
• Performance improvements
• Oracle services / SQL tuning
• Platform migration
• Programming / Custom engineering
• Custom training

3
OpenVMS Backup V8.3 New Features

• Guy Peleg
• President
• Maklee Engineering guy.peleg_at_maklee.com

4
Agenda

• Main Algorithms
• Performance considerations
• V8.3 New Features

5
30,000 feet overview

• Reads data from disk into buffers in memory
• Writes buffers from memory to target saveset
• Savesets are sequential files
• Fixed length record
• RMS is used when writing/reading savesets to/from
  disks
• SYSQIO is used when writing/reading savests
  to/from tape drives

6
BACKUP

• BACKUP.EXE
• DCL Utility to parse qualifiers and call the main
  shareable image
• SYSUPDATEBACKUP.CLD
• BACKUPSHR.EXE
• parses API and branches to action routines
• BACKUP operates in 3 main phases
• Init phase
• Open File phase
• Map I/O phase

7
Init Phase

• Decides on useable buffer space
• WSQUOTA - WSQUOTA/64 300 JPI_FILCNT 512
• Creates each buffer using EXPREG
• Watch out This is R/W virtual address space
  charged against PGFLQUOTA
• Opens and reads in first directory

8
Open File Phase

• Looks for the next input file
• Open file and read file header for mapping
  pointers
• Map file to buffers
• Maps each file extent
• Breaks down I/O to fill a buffer
• Max. buffer size is 63.5 KB
• Even for a nicely contiguous file
• Stops when either FILLM has been reached or all
  buffers have file I/O mapped (WSQUOTA)
• No file data I/O at this point
• No I/O to output save set at this point

9
Map and I/O Phase

• Sort all mapped I/Os by increasing LBN numbers
• Without it performance decreases by 10
• Submit I/Os in LBN order in a tight loop
• Pre V8.3 algorithm (or latest V7.3-2, V8.2 ECOs)
• If DIOLM is reached wait 10ms
• Large DIOLM significantly impacts performance
• Wait for I/Os to complete
• Write data to output device
• Goto phase 1 if last buffer written

10
Disk Queue Load

• BACKUP uses a large buffer pool to read file data
  in an optimized manner
• Algorithm was designed 20 years ago
• All reads for the entire buffer pool are issued
  concurrently
• Over time buffer pools have gotten larger the
  I/O subsystems tolerance of being flooded with
  large number of I/Os has decreased
• Especially true with EVA XP storage controllers
• A picture is worth a thousand words.

11
FC Queue Depth
Very high peak, BACKUP blocks production I/O
12
FC Queue Depth Better Balance
Low peak, BACKUP no longer blocks production I/O
13
Port Queue Depth vs. Read KBS
Note the Read KBS (blue) value on this Port
14
Port Queue Depth vs. Read KBS
Read KBS on this Port remains the same. Lower
DIOLM value keeps a similar BACKUP throughput.
15
Impact of High FC Queue Depth
During periods of high queue depth, VDisk
Latencies exceed 100 milliseconds (thats
1/10th of a second!!)
16
Impact of Lowered FC Queue Depth
With lowered queue depth, VDisk Latencies only
once exceed 10 milliseconds (thats 1/100th of
a second much better)Production I/O workload
does not suffer due to BACKUP jobs.
17
Disk Queue Load

• Pre V7.3-2 V8.2 (plus patches) behavior
• issue as much I/Os possible allowed by DIOLM
• Continue issuing I/Os until we hit SS_EXQUOTA
• Wait for I/Os to complete and flood the I/O
  subsystem again
• The New algorithm
• Issue the disk reads in n parallel AST threads
• The completion AST of each thread issues the next
  I/O
• By default use 8 parallel I/O threads
• Number of threads controlled by new /IO_LOAD
  qualifier

18
Disk Queue Load

• Better performance
• Idle EVA controller showed 15 reduction in
  elapsed time
• Results are not linear busy controllers will
  witness more significant (dramatic !) improvement
• Direct attached SCSI disk showed 5 improvement
• YMMV

19
Performance Considerations
20
Performance Considerations

• Why BACKUP performance is an issue?
• Only 24 hours a day for incremental BACKUP
• Small window for backup
• Even smaller window for restore in case of
  emergency
• Always test your restore procedures !

21
Process Quotas

• System Managers Manual Volume 1 Essentials
• Chapter 11.7
• WSQUOTA
• Initial value of 10,000 pagelets, increment by
  5,000
• Over 100,000 performance decreases
• FILLM
• Equal to 100 (mostly sufficient), increment by 10
• Use larger values if input disk contains many
  very small files or is highly fragmented
• Effectiveness limited by WSQUOTA
• DIOLM
• Equal to 100 (mostly sufficient), increment by 10
• 8 for Modern storage controllers
• Larger values can cause disk I/O subsystem hangs
  or resets without any performance advantage

22
Process Quotas
Second option is 15 faster

• Less is more
• DIOLM32767, BIOLM32767, WSDEFWSQUOTAWSEXTENT3
  000000 VS.
• DIOLM8, BIOLM150, WSDEF10000,
  WSQUOTA20000, WSEXTENT30000

23
Media Compaction

• INIT/MEDIACOMPACTION
• BACKUP/MEDIA_FORMATCOMPACTION
• Compaction performed by the hardware
• Default is no compaction
• Compaction ratio depends on data and drive
• Tape drives are optimized to use compaction

24
SDLT 320 Compaction Performance
MBs Per Second (More is better)
25
Performance Considerations

• BACKUP / GROUP
• Defines the number of blocks backup places in
  each redundancy group.
• /GROUP_SIZE 10 is the default
• Modern drive technology makes the XOR group
  feature obsolete
• /GROUP_SIZE 0 to save 10 tape space
• BACKUP/CRC
• Stores CRC in each backup buffer
• Does not increase the tape utilization as cell is
  part of the record header
• Increases CPU consumption
• With modern tape drives consider using /NOCRC
  (but need to understand the possible risk)

26
Performance Considerations

• Tape drives are getting faster.
• Ultrium-960 can write _at_160 MB/sec
• With Ultrium drives elapsed time for Backup will
  be shorter but during this time the CPU may get
  overloaded (calculating CRC)
• 90 CPU utilization on DS25 writing to
  Ultrium-460 drive (_at_ 40MB/sec)
• May impact the availability of other applications
  on the system

27
Performance Considerations

• /BLOCK_SIZE
• Specifies the size of output block size in bytes
• Larger blocks -gt less I/Os -gt better performance
• Default of 32256 when writing to disks
• Default of 8192 when writing to tapes
• Modern tape drives optimized for large I/Os use
  maximum supported block size 65535
• BACKUP is optimized to save large files
• Same amount of data in small files vs. one large
  file may show up to 40 times difference.

28
RMS Optimizations

• SET RMS
• /BLOCK127 124
• /BUFFER 8
• /EXTEND65535 (for large savesets)
• /NETWORK127
• RMS_SEQFILE_WBH (Prior to V8.3)
• With recent versions VMS now sets RAH WBH

29
BACKUP/FAST

• Fast file scan
• Large number of input files
• Small number of files meet selection criteria

Elapsed time to backup all files created today
(less is better)
30
New Features
31
Improved CTRL-T information

• Traditional CTRL-T information showed the name of
  the current file being saved/restored and the
  total number of save-set blocks processed
• backup IA64KITSI64XB37.BCK/sav
  5dka100/ima
• MIKAXP_VTA61 143638 BACKUP
  CPU000000.86 PF908 IO2192 MEM256
• Restoring file 5DKA100DWMOTIF_SUPPORT_I64XB3
  7.KITHP.SICOMPRESSED1
• Saveset volume1, saveset block266 (32256 byte
  blocks)
• Can you tell how much data restored so far?
• Can you tell when the restore will be done?
• Introducing the new improved CTRL-T

32
Improved CTRL-T information

• IPL31_VTAT7 144627 BACKUP CPU000003.40
  PF6298 IO18408 MEM465
• Restoring file DKA100DWMOTIF_SUPPORT_I64XB37.K
  ITHP.PCSICOMPRESSED1
• Saveset volume1, saveset block720 (32256 byte
  blocks)
• 22.14MB restored out of 1.18GB, 1 completed
• Restore rate 965KB/sec, estimated completion
  time 150731.85

33
Improved CTRL-T information

• Note the message says ESTIMATED !!!
• Fancy CTRL-T message displayed when
• Restoring a saveset
• Creating an image backup
• Counters are updated when a file marked nobackup
  is encountered
• When the amount of data to be saved is unknown,
  only the rate and total amount of data processed
  so far is displayed
• CTRL-T can not be used in batch.and BACKUP is
  usually running in batch

34
BACKUP/PROGRESS_REPORTn

• /PROGRESS_REPORTS writes CTRL-T style message to
  the output device every given interval
• n is the number of seconds between intervals
• back IA64KITSI64XB37.BCK/sav
  dka100/ima/progress10
• BACKUP-I-PROGRESS, progress report generated at
  4-JAN-2006 150054.47
• Restoring file DKA100DWMOTIF_SUPPORT_I64XB37.K
  ITHP1.PCSICOMPRESSED1
• Saveset volume1, saveset block170 (32256 byte
  blocks)
• 5.22MB restored out of 1.18GB, 0 completed
• Restore rate 535KB/sec, estimated completion
  time 153928.28

35
lost saveset attributes

• A saveset transferred using FTP or compressed and
  decompressed using ZIP will lose its RMS
  attributes
• An attempt to process the saveset will fail
  
• BACKUP-F-NOTSAVESET
• Fortunately the correct RMS settings are stored
  in the saveset header

36
BACKUP / REPAIR

• /REPAIR instructs BACKUP to attempt and restore
  the correct RMS attributes
• backup images.bck/sav .exes/repair
• BACKUP-I-REPAIRED, saveset attributes changed to
  RFMFIX, MRS32256
• IPL31GUY 145858 BACKUP CPU000010.89
  PF7765 IO71628 MEM409
• Restoring file UPS000000.EXESBACKUPSHR.EXE16
  

37
Compression Support in BACKUP

• BACKUP supports data compression using ZLIB.
• ZLIBs advantages
• Free
• Legally unencumbered
• May be included in commercial applications
• Lossless data compression library
• Never expands the data
• Unlike DCX may be used to compress stream of
  bytes
• Does not need to analyze the file in advance
• Maximum compression factor 11000
• 50MB file filled with zeros compressed to 49KB
• Realistic numbers are in the range of 21 to 51
• http//www.zlib.net

38
Compression support in BACKUP

• ZLIB routines are shipping in a new shareable
  image
• SYSLIBRARYCOMPRESSSHR.EXE
• SYSLIBRARYCOMPRESSSHR_EV6.EXE (alpha only)
• New qualifier - /DATA_FORMATCOMPRESSED
• Instructs BACKUP to create a compressed save-set
• Does not need to be specified on the restore
  command
• Initially viewed as a feature to save space.but
  turned out to be a significant performance
  improvement (wellat least in some cases)

39
Compression support in BACKUP

• The slower the I/O subsystem the bigger the
  performance win
• Big win for network operations
• Big win for MSCP served devices
• doubles the CPU consumption
• On average BACKUP completes 2-5 times faster
• Useful when shipping dump files
• 800MB dump file was compressed to 75MB
• No support for writing compressed savesets to
  tapes
• Do not encrypt compress..at least for now

40
Compression support in BACKUP

• Compressed save-sets have variable length records
• Instead of fix
• First record is not compressed
• Detect compressed savesets
• Force largest record value of /BLOCK_SIZE
• To fix attributes after file transfer
• SET FILE X.CBCK/ATTRIB(RFMVAR,LRL32256,MRS322
  56)

41

• IPL31gt dir cxx.exe/sizall
• Directory SYSCOMMONSYSEXE
• CXXCOMPILER.EXE1 39.68MB/39.68MB
• CXXDEMANGLE.EXE1 41KB/42KB
• CXXLINK.EXE1 166KB/166KB
• Total of 3 files, 39.89MB/39.89MB
• IPL31gt backup cxx.exe cxx.bck/sav/log
• BACKUP-W-NOFILES, no files selected from
  DSA5SYS0.SYSEXECXX.EXE
• BACKUP-S-COPIED, copied DSA5SYS0.SYSCOMMON.SY
  SEXECXXCOMPILER.EXE1
• BACKUP-S-COPIED, copied DSA5SYS0.SYSCOMMON.SY
  SEXECXXDEMANGLE.EXE1
• BACKUP-S-COPIED, copied DSA5SYS0.SYSCOMMON.SY
  SEXECXXLINK.EXE1
• IPL31gt dir cxx.bck/siz

42

• IPL31gt dir cxx.exe/siz
• Directory SYSCOMMONSYSEXE
• CXXCOMPILER.EXE1 39.68MB
• CXXDEMANGLE.EXE1 41KB
• CXXLINK.EXE1 166KB
• Total of 3 files, 39.89MB
• IPL31gt backup cxx.exe cxx.cbck/sav/datacomp/log
• BACKUP-W-NOFILES, no files selected from
  DSA5SYS0.SYSEXECXX.EXE
• BACKUP-S-COPIED, copied DSA5SYS0.SYSCOMMON.SY
  SEXECXXCOMPILER.EXE1
• BACKUP-S-COPIED, copied DSA5SYS0.SYSCOMMON.SY
  SEXECXXDEMANGLE.EXE1
• BACKUP-S-COPIED, copied DSA5SYS0.SYSCOMMON.SY
  SEXECXXLINK.EXE1
• BACKUP-I-COMPRESS, data compressed by 66
• IPL31gt dir cxx.cbck/siz

Compression support in action
43
Encryption support in BACKUP

• BACKUP uses the Encrypt LP for data encryption
• How does it work?
• At run-time, BACKUP generates a random encryption
  key used for encrypting the save-set records
• The encryption key is being encrypted using a
  user provided key and stored in the save-set
  header
• Decryption the encryption key is retrieved by
  decrypting the key stored in the header using the
  user provided key
• BACKUP/ENCRYPT()
• NAME
• ALGORITHM
• VALUE

44
Encryption support

• Existing limitations
• Uses DESCBC algorithm (old, slow expensive)
• User provided algorithm is only used for
  encrypting the key
• DESCBC is still used for encrypting the data
• V8.3 adds AES encryption support to BACKUP
• Modern stronger encryption
• User provided algorithm is used for encrypting
  the data
• DESCBC is still used by default (to maintain
  backword compatibility)
• The following algorithms supported by BACKUP
• AESCBC128, AESCBC192, AESCBC256, AESECB, AESCFB
  and AESOFB

45
Encryption support - examples

• Create an encrypted save-set, specify the
  encryption key at run-time
• backup .com coms.bck/sav/encryptalgorithmaes
• Enter key value
• Verification
• Create an encrypted save-set, create the
  encryption key from DCL
• encryp/create_key guy "This is a nice key
  called guy"/aes
• backup .com coms.bck/sav/encrypt(algaes,name
  guy)
• backup coms.bck/sav /encrypt(algaes,nameg
  uy)

46
Compaction Encryption

• Compaction Encryption require additional CPU
  work
• Elapsed time will be impacted

47
Saving 2 GB file
rx2600 1.3 GHz OpenVMS V8.3 HSG80
Seconds to save a file (Less is better)
48
BACKUP DVE

• OpenVMS V7.3-2 added Dynamic Volume Expansion
  (DVE) support
• Two new terms introduced Logical Volume size and
  Expansion Volume size
• Controlled by the /SIZE /LIMIT qualifiers to
  the INITIALIZE SET VOLUME commands
• Error count 0 Operations
  completed 8936
• Owner process "_VTAT7" Owner UIC
  NPAR_BUILD
• Owner process ID 2817CDDC Dev Prot
  SRWPL,ORWPL,GR,W
• Reference count 2 Default
  buffer size 512
• Current preferred CPU Id 0 Fastpath
  1
• Total size 33.91GB Sectors
  per track 96
• Total cylinders 7719 Tracks
  per cylinder 96
• Logical Volume Size 2.38GB Expansion
  Size Limit 190.96GB
• Allocation class 5

49
BACKUP DVE

• BACKUP had no knowledge about DVE
• DVE characteristics of a device were lost when
  image backup performed
• Customers had to manually set DVE characteristics
  (assuming they noticed it was lost -)
• BACKUP now fully supports DVE
• VMS732_BACKUP-V0600

50
BACKUP DVE Expansion size

• The volume expansion size is being recorded in
  the save-set header
• BACKUP/LIST displays the expansion size if it
  exists in the save-set
• When restoring a save-set (using /image) or
  performing disk-to-disk image backup, the target
  device inherits the expansion size limit of the
  input device
• New qualifiers
• /IGNORELIMIT prevents the target device from
  inheriting the expansion size
• /LIMITn added to allow overriding the expansion
  size stored in the save-set header
• Corresponding to INIT/LIMIT

51
BACKUP DVE Logical size

• By default logical size is not preserved
• Restoring image backup of 4GB disk to a 36GB disk
  will only result in 4GB of usable disk space
• BACKUP/SIZE
• Instructs BACKUP to preserve the logical volume
  size during a restore operation
• BACKUP/SIZEn
• Instructs BACKUP to initialize the target device
  to have a logical volume size of n

52
Questions?

• See us at www.maklee.com for
• Performance improvements
• Oracle Tuning
• Platform Migration
• Custom Engineering solutions
• Custom Training