Computer Memory Systems

1
BCN 1043Computer Architecture and Organization
Chapter 5 Computer Memory
2
(No Transcript)
3
COMPUTER MEMORY

• Characteristics
• Memory Hierarchy
• Semiconductor memory
• Error Checking
• Cache Memory
• External Memory

4
Characteristics

• Location
• Capacity
• Unit of Transfer
• Access method
• Performance
• Physical type
• Physical characteristics
• Organization

5
Characteristics

• Location whether memory is internal or external
  to the computer. The processor also requires its
  own memory.
• CPU
• Internal
• External
• Capacity the amount of information that can be
  contained in a memory unit -- usually in terms of
  words or bytes
• Word the natural unit of organization in the
  memory, typically the number of bits used to
  represent a number
• Addressable unit the fundamental data element
  size that can be addressed in the memory --
  typically either the word size or individual
  bytes
• Unit of transfer The number of data elements
  transferred at a time
• usually bits in main memory (governed by data bus
  width)
• and blocks in secondary memory

6

• Access Method how are memory contents accessed
• Random access
• Each location has a unique physical address
• Locations can be accessed in any order and all
  access times are the same
• Example main memory
• Sequential access
• Data does not have a unique address
• Must read all data items in sequence until the
  desired item is found
• Access times are highly variable
• Example tape drive units

7

• Direct access
• Data items have unique addresses
• Access is done using a combination of moving to a
  general memory area followed by a sequential
  access to reach the desired data item
• Access is by jumping to vicinity plus sequential
  search
• Example disk drives

8

• Associative access
• A variation of random access memory
• Data items are accessed based on their contents
  rather than their actual location
• Search all data items in parallel for a match to
  a given search pattern
• All memory locations searched in parallel without
  regard to the size of the memory
• Extremely fast for large memory sizes
• Cost per bit is 5-10 times that of a normal RAM
  cell
• Example some cache memory units

9

• Performance
• Access time
• Rate at witch data transfer to/from the memory
  device
• Time between presenting the address and getting
  the valid time
• For RAM, the time to address the unit and perform
  the transfer
• For non-random access memory, the time to
  position the R/W head over the desired location
• Memory cycle time
• Access time plus any other time required before a
  second access can be started
• Transfer rate
• rate at which data transfer to/from the memory
  device

10

• Physical Types
• Semiconductor (RAM)
• Magnetic (Disk Tape)
• Optical (CD DVD)
• Physical Characteristics
• Volatile/nonvolatile
• Erasable/nonerasable
• Power consumption
• Organisation
• Physical arrangement of bits to form words

11
Memory Hierarchy

• Major design objective of any memory system is
• to provide adequate storage capacity (how much?)
• at an acceptable level of performance-access
  time(how fast?)
• at a reasonable cost (how expensive?)
• Trade-off among three key characteristics of
  memory cost,capacity and access time.
• Interrelated way to meet this goal
• Use a hierarchy of storage devices
• Characteristics of the memory hierarchy
• Consists of distinct levels of memory components
• Each level characterized by its size, access
  time, and cost per bit
• Each increasing level in the hierarchy (go down
  memory hierarchy) consists of modules of larger
  capacity, slower access time, and lower cost per
  bit

12

• Hierarchy List
• Registers
• L1 Cache
• L2 Cache
• Main memory
• Magnetic disk
• Optical disk
• Magnetic tape

-Increasing capacity -Decreasing cost/bit -Slower
(increasing) access time
13

• Hierarchy description
• Computer memory is organized into a hierarchy. At
  the highest level (closest to the processor) are
  the processor register. Next comes one or more
  levels of cache (L1,L2,etc).Next comes main
  memory. All of these are considered internal to
  computer system. The hierarchy continues with
  external memory, with the next level typically
  being a fixed hard disk, and one or more levels
  below that consisting of removable media such as
  optical disk, magnetic tape and ZIP cartridges.
• As one goes down the memory hierarchy, one could
  find increasing capacity, decreasing cost/bit and
  slower access time. It would be nice to use only
  the fastest memory, but because that is the most
  expensive memory, computer designer trade off
  access time for cost by using more of the slower
  memory. The trick is to organize the data and
  programs in memory so that the memory words
  needed are usually in the faster memory.

14
Semiconductor Memory

• Several types of semiconductor memory
• RAM (random-access memory)
• ROM (read only memory)
• Characteristics of RAM
• New data can be read or write from this memory
  with easily and rapidly
• Volatile
• -Must be provided with constant power supply
• -Data are lost if the power is turn off
• Can be used only as temporary storage information
• There are two basic types of RAM
• -Dynamic RAM (DRAM)
• -Static RAM (SRAM)

15
DRAM vs. SRAM

• Static RAM
• Bits stored as on/off switches
• No charges to leak
• No refreshing needed when powered
• More complex construction
• Less dense (less cell per unit area)
• More expensive
• Does not need refresh circuits
• Faster access time
• Consumes more power
• Application Cache memory

• Dynamic RAM
• Bits stored as charge in capacitors
• Charges leak
• Need refreshing even when powered
• Simpler construction
• More dense (smaller cell-more cell per unit area)
• Less expensive
• Need refresh circuits
• Slower access time
• Low power
• Application Main memory

16

• Characteristics of ROM
• Permanent type of memory storage.
• It is nonvolatile. (Data will not lose if the
  power is turn off)
• Data stored in ROM is either unchangeable or
  required a special operation to change it.
• Example of application in computer BIOS
• Types of ROM
• ROM
• PROM
• EPROM
• EEPROM
• Flash Memory

17

• ROMs
• Data is wired in during fabrication at a chip
  manufacturers plant
• Permanent pattern of data that cannot be change.
• Not possible to write new data into it.
• PROMs
• Programmable ROM
• Data can be written once by the user employing a
  PROM programmer
• Useful for small production runs
• EPROM
• Erasable PROM
• Programming is similar to a PROM
• Can be rewritten many times.
• Erase using ultraviolet (UV) light.

18

• EEPROM
• Electrically erasable PROMs - It contents are
  erased by applying a specific electrical voltage
• Can be written to many times while remaining in a
  system
• Does not have to be erased first
• Program and erase by individual bytes
• Used in systems for development, personalization,
  and other tasks requiring unique information to
  be stored
• Flash Memory
• Similar to EEPROM in using electrical erase
• Fast erasures, erase by block
• Higher density than EEPROM

19

• Memory Organisation
• Memory were divided to group of words( 8, 16 bit)
• 1 bit is a basic location on memory chips
• Basic element of a semiconductor memory is the
  memory cell
• Exhibit two states, which can be used to
  represent binary 1 and 0
• Capable of being written into (at least once), to
  set the state
• Capable of being read to sense the state

Control
select
Data in/out
20
Error Checking

• Most memory available today is highly reliable.
• Most systems simply have the memory controller
  check for error detection.
• Memory chips with built-in error-checking
  typically use a method known as parity to check
  for errors.
• High-end servers often have a form of
  error-checking known as error-correction code
  (ECC).
• ROM differ from RAM with no write control
• Parity Checking
• Parity chips have an extra bit for every 8 bits
  of data.
• Parity can be divided into even parity and odd
  parity.

21

• Example for Even Parity
• If the total number of 1s is odd, the parity
  bit is set to 1
• If the total is even, the parity bit is set to
  0
• When the data is read back, the total is added
  up again and compared to the parity bit.
• If the total is odd and the parity bit is 1,
  then the data is assumed to be valid and is sent
  to the CPU.
• But if the total is odd and the parity bit is
  0, the chip knows that there is an error (parity
  error) somewhere in the 8 bits and dumps the
  data.
• Odd parity works the same way, but the parity bit
  is set to 1 when the total number of 1s in the
  byte are even.
• Parity error can be caused by RAM chips that are
  unable to hold data reliably. This happens when
  the chips overheat or power falters.
• The problem with parity is that it discovers
  errors but does nothing to correct them.

22

• Error-correction code (ECC)
• Computers in critical positions need a higher
  level of fault tolerance.
• High-end servers often have a form of
  error-checking known as error-correction code
  (ECC).
• Like parity, ECC uses additional bits to monitor
  the data in each byte.
• The difference is that ECC uses several bits for
  error checking -- how many depends on the width
  of the bus -- instead of one.
• ECC memory uses a special algorithm not only to
  detect single bit errors, but actually correct
  them as well.

23
Cache Memory

• Cache memory is a critical component of the
  memory hierarchy
• Compared to the size of main memory, cache is
  relatively small
• Operates at or near the speed of the processor
• Very expensive compared to main memory
• Cache sits between normal main memory and CPU
• Cache contains copies of section of main memory
• When the processor attempts to read a word of
  memory, a check is made to determine if the
  word is in the cache.
• If so, the word is delivered to the processor.
• If not, a block of main memory, consisting some
  fixed number of words is read into the cache and
  then the word is delivered to the processor.

24
Cache Main Memory Interface

• Assume an access to main memory causes a block of
  K words to be transferred to the cache
• The block transferred from main memory is stored
  in the cache as a single unit called a slot,
  line, or page
• Once copied to the cache, individual words within
  a line can be accessed by the CPU
• Because of the high speeds involved with the
  cache, management of the data transfer and
  storage in the cache is done in hardware O/S
  does not know about cache
• If the are 2n words in the main memory, then
  there will be M (2n /K) blocks in the memory
• M will be much more greater than the number of
  lines, C in the cache
• Every line of data in the cache must be tagged in
  some way to identify what main memory block it is

25
Factors in Cache Design

• Size
• Mapping Function
• Direct
• Associative
• Set Associative
• Line Replacement Algorithm
• Least recently used (LRU)
• First in first out (FIFO)
• Least Frequently used (LFU)
• Random

• Write Policy
• Write through
• Write back
• Block/Line Size
• Number and type of Caches
• Single or two level
• Unified or split

26
Mapping Function

• Mapping functions -- since MgtgtC, how are blocks
  mapped to specific lines in cache
• Using three mapping techniques
• Direct mapping
• Associative mapping
• Set associative mapping

27
Direct Mapping

• Each main memory block is assigned to a specific
  line in the cache
• i j modulo C
• where i is the cache line number assigned to main
  memory block j
• Example If M64, C4
• Line 0 can hold blocks 0, 4, 8, 12, ...
• Line 1 can hold blocks 1, 5, 9, 13, ...
• Line 2 can hold blocks 2, 6, 10, 14, ...
• Line 3 can hold blocks 3, 7, 11, 15, ...

28

• Direct mapping cache treats a main memory address
  as 3 distinct fields
• Tag identifier
• Line number identifier
• Word identifier (offset)
• Word identifier specifies the specific word (or
  addressable unit) in a cache line that is to be
  read
• Line identifier specifies the physical line in
  cache that will hold the referenced address
• The tag is stored in the cache along with the
  data words of the line
• For every memory reference that the CPU makes,
  the specific line that would hold the reference
  (if it is has already been copied into the cache)
  is determined
• The tag held in that line is checked to see if
  the correct block is in the cache

29
Direct Mapping Cache Organization
30
Example

• Memory size of 1 MB (20 address bits) addressable
  to the individual byte
• Cache size of 1 K lines, each holding 8 bytes
• Word id 3 bits
• Line id 10 bits
• Tag id 7 bits
• Where is the byte stored at main memory location
  ABCDE stored?
• ABCDE 1010101 1110011011 110
• tag id55 line id39B word id6,

Tag id
Line id
Word id
7 bit
3 bit
10 bit
39B
55
6
31
Advantages and disadvantages of Direct Mapping

• Advantages of direct mapping
• Easy to implement
• Relatively inexpensive to implement
• Easy to determine where a main memory reference
  can be found in cache
• Disadvantage
• Each main memory block is mapped to a specific
  cache line
• It is possible to repeatedly reference to blocks
  that map to the same line number. These blocks
  will be constantly swapped in and out of cache,
  causing the hit ratio to be low

32
Associative mapping

• A main memory block can load into any line of
  cache
• Overcomes direct mappings main weakness Must
  examine each line in the cache to find the right
  memory block
• Examine the line tag id for each line
• Slow process for large caches!
• Line numbers (ids) have no meaning in the cache
• Parse the main memory address into 2 fields (tag
  and word offset) rather than 3 as in direct
  mapping
• AdvantagesFast Flexible
• DisadvantageImplementation cost

33
Fully Associative Cache Organization
34

• Example
• Memory size of 1 MB (20 address bits) addressable
  to the individual byte
• Cache size of 1 K lines, each holding 8 bytes
• Word id 3 bits
• Tag id 17 bits
• Where is the byte stored at main memory location
  ABCDE stored?
• ABCDE 10101011110011011 110
• tag id1579B word id6

Tag id
Word id
3 bit
17 bit
1579B
6
35
Set Associative Mapping

• Compromise between direct and fully associative
  mappings that builds on the strengths of both
• Divide cache into a number of sets (v), each set
  holding a number of lines (k)
• A main memory block can be stored in any one of
  the k lines in a set such that
• set number j modulo v
• If a set can hold X lines, the cache is referred
  to as an X-way set associative cache
• Most cache systems today that use set associative
  mapping are 2- or 4-way set associative
• 2 lines per set (2 way set associative mapping)

36
Two Way Set Associative Cache Organization
37
Example

• Memory size of 1 MB (20 address bits) addressable
  to the individual byte
• Cache size of 1 K lines, each holding 8 bytes
• Assume the 1024 lines are 4-way set associative
  1024/4 256 sets
• Word id 3 bits
• Set id 8 bits
• Tag id 9 bits
• Where is the byte stored at main memory location
  ABCDE stored?
• ABCDE 101010111 10011011 110
• tag id157 set id9B word id6

Tag id
Set id
Word id
9 bit
3 bit
8 bit
9B
157
6
38

• Line Replacement Algorithms
• When an associative cache or a set associative
  cache set is full, which line should be replaced
  by the new line that is to be read from memory?
• Least Recently used (LRU)
• e.g. in 2 way set associative Which of the 2
  block is LRU?
• First in first out (FIFO)
• Replace block that has been in cache longest
• Least frequently used
• Replace block which has had fewest hits
• Random

39

• Not a problem for direct mapping since each block
  has a predetermined line it must use
• No choice
• Each block only maps to one line
• Replace that line

40

• Write Policy
• When a line is to be replaced, must update the
  original copy of the line in main memory if any
  addressable unit in the line has been changed.
• Example of write policy
• Write through
• Write back

41

• Write through
• Anytime a word in cache is changed, it is also
  changed in main memory
• Both copies always agree
• Generates lots of memory writes to main memory

42

• Write back
• During a write, only change the contents of the
  cache
• Update main memory only when the cache line is to
  be replaced
• Causes cache coherency problems --different
  values for the contents of an address are in the
  cache and the main memory
• Complex circuitry to avoid this problem

43

• Block / line sizes
• How much data should be transferred from main
  memory to the cache in a single memory reference
• Complex relationship between block size and hit
  ratio as well as the operation of the system bus
  itself
• As block size increases,
• Additional information transferred will likely be
  used and thus increases the hit ratio - (good)
• Number of blocks in cache goes down, limiting the
  total number of blocks in the cache - (bad)
• As the block size gets big, the probability of
  referencing all the data in it goes down (hit
  ratio goes down) - (bad)
• Size of 4-8 addressable units seems about right
  for current systems

44

• Number of caches
• Single vs. 2-level
• Modern CPU chips have on-board cache (L1)
• 80486 -- 8KB
• Pentium -- 16 KB
• Power PC -- up to 64 KB
• L1 provides best performance gains
• Secondary, off-chip cache (L2) provides higher
  speed access to main memory
• L2 is generally 512KB or less -- more than this
  is not cost-effective

45

• Type of caches
• Unified vs. split cache
• Unified cache stores data and instructions in 1
  cache
• Only 1 cache to design and operate
• Cache is flexible and can balance allocation of
  space to instructions or data to best fit the
  execution of the program -- higher hit ratio
• Split cache uses 2 caches -- 1 for instructions
  and 1 for data
• Must build and manage 2 caches
• Static allocation of cache sizes
• Can out perform unified cache in systems that
  support parallel execution and pipelining
  (reduces cache contention)

46
External Memory

• Types of External Memory
• Magnetic Disk
• Hard disk
• RAID
• Removable
• Optical
• CD-ROM
• CD-Writable
• CD-R/W
• DVD
• Magnetic Tape

47

• Magnetic Disk
• The disk is a circular platter constructed of
  nonmagnetic material called the substrate, coated
  with magnetizable material.
• Data is recorded onto and later read from the
  disk using a conducting coil, the head.
• Substrate used to be aluminium.
• Now glass
• Improved surface uniformity
• Increases reliability
• Reduction in surface defects
• Reduced read/write errors
• Better stiffness (maintain data)
• Better shock/damage resistance (avoid surface
  damage)

48
(No Transcript)
49
Read and Write Mechanisms

• Recording and retrieval via conductive coil
  called a head
• May be single read/write head or separate ones
• During read/write, head is stationary, platter
  rotates
• Write
• Current through coil produces magnetic field
• Pulses sent to head
• Magnetic pattern recorded on surface below
• Read (traditional)
• Magnetic field moving relative to coil produces
  current
• Coil is the same for read and write
• Read (contemporary)
• Separate read head, close to write head
• Partially shielded magneto resistive (MR) sensor
• Electrical resistance depends on direction of
  magnetic field
• High frequency operation - higher storage density
  and speed

50

• Data Organization and Formatting
• Data is organized into concentric rings, called
  tracks, on the platter. Each track is the same
  width as head.
• Adjacent tracks are separated by gaps.This gaps
  is to prevent or minimize errors due to
  misalignment of the head or interface of magnetic
  fields.
• Data are transferred to and from disk in sectors.
• Adjacent sectors are separated by intersector
  gaps.
• Disk rotates at a constant speed constant
  angular velocity.
• Number of data bits per track is a constant.
• Data density is higher on the inner tracks.
• Sectors are identified on each track during the
  formatting process.

51
(No Transcript)
52
Physical Characteristics

• Fixed(rare) or movable head
• Removable or fixed
• Single or double (usually) sided
• Single or multiple platter

53
Fixed/Movable Head Disk

• Fixed head
• One read write head per track
• Heads mounted on fixed ridged arm
• Movable head
• One read write head per side
• Mounted on a movable arm

54
Removable or Not

• Removable disk
• Can be removed from drive and replaced with
  another disk
• Provides unlimited storage capacity
• Easy data transfer between systems
• E.g. Floppy disk and ZIP cartridge
• Nonremovable disk
• Permanently mounted in the drive
• E.g. Hard disk in PC

55
Multiple platter

• One head per side
• Heads are joined and aligned
• Aligned tracks on each platter form cylinders
• Data is striped by cylinder
• reduces head movement
• Increases speed (transfer rate)

56
Disk Performance Parameters

• Seek time - the time it takes to position the
  head at the right track
• Rotational delay - the time it takes for the
  beginning of the sector to reach the head
• Access time - the sum of the seek time and
  rotational delay
• Transfer time - time to read the block (sector)
  from the disk and transfer it to main memory

57
RAID

• Redundant Array of Independent Disks
• Set of physical disks viewed as single logical
  drive by O/S
• Data distributed across physical drives
• Can use redundant capacity to store parity
  information
• 6 levels in common use (0-5)
• Not a hierarchy

58
RAID 0

• No redundancy techniques are used
• Data is distributed over all disks in the array
• Data is divided into strips for actual storage
• Similar in operation to interleaved memory data
  storage
• Can be used to support high data transfer rates
  by having block transfer size be in multiples of
  the strip
• Can support low response time by having the block
  transfer size equal a strip -- support multiple
  strip transfers in parallel

59
(No Transcript)
60
RAID 1

• Mirrored Disks
• Data is striped across disks
• 2 copies of each stripe on separate disks
• Read from either
• Write to both
• Recovery is simple
• Swap faulty disk re-mirror
• No down time
• Expensive

61
(No Transcript)
62
RAID 2

• All disks are used for every access -- disks
• are synchronized together
• Very small stripes
• Often single byte/word
• Error correction calculated across corresponding
  bits on disks
• Multiple parity disks store Hamming code error
  correction in corresponding positions
• Lots of redundancy
• Expensive
• Not used

63
(No Transcript)
64
RAID 3

• Similar to RAID 2
• Only one redundant disk, no matter how large the
  array
• Simple parity bit for each set of corresponding
  bits
• Data on failed drive can be reconstructed from
  surviving data and parity info
• Very high transfer rates

65
RAID 4

• Each disk operates independently
• Good for high I/O request rate
• Large stripes
• Bit by bit parity calculated across stripes on
  each disk
• Parity stored on parity disk

66
(No Transcript)
67
RAID 5

• Like RAID 4
• Parity striped across all disks
• Round robin allocation for parity stripe
• Avoids RAID 4 bottleneck at parity disk
• Commonly used in network servers

68
(No Transcript)
69
Optical Disk

• Data is stored on an optical medium (such as
  CD-ROM or DVD), and read with a laser beam.
• Its not practical for use in computer
  processing, but it is an ideal solution for
  storing large quantities of data very
  inexpensively, and more importantly, transporting
  that data between computer devices.
• Optical disk products
• CD-ROM (Compact disk read-only memory)
• CD-R (CD Recordable)
• CD-RW (CD Rewritable)
• DVD (Digital Versatile Disk)

70
CD-ROM

• Originally for audio
• 650Mbytes giving over 70 minutes audio
• Polycarbonate coated with highly reflective coat,
  usually aluminum
• Data stored as pits and bumps
• Read by reflecting laser
• Constant packing density
• Constant linear velocity

71
CD Operation
72

• CD-Writable
• WORM (Write Once, Read Many disks)
• Now affordable
• Compatible with CD-ROM drives
• CD-RW
• Erasable
• Getting cheaper
• Mostly CD-ROM drive compatible

73

• DVD
• Digital Versatile Disk
• Will read computer disks and play video disks
• Multi-layer
• Very high capacity (4.7G per layer)
• Full length movie on single disk
• Using MPEG compression
• Finally standardized
• Movies carry regional coding
• Players only play correct region films
• Can be fixed

74
Magnetic Tape

• Use the same reading and recording techniques as
  magnetic disk system.
• Data on the tape are structured as a number of
  parallel tracks running lengthwise.
• The first kind of secondary memory
• Still widely used
• very cheap
• very slow
• Sequential access
• Data is organized as records with physical air
  gaps between records
• One words is stored across the width of the tape
  and read using multiple read/write heads