Memory Hierarchy and Cache Memory

1
Memory Hierarchy and Cache Memory

• Jennifer Tsay
• CS 147
• Section 3
• October 8, 2009

2
Two Basic Types of Memory

• RAM (Random Access Memory)
• Used to store programs and data that computer
  needs when executing programs
• Volatile and loses information once power is
  turned off

3
2 Basic Types of Memory

• ROM (Read-Only Memory)
• Stores critical information necessary to operate
  the system, such as program necessary to boot
  computer
• Not volatile and always retains its data
• Also embedded in systems where programming does
  not need to change

4
Memory Hierarchy

• Hierarchal Memory
• Approach in which computer systems use
  combination of memory types to provide best
  performance at best cost
• Basic types that constitute hierarchal memory
  system include registers, cache, main memory and
  secondary memory

5
Memory Hierarchy

• Todays computers each have small amount of very
  high-speed memory, called cache where data from
  frequently used memory locations may be
  temporarily stored
• Cache is connected to main memory, which is
  typically medium-speed memory
• Main memory is complemented by secondary memory,
  composed of hard disk and various removable media

6
The Memory Hierarchy
7
How Do We Classify Memory?

• Based on its distance from the processor
• Distance is measured by the number of machine
  cycles required for access
• The closer memory is to the processor, the faster
  it should be

8
Locality of Reference

• Clustering of memory references into groups
• Important because through use of cache technique,
  locality of reference can optimize performance

9
Cache Memory

• Small, temporary, but fast (and thus high-cost)
  memory
• Processor uses cache memory for information it is
  likely to need again in very near future
• Non-computer Examples
• Cell Phone Address Book vs. Phone Book
• Desk vs. File Cabinet

10

• MAIN MEMORY

• CACHE

11
Cache Mapping Schemes

• For cache to be functional, it must store useful
  data
• This data becomes useless if CPU cant find it
• When accessing data or instructions, the CPU
  first generates main memory address
• If data has been copied to cache, address of data
  in cache is not same as main memory address

12
Types of Mapping Schemes

• Direct Mapped Cache
• Each main memory block has specific location to
  which it maps in cache
• If block already occupies cache location where
  new block must be placed, block currently in
  cache is removed

13
Types of Mapping Schemes

• Fully Associative Cache
• Allows main memory block to be placed anywhere in
  cache
• When cache is full, need replacement algorithm to
  decide which block to throw out of cache

14
Replacement Algorithms

• LRU (Least Recently Used)
• Keep track of the last time each block was
  accessed
• Select the block that has been used least
  recently to be the block that gets removed
• Requires system to keep history of accesses for
  every cache block which requires significant
  space and slows down operation of cache
• FIFO (First-in, First out)
• Block that has been in cache the longest
  (regardless of how recently it has been used)
  would be selected as block to be removed from
  cache memory

15
Measuring Performance

• How do we measure performance of two-level
  hierarchical memory?
• By its Effective Access Time (EAT), or average
  time per access
• EAT H x AccessC (1 H) x AccessMM
• H cache hit rate AccessMM main memory access
  time
• AccessC cache access time

16
Example

• Lets say that
• AccessC 10ns
• AccessMM 200ns
• H 99
• EAT H x AccessC (1 H) x AccessMM
• EAT 0.99 x 10ns (0.01) x 200ns 12ns

17
Cache Writing Policies

• Designers have to determine what to do with
  blocks that have been modified
• Cache write policy determines when actual main
  memory block is updated to match cache block
• Write-Through
• Updates both cache and main memory simultaneously
  on every write
• Write-back
• Only updates blocks in main memory when cache
  block is selected to be removed from cache

18
Data and Instruction Caches

• Data Cache
• Instruction Cache
• Advantages of separating
• Unified cache has only one port for data and
  instructions, resulting in conflicts between the
  two
• Allows accesses to be less random and more
  clustered

19
Levels of Cache

• Multilevel Cache Memory
• Caches using caches for increased performance
• Level 1 (L1) cache is term used for cache
  resident on chip and is fastest, smallest cache
• L1 is always checked first, but if the data is
  not found in L1, L2 cache is checked
• L3 cache refers to extra cache between processor
  and memory on processors

20
References

• http//en.wikipedia.org/wiki/Locality_of_reference
  
• http//images.google.com/url?sourceimgresctref
  qhttp//www.viewpoints.com/HON-210-Series-28-1-2-
  Deep-Vertical-File-Cabinet-review-a897usgAFQjCNF
  ogrb2UbcND7bTPbDviC8DvlTpeQ
• http//i.ehow.com/images/GlobalPhoto/Articles/4799
  688/IMGP1527edited-1-main_Full.jpg
• www.leoncountyfl.gov/recycling/
• www.calgary-city-maps.com/Calgary-phone-book.html
• tjliu.myweb.hinet.net/COA_CH_6.htm
• http//www.kidscreen.com/planetpreschool/wp-conten
  t/uploads/2009/08/2_mel-at-messy-desk.jpg
• http//images.google.com/url?sourceimgresctref
  qhttp//www.novopc.com/2008/09/ram-random-access-
  memory/usgAFQjCNF9SaCi-7adm2Vu2fdpZF-hNVu4sw
• Null, L. Lobur, J. The Essentials of Computer
  Organization and Architecture. 2006 Sudbury.