Mechanism

For Improving
System
Performance
 Mechanism For Improving System
Performance

Cache
Mechanism used to buffer frequently used
data from external memory into fast speed
memory.
Pipelining
Mechanism used to overlap the execution of
program statements in order to reduce overall
program execution time.

2
 Cache
Technology
 Cache Technology Topics

• Rationale for Cache

• Cache Terminologies/Performance
• Cache Design Considerations
• Cache Mapping
• Cache Replacement/Update Policy

4
 Why Cache?

Analysis of a large number programs have found

that references to the computer's memory tends to
focus on a certain area.

This phenomena is known as the property of locality

of reference.

The argument for a cache, with respect to this

phenomena, is to place these commonly located
data into a high speed memory (cache), which can
improve the system's overall access time.

5
 Computer With Cache

Data Bus

Cache
Main
Processor Memory
Memory

Address Bus

6
 Cache Terminologies

Hit
A condition whereby when an access is made to
obtain a piece of data from memory, the piece of
data is found in the cache.
Miss
A condition whereby when an access is made to
obtain a piece of data from memory, the piece of
data is not found in the cache.

7
 Cache Performance

Hit Ratio
The ratio (percentage) in which access to a
piece of data, is successfully found in the
cache.
The effectiveness of a cache can be measured
by looking at the hit ratio and miss ratio, that is,
hit = 1 - miss

8
 Can Cache Improve A System's Performance?

Take The Following Access Time Of A Computer

System,

Tma = 500ns

Based on the above, the overall access time (Tacc )

to the memory is,

Tacc = Tma = 500ns

9
 Can Cache Improve A System's Performance?

If the system has a cache with an access time of

50ns (Tca ) and a hit ratio (H) of 90%, the overall
access time (Tacc ) is calculate as,

Tacc = (Tca X H) + (Tma X (1- H))

Tacc = (Tca X 0.90) + (Tma X (1-0.9))

Tacc = (50 X 0.90) + (500 X 0.10)

Tacc = (45) + (50)

Tacc = 95ns 10
 Cache Design Consideration

Implementing a cache requires due consideration

and tradeoffs between the following:-
• The Hit Ratio
• Access Time on a Hit
• Delay Time on a Miss
• Cost

11
 Question 1 - Cache Performance

You are given that a computer system has a cache

with an access time of 15ns and the main memory
access time is 70ns.

If the hit ratio of the cache system is 75%, what is the

average memory access time of the system?

12
 Question 2 - Cache Performance

If a computer system has a cache which has an

access time of 12ns and the access time of the main
memory is 70ns, what would the hit ratio be if the
overall memory access time of the system is 30ns?

13
 Question 3 - Cache Performance

You are given that a computer has an overall memory

access time of 40ns.

If the system has an access time of 90ns for the main

memory and the hit ratio of 85%, what would the
access time of the cache be?

14
 Question 4 - Cache Performance

If a system has an overall memory access time of

25ns and 10ns as access time for the cache, what
would the access time of the main memory be if the hit
ratio of the cache is 70%?

15
 Types of Cache Mapping

Since the cache is always smaller than main

memory, there must be some way which main
memory can be mapped onto the smaller cache.
The three general methods are:-
• Direct Mapping

Main Memory

Cache
• Full Associative Mapping
Mapping
• Set Associative Mapping

16
 Cache : Direct Mapping

The basis for accessing a computer's memory is

through the address bus.
Therefore, the address bus width determines the
amount of accessible memory.
Example:-
If a system has an address bus width of 15 bits,
the total amount of addressable locations is:-
n = 15
n
2 = locations
215 = 32768
17
 Cache : Direct Mapping

0
Example of how the main
memory is partitioned based
on the address bus width.

CPU MEMORY

32767
18
 Cache : Direct Mapping

Some terms used:-

n = Address bus width of main memory
k = Address bus width of cache memory
2n = Number of words in main memory
2k = Number of words in cache memory
(2n)-1 = Address of the last entry/location in the
main memory
(2k)-1 = Address of the last entry/location in the
cache memory
2(n-k) = Number of TAGS (or SEGMENTS) in the
main memory
TFS =n-k 19
 Cache : Direct Mapping

In order for the main memory to be mapped onto

the cache, the address field has to be divided into
two parts, the tag field and the index field.

15 bits
TAG INDEX

The address field refers to the memory address of

a specific location within the computer's main
memory.
20
 Cache : Direct Mapping

TAG The part of the address that indicates

which particular segment of the main
memory the data is coming from.

INDEX The part of the address that indicates

which particular offset location within the
particular segment of the main memory,
the data is coming from.

21
 Cache : Direct Mapping

To generalise, for any given memory variable, the

memory address is formatted as follows:-

n
TAG INDEX
TFS k

22
 Cache : Direct Mapping

Assume the following:-

A system has an address bus width of 15 bits and a cache

with an address bus width of 9 bits. Also assume the
system has a data bus of width of 12 bits.

This would mean:-

n = 15 15 bits
k=9 TAG INDEX
Tag Field Size = n - k 6 bits 9 bits
=6

23
 Cache : Direct Mapping

Example
0 0 Cache

512 X 12
Main Memory
Address = 9 bits
32K X 12 511 Data = 12 bits
Address = 15 bits
Data = 12 bits

32767

24
 Cache : Direct Mapping

Main Memory Cache

0 0

Tag 0
511 511
0 6 Bits 12 Bits

Tag 1
511

Tag Data is
0 Field stored
here
Tag 63
511
12 Bits
25
 Cache : Direct Mapping

Question.

If the concerned system has a piece of data (as

an example, a numerical value of 45) that is
located at memory address 291 and is frequently
accessed, how would it look like in the cache?

26
 Address (Tag & Index)

Decimal Form
(Address) Binary Form (Address)
291 000000100100011
Tag Field Index Field

27
 Address (Tag & Index)

Decimal Form
(Address) Binary Form (Address)
291 000000100100011
Tag Field Index Field
000000 100100011

28
 Address (Tag & Index)

Tag Field 000000 => 0 (TAG location)

Index Field 100100011 => 291 (Index Position)

29
 Address (Tag & Index)

In summary, a variable located in memory

address 291 can be found at TAG 0, index
position 291 within the computer's main
memory.

30
 Address (Tag & Index)

To calculate the TAG and INDEX, the following

formula can be used:-

TAG = memory address DIV 2k

INDEX = memory address MOD 2k

The DIV represents an integer division and the

MOD represents a division which returns the
remainder/balance (after the division).

31
 Cache : Direct Mapping

Main Memory Cache

0 0

Tag 0
45 291 0 45 291
511 511
0 6 Bits 12 Bits

Tag 1
511

Memory Index
Address Value
0

Tag 63
511
12 Bits
32
 Full Associative Mapping

The full associative mapping method is a method

that does not require extensive calculations.

This is due to the fact that the cache stores the full
memory address of frequently accessed data.

As such, the TFS (tag field size) within the cache

takes on the same size as the address bus width
of the main memory, where

TFS = n
33
 Full Associative Mapping

Example:-
If two memory locations are frequently
accessed, that is, memory address 291 (with
content 12) and memory address 803 (with
content 65), what would the computer's
memory and cache look like?

34
 Full Associative Mapping

Main Memory Cache

291 12 291 12

803 65 803 65

15 Bits 12 Bits

Memory Full address is Data is stored

Address stored here here
12 Bits
Location
35
 Set Associative Mapping

For set associative mapping method, the

calculations involved are identical to that of
the direct mapping method.
The only difference between the two methods
is that the set associative mapping method
would have a cache that effectively doubles
the cache size (this depends on the number
of ways/dimensions of the cache).

36
 Set Associative Mapping

Example:-
If two memory locations are frequently
accessed, that is, memory address 291 (with
content 12) and memory address 803 (with
content 65), what would the computer's
memory and cache look like?

37
 Address (Tag & Index)

Decimal Binary
Form Form
(Address) (Address)

291 000000100100011
Tag Field Index Field

803 000001100100011
Tag Field Index Field
38
 Set Associative Mapping

Main Memory Cache

0 0

Tag 0
12 291 0 12 1 65 291
511 511
0 6 Bits 12 Bits 6 Bits 12 Bits

Tag 1 65 803
511

Index
Memory
Value
0 Address

Tag 63
2-Way Set Associative Cache
511
12 Bits
39
 Set Associative Mapping

Main Memory Cache

291 12 0 12 1 65

803 65

Tag Value is Data is stored

stored here here

2-Way Set Associative Cache

40
 Sample Question
You are given that a computer has a cache
system (utilising the direct mapping technique)
which has 9 bits in terms of its address bus
width.
The system also has an address bus width of 16
bits for its main memory. If the system allows 12
bits data to be stored in both the main memory
and the cache, what would the system look like?
Provide an appropriate illustration which can
clearly depict this system.
41
 Sample Workings

Some terms that can be identified................

n = address bus width of the main memory
2n = number of words in main memory
k = address bus width of the cache
2k = number of words in cache
12 = number of bits per word (main memory
& cache)

42
 Sample Workings

Based on the details provided, we know

that................

n = 16
k = 9

43
 Sample Workings

Since we know the address bus width of main

memory, we can also calculate the total entries
within the main memory.............

= 216
= 65536

44
 Sample Workings

Since we know the address bus width of the

cache, we can also calculate the total entries
within the cache.............

= 29
= 512

45
 Sample Workings

To calculate the total number of TAGS within the

main memory................

2n
TAGS = k
2
216
TAGS = 9
2
65536
TAGS =
512
TAGS = 128

46
 Sample Workings

To calculate the Tag Field Size (TFS) of the

cache................

TFS = n-k
TFS = 16 - 9

TFS = 7

47
 Sample Workings

Main Memory Cache

0 0

Tag 0
511 511
0 7 Bits 12 Bits

Tag 1
511
Data is
Tag Field Stored
here

0
What the system would
Tag 127
look like.................
511
12 Bits
48
 Question 1 - Cache Mapping

You are given that a computer system has a direct mapped

cache system. The main memory has an address bus
width of 32 bits. The data kept in both the main memory
and the cache are 16 bits in length.
If the cache has a limit of 512 entries (that is, addressable
locations) what is the tag field size?
Draw out a diagram which can clearly depict the
organisation of this cache system.
Your illustration must include references to the way the
main memory is divided into segments/tags/portions and
the cache must show the number of addressable locations.
49
 Question 2 - Cache Mapping

You are given that a computer system employs a

full associative cache mapping system and that
the address bus width of the main memory is 20
bits.
Given that the data stored is 16 bits in length and
that the cache has a maximum of 256 entries,
draw out an appropriate illustration which can
clearly depict this cache organisation.

50
 Question 3 - Cache Mapping

You are given that a machine has 16 bits in terms

of its address bus width (main memory). The data
stored within the system is 16 bits in length.
If the cache system employed is the direct mapping
method and the cache has 28 words of storage,
what is the total number of entries in the cache
(addressable locations)?
Provide an illustration which can clearly represent
the cache system outlined above.

51
 Question 4 - Cache Mapping

You are given that a machine has a main memory,

capable of storing 16 bit data.
The machine also has an address bus width of 32
bits in length (main memory).
If the cache system has a tag field size of 8 bits and
that the cache system employs a two-way set-
associative mapping technique, provide an
appropriate illustration which can clearly represent
the above cache system.

52
 Question 5 - Cache Mapping
A machine has an address bus width of 16 bits for its main
memory. You have been given that the number of entries within
the cache is 256 entries.
What is the tag field size?
If the cache system were to employ a four-way set-associative
mapping system, what would the system look like?
What is the size (in terms of bits) of a single entry within the cache
if the system stores its data 16 bits in both the main memory and
the cache?
What is the total size of the cache (in terms of both bits and
bytes)?
What is the size of the cache (in terms of both bits and bytes)
from the perspective of the data only?

53
 Question 6 - Cache Mapping
You are given that a computer's main memory has an
address bus width of 64 bits. The cache system which
accompanies this computer has an address bus width
of 16 bits.
The cache system that is utilised within this computer is
the direct mapped technique.
If both the main memory and the cache can allow data
of 16 bits to be stored per entry, what is the tag field
size?
Provide an illustration which can clearly depict this
cache system.
54
 Question 7 - Cache Mapping

You are given that a computer system has a main

memory which has a total of 16777216 addressable
locations.

The cache found in the machine has a total of 1024

addressable locations.

If the data stored within both the main memory and

the cache is 16 bits and that the cache system
employed is the full associative mapping technique,
what would the cache system look like?
55
 Question 8 - Cache Mapping

You are given that a system employs a four-way

set-associative cache mapping technique for
managing the cache.
The main memory of this system has a total of
1048576 addressable locations. If the tag field size
is 8 bits and that the data stored within the main
memory and cache is 16 bits, what is the address
bus width of the cache?
Draft out an appropriate illustration which can
clearly depict the above-mentioned system.
56
 Question 9 - Cache Mapping
You are given that a computer system has a total of
1024 number of entries within a single TAG. The TAG is
located within the computer's main memory. The data
bus width is given as 16bits.
If the address bus width of this computer's main
memory is 20bits and that the cache system utilises a
two-way set associative cache mapping method, what
is the address bus width of the cache?
What is the tag field size of this cache?
Draft out an illustration to depict this cache mapping
system.
57
 Question 10 - Cache Mapping
You are given that a computer system has a total of
134217728 bytes (128MB) of main memory. The main
memory and cache allows you to store data in two
bytes per storage location.
If the address bus width of the cache is 10bits and the
cache method employed is a four-way set associative
cache system, what is the address bus width of the
main memory?
What is the tag field size of this cache?
Draft out an illustration to depict this cache mapping
system.
58
 Question 11 - Cache Mapping
You are given that a computer system has an address
bus width of 24bits and employs a full associative
cache mapping system. Data stored within this system
is 32bits in size. The address bus width of the cache is
12bits.
If a variable is frequently accessed and is stored in
memory location 6189 and has a value/content of 60,
where does it go?
Draft out an illustration to depict this cache mapping
system, indicating the variable and its location within
the main memory and the cache.

59
 Question 12 - Cache Mapping
You are given that a computer system has a total of
67108864 bytes (64MB) of main memory. If each memory
location stores 16bits worth of data and the total number of
TAGs within the main memory is 1024, what is the address
bus width of both the main memory and the cache? What
is the tag field size of the cache? You are also given that
two variables are frequently accessed, memory location
170169 (value 32) and memory location 268473 (value
88).
If the mapping system employed is a 3-way set associative
mapping method, draft out an illustration to depict this
cache mapping system, indicating the variables and its
location within the main memory and the cache.
60
 Question 13 - Cache Mapping
You are given that a computer system has a total of
1024 TAGs within the computer's main memory.
The cache mapping method employed is a 4-way
set associative cache mapping technique.
If the computer's main memory and cache have a
data bus width of 16bits and that the main
memory's total capacity is 1048576 bytes, what is
the address bus width of the cache?
Draft out an appropriate and complete illustration to
depict this cache mapping system.

61
 Question 14 - Cache Mapping

You are given that a computer system has a 2-way

set associative cache system with a total memory
capacity of 1048576 bytes (only data).
If the data bus width of this computer is 16bits and
that the computer's main memory is 256 times
larger than the cache (based on the data only),
what would this cache mapping system look like?
Draft out an appropriate and complete illustration to
depict this cache mapping system.

62