E C E N

4 2 4 3

C o m p u t e r

A r c h i t e c t u r e

Pipeline Hazards
The ideal pipeline executes instructions during each clock cycle so that the pipeline hardware is always doing something useful. Real pipelines with real instructions can encounter situations where an instruction would not execute correctly because of problems with other instructions already in the pipeline. These situations are called hazards. 1. Structural hazards - When hardware units are being used by instructions already in the pipeline, these units will not be available for use by other instructions. Any situation where there is not enough hardware is a structural hazard. 2. Data hazards - Data hazards can occur when two instructions use data from the same register. Data hazards are divided into three categories. a. RAW (read after write) hazards - the current instruction must wait to read data until after a previous instruction writes the correct data. b. WAR (write after read) hazards - the current instruction must wait to write data until after a previous instruction reads the old data. c. WAW (write after write) hazards - the current instruction must wait to write data until after a previous instruction writes to the same register. This hazard is more subtle in that neither instruction executes incorrectly. However, subsequent instructions can be incorrect if the writes occur out of order. Note that RAR (read after read) is not really a hazard because it makes no difference which order the same operand is read. 3. Control hazards - In the ideal pipeline, we fetch instructions one after another in order. As long as location of the next instruction is known, this process can go forward. When a branch instruction is fetched, the next instruction location is not known until the branch instruction finishes execution. Thus, we may have to wait until the correct location of the next instruction is known before fetching more instructions. This is a control hazard. As we shall see, with clever design, we can handle many hazards without slowing the processor down. Unfortunately, this greatly increases the complexity (and expense) of the processor hardware. It may be quicker and cheaper to handle the hazards by stalling the processor. A stall occurs when the hardware does nothing useful while waiting for a hazard to go away. Stalls can be implemented by inserting NOPs (no operation codes) into the instruction stream. This can be done statically by the optimizing compiler or dynamically by the processor hardware. Pipeline Performance with Stalls. If an instruction causes a pipeline stall, then it takes additional clock pulses for that instruction,

CPI i = ICPI i + SPI i

Pipeline Hazards February 20, 2008 page 1 of 4

E C E N

4 2 4 3

C o m p u t e r

A r c h i t e c t u r e

where ICPI is the ideal CPI when there are no stalls and SPI is the stall cycles per instruction. For a single pipeline (we will discuss multiple parallel pipelines next semester), the ICPI is 1 for all instructions. SPIi can be different for each instruction depending on how many stall cycles an instruction causes and also on how frequently stalls occur. The frequency of stalls depends on whether hazards occur with other instructions in the pipeline. What other instructions are in the pipeline depends on what program is being run. Unfortunately, this makes SPI program dependent as well as instruction dependent. The impact of stalls on the average CPI can be found as follows.

CPI piped = =

( IC i ) ------------ CPI i, piped ( IC ) ( IC i ) ------------ ( ICPI i + SPI i ) ( IC )

= 1+

i This is a very useful relationship which can be used to evaluate pipeline performance. Stalls reduce the performance benefit of pipelining.

( IC i ) ------------ ( SPI i ) ( IC )

CPUtime unpiped Speedup pipelining = ---------------------------------------CPUtime piped ( IC ) ( CPI unpiped ) ( T unpiped ) = ------------------------------------------------------------------( IC ) ( CPI piped ) ( T piped ) ( CPI unpiped ) (T ) - unpiped = ------------------------------------------------------ -----------------------( T piped ) ( IC i IC ) ( SPI i ) 1+
i In the ideal case with no stalls, the denominator of the first fraction evaluates to 1. If we compare the ideal pipelined design with the single cycle unpipelined design, then CPIunpiped is also 1. In the ideal case,

( T unpiped ) Speedup ideal pipelining = -----------------------( T piped )

Pipeline Hazards February 20, 2008 page 2 of 4

E C E N

4 2 4 3

C o m p u t e r

A r c h i t e c t u r e

and in the general case

Speedup ideal pipelining Speedup pipelining = ------------------------------------------------------1+ ( IC i IC ) ( SPI i )

Note: 1. Pipeline speedup depends on the program being run since SPI depends on the program being run. 2. All other pipeline performance measures, for example CPI and throughput, also depend on the program being run for the same reason. Structural Hazards Structural hazards can occur when two instructions might need the same hardware at the same time. In a general purpose machine, it is very expensive to provide all of the hardware all of the time that instructions might need. There is usually a trade-off between providing extra hardware to prevent structural hazards and a decrease in usage of the extra hardware. In the MIPS pipelined architecture (fig. 6.11, p. 388), having a separate instruction memory and data memory does prevent structural hazards for load/store instructions. However, the data memory interface is not used very often; only load/store instructions use it (about 30% usage in the benchmarks). Suppose the designers of an alternative MIPS decide that it is too expensive to provide a separate data memory interface that is not used very often. Instead a single memory interface for both instructions and data is used. In this case, a structural hazard occurs every clock cycle that a load/store instruction is reading/writing memory at the same time another instruction is trying to start instruction fetch. The only way the alternative MIPS can avoid the structural hazard is to delay fetching another instruction (stall) when memory is being used by the data transfer for a load/store instruction. We will discuss how to implement stalls in more detail later. Lets compare the performance of the standard MIPS (separate instruction and data memory) with the alternative MIPS (single memory). Lets assume that the clock rates are the same. Lets assume further that there are no stalls except those caused by the structural hazard (this is not true for either version as we will see in the next section). Since there are

Pipeline Hazards

February 20, 2008

page 3 of 4

E C E N

4 2 4 3

C o m p u t e r

A r c h i t e c t u r e

no stalls for the structural hazard in the standard MIPS, SPI = 0.

CPI standard = 1 + = 1

( ICi IC ) ( 0 )
i

Speedup ideal pipelining Speedup standard = ------------------------------------------------------CPI standard = Speedup ideal pipelining
In the alternative MIPS, every load has SPIi = 1 (even though the load instruction itself does not get stalled, it is the one that causes the stall), every store has SPIi = 1 (a store causes a stall too because memory cant do a read and write at the same time), and all other instructions have SPI = 0.

CPI alternative = 1 +

( ICi IC ) ( SPIi )
i

= 1 + ( IC load IC ) 1 + ( IC store IC ) 1 Speedup ideal pipelining Speedup alternative = ------------------------------------------------------CPI alternative

Typical MIPS instruction frequency data is

0.21, gcc(integer) ( IC load IC ) = 0.31, spice(floating point) 0.12, gcc(integer) ( IC store IC ) = 0.11, spice(floating point)
therefore,

1.33, gcc(integer) CPI alternative = 1.42, spice(floating point) Speedup ideal pipelining -------------------------------------------------------, gcc(integer) 1.33 Speedup alternative = Speedup ideal pipelining -------------------------------------------------------, spice(floating point) 1.42

Pipeline Hazards

February 20, 2008

page 4 of 4