IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. COM-29, NO.

9, SEPTEMBER 1981 1399

Correspondence

Memory Management in a Viterbi Decoder VITERBI DECODER BASICS

The basic theory behind theViterbi decodingalgorithm
CHARLES M. RADER is widely available in theliterature. This sectionrepeatsit
in simplified form as a convenient way of introducing an
Absrracr-Management of the memory contentsin a Viterhi decoder unambiguous notation and terminology.
is amajordesignproblemforbothhardwareandsoftwarereali- A convolutional encoder has the simple structure of Fig. 1.
zations. In a naive implementation, every hit in the memory must be Theinput message symbolsmiare M-ary and theencoded
changed (read,modified, andrewritten)foreachmessage bit de- data ci is a function of the present symbol mi and k previous
coded,and, inaddition,somedoublebuffering is required. An symbols which are stored in a shift register. After ci is deter-
especially annoying featureis the needto read and rewrite long words,
mined, the shift register is clocked and its state changes from
fortybits in atypicalcase. In this notewe describe a memory
organization which overcomes these problems. (mi-1, mi-2, ... m i - k ) t o (mi, mi-1, *..,mi-k+1).
The techniques described here are not novel, hut neither are they At the decoder, a table (memory) is maintained with two
widely known. entries (subwords) for each of M k hypotheses. Each hypothe-
sis corresponds to a particular sequence of message symbols
up to and including the ith; for each of theMk possible states
of the encoding shift register there is one most likely decoder
INTRODUCTION hypothesis for which that state (equivalent to the last k digits
Data which has been convolutionally encoded for transmis- of the message) could exist. One entry in the table is the path
sion over a noisy channel is often decoded using the Viterbi metric, which indicates the likelihood of the hypothesis. The
algorithm, which, for a given code and a memoryless channel, other entry represents the earlier symbols, already shifted out
can acheive the optimum (lowest) possible error rate if suf- of the encoder, which led to the presumed encoder state. This
ficient memory is provided and sufficient decoding delay ac- is called the survivor path. Since the survivor path subword
cepted.Thememory used inaViterbidecoder storestwo would grow indefinitelylong as time proceeds, it must be
kinds of information related t o hypotheses about what data periodically truncated and the oldest symbols discarded. Those
was encoded. One kind of information records which hypothe- discarded symbols tend to agree and t o match the truemessage-
ses are actually being considered-as the algorithm is run the they are the output of the Viterbi decoder.
number of potential hypotheses steadily increases but those no The rule for updating the path metrics is straightforwLard.
longer being consideredare constantlypruned;the memory Consider hypothesis ii+l (a vector), whose last k symbols are
must keep a record of which hypotheses have survived. The
other kind of informationstored is aset of scores (called (it+ 1, ii, it- 1, ii- k+2).
path metrics) for the hypotheses still under consideration. As
the algorithm proceeds, the path metrics of surviving hypothe- Thehypotheses which couldlead t o are thosefor which
ses are derived from the path metrics of the hypotheses from the ith message digit was m i = ji+1, and for which the shift
which the survivors are descended, by adding appropriate register contained (ji, ji- 1, -.*j i - k + 2 , x). Here x could be any
transitionmetrics to the path metrics. In a naive implemen- of M symbol values so there are M possible precursor hypothe-
tation of a .Viterbi decoder every bit in the memory must be ses. For each, the path metricis looked upin the table and is in-
changed (read, modified, and rewritten) for each message bit cremented by a measure of unlikelihood (the transition
decoded and, in addition, some double buffering is required. metric)-this is based on the difference between what is seen
The management of the memory contents is the major design on the channel and what would be on the channel with the
problem in a Viterhi decoder implementation. This is the case input ji+ 1 and the shift register contents (ji, ji- 1, --,j i - k + 2 , X).
whether the decoder is implemented in special purpose hard- TheLwest resulting sum is the new path metric for hypothe-
ware, or insoftware. An especially annoyingfeature of the sis ji+l, and the old symbols in the survivor path of the win-
naive implementation is the need t o read andrewritelong ning ancestor, with x appended, form its new survivor path.
words, forty bits in a typical case. In this note, we describe a The table, containing a survivor path anda path metric for
memoryorganization which overcomestheseproblems.
though the size of the memoryneeded is not reduced,this
Al- each of the M k possible hypotheses z, is updated once for
each received channel measurement-this is called a decoding
organizationeliminates the need fordouble buffers andfqr cycle. To “prune” the survivor path, the hypothesis with the
reading and rewriting long words. lowest path metric is identifiedand its oldsymbolsare the
decoder output.’The oldestsymbols may then be dropped
from all survivor paths. A symbol may be pruned from the
Paperapprovedby the Editor for Communication Theory of the path memory once each decodingcycle, or p symbols may
IEEE Communications Society for publication without oral presenta- be pruned away after every p decoding cycles.
tion.ManuscriptreceivedJanuary 30, 1981; revisedApril 3, 1981.
This work was sponsored by the Departmentof the Air Force. The U.S.
Government assumes no responsibility for the information presented. Thedecoder output may use the droppedsymbolsfromsome
TheauthoriswiththeMassachusetts Institute of Technology, othersurvivor path, givingdecreasedperformance but a simpler in-
Lincoln Laboratory, Lexington, MA 02173. plementation.

0 1981 IEEE
0090-6778/81/0900-1399$00.75
1400 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. COM-29, NO. 9, SEPTEMBER 1981

m-'-p:m-bci
mi-, mi-2 mi-k

MAPPING

Fig. 1. A convolutionalencoder.
Fig. 2. Evolution of contents of path memory for in-placeorganiza-
MEMORY ORGANIZATION FOR PATH METRICS tion (rn = 2, k = 3) for three decoding cycles.
In a naive realization of the decoder,since path metrics
at time i 4- 1 are computed using path metrics at time i, it is M addresses used together. This tells us how to distribute the
necessary t o doublebufferthepath metricmemory. How- metrics in M physically separate memories.
ever, an attractive alternative is an in-place computation In summary,it is not necessary to use temporaryextra
similar to that which is often used to compute the discrete
Fourier transform. We observe that metrics for the M hypothe-
x
storage (beond Mwords) forthepath metricsassociated
with the M hypotheses. By cyclically shifting the natural ad-
ses ( j i , j i - l ,
**e, j i - k + 2 , x)-for M choices of x-are needed dresses before using the addresses t o access a memory, an in-
only to compute metrics for the M hypotheses ( y , j i , j i - l , e-*,
place computation follows easily.
ji-k+2)-for M choices of y . If the M metrics needed are read
from memory, then M memory locations become available to MEMORY ORGANIZATION FOR SURVIVOR PATHS
store the M newly computed metrics, and no double buffering For minimum error rate, the length of the survivor path
is required for metrics. memory field should be made as large as possible. There is a
There is a price t o be paid for this memory economy. It is rule of thumbthatfouror five constraintlengths is ade-
natural to treat the contentsof the shift register, ( a k , ak- 1, --, quate-for M ,= 2, the constraint length is k i- 1. A practical
a l ) , as a k-digit M-ary number and use this number t o index case has k = 6 , so a survivor path memory field of 35 bits is
the table of path metrics. Such an addressing scheme is incon- implied. It is inconvenient t o handle such a long field all at
sistent with writing new metrics over old metrics. Consider the once, although the operations needed are quitesimple.
example of M = 2, k = 3. The decoder has eight hypotheses The need for a long field does not imply the need to handle
endingin000=0;001=1;010=2;~~~;lll=7.Ournatural many bits at once. The operations involved in computing the
order would store stage i metrics in table locations 0 through survivor path field forhypothesis consistof two parts, the
7. But the two successors to, say, 000 and 001 are 000 and designation of an ancestor and the appending of a symbol t o
100. This means we read metrics from0 and 1 and write them- its path. To avoid handling the entire field we could adopt an
by definition of natural order-in 0 and 4. This is not in-place. approachfrequently used in computer science, the use of
To put the metrics in-place, suppose they were in nature1 pointers. Forhypothesis we would storethe new M-ary
order t o begin with. After one, two, and three stages of de- symbol appended to its path, and a pointer to a path for stage
coding we wouldsee the evolution of memory organization i. This stage i path would similarly consist of one M-ary sym-
for path metrics illustrated in Fig. 2. bol and a pointer t o a path at state i - 1. Organization of the
We see that computing metrics in-place leads, inexorably, memory in this way is especially desirable since it defers the
to an addressing scheme which changes after eachdecoding need t o actually read thepathmemoryuntilthe decoding
cycle. If the path metric of the hypothesis with shift register (path memory pruning)-at that time only the path leading to
contents (a, b , c) at time i is in memory location 4a -t- 2b i- c, the decoded symbols must actually be read at all. Because the
then the path metric of the hypothesis with shift register con- path memory does not have to be read during the updating
tents (a, b , c) at time i i- 1 will be in location 412 i- 2a i- b. process, the issue of in-place computation does not arise and
In general, the metrics accessed together are found by gen- therefore it is not necessary t o use an unnatural address to
erating their natural addresses but rotating the bits of these store new information into the path memory.
addresses by i places before reading (or writing) the metrics At first examination, the use of pointers seemscostly of
from (or into) the memory. The natural addresses can be gen- memory. Actually, since each pointer canonly pointto M
erated from a counter, for example. A cyclic shift of i places ancestors, the pointer can be abbreviated t o an M-ary symbol.
is identical to a cyclic shift of i modulo k places. This M-ary symbol is identical to the M-ary symbol which is
If, for extra speed, it is desired to use some parallelism in appended to the path. Therefore no extra storage is needed for
metric computing hardware, one can read, simultaneously, the the pointers-it is only necessary to interpret the path memory
M metrics used together, compute their M successors in paral- contents as pointers. During the decoding cycle an M-ary
lel, and write the results in memory simultaneously. This re- choice is recorded for each hypothesis in the ith digit position
quires, of course, that metricsneededsimultaneouslyarein of the survivor path field for each of the Mksurviving hypothe-
physically separate memories. To assure this, let the modulo sis. Suppose, for the hypothesis with shift register contents (ai,
Msum of the M-ary digits of an address be considered-for ai- 1 , .-, a i - k + l ) the symbol stored is x . Then, t o find its
M = 2 this is the parity. It is clear that the parity of each of predecessor we look in digit position i - 1 of thememory
the pair of addresses used must be different since exactly one word whose address is (ai- 1, -e, x ) ; if we read
a y
bit differs between them. Thus, one metric will always be read there, we look in digit position i - 2 of the word whose ad-
from an address with even parity and one metric will always dress is ( a i - 2 , **., a i - k + l , x , y ) ; finding a z , we continue the
be read from an address with odd parity. The same distinction process. This represents a path x , y , z , e-..

applies in writing new metrics back. In the M-ary case, each of In Fig. 3, we give an an example, with M = 2, k = 3, of how
the M possible modulo M digit sums will occur for one of the the contents of the path memory (1 bit abbreviated pointers)
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. COM-29, NO. 9, SEPTEMBER 1981
ADDRESS I DIGIT 8 7 6 5 4 3 2 1
000
001
010
01 1
100
101
110
111
Fig. 3. Interpretation of pointer-organized survivor path memory con-
tents (rn = 2, k = 3).
indicatethe paths.Note that whenever two survivor paths
agree on k successive pointers they must necessarily converge.
We need t o trace back such a path to prune and decode. If
the path memory field is L M-ary symbols wide, we might de-
code after p decoding cycles, obtaining p decodedsymbols,
then overwrite new path symbols into the newly freed digit
positions onthenext p decoding cycles. A new symbol
will be stored in digit position i mod L of the path during de-
coding cycle i. Achoice of p > l slightly increases the re-
quired path field lengthandgreatly decreases the computa-
tional cost of tracing paths backwards. Another consideration
is that the equipment towhich decoded data is transfered may
require four bit or eight bit transfers.
For M = 2, k = 6, a common practical design, the above
scheme writes one bit ineach of 64 words eachdecoding
cycle, whereas a naive approach would require a 35 bit read-
modify-write operation for each of the 64 words. Especially
for a CMOS memory, for which the power consumption de-
pends primarily on reading or writing information, as opposed
t o simply retaining it,our proposedschemereducespower
consumption 70: 1, ignoring overhead.
In summary, instead of storing entire paths ateach decoding
cycle, it is only necessary to store the symbols appended t o
the surviving paths. These symbols act as pointers t o ancestral
paths. The interpretation of the symbols as pointers removes
the necessity t o move data in the path memory.
DISCUSSION
Memory management is the primary task of the designer of
aViterbidecoder. A straightforward implementation of. the
Viterbi algorithm is relatively costly. Much of the published
literature treats the details of memory management sketchily.
Apaper by Forney [ 11 suggests the analogy with the Fast
Fourier Transform and implies essentially the memory manage-
ment we have described for the path metrics. This is also de-
scribed in a NASA 197 l report [ 21 in excellent detail. Private
discussion with leading experts in the field reveals thatthe
path memory management scheme we propose is known, but
not widely known. It does not appear to have been described
in the literature.
In design studies recently undertaken by the author and by
W. King, the techniques explained here, in-place computation
of scores, use of addressparity t o permit parallelism, path
memorymanagement by implicit pointers, anddecoding p
0090-6778/81/0900-1401$00.75
1401
symbols.at a time, have permitted substantial improvements
in ‘fh.e speed-power product of a Viterbi decoder when com-
pare& with an earlier straightforward design.
REFERENCES
G. IS. Forney, Jr., “The Viterbi algorithm,” Proc. IEEE, vol. 61,
pp. 218-278, Mar. 1973.
K. S . Gilhousen et a l . , “Coding systems study for high data rate
telemetry links,” NASA Rep., Jan. 1971. Prepared under contract
NAS2-6024 by Linkabit Corp., San Dieso. CA.
Sliding Block Decoding of Convolutional Codes
KOU-HU TZOU AND JAMES G. DUNHAM
Abstrucr-It is shown that a convolutional code canbe decoded with
a sliding-block decoder, a time-invariant nonlinear digital filterwith
finite memory and delay. If the memory and delay are small, then the
sliding-block decoder canbe implemented as atable lookup procedure
in ROM, resulting in a low cost,highspeed,andhighreliability
decoder.
I. INTRODUCTION
Convolutional codes with Viterbi decoding are widely used
on additive white Gaussian noise channels with binary phase-
shift keyed modulation. When decoding a block of informa-
tion by minimizing the block probability of error, the encoder
must insert additional bits t o initialize each block, the decoder
requiresblock synchronization, and thedecoder must wait
until the entire block has been received before it can begin.
For a long block the overhead for block synchronization is
low but the delay is long while for a short block the delay is
short but the overhead for block synchronization is high.
An approach t o avoiding theseproblems istointroduce
real-time constraints into thedecoding procedure. As discussed
by Viterbi [ 1 1, path memory truncation is an effective solu-
tion to the problem. For a truncation length or delay of D
bits, we decode the information bit on the branch at depth J
in the trellis which corresponds to the survivor with the best
metric value at depth J -I-D . Fomey [2] used random coding
arguments to show that, on theaverage, a truncation length of
about five constraint lengths will result in an additional error
probability which is comparable totheViterbi algorithm
errorprobability. Hemmati and Costello [3] developed .an
upper bound on the bit error probability and showed that the
truncation length should be chosen as a function of the code’s
free distance.
Lee [4] showed that aconvolutional code couldalso be
decoded under aminimumbit error probability criterion.
PaperapprovedbytheEditor for CommunicationTheoryofthe
IEEE Communications Society for publication without oral presenta-
tion. Manuscript received December 29, 1980; revised April 10, 1981.
This work was supported by the National Science Foundation under
Grant ENG-791001.
The authors arewiththeDepartment of ElectricalEngineering,
Washington University, St. Louis,MO 63130.
0 1981 IEEE