(An Alternate View of) Turing Machines

This handout was written by Maggie Johnson and Mehran Sahami

Introduction
Previously, you were introduced to the most powerful formal model of computation we know,
called a T uring M achine1. This model is used as a basic formalism for modern computers. A
study of Turing Machines will enable us to study the theoretical limitations on the computations
that computers can perform. While some of the material in this handout overlaps with the
chapters on "Turing Machines" and "Uncomputable Functions" that you read previously, it
provides an alternative (graphical) representation of Turing Machines, which can sometimes be
easier to understand. Thus, it might be an easier way of learning into this material as well as
discussing some extensions to the model.

In the prior introduction to Turing Machines, we discussed the input tape, which is used to
stores symbols that the Turing Machine manipulates. Each symbol on the input tape goes in a
location called a cell. The tape is infinitely long (since the input string can be potentially
infinitely long). In the presentation here, we use the symbol 'B' to indicate an empty cell (which
will be the contents of most of the cells on this infinite tape). The input tape actually acts as a
random access memory. The automata will be able to scan up and down this tape, examining and
potentially changing the elements in the cells scanned.

More formally, we can define a Turing Machine as follows:

A T uring M achine (TM) is a 6-tuple < Q , A T, d, q0, B, F > where:
1. Q is a finite set of states.
2. A T is a finite set of tape symbols. (Previously these are always 0 or 1.)
3. d is a transition function that maps a tape symbol and a state into a new state and an action.
An action consists of writing a new tape symbol and moving the tape head to the left (L) or
right (R), or staying in place (-).    (For  example,  the  transition  “a/b  R”  means  that  if  the 
machine sees an 'a', it writes a 'b' in its place and then moves the tape head one space to the
right.) If there is no mapping to a new state for the current combination of tape symbol and
state, then the machine halts. (Previously, the "stay in place" action (-) is not used.)
4. q0 (sometimes also referred to as Q0) is the start state. (This was always state 1 previously.)
5. B is a special symbol denoting a blank/empty cell. (This was not used previously.)
6. F is a finite set of final states. (This was state 0 previously.)

In this formulation of a Turing Machine, we will assume that any input for the TM to process has
been put on an input tape already. Thus, we can imagine a Turing machine as a robot that
                                                 
1 Turing Machines are named after Alan Turing (1912­1954) who was one of the most influential researchers in the 
history of Computer Science.  Turing, the  namesake for the prestigious Turing Award (the highest award given in 
Computer Science), made many fundamental developments in the theory of computing, including the invention of the 
theoretical  model  of  computation  that  now  bears  his  name,  as  well  as  what  has  come  to  be  known as the “Turing 
Test”  –  a  characterization  of  what  it  means  for  a  computer  to  have  reached  human­level  intelligence,  and  thus  be 
characterized  as  having  attained  Artificial  Intelligence.    Sadly,  Turing  died  before  his  time,  committing  suicide 
(although this point is debated by some) through ingesting cyanide at the age of 41.  Turing was voted one of the 
"Most Influential People of the 20th Century" in a Time magazine poll.  As far as we know, however, he never won 
of People magazine's prestigious "Best Dressed" award. 
    – 2 – 

shuttles back and forth across a tape, following whatever actions are dictated by its current state
and the tape symbol at which it is looking. We generally refer to this "robot" as the tape head of
the TM, which indicates which cell in the tape the TM is currently examining. We can visually
depict this notion of a TM as follows:

... B B a a a b b b c c c B B B ...
Tape
head

States
and
Transition
Table

Note, that in this depiction the input tape is infinite in both directions. Generally, this is how the
input tape for a TM is thought of. In fact, the computational power of TMs would remain the
same if we considered an input tape that was only infinite in one direction, but for the sake of
brevity, we do not provide a proof of this equivalence here.

Note that if the TM ever reaches a state in which there is not a valid transition rule specified for
the input that the tape head is currently examining, the machine immediately halts. In the
chapters you read previously, the machine halted when it was explicitly sent to state 0. In the
more general case, we simply say that the TM halts when there is no valid transition for it to take
out of a state.

Formal L anguages and T uring M achines

We can define a formal language (not to be confused with a natural language, like English) as 
simply  a  (possibly  infinite)  set  of  strings  that  are  the  "valid"  words  in  that  language.    For 
example,  we  might  have  the  formal  language  that  is  the  set  of  all  strings  comprised  of  the 
symbols 0s and 1s where there are equal numbers of 0's and 1's in the string. 
 
For  a  broad  range  of  formal  languages,  we  can  use  Turing  Machines  as  language  recognizers.  
That is, when given some string as input, the Turing Machine that determine whether that string 
should be considered part of a particular language or not. 
 
In  order  to  do  this,  we  need  to  expand  the  TM  formalism  in  a  few  ways.    First,  we  designate 
some  subset  of  the  states  in  a  TM  as  being  "accepting  states".    If  the  TM  halts  in  one  of  the 
accepting states when presented with a particular input, then that input string is considered to be 
part of the languages the TM is built to recognize.  Otherwise, if the TM halts in a non­accepting 
state, then we say that they string presented is not part of the language the TM recognizes.  Thus, 
the  set  of  all  strings  that  cause  the  TM  to  halt  in  an  accepting  state  is  called  the  language 
accepted or defined by the TM. 
 
The  example  below  illustrates  how  a  TM  can  be  used  to  recognize  or  accept  strings  of  a 
particular language.
    – 3 – 

E xample 1
Say we have a language L defined over the symbols 0 and 1 that includes only strings of the form
0n1n for all n > 0. That is, L is the set of all strings that start with some positive number of 0s
followed immediately by exactly the same number of 1s.

Consider the Turing Machine below that accepts the language L .

0/0 R
Y/Y R Y/Y L Y/Y R

Q0 0/X R Q1 1/Y L Q2 X/X R Q4

B/B R

X/X R Q3 0/0 L Q5

0/0 L

Here we represent the TM using a transition diagram, where:

 Each state is represented by a node (circle) in the graph. Any states that have double
rings are "accepting states" (such as Q5 in the diagram below), and all other (single ring)
states are non-accepting states.
 Transitions from one state to another are represented by directed arcs (arrows) labeled
with a transition of the form "/ ", where  is the symbol that the tape head is currently
on,  is the new symbol written in the cell the tape head is currently on and  represented
the direction, (L)eft, (R)ight, or (-)Unchanged, that the tape head moves after writing the
symbol .
 The TM always stares in states Q0.
 The TM halts in any state for which the input symbol under the tape head does not match
 in any of the transitions "/ " out of that state.

Now, say we are given the following input tape, where the tape head is always guaranteed to be
at the first non-blank symbol on the tape.

... B B 0 0 1 1 B B ...

The basic strategy for our TM is to start with the first 0, and replace it with an X. Then we move
right on the tape until we find the first 1, and replace it with a Y. We then move left back over
the 0s until we find the second 0, and change it to an X also. Then, we move to the right again
until we find the second 1, and change it to a Y, etc. If the number of 0s equals the number of 1s,
    – 4 – 

all we will have on the tape are Xs and Ys, which eventually puts us in state Q4. If there is an
unequal number of 0s and 1s, we will crash in the loop comprising states Q0 through Q3. Notice
that we have changed the input string in the course of processing it. This happens very
frequently in Turing Machines. It is a very simple task to change the tape back to its original
form by adding additional states to the TM that will take a pass over the tape and convert any Xs
to 0s and any Ys to 1s.

Note that if a string that is not part of the language 0n1n for all n > 0 is processed by the machine,
we will find ourselves in a non-final state that we cannot get out of. When we are in a state that
does not have a valid transition out for the next input symbol, the machine halts. Since the
machine will have halted in a non-final state, the input string is not accepted.

E xample 2
We will now show that a TM can be built to accept the language L = {anbncn | n >= 0}. We will
build the TM assuming that the input string has been written on the tape, and the tape head of the
TM starts out at the position of the first symbol in the string (which then extends to the right).
The basic strategy in the TM will be to make several passes up the tape from left to right. On
each pass, the first occurrence of each a, b, and c will be replaced with an X, Y, and Z,
respectively. After completing each pass, the machine will move leftwards until it reaches the
first X. At this point it starts the cycle again (this is denoted by the arc from Q3 to Q0). If after
completing this process, the tape contains only Xs, Ys, and Zs, we know that there was an equal
number of each symbol (a, b, and c) on the original tape. We check for this final property in
states Q4 and Q5.

X/X R 

Q0  a/X R  Q1  b/Y R  Q2  c/Z L  Q3 

B/B R  a/a L 
a/a R  b/b R 
Y/Y R  b/b L 
Y/Y R  Z/Z R 
Y/Y L 
Z/Z L 
Q4  Q5 
B/B R 
Y/Y R 
Z/Z R 

Note that this technique outlined above could be used, with slight modification, to handle the
problem of accepting the languages {anbncndn}, {anbncndnen}, etc.
    – 5 – 

T he Power of T uring M achines

As far as language recognizers go, Turing machine can recognize a larger set of languages than
any other formal model of computation we know. Consider also that once a TM recognizes a
language pattern, it can perform some action associated with the pattern. This is the essence of a
computer's actions.

To build a Turing machine, it is necessary to keep in mind all the possible things that can be done
using the tape. The tape head can be moved to any cell on the tape, as long as it can be located
from the present position. Any character can be written on a particular cell, or the contents of the
cell can be erased. So, the first step in building a TM is to design an algorithm using the tape and
read/write capabilities. Then, the algorithm is translated into a TM representation. For example,
the above TM uses X, Y and Z to keep track of the size of n.

T uring M achines and Computers

So this little mechanism hardly looks like a computer, yet we said previously that TMs and
computers are equivalent. We need to convince ourselves that TMs are indeed like computers.
One step is to show that TMs can compute functions, which you saw previously in the chapters
that you read. For example, we can put the argument to the function on the input tape
(potentially in unary notation where, for example, 3 = 111), run the machine, and the value of
f(x) is on the tape (again, perhaps in unary notation) after the machine halts. If your function
requires more than one argument, just delimit the sequences of 1s for each argument with an X or
some other symbol.

E xample 3
In the chapters you read previously, a TM was presented that computed the function f(n) = 2n.
Here is another representation of such a machine, using the graphical transition diagram notion
presented here. Note that while the TM described below computes the same function as the TM
presented in the previous readings, the two TMs use slightly different algorithms to do the same
computation! This is just a simple example of how much variation there can be in writing
computer programs to solve the same problem.

We assume the TM tape contains n in unary notation, with that tape head at the first 1. The TM
should halt with the value 2n on the tape in unary notation and the tape head at the start of the
resulting block of 1s.
1/X R 

Q0  1/X R  Q1  B/Y L  Q2  X/X R  Q3 

B/B ­  1/1 R  1/1 L  Y/Y R 

Y/Y R  Y/Y L 

Q6  B/B R  Q5  B/B L  Q4 

X/1 L  Y/Y R 
Y/1 L 
    – 6 – 

Note that the TM above uses Q6 as its halt state, since this machine does not have any transitions
out of that state. So, Q6 is a state that the TM will generally halt in when given a legal input.

Universal T uring M achines

The concept of a Universal Turing Machine further helps to show that TMs have the power to
compute anything that we can compute with a modern computer. We can define this notion more
formally as follows:
For every Turing Machine T , there is a string of symbols dT that describes the states and
transitions of T . There exists a Turing Machine U (Universal T uring M achine) such that, given
a unary number x written on the tape followed by a delimiter symbol and the symbols dT, if U is
started in state Q0 on the leftmost symbol of dT, then when U stops, f(x) will be on the tape,
where f(x) is the output of T .

A Universal Turing Machine (UTM) does exactly what a real modern computer does. Namely, a
UTM executes some algorithm (encoded in the Turing Machine T ) on an input (the number x),
and outputs the results. It is beyond the scope of this class to define how a UTM works, but you
can always take CS154 if you’re really interested.

C hurch's T hesis Revisited

So, just how powerful are Turing machines? Based on the notion of a Universal Turing
Machine, we might be inclined to ask if there are any algorithms we can write in some
programming language that cannot be translated into a Turing machine? The answer is "we don't
think so." In other words, Turing machines stand at the pinnacle of computational power (at least
for finite state systems, and since we can’t construct a real computer with infinite states, this isn’t 
really a germane restriction). We are now in fuzzy territory, but no one has yet come up with a
problem that can be solved algorithmically that cannot be solved by a Turing machine, and there
is considerable empirical evidence to suggest that no one ever will. The cause of the uncertainty
about what is computable lies in the uncertain definition of an algorithm. Computers, after all,
can only do what an algorithm tells them to do. We all have an intuitive notion of what an
algorithm is, but we do not have a firm formal definition. Programs written in C++, for example,
seem like a precise statement of an algorithm, but we would not want to say that the only thing
computers can do is follow C++ programs. Java programmers, among others, would be
somewhat disconcerted if this were true. Some of them might even get downright nasty (and we
all know what an unpleasant experience that can be).

E ffective Procedures
We will not attempt to formally define an algorithm here, but we will at least give it another
name (calling it an Effective Procedure) and try to characterize its behavior. We describe the
characteristics of an Effective Procedure below.
    – 7 – 

An E ffective Procedure is another name for an algorithm. There is no precise definition of an

effective procedure, but we expect it to have these properties:
1. Its description is finite.
2. It will execute in a finite amount of time.
3. It requires a finite amount of storage.
4. It executes in a completely mechanical fashion, without the benefit of insight, cleverness,
luck, or the intervention of supernatural beings (e.g., oracles).
5. It is deterministic (which is just another way of restating #4 above). In this context,
deterministic means that it always produces the same answer for a given input.

A truly amazing fact about Turing machines is that they seem to be capable of solving any
problem for which an effective procedure exists. This notion was stated by Alonzo Church in
what has since come to be known as Church’s Thesis.

Church's Thesis: 
There is an effective procedure to solve a problem if and only if there is a Turing machine that 
solves the problem. 

The statement above is called a thesis rather than a theorem because it is fundamentally
unprovable. (How would you go about proving it? Good luck trying to use induction.)
Unprovable  or  not,  Church’s  Thesis  is  generally  accepted  to  be  true.    In  fact, many computer
scientists define an effective procedure as something for which a Turing machine exists.

This is not as rash as it may seem, since virtually every programming environment used in the
real world is equivalent to a Turing machine (we say that they are T uring complete). This list
includes C, C++, Java, Pascal, Lisp and the usual suite of popular languages. The list also
includes  some  surprising  entries,  such  as  Karel  the  Robot.    Yes,  that’s  right,  Karel  the  Robot, 
who can only turn left, is Turing complete! The implications of this are impressive (and
somewhat staggering): since all of these languages are Turing complete, they are all
fundamentally equivalent. This means that any program written in a "powerful" language like
C++ could also be written with Karel the Robot (albeit with considerably more effort).
Additionally, some dedicated soul with a lot of extra time their hands could translate every single
program ever written into Turing machines. Of course, they would likely go insane during the
process, but that’s the price you sometimes have to pay in doing research.

A lternative T uring M achines

Given the preceding section, it seems pointless to add extensions to a Turing Machine (but you
are certainly welcome to try). The basic version of Turing Machines defined earlier has a single
two-way infinite tape that is one-dimensional. For convenience, however, Turing machines are
sometimes modeled with two or more tapes, or even with a multi-dimensional tape.
Interestingly, none of these features adds any power to the model. These enhancements just
make the representation of certain language patterns and, therefore algorithms, easier.
    – 8 – 

Undecidable Problems
Now that we have a model that is as powerful as a real computer, we can begin to reason about
the capabilities of computers. Are computers omnipotent, or are there problems that they just
cannot solve? In other words, if we really could build a Turing machine with as much tape as we
would ever need, and if we had a lot of time and patience, could it solve any problem we put to
it? Now we have a concrete methodology for thinking about these questions.

Before we try to answer the question, we will first restrict ourselves to a special class of problems
called decision problems. A decision problem is a problem with a yes or no answer. (This may
seem very restrictive, but in fact it is not.) With this notion in mind, we can now define what it
means for a problem to be "decidable."

A decision problem is decidable if it is possible to build a Turing Machine that will always halt
in a finite amount of time, producing a "yes" or "no" answer. A decision problem is undecidable
if a Turing Machine may run forever without producing an answer.

We may wonder whether or not there are any undecidable problems. The answer, sadly, is yes.

Consider the following practical application of decidability:

1. Suppose  we  write  a  program  to  solve  an  important  and  very  complex  problem.    After 
three weeks, it is still running and has not produced an answer yet.  Should we stop the 
run? 

2. In the case of testing a program, it would be nice for programmers to be able to see if they 
have  a  major  error,  such  as  an  infinite  loop,  in  their  program  by  having  their  program 
analyzed by another program. 
 
While  having  a  systematic  way  of  determining  if  a  program  halts  would  be  very  useful,  it  is 
unfortunately an undecidable problem.  In fact, this is the most (in)famous undecidable problem
known as the halting problem, which we define more formally below:

The Halting Problem 
It is impossible to build a Turing Machine that can predict if another arbitrary Turing Machine 
will halt for an arbitrary input tape.  In other words, the Halting Problem is undecidable. 

There is much more discussion of the Halting problem in the course reader, and now would
probably be a good time for you to read Chapter 23 in the reader, if you haven't done so already.
If the halting problem (or the Busy Beaver problem, also described in the course reader) were the
only undecidable problem in the universe, then we might feel better. Unfortunately, there is any
number of very useful problems we would like to solve that happen to be undecidable.  That’s a 
little disappointing now, isn’t it?!

Another famous undecidable problem is the E quivalence Problem. Imagine that you had a
number of programs that had run successfully for several years. You now want to replace these
programs with new programs that are supposed to run faster, but you want to make sure they
    – 9 – 

work. Is it possible to write an algorithm that will determine if two programs are equivalent
(they always produce the same output for the same input)? Unfortunately, it can be shown that a
solution to the equivalence problem would imply a solution to the halting problem — namely, if
you cannot write a program to determine if a program halts, how are you going to write a
program to determine if two programs halt (and are thus equivalent). The programs have to both
halt to generate the same output so, somewhere in this equivalence-finding program, you have to
determine that both programs halt. Since the halting problem is undecidable, then so must the
equivalence problem be. Along these same lines, tragically, it also turns out to be impossible to
write a program that will calculate the complexity (running time) of another arbitrary program.

It is worth emphasizing that undecidable problems are not merely open problems in computer
science. These are problems that are fundamentally unsolvable. No one has found a solution
because no general solution can exist. Sometimes, that's just life in the city.

Bibliography
Turing Machines and the proof of the undecidability of the Halting Problem come from:
[1] A. Turing, "On Computable Numbers with an Application to the Entscheidungsproblem," Proceedings of the
London Mathematical Society, 2, No. 42 (1936), 230-265.
Note: Entscheidungsproblem means “decision problem”, or the problem of decidability.

Church's Thesis comes from:

[2] A. Church, "An Unsolvable Problem of Elementary Number Theory," American Journal of Mathematics, 58
(1936), 345-363.

The above papers and many other seminal papers on undecidability can be found in the
collection:
[3] M. Davis, The Undecidable , Hewlett, NY: Raven Press, 1965.

For more on Turing Machines and undecidable problems see:

[4] M. Davis, Computability and Unsolvability, New York: McGraw-Hill, 1958.
[5] J. Hopcroft, J. Ullman, Introduction to Automata Theory, Languages, and Computation , Reading, MA:
Addison-Wesley, 1979.
[6] H. Lewis, C. Papadimitriou, Elements of the Theory of Computation, Englewood Cliffs, NJ: Prentice Hall, 1981.
[7] M. Minsky, Computation: F inite and Infinite Machines, Englewood Cliffs, NJ: Prentice Hall, 1967.