Chapter 0

Introduction
The word Algorithm is attributed to the great 9th century Iranian
mathematician and astronomer Khowrezmi * . Algorithm is a sequence of
clear instructions for solving a problem, i.e. for obtaining a required output
for any legitimate input in a finite amount of time. Note that:
The range of inputs for which an algorithm works must be specified.
The same algorithm can be represented in several different ways.
The same algorithm, with possibly small modification, maybe able to
solve several different problems.
Several algorithms may exist for solving the same problem.
Algorithms for the same problem can be based on very different ideas
and can solve the problem with very different speeds and efficiency.

Musa Khowrazmi (also known as Al-Khowrezmi) was born in the province of Khorasan,
northwest of Iran, and lived about 780-850 A.D. He became the chief mathematician in the
"House of Wisdom", the academy of sciences of the time. The word "algebra" comes from
the title of his book "Kitab al muhtasar fi hisab algabr w'al muqubalah". This title has been
translated as "A compact book on calculation using rules of completion and reduction." When
Al-Khorezmi's book on the new science (algebra) reached Europe, the Europeans called its
use "algorism" or "algorithm," a corruption of the author's name. For more information see a
long and charming article about Khorezmi, his work and his times, by Heinz Zemanek of
Vienna, Austria. In September 1979, in what was the 1200th anniversary of Al-Khorezmi's
birth, mathematicians marked the occasion by convening at the site of the Khorazem oasis a
conference "Algorithms in Modern Mathematics and Computer Science." Dr. Zemanek's
article opens the bound proceedings of that conference, edited by A.P. Ershov and D.E.
Knuth and published in 1981 by Springer Verlag as volume 122 of "Lecture Notes in
Computer Science."
Unfortunately during the recent invasion and occupation of Iraq, and the pillage of Baghdads
museums and libraries, almost all the art and science treasures including mathematical
manuscripts were looted under the watchful eyes of the occupation forces. See the lead
article in the Newsletter of the British Society for History of Mathematics (BSHM
Newsletter, vol. 49, Autumn 2003, pp.1-9) by Dr. Eleanor Robson where the author attempts
to assess the damage to Iraqs mathematical and cultural heritage based on the expert
assessments and eyewitness reports.

1. 1 Algorithmic Problem Solving

We can consider algorithms to be procedural solutions to problems. These
solutions are not answers but rather specific instructions for getting answers.
The sequence of steps one typically must go through in designing and
analyzing of an algorithm are:
(a) Understanding the problem:
Read the problems description
do a few simple examples by hand, think about special cases.
Specify range of instances algorithm needs to handle.
An input to an algorithm specifies an instance of the problem that the
algorithm solves.
(b) Computer to be used:
Sequential algorithms Ordinary computer architecture (i.e. von
Neumann machine) - based on random access machine (RAM) in
which the instruction are executed one after the other, one operation at
a time. Most algorithm are sequential.
Parallel algorithms Some newer computers can execute operations
concurrently, i.e. in parallel.
(c) Exact and approximation algorithms:
Exact: Most practical algorithms can be implemented to solve the
problem exactly.
Approximation : For problems that cannot be solved exactly because
of numerical difficulties, or because finding the exact solution can
take considerable computer time.
(d) Data Structures: Some algorithms require simple data structure.
Others require a proper choice of suitable data structures. In the new
paradigm of object oriented programming, data structures have become
very important for both design and analysis of algorithms.
(e) Algorithm design technique: An algorithm design technique is a
general strategy or approach to solving problems algorithmically that is
applicable to a variety of problems.
1

(f) Specifying an algorithm: After choosing basic strategy of the algorithm

specify steps, or pseudo-code. A pseudo-code is a mixture of a natural
language and programming language-like constructs, and is more precise
than a natural language.
(g) Proving algorithms correctness: To show that the algorithm yields the
required result for every legitimate input in finite time. A common
method for proof is to use mathematical induction.
Note: Examples are useful for tracing the code for a few inputs they are
not sufficient to show the correctness. Always remember that one or
more examples do not constitute a proof.

1.2 Algorithm Analysis

The goal of algorithm analysis is to improve algorithm efficiency or to
choose the most efficient algorithm among several algorithms.
Algorithm/program efficiency is a measure of :
Amount of work needed by the computer. This is called complexity of
the algorithm.
Amount of computer space needed
Simplicity
Optimality
Amount of work or complexity:
Must be independent of particular computer used
Must reflect execution time
Must reflect the amount of basic or fundamental operations such as
comparisons, multiplication, etc.
Must be independent of implementation details such as initialization of a
variable
Must reflect the size of the input data (n)
We will express the algorithm complexity (or amount of basic operations) as
a function of input size, i.e. C(n).
2

Amount of Space:
The number of bytes of memory needed for the program is the amount of
space.
Some programs require making copies of data during execution, and thus
need more space.
In modern computers space is not a critical factor for most programs.
Simplicity:

A simple and straight forward algorithm is not usually the most efficient
one.
It is desirable to have a simple algorithm, but efficiency is usually the
deciding factor.
Optimality:
A program is optimal for a given problem if there is no other algorithm
that performs fewer basic operations.
Each problem has an inherent least complexity, i.e. it needs some
minimum amount of work to solve it.

1.3 Problem Types

There are several problems that have attracted considerable interest because
of their practical importance, and at times due to their challenge as a
research topic.
Sorting: As a practical matter we need to sort (or arrange in particular
order) list of numbers, characters, strings, and records (e.g. student records
in university, customer records in bank, etc.).
Ordering makes many questions about the data easier to answer. Dozens of
sorting algorithms have been developed, some for particular cases or
applications but most for general sorting. The most efficient sorting

methods are those that require (n lg n ) in the worst case and for any set of
inputs. However, there is no sorting method that best fits all situations.
Note: This course assumes that you are already well familiar with sorting
algorithms and their analysis, using asymptotic notations, , , , and
that you have some familiarity with recurrence relations.
Searching: Finding a key in a given set. Simple sequential search to
efficient binary search and interpolation search. Some are fast but are
applicable only to sorted arrays, others may require more memory. Some
are suitable for static dictionary search, i.e. items are fixed. Others, such as
balanced search trees, are applicable to the dynamic dictionary search where
insertion, deletion and search are supported.
Note: This course assumes that you have covered in other courses the main
searching methods including binary search, binary search trees, and
balanced search trees.
String Processing: The rapid increase in applications such as internet
search engines and bioinformatics, dealing with non-numerical data has
brought about interest in string handling algorithms. One particular
problem, that of searching for a given word in a text, has attracted
considerable attention. It is called string matching and several algorithms
exist for this problem.
Graph Problems: One of the oldest and most useful area in algorithms is
graphs. Graphs can be used for modeling a wide variety of real-life
applications, including transportation, communication networks, project
scheduling, and games. Basic graphs algorithms include graph traversal,
shortest path, topological sorting, minimal spanning tree, connectivity and
strong connectivity, etc.

Some graph algorithms are computationally very hard, and that only very
small instances of such problems can be solved in reasonable amount of time
even on fastest computers.
Note: This course assumes that you know the basic graph algorithms listed
above. A review of the basic graph problems can be found in Chapter -1

Combinatorial Problems: These are problems that ask to find a

combinatorial object such a permutation, combination or subset that satisfy
certain constraints and certain desired properties (e.g. maximizing a values,
etc). Theses problems are the most difficult problems in computing since
the number of combinatorial objects typically grow very fast (e.g.
exponentially) with a problem size. In addition, there are no known
algorithms for solving most of these problems in reasonable amount of
computer time.
Geometric Problems: Deal with objects such as points, lines and polygons.
Ancient Egyptian and Greeks were very much interested in these problems.
Then for about 2000 years the interest in geometric algorithms disappeared,
but recently there has been renewed interest with applications such as
graphics, robotics and tomography.
Numerical problems: This is a very large area of applications, and involves
mathematical objects of continuous nature solving linear and nonlinear
equations, computing integral, evaluation functions, etc. The majority of
these problems can be solved only approximately. Many sophisticated
algorithms have been developed and continue to play a major role in many
scientific and engineering applications. In the last two decades, the
numerical problems have shifted to business applications which require
algorithms for information storage, retrieval, transportation and networks.