You are on page 1of 55

Algorithms

Readings: [SG] Ch. 2 & 3

Chapter Outline:
1. Chapter Goals
2. What are Algorithms
1. Real Life Examples (origami, recipes)
2. Definition
3. Example: A = B + C

3.
4.
5.
6.

Expressing Algorithms Pseudo-Code

Simple Algorithms
Recursive Algorithms
Time Complexity of Algorithms
(UIT2201: Algorithms) Page 1

Algorithms
Computing

How to explain to a dumb mechanical /

computing device how to solve a problem

How

a small, basic set of primitive

instructions.
Complexity

of Solving Problems.
(UIT2201: Algorithms) Page 2

To introduce notion of algorithm as means of

specifying how to solve a problem

To introduce and appreciate approaches for

defining and solving very complex tasks in

LeongHW, SoC, NUS

Chapter Outline:
1. Chapter Goals
2. What are Algorithms
1. Real Life Examples (origami, recipes)
2. Definition of Algorithm
3. Example: A = B + C

3.
4.
5.
6.

Expressing Algorithms Pseudo-Code

Simple Algorithms
Recursive Algorithms
Time Complexity of Algorithms

Computer

as study of algorithms including

their formal properties
Their hardware and software realisations
Their applications to diverse areas

Algorithms: Real Life Examples

Many

Real-Life Examples

Cooking: Recipe for preparing a dish

Origami: The Art of Paper Folding
Directions: How to go to Changi Airport
Remember:

Framework: How to give instructions;

Alg: The actual step-by-step instructions;
Abstraction: Decomposing / Simplifying

Framework:

Algorithm:

Recipe for a Dish

(step by step instructions on how to cook a dish)

Problem

Decomposition

Preparing Ingredients;
Preparing Sauce;

Framework:

Algorithm:

Sequence of Paper-Folding Instructions

(step by step instructions for each fold)

Problem

Decomposition

Finish the Legs; Finish the Tail;
(UIT2201: Algorithms) Page 8

Real Life Examples: Issues

Problems/Difficulties:

Imprecise Instructions;
Job can often be done even if instructions are
not followed precisely
Modifications may be done by the person
following the instructions;
But,

Needs to told PRECISELY what to do;

Instructions must be PRECISE;
Cannot be vague or ambiguous

LeongHW, SoC, NUS

Definition of Algorithm:
An

a finite sequence of unambiguous, executable

steps or instructions, which, if followed would
ultimately terminate and give the solution of
the problem.

Finite set of steps;

Unambiguous;
Executable;
Terminates;
(UIT2201: Algorithms) Page 10

LeongHW, SoC, NUS

Algorithm Examples?

Problem 1: What is the largest integer

INPUT: All the integers { -2, -1, 0, 1, 2, }
OUTPUT: The largest integer
Algorithm:
Arrange all the integers in a list in decreasing order;
MAX = first number in the list;
Print out MAX;

WHY is the above NOT an Algorithm?

(Hint: How many integers are there?)

Problem 2: Who is the tallest women in the world?

Algorithm: Tutorial...
(UIT2201: Algorithms) Page 11

LeongHW, SoC, NUS

Input: Two positive m-digit decimal numbers
(a and b)

am, am-1, ., a1
bm, bm-1, ., b1
Output: The sum c = a + b

cm+1, cm, cm-1, ., c1

(Note: In the textbook, it was am-1,,a1,a0 )
(UIT2201: Algorithms) Page 12

How

do we give the algorithm?

A step-by-step instruction
to a dumb machine
Try

an example:

3492
8157
Imagine you looking at yourself solving it
(UIT2201: Algorithms) Page 13

Algorithm: Finding sum of A & B

Step 1: Set the value of carry to 0
Step 2: Set the value of i to 1.
Step 3: Repeat steps 4 through 6 until the value of i is > m.
Step 4: Add ai and bi to the current value of carry, to get xi.
Step 5: If xi < 10 then
let ci=xi and reset carry to 0.
Else (* i.e. xi 10 *)

let ci=xi - 10 and reset carry to 1.

Step 6: Increase the value of i by 1.

Step 7: Set cm+1 to the value of carry.

Step 8: Print the final answer cm+1, cm, ., c1
Step 9: Stop.
(UIT2201: Algorithms) Page 14

LeongHW, SoC, NUS

Chapter Outline:
1. Chapter Goals
2. What are Algorithms
3. Expressing Algorithms Pseudo-Code
1.
2.
3.
4.
5.

Communicating Alg to computer

Pseudo-Code
Primitive Operations and examples
Variables and Arrays
Algorithm C=A+B in pseudo-code

4. Simple Algorithms
5. Recursive Algorithms
6. Time Complexity of Algorithms
(UIT2201: Algorithms) Page 15

To

Need way to represent the algorithm

Cannot use English
Can

machine language and

programming languages (Java, Pascal, C)
But, these are too tedious (&technical)
Use

Pseudo-Code

Mixture of computer language and English

Somewhere in between
precise enough to describe what is meant without
being too tediuos

Examples:
Let c be 0;
Sort the list of numbers in increasing order;

Need

to know both

syntax representation
semantics meaning
(UIT2201: Algorithms) Page 17

LeongHW, SoC, NUS

Primitive Operations
To

describe an algorithm, we need some

well-defined programming primitives
Assignment primitive:
assignment statement, input/output statements

Conditional primitive:
if statement
case statement

for loop,
while loop,

Statements

are executed one-by-one

(UIT2201: Algorithms) Page 18

LeongHW, SoC, NUS

Conditional Primitives
if

statement

to take different actions based on condition

Syntax
if (condition)
then (Step A)
else (Step B)
endif

true
Step A

condition?

false
Step B

if (condition)
then (Step A)
endif

Semantics
(UIT2201: Algorithms) Page 19

Examples -- (if statement)

Let mark be the total-mark obtained
if (mark < 40)
then (print Student fail)
else (print Student pass)
endif

read in mark (*from the terminal*)

if (mark < 40) then (Grade F)
else if (mark < 50) then (Grade
else if (mark < 60) then (Grade
else if (mark < 70) then (Grade
else if (mark < 80) then (Grade
endif

D)
C)
B)
A);

Looping Primitive while-loop

the while-loop
loop a variable
number of times

Syntax
while (condition) do
(some sequence
of statements)

condition?

false

true
Some sequence
of statements;

endwhile

Semantics
(UIT2201: Algorithms) Page 21

First, the for-loop

loop a fixed or (predetermined) number of
times

Syntax
for j a to b do
(some sequence
of statements)

j a;

(j <= b)?

false

true
Some sequence
of statements;

endfor

Semantics

j j+1;

Exercising the alg: for and while

for j 1 to 4 do
print 2*j;
endfor
print --- Done ---
Output:
2
4
6
8
--- Done ---

j 1;
while (j <= 4) do
print 2*j;
j j + 1;
endwhile
print --- Done ---

Output:
2
4
6
8
--- Done --(UIT2201: Algorithms) Page 23

In

the computer, each variable is assigned

a storage box
can store one number at any time
eg: sum, j, carry

Arrays:

Often deal with many numbers

Such as A1, A2, A3, , A100
Store as an array A, A, , A
we treat each of them as a variable,
each is assigned a storage box
(UIT2201: Algorithms) Page 24

Algorithm: A = B + C (in pseudo-code)

We can re-write the C=A+B algorithm as follows:
Alg. to Compute C = A + B:
(*sum two big numbers*)
carry 0;
for i 1 to m do
x[i] a[i] + b[i] + carry ;
if (x[i] < 10)
then ( c[i] x[i]; carry 0; )
else ( c[i] x[i] 10; carry 1; )
endfor;
c[m+1] carry;
Print c[m+1], c[m], ., c
(UIT2201: Algorithms) Page 25

LeongHW, SoC, NUS

Chapter Outline:
1.
2.
3.
4.

Chapter Goals
What are Algorithms
Expressing Algorithms Pseudo-Code
Simple Algorithms
1. Simple iterative algorithms
Computing Sum, Find Max/Min
2. Modular Program Design
3. Divisibility and Prime Numbers

5. Recursive Algorithms
6. Time Complexity of Algorithms
(UIT2201: Algorithms) Page 26

Note: Store numbers in array A, A, , A[n]

Sum(A, n);
begin
Sum_sf 0;
k 1;
while (k <= n) do
Sum_sf Sum_sf + A[k];
k k + 1;
endwhile
Sum Sum_sf;
Print Sum is, Sum
end;
(UIT2201: Algorithms) Page 27

Input:

A A A A A A

2
5
10
3
12
24

Processing:

Output:

k
?
1
2
3
4
5
6
6

Sum-sf
0
2
7
17
20
32
56
56

n=6

Sum
?
?
?
?
?
?
?
56

Sum is 56
(UIT2201: Algorithms) Page 28

We can also use a while-loop instead of a for loop.

Sum(A, n);
(* Find the sum of A1, A2,, An. *)
begin
Sum_sf 0;
for k 1 to n do
Sum_sf Sum_sf + A[k];
endfor
Sum Sum_sf;
Print Sum is, Sum
end;

HW: (a) Note the differences

(b) Modify it to compute the average?
(UIT2201: Algorithms) Page 29

Note the three stages:

1. Initialization
Set some values at the beginning

2. Iteration
This is the KEY STEP
Where most of work is done

3. Post-Processing or Cleanup

Can use this setup for other problems

Calculating average, sum-of-squares
Finding max, min; Searching for a number,
(UIT2201: Algorithms) Page 30

Software are complex

HUGE (millions of lines of code) eg: Linux, Outlook
COMPLEX; eg: Flight simulator

Idea: Divide-and-Conquer Method

Complex tasks can be divided and each part solved
separately and combined later.

Modular Program Design

Divide big programs into smaller modules
The smaller parts are
called modules, subroutines, or procedures
Design, implement, and test separately

Modularity, Abstraction, Division of Labour

Simplifies process of writing alg/programs
(UIT2201: Algorithms) Page 31

Algorithm to determine if n is prime

Given: A positive integer n
Question: Is n a prime number?

What do we know?
A number n is prime iff
n has no positive factors except 1 and n

Idea: Express it algorithmically

n is not divisible by any positive number k
such that 1 < k < n. or k=2,3,4,,(n-1)

A module Divisible(m,n) to check divisibility
We can use it as a primitive
(UIT2201: Algorithms) Page 32

Pseudo-Code for Prime:

Prime(n)
(* To determine if n is prime *)
begin
for k 2 to (n-1) do
if Divisible(k,n)
then (Print False & exit)
else (* Do nothing *)
endfor
Print True & Stop
end;

Note: Prime uses the module Divisible(k,n)

Exercise it with:
Prime (5); Prime (4);
Prime (55); Prime (41);
(UIT2201: Algorithms) Page 33

A

Given: Two positive integers m and n

Question: Is n divisible by m?
or Algorithm to compute Divisible(m,n)

Divisible(m, n)

Algorithm

{False, if n is not divisible by m

True,

if n is divisible by m

Idea:

A positive integer n is divisible by m iff

for some positive integer k n, m*k = n.
(UIT2201: Algorithms) Page 34

Algorithm (in pseudo-code)

Exercise it with:
Divisible (3, 9);

Divisible (3, 5);

Divisible (3,101);

Divisible(m,n)
(* To compute if n is divisible by m *)
begin
D false; (* assume false, first *)
for k 1 to n do
if (m*k = n) then (Dtrue; exit-loop)
else (* Do nothing *)
endfor
Divisible D; (* true or false *)
end;
(UIT2201: Algorithms) Page 35

LeongHW, SoC, NUS

Chapter Outline:
1.
2.
3.
4.
5.
6.

Chapter Goals
What are Algorithms
Expressing Algorithms Pseudo-Code
Simple Algorithms
Recursive Algorithms
Time Complexity of Algorithms
1. Sequential search algorithm
2. Binary search algorithm
3. Analysis of Time Complexity

7. Summary
(UIT2201: Algorithms) Page 36

Search: sequential search algorithm

List of numbers: A1, A2, A3, ., An
and a query number x;

Given:

Question: Search for x in the list;

Sequential-Search(A, n, x);
(* Search for x in A1, A2,, An. *)
begin
for k 1 to n do
if (x = A[k])
then (Print Yes; Stop)
else (* do nothing *)
endfor
Print No; Stop
end;
(UIT2201: Algorithms) Page 37

How fast is it to search for x?

How many comparisons do we need?
Example: 13, 38, 19, 74, 76, 14, 12, 38, 22, 55
When x=14, need 6 comparisons
When x=13, need only 1 comparison BEST CASE
When x=55, need 10 comparisons WORST CASE
When x=5, need 10 comparisons WORST CASE

In general, given n numbers, A1,,An

Best Case: 1 comparison
Worst Case: n comparisons
Average Case: (n+1)/2 comparisons
(UIT2201: Algorithms) Page 38

Binary Search
If

the List is sorted, that is

A1 A2 A3 . An

Then,

we can do better,

Binary Search

Imagine

n=100

After 1 step, size is 50

After 2 steps, size is 25
After 3 steps, size is 12
After 4 steps, size is 6
After 5 steps, size is 3
After 6 steps, size is 1
DONE!!

What

if n=1000? 1,000,000?
(UIT2201: Algorithms) Page 41

Binary Search Algorithm

Sorted List A1 A2 A3 . An
A number x.
Question: Is x in the list?
Binary-Search(A,n,x);
1. First 1
Last n
2. While ( First Last ) do
mid (first+last) / 2
If ( x = A[mid] )
then (output Yes and Stop)
else If ( x < A[mid] )
then Last mid-1
else If ( x > A[mid] ) then First mid+1
EndWhile
3. Output False and Stop
4. End
Input:

Time Complexity Analysis

Sequential

Search (Alg):

Worst Case:
n comparisons
Best Case: 1 comparison
Avg Case: n/2 comparisons
Binary

Search (Alg):

Worst Case: log2 n comparisons

Best Case: 1 comparison
Avg Case: about log2 n comparisons
How

using mathematical analysis

(UIT2201: Algorithms) Page 43

LeongHW, SoC, NUS

Complexity of Algorithm

Binary Search

A Linear Time Algorithm

Algorithm Sum(A,n) -- O(n) time
Algorithm Sequential-Search(A,n,x) O(n) time

Simple Median-Find (T2-Q3)

All-Subsets(A,n)

LeongHW, SoC, NUS

Chapter Outline:
1.
2.
3.
4.
5.

Chapter Goals
What are Algorithms
Expressing Algorithms Pseudo-Code
Simple Algorithms
Recursive Algorithms
1. Recursion the idea
2. Fibonacci sequence
3. Tower of Hanoi

5. Recursion

A problem solving method of

decomposing bigger problems into
smaller sub-problems that are identical to itself.

General Idea:
Solve simplest (smallest) cases DIRECTLY
usually these are very easy to solve

Solve bigger problems using smaller sub-problems

that are identical to itself (but smaller and simpler)

Abstraction:
To solve a given problem, we first assume that we
ALREADY know how to solve it for smaller instances!!

Dictionary definition:
recursion
see recursion
(UIT2201: Algorithms) Page 46

Definition of Fibonacci numbers

1. F1 = 1,
2. F2 = 1,
3. for n>2, Fn = Fn-1 + Fn-2

The above is a recursive definition.

Fn is computed in-terms of itself
actually, smaller copies of itself Fn-1 and Fn-2

Actually, Not difficult:

F3 = 1 + 1 = 2
F4 = 2 + 1 = 3
F5 = 3 + 2 = 5

F6 = 5 + 3 = 8
F7 = 8 + 5 = 13
F8 = 13 + 8 = 21

F9 = 21 + 13 = 34
F10 = 34 + 21 = 55
F11 = 55 + 34 = 89

1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144,

(UIT2201: Algorithms) Page 47

Fibonacci Numbers: Recursive alg

Fibonacci(n) (* Recursive, SLOW *)
begin
if (n=1) or (n=2)
then Fibonacci(n) 1 (*simple case*)
else Fibonacci(n) Fibonacci(n-1) +
Fibonacci(n-2)
endif
end;

The
It

is simple to understand and elegant!

But,

very SLOW
(UIT2201: Algorithms) Page 48

How

slow is it?

F(6)
F(5)

F(4)

F(4)
F(3)
F(2)

F(3)
F(2)

F(2)

F(3)
F(1)

F(2)

F(2)
F(1)

F(1)

HW: Can we compute it faster?

(UIT2201: Algorithms) Page 49

A

Given: Three Pegs A, B and C

Peg A initially has n disks, different size, stacked up,
larger disks are below smaller disks

Problem: to move the n disks to Peg C, subject to

1. Can move only one disk at a time
2. Smaller disk should be above larger disk
3. Can use other peg as intermediate
(UIT2201: Algorithms) Page 50

Tower of Hanoi

How to Solve: Strategy

Generalize first: Consider n disks for all n 1
Our example is only the case when n=4

Look at small instances

Of course, just Move disk 1 from A to C

1. Move disk 1 from A to B
2. Move disk 2 from A to C
3. Move disk 1 from B to C
(UIT2201: Algorithms) Page 51

Tower of Hanoi (Solution!)

General Method:
First, move first (n-1) disks from A to B
Now, can move largest disk from A to C
Then, move first (n-1) disks from B to C

Try this method for n=3

1. Move disk 1 from A to C
2. Move disk 2 from A to B
3. Move disk 1 from C to B
4. Move disk 3 from A to C

5. Move disk 1 from B to A

6. Move disk 1 from B to C
7. Move disk 1 from A to C
(UIT2201: Algorithms) Page 52

Algorithm for Towel of Hanoi (recursive)

Recursive Algorithm
when (n=1), we have simple case
Else (decompose problem and make recursive-calls)

Hanoi(n, A, B, C);
(* Move n disks from A to C via B *)
begin
if (n=1) then Move top disk from A to C
else (* when n>1 *)
Hanoi (n-1, A, C, B);
Move top disk from A to C
Hanoi (n-1, B, C, A);
endif
end;
(UIT2201: Algorithms) Page 53

LeongHW, SoC, NUS

Characteristics of Algorithm

Correctness

Complexity --- time, space (memory), etc

Ease of understanding

Ease of coding

Ease of maintainence

If