You are on page 1of 55

Algorithms

Readings: [SG] Ch. 2 & 3

Chapter Outline:
1. Chapter Goals
2. What are Algorithms
1. Real Life Examples (origami, recipes)
2. Definition
3. Example: A = B + C

3.
4.
5.
6.

Expressing Algorithms Pseudo-Code


Simple Algorithms
Recursive Algorithms
Time Complexity of Algorithms
(UIT2201: Algorithms) Page 1

LeongHW, SoC, NUS

Algorithms
Computing

devices are dumb

How to explain to a dumb mechanical /


computing device how to solve a problem

How

to solve a problem using

a small, basic set of primitive


instructions.
Complexity

of Solving Problems.
(UIT2201: Algorithms) Page 2

LeongHW, SoC, NUS

1. Goals of Algorithm Study

To develop framework for instructing


computer to perform tasks

To introduce notion of algorithm as means of


specifying how to solve a problem

To introduce and appreciate approaches for


defining and solving very complex tasks in
terms of simpler tasks;

(UIT2201: Algorithms) Page 3

LeongHW, SoC, NUS

Chapter Outline:
1. Chapter Goals
2. What are Algorithms
1. Real Life Examples (origami, recipes)
2. Definition of Algorithm
3. Example: A = B + C

3.
4.
5.
6.

Expressing Algorithms Pseudo-Code


Simple Algorithms
Recursive Algorithms
Time Complexity of Algorithms

(UIT2201: Algorithms) Page 4

LeongHW, SoC, NUS

2. Computer Science and Algorithms


Computer

Science can be considered

as study of algorithms including


their formal properties
Their hardware and software realisations
Their applications to diverse areas

(UIT2201: Algorithms) Page 5

LeongHW, SoC, NUS

Algorithms: Real Life Examples


Many

Real-Life Examples

Cooking: Recipe for preparing a dish


Origami: The Art of Paper Folding
Directions: How to go to Changi Airport
Remember:

Framework: How to give instructions;


Alg: The actual step-by-step instructions;
Abstraction: Decomposing / Simplifying

(UIT2201: Algorithms) Page 6

LeongHW, SoC, NUS

Real Life Example: Cooking


Framework:

Cooking or Recipe language


Algorithm:

Recipe for a Dish


(step by step instructions on how to cook a dish)

Problem

Decomposition

Preparing Ingredients;
Preparing Sauce;

(UIT2201: Algorithms) Page 7

LeongHW, SoC, NUS

Real Life Example: Origami


Framework:

Origami or Paper-Folding language


Algorithm:

Sequence of Paper-Folding Instructions


(step by step instructions for each fold)

Problem

Decomposition

Start with a Bird Base;


Finish the Head;
Finish the Legs; Finish the Tail;
(UIT2201: Algorithms) Page 8

LeongHW, SoC, NUS

Real Life Examples: Issues


Problems/Difficulties:

Imprecise Instructions;
Job can often be done even if instructions are
not followed precisely
Modifications may be done by the person
following the instructions;
But,

NOT for a Computer

Needs to told PRECISELY what to do;


Instructions must be PRECISE;
Cannot be vague or ambiguous

(UIT2201: Algorithms) Page 9

LeongHW, SoC, NUS

Definition of Algorithm:
An

algorithm for solving a problem

a finite sequence of unambiguous, executable


steps or instructions, which, if followed would
ultimately terminate and give the solution of
the problem.

Note the keywords:

Finite set of steps;


Unambiguous;
Executable;
Terminates;
(UIT2201: Algorithms) Page 10

LeongHW, SoC, NUS

Algorithm Examples?

Problem 1: What is the largest integer


INPUT: All the integers { -2, -1, 0, 1, 2, }
OUTPUT: The largest integer
Algorithm:
Arrange all the integers in a list in decreasing order;
MAX = first number in the list;
Print out MAX;

WHY is the above NOT an Algorithm?


(Hint: How many integers are there?)

Problem 2: Who is the tallest women in the world?


Algorithm: Tutorial...
(UIT2201: Algorithms) Page 11

LeongHW, SoC, NUS

Example: Adding two (n-digit) numbers


Input: Two positive m-digit decimal numbers
(a and b)

am, am-1, ., a1
bm, bm-1, ., b1
Output: The sum c = a + b

cm+1, cm, cm-1, ., c1


(Note: In the textbook, it was am-1,,a1,a0 )
(UIT2201: Algorithms) Page 12

LeongHW, SoC, NUS

How to derive the algorithm


Adding

is something we all know

done it a thousand times, know it by heart


How

do we give the algorithm?

A step-by-step instruction
to a dumb machine
Try

an example:

3492
8157
Imagine you looking at yourself solving it
(UIT2201: Algorithms) Page 13

LeongHW, SoC, NUS

Algorithm: Finding sum of A & B


Step 1: Set the value of carry to 0
Step 2: Set the value of i to 1.
Step 3: Repeat steps 4 through 6 until the value of i is > m.
Step 4: Add ai and bi to the current value of carry, to get xi.
Step 5: If xi < 10 then
let ci=xi and reset carry to 0.
Else (* i.e. xi 10 *)

let ci=xi - 10 and reset carry to 1.


Step 6: Increase the value of i by 1.

Step 7: Set cm+1 to the value of carry.


Step 8: Print the final answer cm+1, cm, ., c1
Step 9: Stop.
(UIT2201: Algorithms) Page 14

LeongHW, SoC, NUS

Chapter Outline:
1. Chapter Goals
2. What are Algorithms
3. Expressing Algorithms Pseudo-Code
1.
2.
3.
4.
5.

Communicating Alg to computer


Pseudo-Code
Primitive Operations and examples
Variables and Arrays
Algorithm C=A+B in pseudo-code

4. Simple Algorithms
5. Recursive Algorithms
6. Time Complexity of Algorithms
(UIT2201: Algorithms) Page 15

LeongHW, SoC, NUS

3. Expressing Algorithms to Computer


To

communicate algorithm to computer

Need way to represent the algorithm


Cannot use English
Can

use computer language

machine language and


programming languages (Java, Pascal, C)
But, these are too tedious (&technical)
Use

Pseudo-Code instead

(UIT2201: Algorithms) Page 16

LeongHW, SoC, NUS

Pseudo-Code to express Algorithms


Pseudo-Code

Mixture of computer language and English


Somewhere in between
precise enough to describe what is meant without
being too tediuos

Examples:
Let c be 0;
Sort the list of numbers in increasing order;

Need

to know both

syntax representation
semantics meaning
(UIT2201: Algorithms) Page 17

LeongHW, SoC, NUS

Primitive Operations
To

describe an algorithm, we need some


well-defined programming primitives
Assignment primitive:
assignment statement, input/output statements

Conditional primitive:
if statement
case statement

Looping (iterative) primitive:


for loop,
while loop,

Statements

are executed one-by-one


(UIT2201: Algorithms) Page 18

LeongHW, SoC, NUS

Conditional Primitives
if

statement

to take different actions based on condition


Syntax
if (condition)
then (Step A)
else (Step B)
endif

true
Step A

condition?

false
Step B

if (condition)
then (Step A)
endif

Semantics
(UIT2201: Algorithms) Page 19

LeongHW, SoC, NUS

Examples -- (if statement)


Let mark be the total-mark obtained
if (mark < 40)
then (print Student fail)
else (print Student pass)
endif

read in mark (*from the terminal*)


if (mark < 40) then (Grade F)
else if (mark < 50) then (Grade
else if (mark < 60) then (Grade
else if (mark < 70) then (Grade
else if (mark < 80) then (Grade
endif
print Student grade is, Grade

D)
C)
B)
A);

(UIT2201: Algorithms) Page 20

LeongHW, SoC, NUS

Looping Primitive while-loop

the while-loop
loop a variable
number of times

Syntax
while (condition) do
(some sequence
of statements)

condition?

false

true
Some sequence
of statements;

endwhile

Semantics
(UIT2201: Algorithms) Page 21

LeongHW, SoC, NUS

Looping Primitive for-loop

First, the for-loop


loop a fixed or (predetermined) number of
times

Syntax
for j a to b do
(some sequence
of statements)

j a;

(j <= b)?

false

true
Some sequence
of statements;

endfor

Semantics

j j+1;

(UIT2201: Algorithms) Page 22

LeongHW, SoC, NUS

Exercising the alg: for and while


for j 1 to 4 do
print 2*j;
endfor
print --- Done ---
Output:
2
4
6
8
--- Done ---

j 1;
while (j <= 4) do
print 2*j;
j j + 1;
endwhile
print --- Done ---

Output:
2
4
6
8
--- Done --(UIT2201: Algorithms) Page 23

LeongHW, SoC, NUS

Variables and Arrays


In

the computer, each variable is assigned


a storage box
can store one number at any time
eg: sum, j, carry

Arrays:

Often deal with many numbers


Such as A1, A2, A3, , A100
Store as an array A[1], A[2], , A[100]
we treat each of them as a variable,
each is assigned a storage box
(UIT2201: Algorithms) Page 24

LeongHW, SoC, NUS

Algorithm: A = B + C (in pseudo-code)


We can re-write the C=A+B algorithm as follows:
Alg. to Compute C = A + B:
(*sum two big numbers*)
carry 0;
for i 1 to m do
x[i] a[i] + b[i] + carry ;
if (x[i] < 10)
then ( c[i] x[i]; carry 0; )
else ( c[i] x[i] 10; carry 1; )
endfor;
c[m+1] carry;
Print c[m+1], c[m], ., c[1]
(UIT2201: Algorithms) Page 25

LeongHW, SoC, NUS

Chapter Outline:
1.
2.
3.
4.

Chapter Goals
What are Algorithms
Expressing Algorithms Pseudo-Code
Simple Algorithms
1. Simple iterative algorithms
Computing Sum, Find Max/Min
2. Modular Program Design
3. Divisibility and Prime Numbers

5. Recursive Algorithms
6. Time Complexity of Algorithms
(UIT2201: Algorithms) Page 26

LeongHW, SoC, NUS

Simple iterative algorithm: Sum

Given: List of numbers: A1, A2, A3, ., An

Output: To compute the sum of the numbers

Note: Store numbers in array A[1], A[2], , A[n]


Sum(A, n);
begin
Sum_sf 0;
k 1;
while (k <= n) do
Sum_sf Sum_sf + A[k];
k k + 1;
endwhile
Sum Sum_sf;
Print Sum is, Sum
end;
(UIT2201: Algorithms) Page 27

LeongHW, SoC, NUS

Exercising Algorithm Sum:


Input:

A[1] A[2] A[3] A[4] A[5] A[6]


2
5
10
3
12
24

Processing:

Output:

k
?
1
2
3
4
5
6
6

Sum-sf
0
2
7
17
20
32
56
56

n=6

Sum
?
?
?
?
?
?
?
56

Sum is 56
(UIT2201: Algorithms) Page 28

LeongHW, SoC, NUS

Algorithm for Sum (with for-loop)

We can also use a while-loop instead of a for loop.


Sum(A, n);
(* Find the sum of A1, A2,, An. *)
begin
Sum_sf 0;
for k 1 to n do
Sum_sf Sum_sf + A[k];
endfor
Sum Sum_sf;
Print Sum is, Sum
end;

HW: (a) Note the differences


(b) Modify it to compute the average?
(UIT2201: Algorithms) Page 29

LeongHW, SoC, NUS

Remarks about the iterative algorithm

Note the three stages:


1. Initialization
Set some values at the beginning

2. Iteration
This is the KEY STEP
Where most of work is done

3. Post-Processing or Cleanup

Can use this setup for other problems


Calculating average, sum-of-squares
Finding max, min; Searching for a number,
(UIT2201: Algorithms) Page 30

LeongHW, SoC, NUS

Modular Program Design

Software are complex


HUGE (millions of lines of code) eg: Linux, Outlook
COMPLEX; eg: Flight simulator

Idea: Divide-and-Conquer Method


Complex tasks can be divided and each part solved
separately and combined later.

Modular Program Design


Divide big programs into smaller modules
The smaller parts are
called modules, subroutines, or procedures
Design, implement, and test separately

Modularity, Abstraction, Division of Labour


Simplifies process of writing alg/programs
(UIT2201: Algorithms) Page 31

LeongHW, SoC, NUS

Simple Algorithms: Prime Numbers

Algorithm to determine if n is prime


Given: A positive integer n
Question: Is n a prime number?

What do we know?
A number n is prime iff
n has no positive factors except 1 and n

Idea: Express it algorithmically


n is not divisible by any positive number k
such that 1 < k < n. or k=2,3,4,,(n-1)

What we already have:


A module Divisible(m,n) to check divisibility
We can use it as a primitive
(UIT2201: Algorithms) Page 32

LeongHW, SoC, NUS

Pseudo-Code for Prime:


Prime(n)
(* To determine if n is prime *)
begin
for k 2 to (n-1) do
if Divisible(k,n)
then (Print False & exit)
else (* Do nothing *)
endfor
Print True & Stop
end;

Note: Prime uses the module Divisible(k,n)

Exercise it with:
Prime (5); Prime (4);
Prime (55); Prime (41);
(UIT2201: Algorithms) Page 33

LeongHW, SoC, NUS

A Simple Module: Divisibility


A

module (procedure) for divisibility

Given: Two positive integers m and n


Question: Is n divisible by m?
or Algorithm to compute Divisible(m,n)

Divisible(m, n)

Algorithm

{False, if n is not divisible by m


True,

if n is divisible by m

Idea:

A positive integer n is divisible by m iff


for some positive integer k n, m*k = n.
(UIT2201: Algorithms) Page 34

LeongHW, SoC, NUS

Module for Divisibility:

Algorithm (in pseudo-code)

Exercise it with:
Divisible (3, 9);

Divisible (3, 5);

Divisible (3,101);

Divisible(m,n)
(* To compute if n is divisible by m *)
begin
D false; (* assume false, first *)
for k 1 to n do
if (m*k = n) then (Dtrue; exit-loop)
else (* Do nothing *)
endfor
Divisible D; (* true or false *)
end;
(UIT2201: Algorithms) Page 35

LeongHW, SoC, NUS

Chapter Outline:
1.
2.
3.
4.
5.
6.

Chapter Goals
What are Algorithms
Expressing Algorithms Pseudo-Code
Simple Algorithms
Recursive Algorithms
Time Complexity of Algorithms
1. Sequential search algorithm
2. Binary search algorithm
3. Analysis of Time Complexity

7. Summary
(UIT2201: Algorithms) Page 36

LeongHW, SoC, NUS

Search: sequential search algorithm


List of numbers: A1, A2, A3, ., An
and a query number x;

Given:

Question: Search for x in the list;


Sequential-Search(A, n, x);
(* Search for x in A1, A2,, An. *)
begin
for k 1 to n do
if (x = A[k])
then (Print Yes; Stop)
else (* do nothing *)
endfor
Print No; Stop
end;
(UIT2201: Algorithms) Page 37

LeongHW, SoC, NUS

Remarks on Sequential Search Alg

Analogy: House-to-house search

How fast is it to search for x?


How many comparisons do we need?
Example: 13, 38, 19, 74, 76, 14, 12, 38, 22, 55
When x=14, need 6 comparisons
When x=13, need only 1 comparison BEST CASE
When x=55, need 10 comparisons WORST CASE
When x=5, need 10 comparisons WORST CASE

In general, given n numbers, A1,,An


Best Case: 1 comparison
Worst Case: n comparisons
Average Case: (n+1)/2 comparisons
(UIT2201: Algorithms) Page 38

LeongHW, SoC, NUS

Binary Search
If

the List is sorted, that is


A1 A2 A3 . An

Then,

we can do better,

actually a lot better.

(UIT2201: Algorithms) Page 39

LeongHW, SoC, NUS

Binary Search

1, 4, 9, 11, 14, 43, 78

(UIT2201: Algorithms) Page 40

LeongHW, SoC, NUS

Binary Search (How fast is it?)


Imagine

n=100

After 1 step, size is 50


After 2 steps, size is 25
After 3 steps, size is 12
After 4 steps, size is 6
After 5 steps, size is 3
After 6 steps, size is 1
DONE!!

What

if n=1000? 1,000,000?
(UIT2201: Algorithms) Page 41

LeongHW, SoC, NUS

Binary Search Algorithm


Sorted List A1 A2 A3 . An
A number x.
Question: Is x in the list?
Binary-Search(A,n,x);
1. First 1
Last n
2. While ( First Last ) do
mid (first+last) / 2
If ( x = A[mid] )
then (output Yes and Stop)
else If ( x < A[mid] )
then Last mid-1
else If ( x > A[mid] ) then First mid+1
EndWhile
3. Output False and Stop
4. End
Input:

(UIT2201: Algorithms) Page 42

LeongHW, SoC, NUS

Time Complexity Analysis


Sequential

Search (Alg):

Worst Case:
n comparisons
Best Case: 1 comparison
Avg Case: n/2 comparisons
Binary

Search (Alg):

Worst Case: log2 n comparisons


Best Case: 1 comparison
Avg Case: about log2 n comparisons
How

to get the Average Case?

using mathematical analysis


(UIT2201: Algorithms) Page 43

LeongHW, SoC, NUS

Complexity of Algorithm

Logarithmic Time Algorithm


Binary Search

A Linear Time Algorithm


Algorithm Sum(A,n) -- O(n) time
Algorithm Sequential-Search(A,n,x) O(n) time

A Quadratic Time Algorithm


Simple Median-Find (T2-Q3)

An Exponential Time Algorithm


All-Subsets(A,n)

(UIT2201: Algorithms) Page 44

LeongHW, SoC, NUS

Chapter Outline:
1.
2.
3.
4.
5.

Chapter Goals
What are Algorithms
Expressing Algorithms Pseudo-Code
Simple Algorithms
Recursive Algorithms
1. Recursion the idea
2. Fibonacci sequence
3. Tower of Hanoi

6. Time Complexity of Algorithms

(UIT2201: Algorithms) Page 45

LeongHW, SoC, NUS

5. Recursion

A problem solving method of


decomposing bigger problems into
smaller sub-problems that are identical to itself.

General Idea:
Solve simplest (smallest) cases DIRECTLY
usually these are very easy to solve

Solve bigger problems using smaller sub-problems


that are identical to itself (but smaller and simpler)

Abstraction:
To solve a given problem, we first assume that we
ALREADY know how to solve it for smaller instances!!

Dictionary definition:
recursion
see recursion
(UIT2201: Algorithms) Page 46

LeongHW, SoC, NUS

Example: Fibonacci Numbers

Definition of Fibonacci numbers


1. F1 = 1,
2. F2 = 1,
3. for n>2, Fn = Fn-1 + Fn-2

Problem: Compute Fn for any n.

The above is a recursive definition.


Fn is computed in-terms of itself
actually, smaller copies of itself Fn-1 and Fn-2

Actually, Not difficult:


F3 = 1 + 1 = 2
F4 = 2 + 1 = 3
F5 = 3 + 2 = 5

F6 = 5 + 3 = 8
F7 = 8 + 5 = 13
F8 = 13 + 8 = 21

F9 = 21 + 13 = 34
F10 = 34 + 21 = 55
F11 = 55 + 34 = 89

1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144,


(UIT2201: Algorithms) Page 47

LeongHW, SoC, NUS

Fibonacci Numbers: Recursive alg


Fibonacci(n) (* Recursive, SLOW *)
begin
if (n=1) or (n=2)
then Fibonacci(n) 1 (*simple case*)
else Fibonacci(n) Fibonacci(n-1) +
Fibonacci(n-2)
endif
end;

The
It

above is a recursive algorithm

is simple to understand and elegant!

But,

very SLOW
(UIT2201: Algorithms) Page 48

LeongHW, SoC, NUS

Recursive Fibonacci Alg -- Remarks


How

slow is it?

Eg: To compute F(6)


F(6)
F(5)

F(4)

F(4)
F(3)
F(2)

F(3)
F(2)

F(2)

F(3)
F(1)

F(2)

F(2)
F(1)

F(1)

HW: Can we compute it faster?


(UIT2201: Algorithms) Page 49

LeongHW, SoC, NUS

Example: Tower of Hanoi


A

Given: Three Pegs A, B and C


Peg A initially has n disks, different size, stacked up,
larger disks are below smaller disks

Problem: to move the n disks to Peg C, subject to


1. Can move only one disk at a time
2. Smaller disk should be above larger disk
3. Can use other peg as intermediate
(UIT2201: Algorithms) Page 50

LeongHW, SoC, NUS

Tower of Hanoi

How to Solve: Strategy


Generalize first: Consider n disks for all n 1
Our example is only the case when n=4

Look at small instances


How about n=1
Of course, just Move disk 1 from A to C

How about n=2?


1. Move disk 1 from A to B
2. Move disk 2 from A to C
3. Move disk 1 from B to C
(UIT2201: Algorithms) Page 51

LeongHW, SoC, NUS

Tower of Hanoi (Solution!)

General Method:
First, move first (n-1) disks from A to B
Now, can move largest disk from A to C
Then, move first (n-1) disks from B to C

Try this method for n=3


1. Move disk 1 from A to C
2. Move disk 2 from A to B
3. Move disk 1 from C to B
4. Move disk 3 from A to C

5. Move disk 1 from B to A


6. Move disk 1 from B to C
7. Move disk 1 from A to C
(UIT2201: Algorithms) Page 52

LeongHW, SoC, NUS

Algorithm for Towel of Hanoi (recursive)

Recursive Algorithm
when (n=1), we have simple case
Else (decompose problem and make recursive-calls)

Hanoi(n, A, B, C);
(* Move n disks from A to C via B *)
begin
if (n=1) then Move top disk from A to C
else (* when n>1 *)
Hanoi (n-1, A, C, B);
Move top disk from A to C
Hanoi (n-1, B, C, A);
endif
end;
(UIT2201: Algorithms) Page 53

LeongHW, SoC, NUS

Characteristics of Algorithm

Correctness

Complexity --- time, space (memory), etc

Ease of understanding

Ease of coding

Ease of maintainence

(UIT2201: Algorithms) Page 54

LeongHW, SoC, NUS

If

you are new to algorithms

read the textbook


try out the algorithms
do the exercises

The End

(UIT2201: Algorithms) Page 55

LeongHW, SoC, NUS