You are on page 1of 55

# Algorithms

## Readings: [SG] Ch. 2 & 3

Chapter Outline:
1. Chapter Goals
2. What are Algorithms
1. Real Life Examples (origami, recipes)
2. Definition
3. Example: A = B + C

3.
4.
5.
6.

## Expressing Algorithms Pseudo-Code

Simple Algorithms
Recursive Algorithms
Time Complexity of Algorithms
(UIT2201: Algorithms) Page 1

Algorithms
Computing

## How to explain to a dumb mechanical /

computing device how to solve a problem

How

## a small, basic set of primitive

instructions.
Complexity

of Solving Problems.
(UIT2201: Algorithms) Page 2

## To introduce notion of algorithm as means of

specifying how to solve a problem

## To introduce and appreciate approaches for

defining and solving very complex tasks in

## LeongHW, SoC, NUS

Chapter Outline:
1. Chapter Goals
2. What are Algorithms
1. Real Life Examples (origami, recipes)
2. Definition of Algorithm
3. Example: A = B + C

3.
4.
5.
6.

## Expressing Algorithms Pseudo-Code

Simple Algorithms
Recursive Algorithms
Time Complexity of Algorithms

Computer

## as study of algorithms including

their formal properties
Their hardware and software realisations
Their applications to diverse areas

## Algorithms: Real Life Examples

Many

Real-Life Examples

## Cooking: Recipe for preparing a dish

Origami: The Art of Paper Folding
Directions: How to go to Changi Airport
Remember:

## Framework: How to give instructions;

Alg: The actual step-by-step instructions;
Abstraction: Decomposing / Simplifying

Framework:

Algorithm:

## Recipe for a Dish

(step by step instructions on how to cook a dish)

Problem

Decomposition

Preparing Ingredients;
Preparing Sauce;

Framework:

Algorithm:

## Sequence of Paper-Folding Instructions

(step by step instructions for each fold)

Problem

Decomposition

Finish the Legs; Finish the Tail;
(UIT2201: Algorithms) Page 8

## Real Life Examples: Issues

Problems/Difficulties:

Imprecise Instructions;
Job can often be done even if instructions are
not followed precisely
Modifications may be done by the person
following the instructions;
But,

## Needs to told PRECISELY what to do;

Instructions must be PRECISE;
Cannot be vague or ambiguous

## LeongHW, SoC, NUS

Definition of Algorithm:
An

## a finite sequence of unambiguous, executable

steps or instructions, which, if followed would
ultimately terminate and give the solution of
the problem.

## Finite set of steps;

Unambiguous;
Executable;
Terminates;
(UIT2201: Algorithms) Page 10

## LeongHW, SoC, NUS

Algorithm Examples?

## Problem 1: What is the largest integer

INPUT: All the integers { -2, -1, 0, 1, 2, }
OUTPUT: The largest integer
Algorithm:
Arrange all the integers in a list in decreasing order;
MAX = first number in the list;
Print out MAX;

## WHY is the above NOT an Algorithm?

(Hint: How many integers are there?)

## Problem 2: Who is the tallest women in the world?

Algorithm: Tutorial...
(UIT2201: Algorithms) Page 11

## Example: Adding two (n-digit) numbers

Input: Two positive m-digit decimal numbers
(a and b)

am, am-1, ., a1
bm, bm-1, ., b1
Output: The sum c = a + b

## cm+1, cm, cm-1, ., c1

(Note: In the textbook, it was am-1,,a1,a0 )
(UIT2201: Algorithms) Page 12

How

## do we give the algorithm?

A step-by-step instruction
to a dumb machine
Try

an example:

3492
8157
Imagine you looking at yourself solving it
(UIT2201: Algorithms) Page 13

## Algorithm: Finding sum of A & B

Step 1: Set the value of carry to 0
Step 2: Set the value of i to 1.
Step 3: Repeat steps 4 through 6 until the value of i is > m.
Step 4: Add ai and bi to the current value of carry, to get xi.
Step 5: If xi < 10 then
let ci=xi and reset carry to 0.
Else (* i.e. xi 10 *)

## let ci=xi - 10 and reset carry to 1.

Step 6: Increase the value of i by 1.

## Step 7: Set cm+1 to the value of carry.

Step 8: Print the final answer cm+1, cm, ., c1
Step 9: Stop.
(UIT2201: Algorithms) Page 14

## LeongHW, SoC, NUS

Chapter Outline:
1. Chapter Goals
2. What are Algorithms
3. Expressing Algorithms Pseudo-Code
1.
2.
3.
4.
5.

## Communicating Alg to computer

Pseudo-Code
Primitive Operations and examples
Variables and Arrays
Algorithm C=A+B in pseudo-code

4. Simple Algorithms
5. Recursive Algorithms
6. Time Complexity of Algorithms
(UIT2201: Algorithms) Page 15

To

## Need way to represent the algorithm

Cannot use English
Can

## machine language and

programming languages (Java, Pascal, C)
But, these are too tedious (&technical)
Use

Pseudo-Code

## Mixture of computer language and English

Somewhere in between
precise enough to describe what is meant without
being too tediuos

Examples:
Let c be 0;
Sort the list of numbers in increasing order;

Need

to know both

syntax representation
semantics meaning
(UIT2201: Algorithms) Page 17

## LeongHW, SoC, NUS

Primitive Operations
To

## describe an algorithm, we need some

well-defined programming primitives
Assignment primitive:
assignment statement, input/output statements

Conditional primitive:
if statement
case statement

for loop,
while loop,

Statements

## are executed one-by-one

(UIT2201: Algorithms) Page 18

## LeongHW, SoC, NUS

Conditional Primitives
if

statement

## to take different actions based on condition

Syntax
if (condition)
then (Step A)
else (Step B)
endif

true
Step A

condition?

false
Step B

if (condition)
then (Step A)
endif

Semantics
(UIT2201: Algorithms) Page 19

## Examples -- (if statement)

Let mark be the total-mark obtained
if (mark < 40)
then (print Student fail)
else (print Student pass)
endif

## read in mark (*from the terminal*)

if (mark < 40) then (Grade F)
else if (mark < 50) then (Grade
else if (mark < 60) then (Grade
else if (mark < 70) then (Grade
else if (mark < 80) then (Grade
endif

D)
C)
B)
A);

## Looping Primitive while-loop

the while-loop
loop a variable
number of times

Syntax
while (condition) do
(some sequence
of statements)

condition?

false

true
Some sequence
of statements;

endwhile

Semantics
(UIT2201: Algorithms) Page 21

## First, the for-loop

loop a fixed or (predetermined) number of
times

Syntax
for j a to b do
(some sequence
of statements)

j a;

(j <= b)?

false

true
Some sequence
of statements;

endfor

Semantics

j j+1;

## Exercising the alg: for and while

for j 1 to 4 do
print 2*j;
endfor
print --- Done ---
Output:
2
4
6
8
--- Done ---

j 1;
while (j <= 4) do
print 2*j;
j j + 1;
endwhile
print --- Done ---

Output:
2
4
6
8
--- Done --(UIT2201: Algorithms) Page 23

In

## the computer, each variable is assigned

a storage box
can store one number at any time
eg: sum, j, carry

Arrays:

## Often deal with many numbers

Such as A1, A2, A3, , A100
Store as an array A[1], A[2], , A[100]
we treat each of them as a variable,
each is assigned a storage box
(UIT2201: Algorithms) Page 24

## Algorithm: A = B + C (in pseudo-code)

We can re-write the C=A+B algorithm as follows:
Alg. to Compute C = A + B:
(*sum two big numbers*)
carry 0;
for i 1 to m do
x[i] a[i] + b[i] + carry ;
if (x[i] < 10)
then ( c[i] x[i]; carry 0; )
else ( c[i] x[i] 10; carry 1; )
endfor;
c[m+1] carry;
Print c[m+1], c[m], ., c[1]
(UIT2201: Algorithms) Page 25

## LeongHW, SoC, NUS

Chapter Outline:
1.
2.
3.
4.

Chapter Goals
What are Algorithms
Expressing Algorithms Pseudo-Code
Simple Algorithms
1. Simple iterative algorithms
Computing Sum, Find Max/Min
2. Modular Program Design
3. Divisibility and Prime Numbers

5. Recursive Algorithms
6. Time Complexity of Algorithms
(UIT2201: Algorithms) Page 26

## Note: Store numbers in array A[1], A[2], , A[n]

Sum(A, n);
begin
Sum_sf 0;
k 1;
while (k <= n) do
Sum_sf Sum_sf + A[k];
k k + 1;
endwhile
Sum Sum_sf;
Print Sum is, Sum
end;
(UIT2201: Algorithms) Page 27

Input:

## A[1] A[2] A[3] A[4] A[5] A[6]

2
5
10
3
12
24

Processing:

Output:

k
?
1
2
3
4
5
6
6

Sum-sf
0
2
7
17
20
32
56
56

n=6

Sum
?
?
?
?
?
?
?
56

Sum is 56
(UIT2201: Algorithms) Page 28

## We can also use a while-loop instead of a for loop.

Sum(A, n);
(* Find the sum of A1, A2,, An. *)
begin
Sum_sf 0;
for k 1 to n do
Sum_sf Sum_sf + A[k];
endfor
Sum Sum_sf;
Print Sum is, Sum
end;

## HW: (a) Note the differences

(b) Modify it to compute the average?
(UIT2201: Algorithms) Page 29

## Note the three stages:

1. Initialization
Set some values at the beginning

2. Iteration
This is the KEY STEP
Where most of work is done

3. Post-Processing or Cleanup

## Can use this setup for other problems

Calculating average, sum-of-squares
Finding max, min; Searching for a number,
(UIT2201: Algorithms) Page 30

## Software are complex

HUGE (millions of lines of code) eg: Linux, Outlook
COMPLEX; eg: Flight simulator

## Idea: Divide-and-Conquer Method

Complex tasks can be divided and each part solved
separately and combined later.

## Modular Program Design

Divide big programs into smaller modules
The smaller parts are
called modules, subroutines, or procedures
Design, implement, and test separately

## Modularity, Abstraction, Division of Labour

Simplifies process of writing alg/programs
(UIT2201: Algorithms) Page 31

## Algorithm to determine if n is prime

Given: A positive integer n
Question: Is n a prime number?

What do we know?
A number n is prime iff
n has no positive factors except 1 and n

## Idea: Express it algorithmically

n is not divisible by any positive number k
such that 1 < k < n. or k=2,3,4,,(n-1)

A module Divisible(m,n) to check divisibility
We can use it as a primitive
(UIT2201: Algorithms) Page 32

## Pseudo-Code for Prime:

Prime(n)
(* To determine if n is prime *)
begin
for k 2 to (n-1) do
if Divisible(k,n)
then (Print False & exit)
else (* Do nothing *)
endfor
Print True & Stop
end;

## Note: Prime uses the module Divisible(k,n)

Exercise it with:
Prime (5); Prime (4);
Prime (55); Prime (41);
(UIT2201: Algorithms) Page 33

A

## Given: Two positive integers m and n

Question: Is n divisible by m?
or Algorithm to compute Divisible(m,n)

Divisible(m, n)

Algorithm

## {False, if n is not divisible by m

True,

if n is divisible by m

Idea:

## A positive integer n is divisible by m iff

for some positive integer k n, m*k = n.
(UIT2201: Algorithms) Page 34

## Algorithm (in pseudo-code)

Exercise it with:
Divisible (3, 9);

## Divisible (3, 5);

Divisible (3,101);

Divisible(m,n)
(* To compute if n is divisible by m *)
begin
D false; (* assume false, first *)
for k 1 to n do
if (m*k = n) then (Dtrue; exit-loop)
else (* Do nothing *)
endfor
Divisible D; (* true or false *)
end;
(UIT2201: Algorithms) Page 35

## LeongHW, SoC, NUS

Chapter Outline:
1.
2.
3.
4.
5.
6.

Chapter Goals
What are Algorithms
Expressing Algorithms Pseudo-Code
Simple Algorithms
Recursive Algorithms
Time Complexity of Algorithms
1. Sequential search algorithm
2. Binary search algorithm
3. Analysis of Time Complexity

7. Summary
(UIT2201: Algorithms) Page 36

## Search: sequential search algorithm

List of numbers: A1, A2, A3, ., An
and a query number x;

Given:

## Question: Search for x in the list;

Sequential-Search(A, n, x);
(* Search for x in A1, A2,, An. *)
begin
for k 1 to n do
if (x = A[k])
then (Print Yes; Stop)
else (* do nothing *)
endfor
Print No; Stop
end;
(UIT2201: Algorithms) Page 37

## How fast is it to search for x?

How many comparisons do we need?
Example: 13, 38, 19, 74, 76, 14, 12, 38, 22, 55
When x=14, need 6 comparisons
When x=13, need only 1 comparison BEST CASE
When x=55, need 10 comparisons WORST CASE
When x=5, need 10 comparisons WORST CASE

## In general, given n numbers, A1,,An

Best Case: 1 comparison
Worst Case: n comparisons
Average Case: (n+1)/2 comparisons
(UIT2201: Algorithms) Page 38

Binary Search
If

## the List is sorted, that is

A1 A2 A3 . An

Then,

we can do better,

Binary Search

Imagine

n=100

## After 1 step, size is 50

After 2 steps, size is 25
After 3 steps, size is 12
After 4 steps, size is 6
After 5 steps, size is 3
After 6 steps, size is 1
DONE!!

What

if n=1000? 1,000,000?
(UIT2201: Algorithms) Page 41

## Binary Search Algorithm

Sorted List A1 A2 A3 . An
A number x.
Question: Is x in the list?
Binary-Search(A,n,x);
1. First 1
Last n
2. While ( First Last ) do
mid (first+last) / 2
If ( x = A[mid] )
then (output Yes and Stop)
else If ( x < A[mid] )
then Last mid-1
else If ( x > A[mid] ) then First mid+1
EndWhile
3. Output False and Stop
4. End
Input:

## Time Complexity Analysis

Sequential

Search (Alg):

Worst Case:
n comparisons
Best Case: 1 comparison
Avg Case: n/2 comparisons
Binary

Search (Alg):

## Worst Case: log2 n comparisons

Best Case: 1 comparison
Avg Case: about log2 n comparisons
How

## using mathematical analysis

(UIT2201: Algorithms) Page 43

## LeongHW, SoC, NUS

Complexity of Algorithm

Binary Search

## A Linear Time Algorithm

Algorithm Sum(A,n) -- O(n) time
Algorithm Sequential-Search(A,n,x) O(n) time

Simple Median-Find (T2-Q3)

All-Subsets(A,n)

## LeongHW, SoC, NUS

Chapter Outline:
1.
2.
3.
4.
5.

Chapter Goals
What are Algorithms
Expressing Algorithms Pseudo-Code
Simple Algorithms
Recursive Algorithms
1. Recursion the idea
2. Fibonacci sequence
3. Tower of Hanoi

5. Recursion

## A problem solving method of

decomposing bigger problems into
smaller sub-problems that are identical to itself.

General Idea:
Solve simplest (smallest) cases DIRECTLY
usually these are very easy to solve

## Solve bigger problems using smaller sub-problems

that are identical to itself (but smaller and simpler)

Abstraction:
To solve a given problem, we first assume that we
ALREADY know how to solve it for smaller instances!!

Dictionary definition:
recursion
see recursion
(UIT2201: Algorithms) Page 46

## Definition of Fibonacci numbers

1. F1 = 1,
2. F2 = 1,
3. for n>2, Fn = Fn-1 + Fn-2

## The above is a recursive definition.

Fn is computed in-terms of itself
actually, smaller copies of itself Fn-1 and Fn-2

## Actually, Not difficult:

F3 = 1 + 1 = 2
F4 = 2 + 1 = 3
F5 = 3 + 2 = 5

F6 = 5 + 3 = 8
F7 = 8 + 5 = 13
F8 = 13 + 8 = 21

F9 = 21 + 13 = 34
F10 = 34 + 21 = 55
F11 = 55 + 34 = 89

## 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144,

(UIT2201: Algorithms) Page 47

## Fibonacci Numbers: Recursive alg

Fibonacci(n) (* Recursive, SLOW *)
begin
if (n=1) or (n=2)
then Fibonacci(n) 1 (*simple case*)
else Fibonacci(n) Fibonacci(n-1) +
Fibonacci(n-2)
endif
end;

The
It

## is simple to understand and elegant!

But,

very SLOW
(UIT2201: Algorithms) Page 48

How

slow is it?

F(6)
F(5)

F(4)

F(4)
F(3)
F(2)

F(3)
F(2)

F(2)

F(3)
F(1)

F(2)

F(2)
F(1)

F(1)

## HW: Can we compute it faster?

(UIT2201: Algorithms) Page 49

A

## Given: Three Pegs A, B and C

Peg A initially has n disks, different size, stacked up,
larger disks are below smaller disks

## Problem: to move the n disks to Peg C, subject to

1. Can move only one disk at a time
2. Smaller disk should be above larger disk
3. Can use other peg as intermediate
(UIT2201: Algorithms) Page 50

Tower of Hanoi

## How to Solve: Strategy

Generalize first: Consider n disks for all n 1
Our example is only the case when n=4

## Look at small instances

Of course, just Move disk 1 from A to C

1. Move disk 1 from A to B
2. Move disk 2 from A to C
3. Move disk 1 from B to C
(UIT2201: Algorithms) Page 51

## Tower of Hanoi (Solution!)

General Method:
First, move first (n-1) disks from A to B
Now, can move largest disk from A to C
Then, move first (n-1) disks from B to C

## Try this method for n=3

1. Move disk 1 from A to C
2. Move disk 2 from A to B
3. Move disk 1 from C to B
4. Move disk 3 from A to C

## 5. Move disk 1 from B to A

6. Move disk 1 from B to C
7. Move disk 1 from A to C
(UIT2201: Algorithms) Page 52

## Algorithm for Towel of Hanoi (recursive)

Recursive Algorithm
when (n=1), we have simple case
Else (decompose problem and make recursive-calls)

Hanoi(n, A, B, C);
(* Move n disks from A to C via B *)
begin
if (n=1) then Move top disk from A to C
else (* when n>1 *)
Hanoi (n-1, A, C, B);
Move top disk from A to C
Hanoi (n-1, B, C, A);
endif
end;
(UIT2201: Algorithms) Page 53

## LeongHW, SoC, NUS

Characteristics of Algorithm

Correctness

## Complexity --- time, space (memory), etc

Ease of understanding

Ease of coding

Ease of maintainence

If