Академический Документы
Профессиональный Документы
Культура Документы
Petra University
Faculty of Information Technology
Algorithms
Chapter 1
1 |Page
Introduction
Q: what is the importance of studying algorithms?
1- To know the standard set of important algorithms from
efficiency
3- Compare programs that would not exist without algorithms. 4- The usefulness and importance of algorithms in delivering
analytical skills. Algorithm is the core of computer science and it is relevant to most of science, business, technology, etc. Q: what is an algorithm? Informally, an algorithm is any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values, as output. An algorithm is thus a sequence of computational steps that transform the input into the output. Furthermore: The algorithm could be defined as a sequence of unambiguous instructions for solving a problem to obtain a required output.
Problem
2 |Page
specified carefully.
2- The same algorithm can be represented in several
different ways.
3- Several algorithms for solving the same problem may
exist.
4- Algorithms for the same problem can be based on very
Algorithms can be classified according to: The problem type (sorting, searching, graphs, string matching)
1-
2-
Example 1: Three algorithms that will solve the GCD of 2 integers: The greatest common divisor (GCD) of 2 non-negative & nonboth-zero, denoted by GCD (m, n), Defined as the largest integer that divides both m & n with a remainder zero. Solution: Euclids" proposed an algorithm for solving this problem [Euclids Alg.] GCD (m, n) = GCD (n, m % n). { "(%) mod" is the reminder of division "m" by "n" until n = 0}.
A.
3 |Page
GCD (60, 24): GCD (24, 60%24) GCD (24,12) GCD (12, 24%12) GCD (12,0) = 12
Step1: If n=0 return the value of m as the answer and stop, otherwise proceed to step2. Step2: Divide mod m by n and assign the reminder to r. Step3: Assign the value of n to m & the value of r to n. Step4: Go to step1. We may write the above algorithm in pseudo-code: //Compute GCD (m, n) using Euclid's algorithm //Input: two nun-negative, non-both-zeros integers m & n. //Output: GCD of m & n. While n 0 do r m mod n m n // assign n to m nr return m Consecutive integer checking algorithm computing GCD(m, n):
B.
Step1: Assign the value of min (m, n) to t. Step2: Divide m by t. If the reminder of this division is 0, go to step 3; otherwise go to step 4. Step3: Divide n by t. If the reminder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to step 4. Step 4: Decrease the value of t by 1. Go to step 2.
C.
4 |Page
Step3: find the common factor in the two primes found in step 1 & 2. [If P is a common factors occurring Pn & Pm times in m & n respectively it should be repeated min {Pn, Pm} times]. Step4: compute the product of all common factors and return it as the GCD of given numbers.
The Algorithm for generating consecutive primes not exceeding any given integer "Sieve of Eratosthense" is introduced below:
Algorithm Sieve(n) // Input: An integer n >= 2 // Output: Array L of all prime numbers less than or equal to n For p 2 to n do A[p] p For p 2 to sqtr(n) do If A[p] 0 // p hasnt been eliminated on previous passes J p*p While j n do A[ j ] 0 // mark element as eliminated j j + p
5 |Page
i i + 1 Return L
Homework # 1:
1.
Do some research on al-Khorezmi, the man from whose name the word algorithm is derived? In particular, you should learn what the origins of the words algorithm and algebra have in common.
2.
Euclids algorithm, as presented in Euclids treatise, uses subtractions rather than integer divisions. Write a pseudocode for this version of Euclids algorithm
3.
Implement the 3 algorithm that finds the greatest common divisor (GCD) of 2 non-negative & non-both-zero GCD of to programs in any language.
6 |Page
1- Understand the problem: Before design an algorithm, you should understand the problem completely.
-
- It's helpful to understand how such algorithm works to know its strength & weakness. Especially if you have to choose among several available algorithms.
Understand the problem Decide on: Computational means. Exact against approximately solving. Data structure. Algorithm design techniques.
7 |Page
- The architecture called (RAM) Random Access Machine, its assumption is that instructions are executed one after another. - Algorithms designed to be executed on such machines are called sequential algorithms. Note that the above assumption of RAM does not hold for newer computers that execute operations concurrently in parallel. "Parallel algorithms"
-
solving: - An algorithm is called an exact algorithm when it solves the problem exactly. An algorithm is called approximate when it solves the algorithms approximately.
-
Approximate: Exact:
- In the new world of object oriented programing, data structure are very important for both design & analysis algorithms.
-
It is a general method for mixture of problems from different of computing areas. There are many design techniques including:
8 |Page
Brute force "It is a straight forward technique of solving any problem" usually directly based on the problem statement and definitions of the concepts involved.
Divide and conquer " this technique is probably the best known algorithm design technique and it is achieved according to the following steps:
A. A problem instance is divided into several
smaller instances of the same problem (ideally of the same size). B. The smaller instances are solved typically recursively.
C. Then the solution is obtained from the
Decrease and conquer "This technique is based on exploiting the relationship a solution to given instance and a solution to a smaller instance of the same problem".
Note once such relationship is established, it can be exploited either top down (recursively) or bottom up (without a recursion). Recursion is an approach to problem solving in which the solution to a particular problem depends on solutions to smaller instances of the same problem
9 |Page
Greedy algorithms This technique suggests constructing a solution to an optimization problem through a sequence of steps, each expanding a partially constructed solution obtained so far, until a complete solution to the problem is obtained and reached. Note that on each step, the choice made must be feasible, locally and optimally.
A greedy algorithm is a mathematical process that recursively constructs a set of objects from the smallest possible constituent parts. Greedy algorithms look for simple, easy-to-implement solutions to complex, multi-step problems by deciding which next step will provide the most obvious benefit. Such algorithms are called greedy because while the optimal solution to each smaller instance will provide an immediate output, the algorithm doesnt consider the larger problem as a whole. Once a decision has been made, it is never reconsidered.
Dynamic programming Is an algorithm design technique is like divide and conquer method it solves problems by combining the solutions to subproblems.
Note that divide and conquer algorithms partition the problem into independent subproblems, solve the subproblems recursively, and then combine their solutions to solve the original problem. On the other hand Dynamic programming applicable when the subproblems are not independent, that is when subproblems share subproblems.
Transform and conquer This design method is based on the idea of transformation These methods work as two stage procedures. First in the transformation stage, the problem's instance is modified to, for one reason or another, more amenable (willing) to solution. Then in the second of conquering stage, it is solved.
10 | P a g e
system of linear equations in n can solve such problem by substitution method for solving equations.
6- Methods of specifying an algorithms: Note that once you have designed an algorithm, you need to specify it in some fashion.
-
Also an algorithm in natural language can be written in pseuducode which is mixture of natural language & programming language.
-
Flowchart is a method of expressing an algorithm by collection of representation geometric shapes containing description of an algorithm steps.
7- Proving an algorithm correctness: Note that once an algorithm has been written we have to prove its correctness.
-
It means that the algorithm yields a required result for all the possible input in a finite amount of time [ex. Euclid's Alg.]. Definition
-
8- Analyzing algorithm: - There are two kinds of algorithms efficiency time & space efficiency.
11 | P a g e
Generality: how possible that an algorithm can be used on other machines (sometimes called portability).
9- Coding the algorithm: Algorithms are designed to be implemented on different computers in any language.
-
Q: write an algorithm to find the minimum distance between two elements in an array of elements [using pseuducode]. Solution: // input: array A[0,1, 2, 3., n-1]. // output: minimum distance between two of its elements. Dimn //"can have the value of".
for i = 0 to n-2 do for j = 1 to n-1 do If i j & | A[ i ] A[ j ] | < dimn dimn | A[ i ] A [ j ] | return dimn
12 | P a g e
// Input: A portion of array A[0 ... n 1] between indices l and r (l r) // Output: The index of the largest element in A[l..r] if l = r return l // there is only one element in the array else temp1 MaxIndex (A[l.. (l+r)/2]) //Greatest element in the //1st half of the portion of A temp2 MaxIndex (A[ (l + r)/2 + 1..r]) if A[temp1] A[temp2] return temp1 else return temp2
Homework # 2:
1. Write a pseudocode for an algorithm for finding real roots of equation ax2 + bx + c = 0 for arbitrary real coefficients a, b, and c. (You may assume the availability of the square root function sqrt(x).) 2. Describe the standard algorithm for finding the binary representation of a positive decimal integer a. in English. b. in a pseudocode.
13 | P a g e
1-Sorting: Sorting problems means to rearrange the items of given list in an increasing or decreasing order.
Importance of sorting: The list becomes easier to search. In case of records "contains more than one item of information". We need to choose a piece of information to guide sorting. Key
Example: When we want to sort students records in alphabetical order of names or by student number or by student grads points' average. the chosen piece of information called a KEY.
2-Searching problems:
-
The searching problems deals with finding a given value called a searching key in the list.
14 | P a g e
There are many searching algorithms to choose from, the simplest is sequential search algorithm. Note that some algorithms work faster than others, but requires more memory. Some algorithms are very fast but applicable only to sorted arrays.
3-String processing: -
String is a sequence of character from an alphabet. Text strings comprise letters, numbers, and special characters. Bit strings comprise zeros and ones. String matching: a problem of searching a given word in a text.
4-Graph problems:
-
One of the oldest and most interesting areas in algorithms is graphs algorithms. Graphs can be defined as a collection of points called vertices, some of which are connected by line segments called edges (links). Graphs can be used for modeling wide kinds of real life problems and applications including transportation and common networks, projects scheduling and games. One of the most important criteria in graphs is diameter, (which is the maximum number of links one need to follow to search one web-page from another by the most direct route between them). Example S3 Graph traversal algorithm: how can one visit the points in a network? "Shortest path algorithm what is the best route between two cities." Traveling sales man problem (TSP) is the problem of finding the shortest tour through cities that visits every city exactly once.
15 | P a g e
Graph-coloring problem: assign the smallest number of colors to vertices of a graph and no two adjacent vertices are the same color.
5- Combinatorial problem: -
These problems asks (explicitly or implicitly) to find combinatorial objects such as a permutations, a combinations or a subnet that satisfies a certain constraints such as maximize a value or minimize a cost. In more abstract perspective the (TSP) and graph coloring problem are example of combinatorial problem.
6-Geometric problems:
-
A geometric algorithm deals with geometric objects such as points and lines. Closest-pair problem: given n points in the plane, find the closest pair among them.
7-Numerical problems:
-
Numerical problems are application problems that involve mathematical objects of continuous nature: solving equations and system of equations, computing definite integrals, evaluation functions and so on.
An algorithm for the sorting problem that sorts an array by counting for each of its elements, the number of smaller elements, then uses this information to put the elements in its appropriate position in the sorted array. Homework:
16 | P a g e
1. Consider the algorithm for the sorting problem that sorts an array by counting, for each of its elements, the number of smaller elements and then uses this information to put the element in its appropriate position in the sorted array:
Algorithm comparison counting sort (A [0, 1, 2, , n-1]) // Sort an array by comparison counting // Input: array A [0, 1, .,n-1] of ordered values // Output: array S [0, 1, ., n-1] of A elements sorted in a // non-decreasing order For i 0 to n-1 do Count [i] 0 For i 0 to n-2 do For j = i+1 to n-1 do If A [i] < A [j] Count [j] count [j] + 1 Else count [i] count[i] + 1 For i 0 to n-1 do S [count [i]] A[i] Return S
Homework # 3:
a. Apply the above algorithm to sorting the list 60, 35, 81, 98, 14, 47.
17 | P a g e
* Hamiltonian Circuit (HC): A path of visits all graphs vertices exactly once before returning to the starting vertex. Question: Find and draw the HC for the graph bellow.
13 14 12 20 19 18 1 2 4 3 9 7 8 10 15 11
17
5 6
16
Consider the following map: a. Explain how we can use the graph-coloring problem to color the map so that no two neighboring regions are colored the same.
R Y
G R B G
18 | P a g e
Data structure can be defined as particular scheme of organizing related data items. Data structure can range elementary data types (integers, characters) to data structure (one-dimension-array). Linear data structure: the most elementary data structure is the array & linked list. One-dimension-array is sequence of items of the same data type by specifying value of array index. Usually one-dimensional-arrays are used for implementing matrices
Item [0]
Item [1]
Item [2]
Item [n-1]
Note that in most cases, the index can be integer between of (0 & n-1) or (1 & n). Each array element of an array occurrence one time [this feature distinguishes the array from list regardless where the element is located in the array].
Linear Data Structures: The most important elementary data structures are the array and linked list.
Each element in the array occupies the same amount of computer memory.
19 | P a g e
Arrays are used for implementing a variety data structure as strings [sequence of characters] binary or bit strings which composed of zeros and ones.
Notes: Special pointer: Called null used to indicate the absence of a node's successor.
Single linked list: Each node expects the last one contains a single pointer to the next element.
Item1 Item2 Item3 Itemn-1 null
The time needed to access an element on singly linked list, we start with the first node & Travers the pointer chain until the particular node is reached [disadvantage]. Linked list do not require any pre reservation of computer memory also insertion & deletion can be made efficiently in a linked list by reconnecting a few appropriate pointers We start a linked list with a special node called the header, it contains information such as: the length of the linked list; pointer to the first element and pointer to the linked list's last element.
Double linked list: Every node except 1st & last contain pointers to both its successor & predecessor nodes.
Item1 Item2 Item3 Item n-1 null
Notes:
20 | P a g e
Note that the array and linked list are two main choices in represent APS (Abstract Data Structure) called liner list or list.
List is finite sequence of data items which arranged by certain liner order
The basic operations performed on List are searching, inserting and deleting an element. Stack: is a list in which insertion & deletion can be done only at the end (top). - LIFO
Queue: is a list from which elements are deleted from one end of structure (front), this operation called dequeue.
In queue new elements are added to the other end (rear-this operation called enqueue [FIFO]).
Priority queue is a collection of data items from a totally order universe (integers or real numbers). The most operations done on priority queue is finding its largest element, deleting largest element.
21 | P a g e
Graphs
Definition: A graph is a collection of points in the plane called vertices or nodes; some of them connected by line segment called edges or arcs. Formally Graph G (V, E) is a defined by pair of two sets: a finite set V of items called vertices & set E of pairs of these vertices called edges. A pair of vertices (u, v) is the same pair (v, u) the vertices u & v are adjacent to each other & they are connected by undirected edge (u, v).
Note that if a pair of vertices (u, v) is not the same as the pair (v, u), we say that the edge (u, v) is directed from u to the vertex v.
Example:
a c b a c b
e (A)
e (B)
V = {a, c, b, f, e, d}. E = {(a, c), (c, b), (c, e), (b, f), (e, f), (e, d), (a, d)}.
The following inequality for the number of edges | E| in undirected graph 0 |E| (|V| (|V|-1)) /2 this inequality give us the largest number of edges in the graph if there is an edge connecting each of its vertices
22 | P a g e
with all |V|-1 other vertices [complete graph or fully connected graph].
Graph representation Graphs for computer algorithms can be represented in two main ways: Adjacency matrix of graph with n vertices in nby-n Boolean matrix with one row & one column for each of the graph vertices in which the elements, in which the elements in the row & the column is =1, if there is an edge the ith vertex to the vertex and =0 otherwise.
23 | P a g e
ecdf fbe
Weighted graph or diagraph Defined as a graph with numbers assigned to its edges, these numbers called weights or costs.
The following figure shows: 1) weighted graph 2) Its weight matrix 3) Its adjacency lists.
a c 5 1 7 2 d e 4
a b c d
a 5 1
b 5 7 4
c 1 7 2
d 4 2
a b,5 c,1 b a,5 c,7 d,4 c a,1 b,7 d,2 d b,4 c,2
Paths and cycles A path from vertex u to vertex v of a graph G can be defined as a sequence of adjacent vertices that starts at u and ends at v.
-
- A directed path is a sequence of vertices in which every consecutive pair of the vertices is connected by an edge directed from the vertex listed first to the vertex listed next.
24 | P a g e
The length of a path is total of vertices in a vertex sequence defining the path minus one.
-
Example: a, c, e, f is a directed graph from "a to f in a previous graph. Graph is said to be connected if for every pair of its vertices u & v there is a path from u to v.
-
Example: the following figures A & B has 2 connected components with vertices {a, b, c, d, e} & {f, j, h, i}
a (A) b c e d (B) b e a d
A cycle is a path of positive length that starts & ends at the same vertex and doesn't traverse the same edge more than once.
-
Sub graph: subgraph of a given graph G = (V,E) is a graph G (V, E) such that V V & E E.
Trees:
Tree (free tree) is a connected cyclic graph (Figure A below).
-
Graph that has no cycle, but is not necessary connected is called a forest (Figure B); each of its connected components is a tree. a h
b a b c c f (A) 25 | P a g e d g (B) f d g e i j
Number of edges in a tree is always one less than the number of its vertices. E = V -1
-
Rotated tree On very important property of a tree is that for every 2 vertices in a tree there is always found exactly one path from one of these vertices to the other one.
-
From the above property it is possible to select an arbitrary vertex in a free tree and consider it as the root of the so-called root tree. (figure below)
top. The rooted tree is achieved by placing its root on the
Obvious applications of trees are hierarchies, from file directories to organizational charts of enterprises. a Level (0) i h
Level (1)
c h
b g (A)
e f
Level (2)
If (u, v) is the last edge of the sample path from Level (3) i h the root to vertex the u & v v, u is said to be the parent of v and v is called the child of u.
-
(B)
Vertex with no children is called leaf. Vertex with at least one child called parental.
26 | P a g e
The depth of the vortex v is the length of the length of the simple path form the root to v.
-
Length of the tree is the longest simple path from the root to the leaf.
-
Ordered Trees:
- An ordered tree is a rooted tree in which all the children of each
99
1 2
10
(A)
(B)
binary tree in A. All numbers assigned to each parent at vertex is larger than all numbers in the left sub tree & smaller than all numbers in the right sub tree, called binary search trees. - Binary trees are binary search trees have many applications in CS.
- A binary tree usually implemented for computing purpose by
27 | P a g e
node contains some information associated with the vertex and two pointers to the nodes representing the left and right child of the vertex.
null
12
ull
null
nul l
null
null
10
null
null
a b
null
null
null
null
null
null
null
28 | P a g e
null
null
29 | P a g e
Example:
This way used to represent a set for computing purposes. Other common way used to represent set for computing purpose is to use the list structure to indicate the set's element.
The two principle distinction between sets and list: A. Set can not contain identical items, a list can
B. Set is unordered collection of items; we may change its element's order. List is an ordered collection of items (we may not change the order of items)
-
List: an ordered collection of objects with several operations that can be performed on them is called an abstract data type (ADT). The List, Stack, Queue the Priority Queue and the dictionary are important example of ADT.
30 | P a g e
Homework:
1. Describe how one can implement each of the following operations on an array so that the time it takes does not depend on the arrays size n. a. Delete the ith element of an array (1 i n). b. Delete the ith element of a sorted array (the remaining array has to stay sorted, of course). 2. a. Show the stack after each operation of the following sequence that starts with the empty stack: push(a), push(b), pop, push(c), push(d), pop b. Show the queue after each operation of the following sequence that starts with the empty queue: enqueue(a), enqueue(b), dequeue, enqueue(c), enqueue(d), dequeue 3. a. Let A be the adjacency matrix of an undirected graph. Explain what property of the matrix indicates that: i. The graph is complete. ii. The graph has a loop, i.e., an edge connecting a vertex to itself iii. The graph has an isolated vertex, i.e., a vertex with no edges incident to it. b. Answer the same questions for the adjacency list representation.
31 | P a g e