Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Algorithm & Data Structure I

Algorithms & Data Structure

Fundamentals of Algorithmic Problem Solving

Albohtori Mohammed Salih

Alzaiem Alazhari University

Department of Electrical Engineering (Computer Engineering)

3rd Lecture, February 19, 2020

1
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Overview
▪ Characteristics of an Algorithm
▪ Algorithm Design and Analysis Process
▪ Important Problem Types
▪ Reasons Using Algorithm
▪ Algorithm Complexity
▪ Definitions
▪ Need for Algorithms and Data Structure
▪ General Approaches in Algorithm Design
➢ Randomized Algorithms
➢ Divide-and-conquer Algorithms
➢ Dynamic-programming solutions
➢ Greedy Algorithms
➢ Approximation Algorithms
▪ Analysis of Algorithms
▪ Expressing Algorithms
▪ Basic Terminology
2
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Algorithms
▪ An algorithm: is a sequence of unambiguous
instructions for solving a problem, i.e., for obtaining a
required output for any legitimate input in a finite
amount of time.

Problem

Algorithm

Input Computer Output

3
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Characteristics of an Algorithm
▪ Unambiguous: Algorithm should be clear and unambiguous. Each
of its steps and their inputs/outputs should be clear and must lead
to only one meaning.
▪ Input: An algorithm should have 0 or more well-defined inputs.
▪ Output: An algorithm should have 1 or more well-defined
outputs, and should match the desired output.
▪ Finiteness: Algorithms must terminate after a finite number of
steps.
▪ Independent: An algorithm should have step-by-step directions,
which should be independent of any programming code.

4
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Algorithms
Algorithm Design and Analysis Process:

5
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Algorithms
Algorithm Design and Analysis Process:
▪ Understanding the problem.
▪ Ascertain the capabilities of a computational device.
▪ Choose between exact and approximate problem solving.
▪ Choose a design technique and specify your algorithm.
▪ Prove the correctness of your algorithm.
▪ Analyse your algorithm.
▪ Code your algorithm.

6
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Algorithms
Important Problem Types:
▪ The sorting problem asks us to rearrange the items of a given list.
e.g. in ascending or descending order.
▪ The searching problem deals with finding a given value, called a
search key, in a given set or a multi-set.
▪ Insert, Update or Delete : Algorithms to insert/ update or delete
item in a data structure.
▪ String Processing.
▪ Graph problems.
▪ Geometric algorithms.
▪ Numerical problems.
7
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Reasons Using Algorithm

▪ Efficiency:
✓ Time, cost.
▪ Abstraction:
✓ Complicated problems can be distilled into simpler
ones for which well known algorithms.
▪ Reusability:
✓ Algorithms are often reusable in many different
situations.

8
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Algorithm Complexity
▪ Suppose X is an algorithm and n is the size of input data,
the time and space used by the algorithm X are the two
main factors, which decide the efficiency of X.
▪ Time Complexity:
✓ Time is measured by counting the number of key operations
such as comparisons in the sorting algorithm.
▪ Space Complexity:
✓ Space is measured by counting the maximum memory space
required by the algorithm.

9
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Definitions
▪ An algorithm:
✓ Sequence of steps to be performed in order to solve a
problem by the computer.
▪ A data structure:
✓ Is a collection of data organized in some fashion.

▪ Program Design:
✓ Steps a programmer should do before they start coding the
program in a specific language.
☻in Proper program design helps other programmers to maintain the program
the future.

10
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Need for Algorithms & Data Structure

▪ As applications are getting complex and data rich, there
are three common problems that applications face now-
a-days.
▪ Data Search: Consider data consist of 1 million(106) items. If the
application is to search an item, it has to search an item in 1
million(106) items every time slowing down the search. As data
grows, search will become slower.
▪ Processor speed: Processor speed although being very high,
falls limited if the data grows to billion records.
▪ Multiple requests: As thousands of users can search data
simultaneously on a web server, even the fast server fails while
searching the data.

11
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

General Approaches in Algorithm Design

Many algorithms approach problems in the same way. Thus, it’s
better to classify them based on the approach they employ.
1) Randomized Algorithms:
✓ Randomized algorithms rely on the statistical properties of
random numbers. One example of a randomized algorithm is
quicksort.
2) Divide-and-conquer Algorithms:
✓ This algorithms revolve around 3 steps: divide, conquer and
combine. In the divide step, we divide the data into smaller, more
manageable pieces. In the conquer step, we process each division
by performing some operations on it. In the combine step, we
recombine the processed divisions.
✓ An example of the divide-and conquer algorithm is merge sort.
12
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

General Approaches in Algorithm Design

3) Dynamic-programming solutions:
✓ Dynamic-programming solutions are similar to divide-and
conquer methods in that both solve problems by breaking larger
problems into sub-problems whose results are later recombined.
✓ However, the approaches differ in how sub-problems are related.
In divide-and-conquer algorithms, each sub-problem is independent
of the others.
✓ An example of dynamic-programming is finding the shortest
path to reach a point from a vertex in a weighted graph.

13
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

General Approaches in Algorithm Design

4) Greedy Algorithms:
✓ Greedy algorithms make decisions that look best at the
moment. In other words, they make decisions that are locally
optimal in the hope that they will lead to globally optimal solutions.
✓ It is not give accurate answer to many problems. But when it
works, it will be the fastest method.
✓ One example of a greedy algorithm is Huffman coding, which is
an algorithm for data compression.
5) Approximation Algorithms:
✓ Approximation algorithms are algorithms that do not compute
optimal solutions; instead, they compute solutions that are “good
enough”.

14
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Analysis of Algorithms
Algorithm can be analyzed at two different stages, before
implementation and after implementation.
▪ A Priori Analysis: This is a theoretical analysis of an algorithm.
Efficiency of an algorithm is measured by assuming that all other
factors, for example, processor speed, are constant and have no
effect on the implementation.
▪ A Posterior Analysis: This is an empirical analysis of an
algorithm. The selected algorithm is implemented using
programming language. This is then executed on target
computer machine. In this analysis, actual statistics like running
time and space required, are collected.

15
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Analysis of Algorithms
▪ Analysis of algorithms: is the theoretical study of computer
program performance and resource usage, without use of
specific programming language or implementation.
▪ The practical goal of algorithm analysis is to predict the
performance of different algorithms in order to guide program
design decisions.
▪ Whether we are designing an algorithm or applying one that is
widely accepted, it is important to understand how the
algorithm will perform.
▪ There are a number of ways we can look at an algorithm’s
performance, but usually the aspect of most interest is how fast
the algorithm will run, And storage area occupied by algorithm,
if we interested in its space requirement as well.
16
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Analysis of Algorithms
Worst-Case Analysis:
▪ Most algorithms do not perform the same in all cases;
▪ Normally an algorithm’s performance varies with the data
passed to it.
▪ Typically, three cases are recognized: the best case, average
case and worst case.
▪ Understanding each of these cases is an important part of
analysis because performance can vary significantly between
them.
▪ A basic understanding of how an algorithm performs in all
cases is important, but usually we are most interested in how
an algorithm performs in the worst case.

17
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Analysis of Algorithms
O-notation:
▪ O-notation, also known as Big O-notation, is the most common
notation used to express an algorithm’s performance in a
formal manner.
▪ Formally, O-notation expresses the upper bound is a function
within a constant factor.
▪ Suppose we have an algorithm whose running time is described
by the function T(n) = 3n2 + 10n + 10. Using the rules of O-
notation, this function can be simplified to:
O(T(n)) = O(3n2 + 10n + 10) = O(3n2) = O(n2)

18
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Expressing Algorithms
We can describing algorithms in many different notations:
natural languages, pseudocode, flowcharts.

▪ Natural language: tend to be verbose and

ambiguous (rarely used for complex algorithm).

▪ Pseudocode and flowcharts: are structured ways, avoid

many ambiguities common in natural language
statements.

19
Algorithm & Data Structure I – AAU – Instructor: Albohtori Mohammed

Basic Terminology
▪ Data: Data are values or set of values.
▪ Data Item: Data item refers to single unit of values.
▪ Group Items: Data items that are divided into sub
items are called as Group Items.
▪ Field: Field is a single elementary unit of information
representing an attribute of an entity.
▪ Record: Record is a collection of field values of a given
entity.

20