Chapter 1: Introduction Algorithms and Conventions

Read the Preface of the text. The design and analysis of algorithms is the core subject matter of Computer Science. Given a problem, we want to o o o o o find an algorithm to solve the problem (create) prove that the algorithm solves the problem correctly (validate) prove that we cannot solve the problem any faster (analyse) implement the algorithm test the program (debugging and profiling)

Designing an algorithm for a computational problem involves knowledge of the problem domain, a thorough knowledge of the data structures that are available and suitable, and quite a lot of creativity. This class covers the design of algorithms for various types of problems, as well as a mathematical analysis of those algorithms done independently of any actual computational experiments (a theoretical vs. empirical study). The purpose of the design of algorithms is obvious: one needs an algorithm in order to write a program. Analysis of algorithms is less obviously necessary, but has several purposes: o o Analysis can be more reliable than experimentation. If we experiment, we only know the behavior of a program on certain specific test cases, while analysis can give us guarantees about the performance on all inputs. It helps one choose among different solutions to problems. As we will see, there can be many different solutions to the same problem. A careful analysis and comparison can help us decide which one would be the best for our purpose, without requiring that all be implemented and tested. We can predict the performance of a program before we take the time to write code. In a large project, if we waited until after all the code was written to discover that something runs very slowly, it could be a major disaster, but if we do the analysis first we have time to discover speed problems and work around them. By analyzing an algorithm, we gain a better understanding of where the fast and slow parts are, and what to work on or work around in order to speed it up.

What is an algorithm?
An algorithm is an outline or idea behind a program. Generating an algorithm, itself, is a step-by-step process, getting more specific with each version. It usually starts with a precise statement to solve a problem on a computer but ultimately consists of a sequence of definite instructions to do a certain job. We express algorithms in pseudo-code: something resembling C or Pascal, but with some statements in English rather than within the programming language. It is expected that one could translate each pseudo-code statement to a small number of lines of actual code, easily and mechanically. From the text we see: Definition 1.1 [Algorithm]: An algorithm is a finite set of instructions that, if followed, accomplishes a particular task. In addition, all algorithms must satisfy the following criteria:

1. Input. Zero or more quantities are externally supplied. 2. Output. At least one quantity is produced. (function vs procedure)

3. Definiteness. Each instruction is clear and unambiguous. 4. Finiteness. If we trace out the instructions of an algorithm, then for all cases, the algorithm
terminates after a finite number of steps.

5. Effectiveness. Each instruction must be basic so that it can be carried out, in principle, by a
person using only pencil and paper. It is not enough that each operation be definite as in criterion 3; it also must be feasible.

Design of Algorithms
Devising the algorithm (i.e, method) Expressing the algorithm (computer language) Validating the algorithm (proof of correctness) o translational semantics o operational semantics o denotational semantics o axiomatic semantics

Analysis of Algorithms
Determination of time and space requirements

Implementation and Program Testing

Not in this class

Pseudocode Conventions
See text for these conventions. They should not be all too surprising as they are similar to most highlevel programming syntax rules. 1. 2. 3. Comments: // Blocks: {} Identifiers and Records: Since pseudocode, code is not object-oriented. The use of records is how pre-OO languages grouped information into compound datatypes or structures. Note the use of * to indicate links to other records. Assignment: <variable> := <expression> Booleans: true and false Arrays: A[i,j] Loops: Use of for,while,repeat-until Conditionals: o if<condition> then<statement1>else<statement2 o Case statement (switch) case { : <condition 1> : <statement 1> . . . : <condition n> : <statement n> : else : <statement n+1> }

4. 5. 6. 7.
8.

9. I/O: since pseudocode use read and write rather than specific call

10. Procedures: Algorithm Name (<paramenter list>)

Use of return Algorithm could mean a Function (returned value) or Procedure (no returned value but "side effects" that achieve some purpose). The book is written with the pseudocode. Note that my notes may contain the Pseudocode conventions or I might accidently fall into Java code. For example, most likely there will be times where I will write code and in assignments not have the colon (:=), or sometimes use an arrow. You are graduate students. Please use your own expertise to comprehend simple slips concerning pseudocode .

Example of Algorithms
Suppose you had the SelectionSort identified in the text (Example1.1). Check out how the text will "walk-through" the conceptualization (Algorithm 1.1 and 1.2)through proof of the algorithm (Thm. 1.1). In addition, the OO code would look something like this:

void SelectionSort(Type a[], int n) // Sort the array a[1:n] into nondecreasing order. { for (int i=1; i<=n; i++) { int j = i; for (int k=i+1; k<=n; k++) if (a[k]

Recursive Algorithms

Chapter 1: Recursion
Recursion is a general method of solving problems by reducing them to simpler problems of a similar type.
General Algorithm (problem solving tool) of Divide and Conquer Postpone work (if defined recursively (inductively), work is easy) For a web-oriented example, consider designing a "web-crawler" that will search every hyperlink that is accessible from a specific page. Pseudo-code - yes, I do know that this little example would have a problem with circular links Recursive_URL_search(link) repeat find next_link Recursive_URL_search (next_link) until No_more_links

Three important aspects to always remember in code

consider halt

make sure get to halt must call self

When ... why recursion? If problem itself is recursively defined To describe a backtracking procedure Provability of correctness (induction) Ease of programming Why not?

space efficiency time efficiency

One can always take a recursive program and do it non-recursively. Recursion Example Problem: n factorial n! n! = n * (n-1) * (n-2) * ... * 1 0! = 1! = 2! = 3! = ... 1 1 2 * 1 = 2 3 * 2 * 1 = 6

Iterative procedure: explicit repetition of some process until certain condition is met Algorithm fact(n) { x := n; prod := 1; while (x != 0) do { prod := x * prod; x --; } return prod; }

But note: 0! = 1 1! = 1 * 0! 2! = 2 * 1! 3! = 3 * 2! 4! = 4 * 3! ... n! = n * (n-1)! if n>0

recursive definition (circular definition ...uses itself) (e.g., 5! = 5 * 4!) ...

Recursive procedure Algorithm fact(n){ if (n ==0) then return 1 // recursion must stop! else { x = n-1; // decrement to get to halt y = n * fact(x); //essential to recursion-must call self return y; } }

For any recursive problem one needs to consider two features:

1. Are solutions easy to give for special states (stop state)? (need to identify a "trivial" case) 2. Given a state (not the stop state) are there clear rules for proceeding to a new state that is either a. a stop state b. leads to a stop state
I.e., find a method to solve the "complex" case in terms of a "simpler" case (the conquer is easier if divide)

Exercises: Binary Search, Book page 13, # 5, 6, 7, 8, 12

Exercise 6: The Fibonacci numbers

From David Eppstein's lecture notes at UC Irvine here but don't freak out. We will not get that far into it.
An interesting problem for recursive algorithms is the computation of Fibonacci numbers. It's one you probably wouldn't need to actually solve, but simple enough that it's easy to understand and maybe surprising that there are many different solutions. The Fibonacci story: Leonardo of Pisa was also known as Fibonacci. He invented the Fibonacci numbers while studying the population dynamics of rabbits. More "scientifically", he was interested in many things, including a subject we now know as population dynamics: For instance, how quickly would a population of rabbits expand under appropriate conditions? As is typical in mathematics (and analysis of algorithms is a form of mathematics), we make the problem more abstract to get an idea of the general features without getting lost in detail:

We assume that a pair of rabbits has a pair of children every year. These children are too young to have children of their own until two years later. Rabbits never die.

(The last assumption sounds stupid, but makes the problem simpler. After we have analyzed the simpler version, we could go back and add an assumption e.g. that rabbits die in ten years, but it wouldn't change the overall behavior of the problem very much.)
We then express the number of pairs of rabbits as a function of time (measured as a number of years since the start of the experiment):

F(1) = 1 -- we start with one pair F(2) = 1 -- they're too young to have children the first year F(3) = 2 -- in the second year, they have a pair of children F(4) = 3 -- in the third year, they have another pair F(5) = 5 -- we get the first set of grandchildren

In general F(n) = F(n-1) + F(n-2): all the previous rabbits are still there (F(n-1)) plus we get one pair of children for every pair of rabbits we had two years ago (F(n-2)).
Sounds confusing, but all you have to do is take that equation and provide a recursive algorithm. That is easy!

Induction
We often validate algorithms through proofs by induction on the input size. When algorithms are recursive, it makes them quite easy to prove with induction.