Algorithms and Data Structures: Priority Queues and Heaps

Priority Queues and Heaps
Page 1
Berner Fachhochschule - Technik und Informatik
Algorithms and Data Structures

Dr. Rolf Haenni Fall 2008
Berner Fachhochschule Technik und Informatik
Rolf Haenni Algorithms and Data Structures
Page 2
Outline
Priority Queues Heaps Heap-Based Priority Queues Bottom-Up Heap Construction
Priority Queues
Page 3
Outline
Priority Queues
Page 4
Priority Queue ADT

A priority queue stores a collection of items An item is a pair (key , element) Characteristic operations:
insertItem(k,e): inserts an item with key k and element e removeMin(): removes the item with the smallest key and returns its element minKey(): returns the smallest key of an item (no removal) minElement(): returns the element of an item with smallest key (no removal)
General operations:
size(): returns the number of items isEmpty(): indicates whether the priority queue is empty
Two distinct items in a priority queue can have the same key
Berner Fachhochschule Technik und Informatik Rolf Haenni Algorithms and Data Structures
Priority Queues
Page 5
Total Order
Keys in a priority queue can be arbitrary objects on which a total order is dened Mathematically, a total order is a binary relation dened on a set K which satises three properties:
Totality: x y or y x, for all x, y K Antisymmetry: x y and y x implies x = y , for all x, y K Transitivity: x y and y z implies x z, for all x, y , z K
Totality implies Reexivity: x
x, for all x K
Examples: or for R, alphabetical or reverse alphabetical order for {A, . . . , Z }, lexicographical order for {A, . . . , Z } , etc.
Priority Queues
Page 6
Comparator ADT
A comparator encapsulates the action of comparing two keys according to a given total order relation A generic priority queue uses an auxiliary comparator (passed as a parameter to the constructor) When the priority queue needs to compare two keys, it uses its comparator Operations (all with Boolean return type):
isLessThan(x,y) isLessThanOrEqualTo(x,y) isEqualTo(x,y) isGreaterThan(x,y) isGreaterThanOrEqualTo(x,y) isComparable(x)
Priority Queues
Page 7
Sequence-Based Priority Queue

There are two ways to implement a priority queue with a sequence (list or vector) Using an unsorted sequence
insertItem(k,e) runs in O(1) time, since we can insert the item at the beginning of the sequence removeMin(), minKey(), minElement() run in O(n) time since we have to traverse the entire sequence to nd the smallest key
Using a sorted sequence

insertItem(k,e) runs in O(n) time, since we have to nd the place where to insert the item removeMin(), minKey(), minElement() run in O(1) time since the smallest key is at the beginning or end of the sequence
Priority Queues
Page 8
UML Diagram: Composition

<<interface>> <<interface>>
Comparator
isLessThan(x,y) isEqualTo(x,y) etc.
BasicSequence
isEmpty() size()
<<interface>>
<<interface>>
<<interface>>
PriorityQueue
insertItem(k,e) removeMin() minKey() minElement() etc.
List
etc.
Position
SortedPQ
list: SinglyLinkedList C: Comparator
UnsortedPQ
list: SinglyLinkedList C: Comparator
SinglyLinkedList
rst: ListNode n: Integer
ListNode
element: Object next: ListNode
Priority Queues
Page 9
UML Diagram: Inheritance

<<interface>> <<interface>>
Comparator
BasicSequence
isEmpty() size()
<<interface>>
<<interface>>
<<interface>>
PriorityQueue
insertItem(k,e) removeMin() minKey() minElement() etc.
List
etc.
Position
SinglyLinkedList
rst: ListNode n: Integer
ListNode
element: Object next: ListNode
SortedPQ
C: Comparator
UnsortedPQ
C: Comparator
Priority Queues
Page 10
Sorting with a Priority Queue

We can use a priority queue to sort a collection of comparable elements
Insert the elements one by one with a series of insertItem(e,e) operations Remove the elements in sorted order with a series of removeMin() operations
Depending on how the priority queue is implemented (sorted or unsorted), this leads to two well-known sorting algorithms
Selection-Sort Insertion-Sort
Priority Queues
Page 11
PQ-Sort Algorithm
Solution in pseudo-code: Algorithm PQSort(S,C ) P new priority queue with comparator C while not S.isEmpty() do e S.removeElement(S.first()) P.insertItem(e,e) while not P.isEmpty() do e P.removeMin() S.insertLast(e) return S
Priority Queues
Page 12
Selection-Sort
Selection-sort is the variation of PQ-sort where the priority queue is implemented with an unsorted sequence The main task is to select the elements from the unsorted sequence in the right order Running time analysis:
Inserting the elements into the priority queue with n insertItem(e,e) operations runs in O(n) time Removing the elements in sorted order from the priority queue with n removeMin() operations takes time proportional to n + . . . + 2 + 1, and thus runs in O(n2 ) time
Selection-sort runs in O(n2 ) time
Priority Queues
Page 13
Insertion-Sort
Selection-sort is the variation of PQ-sort where the priority queue is implemented with an sorted sequence The main task is to insert the elements at the right place Running time analysis:
Inserting the elements into the priority queue with n insertItem(e,e) operations takes time proportional to 1 + 2 + . . . + n, and thus runs in O(n2 ) time Removing the elements in sorted order from the priority queue with n removeMin() operations runs in O(n) time
In general (worst case), insertion-sort runs in O(n2 ) time If the input sequence is in reverse order (best case), it runs in O(n) time
Priority Queues
Page 14
In-Place Sorting
Instead of using external data structures, we can implement insertion- and selection-sort in-place A portion of the input sequence itself serves as priority queue For in-place insertion-sort we
keep the initial portion of the sequence sorted use swapElements(p,q) instead of modifying the sequence
5 5 4 2 2 1
4 4 5 4 3 2
2 2 2 5 4 3
3 3 3 3 5 4
1 1 1 1 1 5
Heaps
Page 15
Outline
Heaps
Page 16
The Heap ADT

A heap is a specialized binary tree storing keys at its nodes (positions) and satisfying the following properties:
Heap-Order: key (p) key (parent(p)), for every node p other than the root Completeness: let h be the height of the tree 1) there are 2i nodes of depth i for i = 0, . . . , h 1 2) at depth h, nodes are lled up from the left
The last node of a heap is the rightmost node of depth h Heap operations:
insertKey(k): inserts a key k into the heap removeMin(): removes smallest key and returns it minKey(): returns the smallest key (no removal)
Heaps
Page 17
Heap Example
Depth 2 5 9 13 10 9 7 8 Last node 12 11 14 9 6 8 11 0 1 2 3 4
17 15 12
A heap storing n keys has height O(log n)
Heaps
Page 18
Insertion to a Heap
The insertion of a key k consists of 3 steps:

Add a new node q (the new last node) Store k at q Restore the heap order property (discussed next)
5 9 7 q
2 5 9 7 1 6
Heaps
Page 19
Upheap
Algorithm upheap restores the heap-order property
Swap k along an upward path from the insertion node Stop when k reaches the root or a node whose parent has a key smaller than or equal to k Since the height of the heap is O(log n), upheap runs in O(log n) time
5 9 7 6
2 1
1 5 9 7 6 2
insertKey(k) runs in O(log n) time
Heaps
Page 20
Removal from a Heap
The removal algorithm consists of 4 steps:

Return the key of the root Replace the root key with the key k of the last node Delete the last node Restore the heap order property (discussed next)
9
5 7
6 Last node
7 5 9 6
Heaps
Page 21
Downheap
Algorithm downheap restores the heap-order property
Swap k along an downward path from the root (always with the smaller key of its children) Stop when k reaches a leaf or a node whose children have keys greater than or equal to k Since the height of the heap is O(log n), downheap runs in O(log n) time
5 9
7 6
5 7 9 6
removeMin() runs in O(log n) time

Heaps
Page 22
Finding the Insertion Node

Starting from the last node, the insertion node can be found by traversing a path of O(log n) nodes
If the last node is a left child, return the parent node While the current node is a right child, go to the parent node If the current node is a left child, go to the right child While the current node is internal, go to the left child
2 5 9 13 10 9 7 8 11 Last node 6 8 Insertion node
Heaps
Page 23
Array-Based Heap Implementation

We can represent a heap with n keys directly by means of an array of length N > n
By-pass the binary tree ADT No explicit position ADT needed
Idea similar to array-based implementation of binary trees

The The The The array cell at index 0 is unused root of the heap is at index 1 left child of the node at index i is at index 2i right child of the node at index i is at index 2i + 1
The insertion operations corresponds to inserting at index n + 1 and thus runs in O(1) time
Heaps
Page 24
Array-Based Heap: Example

2 5 9 13 10 9 7 8 11 Last node 6 8
2 5 6 9 7 11 8 13 10 9 8
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Heap-Based Priority Queues
Page 25
Outline
Page 26

We can use a heap to implement a priority queue by storing a pair item = (key , element) at each node of the heap Running times for dierent implementations
Operation size, isEmpty minElement, minKey insertItem removeMin PQ-Sort Unsorted Sequence 1 n 1 n n2 Sorted Sequence 1 1 n 1 n2 Heap 1 1 log n log n n log n
In the long run, the heap-based implementation beats any sequence-based implementation
Page 27
UML Diagram: Composition

<<interface>> <<interface>> <<interface>>
Tree
etc.
Comparator
BasicSequence
isEmpty() size()
<<interface>>
<<interface>>
<<interface>>
BinaryTree
etc.
Heap
insertKey(k) removeMin() minKey()
PriorityQueue
insertItem(k,e) removeMin() minKey() minElement()
ArrayBinaryTree
T: Array n: Integer
BinaryTreeHeap
T: ArrayBinaryTree C: Comparator
ArrayHeap
A: Array C: Comparator
HeapPQ
H: ArrayHeap C: Comparator
Page 28
Heap-Sort
Heap-sort is the variation of PQ-sort where the priority queue is implemented with a heap Running time analysis:
Inserting the elements into the priority queue with n insertItem(e,e) operations runs in O(n log n) time Removing the elements in sorted order from the priority queue with n removeMin() operations runs in O(n log n) time
In general (worst case), heap-sort runs in O(n log n) time For large n, heap-sort is much faster than quadratic sorting algorithms such as insertion-sort or selection-sort
Page 29
In-Place Heap-Sort
To implement Heap-Sort in-place, use the front of the original array to implement the heap
Step 1: Use the reverse comparator (e.g. instead of ) to build up the heap (by swapping elements) Step 2: Iteratively remove the maximum element from the heap and insert it in front of the sequence
Phase 1: 4 7 2 5 3 4 7 2 5 3 4 4 7 5 4 Phase 2: 3 2 3 5 4 2 3 7 4 5 2 3 4 2 2 3 2 4 5 7 4 3 2 5 7 3 2 2 3 4 5 7 2 3 4 5 7 7 4 2 5 3 7 4 7 4 2 5 3 7 2 4 5 7 5 2 4 3 7 2 4 5 3 7 5 2 4 3 7 2
7 5 2 4 3
Bottom-Up Heap Construction
Page 30
Outline
Page 31
Bottom-up Heap Construction

We can construct a heap storing n = 2h 1 keys using a recursive bottom-up construction with h 1 phases In phase i, pairs of heaps with 2i 1 keys are merged into heaps with 2i+1 1 keys Merging two heaps (and a new key k):
Create a new heap with the root node storing k and with the two heaps as subtrees Perform downheap to restore the heap-order property
2i+11 2 1
i
2 1
Page 32
Recursive Algorithm in Pseudo-Code

Algorithm bottomUpHeap(S) // sequence of keys of length 2h1 if S. isEmpty() then return new heap else k S. elemAtRank(0) S. removeAtRank(0) H1 bottomUpHeap(firstHalf(S)) // 1st recursive call H2 bottomUpHeap(secondHalf(S)) // 2nd recursive call return mergeHeaps(k, H1 , H2 ) // merge the results
In the general case, i.e. if the sequence is not of size n = 2h 1, we can adapt the splitting of the sequence accordingly
Page 33
Example: Phase 1
16
15
12
23
20
25 16 15 4
5 12 6
11 9 23
27 20
15 16 25 5
4 12 11
6 9 23
20 27
Page 34
Example: Phase 2
15 16 25 5
4 12 11
6 9 23
20 27
7 15 16 25 5 4 12 11 6 9
8 20 23 27
4 15 16 25 7 5 12 11 8 9
6 20 23 27
Page 35
Example: Phase 3
4 15 16 25 7 5 12 11 8 9 23 6 20 27
10 4 15 16 25 7 5 12 11 8 9 23 6 20 27
4 5 15 16 25 10 7 12 11 8 9 23 6 20 27
Page 36
Complexity Analysis
The recursive BottomUpHeap algorithm decomposes a problem of size n into two problems of size n1 2
Rule D2 with q = r = 2 (see Topic 2, Page 31)
The running time f (n) of each recursive step is O(log n)

with an array-based sequence implementation, firstHalf() and secondHalf() run in O(1) time the necessary downheap in mergeHeaps runs in O(log n) time
Apply the second column of D2, i.e. bottom-up heap construction runs in O(n) Speeds up the heap construction (Phase 1) of the Heap-Sort algorithm, but not Phase 2

Algorithms and Data Structures: Priority Queues and Heaps

Загружено:

Сведения о документе

Исходное описание:

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Algorithms and Data Structures: Priority Queues and Heaps

Загружено:

Авторское право:

Доступные форматы

Priority Queues and Heaps

Berner Fachhochschule - Technik und Informatik