CS2201 DATA STRUCTURES

UNIT IV

UNIT IV HASHING AND SET SYLLABUS: Hashing Separate chaining open addressing rehashing extendible hashing - Disjoint Set ADT dynamic equivalence problem smart union algorithms path compression applications of Set

CHAPTER 1-HASHING Hashing: * Again, a (dynamic) set of elements in which we do search, insert, and delete Linear ones: lists, stacks, queues, Nonlinear ones: trees, graphs (relations between elements are explicit)

Now for the case relation is not important, but want to be efficient for searching (like in a dictionary)! * Generalizing an ordinary array, * direct addressing! An array is a direct-address table

A set of N keys, compute the index, then use an array of size N Key k at k -> direct address, now key k at h(k) -> hashing

* *

Basic operation is in O(1)! To hash (is to chop into pieces or to mince), is to make a map or a transform

Hash Table: * Hash table is a data structure that support * * Finds, insertions, deletions (deletions may be unnecessary in some applications) The implementation of hash tables is called hashing A technique which allows the executions of above operations in constant average time Tree operations that requires any ordering information among elements are not supported findMin and findMax Successor and predecessor Report data within a given range List out the data in order

CS2201 DATA STRUCTURES

UNIT IV

1.1 General Idea: * The ideal hash table data structure is an array of some fixed size, containing the items * * * A search is performed based on key Each key is mapped into some position in the range 0 to TableSize-1 The mapping is called hash function

Unrealistic Solution: * Each position (slot) corresponds to a key in the universe of keys T[k] corresponds to an element with key k If the set contains no element with key k, then T[k]=NULL

* *

Insertion, deletion and finds all take O(1) (worst-case) time Problem: waste too much space if the universe is too large compared with the actual number of elements to be stored E.g. student IDs are 8-digit integers, so the universe size is 108, but we only have about 7000 students

CS2201 DATA STRUCTURES

UNIT IV

Usually, m << N h(Ki) = an integer in [0, , m-1] called the hash value of Ki The keys are assumed to be natural numbers, if they are not, they can always be converted or interpreted in natural numbers. EXAMPLE APPLICATIONS: * * Compilers use hash tables (symbol table) to keep track of declared variables. On-line spell checkers. After prehashing the entire dictionary, one can check each word in constant time and print out the misspelled word in order of their appearance in the document. Useful in applications when the input keys come in sorted order. This is a bad case for binary search tree. AVL tree and B+-tree are harder to implement and they are not necessarily more efficient.

1.2 HASH FUNCTION: * With hashing, an element of key k is stored in T[h(k)]

h: hash function * * * maps the universe U of keys into the slots of a hash table T[0,1,...,m-1] an element of key k hashes to slot h(k) h(k) is the hash value of key k

In programming each program is breakdown into modules, so that no routine should ever exceed a page. Each module is a logical unit and does a specific job modules which inturn will call another module.

Collision:

CS2201 DATA STRUCTURES

UNIT IV

o Problem: collision occurs when two keys may hash to the same slot. o can we ensure that any two distinct keys get different cells? No, if N>m, where m is the size of the hash table

Task 1: Design a good hash function that is fast to compute and can minimize the number of collisions Task 2: Design a method to resolve the collisions when they occur DESIGN HASH FUNCTION: * A simple and reasonable strategy: h(k) = k mod m e.g. m=12, k=100, h(k)=4 Requires only a single division operation (quite fast). * Certain values of m should be avoided e.g. if m=2p, then h(k) is just the p lowest-order bits of k; the hash function does not depend on all the bits Similarly, if the keys are decimal numbers, should not set m to be a power of 10

* *

Its a good practice to set the table size m to be a prime number Good values for m: primes not too close to exact powers of 2 o e.g. the hash table is to hold 2000 numbers, and we dont mind an average of 3 numbers being hashed to the same entry choose m=701

Dealing with String-type Keys: * * Can the keys be strings? Most hash functions assume that the keys are natural numbers * if keys are not natural numbers, a way must be found to interpret them as natural numbers

Method 1: Add up the ASCII values of the characters in the string o Problems: Different permutations of the same set of characters would have the same hash value

CS2201 DATA STRUCTURES

UNIT IV

If the table size is large, the keys are not distribute well. e.g. Suppose m=10007 and all the keys are eight or fewer characters long. Since ASCII value <= 127, the hash function can only assume values between 0 and 127*8=1016

Method 2

o If the first 3 characters are random and the table size is 10,0007 => a reasonably equitable distribution o Problem * Method 3 computes English is not random Only 28 percent of the table can actually be hashed to (assuming a table size of 10,007)

KeySize 1

i =0

Key[ KeySize i 1] * 37 i

involves all characters in the key and be expected to distribute well

The collision can be handled by different collision handling techniques. They are: Separate chaining Open addressing Multiple Hashing

1.3 SEPARATE CHAINING: Like equivalent classes or clock numbers in math Instead of a hash table, we use a table of linked list keep a linked list of keys that hash to the same value

CS2201 DATA STRUCTURES

UNIT IV

Keys: Set of squares Hash function:h(K) = K mod 10

Separate Chaining Operations: * To insert a key K Compute h(K) to determine which list to traverse If T[h(K)] contains a null pointer, initiatize this entry to point to a linked list that contains K alone. If T[h(K)] is a non-empty list, we add K at the beginning of this list. To delete a key K compute h(K), then search for K within the list at T[h(K)]. Delete K if it is found.

Separate Chaining Features: * Assume that we will be storing n keys. Then we should make m the next larger prime number. If the hash function works well, the number of keys in each linked list will be a small constant. Therefore, we expect that each search, insertion, and deletion can be done in constant time. Disadvantage: Memory allocation in linked list manipulation will slow down the program. Advantage: deletion is easy.

* * *

typedef struct list_node *node_ptr; struct list_node { element_type element; node_ptr next; };

CS2201 DATA STRUCTURES

UNIT IV

typedef node_ptr LIST; typedef node_ptr position; /* LIST *the_list will be an array of lists, allocated later */ /* The lists will use headers, allocated later */ struct hash_tbl { unsigned int table_size; LIST *the_lists; }; typedef struct hash_tbl *HASH_TABLE; Initialization routine for open hash table HASH_TABLE initialize_table( unsigned int table_size ) { HASH_TABLE H; int i; /*1*/ if( table size < MIN_TABLE_SIZE ) { /*2*/ error("Table size too small"); /*3*/ return NULL; } /* Allocate table */ /*4*/ H = (HASH_TABLE) malloc ( sizeof (struct hash_tbl) ); /*5*/ if( H == NULL ) /*6*/ fatal_error("Out of space!!!"); /*7*/ H->table_size = next_prime( table_size ); /* Allocate list pointers */ /*8*/ H->the_lists = (position *) malloc( sizeof (LIST) * H->table_size ); /*9*/ if( H->the_lists == NULL ) /*10*/ fatal_error("Out of space!!!"); /* Allocate list headers */ /*11*/ for(i=0; i<H->table_size; i++ ) { /*12*/ H->the_lists[i] = (LIST) malloc ( sizeof (struct list_node) ); /*13*/ if( H->the_lists[i] == NULL ) /*14*/ fatal_error("Out of space!!!"); else /*15*/ H->the_lists[i]->next = NULL; } /*16*/ return H; } Find routine for open hash table position find( element_type key, HASH_TABLE H ) { position p;

CS2201 DATA STRUCTURES

UNIT IV

LIST L; /*1*/ L = H->the_lists[ hash( key, H->table_size) ]; /*2*/ p = L->next; /*3*/ while( (p != NULL) && (p->element != key) ) /* Probably need strcmp!! */ /*4*/ p = p->next; /*5*/ return p; } Insert routine for open hash table void insert( element_type key, HASH_TABLE H ) { position pos, new_cell; LIST L; /*1*/ pos = find( key, H ); /*2*/ if( pos == NULL ) { /*3*/ new_cell = (position) malloc(sizeof(struct list_node)); /*4*/ if( new_cell == NULL ) /*5*/ fatal_error("Out of space!!!"); else { /*6*/ L = H->the_lists[ hash( key, H->table size ) ]; /*7*/ new_cell->next = L->next; /*8*/ new_cell->element = key; /* Probably need strcpy!! */ /*9*/ L->next = new_cell; } } } 1.4 OPEN ADDRESSING: * * Instead of following pointers, compute the sequence of slots to be examined. Open addressing: relocate the key K to be inserted if it collides with an existing key. We store K at an entry different from T[h(K)]. Two issues arise * * * what is the relocation scheme? how to search for K later?

Three common methods for resolving a collision in open addressing Linear probing Quadratic probing

CS2201 DATA STRUCTURES

UNIT IV

Double hashing

Open Addressing Strategy: * * To insert a key K, compute h0(K). If T[h0(K)] is empty, insert it there. If collision occurs, probe alternative cell h1(K), h2(K), .... until an empty cell is found. hi(K) = (hash(K) + f(i)) mod m, with f(0) = 0

1.4.1 LINEAR PROBING: * f(i) =i * cells are probed sequentially (with wrap-around) hi(K) = (hash(K) + i) mod m

Insertion: o Let K be the new key to be inserted, compute hash(K) o For i = 0 to m-1 compute L = ( hash(K) + I ) mod m T[L] is empty, then we put K there and stop.

o If we cannot find an empty entry to put K, it means that the table is full and we should report an error. Linear Probing Example: * * hi(K) = (hash(K) + i) mod m E.g, inserting keys 89, 18, 49, 58, 69 with hash(K)=K mod 10

To insert 58, probe T[8], T[9], T[0], T[1] To insert 69, probe T[9], T[0], T[1], T[2] Primary Clustering:

CS2201 DATA STRUCTURES

UNIT IV

* *

We call a block of contiguously occupied table entries a cluster On the average, when we insert a new key K, we may hit the middle of a cluster. Therefore, the time to insert K would be proportional to half the size of a cluster. That is, the larger the cluster, the slower the performance. Linear probing has the following disadvantages: Once h(K) falls into a cluster, this cluster will definitely grow in size by one. Thus, this may worsen the performance of insertion in the future. If two clusters are only separated by one entry, then inserting one key into a cluster can merge the two clusters together. Thus, the cluster size can increase drastically by a single insertion. This means that the performance of insertion can deteriorate drastically after a single insertion. Large clusters are easy targets for collisions.

1.4.2 QUADRATIC PROBING: * * * f(i) = i2 hi(K) = ( hash(K) + i2 ) mod m E.g., inserting keys 89, 18, 49, 58, 69 with hash(K) = K mod 10

To insert 58, probe T[8], T[9], T[(8+4) mod 10] To insert 69, probe T[9], T[(9+1) mod 10], T[(9+4) mod 10] * Two keys with different home positions will have different probe sequences e.g. m=101, h(k1)=30, h(k2)=29 probe sequence for k1: 30,30+1, 30+4, 30+9 probe sequence for k2: 29, 29+1, 29+4, 29+9

10

CS2201 DATA STRUCTURES

UNIT IV

If the table size is prime, then a new key can always be inserted if the table is at least half empty.

Secondary clustering Keys that hash to the same home position will probe the same alternative cells Simulation results suggest that it generally causes less than an extra half probe per search To avoid secondary clustering, the probe sequence need to be a function of the original key value, not the home position

Type declaration for closed hash tables * enum kind_of_entry { legitimate, empty, deleted }; * struct hash_entry * { * element_type element; * enum kind_of_entry info; * }; * typedef INDEX position; * typedef struct hash_entry cell; * /* the_cells is an array of hash_entry cells, allocated later */ * struct hash_tbl * { * unsigned int table_size; * cell *the_cells; * }; * typedef struct hash_tbl *HASH_TABLE; * * * * * * * * * * * * * * * * * * Routine to initialize closed hash table HASH_TABLE initialize_table( unsigned int table_size ) { HASH_TABLE H; int i; /*1*/ if( table_size < MIN_TABLE_SIZE ) { /*2*/ error("Table size too small"); /*3*/ return NULL; } /* Allocate table */ /*4*/ H = (HASH_TABLE) malloc( sizeof ( struct hash_tbl ) ); /*5*/ if( H == NULL ) /*6*/ fatal_error("Out of space!!!"); /*7*/ H->table_size = next_prime( table_size ); /* Allocate cells */ /*8*/ H->the cells = (cell *) malloc ( sizeof ( cell ) * H->table_size );

11

CS2201 DATA STRUCTURES

UNIT IV

* * * * * *

/*9*/ /*10*/ /*11*/ /*12*/ /*13*/ }

if( H->the_cells == NULL ) fatal_error("Out of space!!!"); for(i=0; i<H->table_size; i++ ) H->the_cells[i].info = empty; return H;

Find routine for closed hashing with quadratic probing position find( element_type key, HASH_TABLE H ) { position i, current_pos; /*1*/ i = 0; /*2*/ current_pos = hash( key, H->table_size ); /* Probably need strcmp! */ /*3*/ while( (H->the_cells[current_pos].element != key ) && (H->the_cells[current_pos].info != empty ) ) { /*4*/ current_pos += 2*(++i) - 1; /*5*/ if( current_pos >= H->table_size ) /*6*/ current_pos -= H->table_size; } /*7*/ return current_pos; } Insert routine for closed hash tables with quadratic probing void insert( element_type key, HASH_TABLE H ) { position pos; pos = find( key, H ); if( H->the_cells[pos].info != legitimate ) { /* ok to insert here */ H->the_cells[pos].info = legitimate; H->the_cells[pos].element = key; /* Probably need strcpy!! */ } } 1.4.3 DOUBLE HASHING: * * To alleviate the problem of clustering, the sequence of probes for a key should be independent of its primary position => use two hash functions: hash() and hash2() f(i) = i * hash2(K) E.g. hash2(K) = R - (K mod R), with R is a prime smaller than m

Double Hashing Example:

12

CS2201 DATA STRUCTURES

UNIT IV

* * *

hi(K) = ( hash(K) + f(i) ) mod m; hash(K) = K mod m f(i) = i * hash2(K); hash2(K) = R - (K mod R),

Example: m=10, R = 7 and insert keys 89, 18, 49, 58, 69

To insert 49, hash2(49)=7, 2nd probe is T[(9+7) mod 10] To insert 58, hash2(58)=5, 2nd probe is T[(8+5) mod 10] To insert 69, hash2(69)=1, 2nd probe is T[(9+1) mod 10] Choice of hash2(): * * Hash2() must never evaluate to zero For any key K, hash2(K) must be relatively prime to the table size m. Otherwise, we will only be able to examine a fraction of the table entries. E.g.,if hash(K) = 0 and hash2(K) = m/2, then we can only examine the entries T[0], T[m/2], and nothing else! One solution is to make m prime, and choose R to be a prime smaller than m, and set hash2(K) = R (K mod R)

Quadratic probing, however, does not require the use of a second hash function likely to be simpler and faster in practice

Deletion in Open Addressing: * Actual deletion cannot be performed in open addressing hash tables otherwise this will isolate records further down the probe sequence

13

CS2201 DATA STRUCTURES

UNIT IV

Solution: Add an extra bit to each table entry, and mark a deleted slot by storing a special value DELETED (tombstone)

Perfect hashing: * * * Two-level hashing scheme The first level is the same as with chaining Make a secondary hash table with an associated hash function h_j, instead of making a list of the keys hashing to the same slot

1.5 REHASHING If the table gets too full, the running time for the operations will start taking too long and inserts might fail for closed hashing with quadratic resolution. This can happen if there are too many deletions intermixed with insertions. A solution, then, is to build another table that is about twice as big (with associated new hash function) and scan down the entire original hash table, computing the new hash value for each (non-deleted) element and inserting it in the new table. As an example, suppose the elements 13, 15, 24, and 6 are inserted into a closed hash table of size 7. The hash function is h(x) = x mod 7. Suppose linear probing is used to resolve collisions. The resulting hash table appears in Figure 5.19. If 23 is inserted into the table, the resulting table in Figure 5.20 will be over 70 percent full. Because the table is so full, a new table is created. The size of this table is 17, because this is the first prime which is twice as large as the old table size. The new hash function is then h(x) = x mod 17. The old table is scanned, and elements 6, 15, 23, 24, and 13 are inserted into the new table. The resulting table appears in Figure 5.21. This entire operation is called rehashing. This is obviously a very expensive operation -- the running time is O(n), since there are n elements to rehash and the table size is roughly 2n, but it is actually not all that bad, because it happens very infrequently. In particular, there must have been n/2 inserts prior to the last rehash, so it essentially adds a constant cost to each insertion.* If this data structure is part of the program, the effect is not noticeable. On the other hand, if the hashing is performed as part of an interactive system, then the unfortunate user whose insertion caused a rehash could see a slowdown. *This is why the new table is made twice as large as the old table.

14

CS2201 DATA STRUCTURES

UNIT IV

Closed hash table with linear probing with input 13,15, 6, 24

Closed hash table with linear probing after 23 is inserted

Closed hash table after rehashing Rehashing can be implemented in several ways with quadratic probing. One alternative is to rehash as soon as the table is half full. The other extreme is to rehash only when an insertion fails. A third, middle of the road, strategy is to rehash when the table reaches a certain load factor. Since performance does degrade as the load factor increases, the third strategy, implemented with a good cutoff, could be best. Rehashing frees the programmer from worrying about the table size and is important because hash tables cannot be made arbitrarily large in complex programs. The exercises ask you to investigate the use of rehashing in conjunction with lazy deletion. Rehashing can be used in other data structures as well. HASH_TABLE rehash( HASH_TABLE H ) { unsigned int i, old_size; cell *old_cells; /*1*/ old_cells = H->the_cells; /*2*/ old_size = H->table_size; /* Get a new, empty table */ /*3*/ H = initialize_table( 2*old_size );

15

CS2201 DATA STRUCTURES

UNIT IV

/* Scan through old table, reinserting into new */ /*4*/ for( i=0; i<old_size; i++ ) /*5*/ if( old_cells[i].info == legitimate ) /*6*/ insert( old_cells[i].element, H ); /*7*/ free( old_cells ); /*8*/ return H; } 1.6 EXTENDIBLE HASHING The case analysed here is where the amount of data is too large to fit in main memory. The main consideration then is the number of disk accesses required to retrieve data. As before, we assume that at any point we have n records to store; the value of n changes over time. Furthermore, at most m records fit in one disk block. We will use m = 4 in this section. If either open hashing or closed hashing is used, the major problem is that collisions could cause several blocks to be examined during a find, even for a well-distributed hash table. Furthermore, when the table gets too full, an extremely expensive rehashing step must be performed, which requires O(n) disk accesses. A clever alternative, known as extendible hashing, allows a find to be performed in two disk accesses. Insertions also require few disk accesses. As m increases, the depth of a B-tree decreases. We could in theory choose m to be so large that the depth of the B-tree would be 1. Then any find after the first would take one disk access, since, presumably, the root node could be stored in main memory. The problem with this strategy is that the branching factor is so high that it would take considerable processing to determine which leaf the data was in. If the time to perform this step could be reduced, then we would have a practical scheme. This is exactly the strategy used by extendible hashing. Let us suppose, for the moment, that our data consists of several six-bit integers. The root of the "tree" contains four pointers determined by the leading two bits of the data. Each leaf has up to m = 4 elements. It happens that in each leaf the first two bits are identical; this is indicated by the number in parentheses. To be more formal, D will represent the number of bits used by the root, which is sometimes known as the directory. The number of entries in the directory is thus 2D. dl is the number of leading bits that all the elements of some leaf l have in common. dl will depend on the particular leaf, and dl D.

16

CS2201 DATA STRUCTURES

UNIT IV

Suppose that we want to insert the key 100100. This would go into the third leaf, but as the third leaf is already full, there is no room. We thus split this leaf into two leaves, which are now determined by the first three bits. This requires increasing the directory size to 3. Notice that all of the leaves not involved in the split are now pointed to by two adjacent directory entries. Thus, although an entire directory is rewritten, none of the other leaves are actually accessed. If the key 000000 is now inserted, then the first leaf is split, generating two leaves with dl = 3. Since D = 3, the only change required in the directory is the updating of the 000 and 001 pointers. See Figure 5.25. This very simple strategy provides quick access times for insert and find operations on large databases. There are a few important details we have not considered. First, it is possible that several directory splits will be required if the elements in a leaf agree in more than D + 1 leading bits. For instance, starting at the original example, with D = 2, if 111010, 111011, and finally 111100 are inserted, the directory size must be increased to 4 to distinguish between the five keys. This is an easy detail to take care of, but must not be forgotten. Second, there is the possibility of duplicate keys; if there are more than m duplicates, then this algorithm does not work at all.

17

CS2201 DATA STRUCTURES

UNIT IV

Extendible hashing: after insertion of 000000 and leaf split

CHAPTER 2 DISJOINT SET ADT

2.1. Equivalence Relations A relation R is defined on a set S if for every pair of elements (a, b), a, b S, a R b is either true or false. If a R b is true, then we say that a is related to b. An equivalence relation is a relation R that satisfies three properties: 1. (Reflexive) a R a, for all a S. 2. (Symmetric) a R b if and only if b R a. 3. (Transitive) a R b and b R c implies that a R c. We'll consider several examples. The <= relationship is not an equivalence relationship. Although it is reflexive, since a <= a, and transitive, since a <= b and b < = c implies a < = c, it is not symmetric, since a <= b does not imply b <= a. Electrical connectivity, where all connections are by metal wires, is an equivalence relation. The relation is clearly reflexive, as any component is connected to itself. If a is electrically connected to b, then b must be electrically connected to a, so the relation is symmetric. Finally, if a is connected to b and b is connected to c, then a is connected to c. Thus electrical connectivity is an equivalence relation. Two cities are related if they are in the same country. It is easily verified that this is an equivalence relation. Suppose town a is related to b if it is possible to travel from a to b by taking roads. This relation is an equivalence relation if all the roads are two-way.

18

CS2201 DATA STRUCTURES

UNIT IV

2.2. The Dynamic Equivalence Problem Given an equivalence relation ~, the natural problem is to decide, for any a and b, if a ~ b. If the relation is stored as a two-dimensional array of booleans, then, of course, this can be done in constant time. The problem is that the relation is usually not explicitly, but rather implicitly, defined. As an example, suppose the equivalence relation is defined over the five-element set { a1, a2, a3, a4, a5}. Then there are 25 pairs of elements, each of which is either related or not. However, the information a1 ~ a2, a3 ~ a4, a5 ~ a1, a4 ~ a2 implies that all pairs are related. We would like to be able to infer this quickly. The equivalence class of an element a S is the subset of S that contains all the elements that are related to a. Notice that the equivalence classes form a partition of S: Every member of S appears in exactly one equivalence class. To decide if a ~ b, we need only to check whether a and b are in the same equivalence class. This provides our strategy to solve the equivalence problem. The input is initially a collection of n sets, each with one element. This initial representation is that all relations (except reflexive relations) are false. Each set has a different element, so that Si Sj =; this makes the sets disjoint. There are two permissible operations. The first is find, which returns the name of the set (that is, the equivalence class) containing a given element. The second operation adds relations. If we want to add the relation a ~ b, then we first see if a and b are already related. This is done by performing finds on both a and b and checking whether they are in the same equivalence class. If they are not, then we apply union. This operation merges the two equivalence classes containing a and b into a new equivalence class. From a set point of view, the result of U is to create a new set Sk = Si U Sj, destroying the originals and preserving the disjointness of all the sets. The algorithm to do this is frequently known as the disjoint set union/find algorithm for this reason. This algorithm is dynamic because, during the course of the algorithm, the sets can change via the union operation. The algorithm must also operate on-line: When a find is performed, it must give an answer before continuing. Notice that we do not perform any operations comparing the relative values of elements, but merely require knowledge of their location. For this reason, we can assume that all the elements have been numbered sequentially from 1 to n and that the numbering can be determined easily by some hashing scheme. Thus, initially we have Si = {i} for i = 1 through n. Our second observation is that the name of the set returned by find is actually fairly abitrary. All that really matters is that find(x) = find(b) if and only if x and b are in the same set. These operations are important in many graph theory problems and also in compilers which process equivalence (or type) declarations. There are two strategies to solve this problem. One ensures that the find instruction can be executed in constant worst-case time, and the other ensures that the union instruction can be executed in constant worst-case time. It has recently been shown that both cannot be done simultaneously in constant worst-case time.

19

CS2201 DATA STRUCTURES

UNIT IV

We will now briefly discuss the first approach. For the find operation to be fast, we could maintain, in an array, the name of the equivalence class for each element. Then find is just a simple O(1) lookup. Suppose we want to perform union(a, b). Suppose that a is in equivalence class i and b is in equivalence class j. Then we scan down the array, changing all is to j. Unfortunately, this scan takes O(n). Thus, a sequence of n - 1 unions (the maximum, since then everything is in one set), would take O( n2) time. If there are (n2) find operations, this performance is fine, since the total running time would then amount to O(1) for each union or find operation over the course of the algorithm. If there are fewer finds, this bound is not acceptable. One idea is to keep all the elements that are in the same equivalence class in a linked list. This saves time when updating, because we do not have to search through the entire array. This by itself does not reduce the asymptotic running time, because it is still possible to perform O( n2) equivalence class updates over the course of the algorithm. 2.3. Basic Data Structure Recall that the problem does not require that a find operation return any specific name, just that finds on two elements return the same answer if and only if they are in the same set. One idea might be to use a tree to represent each set, since each element in a tree has the same root. Thus, the root can be used to name the set. We will represent each set by a tree. (Recall that a collection of trees is known as a forest.) Initially, each set contains one element. The trees we will use are not necessarily binary trees, but their representation is easy, because the only information we will need is a parent pointer. The name of a set is given by the node at the root. Since only the name of the parent is required, we can assume that this tree is stored implicitly in an array: each entry p[i] in the array represents the parent of element i. If i is a root, then p[i] = 0. In the forest in the example, p[i] = 0 for 1 i 8. As with heaps, we will draw the trees explicitly, with the understanding that an array is being used.

To perform a union of two sets, we merge the two trees by making the root of one tree point to the root of the other. It should be clear that this operation takes constant time.

20

CS2201 DATA STRUCTURES

UNIT IV

A find(x) on element x is performed by returning the root of the tree containing x. The time to perform this operation is proportional to the depth of the node representing x, assuming, of course, that we can find the node representing x in constant time. In our routine, unions are performed on the roots of the trees. Sometimes the operation is performed by passing any two elements, and having the union perform two finds to determine the roots. The average-case analysis is quite hard to do. The least of the problems is that the answer depends on how to define average (with respect to the union operation). For instance, in the forest in the example ,we could say that since there are five trees, there are 5 *4 = 20 equally likely results of the next union (as any two different trees can be unioned). Of course, the implication of this model is that there is only a 2/5 chance that the next union will involve the large tree. Another model might say that all unions between any two elements in different trees are equally likely, so a larger tree is more likely to be involved in the next union than a smaller tree. In the example above, there is an 8/11 chance that the large tree is involved in the next union, since (ignoring symmetries) there are 6 ways in which to merge two elements in {1, 2, 3, 4}, and 16 ways to merge an element in {5, 6, 7, 8} with an element in {1, 2, 3, 4}. typedef int DISJ_SET[ NUM_SETS+1 ]; typedef unsigned int set_type; typedef unsigned int element_type; Disjoint set type declaration void initialize( DISJ_SET S ) { int i; for( i = NUN_SETS; i > 0; i-- ) S[i] = 0;

21

CS2201 DATA STRUCTURES

UNIT IV

} Disjoint set initialization routine /* Assumes root1 and root2 are roots. */ /* union is a C keyword, so this routine is named set_union. */ void set_union( DISJ_SET S, set_type root1, set_type root2 ) { S[root2] = root1; } Union (not the best way) set_type find( element_type x, DISJ_SET S ) { if( S[x] <= 0 ) return x; else return( find( S[x], S ) ); } A simple disjoint set find algorithm 2.4. Smart Union Algorithms The unions above were performed rather arbitrarily, by making the second tree a subtree of the first. A simple improvement is always to make the smaller tree a subtree of the larger, breaking ties by any method; we call this approach union-by-size. The three unions in the preceding example were all ties, and so we can consider that they were performed by size. If the next operation were union (4, 5), then the forest will be as below.

Result of union-by-size

Result union

of

an

arbitrary

22

CS2201 DATA STRUCTURES

UNIT IV

Worst-case tree for n = 16 We can prove that if unions are done by size, the depth of any node is never more than log n. To see this, note that a node is initially at depth 0. When its depth increases as a result of a union, it is placed in a tree that is at least twice as large as before. To implement this strategy, we need to keep track of the size of each tree. Since we are really just using an array, we can have the array entry of each root contain the negative of the size of its tree. Thus, initially the array representation of the tree is all -1s. When a union is performed, check the sizes; the new size is the sum of the old. Thus, union-by-size is not at all difficult to implement and requires no extra space. It is also fast, on average. An alternative implementation, which also guarantees that all the trees will have depth at most O(log n), is union-by-height. We keep track of the height, instead of the size, of each tree and perform unions by making the shallow tree a subtree of the deeper tree. This is an easy algorithm, since the height of a tree increases only when two equally deep trees are joined (and then the height goes up by one). Thus, union-by-height is a trivial modification of union-by-size. The following figures show a tree and its implicit representation for both union-by-size and union-by-height.

Code for union-by-height (rank) /* assume root1 and root2 are roots */ /* union is a C keyword, so this routine is named set_union */ void set_union (DISJ_SET S, set_type root1, set_type root2 ) { if( S[root2] < S[root1] ) /* root2 is deeper set */ S[root1] = root2; /* make root2 new root */ else

23

CS2201 DATA STRUCTURES

UNIT IV

{ if( S[root2] == S[root1] ) /* same height, so update */ S[root1]--; S[root2] = root1; /* make root1 new root */ }} 8.5. Path Compression The union/find algorithm, is quite acceptable for most cases. It is very simple and linear on average for a sequence of m instructions (under all models). However, the worst case of O(m log n ) can occur fairly easily and naturally. For instance, if we put all the sets on a queue and repeatedly dequeue the first two sets and enqueue the union, the worst case occurs. If there are many more finds than unions, this running time is worse than that of the quick-find algorithm. Moreover, it should be clear that there are probably no more improvements possible for the union algorithm. This is based on the observation that any method to perform the unions will yield the same worst-case trees, since it must break ties arbitrarily. Therefore, the only way to speed the algorithm up, without reworking the data structure entirely, is to do something clever on the find operation. The clever operation is known as path compression. Path compression is performed during a find operation and is independent of the strategy used to perform unions. Suppose the operation is find(x). Then the effect of path compression is that every node on the path from x to the root has its parent changed to the root.

The effect of path compression is that with an extra two pointer moves, nodes 13 and 14 are now one position closer to the root and nodes 15 and 16 are now two positions closer. Thus, the fast future accesses on these nodes will pay (we hope) for the extra work to do the path compression. set_type find( element_type x, DISJ_SET S ) { if( S[x] <= 0 ) return x; else return( S[x] = find( S[x], S ) ); }

24

CS2201 DATA STRUCTURES

UNIT IV

Code for disjoint set find with path compression The code shows , path compression is a trivial change to the basic find algorithm. The only change to the find routine is that S[x] is made equal to the value returned by find; thus after the root of the set is found recursively, x is made to point directly to it. This occurs recursively to every node on the path to the root, so this implements path compression. When unions are done arbitrarily, path compression is a good idea, because there is an abundance of deep nodes and these are brought near the root by path compression. Path compression is perfectly compatible with union-by-size, and thus both routines can be implemented at the same time. Since doing union-by-size by itself is expected to execute a sequence of m operations in linear time, it is not clear that the extra pass involved in path compression is worthwhile on average. Indeed, this problem is still open Path compression is not entirely compatible with union-by-height, because path compression can change the heights of the trees. Then the heights stored for each tree become estimated heights (sometimes known as ranks), but it turns out that union-by-rank (which is what this has now become) is just as efficient in theory as union-by-size. Furthermore, heights are updated less often than sizes. As with union-by-size, it is not clear whether path compression is worthwhile on average. 8.6. Application of sets: We have a network of computers and a list of bidirectional connections; each of these connections allows a file transfer from one computer to another. Is it possible to send a file from any computer on the network to any other? An extra restriction is that the problem must be solved on-line. Thus, the list of connections is presented one at a time, and the algorithm must be prepared to give an answer at any point. An algorithm to solve this problem can initially put every computer in its own set. Our invariant is that two computers can transfer files if and only if they are in the same set. We can see that the ability to transfer files forms an equivalence relation. We then read connections one at a time. When we read some connection, say (u, v), we test to see whether u and v are in the same set and do nothing if they are. If they are in different sets, we merge their sets. At the end of the algorithm, the graph is connected if and only if there is exactly one set. If there are m connections and n computers, the space requirement is O(n). Using union-by-size and path compression, we obtain a worst-case running time of O(m (m, n)), since there are 2m finds and at most n - 1 unions. This running time is linear for all practical purposes. Disjoint set data structures have lots of applications. For instance, Kruskals minimum spanning tree algorithm relies on such a data structure to maintain the components of the intermediate spanning forest. Another application is maintaining the connected components of a graph as new vertices and edges are added. In both these applications, we can use a disjoint-set data structure, where we maintain a set for each connected component, containing that components vertices. Disjoint-set data structures model the partitioning of a set, for example to keep track of the connected components of an undirected graph. This model can then be used to determine whether two vertices belong to the same component, or whether adding an edge between them would

25

CS2201 DATA STRUCTURES

UNIT IV

result in a cycle. The Union-Find algorithm is used in high-performance implementations of Unification. This data structure is used by the Boost Graph Library to implement its Incremental Connected Components functionality. It is also used for implementing Kruskal's algorithm to find the minimum spanning tree of a graph. UNIT-IV HASHING AND SET PART A 1. Define collision resolution The process of finding another position for the collide record.Various techniques are Separate chaining Open addressing Multiple hashing 2. When does a collision occur in hashing? When 2 key values hash to the same value or position collision occurs.. 3. How will you generate a Hash function? h(k) = k mod m where h(k) is the hash value for the key value k. m is the table size. 4. What is separate chaining? In separate cahining: Instead of a hash table, we use a table of linked list keep a linked list of keys that hash to the same value

5. What are the advantage and disadvantage of separate chaining? * Disadvantage: Memory allocation in linked list manipulation will slow down the program. * Advantage: deletion is easy.

6. What are the Three common methods for resolving a collision in open addressing? Linear probing Quadratic probing Double hashing

7. What are the disadvantages in linear probing? * Linear probing has the following disadvantages: Once h(K) falls into a cluster, this cluster will definitely grow in size by one. Thus, this may worsen the performance of insertion in the future.

26

CS2201 DATA STRUCTURES

UNIT IV

If two clusters are only separated by one entry, then inserting one key into a cluster can merge the two clusters together. Thus, the cluster size can increase drastically by a single insertion. This means that the performance of insertion can deteriorate drastically after a single insertion. Large clusters are easy targets for collisions.

8. How is deletion performed in open addressing? * Actual deletion cannot be performed in open addressing hash tables * * otherwise this will isolate records further down the probe sequence Solution: Add an extra bit to each table entry, and mark a deleted slot by storing a special value DELETED (tombstone)

9. What is rehashing? Rehashing is the process of constructing the hash table double the size of the original hash table. 10. What is an equivalence Relation? An equivalence relation is a relation R that satisfies three properties: (Reflexive) a R a, for all a S. (Symmetric) a R b if and only if b R a. (Transitive) a R b and b R c implies that a R c.

11. What is a disjoint Set? . Each set Si and S j has a different element, so that Si Sj =; this makes the sets disjoint.

12. How is union of 2 sets is performed? To perform a union of two sets, the two trees are merged by making the root of one tree point to the root of the other. It should be clear that this operation takes constant time. Example:

13. What are the different unions performed in a set?

27

CS2201 DATA STRUCTURES

UNIT IV

Union by height Union by size Arbitrary union

14. What are the Applications of Hashing? * Compilers use hash tables (symbol table) to keep track of declared variables. * On-line spell checkers. After prehashing the entire dictionary, one can check each word in constant time and print out the misspelled word in order of their appearance in the document. Useful in applications when the input keys come in sorted order. This is a bad case for binary search tree. AVL tree and B+-tree are harder to implement and they are not necessarily more efficient.

15. What are the applications of sets? Disjoint set data structures have lots of applications. For instance, Kruskals minimum spanning tree algorithm relies on such a data structure to maintain the components of the intermediate spanning forest. Another application is maintaining the connected components of a graph as new vertices and edges are added. In both these applications, we can use a disjoint-set data structure, where we maintain a set for each connected component, containing that components vertices. PART B (BIG QUESTIONS) 1. Explain in detail about Open Addressing. Refer Page No: 8 to 13 2. What is hashing? Explain in Detail about the Separate Chaining. Refer Page No: 5 to 8 3. What is Rehashing and Extendible Hashing. Explain in detail. Refer Page No: 13 to 17 4. What is disjoint set ADT? Explain in detail about the smart union algorithms? Refer Page No: 17 to 21 5. Explain in detail about the path Compression. Refer Page No: 23 to 24

28

CS2201 DATA STRUCTURES

UNIT IV

29