Академический Документы
Профессиональный Документы
Культура Документы
FL J. ENBODY
Department of Computer Science, Michigan State University, East Lansing, Michigan 48823
H. C. DU
Department of Computer Science, University of Minnesota, Minneapolis, Minnesota 55455
A new type of dynamic file access called dynamic hushing has recently emerged. It
promises the flexibility of handling dynamic tiles while preserving the fast access times
expected from hashing. Such a fast, dynamic file access scheme is needed to support
modern database systems. This paper surveys dynamic hashing schemes and examines
their critical design issues.
Permission to copy without fee all or part of this material is granted provided that the copies are not made or
distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and ita
date appear, and notice is given that copying is by permission of the Association for Computing Machinery. TO
copy otherwise, or to republish, requires a fee and/or specific permission.
0 1988 ACM 0360-0300/88/0600-0085 $01.50
a-e
I
n-t n-t n-t P-t
U-7, U-Z
u-z
access time exceeds an acceptable limit. addition, as a file of records changes in size,
Another solution in static hashing is to it should no longer be necessary to remap
develop a new function and remap all the all the records; that is, restructuring, which
records. In this case we could create a new can take a long time, should be kept to a
function that maps into five ranges: a-e, minimum. One way to save restructuring
f-j, k-o, p-t, and u-z. If the function used time is to restructure in an incremental
the first character and the new key mark fashion on the parts of the file.
was added to the original keys (beth, julie,
sue, and wally), the records would be dis-
2. DEFINITIONS AND TERMINOLOGY
tributed evenly, as shown in Figure Id. If
the new key was bob, harry, Sharon, or To make our presentation of dynamic hash-
Wendy, however, we would still have a col- ing more precise, we define some terms.
lision and overflow. A different function Implicit in these definitions is the assump-
would be necessary to handle those keys tion that the volume of data is large and
without overflow; a function to handle all therefore stored in random-access second-
possible name keys would be difficult, if not ary storage devices such as disks.
impossible, to determine.
(1) In this paper we only consider single-
The preceding example shows that if we
key retrieval in which a record is rep-
know the keys in advance, we can often
resented by a key k E K, where K is the
design a static hash function to satisfy our
key domain.
needs. However, in a dynamic environment
in which the keys, particularly the number (2) A file F C K is a collection of records.
of keys, would be unknown, a file of records (3) A bucket (or page) corresponds to one
might need to be restructured many times or several physical sectors of a second-
in order to accommodate the variety of ary storage device such as a disk. Let
keys. The goal of dynamic hashing is to the capacity of a bucket be b records.
design a function and file structure that (4) Since the time for disk accesses is con-
can adapt in response to large, unpredicta- siderably longer than the time for main
ble changes in the number and distribution memory access or internal machine in-
of keys. Tree structures can adapt to such structions, the time taken to respond
changes, but they tend to have longer re- to a query is simply measured in terms
trieval times than hashing schemes. Dy- of the number of distinct disk accesses
namic hashing attempts to be adaptable issued. It is assumed that each disk
while maintaining fast retrieval times. In access will fetch a page (bucket) of data.
(4 (b)
Figure 3. Tries.
EEZ
00 A
01 B
(a)
10
11 l--l=
I I
(b)
Figure 4. Collapsing a trie to a directory.
in page B, and those with bit sequence bits, but creating a minimum depth trie is
1 . . . are placed in page C. If page A over- NP-complete [Comer and Sethi 19771. Dy-
flows, it can be split into two pages, as namic hashing attempts to address both of
shown in Figure 3b. An additional bit is these problems. To begin with, the search
needed to divide the records that were all is shortened by collapsing the trie into a
in A between A and the new page D. Now directory. This directory can then be ad-
*page A contains records beginning with bit dressed by a hash function. If the chosen
sequence 000 . . . , and those with sequence function is uniform, the implied, collapsed
001 * - * will be mapped to page D. trie will be balanced, which, as we shall see,
The trie shown in Figure 3 is very flexible will tend to keep the directory small.
and can grow and shrink as the number of Collapsing a trie to a directory is illus-
records grows and shrinks. The trie, how- trated in Figure 4. The depth of the trie in
ever, must be searched in order to find a Figure 3a was 2 because 2 bits were needed
desired record. If the trie is skewed owing to decide how to distribute the records
to a skewed key distribution, that can lead among the pages. The second bit was not
to long access times. The trie could be needed for page C in the trie, but it is shown
balanced by choosing a different order of to illustrate how the complete directory is
000
w 001
1
0
0
B 010
1
P 9
011
100
101
110
111
(b)
(a)
Figure 5. Doubling the directory.
formed. Once a complete trie has been Now consider the situation in which
formed as in Figure 4a, the bits on the page A overflows and an additional page D
search path can become indexes to the is added. This solution is similar to what
directory. The trie in Figure 4a can then was done in Figure 3. The trie from
be collapsed into the directory shown in Figure 3b can be filled in, as in Figure 5a,
Figure 4b. The first 2 bits of the keys are and then collapsed into the directory shown
now an index into the directory, so there is in Figure 5b. Note that, since an extra bit
no tree to be searched. There are, however, was required to differentiate between pages
redundant directory entries since two direc- A and D, 3 bits and hence three levels were
tory entries are used for page C. This prob- needed. These extra bits result in a direc-
lem of redundant directory entries is tory with eight entries (23). Of those, many
exacerbated by a skewed trie. If a uniform are redundant. Note that page C has four
key distribution can be found, the trie will directory entries.
be balanced, which will minimize the There is a problem with having a direc-
redundancy. We see later that a hash func- tory: It adds a potentially undesirable level
tion can be used to generate such keys, of indirection that can increase access time.
which in turn can be used as indexes to the A different approach to collapsing the trie,
directory. shown in Figure 6, eliminates the directory.
01 B
3
10 c,
11 G
(4 (b)
The directory can be avoided by maintain- basic dynamic file systems. The dynamic
ing the pages in a contiguous address space. hashing systems can be categorized as
Eliminating the directory, however, also either directory schemes or directoryless
eliminates the indirection allowed by the schemes that correspond directly to the trie
pointers in the directory. Again, consider models presented in the previous section.
page C in the directory scheme in Figure 4. Classifying the systems will give us a basis
The two pointers to page C were reproduced for describing the important issues for dy-
in the directory from the lowest level of the namic schemes.
trie. With the directory removed, the search In order to have a hashed file system that
path in the trie forms the address of the dynamically expands and contracts, we
page rather than the index to the directory. need a hash function whose range of values
This addressing method requires that there can change dynamically. Most of the dy-
be as many pages in the contiguous address namic schemes use a function that gener-
space as there were entries in the directory. ates more key bits as the file expands and
The result is that addresses 10 and 11, fewer key bits as the file contracts. One
which through the indirection of the direc- such hash function can be constructed us-
tory both pointed to page C, now must point ing a series of functions hi(k), i = 0, 1, . . . ,
to two different pages. Thus, the contents which map the key space K into the integers
of page C must be split across two pages, as (0, 1, . . . , 2’ - 1) such that for any k E K,
shown in Figure 6. Now when the trie is either
collapsed, no directory is necessary. The 2
hi(k) = hi-,(k)
bits that previously indexed the directory
can now be used to address the pages them- or
selves, as shown in Figure 6b.
As with the directory scheme, there must hi(k) = hi-l(k) + 2’-‘.
be a way to expand (and contract) the file There are many ways to obtain the func-
dynamically. However, when a page over- tions hi(k). We might typically use a func-
flows in the directoryless scheme, the file tion H(k), which maps the key space into
expands one page at a time rather than random bit patterns of length m, for m
doubling. We defer explanation of this ex- sufficiently large. Then hi(k) may be de-
pansion until the addressing technique has fined as the integers formed by the last i
been properly defined. bits of H(k). Thus, if
H(k) = b,-1 . . . bzblb,,,
5. BASIC DYNAMIC HASHING SCHEMES
where each bi is either 0 or 1, then
In this section we present a general descrip-
tion of a dynamic hash function and two hi(k) = bi-1 *** blbo.
(a) (b)
Figure 7. Extendible hashing scheme. (a) Before split. (b) Split a into a and x. (c) Split z into z
and y.
Note that hi defined using the last i bits pages). Consider a directory scheme first.
is a different hi than was implied in At this dth expansion stage, the record with
Figures 3-6. We used the prefix rather than key k will be found in the primary page
suffix because the intuition of dynamic whose address is stored in the directory at
hashing based on tries is easier to explain index hd(k). With a directoryless scheme,
using the first bits of H(k). However, the hd(k) will be the address of the primary
details of the schemes are easier to under- page itself. If a primary page overflows, we
stand if we define hi using the last bits. may have to look further to find the record.
Therefore, we will use the suffix for the The first scheme we consider has a direc-
remainder of this survey. tory (hash table) with 2d entries (extendible
In the dynamic hashing schemes, the file hashing [Fagin et al. 1979]), as shown in
goes through a series of expansions. We can Figure 7. As mentioned above, hd(k) will be
begin with expansion 0, a file with two a d-bit number that maps to a directory
pages addressed by 1 bit (bo). Doubling the entry that points to the primary data page
size of the tile (to expansion 1) will create that contains the record with key k. An
a file with four pages addressed by bits bl important characteristic of this directory is
and bo. At the start of the dth (full) expan- that several directory entries may point to
sion, the file contains s = 2d directory en- the same primary data page. If exactly two
tries or, for a directoryless scheme, primary entries point to the same page, all records
data pages (i.e., not counting overflow in that page will have the same value in
bits bd+ - - - bl b,; that is, they differ only is, use hd+l(lz), which maps one more bit,
in bit bdel; for example, in Figure 7a, all doubling the number of directory entries.
records in page a will have 00 as their last If this extra bit differentiates between the
2 bits. If four entries point to the same records in page X, those records can be split
page, the records will have the same value between two pages, x and y. This extra bit
in bits bde3 - - . bl b,,, and so on; for example, in the hashing function may not be enough
in Figure 7a, all records in page c will have to divide the records into both pages. We
a 1 as the last bit. may still hash all the records into one page,
To simplify this introduction, we will not which will still overflow. The only solution
allow overflow pages. As a result, if a page is to split again with the hope that the
overflows, a new primary data page must additional bit will be sufficient to differen-
be added. This page can be added in one of tiate among the pages. The probability of
two ways. Assume that page a, which has this situation occurring is smaller with uni-
two pointers pointing to it, overflows as form distributions.
shown in Figure 7a. A new page x will be One problem with splitting a node is
added so that each of those two pointers knowing whether we need to double the
will point to their own page. The records directory when we make a split. This deci-
that were in page a will now be split be- sion can be aided with the addition of
tween the two pages a and x according to another field in each data page and one
their &(k) values. In effect, they will be additional entry for the whole directory.
split using bit ddvl, as shown in Figure 7b. The directory must keep track of the
Remember that hd is defined as the last bits “global depth” d, that is, the number of bits
of H rather than the first, as was indicated needed for addressing. The “local depth” dl
in earlier figures using tries. After the split, stored within each data page is the number
all records in page a have 000 as the last 3 of bits used in addressing that page. There
bits of the hash function H(h). Similarly, are 2d-di pointers to each data page. When
all records in page x have 100 as their last we split page a in Figure 7a, the local depth
3 bits. As a result of the split, there is an was 2 and the global depth was 3. Hence,
implicit buddy system based on the bits of only 2 of the 3 bits being used in the direc-
the hash values. The records in this pair of tory were significant for accessing this
pages have 00 as the last bits of the function page. We can split page a and create two
H(h) and differ in the third from last bit. pages with local depth 3 without doubling
We say the overflowing page split into two the directory. If, however, the global and
pages. In a similar way those two buddy local depths are the same, then all the
pages can be combined into one page if the directory bits are used to access that page.
file were to shrink. This was the case when page x needed to
Consider the case in which page x over- be split, in Figure 7b. If we split such a
flows. This page has only one directory page, the directory must double to get an
entry pointing to it, so it cannot split to extra bit to distinguish among the records
form two buddy pages. An option at this within the page. Notice in Figure 7c that
point would be to use one of the overflow when we double the directory only the local
techniques from static hashing. For exam- depths within the split pages are changed.
ple, we could use separate chaining and add The other pages can be distinguished using
an overflow page with a pointer to it in page the same number of bits as before, so
X. These links, however, can increase re- their local depths remain the same.
trieval time. The solution taken here is to Another problem emerges here from this
increase the number of directory entries so information.
there will be at most one page for every It is through doubling the size of the
directory entry. To do that, the whole di- directory that the file can expand dynami-
rectory must be split, causing the directory cally. In a similar way the file can contract.
to double in size, as shown in Figure 7c. If pairs of pages with single pointers can
This doubling can be accomplished by using map to and tit in single pages, then the file
the function hd+, (k) in place of hd(k); that can shrink by combining those pages and
contracting the directory to half its size. A order. We will give an example to illustrate
page can be combined with the “buddy” this solution.
page that was formed when a page split. Figure 8 shows an expansion of a direc-
After pages are combined, the directory can toryless scheme. We begin with four pages
be contracted when the local depth of all shown in Figure 8a, that is, d = 2. The 2
pages is less than the global depth. As be- bits needed to address those four pages are
fore, the other data pages will not be af- shown in the figure. In this example the file
fected; there will merely be a decrease in will expand whenever any page overflows.
the number of pointers to their data pages. Let page b be the first page to overflow.
The hash function &(k) will become The file will expand by adding page A and
hdml (lz), so the index will be referenced with splitting the records of page a into pages a
one less bit. The directory in Figure 7a, for and A. As before, in order to differentiate
example, could be contracted since only 2 between records for page a and those for
bits are required to differentiate records in page A, an extra bit is needed. Therefore,
any page. h2(k) is still used to address pages b, c, and
The other basic scheme is a directoryless d, but h,(k) is used for pages a and A. This
scheme. We will describe the logical map- situation is illustrated in Figure 8b. Since
ping assuming that the file will be stored in page a rather than page b split, an overflow
contiguous logical address space, as shown page (w ) is needed for page b. After a num-
in Figure 8. In Figure 8a the file is at the ber of insertions, let page d overflow next
beginning of the second expansion (d = 2), (Figure 8b). It is page b’s turn to split, so a
so the tile contains s = 2d = 4 primary data page B is added to the file and the records
pages. In this scheme the directory has been from page b and its overflow page w are
eliminated, so hd(k) will be the address of split between pages b and B. hz(k) is still
the desired primary data page. used to address pages c and d, but hs(k) is
The expansion of this scheme is similar used for pages a, b, A, and B. Assume more
to the previous one in that overflows can insertions, and let page d overflow again.
cause pages to split. It differs, however, in Page C will now be added, and page c will
that the overflowing page need not be the split as shown in Figure 8~. Finally, let page
one that gets split. Instead, pages are split B overflow causing page D to be added and
in sequential order with no regard for page d to split as shown in Figure 8d. Now
whether the page that is split is the one all pages require 3 bits for their addresses;
that actually overflowed. In this directory- that is, all are using function h,(k). This
less scheme, the hash function directly ad- point marks the completion of expansion 2
dresses the pages, so there must be a page and the beginning of expansion 3. Expan-
for each function value, whereas in the sion continues in the same way; contraction
directory scheme there was a directory en- is done in an analogous manner.
try for each function value. The pointers in There are some issues to consider with
the directory scheme provide a level of “in- this scheme. The first is the splittingpolicy.
direction” in the addressing that, for ex- In our example, we split whenever any page
ample, allows fewer pages than directory overflowed. Splitting could be done after a
entries. If we perform an expansion in a fixed number of insertions or overflows, or
directoryless scheme using hd+l, we will after the pages have reached some average
double the number of pages in the same percentage of their total capacity. Another
way in which a directory doubled in the issue is overflow handling. As in static hash-
previous scheme. However, doubling the ing, there are a number of ways to handle
directory in the previous scheme did not overflows. Finally, expansion can be
double the number of pages. Doubling the smoothed out further. In our example, we
number of pages, as opposed to doubling split one page into two pages-the original
the number of directory entries, does not page and a new one. It is possible to arrange
provide the desired smooth growth of the the expansion so a page is split across more
file. The solution is to not double the file than two pages. We examine these issues
but to add pages one at a time in sequential in detail later.
(a) lalb/cldj
00 01 10 11
(4
a b C d A 13 C
I
000 001 010 11 - loo 101 110
I
f
(4
a b C d A B C D
Figure 9 shows the notation used for Since there are s = 2d pages, the new ap-
directoryless schemes, also known as linear pended page will be page s + p. The page p
hushing. It shows a file at the beginning of is split so that the records that previously
the dth expansion, so it will contain s = 2d were in page p will be distributed between
primary data pages and will be addressed pages p and s + p using the hash function
by hd(k). As more records are inserted into hd+l(k). The pointer p will then be incre-
the file, new pages will be added to the end mented by 1. When the file size has doubled
of the file. To determine which page will be (i.e., the split pointer p reaches s), there
split into this new page, we introduce an will be 2d+1 pages in the file. We then
index p. At the start of the dth expansion, increment d and return the split pointer
the pointer p will point to page 0 of the file. to the beginning of the file, and the next
When another primary page is appended to expansion begins.
the file, page 0 (pointed to by p) will be With directory schemes, addressing is
split between page 0 and the new page. trivial. The function hd(k) indexes the
(4
Figure 9. Linear hashing.
4 c
..
0 8- 1 a+fJ
PM
(b)
directory where the pointer is followed to directory scheme without overflows can
the desired page. Directoryless schemes, guarantee two accesses for every request,
however, use both h&) and hd+l(k) to one for the directory and then one for the
access the primary data page. The proce- desired page. A directoryless scheme, on
dure is as follows: the other hand, can keep many requests to
begin
only one access because the directory fetch
if hd(k) 2 p then page := hd(k) has been eliminated. With a directoryless
else page := hd+l (k); scheme, however, overflows are required
if necessary, chase the overflow chain because the pages are split in some prede-
end fined order. The result of having these ov-
erflows is that some requests will require
The basic idea is that to the left of the two or more accesses. This comparison gets
pointer p each page has been split. For each more complicated if one also allows over-
page to the left of p that has been split, flows in a directory scheme. The distinction
there is a corresponding buddy page to the is that directoryless schemes must allow
right of page s - 1; see Figure 9. In order overflows, whereas the directory schemes
to decide which of the corresponding pair may allow overflows.
of pages to access, an extra bit is needed.
The function hd+l (Iz) provides that discrim- 6. SPACE UTILIZATION
inating bit. For records that hash into the
unsplit pages between the pointer p and In this section we deal with the issue of
page s - 1, the function hd(k) is still used. increasing space utilization in the various
Notice that, since hd(k) only uses d bits, it dynamic hashing schemes. Given n records,
cannot reference to the right of s - 1, that w data pages, and b records per page, we
is, the right edge of the file at expansion d. define utilization to be n/(wb); that is, uti-
At the end of the current expansion d, all lization is the ratio of records to space
pages will be addressed by hd+,(k). When reserved for records. A large utilization ra-
we move the pointer p back to point to page tio is desirable because it means that there
0, we will also increment d, so in the next is little wasted space; that is, each page will
expansion the function hd(k) is the same be close to full. There is, however, a price
as the previous expansion’s hd+l (k). to be paid for increasing utilization. Within
Now that both basic schemes have been any scheme, space utilization can be in-
introduced, one of the fundamental differ- creased by modifying the overflow struc-
ences between them can be pointed out. A ture or by adding overflows if they do not
After Split
a2
(a) (b)
Figure 10. Overflow pages are used to increase space utilization. (a) Split
without overflow. (b) Split with overflow.
Before Split
. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . .._.................
a1
(a) lb)
Figure 11. Using overflow pages of smaller size. (a) Split with one small overflow. (b) Split
with two small overflows.
assume that the overflow pages are the which give an indication of the relationship
same size as the primary data pages. If both among utilization, primary page size, and
a primary page and its overflow page are secondary page size for a directoryless
full before splitting, that is, a total of 2b scheme. Two conclusions are apparent
records, then after splitting the utilization from those tables. One is that for low uti-
will be 2b/3b = 66%. This situation is il- lizations (e.g., 80% or less), a wide range of
lustrated in Figure lob. There are three overflow page sizes will work. As the utili-
pages after the split because we expect one zation increases, however, the range of good
of the pages still to be overflowing since an page sizes becomes quite narrow. The other
exact split is not guaranteed. This tech- conclusion is that to achieve high utiliza-
nique yields an improvement over the 50% tion, longer overflow chains will be neces-
per split page without overflows, so we sary. These chains have the undesirable
would expect the overall utilization to im- effect of slowing down access time. How-
prove, too. Further improvement can be ever, a side effect for directory schemes is
expected if we have overflow pages that are that the use of overflows can keep the size
smaller than the primary page. For exam- of the directory down. This improvement
ple, let the overflow page size be b/2. If a is especially noticeable if the distribution
primary page and its overflow page are full of records is not uniform.
with a total of 3 b/2 records, then, when A point to consider with respect to direc-
split, there is a good chance that no over- tory schemes is how large a directory is
flow will be needed, as shown in Figure lla. needed for a given number of keys. Fagin
Those 3b/2 records, spread over two pri- et al. [1979] in their original introduction
mary pages with capacity 2 b, yield a utili- to extendible hashing estimated that 2 .
zation of (3b/2)/2 b = 75%. If an overflow Ilog(k/b In 2)l directory entries would be
were needed after splitting, this utilization needed given n records. They assumed that
could be as low as 60%. If there were two the number of records that fell within an
overflow pages before splitting, however, address range was Poisson distributed and
then, with those 2b records, we would ex- showed that the probability of the local
pect an overflow after splitting with a re- depth differing from the global depth by
sulting utilization of 2b/(5b/2) = 80%, as more than 2 was very small.
shown in Figure llb. A better estimate of the size of the direc-
There are many possible overflow page tory was determined by Flajolet [ 19831. He
sizes, each with a different effect on utili- assumed a Bernoulli model that intuitively
zation. The preceding discussion gives a corresponds to the uniform measure over
feeling for that; more details are available n-tuples of the (infinite) binary sequences
in the tables at the end of Larson [1982], that make up the hash keys. He determined
...
Primary Data Pages I
so-1 so+po
O Pf
M 6-2-l s2+p2
P2
lower level is made of overflows from higher Rather than have overflow pages as sepa-
levels, we expect the lower level files to be rate entities, he allowed the primary page
smaller. In a linear hashing implementa- to expand over a number of contiguous
tion by Ramamohanarao and Sacks-Davis pages. The page then became a “multipage
[ 19841, all records stored at level i + 1 node,” as shown in Figure 13. A fixed upper
whose primary page at level i is q will be limit was placed on the size of the directory,
stored in a single page at level i + 1, and and once that limit was reached it was the
the address of this page is determined by q. nodes and not the directory that expanded.
Records are addressed in recursive linear A record was accessed first by consulting
hashing using the following procedure. The the directory and then by searching the
procedure search(p, 1, k, status), which corresponding multipage node.
searches page p of level 1 for a record with The multipage node can be managed in
key k and returns with status either found a number of ways. The simplest would be
or not-found, is assumed to exist. Assume to keep the records in the same order in
that the current number of levels in the file which they were inserted and perform a
is max-level: sequential search. A more complicated ap-
begin
proach would be to manage each multipage
1 := 0; (* initialize level to zero *) node as a dynamically hashed tile. For ex-
repeat ample, the number of pages in the node
I:= 1+ 1; could be kept in the directory, which would
if&(k) 2 p then page := hd(k) allow one to manage the contiguous multi-
else page := hd+l (12); page node as a directoryless hashing
search(page, 1, k, status); scheme. Each multipage node would grow
until status = found or 1= max-level and shrink as a normal directoryless hashed
end file. One could even allow overflows within
The recursive scheme has been imple- the multipage node directoryless file.
mented as a linear hashing scheme, but a A different multipage node proposal was
directory scheme can be implemented in a made by Ramakrishna and Larson [1988].
similar way. Their composite perfect hashing is a static
hashing scheme that can achieve high uti-
lization without overflows. With no over-
7.3 Multipage Nodes
flows, only one disk access is required, thus
Lomet [1983] proposed a different way of the name “perfect” hashing. Using this
handling overflows in directory schemes. perfect hashing with a directory scheme
01
10
11
preserves the two-disk access bound found details of that idea, however, have not been
in extendible hashing. Expansion of com- published.
posite perfect hashing requires rehashing,
but the amortized cost is kept low by keep-
ing the frequency of expansions small. 8. EXPANSION TECHNIQUES
Dynamic hashing schemes can have oscil-
7.4 Internal Storage of Overflows
latory performance. If the hash function
can distribute the hashed keys uniformly
Some static hashing techniques have been over the pages of the file, the pages fill up
directly applied within the dynamic hash- uniformly and become completely filled al-
ing framework. For example, the contig- most simultaneously. Within a small period
uous address space of directoryless dynamic of further file growth, a majority of file
hashing lends itself to open addressing pages overflow, and their entries must be
mentioned earlier. Larson [1985] proposed split over pairs of pages. The result is that
that open addressing be used for storage of utilization increases from 50 to almost
overflows in a directoryless scheme. Open 100% and then suddenly falls to 50% during
addressing searches the file from the over- the short splitting period. In addition, the
flowing page in a linear fashion to find a cost of doing an insertion is comparatively
nonfull page in which to place the overflow- low at low utilizations but is considerably
ing record. In dynamic hashing the problem higher during the splitting period because
is that pages in the unsplit region will tend so many insertions lead to page splitting.
to need the most overflow pages, and few Finally, if overflow records are required by
nonfull pages can be expected until some the technique, then the frequency of occur-
split pages are reached, for example, to the rence of overflow increases as utilization
right of page s - 1 in Figure 9. The solution approaches lOO%, resulting in a sharp in-
is to order the splitting of pages so that the crease in the cost of insertions and searches
split and unsplit pages will be interleaved as overflow page accesses increase.
throughout the file. That will tend to put Fortunately, random variations in hash-
nonfull split pages (with room for over- ing functions, nonperfect uniformity of the
flows) near the full unsplit pages, which functions, and large files all work to keep
will keep the overflows nearby. Access time those oscillations in check. However, a cou-
is kept reasonable by keeping the overflows ple of techniques have been proposed to
close, which prevents the overflow search explicitly smooth out the expansion. First
from getting too long. we look at one that deals with a way to
Chaining can also be used. Mullin [ 19811 smooth out the expansion given a uniform
proposed a directoryless scheme that stores distribution. The second smooths out the
all overflows in the empty space of primary process by changing the distribution to an
data pages that are not full. These over- exponential one. We then look at directo-
flows are then chained together. Vitter ryless and directory schemes that have
[1982] mentioned that coalesced chaining been designed to handle such nonuniform
may be applicable to dynamic hashing. The distributions.
group 0
0 1 2 3 4 5 6 7 8 9 101112131415
(a)
(a)
group 0
0 1 2 3 4 5 6 7 8 9 101112131415
~~
P
b)
bc
bC+l
Figure 18. Graph of bc versus G.
b”
Figure 19b, the file has expanded, deleting actual pages used for the logical pages of
page 1 and adding pages 3 and 4. The Figure 19. The numbers within the pages
mapping on the right shows how all records are the logical numbers of the pages; the
within the interval (c, c + 1) map into pages numbers outside the pages are the actual
2-4. Shifting the interval (c, c + 1) to the numbers of the pages. In the first expan-
right, changes the mapping so logical page sion, when logical page 1, which resides in
addresses fall within pages 2-4. The param- actual page 0, is mapped into logical pages
eter c is selected so that page 1 can be 3 and 4, the actual page 0 is reused for the
eliminated. The formula to determine the logical page 3. Logical page 4 (physical page
new c is c = logb( first + l), where first is 2) is added to the (right) end of the file. In
the leftmost page, that is, the page to the second expansion, when logical page 2
be eliminated. In Figure 19b, first = 1 so is deleted and logical pages 5 and 6 are
c = log,(l + 1) = 1. A further expansion is added, actual page 1 will be reused for log-
shown in Figure 19c, with page 2 deleted ical page 5. Logical page 6 will be added to
and pages 5 and 6 added, the corresponding the end of the file. An additional expansion
mapping is shown on the right with is shown in Figure 20d: Logical page 3 is
c = log,(2 + 1) = log,3. Contraction is deleted, and logical pages 7 and 8 are added.
performed in an analogous fashion, with The procedure to find the actual page
the interval (c, c + 1) sliding to the left. from the logical page is as follows: If the
One apparent problem in what has been logical page resides in the first instantiation
described so far is that the file moves of the actual page, then the actual page is
through the address space. Unless the space the difference between the last logical page
taken up by the “deleted” pages can be and the current first logical page, that
reused, that space can be wasted in most is, the current number of logical pages.
operating systems. The solution is to map For example, the actual page for logical
these logical pages into actual pages. The page 8 is 8 - 4 = 5. To find a reused page,
result is a file structure that looks like our such as where logical page 7 resides, one
basic directoryless scheme. must search back through the logical in-
Let us return to the previous example to stantiations to find when the actual page
illustrate how this mapping can be done. was instantiated. For example, we know
When a page is “deleted” it should be that logical page 7 is in actual page 0.
reused immediately. Figure 20 shows the That actual address can be determined as
(a) p 3
a
1 f!
g 2
e
c H-1 G
(b)
p 3
a
g 2
e
c c+l
7
T /G
follows: The records in logical page 7 were remember that logical-page = Lb’J:
previously mapped into logical page 3 be-
FUNCTION actual-page (logical page)
fore the last expansion. Before that expan- (* determine immediate “ancestor” pages *)
sion, the records in logical page 3 had high := L(1 + logical-page)/bl
mapped to logical page 1, which resided in low := Llogical-page/bl
the initial actual page 0. Therefore, logical if low < high
page 7 resides in actual page 0. then
The recursive function to determine the (* reused page; keep searching *)
actual page given the logical page follows; actual-page := actual-page(low)
rbl
tial Hashing (Section 7.3) was proposed as
a way of handling nonuniform distributions
by expanding some nodes more than others;
(b) 3 2 4 Figure 13 shows such a distribution. In that
figure there is an exponential distribution
of pages (and records), yet the directory is
fully utilized.
A different way of handling a nonuniform
distribution with a directory is a scheme
presented by Larson [1978]. His approach
is a variation of our basic directory scheme
in that it does not store its directory as a
simple table. Larson proposed a forest of
index tries whose roots are located by hash-
ing to a table. The data pages are leaf nodes.
Each index trie can grow or shrink accord-
ing to the load on the tile. Whenever a leaf
node’s page gets too full, the node can split
000 001 010 011 100 into two. A page is addressed by first using
a subset of the key to hash into the table
Figure 20. Mapping of logical pages to actual pages. to locate the root of the appropriate trie.
The (binary) trie is then searched using the
bit values to determine which branch to
else take. Larson did not let his directory grow
(* low = high; found first instantiation or shrink, only the index tries did that. His
of page *) scheme, however, can be modified to allow
actual-page := logical-page-low a dynamic directory. This modification is
end possible because the directory can be
Mullin [1984] presented a scheme to viewed as a trie collapsed into a table.
unify linear hashing (our basic directoryless Otto [1988] has proposed a method of
scheme), linear hashing with partial expan- implementing an extendible hashing
sions, and spiral storage. By using a piece- scheme using a balanced m-way tree as a
wise linear curve with a shape that is directory. This results in a linear growth of
similar to the curve in Figure 18, he showed the directory even in the case of nonuni-
how to implement linear hashing and linear form distributions.
hashing with partial expansions using the
spiral approach. 9. PHYSICAL IMPLEMENTATION
system do most of the work. Let the dy- (size 2q+1) than the block thatp is currently
namic file system map into a contiguous in (size 2q) and store pages that have al-
logical space and let the operating system ready been split, thus the need for twice as
handle the mapping to a dynamic, possibly many pages. The last directory entry (2’) is
noncontiguous, physical address space. a temporary storage area in which pages
In this section we examine a proposal by currently being split can be stored. Pages
Ramamohanarao and Sacks-Davis [1984] from the block currently containing p are
that considers a mapping of a contiguous split into this temporary storage area, thus
directoryless space to a physical address the need for twice as many pages. The
space that is allocated in quanta greater shaded region to the left of p represents
than single pages, but smaller than the pages currently not being used. They have
whole address space of the file. This map- been split and remapped into the unshaded
ping can be done with a fixed table of region in the left part of the temporary
pointers to contiguous blocks of memory. storage page. The shaded part of the tem-
Figure 21 shows a scheme without partial porary storage page contains the pages that
expansions. Quanta of memory are allo- the rest of the pages in the block currently
cated in blocks with 2” pages for various containing p will be split into. A key part
values of x. At the beginning of a full ex- of this technique is that when a page is split
pansion, the physical blocks will all be the the resulting pair of pages will be stored in
same size; as the dynamic hash file ex- physically adjacent pages.
pands, larger blocks of memory will be used. A closer look at how addressing is done
The figure shows two different size blocks will make the structure clearer. The depth
in use, one with 2q pages and the other with will be i + q, where each block at the current
2 q+l pages. The blocks that are adjacent in level is of size 2q, and there are i directory
the table are adjacent in the logical scheme. entries. Let x be the number of pages to the
The pointer p in Figure 21 is the same split left of p, that is, the shaded region in Figure
pointer shown in Figure 9. Blocks above 21. Since p denotes the number of pages
the split pointer p in Figure 21 are larger that have been split during the current