Sorting Strings and Constructing Digital Search Trees in Parallel
(Extended Abstract)
Joseph F. JEiJEi" Kwan Woo Ryu*it
Dept. of Electrical Eng. Dept. of Computer Eng.
Inst. for Adv. Comp. Studies Kyungpook Natl U., KOREA Inst. for Adv. Comp. Studies
Inst. for Systems Research Inst. for Adv. Comp. Studies University of Maryland
University of Maryland University of Maryland
College Park, MD 20742 College Park, MD 20742
Abstract
W e descrzbe two simple oplimal-work parallel al-
gorithms for sorting a list L = ( X I ,X z , . . . , X,)
8
o ,m strings over an arbitrary alphabet C, where
i = l lX;l = n. The first algorithm is a deterministic
algorithm that runs in O( ,$:I,") time and the sec-
ond is a randomized algorithm that runs zn O(logm)
+
lime. Both algorithms use O(m log m n ) operations.
Compared t o the best known parallel algorithms for
sorting strings, our algorithms offer th.e followin,g im-
provements:
1. The total number of operations used b y our algo-
rithms is optimal while all previous parallel algo-
rith.ms use a n,on-optimal numher of operdions.
2. W e make no assumption about the alphabet while
the previous algorithms assume that the alphabet
is restricted t o { 1 , 2 , . . . , n O ( l ) } .
3. The computation model assumed b y our algo-
rithms is the Common C R C W P R A M unlike
the known algorithms that assume the Arbitrary
CRCW PRAM.
4 . Our algorithms use O ( mlog m + n ) space, while
th.e previous parallel algorith,ms use O(n.l+')
space, where E is a positive consiant.
W e also present optimal-work parallel algorith,ms to
construct a digital search tree for a gzven set of strings
and to search for a string in a sorted list of strings. We
use our parallel sorting algorithms to solve the problem
of determining a minimal starting point of a circular
'Partially supported by the National Science Foundation,
Grant No. CCR-9103135, and by the National Science Founda-
tion Engineering Research Center Program NSFD CD 8803012.
isupported in part by NONDIRECTED RESEARCH
FUND, Korea Research Foundation, 1992.
IPartially supported by NSF grants CCR-8906949 and CCR-
9111348.
0 1994 IEEE
0-8186-5602-6/94
Uzi Vishkint
Dept. of Electrical Eng.
College Park, MD 20742
Tel Aviv University, Israel
string with respect t o lexicographic ordering. Our so-
lution improves upon the previous best known result t o
solve this problem.
1 Introduction
The problem of sorting has been studied extensively
in the literature because of its many important appli-
cations and because of its intrinsic theoretical signif-
icance. Given n elements drawn from a linearly or-
dered set, the sequential time complexity for sorting
these n elements is 0 n log n ) on the comparison tree
b
model. In addition, t ere are several parallel sortin
algorithms that run in O(log n ) time using O ( nlog n!
operations (e.g. [2, IO]).
In this paper, we consider the problem of sort-
ing a set of strings containing a total of n symbols
from an arbitrary alphabet C. More formally, given a
list L = ( X I ,X z , . . . , X,) of m strings over an arbi-
trary alphabet C, where [Xi1 = n, we are sup-
posed to rearrange L into a fexicographically sorted
list of strings. This problem has been considered be-
fore in the case where the input alphabet is restricted
to { 1 , 2 , . . .,no(1)} ([1, 13, 161). In particular, Aho
et. al. [l] provide a sequential algorithm that runs
in O ( n ) time; Hagerup and Petersson [13] provide a
parallel algorithm that runs in O( , ~ ~ ~ , " ,time) us-
ing O ( nlog log n ) operations on the Arbitrary CRCW
PRAM, and JbJb and Ryu [16] provide an improved
parallel algorithm that runs in O(1og n ) time using the
same number of operations on the same model.
The problem of sorting strings can be solved in
O(logm) time using O(n1ogm) operations on the
Common CRCW PRAM by directsly applying the par-
allel sorting algorithm of [lo]. Each comparison of the
algorithm in [lo] is now a comparison between two
strings which can be achieved in constant time and
linear work on the Common CRCW PRAM [12], and
each string participates in at most O(1og m) compar-
isons. However, the total work is not optimal.
We present in this paper two simple optimal-
work parallel algorithms to solve our sorting prob-
349
lem. T h e first is a deterministic algorithm that runs
+
in O( IO " m time using O ( m log m n ) operations.
To achieve these bounds, we make use of the digital
search tree data structure for representing the sorted
strings compactly. We show how to merge such data
structures within a sort-by-merge paradigm to allow
effective application of pipelining. Our second algo-
rithm is a randomized algorithm that runs in O(log m)
time using O ( mlog m + n ) operations. This algorithm
is similar to Reischuk's parallel sorting algorithm [20 .
Both algorithms run on the Common CRCW PRA ,
and use only O(m lo m + n ) memory locations. Note
IJ
that the time compfexities are functions of m only.
Our algorithms improve upon the best known previ-
ous parallel algorithms in four ways:
1. The total number of operations used by our algo-
rithms is optimal.
2. The input alphabet is arbitrary, unlike the as-
sumptions made in [13, 161 where the alphabet is
restricted to { 1 , 2 , . . . , n o ( 1 ) } .
3. The parallel computation model assumed by our
algorithms is weaker. T h e model used by the
algorithms of [13, 161 is the Arbitrary CRCW
PRAM.
4 . Our algorithms use O(m1ogm + n ) space while
those of [13, 161 use O(n'+') space, where 6 is a
positive constant.
Our randomized parallel algorithm is based on an
optimal algorithm for solving the string search prob-
lem. This problem is to find a string Y in a sorted list
of m strings C = (Xi., Xa,.. . . ,X m ) , when 1x11 = . . .=
IX,I = = 1. A naive binary search takes O ( f . l o g m )
time. Kosaraju [18] provides an O(1-) time algo-
rithm. We develop in this paper an O(1ogm 1) time + o(,,',"P,;,) time using ~ ( n operations
sequential algorithm for the case where the input list
is properly preprocessed. We also present a very fast
parallel algorithm to preprocess the input list using a
linear number of operations.
In addition, we provide an algorithm for con-
structing a digital search tree for a sorted list C =
( X l , X z , .. . , X m ) , where lXil = n. The dig-
ital search tree can be deterministically constructed
in O( ,lz:,mm) time using ~ ( log +
mm n ) operations.
Note that the time complexity does not depend on the
lengths of the input strings.
We show how to use our parallel sorting algorithms
to find a minimal starting point (m.s.p. in short)
of a circular string with respect to lexicographic or-
dering. This problem is known to admit a sequen-
tial linear time algorithm [8, 211. When the input
alphabet is restricted to { 1,2, . . . , n o ( ' ) } , Iliopoulos
and Smyth [14] provide a parallel algorithm that runs
in O(1og n log log n ) time using O ( n log log n ) opera-
tions on the Arbitrary CRCW PRAM, and JbJb and
Ryu [16] independently provide a.n improved algo-
rithm that runs in O(1ogn) time using the same num-
ber of operations on the same model. A deterministic
350
parallel algorithm that runs in O( l & ~ ~ ~time
, ) and a
randomized parallel algorithm that runs in O(1og n)
time are presented; both algorithms use O ( n ) opera-
tions and do not impose any restriction on the input
alphabet.
The rest of the paper is organized as follows. We
review in Section 2 the PRAM model and some re-
lated results, and introduce some basic terminology
that will be used in the remainder of this paper. The
deterministic parallel algorithms for sorting and for
constructing digital search trees are presented in Sec-
tions 3 and 4. T h e randomized parallel sorting algo-
rithm is presented in Section 5. Finally, an optimal
parallel algorithm for finding a minimal starting point
of a circular string is presented in Section 6.
2 Preliminaries
6
A PRAM Parallel Random Access Machine) con-
Pi, . . . ,Pp-r, all
sists of p sync ronous processors, PO,
having access to a large shared memory. In a single
cycle, each processor may read or write from or into
a shared memory cell, or perform a local operation.
Each processor Pi, 0 5 i 5 p - 1, is uniquely identified
by an index i which can be referred to in the program.
Let T A ( ~denote ) the .time required by a parallel
algorithm A to solve a problem of size n and let W A ( ~ )
be the total number of operations used. We say that A
I
is work optimal i.f W A n ) is @ ( S e q ( n ) ) where
is the running time o the best sequential algorithm
for solving the same problem.
, Seq(n)
The following known results on the PRAM will be
used throughout the paper. Consider a sequence of n
integers ( a l ,a ~. .,. , an) each of length O(1ogn). The
prefix sums of this sequence are the n partial sums
d e f i n e d b y s i = a l + a z + . . . + a ; , l < i < n . Thenwe
have the following lemma.
Lemma 2.1 [11] The prefix sums of a sequence of
n integers each of length O(1ogn) can be found in
) on the Com-
mon CRCW PRAM. 0
A slight modification of the above prefix sums al-
gorithm can be used to solve the following seg-
mented prefix sums problem. Given a sequence
A = a l , a z , . . . , a n ) of n integers each of length
O(logn\ and a Boolean array B of length n such that
bl = b, = 1; for each il < i2 such that bi, = bi, = 1
and bj = 0 for all il < j < iz, we wish to compute the
prefix sums of the subarray (ai,+l, . . . , ai3) of A .
Corollary 2.1 The segmented prefix sums of a se-
quence of n integers each of length O(1ogn) can be
found in O(,d,"P,;,) time using O ( n ) operations on
the Common CRCW P R A M . 0
The following four lemmas state known results for
finding the first 1 in a Boolean array, solving the all
nearest one bit problem, merging two sorted lists, and
sorting a list of n elements respectively.
Lemma 2.2 [12] Given a Boolean array B[1 : n] of
size n , the index i such that B[i] = 1 and B[j] = 0 ,
f o r all 1 5 j 5 i - 1, can be found in 0 1) time using
O(n) operations on the Common C R C PRAM. 0 6 X and Y differ). Sorting the list of input strings im-
Lemma 2.3 [5] Given a Boolean array B 1 : n] =
I,
( b l , bz, . . . , b,) of size n , f o r all 1 5 i 5 n , t e nearest
one bit of bi t o its lefi can be found in O(cr(n))time
using O ( n ) operations, where cr denotes the extremely
slowly increasing uinverse-Ackermann” function. 0
Lemma 2.4 [19, 61 Given two sorted lists ofm and
n elements respectively drawn from a linearly ordered
set, the two lists can be merged in O(loglog(m n ) ) + alphabet C, and is directed away from the root.
time using O ( n + m ) operations on the C R E W P R A M .
0 2. No two edges emanating from the same node have
Lemma 2.5 [lo] Given a list of n elements drawn
from a linearly ordered set, the list can be sorted
in O(1ogn) time using O(n1ogn) operations on the
E R E W PRAM. 0
We now introduce the basic terminology needed
for describing our operations on strings. Let C be
an alphabet consisting of an arbitrary set of sym-
bols. Assume that C is equipped with a total order
4 that can be evaluated in constant time by a single
processor for any given pair of symbols from E. A
string X = ( a , , a2, .. . ,
denoted by 1x1.
h is a finite tuple of sym-
bols from C, where k is t e length of X which is
Let X [ i ] be the i-th symbol of X I
i.e., X [ i ] = a,, where 1 5 i 5 1x1.For any pair of
indices i and j such that 1 5 i , j 5 k , a consecutive
portion of X , X[i : j ] = ( a i , a i + l , . . . , U , ) , defines a
substring of X . Note that X i : j ] is the empty
string 6 if i > j , while X [ i :, j \ is a nontrivial sub-
<
string of X if i j . A substring defined by X[l : i],
for some i such that 1 5 i 5 k , is called a pre-
fix of X ,whereas a substring of the form X[j : IC
for some j such that 1 5 j 5 t , is called a SUI:
fix of X. If X = ( a l l . . . , a k ) and Y = ( b l , . . . , b l )
are strings, the concatenation of X and Y is the
string X Y = ( ~ 1 , .. . , a k , b l , . . . , bl). Let Xi be the
string consisting of X concatenated with itself i times,
for i 2 0. Then X o = 6,and X’ = X X ’ - ’ , for
i > 0. For a string X = ( 0 1 , a 2 , . . . , a k ) , we consider
at+l = u k + 2 = . . . = #, where # is a symbol not oc-
curring in C (“blank symbol”), and extend the total
order 4 to the alphabet C U {#} such that # 4 U ,
Vu E c.
Given a list of strings C = ( X I , X?, . . ., Xm), let
ILL= m be the number of striiigs i n C, and let
= n be the total number of symbols i n the
ithout loss of generality, w e assume that none
of the strings is the empty string 6 and that the input
strings are pairwise distinct since we can add a unique
marker to the end of each string. Hence, for any two
distinct strings X = ( ~ 1 , .. . , a k hand Y = l b l , . . . , bl)
of C,there is a position j in whic X and Y iffer. The
total order on C induces a lexicographic order on
the list of strings as follows. When x = ( a l , . . . , a t )
and Y = ( 6 1 , . . ., b,) are any two distinct strings, we
say that X 4 Y if and only if ai = b i , for 1 5 i < j ,
and aj 4 bj ( j is the most significant position in which
351
plies rearranging the strings in a sorted list with re-
spect to the lexicographic ordering.
One of our algorithms makes use of a slight varia-
tion of the digital search tree as defined, for exam-
ple, in [15]. Let C = ( X I ,X 2 , . . . , X m )be a sorted list
of distinct strings over the alphabet C. Assume that
no X i is a prefix of some X j , whenever i # j . We
define the digital search tree T associated with C to
be an ordered rooted tree with m leaves such that
1. Each edge of T is labeled with a symbol from the
the same symbol as their labels.
3. All the edges from each node appear in sorted
order with respect to their labels.
4. Each leaf node U is uniquely identified with a
string X i , in the sense that the concatenation of
the labels on the path from the root to U is X i .
Notice that condition 3 of our definition does not ap-
pear in the usual definition of digital search trees and
is needed here to describe our deterministic sorting al-
gorithm. Since the reconstruction of the sorted list of
the input strings from the corresponding digital search
t,ree can be done quite easily, we can define the sort-
ing problem as follows: Given a list of strings over an
arbitrary alphabet, construct the digital search tree
associated with the corresponding sorted list. We use
this definition in the next section to obtain the opti-
mal parallel algorithm when all the input strings are
of the same length 1 and 1 is small ( 1 5 logm). We
will then generalize this algorithm to sort strings of
different lengths.
3 A Deterministic Sorting Algorithm
In this section, we present a simple opti-
mal algorithm that lexicographically sorts a list
L = ( X 1 , X 2 , . . .,X,) of m input strings, where
lX,l= n. This algorithm runs in O(l$IEm)
time using O(m log m + n ) operations. Note that the
time complexity depends only on m. During the ex-
ecution of our algorithm, digital search trees are con-
structed for sorting substrings of the same length.
Without loss of generality, we assume that the input
strings are pairwise distinct and no string is a prefix
of another string.
Before going further, we introduce a straightfor-
ward algorithm to sort a list of m strings. Note that
by Lemma 2.2, any two strings of len ths 11 and 12
+
can be compared in O( 1) time using 0 8 1 1 2 ) opera-
tions. Hence, if we apply Cole’s mergesort algorithm
(Lemma 2.5) to solve our string sorting problem, each
of its O(lo m) iterations can be executed in O(1) time
using O(nf operations. Thus, we have the following
lemma.
Lemma 3.1 A list of m input strings whose total
n.umber of symbols i s n can be sorted in O(1og m time
using O ( n log m) operations O R the Common ARC W
PRAM. 0
The total number of operations used by the modified
Cole's mergesort algorithm is nonoptimal in the sense
that the problem can be solved in O(m log m+n) oper-
ations which is strictly less than O(n logm) whenever
m = o ( n ) and m is not a constant.
The basic strategy of our optimal algorithm fol-
lows the mergesort technique. Given t w o digital search
trees with m, and mt leaves respectively, we show how
to merge these two trees into a digital search tree with
+
m, mt leaves. We begin by describing a relatively
slow algorithm that merges the two digital search trees
by sweeping through these trees from top to bottom
in a breadth-first search manner.
Algorithm Merging-Two-Trees
Input: Digital search trees S and T with m, and mt
leaves corresponding to two sorted lists of m, and mt
strings respectively.
Output: The merged digital search tree R with m, =
+
m, mt leaves.
Let the roots be at level 0. Let s be a vertex of S
and let t be a vertex of T . The vertices s and 1 are
pairwise ambiguous if the string defined by the path
from the root of S to the vertex s is equal to the string
defined by the path from the root of T to the vert.ex
1 . In this case, the vertices s and 1 will be merged to
one vertex r in R.
begin
The algorithm works in iterations i 2 1.
Input to iteration ( i 2 1 ) : All ambiguous pairs
( s l , t l ) , .. . , ( s k , t k ) a t level i - 1, where vertices
S I , ,. . , s k belong to S and vertices t l , . . . , i k belong
to T . The children of each s, in S are i n sorted order
according to the i-th symbols of the strings leading
to them. Similarly, the children of each t i n T are
in sorted order according to the i-t.h symbols of the
strings leading to them.
Iteration i:
for every input ambiguous pair ( s , t ) do in parallel
Merge the list of children of s with the list of chil-
dren of i, using a parallel merging algorithm. Identify
ambiguous pairs at level i . For each input ambiguous
pair, form a new vertex r in R.
Implementation Remark: We store each tree in a
one dimensional array. The children of each vertex are
stored in sorted order in consecutive memory locations
so that the merging of the childrens of two vertices
can be done in a linear number of operations. Prior to
advancing from iteration i- 1 to iteration i, we apply a
prefix sums routine so that all the ambiguous pairs of
level i - 1 are given in consecutive memory locations.
end
Let n, and nt be the numbers of vertices of the input
digital search trees S and T respectively. Similarly,
let n, be the number of vertices of the output digitsal
search tree R. Then, clearly n, + n l > n,, and one
can show the following lemma.
352
Lemma 3.2 Given t w o digital search trees S and T
with m, and mt leaves respectively, Algorithm Merg-
ing-Two-Trees merges the t w o trees into a single dig-
ital search tree R with m, = m, +mt leaves in
O ( d , ~ ~time ~ using
~ ~ O(mr
T ) + +
( ( n , n t ) - 12,)) op-
erations, where d = min {depth(S), depth(T)}, and
where depth@) is the length of the longest path f r o m
the root i n tree 2 . 0
We use our tree merging algorithm to derive the fol-
lowing mergesort algorithm. We first consider the case
where all the m input strings are of the same length
1 = 2, We then generalize our algorithm.
Algorithm Merge-Sort
Input: m digital search trees. Each tree is a chain of
length 1 = 2 which represents a string of length 1. For
simplicity, we assume that m = 2 h , for some positive
integer h.
Output: The digital search tree for the sorted list of
t.he input strings.
begin
for j + 1 to logm do
for i +- 1 to 9 do in parallel
Merge the two adjacent trees Tzi-1 and T2i
each with 2j-l leaves;
Let the resulting tree be Ti;
end
By Lemma 3.2, iteration j of this algorithm runs
in O(llo]poRo~,,,)time, and hence the total time for
the a.lgorithm is O(ll:$,",). During iteration j ,
the number of operations necessary to, merge chil-
drens of 2 pairs of trees each with 23-' leaves is
O ( 2 . 2j-l . 2) = O(m). Thus, the total number of
operations required for merging the children by this
algorithm is O(m1ogm). Notice that the number of
ambiguous pairs during iteration j is the difference be-
tween the number of vertices in the trees before the it-
eration and that in the trees after the iteration. Thus,
the total number of ambiguous pairs during the execu-
tion of the algorithm is bounded by the total number
of vertices in the input m trees which is O ( n ) , and
hence the total number of operations for manipulat-
ing the ambiguous pairs is O ( n ) . It follows that the
total number of operations is O ( m log m + n). There-
fore, we have the following lemma.
Lemma 3.3 Given a list of m strrngs each of length
m = I, Algorilhm Merge-Sort finds the sorted list of
2
the strings in 0 ( 1 , ~ $ ~ , ) time using O(mlogm n )+
operations. 0
We can make the algorithm faster by pipelining the
operations of the two loops. Once the first iteration
of the inner loop terminates, we can proceed with the
next outer iteration while trying to complete the in-
ner loop of the previous iteration. Thus, we have the
following lemma.
 Lemma 3.4 Given a list of m strings each of length
m = I , the sorted list of the input strings can be deter-
11
mined in O(([+log m ) lo]gDpo~m)
lime using O ( mlog m+
n) operations. D
the time complexity to O( Iztgmm)
We can improve the previous algorithm by reducing
regardless of the [Step 3 ] For each 1 < j 5 si and each substring Xi,j
value I as stated in the next theorem. See [17] for the
proof.
Theorem 3.1 Given a list C = ( X I ,XZ,. . . , X,) of
m strings, where lXil = n, the sorted list o f t h e
input strings can be found in O( l ~ $ ~ m )dime vsing
+
U ( mlog m n) operations. O
Remark: One can get a somewhat simpler optimal
time sequential algorithm. We can just merge the in-
put m digital search trees of different lengths two by
two by using a sequential algorithm similar to Algo-
r i t h p Merging-Two-Trees. This sequential algorithm
mqrges two digital search trees by processing them in
a breadth-first search or a depth-first search manner.
4 Constructing a Digital Search Tree
Given a lexicographically
sorted list L = (Xi, Xz,. . . , X,) of m input strings,
where IXiJ= n, we show how to construct a dig-
ital searc tree for the list within the same complexity
bounds as those of the sorting algorithm. We have al-
ready presented an optimal-work parallel algorithm to
construct a digital search tree when each input string
is of length logm. In this section, we generalize this
result to an optimal-work deterministic parallel algo- [Step 4 ] We construct in parallel a digital search tree
rithm when the input strings are of different lengths.
If the input strings are not sorted, we can sort them
by using the sorting algorithm in the previous section
without affecting the overall complexity of the algo-
rithm. Without loss of generality, we assume that the
input strings are pairwise distinct and no string is a
prefix of another string. The following simple algo-
rithm constructs the digital search tree in O( ~ ,$"),
+
time using O ( mlogm n ) operations.
Algorithm Digital-Search-Tree
Input: A sorted list L = ( X I ,Xz,. . . , X,) of m input [Step
strings, where gLl lXil = n.
Output: The igrta search tree for the list L .
begin
[Step 1 ] Partition each string Xi = (u,,1,ui,z,.. . , u i , ~ , ) ,
1 5 i 5 m, into si = [ elsub-
strings, say, Xi,l = ( a i , l ,ai,2 . ai,logm)r
' 7
Xi,z = (~i,logm+ir...,ui,21ogm),. . . ! Xi,s, =
('ii(s.--l)logm+l,. . ' i n l a ) ;
. ? hence Ixi,jl = logm,
for 1 5 j 5 si - 1, and !Xi,,,[5 logm.
[Step 2 ] For each substring Xi,, of Xi, where 1 5 i 5 m
and 1 5 j 5 s i , find the minimum interval ( l , ~ ) ,
where 1 5 i 5 r, and such that XI,, exists but
Xl-1,j does not, and X,,, exists, but X,+l,j does
not. This step can be performed in O(1og m) time
using U ( n ) operations with the doubling tech-
nique, and enables allocation of processors for ex-
ecuting the following steps.
of Xi, 1 5 i <
m, find the maximum inter-
val ( l ' , ~ ' ) ,where I' 5 i <r', and such that
Xp,h = Xlt+l,h = . . . = X,r,h,.fOr each 1 h < j .
We do this in two substeps. First we use an auxil-
<
iaryarray A a n d s e t A ( i , j ) := O,ifXi,j = X i - l , j
:= 1, otherwise; we also set the fol-
values: A ( 1 , l ) := 1. Now for
each i separately, find the smallest k for which
= 1; we conclude that for every j < k,
= A ( i , j - l ) , and we set B ( i , j ) = 0 for
k ; we also conclude for every j 2 k,
that A ( i , j ) # A ( i , j - l ) , and we set B ( i , j ) = 1
for all j 2 k. This computation can be done in
O(1) time and O ( n ) operations on the Common
CRCW by Lemma 2.2. The second substep fin-
ishes the computation of this step. Let the inter-
val (l', T ' ) of Xi,l be (1, m). Determining I' for the
interval (l', r') of Xij, for j 2 2, is done by finding
the largest integer k 5 i for which B ( k ,j - 1) = 1.
h
The value r' of l',r') can be also determined in
a similar way. T is step can be done in O ( a ( m ) )
time and O(n operations by Lemma 2.3. Note
that the two dimensional arrays A and B of this
step are used for ease of explanation and are not
really needed.
of depth a t most logm for each interval de-
termined in Step 3 in the following way. Let
(l',.') and ( l " , ~ " ) be such intervals of sub-
strings Xi,, and X;,,-l, respectively. Clearly,
I" 5 1' and r' 5 r". Then, the ( j - 1)st
substrings {XI(;- 1 , XIt+l,j- I., . . . , Xrt,i-l} form
a single path of length logm in the digital search
tree corresponding to the interval (l", r"), and the
j-th substrings {XI),,,Xlt+l,j!.. . , X,),j} form a
digital search tree corresponding to the interval
( 1 ' - r'). \ I I
1
For each interval (l',r') of Xi,j, let the
eaf node corresponding to the substrings
{Xpi-l, Xp+l,j-l,,. ., X,1,j-1} be the root node
of t e tree corresponding to the substrings
{XIf,j,XII+l,j,...,X,/.j}.
end
One can show the following theorem concerning the
above algorithm,
Theorem 4.1 Given a sorted list of m strings whose
total length is n , the corresponding digital search
be constructed i n O(ld",Y:,", ) ' a m e using
O ( m l o g m + n ) operations.
353
5 A Randomized Sorting Algorithm
In this section, we present a simple randomized al-
gorithm that sorts a list C = ( X I ,X Z ,. . . , X,) of m
input strings, where ELl lX,l= n , in O(1ogm) time
+
using O(m1ogm n ) operations, with high probabil-
ity. As in Section 3, we first consider the case where
all the input strings are of the same length 1 = 2,
and later generalize the result. We also assume that
the input strings are stored in a n array X[l..m; 1 . 4 ,
where 1 = 2.
Our randomized algorithm requires an efficient al-
gorithm for the following string search problem. Given
a sorted list C’= (Xi , Xi,. . . ,X’,) of m’ strings, each
of length I , and a string Y of tge same length I , we
need an algorithm to search for Y in L’ serially. A
naive binary search algorithm will take O(1 . log m‘)
time. Kosaraju [18] provides an O ( I d m - time ) al-
gorithm. We now develop a very simple algorithm that
+
takes O(1ogm’ I ) time given that the input list C’
is properly preprocessed. The following algorithm for
preprocessing the input list is similar to Step 3 of Al-
gorithm Digitalsearch-Tree of previous section, and
can be done in O(cu(m)) time using O ( n ) operations.
A l g o r i t h m Binary-Search-Preprocessing
Input: A sorted list L’ = ( X i , Xi,. . . , XL,) of
m’ strings, each of length 1. C’ is stored in array
X’[l..m’; l..l]. We use two auxiliary Boolean arrays
A[l..m’; 1.4 and E[l..m’; 1 . 4 .
O u t p u t : An array A’[l..m’; 1..1]. For each i and j ,
A ’ [ i , j ] contains a pair of integers ( r l r r 2 ) such that
rl 5 i 5 rz and (r1 , T Z ) is the maximum interval such
+
that X ’ [ r I , k ] = X ’ [ q 1, k ] = . . . = X’[rz,k], for all [Step 1 ] Pick randomly
IC such that 1 5 k 5 j .
begin
1. forall 2 5 i 5 m’ and 1 5 j 5 1 do i n parallel
if X ’ [ i - l , j ]= X’[i,j] then A[i,j] +-0
A[1,1] = 1;
else A [ i ,j ]
+- 1;
2. forall 1 5 i 5 m’, find the smallest k i
such that A [ i , k ]= 1 in parallel;
3. forall 1 5 i 5 m’ and 1 5 j 5 1 do i n parallel
if j < ki then E [ i , j ] t 0 else B [ i , j ] 1;
4. forall 1 5 i 5 m‘ and 1 5 j 5 1 do i n parallel
+-
compute r1 for the interval ( T I , r 2 ) of A ’ [ i ,j]
by finding the largest integer
k 5 i for which B[k,j] = 1;
compute r:!for the interval similarly;
end
Step 2 of the above algorithm can be done in O( 1)
time using O(m’1) operations by Lemma 2.2. Steps
1 and 3 can be also performed within the same com-
plexity bounds. Step 4 can be done in O(cu(m’))time
and O(m’1) operations by Lemma 2.3. Thus, we have
the following lemma.
L e m m a 5.1 Given a sorted 12.52
C’ = ( X i , Xi,.. . , X L x , ) of m’ strzngs each of length 1,
Algorithm Binary-Search-Preprocessing correctly con-
structs the auziliary array A’[l..m’; l..I] in O(a(m‘))
time using a linear number of operations. 0
We now provide a binary search algorithm that opti-
mally finds a given string in a sorted list of strings.
The algorithm starts by comparing the given string Y
with the middle string in the sorted list. The first posi-
tion in which these two strings differ gives a restricted
range where Y can occur. We use the information
stored in A‘ to determine the new search range for Y.
The detailed algorithm and the proof of the following
lemma are in [17].
L e m m a 5.2
Given a sorted list C‘ = (Xi, Xi, . . . ,XL,),a string
Y , and the auziliary array A’[l..m’; 1 . 4 , the position
of Y in C‘ can be determined in O(1ogm’ + I ) time
sequentially. 0
The following randomized algorithm uses the above
search algorithm to sort a list of m strings, each of
+
length 1, in O(1ogm I ) time using O(m1ogm n ) +
operations, with high probability. This algorithm can
be modified t o sort m strings of different lengths in
O(1og m) time using O ( m log m + n) operations as in
Section 3.
Algorithm RandomizedSorting
I n p u t : A list C = ( X l , X a , . . .,X,)of m strings,
where 1x11= . . . = IX,I = = 1. t
o u t p u t : The list C in sorted order.
begin
& strings and sort them de-
terminist ically.
[Step 2 ] Preprocess the sorted list of the $ & strings
introduced in Step 1 and determine the position
of each of the m input strings in the sorted list.
[Step 3 ] Partition the set of input strings into (e
+
1) sets according to their positions in the sorted
list. Each such set consists of at most O(log2m)
strings with high probability.
[Step 4 ] Sort all the
parallel.
(e
+ 1) sets independently in
end.
L e m m a 5.3 Let m’ be the number of strings in any
s e t after Step 9 ofthe above algorithm. Then Pr{m’ 2
8 log2 m > 5 m-4.
L e m m a 5.4 Given a list of m strings each of length
11- - 1, Algorithm Randomized-Sorting sorts the in-
m
put strings in O(1og m + 1 ) time using O(m log m n ) +
operations with high probability. 0
L e m m a 5.5 Given a list of m strings each of length
1
m
1 = 1, th,e sorted list of th.ese strin,gs can be found in
O(logm) time using O(m1ogm + n ) operations with
high probability. 0
354
Finally, we generalize this result to the case where the
input strings are of different lengths. The proof of the
following theorem is in [17].
, = ( X 1 , X 2 , .. . , X m ) of
Theorem 5.1 G i v e n a list C
m strings, where E r l lXil = n , the sorted list of
the input strings can be found in O(1ogm) t i m e us-
ing O(m1ogm + n ) operations with high probability.
0
We next consider an application of our sorting algo-
rithms, namely that of determining the minimal start-
ing point of a circular string.
6 Finding a Minimal Starting Point of
a Circular String
c 1 , . . . , c n - l ) be a strin over the ar-
Let C = (CO,
bitrary alphabet C. The string is cafled circular
if the following n strings are considered equivalent;
(cj,.. . ., cn-l, CO, . . . , c j - I ) , j = 0 , . . ., n - 1. Let
C ( J O=) [cj,, . . . , c n - l , CO, . . . , cj,-1] be a linear string
of C with j~ as a starting point. Given C ( i ) and C(j)
we define C ( i ) 4 C ( j ) i%f [ c i , . . . , c n - l , c9,. . . , ~ i - ~ j compare the two strings of length 2' each starting from
precedes [ c j , . . . , c n - l , CO, . . . , cj-11 in lexicographic
order, and C ( i ) 5 C ( j ) iff C ( i ) + C ( j ) or C ( i ) =
C ( j ) . C(j0) is minimal if C(j0) 5 C(j) for all
j = 0 , . . . , n - 1. The index ju is called a minimal
starting point of C (m.s.p. in short). A circular
string C is defined to be repeating if i t has more
than one m.s.p.
Given a circular string, the problem of finding a
minimal starting point can be solved in linear se-
quential time [8, al]. This problem can be also
solved in O(lognlog1o n ) time [14] or O(1ogn) time
[16] using O ( n loglogng) operations on the Arbitrary
CRCW PRAM if the input alphabet is restricted to
{ 1 , 2 , . . . , no(1)}. In this section, we show that the two
parallel algorithms of Sections 3 and 5 can be used to
that runs in O( ,:$z:n)
derive an optimal parallel algorithm for this problem
time deterministically or in
O(1og n ) time with high probability. The correspond-
ing algorithm provides four kinds of improvement over
the algorithms of [14, 161. The first improvement is
that the input alphabet is not restricted; the second is
that the new algorithm uses O ( n )operations; the third [Step 1 ] Partition C into
is that the algorithm runs on a weaker model, namely
the Common CRCW PRAM; and the fourth is that
the algorithm use O ( n ) space while those of [14, 161
use O(n'+') space, where is a positive constant.
In this section, we assume that the input circular
string is not repeating, since if it is repeating, we can
find its smallest repeating prefix in O(1ogn) time and
O ( n )operations [9, 221. Clearly, the m.s.p. of a small-
est repeating prefix is also an m.s.p. of the original
string. One can easily check that there is only one [Step 3 ] Find the m.s.p. of Ci serially, for each i =
m.s.p. in a non-repeating circular string. We be-
gin with a simple O(1ogn) time algorithm t,hat uses
O(n1ogn) operations. We will use it later to derive
a more efficient algorithm. This simple a.lgorithm is
based on the following lemma from [21].
Lemma 6.1 Let C = (cO,cl, . , c n - l ) be a circular
string with a unique m . s . p . If ci+l = C j + l , f o r all
+ + +
1 = 1 , .. . , s, then any of {i 1 , . . . , i s 1) n { j +
+ +
1 , . . .,j s 1) can not be the m . s . p . 0
Algorithm Simple-MSP
Input : A non-repeating circular string C =
( c O I c l , ..., c , , - ~ ) . (Assume for convenience that n =
2k, for some integer IC > 0.)
Output : The m.s.p. of the input string.
begin
1 . Mark all n positions as candidates for the m.s.p.;
2. for i = 1 to logn do
for each block of size 2' do in parallel
Compare the two strings of length 2'
starting at the candidate positions,
and eliminate one of them;
end
We assign 2' processors to each block of size 2', and
its candidate position. If the two strings are differ-
ent, we find the smaller string and mark its starting
position as a further candidate. If the two strings
are the same, we mark the first of the two candi-
dates as a further candidate by using Lemma 6.1.
This can be done in constant time using O(2') op-
erations by Lemma 2.2. Thus, the above algorithm
finds the m.s.p. of a non-repeating circular string in
O(1ogn) time and O(nlogn) operations on the Com-
mon CRCW PRAM. Now, we describe the optimal
algorithm.
Algorithm Optimal-MSP
Input : A non-repeating circular string C =
( C O , c1,. . . , cn-l). (Assume for convenience that n =
2', for some integer A > 0, and that n is a multiple of
log n .)
Output : The m.s.p. of the input string.
begin
& = m blocks, B1,
B2, ..., Bm, where Bi = ( C ( i - l ) l o g n ,
C(i-l)logn+l,...,Cilogn-l), 1 5 i I m.
[Step:! ] Let $ be a special symbol not in C such that
VQ E C, a < $. Then construct m circular strings,
C1, . . ., C, each of length 2 log n + 1, where Ci =
( B i ,B ( ; + l ) m o d m , $ ) ,1 5 i 5 m. Clearly, for each
i , Ci is non-repeating and has only one m.s.p.
1, . . . , m. We can easily prove that this posi-
tion is better than any other position of {(i -
l ) l o g n , ( i - l ) l o g n + l , . , . , i l o g n - - 1) in C by
using Lemma 6.1. If the m.s.p. is in B i , mark the
symbol. Otherwise, do not mark any symbol in
Bi .
355
[Step 4 Now, there are a t most m candidates which have
een marked in Step 3. Construct a t most m
strings each of which starts a t a candidate and
ends at just before the next candidate.
[Step 5 ] Sort the strings constructed in Step 4 by using
one of the algorithms developed in Sections 3 and
5.
[Step 6 ] Replace each of the strings sorted in Step 5 with
its ranking in the sorted list from Step 5. Then C
is reduced into a circular string of length at most
m =Logn L.
[Step 7 ] Apply Algorithm Simple-MSP to the resulting
string and find the final m.s.p.
end
Lemma 6.2 The m.s.p. of the modified circular
string afler Step 6 of Algorithm OptimaLMSP main-
tains the same m.s.p as the original one. 0
Clearly, all the steps of the above algorithm can be
done in O(1og n ) time using O ( n )operations determin-
istically except for Step 5. Thus, we have the following
theorem.
Theorem 6.1 Given a circular string of length n , Al-
string in O( ,zz:,,)
gorithm Optimal-MSP finds the m s . ~ . of the input
time deterministically or O(1ogn)
time randomized, using O(n) operations on the Com-
mon C R C W PRAM. 0
References
[l) A.V. Aho, J.E. Hopcroft, and J.D. Ullman, The
Design and Analysis of Computer Algorithms,
Addison-Wesley, Reading, MA, pp. 78-84, 1974.
[a] M . Ajtai, J . Komlos, and E. Szemeredi, A n
O(n1ogn) Sorting Network, Proc. 15th ACM
Symp. on Theory of Computing, pp. 1-9, 1983.
[3] R.J. Anderson and G.L. Miller, Optimal Parallel
Algorithms for List Ranking, 3rd Aegean Work-
shop in Computing, Lecture Notes in Computer
Science 319, 1988, Springer-Verlag, pp. 81-90,
1988.
[4] A. Apostolico, C. Iliopoulos, G.M. Landau, B.
Schieber, and U . Vishkin, Parallel Construction
of a Suffix Tree with Applications, Algorithmica
3(3), pp. 347-365, 1988.
(51 0. Berkman and U . Vishkin, Recursive star-
Tree Parallel Data Structure, Proc. 30th Annual
Symp. Foundations of Computer Science, pp.
196-202, 1989. Also, SIAM J . Computing, 1993,
to appear.
[6] A. Borodin and J .E. Hopcroft, Routing, Merging,
and Sorting on Parallel Models of Computation,
Journal of Computer and System Sciences 30, pp.
130-145, 1985.
356
[7] P.C.P. Bhatt, K. Diks, T . Hagerup, V.C. Prasad,
T. Radzik, and S. Saxena, Improved Determin-
istic Parallel Integer Sorting, Information and
Computation 94, pp. 29-47, 1991.
[8] K.S. Booth, Lericographically Least Circular Sub-
strings, Information Processing Letters 10, pp.
240-242, 1980.
[9] D. Breslauer and Z. Galil, A n Optimal
O(log logn) time Parallel String Matching Algo-
rithm, SIAM J. Computing 19(6), pp. 1051-1058,
1990.
[lo] R. Cole, Parallel Merge Sort, SIAM J . Computing
17(4), pp. 770-785, 1988.
[ l l ] R. Cole and U. Vishkin, Faster Optimal Prefix
Sums and List Ranking, Information and Com-
putation, 81(3), pp. 334-352, 1989.
[12] F.E. Fich, P.L. Ragde, and A. Wigderson, Rela-
tions between Concurrent- Write Models of Paral-
lel Computation, SIAM J. Computing 1, pp. 606-
627, 1988.
[13] T . Hagerup and 0. Petersson, Merging and Sod-
ing Strings in Parallel, Mathematical Founda-
tions of Computer Science 1992, 17th Int’l Symp.,
Lecture Notes in Computer Science 629, Springer-
Verlag, pp. 298-306.
1141 C.S. Iliopoulos and W.F. Smyth, Optimal Algo-
rithms f o r Computing the Canonical Form of a
Circular String, personal communication, 1992.
[15] J . JhJh, A n Introduction to Parallel Algorithms,
Addison-Wesley, Reading, MA, 1992.
[16] J . JhJh and K.W. Ryu, A n Eficient Parallel Al-
gorithm for the Single Function Coarsest Partz-
tion Problem, Proc. 5th ACM Symposium on Par-
allel Algorithms and Architectures, 1993.
[17] J . JdJd, K.W. Ryu, and U. Vishkin, Sorting
Strings and Constructing Digital Search fiees in
Parallel, CS-TR-3073, UMIACS-TR-93-39, Univ.
of Maryland, 1993.
[18] S.R. Kosaraju, On a Multidimensional Search
Problem, Proc. 11th Annual ACM Symp. Theory
of Computing, pp.67-73, 1979.
[19] C.P. Kruskal, Searching, Merging, and Sorting in
Parallel Compuiation, IEEE Trans. on Comp., C-
32(10), pp. 942-946, 1983.
[20] R. Reischuk, Probabilistic Parallel Algorithms
for Sorting and Selection, SIAM J . Computing,
14(2), pp. 396-409, 1985.
[21] Y. Shiloach, Fast Canonization of Circular
Strings, J . Algorithms 2, pp. 107-121, 1981.
[22] U . Vishkin, Optimal Parallel Matching in Strings,
Information and Control 67, pp, 91-113, 1985.