c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 382396

journal homepage: www.intl.elsevierhealth.com/journals/cmpb

Fast computation of sample entropy and approximate

entropy in biomedicine
Yu-Hsiang Pan a, , Yung-Hung Wang b , Sheng-Fu Liang c , Kuo-Tien Lee a
a

Department of Environmental Biology and Fisheries Science, National Taiwan Ocean University, 2 Pei-Ning Road, Keelung 20224,
Taiwan
b Society of Streams, R.O.C., 1F., No. 15, Alley 70, Lane 12, Sec. 3, Bade Rd., Taipei City 10558, Taiwan
c Department of Computer Science and Information Engineering, National Cheng Kung University, No. 1, Ta-Hsueh Road, Tainan,
Taiwan, ROC

a r t i c l e

i n f o

a b s t r a c t

Article history:

Both sample entropy and approximate entropy are measurements of complexity. The two

Received 2 April 2010

methods have received a great deal of attention in the last few years, and have been suc-

Received in revised form

cessfully veried and applied to biomedical applications and many others. However, the

28 November 2010

algorithms proposed in the literature require O(N2 ) execution time, which is not fast enough

Accepted 6 December 2010

for online applications and for applications with long data sets. To accelerate computation,
the authors of the present paper have developed a new algorithm that reduces the com-

Keywords:

putational time to O(N3/2 )) using O(N) storage. As biomedical data are often measured with

Approximate entropy (AE )

integer-type data, the computation time can be further reduced to O(N) using O(N) storage.

Computational geometry

The execution times of the experimental results with ECG, EEG, RR, and DNA signals show

Multiscale entropy (MSE)

a signicant improvement of more than 100 times when compared with the conventional

Sample entropy (SE )

O(N2 ) method for N = 80,000 (N = length of the signal). Furthermore, an adaptive version of

Sliding kd tree (SKD)

the new algorithm has been developed to speed up the computation for short data length.
Experimental results show an improvement of more than 10 times when compared with
the conventional method for N > 4000.
2010 Elsevier Ireland Ltd. All rights reserved.

1.

Introduction

Both approximate entropy (AE ) [1] and sample entropy (SE ) [2]
are measurements of a systems complexity, which are important for the analysis of biomedical signals [3,4] and signals in
other elds [5,6]. Pincus introduced approximate entropy, a set
of measures of system complexity closely related to entropy,
which is easily applied to biomedical signals and others. However, AE has two deciencies. First, AE strongly depends on
the record length and is uniformly lower than expected for
short records. Second, AE lacks relative consistency. The sample entropy was introduced by Richman et al., which requires

much shorter data sets than the approximate entropy. Both

AE and SE quantify the regularity (orderliness) of a time series.
However, both the measurements indicate a higher complexity for certain pathologic processes associated with random
outputs than for healthy dynamics exhibiting long-range correlations. For this reason, Costa et al. [3,7,8] introduced the
multiscale entropy (MSE), which measures complexity by taking into account multiple time scales. For each scale, the
sample entropy is computed. All these quantities are dened
in Section 2.
The algorithms for computing complexity proposed in the
literature require O(N2 ), which is not realistic for applications
with long data sets or for online diagnosis of a time-varying

Corresponding author. Tel.: +886 987 153 779; fax: +886 2 82366969.
E-mail addresses: D95310001@mail.ntou.edu.tw, schpeter99@gmail.com (Y.-H. Pan).
0169-2607/$ see front matter 2010 Elsevier Ireland Ltd. All rights reserved.
doi:10.1016/j.cmpb.2010.12.003

383

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 382396

system, especially for data collected with high sample rates. In

addition, to nd the statistical meaning of the signals, a large
amount of data and many parameters are typically collected,
from which many parameters are determined.
The method developed by Manis [9] has reduced the execution time for computing approximate entropy. He used a
bucket-assisted technique, similar to a bucket sort, to exclude
impossible matches of similarities early in the process. This
improved the execution time, but it was still an O(N2 ) algorithm. To monitor the change in SE with time for the online
monitoring of a systems health, Sugisaki and Ohmori [10]
derived a recursive sample entropy algorithm for overlapped
rolling windows. However, this algorithm has two drawbacks:
rst, the computed sample entropy is only approximate; second, the computational efciency decays with a decreasing
overlap length, with the improvement reaching zero when the
overlap length is zero.
Bucket-assisted and recursive sample entropy algorithms
are O(N2 ) algorithms. Does a nonquadratic algorithm exist?
If the time series consists only of binary numbers 0 and 1
[11], then the distance r in the approximate (sample) entropy
term disappears. In this situation, a linear time algorithm
exists. In [12], Pincus stated: The discreteness of the state
space affords the possibility of such linear time calculations,
in contrast with inherently quadratic time algorithms for
continuous state space. However, Pincus did not give a theoretical proof. Pan et al. [13] proved that the nonquadratic
execution time can be achieved by transforming the nm
(n m
i )
i
term in the sample (approximate) entropy into an orthogonal
range-counting problem by transforming the template series
into m-dimensional space point sets. All these quantities are
dened in Section 2. The kd tree, or the range tree combined
with the fractional cascading technique, is then applied to
compute the sample and the approximate entropies. These
improve the time complexity to O(N5/3 ) using linear storage
(memory) or O(N log2 N) using O(N log2 N) storage for m = 2.
Although the range tree affords less time complexity, the storage is not linear; it requires a lot of overhead, and hence,
performs worse than the kd tree algorithms for signals with
a practical length (N must be greater than a certain threshold).
This paper has developed for the rst time a sliding kd
tree (SKD) algorithm that reduces the time complexity of a
kd tree algorithm by one dimension while using linear storage. In other words, the time complexity for SKD is O(N3/2 ) for
m = 2 for the real-type data, and is O(Bm1 N) for the integertype data as in digitalized biomedical signals, where B is the
resolution of the data as described in Section 3.3. Second,
for N under a certain threshold, the performance of the SKD
algorithm is inferior to the conventional brute force method.
For this reason, the authors have developed an adaptive SKD
algorithm to accelerate the computational time for short data
length.
The remainder of the paper is organized as follows: Section
2 provides a review of the sample entropy, the approximate
entropy, and the multiscale entropy, and a review of the kd
tree algorithm. In Section 3, the SKD algorithm and its adaptive version are developed to compute the entropy. Section 4
presents experimental results to illustrate the effectiveness of
the SKD algorithm, and in Section 5 conclusions are given.

2.
Review of the computation of sample
entropy and approximate entropy
2.1.
Review of sample entropy, approximate entropy,
and multiscale entropy
Consider a time series of length N: X ={X1 . . . Xi . . . XN }. A pattern length m (length of sequences to be compared) is selected,
and, for each i, a vector (template) of size m is dened:

(Xi )m = {Xi , Xi+1 , . . . , Xi+m1 }

(1)

Two vectors (Xi )m and (Xj )m are dened as similar if their

distance is less than r, where r is dened as the tolerance for
accepting matches.
|(Xi+k ) (Xj+k )| < r,

k, 0 k m 1

(2)

A variable (i,j,m,r) is dened; its value is 1 if Eq. (2) holds;

otherwise, its value is zero.

Nm+1

Furthermore,

n i =
m

(i, j, m, r) is dened

(3)

j=1

which is the number of vectors (Xj )m within r of (Xi )m .

Then, the probability that any vector (Xj )m is within r of
(Xi )m is
Cm
i (r) =

n m
i
Nm+1

Then,
Nm+1
log Cm
(r), which is the averm (r) = 1/N m + 1 i=1
i
(r).
age of the natural logarithms of Cm
i
Subsequently, the approximate entropy [1] is calculated as
follows:
AE (m, r, N) = m (r) m+1 (r)

(4)

The sample entropy SE [2] is computed as:

 

Nm m
n 
ni
n
i=1
SE (scale, m, r, N) = ln Nm
= ln
m+1
n
i=1

ni

(5)

where

Nm

nm
i =

(i, j, m, r)

(6)

j=i+1

in Eq. (6) differs from n m

nm
i in Eq. (3) to the extent that, for
i
/ j) and i ranges from 1 to
SE , self matches are not counted (i =
(N m).
Description of the MSE analysis [3]: rst, the original time
series is divided into nonoverlapping windows of length , and
second, the data points inside each window are averaged. In
general, each element of a coarse-grained time series is cal-

384

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 382396

culated according to the equation

ECG TIme series

250

Xj =

j


1


Xi ,

1j

N
,


N>

200

(7)

i=(j1)+1

150

2.2.

100

amplitude

Then, for each time scale , SE is computed using Eq. (5).

This article has focused on the computation of SE and AE ,
which is similar as seen in Eqs. (3) and (6).

kd tree algorithm

-50

In [13], it was shown that the computation of the nm

term can
i
be transformed into an orthogonal range-counting problem,
and the procedure is briey reviewed here. For each i, the template (Xi )m has been transformed into an m-dimensional point
set Pi by setting
xi = Xi ,

yi = Xi+1 ,

50

zi = Xi+2

(8)

-100
-150
0

1000

2000

3000

4000

5000

6000

7000

8000

9000 10000

points

Fig. 1 Time series for the rst 10,000 points of an ECG

signal.

Then, nm
is equivalent to the number of points inside the
i
bounding box Wi :

x-y plot for ECG signal

250

Wi = [(xLB )i : (xUB )i ] [(yLB )i : (yUB )i ] [(zLB )i : (zUB )i ]

(9a)
200
1d search

where the subscript LB and UB stand for the lower bound and
upper bound of the box, and

150
100

(xUB )i = xi + r
(yUB )i = yi + r
(zUB )i = zi + r

2d search

(9b)
...

Given an orthogonal range (bounding box) in the ddimensional space, the number of points in each box is
queried, which is called an orthogonal range-counting problem in the eld of computational geometry. Hence, for each
i , the computation
point Pi and its associated bounding box W
m+1
(n
)
is
equivalent
to
an
m
(m
+
1)
dimensional
orthogoof nm
i
i
nal range-counting problem. Once nm
and nm+1
are computed,
i
i
nn , nd , and SE can be calculated directly from Eq. (5). Comwith nm
, the time complexity
paring the dimension of nm+1
i
i
is dominated by the rst term. Hence, the time complexity
of computing SE is determined by nm+1
. Fig. 1 shows the rst
i
10,000 points of an ECG (electrocardiographic) signal. For a
given point Pi and distance r, Fig. 2 demonstrates the computation of n1i from a geometric point of view. In the geometric
view, the algorithm proposed in [3] can be interpreted as follows. For each point Pi , calculate its bounding box Wi . Traveling
is equivalent to the
each point pj with index j > i (Eq. (6)), nm
i
number of points in Wi . This algorithm requires double loops
i and j. Hence, it is an O(N2 ) algorithm and can be interpreted
as a brute force algorithm, because it does not preclude any
impossible queries (matches).
The kd tree [14,15] can be applied to the orthogonal rangecounting problem [13] and it showed that it is an effective
algorithm in computing SE . The fundamental concept is to
store the point sets {P} in a specially designed data structure;
subsequently for a given box, the query would be faster. The
kd tree, proposed by Bentley in 1975, is a binary tree, whose
each node v is associated with a rectangle Bv. If Bv contains

(xLB )i = xi r,
(yLB )i = yi r,
(zLB )i = zi r,

50
point i

-50
-100
-150
-150

-100

-50

50

100

150

200

250

x
Fig. 2 Demonstration of the geometric view for an ECG
signal for m = 1. Computing n1i is equivalent to a
two-dimensional search and computing n0i is equivalent to
a one-dimensional search.

only point in its interior, v is a leaf. Otherwise, Bv is partitioned

into two rectangles by drawing a horizontal or a vertical line,
such that each rectangle contains, at most, half of the points;
the splitting lines are alternately horizontal and vertical. A
kd tree can be extended to higher dimensions in an obvious
manner.
Table 1 summarizes the performance of the kd tree algorithm.

Table 1 Time and storage complexity of the kd tree

algorithms.
Construct tree time
Search time
Storage

O(N log N)
O(log N) for d = 1
O(N1(1/d) ) for d > 1
O(N)

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 382396

3.

The sliding kd tree algorithm

3.1.

Developing the SKD algorithm

385

An algorithm called the sliding kd tree (SKD) [16] was

originally developed to speed up the range search in very
large-scale integration systems (VLSI) in electrical engineering. However, its restriction is that the length of the query box
can be xed in any direction. It can be observed that in Eq. (9),
the distance r is identical for all directions, and one can subsequently apply the SKD to further reduce the time complexity
for computing MSE. We focus on computing nm
, nm+1
can be
i
i
computed in the same way.
For convenience, rewrite Eq. (9) as Wi = (Wx )i (Whd )i where
(Wx )i = [(xLB )i : (xUB )i ]

(10a)

and
(Whd )i = [(yLB )i : (yUB )i ] [(zLB )i : (zUB )i ] . . .

(10b)

have been dened. The subscript x and hd in Eq. (10) stand

for the bounding box in the x and higher (d 1) dimension
respectively, and subscript LB and UB denote the lower and
upper bound of the box. Also, index i denotes the points index.
The idea: The d (d = m) dimensional box Wi is a one dimensional box (Wx )i intersected with a (d 1) dimensional box
can be interpreted as the number of points
(Whd )i . Thus nm
i
rst in (Wx )i and then in (Whd )i as demonstrated in Fig. 2. The
SKD algorithm is composed of a one dimensional and a (d 1)
dimensional range counting, and the rime complexity is determined by the latter one. The SKD algorithm of computing nm
i
is presented as follows.
Step1: Sort the point sets {p} by the x (rst) component in
ascending order, and then index each point in the sorted
array. Unless otherwise specied, {p} represents the sorted
array hereafter.
Step 2: For i = 1:N m, report the points inside (Wx )i , which is
equivalent to nd the points with index j satisfy
iLB(i) j iUB(i)

(11)

where iLB(i) and iUB(i) represent the lower and upper index
of points inside (Wx )i .
Step 3: Build the (d 1) dimensional kd tree. The tree initially
contains points in (Wx )i=1 and only the higher dimension
coordinates (y, z, . . .) are stored into the trees node.
Step 4: Begin with i = 1, nm
is equivalent to the number of
i
points inside (Whd )i which is already in (Wx )i as illustrated in
Fig. 2. The d 1 dimensional kd tree search (search y, z, . . .)
.
is applied to obtain nm
i
Step 5: Slide (move) from point i to i + 1, query the number
of points in (Whd )i+1 intersecting (Wx )i+1 . Firstly, report the
points in (Wx )i+1 using the points in (Wx )i as a clue. Because
the points within (Wx )i and (Wx )i+1 may be different, old
points (points in (Wx )i , but not in (Wx )i+1 ; that is, points with
indexes iLB(i 1) j iLB(i) 1) have to be removed from the
tree, and new points (points in (Wx )i+1 , but not in (Wx )i ; that is,

Fig. 3 Demonstration of moving the box from point p i to

p i+1 ; insertion of the new points and removal of the old
points.

points with indexes iUB(i) j iUB(i + 1)) must be inserted into

the tree, as illustrated in Fig. 3. After the deleting and inserting operations, one obtains the points in (Wx )i+1 . Secondly,
the d 1 dimensional kd tree search is applied to obtain nm
.
i+1
Step 6: Repeat Step 5 for i = 1 to N m.
Discussion and modications:
Step 1: Quick sort can be used to sort the point sets {p}.
Step 2: Find iLB(i) and iUB(i) for i = 1 to N m:
Given point pi , iLB(i) is the index of the rst point in {p} satisfying (xLB )i x (xUB )i . That is, j = iLB(i) is the index satises
xj1 < (xLB )i

and xj (xLB )i

(12a)

since the boxs length (2r) in the x direction is a constant

and {p} is a sorted array, then iLB(i) must be in ascending
(strictly speaking, non-descending) order. Thus, we can visit
j (the index to record iLB(i)) from 1 to N m to nd all iLB(i)
for i = 1:N m. Starting from i = 1 and j = 1, if the condition
xj1 < (xLB )i and xj (xLB )i holds, then record iLB(i) = j and
increase i by one; Otherwise increase j by one. Similarly, iUB(i)
is the index j which satises
xj (xUB )i

and

xj+1 > (xUB )i

(12b)

The same procedure is applied to nd iUB(i) for i = 1 to N m.

Step 5: The structure of the kd tree is described in Section
2.2. The kd tree is, in nature, static and a balanced binary
tree. The deletion and insertion operations would destroy the
already balanced structure of the kd tree, hence increase the
tree depth and search time. This difculty can be circumvented as follows. Initially, build the d 1 dimensional tree
using {p} (all the points in the domain). Once the tree is built,

386

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 382396

remove all the points from their corresponding leaf nodes

(points are stored in the leaf nodes), and only retain the tree
structure. A counter is appended to each node to count the
number of points belonging to this node. When a new point
is inserted into the tree, it is inserted into the leaf node of the
tree. Then travel from that leaf node upward to the highest
level node containing this point. Each counter on the nodes
in the path is incremented by one. When a point is deleted
from the tree, we need to search that points corresponding
leaf node in the tree. This can be done by linking points in
{p} to their corresponding leaf nodes during the building the
tree. A similar procedure is applied to remove points from
the tree.
Step 6: iLB(i) and iUB(i) are both in ascending order. This property guarantees that once points deleted from the tree would
never be inserted into the tree again during the sliding process. From this we conclude that there are exactly N deletion
and N insertion operations when the sliding procedure goes
from the leftmost to the rightmost point. In another word,
there are, on the average, one deletion and one insertion
operation required for each sliding process (for each point i).

3.2.

Storage and time complexity analysis

This section analyzes the storage and the time complexity of

the SKD algorithm.
(1) The time complexity for the sorting process is O(N log N).
(2) iLB(i) and iUB(i) for i from 1 to N m: as discussed in Section
3.1, we need to travel i from 1 to N m. Hence, this step
takes O(N).
(3) kd tree building and search: the storage and the time complexity of the building tree are the same as those of the kd
tree. Table 1 shows that the time cost is T(N) = O(NlogN)
and the storage cost is S(N) = O(N). The dimension of the
tree for the SKD is one less than that for the kd tree algorithm. Thus, the time complexity of the SKD query is less
than that of the kd tree algorithm by one dimension. From
Table 1, with d = m
Tq (N) = O(N1(1/d) ) for

d2

Tq (N) = O(log N) for d = 1

(4) The time complexity of deleting processes is O(N log N).
As discussed in Section 3.1, the deleting process travels
the tree upward, starting from the bottom node, and the
depth of a balanced tree is O(log N). Sliding from the leftmost to the rightmost point, there are exactly N points
deleted from the tree, because once the points are deleted
from the tree, they are never inserted into the tree again,
as discussed in Section 3.1. Therefore, the time complexity
for deleting/inserting processes is O(N log N).
To summarize, the time complexity of the SKD algorithm
is dominated by step (3). By applying the query N m times,
the time complexity is
T(N) = O(N log N)

for

= O(N N1(1/m) )

m = 1,
for

T(N) = O(N N1(1/d) )

m>1

(13)

3.3.

Integer-type data

For the integer-type data as seen in the digitalized biomedical signal, the time complexity of the SKD algorithm can be
further reduced to O(Bm1 N).
From Table 1, it can be noted that the time complexity for
one query by the kd tree algorithm is Tq (N) = O(N1(1/d) ) where
the subscript q stands for one query.
For the SKD algorithm, d = m, as discussed in Section 3.2.
Thus,
Tq (N) = O(N1(1/d) ) = O(N1(1/m) ) for m 2

(14)

Data {X} measured from digital device with resolution B

can be represented as X = B + , B = 2b , where b is the number
of bits, and and are some constants.
Without loss of generality, the data can be represented by a
non-negative integer from 0 to B 1. In a d-dimensional space,
the maximum number of possible states (number of points
with different coordinates) is Bd .
If N > Bd , there must be repeated points in the trees leaf
1(1/d)
node. Eq. (14) can be written as Tq (N) = O(Ndiff
), where Ndiff
is the number of states or the number of points with different
coordinates for the given signal. Thus, Ndiff  Bd and the worst
case is Ndiff = Bd = Bm . Subsequently,
1(1/d)

Tq (N) = O(Ndiff

) O(Bd(1(1/d)) ) = O(Bd1 ) = O(Bm1 )

(15)

which is nite and is, therefore, O(1). To compute nd in Eq.

(5), one needs to apply the query N m
= N times, and thus,
the total time is T(N) = O(Bm1 N) = O(N). For example, if m = 2
and B = 256 (8 bits), then T(N) = O(256 N). Similarly, one can
prove that the time complexity of the kd tree algorithm is
T(N) = O(Bm N). Hence, for the integer-type data, the SKD algorithm is observed to improve the kd tree algorithm in that the
coefcient term is further reduced from O(Bm ) to O(Bm1 ).
Biomedical signals are cyclic in nature; the data points
often change smoothly from one point to the next. Therefore, in practice, Ndiff is much less than Bd . The rst 10,000
points in the time series of an ECG signal are shown in Fig. 1.
It is transformed into two-dimensional space point sets (m = 2)
dened as the Poincare plot [17], and is shown in Fig. 2. The
gure shows that the x and the y coordinates of the point sets
are strongly correlated, and the points are mostly distributed
around the line x = y. Therefore, Ndiff is much less than B2 .
This explains why, from numerical experiments, the actual
execution time is much less than that predicted by the order
analysis.
For data with nite resolution B, the SKD is a linear algorithm, but the execution time increases with B. When B
increases and approaches innity (real number), the time
complexity of the SKD algorithm switches smoothly from
O(B(m1) N) to O(N1(1/m) N).

3.4.

Pseudo-code of the SKD algorithm

In this section, we present the pseudo code of computing SE .

The building kd tree, deletion and insertion operations are
modied as described in the discussion sub-section in Section
3.1. To accelerate the execution time for handling integer-type

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 382396

Algorithm 1:.
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
29:

r = r * SD(X); % Normalize r by the standard deviation (SD) of

the time series {X}
for scale = 1: max do
X = coarseGrainSeries(scale, X) % nd the coarse-grain
series s using Eq. (7)
pointArray = transform to space(m, X)
for (k = m 1:m) do
d = k;
pointArray = sort points by x coord(pointArray); % step
1
[iLB,iUB] = nd lower upper index(r, pointArray) % Eq.
(12)
pointArray = compress data(pointArray);
tree = buildKDTree(d, pointArray); % build kd tree
structure
for (j = 1: iUB[1] 1)
insert new points(pointArray[j],tree); % insert point
pj into the tree
end do
for (i = 1: N m) do % The Sliding Loop
delete old points([pointArray[i]],tree); % delete point pi
from the tree
(Whd )i = nd bounding box(m,r,PointArray[i]) % using
Eq. (10b)
nm
= kd search((Whz )i , tree);
% computenm
(k =
i
i
(k = m);
m 1) or nm+1
i
if (k = = m 1)
nn (i) = nn (i) + nm
% numerator term in Eq. (5)
i
else
nn (i) = nn (i) + nm
% denominator term in Eq. (5)
i
endif
for (j = iUB[i] + 1: iUB[i + 1]) do
insert new points(pointArray[j],tree); % insert new
points to tree
end do % j
end do % i
end do % k
SE (scale, m, r) = ln(nn /nd ) % Eq. (5)
release tree memory(tree);
end do % scale

data as discussed in Section 3.2, the data compression technique is applied in this algorithm.
Input:  max : the maximum scale, m: the pattern length, r:
the distance of accepting similarity, and the time series {X}
dened in Section 2.1.
Ouput: SE for scale = 1: max .
We explain the details of the implementation of Algorithm
1 in computing the MSE in natural language.
Line 1: Normalize r by the standard deviation of the original
time series in Eq. (1).
Line 3: Calculate the coarse-grained time series from the original time series by Eq. (7).
Line 4: For each (X )i , transform the template vector (Xi )m into
a d dimensional space point sets using Eq. (8).
Line 7: Input point array in space, output the indexed sorted
point array {p}. Points are stored in a structure containing
their (x,y,z. . .) coordinate. Quick sort algorithm is applied to
sort the point sets by the their x coordinate.

387

Line 8: Input {p} and r, Output the iLU and iUB array dened
Eq. (11). The detailed is discussed in step 2 in Section 3.1.
Line 9: compress data(): Input the sorted point array {p}. Output the new compressed point array. Compress the point
array. If points have the same coordinates, only the rst one
is stored in the database. The counter in the trees node
described in Step 5 in Section 3.1 is used to record the number
of points in a compressed point.
To operate the kd tree, the following tree functions are
needed:
(1) build kd tree(): Input (d 1) dimensional points; Ouput the
kd tree structure;
(2) delete old points(): this function is to delete points from
the tree. Input points to be deleted from the tree. The
tree nodes counters corresponding to these points are
implicitly subtracted from the tree and these points are
disjointed from the leaf node as discussed in Step 5 in
Section 3.1.
(3) insert new points(): similar to the delete old points().
(4) kd search(): Input the bounding box (Whd )i and the tree;
output: nm
i
(5) release tree memory(): this function is to release the trees
memory once the tree operations are nished. Input the
tree, and the memory is implicitly cleaned in this function.
The kd tree is a popular algorithm. Free computer programs
can be found in the public domain.
Line 10: build the kd tree structure.
Line 1113: for i = 1, insert the point sets into the tree.
Line 15: is the deletion operation.
Line 16: input the point array, m, and r. Output the bounding
box (Whd )i dened in Eq. (10b).
and nm+1
for each i by the (d-1) dimenLine 17: compute nm
i
i
sional kd tree algorithm as described in Step 4 in Section
3.1.
Line1822: compute nn (nd ) by taking the contribution from
(nm+1
) for each i.
nm
i
i
Line 2325: the insertion operation.
Line 28: compute SE from nn and nd using Eq. (5)
Line 29: release the memory after the computation of SE is
nished.
From Eq. (6), the index j starts from i + 1 in computing SE .
However, from Eq. (3), the index j starts from 1 in computing
AE . Then, line 15 must be modied as the following 3 lines:
for (j = iLB[i-1]:iLB[i] 1)
deleteOldPoints(pointArray[i]);
end do

3.5.

The adaptive SKD algorithm

3.5.1.

Developing the adaptive SKD algorithm

When the data length N is short (under certain threshold),

the brute force algorithm performs better than the SKD algorithm because both building and operating the tree involve
a lot of overheads. The threshold is around N = 1000 with

388

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 382396

m = 2 from numerical experiments. For short data length, the

computational time can be further improved by the following
algorithm.
Keep the sorting algorithm for the rst dimension in step (1)
given in Section 3.1. Replace the kd tree search in steps (2)(3)
given in Section 3.1 for the higher dimension by the brute force
algorithm. The computer program, which adaptively switches
from SKD to this algorithm when N is smaller than a certain
threshold, can be implemented. Therefore, this algorithm is
called the adaptive SKD algorithm. Now, the details of the
implementation of the adaptive SKD algorithm in computing
the SE in general language are explained.

3.5.2.

Algorithm 2: the adaptive SKD algorithm

In this section, we present the pseudo code of computing the

nn dened in Eq. (5) by the adaptive SKD algorithm. It can be
easily modied to compute SE as described in Algorithm 1.
Replace m by m + 1, we obtain nd . Then SE can be computed by
Eq. (5).
Input: m: the pattern length, r: the distance of accepting similarity, and the time series {X} dened in Section 2.1.
Ouput: nn
1:
pointArray = sort points by x coord(X);
[iLB,iUB] = nd lower upper index (r, pointArray)
2:
for (i = 1: N m) do
3:
4:
i idx = pointArray[i].id
for (j = i + 1:iUB[i]) do
5:
6:
j idx = pointArray[j].id
for (k = 1: m)
7:
if (abs(y[i idx + p] y[j idx + p])
8:
9:
nn (i) = nn (i) + 1
else
10:
11:
break;
12:
end do
end do % k
13:
14:
end do % j
end do % i
15:
Line 1 is exactly the same as line 7 in Algorithm 1 except
that the (y, z, . . .) coordinate is not needed in the pointArray
structure. Line 2 is exactly the same as line 8 in Algorithm
1. The pointArray structure in line 1 also contains the original index (before sorting). Thus we can remember its original
index after sorting. Line 3 to line 15 is to compute the number
of the points inside the bounding box in Eq. (9). Line 5 to 14 is to
, which is quite similar to the brute force algorithm
compute nm
i
provided in [3] except that we must transform from the sorted
coordinate system back to its original coordinate system (line
4 and 6) to perform the similarity testing for k = 1,2,3 (line 7 to
13). To compute the AE , the index j starts from 1 instead of i + 1,
then we must modify line 5 to for (j = iLB[i]:iUB[i]) do.

3.5.3.

Time complexity analysis

From the pseudo code of Algorithm 2, it can be observed that

three loops are required, namely, 1 i N m, i + 1 j iUB[i],
and 1 k m. Hence, the time complexity is
T(N) = O(m Nnbr N) = O(m Nnbr N)

(16)

where Nnbr is the average number of [iUB(i) iLB(i)] in Eq. (12),

which is approximately equal to rN where r < 1. Then, the
time complexity is T(N) = O(mrN2 ). It can be easily shown that
T(N) = O((m + 1)N2 ) for the brute force algorithm. On comparing these two algorithms, it can be observed that they are
both O(N2 ) algorithms, but the adaptive SKD algorithm has
a smaller coefcient and, hence, is faster than the brute force
algorithm. The required storage is the data and the pointArray structure, which are both O(N). Therefore, the memory
consumption is O(N).

4.

Experiments

This section presents the experimental results to verify the

performance of the SKD algorithms and its adaptive version in
computing sample, approximate, and multiscale entropy for
different parameters. These parameters include the nature of
the signal, data length (N), scale, pattern length (m), distance
(r), and data resolution (B). The following signals are tested in
this experiment:
1. an 11-bit ECG signal (from MIT-NIH database) (Fig. 1)
2. an RR (cardiac interbeat interval) time series, which is
extracted from the ECG signal sampled at 250 Hz
3. an 8-bit EEG (electroencephalography) signal
4. integer-type random signals uniformly distributed
between 0 and 1 (white noise)
5. a real-type pink noise
6. a 16-bit electro-oculography (EOG) signal.
First, the execution time versus N plot as the TN plot is
dened. Subsequently, Tbruteforce , Tbucket , Tkd , TSKD , and Tadp skd
as the execution time of the brute force [3], bucket-assisted
[9], kd tree [13], SKD, and adaptive SKD algorithm, respectively, are dened. The efciency of an algorithm is (T )algo =
Tbruteforce /Talgo , where Talgo is the execution time for an algorithm. The brute force algorithm takes into account the
symmetric property of Eq. (2) and is highly optimized. It is
equivalent to the improved basic algorithm dened in [9].
A higher (T ) indicates that the algorithm is faster. The execution time and the efciency of the SKD algorithm and its
adaptive version have been compared with those of the algorithms proposed in the literature. The numerical experiment
is performed on a 1.5-GHz Intel CPU with 1.5 GRAM, running
Windows XP. The computer program was developed in the C
language.

4.1.

Experiment 1

This experiment has tested the performance of the SKD algorithm in computing SE (scale = 1, m = 2, r = 0.15 SD).
Fig. 4 shows the TN plot for the RR series, while Fig. 4a
shows the TN plots for the brute force, kd-tree, and the
SKD algorithm. It can be observed that both the kd and
the SKD algorithms are much faster than the brute force
algorithm. However, the comparison of the SKD algorithm
with the kd tree algorithm cannot be clearly observed from
Fig. 4a. Thus, they are re-plotted n Fig. 4b. For N = 80,000,
Tbruteforce = 55 s, Tkd = 2.1 s, and TSKD = 0.45 s. In other words,

389

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 382396

RR
120

SampEn time (seond.)

100
80
60
40

Brute

20

kd
0

SKD
10000

20000

30000

40000

50000

60000

70000

80000

90000

1e+05

1.1e+05

90000

1e+05

1.1e+05

RR

3.5

SampEn time (second)

3
2.5
2
1.5

kd
1

SKD
0.5
0
10000

20000

30000

40000

50000

60000

70000

80000

N
Fig. 4 (a) Execution times versus N for RR intervals and SE (scale = 1, m = 2, r = 0.15 SD). The black squares and line represent
the execution times of the brute force algorithm, the blue crossed lines represent the corresponding values for the kd tree
algorithm, and the red crossed line represents the corresponding values for the SKD algorithm. (b) Execution times versus N
for RR intervals and SE (scale = 1, m = 2, r = 0.15 SD). The blue circles and line represent the corresponding values for the kd
tree algorithm, and the red crossed line represents the corresponding values for the SKD algorithm.

SKD = 122 and kd = 26, which shows that the SKD algorithm
is signicantly faster than the kd tree algorithm and the brute
force algorithm. Furthermore, Fig. 4b shows that the SKD is
approximately an O(N) algorithm, as can be seen that the SKD
curve is approximately a straight line.
Figs. 57 show the TN plot for the ECG, EEG, and the 8-bit
random signal. Similar results are obtained as in the RR series.
In these three cases, the SKD is found to outperform the kd tree
algorithm by 2.55 times and the brute force algorithm by more
than 100 times for N = 80,000. Note that for N = 80,000, TSKD for
the 8-bit random signal is about 3 times slower than the others.
For the random signal, x and y are not correlated; thus, the
points are evenly distributed in the Poincare plot (see Fig. 2).
Therefore, Ndiff for the random signal is much larger than that
of a biological signal, thereby taking longer execution times.
Fig. 8(a) shows the TN plot for different r. It shows that TSKD
increases slightly with r. As nm
increases with r, the depth of
i
the kd tree also increases. Therefore, (T)SKD increases with r.

Fig. 8(b) shows the TN plot for different m. It shows that

the (T)SKD increases with m initially and converges (stabilizes)
to one curve nally (m 8).
Fig. 8(c) shows the TN plot for different b (bit number) for
the random signal. The time complexity of the SKD for m = 2,
as discussed in Section 3.3, is bounded by O(B N) if N > B2 . The
following is observed: rst, (T)SKD increases with b for xed N.
Second, if N
B2 , then the TN plot is close to a straight line.
Third, if the condition N
B2 fails, then the SKD algorithm
would switch to an O(N3/2 ) algorithm, which is then independent from b. For this reason, the TN plots converge (stabilize)
to one curve for N > 80,000 and b > 10.

4.2.

Experiment 2

This experiment tests the performance of the SKD in computing AE (scale = 1, m = 2, r = 0.2 SD). Fig. 9a and b shows the
TN plot for the RR series and the EEG signal, respectively. To

390

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 382396

ECG Signal

SampEn time (second)

120
100
80
60
40

Brute

20

kd

SKD
10000

20000

30000

40000

50000

60000

70000

80000

90000

1e+05

1.1e+05 1.2e+05

number of points(N)

ECG Singal

SampEn time (second)

kd

SKD

0
10000

20000

30000

40000

50000

60000

70000

80000

90000

1e+05

1.1e+05 1.2e+05

N
Fig. 5 (a) Execution times versus N for the ECG signal and SE (scale = 1, m = 2, r = 0.15 SD). The black circles and line
represent the execution times of the brute force algorithm, the blue crossed lines represent the corresponding values for the
kd tree algorithm, and the red crossed line represents the corresponding values for the SKD algorithm. (b) Execution times
versus N for the ECG signal and SE (scale = 1, m = 2, r = 0.15 SD). The blue circles and line represent the execution times for
the kd tree algorithm and the red crossed line represents the corresponding values for the SKD algorithm.

make a fair comparison of the performance of the SKD with

that of the bucket-assisted algorithm developed by Manis [9],
(T )SKD and the estimated (T )bucket assist were measured. As can
be seen from Fig. 4 of [9], (T )bucket assist for both RR and EEG signals are less than 4 for N = 80,000. Fig. 9(a) and (b) shows that
(T )SKD = 106 s/0.61 s = 173 for RR, and 105 s/0.38 s = 276 for EEG,
which indicates a signicant improvement over the bucketassisted algorithm. On comparing Fig. 4 with Fig. 9(a) and Fig. 6
with Fig. 9b, one can observe that computation of AE is about
two times more than that of SE . The reason is that when using
Eq. (6) in computing AE , the index j starts from 1 instead of j + 1
as in computing SE , and more logarithms are involved in Eq. (4).
To mimic the computations of the complexity of the DNA
sequence [3,11], the approximate entropy AE (scale = 1, m = 2)
of a 1-bit random signal is computed. Fig. 9(c) shows the TN
plot using the SKD algorithm. Table 2 shows the execution
times versus m for N = 106 using the SKD algorithm. It shows
that (T)SKD < 3 s for N = 106 , while Tbruteforce is about 12 hours by
extrapolation from smaller data sets. In other words, (T )SKD
=

1.6 104 . It is hard to compare the execution time of the SKD

algorithm with that of the algorithms used in [12], because
they use different hardware devices, and the algorithm is not
provided in that paper. Nevertheless, the execution time was
approximately 2 minutes in [12] and less than 3 s by the SKD
algorithm for N = 106 . Section 3.3 shows that the SKD algorithm
is bounded by O(Bm1 N). For the binary sequence, the resolution B = 2. From Eq. (15), the computational cost for the SKD is
bound by O(2m1 N). That is, O(2N) for m = 2 and is O(1 N) for
m = 1, which is close to the theoretical limit: O(1 N).

Table 2 Execution times versus m for a binary

sequence for AE (scale = 1, N = 106 ).
m

SKD (s)

1
2
3

2.31
2.66
2.86

391

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 382396

EEG signal

a
SampEn time (second)

120
100
80

Brute

60
40
20

kd
0

SKD
10000

20000

30000

40000

50000

60000

70000

80000

90000

1e+05

1.1e+05

1.2e+05

1e+05

1.1e+05

1.2e+05

EEG Signal

1.2

SampEn time(second)

1
0.8
0.6

kd

0.4

SKD

0.2
0
10000

20000

30000

40000

50000

60000

70000

80000

90000

N
Fig. 6 (a) Execution times versus N for the EEG signal and SE (scale = 1, m = 2, r = 0.15 SD). The black squares and line
represent the execution times of the brute force algorithm, the blue crossed lines represent the corresponding values for the
kd tree algorithm, and the red crossed line represents the corresponding values for the SKD algorithm. (b) Execution times
versus N for the EEG signal and SE (scale = 1, m = 2, r = 0.15 SD). The blue circles and line represent the execution times for
the kd tree algorithm and the red crossed line represents the corresponding values for the SKD algorithm.

4.3.

Experiment 3

This experiment is to test the performance of the SKD and

the adaptive SKD algorithms. The rst test is the EEG signal
in computing SE (scale = 1:1, m = 2, r = 0.15 SD). The results for
N 20,000 are listed in Table 3. Table 3 shows that the adaptive
SKD algorithm is the fastest one in the range of approximately
3008000. (T )adp skd is around 5 for N = 500, and is around 10 for
N around 4000. Furthermore, (T )adp skd is higher than (T )SKD
for N approximately less than 8000 and is higher than (T )kd
for N approximately less than 10,000.
The second test is the 1/f (pink) noise in computing MSE
(scale = 1:20, m = 2, r = 0.15 SD). The results for the data length
ranging from 300 to 0.8 million are listed in Table 4. Table 4
shows that (T )SKD is around 40 for N = 100,000 and 160 for
N = 800,000. For N 1000, the benet of SKD is not clear, as discussed in Section 3.5. It can be observed that the adaptive SKD

algorithm is faster than the SKD algorithm for N < 20,000, and
is faster than the brute force algorithm for N approximately
greater than 400. Furthermore, the adaptive SKD algorithm
is faster than the kd tree algorithm for N < 8 105 . The result
is surprising, because the adaptive SKD algorithm has been
designed for only handling short data length. On comparing
Tbruteforce and Tadp skd , the adaptive SKD is consistently 1015
times faster than the brute force algorithm for N 4000. The
O(N2 ) property of the adaptive SKD can be veried as follows.
The execution time increases approximately by 4 times when
N is doubled as can be seen from Tables 3 and 4.

4.4.

Experiment 4

To test the SKD algorithm and its adaptive version in the handling of time-varying MSE, 16-bit EOG data collected overnight
are partitioned into nonoverlapping windows to analyze the

392

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 382396

Random

SampEn execution time(sec.)

140
120
100
80
60

Brute

40
20

kd

0
10000

20000

30000

40000

50000

SKD

60000

70000

80000

90000

1e+05

1.1e+05 1.2e+05

1e+05

1.1e+05 1.2e+05

Random: 8 bit

SampEn time(second)

kd
4

SKD
0
10000

20000

30000

40000

50000

60000

70000

80000

90000

N
Fig. 7 (a) Execution times versus N for the 8-bit uniform distributed random signal and SE (scale = 1, m = 2, r = 0.15 SD). The
black squares and line represent execution times of the brute force algorithm, the blue circle lines represent the
corresponding values for the kd tree algorithm, and the red crossed line represents the corresponding values for the SKD
algorithm. (b) Execution times versus N for the 8-bit uniform distributed signal and SE (scale = 1, m = 2, r = 0.15 SD). The blue
circles and line represent the execution times for the kd tree algorithm and the red crossed line represents the
corresponding values for the SKD algorithm.

relation between the MSE and the sleep stages. Each window contains 7680 (30 s 256 Hz) data points and the MSE
procedure is repeated for each window. The MSE analysis of
the rst 30,000 s of one subjects EOG is shown in Fig. 10.

Fig. 10(a) shows the MSE values of different scales (x-axis:

time, y-axis: scale, color: MSE value), while Fig. 10(b) shows
the results of averaging the 20 MSE values in each window.
Fig. 10(c) shows the manual sleep scoring by the expert, and

Table 3 Execution times versus N for EEG for m = 2, r = 0.15 SD, scale = 1 for the brute force, kd tree, SKD, and adaptive
SKD algorithms.
Number of points

Brute force (s)

kd tree (s)

300
500
1000
2000
4000
8000
20,000

8.5 104
2.5 103
9.1 103
3.6 102
0.14
0.57
3.63

2.3 103
4.2 103
6.5 103
1.4 102
0.031
0.066
0.18

SKD (s)
1.8 103
1.8 103
4.0 103
7.3 103
0.018
0.035
0.077

Adaptive SKD (s)

2.4 104
5.0 104
1.3 103
3.5 103
0.014
0.041
0.23

393

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 382396

EEG
0.7

SampEn time (second)

0.6

r=0.3

0.5

r=0.2

0.4

r=0.15

0.3

r=0.1
0.2

r=0.05
0.1

20000

30000

40000

50000

60000

70000

80000

90000

1e+05

1.1e+05

1.2e+05

RR
5

SampEn time (second)

m=8,10
3

m=4

m=3

m=2
0

m=1

20000

30000

40000

50000

60000

70000

80000

90000

1e+05

1.1e+05

1.2e+05

SampEn time (second)

Random Signal
2

b=14,15
b=12

1.5

b=10

b=8
0.5

0
20000

b=6
b=1
30000

40000

b=2
50000

b=4
60000

70000

80000

90000

1e+05

1.1e+05

1.2e+05

Fig. 8 (a) Execution times versus N for the EEG signal for different r using the SKD algorithm in computing SE (scale = 1,
m = 2). (b) Execution times versus N for the RR series for different m using the SKD algorithm in computing SE (scale = 1,
r = 0.15). (c) Execution times versus N for the random signal for different number of bits using the SKD algorithm in
computing SE (scale = 1, m = 2, r = 0.15).

394

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 382396

RR

a
7

Apen time (second)

Brute

5
4
3
2
1
0

SKD
10000

20000

30000

40000

50000

60000

70000

80000

90000

1e+05

1.1e+05 1.2e+05

EEG

b
7

Apen time (second)

6
5
4
3

Brute

SKD

1
0
10000

20000

30000

40000

50000

60000

70000

80000

90000

1e+05

1.1e+05 1.2e+05

Binary

Apen execution time (sec)

30
25
20
15
10
5

SKD
0
1e+06

2e+06

3e+06

4e+06

5e+06

6e+06

7e+06

8e+06

9e+06

1e+07

Fig. 9 (a) Execution times of computing AE (scale = 1, m = 2, r = 0.2 SD) versus N for the RR intervals. The black squares and
line represent the execution times of the brute force algorithm, and the red crossed line represents the corresponding
values for the SKD algorithm. (b) Execution times versus N for the EEG intervals and AE (scale = 1, m = 2, r = 0.2 SD). The black
circle and line represent the execution times of the brute force algorithm, and the red crossed line represents the
corresponding values for the SKD algorithm. (c) Execution time versus N for the 1-bit random signal for AE (scale = 1, m = 2).
The red crossed line represents the corresponding values for the SKD algorithm.

395

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 382396

Table 4 Execution times versus N for 1/f noise for m = 2, r = 0.15 SD, scale = 1:20 for the brute force, kd tree, SKD, and
adaptive SKD algorithms.
Number of points

Brute force (s)

kd tree (s)

SKD (s)

Adaptive SKD (s)

300
500
1000
2000
4000
8000
20,000
50,000
105
2 105
4 105
8 105
16 105
32 105

0.0019
0.0075
0.015
0.059
0.29
0.921
5.39
32.4
128.4
501.0
2063
8716

0.014
0.017
0.032
0.059
0.13
0.29
1.01
4.18
15.16
51.3
176.7
655.0

0.012
0.010
0.02
0.039
0.08
0.16
0.46
1.34
3.15
7.35
18.5
52.5
153
411

0.0021
0.00521
0.0066
0.0093
0.026
0.07
0.42
2.14
9.01
34.2
146
540

the scores dening wake, rapid eye movement (REM), stages 1

(S1) and 2 (S2), and slow wave sleep (SWS) are 0, 1, 2, 3,
and 4, respectively. The correlation coefcient between the
average MSE values and the manual scoring of sleep stages
can reach 0.7628. This result demonstrates the feasibility of
applying MSE to sleep staging. The execution times of the

brute force method, the SKD, and the adaptive SKD algorithms to analyze 8-h EOG data (7,365,376 samples) were 320,
40.0, 24.5 s, respectively. With such a level of performance,
this experiment demonstrates that the SKD algorithm can be
integrated into a chip in a consumer product for online diagnosis.

Fig. 10 The relation between the MSE and the sleep stages. (a) The MSE values using EOG of different scales of one subject;
(b) results after averaging the 20 MSE values in each epoch; (c) manual sleep staging reviewed by the expert.

396

4.5.

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 382396

Summary

The experiment veries that the SKD algorithm and its adaptive version are robust for all parameters including different
biomedical signals, data length (N), data resolution (B), pattern
length (m), distance (r), and scales.
The SKD algorithm is O(N3/2 ) for m = 2 and for the real-type
data, and is O(N) for the integer-type data as in the digitalized biological signal. From all the experiments, it is observed
that the SKD algorithm is signicantly better than the kd tree
algorithm, the bucket-assisted algorithm, and the brute force
algorithm in the literature by a factor of 26, more than 20, and
more than 100, respectively, for N = 80,000. For small N, the SKD
is observed to be inferior to the brute force algorithm. For this
reason, an adaptive SKD algorithm has been developed and it
is more than 10 times faster than the brute force algorithm for
r = 0.15 and N 2000.

5.

Conclusions

Sample entropy and approximate entropy are measurements

of complexity used to analyze biomedical and other signals. The time complexity of the algorithms proposed in
the literature requires O(N2 ), which is not fast enough for
many applications. This paper has developed a sliding kd
tree algorithm to reduce the time complexity. For a typical
value of pattern length m = 2, the time complexity is reduced
to O(N3/2 ) for the real-type data, and is reduced to O(BN)
for the integer-type data as in the digitalized biomedical
data. Experimental results show a signicant improvement
more than 100 times when compared with the conventional brute force method, and 26 times than that of
the kd tree algorithm for N = 80,000. The adaptive SKD algorithm was also developed to handle short data length. The
adaptive SKD algorithm is more than 10 times faster than
the conventional brute force method for N 4000. As the
execution time is substantially reduced, the SKD (adaptive
SKD) algorithm can be integrated into a chip in a consumer
product for online diagnosis and can efciently compute
the complexity of biological signals over long periods of
time.

Conict of interest
None.

Acknowledgment
This work was funded in part by the Industrial Development
Bureau Ministry of Economic Affairs, Taiwan (ROC).

references

[1] S.M. Pincus, Approximate entropy as a measure of system

complexity, Proceedings of the National Academy of
Sciences 88 (1991) 22972301.
[2] J.S. Richman, J.R. Moorman, Physiological time series
analysis using approximate entropy and sample entropy,
American Journal of Physiology 278 (6) (2000) H2039H2049.
[3] M. Costa, A.L. Goldberger, C.K. Peng, Multiscale entropy
analysis of biological signals, Physical Review E 71 (2005)
021906.
[4] L. Angelini, R. Maestri, D. Marinazzo, L. Nitti, M. Pellicoro, G.
Pinna, S. Stramaglia, S.A. Tupputi, Multiscale analysis of
short term heart beat interval, arterial blood pressure, and
instantaneous lung volume time series, Articial
Intelligence in Medicine 41 (2007) 237250.
[5] R. Hernndez-Prez, L. Guzmn-Vargas, A. Ramrez -Rojas, F.
Angulo-Brown, Multiscale entropy analysis of electro
seismic time series, Natural Hazards and Earth System
Sciences 8 (2008) 855860.
[6] R. Yan, R.X. Gao, Approximate entropy as a diagnostic tool
for machine health monitoring, Mechanical Systems and
Signal Processing 21 (2007) 824839.
[7] M. Costa, A.L. Goldberger, C.K. Peng, Multiscale entropy
analysis of complex physiologic time series, Physical Review
Letters 89 (2002) 068102.
[8] M. Costa, C.K. Peng, A.L. Goldberger, J.M. Hausdorff,
Multiscale entropy analysis of human gait dynamics,
Physica A 330 (2003) 5360.
[9] G. Manis, Fast computation of approximate entropy,
Computer Methods and Programs in Biomedicine 91 (2008)
4854.
[10] K. Sugisaki, H. Ohmori, Online estimation of complexity
using variable forgetting factor, in: Proceedings of the SICE
Annual Conference, 2007, pp. 16.
[11] S. Pincus, B.H. Singer, Randomness and degrees of
irregularity, Proceedings of the National Academy of Science
93 (March) (1996) 20832088.
[12] S. Pincus, R.E. Kalman, Not all (possibly) random
sequences are created equal, Proceedings of the National
Academy of Science 94 (1997) 35133518.
[13] Y.H. Pan, W.Y. Lin, Y.H. Wang, K.T. Lee, Computing multiscale
entropy with orthogonal range search, Journal of Marine
Science and Technology 19 (2011).
[14] J.L. Bentley, Multidimensional binary search trees used for
associative searching, Communications of the ACM 18 (9)
(1975) 509517.
[15] J.L. Bentley, Decomposable searching problems, Information
Processing Letters 8 (1979) 244251.
[16] Y.H. Pan, Y.H. Wang, K.T. Lee, An optimal two-dimensional
orthogonal range search algorithm in VLSI design
automation, in: Proceedings of the International
Symposium on Computer, Communication, Control and
Automation (3CA IEEE), May, 2010, 2010, pp. 5356.
[17] F. Marcaino, M.L. Migaux, D. Acanfora, G. Furgi, F. Rango,
Quantication of Poincare maps for the evaluation of heart
rate variability, IEEE Computer in Cardiology (1994) 577580.