CW - COMP1108 - 520 - hh0764s - 20150520 - 141707 - 1415.PDF (1.23MB) - UpFile - VN

Final Year Project
EXTRACTING HEART BEAT THROUGH HEAD MOTION

Student: Ha Trung Hieu
Banner ID: 000865979
Supervisor: MSc Pham Van Vung
May 2015
Words count: 8400
Ha Trung Hieu - EXTRACTING HEART BEAT THROUGH HEAD MOTION

331 - FPT UNIVERSITY, HANOI, VIETNAM | HIEUHTGT00342@FPT.EDU.VN
BSC (HONS) COMPUTING
1. ABSTRACT
One of the indicators of health are of primary concern to evaluate and determine the health
status of the person that is the heartbeat. There are many ways to measure the heart rate of people
rely on the achievements of science and technology, they are applied on a heart rate monitor such
as ECG (Electrocardiography). Image processing technology applied to the recognition of the change
of capillaries on the fingers, the color change on the forehead which is also to measure heart rate.
Recently, scientists at MIT have devised a study can get your heart rate recorded through head
movement from the video was recorded by phone, camera, etc. This project will develop technology
based on the results of research and scientific reports of the scientists at MIT. The MITs paper is
Detecting Pulse from Head Motions in Video [1]
Technical extract from the video is the heart rate is based on measuring the motion of the
head due to the impact of the flow of blood to be pumped at each beat. The movement of the head
and the flow of blood to comply with Newton's laws. Method of scientists at MIT are monitoring
points and doing principal component analysis (PCA) to degrade their trajectory into a set of moving
parts. Then select the best components corresponding to the heart rate based on its frequency.
This document will briefly discuss the approach and conduct research based on the research
of scientists at MIT. This project has been studied and implemented in the python script and using
graphics processors OpenCV library. It will then detail the steps that were studied during the project
implementation and the role of research applications. Finally, a conclusion will be drawn as the pros
and cons of using this technology in the measurement of the heart rate.
HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION
2. ACKNOWLEDGEMENTS
I want to thank to all those who help and support their enthusiasm in this project:
My supervisor Pham Van Vung with invaluable lessons, knowledge, the enthusiastic guidance and
feedback quickly. My best friend Vu Cong Thanh who always feedback during the analysis, design,
development and testing stages of this project.
Thanks to my grandmother, who brought ideas and goals to implement this Project. I choose a
project related to health, after completion, it will be a tool to help me and the family can monitor
her health and everyone in the family a simple and intuitive way possible.
Thank the teachers at universities FPT (FPT Greenwich) who have given us many lessons and
experiences. Thank the friends, parents, brothers and sisters in the family spent time caring and
enabling me to fulfill all these projects. They are not afraid of difficulties and hard to help me. I
sincerely thank you!
3. TABLE OF CONTENTS
1.
ABSTRACT ................................................................................................................................................................. 1
2.
ACKNOWLEDGEMENTS ............................................................................................................................................ 2
3.
TABLE OF CONTENTS ................................................................................................................................................ 3
4.
INTRODUCTION ........................................................................................................................................................ 4
5.
LITERATURE REVIEW ................................................................................................................................................ 6

5.1
Background ....................................................................................................................................................... 6
5.1.1
Head Movement ........................................................................................................................................... 6
5.1.2
Beat-to-Beat Variability................................................................................................................................. 8
5.2
Method Technology. ......................................................................................................................................... 9
5.3
Face Detecting With Viola Jones Algorithm And Region Selection. ................................................................ 10
5.4
Selection and Tracking Feature Points ............................................................................................................ 14
5.4.1
Feature Points Selection ............................................................................................................................. 14
5.4.2
Feature Points Tracking Using Lucas Kanade Optical Flow Algorithm ........................................................ 18
5.5
Applying Cubic Spline Interpolation Algorithm to Increase Sampling Rate. ................................................... 22
5.6
Temporal Filtering Using 5th Order Butterworth Filter. .................................................................................. 24
5.7
Retaining Most Stable Features Using Confidence Intervals of Distribution. ................................................. 25
5.8
PCA Decomposition......................................................................................................................................... 27
5.9
Peak Detection ................................................................................................................................................ 30
6.
PROBLEM AREA ...................................................................................................................................................... 31
7.
EVALUATION ........................................................................................................................................................... 31
8.
CONCLUSION .......................................................................................................................................................... 34
9.
TABLE OF FIGURES .................................................................................................................................................. 35
10.
REFERENCES ........................................................................................................................................................ 36
4. INTRODUCTION
Heart rate is an important indicator of medical diagnosis. This date with the development of
society, there is a lot of heart disease, blood pressure seriously affects the health and longevity of
people in the world. With the development of science and technology, more and more interest in
the dissection of the heart rate information and do not need direct contact with objects such as
other ECG traditional gauges. The application of these techniques do not measure the impact on the
body will be very useful when doctors performed on infants, elderly people who are very difficult to
work with them like other adults, or those with vulnerable skin by traditional measurement
methods. There are many ways to measure heart rate and track new changes in the capillaries on
the forehead or the fingertips, combined with image processing technology. Moreover, as the
population is aging quickly, the regular monitoring of the equipment outside the hospital can give
physicians timely information patterns but also easily identify trends and statistical analysis. The
techniques in the future will reduce the cost of transportation, procurement of equipment for
hospitals and health centers, which will be easily accessible to the poor residential areas, and lack
of facilities.
In this project, we exploit fluctuations in a very "delicate" cycle of head to extract information
about the activities of the heart from a video was filmed by
telephone or a device average normal recording. It's a very
interesting study from scientists at MIT in the past year. In
addition to providing an unobtrusive way to measure heart rate,
this method can also be used to extract useful clinical
information about activity of the heart, such as the change in
length each heart rate, which is related to the health of the
autonomic nervous system.
In addition to moving a deliberately controlled by the
brain or the external impact, our head there is an unintentional
movement. These motions cyclical of blood pumped from the
heart to the head via the abdominal aorta and carotid arteries.
FIGURE 1 BLOOD WAS FLOWING FROM THE

HEART PUMPING AND FLOW TO THE HEAD
THROUGH TWO ARTERIES ON EITHER SIDE [2]
(Figure 1). The position of relatively deep artery, at the wrist only, live legs, temples, and both sides
of the neck relatively shallow. At the site can see circuit basis, which the doctors can know the
situation of the arteries. Far the largest artery through the heart valve. The blood vessels that due
to multiple muscle elasticity constitutes, can stretch or bend. In arterial blood is bright red under
pulsed flow. Therefore, we can easily self-heartbeat measured through the comments on both sides
of the front of the neck just below the angle of the jaw. To limit the physical impact on the body, the
measurement of the change of the head position will help us to measure heart rate. The algorithm
is applied to detect the pulse signal from the time blood is pumped from the heart to the arteries.
The approach of the original algorithm that uses an algorithm to determine the Viola Jones
faces, since faces are recognizable place and occupied most of the area on head. Then, use Lucas
Kanade algorithm to figure out the best features in the handle that so that tracking the variability
through each frame in a video image. After the data has been the characteristic of a person, the next
step of the algorithm is the velocity filter of a band, techniques used in this section is Temporal
Filtering using 5th order butter worth filter. Finally the algorithm will use the principal component
analysis (PCA) to find a best periodic signal caused by the pulse in the arteries. Then apply the
algorithm to find the highest point of the pulse, which will have an average value of heart rate.
Before implement the project, the techniques will be described in the following section it will
help to analyze the pulse rate based on a series of rhythm from the recording sequence of head
movements. The results compare with other devices is relatively accurate.
5. LITERATURE REVIEW
5.1 Background
In this background section will provide brief information on some key topics related to the
study. First, the key elements of a cardiac cycle, what they are, and how they work, and thanks
to the elements that make the body and head movements. Finally, some clinical significance of
heart rate variability are discussed.
5.1.1 Head Movement

As stated above in the introduction, when a person is at rest, for example, sit on a chair. The
first part of them will move, and the movement includes two movements are conscious
movement, and the movement has no consciousness (other ingredients that act on the motion).
The conscious movement is deliberate change of body state as shaking his head, leaned forward,
backwards, left or right, this is the big moves easily see with the naked eye. The no consciousness
movement is the movement by acting through the heart pump blood from passing through the
abdominal aorta and carotid arteries in the neck and on the head. This motion to change the very
small, very hard to detect with the naked eye. When the movement that happens, the body that
tend to gravitate towards the front or the rear, with the command of the brain controls the
muscles for balance. From then make a series of moves in a certain period without interruption.
According to the biological knowledge about the human body, a healthy heart is a hollow, it's
just a bit bigger than the fist. It pumps blood continuously through the circulatory system through
the body by repeating the contraction in a certain cycle. The circulatory system is a network of
elastic blood vessels that carry blood through the body. It includes the heart, lungs, aorta, artery
(small aorta) and capillaries (tiny blood vessels). The aorta carries blood to turn the upper body,
such as head, neck through the common carotid artery, carotid artery from which the blood will
continue to migrate to other regions of the head.
Research discovers the vertical direction is the best shaft to determine the motion of the head
on the body caused by the pumping process. While the horizontal axis is the change of state
swaying. To see the details of the move, we made some simple calculations to apply Newton's
second law and III, ignoring many details about fluid dynamics and physiology. We have:
With the heart of a healthy body, in the process pump may be 13ml (0.013 kg) blood in speed
is 0.02 m/s in 0.1s, and the acceleration is 0.9 m/s2 [5]. Assuming both carotid arteries are the
same, and the heads weight is 5kg. In III Newton's law indicates that the individual does not
appear forces that appear in pairs-jet dynamics. In other words, the force appears only when
there is an interaction between two or more things together. Pair this force, the law more clear
III, the driving force for the pair. We have the same magnitude but opposite directions. In the
interaction between two objects Head and B, if A exert a force to B, then B also causes a
force to A and
=
Apply II and III Newtons law we have.
| | = | |
| | = | |
| 5 | = |0.9 2 0.013 |
| | = | (0.9 * 0.013) / 5) |
| | = 0.00234 ( 2)
Since the acceleration of the movement of the head, we determine the distance traveled by
the head of the body. By transporting blood from the heart to the arteries is a modified linear
motion are, we consider the trajectory of the blood flow from the heart to the head is straight
and instantaneous velocity of blood or raise them or is declined steadily over time. From there
we have the formula to calculate the distance change were:
=
=
1
2
1
2
2
0.00234 0.12 *1000
s = 0.0117 millimeters
We can see, the movement is very small.
5.1.2 Beat-to-Beat Variability.

The heart rate variability or HRV for short, is a stage of critical health check possible you've
never experienced. HRV involves a series risk such as stress, depression, diabetes, cardiovascular
disease, sleep disorders and abdominal fat. For some people, it acts as a catalyst for major HRV
will be the cause of many problems and diseases is a red alert.
Moreover, according to the cardiologist, ideal HRV indices associated with frequent orgasms!
Speaking more understandable, HRV is some change in a certain time period of the heartbeat.
This can mean anything? Imagine your heart rate, for examples, the number of times your heart
beats per minute, the same rhythm to a song, to extend and keep pace. HRV notice more
complex rhythms, enabling acceleration or deceleration flexibility by changing the elapsed time
between each pulse.
To understand how reduced HRV increases, the first need to have an autonomic nervous
system (ANS). ANS would strongly affect the main parts of the body, including the heart, there
are divided into two branches. Affiliates parasympathetic slows the heart rate and sympathetic
branch helps raise your heart rate. The two leaders discussed the tasks depending on whether
you're tense, relax or battling illness.
If you are in a state of
stress or prolonged sick, the
sympathetic
branch
will
govern, putting you in the

mood amidst fighting and
FIGURE 2 AN EXAMPLE OF THE ECG SIGNAL FROM THE MACHINE. THE DIFFERENCE
BETWEEN THE R PEAK IN THE PULSE IS USED TO CALCULATE HEART RATE VARIABILITY OR
HRV [3]
give up, will reduce your

HRV. Low HRV is a sign
warning you may develop
other symptoms. Patients with decreased HRV increased risk of arrhythmia, since it is very easy
to kill.
5.2 Method Technology.

This method requires the input
request is a recorded video changes a
person's head and then the result
returned is a heart rate as well as a series
of point beating of the heart rate. Since it
can be used to analyze the beat-to-beat
variability. The first step of the method is
to identify the face to be treated, then
proceed to take the facial features based
on an algorithm. Then proceed to monitor
the change of position x and y coordinates
of each frame that point until the end of
the length of the input video above. After
having been gathered variation of the
feature points, the next step is to increase
the density (smoothing) which gathers
variation
by
interpolation
method.
Because, above input video file, the video

recorded with handheld devices such as
travel cameras, phones, most of them
have very low speed, only about 25-30
frames/s. After proceedings interpolated
smoothing the data, the data set that is
FIGURE 3 OVERVIEW OF THE TECHNIQUE. [4]
passed through the interference filters,
filter change point in an unusual way. Then proceed PCA algorithm to select the set of variations
in the most appropriate location. The Figure 3 show the overview of the method. From the data
after running the PCA algorithm, data is transformed into the frequency spectrum, from which
to select a clear frequency spectrum. The average heart rate is determined using these
frequencies.
5.3 Face Detecting With Viola Jones Algorithm And Region Selection.
The problem of determining a human face (Face detection) is a computer technique for
determining the positions and the size of the face in any photo (digital image). This technique
identifies the featured faces and ignore other things such as buildings, trees, body, etc.
Some applications of the problem is determined faces: system interaction between humans
and machines (Control computer through facial movements), identification system (enables
security agencies managed humans), observing system monitoring and management system of
the entry for agencies and companies, the system checks the driver is asleep or not, the system
analysis of facial emotions, and facial recognition system for digital cameras, etc.
There are so many methods of facial recognition, which is based on the nature of the method,
can be divided into four main approaches are as follows: The approach is based on knowledge:
knowledge of coding people in all kinds of faces and create sets of rules to determine the face.
The approach is based on the book description does not change: the goal of finding algorithm
described structure featured a human face (the book represents no change to the position,
location of capture device, brightness to change, etc.). The approach is based on the matched
sample: Using the standard sample or the faces of people featured. The approach is based on
appearance: methodological training from a set pattern to identify human faces.
Program to locate human faces in this article using approaches based on appearance, using
strong AdaBoost classifier is a combination of the weak classifier based on Haar-like poses
challenge to determine face. Method development objects and Paul Viola Michael Jones has
suggested in their article "Rapid Object Detection using a Boosted Cascade of Simple Features"
in 2001. It is a machine learning method based on numerous data sets positive train (there
appear face), negative (no facial appearance).
To algorithm works well, we need a large amount of training data set includes positive and
negative images. Then we conducted dissection of features from them. With this algorithm, Viola
and Join announced an algorithm to detect the characteristic is known under the name Haarlike. There are four basic characteristics to identify human faces. Each Haar-like feature is a
combination of two or three rectangular "white" or "black" as shown below:
10
FIGURE 4 FOUR BASIC CHARACTERISTICS TO IDENTIFY HUMAN FACES

To use this characteristic to identify human faces, 4 featured Haar-like base is expanding, and is
divided into 3 features set as follows:
FIGURE 5 3 FEATURES SET OF HAAR-LIKE

Each feature is a single value is calculated by subtracting the total number of pixels of the
black rectangle and the total number of pixels of white rectangle. Thus we can see that, for
calculating the value of Haar-like features, one must calculate the sum of the pixels in the image
area. But to calculate the value of the Haar-like feature for all positions on the image requires
considerable computing costs, not met for applications that require run-time. Therefore Viola
and Jones launched a concept called Integral Image, that is a 2-dimensional array with a size
equal to the size of the image to calculate the Haar-like features, with each element of the array
is calculated by aggregating upper pixel (row-1) and left (column-1) of it. Starting position the
upper left to the lower position, right image, this calculation solely on simple addition integer,
so the execution speed is very fast.
11
FIGURE 6 INTEGRAL IMAGE

Having calculated the Integral Image, the sum of the gray values of a particular area on any
image made very simple way:
Suppose we need to calculate the total value of the gray level D as in Figure 7, can be
calculated as follows:
D = A + B + C + D - (A + B) - (A + C) + A
With A + B + C + D is the value at the point P4 on Integral Image, likewise A + B is the value at
point P2, A + C is the value at point P3, and A is the value at point P1. So we can rewrite the
above expression D calculated as follows:
FIGURE 7EXAMPLES OF FAST CALCULATION OF THE GRAYSCALE VALUES OF POINT D IN THE IMAGE
Next, to choose the Haar-like features for setting thresholds, Viola and Jones uses a machine
learning method AdaBoost. AdaBoost will incorporate the weak classifier to form a strong
classifier.
12
The weakness of this algorithm is a very long process of learning machine, but strong again shows
that enjoyment is the face detection in images very quickly, in line with the current needs of the
Project. OpenCV has hosted a lot of money training classifications for the identification and
classification of face, eyes, smile, etc. The XML file data is stored in training OpenCV. We need to
do is load the XML classification, then transmit an image input gray.
After you have identified the face, in the course of running the algorithm, there may be errors
in a frame as there appear two faces have different sizes (In fact, the only one), we will proceed
to remove the face with a smaller area, just take the face with the largest area. Because we use
cameras placed close to the subject's face to record the faces of that object.
We will choose 50% of the width of the rectangle
from the midpoint of edge width and 90% height of
the upper edge from the first term to ensure that the
whole rectangle lies in the facial area. To avoid
creating in additional interference signals from the
unwanted Movements like the blink of an eye, the
movement of the pupil. We proceed to remove the
rectangle containing the left and right eyes. To do
this, we remove the rectangle with width sizes as
above, but the length of the start from 20% to 55%
FIGURE 8 SELECTION AREA
compared with the original length. In the Figure 8.
13
The red area is a removing region, the rectangle green border is a selection region. We will
extract features points of this region.
FIGURE 9 REGION FACE SELECTION
5.4 Selection and Tracking Feature Points

5.4.1 Feature Points Selection
As we have calculated above in section 5.1.1, when blood is pumped from the heart to, the
head will be moved compared to original position a distance of about 0.0117 millimeters. This
gap is very small to be seen with the naked eye.
After identifying the faces and the data processing algorithm is presented in section 5.3
above. We have been measuring the movement of the head through the video by searching out
new features in the face processing and tracking their changes through frame by frame. In the
process of understanding, being aware of the feature points to get the point using a recognition
14
algorithm which is Shi-Tomasi Corner Detector. This algorithm based on Harris Corner Detector
which are some small changes make a better result.
Mathematical model of Harris Corner Detector algorithm is presented summarized as follows:
Suppose we have a grayscale image I, with each point (u, v) and displacement (x, y) we can
calculate the deviation when a window move from (u, v) to (u + x, y + v) as follows:
(1)
In particular, S(x, y) Total value deviation squared, w(u, v) is a window in (u, v), I(u, v) and
I(u + x, v + y) is the light intensity value of the pixel at the position (u, v) and I (u + x, v + y).
I(u + x, v + y) may develop Taylor as follows:
(2)
Ix, Iy is the derivative components of x, y. Since then, (1) can be rewritten as follows:
(3)
If expressed as a matrix, then S (x, y) as follows:
(4)
In particular, A is a tensor structure as follows:
1 and 2 are called the eigenvalues of A. Then, the expression:

(5)
The value Mc will decide if the window w containing corner or not. In late 1994, J.Shi and
C.Tomashi has researched and produced a small change in the formula (5) above:
= min(1 , 2 )
15
If this ratio is greater than a threshold value, then it is considered a corner. To demonstrate the
algorithm in a more figurative way. Suppose we have a picture, see the movement of the
windows in the following figure:
FIGURE 10 ILLUSTRATES USE SHI-TOMASI ALGORITHM DETECTOR CORNER IN PICTURES OF HO CHI MINH MAUSOLEUM [7]
The green window corresponding displacement amounts (x, y) from the red window. On the
picture we see no significant differences between the two windows. But consider the following:
FIGURE 11 ILLUSTRATES USE SHI-TOMASI ALGORITHM DETECTOR CORNER IN PICTURES OF HO CHI MINH MAUSOLEUM [7]
Meanwhile with any small movement, S (x, y) will be changed quite large. The purpose of a series
of equations (2), (3), (4), (5) is to find out the change S (x, y) the largest (corresponding to the
appearance of the corner). From equation (5), if Mc larger than a certain value, the window will
contain the corner.
16
The algorithm Shi-Tomasi Corner Detector has been implemented in the OpenCV library (the
cv2.goodFeaturesToTrack() function). As usual, this function will require the input is a grayscale
image, then you determine N number of strongest corners in that gray picture. Then continue
you determine the level of quality, the minimum Euclidean distance between the detection
corners. Level of quality is a value from 0 to 1, the value used to determine a window there is a
corner or not. If the value of Mc in formula (5) is greater than this value, it will be a corner. Even
if smaller, it will be removed.
In implementation of this project, the number of corners should consider 1500 in face
processing areas. Level of quality is 0.05 (this coefficient is the empirical coefficient). The
Euclidean distance is 0.5.
FIGURE 12 EXAMPLE OF THE RESULT OF APPLYING SHI-TOMASI CORNER DETECTOR
17
The feature points (red points) will be used to monitor changes in the next frame position
compared to their original position through each frame of video. From then shows the variation
of the point position, it is also regarded as the movement of the head.
5.4.2 Feature Points Tracking Using Lucas Kanade Optical Flow Algorithm
From the feature points (the red point) in the results of the above algorithm, we conduct such
tracking points in the video frame by frame. In Frame 1 we obtain the feature points (e.g. the
feature points in the results above), we conducted tracking them to see the change from frame
t=2...T. (T is the total frame in a video). In each frame t, we have N of the feature points, each
point n we feature our coordinates were ( Xn(t), Yn(t) ). The tracking is done by algorithm Lucas
Kanade Optical Flow.
Optical flow is a concept that refers to the relative motion of the points on the surface of an
object, the object that caused, in terms of a landmark observation (eye, camera, etc.). The
movement of the object (which may actually be considered as the movement of the points on
the surface of that object) in 3-dimensional space, they are projected onto a 2D plane observers
called motion field. Generally, the purpose of the optical flow Estimation method is to determine
(approximately) motion field from a set of frame image changes over time, they are widely used
in object segmentation problems, motion detection, tracking, etc.
The key idea of the method of calculation of optical flow based on the following assumptions
gradient: Appearance of objects without much change (in light intensity - intensity) when
considered from frame t to frame t+1. There are a lot of method determine motion field. LucasKanade method is one effective method to calculate the motion. The main content method is
summarized as follows:
Some conditions of the algorithm:
1. Light intensity of pixel constant t,
2. Slightest movement
3. Neighboring pixels and moving at the same speed.
About conditional offer of algorithms, data, we fully met, the facial movements are small, and
people sit in front of the camera is guaranteed to light intensity, without any significant changes.
18
At time t pixel (x, y) color value is I (x, y, t), after time t of pixel position is (x + x, y + y) and I
(x, y, t) = I (x + x, y + y, t + t) (for conditions 1).
That I(x + x, y + y, t + t) I (x, y, t) +
Therefore, we have
Or
Put =
We have
; =
= 0
=0
= 0 (*)
The equation (*) is a quadratic equation in two unknowns least Vx, Vy. The 3rd condition
provided, we can use the pixel values adjacent to the above equation. Then we have m equations
system with m>= 2
Solve simultaneous equations so determined by the minimum squares method (Least-Quares)

we get the velocity vector of a moving object.
19
FIGURE 13 FLOW CHART OF IMPLEMENT LUCAS KANADE ALGORITHM [6]
Algorithms have been deployed strong support of OpenCV library, using a function called
calcOpticalFlowPyrLK() in OpenCV library. The parameter passed to the algorithm includes an
initial gray image, the feature points of image points that gray (The collection of feature points
have been processed from step 5.4.1), and a next frame gray image. The algorithm will return a
list of the status of the feature points in the new gray photo frame. The status of the feature
point includes 0 and 1, which is the first state means the feature point which has appeared in the
next frame, 0 means lost. We iteratively pass these next points as previous points in next step.
Below is a graph of statistical variation first three feature points throughout the course of
handling video. Y-axis coordinate of the Y, X axis coordinates of video time.
20
FIGURE 14 TRACKING POINT NUMBER 1
The algorithm uses a cell window size 40x40 pixels around each of the feature points to follow
the movement of their location. The size of this window is a rectangular area 1/125 faces found
in section 5.3. It is large enough to get the point feature in the area around the nose and mouth.
Not all the features are appearing in point the next frame, the majority of them will be lost, and
the loss of the feature point does not change too much the quality of the calculations in this
project. The cause of the loss of information of the feature points were:
1. A feature point falls outside of the image.
2. An image patch around the tracked point varies too much between images
21
A video of us have about 10 seconds equivalent length 300 frames (30 frames / s), after applying
the tracking algorithm, we obtained the following results tracking (the majority of the feature
points will decrease)
Phase 3: Processing Tracked Points
-- Tracked 1014 Feature Point in frame 0/300
5.5 Applying Cubic Spline Interpolation Algorithm to Increase Sampling Rate.

With camera phones now, most of the technology only supports video recording with up to
30 frames per second (about 30Hz). With the average number of frames like this, there can be
fully recorded information on heartbeat. Looking at Figure 12, 13, and 14. We can see, the data
is very fragmented and varies a great way. With data like this, the application of the technique
will face limitations on data quality, can occur many errors or even not to get the desired results.
While the ECG machine, they operate approximately 250Hz range to capture information
about heart rate variability. To get much more data sample, the study of the document indicates
that it applies an interpolation technique to increase the data sample. The technique is Cubic
Spline Interpolation in this study.
In the method using a straight line
connecting two points can be considered as
a first-order polynomial (first derivative is a
straight line). If instead of using the most
we use higher order, the most common is
tertiary, it can force the third derivative of
this first derivative at the junction equal.
This method is commonly known under the
FIGURE 17 DIFFERENCE BETWEEN LINEAR AND CUBIC SPLINE
name "cubic spline", which is very popular
INTERPOLATION
22
in engineering way. The graph above shows the difference between linear and cubic spline
interpolation.
This algorithm has been deployed in scipy library using python script. The interp1d class in
scipy.interpolate is a convenient method to create a function based on fixed data points which
can be evaluated anywhere within the domain defined by the given data using linear
interpolation. An instance of this class is created by passing the 1-d vectors comprising the data.
The instance of this class defines a __call__ method and can therefore by treated like a function
which interpolates between known data values to obtain unknown values. Behavior at the
boundary can be specified at instantiation time. The following example demonstrates its use, for
cubic spline interpolation.
FIGURE 18 IMPLEMENT CUBIC SPLINE INTERPOLATION USING SCIPY PYTHON
The code above is deployed in this project, with the aim of increasing the amount of data samples
obtained from the processing of video. For example here, the video was filmed in about 10
seconds with a frame rate of 30 frame / s (equivalent to 10 * 30 = 300 frames). But meanwhile,
the amount of data we need here is about 250 frames / s (equivalent to 10 * 250 = 2500 frames).
Therefore, to make the data more finely, we need to split the density axis of time (t) up. The
density was calculated using the formula 30/250 = 0.12. The three charts below are the results
after Cubic Spline Interpolation algorithms run to increase the sample data of three feature
points are taken from Section 5.4.2.
23
FIGURE 19 FEATURE POINT NUMBER 1 APPLIED CUBIC SPLINE
FIGURE 20 FEATURE POINT NUMBER 2 APPLIED CUBIC SPLINE
INTERPOLATION
INTERPOLATION
FIGURE 21 FEATURE POINT NUMBER 3 APPLIED CUBIC SPLINE INTERPOLATION
5.6 Temporal Filtering Using 5th Order Butterworth Filter.

Not all the frequencies of the change of position is useful to detect the pulse. In a normal adult
in a resting state, the average heart rate usually falls in the range [0.75, 2] Hz, equivalent to [45,
120] beats / minute. According to research by MIT scientists, the changes in the lower frequency
0.75Hz usually very adversely affect the results of the program returns. The cause of this motion
because respiration and movement of the head should not have been said in section 5.1.1. But
conversely, the higher frequency 2Hz to provide a lot of useful information about the heart rate,
which in order to find the peak. Because we only consider the location coordinates y (t) of each
feature point. Should we proceed through the techniques 5th Order Butterworth Filter to conduct
data filter frequency range is [0.75, 5] Hz. Another study showed that, for infants, they can beat
faster than adults, so bands applied to the filter to be changed to [1.25, 5] Hz. This algorithm has
been developed in scipy.signal.butter library python, we will use this library to create a band pass
24
filter Butterworth. To generate the filter coefficients for a band pass filter, we using butter()
function the filter order, the cutoff frequencies Wn=[low, high] (expressed as the fraction of the
Nyquist frequency, which is half the sampling frequency) and the band type btype="band". The
low frequency has been set 0.75, and high has been set 5 Hz. The chart below gives us the results
after applying the filter method of 1 feature points, better data and better quality.
FIGURE 22 DATA OF A FEATURE POINT AFTER APPLYING 5TH BUTTER WORD BANDPASS FILTER
5.7 Retaining Most Stable Features Using Confidence Intervals of

Distribution.
Many feature points have moved unstable variable, points to retain the most stable feature,
we use a method that is relatively classic distribution based on Gausses. After calculating the
25
maximum distance traveled between successive frames of the feature points. We just took the
feature point with a distance exceeding the 75th percentile.
A popular normal distribution problem involves finding percentiles for X. That is, you are given
the percentage or statistical probability of being at or below a certain x-value, and you have to
find the x-value that corresponds to it. A percentile isn't a percent. A percent is a number
between 0 and 100; a percentile is a value of X (maximum distance).
This process is carried out as follows:
1. Find out the list of the greatest distance of the points under the frame.
2. Then look mean and sigma modeled distributions Gause.
3. Calculate the confidence interval based on value index and mean sigma which has been
found above.
4. Get the feature points have been validated through normal distribution.
Probability distribution classes are located in scipy.stats. The standard deviation is the square
root of the average of the squared deviations from the mean, i.e., std = sqrt(mean(abs(x x.mean())**2)). The average squared deviation is normally calculated as x.sum() / N, where N =
len(x). Compute the standard deviation along the specified axis using numpy.std function. This
function will be returned the standard deviation, a measure of the spread of a distribution, of the
array elements. The standard deviation is computed for the flattened array by default, otherwise
over the specified axis. Compute the arithmetic mean along the specified axis using numpy.mean()
function. This function will be returned the average of the array elements. The average is taken over
the flattened array by default, otherwise over the specified axis.
When we perform the analysis on a sample and average values of the results obtained, we
estimate the real value of the sample. Although this average value represents the best estimate of
the true value but it is still only an estimate. We can calculate confidence intervals for these analyzes
to demonstrate the accuracy of the analysis. The confidence interval is determined by limiting the
upper and lower confidence levels. The chart below represent the distribution of a distance of the
feature points, and determine confidence intervals.
26
FIGURE 23 RETAINING MOST STABLE FEATURES USING CONFIDENCE INTERVALS OF DISTRIBUTION.
5.8 PCA Decomposition

(Principal Component Analysis - PCA) is one method of multivariate data analysis simple. In
statistics, the usual need to "study" the data before building the model inference based on that
data. But sometimes high dimensional data, cannot visualize in 2 or 3-dimensional space, so it
needs to find ways to put the data space of smaller dimension.
PCA is one such method, but more than that, PCA was like a Swiss knife with many good
features:
27
1. Help reduce the dimensionality of the data.

2. Instead of retaining the axes of the old space, PCA build a new low-dimensional space,
but it also is capable of performing data equivalent good old space, means ensuring
variation (variability) of data on each new dimension.
3. The axes of the new space is a linear combination of the old space, so in terms of
semantics, PCA build new features based on the features observed. Point or are still
performing well this feature original data.
4. In the new space, the potential link data can be discovered, which if put in the old space,
it is difficult to detect, or the link thus not reflected.
Briefly, the objective of PCA is to find a new space (with smaller dimensional space of the
former). The axes of the new space is built so that on each axis, the variance of the data on it is
the largest possible. The objective of the PCA is: maximize the variability.
FIGURE 24 ILLUSTRATIONS PCA: PROJECTION ON DIFFERENT AXES CAN GIVE VERY DIFFERENT VIEWS OF THE SAME DATA.
A classic example is the picture of the camel. Together as a camel but if viewed from
the side, we have enough information, while viewed from the front, it's hard to say it
is camel. A more convincing example is illustrated in the following figure.
28
FIGURE 25 ILLUSTRATIONS PCA: FIND NEW COORDINATE DATA SO THAT THE HIGHEST VARIABILITY
Assuming the initial data set (file green points) was observed in 3-dimensional space
(black shaft) as shown on the left. Clearly, this 3-axis is not performing the best level of
data variability. PCA will therefore find new coordinate system (the red axes in the picture
left). After finding a new space, the data will be moved into this space to be represented
as shown on the right. Apparently the right image just 2 axes but better representation of
data variability compared with the original 3-D axes. A very nice point of PCA are more
quadrants in the new space ensures double orthogonal to each other, although in the original
space, the axels may be not orthogonal.
The basic signal source is the motion of the head, which are caused by the cardiovascular
pulse. The trajectory of the feature points is a collection of many of the movement change as
respiratory, expression changes in the face, or other deliberate action. We have to decompose
the mix moves into a smaller collection to isolate the signal pulse. To do this we consider the
multidimensional position of the head at each frame as a separate data point and use PCA to find
a set of main dimensions along which the position varies. We then select a dimension on which
to project the position time-series to obtain the pulse signal.
Formally, given N feature points, we represent the N dimensional position of the head at
frame t as m t = [y 1 (t), y 2 (t), , y N (t)]. The mean and the covariance matrix of the positions
are:
29
=
||
=1
1
= (
)(
)
=1
PCA finds the principal axes of variation of the position as the eigenvectors of the covariance
matrix:
=
Where m denotes a diagonal matrix of the eigenvalues 1, 2 N corresponding to the
eigenvectors in the columns of m, 1, 2, , N. Each eigenvector represents the Ndimensional direction and magnitude of movement for the feature points. The eigenvectors
differ for each subject. We obtain the 1-D position signal si (t) by projecting the position timeseries onto i:
We picked out five objects s1, s2, s3, and s4, s5 after the PCA, to proceed with identifying
peaks and find the heart rate. The maximum frequency of the chosen signal as an approximate
the pulse rate as 60 / fpulse beats per minute.
5.9 Peak Detection

A heartbeat is not lonely, to fully evaluate a complete cardiovascular system, doctors often
consider the index beat-to-beat (presented in section 5.1.2). To do so, we need to get the signal
peak on the newly selected PCA. We now have almost enough data because after applying a
Cubic Spline algorithm Interpolation to increase value from 30Hz up to 250Hz sample (As
according to the ECG machine). The peaks are close to
1
fpulse
seconds apart with some variability
due to the natural variability of heartbeats, variations of the head motion, and noise. We label
each sample in the signal as a peak if it is the largest value in a window centered at the sample.
30
We set the length of the window (in samples) to be round (
250
). The figure below is an example
fplulse
of beat-to-beat data from PCA chose.
FIGURE 26 EXAMPLES OF MOTION SIGNALS OUTPUTTED BY OUR METHOD. WHITE CIRCLES ARE PEAKS.
6. PROBLEM AREA
The biggest difficulty when implementing this project is the difficulty in determining the
topics and self-study techniques that the scientists at MIT have made. With a huge amount of
knowledge to really catch fast in such a short time: The treatment of facial recognition and
detection of feature points based on the tracking algorithm, polynomial interpolation, PCA, etc.
Second, based on the ideas of inventors at MIT, there is no precedent or applications outside
of life, should experience performing fledgling. The papers and documents relating limited and
unclear information.
Finally the time limit for implementing the scheme, in short time, cannot avoid making
mistakes.
7. EVALUATION
In modern society, the application of advanced techniques for monitoring the health of the
people is indispensable. With the rise of science and technology as well as the popularity of smart
phones. The implementation of this project to the smart device is feasible and in the near future.
The heart rate monitors or monitor current health equipped primarily in large hospitals, they
have cost the expensive and bulky hard move, depends very much on the electrical equipment.
It is these weaknesses, the health problems in remote areas, far not been adequately care.
In addition, the use of a heart rate monitor to present a direct impact on people, such as
attaching the sensor to the people, this will cause difficulty and monitor health care for children
and adults old.
31
By analyzing video just to get the heart rate information, which will promise to bring a new
era of health monitoring, detection of crime, etc. Doctors do not need to directly measure the
heart rate of patients, they just sit at home and watch the patient via online video.
There's also a number of other advantages when conducting manufacture or use of heart rate
measurement technology compact and convenient:
1. Walk, exercise: Not only athletes needed new peak heart rates were measured by the normal
exercise can also make use of training to increase efficiency. By maintaining optimal heart
rate range for the reduction of fat, you can optimize training time to burn calories more.
2. Jogging: Measuring heart rate helps you maintain the intensity at the highest level in the days
weight training or aerobic hold mode set lighter these days. Some types also have more
advanced warning function dehydration or hypoglycemia.
3. Bicycles: With heart rate measuring device, you can keep track of the physical situation in the
different training regime type as downhill racing, sprint, on different terrains as asphalt,
terrain, roads Pass. The dedicated type also provides many other parameters when
connected to a bicycle speed meter.
4. Mountaineering, skiing, adventure: For hiking, using cardio equipment help deliverables
reasonable effort to reach the top. For skiers, it helps to check the excitement and thrill
overcome dangerous obstacles
5. Weight loss: Measuring heart rate improves daily exercise and set reasonable diet. Most of
the devices are functional display calorie released when the collective wage and help
determine light intensity consistent weight training to achieve the best weight loss effect.
6. Recovering from injury: With the ability to display information instantly, a heart rate monitor
is useful for physicians to help patients recover from injury or illness, particularly concerning
cases of heart. Thereby can guarantee health recovery safely and stable.
Within the scope of this project, we only provide a prototype version. See this prototype was
developed on the platform python language, and the OpenCV library. Some features are
presented in this report, has not been completed in the prototype as Signal Selection, Peak
Detection function still works less well.
32
In the near future, after the end of this project, the entire source code of the program will be
shared free of charge to the community under the GNU GPL license. These missing features or
activities are not well developed and perfected. With the desire to create an application that
serves a medical purpose to the community, to help track diseases and health problems of the
places and the poor.
33
8. CONCLUSION
Based on research by computer scientists from MIT, they have discovered a link
between the heart's rhythm and bean variation of head and body. From this infinitesimal
movement, has helped us to obtain information about the heartbeat and the health-related
information.
Through monitoring the movement of the feature point by a motion tracking algorithm, then
perform analysis principal component (PCA), to identify significant shifts and most close to
your heart rate. Thereby heartbeat selected based on the frequency spectrum of its time.
Through assessment and testing on 10 subjects, the results of this project is nearly identical
to the heart rate is measured by the ECG machine. From useful medical information, the
doctors can get information on clinical heart rate variability. Thereby making preliminary
conclusions about heart disease.
34
9. TABLE OF FIGURES
FIGURE 1 BLOOD WAS FLOWING FROM THE HEART PUMPING AND FLOW TO THE HEAD THROUGH TWO ARTERIES ON EITHER SIDE [2] .............................. 4
FIGURE 2 AN EXAMPLE OF THE ECG SIGNAL FROM THE MACHINE. THE DIFFERENCE BETWEEN THE R PEAK IN THE PULSE IS USED TO CALCULATE HEART RATE
VARIABILITY OR HRV [3] .................................................................................................................................................................. 8
FIGURE 3 OVERVIEW OF THE TECHNIQUE. [4] ................................................................................................................................................ 9
FIGURE 4 FOUR BASIC CHARACTERISTICS TO IDENTIFY HUMAN FACES ................................................................................................................ 11
FIGURE 5 3 FEATURES SET OF HAAR-LIKE .................................................................................................................................................... 11
FIGURE 6 INTEGRAL IMAGE ...................................................................................................................................................................... 12
FIGURE 7EXAMPLES OF FAST CALCULATION OF THE GRAYSCALE VALUES OF POINT D IN THE IMAGE ......................................................................... 12
FIGURE 8 SELECTION AREA ...................................................................................................................................................................... 13
FIGURE 9 REGION FACE SELECTION ............................................................................................................................................................ 14
FIGURE 10 ILLUSTRATES USE SHI-TOMASI ALGORITHM DETECTOR CORNER IN PICTURES OF HO CHI MINH MAUSOLEUM [7] ...................................... 16
FIGURE 11 ILLUSTRATES USE SHI-TOMASI ALGORITHM DETECTOR CORNER IN PICTURES OF HO CHI MINH MAUSOLEUM [7] ...................................... 16
FIGURE 12 EXAMPLE OF THE RESULT OF APPLYING SHI-TOMASI CORNER DETECTOR............................................................................................ 17
FIGURE 13 FLOW CHART OF IMPLEMENT LUCAS KANADE ALGORITHM [6] ........................................................................................................ 20
FIGURE 14 TRACKING POINT NUMBER 1..................................................................................................................................................... 21
FIGURE 15 TRACKING POINT NUMBER 2 ..................................................................................................................................................... 21
FIGURE 16 TRACKING POINT NUMBER 3 ..................................................................................................................................................... 21
FIGURE 17 DIFFERENCE BETWEEN LINEAR AND CUBIC SPLINE INTERPOLATION ..................................................................................................... 22
FIGURE 18 IMPLEMENT CUBIC SPLINE INTERPOLATION USING SCIPY PYTHON ...................................................................................................... 23
FIGURE 19 FEATURE POINT NUMBER 1 APPLIED CUBIC SPLINE INTERPOLATION .................................................................................................. 24
FIGURE 22 DATA OF A FEATURE POINT AFTER APPLYING 5TH BUTTER WORD BANDPASS FILTER .......................................................................... 25
FIGURE 23 RETAINING MOST STABLE FEATURES USING CONFIDENCE INTERVALS OF DISTRIBUTION....................................................................... 27
FIGURE 24 ILLUSTRATIONS PCA: PROJECTION ON DIFFERENT AXES CAN GIVE VERY DIFFERENT VIEWS OF THE SAME DATA. ........................................... 28
FIGURE 25 ILLUSTRATIONS PCA: FIND NEW COORDINATE DATA SO THAT THE HIGHEST VARIABILITY ........................................................................ 29
FIGURE 26 EXAMPLES OF MOTION SIGNALS OUTPUTTED BY OUR METHOD. WHITE CIRCLES ARE PEAKS. ................................................................... 31
35
10.
REFERENCES
[1] Balakrishnan, G., Durand, F. and Guttag, J. (2013). Detecting Pulse from Head Motions in Video. 2013 IEEE
Conference on Computer Vision and Pattern Recognition.
[2] Institute, M. (2015). Angiogram. [online] Mayfieldclinic.com. Available at: http://www.mayfieldclinic.com/PEANGIO.htm#.VVwKEdLtmko [Accessed 20 May 2015].
[3] Ni.com, (2015). Using LabVIEW for Heart Rate Variability Analysis - National Instruments. [online] Available at:
http://www.ni.com/example/30832/en/ [Accessed 20 May 2015].
[4] Balakrishnan, G., Durand, F. and Guttag, J. (2013). Detecting Pulse from Head Motions in Video. 2013 IEEE
Conference on Computer Vision and Pattern Recognition, Figure 2 - page 3.
[5] HN. Sabbat et al. Noninvasive evaluation of left ventricular performance based on peak aortic
blood acceleration measured with a continuous-wave Doppler velocity meter. Circulation, 74,
1986
[6] Kanade - Lucas - Tomasi (KLT) Feature Tracker - Computer Vision (EEE6503) Fall 2009, Yonsei
Univ.
[7] Wikimapia.org, (2015). Lng Ch tch H Ch Minh (Lng Bc) - Wikimapia. [online] Available at:
http://wikimapia.org/14596244/vi/L%C4%83ng-Ch%E1%BB%A7-t%E1%BB%8Bch-H%E1%BB%93-Ch%C3%ADMinh-L%C4%83ng-B%C3%A1c [Accessed 20 May 2015].
36

CW - COMP1108 - 520 - hh0764s - 20150520 - 141707 - 1415.PDF (1.23MB) - UpFile - VN

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

CW - COMP1108 - 520 - hh0764s - 20150520 - 141707 - 1415.PDF (1.23MB) - UpFile - VN

Загружено:

Авторское право:

Доступные форматы

Final Year Project

EXTRACTING HEART BEAT THROUGH HEAD MOTION

Ha Trung Hieu - EXTRACTING HEART BEAT THROUGH HEAD MOTION

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

TABLE OF CONTENTS ................................................................................................................................................ 3

LITERATURE REVIEW ................................................................................................................................................ 6

Head Movement ........................................................................................................................................... 6

Method Technology. ......................................................................................................................................... 9

Selection and Tracking Feature Points ............................................................................................................ 14

Feature Points Selection ............................................................................................................................. 14

Applying Cubic Spline Interpolation Algorithm to Increase Sampling Rate. ................................................... 22

Temporal Filtering Using 5th Order Butterworth Filter. .................................................................................. 24

Retaining Most Stable Features Using Confidence Intervals of Distribution. ................................................. 25

Peak Detection ................................................................................................................................................ 30

PROBLEM AREA ...................................................................................................................................................... 31

TABLE OF FIGURES .................................................................................................................................................. 35

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

FIGURE 1 BLOOD WAS FLOWING FROM THE

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

5.1.1 Head Movement

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

0.00234 0.12 *1000

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

5.1.2 Beat-to-Beat Variability.

govern, putting you in the

give up, will reduce your

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

5.2 Method Technology.

Because, above input video file, the video

passed through the interference filters,

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

FIGURE 4 FOUR BASIC CHARACTERISTICS TO IDENTIFY HUMAN FACES

FIGURE 5 3 FEATURES SET OF HAAR-LIKE

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

FIGURE 6 INTEGRAL IMAGE

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

compared with the original length. In the Figure 8.

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

FIGURE 9 REGION FACE SELECTION

5.4 Selection and Tracking Feature Points

1 and 2 are called the eigenvalues of A. Then, the expression:

FIGURE 12 EXAMPLE OF THE RESULT OF APPLYING SHI-TOMASI CORNER DETECTOR

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

Solve simultaneous equations so determined by the minimum squares method (Least-Quares)

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

FIGURE 13 FLOW CHART OF IMPLEMENT LUCAS KANADE ALGORITHM [6]

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

FIGURE 14 TRACKING POINT NUMBER 1

FIGURE 15 TRACKING POINT NUMBER 2

FIGURE 16 TRACKING POINT NUMBER 3

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

5.5 Applying Cubic Spline Interpolation Algorithm to Increase Sampling Rate.

name "cubic spline", which is very popular

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

FIGURE 18 IMPLEMENT CUBIC SPLINE INTERPOLATION USING SCIPY PYTHON

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION

FIGURE 19 FEATURE POINT NUMBER 1 APPLIED CUBIC SPLINE

FIGURE 20 FEATURE POINT NUMBER 2 APPLIED CUBIC SPLINE

FIGURE 21 FEATURE POINT NUMBER 3 APPLIED CUBIC SPLINE INTERPOLATION

5.6 Temporal Filtering Using 5th Order Butterworth Filter.

5.7 Retaining Most Stable Features Using Confidence Intervals of