A Review on Eulerian Video Magnification

for Heart Beat Extraction

DonnyMirza Adhitama
School of Electrical Engineering and Informatics, Institut Teknologi Bandung, Bandung
Indonesia
dm.adhitama@gmail.com
Abstract This paper reviews the many applications for
extracting the human heart beat or usually represent as
pulse rate with indirect measurement method. The
method takes a standard video with image sequence in
it as an input, applied spatial decomposition following
by temporal filtering that is called Eulerian Video
Magnification. With the method, some parameters
from the human body can be the source of data for
human heart beat extraction. There are many sources
that have been researched with some researcher, like
head motion, facial color variation, and neck & wrist
color variation. Using the method, all of those sources
would amplify those subtle changes both of motion and
color variation and amplify it.
Keywords Heart Beat, Pulse Rate, Eulerian Video
Magnification, Spatial-temporal analysis

I. INTRODUCTION

Human body has many information that we can

obtain, especially the information about their health.
Sometimes to obtain the information, there will be
required various devices with sensor or electrode
patch on the human body directly so the device can
read and process the signal obtained by the sensor.
But in the last decade, the indirect obtaining
information procedure for the human body has
getting considerable to be used in medical facility for
flexibility and efficiency purpose, so eventually
medical employer wouldnt have to patch the sensor
device directly to the patient body. They just install
the stationary camera in the addressed room, and at
the display of the camera there will be some health
information of the patient.
With the Eulerian Video Magnification (EVM)
method, this very information can be revealed
considering to obtain the information of the source is
usually invisible for human eye, and it is have to be a

variation like motion or movement and color

variation, it has to be amplified so the variation could
be see with the naked eyes. As we see the variation of
the source, then we can consider it as a source of the
information that we will extract with this method.
II. GENERAL EVM METHODS

EVM method combines the spatial and temporal

processing to obtained many invisible information in
the human body. After this combination processing,
there would be a subtle change, either motion or
another variation like facial color variation, and so
on. Then, this subtle change will be amplified so we
can see the change with naked eyes.
Here the input is the video sequence from a stationary
camera that record any part of body. Before we enter
the EVM process, firstly the system segments a
reference frame (pixels) based on properties, for
example like the similarities in the color and/or the
correlated motions. The filtering later will allow to
find the highest correlation, both in the temporal and
spatial domain. That will be most of EVM method
that be used as shown in figure 1.

Figure 1. Overview of the EVM framework [3]

1|Donny Mirza Adhitama-23214341

The method uses Eulerian specification flow field

theory, that allows the observer to looking to the one
specific point or location of the flow (river stream,
etc) in the space that flows as the time passes to know
how the fluid motion is. With this observation theory,
EVM method has the same way of observation. This
EVM method, takes a standard video with the image
sequence as a input, detect the location of interest of
the video sequence as the observation point, applies
spatial decomposition, followed by temporal filtering
to the frames. The resulting signal of the previous
process is then amplified to reveal its hidden
information, that is why it is called video
magnification.
For the application example, Balakrisnan et al [8]
exploit the very subtle head motion or oscillation that
represent its cardiac cycle that carries information
about cardiac activity and then they extract its
information. The information which they obtained is
the heart rate. It means that this very subtle motion
change of the head will be amplified so its change
will be revealed. But as the general EVM methods
that usually being used, before amplify the subtle
motion, first they extract the motion of the head using
feature tracking. It allows the system to tracking the
core parameter that the system will process to extract
its final output.
Besides the motion variation, the other source which
can be used for the heart beat extraction is color
variation. For example is a color variation in human
skin. This color varies at some short time in the
subtle change with the blood circulation inside the
skin and this variation is invisible to the naked eye
and also it can be exploited to be extracted in pulse
rate[2]. Its color variation located in many region of
the human limb. In [5], the authors introduce another
application of using EVM method to gain some
information with the human body as a source of
information. They exploit the change of facial color
variation that also has subtle change. With this small
change they simply need to amplify the variation of
the facial color via temporal filtering. Using the
amplified variation, the signal output could be
processed to gain the pulse rate with their postprocessing in front of EVM method thereafter.

There is also innovation, in a novel technique from

Poh et al.[2], applied in the same application as we
mentioned before. At the very first demonstration,
they were using low-cost webcam for recording the
data of the face. This new technique performed heart
rate measurement from three participants recording
video simultaneously, with an automated contact-free
heart rate measurement, and motion-tolerant.
The last, for heart beat extraction application is with
the neck and wrist variation color as the source for
acquisition data. It is using the same way as the
previous application. Using the color variation for
each of these subject and then amplify its color to get
the information about the heart beat of the subject.

III. PULSE DETECTION FROM HEAD MOTION

Balakrisnan et al. explain that the human head

actually has a subtle motion caused by the Newtonian
reaction (Newtons Third Law) to the influx of the
blood at the each heart beat. It motion oscillate which
is represent the cardiac cycle of the human body, and
it is also contain about the information of the cardiac
activity. That is the one of many things that they can
obtained for the research, to gain the heart beat.

Figure 2. Blood flows from the heart to the head via the carotid
arteries on either side of the head [8]

2|Donny Mirza Adhitama-23214341

Balakrisnan et al. were using the algorithm which

detects pulse from the cylical movement. The basic
approach that they were made is to track the feature
points on a human head, filter the velocities by a
temporal frequency band of interest, and then use
principal component analysis (PCA) to find a
periodic signal that caused by pulse.After the
periodic signal is found, extract an average pulse rate
from this signal by examining its frequency spectrum
and obtain exact beat locations with a simple peak
detection algorithm. This is the very first they
describe a novel technique that extracts a pulse rate
and series of beat sequences from video recordings of
head motion.
As explained before, PCA is being used to
decompose the trajectories of the head motion into a
set of independent source signals that describe the
main elements of the motion. To choose the correct
source for analysis and computation of the duration
of individual beats, the frequency spectrum is
examined and the source with clearest main
frequency is selected. Next step, the pulse rate that
being identified using this frequency is being
averaged. To be more specific analysis and exact
calcution of beat durations, the system performed
peak detection in time-domain.
In this work, they considered the frequency and
variability of the pulse signal. However, head
movement can offer many other information about
the cardiac cycle besides the heart rate information.

Figure 3. Overview of pulse estimation approach. (a) A region is

selected within the head and feature points are tracked for all
frames of the video, (b) The vertical component is extracted from
each feature point trajectory, (c) Each trajectory is then temporally
filled to remove extraneous frequencies, (d) PCA decomposes the
trajectories into a set of source signals s1, s2, s3, s4, s5, (e) The
component which has clearest main frequency is selected, and (f)
Peak detectin identifies the beats of the signal[8].

The movement of the flows is cylical. It is derived

from the blood which is take place in the heart and
then move to the head via abdominal aorta and
carotid arteries, as shown as figure 2, and that is why
the human head moves in a periodic motion.

IV. PULSE DETECTION FROM FACIAL COLOR

VARIATION

In this chapter, other subtle change of the human

body part will be explained. This change is facial
color variation. In [5], the authros had presented the
application of EVM for facial blood flow assesment
using magnification of the facial changes variation
of redness color face for pulse rate detection.

3|Donny Mirza Adhitama-23214341

Their basic approach is to consider the time series of

color values at any spatial location, then its variation
in the temporal frequency band of interest is
amplified. For example in Figure 4, a band of
temporal frequencies that includes human heart rates
is being automatically selected, and then amplified.
The amplification reveals the variation of redness
color as the blood flows that going through the face.

result of the analysis is presented on the upper left panel (red color
variation with the detected peaks). The lower pael shows the ECG
recording analysis that resulted in R-peaks detection[1].

Miljkovic et al. in [1] demonstrated the same way

using a simple webcam for pulse rate detection. As
showed in Figure 4, they were done offline in Matlab
(The Mathworks, Natick, USA) Electrocardiography
signals that filtered with notch filter (50 Hz), and
with bandpass Butterworth 10th order filter with the
cut-off frequencies of 1 Hz and 100 Hz in order to
reduce the artifacts. Then, a simple threshold R-peaks
detection in order to obtain the pulse rate is applied.
All detected R-peaks were visually investigated for
the accuracy (the detected R-peaks) in ECG are
presented in lower right panel in Figure 4.
Next, the resulting video signal with 2nd order
bandpass Butterworth filter with cut-off frequencies
of 0.67 and 4 Hz as recommended in [9] is being
filtered. Then, the summation of red color pixels
within each frame of filtered video sequence is
calculated. The resulting forms a 1-dimension signal
that represent red color variation. Afterwards, FFT on
1-dimension signal and calculated the frequency of
the maximal peak (FMP) within 0.67 4 Hz range is
applied.

Figure 3. An example of using EVM framework for visualizing the

human pulse. (a) Four frames from the original video sequence
(face); (b) The same four frames with the subjects pulse signal
amplified; (c) A vertical scan line from the input (top) and output
(bottom) videos plotted over time shows how our method amplifies
the periodic color variation. In the input sequence the signal is
imperceptible, but in the magnified sequence the variation is clear.
The complete sequence is available in the supplemental video[5].

In order to obtain the pulse rate, there were sliding

window (4s width with 0.5s shift) which used and
calculated the FFT of each window in order to
determine the FMP within the window (wFMP).
Then, the smoothing procedure as recommended in
[9] is applied. Afterwards the resulting signal
introduced as the magnified red color variation with
detected peaks that correspond to pulse rate.
In [2], Poh et al. introduced the low-cost accurate
video-based method for non-contact heart rate
measurement that is automated, motion-tolerant, and
also capable of performing simultaneously
measurement up to three persons. This novel
technique obtained the heart rate information from
the video images based on blind source separation
(BSS).

Figure 4. Processing steps for pulse rate detection. The upper panel
corresponds to video sequence analysis starting with EVM. The

4|Donny Mirza Adhitama-23214341

There are several methods of BSS, but in this

application Independent Component Analysis (ICA)
is being used. ICA is the technique for uncover the
independent source signals from a set of observation
signals that are mixed linearly of the underlying
sources. Also, face tracker system is being used for
automaticly detect the region of interest (ROI).
Afterwards, the ROI is being separated into the three
RGB layers and spatially averaged all over the pixels
in the ROI resulting of each RGB measurement point,
each frame, and then form the traces.

Figure 6. Recovery of the cardiac pulse from a webcam video

recording of a participant at rest. (a) 30s raw RGB traces and (b)
their respective power spectra. (c) The independent components
recovered using ICA along with the reference finger BVP signal
and (d) their respective power spectra. (e) A single-frame excerp
from the webcam video recording with localized ROI. (f)
Evolution of the localized ROI over 1 minute.[2]

Figure 5. Cardiac pulse recovery methodology. (a) The region of

interest (ROI) is automaticly detected using a face tracker; (b) The
ROI is decomposed into the RGB channels and spatially averaged
to obtain (c) the raw RGB traces. ICA then applied on the
normalized RGB traces to recover (d) three independent source
signals.[2]

Finally, they Fast Fourier Transform (FFT) is applied

on the selected source signal to obtain the power
spectrum. The pulse frequency was designated as the
frequency that corresponded to the highest power of
the spectrum.

V. PULSE DETECTION FROM NECK & WRIST COLOR

VARIATION

Xiaochuan et al.[3] explained that the basic approach

of their research is to exploit EVM framework to
measure the pulse transit time. The video was
captured by a simple webcam and the time interval
was calculated from the moments when the flow of
blood reached the wrist and the neck, respectively,
which could be determined by the color variation
changing that caused by the blood flow.
Pulse transit time is a simple parameter for
continuous monitoring and it is highly related to
pulse wave velocity[10]. Pulse transit time is
propagation time delay between two points. The
conventional way to measure the PTT, it is to
calculate the time interval between the peak of QRScomplex of ECG signal and the corresponding
Photoplethysmogram (PPG) measurement.

5|Donny Mirza Adhitama-23214341

EVM is used to amplify the color variation of the

human skin caused by blood flow. Figure 7 illustrate
the two areas they were looking into in the
application. One is on the wrist, the other one is on
the neck. Obviously, the moments of blood flow
reaches these two areas are slightly different. The
color variation can be revealed by EVM. Then, the
blood pulse can be calculated the PTT.

Figure 8. Two frames from one of the original video sequence

showing the color changing on the wrist and neck caused by blood
pulse.

Figure 9. Pulse detection by using Eulerian color magnification

framework on wrist and neck simultaneously

Figure 7. Selected pulse detection area of interest.

Normally, the color variations of the human skin

along with the blood pulse and the respiration are
invisible by naked eye. By applying the EVM
framework, the changes of color can be seen and the
amplified signal is very useful for vital biomedical
signals estimation and health-care monitoring. As
shown in Figure 8, the color of the skin changes with
the blood flow. By extracting the color signal and
locating the peaks of the signal, they also determined
both of the heart rate and the pulse transit time.

The green curve and the black curve in Figure 9 is the

amplified color variation on the wrist and the neck,
respectively. After the amplification, the subtle
changes of the color can be observed clearly and the
blood pulse can be obtained easily. The red triangles
and the blue triangles stand for the moment when the
blood flow reaches the measurement sites. From
these triangles, not only the pulse transit time can be
calculated, but also the heart rate can be obtained.
From the Figure 9 it can seen that in 18 seconds there
are 16 pulses occur on both site of interest.

VI. CONCLUSION

The EVM methods magnify the subtle color of the

invisible motion and color variation by using spatialtemporal process to reveal both of the changes from
the recorded video. With this method, some part of
the human body can be as the source of the
information of the heart rate. Head motion combine

6|Donny Mirza Adhitama-23214341

frequency filtering and PCA (in the other research

they used ICA) after EVM method to identify the
component of motion that corresponding to the pulse
rate, so heart beat of the subject can be extracted.
From the facial, neck, and wrist color variation can
also being magnified and then processed to extract
the heart beat. In facial part, they were using face
tracker to focused on process in the ROI, so the
process can be exact at the focus point. Also, there is
some enhancement at the processing procedur that
allows the system to process three participants
simultaneously. In the neck & wrist part, they used
the same way as in facial part, but there was only one
research at those area of human part. Both of the neck
and wrist had an output of pulse rate extracted with
the same number of the R-peaks.

[1].

[2].

[3].

X. He, Rafik A, Goubran, and Xiaoping P. Liu; 2014;

Using Eulerian Video Magnification Framework to
Measure Pulse Transit Time; Carleton University,
Canada.
[4]. A. Alzahrani and Anthony Whitehead; 2015;
Preprocessing Realistic Video for Contactless Heart
Rate Monitoring using Video Magnification; Carleton
University, Canada.
[5]. H.Y. Wu, Michael Rubinstein, Eugene Shih, John
Guttag, Fredo Durand, and William Freeman; 2012;
Eulerian Video Magnification for Revealing Subtle
Changes in the World; Massachusetts Institute of
Technology, USA.
[6]. P.J. Burt and Edward H. Adelson; 1983; The
Laplacian Pyramid as a Compact Image Code; IEEE
Transactions on Communications, Vol. COM-31, No.
4.
[7]. G. Balakrishnan, Fredo Durand, and John Guttag;
2013; Detecting Pulse from Head Motions in Video;
Massachusetts Institute of Technology, USA.
[8]. P. Lynch.; 2007; Human Head Anatomy with
External
and
Internal
Carotid
Arteries;
http://www.flickr.com/photos/patrlynch/450142019.
REFERENCE
[9]. P.B. Chambino, 2013, Android-based Implementation
of Eulerian Video Magnification for Vital Signs
N. Miljkovic and Dragan Trifunovic; 2014; Pulse
Monitoring, Faculdade de Engenharia da
Rate Assessment: Eulerian Video Magnificatin vs.
Universidade do Porto.
Electrocardiography Recordings;
University of [10]. J. Sola, M. Proenca, D. Ferrario, J. A. Porchet, A.
Belgrade, Serbia.
Falhi, O. Grossenbacher, Y. Allemann, S. F. Rimldi,
M.Z. Poh, Daniel J. McDuff, and Rosalind W. Picard;
and C. Sartori, 2013, Non Invasive and Non
2010; Non-contact, Automated Cardiac Pulse
Occlusive Blood Pressure Estimation Via A Chest
Measurements using Video Imaging and Blind Source
Sensor, IEEE Transactions on Biomedical
Separation; Massachusetts Institute of Technology,
Engineering, vol. 60.
USA.

7|Donny Mirza Adhitama-23214341