Virtual Reality Therapy for Adults Post-Stroke: A

Systematic Review and Meta-Analysis Exploring Virtual

Environments and Commercial Games in Therapy
Keith R. Lohse
1,2
*, Courtney G. E. Hilderman
3
, Katharine L. Cheung
3
, Sandy Tatla
4
,
H. F. Machiel Van der Loos
5
1School of Kinesiology, Auburn University, Auburn, Alabama, United States of America, 2School of Kinesiology, University of British Columbia, Vancouver, British
Columbia, Canada, 3Department of Physical Therapy, University of British Columbia, Vancouver, British Columbia, Canada, 4Department of Occupational Science and
Occupational Therapy, University of British Columbia, Vancouver, British Columbia, Canada, 5Department of Mechanical Engineering, University of British Columbia,
Vancouver, British Columbia, Canada
Abstract
Background: The objective of this analysis was to systematically review the evidence for virtual reality (VR) therapy in an
adult post-stroke population in both custom built virtual environments (VE) and commercially available gaming systems
(CG).
Methods: MEDLINE, CINAHL, EMBASE, ERIC, PSYCInfo, DARE, PEDro, Cochrane Central Register of Controlled Trials, and
Cochrane Database of Systematic Reviews were systematically searched from the earliest available date until April 4, 2013.
Controlled trials that compared VR to conventional therapy were included. Population criteria included adults (.18) post-
stroke, excluding children, cerebral palsy, and other neurological disorders. Included studies were reported in English.
Quality of studies was assessed with the Physiotherapy Evidence Database Scale (PEDro).
Results: Twenty-six studies met the inclusion criteria. For body function outcomes, there was a significant benefit of VR
therapy compared to conventional therapy controls, G=0.48, 95% CI =[0.27, 0.70], and no significant difference between VE
and CG interventions (P =0.38). For activity outcomes, there was a significant benefit of VR therapy, G=0.58, 95% CI =[0.32,
0.85], and no significant difference between VE and CG interventions (P =0.66). For participation outcomes, the overall
effect size was G=0.56, 95% CI =[0.02, 1.10]. All participation outcomes came from VE studies.
Discussion: VR rehabilitation moderately improves outcomes compared to conventional therapy in adults post-stroke.
Current CG interventions have been too few and too small to assess potential benefits of CG. Future research in this area
should aim to clearly define conventional therapy, report on participation measures, consider motivational components of
therapy, and investigate commercially available systems in larger RCTs.
Trial Registration: Prospero CRD42013004338
Citation: Lohse KR, Hilderman CGE, Cheung KL, Tatla S, Van der Loos HFM (2014) Virtual Reality Therapy for Adults Post-Stroke: A Systematic Review and Meta-
Analysis Exploring Virtual Environments and Commercial Games in Therapy. PLoS ONE 9(3): e93318. doi:10.1371/journal.pone.0093318
Editor: Terence J. Quinn, University of Glasgow, United Kingdom
Received January 17, 2014; Accepted February 28, 2014; Published March 28, 2014
Copyright: 2014 Lohse et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This research was funded by #11-079 from the Peter Wall Solutions Initiative at the University of British Columbia awarded to H.F.M. Van der Loos. The
funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing Interests: The authors have declared that no competing interests exist.
* E-mail: kelopelli@gmail.com
Introduction
Stroke is a leading cause of death and disability around the
world, and the majority of survivors experience chronic motor
deficits associated with reduced quality of life [1]. Neurophysio-
logical data suggest considerable amounts of practice are required
to induce neuroplastic change and functional recovery of these
motor deficits [2][4]. This requisite high repetition is problem-
atic, however, because observational data show that clients
generally perform a very limited number of movement repetitions
in traditional therapy sessions [5]. Furthermore, many logistical,
financial, environmental, and individual barriers limit the efficacy
of conventional therapy for adults post-stroke [6],[7]. Conse-
quently, research is often focused on optimizing an individuals
potential amount of recovery for a given amount of time in
therapy. One proposed method for optimizing the effects of
therapy is the use of virtual reality (VR). VR can be defined as a
type of user-computer interface that implements real-time
simulation of an activity or environment allowing user interaction
via multiple sensory modalities [8]. VR therapies are an appealing
avenue of research because they can provide patients and
therapists with additional feedback during therapy, increase
patient motivation, and dynamically adjust the difficulty of
therapy [9][11].
Increasingly, VR therapies have been compared to "usual care"
or "conventional therapy" (CT) as sophisticated technologies have
PLOS ONE | www.plosone.org 1 March 2014 | Volume 9 | Issue 3 | e93318
become more readily available and affordable. VR therapy refers
to a broad class of interventions, but can generally be defined as
technological interventions that alter properties of the physical
world. These properties might be perceptual, such as providing
clients with additional sensory feedback about their movement in a
virtual environment (VE). At times, VE training is integrated with
exogenous forms of support such as robotic assistance or resistance
[12],[13], but we restricted our review to interventions that did not
include robotic assistance. Moreover, the advent of movement-
controlled videogames such as the Wii (Nintendo), Move (Sony),
and Kinect (Microsoft) has also allowed therapists to integrate
commercial gaming (CG) systems into therapy. Although only a
small number of randomized controlled CG studies exist [14]
[17], CG research is appealing because these interventions offer
some of the benefits of VE interventions [18], but have greater
availability and a significantly reduced cost. Thus, a major
objective for the current review was to quantitatively explore the
effectiveness of VE and CG interventions compared to CT.
Previous reviews comparing VR therapy to CT exist [19][21],
and while they indicate moderate positive benefits of VR therapy,
overall there is considerable variability in the observed effects.
Potential sources of variability include the type and parameters of
intervention, the type of outcome being measured, and the
demographics of clients being studied, such as the time from stroke
to intervention onset and the initial severity of the motor deficit.
This review adds to the current body of knowledge about VR
therapy by: (1) including new data comparing VR therapies to CT
control groups; (2) exploring how VR therapies affect different
outcomes according to the International Classification of Func-
tion, Disability, and Health (ICF); and (3) exploring how different
types of VR therapy affect outcomes, or more specifically, how
custom-built VE systems compare to interventions using CG
technology.
Methods
Prior to data collection, the review was registered with the
Prospero registry for systematic reviews (#CRD42013004338;
http://www.crd.york.ac.uk/NIHR_PROSPERO/). Objectives
were defined according to a PICO model (Population, Interven-
tion, Comparison, Outcome). The population of interest was
adults post-stroke. Interventions considered were VR therapies
that did not include exogenous stimulation (such as functional
electrical stimulation) or robotic assistance. Comparison groups
included "usual care", "standard care" or "conventional therapy",
and could involve physical therapy (PT) and/or occupational
therapy (OT). (See Table 1 for a description of control therapies.)
Primary and secondary outcomes from all studies were considered,
provided that these outcomes were behavioural assessments in one
of the ICF domains (i.e., body structure, body function, activity,
participation). Self-report measures such as the Motor Activity Log
(e.g., Housman et al. [22]) or the ABILIHAND inventory (e.g.,
Piron et al. [23]) were excluded. Restricting our analysis to
behavioural measures of function or impairment that compared
VR and conventional therapy makes these outcomes more
comparable for the purpose of meta-analysis. Further stratifying
these results by ICF classification increases comparability, however
there are still concerns about differences in the types of CT
provided in control groups. These concerns are discussed below.
Search Strategy
Relevant literature was first identified through electronic
searches. A liaison librarian within the Faculty of Medicine at
the University of British Columbia was consulted in selecting
appropriate databases and developing the search strategy,
including identifying key words and medical subject headings
(MeSH terms). On April 4, 2013, electronic searches were
conducted from the earliest available date in Medline, CINAHL,
EMBASE, ERIC, PSYCInfo, DARE, PEDro, the Cochrane
Central Register of Controlled Trials, and the Cochrane Database
of Systematic Reviews. Population search terms were restricted to
stroke and stroke synonyms, and intervention search terms
included "video game", "virtual reality", and "augmented reality".
Further relevant articles were identified by manually searching the
bibliographies of retrieved papers. See Appendix S1 for the full
search strategy.
Study Selection
Following removal of duplicate publications, 4512 records were
screened for eligibility (See Figure 1). The following exclusion
criteria were used to screen the studies: (a) studies of children (,18
years old), (b) studies where fewer than 70% of subjects were adults
post-stroke (e.g., studies involving cerebral palsy, traumatic brain
injury, and other neurological disorders were excluded), (c) studies
that did not use CT control conditions (e.g., studies comparing
robotic assistance in combination with virtual reality to robotic
assistance alone were excluded), (d) studies that did not use
randomization or quasi-randomization with an appropriate
control (e.g., case reports, case series, and uncontrolled trials were
excluded), and (e) studies not published or translated into English
were not searched. (Note, non-English studies were not excluded,
but only studies published in English or translated into English
were searched. Thus, relevant non-English studies may exist, but
were not included, in our search. Despite this last criterion, the
pool of included studies was highly international with studies from
Canada, USA, Japan, Taiwan, Sweden, Italy, and Brazil.)
One author (CH) screened articles by title and abstract
according to these criteria. Next, four authors used these criteria
to screen the remaining articles by full text for inclusion. When
there was disagreement, authors discussed the articles in question
until consensus was reached. A total of 26 trials remained and
were included in the assessment of study quality, but two of these
articles were subsequently excluded for a lack of necessary data
[25],[26], leaving 24 randomized controlled trials (RCTs) in the
quantitative analysis.
Quality Assessment
Three authors (CH, KC, ST) assessed the methodological
quality of individual studies using the Physiotherapy Evidence
Database Scale (PEDro; www.pedro.org.au), a criterion based
measure of quality for randomized controlled trials. PEDro
assesses 11 criteria to determine the selection, performance,
detection, and attrition biases present within a study. For this
reviews quality assessment, a sample of 5 studies was extracted
and all authors provided ratings. Across the 5 studies and 11 items
of the PEDro Scale, reviewers had 93% initial agreement.
Differences were discussed until 100% agreement was reached
and authors proceeded to independently code the remaining
studies.
Quantitative Analysis
Three authors (CH, KC, ST) extracted data relevant to sample
size, participant characteristics, intervention protocols, and out-
come measures. One author (KL) extracted initial statistical data.
All statistical data were then corroborated by an additional author;
CH, KC, or ST. All calculations were based on data in the
published manuscript except in one case [27], where additional
Virtual Reality Therapy
PLOS ONE | www.plosone.org 2 March 2014 | Volume 9 | Issue 3 | e93318
Table 1. Characteristics of trials comparing virtual reality therapy to conventional therapy in adults post-stroke.
Reference Intervention VR Intervention Ctrl Intervention VR Type
Extracted
Outcomes
Outcome
Classification
Broeren, 2008 [32] VE training +
CT vs. CT*
3-D computer games with
UL unsupported, with
rehabilitation personnel
Creative crafts, social and
physical activities at
activity centre.
VE BBT, movement
time, hand-path
ratios
ACT, BF, BF
Cho, 2013 [36] VE walking +
standard therapy
vs. CT + standard
therapy
Virtual walking training
program with video
recording, Co-intervention:
Standard therapy:
Therapeutic exercise,
functional therapy, OT, FES
Treadmill gait training
Co-intervention: Standard
therapy: Therapeutic exercise,
functional therapy, OT, FES
VE BBS, TUG ACT, ACT
Cikajlo, 2012 [37] VE balance training
vs. CT
VR supported balance
training in standing frame,
(2 week in clinic & 1 week
in home) with PT supervision.
Balance training without
VR (in clinic only).
VE BBS, TUG,
10mWT
ACT, ACT, ACT
Crosbie, 2012 [38] VE therapy vs. CT VR tasks focused on UL
reaching and grasping
with therapist.
Standard UL therapy,
including muscle facilitation,
stretching, strengthening
and functional tasks with PT.
VE Mobility Index,
ARAT
BF, ACT
da Silva Cameirao,
2011 [39]
VE game +
Standard Therapy
vs. CT + Standard
Therapy
Rehabilitation gaming
system targeting UL
speed, range of motion,
grasp and release.
Co-intervention:
Standard OT & PT.
One of two treatments: 1)
Pure occupational therapy
targeting object
displacement, grasp, and
release; or 2) Wii games.
Co-intervention: Standard
OT & PT.
VE Mobility Index,
FMA, CAHAI
BF, BF, ACT
Gil-Go mez, 2011 [15] Wii balance board
therapy vs. CT
Easy balance VR system
with Wii balance board
(eBaViR).
Traditional rehabilitation
balance exercises individually
or in group)
CG BBS, BBA ACT, ACT
In, 2012 [40] VE + Standard
Therapy vs. Sham
+ Standard Therapy
VR reflection therapy
for UL movements
(with caregiver).
UL movements using
unaffected limb (no VR
component) (with
caregiver).
VE FMA, BBT,
JTHF
BF, ACT, PART
Jung, 2012 [30] VE treadmill vs.
treadmill
VR (with head mounted
device) treadmill training.
Treadmill training. VE TUG ACT
Katz, 2005 [41] VE street-crossing
vs. visual training
Desktop VR street-crossing
cognitive training.
Computer-based visual
scanning tasks.
VE FIM,
VR-performance,
Real street
crossing.
ACT, ACT, PART
Kihoon, 2012 [34] VE + Standard
Therapy vs. CT*
Interactive Rehabilitation
& Exercise System (IREX)
VR targeting UL and
visual impairments.
Traditional therapy
(unspecified).
VE WMFT, MVPT ACT, BF
Kim, 2009 [33] VE + CT vs. CT* IREX VR balance therapy
+ CT.
Standard PT, involving
neurofacilitation.
VE BBS, MMAS,
10mWT
ACT, ACT, ACT
Kim, 2012 [14] Wii games vs.
no gaming
Nintendo Wii for balance
and motor control +
general exercise
(unspecified) and electrical
stimulation before each
session.
General exercise
(unspecified) and
electrical stimulation
before each session.
CG FIM, PASS, MASS ACT, BF, BF
Kiper, 2011 [42] VE therapy vs.
CT
Virtual Reality Rehabilitation
System (VRRS) training
targeting UL functional
tasks (turning, pouring,
using a hammer, etc.)
with PT.
Traditional neuromotor
rehabilitation (postural
control, in-hand
manipulation, fine
motor control and
coordination) with PT.
VE FIM, FMA, MAS ACT, BF, BF
Kwon, 2012 [35] VE + CT vs. CT* IREX VR UL training with
OT + CT.
Routine OT & PT (gait &
balance training, tabletop
activities, UL strengthening
and functional tasks.
VE FMA, MFT, MBI BF, ACT, ACT
Lam, 2006 [43] VE skills training
vs. CT vs. no
treatment
2-D VR program targeting
various cognitive functions
over 10 sessions.
Psychoeducational training
(instruction + video
modeling) over 10 sessions.
VE Behavioural
assessment
of mass transit
skills.
PART
Virtual Reality Therapy
PLOS ONE | www.plosone.org 3 March 2014 | Volume 9 | Issue 3 | e93318
data was requested and subsequently provided from the original
authors.
Multiple outcome variables from each study were extracted in
order to conduct separate analyses for each ICF category (see
Table 1). Because the dependent measures fell into three ICF
categories (viz. body function, activity, participation), each study
had to contribute at least one and no more than three outcome
variables. Outcomes were selected based on ICF category, and
then precedence was given to primary outcomes. Thus, a study
could report a body function outcome, an activity outcome, and a
participation outcome. Or, if a study reported activity outcomes
and two body function outcomes, the body function outcomes
would be averaged together to create a single standardized effect
size. This method was selected because it allows multiple outcomes
to be selected from each study up to the maximum of one
participation, one activity, and one body function outcome, or a
maximum of three outcome measures from a single study (if not all
ICF categories were measured).
Means, standard deviations, and sample sizes for the experi-
mental group and the control group were entered into an Excel
2010 (Microsoft) spreadsheet and standardized effect-sizes (Hedg-
es G) and effect-size variability (V
G
) were calculated according to
Borenstein, Hedges, Higgins, and Rothstein [28]. Effect-size
calculations were arranged such that effects favouring VR therapy
always had a positive value and effects favouring CT had a
negative value. An effect size of zero indicating no difference
between VR and CT. (The full dataset is provided in Appendix
S2.) Effect-size measures and demographic information were
imported into the statistical analysis software R (cran.r-project.org)
and analyzed using the "metafor" package [29]. Custom scripts
Table 1. Cont.
Reference Intervention VR Intervention Ctrl Intervention VR Type
Extracted
Outcomes
Outcome
Classification
Mirelman, 2010 [44] VE training vs.
Non-VE training
Rutgers ankle rehabilitation
system (robotic gait
training with VR
stimulation), involving
various ankle movements,
with therapist.
Ankle movements without
VR under therapist
supervision.
VE Gait speed,
ankle movement,
ankle power
ACT, BF, BF
Piron, 2007 [45] VE therapy vs.
CT
Reinforced feedback in
VR environment for UL
training with PT.
Conventional UL therapy
(unspecified) with PT.
VE FMA, FIM BF, ACT
Piron, 2009 [23] VE tele-rehab vs.
CT
VR with telemedicine
(VRRS.net) for upper limb
training. Therapist
supported through
videoconferencing.
Conventional UL therapy
progressing in complexity
from postural control to
postural control with
complex motion.
VE FMA, Ashworth
Scale
BF, BF
Piron, 2010 [46] VE therapy vs.
CT
Reinforced feedback
in VR environment for
UL training with therapist.
Conventional UL therapy
progressing in complexity
with PT.
VE FMA, FIM BF, ACT
Saposnik, 2010 [16] Wii games +
Standard therapy
vs. table top games
+ Standard therapy
VR Wii therapy targeting
UL. Co-intervention:
Conventional OT & PT 1
hr each per day.
Leisure activities, such as
playing cards, Bingo, or Jenga.
Co-intervention: Conventional
OT & PT 1 hr each per day.
CG WMFT, BBT, SIS
(hand items)
ACT, ACT, BF
Subramanian, 2013
[27]
VE training vs.
physical training
VR based UL training
(reaching for 6 targets).
Reaching for 6 targets in
non-VR environment.
VE WMFT, RPSS
(close, far items)
ACT, BF, BF
Yang, 2008 [31] VE treadmill
vs. treadmill
VR based treadmill
training designed to
simulate typical community
in Taipei (lane walking,
street crossing, stepping
over obstacles).
Treadmill training while
executing different tasks
(lifting legs to simulate
walking over obstacles,
uphill, downhill and fast
walking).
VE Gait speed,
walking time
in community
BF, ACT
Yavuzer, 2008 [17] Playstation EyeToy
games + Standard
therapy vs. sham +
Standard therapy
Playstation EyeToy games
targeting UL movements.
Co-intervention:
Conventional OT, PT,
and SLP.
Watched Playstation
EyeToy games but did
not play. Co-intervention:
Conventional OT, PT, and
SLP.
CG FIM (self care
items), Brunnstrom
stages (hand, UE
items)
ACT, BF, BF
You, 2005 [47] VE exercise games
vs. CT
IREX VR system targeting
range of motion, balance,
mobility, stepping and
ambulation.
No treatment. VE FAC, MMAS
(walking items)
ACT, ACT
Abbreviations: ACT, activity; ARAT, Action Research Arm Test; BBA, Brunel Balance Assessment; BBS, Berg Balance Scale; BBT, Box and Block Test; BF, body function;
CAHAI, Chedoke Arm and Hand Activity Inventory; CG, commercial gaming; CT, conventional therapy; FES, Functional Electrical Stimulation; FIM, Functional
Independence Measure; FMA, Fugl-Meyer Assessment; ICF, International Classification of Function, Disability, and Health; JTHF, Jebsen-Taylor Hand Function Test; MBI,
Modified Barthel Index; MFT, Manual Function Test; MMAS, Modified Motor Assessment Scale; MSS, Motor Status Scale; MVPT, Motor-free Visual Perception Test; OT,
occupational therapy; PART, participation; PASS, Postural Assessment Scale; PT, physiotherapy; RA, robotic assisted therapy; RPSS, Reaching Performance for Stroke
Scale; SIS, Stroke Impact Scale; SLP, speech and language therapy; TUG, Time Up-and-Go test; UL, upper limb; VE, virtual environments; VR, virtual reality; WMFT, Wolf
Motor Function Test; 10mWT, 10-metre Walk Test.
* = control group was not matched for time to the experimental group.
doi:10.1371/journal.pone.0093318.t001
Virtual Reality Therapy
PLOS ONE | www.plosone.org 4 March 2014 | Volume 9 | Issue 3 | e93318
(Appendix S3) were written to test random-effects models for the
overall effect of VR therapy compared to CT and meta-regression
models to explore the influence of moderator variables on any VR
therapy advantage. In these regressions, we tested the effect VR
therapy type (VE versus CG) and the effect of time (in years) from
stroke to onset of intervention.
Results
Of the 24 VR studies included in the quantitative analysis, only
four studies (16.7%) used CG [14][17] and the remaining 20
studies (83.3%) used VE. Often, studies used these VEs in
conjunction with another apparatus, such as simulated environ-
ments during treadmill walking [30],[31]. In four studies (16.7%)
[32][35], confounding conditions were present in the experi-
mental methods. In these studies, experimental groups received
VE therapy in addition to CT whereas the control group received
CT alone, without being matched for time. Consequently, in these
four studies, it remains unclear how much of the benefit of therapy
can be attributed to the VE versus the additional time in therapy.
With respect to the ICF categories that were explored, 32 outcome
variables were measures of activity; 24 were measures of body
function; and three were measures of participation. See Table 1.
Methodological Quality
PEDro scores for the various studies were moderate, with a
mean of 5.42 and SD of 1.60. The number of studies meeting each
PEDro criterion is shown in Table 2. Studies generally met criteria
for explicitly stating patients eligibility (88.5%), random allocation
to groups (84.6%), statistical comparisons of treatment and control
groups (84.6%), and providing means/SDs for important variables
(96.2%). A moderate number of studies met criteria for blinding of
assessors (61.5%), achieving follow-up assessments for more than
85% of study participants (76.9%), and having comparable groups
determined by baseline measurements (61.5%).
Areas of weakness across studies were concealment of partic-
ipant allocation (34.6%), blinding of participants (19.2%) and
therapists (3.8%) to conditions, and following an intention to treat
(ITT) analysis (19.2%). Proper concealment and ITT analysis are
particularly important considerations; studies may have actually
fulfilled these criteria but lacked explicit description in their
Methods sections. Lack of blinding for both participants and
therapists was also a limitation of the studies. Although it is not
feasible to truly "blind" participants to the fact that they are
receiving VR therapy, keeping patients and therapists naive to the
experimental hypotheses would be a useful step to add experi-
mental rigour and should be reported if it was achieved. A step
further would be to use control conditions that also control for the
social context or novelty of the VR therapy. For example,
Saposnik and colleagues [16] compared Wii games and CT to a
control group who engaged in tabletop games and CT, in an effort
to control for the novelty, cognitive demands, and social context of
the gaming intervention. Future research should attempt similar
controls; the exact nature of these control groups would be
dependent on the intervention.
Demographic Characteristics of Included Studies
Sample sizes were quite small in the included studies, ranging
from 5 to 40 participants per group (median was 11 participants
per group; see Table 3). The intensity (min/day), frequency (days/
week) and duration (weeks) of the interventions varied consider-
ably. Interventions across studies ranged from 20-minute sessions
[26],[39] to two-five hours of therapy per day (combined VR
therapy, occupational and physical therapy) at frequencies of three
to five sessions per week [17], and durations from two [16] to 12
weeks [39]. Multiplying intensity 6 frequency 6 duration yields
total time scheduled for therapy in minutes. For total time, the
shortest time scheduled for the VR therapy was 180 min [31] and
the longest was 1800 min [45] (the median was 570 min). There
was also considerable variability in the average years post-stroke
for each study. The shortest average latency between stroke and
study onset was 0.04 years [39] and the longest was 6.02 years [31]
(the median was 1.05 years). The minimum average age for
participants in these studies was 47.45 years and the maximum
was 71.37 years (median average age was 61.30 years).
Meta-Analysis: ICF Categories
In order to quantify effects of VR therapy we conducted
separate random-effects meta-analyses for each ICF category.
Separate analyses were used to ensure that different outcomes
from the same study were analyzed independently. When studies
had multiple outcomes within the same category (e.g., two activity
outcomes) these effect-sizes were averaged together. Thus, each
study contributed one data-point (at most) to the body function
analysis, the activity analysis, and the participation analysis.
Body Function Outcomes: VE and CG Combined. For
body function outcomes combining VE and CG interventions, the
overall Hedges G=0.48, 95% Confidence Interval =[0.27, 0.70],
which was significant, Z
obs
=4.33, P,0.001. The random-effects
model, estimated using restricted maximum likelihood, had a
t
2
=0.05 (which is the estimate of variance between effects),
I
2
=24.79% (which is the % of total variability due to heteroge-
neity), and H
2
=1.33 (which is the proportion of total variability to
Figure 1. Screening of articles. Four-phase PRISMA flow-diagram for
study collection [24], showing the number of studies identified,
screened, eligible, and included in the review and analysis.
doi:10.1371/journal.pone.0093318.g001
Virtual Reality Therapy
PLOS ONE | www.plosone.org 5 March 2014 | Volume 9 | Issue 3 | e93318
T
a
b
l
e
2
.
S
t
u
d
i
e
s
t
h
a
t
m
e
e
t
t
h
e
c
r
i
t
e
r
i
a
o
f
t
h
e
P
E
D
r
o
s
c
a
l
e
.
S
e
l
e
c
t
i
o
n
B
i
a
s
P
e
r
f
o
r
m
a
n
c
e
B
i
a
s
D
e
t
e
c
t
i
o
n
B
i
a
s
A
t
t
r
i
t
i
o
n
B
i
a
s
F
i
r
s
t
A
u
t
h
o
r
Y
e
a
r
C
1
C
2
C
3
C
4
C
5
C
6
C
7
C
8
C
9
C
1
0
C
1
1
B
r
o
e
r
e
n
2
0
0
8
1
0
0
1
0
0
0
1
0
0
1
C
h
o
2
0
1
3
1
1
1
1
0
0
1
1
0
1
1
C
i
k
a
j
i
l
o
2
0
1
2
1
0
0
0
0
0
1
1
0
1
1
C
r
o
s
b
i
e
2
0
0
8
1
1
0
1
0
0
1
1
1
1
0
C
r
o
s
b
i
e
2
0
1
2
1
1
1
1
0
0
1
1
1
1
1
d
a
S
i
l
v
a
C
a
m
i
e
r
o
2
0
1
1
1
1
0
0
0
0
1
0
0
1
1
G
i
l
-
G
o
m
e
z
2
0
1
1
1
1
0
1
0
0
1
1
0
1
1
I
n
2
0
1
2
1
1
0
1
0
0
0
0
0
1
1
J
u
n
g
2
0
1
2
1
1
0
0
1
0
1
1
0
1
1
K
a
t
z
2
0
0
5
0
0
0
1
0
0
0
1
0
1
1
K
i
h
o
o
n
2
0
1
2
1
1
1
1
1
0
0
1
1
0
1
K
i
m
2
0
1
1
1
1
0
1
0
0
1
1
0
1
1
K
i
m
2
0
1
2
1
1
0
0
0
0
0
0
0
1
1
K
i
p
e
r
2
0
1
1
1
1
0
1
0
0
0
1
1
1
1
K
w
o
n
2
0
1
2
1
1
0
1
1
1
1
1
0
1
1
L
a
m
2
0
0
6
1
1
1
1
1
0
0
1
0
0
1
M
i
r
e
l
m
a
n
2
0
1
0
0
1
0
0
1
0
0
1
0
1
1
P
i
r
o
n
2
0
0
7
0
0
0
0
0
0
0
1
0
0
1
P
i
r
o
n
2
0
0
9
1
1
1
0
0
0
1
1
0
1
1
P
i
r
o
n
2
0
1
0
1
1
1
0
0
0
1
1
1
1
1
S
a
p
o
s
n
i
k
2
0
1
0
1
1
0
0
0
0
1
0
0
1
1
S
u
b
r
a
m
a
n
i
a
n
2
0
1
3
1
1
1
1
0
0
1
1
0
1
1
Y
a
n
g
2
0
0
8
1
1
1
1
0
0
1
0
0
1
1
Y
a
n
g
2
0
1
1
1
1
0
0
0
0
1
0
0
1
1
Y
a
v
u
z
e
r
2
0
0
8
1
1
1
1
0
0
1
1
0
1
1
Y
o
u
2
0
0
5
1
1
0
1
0
0
0
1
0
1
1
N
o
t
e
.
A
"
1
"
i
n
d
i
c
a
t
e
s
t
h
a
t
a
s
t
u
d
y
m
e
t
t
h
a
t
p
a
r
t
i
c
u
l
a
r
c
r
i
t
e
r
i
o
n
,
a
"
0
"
i
n
d
i
c
a
t
e
s
t
h
a
t
a
s
t
u
d
y
d
i
d
n
o
t
m
e
e
t
t
h
a
t
c
r
i
t
e
r
i
o
n
o
r
t
h
a
t
n
o
t
e
n
o
u
g
h
i
n
f
o
r
m
a
t
i
o
n
w
a
s
g
i
v
e
n
t
o
m
a
k
e
a
n
a
s
s
e
s
s
m
e
n
t
.
C
1
=
E
l
i
g
i
b
i
l
i
t
y
c
r
i
t
e
r
i
a
w
e
r
e
s
p
e
c
i
f
i
e
d
;
C
2
=
P
a
r
t
i
c
i
p
a
n
t
s
w
e
r
e
r
a
n
d
o
m
l
y
a
l
l
o
c
a
t
e
d
t
o
g
r
o
u
p
s
;
C
3
=
T
r
e
a
t
m
e
n
t
a
l
l
o
c
a
t
i
o
n
w
a
s
c
o
n
c
e
a
l
e
d
;
C
4
=
G
r
o
u
p
s
w
e
r
e
s
i
m
i
l
a
r
a
t
b
a
s
e
l
i
n
e
;
C
5
=
B
l
i
n
d
i
n
g
o
f
p
a
r
t
i
c
i
p
a
n
t
s
;
C
6
=
B
l
i
n
d
i
n
g
o
f
t
h
e
r
a
p
i
s
t
s
a
d
m
i
n
i
s
t
e
r
i
n
g
t
r
e
a
t
m
e
n
t
;
C
7
=
B
l
i
n
d
i
n
g
o
f
a
s
s
e
s
s
o
r
s
f
o
r
o
u
t
c
o
m
e
m
e
a
s
u
r
e
s
;
C
8
=
M
e
a
s
u
r
e
m
e
n
t
o
f
k
e
y
o
u
t
c
o
m
e
f
r
o
m
.
8
5
%
o
f
p
a
r
t
i
c
i
p
a
n
t
s
;
C
9
=
I
n
t
e
n
t
i
o
n
t
o
t
r
e
a
t
a
n
a
l
y
s
i
s
;
C
1
0
=
B
e
t
w
e
e
n
-
g
r
o
u
p
s
s
t
a
t
i
s
t
i
c
a
l
c
o
m
p
a
r
i
s
o
n
i
s
r
e
p
o
r
t
e
d
f
o
r
k
e
y
o
u
t
c
o
m
e
;
C
1
1
=
M
e
a
s
u
r
e
s
o
f
c
e
n
t
r
a
l
t
e
n
d
e
n
c
y
a
n
d
v
a
r
i
a
b
i
l
i
t
y
a
r
e
p
r
o
v
i
d
e
d
.
A
s
p
e
r
P
E
D
r
o
g
u
i
d
e
l
i
n
e
s
,
t
h
e
"
t
o
t
a
l
"
s
c
o
r
e
i
s
b
a
s
e
d
o
n
C
2
t
h
r
o
u
g
h
C
1
1
.
d
o
i
:
1
0
.
1
3
7
1
/
j
o
u
r
n
a
l
.
p
o
n
e
.
0
0
9
3
3
1
8
.
t
0
0
2
Virtual Reality Therapy
PLOS ONE | www.plosone.org 6 March 2014 | Volume 9 | Issue 3 | e93318
sampling variability). The test for heterogeneity was not signifi-
cant, Q(15) =21.55, P=0.12. We tested years post-stroke as a
potential moderating factor, but time post-stroke did not
significantly affect outcomes (P=0.76). We also tested the type
of VR therapy used as a moderating factor (CG interventions were
coded as 0 and VE interventions were coded as 1 in the
regression), but type of therapy did not significantly affect
outcomes (P=0.38). Thus, there was an overall benefit of VR
therapy for body function outcomes in adults post-stroke and we
found no evidence that this effect was attenuated by the time post-
stroke or the type of therapy given. Individual analyses for VE and
CG studies are provided below.
Body Function Outcomes: Virtual Environments. For
VE studies only (13 studies, 401 total participants, see Figure 2),
the overall effect size was G=0.43, 95% CI =[0.22, 0.64], which
was significant, Z
obs
=3.97, P,0.001. The random-effects model,
estimated using restricted maximum likelihood, had a t
2
=0.02,
I
2
=11.03%, and H
2
=1.12. The test for heterogeneity was not
significant, Q(12) =14.64, P=0.26.
Body Function Outcomes: Commercial Games. For CG
studies only (3 studies, 58 total participants, see Figure 3), the
overall effect size was G=0.76, 95% CI =[20.17, 1.70], which
approached significance, Z
obs
=1.60, P=0.10. The random-
effects model, estimated using restricted maximum likelihood,
had a t
2
=0.45, I
2
=66.30%, and H
2
=2.97. The test for
heterogeneity approached significance, Q(2) =5.85, P=0.05.
Activity Outcomes: VE and CG Combined. For activity
outcomes, the overall effect size was G=0.58, 95% CI =[0.32,
0.85], which was significant, Z
obs
=4.32, P,0.001. The random-
effects model had a t
2
=0.21, I
2
=55.23%, and H
2
=2.23. The
test for heterogeneity was significant, Q(21) =49.18, P,0.01, thus
there was significantly more variability in activity outcomes than
would be predicted by sampling variability alone. Again, we tested
time post-stroke as a moderating factor, but it was not significant
(P=0.65). We also tested the type of VR therapy used a
moderating factor, but type of therapy did not significantly affect
outcomes (P=0.66). Thus, there was an overall benefit of VR
therapy for activity outcomes in adults post-stroke and we found
no evidence that this effect was attenuated by the time post-stroke
or the type of therapy given. Individual analyses for VE and CG
studies are provided below.
Activity Outcomes: Virtual Environments. For VE studies
only (18 studies, 479 total participants, see Figure 4), the overall
effect size was G=0.54, 95% CI =[0.28, 0.81], which was
significant, Z
obs
=4.00, P,0.001. The random-effects model,
estimated using restricted maximum likelihood, had a t
2
=0.15,
I
2
=49.18%, and H
2
=1.96. The test for heterogeneity was
significant, Q(17) =35.99, P,0.01.
Table 3. Demographic statistics for the included studies.
Reference VR Type
Time Scheduled for VR
Intervention (min)
Experimental
Group N
Control
Group N
Years Post-Stroke
(average)
Average Patient Age
(yrs)
Broeren, 2008 [32] VE 45*3*4 =540` 11 11 5.87 NR; range: 44-85
Cho, 2013 [36] VE 30*3*6 =540 7 7 0.82 64.85
Cikajlo, 2012 [37] VE 20*5*3 =300 6 20 0.36 58.50
Crosbie, 2012 [38] VE 37.5*3*3 =337.5 9 9 0.90 60.35
da Silva Cameirao, 2011 [39] VE 20*3*12 =720 8 8 0.04 61.37
Gil-Go mez, 2011 [15] CG 60*20 sessions =1200 9 8 1.58 47.45
In, 2012 [40] VE 30*5*4 =600 11 8 1.11 63.97
Jung, 2012 [30] VE 30*5*3 =450 11 10 1.17 62.05
Katz, 2005 [41] VE 45*3*4 =540 11 8 0.11 62.85
Kihoon, 2012 [34] VE 30*3*4 =360` 15 14 NR 63.85
Kim, 2009 [33] VE 30*4*4 =480` 12 12 0.07 52.09
Kim, 2012 [14] CG 30*3*3 =270 10 10 1.05 48.15
Kiper, 2011 [42] VE 60*5*4 =1200 40 40 0.48 64.00
Kwon, 2012 [35] VE 30*5*4 =600` 13 13 0.67 57.54
Lam, 2006 [43] VE NR 20 16 4.74 71.37
Mirelman, 2010 [44] VE 60*3*4 =720 9 9 .2.00{ 62.00
Piron, 2007 [45] VE 60*5*6 =1800 25 13 0.22 61.50
Piron, 2009 [23] VE 60*5*4 =1200 18 18 1.11 65.20
Piron, 2010 [46] VE 60*5*4 =1200 27 20 1.27 60.50
Saposnik, 2010 [16] CG 60*8 sessions =480 9 9 0.07 61.30
Subramanian, 2013 [27] VE 45*3*4 =540 16 16 3.35 61.00
Yang, 2008 [31] VE 20*3*3 =180 9 11 6.01 58.17
Yavuzer, 2008 [17] CG 30*5*4 =600 10 10 0.33 61.10
You, 2005 [47] VE 60*5*4 =1200 5 5 1.57 57.10
Note. Time scheduled for the VR intervention is given as (min/day) * (days/week) * (weeks) = total time in minutes. NR = not reported.
{
= this study did not report an average time post-stroke, so the minimum time was used instead.
`
= control group was not matched for time to the experimental group.
doi:10.1371/journal.pone.0093318.t003
Virtual Reality Therapy
PLOS ONE | www.plosone.org 7 March 2014 | Volume 9 | Issue 3 | e93318
Activity Outcomes: Commercial Gaming. For CG studies
only (4 studies, 75 total participants, see Figure 5), the overall effect
size was G=0.76, 95% CI =[20.25, 1.76], which was not
significant, Z
obs
=1.48, P=0.14. The random-effects model,
estimated using restricted maximum likelihood, had a t
2
=0.80,
I
2
=77.17%, and H
2
=4.38. The test for heterogeneity was
significant, Q(3) =12.56, P,0.01.
Participation Outcomes. For participation outcomes (3
studies, 74 total participants, see Figure 6), the overall effect size
was G=0.56, 95% CI =[0.02, 1.10], which was significant,
Z
obs
=2.02, P=0.04. The random-effects model had a t
2
=0.06,
I
2
=26.75%, and H
2
=1.37. The test for heterogeneity was not
significant, Q(2) =2.82, P=0.24. Also, given the small number of
studies in this category, these results should be interpreted with
caution. Due to the lack of sufficient data points, we were unable
to test the moderating effect of time post-stroke or the effects of
different VR interventions (all participation outcomes came from
studies using VE interventions). These findings provide prelimi-
nary evidence that VR therapy has a positive effect on
participation outcomes, but this is an understudied area of
research, and more participation outcomes should be included
in future studies.
Discussion
This meta-analysis and systematic review is the first to examine
the effects of VR across levels of the ICF and to compare effect-
sizes as a function of the type of VR therapy implemented. These
findings build upon previous reviews that have explored VR
therapy compared to CT in general. This review adds to the
current body of literature in three key areas: (1) 14 new RCTs
have been published since previous reviews and are included in
our analysis; (2) this review found positive effects of VR therapy
across domains of the ICF; and (3) VR therapies were found to be
effective when delivered as VE or CG. In the current analysis, time
post-stroke and the type of VR intervention were not found to
significantly affect outcomes. However, the small number of CG
studies all had poor precision (shown in Figures 3 and 5), so larger
trials with carefully designed control groups using CG interven-
tions are needed before conclusions can be drawn about the
efficacy of CG interventions.
Figure 2. Body function outcomes in VE studies. The funnel plot (top) for body function outcomes showing effect-sizes (G) as a function of
precision (standard error) in each virtual environment study. The forest plot (bottom) showing the effect-sizes and 95% confidence intervals for each
study and the summary effect-size from the random-effects model. Positive values show a difference in favour of VE therapy. Negative values show a
difference in favour of CT. Abbreviations: VE, virtual environments; RE, random effects.
doi:10.1371/journal.pone.0093318.g002
Virtual Reality Therapy
PLOS ONE | www.plosone.org 8 March 2014 | Volume 9 | Issue 3 | e93318
Review identifies new trials
The most recent previous reviews summarizing the evidence for
VR therapy included searches of the literature up to March and
July 2010 [19],[20]. Fourteen trials (58.3%) included in our review
were published after 2010 and had not yet been included in a
meta-analysis. Furthermore, previous reviews [19] included
observational studies, whereas we selected only randomized
controlled trials to ensure robustness of the evidence. All previous
reviews of this topic have demonstrated a moderate effect in favour
of VR therapy over CT [19][21] however, the heterogeneity of
trial parameters requires that all effect sizes and conclusions be
interpreted with caution. Similarly, our review suggests VR
therapy has a moderate effect on outcomes for adults after stroke,
but many sources of variability exist in the interventions and
outcomes of the included trials.
Sources of variability within interventions
There was considerable variability in how VR interventions
were delivered with respect to intensity, frequency and duration of
the intervention. VR interventions were also inconsistently
conducted in conjunction with other PT/OT treatments. As such,
we are unable to comment on optimal prescribing dosage for VR
therapies.
Studies lacked detail about the content of the CT being
compared to VR. Studies were inconsistent in their reporting of
the role(s) of therapists, rehabilitation assistants, caregivers, and/or
other personnel; future research should ensure sufficient informa-
tion is given to readers to allow for accurate comparisons.
Individual studies did often schedule equal time in therapy for
experimental and control groups, but most studies did not ensure
true dosage matching of groups (e.g., matching active time in
therapy or numbers of repetitions). Subramanian and colleagues
[27] explicitly matched arm-reaching repetitions between the
experimental and control groups, and they also controlled for the
amount of feedback (knowledge of results and performance)
provided. Repetitions were controlled in that study, and there was
no overall benefit of VR therapy beyond CT. However, VR
training did lead to larger improvements in participants with mild
impairments compared to CT, and VR training reduced
compensatory movements in moderate-to-severely impaired par-
ticipants compared to CT. Future studies should use similar
methods for controlling repetitions when investigating VR
therapies to clarify our understanding of the benefits of VR in
therapy.
Another source of variability could be the degree to which
participants felt motivated and engaged during therapy. It has
been suggested that VR therapies are advantageous to CT in part
because of the motivating influence of using novel technologies or
games [10],[11]. Unfortunately, most of the trials included in this
review do not discuss motivation, use motivation as an outcome
measure, or control for the motivating or novel components of VR
therapy. Some studies did attempt to control for this factor using
Figure 3. Body function outcomes in CG studies. The funnel plot (top) for body function outcomes showing effect-sizes (G) as a function of
precision (standard error) in each commercial gaming study. The forest plot (bottom) showing the effect-sizes and 95% confidence intervals for each
study and the summary effect-size from the random-effects model. Positive values show a difference in favour of CG therapy. Negative values show a
difference in favour of CT. Abbreviations: CG, commercial gaming; RE, random effects.
doi:10.1371/journal.pone.0093318.g003
Virtual Reality Therapy
PLOS ONE | www.plosone.org 9 March 2014 | Volume 9 | Issue 3 | e93318
card games [16] or cognitive computer games [33] in their control
group. Arguably, interactive video games may still be considered
more novel to an older population.
No differences found for VR therapy types
A major objective of our review was to compare the effects of
commercial gaming systems (CG) to rehabilitation-specific virtual
environments (VE) in a therapy context for adults post-stroke. We
found no evidence for differences between VE and CG games in
the current analysis, but CG interventions have been too few and
too small to draw conclusions. Four trials (16.7%) examined the
effects of CG therapy [14][17] and 20 trials (83.3%) researched
VE therapy compared to CT for adults post-stroke. Our meta-
analysis provides strong evidence for the effectiveness of VE
interventions and demonstrates promising initial data for the
effectiveness of CG interventions. More data needs to be collected
to see if gains for CG interventions are reliable and to see if the
moderate effect-sizes observed (from G=0.40.7) translate into
clinically meaningful results. These results suggest larger RCTs
using CG interventions are justified; we recommend RCTs
compare CG directly to VE and CT groups.
Movement-controlled games are increasingly investigated as
therapeutic tools for individuals with neurological disorders such
as cerebral palsy [48] and stroke [49]. An appealing aspect of
movement-controlled games is combining aerobic exercise and
motor skills practice, which may increase neuroplasticity during
motor rehabilitation [50]. As a result, commercial games have
been investigated as tools for learning motor skills and for
improving cardiovascular fitness. For example, the game Dance
Dance Revolution has been shown to increase energy expenditure
in adolescents up to 5.4 (1.8 SD) Metabolic Equivalent Tasks
(METs) [51]. In healthy adults, Wii Sports tennis requires 2.1 (1.2
SD) METs, baseball 2.8 (0.9 SD) METs, and boxing 4.7 (1.4 SD)
METs [48]. However, in adults with cerebral palsy, the same
Figure 4. Activity outcomes in VE studies. The funnel plot (top) for activity outcomes showing effect-sizes (G) as a function of precision
(standard error) in each virtual environment study. The forest plot (bottom) shows the effect-sizes and 95% confidence intervals for each study and
the summary effect-size from the random-effects model. Positive values show a difference in favour of VE therapy. Negative values show a difference
in favour of CT. Abbreviations: RE, random effects.
doi:10.1371/journal.pone.0093318.g004
Virtual Reality Therapy
PLOS ONE | www.plosone.org 10 March 2014 | Volume 9 | Issue 3 | e93318
games all increase energy expenditure to over 3 METs [48],
suggesting they can help these individuals meet recommended
guidelines for physical activity. In addition to increasing energy
expenditures, commercial movement games have been used
therapeutically to improve balance, strength and coordination
[49],[52].
Increased availability and lower cost are also potential
advantages of using commercial games over virtual reality systems
that have been designed specifically for rehabilitation. For
example, the Nintendo Wii (Nintendo Co., Kyoto, JP) has been
sold to over 100 million customers worldwide [53], and the
console retails below US$150. As a comparison, the GestureTek
IREX (GestureTek, Toronto, CA) system, used in the study by
Kwon et al. in 2012 [35], is only available through specialized
rehabilitation equipment distributors and retails at more than
US$15,000 [54]. Understanding the benefits of CG and VE
systems, relative to their costs, thus has significant implications for
therapists and clients facing budget constraints.
Positive effects of VR therapy across ICF categories
The ICF provides a framework and a comprehensive perspec-
tive of functioning and disability in research and clinical practice
[55]. The overarching goal of rehabilitation for adults post-stroke
is to restore the persons ability to participate in normal life roles
with as much independence as possible. Impairments at the body
structure and function level may influence activity limitations, and
activity limitations may influence participation restrictions [56].
However, impairments and activity limitations do not necessarily
affect the enjoyment of participation by individuals in various life
situations [57],[58]. It is, therefore, important for researchers and
clinicians to be clear about which ICF domains an intervention
intends to, and actually does, impact.
Our review did not identify any trials that examined outcomes
related to body structures, personal factors, or environmental
factors and only three trials (12.5%) that examined participation
outcomes. There was a moderate but reliable advantage of VR
therapy over CT in the categories of body function and activity,
but outcomes from other ICF categories should be included in
future research.
Limitations
Our review included studies conducted in all stages of stroke
recovery, from acute inpatient to chronic outpatient settings
(average time post-stroke of the study participants ranged from
0.04 years [39] to 6.02 years [31]). However, our analysis was
based on a small number of studies making statistical power a
concern for these regression analyses. Furthermore, the small
number of studies limits our ability to control for other moderating
factors such as the initial severity of stroke and the effect of time
post-stroke on conventional therapy outcomes.
This review is limited by some risk of publication bias in the
included studies. Visual inspection of the funnel plots in the figures
reveals highly positive studies with low precision for both activity
and body function outcomes. For activity outcomes, two outlying
Figure 5. Activity outcomes in CG studies. The funnel plot (top) for activity outcomes showing effect-sizes (G) as a function of precision
(standard error) in each commercial gaming study. The forest plot (bottom) shows the effect-sizes and 95% confidence intervals for each study and
the summary effect-size from the random-effects model. Positive values show a difference in favour of CG therapy. Negative values show a difference
in favour of CT. Abbreviations: CG, commercial gaming; RE, random effects.
doi:10.1371/journal.pone.0093318.g005
Virtual Reality Therapy
PLOS ONE | www.plosone.org 11 March 2014 | Volume 9 | Issue 3 | e93318
studies (8.3%) [17],[36] had small numbers of subjects (N=20 and
14, respectively) but relatively good study quality (7/10 for both
studies on PEDro criteria). Similarly for body function outcomes,
two studies (8.3%) [17],[40] had small numbers of subjects (N=20
and 19, respectively) and while Yavuzer et al. [17] had good study
quality (7/10), In et al. [40] had poor study quality (4/10).
Conclusions
This review updated the evidence for virtual reality therapy to
include the most recent trials, and is the first to investigate the
effects of VR therapy across ICF domains and between VR
therapy types. Virtual reality therapy demonstrates a significant
moderate advantage in body function and activity outcomes when
compared to CT. Research on participation outcomes is limited,
but initial data show a positive benefit of VR therapy compared to
CT. No significant differences were found between the VE and
CG therapy types, and there was no evidence that time post-stroke
attenuated the benefits of VR therapy, but these findings are
limited by a high degree of variability between studies. To date,
CG interventions have been too few and too small to draw strong
conclusions about their efficacy. Larger RCTs investigating CG
interventions would provide better evidence for their use in
therapy as a potentially effective and cost-efficient method of
increasing motor repetitions in a motivating way. Given the
relationship between participation and quality of life, it is also
recommended that future trials include participation outcome
measures in their investigations.
Supporting Information
Appendix S1 Comprehensive search strategy used in
the systematic review.
(DOC)
Appendix S2 Full data-set of effect-sizes and demo-
graphic information for each study.
(XLSX)
Appendix S3 Scripts for analysis. Written in "R" using the
"metafor" package.
(TXT)
Checklist S1 PRISMA checklist.
(DOC)
Author Contributions
Conceived and designed the experiments: KRL. Performed the experi-
ments: KRL CGEH KLC ST. Analyzed the data: KRL CGEH KLC ST.
Wrote the paper: KRL CGEH KLC ST HFMVdL.
References
1. Nichols-Larsen DS, Clark PC, Zeringue A, Greenspan A, Blanton S (2005)
Factors influencing stroke survivors quality of life during subacute recovery.
Stroke 36(7):14804.
2. Nudo RJ, Milliken GW (1996) Reorganization of movement representations in
primary motor cortex following focal ischemic infarcts in adult squirrel monkeys.
J Neurophysiol 75(5):21449.
3. Plautz EJ, Milliken GW, Nudo RJ (2000) Effects of repetitive motor training on
movement representations in adult squirrel monkeys: role of use versus learning.
Neurobiol Learn Mem Jul;74(1):2755.
4. Kwakkel G (2006) Impact of intensity of practice after stroke: Issues for
consideration. Disabil Rehabil 28(July):82330.
5. Lang CE, Macdonald JR, Reisman DS, Boyd L, Jacobson Kimberley T, et al.
(2009) Observation of amounts of movement practice provided during stroke
rehabilitation. Arch Phys Med Rehabil 90(10):16928.
6. Langhorne P, Coupar F, Pollock A (2009) Motor recovery after stroke: a
systematic review. Lancet Neurol 8(8):74154.
7. Foley N, Teasell R, Bhogal S, Speechley M (2003) The efficacy of stroke
rehabilitation. Top Stroke Rehabil 10(2):118.
8. Adamovich SV, Fluet GG, Tunik E, Merians AS (2009) Sensorimotor training in
virtual reality: a review. NeuroRehabilitation 25: 2944.
9. Mirelman A, Bonato P, Deutsch JE (2009) Effects of training with a robot-virtual
reality system compared with a robot alone on the gait of individuals after stroke.
Stroke 40(1):16974.
10. Burke JW, McNeill MDJ, Charles DK, Morrow PJ, Crosbie JH, et al. (2009)
Optimising engagement strategies for stroke rehabilitation using serious games.
Visual Computing 25: 108599.
11. Shirzad N, Van der Loos HFM (2012) Error amplification to promote motor
learning and motivation in therapy robotics. In: Proceedings of the Annual
International Conference of the IEEE Engineering in Medicine and Biology
Society: 2012 Aug 28 - Sep 1; San Diego, USA: IEEE Engineering in Medicine
and Biology Society.
12. Deutsch JE, Merians AS, Adamovich S, Poizner H, Burdea GC (2004)
Development and application of virtual reality technology to improve hand use
and gait of individuals post-stroke. Restor Neurol Neurosci 22(35):371386.
13. Patton J, Dawe G, Scharver C, Mussa-Ivaldi F, Kenyon R (2006) Robotics and
virtual reality: a perfect marriage for motor control research and rehabilitation.
Assist Technol 18: 181195.
14. Kim EK, Kang JH, Park JS, Jung BH (2012) Clinical feasibility of interactive
commercial nintendo gaming for chronic stroke rehabilitation. J Phys Ther Sci
24: 9013.
15. Gil-Gomez J-A, Llorens R, Alcan iz M, Colomer C (2011) Effectiveness of a Wii
balance board-based system (eBaViR) for balance rehabilitation: a pilot
randomized clinical trial in patients with acquired brain injury. J Neuroeng
Rehabil 8(1):30.
16. Saposnik G, Teasell R, Mamdani M, Hall J, Mcilroy W, et al. (2010)
Effectiveness of virtual reality using Wii gaming technology in stroke
rehabilitation. Stroke 147784.
17. Yavuzer G, Senel A, Atay MB, Stam HJ (2008) "Playstation eyetoy games"
improve upper extremity-related motor functioning in subacute stroke: a
randomized controlled clinical trial. Eur J Phys Rehabil Med 44(3):23744.
18. Lohse KR, Shirzad N, Verster A, Hodges NJ, Van der Loos HFM (2013)
Videogames and rehabilitation: Using design principles to enhance patient
engagement. J Neurol Phys Ther 37: 166175.
Figure 6. Participation outcomes in VE studies. The funnel plot
(top) for participation outcomes showing effect-sizes (G) as a function
of precision (standard error) in each study. The forest plot (bottom)
shows the effect-sizes and 95% confidence intervals for each study and
the summary effect-size from the random-effects model. Positive values
show a difference in favour of VE therapy. Negative values show a
difference in favour of CT. Abbreviations: VE, virtual environments; RE,
random effects.
doi:10.1371/journal.pone.0093318.g006
Virtual Reality Therapy
PLOS ONE | www.plosone.org 12 March 2014 | Volume 9 | Issue 3 | e93318
19. Saposnik G, Levin M (2011) Virtual reality in stroke rehabilitation: a meta-
analysis and implications for clinicians. Stroke 42(5):13806.
20. Laver K, George S, Thomas S, Deutsch JE, Crotty M (2011) Virtual reality for
stroke rehabilitation. Stroke 43(2):e20e21.
21. Henderson A, Korner-Bitensky N, Levin M, (2007) Virtual reality in stroke
rehabilitation: a systematic review of its effectiveness for upper limb motor
recovery. Top Stroke Rehabil 14(2):5261.
22. Housman SJ, Scott KM, Reinkensmeyer DJ (2009) A randomized controlled
trial of gravity-supported, computer-enhanced arm exercise for individuals with
severe hemiparesis. Neurorehabil Neural Repair 23(5):50514.
23. Piron L, Turolla A, Agostini M, Zucconi C, Cortese F, et al. (2009) Exercises for
paretic upper limb after stroke: a combined virtual-reality and telemedicine
approach. J Rehabil Med 41(12):1016102.
24. Moher D (2009) Preferred reporting items for systematic reviews and meta-
analyses: The PRISMA statement. Ann Intern Med 151(4):264.
25. Crosbie JH, Lennon S, Mcgoldrick MC, Mcneill MDJ, Burke JW, et al. (2008)
Virtual reality in the rehabilitation of the upper limb after hemiplegic stroke: a
randomised pilot study. Clin Rehabil 22935.
26. Yang S, Hwang W-H, Tsai Y-C, Liu F-K, Hsieh L-F, et al. (2011) Improving
balance skills in patients who had stroke through virtual reality treadmill
training. Am J Phys Med Rehabil 90(12):96978.
27. Subramanian SK, Lourenco CB, Chilingaryan G, Sveistrup H, Levin MF (2013)
Arm motor recovery using a virtual reality intervention in chronic stroke:
randomized control trial. Neurorehabil Neural Repair 27(1):1323.
28. Borenstein M, Hedges L V., Higgins JPT, Rothstein HR (2011) Introduction to
meta-analysis [eBook online]. John Wiley & Sons [cited 2013 Jul 20]. p. 450.
Available from: URL: http://books.google.com/books?hl =en&lr =&id =JQg
9jdrq26wC&pgis =1
29. Viechtbauer W (2010) Conducting meta-analyses in R with the metafor
Package. J Stat Softw 36(3):148.
30. Jung J (2012) Effects of virtual-reality treadmill training on balance and balance
self efficacy in stroke patients with a history of falling. J Phys Ther Sci
24(5):11336.
31. Yang Y-R, Tsai M-P, Chuang T-Y, Sung W-H, Wang R-Y (2008) Virtual
reality-based training improves community ambulation in individuals with
stroke: a randomized controlled trial. Gait Posture 28(2):2016.
32. Broeren J, Claesson L, Goude D, Rydmark M, Sunnerhagen KS (2008) Virtual
rehabilitation in an activity centre for community-dwelling persons with stroke.
The possibilities of 3-dimensional computer games. Cerebrovasc Dis 26(3):289
96.
33. Kim JH, Jang SH, Kim CS, Jung JH, You JH (2009) Use of virtual reality to
enhance balance and ambulation in chronic stroke: a double-blind, randomized
controlled study. Am J Phys Med Rehabil 88(9):693701.
34. Kihoon J (2012) Effects of virtual reality-based rehabilitation on upper extremity
function and visual perception in stroke patients: A randomized control trial.
J Phys Ther Sci 24: 12058.
35. Kwon J-S, Park M-J, Yoon I-J, Park S-H (2012) Effects of virtual reality on
upper extremity function and activities of daily living performance in acute
stroke: a double-blind randomized clinical trial. NeuroRehabil 31(4):37985.
36. Cho KH, Lee WH (2013) Virtual walking training program using a real-world
video recording for patients with chronic stroke: a pilot study. Am J Phys Med
Rehabil 92(5):37184.
37. Cikajlo I, Rudolf M, Goljar N, Burger H, Matjacic Z (2012) Telerehabilitation
using virtual reality task can improve balance in patients with stroke. Disabil
Rehabil 34(1):138.
38. Crosbie JH, Lennon S, McGoldrick MC, McNeill MDJ, McDonough SM (2012)
Virtual reality in the rehabilitation of the arm after hemiplegic stroke: a
randomized controlled pilot study. Clin Rehabil 26(9):798806.
39. Da Silva Cameirao M, Bermudez I, Badia S, Duarte E, Verschure PF (2011)
Virtual reality based rehabilitation speeds up functional recovery of the upper
extremities after stroke: a randomized controlled pilot study in the acute phase of
stroke using the rehabilitation gaming system. Restor Neurol Neurosci
29(5):28798.
40. In TS, Jung KS, Lee SW, Song CH (2012) Virtual reality reflection therapy
improves motor recovery and motor function in the upper extremities of people
with chronic stroke. J Phys Ther Sci 24(4):33943.
41. Katz N, Ring H, Naveh Y, Kizony R, Feintuch U, et al. (2005) Interactive
virtual environment training for safe street crossing of right hemisphere stroke
patients with unilateral spatial neglect. Disabil Rehabil 27(20):123543.
42. Kiper P, Piron L, Turolla A, Sto J, Tonin P (2011) The effectiveness of
reinforced feedback in virtual environment in the first 12 months after stroke.
Neurol Neurochir Pol 43644.
43. Lam YS, Man DWK, Tam SF, Weiss PL (2006) Virtual reality training for
stroke rehabilitation. NeuroRehabil 21(3):24553.
44. Mirelman A, Patritti BL, Bonato P, Deutsch JE (2010) Effects of virtual reality
training on gait biomechanics of individuals post-stroke. Gait Posture 31(4):433
7.
45. Piron L, Tombolini P, Turolla A, Zucconi C, Agostini M, et al. (2007)
Reinforced feedback in virtual environment facilitates the arm motor recovery in
patients after a recent stroke. In: Virtual Rehabilitation [serial online]. 2007 Sep
2729; Venice, Italy. 2007. p 1213. Available from: URL: http://ieeexplore.
ieee.org/lpdocs/epic03/wrapper.htm?arnumber =4362151
46. Piron L, Turolla A, Agostini M, Tonin P, Dam M (2010) Motor learning
principles for rehabilitation: A pilot randomized controlled study in poststroke
patients. Neurorehabil Neural Repair 24(6):5018.
47. You SH, Jang SH, Kim Y-H, Hallett M, Ahn SH, et al. (2005) Virtual reality-
induced cortical reorganization and associated locomotor recovery in chronic
stroke: an experimenter-blind randomized study. Stroke 36(6):116671.
48. Taylor MJD, McCormic D, Shawis T, Impson R, Griffin M (2011) Activity-
promoting gaming systems in exercise and rehabilitation. J Rehabil Res Dev
48(10):11711186.
49. Deutsch JE, Brettler A, Smith C, Welsh J, John R, et al. (2011) Nintendo Wii
Sports and Wii Fit game analysis, validation, and application to stroke
rehabilitation. Top Stroke Rehabil 18)6):70119.
50. Mang CS, Campbell KL, Ross CJD, Boyd LA (2013) Promoting neuroplasticity
for motor rehabilitation after stroke: Considering the effects of aerobic exercise
and genetic variation on brain-derived neurotrophic factor. Phys Ther [online
first] [cited 2013 Sep 14]. Available from: URL: http://ptjournal.apta.org/
content/early/2013/07/31/ptj.20130053
51. Bailey BW, McInnis K (2001) Energy cost of exergaming: A comparison of the
energy cost of 6 forms of exergaming. Arch Pediatr Adolesc Med 165(7):597
602.
52. Agmon M, Perry CK, Phelan E, Demiris G, Nguyen HQ (2011) A pilot study of
Wii Fit exergames to improve balance in older adults. J Geriatr Phys Ther 34:
1617.
53. Nintendo Co., Ltd. Consolidated sales transition by region. Nintendo Web site:
http://www.nintendo.co.jp/ir/library/historical_data/pdf/consolidated_sales_
e1306.pdf. Accessed August 21, 2013.
54. Flaghouse.ca. GestureTek IREX systems. Flaghouse.ca Web site: http://www.
flaghouse.ca/itemdy00.asp?T1 =39081. Accessed August 21, 2013.
55. World Health Organization (2002) Towards a common language for
functioning, disability and health. World Health Organization Web site:
http://www.who.int/classifications/icf/training/icfbeginnersguide.pdf. Ac-
cessed September 14, 2013.
56. Sullivan KJ, Cen SY (2011) Model of Disablement and Recovery: Knowledge
translation in rehabilitation research and practice. Phys Ther 91: 18921904.
57. Harding J, Harding K, Jamieson P, Mullally M, Politi C, et al. (2009) Children
with disabilities perceptions of activity participation and environments: A pilot
study. Can J Occup Ther 76(3):133144.
58. Majnemer A, Shevall M, Law M, Birnbaum R, Chilingaryan G, et al. (2008)
Participation and enjoyment of leisure activities in school- aged children with
cerebral palsy. Dev Med Child Neurol 50: 751758.
Virtual Reality Therapy
PLOS ONE | www.plosone.org 13 March 2014 | Volume 9 | Issue 3 | e93318