DOI: 10.1111/1471-0528.

14369 Intrapartum care

www.bjog.org

Simulation-based team training for multi-

professional obstetric care teams to improve
patient outcome: a multicentre, cluster
randomised controlled trial
AF Fransen,a,b J van de Ven,a E Schuit,c,d AAC van Tetering,a BW Mol,e SG Oeia,f
a
Department of Obstetrics and Gynaecology, Maxima Medical Centre, Eindhoven-Veldhoven, the Netherlands b Department of Obstetrics and
Gynaecology, Maastricht University Medical Centre, Maastricht, the Netherlands c Julius Center for Health Sciences and Primary Care,
University Medical Center Utrecht, Utrecht, the Netherlands d Stanford Prevention Research Center, Stanford University, Stanford, California,
USA e The Robinson Institute, School of Paediatrics and Reproductive Health University of Adelaide and the South Australian Health and
Medical Research Institute, Adelaide, South Australia, Australia f Department of Electrotechnical Engineering, Eindhoven University of
Technology, Eindhoven, the Netherlands
Correspondence: AF Fransen, Postbus 7777, 5500 MB Veldhoven, the Netherlands. Email annemariefransen@hotmail.com

Accepted 24 August 2016. Published Online 10 October 2016.

Objective To investigate whether simulation-based obstetric team
training in a simulation centre improves patient outcome.
Design Multicentre, open, cluster randomised controlled trial.
Setting Obstetric units in the Netherlands.
Population Women with a singleton pregnancy beyond 24 weeks
of gestation.
Methods Random allocation of obstetric units to a 1-day, multi-
professional, simulation-based team training focusing on crew
resource management (CRM) in a simulation centre or to no
such team training. Intention-to-treat analyses were performed at
the cluster level, including a measurement 1 year prior to the
intervention.
Main outcome measures Primary outcome was a composite
outcome of obstetric complications during the first year post-
intervention, including low Apgar score, severe postpartum
haemorrhage, trauma due to shoulder dystocia, eclampsia and
hypoxic-ischaemic encephalopathy. Maternal and perinatal
mortality were also registered.
Results Each study group included 12 units with a median unit size of
1224 women, combining for a total of 28 657 women. In total, 471
medical professionals received the training course. The composite
outcome of obstetric complications did not differ between study
groups [odds ratio (OR) 1.0, 95% confidence interval (CI) 0.80–1.3].
Team training reduced trauma due to shoulder dystocia (OR 0.50,
95% CI 0.25–0.99) and increased invasive treatment for severe
postpartum haemorrhage (OR 2.2, 95% CI 1.2–3.9) compared with
no intervention. Other outcomes did not differ between study groups.
Conclusion A 1-day, off-site, simulation-based team training,
focusing on teamwork skills, did not reduce a composite of
obstetric complications.
Keywords Multi-professional training, obstetric care, patient
outcome, simulation, team training, teamwork skills.
Tweetable abstract 1-day, off-site, simulation-based team training
did not reduce a composite of obstetric complications.
Linked article This article is commented on by T Draycott, p. 651
in this issue. To view this mini commentary visit http://
dx.doi.org/10.1111/1471-0528.14391.

Please cite this paper as: Fransen AF, van de Ven J, Schuit E, van Tetering AAC, Mol BW, Oei SG. Simulation-based team training for multi-professional
obstetric care teams to improve patient outcome: a multicentre, cluster randomised controlled trial. BJOG 2017;124:641–650.

individual’s performance.1 Care teams with a good clinical

Introduction
Clinical performance of obstetric care teams is associated
with the quality of their teamwork skills instead of an
Trial Registration: the Dutch Trial Register, NTR 1859; http://www.trialreg
ister.nl/trialreg/admin/rctview.asp?TC=1859
performance are more likely to state the emergency earlier,
use more structured handovers and closed loop communi-
cation.1 A lack of such teamwork behaviours is identified
as a main contributing factor in approximately 75% of pre-
ventable medical errors.2 As these errors undermine safety
of patients,3,4 improvement of teamwork skills with

ª 2016 Royal College of Obstetricians and Gynaecologists 641

 Fransen et al.
simulation-based team training is recommended by various
healthcare institutes.4,5
Simulation-based medical education is recommended as
it is a more effective learning method than traditional edu-
cation for teaching medical technical skills.6,7 For teamwork
skills, the learning effect seems to be comparable.8–10 The
strength of simulation-based education lies in the opportu-
nity for sustained, goal-oriented and deliberate practice for
the acquisition of knowledge, skills and attitudes without
any risks for patients.11,12 To evaluate the effect of such
training courses, Kirkpatrick’s classification is often used.
This consists of four levels, starting with (1) participants’
perception, followed by (2) assessment of knowledge and
skills, to (3) actions in clinical practice and (4) patient out-
come.13–15
Despite the acknowledged importance of simulation-
based team training in obstetrics, evidence from ran-
domised controlled trials demonstrating improvement of
maternal and perinatal outcome is still lacking.9,10 To fill in
this gap, we set up a cluster randomised controlled trial to
investigate the effectiveness of a 1-day, off-site, simulation-
based obstetric team training.16 We previously showed that
this simulation-based training was effective on level three
of Kirkpatrick with both significant improved teamwork
skills (7.5 versus 6.0 on the Clinical Teamwork Scale;17
P = 0.014), and increased use of essential medical technical
skills (83% versus 46%, P = 0.009; RR 1.8, 95% CI 1.1–
3.0).16 These results correspond with results of other stud-
ies.18,19 The main question is whether this kind of training
course will also lead to fewer obstetric complications (Kirk-
patrick level four).
In this paper, we evaluated whether a 1-day, simulation-
based obstetric team training in a simulation centre affects
the number of obstetric complications in comparison with
the absence of such team training during a 1-year evalua-
tion.
Methods
Trial design and study population
Between November 2009 and July 2011, we conducted an
open, multicentre, parallel, cluster randomised controlled
trial to evaluate the effectiveness on obstetric complications
of a 1-day, simulation-based obstetric team training in a
medical simulation centre in comparison with the absence
of such team training (TOSTI study).20 The simulation-
based obstetric team training focused on crew resource
management (CRM) skills in obstetric emergency situa-
tions. As this was an open randomised trial, no one
involved in the study was blinded to the allocation. As the
allocation of the intervention was at the cluster level, the
institutional review board of the Maxima Medical Centre
decided that ethical approval was not necessary. The
642
original study protocol was described online in the Dutch
Trial Register (NTR no. 1859)21 and a study protocol paper
was published.20
The study was performed in Dutch hospitals, each fea-
turing one obstetric unit that represented one cluster.
Obstetric units already participating in multi-professional
team training were not eligible for this study. Before ran-
domisation, trial consent was approved by a local obstetri-
cian. The included obstetric units were randomly allocated
in a 1:1 ratio to simulation-based obstetric team training
or to the control group. This was done by an independent
researcher using a computer-generated list. Randomisation
was stratified for units being situated in teaching or non-
teaching hospitals, as the presence of residents might affect
patient care and the effectiveness of the training interven-
tion. The obstetric units in the control group agreed to
conduct no simulation-based team training courses during
the complete study period. All included units were allowed
to continue existing local individual skills training pro-
grammes as long as they did not include team training.
All women with a singleton pregnancy beyond 24 weeks
of gestation were included during a 1-year evaluation. The
evaluation period of 1 year started whenever all staff mem-
bers had received the intervention. To determine the study
period for the units in the control group, each unit was
linked to a unit in the intervention group. As cluster ran-
domised controlled trials are commonly affected by an
imbalance in baseline characteristics on patient level, and
as the potential effect of team training depends on the pre-
intervention morbidity rates, an additional pre-intervention
measurement was conducted retrospectively over a period
of 1 year prior to the intervention.
At the time of the study period, team training courses
were not mandatory in Dutch obstetric units, healthcare
professionals were obliged to report their perinatal data to
a national perinatal registry (Perined)22 and national peri-
natal audits were started in 2010. As a randomised design
was applied, an even distribution of these factors among
the study groups was expected.
Intervention
Units allocated to the intervention group were scheduled
for team training. Units allocated to the control group
received no team training. The intervention comprised a 1-
day (8-hour), simulation-based, multi-professional obstetric
team training, focusing on CRM skills (i.e. teamwork
skills), in a medical simulation centre (Medsim) in Eind-
hoven, the Netherlands. The training course was provided
once. The entire multi-professional staff of the obstetric
units of the intervention group were obliged to participate.
They were divided into several multi-professional obstetric
teams consisting of a gynaecologist/obstetrician, a sec-
ondary care midwife and/or a resident, and nurses. Primary
ª 2016 Royal College of Obstetricians and Gynaecologists

care midwives and maternity assistants did not participate
in the team training, as they work in private practices and
are not part of the obstetric unit in the hospital. Anaesthe-
siologists were not included because they are not part of a
standard care team in the delivery room in the Nether-
lands.
Teamwork training was delivered within the context of
clinical training. The main learning objective of the training
course concerned the improvement of CRM and communi-
cation skills between team members in obstetric emergen-
cies, including the following: leadership and role clarity,
distribution of work load, decision making, situational
awareness, directed and closed-loop communication, pre-
vention and management of fixation errors and Situation,
Background, Assessment, Recommendation (SBAR) han-
dover. Approximately 80% of the training course was spent
on teamwork skills and the remaining 20% on medical
technical skills. This was reflected in the debriefing sessions
and the oral presentations after the first scenario about
CRM (30 minutes) and medical technical skills [definition
of cardiotocographic nomenclature (5 minutes) and shoul-
der dystocia (10 minutes)]. To provide the opportunity to
rehearse teamwork skills, five clinical scenarios were pre-
sented with an increasing difficulty level: shoulder dystocia,
eclampsia, umbilical cord prolapse, postpartum haemor-
rhage and resuscitation of a pregnant woman. In contrast
to the protocol paper, a scenario concerning fetal distress
was not included.20 The treatment protocol of these scenar-
ios was based on common national [Dutch Society of
Obstetrics and Gynaecology (NVOG)] and international
guidelines [Royal College of Obstetricians and Gynaecolo-
gists and Managing Obstetric Emergencies & Trauma
(MOET)]. As these guidelines are widely recognised, similar
treatment protocols were guaranteed in all included units.
A more thorough description of the scenarios, learning
goals and instructions for facilitators is available online
(Appendix S1).
The medical simulation centre was equipped with four
simulation and (de)briefing rooms, interactive full-body
simulators, a control room, and audio-visual products.
Each training day was facilitated by two facilitators, out of
a group of ten, with several years of experience: an obstetri-
cian and a communication expert. All facilitators with a
medical background underwent instructor training for
facilitating a simulation-based team training course (Euro-
pean Simulator Instructor Course provided by EUSIM)23
and some of them took an additional instructor course
(Generic Instructor Course provided by MOET).24 The
communication experts were experienced team coaches
working for an international company specialised in learn-
ing and development (Schouten & Nelissen, part of Schou-
ten Global). Two full-body mannequins were chosen for
their high-fidelity characteristics [NoelleTM (Gaumard,
ª 2016 Royal College of Obstetricians and Gynaecologists
Simulation-based team training and patient outcome
Miami, FL, USA) and the Emergency Care Simulator ECSTM
(METI, Sarasota, FL, USA)].
Each training day started with a 15-minute orientation
with the simulation centre and simulation equipment. Dur-
ing this introduction, the facilitator emphasised that indi-
vidual performances should not be discussed outside the
simulation centre to assure the safety of all participants.
The simulation rooms were furnished as a delivery room,
including a delivery bed, resuscitation equipment, a CTG-
monitor and a telephone. A table with the usual labour
ward equipment was present in the room. Before the start
of each scenario, the participants received work clothes
(white coats) to put over their normal clothes. The scenar-
ios were introduced to the participants who would encoun-
ter the patient in the first place by a short (approximately
5 minutes) standardised briefing video. These videos
involved actors who illustrated the situation prior to the
hospital submission. Directly after the video, the partici-
pants entered the simulation room where the simulated
scenario took place. Each scenario ended when the clinical
situation was managed (for example after the delivery of
the baby in the case of a shoulder dystocia), or when the
scenario was under control after approximately 15 minutes.
The difficulty of the scenarios could be tailored to the per-
formance of the care team. Each scenario was debriefed by
two facilitators (approximately 30 minutes), supported by
relevant video recordings. During the debriefing, partici-
pants’ reactions, a deeper analysis of performance (by
group discussion), and take-home messages were discussed.
To maintain the learning effect, every unit received hard-
copy flowcharts outlining the addressed clinical protocols
and SBAR handover (Appendix S1).
Outcomes
Primary outcome was a composite outcome of the number
of obstetric complications, i.e. low Apgar score, severe
postpartum haemorrhage, trauma due to shoulder dystocia,
eclampsia and hypoxic-ischaemic encephalopathy (HIE).
The obstetric complications, with their definitions, are
listed in Table 1 and are thoroughly described in
Appendix S1. Additional secondary outcomes of interest
were perinatal and maternal mortality. Perinatal mortality
was defined as fetal or neonatal mortality beyond 24 weeks
of gestation and within 7 days postpartum. Maternal mor-
tality was defined as the death of a woman during or
within 6 weeks after pregnancy. The intervention and all
outcome measures were evaluated at the cluster level. All
outcomes were collected for the pre- as well as the post-
intervention period, both of which lasted for 1 year.
Data collection
All participating units prospectively registered cases in
which the outcome measures occurred during the first year
643
 Fransen et al.
Table 1. Components of composite outcome
Components of Definition
composite outcome
Low Apgar Score 5-minute Apgar score <7*
Severe postpartum Administration of >4 packed cells blood
haemorrhage transfusion, or the performance of an
embolisation or hysterectomy
in case of a postpartum haemorrhage.
Trauma due to A brachial plexus injury, clavicle fracture,
shoulder dystocia humeral fracture or other injuries
(e.g. other fractures, pneumothorax,
hypoxia/acidosis, HIE or mortality)
due to shoulder dystocia
Eclampsia Eclamptic convulsions
Hypoxic-ischaemic HIE, described using Sarnat stages
encephalopathy (HIE)
*This outcome measure differs from its prespecified measure
‘perinatal asphyxia’ defined as ‘Apgar score after 5 minutes <7 and/
or pH of the umbilical artery <7.05’.20 This change was made
because the units used different protocols for taking umbilical blood
samples. The adjustment of the outcome measure was made before
analysis.21 The original definition was added as a secondary
outcome measure.
after intervention. A local gynaecologist/obstetrician and a
staff member were responsible for registering the outcome
measures with provided registration lists to which we pro-
vided a monthly update. As data collection by local staff
might be open to imprecision, the validity of the data col-
lection for low Apgar score, perinatal and maternal mortal-
ity was verified and complemented with data from
Perined.22 The verification of the other outcome measures
was done by visiting the hospitals to check their local regis-
trations for missing primary outcome data. Whenever cases
were identified which were not registered by the local staff,
they were added to the database. These activities were done
to increase the reliability of the database as much as possi-
ble. Case report forms were designed and filled out to
gather more information about the cases. These data were
transformed into electronical data files automatically. The
demographic characteristics of all deliveries, including the
non-complicated ones, were provided by Perined.22 Three
researchers were responsible for the data collection (A.F.,
J.V. and A.T.). A.F. was responsible for the development of
the database.
Statistical analysis and sample size calculation
In the original study protocol, the sample size was set at
8000 women in each study group. This would allow us to
assess a reduction in ‘perinatal asphyxia’ (defined as Apgar
score after 5 minutes <7 and/or pH of the umbilical artery
<7.05) from 1 to 0.6%.21 Since the start of recruitment for
644
our study, the sample size has been adjusted. In the recal-
culation of the sample size, we assumed an intra-cluster
correlation coefficient (ICC) of 0.08. We calculated that we
needed a sample size of 24 clusters, 12 clusters in each
study group, with a cluster size of at least 200 deliveries for
a power of 80% and an alpha value of 0.05 (two-sided).20
This was based on a reduction from 15 to 5% in the com-
posite outcome. The 15% outcome rate in the control
group was derived from data from Perined and NVOG
guidelines. The definition of the primary outcome has not
been changed between the original protocol, trial registra-
tion, the published BMC protocol and the current manu-
script. The final sample size calculation was finalised before
all hospitals had received the intervention (or control) in
July 2010, prior to the statistical data analysis, and was for-
mulated in the protocol submitted to BMC in January
2010.
Analyses were performed according to intention-to-treat.
The difference in morbidity and mortality rates (at the
cluster level) between the two study groups was assessed by
calculating the odds ratio (OR) with 95% confidence inter-
vals (95% CI) using a random intercept logistic regression
model. The use of a random intercept per hospital was
required, as the pre- and post-intervention measurement
comprised different women. Stratified randomisation was
taken into account by including the presence of residents
in a hospital as a covariate to the regression model.25 Fur-
thermore, to account for possible imbalances in baseline
characteristics, the pre-intervention morbidity and mortal-
ity rates were added to the model as a covariate as well. To
allow comparison with the study of Draycott et al.26 and as
the training could possibly be most effective in term preg-
nancies, we performed an additional analysis including only
women with a term singleton pregnancy in cephalic presen-
tation and without an elective caesarean. The idea for this
subgroup analysis was conceived before data analysis was
performed. We performed one post-hoc sensitivity analysis
in which treatment for severe postpartum haemorrhage was
excluded from the composite outcome, as this outcome
measure seemed to relate to a behaviour rather than an
actual patient outcome.
Results
We approached 36 obstetric units: eight did not meet the
inclusion criteria and four units declined participation.
The remaining 24 units were randomised and included in
the analysis (Figure 1). Each study group contained 12
obstetric units, five teaching and seven non-teaching units.
All obstetric units in the intervention group received the sim-
ulation-based team training between November 2009 and
July 2010. Both post- and pre-intervention periods for each
unit are displayed in Figure S1. All teams belonging to one
ª 2016 Royal College of Obstetricians and Gynaecologists

Enrolment
Allocated to intervention group (n = 12)
♦ Received allocated intervention (n = 12)
♦ Did not receive allocated intervention (n = 0)
Lost to follow-up (n = 0)
Discontinued intervention (n = 0)
Analysed units (n = 12)
♦ Excluded from analysis (n = 0)
Figure 1. CONSORT flow diagram of enrolment of Dutch obstetric units in the cluster randomised design.
obstetric unit received the training within a time period of
4 weeks. No units were excluded after randomisation.
In the intervention group with 12 units in total, 74 mul-
ti-professional teams, comprising 471 healthcare profession-
als, received the training: 74 gynaecologists/obstetricians, 36
residents, 79 secondary care midwives and 282 nurses. The
teams varied between three and nine participants, with a
median size of seven. Whenever a small care team was pre-
sent, one of the facilitators could perform an extra role in
the scenario. Participation rates of the healthcare profes-
sionals of the included obstetric units were around 95%.
The facilitator to participant ratio ranged between 1:2 and
1:4.5.
In the intervention group we studied 14 500 pregnancies
of singletons beyond 24 weeks of gestation compared with
14 157 in the control group in the post-intervention per-
iod. The median cluster size comprised 1224 women [in-
terquartile range (IQR) of 845 to 1509 women]; the data at
cluster level are presented in Figure S1. Demographic char-
acteristics of all deliveries in the pre- and post-intervention
period showed significant differences between the interven-
tion and control group for the parity, presentation at birth,
(medium and high) socio-economic status, and mean
maternal age (Table 2).
Primary and secondary outcomes
The composite of obstetric complications did not differ
between study groups (287/14 500 (2.0%) versus 299/
ª 2016 Royal College of Obstetricians and Gynaecologists
Simulation-based team training and patient outcome
Assessed units for eligibility
(n = 36)
Excluded units (n = 12)
♦ Not meeting inclusion criteria
(n = 8)
♦ Declined to participate (n = 4)
Randomised units
(n = 24)
Allocation
Allocated to control group (no intervention)
(n = 12)
♦ Received no intervention (n = 12)
♦ Received the intervention (n = 0)
Follow-Up
Lost to follow-up (n = 0)
Discontinued intervention (n = 0)
Analysis
Analysed units (n = 12)
♦ Excluded from analysis (n = 0)
14 157 (2.1%); OR 1.0; 95% CI 0.80–1.3). A breakdown of
the primary outcome indicated less trauma due to shoulder
dystocia in the intervention group (0.16% versus 0.25%;
OR 0.50; 95% CI 0.25–0.99) (Table 3), which was mainly
driven by a lower number of clavicle fractures. Further-
more, blood transfusion with >4 packed cells, embolisation
or hysterectomy for massive postpartum haemorrhage
occurred more often in the intervention group than in the
control group (0.28% versus 0.13%; OR 2.2; 95% CI 1.2–
3.9) (Table 3). No differences between the intervention and
the control group were found for low Apgar score or low
Apgar score combined with an arterial umbilical pH <7.05.
The number of women with eclampsia was lower and HIE
higher in the intervention group, but these differences did
not reach statistical significance (Table 3). The perinatal
mortality rate in the intervention group was 0.45% com-
pared with 0.55% in the control group (OR 0.75; 95% CI
0.53–1.07) (Table 3). One maternal death occurred;
this was in the control group and was caused by an
amniotic fluid embolism in a multipara woman with a
term pregnancy. Resuscitation was started and a peri-
mortem caesarean section was performed after more than
5 minutes.
Subgroup analysis of women with singletons in cephalic
presentation at term (exclusion of elective caesareans)
showed results that were similar to the results found in the
analysis of the total study group (Table S1). The post-hoc
sensitivity analysis of a modified composite outcome
645
 Fransen et al.

Table 2. Demographic characteristics of singleton deliveries beyond 24 weeks of gestation: pre- and post-intervention periods for both
intervention and control group

Characteristics Pre-intervention period P-value * Post-intervention period P-value *

Intervention Control group Intervention Control group

group n = 13 971 n = 13 538 group n = 14 500 n = 14 157

Maternal age, mean (SD), years 30.0 (5.0) 30.5 (5.0) <0.0001 30.0 (5.0) 30.4 (5.0) <0.0001
Parity
Nulliparous, n (%) 7318 (52.4) 7387 (54.6) 0.0003 7352 (50.7) 7530 (53.2) <0.0001
Multiparous, n (%) 6653 (47.6) 6150 (45.4) 7145 (49.3) 6624 (46.8)
Gestational age, median (IQR), days 279 (16.0) 279 (16.0) 0.19 278 (15.0) 278 (16.0) 0.56
24w0d–<28w0d, n (%) 20 (0.1) 8 (0.1) 0.12 17 (0.1) 17 (0.1) 0.17
28w0d–<32w0d, n (%) 26 (0.2) 32 (0.2) 19 (0.1) 32 (0.2)
32w0d–<37w0d, n (%) 1020 (7.3) 1013 (7.5) 973 (6.7) 999 (7.1)
≥37w0d, n (%) 12 905 (92.4) 12 485 (92.2) 13 491 (93.0) 13 109 (92.6)
Presentation at birth
Cephalic, n (%) 13 072 (93.6) 12 651 (93.5) 0.02 13 622 (93.9) 13 282 (93.8) 0.0008
Transverse, n (%) 64 (0.46) 47 (0.35) 39 (0.27) 73 (0.52)
Breech, n (%) 735 (5.3) 809 (6.0) 726 (5.0) 778 (5.5)
Birth weight, mean (SD), grams 3426 (582) 3442 (582) 0.03 3429 (564) 3433 (574) 0.54
Socio-economic status
High, n (%) 2448 (17.5) 3415 (25.2) <0.0001 2553 (17.6) 3468 (24.5) <0.0001
Medium, n (%) 7296 (52.2) 5658 (41.8) 7395 (51.0) 5976 (42.2)
Low, n (%) 4072 (29.2) 4112 (30.4) 4408 (30.4) 4340 (30.7)

*P-values are calculated using chi-square tests (for categorical and dichotomous data) and t-test (for continuous data). Data have been analysed
as a complete case analysis.

without postpartum haemorrhage did not lead to different
results.
Discussion
Main findings
Our multicentre, cluster randomised controlled trial
revealed that a 1-day, multi-professional, simulation-based
obstetric team training in a simulation centre, focusing on
teamwork skills, did not reduce a composite of obstetric
complications. When we looked at the components of the
primary outcome, a two-fold reduction in neonatal damage
due to shoulder dystocia and a two-fold increase of treat-
ment with ≥4 packed cells of blood transfusion, embolisa-
tion or hysterectomy in the case of a postpartum
haemorrhage were found. For the other outcome measures
we found no statistically significant differences between the
team training group and the control group.
Strengths and limitations
The most important strength of our study is the cluster
randomised design. A second strength concerns the inter-
vention itself, with a high participation rate, opportunity
for repetition of teamwork skills and a thorough debriefing
by two professional facilitators. Thirdly, we assessed the
training intervention on the level of patient outcome after
we previously observed a positive change in teamwork and
medical technical skills.16 This is desirable, as simulation-
based teamwork training has been recommended by previ-
ous inquiries to improve patient safety.4,5
Limitations of our study should also be considered. First,
we changed our original sample size calculation to take
into account the cluster design of the study by including
an ICC in the sample size calculation. As indicated, this
was done while the intervention was running and prior to
data analysis. Secondly, the actual incidence of the compos-
ite outcome was much lower (2%) than the 15% that was
anticipated. This overestimation was mainly caused as we
first considered all women with postpartum haemorrhages
to suffer the primary endpoint. However, as there was no
indication of a difference in the composite outcome
between the two groups, it is unlikely a larger sample size
would have changed our results. Thirdly, we used a non-
validated composite outcome, which may be a disadvan-
tage, as the underlying process of the individual compo-
nents can be different,27 as was the case for postpartum
haemorrhage. A sensitivity analysis, in which postpartum
haemorrhage was not considered a component of the

646 ª 2016 Royal College of Obstetricians and Gynaecologists

 Simulation-based team training and patient outcome

Table 3. Associations between intervention and patient outcome in singleton pregnancies beyond 24 weeks’ of gestation

Primary and secondary Pre-intervention Post-intervention Odds ratio P-value

outcomes (95% CI)
Intervention Control group Intervention group Control group
group n = 13 971 n = 13 538 n = 14 500 n = 14 157

Composite of obstetric 273 (2.0%) 302 (2.2%) 287 (2.0%) 299 (2.1%) 1.0 (0.80–1.3) 0.90
complications
Low Apgar Score 211 (1.5%) 244 (1.8%) 226 (1.6%) 251 (1.8%) 0.96 (0.74–1.2) 0.72
Severe postpartum haemorrhage 20 (0.14%) 32 (0.24%) 41 (0.28%) 19 (0.13%) 2.2 (1.2–3.9) 0.009
>4 packed cells 16 (0.11%) 31 (0.23%) 34 (0.23%) 18 (0.13%) 2.1 (1.1–3.8) 0.021
Embolisation 8 (0.06%) 9 (0.07%) 10 (0.07%) 3 (0.02%) 4.7 (1.3–17) 0.020
Hysterectomy 1 (0.01%) 4 (0.03%) 10 (0.07%) 1 (0.01%) 10 (0.99–120) 0.051
Trauma due to shoulder dystocia 43 (0.31%) 26 (0.19%) 23 (0.16%) 35 (0.25%) 0.50 (0.25–0.99) 0.048
Brachial plexus injury 13 (0.09%) 9 (0.07%) 8 (0.06%) 6 (0.04%) 1.3 (0.39–4.3) 0.68
Clavicle fracture 23 (0.16%) 11 (0.08%) 13 (0.09%) 26 (0.18%) 0.38 (0.15–0.93) 0.034
Humeral fracture 7 (0.05%) 8 (0.06%) 3 (0.02%) 2 (0.01%) 1.5 (0.25–9.1) 0.65
Other injury 2 (0.01%) 0 (0.0%) 0 (0.0%) 2 (0.01%) NA NA
Eclampsia 4 (0.03%) 6 (0.04%) 6 (0.04%) 12 (0.08%) 0.67 (0.19–2.4) 0.54
HIE 7 (0.05%) 5 (0.04%) 14 (0.10%) 4 (0.03%) 3.2 (0.77–13) 0.11
Perinatal mortality 82 (0.59%) 59 (0.44%) 65 (0.45%) 78 (0.55%) 0.75 (0.53–1.07) 0.11
Maternal mortality 0 (0.0%) 1 (0.01%) 0 (0.0%) 1 (0.01%) NA NA
Low Apgar score and 342 (2.4%) 339 (2.5%) 373 (2.6%) 355 (2.5%) 1.0 (0.77–1.3) 0.98
arterial umbilical
pH <7.05

HIE, hypoxic-ischemic encephalopathy; NA, not applicable.

The components of the composite outcome do not add up to the group total as one individual can have multiple components.

composite outcome, showed results similar to the primary
analysis. A fourth limitation are the differences in the base-
line characteristics at patient level, which were probably
caused by the randomisation on cluster level. However, we
accounted for this by including the pre-intervention out-
come rate as a covariate in our analysis. Finally, all obstet-
ric units were allowed to continue local individual skills
training programmes during the study. As information on
these training programmes was not collected, we were not
able to assess this distribution.
Interpretation
The lack of an effect on obstetric complications is intrigu-
ing, as we previously showed that the applied intervention
successfully improved teamwork skills and the application
of medical technical skills.16 A similar absence of effect on
patient outcome has been observed in a previous ran-
domised controlled trial by Nielsen et al.28 who investi-
gated the effect of a standardised teamwork training
curriculum (without simulation), on the Adverse Outcome
Index (AOI) and Weighted Adverse Outcome Score
(WAOS). Walker et al.29 who investigated the effect of a
low-tech, simulation-based team training course, also found
no effect on maternal complications. Contrary to these
findings, non-randomised studies have shown positive
effects of simulation-based team training on composite
outcome measures like the WAOS8 and the AOI.30 It might
have been interesting to use these validated composite out-
come measures.
The decrease of trauma due to shoulder dystocia in the
intervention group compared with the control group is in
alignment with results from other studies.31,32 In contrast
to most studies, this decrease was mainly caused by a lower
number of clavicle fractures instead of brachial plexus
injury. However, in both study groups the number of
plexus lesions was very low.
An increased use of >4 packed cells of blood transfusion,
embolisation or hysterectomy for the treatment of postpar-
tum haemorrhage was found in the intervention group. As
these interventions were encouraged during the training
course, the increased use might represent a learning effect
comparable to the change in behaviour demonstrated in
our previous publication.16 However, other studies consider
an increase in interventions such as blood transfusion to be
a negative outcome, as it might indicate insufficient man-
agement of postpartum bleeding or over-treatment.33,34
The number of babies with a low Apgar score was com-
parable in both study groups, whereas previous non-rando-
mised studies had reported a decrease of low Apgar score
after simulation-based obstetric team training.26 The

ª 2016 Royal College of Obstetricians and Gynaecologists 647

 Fransen et al.
relatively high number of low 5-minute Apgar scores com-
pared with other literature might have been caused by the
inclusion of women with gestational ages of 24 weeks
onwards.35,36 For eclampsia and HIE, the numbers were
very low. In contrast to Draycott et al.,26 we found a non-
significant higher rate of HIE in the intervention group
than in the control group. This might be explained by a
non-significant decreased number of perinatal mortalities.
Several factors concerning the intervention itself should
be considered to explain the lack of an effect on patient
outcome. First, the training model was limited to a 1-day
training course, in contradiction to the current recommen-
dation of distributed practice and repetition.37–40 Secondly,
the training course focused on the acquisition of teamwork
skills and, as such, time was only provided for repetition of
teamwork skills, not for medical technical skills. Thirdly,
the training course took place at a simulation centre,
whereas ‘in-house’ training courses are currently recom-
mended.41,42 Fourthly, the clinical content of the training
course was partly based on MOET. However, MOET has
not proven to lead to improved patient outcomes in the
Netherlands.43,44 Besides, it could be that CRM training
leads to improved teamwork skills but does not result in
improved patient outcomes.45 Fifthly, the use of patient
actors instead of high-fidelity mannequins could have
resulted in better results for communication skills.46
Finally, anaesthesiologists were not included in the training
as they are not part of a standard obstetric care team in
the Netherlands. Future research should consider their
involvement in obstetric team training.
Another explanation could be that the primary care mid-
wives, who work independently from the hospital, did not
receive the intervention. These midwives initially care for
more than 50% of women during labour in the Dutch
obstetric care system. Of the women in labour who have
been cared for by midwives, 44%47 are transferred to a sec-
ondary care team in the hospital for reasons that include
postpartum haemorrhage, fetal distress and obstructed
labour. Before women are transferred to a secondary care
team, complications such as postpartum haemorrhage or
fetal distress may already have occurred.
Future efforts should focus on the optimisation of the
design of simulation-based team training courses. The
comparison of different training models could be done
using a stepped wedge design.48–50 It would also be inter-
esting to assess patient outcomes at different time points,
to decide the right time for repeating the training course.
For the Netherlands specifically, it would be worth trying
to involve the independently working primary care mid-
wives. This may be feasible in the future, as the Dutch gov-
ernment has recently decided to change the Dutch obstetric
care system to a shared care obstetric system.51
648
Conclusion
This open, multicentre, parallel, cluster randomised con-
trolled trial demonstrated that a 1-day, simulation-based
obstetric team training of hospital teams in a simulation
centre, focusing for 80% on teamwork skills, did not
improve a composite of obstetric complications. Other
training models for simulation-based training
courses should be investigated using the same robust
methodologies.
Disclosure of interest
Full disclosure of interests available to view online as sup-
porting information.
Contribution to authorship
SGO, BWM were involved in the conception and design of
the study. SGO, BMW and AF were responsible for the
integrity of the work. AF, JV and AT were responsible for
data collection. ES was, in close collaboration with AF,
responsible for conducting the analysis. AF drafted the
manuscript. SGO, JV, AT, ES and BWM reviewed the
manuscript and approved submission.
Details of ethics approval
The institutional review board of the Maxima Medi-
cal Centre judged that ethical approval was not
necessary as the intervention was allocated at the cluster
level.
Funding
This study is funded by ZonMW, the Netherlands Organi-
zation for Health Research and Development, grant
(170992303). This organisation was not involved in the
design and conduct of the study; collection, management,
analysis, and interpretation of the data; preparation, review,
or approval of the manuscript; or decision to submit the
manuscript for publication.
Acknowledgements
The authors like to thank A. van Meurs (native English
speaker) for reviewing the manuscript.
Published study protocol
The published study protocol is accessible at: http://
www.biomedcentral.com/1471-2393/10/59 (accessed 16
April 2016).
Supporting Information
Additional Supporting Information may be found in the
online version of this article:
ª 2016 Royal College of Obstetricians and Gynaecologists

Figure S1. Pre- and post-intervention periods for each
unit.
Table S1. Associations between intervention and patient
outcome in term singletons in cephalic presentation (exclu-
sion of elective caesareans).
Appendix S1. Background information about the TOSTI
study. &
References
1 Siassakos D, Fox R, Bristowe K, Angouri J, Hambly H, Robson L,
et al. What makes maternity teams effective and safe? lessons from
a series of research on teamwork, leadership and team training.
Acta Obstet Gynecol Scand 2013;92:1239–43.
2 Martijn L, Jacobs A, Amelink-Verburg M, Wentzel R, Buitendijk S,
Wensing M. Adverse outcomes in maternity care for women with a
low risk profile in The Netherlands: A case series analysis. BMC
Pregnancy Childbirth 2013;13:219.
3 Confidential Enquiry into Maternal and Child Health (CEMACH).
Why Mothers Die 2000–2002. 6th report. London: RCOG Press at
Royal College of Obstetricians and Gynaecologists, 2004.
4 Confidential Enquiry into Maternal Deaths in the United Kingdom
(CMACE). Saving Mothers’ Lives: Reviewing maternal deaths to
make motherhood safer: 2006–2008. The Eighth Report. BJOG
2011;118:1–203.
5 Institute of Medicine. To Err Is Human; Building a Safer Health
System. Washington, DC: National Academy Press, 2000.
6 McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB.
Does simulation-based medical education with deliberate practice
yield better results than traditional clinical education? A meta-
analytic comparative review of the evidence. Acad Med
2011;86:706–11.
7 Daniels K, Arafeh J, Clark A, Waller S, Druzin M, Chueh J.
Prospective randomized trial of simulation versus didactic teaching
for obstetrical emergencies. Simul Healthc 2010;5:40–5.
8 Riley W, Davis S, Miller K, Hansen H, Sainfort F, Sweet R. Didactic
and simulation nontechnical skills team training to improve perinatal
patient outcomes in a community hospital. Jt Comm J Qual Patient
Saf 2011;37:357–64.
9 Meri en AE, van de Ven J, Mol BW, Houterman S, Oei SG.
Multidisciplinary team training in a simulation setting for acute
obstetric emergencies: a systematic review. Obstet Gynecol
2010;115:1021–31.
10 Boet S, Bould MD, Fung L, Qosa H, Perrier L, Tavares W, et al.
Transfer of learning and patient outcome in simulated crisis resource
management: a systematic review. Can J Anaesth 2014;61:571–81.
11 Kneebone R. Simulation in surgical training: educational issues and
practical implications. Med Educ 2003;37:267–77.
12 Ericsson KA. Deliberate practice and the acquisition and
maintenance of expert performance in medicine and related
domains. Acad Med 2004;79:S70–81.
13 Kirkpatrick DL, Kirkpatrick JD. Evaluating Training Programs: The
Four Levels, 3rd edn. San Francisco: Berret-Koehler Publishers Inc.,
2006.
14 Bergh AM, Baloyi S, Pattinson RC. What is the impact of multi-
professional emergency obstetric and neonatal care training? Best
Pract Res Clin Obstet Gynaecol 2015;29:1028–43.
15 Calvert KL, McGurgan PM, Debenham EM, Gratwick FJ, Maouris P.
Emergency obstetric simulation training: how do we know where
we are going, if we don’t know where we have been? Aust N Z J
Obstet Gynaecol 2013;53:509–16.
ª 2016 Royal College of Obstetricians and Gynaecologists
Simulation-based team training and patient outcome
16 Fransen AF, van de Ven J, Meri en AE, de Wit-Zuurendonk LD,
Houterman S, Mol BW, et al. Effect of obstetric team training on
team performance and medical technical skills: a randomised
controlled trial. BJOG 2012;119:1387–93.
17 Guise JM, Deering SH, Kanki BG, Osterweil P, Li H, Mori M, et al.
Validation of a tool to measure and promote clinical teamwork.
Simul Healthc 2008;3:217–23.
18 Crofts JF, Bartlett C, Ellis D, Hunt LP, Fox R, Draycott TJ. Training for
shoulder dystocia: a trial of simulation using low-fidelity and high-
fidelity mannequins. Obstet Gynecol 2006;108:1477–85.
19 Siassakos D, Hasafa Z, Sibanda T, Fox R, Donald F, Winter C, et al.
Retrospective cohort study of diagnosis-delivery interval with
umbilical cord prolapse: the effect of team training. BJOG
2009;116:1089–96.
20 van de Ven J, Houterman S, Steinweg RA, Scherpbier AJJA, Wijers W,
Mol BWJ, et al. Reducing errors in health care: cost-effectiveness of
multidisciplinary team training in obstetric emergencies (TOSTI study);
a randomised controlled trial. BMC Pregnancy Childbirth 2010;10:59.
21 Dutch trial registration. [www.trialregister.nl/trialreg/admin/rctview.
asp?TC=1859]. Accessed 27 July 2016.
22 Perined. [www.perined.nl/wat-doen-we/registratie]. Accessed 27 July
2016.
23 EUSIM, European simulator instructor course. [www.metscenter.
nl/trainingen/training/eusim-basic-eng/] Accessed 27 July 2016.
24 MOET, generic instructor course. [https://moetcursus.nl/generic-
instructor-course] Accessed 27 July 2016.
25 Kernan WN, Viscoli CM, Makuch RW, Brass LM, Horwitz RI.
Stratified randomization for clinical trials. J Clin Epidemiol
1999;52:19–26.
26 Draycott T, Sibanda T, Owen L, Akande V, Winter C, Reading S,
et al. Does training in obstetric emergencies improve neonatal
outcome? BJOG 2006;113:177–82.
27 Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR,
Moher D, et al. Increasing value and reducing waste in research
design, conduct, and analysis. Lancet 2014;383:166–75.
28 Nielsen PE, Goldman MB, Mann S, Shapiro DE, Marcus RG, Pratt SD,
et al. Effects of teamwork training on adverse outcomes and
process of care in labor and delivery: a randomized controlled trial.
Obstet Gynecol 2007;109:48–55.
29 Walker DM, Cohen SR, Fritz J, Olvera-Garcıa M, Zelek ST, Fahey JO,
et al. Impact evaluation of PRONTO mexico: a simulation-based
program in obstetric and neonatal emergencies and team training.
Simul Healthc 2016;11:1–9.
30 Phipps MG, Lindquist DG, McConaughey E, O’Brien JA, Raker CA,
Paglia MJ. Outcomes from a labor and delivery team training program
with simulation component. Am J Obstet Gynecol 2012;206:3–9.
31 Draycott TJ, Crofts JF, Ash JP, Wilson LV, Yard E, Sibanda T, et al.
Improving neonatal outcome through practical shoulder dystocia
training. Obstet Gynecol 2008;112:14–20.
32 Crofts JF, Lenguerrand E, Bentham GL, Tawfik S, Claireaux HA, Odd D,
et al. Prevention of brachial plexus injury-12 years of shoulder dystocia
training: an interrupted time-series study. BJOG 2016;123:111–18.
33 Egenberg S, Oian P, Bru LE, Sautter M, Kristoffersen G, Eggebo TM.
Can inter-professional simulation training influence the frequency of
blood transfusions after birth? Acta Obstet Gynecol Scand
2015;94:316–23.
34 Markova V, Sorensen JL, Holm C, Norgaard A, Langhoff-Roos J.
Evaluation of multi-professional obstetric skills training for postpartum
hemorrhage. Acta Obstet Gynecol Scand 2012;91:346–52.
35 Committee on Fetus and Newborn. American Academy of
Pediatrics, and Committee on Obstetric Practice, American College
of Obstetricians and Gynecologists. Use and abuse of the Apgar
score. Pediatrics 1996;98:141–2.
649
 Fransen et al.
36 Casey BM, McIntire DD, Leveno KJ. The continuing value of the
Apgar score for the assessment of newborn infants. N Engl J Med
2001;344:467–71.
37 Cook DA, Hamstra SJ, Brydges R, Zendejas B, Szostek JH, Wang AT,
et al. Comparative effectiveness of instructional design features in
simulation-based education: Systematic review and meta-analysis.
Med Teach 2013;35:e867–98.
38 Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ.
Features and uses of high-fidelity medical simulations that lead to effective
learning: a BEME systematic review. Med Teach 2005;27:10–28.
39 Bosse HM, Mohr J, Buss B, Krautter M, Weyrich P, Herzog W, et al.
The benefit of repetitive skills training and frequency of expert
feedback in the early acquisition of procedural skills. BMC Med Educ
2015;15:22.
40 McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical
review of simulation-based medical education research: 2003–2009.
Med Educ 2010;44:50–63.
41 Siassakos D, Crofts JF, Winter C, Weiner CP, Draycott TJ. The active
components of effective training in obstetric emergencies. BJOG
2009;116:1028–32.
42 Sørensen JL, van der Vleuten C, Rosthoj S, Østergaard D, LeBlanc V,
Johansen M, et al. Simulation-based multiprofessional obstetric
anaesthesia training conducted in situ versus off-site leads to similar
individual and team outcomes: a randomised educational trial. BMJ
Open 2015;5:e008344.
43 Dijkman A, Huisman CM, Smit M, Schutte JM, Zwart JJ, van
Roosmalen JJ, et al. Cardiac arrest in pregnancy: Increasing use of
perimortem caesarean section due to emergency skills training?
BJOG 2010;117:282–7.
650
44 Woiski MD, Hermens RP, Middeldorp JM, Kremer JA, Marcus MA,
Wouters MG, et al. Haemorrhagia post partum; an implementation
study on the evidence-based guideline of the Dutch society of
Obstetrics and Gynaecology (NVOG) and the MOET (Managing
Obstetric Emergencies and Trauma-course) instructions; the Fluxim
study. BMC Pregnancy Childbirth 2010;10:5.
45 O’Dea A, O’Connor P, Keogh I. A meta-analysis of the effectiveness
of crew resource management training in acute care domains.
Postgrad Med 2014;90:699–708.
46 Crofts JF, Bartlett C, Ellis D, Winter C, Donald F, Hunt LP, et al.
Patient-actor perception of care: a comparison of obstetric
emergency training using manikins and patient-actors. Qual Saf
Health Care 2008;17:20–4.
47 Perined. Perinatale zorg in Nederland 2014. Utrecht: Perined, 2015.
[https://assets.perined.nl/docs/353d9249-9875-4cb3-9c86-f078ae3f7aef.
pdf]. Accessed 27 July 2016.
48 Brown CA, Lilford RJ. The stepped wedge trial design: a systematic
review. BMC Med Res Methodol 2006;6:54.
49 Woertman W, de Hoop E, Moerbeek M, Zuidema SU, Gerritsen DL,
Teerenstra S. Stepped wedge designs could reduce the required sample
size in cluster randomized trials. J Clin Epidemiol 2013;66:752–8.
50 Mdege ND, Man MS, Taylor Nee Brown CA, Torgerson DJ.
Systematic review of stepped wedge cluster randomized
trials shows that design is particularly used to evaluate
interventions during routine implementation. J Clin Epidemiol
2011;64:936–48.
51 Zorginzicht. [www.zorginzicht.nl/bibliotheek/integrale-geboortezorg-
zorgstandaard/Paginas/Home.aspx]. Accessed 27 July 2016.
ª 2016 Royal College of Obstetricians and Gynaecologists