879273 HFSXXX10.

1177/0018720819879273Human FactorsTrust Mediates Reliability–Reliance Relationshipresearch-article2019

Trust Mediating Reliability–Reliance Relationship

in Supervisory Control of Human–Swarm
Interactions
Aya Hussein , Sondoss Elsawah, and Hussein A. Abbass, University of
New South Wales, Canberra, Australian Capital Territory, Australia

Objective: This work aims to further test the theory that Introduction
trust mediates the interdependency between automation reli-
ability and the rate of human reliance on automation. Human trust in automation has gained sig-
Background: Human trust in automation has been the nificant interest as a crucial human factor that
focus of many research studies. Theoretically, trust has been can influence human–automation interaction
proposed to impact human reliance on automation by medi- within a mission, and ultimately mission per-
ating the relationship between automation reliability and the
rate of human reliance. Experimentally, however, the results
formance (J. Y. Chen & Barnes, 2014). Theo-
are contradicting as some confirm the mediating role of trust, retically, it has been widely accepted that trust
whereas others deny it. Hence, it is important to experimen- plays a significant mediation role in the rela-
tally reinvestigate this role of trust and understand how the tionship between automation reliability and
results should be interpreted in the light of existing theory. the rate of human reliance on it (J. Gao & Lee,
Method: Thirty-two subjects supervised a swarm of
unmanned aerial vehicles (UAVs) in foraging missions in which
2006; Lee & See, 2004; Parasuraman & Riley,
the swarm provided recommendations on whether or not to 1997). However, this role has not yet been
collect potential targets, based on the information sensed by empirically confirmed. Only a few research
the UAVs. By manipulating the reliability of the recommen- endeavors (Bustamante, 2009; Chancey, Bliss,
dations, we observed changes in participants’ trust and their Proaps, & Madhavan, 2015; Chancey, Proaps, &
behavioral responses.
Results: A within-subject mediation analysis revealed a
Bliss, 2013; Merritt, 2011; Wiczorek & Manzey,
significant mediation role of trust in the relationship between 2010) were dedicated to the experimental study
swarm reliability and reliance rate. High swarm reliability of the mediation role played by trust. Moreover,
increased the rate of correct acceptances, but decreased the the results of some of these empirical studies
rate of correct rejections. No significant effect of reliability were contradicting to the corresponding well-
was found on response time.
Conclusion: Trust is not a mere by-product of the inter-
established theory on the topic as well as among
action; it possesses a predictive power to estimate the level of each other.
reliance on automation. The objective of this work is to take a step
Application: The mediation role of trust confirms the toward filling this gap. We investigate the related
significance of trust calibration in determining the appropriate literature and contrast experimental findings to
level of reliance on swarm automation.
the theory on the role of trust. Then, we present
Keywords: human–automation interaction, human–swarm
our human experiments, the purpose of which is
interaction, design of experiments, foraging, within-subject to reexamine the hypothesized mediation role of
mediation analysis, recommender systems trust.

Automation Reliability and

Address correspondence to Aya Hussein, School of Trust In Automation
Engineering and Information Technology, University of
New South Wales, Canberra, Australian Capital Territory Lee and See (2004) define human trust in
2610, Australia; e-mail: a.hussein@student.adfa.edu.au. an agent as “the attitude that an agent will help
achieve an individual’s goals in a situation
HUMAN FACTORS
Vol. XX, No. X, Month XXXX, pp. 1­–12 characterized by uncertainty and vulnerability.”
DOI: 10.1177/0018720819879273 Many studies (M. Chen, Nikolaidis, Soh, Hsu,
Article reuse guidelines: sagepub.com/journals-permissions & Srinivasa, 2018; F. Gao, Clare, Macbeth, &
Copyright © 2019, Human Factors and Ergonomics Society. Cummings, 2013; Khasawneh, Bowling, Jiang,
2 Month XXXX - Human Factors
Gramopadhye, & Melloy, 2003; Xu & Dudek,
2015) found that trust is significantly affected
by automation reliability, in terms of the prob-
ability that automation performs its assigned
tasks correctly. Trust is also of a dynamic nature,
as it can drop notably after automation errors
(Akash, Hu, Reid, & Jain, 2017; Desai, Kania-
rasu, Medvedev, Steinfeld, & Yanco, 2013; Xu
& Dudek, 2015). Hence, an accurate estimation
of trust requires repeated measurements during
an interaction (Schaefer, 2016).
The effect of error type (i.e., false alarms vs.
misses) on trust has been investigated in some
studies (Chancey et al., 2015; Merritt, Lee,
Unnerstall, & Huber, 2015) which did not find
this effect to be significant. This result can be
explained using the model in J. Gao and Lee
(2006) in which automation characteristics
affect trust through the perception of perfor-
mance. Therefore, unless automation character-
istics are manipulated to affect performance,
they may not affect trust. Thus, unless one type
of errors is significantly more detrimental to the
performance than the other, trust may not be
impacted by error type.
Trust and Reliance On
Automation
Although trust is a personal attitude toward
an agent, reliance is the actual delegation of a
task to the agent. Lee and See (2004) suggested
that trust in automation is a main factor that
affects human intention to rely on that automa-
tion. Other factors may include workload, self-
confidence, and task-associated risks. In turn,
the intention to rely is a main factor, but not
the sole one, that predicts the reliance behavior,
which is also affected by other factors such as
time constraints.
This distinction between trust and reliance
can lead to situations in which a human trusts
automation but does not rely on it due, for exam-
ple, to some inherent risks. Conversely, the
human may rely on an agent, albeit not highly
trusting it, due to other factors such as experi-
encing a very high workload or the existence of
time constraints. In other words, trust does not
necessarily manifest itself into reliance and reli-
ance is not necessarily a piece of evidence of
trust (Miller et al., 2016). Thus, operationalizing
trust in terms of the reliance behavior confounds
other constructs with trust (Chancey et al.,
2015).
Both overtrust and distrust are believed to be
detrimental to mission performance (Abbass,
Petraki, Merrick, Harvey, & Barlow, 2016; Bou-
bin, Rusnock, & Bindewald, 2017; J. Y. Chen &
Barnes, 2014). According to many researchers
(e.g., Lee & See, 2004; Yang, Unhelkar, Li, &
Shah, 2017), human’s overtrust toward an agent
can result in overrelying on it despite its limita-
tions, which may lead to catastrophic conse-
quences. For instance, Robinette, Li, Allen,
Howard, and Wagner (2016) found that in emer-
gency evacuation scenarios, subjects opted to
follow robot instructions to the exit even when
the robot provided circuitous routes. They attrib-
uted this behavior to subjects’ overtrust in the
robot. On the contrary, distrust has been consid-
ered to lead to automation disuse, hence missing
its potential benefits (Lee & See, 2004). Thus,
maintaining an appropriate level of trust is
deemed important for mission success and opti-
mized performance. Although this stance is
prevalent among human factors researchers, it is
not well backed up by experimental evidence as
mixed results on the mediation role of trust have
been reported in the literature.
The Mediation Role of Trust
The conceptual diagram for a general media-
tion model is depicted in Figure 1. A variable M
is said to mediate the relationship between an
independent variable X and a dependent variable
Y if two conditions are met (Hayes, 2017): (1) X
affects M, which in turn affects Y. For a variable
to affect another variable, it needs to not only
correlate with it but also temporally precede it.
(2) The indirect effect of X on Y through M is
statistically significant from zero.
Trust has been frequently hypothesized to
mediate the relationship between automation
reliability and human reliance on automation (J.
Gao & Lee, 2006; Khasawneh et al., 2003; Lee
& See, 2004; Parasuraman & Riley, 1997).
Nonetheless, experimental results disagree with
each other regarding the mediating role of trust.
Bustamante (2009) examined the mediation
role of trust in the effect of alarm system reli-
ability on human reliance strategy. In that work,
Trust Mediates Reliability–Reliance Relationship 3
Figure 1. A conceptual diagram for the effect of X
on Y. The upper part shows the total effect of X on
Y while M is excluded. The lower part shows the
mediation model.
participants performed primary flight tasks in
which they had to maintain level flight and to
make sure they have adequate fuels. The alarm
system assisted participants with a secondary
monitoring task. Bustamante found that trust
was a by-product of the interaction rather than a
mediator of the reliability–reliance relationship.
However, this result needs to be interpreted
with caution due to two reasons. First, reliabil-
ity was manipulated by changing error type
from false prone to miss prone, yet whether this
change affects performance is questionable. In
that experiment, both the main and secondary
tasks were critical, which means that both false
alarms and misses could be equally detrimental
to the performance, as the former would con-
tinuously interrupt the execution of the main
tasks, whereas the latter would result in missing
safety-critical events. Second, the secondary
tasks were systematically different across the
two experimental conditions, leading to chang-
ing two variables (error type and secondary
task) simultaneously without adjusting for the
potential effects of the confounding variable
(the type of the secondary task), which makes it
difficult to isolate the effect of the main inde-
pendent variable (error type), or lack thereof
(MacKenzie, 2012).
Another endeavor is reported by Wiczorek
and Manzey (2010) who investigated the medi-
ation role of trust in the relationship between
perceived reliability and reliance. They manip-
ulated the reliability of alarm systems across
different experimental conditions and observed
subjects’ trust, perceived reliability, and reli-
ance. They found that reliance was mainly
dependent on perceived reliability but trust was
not a significant mediator in this relationship.
However, these results could have been affected
by their data analysis, as Wiczorek and Manzey
(2010) performed a mediation analysis on the
association between perceived reliability (re-
ported by subjects) and reliance, with trust as a
potential mediator. Nonetheless, they did not
include the actual automation reliability (the
independent variable) in the analysis. Exclud-
ing such an important confounding variable
from the analysis is a serious issue that can have
considerable effects on the results as it can lead
to incorrectly rejecting a true mediation (Hert-
ing, 2002; Li, Bienias, & Bennett, 2007).
Merritt (2011) studied the mediating roles of
trust and automation liking in the relationship
between perceived automation reliability and
reliance, in X-ray screening tasks with two reli-
ability levels: 80% and 90%. Data were col-
lected from participants at five points of time
and were used for fitting five separate statistical
models, one for each point of time. Their results
indicated that the effect of perceived reliability
on reliance was fully mediated by automation
liking in the earlier points of time and by trust in
the later ones.
Chancey et al. (2013) also investigated the
indirect effect of reliability on reliance through
trust. Subjects were loaded with a word search
task and were also provided with an alarm sys-
tem to detect fires. Alarm reliability varied
across the two experimental conditions (20%
and 40%). Subjects’ reliance behavior was mea-
sured using both agreement rate and response
time. Chancey et al. (2013) found that trust did
not mediate the impact of reliability on either
indicators of reliance. Besides, no significant
difference in response time was found among
different reliability conditions. Nevertheless, a
closer look at their experimental design reveals
some factors that could have contributed to
these findings. Subjects were always provided
with the raw sensor data that were used by the
alarm system to generate its recommendation.
4 Month XXXX - Human Factors
Therefore, with the data collection method
used, it was impossible to distinguish between
situations in which subjects indeed relied on the
alarm and accepted its recommendations and
situations in which subjects used the raw data to
decide on the existence of fire themselves with
the results of their processing coinciding with
alarm recommendations. In fact, we suspect
that subjects could have tended to process the
raw data themselves due to the very low reli-
ability levels used in the experiment. In their
literature review, Wickens and Dixon (2007)
showed that a reliability level of 70% is the
crossover point below which automation fails to
benefit the performance. However, Wickens
and Dixon (2007) suggested that subjects may
rely on automation with reliability less than
70% if this is important to maintain the perfor-
mance of the main task. As the reliability levels
(20% and 40%) used by Chancey et al. (2013)
were considerably lower than this crossover
point, we believe subjects might have aban-
doned the alarm system, particularly because of
the importance of fire detection compared with
the word search task.
Chancey et al. (2015) conducted an experi-
ment in which subjects performed primary flight
tasks similar to the ones used in Bustamante
(2009). Besides, a signaling system with reli-
ability levels of 60% and 90% was used to give
the participants advisories about the state of five
gauges. Participants could accept advisories or
request to verify them by checking the gauges
themselves. They found that trust was a signifi-
cant mediator of the reliability–reliance relation-
ship. Reliability had a significant direct effect on
response time, but no significant indirect effect
through trust. Manipulating error type, however,
did not have a significant effect on trust, which
could be due to the same reason we mentioned
for the work in Bustamante (2009). Table 1 pro-
vides a summary of these studies.
Design of Experiments Guidelines
Deficiencies with the experimental design,
data collection, or data analysis could lead to
incorrect inferences. In the light of previous
studies on trust and related literature, we pro-
vide some guidelines that could be helpful to
mitigate errors while examining the role of trust
in human–automation interaction. Experimental
studies investigating the mediation role of trust
should consider the following:
1. The underlying principle in the literature is
that humans rely on automation if there are
considerable benefits of using it. These ben-
efits can be saving a limited resource (e.g.,
time or effort) or maximizing mission objec-
tives. With no such benefits, humans may opt
to abandon automation. Thus, there is no point
in providing humans with an automation that
has no significant benefits to the overall per-
formance.
2. J. Gao and Lee (2006) and Lee and See (2004)
suggest that trust plays a mediation role in the
relationship between automation characteris-
tics that affect task performance and human
reliance on automation. Hence, manipulating
automation characteristics such that it benefits
the performance in a way but equally hurts it
in another is another issue that could have dif-
ferent implications on trust and reliance.
3. The design of the experiment should exclude
confounding variables that change systemati-
cally across experimental conditions (Herting,
2002; Li et al., 2007; MacKenzie, 2012). If a
confounding variable is included, it must be
adjusted for during the data analysis.
4. The data fed into the mediation analysis have to
comply with the temporal precedence require-
ment in establishing the causal effect of trust
on reliance (Bustamante, 2009). That is, only
reliance behavior that follows trust measure-
ment should be considered for the analysis.
5. The dynamic nature of trust should be taken
into consideration. Thus, trust should be
repeatedly measured to get an updated and a
more accurate value of it.
Method
We designed a swarm-based human experi-
ment based on the guidelines described earlier
to avoid the identified weaknesses of previ-
ous studies. A robot swarm is a special type
of robotic systems in which a large number
of robots with relatively limited capabilities
communicate with each other using local inter-
actions which can be utilized to achieve a
wide range of global complex behaviors, such
Trust Mediates Reliability–Reliance Relationship 5

Table 1: Summary of the Experimental Studies on the Mediation Role of Trust

Study Details Guidelines

Study Relation Significant 1 2 3 4 5

Bustamante (2009) Error type → trust → No ✓ X X ✓ X

reliance rate
Wiczorek and Perceived reliability → No ✓ ✓ X ✓ ✓
Manzey (2010) trust → reliance rate
Merritt (2011) Perceived reliability → Yes ✓ ✓ X ✓ ✓
trust → reliance rate
Chancey, Proaps, Automation reliability → No X ✓ ✓ X X
and Bliss (2013) trust → reliance rate
Chancey, Bliss, Error type → trust → No ✓ X ✓ ✓ X
Proaps, and reliance rate
Madhavan (2015)
Chancey et al. Automation reliability → Yes ✓ ✓ ✓ ✓ X
(2015) trust → reliance rate

as flocking, dispersion, and foraging. Robot
swarms have been gaining increasing interest
due to the usefulness of their behaviors in a
variety of applications ranging from agriculture
to space exploration.
Many research studies have used simulated
robot swarms for foraging (e.g., Nunnally et al.,
2012; Pendleton and Goodrich, 2013; Tang
et al., 2017) in which robots are deployed in an
environment to collect target objects. Foraging
is a benchmark problem in the swarm robotics
literature as it resembles many real-world mis-
sions including search and rescue, cleaning, har-
vesting, and land mine clearance (Winfield,
2009).
This research complied with the American
Psychological Association Code of Ethics and
was approved by the Human Research Ethics
Committee (protocol no. HC180308). Informed
consent was obtained from each participant.
Scenario
A swarm of 20 unmanned aerial vehicles
(UAVs) is deployed in an environment contain-
ing two types of initially hidden objects: targets
and nontargets. The swarm autonomously navi-
gates the environment in a predefined path (see
Figure 2a). During its navigation, a UAV can
sense an object within its sensing range (see
Figure 2b), with a probability associated with
that object. Although the UAVs are aimed to
sense only target objects, confusion with non-
target objects is also possible according to some
probability values associated with these nontar-
gets. That is, an object with a detection prob-
ability p is expected to be sensed by an average
of p UAVs that have a sensing range greater than
their distance to that object (nearby UAVs). The
numbers of target and nontarget objects in the
environment are equal. Once an object is sensed
by one or more UAVs, its position is visualized
and the whole swarm starts heading toward it.
Upon arrival, the swarm recommends how the
object should be handled (collected/discarded)
based on the number of UAVs that are able to
sense it (see Figure 2c). If the number of UAVs
sensing the object is at least 10, the swarm
recommends collecting the object, otherwise
it recommends discarding it. The human can
accept a recommendation or request to verify it.
Accepting a recommendation leads the swarm
to execute the recommended action. On the con-
trary, verification is implemented by retrieving
the object’s image so that the human can iden-
tify its type himself or herself (see Figure 3). To
simulate the object identification task, images
containing rectangles are used, where targets
have an even number of rectangles, whereas
nontargets have an odd number of rectangles.
Object collection is a time-consuming opera-
tion that takes 5 s following which the gained
score is visualized (see Figure 2d). Collecting
6 Month XXXX - Human Factors

Figure 2. Scenario description: (a) swarm navigation through the environment starting from the
yellow circle onward; (b) a swarm agent detecting an object within its sensing range; (c) the swarm
arrives at the detected object and recommends collecting it; (d) the gained score after the swarm
finished collecting a target; (e) an indication of missed target after the swarm discarded a target
object.

targets increases the score, whereas collecting
nontargets does not change it. The role of the
human is to make the decisions that maximize
the score of the mission by collecting as many
targets as possible. The time constraint on the
mission implies that collecting nontargets
reduces the time available for finding and col-
lecting targets. Frequent human verification can
also waste the available time as image retrieval
and object identification are time-consuming.
Image retrieval takes 1 s, whereas object identifi-
cation depends on the recognition and judgment
speed of the human. The task is designed such
that proper reliance strategies, with respect to the
level of reliability, yield higher performance.
Two levels of recommendation reliability
(90% and 70%) have been used. These levels
were chosen to be at least 70% (Wickens &
Dixon, 2007) to ensure that automation is ben-
eficial to the performance (Guideline 1). The
gap between the two reliability conditions was
selected to make a notable difference to the
performance (Guideline 2). The two types of
erroneous recommendations (collect nontar-
gets and discard targets) are equally probable
in each reliability condition to avoid the con-
founding effect of error bias (Guideline 3).
Experimental Setup
A within-subject design is employed, in
which each subject supervised the swarm in the
two reliability conditions. The within-subject
design is chosen as it is beneficial to account-
ing for individual differences that can affect
subjects’ behavior (Guideline 3). The order of
the missions was counterbalanced to correct for
ordering effects (Guideline 3). Thirty-two sub-
jects participated in the experiment: 17 females
and 15 males. Their average reported age is
28.13 years (SD = 5.34).
Subjects first read and agreed to the consent-
to-participate form, and then they filled in a short
questionnaire. They were introduced to the task,
given its instructions, and allowed to ask ques-
tions before starting the task. Subjects were noti-
fied that the accuracy of the recommendations
Trust Mediates Reliability–Reliance Relationship 7
Figure 3. A screenshot of the simulation showing a
human verification of the type of an object.
changes across missions, but were not told the
exact accuracy. All subjects were given 2-min
warm-up periods to get used to the simulation
environment. Although no subject complained or
found it hard to interact with the system, we
recorded and compared subjects’ response time
in these 2 min against the rest of the simulation
time (see Figure 4). Given the simplicity of the
environment, the subjects managed to drive the
simulation professionally from the start.
A short trust questionnaire, adapted from
Jian, Bisantz, and Drury (2000), was prompted
every 2 min (Guideline 5) in which subjects
rated their current level of trust in swarm recom-
mendations using a 10-point Likert-type scale.
Subjects were asked to take as much time as
they needed to answer the questions accurately.
The simulation was paused so that the question-
naire does not consume the task time.
Data Analysis and Results
Scenario time has been divided into five
intervals, with the length of each being 2 min.
Four rows of data of the form (reliability, trust,
reliance rate, response time) were extracted
from each scenario, as shown in Figure 5,
such that reliability is the actual recommenda-
tion reliability experienced by a subject in the
interval prior to the questionnaire, trust is the
level of trust reported by the subject in the
questionnaire, reliance rate is the subject’s rate
of accepting recommendations in the interval
following the questionnaire, and response time
is the average time the subject took to react
to a recommendation in the interval following
the questionnaire. Data are constructed in this
way to comply with the temporal precedence
requirement for the mediation analysis (Guide-
line 4). Overall, 256 rows of data (eight repeated
measures for each participant) have been used
for the statistical analyses.
Hierarchical linear modeling has been used to
account for the correlation among data collected
from the same subject due to repeated measure-
ments. Data have been standardized before per-
forming the regression analyses so that beta
coefficients can be calculated. For each depen-
dent variable, we fitted a variance component
model (null model) and one or more random
intercept models; all models contain subject as a
random effect, whereas only random intercept
models contain recommendation reliability (and
possibly trust) as fixed within-subject effects.
We calculated Akaike information criterion
(AIC) to assess the relative quality of candidate
models in terms of their likelihood of correctly
estimating future values, such that lower AIC
values reflect higher model quality. The mar-
ginal and conditional coefficients of determina-
tion Rm2 and Rc2 are calculated to give estimates
of the variance explained by the fixed effect and
by the entire model, respectively. The results of
the statistical analyses are summarized in Tables
2 and 3 and Figures 6 and 7.
Cronbach’s alpha for the trust scale ranged
between α = .7 and α = .81, and the intraclass
correlation coefficient (ICC) for trust measure-
ments is .37. The estimates of the within- and
between-subject variance in the variance com-
ponent model M0 are 0.63 and 0.37, respectively.
In the random intercept model for trust M1, reli-
ability (within-subject factor) is added to the
model, leading to a reduced within-subject vari-
ance of 0.47. The regression of trust on reliabil-
ity is statistically significant with βa = 0.39 and
95% confidence interval (CI) = [0.31, 0.47]. The
marginal and conditional variance explained of
M1 are Rm2 = .16 and Rc2 = .51 , respectively.
AIC for M1 (600.6) is significantly lower than
AIC for M0 (666.6), p < .001.
ICC for the reliance rate is .32. Three models
have been fitted for the reliance rate: the vari-
ance component model and two random inter-
cept models. In the variance component model
8 Month XXXX - Human Factors
Figure 4. Participants’ response time during the experiments. The dashed line indicates the end of the
2-min warm-up period.
Figure 5. The structure of the data used for the analysis. The number inside the parentheses indicates the
row number to which the observation belongs.
M0, the estimates of the within- and between-
subject variance are 0.68 and 0.32, respectively.
This model will be used as a baseline in com-
parison to the models containing predictors, the
random intercept models. Significant differ-
ences between the models can be indicated by
their AIC values.
In the first random intercept model for reli-
ance rate M1, only reliability is incorporated
within the model, leading to a reduced within-
subject variance of 0.51. The decrease in the
within-subject variance reflects the model’s
ability to explain a portion of the variation in
subject’s reliance rate, which is not accounted
for by M0. The regression of reliance rate on reli-
ability is statistically significant with βc = 0.4,
95% CI = [0.31, 0.49], Rm2 = .16 , and Rc2 = .5 .
In the second random intercept model for reli-
ance rate M2, both reliability and trust are incor-
porated within the model, leading to a reduced
within-subject variance of 0.49. The regression
of reliance rate on both reliability (βc1 = 0.34,
95% CI = [0.24, 0.45]) and trust (βb = 0.17, 95%
CI = [0.02, 0.33]) is statistically significant. The
marginal and conditional variance explained of
M2 are Rm2 = .18 and Rc2 = .52 , respectively. The
AIC values for both M1 (615.7) and M2 (612.7)
are significantly lower than that for M0 (681), p <
.001. Moreover, the difference between the AIC
values for M1 and M2 is statistically significant
(p = .03).
ICC for the response time is .6. Three mod-
els have also been fitted for response time. In
the variance component model M0, the esti-
mates of the within- and between-subject vari-
ance are 0.39 and 0.6, respectively. The first
random intercept model for response time M1
incorporates the effect of reliability; however,
this does not affect the within-subject variance.
The regression of response time on reliability is
not statistically significant from zero (β =
−0.02, 95% CI = [−0.1, 0.05]). Furthermore,
adding trust in the second random intercept
model for response time M2 does not affect the
Trust Mediates Reliability–Reliance Relationship 9

Table 2: Descriptive Statistics and the Correlations Between Study Variables

M SD Minimum Maximum 1 2 3 4

1. Actual reliability 0.83 0.14 0.38 1 –  

2. Trust 6.14 1.97 1 10 .43** –  
3. Reliance rate 0.67 0.29 0 1 .38** .29** –  
4. Response time 1.52 0.78 0.3 5.6 –.06 –.12** –.26** –

**p < 0.01.

Table 3: Results of Statistical Analyses

Between-
Response Within-Subject Subject
Rm2 Rc2
Variable Model β Variance Variance

Trust M0 — 0.63 0.37 — .37

  M1 0.39 [0.31, 0.47] 0.47 0.34 .16 .51
Reliance rate M0 — 0.68 0.32 — .32
  M1 0.4 [0.31, 0.49] 0.51 0.35 .16 .5
  M2 0.34 [0.24, 0.45] 0.49 0.35 .18 .52
  0.17 [0.02, 0.33]  
Response time M0 — 0.39 0.6 — .6
  M1 –0.02 [–0.1, 0.05] 0.39 0.6 ≈0 .6
  M2 0.004 [–0.1, 0.11] 0.39 0.59 .01 .6
  –0.07 [–0.21, 0.08]  

Note. Two beta coefficients are listed for model M2, where the first is for reliability and the second is for trust.

The average rates of correct acceptance and

Figure 6. Results of the mediation model.
within-subject variance. The beta coefficients
of both reliability (β = 0.004, 95% CI = [−0.1,
0.11]) and trust (β = −0.07, 95% CI = [–0.21,
0.08]) in M2 are not statistically significant. The
difference in AIC between each pair of models
M0 (AIC = 573.8), M1 (AIC = 575.5), and M2
(AIC = 575.9) is not statistically significant.
A repeated-measures version of the media-
tion analysis described in Hayes (2017) has been
used, with bootstrapped CIs. The indirect effect
of recommendation reliability on reliance rate
through trust is shown to be statistically signifi-
cant (βaβb = 0.07, 95% CI = [0.007, 0.141]).
correct rejection are calculated for each scenario
rather than for each interval, as many intervals
did not include incorrect recommendations.
Paired-samples t tests were conducted to com-
pare the average rates of correct acceptance and
correct rejection in the two conditions. There
was a significant difference in the correct accep-
tance rate among the low-reliability (M = 0.58,
SD = 0.22) and high-reliability (M = 0.79, SD =
0.18) conditions, t(31) = −5.91, p < .001.
Similarly, a significant difference was found in
the rate of correct rejection between the low-
reliability (M = 0.42, SD = 0.23) and high-
reliability (M = 0.22, SD = 0.27) conditions,
t(31) = 3.8, p < .001. The overall performance is
compared in the two conditions, in terms of the
number of targets collected. A paired-samples t
test showed that the difference between the
number of targets collected in the low-reliability
(M = 28.35, SD = 6.33) and the high-reliability
(M = 34.88, SD = 7.13) conditions is statistically
significant, t(31) = −6.8, p < .001.
10 Month XXXX - Human Factors
Figure 7. The average values of subject’s trust and performance in the two scenarios.
Discussion and Conclusion
The aim of this study was to reexamine
the mediation role of trust in the association
between automation reliability and human reli-
ance on automation. However, a true media-
tion role can easily be incorrectly rejected due
to issues with the experimental design or the
statistical analysis employed (Herting, 2002;
Li et al., 2007). Thus, it was important to have
a closer look at previous work examining the
mediation role of trust to identify issues that
could have potentially contributed to errone-
ously dismissing a potential mediation role. By
avoiding such issues, it becomes more probable
to correctly detect or reject the hypothesized
mediation.
The results of our study were able to provide
empirical support to the mediation role of trust
in the relationship between automation reliabil-
ity and the rate of human reliance on automa-
tion. Based on the mediation model fitted to the
experimental data, the indirect effect of reliabil-
ity on the reliance rate through trust accounted
for about 20% of the reliance rate exhibited by
the subjects. These results stress the practical
significance of trust calibration when determin-
ing human reliance on automation. Thus, design-
ing trust-aware human–automation interaction
is expected to benefit mission performance by
regulating the reliance rate.
In our experiment, response time was not a
useful indicator for a subject’s reliance behavior
as indicated by the insignificant change in
response to either reliability or trust. The lack of
these effects could be because subjects were not
loaded with an additional auxiliary task that
would delay their attendance to the recommen-
dations. The estimated between-subject variance
for response time is high (0.6), indicating that
subjects differ widely in their average response
time, possibly due to other individual factors
external to the models.
Increasing the level of reliability resulted in a
significant increase in the rate of correct accep-
tance, which is a desirable effect as it leads to
speeding up task performance. Nonetheless,
this effect was accompanied by a significant
decrease in the rate of correct rejection, which
reflects a decline in human ability to manage
erroneous swarm recommendations. On aver-
age, the performance benefited significantly
from the increase in swarm reliability, as incor-
rect recommendations were much less frequent
than the correct ones. The pattern of subjects’
behavior detected in our study reflects increased
automation-induced complacency (Parasura-
man, Molloy, & Singh, 1993) and suggests that
increased reliability alone is insufficient for
proper use. Human complacent behavior toward
highly reliable automation has been reported in
Trust Mediates Reliability–Reliance Relationship 11
many studies (for instance, Rovira, McGarry, &
Parasuraman, 2007).
The limitation of this study is that the role of
trust is considered in isolation to other factors
that can affect trust and reliance. For instance,
individual differences (e.g., propensity to trust)
could affect the strength of the mediation role of
trust. Automation transparency could also
impact the strength of this role due to its positive
impact on trust (Dorneich et al., 2017; Wang,
Pynadath, & Hill, 2016). Incorporating such fac-
tors while studying the role of trust can help
identify contexts where trust, and thus trust cali-
bration, is of particular importance to the proper
reliance.
Acknowledgment
This work was funded by the Australian Research
Council’s Discovery Grant (no. DP160102037) and
University of New South Wales, Canberra.
Key Points
•• Human factors literature usually considers trust
to be the main mediator between automation reli-
ability and human reliance on automation. This
hypothesized role was not empirically confirmed
by past studies due to their conflicting findings.
•• By scrutinizing the empirical studies on the medi-
ation role of trust, we assessed that their findings
could have been affected by experimental issues
leading to this inconsistency. We designed a few
guidelines that need to be followed by studies on
the mediation role of trust.
•• By adhering to the guidelines in our human–
swarm interaction experiment, our empirical
findings support the theoretical position that trust
plays a significant mediation role in the relation-
ship between automation reliability and human
reliance on it.
ORCID iD
Aya Hussein https://orcid.org/0000-0003-
3204-0712
References
Abbass, H. A., Petraki, E., Merrick, K., Harvey, J., & Barlow,
M. (2016). Trusted autonomy and cognitive cyber symbio-
sis: Open challenges. Cognitive Computation, 8, 385–408.
doi:10.1007/s12559-015-9365-5
Akash, K., Hu, W.-L., Reid, T., & Jain, N. (2017). Dynamic model-
ing of trust in human-machine interactions. In 2017 American
Control Conference (ACC) (pp. 1542–1548). doi:10.23919/
acc.2017.7963172
Boubin, J. G., Rusnock, C. F., & Bindewald, J. M. (2017). Quanti-
fying compliance and reliance trust behaviors to influence trust
in human-automation teams. Proceedings of the Human Fac-
tors and Ergonomics Society Annual Meeting, 61, 750–754.
doi:10.1177/1541931213601672
Bustamante, E. A. (2009). A reexamination of the mediating effect
of trust among alarm systems’ characteristics and human
compliance and reliance. Proceedings of the Human Fac-
tors and Ergonomics Society Annual Meeting, 53, 249–253.
doi:10.1177/154193120905300419
Chancey, E. T., Bliss, J. P., Proaps, A. B., & Madhavan, P. (2015).
The role of trust as a mediator between system characteris-
tics and response behaviors. Human Factors, 57, 947–958.
doi:10.1177/0018720815582261
Chancey, E. T., Proaps, A., & Bliss, J. P. (2013). The role of
trust as a mediator between signaling system reliability
and response behaviors. Proceedings of the Human Fac-
tors and Ergonomics Society Annual Meeting, 57, 285–289.
doi:10.1177/1541931213571063
Chen, J. Y., & Barnes, M. J. (2014). Human–agent teaming for mul-
tirobot control: A review of human factors issues. IEEE Trans-
actions on Human-Machine Systems, 44, 13–29. doi:10.1109/
thms.2013.2293535
Chen, M., Nikolaidis, S., Soh, H., Hsu, D., & Srinivasa, S. (2018).
Planning with trust for human-robot collaboration (arXivpre-
print). Retrieved from https://arxiv.org/abs/1801.04099
Desai, M., Kaniarasu, P., Medvedev, M., Steinfeld, A., & Yanco,
H. (2013). Impact of robot failures and feedback on real-time
trust. In Proceedings of the 8th ACM/IEEE International
Conference on Human-Robot Interaction (pp. 251–258).
doi:10.1109/hri.2013.6483596.
Dorneich, M. C., Dudley, R., Letsu-Dake, E., Rogers, W., Whitlow,
S. D., Dillard, M. C., & Nelson, E. (2017). Interaction of auto-
mation visibility and information quality in flight deck infor-
mation automation. IEEE Transactions on Human-Machine
Systems, 47, 915–926. doi:10.1109/thms.2017.2717939
Gao, F., Clare, A. S., Macbeth, J. C., & Cummings, M. (2013).
Modeling the impact of operator trust on performance in mul-
tiple robot control. In 2013 AAAI Spring Symposium Series.
Retrieved from https://www.aaai.org/ocs/index.php/SSS/
SSS13/paper/view/5741
Gao, J., & Lee, J. D. (2006). Extending the decision field theory to
model operators’ reliance on automation in supervisory control
situations. IEEE Transactions on Systems, Man, and Cybernet-
ics, Part A: Systems and Humans, 36, 943–959. doi:10.1109/
tsmca.2005.855783
Hayes, A. F. (2017). Introduction to mediation, moderation, and
conditional process analysis: A regression-based approach
(2nd ed.). New York, NY: Guilford Publications.
Herting, J. R. (2002). Evaluating and rejecting true mediation mod-
els: A cautionary note. Prevention Science, 3, 285–289.
Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000). Foundations for
an empirically determined scale of trust in automated systems.
International Journal of Cognitive Ergonomics, 4, 53–71.
doi:10.21236/ada388787
Khasawneh, M. T., Bowling, S. R., Jiang, X., Gramopadhye, A.
K., & Melloy, B. J. (2003, November). A model for predict-
ing human trust in automated systems. In Proceedings of
12 Month XXXX - Human Factors
the Eighth Annual International Conference on Industrial
Engineering—Theory, Applications and Practice (pp. 216–
222). Las Vegas.
Lee, J. D., & See, K. A. (2004). Trust in automation: Design-
ing for appropriate reliance. Human Factors, 46(1), 50–80.
doi:10.1518/hfes.46.1.50.30392
Li, Y., Bienias, J. L., & Bennett, D. A. (2007). Confounding in the
estimation of mediation effects. Computational Statistics &
Data Analysis, 51, 3173–3186. doi:10.1016/j.csda.2006.10.016
MacKenzie, I. S. (2012). Human-computer interaction: An empiri-
cal research perspective. Boston, MA: Newnes.
Merritt, S. M. (2011). Affective processes in human–automation
interactions. Human Factors, 53, 356–370. doi:10.1177/00
18720811411912
Merritt, S. M., Lee, D., Unnerstall, J. L., & Huber, K. (2015). Are
well-calibrated users effective users? Associations between cal-
ibration of trust and performance on an automation-aided task.
Human Factors, 57, 34–47. doi:10.1177/0018720814561675
Miller, D., Johns, M., Mok, B., Gowda, N., Sirkin, D., Lee, K.,
& Ju, W. (2016). Behavioral measurement of trust in auto-
mation: The trust fall. Proceedings of the Human Factors
and Ergonomics Society Annual Meeting, 60, 1849–1853.
doi:10.1177/1541931213601422
Nunnally, S., Walker, P., Kolling, A., Chakraborty, N., Lewis, M.,
Sycara, K., & Goodrich, M. (2012, October). Human influence of
robotic swarms with bandwidth and localization issues. In 2012
IEEE International Conference on Systems, Man, and Cybernet-
ics (SMC) (pp. 333–338). doi:10.1109/ICSMC.2012.6377723
Parasuraman, R., Molloy, R., & Singh, I. L. (1993). Performance
consequences of automation-induced “complacency.” The
International Journal of Aviation Psychology, 3(1), 1–23.
Parasuraman, R., & Riley, V. (1997). Humans and automation:
Use, misuse, disuse, abuse. Human Factors, 39, 230–253.
doi:10.1518/001872097778543886
Pendleton, B., & Goodrich, M. (2013, August). Scalable human
interaction with robotic swarms. In AIAA infotech@aerospace
(i@a) Conference. American Institute of Aeronautics and
Astronautics. doi:10.2514/6.2013-4731
Robinette, P., Li, W., Allen, R., Howard, A. M., & Wagner, A.
R. (2016). Overtrust of robots in emergency evacuation sce-
narios. In The Eleventh ACM/IEEE International Conference
on Human Robot Interaction (pp. 101–108). doi:10.1109/
hri.2016.7451740
Rovira, E., McGarry, K., & Parasuraman, R. (2007). Effects of
imperfect automation on decision making in a simulated
command and control task. Human Factors, 49, 76–87.
doi:10.1518/001872007779598082
Schaefer, K. E. (2016). Measuring trust in human robot interac-
tions: Development of the “trust perception scale-HRI.”
In Robust Intelligence and Trust in Autonomous Systems
(pp. 191–218). Springer. doi:10.1007/978-1-4899-7668-0_10
Tang, Q., Ding, L., Yu, F., Zhang, Y., Li, Y., & Tu, H. (2017).
Swarm robots search for multiple targets based on an improved
grouping strategy. IEEE/ACM Transactions on Computational
Biology and Bioinformatics, 15, 1943–1950. doi:10.1109/
tcbb.2017.2682161
Wang, N., Pynadath, D. V., & Hill, S. G. (2016). The impact of
POMDP-generated explanations on trust and performance in
human-robot teams. In Proceedings of the 2016 International
Conference on Autonomous Agents & Multiagent Systems
(pp. 997–1005). Retrieved from http://dl.acm.org/citation.
cfm?id=2936924.2937071
Wickens, C. D., & Dixon, S. R. (2007). The benefits of imper-
fect diagnostic automation: A synthesis of the literature.
Theoretical Issues in Ergonomics Science, 8, 201–212.
doi:10.1080/14639220500370105
Wiczorek, R., & Manzey, D. (2010). Is operators’ compliance with
alarm systems a product of rational consideration? Proceed-
ings of the Human Factors and Ergonomics Society Annual
Meeting, 54, 1722–1726. doi:10.1177/154193121005401976
Winfield, A. F. (2009). Foraging robots. In R. A. Meyers (Ed.),
Encyclopedia of complexity and systems science (pp. 3682–
3700). Springer. doi:10.1007/978-0-387-30440-3_217
Xu, A., & Dudek, G. (2015). OPTIMo: Online probabilistic trust
inference model for asymmetric human-robot collaborations.
In Proceedings of the Tenth Annual ACM/IEEE International
Conference on Human-Robot Interaction (pp. 221–228).
doi:10.1145/2696454.2696492
Yang, X. J., Unhelkar, V. V., Li, K., & Shah, J. A. (2017). Evaluat-
ing effects of user experience and system transparency on trust
in automation. In Proceedings of the 2017 ACM/IEEE Inter-
national Conference on Human-Robot Interaction—HRI 17
(pp. 408–416). ACM Press. doi:10.1145/2909824.3020230
Aya Hussein is a PhD student in the Computer Sci-
ence program at the University of New South Wales,
Canberra, Australia. She received her bachelor’s and
master’s degrees in computer engineering from
Cairo University in 2011 and 2015, respectively.
Sondoss Elsawah is a senior lecturer in the School of
Engineering and Information Technology, Univer-
sity of New South Wales, Canberra, Australia. She
received her PhD in computer science in 2010.
Hussein A. Abbass is a professor in the School of
Engineering and Information Technology, Univer-
sity of New South Wales, Canberra, Australia. He is
a member of the Human Factors and Ergonomics
Society.
Date received: September 17, 2018
Date accepted: September 3, 2019