434
3, AUGUST 2018
Cyber Risk Analysis for a Smart Grid: How Smart is
Smart Enough? A Multiarmed Bandit Approach to
Cyber Security Investment
Matthew David Smith
Abstract—As electric sector stakeholders make the decision to
upgrade traditional power grid architectures by incorporating
smart grid technologies, the benefits of added connectivity must
be weighed against the risk of increased exposure to cyber-attacks.
Therefore, decision makers must ask: How smart is smart enough?
This paper presents a probabilistic risk analysis framework to
address this problem. The goal is to quantify the overall benefit
and risk of adding connections to a network and hiring a number
of cyber defense teams, with the objective to help decision makers
formally assess tradeoffs and set priorities given limited resources.
Central to this approach is a new Bayes-adaptive network
security model based on a reformulation of the “multiarmed
bandits” (MAB) problem. Here, instead of projects with uncertain
probabilities of success as in the classic MAB problem, a network
defender faces the possibility of attacks against network nodes at
uncertain Poisson-distributed rates. This new technique, which by
similarity we call “multinode bandits,” takes a dynamic view of
cyber security investment, exploring how network defenders can
optimally allocate cyber defense teams among nodes. In effect, this
approach entails employing proactively for defensive and informa-
tion gathering purposes teams that traditionally respond to cyber
breaches after they occur. We apply this model to the case study
of an electric utility considering the degree to which they should
integrate demand response into their smart grid network, jointly
identifying both the optimal level of connectivity and the optimal
strategy for the sequential allocation of cyber security resources.
Index Terms—Cyber-physical security, multiarmed bandit
(MAB), smart grid.
I. INTRODUCTION
HE EMERGENCE of the smart grid promises to deliver
T many benefits to the overall operation of the North Amer-
ican electric grid, including increased efficiency, improved
reliability, better incorporation of renewable energy sources,
and more choice for electricity consumers [1]. However, the
same technologies that improve the performance of the smart
grid also expose it to digital threats such as denial of service
attacks, intellectual property theft, invasion of privacy, and
sabotage of critical national infrastructure [2]. Furthermore,
Manuscript received July 14, 2017; revised December 14, 2017; accepted
January 8, 2018. Date of publication February 26, 2018; date of current version
July 17, 2018. Review of this manuscript was arranged by Department Editor
B. Jiang. (Corresponding author: Matthew David Smith.)
The authors are with the Department of Management Science and Engineer-
ing, Stanford University, Stanford, CA 94305 USA (e-mail: mdsmith44@gmail.
com; mep@stanford.edu).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TEM.2018.2798408
0018-9391 © 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.
IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 65, NO.
and M. Elisabeth Paté-Cornell
given the integrated nature of modern cyber-physical systems,
cyber-induced failures of the power grid can cascade to other
critical infrastructure sectors, such as transportation networks,
water treatment, or financial systems, causing extensive
physical damage and economic disruption.
While there is growing recognition across government,
academia, and the private sector of the cyber vulnerability of
the electric grid, the likelihood and consequences of a cyber-
attack are difficult to quantify. Therefore, electric sector stake-
holders have problems determining which investments to make
beyond the minimum required for compliance with mandatory
standards. As a result, current risk management approaches are
generally qualitative or heuristic in nature [3].
This paper presents a probabilistic risk analysis (PRA) ap-
proach to smart grid cyber security. In particular, this paper takes
a dynamic and stochastic view of cyber security investment, ex-
ploring how defenders of smart grid information networks can
optimally allocate cyber defense teams among nodes in their
network. In short, this involves taking teams that traditionally
respond to breaches after they occur, and instead employing
them in a proactive manner for defensive and information gath-
ering purposes. We then show how this model can be used to
identify the optimal level of connectivity, where the benefits of
increased incorporation of smart grid technologies are weighed
against the cyber security risks that these new connections entail.
Given the sequential decision nature of this network de-
fense formulation, we draw insights from multiarmed bandits
(MABs), a class of problems where a decision maker must
sequentially allocate resources among competing projects, typ-
ically with uncertain probabilities of success [4]. In cyber se-
curity settings, network defenders are often concerned not just
with the probability of a compromise, but also the rate at which
nodes in their network can be attacked. Inspired by this notion,
we developed a new variant of the MAB model suited to cyber
security settings. Instead of gaming machine “arms” or projects
with unknown probabilities of success, a decision maker faces
unknown Poisson rates of attack against nodes in their network.
The question at each step is how to employ cyber defense teams
to defend against these attacks, protecting the network against
known threats while also exploring the network to learn about
new, unknown threats. We refer to this new formulation as a
multinode bandit (MNB) model.
Using this MNB network security model, along with systems
and economic analysis of smart grid networks, we solve for
SMITH AND PATÉ-CORNELL: CYBER RISK ANALYSIS FOR A SMART GRID: HOW SMART IS SMART ENOUGH?
the optimal network defense strategy. Based on that strategy,
we quantify the net financial benefit and net cyber risk to a
smart grid information network, as a function of the level of
connectivity. This allows us to gain insights into two related
research questions facing smart grid system operators: (1) How
smart is smart enough, and (2) How much to invest in cyber
security, and where to invest it?
This paper makes a number of analytical and computational
contributions aimed at enhancing the resilience of a smart grid
against cyber threats. Chief among these is the development of
a carefully constructed risk analysis method to enable a system-
atic evaluation of the tradeoff between the benefit and risk of
making the power grid “smarter.” Another principal contribu-
tion of this paper is the definition, development, and use of the
MNB network security model, a Bayes-adaptive framework for
the sequential allocation of cyber defense teams among nodes
in a network. Finally, we apply this model to the case study of
an electric utility considering the degree to which they should
integrate demand response into their smart grid network, jointly
identifying both the optimal level of connectivity and the op-
timal strategy for the sequential allocation of cyber security
teams.
This paper is organized as follows. Section II gives an
overview of the cyber threat to the smart grid, discusses the
research questions, and reviews the relevant literature. After pre-
senting an overview of MABs, Section III develops the MNB
network security model, and discusses how this model fits into
the overall PRA framework to identify the optimal level of
connectivity for a smart grid network. Section IV illustrates the
application of this model to a case study considering the incorpo-
ration of demand response technology into the power networks
of the Sacramento Municipal Utility District (SMUD), identify-
ing the optimal degree of connectivity and the optimal network
defense strategy. Section V discusses the modeling power of
this approach, and offers concluding remarks.
II. SMART GRID CYBER SECURITY
A. Motivation: Cyber Threat to the Smart Grid
The smart grid can be thought of as a modernization of exist-
ing generation, transmission, distribution, and metering infras-
tructures, in which existing systems have been upgraded to a
digital anatomy of microprocessors, software, and network com-
munications channels [5]. In some cases, the new technologies
augment existing components that perform the same function
as before, but have become “smart” by providing and commu-
nicating information about their respective task to a centralized
system. In other cases, the smart grid provides completely new
functionalities that allow human operators and the grid itself to
react intelligently to changing conditions.
In order to reap the benefits of this new level of intelligence,
the smart grid entails an unprecedented level of coupling be-
tween communications and power networks. Similar to security
concerns caused by the proliferation of Internet of Things de-
vices [6]–[8], the growth of interconnections in a smart grid
creates new potential attack vectors into the power system and
therefore new security challenges for utilities. If they could gain
435
Fig. 1. Smart grid cyber-physical networks.
access, hackers could manipulate control systems to disrupt the
flow of electricity, transmit erroneous signals to operators, block
the flow of vital information, or disable protective systems. Con-
cerns over the threat of such a compromise are exacerbated by
the widespread availability of SCADA-specific hacking tools,
which exploit the unique vulnerabilities of industrial protocols
and have created a lower barrier to entry for malicious cyber
activity against the grid [9]. A growing array of threat actors,
including nation states, terrorist groups, criminal organizations,
disgruntled insiders, or common low-level hackers have the po-
tential motivation and capability to inflict various degrees of
harm on power grids [3].
B. Problem Statement
1) How Smart is Smart Enough?: As electricity sector stake-
holders make the decision to upgrade traditional power grid ar-
chitectures by incorporating smart grid technologies and new
intelligent components, the benefits of added connectivity must
be weighed against the risk of increased exposure to cyber-
attacks. Decision makers must thus ask: how smart is smart
enough?
To measure “smartness,” we consider a physical power grid
network consisting of nodes (generators, customers, and sub-
stations), and edges (transmission lines). The system operator
may choose to connect any subset of nodes, creating an overlaid
information network, as illustrated in Fig. 1. Each information
node enables a smart grid technology, but also becomes a po-
tential cyber-attack vector.
Using this framework, we define smartness as the number
of nodes that have been connected to the information net-
work. Therefore, smartness conceptually represents the degree
to which a particular smart grid technology and its external links
has been integrated into the utility’s power network. As a grid
becomes “smarter,” we expect two things to happen.
1) The marginal benefit will decrease. For many smart grid
technologies, most of the benefit can be achieved with
a moderate level of deployment. One reason for the de-
creasing marginal returns is the assumption that nodes are
connected in the optimal order, with the most advanta-
geous nodes selected first. As one example, a recent re-
port found that installing conservation voltage-reduction
436 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 65, NO. 3, AUGUST 2018
technology on 40% of distribution feeders achieves 80%
of the benefit of upgrading all feeders [10].
2) The cyber security risk will increase. While the effects of
the dependency are not immediately apparent, increased
connectivity causes an increase in both the probability of
cyber-attacks given an increased number of attack paths,
and their potential consequences given the tighter cou-
pling between control components in the power network.
When considered together, these two trends have crucial con-
sequences for utilities when evaluating the tradeoff between
the benefit and risks of increased connectivity. Beyond a certain
level, added connectivity yields little additional benefit, and may
be outweighed by the increase in cyber risks.
2) How Much to Invest in Cyber Security?: Numerous cy-
ber security countermeasures and network architectures have
been proposed and developed in recent years to help protect
smart grid information networks. Broadly speaking, these in-
volve a combination of traditional IT countermeasures (e.g.,
antivirus protection, or firewalls that separate control networks
from enterprise networks) as well as additional techniques that
are tailored to the unique security requirements of energy con-
trol systems (e.g., application whitelisting and host data loss
prevention, which require fewer computational resources and
less-frequent patching) [9]. In addition to these technical solu-
tions, there are also a range of management solutions available
to utilities, such as employee training, increased intelligence
gathering, or hiring a chief risk officer.
Despite the range of cyber security tools available to network
defenders, there is significant uncertainty surrounding the risk-
reduction value of the different techniques. For example, given
a fixed security budget, should a network defender allocate the
resources toward improved firewalls, training for employees, or
an increased member of the organization’s cyber security team?
These types of questions are difficult to address, especially due
to the dynamic and uncertain nature of cyber threats. Further-
more, what if the budget is not fixed? Now defenders must ask
how much to invest, in addition to where to invest it. We focus
here on the number and allocation of cyber defense teams, and
their task to protect against both known and new vulnerabilities.
The MNB model, when used as part of an overall PRA frame-
work, provides a way to address both research questions. When
considering a power system with a finite number of nodes, the
model allows system operators to solve for the optimal alloca-
tion of a cyber defense team among nodes in the network. Based
on that strategy, we quantify the cyber risk facing the network
for any number of connected nodes. Comparing this risk to the
benefit curve of increased smartness permits system operators
to identify the optimal level of connectivity. If we allow for the
number of response teams to be variable, we can also address
the question of how much to invest in these teams. Hence, using
the MNB model, system operators can address how smart to
make their network (how many nodes to connect) as well as
how much to invest (how many cyber defense teams to hire).
C. Assumptions and Justification
This section discusses key assumptions that bound the scope
of our analysis and ensure that the MNB network security
model and the overall PRA approach remain both reasonable
and tractable.
Assumption 1: Smartness of a smart grid network is defined
as the number of physical nodes that have been upgraded with a
particular smart grid technology and connected to the informa-
tion network.
The setting for our research model is an electric utility com-
pany faced with the following decision: starting from a baseline
version of their power grid network, they have to decide on
the degree to which they want to integrate some smart grid
technology into their network. As illustrated in Fig. 1, we repre-
sent the physical network by an undirected graph consisting of
nodes (electrically distinct points in the network, which include
generators, substations, and consumers) and edges represent-
ing transmission lines. This representation of power networks
is common in power systems analysis, and modern power net-
works can consist of thousands of such nodes in the case of an
entire regional interconnect, or dozens in the case of municipal
utilities [11].
The utility’s decision amounts to choosing which subset of
nodes to connect with the smart grid technology (and the added
connectivity that entails). In the information network, each
node thus represents an Internet-connected point, for example, a
router that transmits data, or a control center that aggregates in-
formation and makes control decision, and each edge is a direct
communication link [12]. The size of the information network
therefore scales with the increased incorporation of a particular
smart grid technology. As the grid becomes smarter, the infor-
mation network grows to accommodate the increased demand
for real-time monitoring and enhanced control with fast and ef-
ficient algorithms, but the added connectivity also increases the
number of pathways into utility systems. In effect, this network
model discretizes the utility’s decision regarding how smart to
make their system. However, since electricity balancing author-
ities and system operators commonly aggregate customers into
large loads to realize savings, we argue that it makes sense to
bundle the total system load into discrete customer nodes in this
fashion.
Assumption 2: The rate of successful cyber-attacks against
a connected node can be modeled as a Poisson process.
Empirical analysis of cyber security incident data has shown
that the arrival of successful cyber-attacks is well modeled by a
Poisson process. For example, an analysis of recently released
security logs of 1131 cyber intrusions against the U.S. Depart-
ment of Energy from 2010 to 2014 [13] shows that the dis-
tribution of interarrival times is exponential, and thus arrivals
are Poisson, as shown in Fig. 2. Additional theoretical justifi-
cation comes from the Palm–Khintchine theorem, which states
that the aggregate arrivals from many (possibly non-Poisson)
sources approach a Poisson distribution in the limit.
Assumption 3: The rates of attack to different nodes in the
network are independent.
While it is reasonable to think that these rates may be depen-
dent (for example, in the case where an attacker is launching a
coordinated attack using multiple attack vectors), we justify this
assumption by noting that smart grid information networks are
highly distributed and highly heterogeneous. That is, the nodes
in the physical network are geographically dispersed, often
SMITH AND PATÉ-CORNELL: CYBER RISK ANALYSIS FOR A SMART GRID: HOW SMART IS SMART ENOUGH?
Fig. 2. Analysis of cyber security logs of 1131 total cyber intrusions against
the U.S. Department of Energy (DoE) between 2010 and 2014. Interarrival times
are well modeled by an exponential distribution. Interarrivals from 3 to 6 days
are likely overrepresented because incidents are only recorded on weekdays.
separated by many miles, and run by separate system adminis-
trators. As a result, even though the information nodes are all
part of the same broader information network, each node is in
essence its own subnetwork with unique security configurations
and hence unique vulnerabilities that an attacker could exploit.
Assumption 4: The prior probability distribution of each at-
tack rate can be modeled as a gamma distribution.
Gamma distributions are the Bayesian conjugate distributions
to a Poisson, allowing us to track and update the state of knowl-
edge about the uncertain rates of attack against network nodes,
as discussed further in Section III. In addition to the compu-
tational convenience, we can further justify the use of gamma
distributions by noting that they are a good modeling fit for
cyber security settings. First, they effectively capture the “fat-
tail” nature of cyber-attack intensity, as gamma distributions are
leptokurtic (they fall to zero more slowly than a normal distri-
bution). Additionally, the two distribution parameters α and β
allow network defenders to effectively model a wide range of
prior distributions, which may be informed by historical base-
lines or threat reporting.
D. Related Work
There is growing interest in quantitative models and methods
for managing the cyber risk to the electric grid and critical in-
frastructure in general [14]–[17]. However, the task of modeling
the behavior of these complex systems is vast, and current efforts
are still in the early stages. Much of the risk assessment research
in smart grid cyber security utilizes some form of attack trees
or attack graphs [18]–[20], but attack trees are far from the only
modeling approach explored in the literature. Numerous game
theory approaches have been proposed to help smart grid oper-
ators make informed decisions on security strategies given their
potential intelligent adversaries [21]–[23]. In another common
line of research—network science approaches—model compo-
nents are considered as nodes and connections as edges to study
437
the spreading of failures [24] or the defensive value of new
network topologies [25].
Given the inherent tradeoffs involved in defending smart
grid information networks in uncertain threat environments—
specifically the tension between protecting against known cyber
threats (exploitation) versus actively probing the network to
gain a better understanding of unknown threats (exploration)—
we argue that MAB models are a natural framework for cyber
security settings. However, despite the resurgence of research
about MABs in recent years [4], their application to cyber se-
curity is scarce. Relevant to this paper is a recent work that
applies MABs and reinforcement learning techniques to iden-
tify adaptive network defense strategies under uncertain threat
environments [26], [27]. These efforts, however, all use the
standard MAB framework to analyze situations with unknown
probabilities of a compromise. To the best of our knowledge, the
extension of MAB models to handling uncertain Poisson attack
rates is a novel contribution.
III. MULTINODE BANDITS: A BAYES-ADAPTIVE NETWORK
SECURITY MODEL
A. Model Framework and Scope
To assess the benefits and risks of smart grid integration, we
consider an electric utility that wishes to maximize its expected
daily profit. The justification for this choice is that utilities are
typically the entities actually making decisions about how to
invest in cyber security countermeasures, and they will only
do so if they can provide the business case for their financial
investment.
We consider a utility faced with the following situation: Start-
ing from a baseline version of their power grid network, they
have to decide to what degree to integrate some smart grid tech-
nology into their network. Each connected node further enables
the benefits of that smart grid technology, but also becomes a
potential cyber-attack vector, and is subject to successful cyber-
attacks at an uncertain rate λi .
For the defense setting, we assume that a network manager
has the resources to probe one node in the network per day. This
action thwarts any attempted cyber-attacks to that node on that
day, and also provides information that the defender can use to
update his or her belief about the uncertain rates of attack.
Given the sequential decision-making nature of this formu-
lation, this setting lends itself to a Bayes-adaptive Markov
decision process (BAMDP) formulation, a powerful tool for
studying sequential decision problems when there is model un-
certainty. In particular, we draw inspiration from a class of
BAMDPs known as MABs.
B. Multiarmed Bandits
MABs are classic problems in operations research, in which
a decision maker must sequentially allocate efforts among
competing actions, whose values are uncertain at the time of
allocation, but become better understood as time passes. In the
canonical example, a gambler is to choose a sequence of plays
from a finite set of slot machine arms, each with an unknown
probability of success pi . Originally formulated in the 1930s,
438
MABs at first proved to be exceptionally challenging to solve
due to their computational complexity. However, with recent
advances in computational power and reinforcement learning,
MABs have seen a resurgence and are now used in a wide vari-
ety of applications, including clinical trials, managing research
projects in a large organization, and website AB testing [4].
Taking a Bayesian approach, we can model our prior belief
about each uncertain probability pi using a beta distribution with
shape parameters αi and βi . Due to betabinomial conjugacy, our
belief over the uncertain probabilities remains a sequence of beta
distributions after observing the results of pulling a given arm,
but with updated parameters (if we pull the arm of machine i
and that machine wins, then αi → αi + 1; if arm i loses, then
βi → βi + 1). We can thus treat the sequence of belief parame-
ters as the state of the system, e.g., xt = (α1 , β1 , α2 , β2 , α3 , β3 )
in the case of three arms. The Bayesian updating rules provide
the system’s dynamics equations, i.e., the probabilities of tran-
sitioning to a new belief state as the system evolves. This ef-
fectively converts the problem into a Markov decision process,
which we can solve using dynamic programming techniques in
the case of finite time horizons, or allocation indices in the case
of discounted infinite time horizon problems [4].
One of the principle features of MABs is the fundamental
tradeoff between exploitation (choosing the arm that we be-
lieve is best, based on the information that we have gathered
so far) and exploration (choosing an arm about which we are
uncertain, gaining information that will improve the quality of
future decisions). This same fundamental tradeoff is present
when employing cyber security resources, where an effective
strategy must strike a balance between actions aimed at protect-
ing against known threats, and those aimed at learning about
the uncertain threat environment to protect against new and un-
known threats. Inspired by this notion, we develop a new variant
of MAB models suited to network defense settings, as discussed
in the next section.
C. Multinode Bandits
In cyber networks, system defenders are often concerned not
only with the probabilities of attack, but also the rate (frequency)
at which nodes in their network are under attack. Based on
this notion, we develop a new formulation of the MAB model,
where each node in a network is under attack at an uncertain
Poisson distributed rate. To retain the Bayesian formulation, we
model the prior probability distribution for each attack rate as a
gamma distribution, which is a conjugate distribution of a Pois-
son. A gamma distribution for the uncertain rate λ is given
βα −β λ α −1
by gamma(λ; α, β) = Γ(α ) e λ , where α is the shape
parameter, β is the rate parameter, Γ(α) is the gamma func-
tion of α, and the expected value of λ is E[λ] = α/β. Due to
gamma-Poisson conjugacy, the probability distribution of an un-
certain Poisson rate remains a gamma distribution as the system
evolves with new information. After observing k arrivals in t
time steps, our posterior probability distribution for λ is given by
Gamma(λ; α + k, β + t). Therefore, as in the standard MAB
case, we can consider our current belief parameters to be the
“state” of the system.
IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 65, NO. 3, AUGUST 2018
Fig. 3. Conceptual representation of the multinode bandits model, with
gamma probability distributions over uncertain rates of attack, rather than Beta
distributions over uncertain probabilities as in the classic MAB problem.
By similarity with MABs, we refer to this new formulation
as “MNBs”, as depicted in Fig. 3.
1) Basic Formulation: We apply the MNB model to cyber
security settings as follows.
1) A network has N nodes connected to the Internet.
2) For node i as follows.
a) The rate of successful cyber-attacks is a Poisson
process with a constant, but uncertain, rate λi .
b) the prior probability distribution over λi is
gamma(λi ; αi , βi ).
c) Each successful attack incurs an expected
cost ci .
3) At each time period, a cyber defense team probes exactly
1 node.
a) We observe the number of successful cyber-attacks
ki (t) to that node in that time step.
b) We thwart any successful attack to that node in that
time step, thus resulting in a cost saving (or a reward)
of ci · ki (t).
4) The goal is to minimize the overall expected cost over the
finite horizon T.
As in the case of standard MABs, there is a fundamental
tradeoff between exploitation and exploration. Since probing
a node thwarts any current cyber-attacks against that node, a
pure exploitation policy would be to choose the node where the
expected cost inflicted by cyber-attacks, E[λi ci ], is the high-
est. Intuitively, this corresponds to investing security resources,
where the network is most heavily attacked. However, since
probing a node yields information that permits updating our in-
formation about λi , we also have incentives to explore nodes
about which we are more uncertain.
Given the dynamic nature of cyber threats, we choose a finite
time horizon. A short time horizon (T ≤ 60 days) not only
simplifies the computation, but also justifies the assumption that
the rates of attack remain constant (though unknown) throughout
the planning horizon.
2) Cyber Defense Teams: In this paper, we consider the
sequential action in MNB to represent the allocation of cyber
defense teams. As discussed earlier, this involves taking
incident response teams whose traditional role is to respond
to and recover from security breaches after they occur, and
employing them in a proactive manner for defensive and
information gathering purposes. The use of response teams
in a proactive manner is an emerging trend as cyber security
SMITH AND PATÉ-CORNELL: CYBER RISK ANALYSIS FOR A SMART GRID: HOW SMART IS SMART ENOUGH? 439

practitioners in both private and public sectors recognize the

value of investing in preventive measures rather than facing the
costly task of response and remediation after a breach occurs
[28]. While there are different terminologies for this technique
(e.g., Digital Forensic Incident Response teams [29], Cyber
Protection Teams [30], Cyberspace Defense Systems [31]), we
use the generic term “cyber defense team” in this paper.
With each node that they probe, defense teams bring an extra
level of protection on top of the baseline cyber security posture
already in place. Tasks commonly performed in the analysis of a
network node may include in-depth research of historical traffic
reported through sensors, malware, and hard drive analyses to
determine the impact to the network and the initial attack vector,
as well as other analyses of complex datasets such as system
logs, network events, and packet capture data. A typical team
is thus only able to probe one node in the network per day,
and hence must be judiciously allocated among the network
nodes. As a consequence of the deep forensic analysis that they
perform, defense teams glean information that can be used to
update the defender’s assessed probability distribution about the
rate at which that node is attacked. Consequently, as in the classic
MAB problem, an effective strategy must balance exploitation
(protecting against known threats) versus exploration (probing
a node that is poorly understood to learn about new, unknown
threats). We consider in the rest of this paper the problem of
identifying a strategy for the sequential allocation of these cyber
defense teams to the different nodes of a smart grid information
network. Fig. 4. Formal specification of the Markov decision process formulation used
3) Solving MNB Problems: As with standard MABs, we can to solve for the optimal probing strategy in a multinode bandit problems.
solve MNB problems by considering the state of the system to
be the current values of the gamma distribution parameters, Accounting for these challenges, we obtain the MDP formu-
xt = (α1 , β1 , . . . , αN , βN ), where αi > 0 and βi > 0 are the lation shown in Fig. 4.
current parameters of the gamma probability distribution of λi , Given this formulation, we use dynamic programming to
the uncertain rate of attacks against node i. The problem of iden- solve an illustrative MNB problem with n = 2 nodes, a time
tifying the optimal probing strategy of cyber defense teams can horizon of T = 14 days, a cost per attack of ci = $10 k for each
therefore be cast as a Markov decision process. However, solv- node, and parameters of the prior distribution of the rate of attack
ing the resulting Markov decision process presents challenges of each node as αi = 2 and βi = 2 (yielding an initial expecta-
beyond what was required to solve standard MAB problems. tion of E[λi ] = 1 attack per day for each node). The resulting
First, belief updating (i.e., the probability of transitioning from optimal control policy yields an overall maximum reduction of
one information state to the next) is nontrivial. We must find the the cost of attacks of $173.8 k. The intuition of this result is that
probability of observing k arrivals in one time step when our it represents the value of thwarting cyber-attacks over that time
belief over the rate of the Poisson process is gamma(λ; α, β). period. For example, if we did not probe any node, cyber-attacks
This can be shown to be the following: would inflict an expected cost of $280 k on our network over
the 14 days. A naı̈ve strategy, which picked one node at random
βα Γ (α + k) during every time unit, would stop, on average, half the attacks,
Pr (k|α, β) = . (1)
Γ (α) k! (β + 1)α +k achieving a value (cost savings) of $140 k. The optimal probing
strategy thus performs significantly better than the random one,
Second, a more significant computational challenge is the saving $173.8 k, because it adapts the choice of node as system
fact that in one time unit, we can observe any integral num- parameters become better understood.
ber of attacks from a Poisson process, meaning that, in theory, 4) Approximate Solution Methods: Although we are able to
we must keep track of an infinite number of transition pos- solve for the optimal probing policy in simple versions of the
sibilities. To mitigate this, we place a cap on the number of MNB problem, in practice it is rarely possible to find exact so-
arrivals, with m being the most arrivals we allow for a single lutions for larger (more realistic) network problems due to the
node in a single time step. We choose m to balance accuracy exponential increase in computational complexity as the number
with computational complexity, as low values of m may cut off of nodes and planning horizon increase. Therefore, we explore
the high-impact/low-frequency events in which a node is under in this section the effectiveness of various approximate solution
severe attack, but higher values of m will result in exponential methods designed to find nearly optimal solutions while striking
increases in computational complexity. a balance between performance and ease of implementation. We
440 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 65, NO. 3, AUGUST 2018

TABLE I
HEURISTIC-BASED EXPLORATION STRATEGIES FOR MULTINODE BANDIT
PROBLEMS Y i = λi ci

Strategy Choose Node a Intuition

Greedy a = argmax E[X i ] Pick “best” node based

i
on information so far
Boltzmann Pr{a = i} ∝ eE [Y i ]/ B Start out exploratory (high
B is the Boltzmann temperature), then get more
Temperature greedy (cooler temp)
Best Quantile a = argmax FY−1 (q) Better measure of
i
i
q is the quantile “potential upside”
point in CDF
Variance Bonus a = argmax {E[Y i ] Encourages exploration
i
+η(tR − 1) · var(Y i )}tR is time of nodes with more
remaining; η is the bonus factor uncertainty

The variable Y i = λi c i represents the expected cost of attacks against node i, based on
current information state. Fig. 5. Use of value of perfect information to calibrate an approximate value
function, from which we can fit an approximate value function for use in a
one-step look-ahead algorithm.
consider two classes of methods: 1) heuristic-based exploration
strategies, where a node is selected at each step based on some
easily calculated property of the current system state, and 2) ap-
proximate dynamic programming, where we make simplifying
assumptions that enable us to solve a simplified optimization
problem as a proxy to the intractable multiperiod problem.
The first class of approximate solution methods is heuristic-
based exploration strategies. Based on a review of MAB liter-
ature as well as intuition on the unique characteristics of the
MNB variation, we evaluated the solution methods described in
Table I. The parameters of each method are tuned to ensure the
best match with the known optimal policy for the sample two-
node problem from Section III-C3 for which the exact solution
is known. The greedy strategy represents a pure-exploitation
strategy, while the other methods represent different ways to
incorporate exploration into the node-probing strategy.
While the heuristic methods attempt to circumvent the need Fig. 6. Comparison of approximate solution strategies for multinode bandits.
to address the dynamic programming problem, approximate
dynamic programming methods attempt to solve the original
dynamic programming problem by making simplifying as-
sumptions that make it more tractable. We explore here the increase toward VoPI. Indeed, the exact value of various states
use of one such method, a one-step look-ahead algorithm. of information shows this pattern, as presented in Fig. 5.
This means that at any time, rather than performing value A comparison of all the approximate methods considered here
iteration from the terminal horizon back to the current time, (the four heuristic-based strategies as well as the one-step look-
we only consider the expected cost of the next transition, and ahead) is shown in Fig. 6. We compare methods based on three
use an approximate value function to estimate the remaining factors: 1) accuracy (how often do we take the correct action);
value until the end of the time horizon. In order to find a good 2) value of implementing that strategy, as determined by the
approximate value function, we used the concept of Value of Monte Carlo simulation; and 3) computational speed.
Perfect Information, defined as follows. We observe that the one-step look-ahead strategy with a VoPI-
1) Value of Perfect Information (VoPI) = Value we would based approximate value function provides the best value and
get if we knew all uncertain rates of attacks. accuracy, although it is the slowest one to compute. The quantile-
2) Value of No Information (VoNI) = Value we would get if selection and variance bonus strategies perform well, which we
we could not update our beliefs as we got further infor- can attribute to the fact that they capture the “potential upside”
mation by probing each node. of probing a node better than the purely greedy, short-sighted
Intuitively, with one time unit remaining, the best value that strategies. In the rest of this paper, we use the VoPI method
we can achieve is VoNI, since it would be too late to adapt after to gain insights into more complex MNB problems, although
getting new information. With more time remaining, there is it should be noted that the variance bonus method would be
more opportunity to explore, and we expect the probing value to especially useful in settings where a solution is needed quickly.
SMITH AND PATÉ-CORNELL: CYBER RISK ANALYSIS FOR A SMART GRID: HOW SMART IS SMART ENOUGH?
Fig. 7. Overview of probabilistic risk analysis approach to smart grid cyber security.
IV. OPTIMAL CONNECTIVITY AND ALLOCATION OF CYBER
SECURITY RESOURCES
This section illustrates how smart grid system operators can
utilize the MNB network security model as part of a PRA frame-
work to identify the optimal level of connectivity and the optimal
network defense strategy for a smart grid network.
The overall PRA framework is illustrated in Fig. 7. In the first
step, we perform systems analysis to identify plausible classes
of failure scenarios that can be induced by exploiting cyber
vulnerabilities in a smart grid network, as drawn from industry
expert opinion and known incidents. We also specify the initial
threat estimates, which take the form of a set of gamma distri-
bution priors over the uncertain Poisson rates of attack against
the system. Step 2 uses an economic dispatch model to compute
the financial impact of each failure scenario, as well as the ben-
efit of increased network smartness, based on the physics and
economics of their effect on the power system. The outputs of
the economic analysis are numerical inputs to the MNB model,
which is used in step 3 to solve for the optimal strategy for the
sequential allocation of cyber defense teams among nodes in
the network. Based on that strategy, we quantify the net cyber
risk to smart grid information networks that remains after em-
ploying the optimal cyber defense strategy. Finally, in step 4, a
decision-analytic framework combines the benefit of increased
smartness, the cost inflicted on the network by newly introduced
cyber vulnerabilities, and the value of optimal defensive probing
strategies to identify the optimal level of connectivity and the
optimal number of defense teams to hire.
We apply this framework to a case study of the incorporation
of demand response in the management of the SMUD power
network. Demand response is an emerging smart grid tech-
nology, which allows utilities to use price signals to influence
customer behavior, inducing lower electricity demand when op-
441
erating costs are high (e.g., during peak demand) or when system
reliability is in jeopardy (e.g., during an unexpected outage or
other system contingency). Due to the communication overhead
needed to support and enable the technology, demand response
is a quintessential example of a smart grid technology, where
the increased connectivity provides benefits to system operators
while simultaneously introducing new cyber risks [32]. There-
fore, demand response is an appealing case study for application
of our research model, which seeks to identify the optimal level
of connectivity.
A. Case Study Network
We illustrate the MNB network security model and the cas-
cading effects of failures using a schematic 24-node intercon-
nected power network of the SMUD, as shown in Fig. 8. We then
consider the decision regarding the degree to which one should
integrate demand response into any subset of the ten highlighted
consumer nodes, with smartness ranging from 0 to 10.
The reasons for choosing SMUD are threefold. The first ben-
efit is the relative simplicity and isolation of the SMUD power
network, which facilitates illustration of the basic concepts of
the model. Second, as it is a publicly owned utility, more data
are available from resources like the Energy Information Ad-
ministration [33] and California Energy Commission [34] than
for privately owned utilities. Third, SMUD is currently assess-
ing the potential benefits of incorporating demand response into
their network, as evidenced by their recently completed pilot
program [35]. Hence, they are a good candidate to study risks
and benefits of making their network smarter via the incorpora-
tion of demand response.
To model the behavior of the SMUD power network and
the impact of demand response, we utilize the IEEE 24-Bus
Reliability Test System, a simple, adaptable, and well-studied
442
Fig. 8. 24-node schematic representation of the SMUD power network, con-
sisting of customer (C), generator (G), and transformer (T) nodes. The ten
customer nodes may be upgraded with demand response technology.
benchmark of a “typical” utility published by IEEE in 1996 [36].
The original 24-Bus test system contains useful data on the net-
work’s generation capacity, power flow limits, transmission line
failure rates, and customer demand data. We then adjust electric
generation cost and consumer demand parameters to match the
characteristics of the SMUD power network to study the im-
pact of demand response. The complete network specification
is described in the Appendix.
B. Economic Analysis: Risks and Benefits of Connectivity
The risks and benefits of incorporating demand response into
SMUD’s network are calculated using an economic dispatch
model to compute the technology’s impact on the daily operat-
ing profit of the utility. An economic dispatch algorithm uses
an optimal power flow calculation to compute the cheapest way
to dispatch enough power to meet the daily demand, subject to
maintaining system security and satisfying system constraints
such as power line, voltage, and generator limits [37]. Also
referred to as production cost modeling in the literature, this
approach has formed the basis for a number of studies attempt-
ing to quantify the benefit of demand response for smart grid
networks [38], [39]. Economic dispatch calculations are in fact
used by power system operators on a daily basis. Therefore,
this approach has the advantage of introducing cyber security
considerations into the same economic calculus as that used by
electricity providers to make daily operational decisions.
1) Benefits: We evaluate the benefit of demand response in-
tegration by calculating the reduction in the net cost for SMUD
to generate and deliver enough electricity to meet demand. Our
approach is similar to the work of Kwag and Kim [39], who
IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 65, NO. 3, AUGUST 2018
Fig. 9. Benefit curve of increased smartness for SMUD power network. The
curve specifically accounts for the order in which the nodes are connected, with
the most advantageous nodes selected first.
demonstrated a theoretical technique to incorporate demand re-
sponse into the economic dispatch algorithm used to find the
optimal system operating cost for a utility with quadratic gener-
ation cost functions. We apply a similar approach to the SMUD
case study, with additional inputs to address the differing flex-
ibility of commercial versus residential loads, as described in
the Appendix.
In addition to influencing customers to shift their routine daily
energy usage patterns—resulting in a flatter load profile, which
is cheaper for the utility to service—we also consider a benefit
contribution from the quicker detection and mitigation of unex-
pected outages or system contingencies. For example, demand
response sensors provide utilities with the precise location of
an outage. This allows targeted dispatch of restoration crews
and allows operators to switch off noncritical loads to prevent a
major outage [32]. Accounting for the benefit of flattening the
routine daily profile, as well as the benefit of mitigating the im-
pact of unexpected outages of transmission lines (as described
in the Appendix), we find the benefit curve shown in Fig. 9.
2) Risks: To model the risks associated with increased
smartness, we considered five classes of cyber security fail-
ure scenarios related to the incorporation of demand response
technologies, based on those identified in a recent analysis by
the National Electric Sector Cyber security Organization Re-
source (NESCOR) [40]. NESCOR failure scenarios are defined
as “a realistic event in which the failure to maintain confiden-
tiality, integrity, and/or availability of sector cyber assets creates
a negative impact on the generation, transmission, and/or deliv-
ery of power.” The NESCOR study encapsulated the collective
knowledge of numerous industry experts, resulting in a com-
prehensive list of over 100 such plausible cyber security failure
scenarios for the electric grid. Hence, the report serves as a
useful means to introduce systems analysis and expert opin-
ion into our PRA model. Selecting the demand response failure
scenarios that were ranked as at least a medium risk score in
NESCOR’s scoring methodology, we consider the five cyber
security failure scenarios summarized in Table II.
SMITH AND PATÉ-CORNELL: CYBER RISK ANALYSIS FOR A SMART GRID: HOW SMART IS SMART ENOUGH? 443

TABLE II
CYBER FAILURES SCENARIOS INTRODUCED BY INCORPORATION OF DEMAND RESPONSE TECHNOLOGY

Failure Scenario Description Threat Actors Modeling Impact

FS1—Loss of Threat agent compromises demand response (DR) system with Insiders, Criminal Impact of transmission line outages
Situational Awareness custom malware, causing customer system to report false usage and Groups for that node are doubled.
status information to utility. Results in reduced situational awareness
and inhibits a utility’s ability to react proactively, and could increase
the number and duration of failures.
FS2—Local Outage Threat agent injects purpose-built malware into the demand response Terrorist Group, Cost of $3.76 per kWh of demand
Triggered Remotely automation server (DRAS), gaining remote command of the server, Nation States not met [49], taken as highest 4-hour
and blocking or issuing malicious control signals to cause a local or period for that node.
regional outage during peak demand hours.
FS3—Denial of Service Threat agent blocks communications between a demand response Hacktivist, Peak load increase by 2% over
Blocks DR Messages automation server (DRAS) and customer systems. This could be Terrorist Group normal profile (with no demand
accomplished by flooding the communications channel with other response).
messages, or by tampering with the communications channel. These
actions could prevent legitimate DR messages from being received
and transmitted, resulting in increased peak energy usage.
FS4—Price/Meter Threat agent obtains access to the communications channel between Customers, Effective customer prices reduced by
Manipulation the DRAS and the customer DR system, and delivers false Insiders, 1% from nominal values [33],
information to under-report electricity usage or to create artificially Competing Firms reducing revenue for utility.
high prices in the spot power market for financial gain.
FS5—Theft of Private Threat agent compromises DR systems, and then pivots to parts of Criminal Groups Fixed cost of $2.3 k for any node,
Information the information network containing sensitive customer data or based on per capita cost from data
valuable intellectual property, leading to possible fines or breach reports [28]
remediation costs.

TABLE III
COST OF EACH TYPE OF SUCCESSFUL CYBER-ATTACK FOR EACH CUSTOMER
NODE IN SMUD’S NETWORK, BASED ON RECOMPUTING THE DAILY
OPERATING COST SUBJECT TO ADDITIONAL CONSTRAINTS IMPOSED
BY EACH CYBER FAILURE SCENARIO (FS)

Customer FS1 Cost FS2 Cost FS3 Cost FS4 Cost FS5 Cost Net Cost per
Node ($k) ($k) ($k) ($k) ($k) Attack ($k)

C1 1.170 5.250 0.185 3.030 2.3 2.39

C2 1.891 2.336 0.301 1.202 2.3 1.61
C3 1.444 2.237 0.311 1.151 2.3 1.49
C4 1.975 3.961 1.877 2.286 2.3 2.48
C5 1.086 4.621 0.600 3.006 2.3 2.32
C6 0.471 4.706 0.501 3.061 2.3 2.21
C7 4.806 5.257 0.609 3.420 2.3 3.28
C8 5.802 4.737 0.799 4.465 2.3 3.62
C9 2.316 5.296 0.585 3.056 2.3 2.71
C10 3.774 4.027 0.413 2.072 2.3 2.52

The costs of the failure scenarios for each node are computed
by rerunning the economic dispatch model, subject to the addi-
tional constraints imposed by each failure scenario. The average
cost of all failure scenarios then becomes the average cost per
successful cyber-attack to that node, which becomes the nodal
cost parameter ci for use in the MNB model. The net cost per
attack for each node ranges from $1.49 k to $3.62 k, as shown
in Table III.
C. INSIGHTS: HOW SMART AND HOW MUCH TO INVEST?
To identify the optimal level of connectivity, the outputs of
the economic dispatch model—i.e., the benefit curve of Fig. 9
and the cyber-attack costs from Table III—are used as inputs
to the MNB network security model. To complete the network
specification, we consider a time horizon of T = 60 days and a
Fig. 10. Net cyber risk facing the network, accounting for the value of the
optimal defensive probing strategy. The red (upper) curve represents the net
cyber risk that remains after implementing an optimal defensive strategy.
prior probability distribution of gamma(λi ; αi = 2, βi = 2) for
the rate of attacks to each node. At each level of smartness of the
SMUD power network, from 0 candidate nodes connected to all
10, we use the MNB model to determine the value of the optimal
probing strategy of a single cyber defense. Using this strategy,
we evaluate the residual cyber risk facing the network, defined
as the expected value of the cost inflicted by all incoming cyber-
attacks minus the value saved from thwarting attacks. This value
is illustrated in Fig. 10.
Combining this cyber risk curve with the benefit of in-
creased smartness, we assess the tradeoffs as shown in Fig. 11.
These computations show an optimal smartness level where the
marginal benefit of increased connectivity equals the marginal
444
Fig. 11. Tradeoff between the benefits and cyber risk of a smarter grid and
optimum of “smartness”
risk, beyond which point the marginal value of added connec-
tivity decreases. In this case, the optimal “smartness level” is
to connect six nodes with demand response, resulting in a net
increase in daily profit of $7.15 k.
Therefore, the PRA framework developed in this research,
along with the MNB network security model, allowed us to ad-
dress our first research question: How smart is smart enough?
We can also use the model to address our second research ques-
tion: How many teams to hire for cyber security, and how to
allocate their efforts? To do so, we consider a reformulation of
the MNB model, where instead of probing exactly one node
per time unit, we have the option to probe d nodes (by hiring
d cyber defense teams), with d ∈ {0, 1, . . . , n} when there are
n connected nodes. While hiring more teams allows a network
defender to stop more cyber-attacks and achieve a higher sav-
ings value, this value must be weighed against the cost of hiring
more teams to perform the probing, at a cost of $2.88 k per team
per day (based on estimates of a fully loaded annual salary of
$1.05 M for a three-person cyber response team [41]).
We then consider a two-dimensional (2-D) optimization prob-
lem with two decision variables: the number of nodes to connect
(n), and the number of probing teams to hire (d), where d ≤ n
(we cannot probe more nodes than are connected). The net value
at each point is thus a combination of the following:
1) the benefit of a smarter grid;
2) the cost inflicted on the network by cyber-attacks (before
any defensive probing);
3) the value saved by an optimal probing strategy using d
teams;
4) the cost of hiring d cyber defense teams.
In this example, the optimal decision is to connect n = 6
nodes, and hire d = 2 teams, resulting in an increase in daily
profit of $7.85 k, as shown in Fig. 12. Note that solving the
2-D optimization problem resulted in a higher global optimum
($7.85 k increase in profit per day) compared to the 1-D problem,
where we assumed a single defense team (optimal value of
$7.15 k).
IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 65, NO. 3, AUGUST 2018
Fig. 12. Two-dimensional optimization problem showing the net value of a
risk management strategy as a function of the number of connected nodes (n)
and the number of cyber defense teams (d). The pink trace shows the optimal
number of nodes to probe for each level of connectivity. “x” is the optimal point.
The flat area is the infeasible region (we cannot probe more nodes than are
connected).
The MNB network security model, when used as part of a
well-structured PRA framework, can thus allow system opera-
tors for SMUD or any other smart grid network to gain insights
into the questions of how smart to make their network, and how
much to invest in cyber security teams. This type of analysis can
be valuable for a wide variety of electric sector stakeholders. In
addition to utilities looking to maximize expected daily profit,
other entities such as market regulators, load service entities,
or independent system operators can use this model to evaluate
the implications of different policies. For example, if a regulator
were to implement a policy mandating that utilities implement
the maximum amount of cyber security resources, we see that
this can force utilities into negative-profit situations (the “val-
ley” in Fig. 12), where utilities spend more on countermeasures
than the value returned by the investment.
V. DISCUSSION
Summarizing the key results, we have presented a PRA ap-
proach to smart grid cyber security, focusing on the level of
connectivity and the value of the information gained by probing
for threat activity. This probabilistic model allows a systematic
evaluation of the tradeoff between the benefit of smart grid tech-
nologies and the cyber security risks that these new connections
entail. Applying this framework to the case study of the SMUD’s
decision regarding the degree to which to incorporate demand
response into their networks, we found that the optimal strat-
egy is to only connect six of ten possible customer nodes (i.e.,
60% smartness), and employ two cyber defense teams. That is,
“smartness” in the electrical grid is beneficial, but only up to
that point. Above a certain level of connectivity, the marginal
risk of connecting an additional node exceeds the marginal
benefit.
The MNBs model—a novel Bayes-adaptive approach to
network security—provides a powerful framework to model
dynamic network defense strategies under uncertain rates of
SMITH AND PATÉ-CORNELL: CYBER RISK ANALYSIS FOR A SMART GRID: HOW SMART IS SMART ENOUGH?
TABLE IV
SUMMARY OF GENERATOR DATA BY GENERATOR TYPE FOR THE SYNTHETIC POWER NETWORK REPRESENTATION OF SMUD’S POWER SYSTEM
Generator Code Fuel Type Capacity (MW) Number in Network
U12 Solar 12
U20 Hydro 20
U50 Hydro 50
U76 Hydro 76
U100 Wind 100
U155 Natural Gas 155
U197 Geothermal 197
U350 Natural Gas 350
U400 Natural Gas 400
The generator code identifies which generator in the original IEEE 24-bus network these new generators replace.
cyber-attacks. Solving the MNB problem yields a strategy
for the sequential employment of cyber defense teams among
nodes in an organization’s information network, thus protecting
the network against known threats while actively exploring it
to detect and prevent new threats. For simple problems (two
or three nodes, short time horizon T < 15 days), we can solve
for the truly optimal strategy using dynamic programming.
For more complex (and hence more realistic) settings, we need
to use approximate solution methods based on heuristics or
approximate dynamic programming.
We make two final notes on the MNB model. First, the model-
ing power of the MNB framework is not limited to the allocation
of cyber defense teams. In alternative formulations, rather than
a time step of 1 day we could have a time step of t ∼ 1 s (e.g.,
computerized decision support system allocating intrusion de-
tection resources), or even t ∼ 1 month (e.g., long-term planning
to shift organizational resources or plan system upgrades). Sec-
ond, by taking a Bayes-adaptive approach to network security,
the MNB model allows an organization to leverage the techni-
cal ingenuity and expertise of its cyber security professionals to
defend against intelligent adversaries.
Smart grid networks indeed pose unique cyber security chal-
lenges. The Congress of the United States is currently con-
sidering requiring a decrease in the grid’s connectivity level,
essentially unplugging parts of it from the Internet [42]. The
question is: Up to what point? A quantitative analysis such as
that presented here shows that the optimal connectivity can in-
deed be assessed through a risk analysis based on existing attack
data, engineering models, economic analysis, and expert opin-
ion. While it is infeasible to protect against every cyber-attack
vector in systems as complex as the smart grid, this approach
can help enable smart grid stakeholders to prioritize protection
efforts given limited security resources.
APPENDIX
To model the behavior of the SMUD power network, we
utilize the IEEE 24-bus reliability test system, a simple and
well-studied benchmark of a “typical” utility published by IEEE
in 1996 [36]. The original 24-bus test system contains useful
data on generation capacity, power flow limits, transmission
line failure rates, and customer demand data for each of ten
customer nodes. Importantly for our purposes, the test system is,
445
Cost coef. c0 ($/hr) Cost coef. c1 ($/MWh) Cost coef. c2 ($/MW-MWh)
5 631.51 0 0
4 0 19.84 0
6 0 19.84 0
4 0 19.84 0
3 0 13.03 0
4 420.44 23.21 0.05896
3 0 10.43 0
1 769.57 20.61 0.006258
2 1831.09 17.96 0.003351
by design, adaptable to different power system configurations,
allowing us to adjust generation cost and consumer demand
parameters to match the characteristics of the SMUD power
network in a manner suitable to studying the impact of demand
response.
Using generator cost functions and fuel cost data from pub-
licly available sources, we derive the generator mix for the
synthetic SMUD power network, as shown in Table IV. Specif-
ically, the operating cost of each generator is fit to a quadratic
cost function, based on actual operating points for comparable
generators used in California [43], current natural gas fuel prices
[44], recent cost analysis reports for hydro power [45], and the
California Energy Commission’s Cost of Generation Toolkit for
solar, wind, and geothermal generation [46]. Additionally, by
preserving the same nodal generation capacities as the original
IEEE test network, results for the SMUD network may still be
compared with other benchmark studies.
In addition to generator cost functions, the other aspect of
SMUD’s network that we incorporate into the model is the
hourly usage data for end-users. The value of demand response
largely relies on its ability to influence or force users to shift
or reduce their electricity demand from peak to off-peak hours.
Thus, the hourly load profiles play a key role in determining the
benefit of implementing demand response technology. Using
actual hourly energy usage data for the SMUD service area
during the 2012 calendar year from the National Renewable
Energy Lab [47], we obtain the typical aggregate load profile
shown in Fig. 13.
The hourly load profile from Fig. 13, which represents the
aggregate load of the entire power network, is translated to
the individual customer nodes as follows. First, each customer
node is characterized either as a commercial node (where load is
70% commercial, and 30% residential), residential node (30%
commercial, 70% residential), or a mixed node (50% commer-
cial and residential). Introducing three types of customer nodes
allows us to explore how the impact of demand response varies
by end-use sector. Second, the peak load of each customer node
is scaled so that the node’s fractional contribution to the total
system load is the same as in the original IEEE 24-bus network
(as specified in [36, Table V]). The resulting customer load
profiles are summarized in Table V.
To compute the benefit, when customer node Ci in the 24-
node SMUD network is upgraded to a smart node capable of
446 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 65, NO. 3, AUGUST 2018

The impact of a line outage is given by recomputing the daily

economic dispatch algorithm without that transmission line, re-
sulting in an impact ranging from $5140 to $21 126 depending
on the criticality of the link. Multiplying the impact by the daily
probability of failure yields the expected impact. We then as-
sume that the benefit of a demand response node is the sum
of half the expected outage impact of any connected transmis-
sion line. Therefore, nodes that are more connected will achieve
more benefit, due to their enhanced ability to detect and affect
other parts of the network.

REFERENCES
[1] S. Blumsack and A. Fernandez, “Ready or not, here comes the smart
grid!,” Energy, vol. 37, no. 1, pp. 61–68, 2012.
[2] A. Narayanan, “The emerging smart grid: Opportunities for increased
system reliability and potential security risks,” Ph.D. dissertation, Dept.
Eng. Public Policy, Carnegie Mellon Univ., Pittsburgh, PA, USA, 2012.
Fig. 13. Representative daily load profile of the SMUD service area used for [3] M. Hayden, C. Hébert, and S. Tierney, “Cybersecurity and the North
analysis purposes American electric grid: New policy approaches to address an evolv-
ing threat,” Bipartisan Policy Center, Washington, DC, USA, 2014,
TABLE V pp. 33–38.
SUMMARY OF THE TEN CUSTOMER LOADS IN THE SMUD POWER NETWORK [4] M. O. Duff, “Optimal learning: Computational procedures for Bayes-
MODEL adaptive Markov decision processes,” Ph.D. dissertation, Dept. Comp.
Sci., Univ. Amherst, Amherst, MA, USA, 2002.
Customer Node in IEEE Fraction Fraction Peak % of [5] A. Lee, “Cyber security strategy guidance for the electric sector,” Electric
Node 24-Bus Test Commercial Residential Load System Power Research Institute, Palo Alto, CA, 2012.
System (MW) Load [6] Z. Yan, P. Zhang, and A. V. Vasilakos, “A survey on trust management for
Internet of Things,” J. Netw. Comput. Appl., vol. 42, pp. 120–134, 2014.
C1 3 0.5 0.5 362.9 11.4% [7] Q. Jing, A. V. Vasilakos, J. Wan, J. Lu, and D. Qiu, “Security of the Internet
C2 4 0.3 0.7 149.2 4.7% of Things: Perspectives and challenges,” Wireless Netw., vol. 20, no. 8,
C3 5 0.3 0.7 143.2 4.5% pp. 2481–2501, 2014.
C4 6 0.5 0.5 274.2 8.6% [8] J. Hu and A. V Vasilakos, “Energy big data analytics and security:
C5 8 0.7 0.3 344.8 10.9% Challenges and opportunities,” IEEE Trans Smart Grid, vol. 7, no. 5,
C6 9 0.7 0.3 352.8 11.1% pp. 2423–2436, Sep. 2016.
C7 10 0.7 0.3 393.2 12.4% [9] R. S. E. Knapp, Applied Cyber Security and the Smart Grid. Waltham,
C8 13 0.5 0.5 534.3 16.8% MA, USA: Syngress, 2013.
C9 19 0.5 0.5 364.9 11.5% [10] K. P. Schneider, J. C. Fuller, F. K. Tuffner, and R. Singh, “Evaluation
C10 20 0.3 0.7 258.1 8.1% of conservation voltage reduction (CVR) on a national level,” Pacific
Northwest Nat. Lab., Richland, WA, USA, 2010, p. 114.
[11] R. D. Zimmerman, C. E. Murillo-Sanchez, and R. J. Thomas, “MAT-
POWER: Steady-state operations, planning, and analysis tools for power
systems research and education,” IEEE Trans. Power Syst., vol. 26, no. 1,
implementing demand response, the residential and commercial pp. 12–19, Feb. 2011.
loads are shifted by a flexibility factor, representing the fraction [12] M. Parandehgheibi, E. Modiano, and D. Hay, “Mitigating cascading fail-
ures in interdependent power grids and communication networks,” in Proc.
of customers willing and able to shift their energy usage. Based 2014 IEEE Int. Conf. Smart Grid Commun., 2015, pp. 242–247.
on recent analysis of the potential impact of demand response [13] U.S. Department of Energy, Washington, DC, USA, “Freedom of infor-
in the California electricity market [48], we derive flexibility mation request #: HQ-2015-00126-F,” 2015.
[14] Z. Yan, P. Zhang, and A. V. Vasilakos, “A security and trust framework
factors of νC = 3.20% for commercial loads, and νR = 1.35% for virtualized networks and software-defined networking,” Sec. Commun.
for residential. In general, commercial loads are more “flexi- Netw., vol. 9, no. 16, pp. 3059–3069, 2016.
ble” than residential loads, due to their increased diligence to [15] Z. Shu, J. Wan, D. Li, J. Lin, A. V. Vasilakos, and M. Imran, “Security
in software-defined networking: Threats and countermeasures,” Mobile
electricity expenditures and their ability to have on-site tech- Netw. Appl., vol. 21, no. 5, pp. 764–776, 2016.
nologies supporting demand response (e.g., storage devices or [16] F. Hu et al., “Robust cyber-physical systems: Concept, models, and im-
on-site generators that kick in when prices are high). The re- plementation,” Future Gener. Comput. Syst., vol. 56, pp. 449–475, 2016.
[17] R. Deng, G. Xiao, R. Lu, H. Liang, and A. V. Vasilakos, “False
spective loads of any demand response nodes are then shifted data injection on state estimation in power systems—Attacks, impacts,
by a fraction νC or νR from the current to the average value. and defense: A survey,” IEEE Trans. Ind. Informat., vol. 13, no. 2,
The effect is to “flatten” the daily load profile. Recomputing pp. 411–423, Apr. 2017.
[18] N. Liu, J. Zhang, H. Zhang, and W. Liu, “Security assessment for
the economic dispatch algorithm with the new, flatter demand communication networks of power control systems using attack graph
curves will thus result in a cost saving for the utility. and MCDM,” IEEE Trans. Power Del., vol. 25, no. 3, pp. 1492–1500,
To capture the ability of a demand response strategy to mit- Jul. 2010.
[19] T. Sommestad, M. Ekstedt, and L. Nordström, “Modeling security of
igate the impact of unexpected component failures, we intro- power communication systems using defense graphs and influence di-
duce random transmission line failures into the model, with agrams,” IEEE Trans. Power Del., vol. 24, no. 4, pp. 1801–1808,
daily probabilities of failure ranging from 8.22 × 10−4 to Oct. 2009.
[20] A. Hahn and G. Manimaran, “Cyber attack exposure evaluation framework
7.94 × 10−3 , as specified in the original IEEE 24-bus test for the smart grid,” IEEE Trans. Smart Grid, vol. 2, no. 4, pp. 835–843,
system (drawn from transient outage rates in [36, Table 12]). Dec. 2011.
SMITH AND PATÉ-CORNELL: CYBER RISK ANALYSIS FOR A SMART GRID: HOW SMART IS SMART ENOUGH?
[21] N. S. V. Rao, S. W. Poole, C. Y. T. Ma, F. He, J. Zhuang, and D. K.
Y. Yau, “Defense of cyber infrastructures against cyber-physical attacks
using game-theoretic models,” Risk Anal., vol. 36, no. 4, pp. 694–710,
2016.
[22] N. S. V Rao, C. Y. T. Ma, U. Shah, J. Zhuang, F. He, and D. K. Y. Yau, “On
resilience of cyber-physical infrastructures using discrete product-form
games,” in Proc. 2015 18th Int. Conf. Inf. Fusion, 2015, pp. 1451–1458.
[23] Z. M. Fadlullah, Y. Nozaki, A. Takeuchi, and N. Kate, “A survey of game
theoretic approaches in smart grid,” in Proc. 2011 Int. Conf. Wireless
Commun. Signal Process., 2011, pp. 1–4.
[24] Y. Zhu, J. Yan, Y. Sun, and H. He, “Revealing cascading failure vulnera-
bility in power grids using risk-graph,” IEEE Trans. Parallel Distrib. Syst.,
vol. 25, no. 12, pp. 3274–3284, Dec. 2014.
[25] M. Ouyang, L. Duenas-Osorio, and X. Min, “A three-stage resilience
analysis framework for urban infrastructure systems,” Struct. Saf.,
vol. 36–37, pp. 23–31, 2012.
[26] K. Liu and Q. Zhao, “Dynamic intrusion detection in resource-constrained
cyber networks,” in Proc. IEEE Int. Symp. Inf. Theory Proc., 2011,
pp. 970–974.
[27] M. Zhu, Z. Hu, and P. Liu, “Reinforcement learning algorithms for adap-
tive cyber defense against heartbleed,” in Proc. 1st ACM Workshop Moving
Target Defense, 2014, pp. 51–58.
[28] Ponemon Institute, “2016 cost of data breach study: Global analysis,”
Ponemon Inst. Res. Rep., Traverse City, MI, 2016.
[29] P. Henry, J. Williams, and B. Wright, “The SANS survey of digital foren-
sics and incident response,” SANS Whitepaper, 2013.
[30] U.S. Department of Defense, Arlington, VA, USA, “The DoD cyber strat-
egy,” 2015, p. 42.
[31] “Second cyberspace weapon system reaches full operational capabil-
ity status,” 24th Air Force, Air Force Space Command Public Affairs,
Colorado Springs, CO, USA, 2016.
[32] R. Deng, Z. Yang, M. Y. Chow, and J. Chen, “A survey on demand response
in smart grids: Mathematical models and approaches,” IEEE Trans. Ind.
Informat., vol. 11, no. 3, pp. 570–582, Jun. 2015.
[33] U.S. Energy Information Administration, Washington, DC, USA, “Elec-
tricity detailed survey data files,” 2015.
[34] California Energy Commission, Energy Almanac, Sacramento, CA, USA,
“2015 energy supply plans,” 2015.
[35] L. Jimenez, J. Potter, and S. George, “SmartPricing options interim eval-
uation,” SMUD, Sacramento, CA, USA, 2013.
[36] C. Grigg and P. Wong, “The IEEE reliability test system -1996 a report
prepared by the reliability test system task force of the application of
probability methods subcommittee,” IEEE Trans. Power Syst., vol. 14,
no. 3, pp. 1010–1020, Aug. 1999.
[37] D. S. Kirschen, Fundamentals of Power System Economics. Chichester,
U.K.: Wiley, 2004.
[38] M. Hummon et al., “Grid integration of aggregated demand response,
Part 2: Modeling demand response in a production cost model,” National
Renewable Energy Laboratory, Golden, CO, USA, Tech. Rep. DE-AC36-
08GO28308, 2013.
[39] H. G. Kwag and J. O. Kim, “Optimal combined scheduling of generation
and demand response with demand resource constraints,” Appl. Energy,
vol. 96, pp. 161–170, 2012.
[40] A. Lee, “Electric sector failure scenarios and impact analyses,” NESCOR
Technical Working Group 1, Elect. Power Res. Inst., Palo Alto, CA, USA,
2013.
[41] K. J. Soo Hoo, “How much is enough: A risk management approach to
computer security,” Stanford Univ., Stanford, CA, USA, Working paper,
2000, pp. 104–104.
[42] “Lawmakers look to ‘dumb down’ smart grid,” The Hill, 2015.
[43] J. B. Klein, “The use of heat rates in production cost modeling and
market modeling,” Elect. Anal. Office, California Energy Commission,
Sacramento, CA, USA, 1998, pp. 1–124.
447
[44] U.S. Energy Information Administration, Washington, DC, USA, “Natural
gas spot and futures prices (NYMEX),” 2016.
[45] International Renewable Energy Agency, Abu Dhabi, UAE, “Hy-
dropower,” 2012.
[46] California Energy Commission, Energy Almanac, Sacramento, CA, USA,
“Cost of generation report,” 2016.
[47] National Renewable Energy Laboratory, Golden, CO, USA, “Commercial
and residential hourly load profiles for all TMY3 Locations in the United
States,” 2013.
[48] D. J. Olsen et al., “Grid integration of aggregated demand response, Part 1:
Load availability profiles and constraints for the western interconnection,”
Lawrence Berkeley Nat. Lab., Berkeley, CA, USA, LBNL - 6417E, 2013,
p. 101.
[49] K. T. P. Centolella , M. Farber-DeAnda, and L. A. Greening, “Estimates
of the value of uninterrupted service for the mid-west independent sys-
tem operator,” Harvard Elect. Policy Group, Harvard Kennedy School
Government, Cambridge, MA, USA, 2010, pp. 1–49.
Matthew David Smith received the B.S. degree in
physics from the Massachusetts Institute of Tech-
nology, Cambridge, MA, USA, in 2006, the M.S.
degree in electrical engineering from the University
of Southern California, Los Angeles, CA, USA, in
2010, and the Ph.D. degree in management science
and engineering from Stanford University, Stanford,
CA, in 2017.
He is currently serving as an Operations Research
Officer with the U.S. Army, and since 2006 has been
serving in a variety of assignments in the Army’s in-
telligence and research and development communities. His research interests
include the application of mathematical tools and engineering models to en-
hance the resilience of cyber-physical systems against cyber threats.
M. Elisabeth Paté-Cornell received the B.S. degree
in mathematics and physics from the University of
Marseille, Marseille, France, in 1968, the M.S. de-
gree in computer science and applied mathematics
from the University of Grenoble, Grenoble, France,
in 1970, and the M.S. degree in operations research
and the Ph.D. degree in engineering-economic sys-
tems from Stanford University, Stanford, CA, in 1972
and 1978, respectively.
She is the Burt and Deedee McMurtry Professor
with the School of Engineering and a Professor and
Founding Chair (2000–2011) with the Department of Management Science and
Engineering, Stanford University. Her specialty is engineering risk analysis with
application to complex systems (space, medical, offshore oil platforms, etc.).
She has authored more than 100 publications, and is the co-editor of Perspec-
tives on Complex Global Problems (Wiley, 2016).
Dr. Paté-Cornell is a member of the National Academy of Engineering and
the French Académie des Technologies. She was a member of the President’s
Foreign Intelligence Advisory Board from December 2001 to 2008, and of the
board of the Aerospace Corporation (2004–2013) of Draper Laboratory (2009–
2016), and of InQtel (2006–2017).