Вы находитесь на странице: 1из 5

Effect of Load Forecasting Uncertainties on the

Reliability of North American Bulk Power System

Noha Abdel-Karim, Member, IEEE, Elliott J. Nethercutt, Member, IEEE, John N. Moura, Member, IEEE, Thomas
Burgess, Member, IEEE, and Trinh C. Ly Member, IEEE

Index Terms-- Distribution Functions, Load Forecast, Scenario

Analysis, Probabilistic Assessment, and Uncertainty.

HIS paper is motivated by NERCs growing focus
regarding sources of uncertainties in Bulk Power System
(BPS) planning that are multifold and may have potentially
far-reaching effects on overall system reliability. In the past,
only system load forecast was considered to be the main
challenge. More recently, load profiles have changed as
demand has become more elastic, with higher sensitivity to
price signals. This development has been fairly characterized
as a negative generation resource pattern that has supported
increased Demand-Side Management (DSM) adaptation into
the electric system. In particular, the use of smart consumer
electronics, such as compact fluorescent lamps, light-emitting
diodes bulbs, smart thermostats, and other smart appliances that
have changed the behavior of residential load patterns and have
allowed more demand uncertainties to evolve within intra-day,
daily, monthly and annual time frames. Recent literature
discusses incorporating probabilistic and risk assessment
methods into BPS planning and operations [1]-[2]. In [3], the
authors developed a probabilistic, long-term load forecasting
model using hourly load profiles, scenario analysis, and
weather-normalization data to examine the North Carolina
Electric Membership Corporation (NCEMC). Despite point
forecasts, Rob J. Hyndman and Shu Fan proposed a
methodology to forecast long-term peak demand with benefits

978-1-4799-6415-4/14/$31.00 2014 IEEE

of hedging financial risks incurred due to load variability [4].

In addition to load uncertainties and their associated risks, the
BPS is facing a change in its generation fleet, particularly from
coal to natural gas. This shift partially caused by lower
natural gas prices and higher costs of transporting coal has
contributed to a projected 71 GW of fossilfired retirements by
2022 [5]. Figure 1 shows the projected change in the NERCwide generation fleet during the next ten years [6]. On the other
hand, the integration of Variable Generation (VG) resources has
noticeably increased over the past decades, particularly due to
the climate and economic benefits they offer [7], [8]. However,
the variability introduced by adding VG resources to electric
power grids present technological challenges with broad
impacts, ranging from short-term system operations to longterm planning. A recent NERC report focuses on the
approaches taken by the California Independent System
Operator (CAISO) to address emerging risks and ensure
reliability, given the large penetration of VG in California [9].
The solutions being implemented by CAISO builds on existing
NERC Integration of Variable Generation Task Force (IVGTF)
recommendations. In many ways, CAISOs ongoing efforts to
effectively plan and operate a transformed electric grid with
large penetration of VG can be utilized as a model for other
areas to examine and build upon. Accordingly, the report also
addresses the need to better understand expectations of BPS
operational reliability and essential reliability services,
including, but not limited to frequency response, inertial
response, voltage control, and load following capability.
Addressing BPS uncertainty is not something new to system
planners and operators, who have a long history of developed
approaches to cope with load power variability and uncertainty.
Example include generation unit failures, changing in demand
levels, and balancing generation over the time, varying from
minutes to hours. However, increased VG and changing load
shape characteristics, coupled with an uncertain economic
outlook is creating new challenges for planners. This may
require modifying the deterministic approaches previously used
by in NERC assessments, and more reliance on probabilistic
methods to better understand BPS risks and facilitate more
efficient and reliable operation and planning decisions.
Gigawatts (GW)

Abstract-- This paper highlights the importance of demand

uncertainties and their associated risks to system planning and
operations. It presents different approaches to modeling peak
demand uncertainties. The work presented is part of a larger
effort by the North American Electric Reliability Corporation
(NERC) to demonstrate the benefits of applying a probabilistic
approach to all NERC Assessment Areas to better understand
emerging Bulk Power System (BPS) trends and reliability factors
to promote optimum planning. NERC-wide aggregated actual and
forecasted seasonal peak demands, along with their associated
forecasting errors, are presented within this paper. A detailed
individual assessment has been examined, applying statistical
characterization and forecasting models used in a case study
originally released by the Electric Reliability Council of Texas
(ERCOT). For ERCOT, the results show higher forecast accuracy
for the summer season compared to the winter season. This
outcome is due to tail events in the peak demand normal
distribution functions developed and unpredictable weather
patterns. For the purpose of risk assessment validation, a Linear
Predictive Coding (LPC)-based method is used to predict longterm demand. As a result, the developed distribution error has
been utilized to construct scenario analysis for ERCOTs demand
uncertainty. The findings of our analysis suggest the use of more
event-driven outcomes, including extreme weather, unexpected
load increases, and other worst-case scenarios.




Renewable Resources







Pumped Storage





Fig. 1. Cumulative Future-Planned Capacity Change (2014-2023) NERC

LTRA 2013, [6]

The deterministic approaches, currently used in BPS operations

and planning cannot adequately capture all the uncertainties.
Current approaches also lack to accurately consider electricity
market rules needed to adopt and implement more reliable and
cost-effective short- and long-terms market decisions. The
operation and planning reserve costs required to reliably
operate the BPS must be considered. Specifically, the costs
associated with the startup and shut-down of fossil-fired
generators to compensate for the variability caused by increased
levels of VG. Additionally, the costs of load balancing, loadfollowing, and storage and backup supplyall need to be
probabilistically addressed. As part of ensuring system
reliability, NERC recommends that all assessment areas shift
from deterministic to probabilistic approaches [10].
Accordingly, this paper discusses NERC-wide demand forecast
uncertainties. Section II presents annual actual and forecasted
peak demand uncertainties, while Section III shows a statistical
analysis models and examines a prediction approach on
ERCOTs seasonal peak demand. It also applies a scenario
analysis model approach that specifies cases of extreme
weather and system condition. Section IV concludes the
findings and provides recommendations.
NERC prepares seasonal and long-term reliability assessments
that examine the overall reliability and adequacy of the North
American BPS. The NERC footprint is divided into assessment
areas, both within and across the eight Regional Entity
boundaries. The purpose is to evaluate, trend, and assess system
reliability through collecting and analyzing data and
information provided by each Regional Entity, including
forecasts for on-peak demand and energy, as well as projections
for demand response, capacity, and transmission. To support
the 10-year projections of the Long-Term Reliability
Assessment (LTRA), , NERC issues biennial Probabilistic
Assessments (ProbA) to support the enhancement of resource
adequacy metrics provided in the LTRA. Historically, NERC
has gauged resource adequacy through the use of a
deterministic assessment metric- planning reserve margin
(PMR). NERC defines PRM as a measure of available capacity
over and above the capacity needed to meet a normal (50/50)
peak demand forecast. However, because one of the largest
drivers of the PRM is load forecast uncertainty, which has
become more complicated in recent years, the focus of this
paper is addressing NERC-wide load forecast uncertainties.
A. Load Forecast Uncertainty Models
NERCs 2012 Probabilistic Assessment, [10]-[11], reports the
different models and assumptions used by each NERC
Assessment Area. The Florida Reliability Coordinating Council
(FRCC), Midwest Independent System Operator (MISO),
Manitoba, SaskPower, Northeast Power Coordinating Council
(NPCC) and PJM Assessment Areas use weather and economic
parameters to develop forecast uncertainty models. Conversely,
ERCOT, SERC Reliability Corporation and the Southwest
Power Pool Regional Entity (SPP) use a load forecast model
that incorporates weather parameters.

B. Actual Vs. Forecasted Seasonal Peak Demand

Two key tools used by operators and system planners include
load and forecast uncertainty models. In general, the difference
between actual and forecasted values is defined as a measure of

x x

Where is the difference between the true values of a vector

x and the predicted values, x . Note to mention that Eq. (1)

could be also applied to a single data point and not just a vector
of a time series representing certain type of data. It also implies
that a positive variation, , which means that the actual values
are greater than the forecasted ones, which is a case of an
underestimated forecast model. Likewise, negative variation
implies a case of overestimation in which the forecasted values
exceed the actual values.
Figure 2(a) shows NERC-wide annual actual and forecasted
peak demand for the summer season, with percentage
variations, while Figure 2(b) shows NERC-wide actual and
forecasted peak demand for the winter season. The data have
been consolidated from all assessment areas and aggregated to
represent the NERC-wide annual variations from actuals.
It is well understood that the aggregation of NERC-wide peak
demand results in an overall error compensationsome areas
that suffer overestimation while others suffer underestimation.
Figure 3 shows summer and winter forecast errors by Regions.
The peak demand in areas like the Midwest Reliability
Organization (MRO), WECC and NPCC with footprints in the
U.S and Canada, are shown separately.


Figure 2. NERC-wide Demand Uncertainty: a) Summer Peak, b) Winter Peak.

actual and forecasted peak demand with representative

percentage errors from 1993-2011.


The percentage forecast errors have been calculated by using

Equation (2) for both summer and winter peak demands as
Peak DemandS Pea k DemandS

Peak DemandS

Where Peak DemandS stands for actual peak demand in a

certain season S, and Peak DemandS stands for the forecasted
peak demand in a certain season S.
Some Regions are observed as experiencing higher forecasting
error in the winter than the summer season (e.g., ERCOT, SPP
and FRCC). Other Regions have a higher forecasting error in
the summer than the winter.
Therefore, it is important to understand the behavior of
uncertainties through statistical and probabilistic assessment.
The following section presents the ERCOT Region as a case
study of applying a comprehensive analysis to understand the
behavior of peak demand patterns and model uncertainties at
different time frames of interest.

Figure 4. ERCOT Actual and Forecasted Peak Demand with Percentage Error
for: a) Summer Season, b) Winter Season.

The winter forecast error is higher than the summer, due to the
dominating weather parameter in the load model and the hardto-predict variability of demand during the winter. Therefore, it
is important to understand the statistical characterization of
demand and develop a model that best fits its pattern.
A. Statistical Characterization of Peak Demand Uncertainty
Given its importance, and for completeness, a statistical
representation of ERCOTs peak demand has been developed.
In Figure 5, the load duration curves peak demand are presented
for four seasons. The upper figure shows the hourly load
duration curves for the summer, winter, and shoulder seasonal
consumption levels. The lower plot represents the hourly peak
demand from 2006-2012. The greatest load appears during the
summer season, while lowest capacity utilization rate appears
in shoulder months. Hourly peak data has been gathered from
Ventyx, Velocity Suite [13].

x 10

Summer Peak Curve: May - Sept.2012
Winter Peak Curve: Dec.2011 - Feb. 2012

The uncertainty associated with load power has always been

one of the major sources of uncertainty. Seasonal load
variations affect planning decisions on different time horizons.
This section presents the probabilistic assessment applied to
ERCOT Region. The ERCOT load forecast development
process is described in the 2012 longterm demand and energy
forecast document [12]. Methodology, assumptions, and data
used in creating the forecast are presented. ERCOT is a
summerpeaking region. The average weather profile (50/50) is
used for the ERCOT load forecast. Figure 4 shows historical

P M (MW)


Shoulder Peak Curve: March - Apr. 2012

Shoulder Peak Curve: Oct. - Nov. 2012

Figure 3. NERC-Region Percentage Errors for, a) Summer Peak Demand and
b) Winter Peak Demand.

x 10















x 10

Historical ERCOT's Peak Load Duration Curve 2006 - 2012



x 10

Figure 5. Peak Load Duration Curves for ERCOT Region.



= 32,743
= 4,122

= 43,593
= 9,604

for N prediction order. As a result of applying the prediction

algorithm, Figure 8 shows the error distribution of hourly peak

= 2 ,5 8 8 MW
= 2 ,3 2 4 MW


= 32,720
= 5,350

= 32,560
= 5,480






MW Peak Hourly Error


x 10



x 10



x 10





x 10



x 10


= 36,104
= 8,822








x 10

Figure 6 Hourly Peak Load Histogram Curves & Normal Probability Plots for
2012 ERCOT Region: a) Summer, b) Winter, c) Shoulder Month March-Apr.,
d) Shoulder Months Oct.-Nov. e) Total 2006 2012.

x 10


Peak MW Demand

eN (n) x(n) x (n)
Where b is the prediction coefficients, e N is the prediction error





i 1


x 10

Figure 8. Peak Load Forecasting Error.

MW Peak
Hourly Error

Freq. Distribution & Normal Probability

Density Function Fit


x(n) bi,N x(n i) eN (n)


B. Peak Demand Distribution Model

Load is best characterized by a normal distribution model.
ERCOTs seasonal load profile has been represented by normal
fit probability distribution model with different time frames of
interest. Figure 6 shows seasonal and total load histogram plots,
with a near-normal distribution of load obtained for all seasons
at different means and variances. Descriptive characteristics,
such as the 25th and 75th quartiles, median, minimum,
maximum, and outliers are presented in the box plots provided
in Figure 7. As shown, the winter load profile is more skewed
than the summer with outliers. This corresponds lower forecast
accuracy in winter compared to the summer, shown in Fig. 4(b).

Peak Demand Scenario Analysis
analysis is the most informative way in assessing risks
and sensitivities associated with sources of uncertainties.
than computing
2010 an expected
2011value of peak
2012 load, it
represents different possible outcomes and computes the worst
case scenario and its risk implications. By taking generated
error distribution model and evaluating the mean, maximum
and minimum error values, high, low, and medium forecasting
scenarios are generated. The high error scenario represents the
worst-case outcome over which the system may incur the
highest peak demand in a certain period of time. Figure 9 shows
a summer peak demand scenario analysis for ERCOT. This
Scenario was developed using the stochastic representation of
forecasted error, generated using an hourly summer peak
demand data for years 2009-2012. In Figure 9, the worst-case
scenario is given by the high forecast error case that incorporate
risks of extreme demand, due to weather or economic impacts.
Similarly, a best-case scenario is produced using a minimum
forecast error. Finally, the expected demand scenario is
predicted based on ERCOTs normal demand forecast.



Shoulder Mar.-Apr

Shoulder Oct.-Nov.


Figure 7. ERCOT Box Plots of Hourly Peak Load for 2012: a) Summer, b)
Winter, c) Shoulder Month March-Apr., d) Shoulder Months Oct.-Nov, and e)
Total 2006 2012.

C. Peak Demand Long Term Prediction Model

Besides the probability density functions and the statistical
models for incorporating uncertainties into load models, the
stochastic and predictive models provide a more informative,
risk-based approach that ultimately results in better resource
and transmission planning. This sub-section includes a
description of results from ERCOTs long-term demand
prediction model, along with a scenario analysis of considering
worst-case prediction events for long-term planning decisions
[14], [15].
A Linear Predictive Coding (LPC), based on the autocorrelation
method, is applied to ERCOTs hourly peak demand to
determine the coefficients of a forward linear predictor.
Prediction coefficients are calculated by minimizing the
prediction error in the least squares sense [16]. The method
provides the LPC predictor and its prediction error as follows:

Figure 9. Scenario Analysis using Summer Peak Load Forecasting Errors

This paper presents load forecast uncertainty in the NERC
footprint. In particular, it studies the uncertain behavior and
characteristics of demand power using distribution models and
representations of tail events. It then applies a well-defined
linear prediction modeling technique to the ERCOT Region as
part of NERCs probabilistic assessment efforts of identifying

the leading indicators of system stress and determining where

system vulnerabilities exist. Furthermore, the paper presents a
scenario analysis on load forecast uncertainty. Finally, the
paper concludes that only limited information can be
determined by a deterministic PRM; therefore, NERC should
consider increasing the use of probabilistic-based approaches in
future load forecast and resource adequacy assessments.











Billinton, R.; Huiling Bao; Karki, R., "A Joint Deterministic Probabilistic Approach to Bulk System Reliability Assessment,"
Probabilistic Methods Applied to Power Systems, PMAPS '08, 10th
International Conf., pp.1-8, May 2008.
Wenyuan Li, Probabilistic Transmission Planning, Power and Energy
Magazine, IEEE, Oct. 2007.
] Hong, T.; Wilson, J.; Xie, J., "Long Term Probabilistic Load Forecasting
and Normalization With Hourly Information," IEEE Transactions on
Smart Grid, vol.5, no.1, pp.456 - 462, Jan. 2014.
Hyndman, R.J.; Shu Fan, "Density Forecasting for Long-Term Peak
Electricity Demand," IEEE Transactions on Power Systems, vol.25, no.2,
pp.1142 -1153, May 2010.
U.S. Energy Information Administration, Annual Energy Outlook with
Projections to 2040, Report, April, 2013 available online:
NERC, Long Term Reliability Assessment 2013, available at NERC
website, <www.nerc.com>
AWEA U.S. Wind Industry Annual Market Report Year Ending 2010.
Energy Information Administration, EIA Energy Annual Outlook, 2011.
NERC Special Reliability Assessment: Maintaining Bulk Power System
Reliability While Integrating Variable Energy ResourcesCAISO
NERC Probabilistic Assessment Addendum to the 2012 Long-Term
NERC 2012 Long-Term Reliability Assessment, available at:
ERCOT Planning 2012 Long-Term Demand and Energy Forecast
Ventyx, Velocity Suite https://velocitysuite.ventyx.com
Abdel-karim, N., Small, M. and Ilic, M. Short Term Wind Speed
Prediction by Finite and Infinite Impulse Response Filters: A State Space
Model Representation Using Discrete Markov Process, 2009 IEEE
PowerTech Conf., Bucharest, Pg. 1-8.
Noha Abdel-Karim, Marija Ilic, Multi-scale Models for Decomposing
Uncertainties in Load and Wind Power, Engineering IT-Enabled
Sustainable Electricity Services Power Electronics and Power Systems
Book, Volume 30, 2013, pp 171-208, Springer Publication.
P.P. Vaidyanathan, The Theory of Linear Prediction, California
Institute of Technology, 1st ed., Morgan & Claypool, 2008.

Noha Abdel-Karim is a Senior Engineer of
Reliability Assessment responsible for processing and
analyzing NERCs probabilistic assessment reports,
supporting and leading several task forces; Integration
of Variable Generation Task Force (IVGTF),
Geomagnetic Disturbance Task Force (GMDTF), and
Essential Reliability Services Task Force (ERSTF) to
assess overall system reliability. Prior to joining
NERC, Noha worked as a consultant at Pyramid
Consulting International in implement new
operational strategies and identify cost cutting ideas at a fossil fuel plant to
foster an environment of continuous improvement, providing a competitive

advantage to the plant in its designated electricity market while at the same time
ensuring compliance with safety and environmental regulations. She worked as
a post-doctoral fellow at Carnegie Mellon University on developing operation
and planning uncertainty models of electricity systems characterized by large
scale integration of renewable energy. Noha received her Doctor of Philosophy
degree in 2012 from the Engineering & Public Policy Department at Carnegie
Mellon University, B.Sc. and M.Sc. degrees in 2002 and 2006, respectively, in
power systems from the Arab Academy of Science and Technology (AAST) in
Elliott J. Nethercutt is a Senior Analyst responsible
for collecting, processing, and analyzing data for the
NERC reliability assessment reports. Additionally,
Elliott supports the data, analysis, and informationsystem needs for evaluating and reporting on NERC's
bulk power system reliability assessment reports.
Prior, he worked in the Office of Electricity Delivery
and Energy Reliability at the U.S. Department of
Energy. Elliott received an Economics degree from
the University of Colorado at Boulder in 2007 and a
Master of Applied Economics from the Johns Hopkins University in 2012.
John N. Moura is the Director of Reliability
Assessment for NERC where. He leads the electric
reliability organizations efforts to independently
assess and report on the overall reliability, adequacy,
and associated risks of the interconnected North
American bulk power system. NERCs long-term and
continent-wide deterministic and probabilistic
reliability indices for resource adequacy, transmission
planning, as well as comprehensive evaluations of
emerging reliability risks.
Thomas Burgess is the Vice President and Director
of Reliability Assessment and Performance Analysis.
Before joining NERC, he worked at FirstEnergy
Corporation, serving as Executive Director,
Integrated System Planning and Development, where
he was responsible for developing strategic plans that
maximize value for the transmission and generation
business segment, as well as developing integrated
planning strategies that optimize transmission and
generation. Prior, he worked for the Ohio Edison
Trinh Ly is a Junior Engineer of Reliability
Assessment at NERC responsible for collecting,
processing, and analyzing data for NERCs reliability
and special assessment reports. Trinh support data
analysis, information-system needs of NERCs BPS
reliability assessment reports, and provide staff
support to related working groups.