Lance Legel TLEN 5830 Automated Load Forecasting: Algorithms, Adoption Spring 2012 I.

I. INTRODUCTION Prediction of load on energy systems is a core task of utilities and operators around the world. Accurate prediction of loads increases the reliability and cost-efficiency of the power grid, and minimizes resource consumption and environmental impact: spinning-reserves can be scheduled with greater utilization, while automated systems can detect unusual patterns from load profiles to identify and prevent security issues in real-time [1]. It is estimated that a 1% decrease in forecasting error for a 10 GW electric utility can save up to $1.6 million annually [2]. This value proposition has led pioneers to develop advanced load prediction techniques, like neural networks and fuzzy expert systems, for about two decades now [3,4]. Meanwhile, cutting-edge research and technology make adoption of automated load forecasting more potent and economical than ever. Little survey research has been done recently on the adoption and performance of these techniques [5], and there is no dossier covering implementation of these technologies by the 124 balancing authorities listed in the United States [6]. This investigation thus seeks to answer two questions regarding automated load forecasting in the United States: 1. What algorithmic implementations are best for real-world load forecasting? 2. What load forecasting technologies are energy operators using in the United States? To answer the first question, I do an extensive literature review using Google Scholar, starting with a previous technical literature survey from 2002 [5], continuing with work that has done since this survey, and concluding with analysis of industry white papers and documents. To answer the second question, I do case studies of five Independent Systems Operators (ISOs), analyzing documents on their automated load forecasting strategy and publicly available reports. I discuss what companies solutions they are employing. I analyze what algorithms and implementations they are using and how they may or may not be optimal for their circumstances. Page 1 of 10

Lance Legel TLEN 5830 Automated Load Forecasting: Algorithms, Adoption Spring 2012 Before proceeding to the specific algorithm types of interest in load forecasting, we must briefly cover the types of load forecasting performed. In Table 1, we see that load forecasting has short, medium, and long-term varieties. There is a diversity of inputs required to successfully make accurate prediction for each of these varieties. This diversity enables a depth of possible forecasting approaches, ranging from short term inclusions of weather, to medium term preparation for events and holidays, to long-term consideration of economic variables like GMP (gross metropolitan product). The structure of how these variables influence the real outcomes dictates the appropriateness of using certain algorithm types over others. Table 1 Time Horizon Outputs Example Inputs Short-Term 1/12-24 Hours Load Curves Cooling Degree Days Heating Degree Days Current Demand Hour of Day Cloud Cover Humidity II. ALGORITHM TYPES We can reasonably distinguish the algorithms used for load forecasting into three types: statistical time-series, machine learning models, and fuzzy expert systems. Subsets from each have produced strong results in the face of real-world load data, and all are actively in use today. I will present in detail the technical foundation and limitations of regression models, time series models, neural networks, and fuzzy expert systems. Meanwhile, we should remember that some of the most successful models for load forecasting are based on hybrid ensembles of diverse techniques, with weighted outputs for making final decisions. Medium-Term 24 Hours - 8 Weeks Load Curves / Capacity Month Previous Year Data Community Events Holidays Seasonal Trends Similar Data Long-Term 8 Weeks - 20 Years Capacity Demographics Employment Outlook GDP Outlook Historical Trends Retail Price New Efficiencies

Page 2 of 10

Lance Legel TLEN 5830 Automated Load Forecasting: Algorithms, Adoption Spring 2012 II.1. Statistical Time Series Statistical time-series methods range from very simple applications of linear regression to implementations of an autoregressive integrated moving average (ARIMA). We will cover two major types regression and time series models and clarify their relative performance and suitability in solving load forecasting problems. II.1.A. Regression Models John von Neumann, who was regarded as one of the greatest mathematicians in modern history, once said skeptically of multivariate regression: With four parameters I can fit an elephant, and with five I can make him wiggle his trunk [7]. Multivariate regression is indeed a flexible paradigm that is relatively easy to implement. It is based upon identification of some number of endogenous variables that an exogenous variable (i.e. load at a specific time) can be determined to linearly depend upon. While regression models are indeed quite common in load forecasting, Kyriakides and Polycarpou show how there are inherent non-linearities in the factors affecting electrical load [8]. This makes it especially difficult to develop an accurate model. Tests have shown that models deteriorate in performance when the load deviates due to sudden weather changes and load events [9]. So it is not possible to represent the load pattern during certain time periods using a simple linearized model: regression-based approaches require at least some additional engineering or coupling with other models. Advanced regression-based models that incorporate non-linearity have been shown to produce interesting results generally for medium-term load forecasting. One regression approach incorporated weather-dependent variables of temperature, wind speed, rainfall, relative humidity, and hours of sunshine, in addition to the economic factor of gross domestic product (GDP), to produce a mean absolute percentage error (MAPE) of 1.98% on data from 1996 to 2003 [10].

Page 3 of 10

Lance Legel TLEN 5830 Automated Load Forecasting: Algorithms, Adoption Spring 2012 II.1.B. Time Series Models Time series approaches assume that the data follows static patterns, leveraging methods such as autocorrelation and seasonality-fitting. Various approaches include stochastic models, autoregressive moving average (ARMA) and derivations thereof, and state space models. In general, time series prediction seeks to model the load as the sum of two terms, ( ) where ( ) ( )

( ) is derived from time of day and normal weather patterns for the day, seasonally,

while ( ) incorporates data on the current days deviations [11]. We can model ( ) as, ( ) ( ) ( ) ( )

where the first term represents inputs depending on the current load data, middle term represents inputs that depend on weather, and the last term represents uncertain effects that depend on random inputs. The parameters , , along with the summation integers are fitted through

numerical methods using current load and weather data [11]. Among the large variety of time series approaches implemented over the past three decades, most give satisfactory results, and some show unique strength in normal circumstances. The best time series models found reported on short term load data were for hot days between Sunday and Wednesday, with a MAPE of 1.48%; this model performed with 1.99% MAPE on cold days and public holidays [12]. However, like regression models, time series approaches generally fail to adapt well to new circumstances based on new underlying forces, such as environmental or social factors. They have been shown to be numerically unstable under new conditions [8]. This makes it necessary for them, too, to be coupled with other methods. So statistical time series methods, in general, are strong but incomplete solutions to load forecasting.

Page 4 of 10

Lance Legel TLEN 5830 Automated Load Forecasting: Algorithms, Adoption Spring 2012 II.2. Machine Learning Models: Neural Networks Machine learning algorithms are designed to learn correlations automatically in diverse datasets. The models they produce differ from traditional statistical approaches in that they make fewer assumptions about the structure of the underlying domain. They are, by comparison, black boxes because the basis for their predictions cannot typically be translated into real-world considerations. Nonetheless, they have been proven to perform very well, especially in learning structures from uncertain and spontaneous patterns, relative to statistical time series methods. There are many types of machine learning models that have been adapted for load forecasting, including neural networks [13,14,15], genetic algorithms [16], and support vector machines [17]. I will present the most popular machine learning application to load forecasting: neural networks. Neural networks are comprised of neuron-like nodes that develop weighted relationships reflecting the weights of real statistical correlations. In a standard neural network, there is an input layer of nodes, with each node corresponding to data attributes; the input layer is connected to one or more hidden layers of nodes, which are numbered in an amount that optimizes overall performance, and not tied to any explicit knowledge of the domain; and finally there is an output layer in our case, likely to be a single node with optimized weight i , where i represents the current time being forecasted for. The value of i , which may technically range from 0 to 1, translates into a specific quantity of MW predicted of load. In one example of using neural networks to forecast the next hour of load, Hipper et al. found that neural networks with many input and output nodes, and multiple hidden layers, performed exceptionally well with a MAPE of about 2.5% and robustness in handling spreading of errors [18]. Neural networks have been successful in handling holidays [9] and non-linear weather [19], and are known to be more robust

Page 5 of 10

Lance Legel TLEN 5830 Automated Load Forecasting: Algorithms, Adoption Spring 2012 than statistical time-series techniques in the face of sudden uncertainties [7]. A review of different neural network topologies used for load forecasting is detailed in [20]. Meanwhile, neural networks are known to have specific limitations that are not often discussed by researchers in applied domains. One is the long training time on large data sets for most forms of neural networks. It has been my experience that training a neural network on a data set larger than 100 MB requires at least 15 hours of computation on a machine of about 2.5 GHz. However, certain topologies have been shown to train ten to one hundred times faster [21]. Another problem is that neural networks are prime examples of black boxes. Administrators can have essentially zero insight into the mechanical decision making process that these systems go through. So there simply needs to be a level of trust between the administrator of the system and the system itself, just as there may be trust between an administrator and a human expert. Finally, neural networks are known to be wrong in the face of highly skewed/biased data. This means that if the system records that for 85% of all days with certain measures of weather, season, etc. the load is measured to be within some small range of values, but on 15% of those days the load significantly deviates for some unknown reasons, then the network will not generally respond with high accuracy for the 15% minority, unless it has been specially designed to spot this minority. Dealing with skew is a well-known problem in the discipline of machine learning, and some researchers have remarked that the problem lies more in representation of the data (i.e. how does a teacher explain something best?) than errors in computation [22]. It is worth pointing out that neither humans nor other systems can typically predict such minority events with accuracy, either. So neural networks are actually favored in the face of extreme data due to their adaptive abilities [7]. Because neural networks can become specialists at predicting unusual circumstances, they may be best within an integrated forecasting ensemble.

Page 6 of 10

Lance Legel TLEN 5830 Automated Load Forecasting: Algorithms, Adoption Spring 2012 II.3. Expert Systems Expert systems are considered a third, distinct category of forecasting algorithms. Unlike statistical time series and machine learning techniques, expert systems feature a large amount of human intervention and are much more transparent. We will discuss a particularly advanced type of expert system, explaining how it is formed in highlighting its strengths. Expert systems are based upon IF-THEN rules built into traditional programming paradigms. These rules are rigorously extracted from real human experts through interviews and sometimes supervised system learning of the mathematics underlying human decisions. They may correlate any variety and depth of binary or quantifiable variables into the logic. For example, one human expert may be able to articulate the following rule, after intense self-scrutiny:

The rule above is really one that defines a quantitative relationship between variables. Such rules can be effective when they are very carefully tuned to real-world data. In a fuzzy expert system, the logic that interprets all the rules together applies a probabilistic interpretation in formulating outputs. For example, if two rules are considered active, but they are slightly conflicting, fuzzy logic can integrate these two by standardizing the outputs into one common unit (i.e. MW of load) and applying weights to each based upon the likelihood of each rule. Fuzzy expert systems have the advantage of being highly configurable and open to skepticism. When paired with active human administrators, systems can be increasingly tuned for the unique circumstance underlying their regional energy needs. Administrators can request that the software explains why it choose certain values, and consciously choose to trust this, or not. However, fuzzy experts can be as dumbfounded as real experts during departures from the norm.

Page 7 of 10

Lance Legel TLEN 5830 Automated Load Forecasting: Algorithms, Adoption Spring 2012 III. ADOPTION CASE STUDIES To provide a real-world understanding of how large energy utilities and operators are handling the challenge of load forecasting, I go through case studies on five major ISOs and discuss their specific strategies in modeling their regions. The examples provided here are not comprehensive of all the different strategies currently employed in industry, but they do reflect common patterns, and illustrate the diversity of approaches possible for dealing with what are similar (but not necessarily identical) technical problems. III.1. ISO New England (ISO-NE) ISO New England operates models for long-run, peak, and short-term forecasts. Their long-run model is based on autoregression in conjunction with economic and weather variables by state [23]. It includes consumer purchases of more efficient appliances in response to higher energy prices, gradually anticipated over time. Key parameters also include year, price elasticity of energy, real personal income, cooling degree days, heating degree days, and a stochastic residual. The peak model is a monthly prediction of the typical daily peak for each month, and produces forecasts of weekly, monthly, and seasonal peak loads over the next 10 years, by state. Peak loads are modeled as a function of energy consumption, weather, and intercepts and slope shifters for weekends and holidays [24]. A trend-weather variable is included in the regression equation to capture the sensitivity of peak load in the summer months due to use of cooling. ISO-NEs hourly forecast model is an accumulation of multiple software programs supervised by a human forecaster. The model starts by inputting actual weather observations and demand data, then running the Similar Days Load Forecast (SimDay), developed by PRT, Inc. Then the forecaster runs Itrons Metrix ND software [25], which automatically uses the effective temperature during heating months or humidity index during summer months to generate hourly

Page 8 of 10

Lance Legel TLEN 5830 Automated Load Forecasting: Algorithms, Adoption Spring 2012 demand. The Metrix ND program calculates load over a 7 day period, and its outputs are compared to the SimDay program, as well as a neural network that is run independently. One neural network is run with fast learners that emphasize recent load and weather data, and another that balances historical data evenly. The forecaster uses the output of all these models to determine a final load forecast, with weights to each of the outputs based upon discretion. III.2. Energy Reliability Council of Texas (ERCOT) ERCOTs long-term demand and energy forecast is based on a set of econometric and neuralnetwork models. The hourly load in each of 8 regions is described as a function of certain economic variables, such as nonfarm payroll employment, and weather variables, like heating and cooling degree days [26]. Economic and demographic data, including county-level forecast, are obtained on a monthly basis from Moodys Economy.com. Historical monthly economic and demographic data for each county are used back from 1990. Dynamic regression models are employed like the Box-Jenkins, which is based on autoregressive moving average. One neural network used by ERCOT spits out hourly forecast values based upon variables relating to dry bulb temperature, sunset time, year, and ratios of current use to monthly average. Another neural network is trained based upon the month, day of the week, and hour of the day for each of 8 weather zones identified in Texas, and all outputs are integrated by a forecaster. III.3. California ISO CAISO bases its forecasting models on 5 climatic zones: PGE Bay Area and Non-Bay Area, SCE Coastal and Inland, and SDGE. Forecasts are summed to obtain the total ISO forecast. They are based on a combination of neural network and regression models, utilizing about 25 calendar and 25 weather variables per zone [27]. Inputs to the model include hourly weather forecasts from 24 weather stations, specifying temperature, dew point, wind speed, and cloud

Page 9 of 10

Lance Legel TLEN 5830 Automated Load Forecasting: Algorithms, Adoption Spring 2012 cover; as well as half-hour zonal loads. Forecasts adapt every half-hour to load forecast error. CAISO also uses a Very Short Term Load Forecast Predictor (VSTLP) based on neural networks that takes five-minute averages of load over the last 13 months as input [28]. III.4. PJM PJM uses hourly meter data to create monthly and seasonal forecasts for multiple weather zones, regions, and the whole operating system combined [29]. They employ multivariate regressions, which consider calendar effects, weather effects (similar to those previously identified), economic drivers such as the gross metropolitan product, and random possibility inclusion from Monte Carlo simulations. III.5. Southwest Power Pool SPP uses a very short term load forecast for emergency management, based on telemetry data, predicting one hour ahead in five minute intervals. They use a slightly broader short term load forecast that incorporates weather and SCADA data to output an hourly load forecast. The model updates its output hourly, and retrains itself on a daily basis from its new input variables. The two models are based on neural network and economic dispatch methodologies [30]. IV. CONCLUSIONS I have covered the major types of algorithms for automated load forecasting, and presented examples of how these algorithms are used today. The takeaway is that most real-world implementations integrate multiple approaches into ensembles, leveraging the strengths of each. The pattern that emerges is one where a human forecaster will work with multiple software programs, using statistical time series and expert systems to handle the majority of predictions, while continuously consulting with neural networks and machine learning algorithms to quickly detect and respond to unusual circumstances.

Page 10 of 10

Lance Legel TLEN 5830 Automated Load Forecasting: Algorithms, Adoption Spring 2012 V. REFERENCES [1] Kodogiannis, V.S., Anagnostakis, E.M. Soft computing based techniques for short-term load forecasting. Fuzzy Sets and Systems, 128:3, 413-426 (2002). [2] Hobbs, B. F., et al. Analysis of the value for unit commitment of improved load forecasting. IEEE Transactions on Power Systems, 14:4, 1342-1348 (1999). [3] Lu, C.N. Neural network based short term load forecasting. IEEE Transactions on Power Systems, 8:1, 336-342 (1993). [4] Hsu, Y.Y. Fuzzy expert systems: an application to short-term forecasting. IEEE Proceedings on Generation, Transmission and Distribution, 139: 6, 471-477 (1992). [5] Metaxiotis, K. et al. Artificial intelligence in short term electric load forecasting: state-ofthe-art survey for the researcher. Energy Conversion and Management, 44:9, 1525-1534 (2002). [6] North American Electric Reliability Corporation. Balancing Authority Acronym List. Available online at: http://www.nerc.com/filez/ctrlarealist.htm [7] Gibbons, C. Understanding Multivariate Regression. University of California Berkley Economics 140 (2009). [8] Kyriakides, E., Polycarpou, M. Short term electric load forecasting: A tutorial. Trends in Neural Computation, Studies in Computational Intelligence, 35:16, 391-418 (2007). [9]. Papalexopoulos, A. D., Hao, S., Peng, T. M. An implementation of a neural network based load forecasting model for the EMS. IEEE Transactions on Power Systems, 9:4, 1956-1962 (1994).

Lance Legel TLEN 5830 Automated Load Forecasting: Algorithms, Adoption Spring 2012 [10] Hor, C. L., Watson, S. J., Majithia, S. Analyzing the impact of weather variables on monthly electricity demands. IEEE Transactions on Power Systems, 20:4, 2078-2085 (2005). [11] Gross, G., Galiana, F. D. Short-term load forecasting. Proceedings of the IEEE, 75:12, 1558-1570 (1987). [12] Hahn, H., Meyer-Nieberg, S., Pickl, S. Electric load forecasting methods: Tools for decision making. European Journal of Operational Research, 199, 902-907 (2009). [13] Park, D. C., et al. Electric Load Forecasting Using An Artificial Neural Network. IEEE Transactions on Power Systems, 6:2, 442-449 (1993). [14] Chen, S. T., et al. Weather sensitive short-term load forecasting using nonfully connected artificial neural network. IEEE Transactions on Power Systems, 7:3, 1098-1105 (1992). [15] Xiao, Z. et al. BP neural network with rough set for short term load forecasting. Expert Systems with Applications, 36:1, 273-279 (2009). [16] Yasuyuki, G. et al. Daily Peak Load Forecasting by Structured Representation on Genetic Algorithms for Function Fitting, Transactions of the Institute of Electrical Engineers of Japan, 119-B:6, 735-736 (1999). [17] Vapnik, V. N. An overview of statistical learning theory. IEEE Transactions on Neural Networks, 10:5, 988-999 (1999). [18] Hippert, H. S. et al. Large neural networks for electricity load forecasting: Are they overfitted?. International Journal of Forecasting, 21, 425-434 (2005).

Lance Legel TLEN 5830 Automated Load Forecasting: Algorithms, Adoption Spring 2012 [19] Chow, T. W. S., Leung, C. T., Neural network based short-term load forecasting using weather compensation. IEEE Transactions on Power Systems, 11:2, 858-863 (1996). [20] Hippert, H. S. et al. Neural networks for short-term load forecasting: a review and evaluation, IEEE Transactions on Power Systems, 16:1, 44-55 (2001). [21] Hayati, M. Short Term Load Forecasting Using Artificial Neural Networks for the West of Iran. Journal of Applied Sciences, 7, 1582-1588 (2007). [22] German, S., Bienenstock, E., Doursat, R. Neural Networks and the Bias/Variance Dilemma. Neural Computation, 4:1, 1-58 (1992). [23] ISO New England. A General Discussion of the Forecast Model Structures of the ISO New England Short and Long Run Energy and Seasonal Peak Forecasts for the 2009 CELT Report and 2009 Regional System Plan. ISO New England Website (2009). Available online at: http://www.iso-ne.com/trans/celt/fsct_detail/2009/_discussion_of_model_structures.pdf [24] ISO New England. System Operating Procedures: Create Demand Forecast. ISO New England Website (2011). Available online at: http://www.iso-ne.com/rules_proceds/operating/sysop/out_sched/sop_outsch_0040_0010.pdf [25] Itron. MetrixND: Flexible Modeling Tool. Itron Website. Available online at: https://www.itron.com/na/productsAndServices/pages/MetrixND.aspx [26] 2011 ERCOT Planning Team. Long-Term Hourly Peak Demand and Energy Forecast. ERCOT. Available online at: http://www.ercot.com/content/news/presentations/ 2011/2011_Long-Term_Hourly_Peak_Demand_and_Energy_Forecast.pdf

Lance Legel TLEN 5830 Automated Load Forecasting: Algorithms, Adoption Spring 2012 [27] Gaushell, D. Overview of Electric Load Forecasting at CAISO. California ISO: Presentation to Demand Response Working Group (2007). Available online at: http://www.caiso.com/1c57/1c578a8751b30.pdf [28] Makarov, Y. V. et al. Operational Impacts of Wind Generation on California Power Systems. IEEE Transactions on Power Systems, 24:2, 1039-1050 (2009). [29] Resource Adequacy Planning. PJM Manual 19: Load Forecasting and Analysis. PJM (2012). Available online at: http://pjm.com/~/media/documents/manuals/m19.ashx [30] Methaprayoon, K. Generation Planning and Market Operation Strategies in the Southwest Power Pool Energy Imbalance Service Market. PhD Dissertation at University of Texas Arlington (2007).