1

Distributed State Estimation with PMU Using Grid Computing

Qinghua Huang, Student Member, IEEE, Noel N. Schulz, Senior Member, IEEE, Anurag K. Srivastava, Member, IEEE and Tomasz Haupt, Member, IEEE
prevent cascading failures, which is revealed by the blackout in 2003 in the USA. Competition has been introduced for power industries in some parts of the world to provide electricity to the customers with better services at cheaper cost causing further complexity [1]. Limited data can be shared between area/region power system owners. This calls for a reliability coordinator to monitor and control the power system on a larger scale. The ability to monitor the wide area power systems can also benefit day to day operation of the energy market. In addition, it is also extremely beneficial for scheduling the long-distance power transactions between different areas/regions to have a snapshot of the areas where buyers and sellers of power are located [3]. Power system state estimation is an essential tool for system operators to use in monitoring and control of power systems. The state estimator used within an EMS environment has a broad impact on the performance of optimization and control activities within modern power systems as shown in Figure 1.

Abstract-- State estimation is an essential tool in monitoring the power system. As the size of the electric power system continues to grow, a state estimator has to be more computationally efficient and robust. This can be achieved by technical advancement in computational science and improving the state estimation algorithm. Distributed state estimation (DSE) decomposes the power system into smaller subsystems. This provides a way to simultaneously poll the measurements and execute the state estimation for a smaller size power system. PMUs can synchronize the measurements between different areas which are perfectly suited for distributed state estimation. The results obtained by local state estimation will be sent to the coordinator for further processing with consideration of the reference angle and boundary estimates. The phase angles and voltages estimated in each sub-area are usually based on their own area reference bus. Considering there are PMUs in the power system, this paper proposes a new method of calculating the reference angle difference between different areas. Grid computing has been used to help in DSE computation as a novel approach. The preliminary test results have presented for a small scale test case. Index Terms: distributed state estimation, PMU, SCADA, grid computing

I. INTRODUCTION

errestrial power systems have been facing an increased number of challenges recently. With continuous expansion, the power system is becoming more complex highlighting the importance of the wide area monitoring of the power system. Part of the terrestrial power system is operating at its limits for an increased period of time to achieve a specific economic benefit. Additionally, terrorism or natural disaster could possibly cause a sudden change of the power system topology. Improving the reliability and security of the terrestrial power system through efficient monitoring and control is a challenge under such conditions. Since a largescale terrestrial power system is geographically distributed in multiple areas but interconnected at the same time, it is essential that the decision maker have a wide area view of the power system. It is especially important in order to
Q. Huang (email: qh18@ece.msstate.edu), N. N. Schulz (email: schulz@ece.msstate.edu) and A. K. Srivastava (email: srivastava@ece.msstate.edu) are with the Department of Electrical and Computer Engineering, Box 9571, Mississippi State University, Mississippi State, MS 39762 USA. T. Haupt (email: haupt@cavs.msstate.edu) is with Center for Advanced Vehicular Systems at Mississippi State University, Mississippi State, MS 39762 USA.

Figure 1. EMS Functions of ISO [1] State estimation has become a must-run successfully control center function [4]. In Figure 1, it shows that all the

978-1-4244-4241-6/09/$25.00 2009 IEEE

other EMS applications must use the outputs of state estimation. Their execution must wait for the results of the state estimation. The shorter time, these EMS applications take to run, the more frequently and closely the power system can be monitored and analyzed. Therefore, it places a high requirement on the reliability and speed of the state estimation algorithm. An ideal state estimator should provide accurate estimation of the power system at a fast computation speed. It should also demonstrate both numerical stability and less implementation complexity. Sometimes a traditional state estimation algorithm does not converge and the degradation of the reliability of the state estimation is mainly due to the following causes. The size of power system continues to increase. More and more power controlling devices such as FACTS and various compensators are connected to the power systems to improve the power quality. The rapid development of new technologies in the areas of electrical engineering, computer hardware and computer communication have provided the solid foundation and necessary conditions for efficient distributed power system state estimation. The phasor measurement unit (PMU) is a power system intelligent electronic device (IED) that can provide synchronized and more accurate measurements. It can also provide the voltage phase angle, the key solution variable in SE which was not available as a measurement before. PMUs are quickly becoming the ultimate tool for wide-area monitoring [3]. Many power utilities have already placed several PMUs within their systems. It is envisioned that more PMUs will be used for advanced power system applications. Since the PMUs can synchronize the measurements among different areas by using a GPS, they are well suited for distributed state estimation for a wide area power system. In fact, PMU based distributed processing could bring more flexibility and reliability in wide area power system monitoring and control in the following regards. 1. In distributed state estimation, a large power system will be divided into several smaller power systems. State estimation has better convergence performance for smaller power systems. 2. If some part of the power system is causing problems in state estimation convergence, it would be easier to identify the location and problems within that part of the power system. 3. If the state estimation for one subsystem does not converge, the results from other local state estimations can help provide partial states of power system for other EMS applications to use. 4. In distributed state estimation, data polling and state estimation execution can be done in parallel which can reduce the execution time of the state estimation for the overall power system. The saved time can be significant depending on the size of the power system.

II. BACKGROUND A. Integrated state estimation algorithm The power system measurements from the SCADA usually have different levels of error and also some of the useful quantities will not be directly measured. The SE acts to filter errors in the system measurements by optimally computing the bus voltage phasors based on the available redundant raw measurements. A typical algorithm for state estimation is weighted leastsquares (WLS) that minimizes the sum of the squared weighted residuals between the estimated and actual measurements. This method or a variance of this method is popular among commercial state estimators. Suppose: 1. The ith measurements from SCADA system and other IED sensors are represented as Z i ; 2. The function of the state vector x to calculate the value of the measurement is f i ( x) ; 3. The difference between 1 and 2 for the ith measurement is called ei ; Based on the above definitions, for the ith measurement,
Z i = f i ( x ) + ei

For a power system with N measurements, assume the error vector e = [e1 , e2 ...ei ...e N ] to be a standard Gaussian that has a zero mean and independent covariance. The WLS formulation can be formulated as a minimization function as follows [3]:

min J ( x ) =
x i =1

[ Z i f i ( x )]2

i2

where: i2 is the variance of the ith measurement,

1
2

2 2

2 N

= R = E ( e.e T ) = Cov( e)

Cov() is the covariance function and E is the operator of the expected value [14]. J ( x ) is the measurement residual function. The solution x should make J ( x ) minimal. Reference [3] proved that the running time of such an algorithm is N s3 , where N s is the number of buses in the power system. Running time is a term in computer science that describes the approximate execution time of the algorithm. It gives an idea how N will affect the execution time of that algorithm.

B. Integrating Synchronized Phasor Measurement Units (PMU) into state estimation 1) Block Diagram of PMU The block diagram of the PMU shows a classical structure for digital signal processing. As shown in Figure 2, the inputs of the PMU are voltage and current analog signals. For the application in power systems, the analog inputs are mainly from the outputs of the transducers. PMUs output digital signals representing the calculated phasor and frequency information of the inputs. The digital outputs are then transmitted to the control center through the modems or other communication means.

and are treated the same as the conventional measurements in the integrated state estimation. Because they are more accurate, the standard deviations for the phasor measurements are set much lower than the conventional measurements. To integrate the PMU data into the state estimation, it should be considered that the phasor measurements and state estimator do not have a same reference. One solution can be to put a PMU at the reference bus of the state estimator and use this PMU as the reference for other PMUs. This uses the angle difference between any other PMU and the reference PMU in the state estimation algorithm. However, such an arrangement will result in vulnerability where the analysis depends on the signals from the reference PMU. There may be undesired events such as an outage at a measurement location, communication failure, or a failure in the reference PMU. Therefore, reference [12] also lists other methods to reconcile the phasor measurement and state estimator frames.

Figure 2. Block diagram of a PMU [4] The GPS receiver can get GPS signals that work the same as a usual digital clock for a microprocessor. A PMU with GPS signal can provide two additional functions. 1. GPS can make one PMU processing be synchronized with another PMU that is geographically far away. 2. GPS can also provide a global reference for the angle measurements based on the UTC second rollover [1pulse per second (PPS) time signal]. 2) PMU impact on state estimation Depending on PMU measurement accuracy and calibration, the number of PMUs, PMU locations and related SCADA data accuracy, PMU data can benefit the state estimation process [6]. 1. PMU data can improve the estimation of inaccurate measurements close to a PMU location. 2. PMU provides phase angle measurement that could not be measured before. 3. PMU data trend analysis can detect CB/switch status changes in the network, which may improve the topology estimation and error detection. 4. Utilities are willing to share the boundary PMU data. In many instances, poor SE performance has been due to lack of reasonably accurate external area measurements [6]. 5. According to reference [5], PMU can also be synchronized by other broadcasting signals other than GPS. 3) Integrating PMU data into the integrated state estimation As implemented in references [8, 9], the phasor measurements have been added to the measurement vector Z

C. Grid Computing The definition of grid computing given in reference [9] is Grid Computing enables virtual organizations to share geographically distributed resources as they pursue common goals, assuming the absence of central location, central control, omniscience, and an existing trust relationship. Coordination and distribution are two fundamental concepts in grid computing [11]. According to [9], grid computing covers high performance, cluster, peer-to-peer, and Internet computing. There are many separate and distinct Grids. Segmented by organization, grid computing includes enterprise grids, partner grids, and service grids. Enterprise grids are for private resources to be shared within a single enterprise. Partner grids provide extra-networking to enable resource sharing among selected enterprise partners. Service grids are inter-networking to provide public resource sharing on a global scale. The latter of two is still developing and the spending on worldwide grids is shown in Fig. 3. Segmented by function, there are enterprise compute grids, data grids, equipment grids, and application grids.

Figure 3. Worldwide Grid Spending by Type of Organization, 2005-2010($Billion) [10] Enterprise Compute Grids are for computational intensive operations. Data Grids are for controlled sharing and management of a large amount of distributed data. Equipment Grids are for controlling equipment remotely and analyzing the data produced. Application Grids provide shared access to applications. A grid computing application includes three

layers service layer, middleware, and resource layer. Service layer is for computing, storage measurement and control. Middleware tends to provide security, information discovery, resource management, communication, and portability. Different middleware for grid computing have been developed for different use, such as Globus, GRIDBUS, and Narada Brokering, to name a few. The Resource layer for power system design can be SCADA systems, database or computing and simulation software. III. DISTRIBUTED STATE ESTIMATION WITH PMU There are two types of distributed state estimation algorithms. One is called synchronous distributed state estimation, where each area control center (ACC) must coordinate with each other at every iteration of the local state estimation. In this type of approach, each ACC has to wait for other ACC to complete their computations to go to the next iteration and the communication burden will not be minimized. The other type of distributed state estimation is called asynchronous distributed state estimation. The result of an ACC is sent to the coordinator after the convergence is reached. The coordinator consolidates the results from all areas into a complete network result [1] [3] [7]. General steps for performing distributed state estimation include [1] [3] [7]: 1. Split the power system and clarify the boundary of the sub-systems 2. Obtain measurements 3. Execute local state estimations 4. Resolve reference angle differences 5. Assemble the results at the coordinator

diff = pmuBSlack pmuASlack

(2)

where the slack bus in area A is set as the global reference and should also be the reference bus in the state estimation algorithm. There are two issues about the above two methods. First, these two methods do not consider or discuss the condition of PMU data loss due to breakdown or communication failure. For the second method, the reference angle difference calculation only depends on the measurements from the slack bus. Considering the possibility of communication failure, PMU failure or blackout at the substation where the PMU reference bus is located, makes the system vulnerable by using a fixed reference PMU. Secondly, these two methods do not consider the PMU measurement offset issue caused by different vendors including integrated PMU (such as relays have PMU functions). Vendor issues should be considered if these PMUs are to be used for reference angle difference calculation of distributed state estimation. This is because the measurements of PMUs from different vendors have a constant offset at each frequency point as shown in Figure 4.

A. Reference Angle Difference Calculation in the Coordinator for DSE In distributed state estimation, the power system under study needs to be decomposed to several sub-areas first. The phase angles estimated in each sub-area are usually based on a different reference bus. Assume the reference bus in one of the sub-areas is chosen as the global reference bus for the overall power system. The angle difference between the reference bus in sub-area B and the global reference bus in A is defined as the reference phase angle difference for B. Reference [3] has listed four optimization methods for calculating the reference angle difference. The basic idea in this reference is to take the phase angle measurement of any PMU in each subarea ( pmuAn and pmuBm ) and the
corresponding estimated angles to compute the angle difference as shown in (1). diff = pmuAn pmuBm estBm estAn (1) Values computed using equation (1) will be used as an additional term together with other terms from nonsynchronized measurements in the existing algorithm. The only difference is the different standard deviation. References [7] and [8] used only the PMUs from the reference bus in each sub-area which is the slack bus in this case and choose one of the slack buses as the global reference for the PMUs. Therefore the reference angle difference is calculated by (2).

Figure 4. PMU Phase Angle Measurement Errors for Different Vendor PMU [4] Figure 4 shows the phase angle offsets of different PMUs in the frequency range of 54 - 66Hz. This is not an issue as long as all the PMUs are provided by the same vendor. However, when the systems gradually add a larger number of PMU devices, it may not be ensured that the PMUs from a single vendor will be used. This is particularly true for wide area monitoring involving multiple entities and different control areas using PMUs from different vendors [11]. Based on the issues presented above and considering there are PMUs from different vendors that have been installed in the power system, the following equation is proposed for the calculation of the reference angle difference.
diff = ( pmuA0 _ vendor1 pmuAn _ vendor1 + estAn ) ( pmuB 0 _ vendor 2 pmuBm _ vendor 2 + estBm )

(3)

pmuA0 _ vendor1

Where

and

pmuB0 _ vendor 2

are the phasor angle

measurements from area A and area B for the same variable, either current flow in the tie line or voltage angle on the shared bus. This means for the overall system two PMUs would be used closely to the boundary. This is practical where the control area owners are more willing to keep closer watch

on the boundaries.

pmuAn _ vendor1

and

pmuBm _ vendor 2

are the

phasor angle measurements from area A and area B for one of the state of the local state estimation. estAn and estBm are the local estimated phase angle of bus n in area A and bus m in area B. m and n can be the number of any bus in the subareas that has a PMU. These buses with PMU should also have good angle estimates by the local state estimation. IV.
SIMULATION AND RESULTS ANALYSIS

Initial simulations have been done on a small test system to verify this revised algorithm. XTargets Matlab 2 Matlab Toolbox has been used to create the grid of MATLAB computing resources.

The first number in the subscript means from which bus the power flows and the second number means the bus to which the power flows. When dividing the system, for the subsystem A on the left of Figure 6, line power flow P43 and Q43 will be considered as a new load for bus 4; similar modifications will apply to bus 4 and 5 in subsystem B on the right in Figure 6. The measurement information will only contain data for those related to the four buses in each subsystem. After decomposition the bus data for subsystems are as Table 3 and 4 [13]. Table 3 Bus Data for Subsystem A (pu)

A. Test Case Ward-Hale 6 bus system shown in Figure 5 is used as the test case. The network data is shown in Tables 1 and 2.

Table 4 Bus Data for Subsystem B (pu)

Figure 5. Ward-Hale 6 Bus System [computation method] Table 1 Bus data for 6 bus power system (unit: pu) [13]

Note for Tables 1, 3 and 4, Type indicates bus type: 0 indicates slack bus, 1 indicates PV bus, and 2 indicates PQ bus. In the following tables, ISE indicates integrated state estimation and DSE indicates distributed state estimation. Error indicates the difference between the estimated angle by DSE and the actual angle. Reference Angle Difference is the angle difference between the local reference bus and the global reference bus. Table 5 Estimation Results Comparison between ISE and DSE

Table 2 Branch data for 6 Bus Power System (unit: pu) [13]

This test case is divided in two subsystems as shown in Figure 6,

Figure 6 Ward-Hale 6 Bus Power System after Splitting As shown in Figure 6, the 6-bus power system is divided into two 4-bus subsystems, where the four bus subsystems share two buses bus 4 and 5. The power flow between bus 4 and 3 is indicated as P43, Q43, P34 and Q34. The power flow between bus 5 and 2 is represented as P52, Q52, P25 and Q25.

Where Residual= Estimated-Measurement or Residual= Estimated-Actual AR= (Sum of absolute residuals)/(number of measurements) Number of measurements =18 Comparing the values of AR in Table 5, which shows for the estimations on the measurement quantities, ISE has better

results than the DSE. This is because DSE works on 4-bus systems, and the boundary issue affects the DSE results. The results for Table 6 are compared between the different methods of calculating the reference angle difference in distributed state estimation. The standard deviation for the PMU measurements is 0.0001. Since the system is small, the boundary data which is not very accurate have a larger impact on the outputs for each case, especially for the case DSE3. This is because DSE3 uses only the boundary estimates from the 4-bus local systems to calculate the reference angle difference. Table 6 Estimation Error Comparison

[4] [5] [6] [7] [8] [9] [10] [11] [12]

where, DSE1 uses PMU data from only slack bus, DSE2 uses the PMUs located on the tie line, bus 1 and bus 3. In this case bus 3 provides the best local estimation results. DSE3 no PMU has been used. Table 6 also shows the large error caused in DSE without PMU, which are mainly caused by the reference angle difference calculation. V. SUMMARY AND FUTURE WORK This paper proposed an efficient way to calculate the reference angle difference between different areas in distributed state estimation algorithm with PMU. This method considered PMU data loss and angle measurement offset between different vendors. This method can be used as an alternate if any of the slack bus is not available. Grid computing has been used to assist in distributed state estimation. Preliminary results of implementing distributed state estimation with PMU using grid computing have been presented. Further tests need to be done for larger power systems so that boundary estimates will have reduced effects on the results. VI. ACKNOWLEDGEMENT The authors would like to recognize the Office of Naval Research for funding this work through the U.S. Department of Defense (DoD) MURI Fund # N00014-04-1-0404. VII.
[1] EFERENCES

[13] [14]

H. Wu and J Giri, PMU Impact on State Estimation Reliability for Improved Grid Security Transmission and Distribution Conference and Exhibition, 2005/2006 IEEE PES IEEE STANDARD FOR SYNCHROPHASORS FOR POWER SYSTEMS std C37. 118-2005Available at: http://www.phasorrtdms.com/phaserconcepts/phasor_adv_faq.html M. A. Freeman, Multi-area power system state estimation utilizing boundary measurements and phasor measurement units (PMUs) M.S. Thesis at Texas A&M University, August 2006. L. Zhao and A. Abur, Multi Area State Estimation Using Synchronized Phasor Measurements, IEEE Transactions on Power Systems, vol. 20, no. 2, pp. 611-617, May 2005 A. Khatib. Internet Based Wide Area Information Sharing and its Roles in Power System State Estimation, Power Engineering Society Winter Meeting, 2001 IEEE, Vol. 2, 2001, pp. 470 475. Dec. 2001 A. Abbas Grid Computing: A Practical Guide to Technology and Applications Charles River Media; 1 edition (December 2003), ISBN: 978-1584502760. Market report from Insight Research (IR), GRID COMPUTING A VERTICAL MARKET PERSPECTIVE 2005-2010 February 2005. F. Berman, G. Fox, and T. HEY, Grid Computing Making the Global Infrastructure a Reality Wiley (April 8, 2003), ISBN: 978-0470853191 Y. Hu and D. Novosel Progresses in PMU Testing and Calibration DRPT2008 6-9 April 2008 Nanjing China M. Crow, Computational Methods for Electric Power Systems, 1st ed. CRC, 2002. http://www.mathworks.com/access/helpdesk/help/techdoc/index.html?/acce ss/helpdesk/help/techdoc/ref/cov.html&http://www.google.com/search?hl= zh-CN&client=firefox-a&channel=s&rls=org.mozilla%3AzhCN%3Aofficial&hs=CI0&q=covariance%2C+cov&btnG=Google+%E6% 90%9C%E7%B4%A2&lr=

VIII. BIOGRAPHIES
Qinghua Huang is a Ph.D. candidate in the department of Electrical and Computer Engineering at Mississippi State University. She received her Bachelor degree in Electronics Engineering from University of Electronics Science and Technology of China (UESTC) and Masters degree in Electrical Engineering from Mississippi State University. Her research interests include power systems and the applications of distributed computing in state estimation and simulation in power systems. Noel N. Schulz received her B.S.E.E. and M.S.E.E. degrees from Virginia Polytechnic Institute and State University in 1988 and 1990, respectively. She received her Ph.D. in EE from the University of Minnesota in 1995. Dr. Schulz is a Professor in Electrical & Computer Engineering at Mississippi State University. She is the TVA Endowed Professor of Power Systems Engineering. Her research interests are in the computer applications in power systems including power system operations, shipboard power systems and intelligent system applications. She is an NSF CAREER award recipient. She is active in the IEEE Power & Energy Society and served as Secretary for 2004-2007 and Treasurer for 2008-2009. She was the 2002 recipient of the IEEE/PES Walter Fee Outstanding Young Power Engineer Award. Dr. Schulz is a member of Eta Kappa Nu, Tau Beta Pi, ASEE, SWE and NSBE. Anurag K. Srivastava received his Ph.D. degree from Illinois Institute of Technology (IIT), Chicago, in 2005, M. Tech. from Institute of Technology, India in 1999 and B. Tech. in Electrical Engineering from Harcourt Butler Technological Institute, India in 1997. He has been working as an Assistant Research Professor at Mississippi State University since September 2005. Before that, he worked as a research assistant and teaching assistant at IIT, Chicago, USA and as a Senior Research Associate at Electrical Engineering Department at the Indian Institute of Technology, Kanpur, India as well as Research Fellow at Asian Institute of Technology, Bangkok, Thailand. His research interests include real time simulation, power system modeling, power system security, power system deregulation and artificial intelligent application in power system. Dr. Srivastava is member of IEEE, Power & Energy Society, IET, Sigma Xi and Eta Kappa Nu. He is recipient of several awards and serves as reviewer for IEEE Transactions, international journals and conferences. Tomasz Haupt is a physicist by education (Jagiellonian University and Institute of Nuclear Physics in Krakow, Poland), but for the last 20 years has been involved in Computer Science research. At Syracuse University he was

[2] [3]

M. Shahidehpour and Y. Wang, Communication and Control in Electric Power Systems. Wiley-IEEE Press (June 19, 2003), ISBN-13: 9780471453253. H. Louie, and K. Strunz, Hierarchical Multiobjective Optimization for Independent System Operators (ISOs) in Electricity Markets IEEE Transactions on Power Systems, V. 21, No. 4, November 2006. R. F. Jeffers, Techniques for wide-area state estimation in power systems, M.S. thesis at Virginia Polytechnic Institute and State University, 2006.

7 developing methodologies for parallel computing, including HPF language and compiling system, as well as HPF and MPI applications. At Mississippi State University he created and leads the Cooperative Computing Group that develops the cyber-infrastructure to support engineering applications, including Computational Material Design and Design Optimizations, Earthquake Engineering, Power Systems, and Ship Survivability.