The performance -indicator delusion

Vilnis Vesma

Every energy manager needs to be able to show the savings achieved, and to do so in
an objective way that makes due allowance for factors (such as the weather or
production mix and output) which themselves cause consumption to vary. It is
important to be able to separate out the effects of energy efficiency improvements
from these other influences, which distort the picture.

It is sometimes supposed that improvements in performance can only be calculated by

evaluating the difference in an energy performance indicator (EPI) before and after
the introduction of the energy-efficiency measures. Such performance indicators are
commonly expressed as simple specific energy ratios, that is to say, kilowatt-hours
per unit of output (in manufacturing) or per degree-day (in the case of space heating
systems). This article will show that EPIs tend to yield incorrect conclusions, and in
some instances cannot be calculated in the first place. It will go on to explain how,
using a more appropriate protocol, absolute energy savings can nevertheless be
evaluated reliably. The explanation is based on a very typical set of real-life operating
data (reproduced in Appendix 1).

As Figure 1 shows, monthly electricity consumption in the example data set is

correlated with production output, and it is possible to represent the relationship by a
straight line from which expected consumption (on the vertical axis) can be read off
for any given level of monthly output (horizontal axis).

Figure 1

1
Let us examine two particular months, January 2008 and March 2009. Consumption
and production for both months are shown in Table 1, together with their resulting
ratios:

Energy
Month Electricity Production performance
(MWh) (tonne) indicator
(MWh/tonne)
January 2008 96 118 0.814
March 2009 45 52 0.865
Table 1

The obvious conclusion appears to be that March 2009 must have been relatively
wasteful, since it has the higher EPI measured in MWh per tonne. However, when the
months in question are located on the scatter diagram (Figure 2) the truth emerges:
March 2009 falls on the expected-consumption line, while January 2008 falls above it,
and is the wasteful month (to the tune of about 10 MWh). Reliance on EPIs not only
implicated the wrong month, but failed to identify (let alone quantify) the waste which
we can now see occurred in January 2008.

Figure 2

2
The straight-line model of the foundrys expected energy consumption relative to
product output (as in Figures 1 and 2) enables us to compute the deviation from
expected consumption through time, as has been done in Figure 3 below. From this
figure we can see that with rare exceptions performance was relatively consistent
throughout the five years analysed. Three exceptions can be seen, and are highlighted:
August 2006, January 2008 and (worst of the three with a 15 MWh deviation)
November 2008.

Figure 3

3
What conclusions would the advocates of EPIs have reached about the same data?
They would draw Figure 4 from which they would conclude that August 2004, at over
1.3 MWh/tonne, was the worst month, with July 2007 and November 2008 a close tie
for second place. At least they spot November 2008, but they miss January 2008 (10
MWh excess consumption) and August 2006 (12 MWh). Astonishingly August 2004,
which they regard as the worst case, can be seen from Figure 3 (fifth data point) to
have been a good month: the foundry actually used about 10 MWh less electricity
than expected.

Figure 4

This illustration of the shortcomings of EPIs is not artificial or a fluke. EPIs are bound
to give misleading answers because they fail to take account of the fact that there may
be a tranche of consumption which is constant and unrelated to the factor (such as
production or weather) which causes demand to vary through time. When production
output increases (or, in heating system, when the weather is colder) the EPI will
decrease even in the absence of energy efficiency measures. This is because the fixed
component becomes a smaller proportion of the total. EPIs conflate the effects of
energy efficiency with manufacturing efficiency or changing weather, so that nobody
who uses them can really see what is going on.

Before moving on it must also be pointed out that when two or more influences are at
play (say, for example, where a gas meter feeds both a process and a space-heating
system) no simple ratio can even be calculated in the first place. Worse still, in non-
energy-intensive manufacturing, there have been many cases of companies reporting
energy used per unit of output when in fact only the weather (which they ignored) was
affecting consumption.

4
So does this mean that we are unable to quantify improvements in performance? Far
from it. All we need to do is establish what, historically, was the relationship between
consumption and its driving factors. Figure 2 illustrates just such a mathematical
model of expected consumption in a case where there is a single driving factor. To be
precise, it shows that the foundrys electricity consumption consists of 11.5 MWh per
month base load, plus a variable requirement of 0.643 MWh per tonne of output. This
means that for any future month, given the production output, we can compute what
we would have expected the electricity consumption to be in the absence of changes
in performance. After energy-saving measures are introduced the actual monthly
outturns should be less than this historical baseline consumption and we will be able
to compute the cumulative saving, in absolute MWh terms, through time.

In the case of space-heating systems we merely substitute local degree-day values as

the driving factor. The method also works when there are two or more driving factors.
It just requires more terms in the equation for calculating expected consumption Ee:

Ee = k0t + k1D1 + k2D2 + + knDn

D1 , D2 , Dn are the various driving factors. Depending on the context they might
include degree-day values, hours of darkness, numbers of visitors, etc; some might
represent different product grades. They do not even have to be measured in the same
units of measurement. Meanwhile k0 is the fixed consumption per time interval and
k1 , k2 , etc are fixed coefficients and t represents the number of time-intervals
included in the analysis (for example, with a years worth of monthly data, t=12)

If circumstances dictate, even more complex formulae could be used. However for the
majority of applications a simple straight-line formula (Ee = k0t + k1D1) is generally
practical 123.

One final weakness of the EPI methodology which has not been mentioned is the fact
that it yields a ratio, whereas in fact the energy manager normally requires an
indication of the absolute quantity of energy saved, since this can be readily converted
into cash or carbon-dioxide terms. The performance improvement metric propounded
in this article yields the absolute quantity saved.

In conclusion energy performance indicators, even in those cases where they can be
meaningfully calculated in the first place, are not only unreliable but potentially
dangerously misleading. They have no place in energy management.

Vilnis Vesma
vilnis@vesma.com
+44(0)1531 821350
8 May, 2009

1
The efficient use of steam, Lyle O., HMSO 1947, p711
2
Energy monitoring and target setting using Cusum, Harris P, Cheriton Technology Management 1989
3
Gestin energtica, Janna F et al., Universidad Pontificia Bolivariana, 2001

5
Appendix 1
Data used in the example
The example data set is taken from a melting shop but the numerical values have been
multiplied by fixed factors, and the dates altered, so as to protect the confidentiality of
the source without compromising the integrity of the analysis.

Month Year MWh tonne 7 2007 42 42

4 2004 57 77 8 2007 58 76
5 2004 56 64 9 2007 80 105
6 2004 51 64 10 2007 103 140
7 2004 63 85 11 2007 100 135
8 2004 4 3 12 2007 76 103
9 2004 53 59 1 2008 96 118
10 2004 62 83 2 2008 89 119
11 2004 57 92 3 2008 86 117
12 2004 62 79 4 2008 99 139
1 2005 62 76 5 2008 96 129
2 2005 61 76 6 2008 79 120
3 2005 68 88 7 2008 33 35
4 2005 76 94 8 2008 64 85
5 2005 71 86 9 2008 86 124
6 2005 80 101 10 2008 90 146
7 2005 75 93 11 2008 77 76
8 2005 77 84 12 2008 22 29
9 2005 74 96 1 2009 41 43
10 2005 77 105 2 2009 42 47
11 2005 77 98 3 2009 45 52
12 2005 73 95
1 2006 77 93
2 2006 75 93
3 2006 83 113
4 2006 67 78
5 2006 76 101
6 2006 81 105
7 2006 27 42
8 2006 64 74
9 2006 73 93
10 2006 89 115
11 2006 92 124
12 2006 78 108
1 2007 84 110
2 2007 82 109
3 2007 86 113
4 2007 79 108
5 2007 90 118
6 2007 97 145