From Water Boiling Test to Water Heating Test

Case Study of Indonesia


Introduction
Clean cookstove initiatives beginning in the early
1980s recognized the need for improvements in the
performance of biomass fuel cooking stoves.
Recommended test methods were developed at the
University of Eindhoven and Bois de Feu, an NGO in
France. A review of their methods is instructive
because they had resolved several conceptual issues
which were reintroduced as part of a Water Boiling
Test (WBT) by a VITA-led group in 1985. The original
document produced by VITA outlining proposed
testing methods presented three complimentary
approaches: the Water Boiling Test (WBT), Kitchen
Performance Test (KPT) and Controlled Cooking Test
(CCT). All of them contain conceptual errors that
resulted in the mis-rating of stoves, particularly at low
power.
The available test methods at the time were reviewed
1
by Rani, Kandpal and Mullik who noted the problem
with the low power metrics and reported on an
experiment that showed the amount of fuel required to
simmer a pot was independent of the mass of water in
the pot, rendering the validity of several reporting
metrics questionable.
In the meantime, the VITA WBT emerged as a dominant
testing method in the laboratory environment,
particularly in the Western world. Around the same
time, India and China adopted quite different methods
which they have continued to use until now.
The core claim for the WBT is that it supports the
standardization and replicability of controlled
laboratory testing to arguably allow for the
differentiation of cookstoves at their optimum
performance.
1C.S. Rani, T.C. Kandpal, S.C. Mullick, “Preliminary study of water boiling test procedures used for performance evaluation of fuelwood cookstoves,” Energy Conversion
and Management 33(10) (1992): 919-929.
2
The inherent variability of the WBT is about 30%. See: Gorrity, Conference paper, CAU, November 2014.
3
GACC, The Water Boiling Test Version 4.2.3: Cookstove Emissions and Efficiency in a Controlled Laboratory Setting, (19 March 2014): 2.
From Water Boiling Test to Water Heating Test 1
The performance of a stove is contingent on four
factors: (a) the stove, (b) the fuel and its moisture
content, (c) the operating procedure or burn cycle, and
(d) the cooking vessel or vessels.
The last three factors are highly dependent on the local
context, meaning that whatever one chooses for each
variable will strongly affect the relative performance
rating of 'the stove'. Performance rating is therefore an
evaluation of the combination of factors, not only of the
stove. In most cases, a prescribed cooking cycle was
written into the test, resulting in performance ratings
that are only valid for that particular three-phase burn
cycle consisting of a cold start, hot-restart of the fire,
and a 45 or 30 minute simmering period. The test
ignored emissions before placing the pot on the stove
and suffered from many other shortcomings.
Although the WBT has been advertised as simple to
conduct and capable of giving useful information to
designers, it is not as simple to conduct as either the
Indian or Chinese tests and has a larger inherent
2
variability.
Field tests conducted using the VITA CCT were
intended to capture an indication of performance
based on the cooking practices of local users and
“provide an indication of the performance of stoves
during actual use”. From the beginning, VITA struggled
with, and recognized the challenge of, striking a
balance between:
(a) the standardization of testing methods to
ensure replicability across testing locations; and,
(b) the performance of actual cooking practices in
the laboratory environment to better predict the stove's
performance in local usage.3

VITA emphasized the distinction between testing
“done for local use only (for stove users and others)
and testing where the results are intended to be
transmitted to other places”.4
CCT conducted in Yogyakarta, 2009
The core misunderstanding in the stated intention for
the CCT is the idea that the performance of a stove is
inherent to all circumstances. Test results are only
valid for the context in which they were performed.
This has been amply validated by the fact that there is
little, if any, correlation between laboratory tests in
which 'cooking is simulated' and cooking in any place
that does not 'cook like that'. A stove tested and
optimized using dry fuel will almost always perform
badly when tested using wet fuel; whether that is in the
same laboratory, another one, or in the field.
Since the WBT occupied a central the role in laboratory
testing in the West, using a fixed fuel, fixed burn cycle
and fixed cooking vessel, the results proved
untranslatable to different global contexts and unable
to transcend local variations in cooking practices. A
stove 'recommended' by WBT results might not be
accepted when used under different circumstances.
The CCT was supposed to address the issue of
translatability to different cooking contexts. However,
if the CCT is able to provide useful information,
4
Ibid: 31.
5
For information on current testing protocols: http://www.cleancookstoves.org/our-work/standards-and-testing/learn-about-testing-protocols/.
From Water Boiling Test to Water Heating Test 2
why was it not used in the laboratory by reproducing
relevant cooking cycles there? The answer is not clear.
However, there is an apparent belief that performance
is 'inherent': a 'better stove' would always out-perform
a 'worse stove', with cookstove projects opting to
promote the 'better' stove on the basis of the WBT lab
test.
Because any test conducted 'differently' will give a
different performance rating, it is critical that a product
selection process conducted in the lab or the field be
representative of the local context.
1
It is incorrect assume that any characteristics of a
stove are inherent in the design. For example, the
particulate matter (PM) emissions vary considerably
from the same stove if the moisture content is altered
or the fueling rate is changed. Having two different
laboratories agree that a stove performs well with 5%
moisture fuel does not indicate how a stove will
perform with 15% or 25% moisture fuel, levels at
which stove performance may not be optimized.
Testing performed out of context carries little
information about performance, or gives misleading
guidance.
Any test reports the performance under that set of test
conditions. While a properly documented set of
conditions is reproducible, it may not be relevant for
the reader's project area. Where there is no match,
there is no useful information conveyed.
The WBT has been significantly altered from its
original version to the present Water Boiling Test
5
Protocol 4.2.3 and contains additional output metrics.
As a result, test results from different versions are not
directly comparable and may not even have the same
reporting metrics. Importantly, the core problems with
the low power metrics introduced in the original 1985
VITA test have not been addressed. So far as is known,
the current version has never been externally reviewed
by independent experts.

Because of the large inherent variability of the WBT
protocol and the clear differences between the
prescribed lab conditions and the flexible field
conditions, there has been a standing conflict between
WBT and CCT results. The desire to ensure
repeatability and precision in the lab has never been
matched in the the practical field. This has been
exacerbated by the impracticality of conducting
6
precise tests in field conditions.
WBT conduct in Yogyakarta,2005
The developers of the WBT suggest that the
requirement for the stove to perform at low and high
power output provide a solid indication of the stove's
performance during actual cooking.7 While this is
theoretically possible, the fixed burn cycle and the
poor choice of reporting metrics has resulted in its
continued rejection in many circles. The UNFCCC
allows for the CCT to be used but has done so without
conducting an external review.
Furthermore, concepts inherent to test which made
little difference to performance rating in the past have,
because of new inventions, come to the fore as
significant issues. The most important is the old
'power station' idea that the heat transfer efficiency is a
good proxy for the fuel efficiency of a stove. The
development of stoves that make charcoal concurrent
with cooking has resulted in stoves that consume as
much or more fuel than a three stove fire.
6 GACC, 2014: 31.
7 Rob Bailis, Damon Ogle, Nordica MacCarty, and Dean Still (input from Kirk R. Smith and Rufus Edwards), Water Boiling Test, Aprovecho Research Center, Household
Energy and Health Program, Shell Foundation (2003): 13.
8 GACC, 2014.
From Water Boiling Test to Water Heating Test 3
When rated for their 'heat transfer efficiency' (a
calculation of the percentage of heat released from the
fire that enters the pot) there is a mismatch with 'fuel
consumption' since the heat contained in the charcoal
is not released during burning. The WBT rates the
efficiency of the use of energy released, not the
efficiency of the use of the energy in all the fuel
extracted from the forest. Stoves that are 15-20%
efficient from the 'forest's point of view' are being been
credited with 'an efficiency' that is over 50% because
they do not burn all the fuel, it is turned into charcoal.
Whatever happens to that charcoal, or smaller
amounts produced by other stoves, is independent of
the stove that produced it. How to account for this
charcoal requires a programmatic decision apart from
the performance rating.
The idea that 'simmering', or performance at 'low
power', has an 'efficiency' is incorrect, also noted by
Rani et al. Similarly, the idea that the final volume of
water in a pot represents 'cooked food' is incorrect,
particularly for foods cooked in water like potatoes or
corn.
It is recognized by developers and proponents of the
WBT that it does not provide a complete view of stove
performance while performing real cooking tasks in a
particular local context. The stated purpose of the
WBT is to “measure how efficiently a stove uses fuel to
heat water in cooking pot and the quantity of
emissions produced while cooking”8
The core problem is that even if the metrics were
corrected, the idea that a universal test tells us
something useful about how a particular stove will
perform in any circumstance is fundamentally flawed.
This assumption fails to consider, in the lab, the
context of that future use. Only contextual
performance comparisons are relevant for product
selection of biomass stoves.

Alternative testing protocol to WBT
The Water Heating Test (WHT) method developed
through the Indonesian Clean Stove Initiative pilot
program asserts that cooking practices are usually
complex cycles that are not adequately represented by
two measurements at high power and a third at low
power. In place of the standardized WBT cooking
cycle, the pilot program has adopted a method of
creating a contextual burn cycle relevant to the
Indonesian local cooking experience. This has been
paired with the well-known SeTAR Heterogeneous
Testing Protocol (HTP) referenced in the IWA 11:2012.
The characterization of the burn cycle is achieved by
the triplicate replication of various cooking cycles and
measuring the average performance. This is done by
local cooks using local fuels and pots in a laboratory
setting.
Preparing local fuels for WHT
Multiple cooking cycles are combined to create an
'average burn cycle' called a Technical Test which
reproduces the power levels and its variation. It may
include the 'weighting' of cooking cycles to represent
their relative frequency of use.
Thus the WHT is a testing framework based on the HTP
into which any locally relevant cooking cycle or set of
cycles can be placed. The HTP testing method is
thesame from country to country, from lab to lab, but
everything about the cooking is local. A lab in another
From Water Boiling Test to Water Heating Test 4
country could easily reproduce a documented burn
cycle from a catalogue of behaviors. In this manner, a
manufacturer can test a stove product intended for a
foreign market. This accomplishes the intended goal
of the WBT to permit stoves to be tested anywhere, for
any market. All that is required is a documented
Technical Test and the relevant pots and fuels.
Boiling water is a part of cooking cycle
In sum, there are multiple clear differences between
the WHT and the WBT:
- The WHT reports the fuel consumption necessary to
replicate a cooking task or a set of cooking tasks; the
WBT reports the energy consumption, mathematically
treating 'partially burned fuel remaining' as 'unburned
raw fuel.'
- The WHT reports the performance while conducting
locally relevant cooking cycles while still being
restrained to laboratory conditions, thus replicating
'average field testing' in the lab; the WBT reports the
performance using a fixed, three stage burn cycle,
fixed pot and fuel type.
- The WHT calculates the performance using
measurements appropriate for the metric being rated;
the WBT uses somewhat arbitrary metrics taken from
various sources, not all of which are relevant, correctly
calculated or conceptually valid.
- The WHT has a low inherent variability because it
minimizes the influence of experimental variation in
calculations; the WBT has an inherent variability of
about 30%, largely because of using 'water mass
remaining' as a divisor for multiple outputs.
- The WHT has been informally, externally reviewed
multiple times as and whenever it is edited.

It is the aim of the Indonesian initiative that the a
contextually accountable testing process will be
achieved through the incorporation the social
anthropology of cookstove usage into development of
the burn cycles employed at the laboratory. There are
multiple documented cycles available. As knowledge
on local cooking practices increases, it may transpire
that some cycles are relevant to other regions and
countries.
Any description of cooking behavior can be used in
developing a testing protocol. This does not affect the
validity of the measurements made, only the
applicability to the target population the burn cycle is
supposed to represent. The more locally relevant the
information on cooking behavior, fuels, foods and
pots, the more likely the lab-predicted behavior of
candidate stoves will be reflected in the field.
Thus there are two protocols involved: once for the
social science and another for laboratory testing.
INDONESIAN CLEAN STOVE INITIATIVE
This case study sets out to propose an argument for a
lab-based stove testing method that reasonably
predicts future performance when used in a particular
context. One of the aims of the clean stove initiative in
Indonesia is to develop and assess an alternative
testing method (the WHT) that claims to incorporate
critical social factors into the test cycle and therefore
into the rating process.
In 2012, the World Bank in collaboration with the
Directorate of Bioenergy, Ministry of Energy and
Mineral Resources (MEMR), initiated the Indonesian
Clean Stove Initiative (ICSI). The central aim of the
initiative is to scale up access by 2030 to clean
cooking solutions for the 40% of Indonesian
households that will likely continue to rely heavily on
solid biomass fuels for cooking. Over the next 10-20
years, national economic development is expected
9
Referring to the Kerosene-to-LPG Conversion Program and the Indonesia Domestic Biomass Programme (IDBP).
From Water Boiling Test to Water Heating Test 6
to result in the increased adoption of LPG. . It is also
expected that households which continue to use
biomass cooking fuel will do so with a clean stove.
Further, it recognizes that a significant proportion of
LPG users (about 70%) also use biomass fuels to heat
water. As a result of being a highly mixed fuel
economy, the target market of the initiative includes
approximately 70% of the entire Indonesian
population.
Director General of New, Renewable Energy & Energy Conservation,
Mr. Rida Mulyana launched The ICSI program on August 2014
The ICSI comprises four program phases (i) initial
stocktaking and development of the implementation
strategy; (ii) institutional strengthening, capacity
building, and piloting of the strategy; (iii) scaled up
program implementation; and (iv) program evaluation
and dissemination of lessons learned.
The activities completed in Phase I have focused on in-
depth assessments of household cooking fuel
technologies and the existing stove market, review of
the sector policy and institutional framework, and
collation of lessons from the country's two most
9
successful clean cooking programs for application to
the biomass stove initiative.
The next step for the ICSI, Phase II, is to establish
stove standards and testing protocols, strengthen
institutions and build stakeholder capacity, design and
prepare the master plan for rollout of the national
program.
 From Water Boiling Test to Water Heating Test 6

biogas,1140% of Indonesians indicate that their primary

Field studies on biomass stove users
The uptake of clean cookstoves by households
depends on behavioral change to ensure not only the
purchase but the sustained usage of the improved
cooking stove. This uptake hinges on a variety of
factors which cannot be adequately understood
without careful study of local cooking practices. While
establishing the stove standards and testing protocols,
the World Bank CSI's technical team in collaboration
with the Social/Gender team conducted surveys and
group discussions with biomass stove users. The
studies were conducted between 11 March and 7 April
2013 in Yogyakarta and Central Java. Findings from
these field observations and discussions have
provided a more holistic view of the range of
cookstove usage, fuels and cooking practices.
The purpose of these field visits was to provide
“enough ethnographic details about socio-economic
interests and relevant macro-contexts... to recognize
what improvements in stove technology will be most
likely to motivate potential customers belonging to
particular stove using groups to buy and use improved
10
biomass stove products”.
Central findings from these field studies that inform a
contextually relevant testing method include:
fuel source for cooking is from biomass, however this
understates its prevalence. Field survey results from
17 provinces indicate that wood-burning stoves are
12
produced and sold more than any other stove type.
Reliance on biomass fuel for cooking is particularly
pronounced in rural areas and it is unlikely to decrease
in coming years.
Considerations such as time, cost, cleanliness,
micro-enterprise applications and convenience are
central selecting the choice of fuels and stove model.
The typical trend across demographics in Indonesia is
the allocation of more biomass to home industry and a
shift to LPG for non-industrial cooking. While LPG
usage enables families to improve their kitchen
conditions (clean pots, convenience, speed),
households remain dependent on (typically free)
biomass for heating, lighting, and cooking. In
Indonesia, of those surveyed, 70% said that though
they used LPG primarily for cooking, they also use
13
wood for heating water. As biomass is projected to
remain a widely available, free, convenient and
renewable fuel source, it is likely that reliance on
biomass-burning stoves will continue, particularly for
heating water, even if LPG is more widely adopted.

a. Complex demographics of fuel usage in Indonesia

Demographic studies of fuel usage in Indonesia must

be carefully read to provide an adequate image of
distribution of fuel sources among geographically and
socio-economically divided groups. It is common
practice for households to use a combination of fuel
sources, differentiating between fuels according to the
type of cooking task performed. The common usage
of a combination of fuels complicates the
conceptualization of the actual biomass use or Biomass stock in one of household in Bantul, Yogyakarta Special Region
dependency in the country. Despite the Indonesian
Government's conversion programs for both LPG and

10The World Bank, Asia Sustainable and Alternative Energy Program, East Asia and Pacific Clean Stove Initiative Series, Indonesia: Toward Universal Access to Clean
Cooking, (June 2013), Washington, DC.
11 Ibid: 26.
12
Ibid: 16.
13 While the focus of this initiative is not on the development of clean biomass water heating stoves, this example provides valuable insight on just how complex the biomass

economy is in Indonesia.
 From Water Boiling Test to Water Heating Test 7

The allocation of different fuels to particular tasks is b.Convenience and economic costs as determinant
geographically and culturally contingent, which is to factors
say 'the cooking context' is local. An observation from
field studies highlights the importance of biomass Field research revealed the main determining factors
burning stoves for the provision of lighting in the home. affecting the choice of fuel source and type, and stove
Stoves are often used to light the home, notably in rural type. Convenience and economic costs feature
family and peri-rural households. Illumination will be centrally in these choices. Widespread access to free
an important consideration affecting the uptake of a biomass fuel wood as well as the low cost of
stove if it has a window-less door that must be closed traditional cooking stoves make biomass cooking a
to work at optimum efficiency. Will people use the popular option, especially for lower income, rural
stove with the door closed? If not, should the stove be households.
tested with the door open, as people would actually be
using it? This aspect of the local context must be Higher income households have been quicker to adopt
understood when selecting the test cycle to be applied. LPG as well as electric rice cookers which reduce the
time required for gathering firewood and the cleaning
The mixed demographics of fuel usage in Indonesia of soot from pots.
reveal that, despite the introduction of new energy
carriers, biomass remains an integral part of “By cooking rice in an electric cooker, biomass
Indonesian household cooking. Past attempts to using households reduce the amount of
improve indoor air quality were directed at fuel firewood they must gather by about 25% and
14
switching (e.g. from kerosene to LPG). Considering they also reduce the amount of time spent
15
the contextual complexities and the inhomogeneity of cooking for the family by the same factor.”
fuel allocation for cooking tasks, adequately
conceptualizing what local choices will be made is
difficult. sIt is clear from the surveys that there will be a
continued reliance on biomass for some aspects of
domestic cooking, particularly water heating. Evolving
consumer demands and culturally-rooted
performance requirements are intrinsic drivers of
domestic energy choices in Indonesia. The continued
improvement of biomass stoves and the use of
contextually relevant evaluations must become a
permanent facet of the domestic energy policy
framework.

Electric rice cooker

Keeping pots soot-free is important to consumers.16

The desire for convenience, cleanliness and low cost
cannot be neglected when developing a contextually
relevant and user-accountable product evaluation
method.

14
World Bank, 2013: 26.
15
Cecil E. Cook, Report to World Bank CSI Program on Social & Cultural Factors Impacting Stove/Fuel Use in Yogyakarta and Central Java, (2013), South Africa.
16 Kitchens are often divided into a “clean area” and a “dirty area”, with the “clean pots” for LPG use.

The projected continued easy access to free firewood
as well as a lack of economic means in rural areas to
adopt alternative cooking solutions, including LPG fuel
and rice cookers, suggests the long term centrality of
biomass cooking stoves in the domestic energy
market.
c.Avoiding standardization of questionnaires to
assess stove and fuel users
Analysis of the field study revealed concerns about the
Clean Cookstove Questionnaire developed to assess
household fuel and stove usage as well as with the
reasons for the uptake of a particular cooking stove.
The wide systematic variation, as a result of
geographic (forest cover, type of farming, and
altitude), economic status (income level) and cultural
17
variables fundamentally shape the biomass economy.
A random survey is arguably incapable of providing an
informed view of cookstove usage in Indonesia. The
varied realities of the biomass economy in Indonesia
must be incorporated into the survey questionnaire as
well as the testing cycle applied. This can be used to
ensure that an agreeable standardized, countrywide
testing method is adopted, while recognizing the
necessity of identifying locally relevant cooking cycles
to be used with the method. The selection of a single
'national cooking cycle' would fail to address the vast
differences in the Indonesian biomass economy. The
distinction between a test protocol and a cooking cycle
is not well recognized. Therefore, they are often
lumped together. Identified as separate components of
a national regulatory framework, contextual testing
can be done for any community of users based on a
common protocol. This addresses all the geographic,
socio-economic and cultural variables affecting
performance. A well-constructed survey correctly that
informs the cooking cycle to be applied during
assessment.
17Cook, 2013: Section 2:8.
18
The HTP originates from the SeTAR Centre, University of Johannesburg, South Africa.
From Water Boiling Test to Water Heating Test 8
Incorporation of field observations into
ISCI's program approach
With the aforementioned results of the field studies,
the technical team has noted that using the WBT v4.x
method does not provide an adequate, relevant or
holistic view of what is taking place in the biomass
economy. The WBT closely follows the metrics and
'performance tiers' in the Interim Workshop
Agreement (IWA 11:2012). The inability of the test
method and the questionability of some of the metrics
means it cannot rate stoves in a way which reasonably
predicts future performance. This threatens the ability
of the ICSI to provide a classification framework that is
beneficial to the Government, local vendors or users.
The WHT burn cycles and testing framework
developed during this initiative provide meaningfully
accurate relative performance rankings by replicating
average local household practices in a controlled lab
setting. Furthermore, the metrics used and the
calculations made have been reviewed for relevance
and correctness.
a. Distinction between technical vs social solutions
The development of the WHT, a revised version of the
18
more general Heterogeneous Testing Protocol (HTP),
has centered on describing and blending culturally
relevant cooking cycles that can be used in a lab
setting within an already validated testing framework.
This approach overcomes the long-standing and
conflicting demands for a well-controlled, accurate,
technical assessment that can be done while
measuring a realistic cooking event. Previously these
goals have been thought of as incompatible, separate
assessments which usually resulted is incomparable
and/or completely misleading performance ratings,
since what was done in the lab was so different from
what is done in the field. The HTP allows the 'field' to be
brought into the' 'lab'.

A persistent tendency within the stove world has been
to confuse the test and evaluation method with the
burn cycle employed during that test. The WBT
method has historically only used a single three-stage
burn cycle. The method and the burn cycle have often
been confabulated into a 'single thing'. Combined with
the misunderstanding that performance ratings are
somehow valid independent of the fuel properties and
burn cycle, the stove community has been repeatedly
let down when field performance turned out to be very
different from predictions made in the lab.
The measurements and calculations performed by a
protocol are standard no matter what one does with
the stove during the test. One is a measurement
system, the other is the thing being measured. A test
method can be validated independently of the burn
cycle employed. A validated test method used with an
unrepresentative burn cycle will give a misleading
prediction of future performance. A defective protocol
used to evaluate a stove tested while conducting a
locally relevant cooking cycle will still give an
unreliable rating. To make a valid assessment, the test
method must be valid and the burn cycle, fuel and pots
must be relevant to the community in which that stove
will be used. That assessment is not necessarily valid
for any other community unless the pattern of use and
fuels happen to be similar. No assessment is 'generally
tr ue'. No stove has generic per formance
characteristics, and certainly not characteristic fuel
consumption or emissions of smoke since these
variables alter according to local circumstances.
Cooking stoves have been proposed as a viable
solution for a variety of global social problems. This
has been termed the “technologisation of social
issues” involving the use of “technical panaceas” and
19
“encompassing solutions”. The policy focus of the
ICSI public health due to well-established links
between poor air quality and negative health
consequences. The overarching goal of the program is
to provide access to modern technology through
19 Samer Abdelnour, “Technologizing Humanitarian Space: Darfur Advocacy and the Rape-Stove Panacea,” International Political Sociology 8(2) (June 2014): 157.
Available here.
20
Ibid: 157.
From Water Boiling Test to Water Heating Test 9
the adoption of more efficient stoves which reduce
household air pollution (HAP).
Abdelnour's critique of the promotion of “efficient
stoves as a panacea for rape risk” presents useful
insight into the importance of understanding the local
context and the root causes of social problems. He
reminds project teams to remain mindful of the
variations in social context and the importance of
developing a holistic view of the local biomass
economy in order to ensure the needs of local users
are fully met.
“Rather than engage with complex reality,
technical panaceas legitimate the delivery of
universal 'solutions'. As our study
demonstrates, this amounts to a subtle yet
profound shift in humanitarian agendas and the
technologizing of humanitarian space: the
struggle to understand and prevent sexual
violence is replaced by the quest to design,
produce, promote, and deliver the most fuel-
20
efficient stoves.”
The tendency of organizations involved in cookstove
initiatives has on occasion been to attach multiple
agendas to technical solutions that reflect the
aspirations of marketers and manufacturers and a
variety of governmental and non-governmental
organizations. This ultimately overshadows the
needs and aspirations of users. The aspirational
targets of users are contextually dependent and have
been regularly overlooked or misinterpreted by
testers, donors, and proponents of various cookstove
initiatives. These complications must be addressed.
The methods used in this pilot program seek to
address health impacts and the needs of consumers
in the context of Yogyakarta Province. The
incorporation of the social cooking context into the
evaluation method allows results to be weighted both
for technical and social considerations and includes

and social considerations and includes new metrics
that span both fields. This is expected to overcome the
shortfalls of the past and positively impact the eventual
widespread uptake of high performance stoves
certified by the ICSI.
b. Results-based financing to ensure accountability
of initiative to users
The results-based financing used in this CSI will
provide subsidies to manufacturers for their
cookstoves at two levels of consumption and usage:
1.70% of determined subsidy will be granted to the
manufacturer based on the sale of the product.
2.The remaining 30% of the subsidy is granted to the
manufacturer only if field visits confirm that
consumers are using the product.
The purpose of an RBF approach is to ensure the
accountability of manufacturers and provide an
incentive for the maintenance of a high standard
product as classified through testing. Furthermore,
manufacturers are held accountable to consumer
demands and needs through the requirement that the
stoves must be proven to be in use once purchased.
“The promotion of clean cooking solutions
should understand market segmentations,
adapt to local conditions, and be consistent with
21
and adjust to long-term development patterns.”
The initiative's RBF mechanism sets out to ensure that
the policy of the public sector and its objectives are
met, and that the private sector is rewarded for
delivering these desired outcomes. The objective of
the program is not solely to introduce clean
cookstoves to local markets but to ensure that they are
actually used once purchased in order to deliver the
health benefits sought by the public sector.
The behavioral change required by consumers in order
to ensure sustained uptake of the clean cookstoves
certified in the CSI is variable based on current local
21 World Bank, 2013: 38.
22
Abdelnour, 2014: 623.
23
Cook, 2013.
From Water Boiling Test to Water Heating Test 10
practices determined by varied geographical, socio-
economic and cultural factors. The onus of ensuring
the continued use of the stoves is placed in part on the
producer as a result of RBF.
Furthermore, RBF seeks to distort the local market as
minimally as possible and enable local producers to
shape the biomass market economy. This is unlike
traditional subsidy models which “[distort] the supply
of stoves by subsidizing production runs on stove
designs which continue to rely on non-indigenous
22
materials”. The subsidy model which has been
applied commonly by cookstove projects keeps
prices of cookstoves artificially low while providing
few opportunities to local producers to develop
sustainable business practices that meet local needs
through local market exchanges.
c. Involvement of manufacturers in testing process
Phase II of the initiative gives an opportunity to assist
stove producers and marketers to improve the
technical performance and social acceptability for
their stove products after receiving critical feedback
on its performance on CCTs. At this stage,
manufacturers and vendors have the opportunity to
introduce changes to the design, components,
materials, and operating instructions for their stove.
This may dramatically improve either the stove's
technical performance, social acceptance, or both.
This stage fortifies the initiatives' emphasis on local
demands and market control.
The proposal of pre-screening of stoves prior to
inclusion in the field testing has been put forward as a
23
result of the field study inquiry. Pre-screening is done
before the top choices for cooking stoves are
presented to the general public. This is to ensure that it
is not a free-for-all access of manufacturers to local
market, in part to guarantee quality of product as well
as provide a mechanism for the protection of local
manufacturers from global competitors.
 From Water Boiling Test to Water Heating Test 11

“It does not make sense for the CSI Program to
simply start a wild stampede between local
and international stove manufacturers and
vendors without going through a preliminary
multi-stage screening process to decide what
candidate stoves, producers and vendors
(market aggregators) for inclusion in its RBF
program.” 24
Therefore, one of the purposes of pre-screening is to
ensure that local producers are not at a disadvantage
in comparison to international competitors which
could be interested in entering the Indonesian market.
The goal is to predict the in-home performance and
acceptability of a candidate stove.
The cooking cycles that will be replicated in the YDD
lab include:
a. A typical 'breakfast cooking cycle'. Boil 1 liter of
water to parboil rice; bring to boil another pot
containing 2 liter of water preparing the rice steamer;
reducing the heating power to steam the rice; finally,
increasing the heating power to boil of 3 liters of water
for drinking

DEVELOPMENT OF WATER HEATING

TEST (WHT) METHOD

The collation of lessons from field studies and the

resultant socially accountable business plan has
informed the development of the Water Heating Test to
ensure the desired outcomes of all stakeholders are
met. The previous discussion intends to show that
different stove programs, users and distributors have
particular priorities which must be accommodated by
Steaming rice using “Keren”traditional stove
technical testing and functional evaluations. The WHT
Technical Test used in the ICSI lab implements this b. A common cooking cycle for 'coconut milk chicken
holistic, contextual testing approach. soup'. Boil coconut milk to a high power; reduce the
heating power to a low level to simmer the soup for a
a.Replication of local cooking practices in test long time; replace the pot with wok for cooking
cycles "sambal" still using low level to frying the
chilli/sambal.
This pilot program is evaluating a testing methodology
that considers the local context of cooking and
provides consumers with a cookstove classification
system that is a solid indication of whether or not the
stove they are purchasing will serve their cooking
needs. While not the first to break from the WBT, it
specifically recognizes the need for a standard testing
protocol that delineates what to measure and how,
while replicating common cooking cycles. The 'local
context' in this case consists of the common cooking
cycles, pots and fuels popular in Yogyakarta province.
Coconut milk chicken soup

24
Ibid

Note: Other cooking cycles may be selected with the
goal of covering the cooking power range needed in
that culture, as many different cycles as are needed to
achieve this. A stove that exhibits the required range of
functions (such as holding the pots securely, igniting
in a reasonable time, raising, lower or tapering the
heating power level and efficiently combusting the
available fuels) is assumed to be acceptable in the
market as well as in line with the program goals.
Cooking cycles can be combined, as well as weighted
for frequency of use, to produce a numbered Technical
Test heating cycle. It is the appropriate Technical Test
that is replicated in the controlled conditions of the
laboratory.
It is recognized that the Technical Test does not predict
the performance or acceptability of the same stoves at
other locales where the fuels, cooking cycles, pots,
foods and power range requirement are significantly
different. It does however overcome the long-standing
problem of laboratory-based stove performance
ratings made using an arbitrary cooking simulation. An
arbitrary cooking simulation offers little information on
a stove's performance in any community. The WHT
instead prescribes what will be measured, how and
when, and what calculations will be done to provide
the reporting metrics while leaving the simulated tasks
to be defined according to the local context.
For example, one criticism of the high and low power
testing procedure of the WBT is that the sequence
high-low sequence measured when the stove is
already hot gives a performance rating advantage to
25
high-mass stoves. High-mass stoves are able to store
heat from the cold start and use this during the
simmering stage following a second high power start.
Furthermore,
“A stove might operate inefficiently if too much
power is provided for the needed task. Test
results are highly dependent on the power
level chosen. Some guidance should be
25
GACC, 2014: 78.
26
Ibid: 80.
27
Ibid : 59
From Water Boiling Test to Water Heating Test 12
needed to boil the standard 2.5L and 5L of
water. We should collect data on observed
firepower from regional testing laboratories to
26
provide a range for users.”
While the heat storage by a high-mass stove may be
an attractive feature in some cultures, in other
contexts where the cooking cycle is short this is a
disadvantage and results in lowered efficiency.
The Technical Test applied in the ICSI seeks to
evaluate performance during typical use. By using a
cooking cycle based on local practices and utensils,
the test does not advantage any stove unfairly, such
as witnessed previously with the WBT and high-mass
stoves. Rather the stove is tested in the lab how it will
be used, on average. In short, it advantages the
stoves capable of delivering the requirements of local
users.
The Technical Test simultaneously corrects a number
of conceptual and mathematical errors embodied in
the original WBT still affecting the results of its
descendants. This anticipates the work being done by
the Working Groups of ISO Technical Committee 285.
a.Indicators: Particulate matter (PM), carbon
monoxide (CO), and efficiency
The focus of the Indonesian pilot program is on
providing improvements in HAP and related health
benefits. This focus impacts the differential weighting
applied to indicators of “clean cookstoves”.
The WBT test indicates that the pollutants of primary
importance for measurement are CO (carbon
27
monoxide) and PM (particulate matter). While the
measurement of both compounds has value, the
context-specific testing method utilized in this pilot
program urges the careful consideration of the local
environment of risk factors associated with biomass
cookstoves in applying weight to the health impacts of

that will be used 'tomorrow'. That would include any
usable charcoal produced during the which,
depending on the local culture, be saved or discarded.
Actual behavior strongly affects actual fuel
consumption. A WBT always starts with new fuel and
treats remaining charcoal as unburned raw fuel when
determining the 'energy consumption'. No fuel from
previous replications is used in subsequent tests. If
users normally extinguish the charcoal immediately
after cooking (a common practice in some cultures
and for which there are purpose-build clay containers)
the emissions into the home are impacted, thus
affecting the fuel consumption rating.
The WHT measures fuel consumption based on the
behavior of local users by providing a new definition of
'fuel consumption' as distinct from 'energy
consumption'. Where cooks re-use the remnant wood
and charcoal, this behavior is replicated in a series of
tests which disregard the first in the series. Meaning,
the first test is conducted with raw fuel, after which
remaining fuel is carried forward to the subsequent
test. Only the subsequent test results are considered.
Where users allow the charcoal to continue burning
after the cooking cycle (for example to provide light or
dry fuel for the next day) the remaining fuel is allowed
to burn and measurements are taken until the end of
the burning process. This ensures that all emissions
from this post-cooking burning are accounted for in
the measurements of fuel efficiency, CO, and PM. Two
sets of metrics can be reported if desired: the
performance of the stove itself during cooking, and the
influence on air quality of the local culture which
depends on what happens to the remaining fuel.
The ICSI's WHT considers partially used, burnable fuel
and incorporates it into the test cycle. Because not all
stoves can accept remnant fuel, the WHT considers it
'consumed'. If the stove manufacturer indicates what
portion or type of fuel can be 'recycled', the stove is
tested accordingly. While most (but not all) stoves on
From Water Boiling Test to Water Heating Test 14
the market in Indonesia can re-use partly-burned stick
fuel, few are capable using remaining charcoal.
Therefore, it is important to take into consideration the
ultimate disposition of this charcoal after the
completion of cooking cycles. Is it allowed to burn out,
removed from the room, or extinguished?
The WBT (always conducted with new raw fuel)
attempts to deal with the same issue by assigning an
energy content to the remnant charcoal but makes no
provision for dealing with partially burned, dry and re-
usable sticks. The WBT calculates the amount of
energy consumed, not the amount of raw fuel needed
to replicate the burn cycle. This quantity of energy is
not analogous to the consumption of raw fuel because
the energy in remaining fuel, even if in that culture it is
thrown away, is considered to be 'unconsumed fuel' in
the energy accounting calculation.
The WHT breaks with the WBT both conceptually and
practically by determining the actual amount of new,
raw fuel required to replicate any test cycle, in a series
of identical cycles, save the first (which is used to
generate the first 'leftovers').
A detail of the WBT's energy deduction for remaining
charcoal is that the ash remaining is, according to the
protocol, treated as unburned charcoal. Depending on
the fuel type, this can lead to a large error. The
Environmental Protection Agency (EPA) has used a
WBT variation which determines the actual energy
content of the charcoal. However, this is not standard
practice as most labs cannot make this determination.
Other options remain but this is considered a WBT-
specific problem that only arises when one attempts
to determine the energy consumption according to the
WBT, rather than the fuel consumption, which is the
ICSI program goal. The WHT's conceptual and
mathematical treatment of ash correctly rates the
stove regardless of the ash content of the fuel.

different pollutants. CO is arguably of less importance
in the Indonesian context due to the open-air nature of
the typical kitchen in this tropical region. The ceiling
height and permeability of typical homes makes the
concentration of CO at dangerous levels unlikely.
Traditional kitchen in Yogyakarta
Furthermore, the relative importance of pollutants is
politically determined by policy decisions that propose
clean cookstoves as solutions for certain health,
social or environmental issues. In the case of the
Indonesian stove program, the government has
emphasized the role that clean cookstoves can play
improving public health. As a public health policy
initiative, the value of measuring PM outweighs the
perceived value of fuel efficiency due to the correlation
between HAP by particulate matter (PM) from solid
fuel combustion versus the easy availability of
sufficient fuel.
The indicators measured in this pilot program include:
carbon monoxide (CO), particulate matter (PM), and
fuel efficiency. The better the performance, the more
performance 'stars' are awarded to that model of
stove. The stars are allotted based on performance on
these three indicators. As a result of the CSI's
Indonesian context as well as the pilot program's
focus on clean cookstoves for public health benefits,
differential weight is applied to each of the three
indicators in order to provide a classification system
that is reflective of the desired outcome of the ICSI.
From Water Boiling Test to Water Heating Test 13
Due to the high level of air flow and openness to the
outdoors of the typical Indonesian household, CO is a
minor health concern with little risk of high levels of
accumulation. On the other hand, moist fuels tend to
create more particulate matter. Thus PM reduction is
of particular interest to the program and an integral
measurement to ensure that cookstoves which
reduce this type of HAP remain central to the
certification process.
The number of stars available for each of the three
indicators is yet to be confirmed. However, the
expectation is that more stars will be available for
attainment of larger reductions in PM emission
compared with CO. This method proposes the
payment of subsidies to manufacturers based on the
total number of stars earned while allowing for a
different number of attainable stars for each indicator.
The purpose of allocating stars based on the relative
importance of each indicator in the health-focused
pilot program is to ensure greater control over the
translation of stove benefits to potential consumers.
With HAP and its resultant health impacts as the focus
of the program, a contextually-relevant and
consumer-focused initiative must present test
findings to the public in a manner that adequately
indicates if a particular cookstove is capable of
addressing and mitigating the health risks in question.
Therefore the differential weighting of the three
indicators seeks to ensure the maintenance of
emphasis on the health impact of dirty or traditional
cookstoves.
a. Measurement indicators
A stove's fuel efficiency rating is impacted by the
protocol's mathematical treatment of remnant
charcoal. The WBT has not dealt adequately with this
matter causing fuel consumption ratings to conflict
with actual fuel consumption. WBTs do not consider
what happens to re-usable fuel remaining from 'today'

d. Definitions of terms and metrics
This testing method additionally seeks to address
possible confusions with common definitions and
metrics, including those used by the WBT.
a. Fuel Consumption
The amount of new fuel required to initiate and
complete a task within a sequence of repetitive uses.
Residue of char or partially pyrolyzed wood is a
significant element of the calculation – is it discarded,
or is it used in next fire-making cycle?
For calculating GHG emissions, the use of residual
char for combustion in a secondary device or any non-
combustion use is an important factor for some
28
programs.
a. Fuel Consumed (Fc)
The fuel consumption of a biomass burning stove is
defined as the mass [kilograms] of new fuel drawn
from a supply that is sourced outside the cooking
system needed to conduct any one of a series of
25
identical replications of a burn cycle, save the first.
28Annegarn and Pemberton-Pigott, ETHOS 2014, Kirkland WA, (24 January 2014).
29
Bailis et al., 2003: 23.
From Water Boiling Test to Water Heating Test 15
The WBT defines 'wood consumed' as “the mass of
wood that was used to bring the water to a boil found
by taking the difference of the pre-weighed bundle of
wood and the wood remaining at the end of the test
29
phase”; though that is not the whole story. Because of
the conceptual differences in the treatment of remnant
wood and charcoal, the two fuel consumption metrics
are not directly comparable.
c. Heat Available in the Fuel (HF)
The total heat available from the perfect combustion of
the Fuel Consumed calculated from the heating value
per unit mass As Received [AR]. HF is expressed in
25
unit [MJ].
d. Net Heat Gained (HNET)
This is the net heat retained by a cooking vessel during
a burn cycle and is expressed in units of MegaJoules.
It includes the heating of the pot and its contents plus
the heat of evaporation of water, but excludes other
heat flows through the pot, specifically radiative and
25
convective losses from the pot sides and top.
A thermocouple use to measure the temperature inside the pot

e. System Efficiency (ç)
The ratio of the net heat [HNET] gained by a cooking
vessel divided by the heat available [HF] in the fuel
consumed [FC], expressed as %. Synonyms include
Overall Thermal Efficiency and Overall Energy
25
Efficiency.
This is the WHT's thermal efficiency metric. It is
[HNET] divided by [Fc] expressed as %. It is the same
as the fuel consumption (as most people understand
it) and used for comparing the relative fuel
consumption of different stoves performing the same
tasks with the same pots and fuel.
It is significantly different from the 'thermal efficiency'
or 'energy efficiency' numbers of other test methods,
including the WBT, that calculate the heat gained by the
contents of a pot divided by the heat theoretically
released by the fire – a metric analogous to the 'heat
transfer efficiency'. The heat transfer efficiency of a
stove is a poor predictor of its system efficiency (=
fuel efficiency).
f. Carbon Monoxide mass per MJNET
The mass of carbon monoxide emitted per MegaJoule
of energy gained by the pot is the WHT metric used to
rate performance. The number of MJ of energy gained
by the pot is the useful energy. People cook until the
task is completed, not until a mass of fuel is
consumed. Therefore, this metric uses the energy
available for cooking (in the pot), rather than a mass of
fuel or the energy released by the fire in the
denominator.
From Water Boiling Test to Water Heating Test 16
g. PM2.5 mass per MJNET
The mass of PM2.5 emitted per MegaJoule of energy
gained by the pot is the WHT metric used to rate
performance. The number of MJ of energy gained by
the pot is the useful energy. People cook until the task
is completed, not until a mass of fuel is consumed.
Therefore, this metric uses the energy available for
cooking (in the pot), rather than a mass of fuel or the
energy released by the fire in the denominator.
h. Heat Flow Rate (HFR) into the pot, per unit area
(heat flux)
The rate at which heat enters a cooking vessel per unit
area of heated surface, normally taken to be the area
of the bottom of the vessel. It is a measure of cooking
power per unit area expressed in units [J/s/cm2] or
[W/cm2].
The measurement may be made for any diameter of
pot used during a test cycle, but is usually reported for
the largest diameter. The diameter should be reported
together with the HFR value, or indicated by clear
implication in the body of the report. Sunken pots and
skirted pots will be treated differently, with the heated
surface area calculated appropriately.
This metric is used to rate the cooking performance in
order to estimate whether or not the cooks in the locale
of interest would accept the 'cooking power' of the
stove.The HFR is analogous to the 'cooking speed'
and can be applied to any pot size because it reports
the number of watts per square centimeter. It is not the
fire power, or the total cooking power, but is the
cooking power factored for pot size. Acceptable
cooking power is culturally determined so it varies
from region to region. If a stove meets the requisite
HFR, customers will not complain that it is
'underpowered'.

This is also called the Heat Flux or the Density of Heat

Flow depending on which language is used. For
example