Академический Документы
Профессиональный Документы
Культура Документы
ABSTRACT
The growing traction of performance-based efficiency has major implications for the role of
measurement and verification in energy programs. However, the policies that have enabled meter-based
pay-for-performance in three early adopting states (California, Oregon and New York) have
notable differences from each other. In this paper, we summarize the specific pathways of regulatory
and market reform, legislation, and executive order that these three states have followed as means
for adoption. Comparing the similarities and differences of these pathways offers insights for
other states and jurisdictions considering such a transition. In addition, we will examine how the
execution of these policies has also led to differences in the rules and guidelines adopted in each state.
The pathways will be reviewed in three generic categories. The “Market Focus” represents
a transition to meter-based procurements with authorization to scale enabled through regulatory
and market reform. The “Pilot Programs and Procurements” path begins with a smaller-scale effort
that is proposed and tested and allowed to expand to greater procurements, enabled through
legislation that redefines energy savings. Finally, the “Focus on Existing Contractors and Vendors” path
relies on the past performance and program designs of existing contractors to form the basis of an
incremental launch into pay-for-performance, enabled through executive order.
The challenges and opportunities for each path will be presented based on experience in
each state, and the adoption process and early lessons learned will be of particular focus.
Introduction
Meter-based pay-for-performance has been taking shape in several states over the past several
years. In each state, the climate for adoption and the motivating policy drivers have differed but the
opportunities for meter-based pay-for-performance appear consistent.
That consistent, multi-pronged expectation that meter-based pay-for-performance will aid in
aligning incentives to maximize the energy savings per dollar spent on energy efficiency,
improve customer satisfaction, and reduce the amount of administrative oversight required of third-
party program implementers. The promise of pay-for-performance also includes a major
transformation of current energy efficiency models by replacing centralized programs with animated
markets that are flexible, technology-agnostic, and that pay efficiency aggregators for actual savings at
the meter. In this paper we will explore how different parts of this “promise” have led to different
approaches to adapting to this new paradigm.
The states highlighted in this paper have asked, through different policy mechanisms, to carefully
consider, and take action to implement a meter-based pay-for-performance. In this paper we will
1
Any opinions expressed, explicitly or implicitly, are those of the authors and do not necessarily represent those of
the New York State Energy Research and Development Authority.
summarize the process of adoption and the intentional steps of these early adopters to ensure success up
and down the market chain. We will also share some of the early lessons that are emerging from each
jurisdictions’ experience and key issues to monitor into the future. The experiences of these three states
help illustrate the various pathways by which this transition has been initiated and the challenges of
executing on that direction to aid others as they consider similar transitions.
Pay-for-performance has been a part of demand side management program considerations for
over 25 years, and a wide range of lessons learned can be found in the literature (Szinai, 2017). The
particular variant of pay-for-performance discussed in this paper can be characterized as “meter-based”
because the performance payments are based on savings quantified using meter data (monthly, daily, or
hourly). To enable this program design, data and measurement and verification infrastructure are
essential to enable consistent and transparent calculation and adjudication of performance payments. For
the three states discussed in this paper the CalTRACK whole building methods executed through the
OpenEEmeter are used as the basis of the settlement calculations and ongoing savings tracking for the
programs.2
2
CalTRACK (www.caltrack.org) is a set of open source, empirically validated and replicable methods for calculating
energy savings by comparing weather normalized pre-and post-retrofit energy use for a given customer or
portfolio of customers. CalTRACK methods are the basis for the OpenEEmeter, which is an open-source calculation
engine that is available as Python code at https://github.com/openeemeter/openeemeter.org.
Meter-based pay-for-performance, like any other new approach to tackling energy efficiency, has
had to find a path through significant shifts in state policy, pilot adoption, or incremental testing. These
pathways have had an influence on the size, scope, and likely learnings from meter-based pay-for-
performance models in action today; each should be considered as part of the picture when evaluating
the ultimate success of meter-based pay-for-performance.
In most states, a legislative mandate of some sort sets the basis for investment in energy efficiency
and other clean energy initiatives. This may come in the form of an Energy Efficiency Resource Standard,
or broad-brush mandates to capture all cost-effective energy efficiency before investment in other
resources. As of 2018, twenty-seven states in the United States have Energy Efficiency Resource Standards
policies, which establish specific energy savings targets that utilities and program administrators must
meet through customer energy efficiency programs. Seven states require all cost-effective efficiency
(Berg, 2018). The details on how to optimize these interventions are then left to regulators, designated
administrators or utilities to work out. In other jurisdictions, very specific legislation may dictate priorities,
pathways and value structures. Most jurisdictions also have evaluation activities to validate savings and
provide feedback for ongoing program improvement.
The three states highlighted in this paper vary only slightly on these expectations and did not likely
affect their adoption of meter-based pay-for-performance. Last year (2018), New York adopted
significantly accelerated energy efficiency targets, which will double utility energy efficiency achievement
over 2019 to 2025. California falls in the category of capturing all cost-effective efficiency, for
administration by investor owned utilities, under rules established by the California Public Utilities
Commission. Oregon’s investor-owned utility is obligated to capture all cost-effective energy efficiency as
determined through integrated resource planning. All three states have been implementing and
evaluating energy efficiency programs for decades, and can be considered fairly mature markets.
While the mandates for capturing energy efficiency resources are similar in these three states,
the pathways for adoption of pay-for-performance models varied slightly. They ranged from a state wide
initiative for regulatory reform and market-based solutions, to a legislative mandate, to executive order.
These variable origins have influenced who was the lead entity, the structure of partnerships and, to a
degree, the size and nature of the pay-for-performance efforts. All three are using a standardized
measurement and verification approach as a core part of program deployment.
Models of Transition
Focus on Existing
Market Focus Pilot & Procurement
Contractors
Generic Large scale pilot with Large scale pilots and Step-wise testing with
Description focus on market third-party procurements contractors delivering
development existing programs
Market Focus
New York provides the best current example of the general category of “Market Focus”
approach to launching pay for performance. The Reforming Energy Vision (REV) was launched to
optimize the grid overall. The Clean Energy Fund (CEF) was adopted as a core component of REV and
was coincident with the development of working groups responsible for reviewing and considering
various approaches for market innovation. Based largely on that input New York State Energy Research
and Development Authority (NYSERDA) took the lead in establishing pay-for-performance for New York
State.
Adoption Process
The Reforming Energy Vision was adopted in New York by Governor Andrew Cuomo alongside the New
York Public Service Commission (PSC) in April 2014.
“This initiative aims to align electric utility practices and our regulatory paradigm with
technological advances in information management and power generation and distribution.
These developments promise improvements in system efficiency, greater customer choice,
and greater penetration of clean generation and energy efficiency technologies, but only if
barriers to adoption are eliminated and proper regulatory incentives are established.”(New
York Public Service Commission 2014, 2)
While REV provided a focal point for market driven solutions, including for energy efficiency, the
adoption of the CEF was a critical component of REV that identified the need for a change in the status
quo deployment of energy efficiency and also created the Clean Energy Advisory Council.
“The status-quo must evolve to a model that recognizes the appropriate use of targeted
programs combined with spurring private sector involvement to reach the level of scale needed
to realize our objectives. Transitioning from predominately government-directed resource
acquisition approaches to market-based initiatives that intrinsically recognize the value of clean
resources requires careful planning, along with a long-term commitment to the market.” (New
York Public Service Commission 2016, 4)
“…the core of the P4P model is the design and alignment of the performance-based requirements
between the program administrator and the service provider as well as the corresponding
services/requirements between the service provider and the customer. The program design and
service provider contract need to address key features to best ensure performance, cost-reduction,
and meaningfulness of savings.” (New York 2018, p35).
Based on the analysis and recommendations, pay-for-performance was identified as a model that
could create a market eco-system for energy efficiency investment and NYSERDA took the lead on
developing a pilot, reaching out to the utilities to collaborate.
NYSERDA and their utility partners have gathered input from market stakeholders throughout the
process to ensure a successful transition. Multiple stakeholder meetings were held in the early phases of
the pilot development as well as other informal input. In the fall of 2017, the first stakeholder meetings
were held to gather input on the principles for residential and commercial program design. Feedback was
used to inform and guide the Investment Plan necessary for NYSERDA to secure funding. In early 2018,
NYSERDA hosted the second stakeholder meeting that was part of a larger roundtable on Innovative Utility
Strategies and Emerging Energy Efficiency Building Models. At this stakeholder meeting, NYSERDA
released a conceptual framework that laid out a long-term plan for pay-for-performance executed in
three-phased approach that was outlined the Investment Plan. Additional stakeholder meetings held in
2018 supported further design refinements specific to commercial program designs and consultants also
contributed to the design work.
The NYSERDA Investment Plan (NYSERDA, 2018), approved by the Department of Public Service
includes a series of pilots to be rolled out over the next several years. With Con Edison, the Authority will
focus first on small to medium businesses (SMB) in Westchester and Staten Island, and make use of Con
Edison’s deployment of AMI meters in these areas. The residential pay-for-performance pilot with
National Grid that will follow will be focused on Onondaga, Oneida, and Oswego counties. Each pilot is
intended to test different approaches and use cases to create broad learnings that will enable utility
adoption at scale. Staging the deployment of a new pay-for-performance pathway allows all parties to
familiarize themselves with the process and better understand risk.
In May of 2019 NYSERDA and Con Edison jointly released the “Energy Pro Request for Proposals”
publicly to solicit written feedback from interested service providers and other stakeholders. It included
detailed information on how proposals would be scored, the measurement and verification requirements,
data security requirements, and protocols for data access via Green Button Connect. This level of detail
Early Lessons
The broad scope of REV has meant that foundational elements like investment in data
infrastructure are just now getting attention, leading to data standards and infrastructure evolving at the
same time as pilot development. With limited reference points for the design of the pilots (but leveraging
information from Pacific Gas & Electric), creativity, patience, and commitment to learning from other early
adopters is necessary. Limitations of tools and models are revealed as specific use cases are explored.
Sharing early and often has been a key element for progress to date, particularly around boundaries
defining what’s possible and a “good enough” implementation. While the high degree of coordination
required – between NYSERDA, investor-owned utilities, market actors, and the Department of Public
Service – can be challenging, and it also creates the opportunity to develop a unified pay-for-performance
model for the state that will be easier for market actors to navigate. Pilot learnings will continue to build
confidence in the market and will lead to iterations that support a scaled adoption.
As NYSERDA focuses on this market transition, communication with market actors has been
essential to shape a system that can work as effectively as possible for all. Providing guidance, and detailed
information around requisite qualifications in advance serves to prepare market actors for successful bids.
Early adopters, iterating as they grow in experience, inevitably will also be learning firsthand the lessons
to be successful.
The California experience in adopting meter-based pay for performance provides a good example
of a pilot and unique procurement opportunities emerging in close to the same timeframe. Kicked off with
a legislative mandate, regulatory guidance soon followed but continues to be refined as larger scale
applications of meter-based pay for performance emerge.
Adoption Process
The pay-for-performance market adaptations that emerged in California were sparked by state legislation
and developed in a smaller scale launch of meter-based pay-for-performance at Pacific Gas & Electric. The
residential meter-based pay-for-performance program was initiated in response to both legislation passed
in 2015 (SB350 and AB802) and regulations that were finalized in December of 2015 (CPUC, 2016).
The Clean Energy and Pollution Reduction Act of 2015 (Senate Bill 350) called for a doubling of
energy efficiency to meet climate goals as well as the adoption of a carbon-based Integrated Resource
Plan. Among the solutions included in the legislation was the call for the CPUC to: “. . . Authorize pay-for-
performance programs that link incentives directly to measured energy savings.”
In response to the legislation, the California Public Utilities Commission issued a framework, via
ruling, to facilitate the initiation of a series of High Opportunity Projects and Programs (CPUC, 2015).
Pacific Gas & Electric took the lead on submitting one of the first meter-based pay-for-performance
models. The design of this model allows qualified aggregators (companies or other third parties that
bundle efficiency) to bid for the opportunity to join the program (or trusted existing partners) based on
estimated savings, market intervention plan, and the bid price. Those that win will work with contractors
to implement sets of interventions and be paid quarterly based on savings achieved.
“…programs that use NMEC [meter-based] for savings determination and incentive payment
should incorporate a pay-for-performance element that not only provides adequate motivation to
pursue metered savings, but also provides such motivation to the market actors that have access
to performance information and the ability to improve or affect performance as it evolves.”p43
(CPUC, 2018)
Early Lessons
3
The California Energy and Demand Side Management Council held an EM&V Forum, on January 31, 2019 where
the utilities discussed their third-party program solicitations. https://cedmc.org/events/emv-nmec-forum/
The Energy Trust of Oregon provides a good example of an incremental approach to considering
adoption of meter-based pay-for-performance. Their initial steps were to track past performance,
monitor population changes in consumption, and focus on deployment with existing contractors. By
leveraging its long-standing relationships with trusted contractors, Energy Trust aims to evaluate and
understand multiple facets of the transition to meter-based pay-for-performance before “going big.”
Adoption Process
In 2017, the Energy Trust of Oregon hired an advanced measurement and verification firm to
quantify changes in energy consumption for residential customers, and test calculational systems to have
confidence in the approach. Shortly thereafter, Governor Kate Brown signed an executive order calling on
the Oregon Public Utilities Commission and Energy Trust to “expand meter-based savings pilot programs,
including pay-for-performance pilot programs” as one piece of a larger call to action on climate goals
(Oregon, 2017).
The prospect of expanding meter-based pay-for-performance raised several questions for the
Energy Trust that they will answer as they develop and deploy the pilot program, and that provide strong
guidance that California and New York are also considering in their evaluation efforts (Wyman, 2019).
With this long list of questions, the program administrators and internal evaluator teams have
been closely collaborating on the development of the pilot program. Monthly billing data was
consolidated for all residential customers, through a utility data sharing agreement, to support targeting
and understanding where the opportunities for interventions may lie, as well as using past program data
to conduct a “back cast” and use it to inform the program design and performance contract criteria. A
back cast, reviewing meter-based past program performance to understand future performance, provides
the administrator and implementer comparative results of a deemed program versus the same program
Early Lessons
The pilot nature of the Energy Trust pay-for-performance programs is similar to that
contemplated in several other jurisdictions. Taking a pilot-project approach allows program
administrators and implementers to focus on familiarity with the measurement and verification tools and
methods, understand existing customer consumption profiles, and design an incremental adjustment to
the existing program deployment models to answer more discrete questions about value propositions.
Unlike the other case studies, Energy Trust was able to essentially craft an experiment in the absence of
other high-pressure factors to re-invent the energy efficiency paradigm to comply with the executive
order.
The incremental pace of this experiment creates the potential for other lessons. The side-by-side
comparison of methods, or the fit of existing program designs to pay-for-performance, may not present a
realistic view of a full pay-for-performance market because the small-scale approach is not tapping into
the full benefits of potential flexibility or business models that may be better suited to pay-for-
performance. Comparisons with deemed savings and meter-based outcomes side by side will be revealing
and may create discomfort with the savings values from either approach; even if they’re both “right” they
will need to be reconciled.
This test will also reveal some key stress points such as the fact that many of the deemed
measures do not baseline against existing conditions. As a result, meter-based and deemed results will
not create a compatible comparison. Another test point is the quality of utility data. Meter-based savings
may not be applicable where intercepts are missing, for non-routine events, where there are account
changeovers (which limit access to prior usage), or accommodate prior participation and/or staggered
participation. These issues may render pay-for-performance unsuitable for broad deployment without
In reviewing the three case studies in this paper, nearly as many similarities as differences have
emerged. They exhibit some slightly different characteristics that may lead to differences in outcomes,
but it is too early to gauge what those may be. As such, jurisdictions will need to consider the specific
characteristics that are most important to them and that reflect their situation.
States and utilities that have adopted aggressive climate goals or are urgently looking for
pathways to scale distributed energy resources may find that adopting meter-based pay-for-performance
with a market focus supports a key strategy to achieve these objectives and test and improve on best
practices along the way. Jurisdictions that are new to efficiency but have a keen interest in tapping
performance for improved accountability may also wish to consider a bold transition to creating a market
environment to deliver energy efficiency under this model. If this seems like the right fit, four common
steps are revealed in the case studies:
As also revealed in the case studies, pilots are a valuable tool to test assumptions and build market
capacity. A staged implementation strategy with existing trusted contractors may be the best fit for initial
testing of the concepts:
● Step 1. The existing program implementer would start tracking savings at the meter
using consistent, transparent, meter-based methods such as CalTRACK, and develop data
infrastructure that allows them to project and monitor performance in real time.
● Step 2. The program implementer would work with the administrator to set appropriate
performance targets and agree to a percentage of payment that will be based on reaching
desired performance goals.
● Step 3. The administrator will be able to choose whether or not to competitively solicit
bids from a variety of program implementers under a pay-for-performance framework or
continue working with a single implementer with performance-based incentives.
● Step 4. Leverage insights and intelligence gained in the process and from others to
initiate, iterate and improve.
Based on this review of three pathways to pay-for-performance, it is clear that there are a range
of ways to get started. The value proposition for pay-for-performance is essentially the same in all three
cases. The relatively small variation in execution was driven by the nature of the call to action, types of
parties involved, the degree of experimentation. The common denominator is finding the appropriate
paths for managing the transition for the range of market actors involved. In these case studies this has
been accomplished through market engagement and education, practice through pilots and incremental
testing. Each provides valuable lessons learned to date, and will inevitably generate more as they are
launched, reviewed, and continued or discontinued. Any entity with an interest in using meter-based pay-
for-performance can take incremental steps to launch based on the early lessons learned during these
pilots, despite the fact that they are all works in progress.
References
Berg, W., S. Nowak, G. Relf, S. Vaidyanathan, E. Junga, M. DiMascio, and E. Cooper. 2018. The 2018 State
Energy Efficiency Scorecard.” Washington, D.C. American Council for An Energy Efficient Economy
https://aceee.org/research-report/u1808
California Public Utilities Commission. 2016. Pacific Gas and Electric Approved Advice Letter 4813-E-A –
Residential Pay for Performance Program – High Opportunity Programs and Projects. San Francisco, CA.
https://www.pge.com/nots/rates/tariffs/tm2/pdf/ELEC_4813-E-A.pdf
California Clean Energy and Pollution Reduction Act. 2015. Senate Bill 350 Chapter 547. Sacramento, CA.
http://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=201520160SB350
Clean Energy Advisory Council. 2017. REV Energy Efficiency Best Practices Guide. New York Public
Service Commission. Albany, NY.
http://documents.dps.ny.gov/public/Common/ViewDoc.aspx?DocRefId=%7B738AE607-658B-44CB-
BB51-8898FE513533%7D
New York Public Service Commission. 2014. CASE 14-M-0101 - Proceeding on Motion of the Commission
in Regard to Reforming the Energy Vision. Albany, NY.
New York Public Service Commission. 2016. Order Authorizing the Clean Energy Fund Framework.
Albany, NY.
https://www.nyserda.ny.gov/About/Funding/Clean-Energy-Fund
New York EE Advocates. 2017. Energy Efficiency Procurement Proposal. New York
New York State Energy Research and Development Authority. 2018. Clean Energy Fund Investment Plan:
Multi-Sector Solutions Chapter, Section 21.5 Pay for Performance. Albany, NY.
https://www.nyserda.ny.gov/-/media/Files/About/Clean-Energy-Fund/CEF-Multi-Sector-Solutions-
chapter.pdf
New York State Energy Research and Development Authority and Department of Public Service. 2018.
New Efficiency New York. https://www.nyserda.ny.gov/About/Publications/New-Efficiency
Oregon, Office of the Governor. 2017. Executive Order No. 17-20 – Accelerating Efficiency in Oregon’s
Built Environment to Reduce Green House Gas Emissions and Address Climate Change. Salem, OR.
https://drive.google.com/file/d/1UN8oQ6PtM90XaIuWDMPt8s8yPALepA6y/view
Szinai, J., M. Borgeson, E. Levin. 2017. Putting Your Money Where Your Meter Is: A Study of Pay-for-
Performance Energy Efficiency Programs in the United States. Natural Resources Defense Council.
https://www.nrdc.org/sites/default/files/pay-for-performance-efficiency-report.pdf
Wyman, W. 2019. “Pay for Performance” Presentation at Annual Meeting of the Association of Energy
Service Providers. Energy Trust of Oregon. Portland, OR.