08-081

Testing Strategy with Multiple Performance Measures Evidence from a Balanced

Scorecard at Store24
Dennis Campbell Srikant M. Datar Susan L. Kulp V.G. Narayanan

Copyright © 2008 by Dennis Campbell, Srikant M. Datar, Susan L. Kulp, and V.G.
Narayanan Working papers are in draft form. This working paper is distributed for
purposes of comment and discussion only. It may not be reproduced without
permission of the copyright holder. Copies of working papers are available from
the author.
Testing Strategy with Multiple Performance Measures Evidence from a Balanced
Scorecard at Store24* Dennis Campbell Srikant Datar Harvard Business School Susan
L. Kulp George Washington University V.G. Narayanan Harvard Business School
Current Draft: February 2008
ABSTRACT: We analyze balanced scorecard data from a convenience store chain,
Store24, during the implementation of an innovative, but ultimately unsuccessful
strategy. Quarterly strategic reviews, based in part on the firm's balanced
scorecard, led executives at Store24 to identify problems with, and eventually
abandon, this strategy over a two year period. We find that formal statistical
tests of the hypotheses underlying the firm's balanced scorecard and strategy map
reveal problems with the strategy on a timelier basis. We also test alternative
hypotheses to those underlying the firm's formal strategy map and scorecard that
are consistent with concerns expressed by some of Store24's top executives during
the initial stages of implementing the new strategy. Our analysis demonstrates
that this firm's balanced scorecard contained useful and timely information for
distinguishing between these alternatives. These results provide some of the first
field-based evidence on the potential for a firm's balanced scorecard to provide
useful information for detecting problems in its strategy.

I.

Introduction

This study investigates the role of the balanced scorecard in generating useful
information for testing and validating an organization's strategy. Numerous case
studies of balanced scorecard

implementations document their use in translating organizational strategies to

objectives and measures, communicating strategic objectives to employees,
evaluating the performance of business units, and aligning the incentives of
employees across business units and functions.1 Field-based and experimental
research in the accounting literature has also focused on these uses of balanced
scorecards (Malina and

The authors thank Store24 for use of its data. We thank Chris Ittner, Robert
Kaplan, Ken Koga, Joan Luft, Michael Maher, Ella Mae Matsumura Tatiana Sandino,
Philip Stocken, Dan Weiss, two anonymous referees, and seminar participants at the
AAA Annual Meeting in Orlando, Boston University, the EIASM conference, Harvard
University, Management Accounting Section Mid-year Meeting in San Diego, Michigan
State University, Ohio State University, University of Arizona, UCLA, University
of Michigan, University of Southern California, and the University of Wisconsin
for their helpful comments and suggestions. 1 See Kaplan (1998), Campbell and Lane
(2006), or many of the organizations documented in Kaplan and Norton (2006) for
examples.

1
Selto 2001; Lipe and Salterio 2000: Ittner et. al. 2003; Banker et. al. 2004;
Campbell 2008). In addition to these uses, the literature on the balanced
scorecard has long argued that these measurement systems can play a role in
facilitating feedback and learning through the testing, validation, and revision
of the underlying strategic assumptions embedded in the scorecard (Kaplan and
Norton 1996, 2004, 2008). Despite these long-standing claims, there has been
comparatively little research on this potential learning and feedback role of
balanced scorecards. The testing and validation of assumptions underlying balanced
scorecards is an important topic given the considerable uncertainty faced by
decision-makers in designing these measurement systems. The balanced scorecard
framework advocates choosing performance metrics related to key financial and
customer objectives, the firm's internal processes for achieving these objectives,
and organizational capabilities necessary to execute its internal processes.
Further, performance measures should be

explicitly linked via a "strategy-map" of hypothesized "cause-and-effect"

relationships that depict the firm's strategy (Kaplan and Norton 2000; 2004).
Improvements in measures of organizational capabilities are expected to drive
improvements in the execution of internal processes which in turn lead to customer
and financial outcomes. In this way, the balanced scorecard framework explicitly
recognizes

interrelationships between strategy-specific measures of financial and customer

outcomes and inputoriented "performance drivers" related to the firm's internal
processes and organizational capabilities. However, managers must formulate
strategies, and select related objectives and measures, based on exante
expectations about how the strategy will translate into financial performance.
Thus, the strategic objectives and performance measures chosen for an
organization's scorecard are often based on uncertain hypotheses about how
measured performance against these objectives ultimately leads to financial
performance. Proponents of the balanced scorecard concept have long recognized
this uncertainty. To mitigate against it, and as a mechanism for strategic
feedback and learning, they have advocated for formal testing of the hypothesized
linkages among the performance measures included in an organization's scorecard
(Kaplan and Norton 1996; 2004, 2008). Theoretically, multiple performance measures
selected based on

2
an organization's unique strategy, coupled with a set of hypothesized
relationships among these performance measures that describes the strategy (e.g. a
"strategy map" or "business model"), should provide the necessary data for
strategy validation and testing via standard statistical analysis techniques.
Particularly in organizations with large numbers of similar operating units,
statistical analysis can potentially be used to explicitly test the hypothesized
linkages among the performance measures in the balanced scorecard and provide
early feedback about the validity of the underlying strategic assumptions.
However, there are at least three reasons why such an approach may not provide
useful information even in a setting where improvements in nonfinancial
performance measures are expected to lead improvements in financial performance
with a short time-lag. First, nonfinancial performance measures related to a
firm's unique strategy may be noisy indicators of true underlying strategic
performance limiting their usefulness for statistical testing purposes. Consistent
with this notion, the problem of "quantifying qualitative information" has
frequently been noted by decision-makers as a significant challenge in
implementing balanced scorecards (Ittner and Larcker 1998). Second, an

organization's balanced scorecard and associated strategy map may not capture all
dimensions necessary for a strategy to succeed. Explicit (e.g. technology
investment) and implicit (e.g. difficulty in

measurement) costs of information collection may limit the set of performance

measures that are ultimately included in an organization's measurement system to
those related to dimensions of strategy that are easiest to measure (Goold and
Quinn 1990). Finally, statistical analysis of the hypothesized linkages among
performance measures in a balanced scorecard may not yield incremental information
relative to alternative mechanisms used to monitor performance within
organizations. For example, organizations frequently use formal strategy review
meetings to assess whether strategy is progressing as intended. Ongoing monitoring
over time of measured performance against strategic objectives may help decision-
makers implicitly test the hypotheses underlying the organization's balanced
scorecard even absent formal statistical analysis. Given these considerations, the
extent to which balanced scorecards provide useful information for testing and
validating an organization's strategy is an open empirical question.

3
We investigate this issue by analyzing balanced scorecard data from a convenience
store chain, Store24, during the implementation of an innovative, but ultimately
unsuccessful strategy. In FY 1998 Store24 initiated a new store-level strategy to
differentiate itself by improving customer experiences. There was, however,
significant variation in how much and how well individual stores executed against
Store24's implementation plan, in how customers valued this strategy, and in
financial performance across stores. Though store24 monitored store performance
via a set of performance measures formulated in a balanced scorecard, they did not
rely on, nor did they conduct, formal statistical tests of the hypothesized
relationships among the performance measures in the scorecard. Rather, quarterly
strategic reviews, based in part on the firm's balanced scorecard, led executives
at Store24 to identify problems with, and eventually abandon, this strategy over a
two year period after which they reverted back to a traditional strategy that
emphasized speed of service and operational efficiency. Our objectives in this
paper are to explore whether, when, and how information about problems with this
strategy was captured in the firm's balanced scorecard. Doing so may, in turn,
provide evidence for or against claims in the balanced scorecard literature that
these measurement systems provide useful and timely information for testing the
efficacy of an organization's strategy. Additionally, it may

stimulate new theories about the role of multidimensional performance measurement

systems in the strategic feedback and learning processes of organizations and the
conditions under which formal analysis of the data generated by these systems
provides incremental learning relative to standard strategic review practices. To
achieve these research objectives, we exploit a unique feature of our research
setting. Namely, Store24 as a research site offers three natural benchmarks
against which we can gauge the efficacy of the information in the firm's balanced
scorecard in detecting problems in its strategy: (1) the explicit hypotheses
underlying the firm's balanced scorecard and strategy map; (2) implicit
alternatives to these hypotheses based on management concerns about the merits of
the strategy; and (3) perceived problems with the strategy revealed in the firm's
formal strategy review processes.

4
We find that statistical tests of the hypotheses underlying the firm's balanced
scorecard and strategy map reveal problems with the strategy. Contrary to the
explicit hypotheses underlying the firm's balanced scorecard and strategy map,
nonfinancial performance measures related to the strategy are not, and in some
cases are even negative, drivers of financial performance. We further find
evidence for and against several alternative hypotheses to those underlying the
firm's formal strategy map and scorecard that are consistent with concerns
expressed by some of Store24's top executives during the initial stages of
implementing the new strategy. Field-evidence based on interviews at our research
site reveals that concerns about the strategy centered on issues of formulation,
implementation, and fit with the organization's existing level of employee
capabilities. As part of the customer perspective of its balanced scorecard,
Store24 executives measured the extent to which individual stores provided an
entertaining experience (i.e., a strategy-specific customer outcome measure).
Concerns about whether the strategy was well formulated arose from disagreement
among executives about the merits of this choice of strategic objective for
Store24. In particular, some top executives where concerned that, even if the
organization could achieve this strategic objective, financial returns would not
follow. We find that this is indeed the case. On average, store-level performance
on Store24's strategy-specific customer outcome metric is negatively related to
store-level financial performance even after controlling for a variety of
location- and store-specific factors. Concerns about whether the strategy was well
implemented arose from disagreement among executives about the merits of the
operating standards chosen to implement the new strategy. Store24 executives
developed a store-level action plan to implement the strategy, mapped the action
plan into operating standards, and measured store-level conformance with these
standards as part of the internal process perspective of its scorecard (i.e. a
strategy-specific input measure). Thus, all stores worked on executing against
these operating standards to implement the new strategy. There was, however,

significant variation in how well the strategy was implemented in different stores
and in how customers experienced the implementation. Some top executives where
concerned that, even if stores executed

well against these operating standards, this would not result in achievement of
the strategic objective of

5
providing an entertaining in-store experience from the customer's perspective. We
find that the firm's scorecard metrics reveal evidence against this alternative
hypothesis and in favor of the explicit hypothesis underlying the firm's strategy
map. On average, store-level performance on Store24's strategyspecific input-
metric is positively related to store-level performance on the firm's strategy-
specific customer outcome metric. Concerns about the fit of the strategy with the
organization's existing capabilities arose from disagreement among executives
about whether the skill levels of store-level employees were sufficient to
implement, and derive economic benefits from, the strategy. Consistent with these
concerns, our results indicate that cross-sectional differences in measures of
employee capabilities in the firm's scorecard account for differences in the
success of Store24’s strategy. Low employee skill levels do not directly affect
strategy implementation. But in stores with low employee skills, even when outcome
measures are high, financial performance is poor. Conversely, in stores with high
employee skills, when outcome measures are high, financial performance is strong.
These results are consistent with a "poor fit"

hypothesis in which regardless of how thoroughly Store24 implements its strategy,

for the strategy to succeed, store level employee capabilities need to be high.
Collectively, these results provide evidence that this firm's balanced scorecard
contained relevant information for detecting strategic problems and for
distinguishing among implicit alternative hypotheses (relative to those explicitly
articulated in the scorecard) related to strategy formulation, implementation, and
fit problems. However, they do not provide evidence on whether formal analysis of
the data generated by Store24's balanced scorecard provides incremental learning
relative to the firm's quarterly strategic review process which did not rely on
such analysis. Using a sub-sample of quarterly data from almost one-year prior to
the quarter in which Store24 executives decided to abandon the strategy, albeit on
a more limited set of performance measures due to data availability, we find
results that are consistent with those noted above. We view these results as
providing evidence that formal analysis of the data generated by Store24's
balanced scorecard provides timely information about strategic problems relative
to the firm's quarterly strategy review process.

6
Our study makes three contributions to the accounting literature on performance
measurement. First, we provide some of the first field-based evidence on the
potential for a firm's balanced scorecard to provide useful information for
strategy testing and validation. Prior research has largely ignored this potential
role of balanced scorecards and rather focused on their use in communicating
strategic objectives to employees, evaluating the performance of business units,
and aligning the incentives of employees across business units and functions
(Malina and Selto 2001; Lipe and Salterio 2000: Ittner et. al. 2003; Banker et.
al. 2004; Campbell 2008). Related studies in management accounting demonstrate
relationships among financial performance metrics and non-financial measures such
as product quality and customer satisfaction (e.g., Banker, et. al. 2001; Ittner
and Larcker 1998b; Nagar and Rajan 2001). However, these studies do not explicitly
analyze measures of a firm’s strategy and capabilities and, consequently, the
extent to which such measures provide information useful for timely detection of
strategic problems. Second, despite the academic evidence that non-financial
performance measures typically lead financial performance, Ittner and Larcker
(1998b) document that many executives do not tie together firm-specific non-
financial metrics with lagging accounting measures.2 Our paper shows that the

relationships between non-financial performance measures and financial performance

depend on characteristics of the strategy captured by those measures. A lack of a
relationship between firm-specific non-financial metrics and accounting returns
may be informative about (1) the firm’s strategy formulation, (2) its strategy
implementation, and (3) the fit of the formulated strategy with the firm’s
internal capabilities. We provide some of the first field-based empirical evidence
on the potential for a set of strategically linked financial and non-financial
performance measures to distinguish among these three alternatives. Third, we
extend prior research on the relationships between non-financial performance
measures and financial performance by examining the potential moderating effect of
employee capabilities. Prior research suggests that business models are typically
depicted by linear relationships between financial and
Consistent with this, Store24 management did not perform statistical analyses
linking the performance measures together, although the metrics were consistently
collected across stores and across time.
2

7
non-financial performance metrics (Rucci et al. 1998, Kaplan and Norton 1996;
2000). Except for Ittner and Larcker (1998b), prior empirical work typically
ignores potential nonlinearities in relationships among performance measures.
Moreover, these studies do not examine interactions among non-financial
performance measures as a source of nonlinearity that may moderate these
relationships (Ittner and Larcker 1998a). The results in this paper are subject to
the caveat that the field-based nature of our research limits the generalizability
of our findings. However, the unique nature of a firm’s strategy dictates that the
performance measures and links between these measures, articulated in the firm’s
balanced scorecard, are likely to be firm-specific. Future research should provide
additional evidence from other settings of the extent to which business model-
based performance measurement systems such as the balanced scorecard capture
information useful for monitoring strategic progress. The remainder of the paper
proceeds as follows. In section II we present our research site and describe the
firm's strategy and related balanced scorecard implementation. Section III
presents our empirical research design and results. We conclude the paper in
section IV.

II.

Research Setting

Store24 is a privately held convenience store retailer in New England, the 4th
largest in the region. Its stores, located through Massachusetts, New Hampshire,
Rhode Island, and Connecticut, are grouped into nine geographic divisions, each
with its own division manager. Stores are homogenous in many aspects of their
operations including compensation, technology, management structure, and product
pricing, but they vary in size, geographic location, market demographics, and
product mix. The company’s primary product categories include cigarettes,
beverages, snacks, prepared foods, and lottery tickets. Revenues totaled
approximately $180 million in fiscal year 1998 (May 1, 1998 to April 30, 1999).
Store24 employed 800 people including 740 store managers and crew and 60 corporate
level employees. The skills and experience of these employees vary widely overall
and across stores.

8
Store24 operates in a mature environment with competition from convenience stores,
gasoline retailers, and drug stores. Traditionally, convenience store retailing
focused on short-term productivity (e.g., inventory and cash control). As the
convenience store industry matured and competition intensified, marketing,
customer service, and brand name emerged as differentiating factors. Before FY
1998 and after FY 1999, Store24 did not differentiate itself; rather it focused on
excelling at traditional service quality metrics such as physical environment
(cleanliness and store layout) and quality of the customer experience (fast,
friendly service) (Fitzsimmons and Fitzsimmons, 2001). During FYs 1998 and 1999
(that is from May 1, 1998 to April 30, 2000), Store24 formulated a strategy aimed
at increasing same-store sales and margins because growing via new sites was
difficult. “Location is a primary driver of store performance. However, we are
stymied on the growth front due to a lack of acceptable new sites. This has led to
a focus on optimizing our existing sites through an increasing emphasis on store-
level marketing and operations,” explained Store24’s CFO. To achieve its goals,
Store24 changed its strategy to creating entertaining in-store atmospheres that
would differentiate its stores from those of competitors. The Differentiation
Strategy Store24 implemented this new, innovative store-level strategy during the
first quarter of FY 1998 (i.e., beginning May 1, 1998). It aimed to differentiate
its stores while maintaining performance on traditional productivity measures.
Successful retailers, such as Disney stores, offer “fun and interactive” shopping
experiences. Store24’s CEO believed that adopting a similar strategy would improve
financial performance. Store24 provided a fun in-store atmosphere by emphasizing
specific themes. Store-level strategy execution centered on a large display case
(i.e., “endcap”) featuring themeoriented promotional items and store decorations
that fostered employee interaction with customers. For example, during the old
movie theme stores featured life-size cutouts of movie stars, endcaps contained
high-margin videos of old movies, and old movies became a conversation piece. The
themes sought to attract urban adults between the ages of 14 and 29 years, a
growing market segment and Store24’s target market. A senior manager explained,
“The [Differentiation] strategy was really playing off of the urban,

9
young adult market. Marketers know that this demographic gets bored easily and
needs to be stimulated. We wanted this group to always see new and different
things in the store.” In contrast to the basic service quality component, store-
managers were accorded autonomy in implementing the differentiation strategy. That
is, although all stores were required to implement the new strategy, how they
implemented or how much they implemented varied across stores. Corporate defined a
theme and provided the endcaps, but store employees possessed considerable
flexibility in strategy execution. Thus, manager and crew skills were at least as
important as theme choice to the strategy’s success. Store24’s controller
explained, “Our best managers really took the strategy to heart. The strategy
served as an outlet for manager and crew creativity. However, other managers put
minimal effort into this strategy and even stocked traditional items such as chips
on the endcaps saying they needed the product space.” The differentiation
strategy, as originally conceived, centered on the physical environment. But the
interaction between store employees and customers was crucial to the strategy’s
success. Senior management intended the themes and promotions to serve as points
of interaction that would help Store24 establish relationships with customers and
cross-sell high margin products. Explained a senior executive, “The endcaps and
displays under the [differentiation strategy] had the dual intention of building a
rapport with customers and bumping up the average sales per customer. We felt that
store management and crew could use the displays as “ice-breakers” in talking with
customers. In addition, the margins on the promotional items featured under the
[differentiation] strategy were typically two to four times the margins of our
traditional products. When customers were browsing or “window shopping” we

encouraged store crew to direct the customer’s attention to these promotional

items.” Store24 looked to its differentiation strategy to attract new customers
and increase store sales, specifically, sales of highermargin, strategy-specific
products, and thereby boost store profits. Balanced Scorecard Performance
Measurement System Store24 used a balanced scorecard-based performance measurement
system. The company

collected information on a variety of performance measures at various levels of

the organization and at

10
various frequencies. Store24's balanced scorecard along with its performance
dimensions ("parameters"), performance measures, organizational levels of
measurement (e.g. store vs. corporate), and frequencies of measurement (e.g.
quarterly vs. annual) are provided in Figure 1. Performance measures were
organized around the four traditional balanced scorecard perspectives (financial,
customer, internal, and learning & growth). Store24 selected a variety of

traditional accounting based measures of performance for the financial perspective

of its scorecard. In addition to sales growth and asset utilization metrics,
Store24 also monitored several different profitability metrics including gross
profit, controllable contribution, EBITDA, and return on capital deployed. All
financial performance metrics were collected and monitored on a quarterly basis.
With the exception of G&A overhead and return on capital deployed which were
measured at the corporate level, all other financial metrics were collected at the
store-level but might also be aggregated at the regional or corporate level for
monitoring and review purposes.3 Performance measures in the customer perspective
of the balanced scorecard were monitored quarterly but, underscoring the
difficulty of collecting customer information within stores, were measured
primarily at the corporate level. As Store24’s CFO explained “Our customers are
informationally

anonymous. This is a high-transaction, low-ring environment. Our stores see an

average of 8,000 customers per week and an average check-size of $5. The vast
majority of transactions are cash-based.” Store24 contracted with a third-party
research firm which conducted quarterly telephone surveys of selfidentified
convenience store customers in the company’s major markets to assess the
likelihood of customers shopping at Store24, name recognition of Store24, and, for
self-identified Store24 customers, the quality of merchandise, price, and store
cleanliness. In order to measure the extent to which the company was achieving its
strategic objective of making its stores fun, entertaining places to shop, Store24
also captured a strategy-specific customer outcome metric related to its
differentiation strategy: the proportion of self-identified Store24 customers that
rated their shopping experiences at Store24 highly

Store24 eventually moved towards store-level allocation of portions of its capital

investments, valued at historical cost, for measurement of return on invested
capital at the store-level. However, this occurred later than the period we study
in this paper, and we do not have access to data on store-level return-on-capital
metrics.

11
on this dimension. While metrics for the customer-perspective were not routinely
collected at the storelevel as part of the balanced scorecard, a feature we will
exploit for our later empirical tests is that Store24 did commission a one-time
survey project to obtain store-level customer outcome metrics. Between the 1st and
4th quarters of FY 1999, the same third-party research firm that conducted
external surveys for the company solicited feedback from customers at 65 stores
about Store24, its product selection, and other factors that would persuade them
to shop at Store24 more often. Customers ranked unique attributes related to the
differentiation strategy that they found appealing; among these was “fun place to
shop,” “entertaining,” and “unexpected.” The latter measures capture whether
customers observe and value the new strategy. Throughout the remainder of the
paper, we refer to these metrics as strategyspecific customer outcome measures of
the differentiation strategy. The internal perspective of Store24’s balanced
scorecard primarily contained measures of storelevel execution of the company’s
unique operating standards. Store24 translated the components of its strategy into
a set of store-level operating standards and measured store-level conformance to
these standards via walk-through audits. During these announced visits, management
evaluated store

performance on various dimensions including in-store image, in-stock position, and

store appearance. The walk-through audit score quantified the store-level
implementation of Store24’s strategy. For FYs 1998 and 1999, the standards
included for audit reflected both the differentiation strategy as well as
traditional dimensions of basic convenience store operations and service quality.
A store’s differentiation score referred to a separate measure of conformance to
only standards related to the differentiation strategy such as actions in terms of
themes and products that would make Store24 a fun and entertaining place to shop.
This measure of strategy inputs captured the store-level activities that senior
management believed drove the success of the differentiation strategy. Senior and
mid-level corporate management measured performance by conducting walk-through
audits twice per quarter.4 Points were awarded based on compliance with 78
operating standards selected by senior management. A percentage score is

We omit the mystery shop scores due to their correlation with walk-through audit
scores and data availability. We cannot disaggregate mystery shop scores into
basic service quality and differentiation strategy implementation measures.

12
calculated by dividing total awarded points by total potential points. Store24
also measured conformance to store-level operating standards through monthly
surprise visits or “mystery shops.” The primary role of the mystery shop review,
which consisted of twenty high-level questions, was to ensure the validity of the
more detailed walk-through audit scores. Scores on the announced and unannounced
visits are significantly and positively correlated. Reflecting one focus of the
differentiation strategy on driving sales of higher-margin, non-traditional
products, the company also monitored net gross profit from new concepts at the
corporate level as part of the internal perspective of its scorecard. Senior and
division management considered employee capabilities critical to consistent
implementation of the store-level strategy. Accordingly, Store24 selected various
measures of employee capabilities in the learning & growth perspective of the
scorecard. Measures of the tenure of store managers and crew were included in
Store24’s scorecard consistent with management’s belief that the retention of
experienced employees was necessary for strong store-level operational
performance. Employee capabilities were also directly measured through bi-annual
evaluations of manager and crew skill levels. These evaluations were conducted
during the 2nd and 4th quarters of each fiscal year. Managers were rated, on a
five-point scale, on many dimensions including ability to retain, train, and
interact with crew; customer service; merchandising; time and labor management;
maintaining store safety; and technology use. A store manager’s skill rating was
the average score across all dimensions. Crew skills were rated on a five-point
scale along similar dimensions; all non-management employee scores were averaged
to devise a store’s crew skill rating. In addition to measures of employee

capabilities, the learning & growth perspective of Store24’s balanced scorecard

contained corporate and regional measures of employee satisfaction and information
technology use. Store manager and crew compensation was tied to, for example,
store-level profit and strategy implementation measures. To encourage
implementation of the differentiation strategy specifically,

employee rewards were based on both the differentiation score and total walk-
through audit score. As a result of these measures and incentives, all stores
implemented the new strategy. But, implementation of the differentiation strategy
was not straightforward. Beyond the physical environment and stocking of

13
new products, it required store staff to establish relationships with customers
and sell high-margin products. Implementation of the strategy varied significantly
among stores. Even when stores

implemented the strategy well, there was variation in how customers experienced
the new strategy. There was also significant variation across stores in
profitability. We exploit these variations in our empirical tests to examine
whether Store24’s balanced scorecard contained useful information for drawing
conclusions about both strategy formulation and implementation. Strategy Map and
Hypotheses Underlying the Balanced Scorecard The performance measures included in
Store24’s balanced scorecard were selected based on an underlying “strategy map”
which described senior management’s assumptions about cause-and-effect
relationships across the four perspectives of the scorecard. In particular, the
strategy map detailed how input metrics selected for the learning & growth and
internal process perspectives of the scorecard were linked to outcome metrics
selected for the financial and customer perspectives of the scorecard via a
hypothesized set of cause-effect relationships. Store24’s strategy-map, including
the objectives and related performance measures in each perspective, is
illustrated in Figure 2. The strategy map at Store24 had a simple structure.
Starting with the learning & growth

perspective, measures and objectives in each perspective were hypothesized to be

drivers of those in the next perspective. The strategy map did not capture all
possible relationships among the performance measures in the balanced scorecard
but, rather, focused on management’s primary hypotheses about how their chosen
objectives ultimately led to financial performance. The strategy map reflected
several straightforward and specific hypotheses about how the differentiation
strategy would result in financial performance. First, improvements in measures of
the unique internal processes chosen by senior management to implement the
strategy were expected to lead to improved financial performance via a two-step
process: improvements in measures of strategy-inputs would lead to improvements in
strategy-specific customer outcome measures which would lead, in turn, to improved
financial performance. As a starting point, it is worth considering the assumed
link between

14
the internal process and financial perspectives of the scorecard (the dashed line
in Figure 3) before considering the intermediate performance measures in the
customer perspective. H1: Ceteris Paribus strategy inputs are positively related
to financial performance. Figure 3 illustrates this and subsequent hypotheses
underlying Store24’s strategy map and balanced scorecard. H1 could be rejected if
the input metrics show no (or a negative) relationship with financial performance.
Underscoring the importance of the assumed linkages across the perspectives of
Store24’s balanced scorecard, this could occur if either of the following two
hypotheses in the strategy map were rejected. H2: Ceteris Paribus strategy inputs
are positively related to strategy-specific customer outcomes. H3: Ceteris Paribus
strategy-specific customer outcomes are positively related to financial
performance. As discussed below, the extent to which these hypotheses are
validated depends on whether the differentiation strategy was well formulated
and/or well implemented. Finally, improvements in measures of organizational
capabilities were expected to drive improvements in measures of the unique
internal processes chosen by senior management to implement the strategy (e.g.
strategy inputs). H4: Ceteris Paribus measures of employee capabilities are
positively related to measures of strategy inputs. H1-H4 collectively represent
the explicit hypotheses underlying Store24’s strategy map and balanced scorecard.
Alternatives to the Hypotheses Underlying the Strategy Map and Balanced Scorecard
We interviewed several members of the top management team at Store24 including the
CEO, CFO, COO, Controller, V.P. of Marketing, and others to gauge the extent to
which there was consensus about the differentiation strategy at the time the
strategy map was developed and/or in the early stages of implementing the
differentiation strategy. Our interviews suggest that views about the merits of
the differentiation strategy were not unanimous among top management even once the
strategy map was

15
developed. The interviews reveal that concerns about the differentiation strategy
centered on issues of its formulation, implementation, and fit with Store24's then
existing level of employee capabilities. These concerns were discussed openly
among the top management team early in the strategy development process. Concerns
about whether the strategy was well implemented arose from disagreement among
executives about the merits of the operating standards chosen to implement the new
strategy. In

particular, some executives at Store24 were uncertain early on about whether the
operating standards for the differentiation strategy were specific enough to
deliver the differentiated customer experience intended by the strategy. In a
representative comment, Store24’s CFO noted “…there was a potential disconnect
between how we intended to execute the strategy at the operational level and
understanding how the strategy helped customers and sales.” This uncertainty among
the top management team

suggests an implicit alternative to H2: strategy inputs are unrelated to strategy-

specific customer outcomes. Thus, rejection of H2 would provide evidence that the
differentiation strategy was not well implemented in the sense that the store-
level execution of the action plan and internal processes selected by senior
management to implement the strategy, as captured by the strategy input measure,
do not result in the strategy-specific customer outcomes intended by the strategy.
In the customer perspective of its balanced scorecard, the performance metric that
captured the primary strategic objective of the differentiation strategy measured
the extent to which individual stores provided an entertaining experience (i.e., a
strategy-specific customer outcome measure). Concerns about whether the strategy
was well formulated arose from disagreement among executives about the merits of
this choice of strategic objective for Store24. In a representative comment
expressing the uncertainty that surrounded this choice of strategic objective, one
senior manager noted that “…we needed to differentiate our stores, but some of us
weren’t sure this is what our customers wanted. We were throwing a wild card of
entertainment into a business that is about fast-efficient service.” Reflecting
this uncertainty, some top executives where concerned that, even if the
organization could achieve this strategic objective, financial returns would not
follow or might even decline. This suggests an implicit alternative to H3:
strategy-

16
specific customer outcome measures are not or are negatively related to financial
performance. Thus, rejection of H3 would provide evidence that the differentiation
strategy was not well formulated in the sense that achieving the strategic
objective of the differentiation strategy, as captured by the strategyspecific
outcome measure, does not result in the achievement of Store24’s financial
objectives. Questions about the fit of the strategy with the organization's
existing capabilities arose from uncertainty among executives about whether the
skill levels of store-level employees were sufficient to implement, and derive
economic benefits from, the strategy. Senior management believed successful store-
level implementation of this strategy required performance in complementary,
difficult to measure activities. To leverage the environment into financial
performance, skilled employees needed to establish customer relationships. Senior
management believed that high skill levels enhanced and low skill levels limited,
the relationship between implementation of the differentiation strategy and store
performance. Explained Store24’s CFO, “Managers and crew that were already skilled
in our core [efficiency based] strategy and other basic store operations such as
cash, labor, and inventory control, were able to devote considerably more time to
implementing the [differentiation] strategy and to tailor this strategy based on
knowledge of their customers. These skills made it easier to build the
[differentiation] strategy on top of the basics.” The success of local strategy
implementation relied on manager and crew interactions with customers and local
market knowledge. Absent these complementary activities, differentiation

implementation might not translate into improved store-level customer outcomes and
financial performance, and might, in fact, adversely affect performance.
Interestingly, there appears to have been a high degree of consensus early on
about the importance of employee capabilities not just in ensuring store-level
execution of the strategy (e.g. H4), but also in ensuring that even if executed
well at the local level, the differentiation strategy ultimately translated into
the desired customer and financial outcomes. The following hypotheses capture
these assumptions about the importance of employee capabilities. H5: Ceteris
Paribus the impact of increases in strategy inputs on strategy-specific customer
outcomes is positively related to the level of employee capabilities. H6: Ceteris
Paribus the impact of increases in strategy-specific customer outcomes on
financial performance is positively related to the level of employee capabilities.

17
Despite the relative degree of consensus among top management about the role of
employee capabilities in the differentiation strategy, the strategy map did not
directly reflect these hypotheses. Rather, these hypotheses remained implicit and
outside of the formal strategy map. Strategy Change and the Quarterly Review
Process Store24 followed the differentiation strategy during FYs 1998 and 1999.
During this time, management monitored the scorecard. Formal monitoring of the
scorecard metrics took place during quarterly strategic review meetings which, in
actuality, focused on both strategic and operational issues within Store24. These
meetings were attended by corporate and regional management with the intent of
monitoring strategic progress and identifying any operational issues that might
hinder such progress. The balanced scorecard was the primary, but not sole, source
of performance information in these meetings. During these meetings, the balanced
scorecard served as a focus for identifying areas in need of improvement. During
each quarterly meeting, overall performance on each of the different perspectives
was given a grade of A+/-, B+/-, C+/-, D+/-, or F. The grades given for
performance in each perspective reflected the consensus judgment of management in
attendance5. Once each perspective was graded, reasons for abnormally high or low
performance were discussed to identify whether the strategy represented in the
scorecard and strategy map were progressing as intended. For example, it was not
uncommon for financial performance to be strong while nonfinancial metrics in the
customer, internal, or learning & growth perspectives fell below targeted
performance or vice versa. During the two year period the strategy was being
implemented patterns were observed in the scorecard metrics that led top
management to solicit additional customer feedback and, ultimately, to question
the validity of their strategic assumptions. Store-level execution of operating
standards

(strategy-inputs) declined and then gradually increased over this period (Figure
4), and the strategyspecific customer outcome measure followed the same pattern.
In each quarter of FY 1999 Store24 posted a higher profit than in the
corresponding quarter of FY 1998 (Figure 4). Store24 management,
We do not have historical information on the grade awarded for each perspective in
each quarter. The grades were not formally recorded for use outside of the
quarterly meetings. Rather, they primarily served as a way to focus management
attention during the meetings on areas of needed improvement.
5

18
however, did not feel that they could attribute the strong financial performance
to the new strategy for two reasons. First, performance on the strategy input and
outcome metrics showed a declining pattern during most quarters suggesting a
potential disconnect between the strategy and observed financial performance.
Second, growth in profits closely tracked industry averages suggesting the
possibility that the strategy was not differentiating Store24 as strongly as
intended. In FY 2000, based on negative customer

feedback, Store24 concluded that the differentiation strategy had failed and
refocused its strategy on traditional service quality activities.6 See Figure 5
for a timeline of events related to Store24’s strategy change. Based only on
trends in the balanced scorecard metrics, it was difficult for management to
definitively disentangle problems with strategy formulation from those with
strategy implementation. That is, it wasn’t easy to pinpoint why the strategy
failed.

III.

Empirical Tests and Results

In this section, we test the explicit and implicit hypotheses underlying Store24’s
balanced scorecard to examine whether, when, and how its multiple performance
measures captured information about problems with the differentiation strategy.
Our sample consists of financial, non-financial and

customer performance measures for 65 stores during fiscal years 1998 and 1999
(i.e., during implementation of the differentiation strategy). To obtain scores on
store-level differentiation, we

disaggregate the walk-through audit scores into their constituent components. We

have data for storelevel implementation of the differentiation strategy for the
fourth quarter of FY 1998 and the second and third quarters of FY 1999.7 We
supplement Store24’s balanced scorecard data with information on store competition
and demographics gathered during the same time period. Empirical Variables
Financial Performance

6 7

Store24 received negative feedback from in-store comment cards, telephone surveys
and focus groups. Unfortunately, after abandoning the differentiation strategy,
Store24 did not maintain consistent historical data on the performance measures
related to this strategy. We were only able to obtain data on these performance
measures for these quarters.

19
To improve its financial performance, Store24 can: i) increase customers; ii)
increase spending per customer; or iii) increase the efficiency and effectiveness
of store personnel (decrease costs). The measure of controllable contribution
(Profit) from the financial perspective of Store24’s scorecard summarizes these
categories at the store level; it is defined as revenues (Sales) from general
merchandise, lottery tickets, money orders, and phone cards less cost-of-goods
sold, utilities expense, and labor expense. This measure reflects the financial
components that Store24 believes store-level management can influence and is the
primary measure used by management to evaluate overall store financial
performance. We measure Profit as annual operating profit during FY 1999. This is
the period we are able to match with available strategy input measures, strategy
outcome measures, and measures of employee capabilities. FY 1999 is the second
year of Store24's differentiation strategy, allowing enough time for any start-up
problems in implementation to be worked out. In all analyses, we scale Profit by
square feet of store selling space.8 Non-financial Performance Measures Measure of
Strategy Inputs. We disaggregate stores’ total operational audit scores into
scores that reflect the store’s compliance with operating standards (strategy
input measures) for the differentiation strategy.9 Input_Diff reflects a store’s
percentage score on operating standards related to the

differentiation strategy; it reflects how well each store executed this strategy.
We use the strategy input measure taken at the beginning of FY 1999 in all our
empirical analyses (Input_Diff). 10 Measure of Basic Operational Compliance.
During the walk-through audit, Store24 management also measures basic service
quality items such as in-store image, fast service, and in-stock position.
Input_Basic is the average percentage score on operating standards related to
basic service quality taken over the same period as our measure of strategy-
specific inputs.

We find similar results for all of our subsequent analyses when store-level EBITDA
is used rather than controllable contribution. 9 Due to extra credit points for
strong implementation of Differentiation, a store’s score on Input_Diff can reach
135%. Employees were compensated based on a separate measure of this strategy
normalized by total available points. Thus, they were induced to invest in this
implementation. 10 Mystery shop scores are positively and significantly correlated
with walk-through audit scores and cannot be disaggregated. Adding mystery shop
scores to the analyses does not change the results.

20
Measure of Strategy-Specific Customer Outcomes. A third-party research firm
conducted in-store customer interviews at a subset of stores throughout FY 1999.11
Customers rated the attributes they “liked most about this particular Store24,”
including whether Store24 was “entertaining,” “a fun place to shop,” and
“unexpected.” We collect the metrics specific to the differentiation strategy;
these metrics comprise a reliable set as evidenced by a Cronbach’s coefficient
alpha of 0.9596. Each attribute is measured as the proportion of surveyed
customers who stated that they liked this characteristic about a particular store;
Outcome_Diff is the average of these measures. Outcome_Diff reflects whether

customers observe and value the new strategy; it represents a strategy-specific

customer outcome measure resulting from implementation of the differentiation
strategy (strategy input measures). Employee Capabilities. We take the measures of
manager and crew skills (MgrSkill, CrewSkill) from the learning & growth
perspective of the scorecard as our primary measures of the firm’s employee
capabilities. The inclusion of tenure metrics in Store24’s scorecard reflects the
idea that experience may capture dimensions of employee capabilities that are not
directly measured in the skill ratings. For this reason, we also include Store24’s
measures of manager and crew tenure (MgrTenure, CrewTenure) in our analyses. In
all subsequent empirical tests, we use the skill and tenure metrics taken in the
beginning of FY 1999.12 Were Store24’s senior management simply to infer skill
ratings from actual store performance, a store’s manager and crew skills ratings
would reflect store performance rather than exogenous skill levels. As shown in
Table 2, neither manager nor crew skills exhibit significant univariate
correlations with Profit. Thus, on average, senior executives do not provide
higher skill ratings to employees in better performing stores. Data on individual
employee skill ratings for a sample of 20 stores reveals variation in skill
ratings across individual employees within a particular store, reflecting senior
management’s desire to identify individual skills rather than infer skill-level
from store performance.

11 12

Data was collected for approximately 15-20 stores per quarter. Our results are
invariant to the use of average skills and tenure throughout FY 1999 rather than
taking the skill metrics at the beginning of FY 1999.

21
Control variables. Store24 collects demographic information for the half-mile
radius around each store. Many of these demographics relate to population and foot
traffic in the trading area of a given store and are highly correlated. Because
many of these variables are correlated we use factor analysis to identify the
underlying constructs and find one population factor with an eigenvalue greater
than one. Population represents daily activity around the store location. It
comprises primarily the student population (pre-high school, high school, and
college), pedestrian count rating, and population density. Income is an estimate
of the median level of annual disposable income available to a family for grocery
and convenience store purchases in the surrounding area which Store24 obtains from
a third-party research firm. Because we expect high income and/or large population
areas to offer more sales potential, these variables should relate positively to
financial performance. Finally, having more competing stores in the area is
expected to be associated with lower financial performance. To control for this
effect, we include Competition which reflects the number of competing stores
within a half-mile radius of each store. We also control for unobservable location
characteristics by including rent per square foot (Rent). Store24 pays a premium
to rent facilities in locations with, for example, high visibility. Cross-
sectional differences in Rent should capture store location differences which we
do not directly control for in our analyses. Finally, we include a measure of
store size (SQFT), measured as square feet of retail selling space, and a variable
that indicates whether a store is open 24 hours per day (24Hours). Methodology We
test the baseline hypothesis in Store24’s balanced scorecard, H1, by estimating
the following equation:
PROFITi t = α 0 + α1 Input _ Diffi + α 2 MgrSkilli + α 3CrewSkilli +α 4 MgrTenurei
+ α 5 CrewTenurei +α 6 Input _ Basici + α 7 Competitioni + α 8 Populationi + α 9
Incomei + α10 24 Hoursi +α11 SquareFeeti +α12 Renti +ε i

(1)

Where PROFITi denotes controllable contribution for store i during FY 1999. We

estimate this equation using OLS on a cross-sectional sample of 65 stores. To
reduce collinearity due to the inclusion of the interaction terms and to maintain
interpretability of the coefficients, we mean center the interaction variables
prior to estimation (Aiken and West 1991).

22
If the strategy-input measure leads to improved financial performance, we expect
α1 to be positive and significant. Finding no (a negative) relationship implies
that improved strategy implementation is not (negatively) associated with improved
performance, signaling potential problems with strategy formulation, strategy
implementation, or strategy fit. Consistent with the explicit and implicit
hypotheses underlying Store24’s strategy map and balanced scorecard, we test for
problems in strategy implementation (H2), strategy formulation (H3), and strategy
fit (H4, H5, and H6) by using OLS to estimate the following equations.
PROFITi = γ 0 + γ 1Outcome _ Diffi + γ 2 Outcome _ Diffi × MgrSkilli + γ 3Outcome
_ Diffi × CrewSkilli + γ 4Outcome _ Diff i × MgrTenurei + γ 5Outcome _ Diff i ×
CrewTenurei + γ 6 MgrSkilli + γ 7 CrewSkilli + γ 8 MgrTenurei + γ 9CrewTenurei + γ
10 Input _ Basici + γ 11Competitioni + γ 12 Populationi + γ 13 Incomei + γ 14 24
Hoursi + γ 15 SquareFeeti + γ 16 Renti + ηi
O u tco m e _ D iff
t i

(2)

= β 0 + β 1 In p u t _ D iff i + β 2 In p u t _ D iff i × M g rS k ill i + β 3 In

p u t _ D iff i × C re w S k ill i + β 4 In p u t _ D iff i × M g rT en u re i + β
5 In p u t _ D iff i × C re w T en u re i

+ β 6 M g rS kill i + β 7 C rew S k ill i + β 8 M g rT e n u re i + β 9 C rew T e

n u re i + ε i
In p u t _ D iff i = α
0

(3 )

+ α 1 M g r S k illi + α 2 C r e w S k illi + α 3 M g r T e n u r e i + α 4 C r e
w T e n u r e i + μ i

(4 )

Equation (2) is analogous to equation (1) where the outcome measure replaces
Store24’s internal strategy input measure13. Equation (3) tests the relationship
between the outcome measure and Store24’s input measure.14 A positive correlation,
β1, indicates relatively good implementation of the differentiation strategy
because the outcome measure correlates with the input metrics. β1>0, γ 1 ≤0 would
provide

evidence in favor of H2 and against H3 implying a good implementation of a bad

strategy. Conformance to operating standards (strategy inputs) leads to the
desired strategy-specific customer outcome (customers view stores as
“entertaining"), but the strategy-specific customer outcome does not translate
into improved store financial performance. β1≤0, γ 1 >0 would provide evidence
against H2 and in favor
In untabulated tests, we estimate equation 2 separately for stores where
Outcome_Diff was measured during the first 6-months and second 6-months of FY 1999
respectively. In these tests, for stores measured in the first (second) 6-months,
we measure manager and crew skills as the average of skills as measured during the
end of the fourth quarter of FY 1998 (second quarter of FY 1999) and the second
quarter of FY 1999 (fourth quarter of FY 1999). The results from estimation of
equation 2 on each of these sub-samples are substantively similar to those
reported in Table 5 on the full sample of stores. These results mitigate the
potential that the findings in our paper are due to any mismatch in performance
measurement periods within Store24. 14 Note that we do not include demographic and
other store location characteristics as controls in Equation 3. There is no a
priori reason to believe that strategy-specific outcomes should be driven by these
factors. However, we have estimated Equation 3 using the same controls as in
Equations 1 and 2 and results are substantively similar.
13

23
of H3; it is consistent with bad implementation of a good strategy. Strategy-
specific customer outcomes (more entertaining stores) are associated with higher
financial performance; however the strategy input measures do not lead to higher
levels of strategy-specific customer outcomes. To test the implicit assumptions
underlying management’s beliefs about the complementary impact of Store24’s
employee capabilities on the relationships between input, outcome and financial
performance measures, we rely on the interaction terms between the strategy-
specific measures and the measures of employee capabilities (e.g. skill and
tenure). Significant coefficients on these variables indicate that the level of
employee capabilities impacts the relationships among input measures, outcome
measures and financial performance (H5 and H6). Finally, we use equation (4) to
investigate the final explicit hypothesis in Store24’s balanced scorecard (H4) by
examining the relationship between performance on the strategy input metric
(Input_Diff) and the level of employee capabilities (MgrSkill, CrewSkill,
MgrTenure, CrewTenure). We include MgrSkill, CrewSkill MgrTenure, and CrewTenure
in equations (2) and (3) to account for any main effects of employee capabilities
on store financial performance.15 Although scaling by store size (Square Feet)
alleviates concerns with heteroskedasticity, we calculate p-values based on both
OLS standard errors and Mackinnon and White’s (1985) heteroskedasticity consistent
“HC3” standard errors with no substantive differences in results.16 RESULTS
Descriptive Statistics Table 1 provides descriptive statistics and Table 2
presents the correlation matrix for the sample of 65 stores. Note that the stores
exhibit wide cross-sectional variability in both Store24’s input measure
(Input_Diff) and outcome measure (Outcome_Diff). Outcome_Diff is negatively
related to Profit.
15

The univariate correlations suggest that

Additionally, the outcome measure is significantly

Managers with high skills may, for example, more effectively manage labor and
inventory costs which would have a direct effect on store-level financial
performance. 16 White’s test for heteroskedasticity is not as reliable in small
samples (Mackinnon and White 1985, Long and Ervin 2000). Long and Ervin (1997)
suggest using the HC3 estimator for standard errors when heteroskedasticity is
suspected. Although we have no a priori reason to suspect heteroskedasticity, we
check p-values based on HC3 estimators for robustness (untabulated).

24
positively related to Store24’s input measure (Input_Diff). Together, this
provides preliminary evidence that the differentiation strategy was well
implemented, as Store24’s view of good implementation corresponds to the customer
outcome, but possibly poorly formulated due to the negative relation of the
customer outcome with financial performance. Since stores vary on other factors
that might affect financial performance (e.g., location and skills) we refrain
from making conclusions based on these univariate tests. Competition, Population,
Income, Sqft and Rent all exhibit significant correlations with Profit. Thus,
these seem to be powerful controls for unobserved location characteristics that
might affect store performance. Tests of H1 (Linking Internal Processes to
Financial Performance) Table 3 reports the results of estimating the relationship
between Profit and Store24’s assessment of stores’ internal conformance with
strategic operating standards. On average, the strategy input metric, Input_Diff,
is not associated with Profit. This suggests that store-level effort to implement
the new strategy was not translating into store-level profits. Manager skills and
tenure significantly and positively relate to profit as does population in the
surrounding area; competition is negatively related to profit. Compliance with
basic operating standards (Input_Basic) is positively and significantly related to
profit with each 1% increase in this measure corresponding with a $2.16 increase
in annual profit per square foot all else equal. These results highlight that the
hypothesized link in the scorecard between internal implementation of the action
plans related to the new strategy and financial performance does not exist (H1).
However, it is unclear whether the strategy was poorly formulated or poorly
implemented. Tests of H2 and H3 (Distinguishing between Problems of Formulation
vs. Implementation) Table 4 contains results from estimation of equation (3). On
average, Store24’s input metric (Input_Diff) positively relates to the outcome
measure (p<0.10). This result provides support for H2 and suggests that the
strategy of creating entertaining stores was well implemented. The operating
standards management selected for the differentiation strategy relate to
customers’ views of stores being innovative and entertaining (strategy-specific
customer outcome). Estimation of equation (2) yields Panel A of

25
Table 5. On average, the outcome measure of the differentiation strategy
implementation is negatively related to Profit (p<0.10). Overall, these results
tend to support H2 and negate H3. Specifically, the relationship between the way
Store24 operationalized the differentiation strategy (input measure) and the way
in which the strategy was viewed by customers (outcome measure) is correlated; the
implementation seems to be good. However, the outcome measures of strategy
implementation and financial performance are negatively correlated. Thus, the
evidence from the multivariate analysis provides only partial validation of the
assumed links underlying Store24’s scorecard and points to the potential validity
of some of the concerns expressed by top management about the objective of the
strategy. In particular, the results imply that although the strategy was well
implemented, the strategy formulation may have been flawed. Tests of H5 and H6
(Identifying Strategic Fit) H5 and H6 focus on Store24’s employee capabilities and
whether the fit between these capabilities and the strategy influences its
performance. In particular, employee skill levels might impact the relationship
between the input and outcome measures and/or the relationships between the
outcome measure and financial performance. Table 4, presents the results for tests
of the skill and tenure interaction (H5). The results imply that the relationship
between input and outcome measures is not contingent on the store-level
capabilities of Store24. Neither the interaction of the input measure and manager
skills (tenure) nor the interaction of the input measure with crew skills (tenure)
is significant at conventional levels. Panel A of Table 5 presents tests of H6.17
On average, crew skills appear to moderate the relationship between the outcome
measure and financial performance. The interaction between crew skills and
Outcome_Diff positively relates to Profit (p<0.001).18 Although Store24 predicted
a positive relationship between the differentiation strategy and Profit, the
results imply that the benefits derived from the strategy depend on the level of
crew skills in a store.
In untabulated results we interact skill metrics with Input_Basic. The interaction
is not significant, and the results are substantively unchanged. 18 We also
investigate, in untabulated results, whether the interaction of Input_Basic and
skills and tenure as well as the interaction of Input_Diff with store location
variables relates to performance. All results hold and the additional interactions
are not significant.
17

26
We further examine the interaction between crew skills and the outcome measure
using post-hoc probing as suggested by Aiken and West (1991).19 Panel B of Table 5
illustrates the estimated relationship between the outcome measure (Outcome_Diff )
and financial performance (Profit), conditional on high (1 point above the mean
rating), mean, and low (1 point below the mean rating) crew skills, respectively.
We compute the standard errors for each estimated relationship in Panel A of Table
5 conditional on the level of crew skills and adjust t-statistics accordingly
prior to inference. The outcome measure negatively impacts Profit in stores with
low and average skills. However, these negative impacts seem to be mitigated in
stores with high crew skills where there is a positive relationship between the
outcome measure and Profit. Overall, the results suggest problems with the fit of
the differentiation strategy with Store24’s employee capabilities. Crew skills
determine the magnitude of the relationship between strategy outcomes and
financial performance, but the relationship is only greater than zero for high
levels of crew skills.20 Results of H4 (Linking the Learning & Growth and Internal
Process Perspectives) The results of tests of the drivers of the input metrics are
presented in Table 6. On average, crew skills and tenure are not significantly
related to strategy execution at the store-level; manager skills, but not manager
tenure, are positively and significantly related to store-level strategy execution
(Input_Diff) (p<0.10). Therefore, we find only mixed support for validation of the
assumed link in Store24’s

scorecard between employee capabilities and strategy execution (H4). Contrary to

the explicit hypotheses underlying the scorecard, but consistent with the implicit
hypotheses of the senior management team, the information in the scorecard reveals
that the primary role of employee capabilities is not necessarily in ensuring
store-level execution of the strategy (e.g. H4), but rather in ensuring that even
if executed well at the local level, the differentiation strategy ultimately
translated into the desired financial outcomes.

Because we mean-center all variables prior to interaction, it is difficult to

interpret the economic significance of the results directly from the tables. 20
The training costs associated with skill improvements should be considered before
the long-run viability of differentiation is evaluated. Because data constraints
preclude this analysis, this study indicates only the benefits of the strategy
gross of training costs.

19

27
Did the Balanced Scorecard Contain Timely Information about Problems with the
Strategy? Collectively, the results reported in Tables 2-6 provide evidence that
this firm’s balanced scorecard contained relevant information for detecting
strategic problems and for distinguishing among implicit alternative hypotheses
(relative to those explicitly articulated in the scorecard) related to strategy
formulation, implementation, and fit problems. However, they do not provide
evidence on whether formal analysis of the data generated by Store24's balanced
scorecard provides incremental learning relative to the firm's quarterly strategic
review process which did not rely on such analysis. To shed light on this issue,
we estimate a version of equation (1) using quarterly, rather than annual, Profit
measured during the first quarter of FY 1999 – almost a full year prior to the
decision to abandon the strategy. Strategy-specific customer outcomes were not
available at the store level at this point, so we focus on the direct link between
the internal process and financial perspectives of Store24’s scorecard. Although,
it would have been possible for Store24 to perform a similar test even sooner,
this is the earliest quarter that we are able to line up Store24’s strategy-input,
employee capability, and financial performance metrics. We supplement the
specification in equation (1) with interaction terms between the strategy-input
and employee capability metrics.21 As shown in Table 7, we find results that are
consistent with those noted earlier. On average,

the input metric, Input_Diff, is not associated with Profit. However, the
relationship between Profit and Input_Diff is increasing in the level of crew
skills. Panel B of Table 7 shows that the relationship between Profit and
Input_Diff is significant and positive in stores with high crew skills and
significant and negative in those with low crew skills. Despite the use of earlier
quarterly data, these results are largely consistent with those reported in Tables
3-5 offering evidence that formal analysis of the data generated by Store24's
balanced scorecard provides timely information about strategic problems relative
to the firm's quarterly strategy review process.

When we run the same analysis, but use the annual rather than quarterly data on
Profit, we obtain qualitatively similar results. That is, if we repeat the
analysis reported in Table 3 with the inclusion of interaction terms between
Input_Diff and measures of employee capabilities, we find no relationship between
Profit and Input_Diff on average and a negative (positive) relationship in stores
with low (high) crew skills.

21

28
What did Store24 Learn about Performance Drivers in the Balanced Scorecard?
Executives at Store24 eventually refocused the company’s strategy on traditional
convenience store operating activities related to speed, efficiency, and service
after abandoning the differentiation strategy. While our analysis provides
evidence that the company’s balanced scorecard contained useful and timely
information for detecting problems in its strategy, the results also suggest that
Store24 executives eventually learned about problems with the strategy despite a
lack of reliance on such formal analysis. In particular, the results in Tables 3
and 7 show that store level execution of the operating standards related to the
differentiation strategy (Input_Diff) are not related to financial performance
while store level execution of basic operating standards (Input_Basic) are
strongly and positively related to financial performance. Thus, the operating
standards which Store24 executives eventually abandoned were not, while those they
retained were, drivers of financial performance. Similarly and perhaps
unsurprisingly given the earlier results in the paper, in untabulated analyses we
find that once the differentiation strategy was abandoned, the updated internal
process metric capturing overall store compliance with operating standards (e.g.
the walk-through audit) becomes a stronger predictor of financial performance.
Furthermore, this increase in predictive ability is

concentrated primarily in stores with low crew skills where the differentiation
strategy was least effective. Overall, these results provide evidence that Store24
executives learned about the underlying drivers of store performance despite a
lack of reliance on formal statistical analysis of the assumed relationships
underlying their scorecard. However, the earlier results in the paper show that
such analysis could have yielded more timely information as well as more detail on
why the strategy was not working as planned.

IV.

Discussion and Conclusion

Our research investigates whether, when, and how information about problems with a
firm’s strategy was captured in the multiple performance measures of its balanced
scorecard. We analyze balanced scorecard data from a convenience store chain,
Store24, during the implementation of an innovative, but ultimately unsuccessful
strategy. Our results demonstrate that formal statistical tests of

29
the hypotheses underlying the firm's balanced scorecard and strategy map reveal
problems with the strategy on a timelier basis relative to the quarterly review
process that eventually led management to question and abandon the strategy. Our
analysis also demonstrates that this firm's balanced scorecard contained useful
and timely information for distinguishing between several alternatives to the
hypotheses underlying the firm’s scorecard and strategy map that are consistent
with concerns expressed by the top management team early in the development and
implementation of the strategy. These results provide some of the first field-
based evidence on the potential for a firm's balanced scorecard to provide useful
information for detecting problems in its strategy. En route, we document that the
extent to which non-financial performance measures predict future financial
performance depends on characteristics of the underlying strategy captured by
those measures. Little, or no, relationship between firm-specific non-financial
metrics and accounting returns may be informative about (1) the firm’s strategy
formulation, (2) its strategy implementation, or (3) the strategy’s fit with
internal capabilities. We provide some of the first field-based empirical evidence
on the potential for the multiple measures in a balanced scorecard to distinguish
between these three alternatives. Companies develop assumptions about the links in
“business-model” based measurement systems like the balanced scorecard based on ex
ante expectations (Ittner and Larcker 1998). Our findings indicate that non-
financial and financial measures and the hypothesized links between them can be
used more extensively for continuous hypothesis testing ex post. Building on prior
research illustrating the use of balanced scorecards data to communicate strategy
(Selto and Malina 2001; Banker et. al. 2004), we use Store24’s balanced scorecard
data to study how the system can be used to test strategy performance. Our
findings suggest that ongoing tests of these relationships are important to ensure
that hypothesized links are valid. Such investigation can potentially reveal
specific aspects of a strategy’s merits as well as its shortcomings; it can help
distinguish between strategic problems related to formulation, implementation, or
fit of the strategy with the firm’s internal capabilities. If a company
consistently applies its scorecard

30
across multiple units, these tests can be performed at an early stage, prior to
collecting an extensive longitudinal sample. The results in this paper are subject
to the caveat that the field-based nature of our research limits the
generalizability of our findings. However, the unique nature of any firm’s
strategy dictates that the performance measures and links between these measures,
articulated in the firm’s business model, are likely to be firm-specific. Future
research should provide additional evidence from other settings of the extent to
which business model-based performance measurement systems, such as the balanced
scorecard, capture information useful for strategy testing and validation.
Throughout the paper, we rely on notions of strategy-formulation, strategy-
implementation, and fit in interpreting our results. We recognize that there is a
strong interrelationship between these

concepts. We define tests related to strategy-formulation as analyzing whether,

given the capabilities available to Store24, their choice of strategy was sound.
Similarly, our tests related to implementation refer to the efficacy of Store24's
unique internal processes in achieving its strategic objectives given its
available capabilities. Our point is not to belabor the distinction between
formulation, implementation, and fit, but rather to identify how multiple measures
in a balanced scorecard might systematically be used to test how well different
drivers of performance are working to achieve strategic objectives and superior
financial performance. References 1. Aiken, L.S. and S.G. West. 1991. Multiple
Regression: Testing and Interpreting Interactions. London: Sage Publications. 2.
Banker, R. D., G. Potter, and D. Srinivasan. 2001. An Empirical Investigation of
an Incentive Plan that Includes Nonfinancial Performance Measures. The Accounting
Review 75 (1). 3. Banker, R. D., H. Chang, and M. Pizzini. 2004. The Balanced
Scorecard: Judgemental Effects of Performance Measures Linked to Strategy. The
Accounting Review 79 (1). 4. Campbell, D. and D. Lane. 2006. "China Resources
Corporation (A): 6S Management."

Harvard Business School Case 107-013.

5. Campbell, D. 2008. Nonfinancial Performance Measures and Promotion-Based
Incentives. Journal of Accounting Research. Forthcoming. 6. Cstore News, 2000. 7.
Fitzsimmons, J.A. and M.J. Fitzsimmons. 2001. Service Management: Operations,
Strategy, and Information Technology. New York: McGraw-Hill. 8. Goold, M. and
Quinn, J. 1990. The Paradox of Strategic Controls. Strategic Management Journal,
11 (1), 43-57 9. Ittner, C.D. and D.F. Larcker. 1998a. Innovations in Performance
Measurement: Trends and Research Implications. Journal of Management Accounting
Research 6: 205-238.

31
10. Ittner, C.D. and D.F. Larcker. 1998b. Are non-financial measures leading
indicators of financial performance?: An analysis of customer satisfaction.
Journal of Accounting Research 36: 1-35. 11. Ittner, C.D. and D.F. Larcker. 2001.
Assessing Empirical Research in Managerial Accounting: A Value-Based Management
Perspective. Journal of Accounting and Economics 32: 349-410. 12. Ittner, C.,
Larcker, D., and Meyer, M. 2003. "Subjectivity and the Weighting of Performance
Measures: Evidence From a Balanced Scorecard." The Accounting Review. 78(3) : 725-
758 13. Ittner, C.D. and D.F. Larcker 2005. Moving from Strategic Measurement to
Strategic Data Analysis. Controlling Strategy: Management, Accounting, and
Performance Measurement. Edited by C. Chapman. Oxford University Press. 14.
Kaplan, R.S. and D.P. Norton. 1992. The Balanced Scorecard – Measures that drive
performance. Harvard Business Review 70 (1): 71-79. 15. ___. 1996 The Balanced
Scorecard: Translating Strategy into Action. Boston, MA: Harvard Business School
Press. 16. Kaplan, R. S. 1998 "Mobil USM&R (A): Linking the Balanced Scorecard."
Harvard Business School Case 197-025. 17. ___. 2000 The Strategy Focused
Organization. Boston, MA: Harvard Business School Press. 18. ___. 2004 Strategy
Maps: Converting Intangible Assets into Tangible Outcomes. Boston:

Harvard Business School Publishing 19. ___. 2006 Alignment: Using the Balanced
Scorecard to Create Corporate Synergies. Boston: Harvard Business School Press 20.
___. 2008 The Execution Premium: Linking Strategy to Operations for Competitive
Advantage. Harvard Business School Press
21. Nagar, V. and M. V. Rajan. 2001. The Revenue Implications of Financial and
Operational Measures of Product Quality. The Accounting Review 76 (4): 495-513.
22. Mackinnon, J.G. and White, H. 1985. Some Heteroskedasticity Consistent
Covariance Matrix Estimators with Improved Finite Sample Properties. Journal of
Econometrics. 29: 53-57. 23. Selto, F. and M. Malina 2001. Communicating Strategy:
An Empirical Study of the Effectiveness of the Balanced Scorecard. Journal of
Management Accounting Research. 13: 47-90

32
FIGURE 1 Store24 Balanced Scorecard
Financial Perspective Parameter Return on Capital Deployed G&A Overhead EBITDA
Controllable Contribution Gross Profit Growth Sales Growth Inventory Turnover
Customer Perspective Parameter Loyalty - Recommend Store24 Primary Convenience
Store Enjoyable Experience Measurement % would recommend Store24 and % will visit
Store24 soon based on telephone survey % stating Store24 as their primary
convenience store based on telephone survey % viewing Store24 as fun and/or
entertaining place to shop based on telephone survey Level Corporate Corporate
Corporate Frequency Quarterly Quarterly Quarterly Measurement EBITDA divided by
value of equipment and leaseholds Average G&A cost per store Controllable
contribution less rental or lease cost Gross profit less utilities and labor
expense Growth in gross profit from same quarter in prior year Growth in sales
from same quarter in prior year Days inventory for general merchandise and
cigarettes Level Corporate Corporate Store Store Store Store Store Frequency
Quarterly Quarterly Quarterly Quarterly Quarterly Quarterly Quarterly

Internal Perspective Parameter Concept Development Operational Excellence Ban

Boredom Measurement Net gross profit $ from new concepts Walk-through audit and
mystery shopper ratings of compliance with basic operating standards Walk-through
audit and mystery shopper ratings of compliance with Ban Boredom implementation
standards Level Corporate Store Store Frequency Quarterly Quarterly Quarterly

Learning and Growth Perspective Parameter Manager Skills Crew Skills Manager
Tenure Crew Tenure Employee Satisfaction Information System Use Measurement Skill
rating of store managers Average Skill rating of non-management store employees
Number of years manager has been with Store24 Averge number of years with Store24
for non-management store employees Gallup survey of employee satisfaction on 5-
point scale Regional manager evaluation of store utilization of front and back-
office technology Level Store Store Store Store Corporate Regional Frequency Every
6-months Every 6-months Quarterly Quarterly Every 6-months Every 6-months

33
FIGURE 2 Store24 Strategy Map
Financial Perspective
ROI Return on Capital Deployed*

EBITDA

EBITDA* Asset Utilization Inventory turns*

Contribution

Controllable Contribution*

Gross Profit Increase Sales

Gross Profit Growth*

Sales or Sales Growth* Basic Requirements

Customer Perspective

Differentiators Enjoyable Experience Interesting Promotions Telephone Surveys*

Quality, Value, Cleanliness, Selection

Friendly Interactions

Telephone Surveys*

Internal Perspective

Differentiate In-Store Experience Create fun, entertaining instore atmospheres Ban

Boredom Walk-Through Audits* Net Gross Profit from New Concepts*

Increase Customer Value Enhance the customer experience with flawless operations
Walk-Through Audits* Mystery shoppers*

Learning & Growth Perspective

Competencies Required competencies are built on capable employees

Technology Focus on technology is on information systems use

Climate for Action Ability to implement relies heavily on employee satisfaction

Gallup poll*

Skills evaluation* Employee Tenure*

Technology evaluation sheet*

* Measures

34
FIGURE 3 Summary of Hypotheses Underlying the Scorecard and Strategy Map
Financial Perspective Financial Performance H3 H1 Customer Perspective H6
Strategy-Specific Customer Outcomes

H2 Strategy Inputs H4 H5

Internal Process Perspective

Learning and growth Perspective

Employee Capabilities

FIGURE 4 Store24’s Scorecard Metrics during Differentiation Period

Strategy-Spcific Customer Outcome Measure

Strategy-Specific Input Measure

6.20 Enjoyable Experience Rating

124% Walk-Through Audit Score 124% 123% 123% 122% 122% 121% 121% 120% 120% Q1 FY99
Q2 FY99 Q3 FY99 Q4 FY99 Quarter

6.10 6.00 5.90 5.80 5.70 5.60 Q1 FY99 Q2 FY99 Quarter Q3 FY99 Q4 FY99

Average Operating Profit

$5,400 $5,200 Operating Profit $5,000 $4,800 $4,600 $4,400 $4,200 $4,000 Q1 FY98
Q2 FY98 Q3 FY98 Q4 FY98 Q1 FY99 Q2 FY99 Q3 FY99 Q4 FY99 Quarter

* **

Operating profit (e.g. controllable contribution) is scaled by the number of weeks

in each respective quarter. Note that operating profit in convenience store
retailing exhibits strong quarterly seasonality.

35
FIGURE 5 Timeline of Events Related to Store24’s Strategy Change
Q1 FY 1998 Q3 FY 1999 Q4 FY 1999 Q1 FY 2000

•Store24 implements
differentiation strategy.

•Customer •Monitors and enforces

store-level strategy implementation using walk-through audits

•Customer focus

•Translates strategy

to a set of operating standards and measures store-level implementation of these

standards using walk-through audits.

feedback surveys suggest differentiation strategy is not resonating with

customers.

groups confirm that differentiation strategy is not resonating with customers.

basic service operations.

•Store24

• Store 24 refocuses on

updates performance measures to only reflect basic service operations

•Monitors customer

feedback about strategy through in-store comment cards and telephone surveys.

TABLE 1 Descriptive Statistics for the Sample of 65 Stores Used in Empirical

Analyses
Variable Mean SD Min Median Max 133.93 54.88 51.88 121.63 349.49 Profit 108.16
22.39 46.43 117.85 135.71 Input_Diff 89.89 5.58 71.21 89.60 99.26 Input_Basic
27.98 9.88 2.56 26.85 51.87 Outcome_Diff 3.27 0.63 1.21 3.27 4.38 MgrSkill 3.35
0.43 2.75 3.24 4.51 CrewSkill 45.2 57.95 0.20 22.51 265.76 MgrTenure 13.4 16.38
2.27 8.48 89.66 CrewTenure 3.87 1.38 1.65 3.68 11.13 Competition -0.06 0.90 -1.27
-0.28 3.06 Population 2,588.35 532.20 1,700.00 2,499.00 4,230.00 Income 0.85 0.36
1.00 1.00 24hours 2,139.05 374.78 1,333.00 2,133.00 2,919.00 Sqft 23.73 15.90 4.76
19.02 85.71 Rent Profit = Revenue from general merchandise, lottery tickets, money
orders, and phone cards less expenses related to cost-of-goods sold, utilities,
and labor, scaled by square feet of the store; Input_Diff = measure of store-level
implementation of Differentiation strategy measured as percentage compliance with
operating standards related to the differentiation strategy; Outcome_Diff =
customer (Outcome) measure of Differentiation strategy; MgrSkill and CrewSkill
=Average of bi-annual measures of the manager and crew skills in basic store
operations, rated on a five-point scale; MgrTenure and CrewTenure = Months of
tenure with Store24 for managers and average months of tenure with the company for
non-management employees (crew) respectively; Input_Basic = measure of percentage
compliance with operating standards related to standard convenience store
operations; Competition = number of competitors within the trading area of a
store; Population = store location factor score capturing items related to
population density and foot traffic around the stores’ trading area; Income =
Measure of median annual disposable income available for grocery and convenience
store purchases in the stores trading area 24hours = 1 if store is open 24 hours
per day, 0 otherwise; Sqft = square footage of the store; and Rent = monthly rent
per square foot for store.

36
TABLE 2 Correlation Matrix for 65 Stores during FY 1999
Profit Outcome_Diff Input_Diff MgrSkill CrewSkill MgrTenure CrewTenure Input_Basic
Competition Population Income 24hours Sqft Rent Profit 1 -0.4125* -0.1293 0.0911
0.0949 0.3460* 0.0815 0.1022 -0.3920* 0.4452* 0.2223* -0.096 -0.5790* 0.6526*
Outcome_Diff 1 0.2433* 0.0682 -0.0401 -0.1274 -0.0133 0.1007 0.2477* -0.0848
-0.4648* 0.0209 0.2280* -0.4003* Input_Diff MgrSkill CrewSkill MgrTenure
CrewTenure Input_Basic Competition Population Income 24hours Sqft

1 0.183 0.0031 0.0575 -0.0046 0.2612* 0.0419 -0.2750* -0.1132 0.2731* 0.1885
-0.3130*

1 0.2551* 0.2162* 0.1847 0.3514* 0.2896* -0.1605 -0.2698* 0.2067* 0.1402 -0.1444

1 -0.1059 0.12 0.1446 -0.1647 0.0515 0.0142 0.0612 -0.0159 0.0329

1 0.3266* 0.0483 0.1677 -0.0363 0.0323 -0.1923 0.0266 0.1956

1 0.1386 -0.0427 -0.0089 0.0118 -0.0949 0.063 0.172

1 0.0138 -0.0592 0.0434 0.2005 0.1552 -0.0654

1 -0.1303 -0.3821* 0.0998 0.3145* -0.4161*

1 0.0281 -0.1776 -0.181 0.3904*

1 -0.0095 -0.0661 0.4206*

1 0.035 -0.1363

1 -0.5487*

37
TABLE 3 Linking the Internal Process and Financial Perspectives (Dependent
Variable = Profit; Adjusted R2 = 0.72)
Intercept Input_Diff MgrSkill CrewSkill MgrTenure CrewTenure Input_Basic
Competition Population Income 24hours Square Feet (00's) Rent per Square Foot
Coefficient 22.49 0.02 16.87 2.02 0.34 -0.41 2.16 -7.91 22.63 0.006 -0.92 -0.06
0.32 Standard Error 89.05 0.19 8.93 7.36 0.08 0.25 1.05 3.56 4.68 0.009 11.42 0.01
0.38 Two-Sided p-Value 0.80 0.92 0.06 0.79 0.00 0.11 0.05 0.03 0.00 0.48 0.94 0.00
0.40

All bolded coefficients are significant at least at the 10% level using a two-
tailed test.

TABLE 4

Linking Internal Processes to Customer Outcomes

(Dependent Variable = Outcome_Diff; Adjusted R2 = 0.13)
Intercept Input_Diff Input_Diff x MgrSkill Input_Diff x CrewSkill Input_Diff x
MgrTenure Input_Diff x CrewTenure MgrSkill CrewSkill MgrTenure CrewTenure
Coefficient 18.10 0.11 -0.11 -0.02 0.002 -0.001 1.99 -2.03 -0.04 0.05 Standard
Error 10.50 0.06 0.13 0.13 0.001 0.004 2.93 2.38 0.03 0.06 Two-Sided p-Value 0.09
0.07 0.39 0.86 0.184 0.834 0.50 0.40 0.11 0.42

All bolded coefficients are significant at least at the 10% level using a two-
tailed test.

38
TABLE 5

Panel A: Linking Customer Outcomes to Financial Performance

(Dependent Variable = Profit; Adjusted R2 = 0.76)
Intercept Outcome_Diff Outcome_Diff x MgrSkill Outcome_Diff x CrewSkill
Outcome_Diff x MgrTenure Outcome_Diff x CrewTenure MgrSkill CrewSkill MgrTenure
CrewTenure Input_Basic Competition Population Income 24hours Square Feet (00's)
Rent per Square Foot Coefficient 37.12 -0.92 0.64 2.00 0.00 -0.02 21.00 3.78 0.28
-0.37 2.47 -7.51 22.50 -0.01 -5.28 -0.07 0.06 Standard Error 88.53 0.46 0.79 0.84
0.01 0.04 8.55 6.99 0.07 0.26 1.00 3.35 4.38 0.009 11.58 0.01 0.38 Two-Sided p-
Value 0.68 0.05 0.42 0.02 0.64 0.65 0.02 0.59 0.00 0.17 0.02 0.03 0.00 0.93 0.65
0.00 0.88

All bolded coefficients are significant at least at the 10% level using a two-
tailed test.

Panel B: Summary of Moderating Effect of Crew Skills Two-sided p-value for test of
γ 1 + γ 3 = 0
Low Crew Skills Mean Crew Skills High Crew Skills Coefficient -2.92 -0.92 1.08
Two-Sided p-Value 0.000 0.098 0.020

TABLE 6

Linking the Learning & Growth and Internal Process Perspectives (Dependent
Variable = Input_Diff; Adjusted R2 = 0.09)
Intercept MgrSkill CrewSkill MgrTenure CrewTenure Coefficient 55.90 16.41 0.14
-0.03 -0.12 Standard Error 34.13 5.14 9.12 0.05 0.18 Two-Sided p-Value 0.11 0.00
0.99 0.53 0.49

All bolded coefficients are significant at least at the 10% level using a two-
tailed test.

39
TABLE 7

Panel A: Did the Balanced Scorecard Contain Timely Information about Problems with
the Strategy?
(Dependent Variable = FY 1998 fourth-quarter Profit; Adjusted R2 = 0.66)
Intercept Input_Diff Input_Diff x MgrSkill Input_Diff x CrewSkill Input_Diff x
MgrTenure Input_Diff x CrewTenure MgrSkill CrewSkill MgrTenure CrewTenure
Input_Basic Competition Population Income 24hours Square Feet (00's) Rent per
Square Foot Coefficient 14.76 0.002 0.09 0.17 0.00 0.00 5.57 0.11 0.09 -0.10 0.39
-2.12 4.27 2.78 -1.60 -0.02 0.10 Standard Error 21.39 0.05 0.07 0.09 0.00 0.00
2.54 2.18 0.02 0.06 0.21 0.60 1.28 1.21 4.32 0.00 0.12 Two-Sided p-Value 0.49 0.98
0.26 0.05 0.73 0.91 0.03 0.96 0.00 0.13 0.07 0.00 0.00 0.03 0.71 0.00 0.40

All bolded coefficients are significant at least at the 10% level using a two-
tailed test.

Panel B: Summary of Moderating Effect of Crew Skills

Low Crew Skills Mean Crew Skills High Crew Skills Coefficient -0.17 0.002 0.18
Two-Sided p-Value 0.098 0.976 0.084

40