Accepted Manuscript

Title: Using Balanced Scorecards for the Evaluation of

Software-as-a-service
Author: Sangjae Lee Sung Bum Park Gyoo Gun Lim
PII:
DOI:
Reference:

S0378-7206(13)00078-5
http://dx.doi.org/doi:10.1016/j.im.2013.07.006
INFMAN 2652

To appear in:

INFMAN

Received date:
Revised date:
Accepted date:

17-8-2012
24-1-2013
11-7-2013

Please cite this article as: S. Lee, S.B. Park, G.G. Lim, Using Balanced Scorecards
for the Evaluation of Software-as-a-service, Information & Management (2013),
http://dx.doi.org/10.1016/j.im.2013.07.006
This is a PDF file of an unedited manuscript that has been accepted for publication.
As a service to our customers we are providing this early version of the manuscript.
The manuscript will undergo copyediting, typesetting, and review of the resulting proof
before it is published in its final form. Please note that during the production process
errors may be discovered which could affect the content, and all legal disclaimers that
apply to the journal pertain.

Using Balanced Scorecards for the Evaluation of

Software-as-a-service

cr

Sung Bum Park,

National Information Society Agency
parksb@nia.or.kr

ip
t

Sangjae Lee
School of Business Administration, Sejong University
sangjae@sejong.ac.kr

an

us

Gyoo Gun Lim

School of Business, Hanyang University
gglim@hanyang.ac.kr

ABSTRACT

In order to overcome the problem of limited resources, increasing numbers of

ed

small and medium-sized companies (SMEs) are adopting Software-as-a-service

(Saas) as an efficient tool for IS implementation. The balanced scorecard (BSC) has
been adopted by SMEs to evaluate Saas, via four measures: learning and growth,

pt

internal business processes, customer performance, and financial performance. The

survey results for 101 Software-as-a-service adopters indicate that learning and

Ac
ce

growth, internal business processes, and customer performance are casually related
to financial performance. The results show that these four key elements for Saas
success are interrelated, supporting the core premise of the BSC.

Key Words: Software-as-a-service (Saas), balanced scorecard (BSC), causal

relationships

Page 1 of 32

*Manuscript

Using Balanced Scorecards for the Evaluation of

Software-as-a-service

ip
t

ABSTRACT
In order to overcome the problem of limited resources, increasing numbers of

cr

small and medium-sized companies (SMEs) are adopting Software-as-a-service

(Saas) as an efficient tool for IS implementation. The balanced scorecard (BSC) has

us

been adopted by SMEs to evaluate Saas, via four measures: learning and growth,
internal business processes, customer performance, and financial performance. The
survey results for 101 Software-as-a-service adopters indicate that learning and

an

growth, internal business processes, and customer performance are casually related
to financial performance. The results show that these four key elements for Saas

success are interrelated, supporting the core premise of the BSC.

Key Words: Software-as-a-service (Saas), balanced scorecard (BSC), causal

ce
pt

ed

relationships

1. Introduction

IS maintenance costs make up a major portion (e.g., 70%) of total IS implementation

Ac

cost. Companies choose to invest their resources and manpower in their core
capability in order to provide products or services. The demand for IT outsourcing
and the Software-as-a-service (Saas) model, which integrates network, hardware,
and software, is increasing as IT sophistication becomes greater (Kern et al., 2002).
Saas can be defined as applications and computer-based services delivered and
managed from a remote center to multiple customers via the Internet or a VPN,
sharing common themes with On-Demand Service (Jeong and Stylianou, 2010).
There is a growing use of other related and advanced platform services, such as
cloud computing, infrastructure-as-a-service (Iaas) and platform-as-a-service (Paas),
representing a large pool of usable resources such as hardware and software that

Page 2 of 32

are easily accessible via the Internet (Vaquero, 2009; Cusumano, 2010). It is
estimated that by 2013 the cloud market will have reached $8.1 billion (BBC, 2010).
Industry leaders predict that revenues from cloud computing enterprises will grow to
$160 billion; experts describe cloud computing as an emerging IT development,
deployment and delivery model, enabling real-time delivery of products, services and

ip
t

solutions over the Internet (Fowler and Worthen 2009).

These attempts at utility computing are taking off because sufficient bandwidth

cr

is now available for such services. Broadband communication has become cheap
and plentiful enough for utilities to deliver computing services with the speed and

us

reliability that businesses previously enjoyed from their local machines. Typical Iaas
offerings include Amazons Elastic Compute Cloud (EC2) and Simple Storage
Service (S3), Joyents Accelerator, and Rackspaces Mosso. The possible reasons

an

for adopting cloud computing include (1) avoiding capital expenditure in hardware,
software, and IT support, and (2) utilizing flexibility and scalability of IT resources.

The major issues of cloud computing include: integrity of services and/or data,
confidentiality of corporate data, and reliability perceptions, to name a few (Behrend
et al., 2011; Catteddu and Hogben, 2009).

ed

The Saas provider is like a mediator, mediating services between independent

software vendors (ISVs). Saas customers do not possess, manage, or maintain the

ce
pt

applications, but only use them as final products by accessing services with IT
support. While Saas is advantageous in that it reduces the repair cost of applicationbased construction and maintenance, the risk of data leakage becomes a major
disadvantage since an application server is constructed by outside companies.

Ac

Saas is one type of ASP (application service provider service) and, after first
being introduced between 1998 and 1999, the ASP service market increased rapidly
between 2000 to 2001 due to excessive expectations and ASP service provider
mergers and acquisitions (M&A). While the number of customers and the market
size are continuously growing, the growth rate is plateauing. As the ASP market
worsens, it is essential to improve ASP planning and management as it is now
harder for many ASP providers to survive (Currie, 2003). For ASP services to be
successful, system and service qualities need to be well prepared (Kim and Kim,
2008). In small and medium-sized companies (SMEs), both ASP service and
information quality are significant factors for enhancing user satisfaction, trust, and

Page 3 of 32

intention to use (Lee et al., 2007). ASPs are said to have achieved considerable
success with big businesses, but their success has been less notable with SME
markets (Altaf and Schuff, 2010). For example, lack of ASP customization options,
and concerns about financial stability, service reliability, and functional capability
flexibility are problems SMEs. For these reasons, SMEs are less likely to adopt ASP

ip
t

services than large companies are, making it is necessary to develop empirical

studies to examine how specialized measures may be applied to SMEs.

cr

Outcome assessment in the traditional field of business administration places

excessive emphasis on financial performance measures (Olve et al., 1999). Since it

us

is impossible to assess an organization in a competitive environment based merely

on financial performance, it is necessary to measure Saas performance by using the
balanced scorecard (BSC) approach, which balances leading and lagging indicators,

an

as well as by using financial and non-financial measures (Kaplan and Norton, 1996).
In order to provide a balanced approach to the measurement of organizational

performance, including sub-areas such as KM (knowledge management), business

processes, and financial performance, a BSC measures four categories: learning
and growth, internal business processes, customer performance, and financial

ed

performance. Small companies develop multiple scoreboards, each tailored to the

strategy and goals of a specific subunit. In order to survive in today's global and

ce
pt

volatile business environment, SMEs are utilizing newer management systems, like
BSCs, to clarify their vision and strategy, and to translate them into action (Kumru,
2012).

BSCs also include a financial perspective because such a perspective can easily

Ac

summarize previous financial activities and yield predictable economic outcomes.

The financial perspective indicates whether a strategic operation contributed to net
profit improvement. The customer category assesses the extent to which the target
market was captured. The internal business processes category focuses on core
processes aimed at customer satisfaction and at financial objective achievement.
The learning and growth category assesses the construction of necessary long-term
growth and improvement infrastructure (Kaplan and Norton, 2001).
BSCs have been applied to the performance measurement of a wide variety of
businesses. For instance, Yawson et al. (2006) demonstrated the application of
BSCs at an institutional level in a collaborative effort to develop a performance

Page 4 of 32

measurement framework for the Food Research Institute (FRI). Withanachchi et al.
(2007) applied BSCs to evaluate an organizational development program (TQM) that
was implemented at a tertiary-care public hospital. Moreo et al. (2009) suggested
that BSCs can be used by managers to quantify the environmental and financial
impact of a company and to help promote environmental quality to stakeholders,

ip
t

including hospitality owners and stockholders. Smandek et al. (2010) developed and
implemented a BSC system for IP management in order to optimize licensing income

cr

generation, cut costs, and keep inventors motivation high. Homburg et al. (2012)
tested and applied BSCs to marketing performance management to show the

us

comprehensive relationship between a marketing performance measurement system

and firm performance as conditional on marketing alignment and market-based
knowledge. Taylor and Baines (2012) applied BSCs to evaluate the performance

an

management of higher education in terms of the formation, monitoring, and

evaluation of strategy and policy, as well as issues of motivation.

While BSCs have been applied in various contexts, empirical studies on the
application of BSCs to specific IT services like Saas are lacking. Further, while SMEs
have difficulty rationalizing their operational practices and strategic processes, there

ed

is a dearth of comprehensive performance management system application by

SMEs (Garengo and Biazzo, 2012). Despite the wide scope of BSC application as a

ce
pt

decision-supporting instrument, cause-and-effect relationships and time-delayed

elements between measures are still elusive (Nielsen and Nielsen, 2012). While
causal relationships among the four BSC measurements are the core focus of BSC,
empirical studies, via the testing of causal relationships among categories, to

Ac

examine how well BSCs may be applied, are almost nonexistent. This study intends
to fill this gap. This study suggests some measures to evaluate Saas; this study also
tests these measures using data collected from companies that have adopted Saas.
The discussion and implications are included in the study.

2. Theoretical Background
2.1 ASP service evaluation models
The criteria considered for ASP service selection were largely determined to be:

Page 5 of 32

credibility, appropriateness, and efficiency, which further includes prior experience,

ASP service expectation, perceived provider performance, and expectationdisconfirmation.
Kern et al. (2002) suggested six propositions based on resource dependency
theory, resource-based theory, transaction cost theory, and agency theory. These

ip
t

propositions are as follows: (1) the use of an ASP service is a strategic decision to
supplement necessary parts of IS; (2) ASP customer service depends heavily on the

cr

ASP service; (3) ASP service generally has lower cost; (4) ASP service prices are
increasing over time; (5) SMEs are more interested in ASP service than are large

us

businesses; (6) the suitability of an ASP service is determined by the capability of the
customer and the variety of the ASP service. Kern et al. articulated specific
characteristics derived from the relationship between ASP service providers and their

an

customers.

This framework for measuring the SME benefits and risks of Internet based

applications is designed for companies that do not have any specific team or
methodology for measuring proposed IT outsourcing. Currie (2003) suggested five
categories for measuring the risk and benefit of applications provided by an ASP

ed

service, including: delivery & enablement, integration, management & operation,

business transformation, and customer / vendor relationship. This framework

ce
pt

provides a difference in viewpoint between ASPs and their customers regarding key
performance indicators (KPIs) of ASP services. Susarla et al. (2003) suggested that
perceived provider performance has a positive impact on user satisfaction with an
ASP; further expectations about ASP services have a significant influence on any

Ac

performance evaluation of ASPs.

Leam and Lee (2004) proposed items for auditing and verifying the reliability of
ASP service through a survey of 35 Korean companies. By using categories
including network, data center, application, and security and customer support, they
evaluated three ASP vendors in Korea; they derived the ASP life cycle and auditing
items based on the evaluation results. The items for auditing ASPs were classified by
function and performance into 12 items, and the relative importance of those 12
items was investigated. The items for auditing an ASP are focused on assessing
usefulness, extensibility, and application usability. Zviran et al. (2005) suggested that
perceived usefulness is one of the factors affecting user satisfaction with enterprise

Page 6 of 32

resource planning (ERP) systems, which are typical examples of ASP services.
Kim et al. (2006) identified three characteristics of ASPs that affect the
satisfaction of ASP customers: stability, IT infrastructure, and service flexibility. Kim
and Kim (2008) suggested five characteristics of ASPs that affect ASP customer
satisfaction: credibility, system currency, security, acceptability, and system support.

ip
t

Susarla et al. (2009) used prior literature in transaction cost economics (TCE) to
posit that the contract design for an ASP service should consider such factors as

cr

uncertainty in specifying service requirements, interdependence between the ASP

application and the IT systems in the client organization, and the need for specific
to

address

transaction

costs

that

result

due

to

contractual

us

investments

incompleteness and opportunism. Schwarz et al. (2009) suggested 10 attributes that

firms consider when deciding on outsourcing of applications and indicated that the

an

three most significant drivers of an IT application service choice were cost, risk, and
vendor capability. Yao et al. (2010) suggested that firms who specialize in providing

software applications to other firms over the Internet should be evaluated in terms of
measures such as the possibility of using an ASP service, software customization
requirements, financial stability, and exit strategies.

ed

But these measures are limited as they are based only on the BSC customers

measures.

ce
pt

perspective. Other BSC perspectives should also be added to Saas evaluation

2.2. The balanced scorecard

Ac

The need for comprehensive performance measurement, including knowledge

components, has motivated the development of the Balanced Scorecard (BSC). The
BSC attempts to integrate all interests of key stakeholders: shareholders, customers,
and employees, on a scoreboard (Kaplan & Norton, 2001).
Recently, studies have been conducted on BSC application in various fields
including organizational or IS performance evaluation. Lee et al. (2008) proposed an
approach based on the fuzzy analytic hierarchy process (FAHP) and balanced
scorecard (BSC) processes for evaluating an IT department in the Taiwanese
manufacturing industry. The results offered guidance for Taiwanese IT departments
in the manufacturing industry regarding strategies for improving department

Page 7 of 32

performance. Huang (2009) suggested the use of an analytic hierarchy process

(AHP) to prioritize the measures and strategies of a BSC framework; this was done
with the intention of developing an intelligent BSC planning and management
decision support system for strategic planning. Wu et al. (2009) suggested a Fuzzy
Multiple Criteria Decision Making (FMCDM) approach for banking performance

ip
t

evaluation based the four perspectives of BSCs. They summarized performance

indexes to fit the banks in order to construct a hierarchical framework of performance

cr

evaluation; Asosheh et al. (2010) adopted an integrated use of balanced scorecard

(BSC) and data envelopment analysis (DEA), and proposed a new approach for IT

us

project selection. This approach utilized BSCs as a comprehensive framework for

defining IT project evaluation criteria and adopted DEA as a nonparametric
technique for ranking IT projects. Velcu (2010) tested the interrelations between

an

strategic alignment, management of ERP implementations, process changes, and

business performance of companies implementing ERP, and their study was based

on a balanced scorecard approach to analyzing business performance. Kunz and

Schaaf (2011) suggested an expert system for health care management for the
formalization of BSC evaluation. They provided a general definition of an indicator

ed

system for each BSC category for clinical application.

BSCs will be able to represent and deliver strategies according to each business,

ce
pt

rather than offering the simple combination of financial and non-financial indicators
sorted by individual categories. The strategy of BSC organization is represented by
performance connected through causal relationships and ongoing performance
indicators. In general, final performance indicators, or lagging indicators, are

Ac

represented by improved financial performance, which is the ultimate goal of the

strategy. Meanwhile, ongoing performance indicators, or leading indicators, inform
all organization members of what will currently be performed in learning and
business processes (Kaplan and Norton, 2001). Since a strategy is the hypothetical
representation of cause and effect regarding the capability of an organization to
correspond to market changes in order to achieve a goal, such a strategic causal
relationship can certainly form a set of causes and effects. Thus, the result (lagging
indicator), focused on each business strategy, may be appropriately connected to
the performance cause (leading indicator), allowing all indicators in each BSC
category to be related to the financial goal.

Page 8 of 32

3. Research Model

ip
t

Insert Figure 1 here

Despite the wide scope of BSC application as a decision-supporting instrument,

empirical studies on the application of BSC to specific IT services like Saas are

cr

relatively absent. Further, it is necessary to support SMEs with specialized

us

performance measurement systems like BSCs. As a way to accomplish these

research objectives, this paper examines causal effects in each BSC category to
indicate how well BSCs can be applied to Saas. BSC proponents expect that

an

companies who continuously improve their capabilities (e.g. by implementing

advanced workplace practices, monitored via innovation and learning) should

achieve better performance for their customers. All such efforts should lead to
improved financial performance. Sim and Koh (2001) positively verified a causal
relationship between several individual performance indicators among the four BSC

ed

categories through relationship and regression analysis; they used data obtained
from 83 electronics companies in the U.S. BSC can be used as a tool to align

ce
pt

business processes with new strategies, moving away from cost reduction and
towards growth opportunities based on more customized, value-adding products and
services. Wu (2012) used a multiple criteria analysis tool, to determine causal
relationships between key performance indicators for each BSC perspective to

Ac

create a visualized strategy map with logical links to improve banking performance.
If cause-and-effect relationships are not adequately reflected on a balanced
scorecard, the scorecard will not translate to a companys vision and strategy. Sohn
et al. (2003) posited that a BSCs merit is that it seeks a balance between financial
and non-financial measures, categorized into financial, customer, and internal
business processes, and innovation and learning factors. Firms using IS must
determine which specific BSC measures to focus on and which to ignore. However,
relatively few studies have been carried out on the specific weights of each BSC
perspective. Northcott and Taulapapa (2012) indicate the need for improved
theorization on several issues that present particular challenges for BSC practice in

Page 9 of 32

the public sector: BSC measure modification, measure design that captures
important qualitative outcomes, "customer" suggestion and/or achievement of a
genuine multi-stakeholder approach, and mapping BSC causality relationships.
In an era of global economic competition, organizations should be creative in
learning. Organizational capability for learning and creating knowledge is the

ip
t

cornerstone for operating businesses. Many researchers in the field of knowledge

management (KM) have emphasized the role of learning in knowledge management

cr

systems (KMS) based on the interrelatedness of learning and knowledge.

Organizational knowledge is defined as the improvement of actions through

us

knowledge acquisition. Organizational learning occurs when individuals and subunits

acquire knowledge after understanding the possibility of organizational change.
Organizational learning improves the potential capability for effective actions of

an

organizations and individuals through improved business processes and customer

service.

In KM, organizational learning is established across entire organizational

hierarchies when the skills and knowledge acquired by individuals are internalized
into organizational entities. Organizations become efficient and skillful initiators of

ed

change by creating, acquiring, and transferring knowledge. Organizational

improvement is determined by learning processes that require the addition or change

ce
pt

of mental models. Organizations can create knowledge continuously from inquiry into
and understanding of new environments (DuToit, 2003).
Improvement in customer service depends on the effectiveness of situational
behaviors in their contingencies. Under rapid, innovative, and discontinuous change

Ac

of environments, the assumptions lying beneath work practices stored in a

knowledge repository should be continuously improved for better customer service.
Thus, organizations that do not engage in creative learning through KM increase
entropy, which ultimately leads to failure in customer service. Top managements
view of the business environment should be adjusted in a timely manner according
to changes in environments; successful customer service companies have better
and more rapid creative learning. Mason (2004) indicated that one crucial measure,
besides financial performance, is learning, which is necessary to assessing the
extent to which organizations acquire new knowledge in order to maintain
competitiveness. Pfeffer (2005) showed that organizational perception can determine

Page 10 of 32

organizational performance and that creative learning can result in innovation and
dramatic performance improvement.
This section examines the effects of learning and growth on customer service
improvement for Saas performance. The learning and growth of companies is a
fundamental force driving customer service performance and customer relationship

ip
t

management. For example, better staff skills will reduce the frequency of bugs in an
application. An application with fewer bugs will be more likely to meet end-user

cr

expectations. This, in turn, will enhance the support of customer service processes.
Nonfinancial indicators such as the customer satisfaction index and customer loyalty

us

are prerequisites and pre-conditional indicators for financial indicators (Behn and
Riley, 1999; Ittner and Larcker, 1998); customer performance has been proven to be
a decisive factor in financial performance, as learning and growth affect financial

an

performance through customer performance. For instance, using data from

multinational corporations, Gonzalez-Padron et al. (2010) used BSCs to assess how

subsequent financial performance.

organizational learning affects actions relating to global marketing strategy and

Thus, organizational change based on learning is a crucial factor for improved

ed

organizational process and customer service. Top managements ability to

understand and learn in uncertain and competitive environments is important for

ce
pt

business processes and customer service. Top management interprets information

on behalf of organizations, and the creative learning of top management likely affects
the creative learning of organizations. Thus, organizations should acquire knowledge
of business environments and perform learning processes faster than competitors in

Ac

order to maintain good business and customer relations. This increases the demand
for efficient processes based on KM tools and new infrastructure capability (Bergman
et al., 2004).

Hypothesis H1: Learning and growth positively affect internal business processes in
the clients organization after Saas is offered.
Hypothesis H2: Learning and growth positively affect the customers performance
after Saas is offered.

In addition, the effect of internal business processes on both customer and financial

Page 11 of 32

performance, and subsequently the customers performances effect on financial

performance can be investigated. The goals of internal business processes in the
BSC model are to innovate and improve the process of identifying and satisfying
customer demand, as well as to provide excellent customer management service
afterward. Customers can then recognize that the service company provides the best

ip
t

customer value and thus customer satisfaction increases. The customers

performance facilitates the market share and the customer profitability in the detailed

cr

market targeted, allowing the financial goals to be achieved.

Previous studies have suggested that customer and financial performance BSC

us

measures are causally interrelated. For instance, Ittner and Larcker (1998) studied
the relationship between customer satisfaction and financial performance by using
various data sorted by company, business, and customer. As a result, the measured

an

value of customer satisfaction turned out to have a significant impact on future

financial performance. Behn and Riley (1999) demonstrated, in their research, the

announcement that customer satisfaction had an effect on American airline

companies future financial performance, i.e. that non-financial indicators function as
leading indicators of quarterly financial performance.

ed

The ultimate aim of many balanced IS scorecards is to support IS management

such that the overall financial outcomes of the enterprise are improved (Martinsons

ce
pt

et al., 1999). This study suggests, based on this premise that balanced IS
scorecards should improve financial performance. In order to realize financial
performance, the customer performance should be improved. The determinants for
customer performance are, in turn, internal business processes and learning and

Ac

growth (Epstein and Manzoni, 1998). Gonzalez-Padron et al. (2010) confirmed that
customer performance affects financial performance. Learning and growth facilitate
internal business processes. This leads to the following hypotheses:

Hypothesis H3: Internal business processes positively affect customer performance

after Saas is offered.
Hypothesis H4: Customer performance positively affects financial performance after
Saas is offered.

Page 12 of 32

4. Research Method
4.1 Measurement of Variables
Most of the variables in the model are measured by items written in the form of a

ip
t

statement with which the respondent is to agree or disagree on a 7-point Likert-type

scale. The latent variables and measurement items used in this study are shown in
Table 1. The measurement items were measured by partially revising the leading

cr

and lagging indicators suggested in the BSC model. The research variables

us

consisted of four measures including learning and growth, internal business

processes, customer performance, and financial performance.

The measures are based on and adapted from previous studies on intellectual

an

capital or BSCs, such as Butler et al. (1997), Epstein and Manzoni (1998), Harvey
and Lusch (1999), Martinsons et al. (1999), Mayo (2000), Petty and Guthrie (2000),

Lipe and Salterio (2000), Mayo et al. (2000), Stewart (2001), and so on.
The measure of learning and growth represents the extent to which using Saas
continued employee learning and development as well as knowledge sharing in

ed

order to make efficient use of business resources. The utilization of Saas and
knowledge sharing can promote learning and development of an organization.

ce
pt

Internal business processes indicate the extent to which value is brought to the
customer via efficient use of business resources. Customer performance indicates
the extent to which the customer market and service are improved. Financial
performance indicates the return on investment and created shareholder value.

Ac

Learning and growth include the percentage of work processed through Saas,
and knowledge sharing within an organization and with other organizations (trading
partners) after Saas is offered. The category of internal business processes is
composed of customer order processing time reduction, observance of customer
request time limits, and the improvement of customer service after Saas is offered.
Customer performance consists of the annual average increase in the customer
market, cost reduction in ordering, and revenue increase per year after Saas is
offered.

Page 13 of 32

Insert Table 1 here

4.2 Data collection

ip
t

500 interviewees were selected randomly among SMEs registered by the Small &
Medium Business Administration in Korea. Data were collected using a structured
questionnaire, in Korean, through telephone and personal interviews with IT staff

cr

members who employed Saas. The sample was composed of companies that are

us

diverse in terms of business area, corporate size, etc. Among the 500 corporations,
101 corporations were included in the final sample after excluding missing values
and incomplete responses. Among the 101 corporations that answered, IT

an

corporations accounted for 51% (51 corporations) while non-IT corporations made
up 49% (49 corporations) of the sample. The descriptive statistics of the sample data

are shown in Table 2. Organizations within the sample had an average of 79

employees and had adopted Saas for an average of three years. The average
percentage of Saas utilization is 45.2%. The majority of applications using Saas are

ed

core businesses (sales, production, purchasing, logistics applications) (71%).

Ac

5. Results

ce
pt

Insert Table 2 here

5.1 Assessment of measurement model

In order to ensure convergent and discriminant validity, exploratory factor analysis
with Varimax rotation was performed to produce a cross-loading matrix. The matrix
shows a correlation between latent variables and measurement items, suggesting
that the measurement items load highly on their theoretically assigned latent
variables while not loading highly on other factors (Gefen et al., 2003). Table 3
shows the cross-loadings of individual items that are compared across all latent
variables. Each item is assigned highly to a respective latent variable, thus ensuring

Page 14 of 32

convergent and discriminant validity.

Insert Table 3 here

Data analysis was performed using the Partial Least Squares (PLS) method. The

ip
t

use of PLS has increased in IS fields (Compeau et al., 1995). PLS, a componentbased structural equation modeling technique, is similar to regression but

cr

simultaneously models paths among variables. A large sample size is required, and
standard distribution is assumed for SEM. In contrast, PLS, introduced by Wold et al.

us

(1985) focuses on maximizing the variance of dependent variables explained by an

independent one (Haenlein and Kaplan, 2004). The PLS approach places minimal

an

restrictions on sample size and residual distribution. In addition, a PLS algorithm

allows each indicator to vary in how much it contributes to the composite score of the
latent variable instead of assuming equal weights for all indicators of a scale. Tobia

(1999) posited that PLS is useful in screening out negligible factors affecting the
dependent variable.

ed

A reliability test was performed using Cronbachs Alpha and Composite Reliability
(CR). Table 4 shows that there is no significant defect in internal consistency. CR
greater than .70 implies that a construct retains both internal consistency and

ce
pt

convergent validity (Werts et al., 1974). Table 4 additionally illustrates how each item
contributes to the total reliability of each construct. Tests were only conducted for the
construct variable LG, which has more than 3 indicators.
To test the validity, factor loading and AVE (Average Variance Extracted) were

Ac

examined. AVE measures the variance percentage captured via a construct by

showing the variance sum ratio captured by the construct and the measurement
variance. It is acceptable when an individual item factor loading is greater than .70
and an AVE exceeds .50 (Gefen et al., 2000).
Table 4 shows individual item factor loading and each latent variables CR and
AVE. All loadings are significant at =.001. Since the CRs of all eight measurement
items shown in Table 4 are higher than .70, all latent variables can be said to have
convergent validity. The factor loadings of all items exceed .70. Therefore, the
research variables have strong reliability and validity.

Page 15 of 32

Insert Table 4 here

The AVE square root can be used to examine discriminate validity as well. As
suggested by Fornell and Larcker (1981), diagonal elements should be larger than

ip
t

the entries in corresponding rows and columns. Table 5 indicates that the AVE
square root of a latent variable (the entry in the diagonal of Table 5) is larger than the

cr

correlation of other latent variables. The correlations among all constructs are well

5.2 Testing of research hypotheses

an

Insert Table 5 here

us

below .70, suggesting that all constructs are distinct from each other.

The structural model estimation and research hypotheses tests were done using the

ed

PLS method, as shown in Figure 2 and in Table 6.

ce
pt

Insert Figure 2 here

Insert Table 6 here

Ac

In the entire sample, three of four path coefficients are higher than 0.5 (LG-IP:
0.746; IP-CP: 0.610; CP-FP: 0.672). All the paths in the research model are
significant and positive. All four research hypotheses are acceptable for the entire
sample as well as for the non-IT corporation sample. This indicates that causal
relationships exist among learning and growth, internal business processes,
customer performance, and financial performance. This implies that for Saas
customer companies, the financial category of BSC measurements focuses on
generating satisfactory return on investment and creating shareholder value while
the three other categories can be explained as determinants of financial success.
Customer performance represents value creation for customers and is important in

Page 16 of 32

explaining how customers perceive their performance. The important way to

increase customer service and market is to have massive numbers of employees
servicing customers, or by having fewer employees who utilize time management
more efficiently and whom management supports with excellent IT based processes.
Creating value for customers only translates into shareholder value if it is based on

ip
t

effective and efficient key internal processes. Finally, the study results indicate that to
make sure that the Saas customer company will still be appreciated by their future

cr

customers and will keep making excellent use of its processes, the organization and
its employees must continue organizational learning growth. The learning and growth

us

category should thus group indicators capturing the companys performance, through
the utilization of Saas, with respect to learning based on information sharing and
innovation.

an

In IT corporations, learning and growth fail to affect customer performance; on the

other hand, learning and growth significantly influence customer performance in non-

IT corporations. Therefore, corporation type, IT vs. non-IT, moderates and affects the
relationship between learning and growth and customer performance. For non-IT
companies, learning and growth are more important for the improvement of customer

ed

service and market than those factors are in IT companies. Knowledge sharing and
Saas utilization are considered natural and taken for granted by IT companies, as

ce
pt

these processes have a lesser influence on customer service and market. For non-IT
companies, there exists more opportunity for improvement in customer service and
market through learning based on information sharing and utilization of Saas.

Ac

6. Implications

6.1 Implications for researchers

Despite rapid growth in the Saas market, no research has been done on Saas
measurement based on BSCs. This study provides an empirical study on BSC
application to Saas and shows the application of comprehensive performance
management systems for SMEs, considering that SMEs have a hard time
rationalizing their operational practices and strategic processes. This study indicates
that significant causal relationships exist among BSC measures in a Saas context,

Page 17 of 32

which strengthens the justification for BSC application as a decision-supporting

instrument given that the causal relationships among the four BSC measures are the
focus of BSC assessment. By showing these causal relationships, this paper
provides a rationale for SMEs to use BSCs to assess Saas.
This study extends previous studies on ASP success in that the causal

ip
t

relationships among Saas financial performance determinants are investigated. This

study provides an assessment of comprehensive performance management systems

cr

for SMEs of specific IT services, such as Saas. Further, this study provides an
empirical basis for using BSC as a decision-supporting instrument by illuminating

us

cause-and-effect relationships and time-delay elements among measures, which

relationships are still elusive (Nielsen and Nielsen, 2012).

This study uses BSCs as a lens to suggest four measures; the intention was to

an

find out which are the leading or lagging indicators. This determination improves the
understanding of leading and lagging indicators of BSC categories based on the

ed

6.2 Implications for Practitioners

balance between financial and non-financial analysis.

SMEs, which comprise the majority of the firms in the study sample, consider

ce
pt

investment in enterprise applications to be costly and tend to avoid taking efforts

away from their main business area. In order to focus more on main business
activities, SMEs are more likely to utilize Saas, as it can provide a viable alternative
to costly in-house information systems. The high utilization of Saas, and the high

Ac

percentage of its use in core applications, show that SMEs use Saas from a strategic
decision to fill a gap in IS resources and capabilities, given that Saas provides not
only a cost efficient but also an extremely fast solution to these difficulties.
Knowledge acquisition about Saas was often informal and ad hoc, usually induced
by vendor hype rather than through a rigorous evaluation of vendor capabilities. To
address this problem, BSCs can provide a template for evaluating the performance
of application outsourcing.
The results of this study suggest that BSCs effectively assess four causally
interrelated measures, i.e., learning and growth, internal business processes,
customer performance, and financial performance. In order to increase financial

Page 18 of 32

measures, customer performance should be promoted; this, in turn, requires the

enhancement of learning and growth and internal business processes. Financial
measurement is the most lagging indicator and is effective in summarizing previous
financial activities into easily predictable economic results. Customer performance
assesses the extent to which the target market is captured. In order to increase

ip
t

financial measures, customer performance should be improved and promoted.

In order to increase customer performance, employee learning and internal

cr

business process efficiency should be improved. Internal business processes are

focused on customer satisfaction and financial objective achievement. Learning and

us

growth assess the necessary infrastructure constructed for long-term growth and
improvement. In a knowledge economy society, learning knowledge is the basis for
creating new capital, and the purpose of business is to create new knowledge. Thus,

an

continuous learning and experimentation are necessary to produce new ideas and
products; it is critical to stress the importance of an organizational culture supportive

of and encouraging learning. The designers and developers of KMS should focus on
developing an integrated KM system that facilitates knowledge acquisition and
application, the two sub-processes affecting creative organizational learning. KMS

ed

should enhance both the efficiency of existing knowledge use by processing

information and the effectiveness of new knowledge creation (Malhotra, 2001). This

ce
pt

study reinforces arguments that KMS developers should concentrate on both

creation of new knowledge and utilization of existing knowledge.

Ac

7. Conclusion

Since it is impossible, in a competitive environment, to assess an organization

merely according to financial performance measures, it is necessary to measure the
performance of Saas by using a BSC, which balances leading and lagging factors,
and which also considers financial and non-financial factors. This paper intends to
examine the causal relationships among the four BSC categories that explain the
performance of Saas.
There are some issues for future research. First, practical case studies of
corporations that have adopted BSC based performance monitoring systems are a
promising avenue for study. This line of research can practically prove that key

Page 19 of 32

performance measures of learning and growth actually improve other measures such
as customer performance and internal business processes as time goes by.
Second, in analyzing moderating effects, other moderating variables such as
corporate strategy type, market positioning type, and main service and product type
can be used. Further, it is necessary to carry out future studies to validate the reason

ip
t

that key measures of learning and growth do not affect customer performance in IT
corporations.

cr

This study has some limitations. First, due to the insufficient sample size, it is not
possible to further sub-categorize the sample in order to test moderating effects.

us

More detailed categories need to be considered in terms of IT usage or

sophistication rather than simply two categories, i.e., IT corporations versus non-IT
corporations, as used in this study. Second, there could be missing or unconsidered

an

measurement items for learning and growth, internal business processes, customer
performance, and financial performance, even though this study developed or

adapted each item based on previous literature. Each Saas-adopting corporation has
its own main business field and performance characteristics, and this may also affect

ce
pt

References

ed

the response data.

Altaf, Farheen and Schuff, David, Taking A Flexible Approach to ASPs,

Communications of the ACM, 53(2), 2010, 139-143.

Ac

Asosheh, A., Nalchigar, S., and Jamporazmey, M., Information technology project
evaluation: An integrated data envelopment analysis and balanced scorecard
approach, Expert Systems with Applications, 37(8), August 2010, 5931-5938.
Bbc News, Cloud computing for business goes mainstream, 06 May, 2010, http://
www.bbc.co.uk/news/10097450.
Behrend, T.S., Wiebe, E.N., London, J.E., and Johnson, E.C., Cloud computing
adoption and usage in community colleges, Behaviour & Information Technology,
30(2), 2011, pp. 231240.

Page 20 of 32

Behn, B.K. and Riley, R.A., Using non-financial information to predict financial
performance: The case of the U. S. airline industry, Journal of Accounting, Auditing
and Finance, 14, 1999, 29-56.
Bergman, J., Jantunen, A., and Saksa, J.M., Managing knowledge creation and
sharing - scenarios and dynamic capabilities in inter-industrial knowledge networks,

ip
t

Journal of Knowledge Management, 8(6), 2004, 63-76.

Long Range Planning, 30(2), 1997, 242-253.

Catteddu,

D.,

and

Hogben,

G.,

Cloud

Computing:

cr

Butler, A., Letza, S.R., and Neale. B., Linking the balanced scorecard to strategy,

Benefits,

risks

and

us

recommendations for information security. European Network and Information

Security Agency (ENISA), 2009, 1125.

Compeau, D.R. & Higgins, C.A., Application of social cognitive theory to training for

an

computer skills. Information System Research, 6(2), 1995, 118-143.

Currie, W.L., A knowledge-base risk assessment framework for evaluating web-

based application outsourcing projects, International Journal of Project Management,

21(3), 2003, 207-217.

Cusumano, M., Technology strategy and management: cloud computing and SaaS

ed

as new computing platforms, Communications of the ACM, 53(4), 2010, 27-29.

Davis, F.D., Bagozzi, R.P. & Warsahw, P.R., User acceptance of computer

ce
pt

technology: A comparison of two theoretical models, Management Science, 35(8),

1989, 982-1003.

DuToit, A., Knowledge: A sense making process shared through narrative, Journal of
Knowledge Management, 7(3), 2003, 27-37.

Ac

Epstein, M. and Manzoni, J.-F., Implementing corporate strategy: From tableaux De

Bord to balanced scorecards, European Management Journal, 16(2), 1998, 190-203.
Fornell, C. and Larcker, D.F., Evaluating structural equation models with
unobservable variables and measurement error, Journal of Marketing Research,
18(1), 1981, 39-50.
Fowler, G.A. and Worthen, B., The internet industry is on a cloud whatever that
may mean. Wall Street Journal Eastern Edition, 253(70), 2009, A1A10.
Frigo, M.L. and Krumwiede. K.R., Automating the Balanced Scorecard - Maximizing
Corporate Performance Through Successful Enterprise-Wide Deployment. Gentia
Software plc, 1998.

Page 21 of 32

Garengo, P. and Biazzo, S., Unveiling strategy in SME through balanced scorecard
implementation: A circular methodology, Total Quality Management & Business
Excellence, 23(1), 2012, 79-102.
Gefen D. and Ridings, C.M., IT Acceptance: Managing User-IT Group, 2003.
Gefen, D., Straub, D.W. and Boudreau, M., Structural equation modeling and

ip
t

regression: guidelines for research practice, Communications of the Association for

Information Systems, 4(7), 2000.

cr

Gonzalez-Padron, T., Chabowski, B., Hult, G.T.M., and Ketchen, David J.,
Knowledge Management and Balanced Scorecard Outcomes: Exploring the

us

Importance of Interpretation, Learning and Internationality, British Journal of

Management; 21(4), 2010, 967-982.

Harvey, M.G. and Lusch, F., Robert, Balancing the intellectual capital books:

an

intangible liabilities. European Management Journal, 17(1), 1999, 85-92.

Hoffecker, J. and Goldenberg. C., Using the balanced scorecard to develop

companywide performance measures, Journal of Cost Management, Fall 1994, 5-17.

Homburg, C., Artz, M., and Wieseke, J., Marketing Performance Measurement
Systems: Does Comprehensiveness Really Improve Performance?, Journal of

ed

Marketing, 76 (3), 2012, 56-77.

Huang, H.-C., Designing a knowledge-based system for strategic planning: A

ce
pt

balanced scorecard perspective, Expert Systems with Applications, 36(1), January

2009, 209-218.

Ittner, C.D. and Larcker, D.F., Are non-financial measures leading indicators of
financial performance? An analysis of customer satisfaction, Journal of Accounting

Ac

Research Supplement, 36, 1998, 1-35.

Jeong, B.-K., & Stylianou, A.C., Market reaction to application service provider (ASP)
adoption: An empirical investigation. Information & Management, 47(3), 2010, 176187.

Johnson, S.D., Identification and selection of environmental performance indicators:

Application of the balanced scorecard approach, Corporate Environmental Strategy,
5(4), 1998, 35-41.
Kaplan, R.S. and Norton R.D., Using the balanced scorecard as a strategic
management system, Harvard Business Review, 1996, 75-85.

Page 22 of 32

Kaplan, R.S. and Norton, D.P., The Strategy-Focused Organization: How Balanced
Scorecard Company thrive in the New Business Environment, Harvard Business
School Press, 2001.
Kern, T., Krijger, J. and Willcocks, L., Exploring ASP as sourcing strategy: theoretical
perspecitves, proposions for practice, Journal of Strategic Information Systems, 11,

ip
t

2002, 153~177.
Kim, G. and Kim, E., An exploratory study of factors influencing ASP (application

cr

service provider) success, Journal of Computer Information Systems, 48(3), 2008,

118.

us

Kim, J.W., Han, H.S., Choi, Y.J., Woo, J.A. and Park, S.C., Evaluation of SME's ASP
Adoption Effects on Economic Performance, National Computerization Agency
(Korea), 2006.

an

Klems, M., Nimis, J., and Tai, S., Do Clouds Compute? A Framework for Estimating
the Value of Cloud Computing. C. Weinhardt, S. Luckner, & J. Stoer (Eds.): WEB

2008, LNBIP 22, 2009, 110123.

Kumru, M., A balanced scorecard-based composite measuring approach to

2012, 821-843.

ed

assessing the performance of a media outlet, Service Industries Journal, 32 (5),

Kunz, H. and Thorsten Schaaf, T. General and specific formalization approach for a

ce
pt

Balanced Scorecard: An expert system with application in health care, Expert

Systems with Applications, 38(3), March 2011, 1947-1955.
Lee, A.H.I., Chen, W.-C., and Chang, C.-J., A fuzzy AHP and BSC for evaluating
performance of IT department in the manufacturing industry in Taiwan, Expert

Ac

Systems with Applications, 34(1), January 2008, 96-107.

Lee, H., Kim, J., Kim, J., Determinants of success for application service provider: An
empirical test in small businesses, International Journal of Human-Computer Studies,
65(9), 2007, 796-815.
Leam, C., and Lee, H., Development of certification and audit processes of
application service provider for IT outsourcing, Technovation, 24(1), 2004, 63-71.
Lipe, M.G. and Salterio. S.E., The balanced scorecard: Judgmental effects of
common and unique performance measures, The Accounting Review, 75(3), 2000,
283-298.

Page 23 of 32

Malhotra, Y., Organizational controls as enablers and constraints in successful

knowledge management systems implementation, in Knowledge Management and
Business Model Innovation, Malhotra, Y. (ed.), Idea Group Publishing, Hershey:PA,
2001, 326-336.
Martinsons, M., Davison, R., and Tse, D., The balanced scorecard: a foundation for

ip
t

the strategic management of information systems, Decision Support Systems, 25(1),

1999, 25-36.

cr

Mason, J., From e-learning to e-knowledge, in KM tools and techniques: practitioners

and experts evaluate KM solutions, Rao, M. (ed.), Butterworth-Heinemann,

us

Oxford:UK, 2004, 320-328.

Mayo, A. 2000, The role of employee development in the growth of intellectual

capital, Personnel Review, 29(4), 2000.

an

Mendoza, C. and Zrihen, R., Measuring Up, Financial Up, 2001, 26-29.
Moreo, A., DeMicco, F.J., Xiong, L., Towards a Model to Measure the Quality of

Environmental Sustainability: The Hospitality Environmental Scorecard, Journal of

Quality Assurance in Hospitality & Tourism, 2009, 10(1), 44-58.
Nielsen, S. and Nielsen, E.H., Discussing feedback system thinking in relation to

23(6), 2012, 436-451.

ed

scenario evaluation in a balanced scorecard setup, Production Planning & Control,

ce
pt

Northcott, D. and Taulapapa, T.M., Using the balanced scorecard to manage

performance in public sector organizations: Issues and challenges, International
Journal of Public Sector Management, 25(3), 2012, 166-191.
Olve, N.-G., Roy, J. and Wetter, M., Performance Drivers: A Practical Guide to Using

Ac

the Balanced Scorecard. Wiley, 1999.

Petty, R. and Guthrie, J., Intellectual capital literature review: Measurement,
reporting, and management, Journal of Intellectual Capital, 1(2), July 2000, 155-176.
Pfeffer, J., Changing mental models: HR's most important task, Human Resource
Management, 44(2), 2005, 123-128.
Roos, G. and Roos, J., Measuring your companys intellectual performance, Long
Range Planning, 30(3), 1997, 413-426.
Schwarz, A., Jayatilaka, B., Hirschheim, R., and Goles, T., A Conjoint Approach to
Understanding IT Application Services Outsourcing, Journal of the Association for
Information Systems, 10(10), 2009, 748-781.

Page 24 of 32

Smandek, B., Barthel, A. Winkler, J., Ulbig, P., Balanced score card implementation
for IP rights management in a public research institution,

Measuring Business

Excellence, 2010, 14(4), 2010, 65-75.

Sim, K.L. & Koh, H.C., Balanced scorecard: A rising trend in strategic performance
measurement, Measuring Business Excellence, 5, 2001, 18-27.

ip
t

Sohn, M., You, T., Lee, S., and Lee, H., Corporate strategies, environmental forces,
and performance measures: a weighting decision support system using the k-

cr

nearest neighbor technique. Expert Systems with Applications, 25(3), 2003, 279-292.
Stewart, W.E., Balanced Scorecard for Projects, Project Management Journal, 32(1),

us

2001, 38-53.

Susarla, A., Barua, A., Whinston, A.B., Transaction Cost Perspective of the
"Software as a Service" Business Model, Journal of Management Information

an

Systems, 26(2), 2009, 205-240.

Susarla, A., Barua, A., Whinston, A.B., Understanding the service component of

MIS Quarterly, 27(1), 2003, 91-123.

application service provision: An empirical analysis of satisfaction with ASP services,

Taylor, J. and Baines, C., Performance management in UK universities:

ed

implementing the Balanced Scorecard, Journal of Higher Education Policy &

Management; 34(2), 2012, 111-124.

ce
pt

Tobia, R.D., An Introduction to Least Squares Regression. SAS Institution Inc., 1999.
Vaquero, L.M., Rodero-Merino, L., Caceres, J., and Lindner, M., A break in the
clouds: Towards a cloud definition, Computer Communication Review, 39(1), 2009,
5055.

Ac

Velcu, O., Strategic alignment of ERP implementation stages: An empirical

investigation, Information & Management, 47(3), April 2010, 158-166.
Werts, C.E., Lin, R.L. and Joreskog, K.G., Intraclass reliability estimates: Testing
structural assumptions, Educational & Psychological Measurement, 34(1), 1974, 2533.
Withanachchi, N., Handa, Y., Karandagoda, K.K.W., Pathirage, P.P., Tennakoon,
N.C.K., Pullaperuma, D.S.P., TQM emphasizing 5-S principles/A breakthrough for
chronic managerial constraints at public hospitals in developing countries,
International Journal of Public Sector Management; 20(3), 2007, 168-177.

Page 25 of 32

Wold, H., Partial Least Squares. In Encyclopedia of Statistical Sciences, in S. Kotz

and N. Johnson (Eds.) Wiley, New York, NY, 1985, 58-591.
Wu, H.-Y., Constructing a strategy map for banking institutions with key performance
indicators of the balanced scorecard, Evaluation & Program Planning, 35(3), 2012,
303-320.

ip
t

Wu, W.-W., Developing an explorative model for SaaS adoption, Expert Systems
with Applications, 38(12), 2011, 15057-15064.

cr

Wu, H.-Y., Tzeng, G.-H., and Chen, Y.-H., A fuzzy MCDM approach for evaluating
banking performance based on Balanced Scorecard, Expert Systems with

us

Applications, 36(6), August 2009, 10135-10147.

Yao, Yurong, Watson, Edward, Kahn, Beverly, Application Service Providers: Market
and Adoption Decisions, Communications of the ACM, 53(7), 2010, 113-117.

an

Yawson, R., Amoa-Awua, W. Sutherland, A. Smith, D. Noamesi, S., Developing a

performance measurement framework to enhance the impact orientation of the Food

Research Institute, Ghana, R&D Management, 36(2), 2006, 161-172.

Zrivan, M., Pliskin, N., Levin, R., Measuring user satisfaction and perceived
usefulness in the ERP context, Journal of Computer Information Systems, 45(3),

Ac

ce
pt

ed

2005, 43-52.

Page 26 of 32

H2
Customers

H1
Internal Business

Financial

H4

Performance

us

Performance

cr

Learning and
Growth

ip
t

Figures and Tables

H3

an

Process

Ac

ce
pt

ed

Figure 1: Research model causal relation among measures of Saas

Page 27 of 32

Table 1: Research variables

Sources

ip
t

Roos and Roos (1997)

Epstein and Manzoni (1998)
Kaplan and Norton (1996,
2001)
Harvey and Lusch (1999)
Lipe and Salterio (2000)
Kim et al. (2006)

Martinsons et al. (1999)

Olve et al. (1999)
Kim et al. (2006)

ce
pt

ed

an

The
work
process
is
processed in a great portion
through Saas (LG1).
There exists a great extent of
knowledge sharing within
organization after Saas is
adopted (LG2).
There exists a great extent of
knowledge sharing with other
organizations
(trading
partners) after Saas is
adopted (LG3).
Internal
The
customer
order
Business
processing time is greatly
Process
reduced
after
Saas
is
Performance adopted(IP1).
(IP)
The customer request time
limit is strictly followed after
Saas is adopted (IP2).
Customers
The customer service is
Performance greatly improved after Saas is
(CP)
adopted (CP1).
The company has a great
annual average increase in
customer market (CP2).

cr

Measurement items

us

Latent
variables
Learning
and Growth
(LG)

Ac

Financial
The ordering cost is greatly
Performance reduced
after
Saas
is
(FP)
adopted (FP1).
The companys revenue is
greatly increased per year
after Saas is adopted (FP2).

Hoffecker and Goldenberg

(1994)
Butler et al. (1997)
Frigo et al. (1998)
Epstein and Manzoni (1998)
Harvey and Lusch (1999)
Martinsons et al. (1999)
Petty and Guthrie (2000),
Lipe and Salterio (2000)
Stewart. (2001)
Mayo (2000)
Hoffecker and Goldenberg
(1994)
Butler et al. (1997)
Johnson. (1998)
Epstein and Manzoni (1998)
Marc et al. (1998)
Harvey and Lusch (1999)
Kaplan et al. (1996, 2001)
Lipe and Salterio (2000)
Mendoza and Zrihen. (2001)

Page 28 of 32

Table 2: Profile of Sample

22
19
51
8
100

22
19
51
8
100

39
42
19
100

39
42
19
100

ed

ce
pt

Ac

cr
us

an

43
38
19
100

ip
t

Percentage (%)

43
38
19
100

22
41
37
100

22
41
37
100

41
20
24
15
100

41
20
24
15
100

36

36

71

71

32
100

32
100

Industry of the company

Manufacturing industry
Retail industry
Service industry
Others
Total
Size of company (Number of employees) (Mean =
79)
1-10
11-50
More than 50
Total
Years of adopting Saas (years) (Mean=3)
1-2
3-4
More than 4
Total
Percentage of Saas utilization in total processes
(%) (Mean = 45.2)
0-10
11-50
More than 50
Total
Monthly Saas usage fee (dollar)
(Mean = 85)
0-10
11-30
31-100
More than 100
Total
Applications using Saas
Supporting applications (accounting, personnel,
administration)
Core business (sales, production, purchasing,
logistics applications)
Business-to-business applications
Total

Number of
companies

Page 29 of 32

Financial
Performance
(FP)

0.372
0.683
-0.013
0.711
0.85
0.966
0.965
0.682
0.571

0.346
0.462
-0.037
0.557
0.593
0.624
0.674
0.949
0.927

cr

ip
t

Customers
performance
(CP)

us

Table 3: Exploratory factor analysis

Learning and Internal
Growth (LG) Business
Process
Performance
(IP)
LG1
0.814
0.383
LG2
0.929
0.651
LG3
0.851
-0.014
IP1
0.674
0.954
IP2
0.752
0.966
C1
0.716
0.825
C2
0.701
0.754
F1
0.586
0.612
F2
0.384
0.505

Ac

ce
pt

ed

an

Table 4: Reliability and validity of research variables

Average
Composite
variance
Measurement Items
Loading
reliability (CR) extracted
(AVE)
Learning and Growth (LG) LG1
0.8290
0.872
0.695
LG2
0.8537
LG3
0.8175
Internal Business
IP1
0.9603
0.960
0.922
Process Performance (IP) IP2
0.9603
Customers performance CP1
0.9659
0.965
0.933
(CP)
CP2
0.9659
Financial
PerformanceFP1
0.9603
0.937
0.922
(FP)
FP2
0.9603

Page 30 of 32

Table 5: Correlations among latent variables

(The numbers in diagonals are square root of the average variance extracted.)

Learning and
Growth (LG)
Internal
Business
Process
Performance
(IP)
Customers
performance
(CP)
Financial
Performance
(FP)

0.834
0.960

0.746

0.818

0.526

0.6

0.966

LG3

0.852**

0.826**

ce
pt

Learning and
Growth

H2

0.610**

H1

0.746**

Ac

Internal Business

0.672**

Customers

Financial
Performance

Performance

0.956**

CP1

Process

IP1

H4

H3

0.279*

0.954**

0.960

0.672

ed

0.814**

FP

an

0.734

LG2

LG1

CP

ip
t

IP

cr

LG

us

Latent
variables

0.965**

CP2

0.950**

FP1

0.927**

FP2

0.966**

IP2

Figure 2: PLS analysis results for entire sample

* p < 0.05, ** p < 0.01

Page 31 of 32

Table 6: Test of research hypotheses

Sample of IT
companies
0.746**

0.279*

0..107

0.610**

0.760**

0.672**

0.576**

0.452**
0.467**
0.773**

Ac

ce
pt

ed

an

us

* p < 0.05, ** p < 0.01

Sample of nonIT companies

0.761**

ip
t

H1: Learning and Growth Internal Business

Process Performance
H2: Learning and Growth Customers
performance
H3 : Internal Business Process Performance
Customers performance
H4: Customers performance Financial
Performance

Entire
sample
0.746**

cr

Research hypotheses

Page 32 of 32