The current issue and full text archive of this journal is available at

http://www.emerald-library.com/ft

Exploring consumer Exploring

consumer
evaluations of e-services: evaluations

a portal site
Allard C.R. van Riel 359
Department of Marketing and Marketing Research, University of Received April 2001
Maastricht, Maastricht The Netherlands Revised July 2001
Accepted Julye 2001
Veronica Liljander
Department of Marketing, Swedish School of Economics and Business
Administration, Helsinki, Finland, and
Petra JurrieÈns
Department of Marketing and Marketing Research, University of
Maastricht, Maastricht, The Netherlands

Keywords Services marketing, Customer loyalty, Customer satisfaction, Internet,

Service quality
Abstract The commercial importance of e-services, accessed via Internet or mobile phone,
grows steadily. There is a clear need to develop a better understanding of how consumers evaluate
these services and develop e-loyalty. An empirical study of an Internet portal was tied in with
extant theories about service quality, customer satisfaction and loyalty. Data were collected by
means of an on-line survey. The effect of customer satisfaction with three service components on
value perception and overall satisfaction was measured. Customers' overall satisfaction was
influenced by their satisfaction with the core service, supplementary services and the user
interface. A strong positive effect of overall satisfaction on the intention to continue using the
portal was found. Noteworthy was the predominant effect of satisfaction with supplementary
services on value perception. No significant direct effects were found of satisfaction with the user
interface on value perception. Some implications as well as future research directions are
provided.

Introduction
In the present paper we investigate the way consumers evaluate an Internet
portal site and identify the major components of the online service offer. We
explore how satisfaction with these components affects value perception,
customer satisfaction and loyalty.
Portal sites are an important emerging and really new type of interactive
ICT-based service (Lehmann, 1997) that can be accessed either via computer
(Internet) or via mobile phone (e.g. via WAP or DoCoMo's I-mode)[1]. Portals
offer many functional advantages over traditional media, such as easy access
to several related services, access to almost unlimited content and excellent
The authors wish to gratefully acknowledge the helpful comments and encouragements of the International Journal of Service
Industry Management,
editor and two anonymous referees, as well as the kind cooperation of the publisher of the portal Vol. 12 No. 4, 2001, pp. 359-377.
site. The data were collected for Petra JurrieÈns' Master's thesis, at the University of Maastricht. # MCB University Press, 0956-4233
IJSIM retrieval facilities. Providers of portal sites manage to capture distinctive
12,4 professional audiences or market segments by combining high tech ICT-
solutions with their market knowledge and editorial skills. A whole range of
customized facilitating and supporting services, such as search engines or
communication forums and newsgroups can add value to a core service. Profits
can be generated by the core service or by complementary commercial services
360 that leverage the targeted market segment. The provider of the core service, or
third parties, can commercialize these complementary services, which could for
example include specialized financial advice, travel information and online
bookings, or the supply of professional equipment. In the publishing industry,
portal sites are rapidly becoming a major alternative to ± or at least important
supplements of ± traditional print media, such as textbooks, journals,
magazines and newspapers.
The acquisition of e-customers is difficult and expensive. Customers can
quite easily evaluate and compare the benefits of competing services and
switching costs are low. As a result, customer retention in e-services ± or e-
loyalty ± is of paramount importance (Reichheld and Schefter, 2000). Relatively
little is known, however, about how consumers perceive and evaluate
electronically offered services and how they develop loyalty to their providers.
Most research into consumer evaluations and quality of services has been
conducted with respect to services that are characterized by personal
interactions between customers and employees. Therefore, its outcomes cannot
be readily applied to services that are characterized by interactions with
technology, i.e. the fourth leg in Parasuraman's pyramid model (Parasuraman,
1996, 1997). The rapid development of information and communication
technologies during the 1990s has enabled companies to introduce more and
more high-tech services. We can think of Internet banking and other completely
new services that add value to existing products by substituting or
complementing personal interactions with service staff by means of
technological solutions. Taking into account these developments, it is evident
that service researchers need to pay more attention to consumer evaluations of
technology-based services (Parasuraman and Grewal, 2000). So far, research on
consumer evaluations of self-service technology (SST) (Bitner et al., 2000;
Dabholkar, 1996; Meuter et al., 2000), like electronic services (e-services), has
been scarce. For example, little attention has been paid to the theoretical
implications of differences between evaluations of service offerings on the
Internet and in the physical marketplace (GroÈnroos et al., 2000).
Research is needed with respect to the influence of technology on all
customer responses, such as perceived value, satisfaction and loyalty
(Parasuraman and Grewal, 2000). Developing insight into the determinants of
value perception, satisfaction and loyalty for a portal site is important for both
marketing researchers and the managers in charge of designing the service.
This paper contributes to services research by empirically exploring customer
satisfaction with different components of an e-service. We study a medical
professional portal site, which for an annual fee offers access to an electronic
journal and a number of supplementary services to general practitioners. The
purpose of the empirical study was to investigate the relative importance of Exploring
service components on customers' value perception, overall satisfaction and consumer
intentions to continue using the portal.
We have structured our paper as follows. First, the current state of
evaluations
knowledge with respect to consumer evaluation of e-services is discussed and
some propositions for the empirical study are developed. Second, we present
the design of the empirical study and analyze our findings. The paper 361
concludes with a discussion, some managerial implications and a number of
directions for future research.

Literature review and development of propositions

Research on e-services is still in its infancy and no generally accepted
theoretical conceptualization for consumer evaluations of these services has yet
emerged. Past conceptualizations of services were created to capture the
interpersonal nature of service encounters (Meuter et al., 2000) and may not
adequately cover the specific characteristics of customer interactions with SST,
as is the case in e-services. On the other hand, past conceptualizations need not
be completely discarded but may instead offer a useful platform for the
theoretical description of e-services. Currently, two approaches to studying e-
services can be distinguished: one that proposes the use of existing service
quality theory as a basis for further empirical research (GroÈnroos et al., 2000;
Parasuraman and Grewal, 2000; Kaynama and Black, 2000; Zeithaml et al.,
2000), and a more empirical approach, which has generated new categories for
general SSTs (Dabholkar, 1996; Meuter et al., 2000); or e-services (Szymanski
and Hise, 2000). Propositions and empirical findings from these studies are
analyzed next.

Conceptualization of e-services
Customer perceived service quality has been theoretically represented as
consisting of two dimensions. Berry and Parasuraman (1991) distinguish a
process and an outcome dimension, whereas GroÈnroos (1990) makes a
distinction between functional and technical quality. The process or functional
quality refers to ``how'' the service is delivered, while the outcome or technical
quality refers to ``what'' customers receive, the benefits of using the service.
GroÈnroos et al. (2000) propose that for services offered on the Internet the
traditional service concept, consisting of the core service, facilitating and
supporting services, needs to be extended with a fourth factor, the user
interface. As noted by GroÈnroos et al. (2000), it is often difficult to differentiate
between facilitating and supporting services. A term that has been coined to
more generally denote services that are not part of the core service is
supplementary services (Anderson and Narus, 1995). Facilitating and
supporting services are both closely connected with the core service. For an on-
line bookseller facilitating services could be search facilities, an invoice archive
and secure payment methods. Book reviews and personal recommendations
could be interpreted as supporting services. In addition, independent, though
related products that are neither facilitating nor supporting the core service can
IJSIM be offered through the portal site. In the case of a medical portal, we could think
12,4 of self-contained products, such as specialized financial services, insurances, a
travel agency, an option to purchase medicine online, etc. We would suggest
preserving the adjective ``complementary'' for those products, in order to
distinguish them from supplementary services. Whereas supplementary
services have the potential to add value to the core service, complementary
362 services are self-contained products that do not add value to the core service,
but have the potential to add value to the portal in its entirety. Finally, the
functionality, outlook, logic and usability of the site constitute the user
interface. Core, supporting and complementary services can be interpreted as
``what'' customers receive. The user interface and, at least to a certain extent,
the facilitating services describe ``how'' the service is delivered to customers.
Supplementary services are used to differentiate the service from similar
competing offerings and to add value to customers (Anderson and Narus, 1995;
GroÈnroos et al., 2000). However, Anderson and Narus (1995) observe that
companies often misuse them by not investigating which supplementary
services customers actually need and value. The authors suggest that
``customers are understandably happier when they are offered services for free''
(Anderson and Narus, 1995, p. 77). However, when supplementary services are
not customized to the targeted customer segment but offered indiscriminately
and unprofitably to everyone, their added value can be limited, both to
consumers and to the firm. Anderson and Narus addressed inter-firm services,
but their observations seem to apply to private customers as well (Zeithaml et
al., 2000). Because of the high costs involved in the upkeep and development of
supplementary services, it seems especially important for companies to find out
whether they are positively valued by customers and how they affect
satisfaction and e-loyalty.
All components of the e-service offer are expected to influence overall
satisfaction with a portal site offering, if customers perceive them to be
superior to their expectations, compared to competitors' e-offerings or similar
services of the ``brick-and-mortar'' kind. It seems likely that the added value of a
complementary service, as defined above, will depend not only on its individual
quality, but also on the degree of fit with the core service offering. This
phenomenon lies beyond the scope of the present study and from now on we
will therefore limit ourselves to studying the effects of core and supplementary
services as well as of the user interface. Because of the lack of past research, no
propositions will be made regarding the individual importance of each element.
We propose:
P1. Satisfaction with each service component is positively related to overall
satisfaction with the portal site.
The quality of all e-service components can also be expected to add value to the
portal. We therefore propose:
P2a. Satisfaction with each service element is positively related to perceived
value of the total offer.
We expect core and supplementary services to be the most important Exploring
determinants of perceived value. At the same time, customers are known to consumer
expect high core service quality and to take this high quality largely for
granted (Gwinner et al., 1998). Therefore, other benefits or incentives, such as
evaluations
supplementary services, will be important in order to add value to the service
(Gwinner et al., 1998; Ravald and GroÈnroos, 1996). Consequently we propose:
P2b. Satisfaction with supplementary services will have a stronger effect on 363
perceived portal value than satisfaction with the core service.

E-service quality
Using a portal site can be described as a complex process that can be divided
into sub-processes, such as retrieving or exchanging information and articles,
or ordering books. Each of the four service components could be described in
terms of process and outcome, and could also be analyzed into sub-dimensions
and quality determinants. For example, customers may be satisfied with the
responsiveness and user-friendliness of a search engine but dissatisfied with
the reliability of the search outcome. We assume that the un/successful process
or outcome of an interaction influences customers' overall satisfaction with the
Web site. However, most likely the customer will not evaluate each sub-process
in detail during a single visit to a Web site, but will perceive the service as an
overall process and outcome, unless one of the service elements stands out as
especially positive or negative. The same applies to a series of interactions
within a customer e-service relationship. In a study of critical incidents with
SST, Meuter et al. (2000) found that positive incidents were rare and that
dissatisfying incidents were often caused by process failure, such as lost
orders.
The process and outcome dimensions of an e-service can thus be divided into
more specific service quality dimensions. The best-known evaluation
dimensions are the five proposed in the SERVQUAL model (Parasuraman et al.,
1988): tangibility, responsiveness, reliability, assurance and empathy.
Parasuraman and Grewal (2000, p. 171) suggest that research is needed on
whether ``the definitions and relative importance of the five service quality
dimensions change when customers interact with technology rather than with
service personnel''. Since the dimensions were developed from, and for,
interpersonal encounters, in our opinion the content description and scale items
would have to be reformulated before they can be meaningfully used in an e-
service context. For example, tangibility could be replaced with the user
interface. Responsiveness could refer to the company's responses to customers'
requests and the speed of order confirmation. Reliability could relate to the on-
time delivery of ordered goods, accurate supply of information and error-
freeness of links. Assurance could refer to the safety of on-line transactions and
the policy for using personal information by the company. Assurance refers to
a general trustworthiness, which has been considered one of the most
important drivers of e-service satisfaction (Petersen, 2001; Urban et al., 2000;
Wingfield and Rose, 2001). Finally, empathy could be interpreted as the degree
of customization of communications and the service provider's awareness of
IJSIM customers' personal needs. Evaluation and relevance of these dimensions may
12,4 vary for different e-services and could even vary between features. For
example, customers expect high responsiveness from a search engine but may
tolerate slower financial transactions if they lead to increased security
(assurance). To what extent the five dimensions actually capture customer
perceptions of e-service quality and which trade-offs might exist remains to be
364 shown. The five dimensions were not tested systematically in the focal study.
Consumer evaluations of services can be impacted by the choice of a
comparison standard (Cadotte et al., 1987; Zeithaml et al., 1993). Meuter et al.
(2000) found that satisfying incidents of SST were most often related to a
comparison with traditional or other SST services, and Szymanski and Hise
(2000) found that convenience in comparison with traditional stores was an
important determinant of overall satisfaction. Thus, customers appear to use
experience-based norms (Cadotte et al., 1987) and traditional services as
comparison standards for SSTs and e-services. However, e-service expectations
seem to be of less importance as a comparison standard (Zeithaml et al., 2000)
Empirical studies have demonstrated that consumer benefits of using SST
include convenience (Meuter et al., 2000; Reichheld and Schefter, 2000;
Szymanski and Hise, 2000; Zeithaml et al., 2000), saving time and money
(Meuter et al., 2000), avoiding interpersonal interaction (Dabholkar, 1996;
Meuter et al., 2000), and being in control (Dabholkar, 1996; Zeithaml et al., 2000).
Features of the user interface, such as site design (Szymanski and Hise, 2000)
and ease of use (Dabholkar, 1996; Meuter et al., 2000; Zeithaml et al., 2000),
affect service quality perception and satisfaction. However, Meuter et al. (2000)
found that technology was only mentioned in the context of dissatisfying
incidents. Similarly, financial security and other assurance aspects may also
affect e-satisfaction only in the case of problems (Szymanski and Hise, 2000;
Zeithaml et al., 2000).
On the other hand, the way a Web site is designed can also make it positively
fun to use. Dabholkar (1996) found a strong positive effect of enjoyment of
using SST on perceived overall service quality. Enjoyment captures the
hedonic dimension of consumption and can be measured as the extent to which
consumers find the SST to be enjoyable, fun and pleasant to use. In this sense,
satisfaction can also be described as a moderately positive affective state
(Russell, 1980), but in services research satisfaction is most often treated as a
composite dependent variable that is influenced by both cognitively perceived
service quality and affective reactions to the service (e.g. Liljander and
Strandvik, 1997). Customers' affective states can be expected to influence and
be influenced by the use of SST. For example, customers' initial mood when
starting to use the SST can affect their evaluation of it, and using the SST may
cause new affective reactions, such as excitement or irritation. Except in
Dabholkar's study, customers' affective reactions to SST have not been
investigated empirically. The relative importance of this factor will probably
depend on the targeted customer category and the type of service. Hedonic
effects will presumably play a more important role for consumers of leisure-
related services than for users of a professional service.
Conceptualization of e-loyalty Exploring
Loyal customers are important, because they contribute to the profitability of consumer
the service provider (Anderson and Mittal, 2000; Anderson and Sullivan, 1993;
Iacobucci et al., 1994; Storbacka et al., 1994). They purchase more than newly
evaluations
acquired customers and can be served with reduced operating costs. They
provide free word of mouth and are likely to pay regular prices. In a recent
update on e-loyalty, Reichheld and Schefter (2000, p. 107) observe that contrary 365
to current beliefs ``price does not rule the Web; trust does''. According to the
authors, referrals are extremely important in e-services and may count for half
of the acquired customers for some services. Furthermore, referred customers
were found to ask for advice and guidance from the loyal customers who
recommended the service to them, thus reducing the company's costs for
providing help.
Loyalty is largely, though not exclusively, satisfaction driven[2] (Oliver,
1999; Rust et al., 1995) and therefore customer satisfaction measurements are
believed to give a better indication of future performance of service firms
(Anderson and Fornell, 1999) than, for instance, financial and accounting based
measures (Kaplan and Norton, 1996). Removing the causes of dissatisfaction is
considered crucial for customer retention (Anderson and Mittal, 2000). At the
same time it must be acknowledged that not all loyal and satisfied customers
need be profitable to a firm (Storbacka, 1994). However, since it may take
several years before acquired e-service customers become profitable it is in the
provider's interest to first work on the retention of all its customers, and then
focus on the most profitable ones. According to Reichheld and Schefter (2000),
retention of e-customers can only be accomplished by providing superior
service quality, not by technology alone. We therefore propose:
P3. Customer satisfaction has a positive effect on the intention to return to
the portal.

Empirical study
Description of the service: a portal for general practitioners
The portal site we studied was a self-contained project initiated by a
renowned medical publisher. In essence the Internet pay site[3] provided its
customers, general practitioners (GPs), with medical information and kept
them up to date. During our study, access to full content was limited to
subscribers, though non-subscribers had the opportunity to explore the full
site for free for a limited period of time. The site consisted of seven elements:
current news, a discussion platform, a medical book show case, information
guide, suggestions for organizing the medical practice, information on new
products and subscription services. A detailed description of the services can
be found in the Appendix.
Apart from these seven elements an option to electronically search the site
was provided, as well as a link to Medline[4] (a US medical site). Knowledge
tests were made available online for GPs' compulsory refresher courses. This
part was freely accessible. Finally, a free newsletter anybody could subscribe to
was distributed to a large number of addresses every week.
IJSIM Based on the literature review, we created a framework for the e-service
12,4 offering. This is presented in Figure 1. The figure illustrates how the service
can be divided into the five e-service components:
(1) The core services.
(2) Facilitating services.
366 (3) Supporting services.
(4) Complementary services.
(5) The user interface, through which the customer accesses the services.
Examples from the medical portal site are given for each of the components.
Figure 1 also illustrates that customers can experience processes and
outcomes for all five components. Each element can also be further analyzed
into its constituent service quality dimensions. Next to value and quality
judgments, customers may also experience distinct emotions with respect to
each of the elements. For example, while the core service may be perceived as
reliable, the supporting services may not, which may cause frustration and
anger. For some customers the basic service process can consist of information
retrieval, for others of product ordering and delivery services. For the service
provider it is therefore important that any service offered on the Web site
satisfies its respective users. Irritation about substandard supporting services
could for example reduce confidence in the core service.

Research design and measurement

A number of key evaluation criteria of the portal site were identified through
interviews with five current and former paying users of the service. Based on

Figure 1.
Conceptualization of an
e-service
our conceptualization of an e-service as consisting of four components, the Exploring
results of the interviews and the traditional service quality literature, a consumer
questionnaire was created.
evaluations
Findings from interviews
The interviews revealed that the factors that mattered most for general
practitioners were the relevance, reliability and trustworthiness of journal 367
content, and of other news and information presented on the Web site. Integrity
of the publisher with respect to their personal data was also important. The
respondents mentioned that if a third party were to get hold of their private
data this would be a reason for canceling their subscription. Responsiveness
was not deemed important, as practitioners showed little interest in contacting
or having a deeper one-to-one relationship with the publishing company. They
were only interested in contacts with other practitioners. They did, however,
expect the publisher to find out about general matters of interest to them. They
expected the portal to provide information that met their specific needs. The
main reason reported for using the service was to fulfill a need of staying up to
date by receiving new information. A problem mentioned by most GPs was a
general lack of time to obtain that information via other means. Speed of
delivery of hard copies of journals was not deemed important because they
were always delivered on time. The doctors had not experienced any problems
with logging on to the Web site and were satisfied with their Internet
connections. The price of the journal was not deemed important either, because
they could subtract it from their taxes. The price of a subscription to the Web
site was by some perceived as not important, because it provided a unique and
invaluable service, while a few felt that the service should be free of charge.

Survey design
The survey included questions on customer satisfaction with different aspects
of the different site components, overall satisfaction with the portal, intentions
to continue using the site in the future and perceived value of the service.
Questions about potential improvements were also included. All independent
variables were measured as the extent to which the participants agreed with
statements on seven-point scales, ranging from ``completely disagree'' to
``completely agree''. We measured our dependent variables with single-item
measures, since no validated multi-item scales were available for value,
satisfaction and loyalty in an Internet context. Perceived value was measured
as agreement with the sacrifice/benefit statement: ``For the money, the portal
delivers good quality''. The measure of overall satisfaction with the portal
ranged from ``very dissatisfied'' to ``very satisfied''. Loyalty, or rather ``the
intention to return', was measured as the likelihood doctors would continue
using the portal in the future, ranging from ``very small'' to ``very big''.
Respondents were also asked if they felt that the site should be free of charge.
The other questionnaire items are presented in connection to the results.
The survey was announced in brief on the portal homepage, where a link
was provided to a Web form containing the questionnaire. Every practitioner
IJSIM visiting the site between 27 November and 31 December 2000 was thus asked
12,4 to participate in the survey. At the same time the purpose of the survey and a
request to participate were published in the free weekly newsletter, distributed
by e-mail. Very few subscribers completed the questionnaire (response rate 7
per cent, or 52 respondents). The fact that so few general practitioners
participated may be due to the combination of a general lack of time reported
368 by the doctors and the length of the questionnaire. We also feel that the
announcement of the survey on the homepage could have been more prominent
and inviting, but we had little control over this. Because of the limited sample
size the study should be considered exploratory.

Results and discussion

Most of the 52 respondents were general practitioners, and only nine of them
had less than five years working experience. Most of them visited the Web site
once a week (24 respondents) or more often (13), and more than half of them (32)
were paying subscribers with access to all available services. To a certain
extent the site was perceived as a substitute for hard copies of subscribed
journals (mean 3.6 on seven-point scale). The reasons practitioners reported for
using the site are presented in Table I.
The response rate was quite low, therefore our sample was small (n = 52)
and problems exist both with respect to generalizability to the full population
as well as with respect to the choice of statistical techniques to be used for the
analysis. Therefore descriptive statistics and correlations played a relatively
important role. Nonetheless, we performed a number of confirmatory
multivariate regressions on a limited number of variables.

Evaluation of the portal

The practitioners generally did not expect the site to be free of charge (mean 2.8
on a seven-point scale) and the item did not correlate with overall satisfaction
or loyalty. This can be explained by the fact that they could deduct the
subscription from their taxes. Correlation between overall satisfaction and
loyalty was 0.66, between perceived value and satisfaction 0.60 and between

Questionnaire items Number in agreementa

Added value of the portal in addition to journals

Always available 39
Search facilities 36
Archive function 35
Daily news 29
Motives for subscribing to the portal
Keep up with current medical developments 38
Good value 7
Table I. Recommended by a colleague 3
Perceived value of the
a
service Notes: More than one option could be chosen; n = 52
value and loyalty 0.39. Table II presents correlations between questionnaire Exploring
items and perceived value, overall satisfaction and the likelihood of continuing consumer
using the portal. Although more items were used in the questionnaire, only the evaluations
ones that correlate strongly with our dependent variables are included in this
table. Significant correlations with perceived value and loyalty are underlined
when they are higher than for satisfaction.
Table II demonstrates that most of the items show higher correlation with 369
overall portal satisfaction than with perceived value or portal loyalty. This may
be an artifact of measuring customer satisfaction with each item (instead of, for
example, quality). Although practitioners in the interviews underlined the
importance of discreetly handling personal data it had no effect on overall

Correlation with service dimensions VALa SATb LOYc Meand

Core service
The portal and journals complement each other well 0.48** 0.84** 0.60** 5.0
I get additional information from the site, with respect to 0.35* 0.80** 0.49** 5.2
what I get from journals
I'm satisfied with the completeness of the offered services 0.39** 0.69** 0.41** 4.9
I'm satisfied with the way services adapt to my interests 0.66** 0.68** 0.37** 5.3
I'm satisfied with the readability of articles on the site 0.37* 0.63** 0.37* 5.4
I'm satisfied with the news on the site 0.33* 0.58** 0.48** 5.5
I'm satisfied with the scientific level of the site 0.54** 0.58** 0.30* 4.7
I'm satisfied with the scientific content of the site 0.40** 0.50** 0.24 4.7
I'm satisfied with the focus of the site on general 0.39** 0.48** 0.46** 5.6
practitioners
Supporting services
I'm satisfied with the possibility of information retrieval 0.59** 0.69** 0.49** 4.6
I'm satisfied with how they inform me about non-medical 0.50** 0.54** 0.31* 4.6
products
I'm satisfied with how they bring medical information from 0.40** 0.52** 0.53** 5.2
other sources to my attention
I'm satisfied with the continuous updating of the site 0.42** 0.48** 0.31* 5.2
I'm satisfied with the offers that they make to subscribers 0.38** 0.46** 0.40** 4.1
I'm satisfied with how the site is integrated with other 0.34* 0.30* 0.21 4.6
products
I'm satisfied with the fact that I get only one bill for the 0.08 0.11 0.23 4.8
different services that are offered on the site
I'm satisfied with the possibility have discussions with 0.06 0.08 0.23 4.6
other doctors
User interface
I'm satisfied with the accessibility of the site 0.45** 0.57** 0.32* 5.0
I'm satisfied with the transparency of the site 0.41** 0.53** 0.24 5.1
I'm satisfied with the user friendliness of the site 0.26 0.50** 0.33* 5.1 Table II.
I'm satisfied with the design of the site 0.32* 0.48** 0.18 4.9 Major items and their
I'm satisfied with the error freeness of the links at the site 0.31* 0.23 0.34* 5.1 correlations with
perceived value, overall
Notes: a VAL = perceived value. b SAT = overall satisfaction. c LOY = likelihood of using satisfaction and
the site in future. d Scale range used: 1 (completely disagree) to 7 (completely agree). intentions to use the
* Significant at the 0.05 level; ** significant at the 0.01 level site in the future
IJSIM satisfaction in the survey sample. It may be the case that discretion is taken for
12,4 granted and dissatisfaction only arises when privacy is violated. The extent to
which the portal and was perceived to complement the journals and the extent
to which it provided additional information were especially important for
overall satisfaction, but also for loyalty. On the other hand, for example, the
scientific level of the site, adaptation to practitioners' interests, and the
370 possibility of information retrieval affected both satisfaction and perceived
value. Another factor that came up in the interviews, being able to discuss with
other general practitioners through the site, was not significantly related to
overall satisfaction. We suspect that this is due to the current newsgroup-like
organization of the ``Platform'' feature, where discussions are not taking place
publicly in real-time, but indirectly by posting messages to the author of a
request. This option currently does not seem to fulfill doctors' needs to
communicate directly with colleagues. From Table III we can conclude as well,
that more direct and interactive communication with other doctors remains a
desired supplementary service.
Some potential supplementary and complementary services were not
available yet on the site, but the publisher was considering adding them to the
portal. Customers were therefore asked if they thought the Web site could be
improved by adding these features. The results are presented in Table III and
show that practitioners' interest in the suggested features was rather neutral.
Only the options of downloading software and communicating with other
doctors, directly or via a private homepage, might improve the attractiveness of
the site.

Data analysis
In order to be able to perform multivariate regressions with our limited sample,
we reduced the number of independent variables by creating factors based on
our four-component model. We took into consideration that it is difficult to
distinguish between supporting and facilitating services and therefore took
them together as supplementary services. We selected the items that correlated
highly (> 0.46) with at least one of the dependent variables to include them in
the factors. These items are listed in Table II. We then excluded items that
decreased the internal consistency of the factors thus formed. Accordingly we
created ``core service'' (COR,  = 0.91) ``supplementary services'' (SUP,  = 0.84)

The quality of the site would be improved by including Item meana

Possibility of downloading software 5.7

Possibility of contacting other doctors 5.2
A home-page for my practice 4.6
Financial information 4.1
Information about charities 3.7
Travel information 3.3
Table III. Possibility of ordering medicine on-line 3.1
Service development
a
desired by customers Note: Scale ranged used: 1 (completely disagree) to 7 (completely agree)
and ``user interface'' (UI,  = 0.87). Only the items that appear in boldface in Exploring
Table II were used in order to create these constructs. consumer
The data were screened for missing values. As a result we had to delete four
cases, where a number of important variables were missing, leaving us with n
evaluations
= 48. The data set was also screened for multivariate outliers, by calculating
the Mahalanobis D2 measure, comparing the position of each value with the
center of all observations. We did not identify outliers. The dependent variables 371
were tested with respect to skewness, kurtosis and normality. No aberrations
were detected.
Confirmatory regression analyses (Hair et al., 1998) were then performed on
the data. The regression equations included two dependent variables: overall
satisfaction (SAT), and perceived value (VAL). The simplified equation can be
represented by:
Y ˆ 1 ‡ 2 UI ‡ 3 CORE ‡ 4 SUPL ‡ " …1†

Where Y is the respective dependent variable, 1 the Intercept and " represents
an error term.
We present the correlation matrix of our key constructs in Table IV. Because
the created independent variables were rather strongly correlated, Ridge
regressions were performed (Hoerl and Kennard, 1970). This led to very
acceptable results. Table V presents the results of the regressions found in
SPSS version 10 (SPSS, 1998). A regression of LOY on VAL and SAT was also
performed. The results are presented in Table VI.
Based on the results of our confirmatory regressions, we found support for
the first proposition. Each of the studied service dimensions was positively

VAL SAT LOY COR SUP

SAT 0.604** 1
LOY 0.390** 0.660** 1
COR 0.561** 0.863** 0.563** 1
SUP 0.591** 0.692** 0.522** 0.806** 1
UI 0.426** 0.609** 0.314* 0.647** 0.621**
Table IV.
Notes: * Correlation is significant at the 0.05 level (2-tailed); ** correlation is significant at Correlations between
the 0.01 level (2-tailed); core constructs

VAL SAT

Dependent F = 8.20*** R2 = 0.36 K = 0.4 F = 32.05*** R2 = 0.69 K = 0.5

Independent -value t-Value -value t-Value
Intercept 0.00 ±0.31 0.00 0.95 Table V.
COR 0.21 2.49 0.42 8.05 Regression coefficients
SUP 0.26 3.14 0.18 3.36 of multivariate Ridge
UI 0.09 1.05 0.16 2.78 regressions
IJSIM related to overall satisfaction with the portal site. The first part of the second
12,4 proposition, positing a positive effect of all elements on perceived value, was
not supported by the data, since satisfaction with the quality of the user
interface did not add directly to the perceived value of the portal. We do
suspect, however, that there might be interaction effects between the quality of
the user interface and perceived value of the offered services. The second part
372 of the second proposition was supported. Supplementary services indeed had
the strongest effect on perceived value. We proposed that there would be a
positive effect of overall satisfaction with the portal site on the intention to
continue using it. This was convincingly supported by the results of the
univariate regression, as presented in Table VI.

Conclusion and managerial implications

In this study, three service components, the core service, supplementary
services and the user interface, were identified. Satisfaction with each of the
elements influenced consumers' overall satisfaction with the portal.
Noteworthy was also the prevalence of the effect of supplementary services
over the core service on perceived value. Furthermore, the added value of
supplementary services could be an important driver of intentions to continue
using the portal. Satisfaction with the user interface had no direct effect on
perceived value. However, it remains possible that the effects of the other
service elements are mediated by the perceived quality of the user interface.
Clearly, as in other service categories, e-loyalty is strongly influenced by
customers' overall satisfaction with the portal.
Since the scope of the study was limited to identifying and ordering the
underlying factors that impact customer responses, no direct recommendations
can be provided on how to increase satisfaction, value or loyalty. Although the
results give an indication of the importance of supplementary services as a
value-adding component, more detailed research is needed on how
improvements can be achieved in the various dimensions of customer
satisfaction.

Limitations and suggestions for further research

There are a number of limitations to the present study. In the first place it only
focused on one particular type of e-service and no more than one portal site has
been studied. Furthermore, the low response rate and resulting small sample
have severely limited the generalizability of the conclusions that can be drawn
from the results. The findings can only with some caution be generalized to

B Standard error -value t-value

Intercept 2.92 0.497 5.87

Table VI. SAT 0.40 0.076 0.314 5.26
Ridge regression on VAL 0.10 0.056 0.105 1.76
``intentions to continue
using the portal'' (LOY) Notes: F = 11.96***; K = 0.9; R2 = 0.35
services that are very similar to the one studied. Therefore, more research is Exploring
urgently needed on the determinants of value perception, satisfaction and consumer
loyalty for a full range of e-services and other self-service technologies. evaluations
Although the user interface had only a small effect on portal satisfaction and
no direct effect on value perception in the current study, its importance should
not be underestimated at this stage. Because the user interface is needed to
access each of the offered service components, it is probable that the interface 373
quality will moderate the effect of the other components on customer
satisfaction. For example, if browsing is slow, the time to retrieve information
is disproportionately long, or if the format is inappropriate or confusing, this
will have a negative effect on satisfaction. We can therefore expect interaction
effects between the perceived quality of the user interface and all other
categories of service quality. Furthermore, the user interface may be a
dissatisfier that only has a negative effect on satisfaction, value and loyalty
when it fails to reach a certain standard. Some evidence from Meuter et al.
(2000) suggests that this may be the case. Therefore, further research is needed
on how the user interface influences customer evaluations in various situations.
We also believe that the measurement of our dependent variables could be
substantially improved by using multi-item scales. Since satisfaction, value
perception and loyalty in an online context have not yet been defined, further
research is needed to validate multi-item scales for e-satisfaction, e-value
perception and e-loyalty. When developing the scales it is important to
differentiate between different types of online services. For example, Zeithaml
(1988) proposes four definitions of value, two of which are based on price.
However, since many online services are offered for free, other sacrifices than
monetary should be considered when defining their value. Similarly, customer
loyalty to services that are free could include, for example, intentions to use the
service in future, extent of current usage and word-of-mouth behavior.
Furthermore, as there are important differences in evaluation mechanisms
between products on both extremes of the services/non-services continuum
(van Riel et al., 2001), other differences between dissimilar e-service categories
can also be expected. The relative influence of the different service elements on
customer evaluations may differ between dissimilar categories of online
services, e.g. between customer services, audiovisual content providers, e-
outlets of brick and mortar stores, and dot-com companies etc. Studies are also
needed on how the quality of a company's online supplementary or
complementary services affects customer loyalty to the brick and mortar
stores, or to physical goods.
Because one Web site may offer different services to different customer
segments, the evaluation and overall importance of different service elements
for satisfaction and loyalty can vary between target segments. Some segments
may also have special needs of online services that are not currently being met
by the company. Therefore, research with respect to the differences between,
for example, professional and private customers, or between customers of
varying profitability, seems appropriate.
IJSIM Affect was not explicitly included in the present study, but its precise role in
12,4 the generation of SST satisfaction and loyalty should be investigated.
Dabholkar (1996) found that enjoyment of using a SST was strongly related to
perceived service quality. However, Meuter et al. (2000) reported that customers
did not tend to experience spontaneous delight with SST, but would be highly
dissatisfied with technical failures. It is possible that once the novelty of using
374 the new service wears off, customers will only experience the contentment of
trouble free service. The lack of highly positive incidents found by Meuter et al.
(2000) could also indicate a lack of affective commitment to SSTs. Since
customer delight is considered important for future loyalty (Rust and Oliver,
2000), sources of positive affective reactions to e-services should be
investigated further. Moreover, since negative emotions may have a stronger
effect on customer responses than positive emotions (Liljander and Strandvik,
1997) e-service companies need to identify and minimize the causes of e-service
failures.
The fact that many online firms have found it difficult to obliterate inter-
personal interactions completely shows how difficult it is for customers to use
SSTs. For instance Meuter et al. (2000), found that 80 per cent of customer
complaints were made in person to the company, either by phone or by visiting
a service facility. When the service fails it has fallen outside the customer's
zone of tolerance (Zeithaml et al., 1993). So far, nothing is known about
customers' tolerance levels for e-services, or customers' propensity to complain
about online service failures. Although Zeithaml et al. (2000) concluded that
customers have no e-service expectations, customers have been found to
compare self-service technologies to competitors' services and to brick and
mortar stores (Meuter et al., 2000; Szymanski and Hise, 2000). Customers may,
for example, compare delivery times and prices and conclude that they would
rather pay a higher price in the brick and mortar store than wait three weeks
for delivery from an online retailer. The zone of tolerance may also be
considerably narrower for e-services than for traditional services, since
competitors' services are only ``one click away''. Although supplementary and
complementary services add to the perceived value of the service, customers
may have a higher tolerance for failure of supplementary services than failure
of the core service. Furthermore, technical failures that are attributed to the
customers' own computer or to the Internet connection may be better tolerated
than technical failures that are attributed to the service provider. Thus,
customers' zone of tolerance for e-services, their immediate reactions to service
failure and their consecutive behavior, such as reduced loyalty or switching
behavior, should be investigated further.
Considering the high costs of acquiring new customers and the apparently
high customer turnover of many online services, it is of the utmost importance
to study the determinants of customer commitment. However, studies should
not be confined to online retailers but be extended to include all types of online
services substituting or complementing both traditional services and physical
products. Also, entirely new e-services that provide unique benefits to
consumers should be investigated. As outlined in this paper, more conceptual
and empirical work is needed to gain a better understanding of consumer Exploring
evaluations of e-services. consumer
Notes evaluations
1. See: http://www.nttdocomo.com/i/index.html
2. In recent literature the concept of customer value has taken an important place. But the
value can be considered strongly correlated with satisfaction since customer-satisfaction is 375
the result of value perception.
3. At the time of our study the price was e100.00 per annum, about equal to the subscription
rate for well-known print medical journals.
4. http://www.medline.com

References
Anderson, E.W. and Fornell, C. (1999), ``The customer satisfaction index as a leading indicator'',
in Iacobucci, D. and Swartz, T.A. (Eds), Handbook of Services Marketing and Management,
Sage, New York, NY.
Anderson, E.W. and Mittal, V. (2000), ``Strengthening the satisfaction-profit chain'', Journal of
Service Research, Vol. 3 No. 2, pp. 107-20.
Anderson, J.C. and Narus, J.A. (1995), ``Capturing the value of supplementary services'', Harvard
Business Review, Vol. 73 No. 1, pp. 75-83.
Anderson, E.W. and Sullivan, M.W. (1993), ``The antecedents and consequences of customer
satisfaction for firms'', Marketing Science, Vol. 12 No. 2, pp. 125-43.
Berry, L.L. and Parasuraman, A. (1991), Marketing Services: Competing through Quality, The
Free Press, New York, NY.
Bitner, M.J., Brown, S.W. and Meuter, M.L. (2000), ``Technology infusion in service encounters'',
Journal of the Academy of Marketing Science, Vol. 28 No. 1, pp. 138-49.
Cadotte, E.R., Woodruff R.B. and Jenkins, R.L. (1987), ``Expectations and norms in models of
consumer satisfaction'', Journal of Marketing Research, Vol. 24, August, pp. 305-14.
Dabholkar, P.A. (1996), ``Consumer evaluations of new technology-based self-service options: an
investigation of alternative models'', International Journal of Research in Marketing,
Vol. 13 No. 1, pp. 29-51.
GroÈnroos, C. (1990), ``Relationship approach to marketing in service contexts'', Journal of
Business Research, Vol. 20 No. 1, pp. 3-11.
GroÈnroos, C., Heinonen, F., Isoniemi, K. and Lindholm, M. (2000), ``The Netoffer model: a case
example from the virtual marketspace'', Management Decision, Vol. 38 No. 4, pp. 243-52.
Gwinner, K.P., Gremler, D.D. and Bitner, M.J. (1998), ``Relational benefits in service industries: the
consumer's perspective'', Journal of the Academy of Marketing Science, Vol. 26 No. 2,
pp. 101-14.
Hair, J.F., Anderson, R.E., Tatham, R.L. and Black, W.C. (1998), Multivariate Data Analysis,
5th ed., Prentice Hall, Upper Saddle River, NJ.
Hoerl, A.E. and Kennard, R.W. (1970), ``Ridge regression: biased estimation for non-orthogonal
problems'', Technometrics, Vol. 12, pp. 55-67.
Iacobucci, D., Grayson, K.A. and Ostrom, A.L. (1994), ``The calculus of service quality and
customer satisfaction: theoretical and empirical differentiation and integration'', in Swartz,
T.A., Bowen, D.E. and Brown, S.W. (Eds), Advances in Services Marketing and
Management, Vol. 3, JAI Press, Greenwich, pp. 1-67.
Kaplan, R.S. and Norton, D.P. (1996), ``Using the balanced scorecard as a strategic management
system'', Harvard Business Review, January-February, pp. 75-85.
IJSIM Kaynama, S.A. and Black, C.I. (2000), ``A proposal to assess the service quality of online travel
agencies'', Journal of Professional Services Marketing, Vol. 21 No. 1, pp. 63-8.
12,4
Lehmann, D.R. (1997), ``A different game: setting the stage'', in Moreau, P. (Ed.), A Different
Game: Really New Products, Evolving Markets and Responsive Organizations, Marketing
Science Institute, Report Number 97-118.
Liljander, V. and Strandvik, T. (1997), ``Emotions in service satisfaction'', International Journal of
Service Industry Management, Vol. 8 No. 2, pp. 148-69.
376
Meuter, M.L., Ostrom, A.L., Roundtree, R.I. and Bitner, M.J. (2000), ``Self-service technologies:
understanding customer satisfaction with technology-based service encounters'', Journal
of Marketing, Vol. 64, July, pp. 50-64.
Oliver, R.L. (1999), ``Whence consumer loyalty'', Journal of Marketing, Vol. 63, Special issue,
pp. 33-44.
Parasuraman, A. (1996), ``Understanding and leveraging the role of customer service in external,
interactive and internal marketing'', presentation at AMA Conference: Frontiers in
Services, Nashville, TN.
Parasuraman, A. (1997), ``Augmented marketing'', Business Today, annual edition.
Parasuraman, A. and Grewal, D. (2000), ``The impact of technology on the quality-value-loyalty
chain: a research agenda'', Journal of the Academy of Marketing Science, Vol. 28 No. 1,
pp. 168-74.
Parasuraman, A., Zeithaml, V.A. and Berry, L.L. (1988), ``SERVQUAL: a multiple item scale for
measuring consumer perceptions of service quality'', Journal of Retailing, Vol. 64 No. 1,
pp. 12-40.
Petersen, A. (2001), ``Private matters: it seems that trust equals revenue, even online'', The Wall
Street Journal Europe, Special section on e-commerce, February 12, p. viii.
Ravald, A. and GroÈnroos, C. (1996), ``The value concept and relationship marketing'', European
Journal of Marketing, Vol. 30 No. 2, pp. 19-30.
Reichheld, F.F., Sasser, J. and Earl, W. (1990), ``Zero defections: quality comes to services'',
Harvard Business Review, September-October, pp. 105-11.
Reichheld, F.F. and Schefter, P. (2000), ``E-loyalty: your secret weapon on the Web'', Harvard
Business Review, July-August, pp. 105-13.
Russell, J.A. (1980), ``A circumplex model of affect'', Journal of Personality and Social Psychology,
Vol. 39 No. 6, pp. 1161-78.
Rust, R.T and Oliver, R.L. (2000), ``Should we delight the customer'', Journal of the Academy of
Marketing Science, Vol. 28 No. 1, pp. 86-94.
Rust, R.T., Zahorik, A.J. and Keiningham, T.L. (1995), ``Return on quality (ROQ): making service
quality financially accountable'', Journal of Marketing, Vol. 58, April, pp. 58-70.
SPSS (1998), SPSS Base 8.0 User's Guide, SPSS, Chicago IL.
Storbacka, K. (1994), ``The nature of customer relationship profitability: analysis of relationships
and customer bases in retail banking'', Publications of the Swedish School of Economics and
Business Administration, Vol. 55, Helsinki.
Storbacka, K., Strandvik, T. and GroÈnroos, C. (1994), ``Managing customer relationships for
profits: the dynamics of relationship quality'', International Journal of Service Industry
Management, Vol. 5 No. 5, pp. 21-38.
Szymanski, D.M. and Hise, R.T. (2000), ``E-satisfaction: an initial examination'', Journal of
Retailing, Vol. 76 No. 3, pp. 309-22.
Urban, G.L., Sultan, F. and Qualls, W.J. (2000), ``Placing trust at the center of your Internet
strategy'', Sloan Management Review, Vol. 42 No. 1, pp. 39-48.
van Riel, A.C.R., Lemmink, J. and Ouwersloot, H. (2001), ``Consumer evaluations of brand Exploring
extensions: differences between goods and services'', Journal of Service Research, Vol. 3
No. 3, pp. 220-31. consumer
Wingfield, N. and Rose, M. (2001), ``Amazon is split on fees to plug books'', The Wall Street evaluations
Journal Europe, 8 February.
Zeithaml, V.A. (1988), ``Consumer perceptions of price, quality and value: a means-end model and
synthesis of evidence'', Journal of Marketing, Vol. 52, July, pp. 2-22.
Zeithaml, V.A., Berry, L. and Parasuraman, A. (1993), ``The nature and determinants of customer
377
expectations of service'', Journal of the Academy of Marketing Science, Vol. 21 No. 1,
pp. 1-12.
Zeithaml, V.A., Parasuraman, A. and Malhotra, A. (2000), ``A conceptual framework for
understanding e-service quality: implications for future research and managerial practice'',
Working Paper, Marketing Science Institute, Report Number 00-115.

Appendix. Description of the seven elements of the portal site

Each day, between two and five newsflashes are posted in Current News. A selection of medical
news from journals, newspapers, radio and TV is provided, as well as an archive. A number of
``files'' are maintained about hot topics. Subscribers have full access, whereas others can only see
headlines and headings of what is in the archives.
``Discussion platform'' maintains between eight and ten different discussions at a time, plus
an archive. Via a mailing list discussions are announced. The ``platform'' is freely accessible for
subscribers and non-subscribers (except for the archives). Interaction does not occur in real-time,
but through posting reactions to an editor and is therefore similar to a newsgroup.
``Content'' provides a medical book showcase. It offers an overview of all published titles,
some of which are available online. The books can also be ordered online. Both subscribers and
non-subscribers can view ``Content''.
``Guide'' provides practical information, helping GPs to find their way in the online medical
world. It contains about 600 links to Web sites of interest and a database of medical institutions,
hospitals, etc. Additionally there is a calendar of events of interest for a GP (education,
congresses, etc.). Furthermore, there are medical files about illnesses. Most of this information is
restricted to non-subscribers.
``Practice'' is set up to help GPs organize their practice and to make them more aware of the
opportunities the Internet could offer them. Sub sections in ``Practice'' are an overview of
vacancies, and a supply and demand section for doctors willing to temporarily replace their
colleagues. Furthermore, advice is given about how GPs should organize their finances and
taxes, about how to implement ICT in the practice and on how to reduce stress and increase
working pleasure.
``Marketplace'' is aimed at providing pharmaceutical information. There is news about the
pharmaceutical industry and innovations. Links to pharmaceutical companies and medical
products are provided here. This part is only accessible for subscribers.
``Service'' provides GPs with information about subscription, and contact information. There
is also information on how to advertise on the portal site.