European Journal of Information Systems (2008) 17, 649–667
& 2008 Operational Research Society Ltd. All rights reserved 0960-085X/08
User satisfaction with business-to-employee
portals: conceptualization and scale
development
Dewi Rooslani Tojib1,
Ly-Fie Sugianto2 and
Sen Sendjaya3
1
Department of Marketing, Monash University,
Victoria, Australia; 2Department of Accounting &
Finance, Monash University, Victoria, Australia;
3
Department of Management, Monash
University, Victoria, Australia
Correspondence: Dewi Rooslani Tojib,
Department of Marketing, Monash
University, Caulfield Campus, Level 7,
Chisholm Tower, 26 Sir John Monash Drive,
Caulfield East 3145, Victoria, Australia.
Tel.: þ 61 3 9903 2686;
Fax: þ 61 3 9903 2900;
E-mail: Dewi.Tojib@buseco.monash.edu.au
Received: 10 May 2007
Revised: 25 June 2008
2nd Revision: 9 October 2008
Accepted: 13 October 2008
www.palgrave-journals.com/ejis/
Abstract
The purpose of this paper is to develop and validate the business-to-employee
portal user satisfaction (B2EPUS) measure. Five sequential stages of scale
development were undertaken to achieve this purpose: conceptual model
development, item generation, content validation, exploratory study, and
confirmatory study. Confirmatory factor analysis confirmed the five dimensions
of the B2EPUS, namely Usefulness, Confidentiality, Ease of Use, Convenience of
Access, and Portal Design as the dimensions of the construct. Competing model
analysis indicates that the second-order factor model is the better fitting for the
B2EPUS construct. The theoretical and practical implications of the study as
well as directions for future research are discussed in the concluding sections of
this paper.
European Journal of Information Systems (2008) 17, 649–667.
doi:10.1057/ejis.2008.55
Keywords: user satisfaction; business-to-employee portals; scale development
Introduction
Since the introduction of the first personalized consumer portal by Yahoo
in 1996 (Brosche, 2002), the global adoption of business-to-employee (B2E)
portals in organizations has been on the increase (Hamerman, 2004). A
B2E portal is the latest generation employee support system to be
developed to facilitate the access and availability of customized and
personalized information for employees (Davydov, 2001). It has two
distinct advantages over web-based Intranets (Ryley, 2001; Smith, 2004).
First, it utilizes user-recognition technology that enables the retrieval of
information personally tailored to individual employees. Second, it
provides integrated access to dynamic content from a variety of sources
in a variety of source formats.
The literature indicates that scant attention has been paid to the
successful implementation of B2E portals, although they have the
potential to deliver significant financial and non-financial benefits in
terms of reduced cost, increased competitive advantage, improved
corporate communication, and higher employee morale and loyalty (Dias,
2001; Benbya et al., 2004; Sugianto & Tojib, 2006).Given the high cost of
portal installation (Bannan, 2002) and significant failure risk of informa-
tion system (IS) projects (Yeo, 2002), the measurement of B2E portal
success is critical to our understanding of the value and efficacy of IS
investments (DeLone & McLean, 2003). The quick and easy solution of
using the return on investment (ROI) as an indicator of B2E portal success,
as recommended in the practitioner-oriented literature (Braue, 2003;
650 User satisfaction with B2E portals
White, 2003; Bisconti, 2004), often poses more logistical
problems in the implementation phase as the portal’s
intangible benefits are difficult to quantity (Zviran &
Erlich, 2003). The shortcomings of the ROI approach in
this context will no doubt restrict its application in the
organizations. Unfortunately, empirical studies on B2E
portals also fail to offer a solution to this conundrum
given their myopic focus on portal adoption (Kotorov &
Hsu, 2001; Corbitt et al., 2005; Rahim, 2007; Remus,
2007). Only very few studies have attempted to investi-
gate the post-implementation stage (Lai & Yang, 2008),
and none of them examined how B2E portal success can
be measured from the users’ perspectives. The study
reported in this paper seeks to address this gap in the
literature by developing and validating a measure of the
B2E portal user satisfaction (B2EPUS).
The focus on the user satisfaction dimension stems
from the fact that the benefits of any IS are largely
dependent on the users’ responses to the system (Au
et al., 2002). Users who perceive that their needs might
not be sufficiently fulfilled even by a technically
advanced system, are usually less motivated to use the
system, which may lead to dissatisfaction and eventually
a failure in the system adoption (Doll & Torkzadeh,
1988). Melone (1990) argued that user satisfaction is the
most widely used and commonly accepted surrogate
measure of system success. In keeping with the rigorous
scale development procedures recommended by the
authorities in the field (Churchill, 1979; Straub, 1989;
DeVellis, 2003; Lewis et al., 2005), this study used five
stages of scale development: conceptual model develop-
ment, item generation, content validation, exploratory
factor analysis (EFA), and confirmatory factor analysis
(CFA); each of which is explained in the following
sections.
User satisfaction studies
The concept of user satisfaction as a surrogate measure of
system success was first proposed by Cyert & March
(1963), then subsequently developed by Powers &
Dickson (1973), Gallagher (1974), Swanson (1974),
Jenkins & Ricketts (1979), Larcker & Lessig (1980), and
Neumann & Segev (1980). However, not until the early
1980s were empirical studies on user satisfaction devoted
to establishing a user satisfaction scale. In order to gain
more understanding of the user satisfaction concept, we
conducted a thorough review of past studies on user
satisfaction. We classified these studies into three broad
streams of research: user satisfaction with the overall ISs,
user satisfaction with a particular IT application, and user
satisfaction with web-based IS. A summary of this review
is provided in Table 1.
As shown in Table 1, the three most commonly used,
and hence validated, user satisfaction measures in the
field of IS were developed by Bailey & Pearson (1983), Ives
et al. (1983), and Doll & Torkzadeh (1988). However,
none of the measures was developed to specifically
measure user satisfaction with B2E portals. The first two
European Journal of Information Systems
Dewi Rooslani Tojib et al
measures were built to assess general user satisfaction in
the traditional data processing environment, which was
different from the B2E portal environment where end
users interact with the portals without the intervention
of IS staff ( Jones & Beatty, 2001). The third measure
developed by Doll & Torkzadeh (1988) was designed to
gauge user satisfaction with specific IT applications in
the end user computing (EUC) environment. While this
measure has been used to assess many different types of
IS applications (e.g., Seddon & Yip, 1992; McHanney &
Cronan, 1998; Zviran et al., 2005), research suggests that
significant modifications are necessary for this scale to
be used in the web-based environment (e.g., Otto et al.,
2000; Simmers & Anandarajan, 2001; Xiao & Dasgupta,
2002; Abdinnour-Helm et al., 2005). Straub (1989) has
long argued that a considerable modification to a
measure will negatively affect its validity and reliability.
In light of this comment, we believe that since B2E
portals operate in a web-based environment, it would not
be psychometrically sound to adopt this measure for B2E
portals. There is a need, therefore, to develop a new user
satisfaction measure for B2E portals in particular.
Similarly, Table 1 suggests that none of the studies
focused on measuring user satisfaction with B2E portals.
The closest are two studies on the B2E benefit system
(Huang et al., 2004) and consumer portals (Xiao &
Dasgupta, 2002). The former examined a system that
provides opportunities for employees to purchase pro-
ducts and services online and the latter investigated
consumer satisfaction with commercial portals. The
concepts and measures addressed in these studies are
not directly relevant to B2E portals whose functionalities
such as search and retrieval processes, work flow systems,
online self-service applications, and collaboration
tools are unique and distinct. This confirms the need
for a new measure to assess B2E portals user satisfaction
(B2EPUS). The following sections outline the B2EPUS
scale development process.
The B2EPUS conceptual model development
Recognizing the importance of the initial domain
identification stage in scale development to ensure
the measure sufficiently captured the specific domain
under examination (Hinkin, 1995), we meticulously
followed the established practice in the field (Sonquist
& Dunkelberg, 1977; Sekaran, 1992; DeVellis, 2003). First,
we scanned the literature for a definition that best fits our
study objective. While many researchers examined user
satisfaction from various perspectives, as seen in Table 1,
it was Doll & Torkzadeh (1988) who specifically argued
that user satisfaction with a system manifests itself in user
attitudes, and that an employee interacts with the system
directly without the intervention of other staff. We
therefore believe that Doll & Torkzadeh’s (1988) under-
standing of user satisfaction with IT applications in the
EUC environment provides the closest parallel to our B2E
portal study, and adapted it accordingly. Hence, the user
satisfaction with B2E portals in our study is defined as an
 User satisfaction with B2E portals
Table 1
Study
I. Studies measuring user satisfaction (US) with overall information systems (ISs)
Bailey & Pearson (1983)*
Ives et al. (1983)
Mahmood & Becker (1985)
Raymond (1987)
Montazemi (1988)
Foley & Newman (1988)
Bergeron & Berube (1988)
Anderson (1989)
Iivari & Karjalainen (1989)
Iivari & Ervasti (1994)
Khalil & Elkordy (1999)
Baroudi & Orlikowski (1988)
Yaverbaum & Nosek (1992)
Glorfeld & Cronan (1992)
Sengupta & Zviran (1997)
Ryker et al. (1997)
Ryker & Nath (1997)
Ang & Soh (1997)
Khalil & Elkordy (1999)
Amoako-Gyampah & White (1993)
Henry & Stone (1994)
Harrison & Rainer (1996)
Gelderman (1998)
Aladwani (2002)
Nath (1989)
II. Studies measuring user satisfaction (US) with particular types of IT applications
Doll & Torkzadeh (1988)*
Chin et al. (1988)*
Hartrum et al. (1989)*
Hiltz & Johnson (1990)*
Benard & Satir (1993)*
Etezadi-Amoli & Farhoomand
(1996)*
Kekre et al. (1995)*
Jones & Beatty (2001)*
Wu et al. (2002)*
Wang (2003)*
Calisir & Calisir (2004)*
Ong & Lai (2004)*
Glorfeld & Cronan (1992)
Dewi Rooslani Tojib et al 651
Summary of studies using user satisfaction measures
Applications type Remarks
Overall IS First reported scale to measure U.S. with overall IS
application.
Overall IS Validated the Bailey & Pearson (1983) scale.
Overall IS Adopted 22 items from the Bailey & Pearson (1983) scale.
Overall IS Adopted 20 items from the Bailey & Pearson (1983) scale.
Computer-based IS Adopted 35 items from the Bailey & Pearson (1983) scale.
Campus wide IS. Adopted 35 items from the Bailey & Pearson (1983) scale.
Microcomputer-based IS Adopted 34 items from the Bailey & Pearson (1983) scale.
Computer-based IS Adopted four items from the Bailey & Pearson (1983)
scale.
IS application Adopted 10 items from the Bailey & Pearson (1983) scale.
21 major IS applications Adopted 13 items from the Bailey & Pearson (1983) scale.
Computer-based IS Adopted the Bailey & Pearson (1983) scale and the Ives
et al. (1983) scale.
Overall IS Validated the Ives et al. (1983) scale.
Computer-based IS Adopted the Ives et al. (1983) scale.
Mainframe applications Adopted the Ives et al. (1983) scale.
Overall IS in hospital Found that the Ives et al. (1983) scale was not suitable for
use.
Computer-based IS Adopted the Ives et al. (1983) scale.
Computer-based IS Adopted the Ives et al. (1983) scale.
Overall IS Adopted the Ives et al. (1983) scale.
Computer-based IS A modified version of the Ives et al. (1983) scale was
adopted.
Overall MIS Adopted the Baronas & Louis (1988) scale.
Computer based medical IS Adopted 10 items from the Doll & Torkzadeh (1988)
scale.
Computer-based IS Adopted the Doll & Torkzadeh (1988) scale.
IS application Adopted the Doll & Torkzadeh (1988) scale.
Overall IS Adopted the Doll & Torkzadeh (1988) scale.
Overall IS Adopted the Nath (1988) scale.
250 different types of IT applications First reported scale to measure user satisfaction with
specific types of IT applications.
Software products A new scale was developed.
Interactive software tool Pre-set questionnaire items were derived from past
literature.
Computer mediated system Pre-set questionnaire items were derived from past
literature.
Executive IS A new user satisfaction index was developed.
Sofware application A new scale was developed.
Software products Pre-set questionnaire items were derived from past
literature.
Electronic data interchange Pre-set questionnaire items were derived from past
literature.
Enterprise resource planning (ERP) Pre-set questionnaire items were derived from past
literature.
Electronic learning systems Pre-set questionnaire items were derived from past
literature.
ERP Pre-set questionnaire items were derived from past
literature.
Knowledge management IS Pre-set questionnaire items were derived from past
literature.
Mainframe applications Adopted the Doll & Torkzadeh (1988) scale.
European Journal of Information Systems
652 User satisfaction with B2E portals Dewi Rooslani Tojib et al

Table 1 Continued

Study Applications type Remarks

Seddon & Yip (1992) General ledger accounting software The Doll & Torkzadeh (1988) scale was found to be a
more useful measure of satisfaction for this type of
application.
Palvia (1996) IT applications in small businesses Adopted the Doll & Torkzadeh (1988) scale and additional
items.
Simon et al. (1996) Computer systems Adopted the Doll & Torkzadeh (1988) scale.
McHaney & Cronan (1998) Decision support systems Adopted the Doll & Torkzadeh (1988) scale.
Downing (1999) Interactive voice response system Selected six items from the Doll & Torkzadeh (1988) scale.
Chen et al. (2000) Data warehouses Adopted the Doll & Torkzadeh (1988) scale and additional
items.
McHaney et al. (2002) Typical business software applications Adopted the Doll & Torkzadeh (1988) scale.
Mackie & Downing (2004) Interactive voice response system Selected six items from the Doll & Torkzadeh (1988) scale.
Zviran et al. (2005) ERP Validated the Doll & Torkzadeh (1988) scale.
Wan & Wah (1990) Office automation Adopted 33 items from the Bailey & Pearson (1983) scale.
Lawrence & Low (1993) Certain application system Made a slight modification to the Bailey & Pearson (1983)
scale to reflect Australian English usage.
Yeo et al. (2002) Microsoft access Adopted the Bailey & Pearson (1983) scale.
Bergeron et al. (1992) Executive IS Adopted items from the Bergeron et al. (1990) scale,
Bergeron (1986) scale, Swanson (1987) scale.
Gatian (1994) Financial accounting system Adopted three dimensions from Jenkins & Ricketts (1985).
Barki & Huff (1985) Decision support systems Adopted the Ives et al. (1983) scale.
Igbaria & Nachman (1990) Microcomputer or computer Modified the Ives et al. (1983) scale to make it relevant to
EUC environment.
Seddon & Yip (1992) General ledger accounting software Result revealed that the Doll & Torkzadeh (1988) scale
was a more useful measure for this type of application.
McKeen et al. (1994) Various computer application Selected 10 items from the Ives et al. (1983) scale.
Yoon et al. (1995) Expert systems Adopted the Raymond (1985) scale.
Abdul-Gader (1996) Computer-mediated communication Selected 13 items from the work of Hiltz & Johnson
(1990).

III. Studies measuring user satisfaction (US) with web-based IS applications

Ho & Wu (1999)* Online shopping A new scale was developed from an extensive review of
past literature.
Cho & Park (2001)* Internet shopping site A new scale was developed from an extensive review of
past literature.
Huang et al. (2004)* B2E benefit systems A new scale was developed from an extensive review of
past literature.
Muylle et al. (2004)* Website A new scale was developed from an extensive review of
past literature.
Otto et al. (2000) Websites Factor timeliness in the Doll & Torkzadeh (1988) scale was
replaced with factor website responsiveness.
Simmers & Anandarajan (2001) Internet Items measuring factor output in the Doll & Torkzadeh
(1988) scale were excluded.
Xiao & Dasgupta (2002) Consumer portals Found the Doll & Torkzadeh (1988) scale needs further
refinement for use in web-based environment.
Abdinnour-Helm et al. (2005) Websites Adopted the Doll & Torkzadeh (1988) scale and
suggested refining factor timeliness in future research.
Note the symbol * denotes original study.

affective attitude toward the B2E portal by an employee who
interacts with the portal directly. This definition served as a
guide concurrently with findings from extant literature
to identify the B2EPUS dimensions.
In order to adequately capture the relevant dimensions
of the B2EPUS, we followed the inductive method
recommended by Webster & Watson (2002) whereby
the literature guides the selection of dimensions which
were conceptually relevant to the B2EPUS domain.
Instead of imposing our knowledge of portal-related
dimensions a priori, we carefully reviewed extant litera-
ture (i.e., user satisfaction, B2E portal technology) and
let it generate the most relevant dimensions. We
also reviewed anecdotal evidence that appears in

European Journal of Information Systems

 User satisfaction with B2E portals
practitioner-oriented and trade magazines published by
portal solution vendors (e.g., Plumtree Software, SAP
Portals, Oracle, etc.), paying specific attention to the
portal characteristics featured therein. Following this
iterative process, we identified nine dimensions of
B2EPUS: Ease of Use, Layout, Convenience of Access,
Information Content, Communication, Timeliness, Efficiency,
Confidentiality, and Security. These nine dimensions can be
grouped into three categories which were typically used in
various models of end user satisfaction (Bailey & Pearson,
1983; Doll & Torkzadeh, 1988; Hoffman & Novak, 1997;
Yoo & Donthu, 2001; Au et al., 2002; Yang et al., 2005; Lin,
2007; Bai et al., 2008), namely information quality, system
quality, and system design quality. The three categories and
nine dimensions corresponded to our focus on, and
definition of, user satisfaction with B2E portals, in that
users of B2E portals will have direct, personal experience
with the quality of the information the portal produces,
the system performance, and the interface design quality.
This generative or inductive method ensured that the
domain of interest was sufficiently captured by the nine
dimensions and conceptually irrelevant dimensions (e.g.,
management support, technology competency, and user
involvement) were excluded from further investigation.
Information content
The inclusion of Information Content as a dimension of
user satisfaction is well established in the literature
(Bailey & Pearson, 1983; Doll & Torkzadeh, 1988; Chen
et al., 2000; Muylle et al., 2004; Ong & Lai, 2004; Yang
et al., 2005; Liang et al., 2007; Wang & Liao, 2007). It is a
critical dimension in our construct as, unlike other types
of employee support systems, B2E portals offer each
employee information which is individualized, current,
relevant, reliable, and accurate, thereby empowering
employees to function more efficiently and effectively.
These portal features ensure that end users are not
overloaded with information that they do not require
(Liaw & Huang, 2003).
Convenience of access
Huang et al. (2004) and Yang et al. (2005) argued that
the extent to which a system is accessible with no time
and geographic barriers is an important dimension of
user satisfaction. B2E portals are typically accessible
by employees at work, from home, and when mobile
through various means of access such as company
intranet, any computer with an internet connection,
and mobile devices such as web-enabled mobile phones
or PDAs. Professionals who often work remotely from the
office benefit from portals in various ways. They can keep
up with the organization’s news and connect with their
colleagues while at the same time performing their work-
related and personal tasks.
Ease of use
Previous user satisfaction studies (e.g., Doll & Torkzadeh,
1988; Harrison & Rainer, 1996; Cho & Park, 2001; Smith,
Dewi Rooslani Tojib et al 653
2001; Wang, 2003; Muylle et al., 2004; Yang et al., 2005;
Wang & Liao, 2007) unanimously retained Ease of Use as
a stable and consistent dimension of user satisfaction
construct (Tate et al., 2007). This dimension was also
included in the Technology Acceptance Model (Davis,
1989). Since the use of B2E portals within organizations is
often not mandatory, user-friendliness is crucial to attract
first-time users. Research suggests, for example, that
end users prefer organized, clearly structured, and easy-
to-navigate sites (Eighmey & McCord, 1998; Smith,
2001), all of which will enhance users’ overall satisfaction
(Huang et al., 2004).
Efficiency
Chen et al. (2000) and Loiacono et al. (2002) affirmed that
the extent to which users believe that the system can
fulfill their needs has an impact on user satisfaction. B2E
portals potentially enhance employees’ efficiency by
streamlining work processes, thereby increasing their
productivity and overall performance (Mawhinney &
Lederer, 1990; Vlahos & Ferratt, 1995).
Security
Previous research has confirmed that Security is an
important aspect of user satisfaction (Bailey & Pearson,
1983; Yoo & Donthu, 2001; Loiacono et al., 2002; Huang
et al., 2004). However, their studies have used the term
Security to represent misappropriation or unauthorized
use of data. In our study, this term taps into a more
encompassing aspect: the ability of the portal to provide
secure access to all applications and facilities provided.
We believe that providing a secure access is critical
considering the nature of information and resources
provided by the portal (e.g., Employee Self-Service)
(Benbya et al., 2004).
Communication
The system’s ability to assist users to interact with others
does influence user satisfaction (Wang, 2003; Ong & Lai,
2004; Yang et al., 2005). This Communication dimension
is also relevant in our study since B2E portals act as a
medium of communication among employees and
between employees and their organizations. Specifically,
they facilitate collaboration among employees through
the use of email, chat, or discussion threads. These
facilities support mobile employees who have less
frequent face-to-face interactions with their colleagues,
thereby helping them to maintain a sense of attachment
to their organization.
Confidentiality
Privacy/confidentiality issues have been a serious concern
in the online environment as Internet users’ personal
information is often retrieved or submitted without
warrant or authorization (Yoo & Donthu, 2001; Loiacono
et al., 2002; Huang et al., 2004; Yang et al., 2005). B2E
portals are not immune to these risks. The confidentiality
and sensitivity of the submitted information in the ESS
European Journal of Information Systems
654 User satisfaction with B2E portals
applications, for example, requires the portal to be
accessible only by authorized individuals.
Timeliness
This dimension was derived from Bailey & Pearson
(1983), Yoo & Donthu (2001), and Muylle et al. (2004)
who confirmed that response time or processing speed is
an indicator of user satisfaction with IT applications.
Similarly, employees working in a competitive work
environment using B2E portals often require on-demand
information reasonably quickly or the information
becomes obsolete. Lengthy response time has been
found to reduce user satisfaction and results in poor
productivity (Hoxmeier & DiCesare, 2000). In our study,
Timeliness is defined as the ability of the portal to deliver
requested information in a reasonable response time.
Layout
The final dimension of the B2EPUS is Layout, defined as
the design of the interface and display of the informa-
tion. Research suggests that interface design is an
important gauge of user satisfaction (Chin et al., 1988;
Cho & Park, 2001; Loiacono et al., 2002; Zazelenchuk &
Boling, 2003; Huang et al., 2004; De Wulf et al., 2006;
Zviran et al., 2006; Lindgaard, 2007). Muylle et al. (2004)
stated that effective website design is vital for the success
of an e-commerce initiative. Given the role that a portal
plays as the interface between the employees and the
organization, effective portal design must be seen as a
prerequisite for successfully implementing B2E initia-
tives. Consequently, an attractive and aesthetic portal
design is proposed as a dimension of the B2EPUS
construct.
To conclude, there are nine proposed dimensions of the
B2EPUS construct. While the terminology of the dimen-
sions may be generic in the existing IS literature, the
constructs measured in each dimension were specifically
relevant for the B2E portal environment. In fact, some
constructs were uniquely developed for B2E portals, such
as Convenience of Access (the accessibility of B2E portals
through different types of media), Communication
(the ability of B2E portals to mediate information sharing
or collaboration), and Efficiency (the extent to which
B2E portals are useful to employees in terms of increasing
productivity). Definitions of each dimension are pro-
vided in Table 2.
Item generation
Following the deductive approach of item generation
proposed by Schwab (1980), the nine dimensions of the
B2EPUS conceptual model were used as a basis to
generate scale items. Referring to the literature, multiple
items were generated for each dimension to ensure
reliability (Allen & Yen, 1979) and internal consistency
(Nunnally, 1978). Whenever appropriate, items from
existing scales which had been empirically tested were
considered and adapted for the newly developed measure
(Chang, 2001). The process established a pool of 47 items,
European Journal of Information Systems
Dewi Rooslani Tojib et al
Table 2 B2EPUS construct – dimensions and definitions
Dimensions Definition
Information quality
Information The relevance, accuracy, currency, and reliabil-
content ity of information presented to each employee
based on his/her role in the organization.
System quality
Convenience of The accessibility level of the portal anytime
access and anywhere through intranet, Internet, or
even mobile devices.
Ease of use The extent to which the portal is perceived to
be user friendly (i.e., learnability, ease of
navigation, training issue, feeling of being in
control).
Efficiency The ability of the portal to assist employees to
perform their tasks better and faster, thereby
improving their productivity and streamlining
work processes.
Security The ability of the portal to provide a secure
access to all applications and facilities provided.
Communication The extent to which the portal can mediate
interaction (i.e., information sharing and
collaboration) between employees and the
business as well as employees and other
colleagues/partners.
Confidentiality The ability of the portal to provide a sense of
assurance that any personal information re-
trieved or submitted from and through the
portal will not be misused by authorized people.
Timeliness The ability of the portal to deliver requested
information in a reasonable response time.
System design quality
Layout The design of the interface and display of the
information.
which were subsequently refined and arranged in a
sequence suitable for the purpose of content validation.
Content validation process
The 47 items were subsequently tested for content
validity. In order to ensure the adequacy of the dimen-
sions and items representing the B2EPUS construct, an
international panel of content experts was asked to
provide their judgment. The panel members were
selected on the basis of the following criteria: (1) hold a
Ph.D. degree or currently pursue a Ph.D. study; (2)
actively research in the domain of interest or have
professional experience in the field of portal develop-
ment (Grant & Kinney, 1992; Grant & Davis, 1997).
First round of content validation
Six content experts were willing to participate in our
study. First, they were asked to rate the importance of
each of the nine dimensions using a 5-point rating scale
(with 1 being ‘Extremely Unimportant’ and 5 ‘Extremely
 User satisfaction with B2E portals Dewi Rooslani Tojib et al 655

Important’). The mean values for each of the nine Table 3 Sample characteristics for exploratory and
dimensions were greater than 3, ranging from 3.67 to confirmatory analyses
4.83, which confirmed that all dimensions were essential
Exploratory analysis Confirmatory analysis
for measuring the B2EPUS construct. Second, they were
(145 cases) (157 cases)
asked to match each item with its nominated dimension
and rank the relevance of the item to the assigned Percentage Percentage
dimension, using a 5-point rating scale (with 1 being ‘not
Gender
at all’ and 5 ‘very well’). We calculated the weighted Male 34.5 39.5
ratios for each item using the cutoff value of less than or Female 65.5 60.5
equal to 0.50 (Fehring, 1987). The process yielded 15
items with average weighted ratios less than 0.50, which Age (years)
were subsequently excluded from further analysis. 18–25 9.0 6.4
The expert qualitative feedback indicated that the list 26–35 29.7 32.5
of dimensions was comprehensive and no additional 36–45 29.0 27.4
dimensions were required. They suggested combining 46 and above 32.4 33.8
dimension Security and dimension Confidentiality as
they considered these two dimensions to be similar. Job categories
Similarly, the experts suggested combining Efficiency Academic staff 24.8 25.5
and Usefulness as they capture a similar concept in the Researcher 6.9 8.3
General staff 68.3 66.2
Technology Acceptance Model (Davis, 1989). To main-
tain the rigor of the scale development process, we
conducted a second round of the content validation
process.
university global email was sent to university staff,
Second round of content validation inviting them to participate in the online survey. Three
In the second round, three panel members agreed to hundred and two useable responses were generated. We
participate. Lynn (1986) argued that three qualified then randomly split them into two data sets: 145 cases for
content experts are sufficient for the purpose of the the exploratory analysis and the remaining 157 cases
content validation. They were asked to rate the relevance for the confirmatory analysis. The sample size for both
of each item to the nominated dimension using a 4-point data sets was considered adequate, following the para-
rating scale ranging from 1 being ‘irrelevant’ and 4 ‘very meter estimate ratio of 5 as suggested by Bentler (1995).
relevant.’ The 4-point rating scale was utilized in this The sample characteristics are outlined in Table 3.
round since the feedback was quantified by calculating
the Content Validation Index (CVI) for each individual
item and the entire scale, following Lynn’s (1986) Exploratory analysis
recommendation. Each item had to be rated as 3 or 4 in The 145 cases in the first data set were subjected to
order to be considered content valid. The CVI value for principal component analysis. The Promax rotation
the entire scale was 91.89 %, which clearly exceeded the method was used as the data demonstrated high correla-
expected minimum CVI of 0.80 for a new scale (Davis, tions among the extracted factors. Three commonly used
1992). Corroborating the first round, the panel experts decision rules were applied to identify the number of
confirmed that the proposed dimensions were adequate factors underlying the B2EPUS construct (Comrey & Lee,
to represent the B2EPUS construct and no additional 1992; Hair et al., 1998; Tabachnick & Fidell, 2001). Items
dimensions were required. An important suggestion with less than 0.45 loading and which cross-loaded on
was to combine Timeliness with Information Content, two or more factors at 0.45 or higher were excluded. An
and Layout with Ease of Use. All in all, a total of 28 items Eigenvalue of 1 was used as the cutoff value for
were retained under Usefulness, Communication, extraction. The iterative sequence of factor analysis and
Convenience of Access, Ease of Use, Information Con- item deletion was repeated, resulting in a final scale of 22
tent, and Security dimensions. items belonging to five distinct factors associated with
the B2EPUS construct. Following Tabachnick & Fidell
Field study data collection: sample and procedure (2001), we analyzed other possible factor solutions (e.g.,
The field study was conducted in Australian universities, three, four, and six) but confirmed that the five-factor
since B2E portals have been widely used in the higher solution is the most fitting solution, accounting for
education sector (Deans & Allmen, 2002). All 38 73.63% of the variance. Table 4 summarizes the factor
Australian universities were contacted, 10 of which loadings for the condensed 22 item scale. The significant
indicated that they had built and implemented B2E loading of all the items on the single factor indicates
portals. In response to our email invitations, five unidimensionality. The fact that no item had multiple
universities agreed to participate in the study. Using cross loading was found to support the preliminary
a non-probability convenience sampling method, a discriminant validity of the scale. Furthermore, the

European Journal of Information Systems

656 User satisfaction with B2E portals Dewi Rooslani Tojib et al

Table 4 Factor loading of the 22 final items

Factor

1 2 3 4 5

Usefulness
Sharing or exchanging information with team member colleagues 0.94
Facilitating collaboration with all colleagues 0.92
Discussing work issues with work colleagues 0.91
Performing more work electronically 0.79
Streamlining work processes 0.77
Sharing information within the whole organization 0.68

Confidentiality
Certainty of appropriate use for submitted information 0.87
Confidentiality of submitted information 0.84
Trustworthy retrieved information 0.79
Dependable retrieved information 0.78
Security of portal 0.75
Provided reliable information 0.64

Ease of use
Self-explanatory use 0.93
Navigatability 0.79
Learnability 0.77
Feeling of being in control 0.63
Convenience of access
Accessible from home 0.89
Gain access easily 0.72
Accessible 24/7 0.69

Portal design
Aesthetic design 0.95
Attractive design 0.95
Availability of help functions and useful button and links 0.52
Corrected Cronbach’s a 0.92 0.89 0.84 0.81 0.89
Eigenvalue 8.92 3.07 2.01 1.16 1.03
Cumulative variance explained (%) 40.54 54.50 63.65 68.94 73.63

reliability coefficients for all five factors were above 0.80,
indicating acceptable reliability (Nunnally, 1978).
The results of the factor analysis revealed a different
pattern from that of the post-content validation model.
Items initially generated to measure dimension Commu-
nication and Usefulness were formed together as Factor
One (Usefulness). While conceptually different, the fact
that these two dimensions emerged together is reason-
able as respondents may perceive the B2E portal as a
useful tool that enables them to perform their work
electronically and to communicate with their colleagues.
Items belonging to dimension Security were amalga-
mated with items associated with dimension Information
Content in Factor Two (Confidentiality). Respondents
may perceive that, as the portal provides a sense of
assurance that any submitted information to the portal
will remain confidential, the portal will present reliable,
dependable, trustworthy information. It is thus logical
to see the integration of these two dimensions as the
co-relationship that may exist between these two dimen-
sions. Although content experts suggested combining
dimension Layout and Ease of Use, our exploratory
analysis demonstrated that respondents were able to
clearly differentiate one-item group as measuring user
friendliness of the portal (Factor Three) and the other
group as measuring the design of the portal (Factor Five).
This finding is consistent with Muylle et al. (2004) who
found that these two dimensions were indeed two
separate constructs. Hence, we named Factor Three as
Ease of Use and Factor Time as Portal Design. All items
measuring Convenience of Access were loaded into
Factor Four, thereby retaining the original name.
Confirmatory analysis
CFA was performed on the remaining 157 cases in the
data set using the maximum likelihood method. This
method detects the unidimensionality of each factor,
which indicates the presence of a single trait or construct

European Journal of Information Systems

 User satisfaction with B2E portals Dewi Rooslani Tojib et al 657

underlying a set of measures (Anderson & Gerbing, 1988).

Final model
(18 items)

Po0.05
258.22

2.07
0.85
0.79
0.17
0.08
0.92
0.88
0.93
350.52
537.11
Following Byrne (2001), we developed a measurement

125
model which included the identified five factors as first-
order factors. The five factors were correlated, each item
having a non-zero loading on its designated factors and
zero loadings on other factors, and the measurement
error terms associated with the item were uncorrelated.
The hypothesized five-factor model turned out to be a

Third revised model

poor model. As shown in Table 5, while the w2/d.f. value
suggested that the initial model was a reasonably

(19 items)

Po0.05
315.67

2.22
0.82
0.77
0.19
0.09
0.90
0.86
0.92
411.67
606.37
good-fitting model, the other fit indices were all below

142
the recommended threshold. Therefore, there was a need
to re-specify the hypothesized model to detect ill-fitting
parameters and achieve a clearer factor structure, in
keeping with Yoo & Donthu’s (2001) recommendation.
Table 5 shows several models that we re-specified,
taking into consideration the empirical and theoretical
rationale in each step including standardized factor

Model fit test results of initial and revised models

Second revised model
loadings (Raghunathan et al., 1999) and Modification
index (Byrne, 2001). We followed Joreskog & Sorbom’s

(20 items)

Po0.05
364.50

2.28
0.81
0.75
0.19
0.09
0.89
0.85
0.91
464.49
667.31
(1988) recommendation to drop one item at a time

160
in order to avoid over modification. At the end of the
re-specification process, four items were deleted. As
shown in Table 5, the final 18-item model had acceptable
fit indices (w2/d.f. ¼ 2.07, CFI ¼ 0.93, TLI ¼ 0.92, RMSEA ¼
0.83, GFI ¼ 0.85, AGFI ¼ 0.79, RMR ¼ 0.17, Normed Fit
Index (NFI) ¼ 0.88). The AIC value for the final model is
the smallest, suggesting that the final model is the most
parsimonious model. The 18 items are provided in
First revised model

Appendix A.
(21 items)

Po0.05
460.25

564.25
775.18
2.57
0.78
0.72
0.21
0.10
0.86
0.88
0.88
179

Investigation of higher-order factor model

Competing model analyses were performed to corrobo-
rate the correlated five-factor model. Following Doll et al.
(1994), four plausible alternative models were specified
(see Figure 1; only representative items are depicted).
Model 1 is a one-factor, 18-item model, which was
Table 5

plausible in light of past studies that assumed that user

Initial model
(22 items)

Po0.05
572.86

680.86
899.90
2.88
0.75
0.69
0.21
0.11
0.83
0.79
0.85

satisfaction is a single first-order construct (Doll et al.,

199

1994). In Model 2, the 18 items were loaded onto five-

correlated factors. On the basis of the result of EFA, this
was the proposed model in the study. In Model 3, the 18
items were loaded onto five uncorrelated factors. Model 4
comprised a second-order factor onto which the five first-
order factors were loaded. This model tested the extent to
which the correlations among the five first-order factors
Smaller is better

Smaller is better
Smaller is better
1ow2/d.f. o3

were accentuated by the second-order factor.

Table 6 compares the fit indices of the four alternative
Threshold

P40.05

models. A null model, which suggests that no latent

40.90
40.80

40.90
40.90
40.90
o0.10
o0.08

factors underlie the observed items and that the correla-

tions between items are zero in the population (Muylle
et al., 2004), was also included to establish zero point for
the NFI (Doll et al., 1994). The null model turned out to
provide a poor fit with the data, as expected. Model 1 and
Model 3 demonstrated a substantial improvement over
P-value

RMSEA
w2/d.f.

CAIC

the null model; however, neither of them indicated a

AGFI
RMR

AIC
NFI
GFI
d.f.

CFI
TLI
w2

reasonable fit with the empirical data.

European Journal of Information Systems

658 User satisfaction with B2E portals Dewi Rooslani Tojib et al

MODEL 1 MODEL 2

Item 1 Item 1
UF
Item 2 Item 2
..
Item 3 B2EPUS .
Item 5
Item 4 CF
Item 6
Item 5 ..
.. .
. Item 10
EU
Item 11
..
.
Item 13
CA
Item 14
..
.
Item 16
DS
Item 17
..
.

MODEL 3 MODEL 4

Item 1 Item 1
UF UF
Item 2 Item 2
.. ..
. .
Item 5 Item 5
CF CF
Item 6 Item 6
.. ..
. .
Item 10 Item 10
EU EU B2EPUS
Item 11 Item 11
.. ..
. .
Item 13 Item 13
CA CA
Item 14 Item 14
.. ..
. .
Item 16 Item 16
DS DS
Item 17 Item 17
.. ..
. .

Figure 1 Alternative models for the B2EPUS scale.

Table 6 Model fit test results of alternative models

Threshold Null model Model 1 Model 2 Model 3 Model 4

w2 Smaller is better 2170.94 1082.57 258.52 520.91 292.93

d.f. 153 136 125 135 130
P value P40.05 Po0.05 Po0.05 Po0.05 Po0.05 Po0.05
w2/d.f. 1ow2/d.f. o3 14.19 7.96 2.07 3.86 2.25
GFI 40.90 0.25 0.51 0.85 0.70 0.82
AGFI 40.80 0.16 0.39 0.79 0.62 0.76
RMR o0.10 0.93 0.29 0.17 0.71 0.19
RMSEA o0.08 0.29 0.21 0.08 0.14 0.09
TLI 40.90 — 0.47 0.92 0.78 0.91
NFI 40.90 — 0.50 0.88 0.76 0.87
CFI 40.90 — 0.53 0.93 0.81 0.92
AIC Smaller is better 2206.94 1152.58 350.52 592.90 374.93
CAIC Smaller is better 2279.95 1294.55 537.11 738.93 541.23

Model 2 provided substantial improvement over Model marginal fit. As for Model 4, the majority of the fit indices
1 and demonstrated reasonable fit, as indicated by the values were within, or close to, the ideal thresholds. Thus,
CFI and TLI values, which were within the recommended Model 2 and Model 4 were both adequate for represent-
thresholds. The other fit indices values also showed ing the underlying structure of the B2EPUS scale.

European Journal of Information Systems

 User satisfaction with B2E portals Dewi Rooslani Tojib et al 659

It should be noted that the second-order factor merely Assessment of reliability

explains the covariation among first-order factors in a Following Hair et al. (1998), the reliability of the B2EPUS
more economical way (Stewart & Segars, 2002). Hence, scale was assessed through the investigation of the
even though the higher-order model is able to explain Cronbach’s a, composite reliability, and the average
the factor covariations, the goodness-of-fit of the higher- variance extracted (AVE). Table 7 indicates that all factors
order model will never surpass that of the corresponding met the recommended threshold value, demonstrating
first-order model (Doll et al., 1994). This was shown in strong reliability level, except for dimension Conveni-
the fit indices of Model 2, which were superior to those of ence of Access, which was only moderate.
Model 4. As such, Model 2 provided a target or optimum
fit for Model 1 and it acted as the target model. However, Assessment of convergent and discriminant
the target coefficient (T) of 0.883 (the ratio of the w2 of validity
Model 2 to the w2 of Model 4) suggested the existence of Convergent validity refers to the extent to which the
the second-order factor model. It indicated that 88.3% of items appear to be indicators of a single underlying
the variation in Model 2 was explained by the second-
order factor. Hence, the factor covariances in Model 2 can Table 7 Assessment of internal consistency
be represented in a more economical manner with the Dimensions No. of Cronbach’s a Composite AVE
existence of the second-order factor. Furthermore, as items (40.70) reliability (40.50)
described in Figure 2, the paths from the second-order (40.70)
factor to the first-order factors were all significant. These
results suggested that the second-order factor model is Usefulness 4 0.93 0.93 0.77
the best fitting model of the B2EPUS construct. In other Confidentiality 5 0.88 0.89 0.61
Ease of use 3 0.81 0.82 0.60
words, the second-order factor model (Model 4) is a more
Convenience of 3 0.70 0.70 0.45
accurate representation of the B2EPUS construct and
access
should be accepted in preference to Model 2 (Segars & Portal design 3 0.92 0.92 0.80
Grover, 1998).

Item 1 0.90

Item 2 0.93
0.92 Usefulness
Item 3
0.74
Item 4 0.63

Item 5 0.61

Item 6 0.94

0.96
Item 7 Confidentiality
0.61 0.60
Item 8 0.73

Item 9
B2EPUS
Item 10 0.87 0.86
0.60
Item 11 Ease of Use
0.83
Item 12
0.66
Item 13 0.50
0.85 Convenience
Item 14 Of Access
0.61
0.81
Item 15

Item 16 0.96
0.83 Portal
Item 17 Design
0.88
Item 18

Figure 2 Second-order factor model of the B2EPUS construct.

European Journal of Information Systems

660 User satisfaction with B2E portals Dewi Rooslani Tojib et al

construct (Salisbury et al., 2002) and is assessed by the and above the threshold value of 0.40 (Gefen et al., 2000)
presence of significant factor loadings (Segars, 1997) and and all factors constituting the B2EPUS scale had
NFI40.90 (Mak & Sockel, 2001). As shown in Table 8, all NFI40.90. These results showed that each factor in the
standardized factor loadings were positive, significant, scale had convergent validity.
Discriminant validity is defined as ‘the degree to which
measures of distinct concepts differ’ (Bagozzi & Phillips,
Table 8 Assessment of convergent validity 1982, p. 469). Discriminant validity between the five
factors of the B2EPUS scale was analyzed by examining
Factor Item Factor loading NFI
factor correlations (Kline, 2001) and performing w2 dif-
Usefulness 0.98 ference tests (Chang & King, 2005). As shown in Table 9,
Item 1 0.90 all factor correlations were below 0.80, confirming the
Item 2 0.93 discriminant validity of the scale (Bhattacherjee, 2002).
Item 3 0.92 Using the w2 difference tests method, the w2 statistic of
Item 4 0.73 the unconstrained CFA model (where all constructs are
Confidentiality 0.98 freely correlated) was compared with that of constrained
Item 5 0.61
models where the correlation between pairs of factors was
Item 6 0.94
set to one. The significant difference in w2 between the
Item 7 0.96
Item 8 0.61
unconstrained and constrained models (w2 diff (1) ¼ 3.84,
Item 9 0.72 Po0.05) established the discriminant validity between
Ease of use 0.92 the constrained pair of constructs.
Item 10 0.88 Tests of all possible pairs of factors were conducted and
Item 11 0.60 the results are presented in Table 10. The results revealed
Item 12 0.82 that all combinations had significant w2 differences,
Convenience of access 0.93 except for two models that had w2 differences far below
Item 13 0.50 the threshold value (i.e., 3.84). They were the models in
Item 14 0.86 which factor Usefulness–Ease of Use and factor Useful-
Item 15 0.61 ness–Portal Design were constrained. These results sug-
Portal design 0.94 gest that factors Usefulness–Ease of Use or Usefulness–
Item 16 0.96 Portal Design should be combined as they seem to
Item 17 0.83
measure a similar construct. However, as shown in
Item 18 0.88
Table 4, these three factors were separately extracted in

Table 9 Assessment of discriminant validity using factor correlations

Usefulness Confidentiality Ease of use Convenience of access Portal design

Usefulness 1
Confidentiality 0.34 1
Ease of use 0.51 0.48 1
Convenience of access 0.29 0.66 0.59 1
Portal design 0.60 0.42 0.72 0.45 1

Table 10 Assessment of discriminant validity: v2 difference test

Variables constrained w2 d.f. w2 difference* w2/d.f. GFI CFI TLI RMSEA

None 258.52 125 — 2.07 0.85 0.93 0.92 0.08

UF, CF 280.28 126 21.76 2.22 0.83 0.92 0.91 0.09
UF, EU 258.79 126 0.27 2.05 0.85 0.93 0.92 0.08
UF, CA 289.12 126 30.60 2.30 0.83 0.92 0.90 0.09
UF, DS 259.40 126 0.88 2.06 0.85 0.93 0.92 0.08
CF, EU 267.76 126 9.23 2.13 0.84 0.93 0.92 0.09
CF, CA 287.73 126 29.21 2.28 0.83 0.92 0.90 0.09
CF, DS 267.25 126 8.73 2.12 0.84 0.93 0.92 0.09
EU, CA 268.44 126 9.92 2.13 0.84 0.93 0.91 0.09
EU, DS 267.79 126 9.27 2.13 0.84 0.93 0.92 0.09
CA, DS 270.460 126 11.94 2.15 0.83 0.93 0.91 0.09
*Po 0.05, UF, Usefulness; CF, Confidentiality; EU, Ease of use; DS, Portal Design; CA, Convenience of access.

European Journal of Information Systems

 User satisfaction with B2E portals
the exploratory study, and factor correlations between
factor Usefulness–Ease of Use and factor Usefulness–
Portal Design (see Table 10) were below the threshold
value of 0.80. Hence, the five factors constituting the
B2EPUS scale clearly discriminated among each other.
Therefore, the unconstrained CFA model was the best
model for describing the B2EPUS construct (w2/d.f. ¼ 2.07,
CFI ¼ 0.93, TLI ¼ 0.92, RMSEA ¼ 0.08, GFI ¼ 0.85).
Assessment of criterion-related validity
The criterion-related validity refers to the extent to which
a scale is related to pre-specified criteria (Yang et al.,
2005). It was assessed by the correlation between the total
scores on the scale (sum for 18 items) and the measures of
the valid criterion (sum for two global items). In this
study, the two global items were adapted from Doll &
Torkzadeh (1988): ‘In general, I am satisfied with the staff
portal’ and ‘In general, the staff portal is successful.’ As
expected, a positive relationship was revealed (r ¼ 0.71,
n ¼ 157, Po0.01), showing that the scale was capable of
measuring the B2EPUS construct.
Assessment of nomological validity
Nomological validity refers to the extent to which
predictions based on the construct being measured are
confirmed within a wider theoretical context or network
of constructs (Bagozzi, 1981). To assess the nomological
validity of the B2EPUS scale, this study assumed that
there was a positive relationship between the user
satisfaction construct and system use (e.g., Baroudi
et al., 1986; Fraser & Salter, 1995; Igbaria & Tan, 1997).
Hence, it was expected that a positive relationship
between user satisfaction (i.e., the B2EPUS construct)
and system use (i.e., the B2E portal use construct) did
exist.
On the basis of the works of Teo et al. (1997) and Doll &
Torkzadeh (1998), we developed six new items to assess
the generic use of B2E portals, namely to check and reply
to emails, to communicate with colleagues, to obtain
organizational news, to search for information, to
perform work-related and personal-related tasks (see
Appendix B). A 7-point Likert-type scale (1 ¼ strongly
disagree, 2 ¼ moderately disagree, 3 ¼ slightly disagree,
4 ¼ neutral, 5 ¼ slightly agree, 6 ¼ moderately agree,
7 ¼ strongly agree) was used to measure these items. The
coefficient a of 0.85 clearly suggested that these six items
were highly reliable. As expected, there was a strong
positive correlation between the two constructs (r ¼ 0.63,
n ¼ 157, Po0.01). This finding corroborated that of prior
research, thereby confirming the nomological validity of
the B2EPUS scale.
Discussions
While IS researchers have been focusing on a qualitative
approach to content validity assessment (Straub et al.,
2004), this study extends IS research by demonstrating
how the quantitative approach can be applied in content
validating the B2EPUS scale. Compared with previous
Dewi Rooslani Tojib et al 661
studies (Bhattacherjee, 2002; Templeton et al., 2002),
which reported only one round of the content validation
process, this study performed its content validity assess-
ment in two rounds to ensure that there was sufficient
rigor in the measure. This step addressed the concern
posed by Hinkin (1995) who argued that many scale
development practices include measures that lack con-
tent validity in the item development stage. We also
believed that a second round was necessary as some
dimensions and items were revised and several items were
added to the scale in the first round.
The finding of the exploratory study suggested that the
rotated five-factor solution met the following three
criteria: simplicity (Sethi & King, 1991), interpretability
(Lederer & Sethi, 1992), and a reasonable percentage of
variance explained (Bernstein, 1988). The CFA under-
taken in this study confirmed the stability of these five
factors. In addition, the competing analysis also revealed
the existence of the second-order factor model in the
B2EPUS construct. This finding provides evidence that
the B2EPUS is a single second-order construct, in
alignment with previous user satisfaction studies (e.g.,
Doll et al., 1994; Huang et al., 2004; Muylle et al., 2004).
Our findings support the view that user satisfaction is not
a single construct.
This study also concluded that the two most frequently
applied user satisfaction scales, the user information
satisfaction (UIS) scale proposed by Ives et al. (1983) and
end user computing satisfaction (EUCS) scale developed
by Doll & Torkzadeh (1988), were not appropriate for
measuring user satisfaction with B2E portals. A compar-
ison of the underlying dimensions among the UIS, EUCS,
and B2EPUS constructs, as shown in Figure 3, reveals that
there are similar dimensions among the three constructs:
(1) Information Content in the UIS and EUCS constructs
appears to be a part of Confidentiality in the B2EPUS
construct; (2) Ease of Use in the EUCS construct is similar
to the one in the B2EPUS construct. These overlapping
B2EPUS
Usefulness
Portal Design Convenience
Of Access
Confidentiality *
In
Pr for **
od ma se
uc tio ofU t*
n
t* n se te
Knowledge & Ea Con
UIS Involvement Timeliness EUCS
Format
EDP Staff &
Services Accuracy
Figure 3 A comparison of underlying dimensions: UIS, EUCS,
and B2EPUS. *Information Product and Content appeared to be
a part of Confidentially in the B2EPUS construct. **Ease of Use
appears in both the B2EPUS and EUCS constructs.
European Journal of Information Systems
662 User satisfaction with B2E portals
dimensions were expected since we used their scales as
references when developing our own scale (see Wang &
Liao, 2007 for another example). The fact that Informa-
tion Content (partially embedded in dimension Con-
fidentiality) and Ease of Use became a part of the B2EPUS
construct was also important because they were the two
most common dimensions of the user satisfaction
construct, thereby strengthening the face validity of the
B2EPUS scale as a user satisfaction measure. More
fundamentally, nearly all dimensions in our measure
(i.e., Confidentiality, Usefulness, Convenience of Access,
and Portal Design) and their respective items are
specifically developed for the B2E portal.
Implications of the study
This study has conceptually defined the domain of the
B2EPUS construct and operationalized this construct into
conceptually distinct indicators that could be observed
and assessed. The B2EPUS scale developed in this
study is exceptionally important in the development of
the B2EPUS construct since a psychometrically sound
measurement is a prerequisite for any theoretical
advancement (Schwab, 1980). The findings provide a
better understanding of the multidimensionality of the
B2EPUS construct, which comprises five empirically
distinguishable dimensions (Ease of Use, Convenience
of Access, Confidentiality, Portal Design, and Usefulness).
The existence of the second-order factor in the B2EPUS
construct also confirms the view that the user satisfaction
construct should not be treated as a single construct, as
commonly suggested. This study may contribute to and
stimulate further interest in the study of user satisfaction
within the web-based environment and its link to other
outcome variables, such as employee loyalty and job
satisfaction.
As for its practical implications, the 18-item B2EPUS
scale is an accessible and easily administered measure of
B2EPUS. Organizations may administer the scale to
evaluate their B2E portal performance and examine
specific dimensions (i.e., Convenience of Access, Useful-
ness, Ease of Use, Portal Design, and Confidentiality) of
the B2E portal. The portal project manager can also
administer the scale over successive periods to assess the
extent to which employees are satisfied with the portal in
both the earlier and later implementation stages. Doing
so enables the organization to strategically manage and
control the most important aspects of B2E portals during
portal implementation.
Study limitations and directions for future
research
Although the resulting scale has been subjected to
rigorous development and validation procedures, the
study is not without limitations. After a series of
re-specifications, our final five-factor model still had a
significant w2 statistics. It is important to note however,
that the w2 index is always sensitive to, and can be
adversely affected by, sample size (Hair et al., 1998).
European Journal of Information Systems
Dewi Rooslani Tojib et al
A good fitting model could be rejected merely because
of small differences between the observed and predicted
covariance matrices if the sample size were large. On the
contrary, ill-fitting models may be accepted as having
adequate fit if the sample size is small. Bentler & Bonett
(1980) recommended nearly three decades ago that
accepting a model with a small w2 signifies nothing but
dubious logic, since one can purposefully produce a small
w2 by simply reducing the sample size. Given the fact that
mere reliance on w2 does not warrant any statistical merit,
alternative types of indices were used in the current
investigation to assess the overall fit of the measurement
model ( Joreskog & Sorbom, 1988; Hair et al., 1998; Gefen
et al., 2000). In the IS literature, it has been common
practice to report a model fit assessment with various fit
indices (e.g., Lewis & Byrd, 2003; Chiou, 2004; Xia & Lee,
2005), and ours is no exception. As seen in Table 5, most
of the fit indices we reported (i.e., GFI, AGFI, RMR,
RMSEA, TLI, NFI, CFI, AIC, CAIC) were within the
recommended thresholds. Consequently, the decision
to retain our model was reasonable and substantiated
(Kim & Umanath, 2005).
Second, the participating organizations were predomi-
nantly Australian universities. While the unit of analysis
is the individual B2E portal user who exhibits characteri-
stics typical of B2E portal users, the generalizability of the
findings remains yet to be determined. The sampling
method may have potential bias since a sample of willing
respondents may not be generalized. Future research
should test the B2EPUS scale with new data sets in other
settings. For instance, researchers could perform cross-
cultural studies using different larger samples from
different industries for greater generalization of the scale.
Test–retest reliability investigation should be considered
in order to examine the stability of the construct over
time (Webster & Martocchio, 1992). Future studies might
also investigate the perceptual differences in their
satisfaction with B2E portals among employees in various
roles. With different types of work arrangements in
today’s workplaces, employees may perceive the B2E
portal differently. For instance, telecommuting home-
based employees may view them as collaboration tools.
The portal’s accessibility through the Internet allows
employees to stay connected with colleagues and retains
their sense of community. On the other hand, office-
based employees may perceive the portal as their desktop
destination since they can perform their allocated tasks,
administrative responsibilities, and individual-related
tasks through the portal. It would be interesting to
examine whether their perceptual differences affect their
level of satisfaction. Such examination is crucial to ensure
that the portal can satisfy the needs and wants of all
employees. Finally, this study has incorporated multiple
rounds of theory building through literature review and
expert opinion. Furthermore, a rigorous approach of
theory testing has been adopted that seems to confirm
the adequacy of measurement. However, since the
establishment of psychometric properties of a measure
 User satisfaction with B2E portals
is a complex and lengthy process, there exists a need for
continued refinement and validation of the measure.
Thus, future work should replicate the current one in a
variety of settings in order to ascertain the generali-
zability of the findings.
Conclusion
The primary aim of this study was to develop and validate
a measure of user satisfaction with B2E portals. Explora-
tory and confirmatory factor analyses established an
About the authors
Dewi Rooslani Tojib is a Research Fellow in the
Department of Marketing at Monash University. She
holds Bachelor of Business Systems (H1) and Doctor of
Philosophy (Ph.D.) from Monash University. Her research
has been published in a number of academic and
practitioner journals as well as presented in several
international conferences. Her research interests include
web-based information systems, mobile technology, con-
sumer behavior, services marketing, experimental de-
signs, and choice modeling.
Ly-Fie Sugianto is a senior lecturer in the Department of
Accounting and Finance at Monash University. She
obtained her Ph.D. in Engineering from Monash and
has received a number of awards and grants for her work
and contribution in optimization and IS research. Her
research reflects her on-going interests in the study and
References
ABDINNOUR-HELM SF, CHAPARRO BS and FARMER SM (2005) Using the end
user computing satisfaction scale to measure satisfaction with a
website. Decision Sciences 36(2), 341–363.
ABDUL-GADER A (1996) The impact of user satisfaction on computer-
mediated communication acceptance: a causal path model. Informa-
tion Resources Management Journal 9(1), 17–26.
ALADWANI AM (2002) Organisational actions, computer attitudes, and
end user satisfaction in public organisations: an empirical study.
Journal of End User Computing 14(1), 42–49.
ALLEN MJ and YEN WM (1979) Introduction to Measurement Theory.
Brooks/Cole, Monterey, CA.
AMOAKO-GYAMPAH K and WHITE KB (1993) User involvement and user
satisfaction: an exploratory contingency model. Information & Mana-
gement 25(1), 1–10.
ANDERSON EE (1989) The implementation of information systems for
workers: a structural equation model. Information & Management
16(4), 171–186.
ANDERSON JC and GERBING DW (1988) Structural equation modelling in
practice: a review and recommended two-step approach. Psychological
Bulletin 103, 411–423.
ANG J and SOH PH (1997) User information satisfaction, job satisfaction
and computer background: an exploratory study. Information &
Management 32(5), 255–266.
AU N, NGAI EWT and CHENG ETC (2002) A critical review of end-user
information system satisfaction research and a new research frame-
work. Omega 30(6), 451–478.
Dewi Rooslani Tojib et al 663
18-item, five-factor measure of B2E portals user satisfac-
tion. Preliminary psychometric evidence suggested that
this measure is reliable and valid, and can therefore be
used with confidence by academics and practitioners.
This scale has the potential to support a wide range of IS
research as a better understanding of what constitute the
B2EPUS construct has been demonstrated. Practitioners
may also use the scale to evaluate the success of their B2E
portal implementations.
development of support tools and techniques for intel-
ligent decision making. Other positions she has held
include serving as program and international board
committee for international conferences and journal
editorial. Ly-Fie has also been appointed as an Expert of
International Standing by the Australian Research Coun-
cil College of Expert.
Sen Sendjaya is a Senior Lecturer in the Department
of Management at Monash University. He earned his
MBA and Ph.D. from Monash University. Sen has
published some of his research in top journals such
as Journal of Management Studies and International
Journal of Human Resource Management. His current
research projects include leadership modeling, servant
leadership, destructive leadership, and positive
emotions.
BAGOZZI RP (1981) An examination of the validity of two models of
attitude. Multivariate Behavioural Research 16(3), 323–359.
BAGOZZI RP and PHILLIPS LW (1982) Representing and testing organiza-
tional theories: a holistic construal. Administrative Science Quarterly 27,
459–489.
BAI B, LAW R and WEN I (2008) The impact of website quality on customer
satisfaction and purchase intentions: evidence from Chinese online
visitors. International Journal of Hospitality Management 27(3),
391–402.
BAILEY JE and PEARSON SW (1983) Development of a tool for measurement
and analysing computer user satisfaction. Management Science 29(5),
530–545.
BANNAN KJ (2002) Personalization and portals: if you build it (right) they
will come. EContent 25(10), 17–21.
BARKI H and HUFF SL (1985) Change, attitude to change, and
decision support systems success. Information & Management 9(5),
261–268.
BARONAS AMK and LOUIS MR (1988) Restoring a sense of control during
implementation: how user involvement leads to system acceptance.
MIS Quarterly 12(1), 111–124.
BAROUDI JJ, OLSON MH and IVES B (1986) An empirical study of the impact
of user involvement on system usage and information satisfaction.
Communications of the ACM 29(3), 232–238.
BAROUDI JJ and ORLIKOWSKI WJ (1988) A short measure of user information
satisfaction: a psychometric evaluation and notes on use. Journal of
Management Information Systems 4(4), 44–59.
European Journal of Information Systems
664 User satisfaction with B2E portals
BENARD R and SATIR AS (1993) User satisfaction with EISS. Information
Systems Management 10(4), 21–30.
BENBYA H, PASSIANTE G and BELBALY NA (2004) Corporate portal: a tool for
knowledge management synchronization. International Journal of
Information Management 24, 201–220.
BENTLER P (1995) EQS Structural Equations Program Manual. Multivariate
Software, Encino.
BENTLER PM and BONETT DG (1980) Significance tests and goodness
of fit in the analysis of covariance structures. Psychological Bulletin 88,
588–606.
BERGERON F (1986) Factors influencing the use of DP charge back
information. MIS Quarterly 10(3), 225–238.
BERGERON F and BERUBE C (1988) The management of the end user
environment: an empirical investigation. Information & Management
14(3), 107–113.
BERGERON F, RAYMOND L, RIVARD L and GARA M-F (1992) Understanding EIS
use: an empirical test of a behavioral model. In Proceedings of the
Twenty-Fifth Hawaii International Conference on System Sciences
(MILUTINOVIC V and SHRIVER BD, Eds), pp 157–165, IEEE Computer
Society Press, Koloa Hawai, United States.
BERGERON F, RIVARD S and DE SERRE L (1990) Investigating the support role
of the information centre. MIS Quarterly 14(3), 247–262.
BERNSTEIN IH (1988) Applied Multivariate Analysis. Springer-Verlag, New
York.
BHATTACHERJEE A (2002) Individual trust in online firms: scale development
and initial test. Journal of Management Information Systems 19(1),
211–241.
BISCONTI K (2004) Determining the value and ROI of an enterprise portal.
Available from http://esj.com/enterprise/article.aspx?EditorialsID ¼
1197 (accessed 7 October 2008).
BRAUE D (2003) Portals: opening new doors to business. Available
from http://www.zdnet.com.au/news/business/soa/Portals-opening-
new-doors-to-business/0,139023166,120280513,00.htm (accessed 7
October 2008).
BROSCHE C (2002) Designing the corporate portal. Unpublished Master
Thesis, University of Gothenburg, Sweden.
BYRNE BM (2001) Structural Equation Modelling with AMOS: Basic
Concepts, Applications, and Programming. Lawrence Erlbaum, Mahwah,
NJ.
CALISIR F and CALISIR F (2004) The relation of interface usability
characteristics, perceived usefulness, and perceived ease of use to
end-user satisfaction with enterprise resource planning (ERP) systems.
Computers in Human Behaviour 20, 505–515.
CHANG JC-J (2001) The development of a measure to assess the
performance of the information systems function: a multiple-
constituency approach. Ph.D. Thesis, University of Pittsburgh, USA.
CHANG JCJ and KING WR (2005) Measuring the performance of
information systems: a functional scorecard. Journal of Management
Information Systems 22(1), 85–115.
CHEN L, SOLIMAN KS, MAO E and FROLICK MN (2000) Measuring user
satisfaction with data warehouses: an exploratory study. Information &
Management 37(3), 103–110.
CHIN JP, DIEHL VA and NORMAN KL (1988) Development of an instrument
measuring user satisfaction of the human-computer interface. In
Proceedings of the ACM CHI 88 Human Factors in Computing Systems
Conference. (ELLIOT S, DOUGLAS F and SHEPPARD SB, Eds), pp 213–218,
Association for Computing Machinery (ACM) Press, New York, United
States.
CHIOU J-S (2004) The antecedents of consumers’ loyalty toward Internet
service providers. Information & Management 41, 685–695.
CHO N and PARK S (2001) Development of electronic commerce user-
consumer satisfaction index (ECUSI) for Internet shopping. Industrial
Management & Data Systems 101(8), 400–405.
CHURCHILL GA (1979) A paradigm for developing better measures
of marketing constructs. Journal of Marketing Research 19(4),
491–504.
COMREY AL and LEE HB (1992) A First Course in Factor Analysis. Lawrence
Erlbaum, New Jersey.
CORBITT B, BRADLEY T and THANASANKIT T (2005) Factors influencing the
implementation and use of a portal for knowledge management in
higher education. In Proceedings of the 9th Pacific Asia Conference on
Information Systems. IEEE Computer Society Press, Bangkok, Thailand.
European Journal of Information Systems
Dewi Rooslani Tojib et al
CYERT RM and MARCH JG (1963) A Behavioural Theory of the Firm. Prentice-
Hall, Englewood Cliffs, NJ.
DAVIS FD (1989) Perceived usefulness, perceived ease of use and
user acceptance of information technology. MIS Quarterly 13(3),
319–339.
DAVIS L (1992) Scale review: getting the most from your panel of experts.
Applied Nursing Research 5, 104–107.
DAVYDOV MM (2001) Corporate Portals and e-Business Integration.
McGraw-Hill, New York.
DEANS KR and ALLMEN S (2002) Poo poo portals at your peril. In
Proceedings of the Eighth Australian World Wide Web Conference.
(ELLIS A, Ed), pp 117–135, Aus Web, Queensland, Australia.
DELONE WH and MCLEAN ER (2003) The DeLone and McLean model of
information systems success: a ten-year update. Journal of Manage-
ment Information Systems 19(4), 9–30.
DEVELLIS RF (2003) Scale Development: Theory and Applications 2nd edn,
Sage Publications Inc., Thousand Oaks, CA.
DE WULF K, SCHILLEWAERT N, MUYLLE S and RANGARAJAN D (2006) The role
of pleasure in web site success. Information & Management 43,
434–446.
DIAS C (2001) Corporate portals: a literature review of a new concept
in information management. International Journal of Information
Management 21(4), 269–287.
DOLL WJ and TORKZADEH G (1988) The measurement of end-user
computing satisfaction. MIS Quarterly 12, 259–275.
DOLL WJ and TORKZADEH G (1998) Developing a multidimensional
measure of system-use in an organizational context. Information &
Management 33, 171–185.
DOLL WJ, XIA W and TORKZADEH G (1994) A confirmatory factor analysis of
the end-user computing satisfaction instrument. MIS Quarterly 18(4),
453–461.
DOWNING CE (1999) System usage behaviour as a proxy for user
satisfaction: an empirical investigation. Information & Management
35(4), 203–216.
EIGHMEY J and MCCORD L (1998) Adding value in the information age:
uses and gratifications of sites on the World Wide Web. Journal of
Business Research 41(3), 187–194.
ETEZADI-AMOLI J and FARHOOMAND AF (1996) A structural model of end
user computing satisfaction and user performance. Information &
Management 30(2), 65–73.
FEHRING RJ (1987) Methods to validate nursing diagnoses. Heart and Lung
16, 625–629.
FOLEY TJ and NEWMAN MA (1988) An empirical study of user satisfaction
with a microcomputer-based campus wide information system. In
Proceedings of the ACM-SIGUCCS XVI. (CRANE P, Ed), pp 91–100,
Association for Computing Machinery (ACM) Press, New York, United
States.
FRASER SG and SALTER G (1995) A motivational view of information system
success: a reinterpretation of DeLone & McLean’s model. In Proceed-
ings of the 6th Australasian Conference on Information Systems 1,
(SPENCER S and JENKINS A, Eds), pp 119–140, Australasian Association
for Information Systems, Adelaide, South Australia.
GALLAGHER CA (1974) Perceptions of the value of management
information system. Academy Management Journal 17, 46–55.
GATIAN AW (1994) Is user satisfaction a valid measure of system
effectiveness? Information & Management 26(3), 119–131.
GEFEN D, STRAUB D and BOUDREAU MC (2000) Structural equation
modelling and regression: guidelines for research practice. Commu-
nications of the AIS 4(7), 1–70.
GELDERMAN M (1998) The relation between user satisfaction, usage of
information systems and performance. Information & Management
34(1), 11–18.
GLORFELD KD and CRONAN TP (1992) Computer information satisfaction:
a longitudinal study of computing systems and EUC in a public
organisation. Journal of End User Computing 5(1), 27–36.
GRANT J and KINNEY M (1992) Using the Delphi technique to examine the
content validity of nursing diagnoses. Nursing Diagnosis 3, 12–22.
GRANT JS and DAVIS LL (1997) Focus on quantitative methods: selection
and use of content experts for instrument development. Research in
Nursing & Health 20, 269–274.
HAIR JF, TATHAM RL, ANDERSON RE and BLACK W (1998) Multivariate Data
Analysis 5th edn, Prentice-Hall, Upper Saddle River, NJ.
 User satisfaction with B2E portals
HAMERMAN P (2004) Creating the online employee experience. Best
Practices. Forrester Research, Inc., 4 October.
HARRISON AW and RAINER Jr. RK (1996) A general measure of user
computing satisfaction. Computers in Human Behaviour 12, 79–92.
HARTRUM TC, MALLARY TC and FOLEY JW (1989) Evaluating user satisfaction
of an interactive computer program. In Journal Title: Proceedings of
the IEEE 1989 National Aerospace and Electronics Conference 22–26
May, pp 508–514, IEE, Inc., Ohio, United States.
HENRY JW and STONE RW (1994) A structural equation model of end user
satisfaction with a computer-based medical information system.
Information Resources Management Journal 7(3), 21–33.
HILTZ SR and JOHNSON K (1990) User satisfaction with computer mediated
communication systems. Management Science 36(6), 739–764.
HINKIN TR (1995) A review of scale development practices in the study of
organisations. Journal of Management 21(5), 967–988.
HO C-F and WU W-H (1999) Antecedents of customer satisfaction on
the Internet: an empirical study of online shopping. In Proceedings of
the 32nd Hawaii International Conference on System Sciences, pp 1–10,
IEEE Computer Society Press, Maui (Hawaii), United States.
HOFFMAN DL and NOVAK TP (1997) A new marketing paradigm for
electronic commerce. The Information Society 13(1), 43–54.
HOXMEIER JA and DICESARE C (2000) System response time and user
satisfaction: an experimental study of browser based application. In
Proceedings of the Association of Information Systems Americas
Conference, Association of Information Systems, California, United
States.
HUANG JH, YANG C, JIN BH and CHIU H (2004) Measuring satisfaction
with business-to-employee systems. Computers in Human Behaviour
20, 17–35.
IGBARIA M and NACHMAN SA (1990) Correlates of user satisfaction with
end user computing: an exploratory study. Information & Management
19(2), 73–82.
IGBARIA M and TAN M (1997) The consequences of information
technology acceptance on subsequent individual performance.
Information & Management 32(3), 113–121.
IIVARI J and ERVASTI I (1994) User information satisfaction: IS implement-
ability and effectiveness. Information & Management 27(4), 205–220.
IIVARI J and KARJALAINEN M (1989) Impact of prototyping on user
information satisfaction during the IS specification phase. Information
& Management 17(1), 31–45.
IVES B, OLSON MH and BAROUDI JJ (1983) The measurement of user
information satisfaction. Communications of the ACM 26(10), 785–793.
JENKINS AM and RICKETTS JA (1979) Development of a scale to measure
user information satisfaction with management information systems.
Unpublished Working Paper, Indiana University, Bloomington.
JENKINS M and RICKETTS J (1985) The development of an MIS satisfaction
questionnaire: an instrument for evaluating user satisfaction with
turnkey decision support systems. In Discussion Paper No. 296,
Indiana University, August.
JONES MC and BEATTY RC (2001) User satisfaction with EDI: an empirical
investigation. Information Resources Management Journal 14(2), 17–26.
JORESKOG KG and SORBOM D (1988) LISREL 7: A Guide to the Program and
Applications. SPSS Inc., Chicago, IL.
KEKRE S, KRISHNAN MS and SRINIVASAN K (1995) Drivers of customer
satisfaction for software products: implications for design and service
support. Management Science 41(9), 1456–1470.
KHALIL OEM and ELKORDY MM (1999) The relationship between user
satisfaction and systems usage: empirical evidence from Egypt. Journal
of End User Computing 11(2), 21–33.
KIM KK and UMANATH NS (2005) Information transfer in B2B procure-
ment: an empirical analysis and measurement. Information & Manage-
ment 42(6), 813–828.
KLINE RB (2001) Principles and Practices of Structural Equation Modelling.
Guilford Press, New York.
KOTOROV R and HSU E (2001) A model for enterprise portal management.
Journal of Knowledge Management 5(1), 86–93.
LAI J-Y and YANG C-C (2008) Effects of employees’ perceived depend-
ability on success of enterprise applications in e-business. Industrial
Marketing Management, Available from http://www.sciencedirect.
com/science?_ob ¼ MImg&_imagekey ¼ B6V69-4S03075-9&_cdi ¼ 5809
&_user ¼ 542840&_orig ¼ search&_coverDate ¼ 03%2F04%2F2008
&_sk ¼ 999999999&view ¼ c&wchp ¼ dGLbVzW-zSkzV&md5 ¼
Dewi Rooslani Tojib et al 665
786a7926012808535387c5fa3ceb8359&ie ¼ /sdarticle.pdf (access 8
October 2008), in press.
LARCKER DF and LESSIG VP (1980) Perceived usefulness of information: a
psychometric evaluation. Decision Sciences 11(1), 121–134.
LAWRENCE M and LOW G (1993) Exploring individual user satisfaction
within user-led development. MIS Quarterly 17(2), 195–208.
LEDERER AL and SETHI V (1992) Root causes of strategic information
systems planning implementation problems. Journal of Management
Information Systems 9, 25–45.
LEWIS BR and BYRD TA (2003) Development of a measure for information
technology infrastructure. European Journal of Information Systems 12,
93–109.
LEWIS BR, TEMPLETON G.F and BYRD TA (2005) A methodology for
construct development in MIS research. European Journal of Informa-
tion Systems 14, 388–400.
LIANG T-P, LAI H-J and KU Y-C (2007) Personalized content recommenda-
tion and user satisfaction: theoretical synthesis and empirical findings.
Journal of Management Information Systems 23(3), 45–70.
LIAW SS and HUANG HM (2003) An investigation of user attitudes toward
search engines as an information retrieval tool. Computers in Human
Behaviour 19, 751–765.
LIN H-F (2007) The impact of website quality dimensions on customer
satisfaction in the B2C E-commerce context. Total Quality Manage-
ment 18(4), 363–378.
LINDGAARD G (2007) Aesthetics, visual appeals, usability, and user
satisfaction: what do the user’s eyes tell the user’s brain? Australian
Journal of Emerging Technologies and Society 5(1), 1–14.
LOIACONO ET, WATSON RT and GOODHUE DL (2002) WebQual: a measure
of website quality. In Marketing Theory and Applications (EVANS KK and
SCHEER LK, Eds), Vol. 13, American Marketing Association, Chicago.
LYNN M (1986) Determination and quantification of content validity.
Nursing Research 35(6), 382–385.
MACKIE BG and DOWNING CE (2004) Viability of electronic service
offerings: user satisfaction in the business-to-employee market.
International Journal of Services Technology and Management 5(1),
14–24.
MAHMOOD MA and BECKER JD (1985) Effect of organizational maturity on
end-users’ satisfaction with information systems. Journal of Manage-
ment Information Systems 2(3), 37–64.
MAK BL and SOCKEL H (2001) A confirmatory factor analysis of IS
employee motivation and retention. Information & Management
38(5), 265–276.
MAWHINNEY CH and LEDERER AL (1990) A study of personal computer
utilization by managers. Information & Management 18(5), 243–253.
MCHANNEY R and CRONAN TP (1998) Computer simulation success: on the
use of the end-user computing satisfaction instrument. Decision
Sciences 29(2), 525–536.
MCHANEY R, HIGHTOWER R and PEARSON J (2002) A validation of the end-
user computing satisfaction instrument in Taiwan. Information &
Management 39(6), 503–511.
MCKEEN JD, GUIMARES T and WETHERBE JC (1994) The relationship between
user participation and user satisfaction: an investigation of four
contingency factors. MIS Quarterly 18(4), 427–451.
MELONE NP (1990) A theoretical assessment of the user satisfaction
construct in information systems research. Management Science 36(1),
76–91.
MONTAZEMI AR (1988) Factors affecting information satisfaction in the
context of the small business environment. MIS Quarterly 12(2),
239–256.
MUYLLE R, MOENAERT M and DESPONTIN M (2004) The conceptualization
and empirical validation of web site user satisfaction. Information &
Management 41, 543–560.
NATH R (1988) A measure of end-user computing success. Working
Paper, MIS Department, Memphis State University, Memphis, TN.
NATH R (1989) Are frequent users more satisfied? Information Processing
and Management 25(5), 557–562.
NEUMANN S and SEGEV E (1980) Evaluate your information systems.
Journal of Systems Management 9(1), 34–41.
NUNNALLY JC (1978) Psychometric Theory 2nd edn, McGraw-Hill,
New York.
ONG CS and LAI JY (2004) Developing an instrument for measuring user
satisfaction with knowledge management systems. In Proceedings of
European Journal of Information Systems
666 User satisfaction with B2E portals
the 37th Hawaii International Conference on System Sciences, IEEE
Computer Society Press, Island of Hawaii (Hawaii), United States.
OTTO JR, NAJDAWI MK and CARON KM (2000) Web-user satisfaction: an
exploratory study. Journal of End User Computing 12(4), 3–10.
PALVIA PC (1996) A model and instrument for measuring small business
user satisfaction with information technology. Information & Manage-
ment 31(3), 151–163.
POWERS RF and DICKSON GW (1973) MIS project management:
myths, opinions, and reality. California Management Review 15(3),
147–156.
RAGHUNATHAN B, RAGHUNATHAN TS and TU Q (1999) Dimensionality of the
strategic grid framework: the construct and its measurement.
Information Systems Research 10(4), 343–355.
RAHIM MM (2007) Identifying barriers to using business-to-employee
(B2E) portals: some lessons learned from an Australian university. In
Proceedings of the 40th Hawaii International Conference on System
Sciences (HICSS’07), IEEE Computer Society Press, Waikoloa (Hawaii),
United States.
RAYMOND L (1985) Organisational characteristics and MIS success in
context of small business. MIS Quarterly 9(1), 37–52.
RAYMOND L (1987) Validating and applying user satisfaction as a measure
of MIS success in small organizations. Information & Management
12(4), 173–179.
REMUS U (2007) Critical success factors for implementing enterprise
portals: a comparison with ERP implementation. Business Process
Management Journal 13(4), 538–552.
RYKER R and NATH R (1997) User satisfaction determinants: the role of
hardware and procedural components. The Journal of Computer
Information Systems 38(2), 44–48.
RYKER R, NATH R and HENSON J (1997) Determinants of computer
user expectations and their relationship with user satisfaction: an
empirical study. Information Processing And Management 33(4),
529–537.
RYLEY S (2001) Corporate portal development: a practical approach
ensures real business benefits. Business Information Review 18(2), 28–34.
SALISBURY WD, CHIN WW, GOPAL A and NEWSTED PR (2002) Research
report: better theory through measurement-developing a scale to
capture consensus on appropriation. Information Systems Research
13(1), 91–103.
SCHWAB DP (1980) Construct validity in organization behaviour. In
Research in Organizational Behaviour (STAW BM and CUMMINGS LL, Eds),
Vol. 12, pp 3–43, JAI Press, Greenwich.
SEDDON PB and YIP S-K (1992) An empirical evaluation of user
information satisfaction (UIS) measures for use with general ledger
accounting software. Journal of Information Systems 6(1), 75–92.
SEGARS AH (1997) Assessing the unidimensionality of measurement:
a paradigm and illustration within the context of information
systems research. International Journal of Management Science 25(1),
107–121.
SEGARS AH and GROVER V (1998) Strategic information systems planning
success: an investigation of the construct and its measurement. MIS
Quarterly 22(2), 139–163.
SEKARAN U (1992) Research Methods for Business: A Skill-Building Approach.
Wiley, New York.
SENGUPTA K and ZVIRAN M (1997) Measuring user satisfaction in an
outsourcing environment. IEEE Transactions on Engineering Manage-
ment 44(4), 414–421.
SETHI V and KING WR (1991) Construct measurement in information
systems research: an illustration in strategic systems. Decision Sciences
22, 455–472.
SIMMERS CA and ANANDARAJAN M (2001) User satisfaction in the Internet-
anchored workplace: an exploratory study. Journal of Information
Technology Theory and Application 3(5), 39–53.
SIMON JS, GROVER G, TENG JT and WHITCOMB K (1996) The relationship of
information system training methods and cognitive ability to end user
satisfaction, comprehension, and skill transfer: a longitudinal field
study. Information System Research 7(4), 466–490.
SMITH AG (2001) Applying evaluation criteria to New Zealand govern-
ment websites. International Journal of Information Management 21,
134–149.
SMITH MA (2004) Portals: toward an application framework for
interoperability. Communications of the ACM 47(10), 93–97.
European Journal of Information Systems
Dewi Rooslani Tojib et al
SONQUIST JA and DUNKELBERG WC (1977) Survey and Opinion Research:
Procedures for Processing and Analysis. Prentice-Hall, London.
STEWART KA and SEGARS AH (2002) An empirical examination of the
concern for information privacy instrument. Information Systems
Research 13(1), 36–49.
STRAUB D, BOUDREAU MC and GEFEN D (2004) Validation guidelines of IS
positivist research. Communication of the Association for Information
Systems 14, 380–426.
STRAUB DW (1989) Validating instruments in MIS research. MIS Quarterly
13(2), 147–169.
SUGIANTO LF and TOJIB DR (2006) Information at your fingertips: B2E
portal as a strategic tool for today’s workforce. In Portals: People,
Processes, and Technology (COX A, Ed), pp 125–137, Facet Publishing,
London.
SWANSON EB (1974) Management information systems: appreciation and
involvement. Management Science 21(2), 178–188.
SWANSON EB (1987) Information channel disposition and use. Decision
Sciences 18(1), 131–145.
TABACHNICK BG and FIDELL LS (2001) Using Multivariate Statistics 4th edn,
Harper Collins, New York.
TATE M, EVERMANN J, HOPE B and BARNES B (2007) Perceived service quality
in a university web portal: revising the e-qual instrument. In
Proceedings of the 40th Annual Hawaii International Conference on
System Sciences (HICSS’07), IEEE Computer Society Press, Waikoloa
(Hawaii), United States.
TEMPLETON GF, LEWIS BR and SNYDER CA (2002) Development of a
measure for the organisational learning construct. Journal of Manage-
ment Information Systems 19(2), 175–218.
TEO TSH, LIM VKG and LAI RYC (1997) Users and uses of the Internet: the
case of Singapore. International Journal of Information Management
17(5), 325–336.
VLAHOS GE and FERRATT TW (1995) Information technology use by
managers in Greece to support decision making: amount, perceived
value, and satisfaction. Information & Management 29, 305–315.
WAN TB and WAH LT (1990) Validation of a user satisfaction instrument
for office automation success. Information & Management 18(4),
203–208.
WANG YS (2003) Assessment of learner satisfaction with asynchronous
electronic learning systems. Information & Management 41, 75–86.
WANG Y-S and LIAO Y-W (2007) The conceptualization and measurement
of m-commerce user satisfaction. Computers in Human Behaviour 23,
381–398.
WEBSTER J and MARTOCCHIO JJ (1992) Microcomputer playfulness:
development of a measure with workplace implications. MIS Quarterly
16(2), 201–226.
WEBSTER J and WATSON RT (2002) Analyzing the past to prepare
for the future: writing a literature review. MIS Quarterly 26(2),
xiii–xxiii.
WHITE C (2003) Determining enterprise portal ROI. Available from http://
www.dmreview.com/issues/20030401/6510-1.html (accessed 7
October 2008).
WU JH, WANG YM, CHIEN MCC and TAI WC (2002) An examination of ERP
user satisfaction in Taiwan. In Proceedings of the 35th Hawaii
International Conference on System Sciences, IEEE Computer Society
Press, Island of Hawaii (Hawaii), United States.
XIA W and LEE G (2005) Complexity of information systems development
projects: conceptualization and measurement development. Journal of
Management Information Systems 22(1), 45–83.
XIAO L and DASGUPTA S (2002) Measurement of user satisfaction with
web-based IS: an empirical study. In Proceedings of the Eight Americas
Conference on Information Systems (WARKENTIN M, Ed), pp 1149–1156,
Association for Information Systems, Texas, United States.
YANG Z, CAI S, ZHOU Z and ZHOU N (2005) Development and validation
of an instrument to measure user perceived service quality of
information presenting web portals. Information & Management 42,
575–589.
YAVERBAUM GJ and NOSEK J (1992) Effects of information system education
and training on end user satisfaction: an empirical evaluation.
Information & Management 32(5), 217–225.
YEO JSJ, AURUM A, HANDZIC M and PARKIN P (2002) When technology
is mandatory-factors influencing user satisfaction. In Proceedings
of the International Conference on Computers in Education (WERNER B
 User satisfaction with B2E portals
Ed), pp 1023–1024, IEEE Computer Society Press, Washington, United
States.
YEO KT (2002) Critical failure factors in information systems project.
International Journal of Project Management 20(1), 241–246.
YOO B and DONTHU N (2001) Developing a scale to measure the
perceived quality of an Internet shopping site (SiteQual). Quarterly
Journal of Electronic Commerce 2(1), 31–45.
YOON Y, GUIMARAES T and O’NEAL Q (1995) Exploring the factors
associated with expert systems success. MIS Quarterly 19(1),
83–106.
Appendix A: The final 18-item of the B2EPUS scale
1. The staff portal enables me to share or exchange
project/task information with my team member
colleagues.
2. The staff portal facilitates my collaboration work
with all colleagues.
3. The staff portal enables me to discuss work or project
issues with my immediate work colleagues.
4. The staff portal enables me to share general informa-
tion through email or on website with other
colleagues in the whole organization.
5. I feel confident in submitting personal information
through the staff portal because it will be properly
used by authorized people.
6. The information presented on the portal can be
trusted.
7. Information presented on the portal is dependable.
8. I feel the staff portal is secure.
Appendix B: Six items measuring the B2E portal use construct
1. Since the staff portal is now available, I prefer to
access the portal to check and reply my emails.
2. Since the staff portal is now available, I prefer
to access the portal to communicate with my
colleagues.
3. Since the staff portal is now available, I prefer to
access the portal to get the latest organizational
news.
Dewi Rooslani Tojib et al 667
ZAZELENCHUK TW and BOLING E (2003) Considering user satisfaction in
designing web-based portals. EDUCAUSE Quarterly 26(1), 35–40.
ZVIRAN M and ERLICH Z (2003) Measuring IS user satisfaction. Commu-
nications of the Association for Information Systems 2003(12), 81–103.
ZVIRAN M, GLEZER C and AVNI I (2006) User satisfaction from commercial
websites: the effect of design and use. Information & Management 43,
157–178.
ZVIRAN M, PLISKIN N and LEVIN R (2005) Measuring user satisfaction and
perceived usefulness in the ERP context. The Journal of Computer
Information Systems 45(3), 43–52.
9. I can rely on the information presented on the staff
portal to carry out my tasks.
10. No training on the use of staff portal is necessary as
the portal use is self-explanatory.
11. The staff portal is easy to navigate, back forward and
backward.
12. When I am navigating the staff portal, I feel that I am
in control of what I am doing.
13. The staff portal is accessible from my home through
Internet connection.
14. Gaining access to the staff portal is easy.
15. The staff portal is accessible 24 h a day, 7 days a week.
16. The staff portal is aesthetically designed.
17. The design of the staff portal is attractive.
18. The staff portal is user friendly with many help
functions and useful button and links.
4. Since the staff portal is now available, I prefer to
access the portal to search for any information
I need.
5. Since the staff portal is now available, I prefer to
access the portal to perform my job-related tasks.
6. Since the staff portal is now available, I prefer to
access the portal to perform my personal-related
tasks.
European Journal of Information Systems