Improvement in Organizational
Performance and Self-Assessment
Practices by Selected American Firms
TON VAN DER WIELE, ERASMUS UNIVERSITY
ALAN BROWN, EDITH COWAN UNIVERSITY
ROBERT MILLEN, NORTHEASTERN UNIVERSITY
DANIEL WHELAN, BOSTON SECTION, ASQ
Quality award models, such as the Malcolm Baldrige
National Quality Award, have stimulated considerable
interest in quality management and provided guide-
lines for organizations seeking to introduce quality
management. The notion of self-assessment has been
adopted by companies throughout the world as a
mechanism for guiding the development of such quality
activities. This involves regular and systematic reviews
of an organization’s activities and performance
against a quality model, usually based on an award,
culminating in planned improvement actions.
In this paper the authors discuss the self-assessment
practices of U.S. companies based on a questionnaire
survey. A range of issues is discussed including why
self-assessment was introduced, how it is used, and
outcomes — including the impact on organizational
performance.
The survey data suggest that self-assessment is linked to
better-performing companies. The data also highlight
some of the self-assessment practices that are used by
organizations in their quest to put meaning to quality.
Key words: improvement process, quality award
models, quality management
8 QMJ VOL. 7, NO. 4/© 2000, ASQ
© 2000, ASQ
INTRODUCTION
As organizations grapple with quality management,
they seek ways of defining and measuring the various
dimensions of quality in their efforts to improve orga-
nizational performance. Quality awards such as the
Malcolm Baldrige National Quality Award (National
Institute of Standards and Technology 1996), the
European Quality Award (EQA), or the Australian
Quality Award (AQA) all provide a similar framework
for organizations to adopt (Nakhai and Neves 1994).
The Baldrige Award in particular has had a substan-
tial impact on organizations, in that each year the
guidelines are requested by hundreds of thousands of
organizations that use them to guide their quality
drive. However, while the guidelines provide general
criteria, they don’t specify exact means of identifying
where an organization might be in terms of these criteria
or show the stages of moving toward higher levels of
quality. This is where self-assessment provides a more
tangible means of guiding the quality drive.
Self-assessment can be defined as “a cyclic, com-
prehensive, systematic, and regular review of an
organization’s activities and results against a model of
business excellence (for example, the total quality
management models of the quality awards) culminating
in planned improvement actions” (European
Foundation for Quality Management 1993a).
Guidelines for the various international awards usually
make reference to the self-assessment process.
Recent research has been conducted in a number of
European countries (Finn and Porter 1994; Van der
Improvement in Organizational Performance and Self-Assessment Practices by Selected American Firms
Wiele 1995) and Australia (Brown and Van der Wiele
1996a, 1996b) on the implementation of self-assessment
practices and their impact on organizational perfor-
mance. A search of the literature revealed, however, that
no survey had yet been reported on the self-assessment
practices of U.S. organizations.
Therefore, for a selection of U.S. firms, the objectives
of the research were to
• Investigate the self-assessment activities utilized.
• Determine the knowledge and awareness of
self-assessment activities.
• Explore reasons for success and failure with self-
assessment.
• Examine linkages between self-assessment and
improvement in business performance.
SELF-ASSESSMENT
The general approach of the self-assessment process
itself is presented in Figure 1. Individual approaches
are influenced by the structure of the organization and
differ in detail.
Several ways of using award models for self-
assessment are possible.
• Taking the award guidelines and following these for
internal use in the same way as described for award
applicants. This generally involves a considerable
workload since fact finding and report writing are
rather time-consuming, but will, on the other hand,
generate a comprehensive picture of the organization.
• Adjusting the criteria and/or the scoring to suit the
specific situation and goals of the organization. The
Baldrige Award model is defined as a generic model
that can be used for the award application by all types
of organizations whether they are large or small,
manufacturing or service, private or public. For inter-
nal use it may be better to interpret some criteria in a
specific way or to give other ratings to the criteria.
• Specifying the path from the present situation to
the end goal of being a world-class enterprise by
identifying the steps required over a longer period
of time. In this way the criteria have been defined
precisely in relation to different levels of quality
maturity of the organization from anecdotal and of
little use, to evidence and sound application.
Many organizations that adopt the third option use
a quality maturity matrix. This allows the position in
terms of quality maturity to be made visible very easily
and communicated throughout the organization. The
concept of a matrix has its origins in Crosby’s (1979)
quality maturity grid where he identifies five stages of
progression in the quality journey, namely; uncertainty,
awakening, enlightenment, wisdom, and certainty.
Figure 2 shows an example of such a matrix as adopted
by an organization.
Research by Brown and Van der Wiele (1996a)
found the use of the matrix to be relatively common in
Australia where the various award criteria form the
horizontal axis while the vertical axis indicates levels of
development along these criteria. Organizations develop
descriptors for each cell in the matrix to help identify
Figure 1 The self-assessment process.
Develop commitment
Plan self-assessment cycle
Establish model and
reporting system
Communicate plan
Review progress
Educate staff
Conduct self-assessment
Establish action plan
© 2000, ASQ
Implement action plan
Source: Self-Assessment Based on the European Model for Total Quality
Management, 1994, Guidelines for Identifying and Addressing Business Excellence
Issues. Brussels, European Foundation for Quality Management, 1993b, p. 5.
VAN DER WIELE, BROWN, MILLEN, AND WHELAN/© 2000, ASQ 9
Improvement in Organizational Performance and Self-Assessment Practices by Selected American Firms

Figure 2: Telstra continuous improvement matrix.

1 2 3 4 5 6 7
Statistical
thinking Process
Customer People Policy and Supplier and data control and
Leadership focus management strategy management management improvement
The collective Our customers Our people are The strategic and Partnerships with Statistical We enhance our
leadership under- are advocates of active advocates business planning suppliers and thinking systems through
stands the complex our organization. of our learning cycle fully contractors is integrated into continuous
relationships bet- We are partners organization. integrates all con- enhance all aspects of our improvement to
7 ween personal and working toward tinuous improve- organizational organization, sustain a
organizational a mutually ment initiatives to performance in ensuring competitive
values,and busi- beneficial sustain a meeting customer business success. advantage.
ness and commu- future. competitive requirements.
nity environments. advantage.
The collective The organization The questioning The planning Long-term The reporting Incremental and
leadership enrolls is adaptive to and challenging process mutually system and the breakthrough
the organization customers’ of values and communicates a beneficial statistical improvements
6 in a common pur- changing needs. assumptions is a common purpose. relationships methods used are expected
pose, optimizing core part of People at all levels are in place with are routinely and recognized
the diversity and organizational and functions are key suppliers. reviewed by the
commitment of life. involved in formu- for relevance. organization.
people. lation, deployment,
and review.
Leaders recognize The customer is Policies and Improvement Suppliers Lead and lag People have a
the complexity of the reason for our procedures are initiatives have a demonstrate indicators are systematic
the business existence. This is aligned to maxi- focus that is in evidence of used to measure approach to
5 environment recognized and mize opportunity harmony with process and improve continually
while balancing acted upon for individual key strategic capability performance in improve and
local and global throughout the and team directions. and continuous all processes. standardize best
issues. organization. contributions. improvement. known methods.
Leaders create a Processes are The organization Performance Choice of The impact of Reducing
supportive envi- aligned and values learning. levels and the suppliers and variation on our variation
ronment that designed to meet Ideas for means to achieve contractors is business is is the basis for
4 encourages the customers’ needs improvement are them are based on understood. We process
seeking of new and expectations. sought and identified, measures that continually improvement
opportunities. They acknowledged. agreed to, and reflect total cost. improve our which results in
provide a clear communicated. performance by higher predictability
context for change. reducing variation. of outcomes.
Leaders strive for Customer Continuous Knowledge of We use key per- We mentor and Systems are in
excellence by perceptions are improvement process formance criteria support by place for sharing
continuously sought and is built on capability to measure actual making information on
3 improving known. This collaboration and and customer performance. decisions improvements,
processes. They knowledge is valid information. needs are key Supply based on the use standardizing the
create an open acted upon. Feelings are inputs to the agreements of statistical way we do things.
and questioning expressible as planning process. are in place with methods.
environment. valid information. key suppliers.
Leaders work People in the We share a Plans are based Key performance A mind-set of Customers’ needs
with teams. They organization common problem on identified criteria and audit common and define the
foster an know the part solving opportunities procedures special cause purpose of our
2 environment they play in the methodology for improvement. for suppliers and variations is system.
that accepts process of throughout the People responsible contractors are used to focus Process measures
change. meeting organization. We for processes mutually on improvement are shaped by
customer needs. address cross- are involved in agreed upon. opportunities. customer criteria.
functional issues. planning.
Leaders see the Customers are Joint problem The PDCA cycle We talk to our An ongoing Work as a process
need for a known and solving is the underpins the suppliers. development plan is understood, peo-
supportive segmented, their basis for consistent They are aware for training in ple are seen as part
1 environment. needs are creatively planning of our needs and statistical tools of the end-to-end
They recognize explored. Market involving people. framework used expectations. and methods is process. Internal
© 2000, ASQ

personal and team information is We develop a throughout the established. customer-supplier

development gathered. plan for doing so. organization. relationships are
needs. identified.

Source: Telstra Quality Approach: Self Evaluation Guide and Workbook, February 1994.
Used with permission of Telecom Australia, Breakthrough Quality Management Branch, Melbourne.

10 QMJ VOL. 7, NO. 4/© 2000, ASQ

Improvement in Organizational Performance and Self-Assessment Practices by Selected American Firms
where they are and give direction to the activities that
have to be developed to grow to the next level of maturity.
Early applications of the matrix in Australia included
BHP (Cleary and Vogel 1989) and Telecom (Telstra
Corporation Limited 1994).
All three approaches to self-assessment involve
considerable training. This will start at the top of the
organization to create awareness and understanding with
senior managers. The training can be cascaded down to
lower management levels and also for the training of the
future assessors for the quality award. Training can be
organized internally, but many consulting firms also
offer training modules for introducing self-assessment.
One interesting aspect of these data is that self-
assessment was seen as linked to achieving ISO 9000
series certification. This may reflect some confusion
about the role of self-assessment against a quality
award model vis-à-vis auditing against the ISO
9000 series standards. The latter involves a type of
self “checklist” in the sense of meeting the standards,
yet, at the same time, the ISO standards also
promote a broader base to quality such as TQM.
In general the self-assessment process demands
considerable time and attention, and, therefore, the
whole process has to be linked and integrated into the
existing planning and review process. This is the only
way of ensuring that the quality management criteria
serve as a guide for the entire organization.
As indicated, the major national award models are
definitions of TQM in a broad sense, regarding the
whole organization and all activities that take place in
it. Research by Brown and Van der Wiele (1995, 1996a,
1996b) in Australia and Europe indicates a strong
interest in using the awards for self-assessment purposes
with many organizations having already implemented
a self-assessment process, or planning to do so. In the
Australian survey some 21 percent of respondents had
conducted self-assessment using the AQA criteria
(Australian Quality Awards Foundation 1996). Of those
who indicated that they had been involved in self-
assessment activities during the past three years, 30
percent said they had been involved with at least one
formal assessment for the AQA.
BENEFITS OF SELF-ASSESSMENT
Many organizations have difficulties with measuring
TQM progress, which has been mentioned as one of the
reasons for the failure of attempts to introduce TQM
(Boyce 1992). Self-assessment on the basis of the award
criteria is one means of measuring the overall effects of
TQM efforts, and to go through the plan-do-check-act
cycle by evaluating the results of the self-assessment
and taking action for the following period.
Benefits of self-assessment mentioned by seven
organizations forming a benchmarking group on
self-assessment in the United Kingdom (European
Foundation for Quality Management 1993a) were
as follows:
• Management teams are setting the agenda for their
improvement activities in the forthcoming period.
• The model generates enthusiasm with managers as
they can make the links between the TQM philosophy
and their business objectives.
• The model helps identify improvement opportunities.
• The model gives a deeper awareness of what a true
TQM company looks and feels like.
• The model has forced managers to find out who is better
than they are at what they do and how they do it.
• The self-assessment process has been effective in
generating real ownership for quality among senior
managers in the business because it forced them to
examine their own activity and develop their own
plans for their own areas in their own way.
Other possible benefits include the following:
• The model promotes a total approach to quality. This
helps to overcome what Lascelles and Dale (1991)
classify as tool pushers, improvers, and drifters. These
organizations don’t have an integrated approach to
quality and often spend most of their time focusing
on improvement teams and quality tools.
• The model provides a guide for all organizational
functions, whereby those operating in strategic or
operational areas can visualize where they are and
where they need to head.
Other research on self-assessment (Hausner and
Arndt 1999) suggests that using quality award models
for self-assessment purposes tends to improve business
VAN DER WIELE, BROWN, MILLEN, AND WHELAN/© 2000, ASQ 11
Improvement in Organizational Performance and Self-Assessment Practices by Selected American Firms
performance for a number of reasons. These include
regular focus on performance data, benchmarking
performance indicators, and the integration of quality
into the strategic planning process.
THE SURVEY
A postal survey was conducted in the greater Boston
region with the endorsement and support of the Boston
Section of the American Society for Quality (ASQ),
formerly known as ASQC. The questionnaire was based
on the version that had previously been used in Europe
and Australia to allow for comparisons.
The questionnaire included a number of items used
by organizations practicing self-assessment. These were
derived from a variety of sources including case study
material from quality award winners in Europe,
Australia, and the United States, interviews with quality
managers in organizations using self-assessment, and
the literature on self-assessment. The questionnaire
was piloted with a sample of organizations in Europe,
Australia, and the United States.
The research sought to: (1) characterize the state of
self-assessment in a sample of U.S. firms; and (2)
determine if self-assessment has contributed to perfor-
mance improvement in firms.
The leadership of the Boston Section provided a list
of its members and cosigned the cover letter asking
for members’ cooperation. Of the more than 2000
members of ASQ in the Boston Section, only those
with managerial responsibilities were sent question-
naires. The resulting list was then examined to ensure
that only one person at any organization was contacted.
This provided a total sample of 640. The sample was
comprised of persons at the managerial level or above.
Specifically, 53 percent held titles of manager, 24 per-
cent held titles of director or corporate director, 12
percent held titles of vice president, 11 percent CEO or
president. Fifty-seven percent had the word quality in
their titles or related wording, such as design assur-
ance, reliability, continuous improvement, or
product assurance.
Nonresponse bias is always a concern in survey
research. One method for testing nonresponse bias is to
test for significant differences between the responses of
12 QMJ VOL. 7, NO. 4/© 2000, ASQ
early and late waves of returned questionnaires
(Lambert and Harrington 1990; Armstrong and
Overton 1977). This method is based on the assumption
that the opinions of late responders are somewhat
representative of the opinions of nonrespondents.
For the present study, the responses from the last
third of the surveys received of those utilizing self-
assessment were compared to the surveys received earlier
across a number of items. The items compared were
the size of the firms, the level of ISO registration
achieved, the level of maturity in quality thinking, the
criteria utilized for self-assessment, and the extent of
improvement attained based on the Malcolm Baldrige
National Quality Award categories. The t-tests yielded
no significant differences among these items. Although
these results do not rule out the possibility of non-
response bias, they do suggest that nonresponse bias
may not be an issue to the extent that late respondents
represent the opinions of nonrespondents.
The study is limited to some extent by the use of a
single respondent from each firm. Social scientists have
long been concerned with the problem of common
method variance because of the bias associated with
using a single informant. Asking a single respondent to
make complex social judgments about organizational
characteristics may increase the subjective propensity of
respondents to seek out consistency in their responses
and increase random measurement error. It is not clear
whether these random error components result from
the reporting process, knowledge deficiencies, inadequate
measures, or some combination of these and other
factors. It is generally concluded that strong assessment
of convergent or discriminent validity cannot be made
when there is a single informant. However, the cost
associated with gaining both participation and consensus
from several individuals from a large number of
organizations is very high.
Research suggests that common method variance
can be mitigated by paying greater attention to infor-
mant selection when practical considerations require
single respondents. In particular, previous studies have
determined that higher-ranking informants are more
reliable than their lower ranking counterparts (see, for
example, Phillips 1981). The managerial level and job
responsibilities (as estimated from their titles) of the
Improvement in Organizational Performance and Self-Assessment Practices by Selected American Firms

sample indicates that this group of managers is in a
position to be aware of their organizations’ efforts in
the area of self-assessment.
In addition, this approach is consistent with recent
exploratory studies of other managerial innovations. For
example, recent empirical studies of the implementation
of advanced manufacturing technology (Ramamurthy
1995; Small and Yasin 1997) and the study of organiza-
tional culture and effectiveness (Denison and Mishra
1995) have employed single informants.
Returns were received from 206 organizations,
representing a response rate of 32 percent. This is
reasonable considering the complex questionnaire
and the sample characteristics, and is better than the
Australian and European response rates (18 percent
and 25 percent, respectively).
Differences Between
Users and Nonusers of
Data on sales turnover for both users and nonusers
are also provided in Table 1. Of companies using self-
assessment, 31 percent had annual sales in excess of
$100 million with only 15 percent of companies using
self-assessment having an annual turnover of less
than $5 million. This compares with 36 percent of
the companies not using self-assessment having the
same turnover, suggesting that self-assessment is more
widespread in companies with higher turnover levels.
The differences are somewhat less apparent when looking
at the number of employees, where self-assessment is
relatively widely practiced in smaller organizations
(less than 100 employees). Thirty percent of those
using self-assessment were in this group as compared
with 45 percent who were not. No statistically significant
difference in either measure of size was found between
the two groups.
Both users and nonusers were asked a series of
questions regarding their quality management pro-
grams in general. The first question was whether their

Self-Assessment
While the main focus of the survey was to identify the Table 1 Size of responding firms.
practices of firms using self-assessment, the authors Percent of firms Percent of firms
were also interested in what differences, if any, existed Number of utilizing not utilizing
employees self-assessment self-assessment
between users and nonusers. Thus, approximately
one-fourth of the questionnaire contained items that Fewer than 51 16% 28%
both groups could respond to effectively; namely, size, 51 to 100 14% 17%
performance, and quality practices in general.
101 to 250 23% 22%
Of the respondents, 71 percent indicated that their
firms were using self-assessment. Of those not using 251 to 500 11% 7%
self-assessment currently, 45 percent reported that their 501 to 1000 12% 3%
firms planned to start self-assessment activities in the
1001 to 5000 16% 18%
next three years.
On average, those firms employing self-assessment More than 5000 8% 5%
had conducted slightly more than 10 formal self- Annual sales turnover
assessment cycles. Approximately half of the respondents
Less than $1M 3% 12%
had been utilizing self-assessment before 1994. Hence,
the respondent group can be considered relatively well $1M to $5M 12% 24%
advanced in this process. $5M to $10M 9% 4%
Employment statistics for both the firms that reported
using self-assessment and not using self-assessment are $10M to $50M 32% 31%

provided in Table 1. Although nonusers tend to be $50M to $100M 13% 8%

© 2000, ASQ

smaller firms, as expected, a number of large organiza-

More than $100M 31% 21%
tions are also represented in this group.

VAN DER WIELE, BROWN, MILLEN, AND WHELAN/© 2000, ASQ 13

Improvement in Organizational Performance and Self-Assessment Practices by Selected American Firms

Table 2 ISO 9000 registration of responding

indicated that less than 20 percent of their company
firms. had been registered. On the other hand, about half of the
users indicated that more than 80 percent of their firm
Percent of firms Percent of firms Percent of firms had been registered. A chi-square test indicated that the
registered to utilizing not utilizing
ISO 9000 series self-assessment self-assessment responses were significantly different ( p
0.05).
The results in Table 2 suggest that, in general, more
Less than 20% 28% 51%
companies practicing self-assessment tend to also be
20% to 40% 8% 0% certified to ISO 9000. Certification may provide the
41% to 60% 7% 9% stimulus for further quality activities such as self-
assessment (Brown and Van der Wiele 1996c) although
61% to 80% 5% 14%

© 2000, ASQ
this is by no means certain. Companies that have a
More than 80% 52% 26% committed approach to quality typically see the adoption
of broad concepts of quality such as TQM and more
process focused approaches like ISO 9000 as comple-
mentary. They are more likely to use ISO 9000 as a basis
Table 3 Responding firms positioning on Crosby’s for ongoing organizational improvement that may be
quality management maturity grid. combined with broader self-assessment processes.
While gaining ISO 9000 certification might normally
Percent of firms Percent of firms
Level of utilizing not utilizing mean that a company is stimulated to continually
maturity self-assessment self-assessment improve its processes and systems, many adopt a mini-
Uncertainty 3% 15% malist approach (Brown and Van der Wiele 1996c)
where certification is gained simply to “get the certificate
Awakening 18% 27%
on the wall.” Many such organizations feel forced to
Enlightenment 61% 48% gain ISO 9000 certification either through market
Wisdom 17% 7% pressure or their suppliers insist on it. Having gained
© 2000, ASQ

such certification, these firms may not undertake

Certainty 1% 3%
firms had quality manuals, and, if so, if the manual
has a reference to self-assessment. As expected, signifi-
cantly more of the users had such manuals than the
nonusers. Of those with a quality manual, users were
three times more likely to have a reference to self-
assessment in their manuals. Specifically, 93 percent of
the users indicated that their firms had a quality man-
ual whereas only 40 percent of the nonusers indicated
that such a manual existed at their firm (difference
significant p
0.01). For those with quality manuals,
75 percent of the users indicated that their manual had
a reference to self-assessment versus only 20 percent for
the nonusers (difference significant p
0.01).
All respondents were also asked to indicate the percent
of their company that had been registered to the ISO
9000 series (see Table 2). About half of the nonusers
regular assessments of their processes, and may simply
focus on meeting the next audit requirement. This
might be a reason for nonusers of self-assessment to
still be certified to ISO 9000.
The next question focused on the managers’ percep-
tion of where their organizations would place on
Crosby’s quality management grid (see Table 3).
Similar to the responses to the question regarding the
extent of ISO registration, users indicated a significantly
greater level of maturity than nonusers did ( p
0.05).
The final questions posed to both users and
nonusers related to the performance of their firms.
Executives were asked the annual trends for sales, market
share, and profits (or return on sales) during the past
three years, and the results are provided in Table 4.
Although no significant differences were found in the
response for annual sales turnover or market share,
more users indicated a positive trend in profits for the
previous three years (difference significant at

14 QMJ VOL. 7, NO. 4/© 2000, ASQ

Improvement in Organizational Performance and Self-Assessment Practices by Selected American Firms

Table 4 Trends for the responding firms over Table 5 Reasons for deciding to conduct
the last three years. self-assessment.
Percent of firms Percent of firms Reason Importance*
Annual sales utilizing not utilizing
tunover self-assessment self-assessment To achieve quality system
registration (e.g., ISO 9000) 4.17
Positive trend 72% 69%
To direct the improvement process 4.16
Remained constant 24% 19%
To manage the business 4.10
Negative trend 4% 12%
To find opportunities for improvement 4.00
Market share
To provide new motivation for the
Positive trend 60% 54% improvement process 3.81

Remained constant 34% 38% To link quality management with

strategic planning 3.71
Negative trend 6% 8%
To strive for cost reduction 3.51
Profits/return on sales
To create focus on the TQM model
Positive trend 72% 52% of management 3.41
Remained constant 19% 22%
© 2000, ASQ

Customers were demanding evidence

of self-assessment 2.64
Negative trend 9% 26%
To benchmark against others 2.62

Internal champion within the unit 2.32

Pressure from headquarters 2.12

p < 0.05). Thus, while both groups reported positive
To stimulate internal competition 1.94
trends in sales, users appear to be obtaining greater
returns from these increased sales. To go for a quality award 1.72

© 2000, ASQ
Competitors were using self-assessment 1.63

Why Self-Assessment?
The importance of various reasons given for using self-
assessment is provided in Table 5. All of the reasons
that were scored at 3.4 or higher are related to the
internal management aspects of working on quality in
the organization; that is, to direct, focus, and motivate
improvement activities. It is also important to see that
quality is linked to strategic processes in organizations.
One interesting aspect of these data is that self-
assessment was seen as linked to achieving ISO 9000
series certification. This may reflect some confusion
about the role of self-assessment against a quality
award model vis-à-vis auditing against the ISO 9000
series standards. The latter involves a type of self
“checklist” in the sense of meeting the standards, yet,
at the same time, the ISO standards also promote a
broader base to quality such as TQM.
* Scores are based on a scale from 1 (not important) to 5 (very important)
External reasons (for example, demanding customers,
competitors using self-assessment, the desire to bench-
mark against others) are scored low by respondents.
Also reasons related to internal competition on quality
issues (for example, creating internal champions,
internal quality award) are given a relatively low score.
Self-assessment provides managers an instrument to
coordinate and give a well-defined direction to all quality
improvement activities going on in the organization. It
helps increase quality awareness in all aspects of the
business, and it is seen as a way to improve business
performance. These were the motivations for introducing
self-assessment that scored highest. Creating focus for
TQM activities and stimulating internal competition

VAN DER WIELE, BROWN, MILLEN, AND WHELAN/© 2000, ASQ 15

Improvement in Organizational Performance and Self-Assessment Practices by Selected American Firms
were two reasons seen as more important ( t-test
statistically significant at 0.05) for larger companies.
Trying to achieve a quality award was not considered
to be a significant reason for using self-assessment
even though the award guidelines were often used as
the basis for self-assessment. This demonstrates the
usefulness of the award criteria for internal self-
assessment purposes. The relatively high profile of such
awards means that they help elevate awareness of
quality management. However, internal championship
and creating best practices for internal benchmarking
are not yet very well developed as part of the initiatives
to carry out self-assessment.
Criteria for Self-Assessment
and Self-Assessment Activities
The extent to which different criteria are used by firms
for self-assessment is provided in Table 6. The two
main criteria used for self-assessment are checklists
and criteria defined within the organization.
Interestingly, the Baldrige Award criteria are not sig-
nificant in this process. From a quality promotion
perspective, promoters of the Baldrige Award should
not see this as a failure, but rather, as a positive result
in terms of heightening quality awareness and wide-
spread recognition that the guidelines serve a purpose
for adapting to individual organizational needs.
Thirty-one percent of respondents were using a
quality matrix for self-assessment purposes. The criteria
used in these matrices are most often strongly related to
the award criteria, with sometimes one or more specific
Table 6 Criteria utilized for self-assessment.
Source of criteria utilized
Checklists (e.g., ISO 9000)
Criteria defined within own organization
Malcolm Baldrige National Quality Award
State or other national quality award
European Quality Award
* Scores are based on a scale from 1 (not utilized) to 5 (fully included)
16 QMJ VOL. 7, NO. 4/© 2000, ASQ
criteria added that reflect the specific situation of the
organization. Organizations perceiving themselves as
further along Crosby’s quality maturity grid were also
more likely to employ a quality matrix and had been
more frequently involved in formal quality manage-
ment self-assessment cycles.
Further analysis of the data showed that companies
with a smaller annual turnover and having carried out
more than three self-assessment cycles tended to make
greater use of a matrix ( p
0.05). More-experienced
self-assessment users also had more criteria in their
matrix. This suggests that less-experienced users tend
to rely on the mainstream Baldrige Award criteria and
not develop their own. The result on company size is
somewhat surprising since the Australian evidence is
that only very large companies generally had the
resources (generally quality specialists or quality
departments) to devote to developing and using a
matrix (Brown and Van der Wiele 1996a).
THE NATURE OF
SELF-ASSESSMENT
Respondents were asked to identify the extent to which
different types of activities formed part of their self-
assessment processes. The results are provided in Table 7.
The top five most common steps selected suggest a
very systematic approach that was used by management
to develop the organization, with particular emphasis
on business unit planning. Self-assessment guidelines
help promote universal agreement on directions
throughout the organization. A further finding is that
the self-assessment process is linked to the business
planning process, which is more widespread among
larger companies.
Further analyses of various subgroups among the
Extent included* sample highlight how some of these features are more
widespread among particular groups (t-test at 0.05
4.42
levels of statistical significance). Larger organizations
4.12 tended to practice the following as part of their self-
2.05 assessment activities reflecting a more formal approach
involving greater time and resources.
1.58
© 2000, ASQ
• Business unit gathered data on all criteria.
1.12
• Assessors from outside the business unit are used,
and these assessors receive training.
Improvement in Organizational Performance and Self-Assessment Practices by Selected American Firms
Table 7 Extent to which activities form part of
self-assessment.
Activities
Business unit employees gather data on
all criteria
Management team of business unit
discusses data gathered on all criteria
Management team of business unit
reaches consensus on strengths and
areas for improvement
Assessors present their findings in
written form to management team of the
business unit
Assessors from outside the organization
are used
Management team of business unit
receives training
Outcomes of the self-assessment process
are linked to the business planning process
Management outside the business unit
monitors improvement targets
Assessors visit the business unit to check
and verify scores
Management team has to present
improvement plan to management outside
the business unit
Report is assessed by assessors from
outside the business unit
Software based checklists produced
internally
Assessors from outside the business unit
receive training
Business unit employees write a full
application report
Assessment meetings with the help of a
computer toolkit
* Scores are based on a scale from 1 (not utilized) to 5 (fully included)
• Assessors visited business units to verify scores and
presented their findings in written form to the man-
agement team of the business unit.
• Management teams presented improvement plans to
management outside their own business unit.
• External management group monitors business
improvement targets.
Extent included*
Other features of self-assessment tended to be found
in organizations experiencing positive trends in profits
and market share. These were
3.83
• Management team of business unit reaches consensus
3.79 on strengths and areas for improvement.
• Management team of business unit discusses data
gathered on all criteria.
3.76 • Outcomes of self-assessment linked to business
planning.
These findings suggest that self-assessment may
3.62
provide specific directions for improvement, which is
then part of the business unit manager’s action plans.
3.48
Self-assessment is also linked to the broader planning
process in these organizations.
3.45
Most of these features also tended to be found in
companies that had progressed further on Crosby’s
3.42
quality maturity grid; that is, they had reached the stage
of enlightenment, wisdom, or certainty. Generally, then,
3.15 these could be regarded as features of self-assessment in
organizations with relatively mature self-assessment
3.04 programs.
Overall, self-assessment appears to have several
characteristics including data gathering and scoring,
2.99
discussing strengths and weaknesses, developing an
improvement plan, and linking it to the business plan.
2.96
These steps promote organizational learning on the
basis of communication and feedback of the self-
2.81 assessment results. Linking the self-assessment results to
the business plan will also support the improvement
2.60 activities, not only because resources will be made avail-
able, but also because the defined improvement goals
2.52 will be an integral part of the regular business review.
© 2000, ASQ
The quality improvement matrix in particular
1.75 provides a methodology for linking quality indicators
into business planning. As noted, the more experienced
a firm was in terms of self-assessment cycles, the more
likely it employed such a matrix. Hence, there appears
to be a level of experience required before firms are
aware of the potential or have the capability to imple-
ment such a methodology.
VAN DER WIELE, BROWN, MILLEN, AND WHELAN/© 2000, ASQ 17
Improvement in Organizational Performance and Self-Assessment Practices by Selected American Firms

Table 8 Extent of improvement regarding Table 9 Extent of improvement regarding

Malcolm Baldrige National Quality specific issues since implementation of
Award categories since implementation self-assessment.
of self-assessment.
Specific issues Extent improved*
Malcolm Baldrige National
Reliability of your operations 3.43
Quality Award categories Extent improved*
Line management better understands
Customer focus 3.71
what TQM is 3.29
Quality of product, process, and service 3.61
Line management better understands
Information and analysis 3.58 importance of TQM 3.28

People 3.42 Errors or defects 3.28

Organizational performance 3.40 Cost savings 3.09

Policy and planning 3.30 Customer complaints 3.03

Leadership 2.99 © 2000, ASQ Return on sales 3.03

Customer retention 2.98

* Scores are based on a scale from 1 (no improvement)
to 5 (considerable Improvement)
Market share 2.98

Cost of [non] quality 2.91

Return on assets 2.90

What Benefits Has Product lead time 2.84

Self-Assessment Given? Inventory turnover 2.80

Analysis of the data on benefits indicates considerable Number of suggestions received

from employees 2.77
improvement on many of the prescribed indicators.
Executives were asked to rate on a 5-point scale the Managers have broadened their own
extent of improvement realized since the implementa- contact network 2.77

tion of self-assessment, where 0  no improvement Internal business units now sharing

and 5  considerable improvement. Out of the seven “best-in-class” practice 2.60

indicators related to the categories of the Baldrige Safety/health of employees 2.50

Award model, “customer focus” scored highest, followed
Employee turnover 2.10
© 2000, ASQ

by “quality of product, process, and service” (see Table

8). All categories apart from leadership scored at 3.3 or Attendance of employees 1.98
above on the 5-point scale, suggesting considerable * Scores are based on a scale from 1 (no improvement)
improvement on these criteria. Improvements in quality to 5 (considerable improvement)

of product, process, and service may also partly reflect

the fact that many of these companies with self-
assessment also have ISO 9000 series quality systems in
place in much of their organizations. The top five items for organizations that reported
A subsequent question addressed the benefits the strongest improvements were
achieved regarding a number of more specific items. 1. Reliability of operations
Seven of the more specific items of improvement 2. Line management better understands what TQM is
resulting from self-assessment scored more than 3 on 3. Line management better understands how important
the improvement index scale (see Table 9). TQM is for the organization

18 QMJ VOL. 7, NO. 4/© 2000, ASQ

Improvement in Organizational Performance and Self-Assessment Practices by Selected American Firms

Table 10 Number of significant correlations ( p
0.05) between level of involvement in self-assessment

activities and level of improvement noted.
Baldrige Award Specific Issues
7 items 19 items
Activity (see Table 8) (see Table 9)

Business unit employees gather data on all criteria 2 12

Assessment meetings with use of computer toolkit 3 19

Business unit employees write full application report 4 9

Assessors from outside the unit receive training 0 3

Report is assessed by assessors from outside unit 3 2

Assessors visit unit to check and verify scores 4 8

Assessors present findings in written form to management team of unit 3 1

Management team has to present improvement plan to management outside unit 1 7

Management outside the unit monitors improvement targets 5 5

Outcomes of the self-assessment process are linked to the business planning process 7 17

Management team of the unit receives training 7 17

Management team of the unit discusses data gathered on all criteria 6 14

Management team of the unit reaches consensus on strengths and areas for improvement 7 16

Software based checklists produced internally 3 5

© 2000, ASQ
Assessors from outside the organization are used 0 0

4. Errors or defects Improvements in reliability, errors, defects, and cost

5. Cost savings
These results support the idea that the self-assessment
process is very useful as a way of defining TQM for the
organization and making it meaningful for all man-
agers in the organization. Managers not only get a better
understanding of what TQM is all about, but also, and
even more crucial, they see the importance for their
business units and job.
Respondents reported only moderate improvements on
issues related to the human aspects in the organization
related to the implementation of quality management
self-assessment. For example, employee turnover, atten-
dance, and the safety and health of employees received
improvement scores of 2.5 or less. This is consistent
with earlier findings (Van der Wiele et al. 1996) that
self-assessment is seen as a management issue and
thus effecting only the management-related indicators.
savings tend to support one of the core principles of
quality management; namely, quality reduces costs.
Better customer retention and fewer customer
complaints is also another core principle of quality
management.
In an effort to gauge the relationship between the
level of involvement in self-assessment activities and
the level of improvement in performance, the correla-
tion between these items was examined as to signifi-
cance. That is, a test of significance was performed on
the correlation between each of the 15 activities and
each of the 26 improvement areas (seven for the
Baldrige Award and 19 more specific areas). The number
of correlations found to be significant is reported in
Table 10.
In general, the results support the contention that
higher levels of self-assessment activity and higher levels

VAN DER WIELE, BROWN, MILLEN, AND WHELAN/© 2000, ASQ 19

Improvement in Organizational Performance and Self-Assessment Practices by Selected American Firms
of improvement are related. Those activities with the
largest number of significant correlations with improve-
ment in the Baldrige Award areas were as follows:
• Outcomes of self-assessment process are linked to
the business planning process.
• Management team of business unit receives training.
• Management team of business unit reaches consensus
on strengths and areas for improvement.
• Management team of business unit discusses data
gathered on all criteria.
• Management outside the business unit monitors
improvement targets.
The results, when the improvement on more specific
issues are examined, is nearly identical. That is, the
activities with the greatest number of correlations
remains almost the same.
Overall, self-assessment appears to have several
characteristics including data gathering and scoring,
discussing strengths and weaknesses, developing an
improvement plan, and linking it to the business
plan. These steps promote organizational learning
on the basis of communication and feedback of the
self-assessment results.
Greater improvements in organizational perfor-
mance (t-test statistically significant at 0.05) had been
realized since implementing self-assessment by compa-
nies experiencing positive trends in profits, market
share, and sales than those who were not. While the
direction of causation is not certain, this gives some
indication that self-assessment may be a process that
impacts positively on organizational performance.
The contributing factors to such enhanced
improvements vary. For companies experiencing positive
trends in profits, they suggest policy and planning,
quality of product, process and service, reliability of
operations, employee suggestions, better understanding
of TQM by line managers, cost savings, and customer
retention among others as reasons. Greater levels of
improvements have also been realized by companies
with positive trends in market share.
20 QMJ VOL. 7, NO. 4/© 2000, ASQ
Larger enterprises, in this case with more than 250
employees, found improvements in the information
and analysis categories and customer focus to a greater
extent than smaller enterprises. In addition, these larger
firms noted larger improvements in the level of errors
or defects and in the sharing of best-in-class across
business units.
CONCLUSIONS
Self-assessment is clearly defined in organizations as
a management issue aimed at increasing quality
awareness, driving the quality improvement activities,
and improving business performance. Many respondents
are familiar with the Baldrige Award model (and others)
and the underlying criteria that help to assess the orga-
nization. Many organizations have adopted these and
other criteria for their internal self-assessment process.
Specifically, this study has found the benefits of self-
assessment to include the following:
• Providing strategic direction on the dimensions of
quality
• Helping to align quality processes and activities
throughout an organization by defining quality in
terms of principles that allows individual operating
units of large organizations to use it as a means of
setting goals and monitoring these
• Developing short- to medium-term targets for the
organization and various business units
• Linking quality to the strategic planning process
• Serving to focus attention on the means of achieving
better organizational performance
Most interestingly, those utilizing self-assessment
reported greater returns on sales than those firms not
utilizing self-assessment.
These findings must be considered tentative in light
of certain limitations to this study. Respondents were
from a limited geographical region; namely, Boston,
which may be considered one of the more progressive
regions in terms of managerial advances. Hence, the
results would not be characteristic of samples conducted
elsewhere. However, this does indicate the need to
examine self-assessment practice in different regions.
Improvement in Organizational Performance and Self-Assessment Practices by Selected American Firms
In addition, even with a response rate in excess of
30 percent, the pool of respondents was still not suffi-
ciently large to conduct highly refined analyses by firm
demographics. For example, the authors were able to
split the sample into “small” and “large” firms, but
not any finer than this. With additional respondents, it
would be possible to examine self-assessment practices
and outcomes across a number of other dimensions,
such as industry type and production process.
Finally, a causal link cannot be established based
on these data. The reported greater return on sales by
those utilizing self-assessment is particularly intriguing.
Yet, it is unlikely that self-assessment alone accounts
for this difference. However, the adoption of self-
assessment may indicate a level of development that
heightens the probability of achieving such results. To
this end, further research is needed of not only those
firms that continue to conduct self-assessment activities,
but also of their nonadopting counterparts. This would
permit the investigation of the link between performance
and self-assessment.
REFERENCES
Armstrong, J. S., and T. S. Overton. 1977. Estimating nonre-
sponse bias in mail surveys. Journal of Marketing Research,
4:396 – 402.
Australian Quality Awards Foundation. 1996. Australian Quality
Awards: 1994 Assessment criteria and application guidelines. St.
Leonards, New South Wales: Australian Quality Awards
Foundation.
Boyce, G. W. D. 1992. Why quality programs aren’t — And how
they could be. Business Quarterly 57, no. 2:57 – 64.
Brown, A., and T. Van der Wiele. 1995. Quality self-assessment:
Managing the quality journey. International Journal of Business
Studies 3, no. 2:109 – 120.
———. 1996a. Quality management self-assessment in Australia.
Total Quality Management 7, no. 3:293 – 308.
———. 1996b. Survey on Australian self-assessment. The Quality
Magazine 5, no. 3:18 – 24.
———. 1996c. A typology of approaches to ISO certification
and TQM. Australian Management Journal 21, no. 1:57 – 72.
Cleary, J., and N. Vogel. 1989. How can progress toward quality
based management be assessed? Paper presented at the First
National Total Quality Management International (TQMI)
Conference, 10 – 11 August, Sydney, Australia.
Crosby, P. B. 1979. Quality is free: The art of making quality
certain. New York: New American Library.
Denison, D. R., and A. K. Mishra. 1995. Toward a theory of
organizational culture and effectiveness. Organization Science 6,
no. 2:204 – 223.
European Foundation for Quality Management. 1993a. Business
improvement through self-assessment: A feedback report on
benchmarking the use of self-assessment by seven member
companies of the EFQM. . Brussels: European Foundation for
Quality Management.
———. 1993b. Self-assessment based on the European model for
total quality management: 1994 Guidelines for identifying and
addressing business excellence issues. Brussels: European
Foundation for Quality Management.
Finn, M., and L. J. Porter. 1994. TQM self-assessment in the UK.
The TQM Magazine 6, no. 4:56 – 61.
Hausner, A., and G. Arndt. 1999. Linking quality management
practices and business performance. Paper presented at the 6th
National Research Conference on Quality Management, 8-10
February, Melbourne, Australia.
Lascelles, D. M., and B. G. Dale. 1991. Leveling out the future.
The TQM Magazine 3, no. 6:325 – 30.
Lambert, D. M., and T. C. Harrington. 1990. Measuring nonre-
sponse bias in mail surveys. Journal of Business Logistics 11, no.
2:5 – 25.
Nakhai, B., and J. S. Neves. 1994. The Deming, Baldrige, and
European Quality Awards. Quality Progress 27, no. 4:33 – 37.
National Institute of Standards and Technology (NIST). 1996.
Malcolm Baldrige National Quality Award: 1996 Award criteria.
Gaithersburg, Md.: National Institute of Standards and
Technology.
Phillips, L. W. 1981. Assessing measurement error in key
informant reports: A methodological note on organizational
analysis in marketing. Journal of Marketing Research 18, no.
11:395 – 415.
Ramamurthy, K. 1995. The influence of planning on
implementation success of advanced manufacturing technology.
IEEE Transactions on Engineering Management 42, no. 1:50 – 61.
Small, M. H., and M. M. Yasin. 1997. Advanced manufacturing
technology implementation policy and performance. Journal of
Operations Management 15, no. 4:349 – 370.
Telstra Corporation Limited. February 1994. Telstra quality
approach: Self-evaluation guide and workbook. 2d edition.
Melbourne, Australia: Telstra Australia, Breakthrough Quality
Management Branch.
Van der Wiele, T. 1995. Pan-European survey on self-assessment.
Paper presented at the European Foundation for Quality
Management (EFQM) Leading Edge Conference, 18-19 May,
Vienna, Austria.
VAN DER WIELE, BROWN, MILLEN, AND WHELAN/© 2000, ASQ 21
Improvement in Organizational Performance and Self-Assessment Practices by Selected American Firms
Van der Wiele, T., A. T. T. Williams, B. G. Dale, G. Carter, F.
Kolb, D. M. Luzon, A. Schmidt, and M. Wallace. 1996. Self-
assessment: A study of progress in Europe’s leading organiza-
tions in quality management practices. International Journal of
Quality and Reliability Management 13, no. 1: 84 – 104.
BIOGRAPHIES
Ton van der Wiele is the co-founder and director of the Strategic
Quality Management Institute at Erasmus University, Rotterdam,
the Netherlands. His specific research interests are in quality
management, especially in developments related to self-assessment
in relation to the internationally, nationally, and regionally
recognized quality award models. He has been involved in
projects sponsored by the European Commission to stimulate
quality management developments in India and Romania. He is
the author of numerous publications on quality management and
general management issues, and has made presentations on his
research throughout the world. He received his Ph.D. from
Erasmus University on the thesis, “Beyond Fads: Management
Fads and Organizational Change with Reference to Quality
Management.”
Alan Brown is professor and head of the School of Management
at Edith Cowan University in Western Australia. His research and
teaching specializations include quality management and human
resource management. He has published papers in a range of
international journals and consulted with public and private
organizations in these areas.
22 QMJ VOL. 7, NO. 4/© 2000, ASQ
Robert Millen is professor of operations management in the
College of Business Administration at Northeastern University,
and is the contact author for this article. He has served as a visiting
faculty member at Monash University (Australia), Groupe HEC
(France), and UCLA. His research, teaching, and consulting
activities focus on implementing and enhancing continuous
improvement programs. He has authored or co-authored more
than 60 articles, and has made numerous presentations on his
research. He continues to consult with a number of organiza-
tions, including several Fortune 1000 firms. He received his Ph.D.
from UCLA, and may be contacted as follows: Management
Science Group, College of Business Administration, Northeastern
University, 314 Hayden Hall, Boston, MA 02115; 617-373-4754;
Fax: 617-373-8628; E-mail: rmillen@cba.neu.edu .
Daniel Whelan, is chair of the Boston Section of the American
Society for Quality (ASQ). He was certified as an ASQ quality
auditor in 1989. Whelan has conducted numerous audits in the
medical device industry and elsewhere, both domestically and in
Europe. He has also conducted training in a number of subjects,
including ISO 9000, quality auditing, and FDA good manufac-
turing practices. Whelan was the 1999 recipient of the Boston
Society Award, a nationally recognized prize for distinguished
achievements in the quality profession. He is an ASQ senior
member, a certified quality manager, and is principal consultant
at Quality Assessment Services, based in Plymouth, MA.