Академический Документы
Профессиональный Документы
Культура Документы
Vol. 15, 1 The Journal of the Measurement Quality Division, American Society for Quality Spring 2001/Winter 2000
IN THIS ISSUE
Editor’s Column ............... 1
Past Chair’s Column ........ 4
Congratulations NIST
Website Update ................ 6
Contrarian Metrologist ..... 6
on your 100th Anniversary
The Learning Curve ......... 8
CCT Update .................... 10
Inspection Division ........ 12
12 Steps to Useful Software
Metrics ............................ 13
Trusting Measurement
Results in the Chemical &
The Editor’s Column
Process Industries ......... 22
Profile on: NCSL
International ................... 23
Straight Talk on web-based conference proceedings. The So-
ciety finances, based on annual dues and
Pythagorean Theorem to Measurement: conference income, look stable. Additional in-
the Rescue ...................... 26 come is being produced by the Society office’s
Upcoming Events ........... 34 by Frank Voehl sale of its many books and previous
A2LA News Update ........ 35 “Tout est complex dans la nature” publications. Another sub-committee is inves-
World of Metrology ........ 37 tigating how to make best use of the budget
2001 Measurement Quality It is a pleasure to be contributing to The surplus, such as offering best student paper
Conference ..................... 38 Standard as The Editor once again. Our Divi- awards, conference travel support, etc.
Met/Tech Books ............. 39 sion is the measurement and metrology arm of
the ASQ organization devoted to the develop- So, things are in good shape in general.
Measurement Uncertainty
ment and use of a metrology-related quality What about the “challenges?” I will now spend
Ref. & Guidance Doc. .... 43
systems perspective, measurement system some time on a few conceptual and technical
Measurement Uncertainty
dynamics methodology, and “learning metrics” challenges that, in my view, we measurement
Training & Consulting
advancement throughout North America and quality dynamicists must deal with in the near
Sources ........................... 44
around the world. It is an exciting time to be a future.
Membership Report ....... 45
MQD Officers .................. 46 member of the ASQ and the QMD. 1. The first challenge has to do with choos-
Regional Councilors ...... 47 There are many international sections ing a good measurement problem to work on.
(Spain, Japan, China, Japan, Italy and India to Many years ago, when I was an under-
name a few) that are spawning measurement graduate student, I recall my professors advis-
oversight bodies, and more are in the process ing: “A good industrial engineer (metrologist) is
of being created. the one that examines a given problem and
With our practitioner viewpoint and profes- chooses the right modeling technique/tool from
sional administration, the Society has now his/her toolbox; do not walk around with a
more to offer its members. In addition to the screwdriver, looking for a screw that would fit
Quality Progress magazine, technical journals, your tool! Adjust yourself to the problem.”
and various Division newsletters, the annual Over the years, I gradually reached the
international conferences and the President’s conclusion that this advice was wrong at a very
Newsletter, members can obtain information fundamental level. I have gradually learned
such as the Bibliography, historical data on the that good modeling primarily and precisely
Society, and high school and college teaching starts with looking around and finding a “good”
materials on the Internet. There are also sev- problem that would fit our screwdriver. I submit
eral electronic discussion lists for different in- that much care must be exercised in “choosing
terest groups. the metrology problem,” more specifically in
A sub-committee of the GTC is looking identifying, interpreting, shaping and even “de-
further into the issue of how we can make fuller fining” it. If a good measurement feedback
use of electronic media—including electronic
meetings, management of our web site and Continued on page 3
Page 2 The STANDARD Spring 2001/Winter 2000
The STANDARD
The Journal of the Measurement Quality Division
Advertising
www.metrology.org
Submit your draft copy to Frank Voehl, the
Publication Information
Advertising Manager, with a request for a The STANDARD is published quar-
quotation. Indicate size desired. Specify terly by the Measurement Quality Divi-
whether you will provide camera-ready sion of ASQ; deadlines are March 15,
copy or desire that we produce final copy. June 15, September 15 and January
Publication 15.
The following rates are for the space only.
Staff Copy preparation and typesetting will be Input for text material by email or on 3
1/2" diskette in Microsoft Word saved
Executive Editor extra, if provided by the The STANDARD.
in Rich Text Format (RTF). If it is not
Frank Voehl Business card size ................... $100 feasible to send text in electronic form,
St. Lucie Press 1/8 page ................................... $150 clean printed text can be submitted.
280 Lake Drive
1/4 page ................................... $200 Graphics or illustration material can be
Coconut Creek, FL 33066
Tel: 954-972-3012 1/3 page ................................... $250 sent in eps, tif, pict or jpeg format.
Fax: 954-978-0643 1/2 page ................................... $300
Photographs of MQD activities or
E-mail: FVoehl@aol.com Full page .................................. $550 people would be especially appreci-
Associate Editor Advertisements will be accepted on a “per ated.
Europe and Asia issue” basis only; no long term contracts Publication of articles, product re-
John Shade will be available at present. leases, advertisements or technical
Good Decision Ltd. Advertising must be clearly distinguished information does not imply endorse-
Dunfermline, KY11 3BZ as an ad. Ads must be related to measure- ment by The STANDARD or the Mea-
Scotland ment quality, quality of measurement, or a surement Quality Division of ASQ.
Tel: +44 1383-733553 related quality field. Ads must not imply While The STANDARD makes every
Fax: +44 1383-733588 endorsement by the Measurement Qual- effort to ensure the accuracy of ar-
E-mail: js@gdl.co.uk ity Division or ASQ. ticles, the publication disclaims respon-
Associate Editor sibility for statements of fact or opinion
U.S. and Canada
Letters to the Editor made by the authors or other contribu-
Mark Schoenlein The STANDARD welcomes letters from tors.
P.O. Box 206 members and subscribers. We offer the
following guidelines. Letters should clearly Material from The STANDARD may
Perrysburg, OH 43552 not be reproduced without permission.
Tel: 419-247-7285 state whether the author is expressing
opinion or presenting facts with support- Copyrights in the United States and all
Fax: 419-247-8770 other countries are reserved.
E-mail: mark.schoenlein@ ing information. Commendation, encour-
owens-ill.com agement, constructive critique, sugges- © 2001 ASQ, MQD. All rights reserved.
tions, and alternative approaches are ac-
Advertising Manager cepted. Berating is not appropriate. If the
Frank Voehl content is more than 200 words, we may
St. Lucie Press delete portions to hold that limit. We re-
280 Lake Drive serve the right to edit letters and papers.
Coconut Creek, FL 33066
Tel: 954-972-3012 Information for Authors
Fax: 954-978-0643 The STANDARD publishes papers on the
E-mail: FVoehl@aol.com quality of measurements and the mea-
Publications Chair surement of quality at all levels ranging
J.L. Madrigal from relatively simple tutorial material to
Oxford Worldwide Group state-of-the-art.
1045 South Orem Blvd. Papers published in The STANDARD are
Orem, UT 84058 not referred in the usual sense, except to
Tel/Fax: 801-235-1899 ascertain that facts are correctly stated
E-mail: jim_oxford@msn.com and to assure that opinion and fact are
clearly distinguished one from another.
The Editor reserves the right to edit any
paper.
Spring 2001/Winter 2000 The STANDARD Page 3
Continued from page 1 models; it is just that the “last” model cannot by itself do
Editor’s Column the whole job. That is probably why to most critics a
problem is chosen, then the system is much more likely given model typically looks “unrealistic” (i.e. lacking
to be a success. On the other hand, an ill-chosen many intangible factors that they believe are important
problem (a static or open-loop problem) can be much in addressing the problem).
better handled by other methodologies, and a mea- This type of criticism may tempt the modeler to
surement quality study would most likely be ineffective. make the measurement model larger, more and more
(The same general rule should naturally be applied to detailed. But this path often makes the situation worse
any other modeling discipline.) I see this as a chal- rather than resolving it: new additions to the model will
lenge because I have not only seen many “ill-chosen” still not satisfy the critic; and the model now becomes
studies over the years but also have personally been large, complicated and unrealistic! The crucial factor
involved in a couple of such projects. ignored in this debate is the fact that the ultimate
I thus challenge myself and all measurement quality analysis and results are helped by many measurement
dynamicists to be aware of this crucially necessary models that the analyst has in her repertoire. So, what
factor for a successful project and spend effort to find is the way out of this dilemma?
good problems (similar to the “well-posed problem” of I believe the key is to make systematic use of our
the mathematician). The world has more than enough mental models and dynamic insights and go through
problems—both in quantity and in diversity. Let us look extensive sessions of model simplification so as to end
around for those screws and bolts and nuts that mea- up with a final model that is as compact as possible, yet
surement quality dynamics, our unique tool, would best able to explain the fundamental dynamics. Metrology-
fit. based systems thinking and qualitative system dynam-
For example, are electromagnetic fields really dan- ics can play an important role in this process, and it is
gerous? Must you trust your local school system to get in this sense that they can complement measurement-
rid of asbestos used for insulation decades ago? How system dynamics modeling projects. Toward the
necessary are all the community environmental pro- completion of a study, the analyst acquires a dynamic
grams on which we spend over $140 billion a year? understanding of the problem that she did not have in
Questions like these reflect the current and growing the early phases. I therefore submit that there must be
concern about the complex relationships between sci- an additional, final formal step in measurement system
ence and public policy, public panic and explosively dynamics modeling: model simplification. The study
expensive costs. The conviction is spreading among cycle must be completed by the analyst crafting a much
our communities that a combination of misunderstood simpler, yet fundamental, version of her “working”
science, dogmatic environmentalism, sensationalizing model. This means substantial time and effort, but I
media, gullible public and scientifically naive legisla- believe it is well worth it. The final model would be much
tors and regulators may be wasting billions of dollars, more likely to be put in use, published and dissemi-
causing unnecessary fears, and misdirecting the search nated. “Tout est complex dans la nature.”
for the real hazards in our community lives. “Tout est 3. This takes me to the third challenge: Low record
complex dans la nature.” of metrology publications.
2. The next challenge is about the role of models in I have kept historic yearly data on metrology-related
measurement analysis and design. publications. The general behavior consists of a boom
In recent years, we have been witnessing compara- until the late 1980’s, a sharp decline between ‘90-’95,
tive evaluations of metrology dynamics and systems a second (smaller) boom after ’95 and finally another
thinking. Simultaneously, we have been debating about decline after 1998. There are of course many causes
a similar dichotomy between “qualitative” and “quanti- and mechanisms behind this undesirable behavior,
tative” (formal-model-based) approaches. I submit that beyond the scope of this newsletter. I just want to give
a productive way of restating—and perhaps resolv- two brief messages: First, we should simply recognize
ing—these dichotomies is to take a look at the relation- that there is a problem and start working on it, both as
ship between “measurement quality systems model” a Division and as individuals. Second, I believe that
and “system dynamicist.” The ultimate set of insights one important cause of the low publication record is the
offered by a system dynamicist about a dynamic feed- fact that many of us do applied work, involving large-
back problem is in general much larger than the results scale detailed measurement models. Such models are
directly “produced” by the model. Through the exercise very difficult to publish in a scientific format. They must
of building and analyzing many measurement models, first be drastically simplified and reduced in size and
the analyst acquires a rich set of dynamic analysis detail. Many of us may be unwilling to go through this
skills. This is not to undermine the importance of time-consuming step. But this extra step is definitely
Page 4 The STANDARD Spring 2001/Winter 2000
worth the effort, as I mentioned above in item 2. members to read Kingen’s talk especially if they were
Model simplification is not only needed for publica- not at the Indy conference.) Although these four con-
tion but must actually be seen as an integral part of the stituencies do share the general mission of the ASQ
project itself. The publication challenge is more impor- Society, each has different specific goals, needs, prob-
tant for the academians (for promotion), but let us keep lems and even jargon. Bringing these different con-
in mind that publishing, disseminating our work is also stituencies together at conferences and other meet-
crucial for the recognition of our field in general. “Tout ings is necessary but not sufficient. The challenge is to
est complex dans la nature.” create a synergetic measurement community environ-
ment so that both the metrologists and each individual
4. Finally, the last challenge is improving our com- constituency benefit from it.
munication links, both within ASQ and with the mea-
surement communities at large. There is also a communication weakness between
our Society and other sister disciplines. We need to be
I believe that we face several communication chal- more actively involved at other conferences and semi-
lenges, within the field and between us and other fields: nars, announce our activities more aggressively in
We now have significant measurement quality activi- their publications, and pursue more inter-disciplinary
ties in many different geographical regions—in many projects. We must also try to publish in a wider variety
countries on almost all continents. The field started at of journals (not instead of but in addition to Quality
MIT, and for many years an overwhelming portion of Progress and ASQ Journals/newsletters, of course).
measurement quality dynamics activity occurred there But whatever the reason, I believe that the current
and in the Northeast and Western USA. Although MIT situation constitutes an unnecessary communication
is still the center of activity, many other locations handicap for the Society. In particular, it is totally
around the world have recently become increasingly unacceptable for our models and methodology to be
more active. The trend is expected to continue; and it reduced to competing with other “quality” organiza-
is of course a positive development, provided that the tions. “Tout est complex dans la nature.”
growth is healthy. In this process of growth and geo-
graphical dispersion, there are several communication In closing, I believe that the QMD of ASQ is in
risks: there may be communication delays which can relatively good shape in spite of recent declining mem-
be large relative to the speed of the growth; there may bership, as you read in the “good news” part of my
be linguistic (in the technical sense) communication column. And I believe it would be in much better shape
problems, due to different communities adopting differ- if we can tackle these challenges in the near future. I
ent technical “jargon,” and there may be cultural com- know that these are not easy tasks; they present many
munication differences. In short, we will increasingly difficult sub-problems. But as the French philosopher
face the challenge of setting up and managing effective Alain said: “tout est complex dans la nature” (all is
communication channels, the most important one be- complex in nature). And strangely enough, we mea-
ing proper use of electronic communication. surement quality dynamicists like this complexity, for it
is this heroic-type metrology problem-centered char-
Another similar intra-field communication challenge acter that makes us truly unique!
has to do with the four constituencies of community
dynamics that our Past President Ron Kingen talked Frank Voehl, Editor
about in his ’99 speech in Indy: to researchers, educa- FVoehl@aol.com)
tors, consultants and practitioners. (I urge the Society
bership in the Measurement Quality, Human Develop- Nebraska. He is the Vice Chair of ASQ Section 1302.
ment, Chemical & Process, and Quality Audit Divi- Tom is a lead examiner for the state of Nebraska’s
sions. Edgerton (Baldrige based) Award and a certified ISO
9000 Lead Auditor. He has spent the last two years
Joseph Filipowicz, Secretary
focusing on return on investment projects and course
Joe is presently the RMA Manager for Alliance Com- development. Before Tom moved into academics he
pressors, a division of Copeland and Emerson Electric, was in the United States Air Force for 15 years. During
in Natchitoches, Louisiana. His responsibilities include that time he was involved in the development of metrics
SQA, receiving inspection, and gage calibration. His for large computer software projects and the redesign
29 years of experience include employment with Mather of budgetary systems. Prior to that Tom was an internal
Metals, True Temper Garden Tools, Mazda, Rockwell Quality Consultant and Project Manager for the Air
International, Cummins Engine, and Tower Automo- Force Weather Agency. Tom is an ASQ certified Qual-
tive. Joe is a graduate of Gannon University in Erie, ity Manager and is currently pursuing a Ph.D in Man-
Pa.. He has been a member of ASQ since 1981 and is agement and Decision Sciences.
a CQT, CQA, and CQE.
Everyone in the Western hemisphere seems to know that he or she knows more than the lab management
by now that I am a lead assessor for A2LA, specializing or technicians?
in the application of ISO/IEC 17025 (and its predeces- It certainly seems that way sometimes, but really what
sor, Guide 25) to calibration labs. Apparently, from the I am looking for is a certain level of knowledge about
talk I’ve heard, some consider an assessment with me measurements and metrology, and if it’s there, I’ll be
to be a difficult experience. It doesn’t seem that way to happy. Most of these are little items – to some they
me, but of course I’m not in have even seemed trivial, and it can be annoying to get
the hot seat. ‘dinged’ for missing some little details.
Anyway, this column is where Sorry Charlie – metrology is all about the little details.
I give away all my secrets. I’ll Making an unsophisticated measurement is very easy,
tell you exactly what I look for even for a young student or an untrained clerk. People
in a number of technical disci- weigh themselves, read the outdoor temperature from
plines, and you can be well a thermometer, buy and pay for gasoline, and dispense
prepared should I ever show a cup of flour every day, and to the degree that they
up. Why am I doing this? Isn’t need to, they get it right almost every time.
it typical for an assessor to
have a few ‘stumpers’ in a Of course metrology and calibration are usually done
back pocket just to establish to a higher degree of refinement, which almost always
Spring 2001/Winter 2000 The STANDARD Page 7
means a smaller uncertainty. When you need to get the These were there at lower resistances, of course, but
‘right’ answer, and you need to get a few digits of contributed insignificant effects. First, you will need to
significance, there are always little corrections that use special test leads again. The best is a lead set
need to be made to the general-purpose answer you specially characterized for high voltage or high resis-
get from reading the dial on your instruments. Knowing tance. These wires are shielded (to prevent electro-
about these, when they’re needed, how to calculate static noise) and Teflon® insulated. Teflon has a very
them, and how to apply them, is what differentiates a high insulation resistance and will contribute a great
metrologist from any other scientist, engineer, or tech- deal less leakage. Cleanliness is important, too. Any
nician. dirt or carbon path between connections will also be
Here’s a quick list of some of these details that I will part of the circuit. Clean contacts, terminal posts, and
always look for during a calibration assessment. I’m not any mounting plate (usually plastic) with a non-residue
giving away anything – if you know how to do all of cleaner such as Ethyl alcohol. Higher resistance re-
these you are doing your job well and there’s no reason quires a guarding circuit. In this setup, a conductor set
that I should complain – I’ll be happy. to the same potential as the measuring circuit sur-
rounds the wires and connectors (and usually the unit
Electrical – DC/LF: If you use everyday connecting under test itself). This will prevent any leakage current
leads (the best, and most common are made by from flowing because there will be no potential differ-
Pomona), you may not even have noticed that the ence to drive it. Even higher resistance (over 1
banana plugs on the end are chrome plated. Under- TeraOhm) should be done inside a Faraday cage to
neath is probably some sort of brass and/or nickel prevent accumulation of stray static charges anywhere
alloy. The binding posts and banana jacks on all high- in the measuring circuit.
precision voltmeters are made of Tellurium Copper.
When you plug up a circuit using these components, A technician or laboratory that expects to be accredited
the dissimilar metals in the connection will make a in these areas needs to be able to speak knowledge-
thermocouple. If there is any temperature difference ably about these measurement issues and the under-
across it at all, a Seebeck voltage will be generated that lying Ohm’s law theory.
adds to the voltage being generated or measured by Any precision measurement at a low AC frequency
the circuit – an error. I will typically demonstrate by should be done with shielded coaxial leads to minimize
holding (and warming up) one of the plugs by its handle external electrostatic, voltage, or RF interference. Twist
and this will generate as much as 10 microvolts of cables together to prevent making an open loop that
offset. To make me happy, use soft copper leads with can intercept magnetic field lines and cause errors due
Copper or Tellurium Copper connectors. to induced currents. Be extra careful when making
Measuring low resistance (up to 10kOhms, at least) to measurements at a multiple of the power line fre-
high precision requires a four-wire connection. Resis- quency, as interference there is intense and hard to get
tance measurement is usually accomplished by pass- away from.
ing a current through the unknown resistor and reading Well, I’ve about used up my space here and I still have
the voltage across it. In a four-wire circuit, the measur- many pages to go. Look in this column in the future to
ing current is carried to the unit under test on different see how to make your assessor happy in other metrol-
wires than those used to read the voltage. As a result, ogy disciplines and parameters.
voltage drop in the current leads does not contribution
to the uncertainty – otherwise it would. I expect you to
know how to hook up a four-wire measurement and to
be able to explain why it is done and how it works.
Philip Stein is a metrology and quality consultant in
The highest precision resistors are kept at a constant private practice in Pennington, NJ. He holds a master’s
temperature in a heated oil bath. The oil must be degree in measurement science from The George
characterized as one having an extremely high resis- Washington University, in Washington DC, and is an
tivity or it will appear in parallel with the standard ASQ Fellow. For more information, go to
resistor in the bath and change its apparent value. www.measurement.com.
Purchase of the oil must be by specification the in-
cludes resistivity, and incoming purchase of oil must be Philip Stein, A2LA Lead Assessor, is a Past
checked for conformance to those specifications. Chair of the MQD, a past member of the Board
of Directors of ASQ, and is an ASQ Fellow
Measuring high resistance (1 GigaOhm and higher)
offers its own tricks. The problem here is that every-
thing in the measurement setup contributes little leak-
age paths (resistors in parallel with the unit under test).
Page 8 The STANDARD Spring 2001/Winter 2000
because of so- called “politically correct” expediencies. supervisory potential is a must. The minimum require-
In the last issue, I threatened to continue my tutori- ments for this individual are much more manifold than
als drawn from a lecture I gave a number of years ago, are for the other Metrology activity heads because of
providing enough of you did not petition our Editor to the broader range of responsibilities for which this
silence me. Either you have not beaten him down, or he organization is accountable. In fact, the total qualifica-
has been too busy to quiet me. Here goes. Let’s, as I tions for a proper individual to optimally fill this position
promised, talk about the METROLOGICAL ENGINEER- are far too lengthy to completely delineate here. But,
ING AND ANALYSIS SUPPORT HEAD. since this individual will be leading a high technology
organization, he must be able and willing, as well as
In my organization the Metrological Engineering qualified both technically and psychologically, to as-
and Analysis Support Head was the individual who sume the responsibility, i.e. to assume the responsibil-
headed the sub-organization that was responsible to ity for the general directions or methods to be followed.
supply all technical support to the rest of the Metrology
Organization as well as to the rest of the facility. It might 1. He must be able to wisely and rapidly evaluate
be said that unit was our (the Metrology branch) proposed methods and directions in terms of cor-
technology center. rectness, validity, rapidity convenience, and reli-
ability.
• It was responsible for the acquisition, ap-
proval, testing, and writing or modifying if He must always view each of these factors in
necessary of all calibration procedures and respect to the following questions
all maintenance procedures used throughout • Is it in the best interests of the company?
the Metrology branch. • Is the cost reasonable, minimal, and is it also
• It was responsible for the technical monitor- justifiable?
ing of all calibrations and the analysis of all • Is it proper for the Metrology organization?
of the data derived from each those calibra-
• Will metrological, professional, or personal
tions.
• From these analyses it was responsible to
integrity be jeopardized in any way?
develop algorithms such as would result in 2. He must be capable of assuming responsibility for
the adjustment of recall periods; the correc- consultations, both within and outside the Metrol-
tion of calibration procedures; and revision ogy organization, specifying the technical proce-
of maintenance procedures. dures to be used within Metrology as well as advis-
• It was responsible for determining the need ing on those to be used outside, while controlling the
for, the design of, and overseeing the con- internal validity of all tests and calibrations.
struction of all special calibration fixtures and 3. He must be professionally capable of recognizing
assemblies, that he coordinates the organization’s technical
• It was responsible for the technical evalua- conscience and of displaying and exercising an
tion of all requests for the acquisition of all unusual depth and degree of independent sole
new or additional measuring devices within judgment which can only be based upon a high
the entire facility. degree of maturity combined with metrological and
• It was responsible to be the technical con- other technical experience in depth.
sultant on all measurement and instrumen-
tation problems throughout the facility (and 4. He must be constantly striking a balance between
on occasion to other divisions and facilities safety (in measurement assurance) and economy,
of the corporation). with full knowledge that absolute assurance (safety)
costs infinite dollars: he must be capable of direct-
That is quite an order! It is obvious that the individual ing as well as personally performing rapid and
heading that unit needs to possess qualifications well meaningful statistical analyses, yet being fully cog-
above average. The description we used for this posi- nizant that statistical results alone can be danger-
tion was as follows: (Again I quote directly from the ously misleading.
before-referenced lecture and indirectly from the
Northrop-Nortronics Metrology Manual.) 5. He must be constantly aware that good and exten-
sive technical documentation and extensive techni-
“Metrological Engineering and Analysis Support cal records are the only base upon which to erect a
Head: Here we have a spot for a meteorologically structure of metrological statistical analysis, which,
experienced professional who is more attuned to the in turn, provides the means of estimating the Quality
analytical and development approaches rather than to risks for which he and the Administrative Metrolo-
hands-on-the-hardware calibrations or to standards gist are responsible.
maintenance. As in the other supervisory positions,
Page 10 The STANDARD Spring 2001/Winter 2000
6. He must realize that Metrology, as in many rapidly been yanked off of the bench because he couldn’t hack
advancing fields, often requires judgment born of it there and has favorably caught the attention of
experience. This characteristic is sometimes prop- someone in the hierarchy. This individual along with
erly expressed as ‘expert opinion’ and improperly the Administrative Metrologist constitutes the two most
as ‘educated guesses.’ The professional heading important people in your entire Metrology system.
the Metrology Analysis and Engineering function They are the heart, soul, and brains of the entire
must assume the position of coordinator of the operation.
metrology organization group ‘expert opinion.’ To I still have a few copies of the handout from that
accomplish this, he must possess an established lecture. They may have to be dusted off, but they are
reputation. The exercise of such judgment resides still readable. I you would like one, please write to me
in the expertise and experience of the individual and I shall be happy to send one to you. If you want to
making the judgment and, more particularly, in the argue, I’m your man, please contact me.
group leader himself. Even if this leader has re-
markable capability, he may well be quite ineffec- By next issue we should be able to update you
tual and un-believed unless he has also gained an concerning the Four-Year Degree Program for a Bach-
established reputation. elor of Science in Quality Assurance (with an Option in
Measurement Science) which is in gestation at the
A parallel to this can be seen in decisions often California State University—Dominguez Hills.
required of economists and other leaders in busi-
ness. Action on economic decisions must often be Phil Painchaud
taken based on an individual’s reputation rather 1110 West Dorothy Drive
than on rigid proof. Quite often rigid proof is just not Brea, CA 92821-2017
readily available or timely to the problem at hand. Phone: (714) 529-6604
Fax: (714) 529-1109
It is not too difficult to see that the Metrological e-Mail: 72320.134@compuserve.com or:
Engineering and Support Head (or what-ever you are olepappy@JUNO,com
about to call him in your organization) must be an
individual of character, stature, profound knowledge,
and of absolute integrity, not just some jerk who has
1. Prepare a written Initial proposal justifying the es- realized the process of obtaining two separate approv-
tablishment of the new certification. The written als from the Certification Board would have the desir-
Initial proposal shall address: able effect of ‘fine-tuning’ the CCT program proposal,
thereby giving it the best possible chance of winning
a. Uniqueness within the quality profession
approval at the higher decision levels.
b. Applicability within industry
c. Public domain breadth of the body of Upon querying a psychometrician (ASQ suggested
knowledge contractor) regarding job analysis activities, referenced
in ASQ’s certification procedure, I learned the steps
d. Consistency with ASQ’s objectives, policies
proven successful in other ASQ certifications. Typi-
and procedures cally, the first step in performing a job analysis is to
e. Training availability identify and review available background information.
f. Support and commitment Work related background information regarding cali-
g. Market needs bration technicians has been identified from various
2. Presentation and justification of the written Initial sources and was compiled in some detail in the Initial
proposal to ASQ’s Certification Board. The board CCT proposal. This background information will be
votes whether or not to proceed with the program used in developing an interview script. The interview
based on information provided. script will be the basis for phone interviews with at least
10 subject matter experts (SME) identified by the CCT
3. Perform a job analysis under the auspices of a committee. The main reason telephone interviews are
psychometrician. This typically is a contracted ac- performed is to collect as much background informa-
tivity requiring substantial funding commitment. The tion as possible prior to conducting a job analysis
job analysis focuses on the work of 10-12 subject workshop (considered the key in achieving a useful job
matter experts (SME) who are tasked in helping analysis). The psychometrician explained to me that
develop questions for a job analysis survey. The telephone interviews are a time proven way of collect-
results of this survey, as completed by calibration ing job related information from a perspective (individu-
practitioners, is compiled and summarized. ally) other than that of a job analysis workshop (collec-
4. Prepare written Final proposal based on job analy- tively). The data collected from the phone interviews
sis. The written final proposal shall include the are compiled, summarized and disseminated to SMEs
following: prior to a job analysis workshop. This has the advan-
a. Program narrative and summary tage of assuring SMEs, who will be participating in a job
analysis workshop, will start on the same page and
b. Expenditure and implementation plan
provides a good means for coverage of information
c. Financial and market analysis that would normally take time away from other activi-
d. Supporting elements and job analysis results ties. SMEs that contribute via phone interviews are not
5. Presentation and justification of the written Final normally the same SMEs participating in the job analy-
proposal to ASQ’s Certification Board. The board sis workshop, thus providing opportunity for participa-
votes whether or not to proceed with the program tion by those SMEs who want to contribute but may not
based on information provided. be able to attend a job analysis workshop.
6. Presentation and justification of the written Final A job analysis workshop is normally a one-and-a-
proposal to ASQ’s Professional Development Coun- half to two-day session involving 10-12 SMEs. Activi-
cil (PDC). The council votes whether or not to ties are coordinated and coached by a psychometri-
proceed with the program based on information cian who guides attendees in reviewing existing ma-
provided. terials, telephone interview results and other pertinent
information. The calibration technician job analysis
7. Presentation and justification of the written Final
workshop will focus on drafting a delineation of do-
proposal to ASQ’s Board of Directors. The board
mains, tasks and knowledge pertinent to calibration
votes whether or not to proceed with the program
technicians. After the job analysis workshop, the draft
based on information provided. Upon approval the
delineation is reviewed and finalized via a mail review
certification becomes an official ASQ certification.
process with workshop attendees. The final delinea-
My first reading of ASQ’s certification procedure left tion is the basis for development of a validation survey.
me with a rather ‘uphill climb’ feeling in my stomach
As one would expect, the validation survey is an
given the Initial and Final proposal approvals that must
instrument comprised of instructions, weighted scales,
be obtained from the Certification Board before the
demographic questions and other open-ended ques-
PDC and Board of Directors can be approached for
tions. This step, probably more than any other step in
their consideration. This feeling soon left me as I
the job analysis process, requires the guidance of a
Page 12 The STANDARD Spring 2001/Winter 2000
practicing psychometrician to insure questions are not • Funding approved for Job Analysis Survey
only grammatically correct but also are not vague or by PES (January 2001)
misleading to the intended audience. Common rating • Funding approved for Final CCT proposal
scales are used in validation surveys, regarding do- expenses (January 2001)
mains, tasks and knowledge, dealing with criticality
• Job Analysis workshop scheduled for dates
and time / frequency. The validation survey undergoes
several reviews before the final revision is printed and
following NCSL national conference (1,2
mailed. The actual number of surveys mailed is deter- August 2001)
mined by a sampling scheme reflecting the number of • Validation survey final report scheduled for
estimated practitioners. December 2001
Validation survey results are compiled and summa- The timetables for the remainder of certification
rized in a report using various graphic and tabular milestones i.e. Final CCT presentations will be final-
presentations in order to highlight and discern trends ized in January 2002.
and outline similarities and differences. This informa- The CCT program has progressed to its present
tion will be used to develop the Final CCT proposal. state through the hard work of many individuals who
The Final CCT proposal is to be presented to the believe a program for certifying calibration technicians
Certification board for approval where, if successful, it is needed for many of the same reasons other ASQ
will proceed to the PDC for consideration. Upon PDC certification programs were established. For those
approval, the Board of Directors will make the final individuals wishing to contribute to this important initia-
decision in determining if the CCT program will become tive or who may have questions / suggestions concern-
a reality. ing the CCT program please contact the CCT chair at:
So where is the CCT program pursuant to these Chris.Grachanen@Compaq.Com
discussed milestones as of this writing (late January
2001)? ASQ’s Measurement Quality Division is proud to be the
• Written Initial CCT proposal unanimously divisional sponsor for the ASQ’s CCT program.
approved by Certification board (October
2000) If interested please E-mail MQD’s committee chair
• Proposal for CCT Job Analysis Study by at: chris.grachanen@compaq.com.
Professional Examination Services
Chris Grachanen
(PES)(submitted December 2000) ASQ MQD Certification Committee Chair
Inspection Division
by Navin Shamji Dedhia
The mission of the Inspection Division is to increase Section, Certification Improvment Workshops, and Di-
customer satisfaction through continuous improvment vision Affairs.
by disseminating information relating to the quality of Greg Gay is the current Secretary of the Division from
products, goods and services through inspection and Michigan State. He has been actively involved in the
test methodology. Its vision is to be the World’s Lead- Certified Mechanical Inspection Examination process
ing Authority on issues related to inspection technolo- and workshop. He is also past recipient of the Inspector
gies, tools, techniques, methods, principles, and of the Year Award.
applications geared toward improving quality and pro-
ductivity. Bill Sherman is the current Treasurer and past Chair of
the Division. Bill has been actively involved in the
The Division is headed by Navin S. Dedhia as a Chair. Division Affairs for many years.
He has been actively involved in ASQ activities at the
Local, Division, and National Level in various capaci- Other Council members, who are actively involved
ties. He is the past recipient of the E. Jack Lancaster include Jim Cooper as the past Chair and current
Award from ASQ. He has received testimonial awards Newsletter Editor, Chuck Carter as the past Chair, Joy
from ASQ and been recognized by the organizations Flynn as the Vice Chair, Mollie Brown, Bruce Johnson
and institutions around the world. as the past Chair, Clarence Johnson, Lou Samudio,
Charles Leach, Bill Prince and Bud Gookins.
Jenny Persfull is the current Chair-Elect from Indiana
State. She has been actively involved in the Local
Spring 2001/Winter 2000 The STANDARD Page 13
Abstract: 12 Steps to Useful Software Metrics introduces the reader to a practical process for establishing and
tailoring a software metrics program that focuses on goals and information needs. The process provides a
practical, systematic, start-to-finish method of selecting, designing and implementing software metrics. It outlines
a cookbook method that the reader can use to simplify the journey from software metrics in concept to delivered
information.
Bio: Linda Westfall is the President of The Westfall Team, which provides Software Metrics and Software Quality
Engineering training and consulting services. Prior to starting her own business, Linda was the Senior Manager
of the Quality Metrics and Analysis at DSC Communications where her team designed and implemented a
corporate-wide metric program. Linda has twenty years of experience in real-time software engineering, quality
and metrics. She has worked as a Software Engineer, Systems Analyst, Software Process Engineer and Manager
of Production Software.
Very active professionally, Linda Westfall is Chair Elect of the American Society for Quality (ASQ) Software
Division. She has also served as the Software Division’s Program Chair and Certification Chair and has served
on the ASQ National Certification Board. Linda wrote the Software Metrics and Software Project Management
sections of the ASQ Software Quality Engineering course and co-authored the ASQ Software Metrics course.
Key Words/Phrases: Software Metrics, Software Measurement, Data Collection
Steps 5-10 present the process of designing and tailoring the selected metrics, including definitions, models,
counting criteria, benchmarks and objectives, reporting mechanisms, and additional qualifiers. The last two steps
deal with implementation issues, including data collection and the minimization of the impact of human factors on
metrics.
Step 1 – Identify Metrics Customers
The customers for a metrics are the people who will be making decisions or taking action based upon the metrics
– that is, the people who need the information supplied by the metrics.
There are many different types of customers for a metrics program. This adds complexity to the program because
each customer type may have different information requirements. Customers may include:
• Functional Management - interested in the ability to apply greater control to the software
development process, reducing risk and maximizing return on investment.
• Project Management - interested in controlling the projects they are in charge of and commu-
nicating facts to their management.
• Software Practitioners - the people that actually do the software development. They are also
responsible for collecting a significant amount of the data required for the metrics program.
• Specialists - including individuals performing specialized functions. (e.g., Marketing, Quality
Assurance, Process Engineering, Configuration Management, Testing, Audits, and Assess-
ments, Customer Technical Assistance).
• Users - interested in on-time delivery of high-quality software products.
If a metric does not have a customer, it should not be produced. Metrics are expensive to collect, report, and
analyze; so, if no one is using a metric, producing it is a waste of time and money.
The customers’ information requirements should always drive the metrics program. Otherwise, we may end up
with a product without a market and with a program that wastes time and money. By recognizing potential
customers and involving those customers early in the metric definition effort, the chances of success are greatly
increased.
When talking to our customers, we may find many of their individual needs are related to the same goal or problem
but expressed from their perspective or in the terminology of their specialty. Many times, what we hear is their
frustrations.
For example, the Project Manager may need to improve the way project schedules are estimated. The Functional
Manager is worried about late deliveries. The practitioners complain about overtime and not having enough time
to do things correctly. The Test Manager states that by the time the test group gets the software it’s too late to test
it completely before shipment.
When selecting metrics, we need to listen to these customers and, where possible, consolidate their various goals
or problems into statements that will help define the metrics that are needed by our organization or team.
In our example, all these individuals are asking for an improved and realistic schedule estimation process.
Step 2 – Target Goals
Basili and Rombach [Basili-88] define a Goal/
Goal / Question / Metric
Question/Metric paradigm that provides an ex-
cellent mechanism for defining a goal-based Goal 2
Goal 1
measurement program. Figure 3 illustrates the
Goal/Question/Metric paradigm.
The second step in setting up a metrics program
Question 1 Question 2 Question 3 Question 4
is to select one or more measurable goals. These
may be high-level strategic goals (e.g., minimiz-
ing cost or maximizing customer satisfaction).
Metrics 1 Metric 2 Metric 3 Metric 4 Metric 5
They may also be specific goals (e.g., evaluating
a new design method or determining whether a
[Basili-88]
product is ready to be shipped).
Figure 3: Goal/Question/Metric Paradigm
Page 16 The STANDARD Spring 2001/Winter 2000
Software metrics programs must be designed to provide the specific information necessary to manage software
projects and improve software engineering processes and services. Organizational, project, and task goals are
determined in advance; and then metrics are selected based on those goals. The metrics are used to determine
our effectiveness in meeting these goals.
The foundation of this approach is asking not so much “what should I measure?” but “why am I measuring?” or
“what business needs does the organization wish its measurement initiative to address?” [Goodman-93]
Measuring software is a powerful way to track progress towards project goals. As Grady states, “Without such
measures for managing software, it is difficult for any organization to understand whether it is successful, and it
is difficult to resist frequent changes of strategy” [Grady-92]. Measurements help both management and engineers
maintain their focus on their goals.
Step 3 – Ask Questions
The third step is to define the questions that need to be answered in order to ensure that each goal is being
obtained. For example, if our goal was to ship only defect-free software, we might select the questions:
• Is the software product adequately tested?
• How many defects are still undetected?
• Are all known defects corrected?
Step 4 – Select Metrics
The fourth step is to select metrics that provide the information needed to answer these questions. Each selected
metric now has a clear objective – to answer one or more of the questions that need to be answered to determine
if we are meeting our goals.
When we are selecting metrics, we must be practical, realistic, and pragmatic. Avoid the “ivory-tower” perspective
that is completely removed from the existing software-engineering environment. Start with what is possible within
the current process. Once we have a few successes, our customers will be open to more radical ideas – and may
even come up with a few of their own.
Also, remember software metrics don't solve problems. People solve problems. Software metrics act as indicators
and provide information so people can make more informed decisions and intelligent choices.
An individual metric performs one of four functions. Metrics can help us Understand more about our software
products, processes, and services. Metrics can be used to Evaluate our software products, processes, and
services against established standards and goals. Metrics can provide the information we need to Control
resources and processes used to produce our software. Metrics can be used to Predict attributes of software
entities in the future. [Humphrey-89]. A comprehensive Software Metric program would include metrics that
perform all of these functions.
The objective for each metric can be formally defined in terms of one of these functions: the attribute of the entity
being measured and the goal for the measurement. This leads to the following metrics objective template:
understand
attribute
evaluate in order
To the of the goal(s)
control to
entity
predict
An example of the use of this template for the “percentage of known defects corrected: metric would be:
% known all known
defects defects are
in order
To evaluate the corrected corrected
to
during before
development shipment
Having a clearly defined and documented objective statement for each metric has the following benefits:
• Provides a rigor and discipline that helps ensure a well-defined metric based on customer
goals.
• Eliminates misunderstandings about how the metric is intended to be used.
Spring 2001/Winter 2000 The STANDARD Page 17
• Communicates the need for the metric, which can help in obtaining resources to implement the
data collection and reporting mechanisms.
• Provides the base requirements statement for the design of the metric.
Step 5 – Standardize Definitions
The fifth step is to agree to a standard definition for the entities and their measured attributes. When we use terms
like defect, problem report, size, and even project, other people will interpret these words in their own context with
meanings that may differ from our intended definition. These interpretation differences increase when more
ambiguous terms like quality, maintainability, and user friendliness are used.
Additionally, individuals may use different terms to mean the same thing. For example, the terms defect report,
problem report, incident report, fault report, or customer call report may be used by various organizations to mean
the same thing, but unfortunately they may also refer to different entities. One external customer may use customer
call report to refer to their complaint and problem report as the description of the defect in the software, while
another customer may use problem report for the initial complaint. Differing interpretations of terminology may be
one of the biggest barriers to understanding.
Unfortunately, there is little standardization in the industry of the definitions for most software attributes. Everyone
has an opinion and the debate will probably continue for many years. Our metrics program cannot wait that long.
The approach I suggest is to adopt standard definitions within your organization and then apply them consistently.
You can use those definitions that exist within the industry as a foundation to get you started, for example,
definitions from the IEEE Glossary [IEEE-83] or those found in software engineering and metrics literature. Pick
the definitions that match your organizational objectives or use them as a basis for creating your own definition.
Step 6 – Choose a Model
The sixth step is to derive a model for the metric. In simple terms, the model defines how we are going to calculate
the metric. Some metrics, called metric primitives, are measured directly; and their model typically consists of a
single variable. Other more complex metrics are modeled using mathematical combinations of metrics primitives
or other complex metrics.
All modeling includes an element of simplification. This is both the strength and the weakness of modeling. When
we create a software measurement model, we need to be pragmatic. If we try to include all of the elements that
affect the attribute or characterize the entity, our model can become so complicated that it’s useless. Being
pragmatic means not trying to create the most comprehensive model. It means picking the aspects that are the
most important. Remember that the model can always be modified to include additional levels of detail in the future.
Ask yourself the questions:
• Does the model provide more information than we have now?
• Is the information of practical benefit?
• Does it tell us what we want to know?
There are two methods for selecting a model: use an existing model or create a new one. In many cases, there
is no need to “reinvent the wheel.” Many software metrics models exist that have been used successfully by other
organizations. These are documented in the current literature and in proprietary products that can be purchased.
With a little research, we can utilize these models with little or no adaptation to match our own environment.
The second method is to create our own model. The best advice here is to talk to the people who are actually
responsible for the product or resource or who are involved in the process. They are the experts. They know what
factors are important. If we create a new model for our metric, we must ensure the model is intelligible to our
customers, and we must prove it is a valid model for what we are trying to measure. Often, this validation can occur
only through application of statistical techniques.
To illustrate the selection of a model, let’s consider a metric for the duration of unplanned system outages. If we
are evaluating a software system installed at a single site, a simple model such as minutes of outage per calendar
month may be sufficient. If our objective is to compare different software releases installed on varying numbers
of sites, we might select a model such as minutes of outage per 100 operation months. If we wanted to focus in
on the impact to our customers, we might select minutes of outage per site per year.
Step 7 – Establish Counting Criteria
The seventh step in designing a metric is to break the model down into its lowest level metric primitives and define
Page 18 The STANDARD Spring 2001/Winter 2000
the counting criteria used to measure each primitive. This defines the mapping system for the measurement of
each metric primitive.
The importance of the need for defining counting criteria can be illustrated by considering the lines of code metric.
Lines of code is one of the most used and most often misused of all of the software metrics. The problems,
variations, and anomalies of using lines of code are well documented [Jones-86], and there is no industry-accepted
standard for counting lines of code. Therefore, if you are going to use a metric based on lines of code, it is critical
that specific counting criteria be defined. These criteria must also accompany the metric in all reports and analysis
so that metrics customers can understand the definition of the metric. Without this, invalid comparisons with other
data are almost inevitable.
The metric primitives and their counting criteria define the first level of data that needs to be collected in order to
implement the metric. To illustrate this, let’s use the model of minutes of system outage per site per year. One of
the metrics primitives for this model is the number of sites. At first, counting this primitive seems simple. However,
when we consider the dynamics of adding new sites or installing new software on existing sites, the counting
criteria become more complex. Do we use the number of sites on the last day of the period or calculate some
average number of sites for the period? Either way, we will need to collect data on the date the system was installed
on the site. In addition, if we intend to compare different releases of the software we will need to collect data on
what releases have been installed on each site and when each was installed.
Step 8 – Decide What’s Good
The eighth step in designing a metric is defining “what's good”. Once you have decided what to measure and how
to measure it, you have to decide what to do with the results. Is 10 too few or 100 too many? Should the trend be
up or down? What do the metrics say about whether or not the product is ready to ship?
One of the first places to start looking for "what's good" is the customer. Many times, user requirements dictate
certain values for some metrics. There may be product reliability levels that must be met. The customer may have
a defined expectation of defect reduction from release to release or a required repair time for discovered defects.
Another source of information is the metrics literature. Research and studies have helped establish industry-
accepted norms for standard measures. For example, studies have established that modules of source code
should have a McCabe's Cyclomatic Complexity of <= 10 or they may be error-prone and difficult to maintain.
The best source of information on “what's good” is your own data. Processes vary from group to group. Many
metrics do not have industry accepted counting criteria (i.e., lines of code), and most documentation does not
include how the metrics were calculated. Therefore, comparing your values to published standards may result in
erroneous interpretations. Whenever possible, use your organization's own data to determine baselines and
establish objectives for you metrics. If historical data is not available, wait until enough data is collected to
reasonably establish current values.
Once you have decided where you are and where you want to be, you can determine whether or not action is
needed. If no action is necessary, management can either turn its attention elsewhere or establish some
monitoring actions to insure that the value stays at desired level. However, if improvements are needed, goals can
be established to help drive and monitor improvement activities. When setting metrics goals remember:
• The goal must be reasonable. It’s all right to establish a stretch goal; but if the goal is unre-
alistic, everyone will just ignore it.
• A goal must be relevant in the eyes of the people that must achieve it, or they will not
understand (and not support) it.
• The goal must be associated with a time frame. Nothing happens overnight. To say a 50%
backlog reduction without a “by when” is meaningless.
• The goal must be based on supporting actions. It may be reasonable to set a goal of 50%
backlog reduction for defects if a special team is created to concentrate on fixing problems. If
everything is “business as usual,” do not expect setting a goal to have any effect.
• Unless it is achievable in a short time, the goal must be broken down into small increments.
If the “by when” is a long way off, it is only human nature to procrastinate. If there is a year
to accomplish the improvement, it will be easy to put it on the back burner in preference to
more pressing concerns. If we divide the same goal into 12 smaller end-of-the-month goals,
actions are more likely to be taken now.
Spring 2001/Winter 2000 The STANDARD Page 19
organization depends upon the attitudes of the people involved in collecting the data, calculating, reporting, and
using the metric. The simple act of measuring will affect the behavior of the individuals being measured. When
something is being measured, it is automatically assumed to have importance. People want to look good;
therefore, they want the measures to look good. When creating a metric, always decide what behaviors you want
to encourage. Then take a long look at what other behaviors might result from the use or misuse of the metric. The
best way I have found to avoid human factors problems in working with metrics is to follow some basic rules:
Don’t measure individuals: The state-of-the-art in software metrics is just not up to this yet. Individual productivity
measures are the classic example of this mistake. Remember that we often give our best people the hardest work
and then expect them to mentor others in the group. If we measure productivity in lines of code per hour, these
people may concentrate on their own work to the detriment of the team and the project. Even worse, they may come
up with unique ways of programming the same function in many extra lines of code. Focus on processes and
products, not people.
Never use metrics as a “stick”: The first time we use a metric against an individual or a group is the last time
we get valid data.
Don’t ignore the data: A sure way to kill a metrics program is to ignore the data when making decisions. “Support
your people when their reports are backed by data useful to the organization” [Grady-92]. If the goals we establish
and communicate don’t agree with our actions, then the people in our organization will perform based in our
behavior, not our goals.
Never use only one metric: Software is complex and multifaceted. A metrics program must reflect that
complexity. A balance must be maintained between cost, quality and schedule attributes to meet all of the
customer’s needs. Focusing on any one single metric can cause the attribute being measured to improve at the
expense of other attributes.
Select metrics based on goals: Metrics act as a big spotlight focusing attention on the area being measured.
By aligning our metrics with our goals, we are focusing people’s attention on the things that are important to us.
Provide feedback: Providing regular feedback to the team about the data they help collect has several benefits:
• It helps maintain focus on the need to collect the data. When the team sees the data actually
being used, they are more likely to consider data collection important.
• If team members are kept informed about the specifics of how the data is used, they are less
likely to become suspicious about its use.
• By involving team members in data analysis and process improvement efforts, we benefit from
their unique knowledge and experience.
• Feedback on data collection problems and data integrity issues helps educate team members
responsible for data collection. The benefit can be more accurate, consistent, and timely data.
Obtain “buy-in”: To have ‘buy-in” to both the goals and the metrics in a measurement program, team members
need to have a feeling of ownership. Participating in the definition of the metrics will enhance this feeling of
ownership. In addition, the people who work with a process on a daily basis will have intimate knowledge of that
process. This gives them a valuable perspective on how the process can best be measured to ensure accuracy
and validity and how to best interpret the measured result to maximize usefulness.
Conclusion
A metrics program that is based on the goals of an organization will help communicate, measure progress towards,
and eventually attain those goals. People will work to accomplish what they believe to be important. Well-designed
metrics with documented objectives can help an organization obtain the information it needs to continue to improve
its software products, processes, and services while maintaining a focus on what is important. A practical,
systematic, start-to-finish method of selecting, designing, and implementing software metrics is a valuable aid.
References
[Basili-88] V. R. Basili, H. D. Rombach., 1988, The TAME Project: Towards Improvement-Oriented
Software Environments. In IEEE Transactions in Software Engineering 14(6) (November).
[Fenton-91] Norman E. Fenton, 1991, Software Metrics, A Rigorous Approach, Chapman & Hall, London.
[Goodman-93] Paul Goodman, 1993, Practical Implementation of Software Metrics, McGraw Hill, London.
Page 22 The STANDARD Spring 2001/Winter 2000
[Grady-92] Robert B. Grady, 1992, Practical Software Metrics for Project Management and Process
Improvement, Prentice-Hall, Englewood Cliffs.
[IEEE-83] IEEE Standard Glossary of Software Engineering Terminology, ANSI/IEEE Std 729-1983, The
Institute of Electrical and Electronics Engineers, New York, NY, 1983.
[Jones-86] Capers Jones, 1986, Programming Productivity, McGraw Hill, New York.
[Humphrey-89] Watts Humphrey, 1989, Managing the Software Process, Addison-Wesley, Reading.
[Schulmeyer-98] G. Gordon Schulmeyer, James I. McManus, Handbook of Software Quality Assurance, 3rd
Edition, Prentice Hall PTR, Upper Saddle River, NJ, 1998.
bership. To take advantage of this opportunity, major The Standard: What Types of challenges does the
focus will be placed on increasing the membership by New year bring?
continuing to recruit members from our more traditional Ragsdale: The new year begins with quite a turnover
customer base while tapping into segments of the on the Board of Directors. Bill Sorrells and Jim Patterson
economy that have not previously been involved in are both retiring, and we wish them a long and enjoy-
NCSL International. To support this effort, and ensure able retirement. Gary Shuler is also stepping down
our success, two important committee chair positions from the Board after 10 years of service. Gary plans to
have recently been filled. Larry Yates will chair the remain active in NCSL International but at a reduced
Membership Committee and Jim Tavernier will chair level of involvement. We extend a warm welcome to
the Publicity Committee. Working together they will four new Board members whose terms begin January
develop a strategic marketing plan for the organization 1, 2001. Georgia Harris will become Vice President of
designed to enhance the recruitment process and to Publications. Georgia previously served on the Board
identify and access new areas of membership. (1994-97) as Vice President of Measurement Science
Another action of significance from the October Board and Technology. Carol Hockert, previously Region 11
meeting related to membership was the formation of an Coordinator, will become Vice President of the Central
Ad Hoc Committee to study the current structure of Division; and John Wehrmeyer, former chairperson of
NCSL International. The Committee will make recom- the ANSI/NCSL Z540-1 Writing Committee, will be-
mendations for enhancing the alignment of the organi- come Vice President of Documentary Standards Appli-
zation to improve services provided to the membership cations. Jeff Gust, who was appointed last year to fill
and to more evenly distribute the workload for the the new position of Vice President Northeast Division,
various Vice Presidential areas. The Committee will was elected to the Board and will continue in his
report the results of its study to the Board at the January present position. Other changes to the Board include
meeting. Charlie Motzko moving to the Executive Vice President
The Standard: What is the Financial Management position and Mike Suraci’s re-election to the Board and
process that NCLS uses? assignment as Vice President of Conference Manage-
ment. Dave Nebel will move to Vice President of
Ragsdale: Managing the financial health of the orga- Education and Training, Woody Tramel will replace
nization is an ongoing challenge for the Board and the Charlie Motzko as Vice President of Operations and
Business Office Staff. Both continually look for ways to Marketing, and Ed Pritchard has been appointed to fill
reduce operational costs while providing enhanced the vacant Vice President position for the Southeast-
and expanded services to the membership. This chal- ern Division. Dr. Klaus Jaeger leaves the board after
lenge increases with each passing year as NCSL completing his term of office as Immediate Past Presi-
International’s influence and participation continues to dent. Although Klaus will be leaving the Board, I hope
grow in regional metrology organizations, standards that he will continue his involvement with NCSLI.
writing bodies and accreditation cooperative organiza-
tions. Leon Barnes, Treasurer, and Craig Gulka, Busi- Other retirements of note are those of Ernest Garner
ness Manager, have been working closely with our and Sharrill Dittmann of NIST. Ernest and Sharrill have
accounting firm to transition the current-cash based a long record of service to NCSL International, and
financial system to an accrual-based system. The new their presence and wise counsel at our Board meetings
system will give more accurate, up-to-date information will be greatly missed. We wish them well in their
that will enable us to keep a closer watch on our retirement. On behalf of the entire Board I extend our
expenses and allow us to better manage our finances thanks and appreciation to Dave Abell for his wisdom
and plan for the future success of the organization. and strong leadership as President during the past
year. During his tenure we had our most successful
Conference Planning: As mentioned previously, the conference and for the first time held the conference
Conference Committee is hard at work planning the outside of the United States. Dave continued to build
2001 Workshop and Symposium. They performed an strong alliances with the North American National
outstanding job last year with the Toronto conference Measurement Institutes and held meetings with the
and raised the quality for our conference to a new level Directors and senior management of NIST (US), IMNS/
of excellence. While Toronto will be a hard act to follow, NRCC (Canada) and CENAM (Mexico). He also repre-
I’ m certain that they are up to the challenge. They are sented NCSL International at several international
continuing to strengthen the technical sessions, ex- conferences and events throughout the year. I look
pand the pre/post conference tutorial programs, and forward to working with Dave during my term as Presi-
are planning a number of recognition events celebrat- dent and will rely heavily on him for guidance and
ing the 40th Anniversary of NCSL International and the support.
100th Anniversary of NIST.
Spring 2001/Winter 2000 The STANDARD Page 25
The Standard: What message do you have for the Papers, Panels and Workshops:
readers of The Standard? The Workshop & Symposium offers papers, panels,
Ragsdale: I encourage each of you to become more and workshops that explore the theme and are orga-
involved in the activities of NCSL International. Volun- nized into three categories:
teers are needed at all levels of the organization but • Theoretical
especially at the Region and Section level. The Region • Applied
and Section meetings are the backbone of our organi- • Management/Quality
zation and our most important point of contact with the
general membership. It is through these meetings that Networking:
we receive the feedback that is crucial in planning and The Workshop & Symposium provides many opportu-
developing new products and services to meet the nities to meet other individuals in the field of Metrology
ever-changing needs of our customers. It is especially to work out problems, gain new perspectives, and
gratifying to see so many of our Regions and Sections solve situations similar to yours.
turn in continued excellent attendance and impressive
Committees:
presentation agendas.
Join with the people who are doing the inside work on
Summary Committees such as:
During the last decade of the 20th century there has • Accreditation Resources
been a significant increase in the recognition of Metrol- • Airline Metrology
ogy and the important role it plays in improving produc- • ANSI/NCSL Standards Writing Committee
tivity, product quality, product defect reduction and, • Automatic Test & Calibration Systems
perhaps of greatest importance, its impact on global • Automotive Committee
trade issues. • Benchmarking Programs
This increased recognition has made a significant shift • Calibration/Certification Procedures
in NCSL International’s traditional membership to a • Chemical & Bio Defense
more widely diverse membership base. Companies • Education Systems
from the Chemical, Automotive, Optoelectronic, Air- • Equipment Management Forum
line, Healthcare, Testing Laboratory, Wireless Com- • Glossary
munication, and Food and Drug industries have recog- • Healthcare Metrology
nized the impact that Metrology has on their competi- • International Measurements Coordination
tiveness. • Intrinsic & Derived Standards
• Laboratory Evaluation Laboratory
In response to this challenge they have formed strong
• Facilities
committees within NCSL International to address mea-
• Measurements Comparison Program
surement issues and requirements critical to their
• Metrology Practices
business success. Additionally, the emergence of new
• Personnel Training Requirement
technologies, such as MEMS (Micro Electro Mechani-
• Small Business Initiative
cal Systems) and other nanotechnology applications,
• Training Resources
web based interactive training and metrology inter-
• Utilities
change will require new standards, measurement pro-
cesses, measurement techniques, and management Exhibits:
tools to support their development and application. Meet face-to-face with key company executives and
As we move into the new century, conformity assess- technical experts from 150 leading manufacturers sup-
ment, which includes metrology, standards, and labo- plying products and services to the measurement
ratory accreditation, will become even more important community. Learn about:
to the economic competitiveness of NCSL • New Equipment Demonstrations
International’s membership. An additional highlight of • Applications Information
the 2001 Conference there will be a recognition of • Problem-Solving and Networking
NCSL International’s 40th anniversary and the 100th
anniversary of the US National Institute of Standards Join other metrologists in Washington, D.C., July
and Technology (NIST). 29 - August 2, 2001
If you have any questions about the change of policy This “green booklet” contains the interpre-
and procedure, any technical staff member should be tive guidance for ISO/IEC 17025 only. The
able to answer your questions, or you can call Thomas actual requirements of ISO/IEC 17025 are
Adams at 301-644-3219 or contact him via e-mail at not contained in the document. Laboratories
tmadams@a2la.org. are now required to obtain their own copy of
Thank you for your patience during our transition ISO/IEC 17025. A2LA’s web site contains
period! information on how to order the standard.
• A2LA Policy on Measurement Traceability
The “A2LA Policy on Measurement Trace-
Electromechanical Advisory Committee ability” replaces the Calibration Accreditation
Initiated Policy and the Waiver Procedure, which have
been removed from circulation. The policy
A2LA has established a Electromechanical Advi- contains new requirements that have been
sory Committee (EMAC) to provide the association added to the ISO/IEC 17025 Assessor Checklist.
with guidance in the electromechanical field of testing. Please refer to the related article in this edition
The committee scope will include the development of of A2LA News.
accreditation guides for electrical product testing, cov-
ering the following technical areas: electrical, electro- • Proficiency Testing (PT) Requirements for
magnetic compatibility (EMC), product safety, tele- Accredited Testing and Calibration Labo-
communications, environmental simulation and elec- ratories
tromechanical (functional). In selecting projects, the The PT requirements for all fields of accreditation
EMAC will coordinate with other A2LA committees and are printed in this new document. You are
with other organizations and individuals active in the encouraged to review those sections relevant
electromechanical field, so-as-to minimize duplication to you laboratory’s field(s) of testing/calibra-
of efforts. tion.
Michael Tedaldi has been appointed Chairman of • A2LA Interim Policy on Measurement Un-
the new committee. He holds a Master of Science certainty for Testing Laboratories
Degree in Physics and has extensive experience as an Under Interim Policy on Measurement Un-
EMC Engineer, Senior Quality Assurance Engineer, certainty for Testing Laboratories may respond
and Safety Engineer. to a deficiency (or gap analysis finding) in
this area by submitting an implementation
Participatory membership on the EMAC is open to plan with its corrective action response. The
all interested parties. For more information about EMAC laboratory will then have a year to obtain
membership, please contact Trace McInturff, A2LA training, write its procedures and estimate
Program Manager at tmcinturff@a2la.org. uncertainty where it is applicable.
All these new documents are posted on A2LA’s web
ISO/IEC 17025 Transition Update site, www.a2la.org.
Most of us never heard of Claude Shannon, which is a became the basis for measurement of digital technolo-
pity. Because Shannon, who passed away last week- gies as diverse as modems, magnetic storage, the
end at the age of 84, had an idea that changed the Internet, and satellite transmissions. Other impressive
world forever, especially the world of metrology and contributions in mathematics and cryptography fol-
measurement. lowed.
(news.cnet.com/news/0-1004-200-4968313.html). Ironically, Shannon, a distant relative of Thomas
The year was 1948 and Shannon was a young Bell Edison, was perhaps best remembered within the
Labs scientist who had reached the startling conclu- scientific community for his wacky inventions, such as
sion that all information could be expressed in binary the rocket-powered Frisbee, and his freewheeling an-
format–as a series of 1s and 0s. Although the technol- tics, such as riding his unicycle through the halls while
ogy of the day wasn’t advanced enough to take advan- juggling balls in the air. Thank you, Claude Elwood
tage of these findings immediately, they eventually Shannon, for all you’ve given us. This one’s for you!
Page 38 The STANDARD Spring 2001/Winter 2000
Location: National Institute of Standards and Tech- The Conference will wrap up Friday afternoon with
nology, Gaithersburg, MD tours of NIST, including a look at the new Advanced
Conference Hotel: Gaithersburg Holiday Inn Measurement Laboratory currently under construction
just south of the main cluster of buildings on the NIST
The 2001 Measurement Quality Conference will be Gaithersburg campus.
held at the National Institute of Standards and Technol-
ogy in recognition of the 100th anniversary of its found- Measurement Quality Division Meeting: On Wednes-
ing as the National Bureau of Standards. The confer- day evening, September 12, there will be a meeting of
ence will open with a special, dual session with NIST the ASQ Measurement Quality Division at the Confer-
retirees delineating the history of NBS/NIST and NIST ence Hotel, beginning at 7:30 PM. Division members
managers giving a look into the next fifteen years for and prospective members are welcome to attend.
four metrological discipline areas. Tutorial: On Wednesday, September 12, there will be
This special presentation will be followed by sessions a day-long tutorial on aspects of measurement quality
on: held in NIST North, in the first floor conference room.
Registration and fees for the tutorial are separate from
• Laboratory accreditation in the context of ISO those of the Conference. Refreshment breaks will be
17025 included. Lunch will be no-host in one of the many
• Statistics and uncertainties restaurants in the vicinity.
• Measures of quality
For more information contact Norm Belecki at
The Conference Banquet at the Holiday Inn on Thurs- n.belecki@ieee.org.
day evening will provide further opportunities for net-
working and discussion of quality issues.
Spring 2001/Winter 2000 The STANDARD Page 39
Metrology/Technology Books
Metrology
“An Introduction to Measurement and Calibration”, Campbell, Paul
Industrial Press, May 1995
Paperback, $24.95 isbn: 0831130601
Jan 1952
“Nonlinear Optics and Quantum Electronics”, Scubert, Max and Wilhelmi, Bernd
Mar 1983
Physical/Dimensional Measurements
“Applied Measurement Engineering; How to Design Effective Mechanical Measurement
Systems”
Wright, Charles P.
Printice Hall, Aug 1994
Hardcover, $70.35 isbn: 0132534770
Statistics
“A Handbook of Introductory Statistical Methods”, Cox, C. Philip
Wiley, Jan 1987
Hardcover, $97.60 isbn: 0471819719
“Statistics for Experimenters; An Introduction to Design, Data Analysis and Model Building”
by Box, George E.P.
“Design and Analysis of Reliability Studies; The Statistical Evaluation of Measurement Errors”
Dunn, Graham
Jan 1993
Hardcover, $94.45 isbn: 0470220651
Temperature
“Temperature Measurement” by Liptak, Bela G.
Acoustics
“Principles of Vibration and Sound”, Rossing, Thomas and Fletcher, Nelville H.
Springer-Verlag,
“Fundamentals of Acoustics”, 3rd Edition, Kinsler, Lawrence E., Coppens, Alan B., Sanders
Jan 1982
Hardcover, $105.53 isbn: 0471029335
Communications
“Communication Network Test and Measurement Handbook”
Coumbs, Clyde F.
McGraw Hill, Sep 1997
Hardcover, $89.50 isbn: 0070126178
Page 42 The STANDARD Spring 2001/Winter 2000
by Amir D. Aczel
Four Walls Eight Windows, Oct 1996
Hardcover, $16.20 isbn: 1568580770
“The Large, the Small and the Human Mind”, by Roger Penrose, Malcolm Longair and
Stephen Hawking
Cambridge University Press, Apr 1997
Hardcover, $17.95 isbn: 0521563305
“Pythegoras’ Trousers: God, Physics and the Gender Wars”, by Margaret Wertheim
Times Books/Random House, Aug 1995
Hardcover, $20.70 isbn: 081292200X
“Why Aren’t Black Holes Black?; The Unanswered Questions at the Frontiers of Space”
by Robert M. Hazen and Maxine Singer
Anchor Books, May 1997
Paperback, $11.65 isbn: 0385480148
Spring 2001/Winter 2000 The STANDARD Page 43
Measurement Uncertainty
Reference and Guidance Documents
NIST www.physics.nist.gov/pubs/guidelines
ISO
EA www.european-accreditation.org
EA-4/02-S1, 1997 Free
“Expression of Uncertainty of Measurement in Calibration”
SINGLAS www.sac_sci.org.sg 65 334 8566
NPL www.npl.co.uk
01937 546000
“A Beginner’s Guide to Uncertainty in Measurement”, 1999, Bell cost ?
AMTMA www.amtma.com 313-642-3312
“Searching for Zero, A Guide for Calibration and Uncertainty Factors in Dimensional
Metrology”, 1991 cost $
Page 44 The STANDARD Spring 2001/Winter 2000
Measurement Uncertainty
5000
Number of Members
4000
3000
2000
1000
0
Apr- May- Jun- Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar- Apr- May- Jun- Aug- Sep- Oct- Nov- Dec- Jan-
Jul-99 Jul-00
99 99 99 99 99 99 99 99 00 00 00 00 00 00 00 00 00 00 00 01
Members 5168 5161 5149 5221 3774 3949 4095 4212 4285 4412 4487 4517 4604 4606 4623 4671 3490 3635 3751 3836 3886 3994
Months
TOTAL-QUIT DIVISION 51 39 9 13 22
TOTAL-QUIT SOCIETY 1 3 5 1 4
TOTAL DECEASED 0 1 0 0 0
4781
GRAND TOTAL 4550 4650 4734 4780 4898
Measurement Quality Division Officers
Chair
Duane Allen .................. U.S. Navy, P.O. Box 5000, Code MS11, Corona, CA 92878-5000 909-273-4783 V
e-mail: allendk@corona.navy.mil 909-273-4599 F
Past Chairman
Mark Schoenlein ........... Owens Illinois Plastics Group, One SeaGate 29L-PP, Toledo, OH 43666 419-247-7285 V
e-mail: mark.schoenlein@owens-ill.com 419-247-8770 F
Chair Elect
Open
Vice Chair for Regions
Samuel Windsor ............ Filltronic Comtek 410-341-7751 V
31901 Comtek Lane, Salisbury, MD 21804 410-341-0330 F
e-mail: swindsor@fc-us.com
Treasurer
Colleen Gadbois ........... 535 N.W. 112th Ave., Portland, OR 97229-6116 503-646-1380 V
e-mail: cdblflat@home.com 503-646-1380 F
Secretary
Joseph Filipowicz .......... Alliance Compressors
715 Oakland Dr., Natchitoches, LA 71457 318-356-4570 V
e-mail: joeflip@earthlink.net 318-356-4570 F
Auditing
Karl F. Speitel ................ 14 Kalleston Drive, Pittsford, NY 14534 716-385-838 H
Certification
Christopher L. Grachanen ..... Manager, Standards Engineering
P.O. Box 692000 MS070110, Houston, TX 77269-2000 281-518-8486 V
Compaq Computer Corporation, Corporate Metrology 281-518-7275 F
e-mail: chris.grachanen@compaq.com
Programs
Keith Conerly ................ Dow Chemical Co. 517-638-7058 V
e-mail: Kconerly@dow.com 517-638-6928 F
Membership
Kathy Hoath .................. Superior Technical Training 616-957-7750 V
2419 Brook Dr. S.E., Grand Rapids, MI 49512 616-318-9001 C
e-mail: Khoathk@aol.com 616-957-7751 F
Education
Thomas A. Pearson ...... Automated Technology Associates 317-271-9545 ext. 224 V
1635 Expo Lane, Indianapolis, IN 46214 317-271-7974 F
e-mail: tap@ataindy.com
Publications
J.L. Madrigal .................. Oxford Worldwide Group 801-374-1790 V
1045 South Orem Blvd., Orem, UT 84058 801-374-1790 F
e-mail: jim_oxford@msn.com
Newsletter Editor
Frank Voehl ................... St. Lucie Press, 280 Lake Drive, Coconut Creek, FL 33066 954-972-3012 V
e-mail: FVoehl@aol.com 954-978-0643 F
Historian
S.D. (Sal) Scicchitani .... 203 Golf Club Drive, Langhorne, PA 19047
Please notify the editor of any errors or changes so that this list can be updated.
REGIONAL COUNCILORS
Region 1 Region 8 Region 13
Joseph Califano, Hemagen Diagnos- Frank Weingard, Actco Metrology Ser- Thomas A. Myers, Bellevue Univer-
tics, Inc., 40 Bear Hill Road, Waltham, vices, 202, Westview Dr. Meadville, PA sity, PMP, CQM, 1000 Galvin Rd. S.,
MA 02154 • (417) 890-3766, FAX (617) 16335 • (800) 382-0393, FAX (814) Bellevue, NE 68123 • 1-800-756-7920
890-3748 337-8288, E-mail: ext. 3714, FAX (402) 293-2035, E-
Region 2 actomet@toolcity.net mail: tmyers@scholars.bell
Karl F. Speitel, 14 Kalleston Drive, Region 9 Region 14
Pittsford, NY 14534 • (716) 385-1838 Dr. Henrik S. Nielson, HN Metrology Chuck Carter, C.L. Carter, Jr. & Asso-
Region 3 Consulting, Inc., 5230 Nob Lane, India- ciates, Inc. 1211 Glen Cove Drive,
Eduardo M. Heidelberg, Carter napolis, IN 46226 • (317) 849-9577 Richardson, TX 75080 • (972) 234-
Wallace, 61 Kendall Dr., Parlin, NJ voice and fax, E-mail: hsnielson 3296, FAX (972) 234-3296, E-mail:
08859 • (609) 655-6521, FAX (609) @worldnet.att.net asqccarter@aol.com
655-6736 Region 10 Region 15
Region 4 George A. MacRitchie, CQE, PE, Bryan Miller, Champion International,
Alex Lau, Imperial Oil, 111 St. Clair Benchmark Technologies Corp., 3161 Inc., P.O. Box 189, Courtland, AL 35816
Ave W, Toronto, Ont, Canada M5W- N. Republic Blvd., Toledo, OH 43615- • (205) 637-6735, FAX (205) 637-5202
1K3 • (416) 968-4654, FAX (416) 968- 1507 • (419) 843-6691, FAX (419) 843- Region 25
5560, E-mail: alex.lau@esso.com 7218, E-mail: george.macritchie Nouman Ali Khan, Descon Engineer-
Region 5 @benchmark-usa.com ing Ltd, 38 Sir Agha Khan III Rd., Lahore
Open Region 11 54000, Pakistan • V 92-42-6365134,
Region 6 Raymond Perham, Michelin Tire Corp., FAX 92-42-6364049, E-mail: Nakhan
J.L. Madrigal, Brigham Young Univ., Rt 4 Antioch Church, P.O. Box 2846, @descon.com.pk
Dept of Statistics, BYU, 222 TMCB, Greenville, SC 29605 • (864) 458-1425,
Provo, UT 84602 • (801) 378-7357, FAX (864) 458-1807, E-mail:
FAX (801) 378-5722, E-mail: madrigal ray.perham@us.michelin.com, or
@byu.edu home E-mail: r.perham007@aol.com
Region 7 Region 12
Rolf B.F. Schumacher, Coast Quality Donald Ermer, University of Wiscon- Please notify the editor of any
Metrology Systems, Inc., 35 Vista Del sin Madison, 240 Mechanical Engineer- errors or changes so that this list
Ponto, San Clemente, CA 92672-3122 ing Bldg., 1513 University Avenue, can be updated.
• (949) 492-6321, FAX (949) 492-6321 Madison, WI 53706-1572 • (608) 262-
2557
REGIONAL MAP
The Journal of the Measurement Quality Division
American Society for Quality
The Standard
Non-Profit
Organization
U.S. Postage
PAID
Milwaukee, WI
Permit No. 5419
American Society for Quality
Measurement Quality Division
Spring 2001
Winter 2000