"USER" - A Four Letter Word

In most situations where ethical problems arise, there are a variety of contributing
factors from the lack of knowledge of the customer to the desire for unreasonable
profits. In the above cases , when we focus on those factors which are in the software
engineer's control, we see a single pattern that significantly contributes to the ethical
problems.

Each case in a case-pair raises similar ethical questions. The first case-pair points to
problems generated by a failure to employ professional skills and knowledge about
interface design and testing. The second set of cases deals with problems that may
arise in modifying a complex system "on the fly." We know that any unanticipated
maintenance tends to degrade a system. And we know that maintenance done quickly
on a very complex system has a greater potential to produce a highly degraded
(failing) system. The third set of cases addresses the failure of the ethical commitment
to adhere to professional standards. In these cases, this failure is based on a non-
ethical issue --economics-- which nevertheless effects our judgement.

The ethical concerns of the second case of each pair of cases(the b. cases) seems more
obvious because they each involve a potential risk to human life. The question of
safety is one of judgements about risk. Risk is generally defined as a measure of the
probability and severity of harm to human heath and a system is considered safe if its
risks are judged to be acceptable. These cases are easily recognizable as causes for
ethical concern because they involve the potential for physical harm. The degree to
which this potential is realized is tied to the quality of the software engineer's
professional judgement.

The first item of each case pair (the a. cases) seems more difficult to deal with. They
do not contain direct threats to human life. It is a mistake, however, to equate ethics in
software engineering just with the threat to human life. The a. cases deal with a threat,
not a threat to human life, but a threat to human well-being. Case 2.1a limits a
manager's freedom to improve his business, and case 2.2a and 2.3a are threats to the
continued existence of the customer's business. These threats are ethically significant,
but are often minimized because they are not issues of safety. The a. cases, just like
the b. cases, involve a significant element of skilled professional judgement. The
events described also have the potential to negatively effect several lives. The major
difference between the a. cases and the b. cases in each sets is that the b. cases have to
be handled quicker. The a. and b. cases contain similar ethical issues involving
professional judgement. Problems like those described in the a. cases arise when
professional judgements fails or is compromised.
I think the pivotal form of professional compromise in these cases is the failure to
keep our responsibility to the user in mind as we develop software artifacts. We have
a responsibility to develop the artifact using the best of our skills, but we also have a
responsibility to develop a usable quality product for the user which will not
negatively effect their well-being. There was little concern for the user in case 1a
where the contractual requirements, about which information to display, were met.
The contractor probably violated no laws. He probably would not be found guilty of
negligence by a court, nor do I think he be would found guilty under tort law --
dealing primary with non-criminal harms. Nevertheless, there is moral culpability
because of a failure to meet his professional responsibility to the user. This moral
culpability does NOT require the intent to harm the user. It merely requires the
conscious decision by the practicing professional not to fulfill the professional
responsibilities to the user. It is this single common failure which significantly
contributes to the ethical problems.

Professional concern for the user is evidenced in our attempt to reduce threats in the
software and our commitment to produce a quality product based on an implicit
agreement with the user to employ our software engineering skills. Unfortunately,
when we do think about the user, almost all of our attention is directed toward threats
to human life. We make a mistake when we ignore threats to human well-being. It is
also a mistake to minimize our standards without first considering its impact on the
user.

The safety issues give us some insight into a broader ethical responsibility which is
not defined by law, but which is defined by our professional standards and our
realization that computing products should be designed with the welfare of the user in
mind.

3.2 Frank S. Marchilena

Raytheon Company, Missile Systems

Software Laboratory Manager
50 Apple Hill Drive
Tewksbury MA 01876-0901

Case 3: Professional Judgement - Experimental fighter flight control software.

Case three has several interesting aspects that must be considered from an ethical
point of view. There is the responsibility of the customer (government in this case) to
provide or accept a set of test cases which will lead to a product that will perform its
intended functions in a safe manner. The second responsibility belongs to the
company which accepted the contract to develop the flight control software. Thirdly,
there is the responsibility of the engineers who are developing the software.

The government must accept its responsibility as the customer and the project
manager of advanced development contracts. The government must:

provide and/or accept responsibility for

the requirements of the system;
ensure that the contractor has the
ability to perform the contract;
ensure that the contractor performs the
contracts.

In case 3 a set of requirements have been developed and accepted by all parties
involved. Likewise a set of tests have been accepted by all parties involved. It is the
customer's (government's) responsibility to see that the software meets all
requirements. In addition the government must provide the necessary oversight to be
assured that all of the tests were performed and that the results prove that all
requirements have been met and that the flight control software will perform its
function.

A company is responsible for providing the climate in which an engineer can perform
her/his duties in a manner which conforms with accepted professional standards. This
can be accomplished in the following manner:

the company should provide a written standard of moral conduct it expects

from its employees in the performance of their
assignments;
the company should provide a written set of engineering development standards
which areexplicit in how a job is to be performed;
the company must provide a means by which an employee can express their
"moral" disapproval for the way the company is
conducting a certain business transaction.

The establishment of an environment in which employees are aware of their moral

duties and obligations is the duty of all companies.

The employee (the quality control engineer) in this case must perform to standards of
the company and the contract. If these standards are not met then it is her
responsibility to report the issue no matter what the consequences are.
The interesting aspect of this case is when all standards of performance have been met
yet the quality control engineer still believes that the software is insufficiently tested.
In this situation the quality control engineer has the responsibility to seek other
professional judgement on the subject. It is her obligation to prevent the software from
being used until she is satisfied that it will perform.