ACM SIGSOFT Software Engineering Notes

Page 1

July 2005

Volume 30 Number 4

Software Engineering Research versus Software Development

Esperanza Marcos
KYBELE Research Group
Universidad Rey Juan Carlos
{esperanza.marcos }@urjc.es
Abstract
Engineering research differs greatly, both in its aims and in its
methods, from traditional scientific research. While Sciences
deal with the study of existing objects and phenomena, be it
physically, metaphysically or conceptually, Engineering is based
on how to do things, how to create new objects. For this reason,
scientific research methods are not always directly applicable to
research problems of an engineering nature.
In the present article, we concentrate on the problems and research
methods of a specific branch of engineering: Software
Engineering, discussing, on the one hand, the nature of the method
in this field while and, on the other, the similarity of the methods
of research in Software Engineering and those of software
development.
Key words: Software Engineering Research, Software
Development, Science, Engineering.
1. Introduction
From the 16th century on, a major advance took place in the
development of science, which also affected research methods and
the criteria for verifying theories. New research methods thus
arose with their origin in Galileos proposals, which were more
suitable for the scientific studies of the day (astronomy, medicine,
mathematics and physics). These methods may be empirical
(inductive and hypothetico-deductive), representing a radical
change with regard to the scholastic methods, or deductive, as
proposed by Decartes, who rejected scholastic logic and developed
new mathematical methods. Nevertheless, since the 16th century,
knowledge has undergone major advances, including the
development of new disciplines like engineering and, with
relevance to this study, Software Engineering (SE). The nature of
engineering knowledge is essentially different from that of the
traditional sciences. Therefore, just as in the 16th century new
methods of research arose in line with the science of the period, it
is now necessary to define others that are applicable to the specific
problems of SE.
There are many and varied classifications of sciences [4], based on
different criteria. Depending on the kind of science, different
research methods are used. None of these methods, however,
seems to be totally suited to SE research. As is proposed in [9], the
branches of engineering do not fit perfectly into the classifications
proposed in the literature, although they are related with most of
the disciplines appearing in them. For this reason, the search for a
method appropriate to SE research and its application to the
development and implementation of Information Systems (IS), is
becoming a research field in its own right [6], [7], [8], [9], [11].
But the method greatly depends on the object of study. Our initial
hypothesis is that the object of study of the branches of
engineering (and SE in particular) differs from the object of study
of the formal and empirical science (be they sociological or

natural). While these subjects deal with the study of existing

phenomena or objects, engineering sciences deal with the study of
the methods and techniques necessary for the creation of new
objects and even with the creation of such methods and techniques.
Insofar as SE research is concerned with creation (of new
techniques and methods), it has important similarities with the
application of SE, which is also concerned with creation (of
software products). Therefore, it is possible to establish a
parallelism between the method of research in SE and method of
research in software development.
2. Software Engineering Research
Perhaps one of the most commonly accepted classifications of
science is the division into formal sciences (logic and
mathematics) and empirical ones (such as biology, chemistry,
etc.). While the former use deductive research methods, the latter
are based on empirical ones (inductive and hypthoteticodeductive).
This is not, though, the only classification of Science. Bunge [4]
distinguishes between pure and applied sciences, including the
technologies in the latter. He thus considers electrical engineering
to be a physical technology and medicine a biological one. We
have no clear idea of how to classify SE and some other branches
of engineering in this system, not only because it is not the direct
application of a single pure science, but also because we consider
that SE, together with the other branches of engineering, are not
merely applications of other sciences. The fundamental difference
between science and engineering is that while the former deals
with the study of what things are like, the latter is concerned with
what they should be like in order to make it possible to construct
new objects.
In recent years, within the philosophy of science,
much insistence has been laid, and rightly, on science being also
action, not just knowledge; correlatively we may say that
engineering is also knowledge, and not just application. The
difference between science and technology lies in the modes of
knowledge and action that they develop, not in one of them
knowing and the other applying [9].
Apart from the philosophers of science, there are also important
authors on SE who claim the need to define a science of
engineering. In this regard, Blum [3], for example, distinguishes
between science and technology and sets out their mutual
relationship, comparing the work of the engineer and the scientist
and their respective knowledge. He states: I reject the narrow
definition of software engineering coming from the computer
sciences; indeed, I intend to design a new computer science for
software engineering... and goes on to define the science of
computational technology as the study of transforming ideas into
operations.
In line with Blum, we propose two computational sciences for
the discipline of Software Engineering, which we shall call
Software Engineering Science and Software Science. While the

ACM SIGSOFT Software Engineering Notes

Page 2

former, as we shall see below, deals with the study of how to

create software, the latter is concerned with studying both,
software and the techniques, models, methods and so forth
enabling its creation. There would be a third science which could
be considered to overlap the disciplines of SE and Information
Systems (IS), which we might call the Science of Information
Systems, which would deal with the implementation and use of
both software and the techniques, models and so on used in its
creation.
2.1. The Aim of the Study
Depending, then, on the aim of each Science, we shall find a
method which is more or less suitable for its study. But, what is
the object of study of SE? The empirical and formal sciences, in
order to find answers to many unanswered questions, are
concerned with the study of existing objects to obtain answers to
those questions via the creation of hypotheses and models, by
observation and by experiment. However, just as an engineer
constructs new objects, engineering-like sciences also construct
new objects (models, techniques and the like), which in turn can
become the object of scientific research. In this regard,
engineering does not only seek knowledge about certain objects,
but mainly know-how. If research is centred on how to construct
new objects, new methods will be needed; if research is centred on
studying those objects (or the objects that they allow us to
construct), then it may be sufficient to adapt one of the methods
used by the traditional sciences, as they study phenomena and
objects of the world regardless of how they were created.
SE research deals, then, with different problems which should be
tackled with different methods. In [7] a comprehensive study is
offered of the problems dealt with in the literature and of the
methods used. This work is based on the classification presented
in the SWEBOK (Guide to the Software Engineering Body of
Knowledge) [13]. The SWEBOK divides SE into such areas of
knowledge as Software Requisites, Software Design, etc.
Nevertheless, for present purposes, no such classification is
necessary as we shall attempt to divide the problems of SE
research according to the nature of the relevant knowledge. In this
regard, an initial approach will lead us to distinguish between:
Research focussed on the construction of new objects (processes,
models, methodologies, techniques and so forth), carried out in the
context of what we have termed Software Engineering Science.
Problems of this type are of an engineering nature, insofar as the
object of study is the construction of new tools (methods, models,
etc.) for software construction.
Research focussed on the study of such objects (metrics,
optimisation, etc.), carried on in the context of what we have
called Software Science. Its object of study does not differ from
that of the traditional sciences except in that the objects studied are
artificial rather than natural.

TYPE A
TYPE B
TYPE C

July 2005

Volume 30 Number 4

Research directed towards the implementation and use of these

new objects. This goes on, however, within what could be called
Science of Information Systems.
In each of the areas of knowledge proposed in the SWEBOK these
three approaches are to be found. Thus, for example, in Software
Design, we shall be able to research into (A) new models for
improving the design process; in (B) we shall concentrate on the
study of models already constructed and in (C) how to implement
the use of these new design models in an organization.
Problems of type (A) concern engineering, those of type (B) are
generally of an empirical nature and those of type (C) deal with the
study of problems of a cultural and social nature. In this study we
shall be dealing with problems of the first type, those concerning
the construction of new techniques, tools, etc., to facilitate the
process of creating new objects (in general, software objects).
Problems of types (B) and (C) are studied from the perspective of
the traditional sciences.
2.2. The Method
Although research methods can be classed in different ways, a
classification currently enjoying widespread acceptance and
which is suitable for our arguments is the division of methods into
quantitative and qualitative [11]. Deductive and empirical methods
could be classed as quantitative research methods and are
especially suited to the study of natural phenomena or objects. The
study of cultural and social phenomena, however, requires another
kind of method, not based on formal experiments or theories but
on interviews, questionnaires, documents, impressions and the
reactions of the researcher, etc. These are called qualitative
methods and they include action research, case studies,
ethnography, etc. We add to these a third type of method, creative
research methods, which are those that resort more to the arts
(although creativity and science are becoming more and more
closely linked).
Although creativity could be considered a characteristic of
research, regardless of the method, our view is that some sciences
require a high level of creativity as opposed to observation or
experimentation. Such is the case of the arts and branches of
engineering, because of their strong artistic component. When
creativity marks out the research process, we speak of creative
methods. These methods are based on such characteristics as
imagination, premonition, visualization and the like, and they
depend more on the researchers creative intelligence than on
rationale.
Logically, as is true of any kind of classification, there will be
problems that fall into more than one of these classes of research
methods and which will therefore require a combination of
different methods. Table 1 shows the main objects of study,
sciences and methods used in SE.

SCIENCE

OBJECT OF STUDY

CHARACTER

METHODS

Software Engineering
Science
Software Science

Construction of new objects

Engineering

Object constructed

Empirical

Qualitative
Creative
Quantitative

Science of Information
Systems

Implementation and uses of

objects constructed

Cultural and
social

Qualitative

Table 1. Summary of problems, sciences and methods in SE

ACM SIGSOFT Software Engineering Notes

Page 3

The problems which we have called Type B seek to answer such

questions as How good is a method? or Is it possible to find a
more efficient method of access?. They are those that are most
similar to the problems dealt with by the traditional (formal and
empirical) sciences. They may be said to have a scientific nature,
as opposed to the engineering nature of problems of Type A. They
can, therefore, be tackled by means of quantitative research
methods, though not exclusively.
Type C problems require the study of social and cultural factors
and seek to answer questions like What factors make a given
software process unacceptable to the company? or Why is one
software development tool more acceptable than another? Such
problems cannot be tackled exclusively with the traditionally
styled scientific methods, that is, purely quantitative methods.
They are problems requiring the use of qualitative methods.
The solution of problems purely concerning engineering, those of
Type A, requires methods of a different kind. In these cases, it is
not possible to apply empirical methods, as the object of study
does not yet exist. It is generally accepted that the object of
empirical science lies outside us and has its existence in the outer
world. Its knowledge is therefore fundamentally experimental and,
moreover, such experimentation should be based on reality. This is
not the case of engineering, whose knowledge has a major
component of creativity, which makes it difficult to draw up a
universal method for solving problems within this field. Apart
from its creativity component, research of this type has a major
social and cultural component regarding the adoption of new
paradigms, teamwork in software processes, collaborative work,
etc. Because of all this, qualitative methods are increasingly being
used in SE research.
It would seem, then that a research method for tackling the
specific problems of SE Science should be based essentially on
qualitative and creative methods. Of course there is not universal
way of solving problems, but that each problem requires its own
method. As Popper stated: Generally, I start my courses on the

July 2005

Volume 30 Number 4

Scientific Method saying to my students that the scientific method

doesnt exist [12]. Indeed, for example, a method for the
specification of a new methodology for the development of Web
Information System (WIS), would consist fundamentally in
studying existing methodologies, reflecting on them to determine
their advantages and disadvantages and proposing a new one,
which, while retaining the advantages of the methodologies
studied, would, as far as possible, lack their shortcomings.
Arriving at a better final proposition would largely depend on the
creativity and common sense applied to the construction of the
new method.
Software development methods, like SE Science research methods,
also have a major qualitative component (human, social and
cultural factors). Furthermore, nobody questions the importance of
creativity in the conception of a new software product.
3. Parallelism between the Method in SE and the Method
in Software Development
To justify this similarity between the method of research in SE
Science and the method of software development, we shall use a
case study, one based on the method of research being used for the
specification of a methodology of the development of WIS [10]. It
should be noted that the problem is of an engineering kind and we
have classed it under Type A. This is the kind of research that we
shall be concentrating on.
The research method is based on the steps proposed by
Bunge [3] for any scientific research (see Figure 1). Although
these steps are based on the hypothetico-deductive method,
because of their general nature they are applicable, with certain
modifications, to any kind of research. The solving stage will be
the one with the most differences as it is here that a choice must be
made between techniques (experiments, interviews, etc.) according
to whether the method is qualitative, quantitative or creative. A
detailed description of the method is available in [9]; here we shall
be concentrating on its main similarities with the methods of
software development. Table 2 summarizes this parallelism.

Figure 1: Research Method

ACM SIGSOFT Software Engineering Notes

Page 4

Stage 0: Information Gathering

The information gathered for research includes documentation on
the problem to be solved and that linked with the method of
solution and validation.
Information gathered for a development includes documentation
on methodologies applied, products used, and so on. On many
occasions, before a development is begun, it is also necessary to
gather information about the specific domain of the product to be
developed.
Stage 1: Determining the Problem
In this stage the problem to be tackled is clearly determined and
defined, from the perspective of unsolved problems in our sphere
of knowledge. This stage is in some ways very similar to the
requisite-finding stage of the software development process. The
finding of requisites makes it possible to analyse the problem to be
tackled and to set the limits of the essential aspects to be taken into
account in the future system.
Stage 2: Creating the Hypothesis
In traditional methods of scientific research, the hypothesis is
formulated in terms of causation (if A happens, then B will
happen). Such hypotheses are conjectures of facts that the
scientific method must compare and check. It is easy to see,
however, that a hypothesis in SE Science research does not fit into
the cause-effect scheme. It must be borne in mind that the object
of study of this type of science is the construction of new objects
(models, techniques or, as in our case study, methods) which, as
they do not yet exist, cannot be experimented on. This is not the
case of research undertaken in what we have called Software
Science, for here study is focussed on the objects created by SE
Science. Here, a valid hypothesis would be if the number of
foreign keys rises in a relational scheme, its maintainability
decreases, a hypothesis in causal terms which could be verified
by means of quantitative methods (normally the experimental
method).
The hypothesis in SE Science will be formulated like the
description of a new object to be constructed, which, in our case,
will be the description of the new methodology of development, of
what systems are to be applied, what stages of the life cycle will
be tackled and what technology it will be based on. And this ties in
with the specification of software requisites, a product obtained as
a result of the requisite-finding stage. The hypothesis will
therefore be the specification of requisites of the new object to be
constructed.
Stage 3: Definition of the Working Method
Although, generally speaking, research methods do not involve the
definition of the method to be followed as a task to carry out for
the solving and verifying of the problem (which Bunges method
does not do, either), we consider it necessary to define it because,
as we shall see in section 2.2, there is no universal method
applicable to any research.
As is the case with the research method, we cannot speak of a
process or the methodology of software development. A
development methodology must be taken as a guide, not as
something rigid; it must be adapted for each of its uses, like the
research method. When research is begun, the methodological

July 2005

Volume 30 Number 4

paradigm to be followed must be chosen (qualitative, quantitative,

etc.), as must the method itself (action research, experimentation,
etc.). Likewise, when a software development is begun, the
methodological paradigm is decided on (structured, object
oriented) as is the definite methodology to be followed (Yourdon,
OMT, etc.).
Feedback between the solution and verification stage and the
definition stage of the working method (shown by a broken line in
Figure 1), shows that the method arises, refining itself, as the
solution to the problem is approached. It can therefore be said that
the definition of the working method is not concluded until the end
of the solution and verification phase. The solution phase will
show us how to solve the problem and, as is the case with the
definition phase of the research method, the method of software
development defined in it will arise and become refined in
successive approximations. Likewise, when software is developed,
the product is not completely finished until the last trials are run
and, generally, it is constantly being modified.
Stage 4: Solution, Validation and Verification
For the solution, validation and verification of the problem before
us, the definition of a methodology for WIS development, it is
necessary to develop, in a very simplified way, the following
activities: specification of the process of software development,
specification of activities to be carried out within it, and the
specification of the techniques to be used. Although for this,
different research methods could be used, in our case we propose,
for each of these activities:
Solution: by means of the analysis of case studies and a process of
imagination and creativity.
Verification: by means of the implementation of a prototype that
makes it possible to eliminate ambiguities and check that the right
choice has been made (here it would be possible to use some other
technique of formal specification which would not need to be
compilable).
Validation: by means of its application to trial cases.
Another recommendable method for solving, validating and
verifying the present research problem is action research [1], [14].
This method would allow us to define the methodology while it is
in use and being refined in real cases. It is especially useful for
specifying, validating and verifying software processes. The
problem with this method lies in the difficulty of working with
companies and the fact that they do not normally allow their
developments to be used as trial cases. The method based on case
studies and trial cases would be similar to laboratory action
research, when the case study and the trial case coincide.
In fact, it will be seen that the method of solution proposed is
nearer to any of the software development methods (successive
refinements) than to the traditional methods of scientific research.
The reason is that the nature of the problem to be solved is also
nearer to the problems arising in the context of SE than to
problems which scientific research deals with. A method of
software development gives us the guidelines for constructing new
objects (software), just as the research method gives us the
guidelines for the construction of new objects (methodologies,
models, etc.):
If we analyse, for example, the philosophy proposed by Beck in
his XP methodology [2], we shall find it very close in outlook to
methods based on action research. In both cases, a product is

ACM SIGSOFT Software Engineering Notes

Page 5

sought (software, or development methodology in our case study),

by means of its direct creation (programming or conception and
design), in collaboration with the users (of the software or
methodology), by trialling it and adapting it continually during the
creation process.
It is also easy to find similarities in a method of software
development with regard to the method of our case study (based in
turn on case studies, trials and creativity). For example, if a digital
library is to be set up, in the first place we shall analyse others and
by applying a process of imagination and creativity, the new
library will be designed. A series of trials will serve to check its
correct functioning. User trialling would also be possible, but this
is not the only way, by means of a prototype.
It would be possible to draw more exhaustive comparisons
between research methods and development methods, but we shall
leave this for a later study.
Traditional scientific research methods, in particular the one
proposed by Bunge [3], point only to the need for checking the
hypothesis set out (or the theory proposed, in the case of formal
sciences). Nevertheless, in the present case, the checking stage
brings together two tasks: validation, checking that the model has
been built according to the hypothesis set, and verification,
checking that it has been built correctly, that is, that it is
consistent. Similarly, we can make a comparison with any method
of software development where validation allows us to check that
the users specifications have been fulfilled and that verification
ensures that the system is correct.
The solution phase could correspond to the creation of the
hypothesis in an empirical method or to drawing up the theory of a
deductive method. Seen this way, verification of the empirical
method would be tantamount to validating our method, while
verification of the theory of a deductive method would be like

Stages
E0: Documentation

July 2005

Volume 30 Number 4

checking the consistency of our method, which is the verification

task itself.
Stage 5: Analysis of the Results and the Formulation of
Conclusions
The hypothesis set at the beginning of the research will be
contrasted with the results obtained. It will be necessary to check
how far objectives have been fulfilled and to what extent the
problem has been solved. In this phase it is very important to
determine the aspects that it has been impossible to solve and any
new problems that may have arisen as a consequence of the
research and which will be the subject of future research.
In methods of a quantitative nature, this aspect appears clear.
When the research is qualitative and the hypothesis formulated
does not fit into a cause-effect pattern, checking the hypothesis
consists basically in seeing how for the requisites existing at the
initiation of the research have been fulfilled: Does the
methodology developed cover all the necessary phases of the
process? Can it be used in the circumstances it was designed for?,
etc. It would be like, in software development, asking the user how
far their expectations have been met and analysing the possibility
of making improvements, even if they had not been contemplated
at the outset of the project.
Stage 6: Writing the Final Report
In the final report, the research undertaken is set out step by step,
with details of the hypothesis, research method, conclusions,
bibliography and any other point considered relevant for the
understanding and evaluation of the work done. In this stage, stress
should be laid on the importance of research ethics.
Likewise, when the development process is over, it should be
documented. Apart from any documentation generated during
development (users requirements, models, etc.), users manuals
and guides will be included to specify both the functioning of the
product and its limitations. Here again, the ethical considerations
of the development are important.

Research Method in SE Science

Problem to be solved

Research method
E1:Determination of the problem Field study

Software Development Method

Field of application
Methodologies, techniques, etc.
Analysis of the field

E2: Creation of the hypothesis

Setting out the problem

Description of the object to be constructed

Requisite finding
Specification of needs

E3: Definition of the

method

Selection of the paradigm (qualitative,

quantitative, etc.)

Selection of the paradigm (structured, OO,

etc.)

Selection of the method (experimental,

action research, etc.)

Selection of the method

(RUP, XP, etc.)

Adaptation to the problem (experimental

techniques, frameworks for checking, etc.)

Adaptation to the problem (techniques,

notations, etc.)

E4: Resolution

Case studies, creativity

Case studies, creativity

Validation

Trial cases

Prototype

Verification

Tool prototype

Set of trial

Action research
Checking the hypothesis

eXtreme Programming
Checking the users requirements

Hypothesis, Method, Conclusions, etc.

Requirements, Users manual, etc.

Resoluton, V&V
E5: Analysis
E6: Final report

Table 2.- Research Methods vs. Development Methods

ACM SIGSOFT Software Engineering Notes

Page 6

When the research is over, the initial body of knowledge has

become modified (normally it will have grown) and the problems
that we set out with are no longer the same. Some will have been
solved, others will have been altered (to become simpler or
otherwise) and also new problems will arise as a result of the
research done. This new body of knowledge with its new problems
will give rise to new research.
When software development is over, the customers initial
knowledge about what he can get will have changed (it will
usually have widened) and the problems that we set out with will
have changed. Some will be have been solved, others altered
(becoming simpler or otherwise) and new problems will have
arisen as a result of the implementation of the software developed.
This new body of knowledge with its new problems will give rise
to other development studies.
4. Conclusions
Summing up, we have discussed the character of SE knowledge,
setting out three scientific fields related to it, which we have called
Software Engineering Science, when the object of study does not
exist outside the researchers mind and the research process
consists chiefly in creating it (a method, model, etc.); Software
Science, when the object of study is any object previously created
by a process of research in SE Sciences; Science of Information
Systems, if the object of study is focussed on the process of
implementation and use of the objects already mentioned. In view
of the object of study of Software Engineering Science, we
conclude that the research methods best suited to problems of this
type are of a qualitative and creative nature.
We have discussed the parallelism between the SE research
method and the software development method. We leave open for
a later study the exhaustive comparison of different methods and
paradigms of research and of development. It would be interesting
to study the evolution of methods in each of these fields (science
and technology), to see how the lessons learnt could be exchanged.
Much still has to be done to establish the bases and foundations of
this new SE Science. It will require in turn an adapted Philosophy
of Science, a Philosophy of Software Engineering, which will have
to be built on the close cooperation of Software Engineers and
Philosophers of Science.
Acknowledgements
This study was carried out within the framework of the MIIS
network (Research Methods in Software Engineering - TIC20003250-E). The author wishes to thank Camino Marcos and Mario
Piattini for their useful comments.
Bibliography
[1] Avison, D., Lan, F., Myers, M. y Nielsen, A. Action
Research. Communications of the ACM, 42(1), 94-97, 1999.
[2] Beck, K. Embracing Change with eXtreme Programming.
Computer , vol. 32, n 10, pp.70-77, 1999.
[3] Blum, B. I. Beyond Programming: To a New Era of Design.
Oxford University Press, 1996.
[4] Bunge, M. Scientific Research. Springer, Berln, 2 vols., 1967.
[5] Cornford, T y Smithson, S. Project Research in Information
Systems. A Students Guide. Ed. MacMillan, 1996.

July 2005

Volume 30 Number 4

[6] Dobson, P. J. The Philosophy of Critical Realism-An

Opportunity for Information Systems Research. Information
Systems Frontiers, 3:2, pp. 199-210, 2001.
[7] Glass, R.L., Vessey, I. y Ramesh, V. (2002). Research in
software engineering: an analysis of the literature.
Information and Software Technology, Elsevier Science B.V.
N.44, pp. 491-506, 2002.
[8] Gregg, D. G., Kulkarni, U. R. y Vinz, A. S. Understanding
the Philosophical Underpinnings of Software Engineering
Research in Information Systems. Information Systems
Frontiers, 3:2, pp. 169-183, 2001.
[9] Marcos, E. y Marcos, A. An Aristotelian Approach to the
Methodological Research: a Method for Data Models
Construction. Information Systems- The Next Generation. L.
Brooks and C. Kimble (Eds.). Mc Graw-Hill, pp. 532-543,
1998.
[10] Marcos, E. Vela, B., Cceres, P. y Cavero, J.M..
MIDAS/DB: a Methodological Framework for Web
Database Design. Conceptual Modeling for New Information
Systems Technologies. Hiroshi Arisawa, Yahiko Kambayashi,
Vijay Kumar, Heinrich C. Mayr, Ingrid Hunt (Eds.). Springer
Verlag, LNCS 2465, pp.227-238, Heidelberg-Germany,
Septembre 2002.
[11] Myers, M. D. Qualitative Research in Information Systems.
MIS Quarterly, 21:2, pp 241-242, June 1997.
[12] Popper, K. Realism and the Aim of Science, Rowman and
Littlefield, Totowa, N.J. 1983.
[13] SWEBOK. Guide to the Software Engineering Body of
Knowledge. IEEE Computer Society and ACM Software
Engineering
Coordinating
Committee,
2001;
http://www.swebok.org/.
[14] Wood-Harper, T. Research Methods in Information Systems:
Using Action Research. Research Methods in Information
Systems. Mumford et al. (Eds.), Amsterdam: North-Holland,
pp. 169-191, 1985.

ACM SIGSOFT Software Engineering Notes

Page 7

July 2005

Volume 30 Number 4