Lecture No.

Empirical Software Engineering

1
 Empirical Software Engineering
What we Know from the last lecture:
- What is research
- What are the expectations from research
- Steps in conducting research
- Quality of research
- What is good and what is bad research
What is Next ?

Characteristics

&

Types of Research
 Characteristics of Research

To qualify as research, the process must have

certain characteristics- it must be:
• Controlled
• Systematic
• Empirical
• Valid, Reliable, Methodical and Verifiable
• Critical
 Characteristics of Research-contd

• Controlled
– In real life there are many factors that affect an outcome.
– A particular event is rarely the result of a one-to-one
relationship.
• Some relationships are more complex than others.
• Most outcomes are a consequence of the interaction of a variety of
relationships and interacting factors.
– In a study of cause and effect relationships, it is important
to be able to link the effect(s) with the cause(s) and vice
versa.
– Study of causation, the establishment of this linkage is
essential, in practice, sometimes it is extremely difficult —
and occasionally impossible — to make the link.
 Characteristics of Research-contd
Controlled:
• Set up your study in a way that eliminates or
minimizes the effects of other factors affecting
the relationship
• Physical Science: Easy to be achieved to a large
extent as most of the research is done in a
laboratory.
• Social Science: Extremely difficult as research is
carried out on issues relating to human beings
living in society
– Laboratory controls are impossible.
• In social sciences, you cannot control external
factors, but you can attempt to quantify their
impact.
 Characteristics of Research-Contd
• Systematic
– A research investigation follows a certain logical
sequence.
• The different steps can’t be taken in a haphazard way.
• Some procedures must follow others.
• Empirical
– Conclusions drawn are based upon evidence
gathered from information collected from real life
experiences and/or observations
 Characteristics of Research-contd

• Valid, Reliable, Rigorous & Verifiable

– Findings are `Correct` and can be verified by you and
others
– Procedures followed to find answers to questions are
relevant, appropriate, and justified.
– Are we measuring what we think we are measuring
(validity)
– Complications to ensure the validity of a study
• Who will decide that an instrument/study is measuring what it is
supposed to measure?
• How can it be established that an instrument is measuring what it
is supposed to measure?
• Validity TESTS (to be discussed)
– The consistent outcome of an experiment (reliability)
– How it is ensured ?-Reliability tests (to be discussed)
 Characteristics of Research-contd

• Critical
– Critical scrutiny of the procedures used and the
methods employed
– The process of investigation must be foolproof
and free from any drawbacks.
– The process adopted and the procedures used
must be able to withstand critical scrutiny.
 Class Discussion
Based on your understanding of the
characteristics of research, discuss why the
studies given in the next slide do not fall in the
domain of research?
 Lect-1 Examples Revisited
(1) Person ‘A’ prepared a report on “computer usage in
secondary schools” after reviewing literature on the
subject available in his university library and called it a
piece of research.
(2) Person ‘B’ says that he has researched and completed
a document which gives information about the age of
students, their semester results, their parents income
and distance of their schools from the university
campus.
(3) Person ‘C’ participated in a workshop on curriculum
development and prepared what he calls, a research
report on the curriculum for building technicians. He
did this through a literature survey on the subject and
by discussing with the participants of the workshop
11
Types of research
 Types of research
• Research can be classified from three perspectives
(1) The application of the research study
– Pure
– Applied
(2) The objectives in undertaking the research
– Descriptive
– Exploratory
– Explanatory
– Co-relational

(3) The type of information sought.

– Quantitative
– Qualitative

• These three classifications are not mutually exclusive

– Research study classified from the viewpoint of “application”
can also be classified from the perspectives of “objectives” and
“type of information sought.”
Types of research-Application Viewpoint

• Application Prospective:
• Two broad categories:
– Basic or fundamental (i.e., pure) research
– Applied research.
• Pure research:
– Involves developing and testing theories and
hypotheses that are intellectually challenging to the
researcher
– May or may not have practical application at the
present time or in the future.
– Such work often involves the testing of hypotheses
containing very abstract and specialized concepts.
 Types of research-Application Viewpoint
• Examples of pure research
– Developing a sampling technique that can be applied to a
particular situation
• There are many sampling tech. However, a new one can be
developed by a researcher purely based on his own interest or
curiosity.
• No immediate application might be foreseen by the researcher or
even possible under current circumstances
– Developing a methodology to assess the validity of a
procedure
• Many validity tests are in practice, though
– Developing an instrument, say, to measure the stress level in
employees
– Finding the best way of measuring people’s attitudes.

• The knowledge produced through pure research is sought

in order to add to the existing body of knowledge of
research methods
 Applied Research
• Applied research is a methodology used to solve a
specific, practical problem of an individual or group.
The study and research is used in business, medicine
and education in order to find solutions that may
cure diseases, solve scientific problems or develop
technology.
• Interventions for specific child behaviors
• How to reverse or manage global warming
• What is causing increased poverty?
 Types of research-Objective view point

• Descriptive:
– To systematically describe something*
• Co-relational:
– To discover or establish the existence of a
relationship
• Explanatory:
– To clarify why and how there is a relationship
between two aspects of a situation or phenomenon
• Exploratory:
– To investigate the possibilities of undertaking a
particular research study.
* situation, problem, phenomenon, service or program
 Examples of Descriptive Research

• The study attempts to describe the type of

services provided by an organization
• The study attempts to describe the
administrative structure of an organization
• The study describes the living conditions of
the people in areas of the former Soviet Union
• How a child feels living in a house with
domestic violence
• The attitudes of employees towards the
management
 Examples of Co-Relational Research
• Is there an impact of an advertising campaign on the sale
of a product?
• Is there a relationship between stressful living and the
incidence of heart attacks?
• What is the relationship between fertility and mortality?
• Is there (or what is) the relationship between technology
and employment?
• What is the effect of a health service on the control of a
disease, or the home environment on educational
achievement?
– These studies examine whether there is a relationship between two or
more aspects of a situation or phenomenon.
– Don’t answer why.
 Examples of Explanatory Research
• Why stressful living results in heart attacks
• Why a decline in mortality is followed by
fertility decline
• How and why the home environment affects
children’s level of academic achievement.
 Examples of Exploratory Research
• Also called a “feasibility study” or a “pilot
study.”
• It is usually carried out when a researcher
wants to explore areas about which s/he has
little or no knowledge.
• A small-scale study is undertaken to decide if
it is worth carrying out a detailed
investigation.
• On the basis of the assessment made during
the exploratory study, a full study may
eventuate.
Objective -The Difference
Types of research-Objective View Point

– Figures (Slides 22-24) show the types of research

studies from the viewpoint of objectives.
Examples Aim Main Type of
Theme Research
•Socio-economic characteristics of residents To describe To describe Descriptive
of a community what is what is Research
•Attitudes of students towards quality of Common common
teaching regarding a/an:
•Types of service provided by an agency •group of people
•Needs of a community •community
•Sale of a product •phenomenon
•situation
•Attitudes of nurses towards death and dying
•program
•Attitudes of workers towards management
•outcome
•Number of people living in a community
•Problems faced by new immigrants
•Extent of occupational mobility among
immigrants
•Students’ likes and dislikes with regard to
their experience at PBA.
•Effects of living in a house with domestic
violence.
Examples Aim Main Type of
Theme Research
•Impact of a program on…. To establish or To Correlational
•Relationship between stressful living and Explore a/an: determine if Research
incidence of heart attacks •relationship there is a
•Impact of technology on employment relationship
•association
•Impact of maternal and child health services •interdepen-
on infant mortality dence
•Effectiveness of a marriage counseling
service on extent of marital problems
•Impact of an advertising campaign on sale of
a product
•Impact of incentives on productivity of
workers
•Effectiveness of an immunization program in
controlling infectious disease
Examples Aim Main Type of
Theme Research
Why does stressful living result in heart To explain Why: To explain Explanatory
attacks? •a relationship, why the Research
•How does technology create association or relationship
unemployment/employment? interdepen- is formed
•How do maternal and child health services dence exists.
affect infant mortality? •a particular
•Why do some people have a positive attitude event
towards an issue while others do not? occurs.
•Why does a particular intervention work for
some and not for others?
•Why do some people use a product while
others do not?
•Why do some people migrate to another
country while others do not?
•Why do some people adopt a program while
others do not?
 Types of research-Inquiry
mode(information sought)
• Type of Information Sought
– From this perspective, research can be classified
as qualitative or quantitative.
– The quantitative—qualitative classification is
dependent on the following criteria:
(1) The purpose of the study
(2) How the `variables` are measured
(3) How the information is being analyzed
Types of research-Information Sought viewpoint
• The study is classified as qualitative if:
– The purpose of the study is primarily to describe a
situation, phenomenon, problem or event
– The information is gathered through the use of
variables measured on nominal scales or ordinal
(sometimes) scales (qualitative measurement
scales)
– If analysis is done to establish the variation in the
situation, phenomenon or problem without
quantifying it.
– The description of an observed situation, the historical
details of events, an account of the different opinions
people have about an issue:
• War on Terror
• Description of the living conditions of a community
 Types of research-Information Sought viewpoint
• The study is classified as quantitative if:
– We quantify the variation in a phenomenon, situation,
problem or issue
– Information is gathered using predominantly quantitative
variables
– Analysis is geared to ascertain the magnitude of the
variation
• Examples of quantitative aspects of a research study are:
• How many people have a particular problem?
• How many people hold a particular attitude?
• The use of Statistics in Quantitative Research
– Act as a test to confirm or contradict the conclusions
drawn on the gathered data
– Helps the researcher to quantify the magnitude of a
relationship, provide an indication of the confidence
you can place in your findings and help you to isolate
the effect of different variables- Very important !!!
 Qualitative vs Quantitative

• Certain disciplines lend themselves mostly either to

qualitative or to quantitative research.
• Disciplines like anthropology, history and sociology are
more inclined towards qualitative research
• Human resources, engineering, operations
management, psychology, medicine, education,
economics, public health and marketing are more
inclined towards quantitative research.
• However, this does not mean that an economist or a
marketing researcher or an engineer never uses the
qualitative approach (they do … a lot), or that an
anthropologist never uses quantitative information.
• There is increasing recognition by most disciplines that
both types of research are important for a good
research study.
Types of research-Information Sought viewpoint
• Both approaches have their strengths and weaknesses
• The measurement and analysis of the variables about
which information is obtained in a research study are
dependent upon the purpose of the study.
• In some studies a researcher might need to combine
both qualitative and quantitative approaches.

• Types of service is the qualitative aspect of the study

as finding out about them entails description of the
services.
• The extent of utilization of the services is the
quantitative aspect as it involves estimating the
number of people who use the services and
calculating other indicators that reflect the extent of
utilization.
Types of Research
 Assignment # 1
Write down two topics in the field of your expertise (or of
your choice) and classify these into:
• Pure Research or Application Research
• Descriptive, exploratory, co-relational or explanatory
study
• Quantitative or qualitative
Submit it to me in hard form (handwritten) on 26th
March, 2019, before 2 p.m. Copied from internet/Friend
will result in zero marks (Both Parties).
Late Assignment is not be accepted at any cost.
 Our Roadmap.....
• Overview of Research- Done.
• Characteristics and types of research-Done.
• The research process
• Literature Review
• Formulating a research problem
• Identifying the variable and constructing the Hypotheses
• Selecting a research study design
• Methods of data collection
• Sampling
• Writing a research proposal
• Writing a research report/paper/thesis
• Research Ethics- If time permits.