Research Methodology KEY CONCEPTS OF THE SCIENTIFIC METHOD

There are several important aspects to research methodology. This is a summary of the key concepts in scientific research and an attempt to erase some common misconceptions in science.

by Experiment-Resources.com (2008)

Steps of the scientific method are shaped like an hourglass - starting from general questions, narrowing down to focus on one specific aspect, and designing research where we can observe and analyze this aspect. At last, we conclude and generalize to the real world. FORMULATING A RESEARCH PROBLEM

Researchers organize their research by formulating and defining a research problem. This helps them focus the research process so that they can draw conclusions reflecting the real world in the best possible way.

HYPOTHESIS

In research, a hypothesis is a suggested explanation of a phenomenon.

A null hypothesis is a hypothesis which a researcher tries to disprove. Normally, the null hypothesis represents the current view/explanation of an aspect of the world that the researcher wants to challenge.

Research methodology involves the researcher providing an alternative hypothesis, a research hypothesis, as an alternate way to explain the phenomenon.

The researcher tests the hypothesis to disprove the null hypothesis, not because he/she loves the research hypothesis, but because it would mean coming closer to finding an answer to a specific problem. The research hypothesis is often based on observations that evoke suspicion that the null hypothesis is not always correct.

In the Stanley Milgram Experiment, the null hypothesis was that the personality determined whether a person would hurt another person, while the research hypothesis was that the role, instructions and orders were much more important in determining whether people would hurt others. VARIABLES

A variable is something that changes. It changes according to different factors. Some variables change easily, like the stock-exchange value, while other variables are almost constant, like the name of someone. Researchers are often seeking to measure variables.

The variable can be a number, a name, or anything where the value can change.

An example of a variable is temperature. The temperature varies according to other variable and factors. You can measure different temperature inside and outside. If it is a sunny day, chances are that the temperature will be higher than if it's cloudy. Another thing that can make the temperature change is whether something has been done to manipulate the temperature, like lighting a fire in the chimney.

In research, you typically define variables according to what you're measuring. The independent variable is the variable which the researcher would like to measure (the cause), while the dependent variable is the effect (or assumed effect), dependent on the independent variable. These variables are often stated in experimental research, in a hypothesis, e.g. "what is the effect of personality on helping behavior?"

In explorative research methodology, e.g. in some qualitative research, the independent and the dependent variables might not be identified beforehand. They might not be stated because the researcher does not have a clear idea yet on what is really going on.

Confounding variables are variables with a significant effect on the dependent variable that the researcher failed to control or eliminate - sometimes because the researcher is not aware of the effect of the confounding variable. The key is to identify possible confounding variables and somehow try to eliminate or control them. OPERATIONALIZATION

Operationalization is to take a fuzzy concept, such as 'helping behavior', and try to measure it by specific observations, e.g. how likely are people to help a stranger with problems.

See also:

Conceptual Variables CHOOSING THE RESEARCH METHOD

The selection of the research method is crucial for what conclusions you can make about a phenomenon. It affects what you can say about the cause and factors influencing the phenomenon.

It is also important to choose a research method which is within the limits of what the researcher can do. Time, money, feasibility, ethics and availability to measure the phenomenon correctly are examples of issues constraining the research.

CHOOSING THE MEASUREMENT

Choosing the scientific measurements are also crucial for getting the correct conclusion. Some measurements might not reflect the real world, because they do not measure the phenomenon as it should. RESULTS SIGNIFICANCE TEST

To test a hypothesis, quantitative research uses significance tests to determine which hypothesis is right.

The significance test can show whether the null hypothesis is more likely correct than the research hypothesis. Research methodology in a number of areas like social sciences depends heavily on significance tests.

A significance test may even drive the research process in a whole new direction, based on the findings.

The t-test (also called the Student's T-Test) is one of many statistical significance tests, which compares two supposedly equal sets of data to see if they really are alike or not. The t-test helps the researcher conclude whether a hypothesis is supported or not. DRAWING CONCLUSIONS

Drawing a conclusion is based on several factors of the research process, not just because the researcher got the expected result. It has to be based on the validity and reliability of the measurement, how good the measurement was to reflect the real world and what more could have affected the results.

The observations are often referred to as 'empirical evidence' and the logic/thinking leads to the conclusions. Anyone should be able to check the observation and logic, to see if they also reach the same conclusions.

Errors of the observations may stem from measurement-problems, misinterpretations, unlikely random events etc.

A common error is to think that correlation implies a causal relationship. This is not necessarily true. GENERALIZATION

Generalization is to which extent the research and the conclusions of the research apply to the real world. It is not always so that good research will reflect the real world, since we can only measure a small portion of the population at a time.

VALIDITY AND RELIABILITY

Validity refers to what degree the research reflects the given research problem, while Reliability refers to how consistent a set of measurements are.

Types of validity: External Validity Population Validity Ecological Validity

Internal Validity Content Validity Face Validity Construct Validity Convergent and Discriminant Validity Test Validity Criterion Validity Concurrent Validity Predictive Validity

A definition of reliability may be "Yielding the same or compatible results in different clinical experiments or statistical trials" (the free dictionary). Research methodology lacking reliability cannot be trusted. Replication studies are a way to test reliability.

Types of Reliability: Test-Retest Reliability Interrater Reliability Internal Consistency Reliability Instrument Reliability Statistical Reliability Reproducibility

Both validity and reliability are important aspects of the research methodology to get better explanations of the world. ERRORS IN RESEARCH

Logically, there are two types of errors when drawing conclusions in research:

Type 1 error is when we accept the research hypothesis when the null hypothesis is in fact correct.

Type 2 error is when we reject the research hypothesis even if the null hypothesis is wrong.

Steps of the research process By Diane C. Blankenship

Scientific research involves a systematic process that focuses on being objective and gathering a multitude of information for analysis so that the researcher can come to a conclusion. This process is used in all research and evaluation projects, regardless of the research method (scientific method of inquiry, evaluation research, or action research). The process focuses on testing hunches or ideas in a park and recreation setting through a systematic process. In this process, the study is documented in such a way that another individual can conduct the same study again. This is referred to as replicating the study. Any research done without documenting the study so that others can review the process and results is not an investigation using the scientific research process. The scientific research process is a multiple-step process where the steps are interlinked with the other steps in the process. If changes are made in one step of the process, the researcher must review all the other steps to ensure that the changes are reflected throughout the process. Parks and recreation professionals are often involved in conducting research or evaluation projects within the agency. These professionals need to understand the eight steps of the research process as they apply to conducting a study. Table 2.4 lists the steps of the research process and provides an example of each step for a sample research study.

Step 1: Identify the Problem

The first step in the process is to identify a problem or develop a research question. The research problem may be something the agency identifies as a problem, some knowledge or information that is needed by the agency, or the desire to identify a recreation trend nationally. In the example in table 2.4, the problem that the agency has identified is childhood obesity, which is a local problem and concern within the community. This serves as the focus of the study.

Step 2: Review the Literature

Now that the problem has been identified, the researcher must learn more about the topic under investigation. To do this, the researcher must review the literature related to the research problem. This step provides foundational knowledge about the problem area. The review of literature also educates the researcher about what studies have been conducted in the past, how these studies were conducted, and the conclusions in the problem area. In the obesity study, the review of literature enables the programmer to discover horrifying statistics related to the long-term effects of childhood obesity in terms of health issues, death rates, and projected medical costs. In addition, the programmer finds several articles and information from the Centers for Disease Control and Prevention that describe the benefits of walking 10,000 steps a day. The information discovered during this step helps the programmer fully understand the magnitude of the problem, recognize the future consequences of obesity, and identify a strategy to combat obesity (i.e., walking).

Step 3: Clarify the Problem

Many times the initial problem identified in the first step of the process is too large or broad in scope. In step 3 of the process, the researcher clarifies the problem and narrows the scope of the study. This can only be done after the literature has been reviewed. The knowledge gained through the review of literature guides the researcher in clarifying and narrowing the research project. In the example, the programmer has identified childhood obesity as the problem and the purpose of the study. This topic is very broad and could be studied based on genetics, family environment, diet, exercise, self-confidence, leisure activities, or health issues. All of these areas cannot be investigated in a single study; therefore, the problem and purpose of the study must be more clearly defined. The programmer has decided that the purpose of the study is to determine if walking 10,000 steps a day for three days a week will improve the individuals health. This purpose is more narrowly focused and researchable than the original problem.

Step 4: Clearly Define Terms and Concepts

Terms and concepts are words or phrases used in the purpose statement of the study or the description of the study. These items need to be specifically defined as they apply to the study. Terms or concepts often have different definitions depending on who is reading the study. To minimize confusion about what the terms and phrases mean, the researcher must specifically define them for the study. In the obesity study, the concept of individuals health can be defined in hundreds of ways, such as physical, mental, emotional, or spiritual health. For this study, the individuals health is defined as physical health. The concept of physical health may also be defined and measured in many ways. In this case, the programmer decides to more narrowly define individual health to refer to the areas of weight, percentage of body fat, and cholesterol. By defining the terms or concepts more narrowly, the scope of the study is more manageable for the programmer, making it easier to collect the necessary data for the study. This also makes the concepts more understandable to the reader.

Step 5: Define the Population

Research projects can focus on a specific group of people, facilities, park development, employee evaluations, programs, financial status, marketing efforts, or the integration of technology into the operations. For example, if a researcher wants to examine a specific group of people in the community, the study could examine a specific age group, males or females, people living in a specific geographic area, or a specific ethnic group. Literally thousands of options are available to the researcher to specifically identify the group to study. The research problem and the purpose of the study assist the researcher in identifying the group to involve in the study. In research terms, the group to involve in the study is always called the population. Defining the population assists the researcher in several ways. First, it narrows the scope of the study from a very large population to one that is manageable. Second, the population identifies the group that the researchers efforts will be focused on within the study. This helps ensure that the researcher stays on the right path during the study. Finally, by defining the population, the researcher identifies the group that the results will apply to at the conclusion of the study. In the example in table 2.4, the programmer has identified the population of the study as children ages 10 to 12 years. This narrower population makes the study more manageable in terms of time and resources.

Step 6: Develop the Instrumentation Plan

The plan for the study is referred to as the instrumentation plan. The instrumentation plan serves as the road map for the entire study, specifying who will participate in the study; how, when, and where data will be collected; and the content of the program. This plan is composed of numerous decisions and considerations that are addressed in chapter 8 of this text. In the obesity study, the researcher has

decided to have the children participate in a walking program for six months. The group of participants is called the sample, which is a smaller group selected from the population specified for the study. The study cannot possibly include every 10- to 12-year-old child in the community, so a smaller group is used to represent the population. The researcher develops the plan for the walking program, indicating what data will be collected, when and how the data will be collected, who will collect the data, and how the data will be analyzed. The instrumentation plan specifies all the steps that must be completed for the study. This ensures that the programmer has carefully thought through all these decisions and that she provides a step-by-step plan to be followed in the study.

Step 7: Collect Data

Once the instrumentation plan is completed, the actual study begins with the collection of data. The collection of data is a critical step in providing the information needed to answer the research question. Every study includes the collection of some type of datawhether it is from the literature or from subjectsto answer the research question. Data can be collected in the form of words on a survey, with a questionnaire, through observations, or from the literature. In the obesity study, the programmers will be collecting data on the defined variables: weight, percentage of body fat, cholesterol levels, and the number of days the person walked a total of 10,000 steps during the class.

The researcher collects these data at the first session and at the last session of the program. These two sets of data are necessary to determine the effect of the walking program on weight, body fat, and cholesterol level. Once the data are collected on the variables, the researcher is ready to move to the final step of the process, which is the data analysis.

Step 8: Analyze the Data

All the time, effort, and resources dedicated to steps 1 through 7 of the research process culminate in this final step. The researcher finally has data to analyze so that the research question can be answered. In the instrumentation plan, the researcher specified how the data will be analyzed. The researcher now analyzes the data according to the plan. The results of this analysis are then reviewed and summarized in a manner directly related to the research questions. In the obesity study, the researcher compares the measurements of weight, percentage of body fat, and cholesterol that were taken at the first meeting of the subjects to the measurements of the same variables at the final program session. These two sets of data will be analyzed to determine if there was a difference between the first measurement and the second measurement for each individual in the program. Then, the data will be analyzed to

determine if the differences are statistically significant. If the differences are statistically significant, the study validates the theory that was the focus of the study. The results of the study also provide valuable information about one strategy to combat childhood obesity in the community.

As you have probably concluded, conducting studies using the eight steps of the scientific research process requires you to dedicate time and effort to the planning process. You cannot conduct a study using the scientific research process when time is limited or the study is done at the last minute. Researchers who do this conduct studies that result in either false conclusions or conclusions that are not of any value to the organization.

This is an excerpt from Applied Research and Evaluation Methods in Recreation.

3-The steps in the design process interact and often occur simultaneously. For example, the design of a measurement instrument is influenced by the type of analysis that will be conducted. However, the type of analysis is also influenced by the specific characteristics of the measurement instrument.

Step 1: Define the Research Problem

Problem definition is the most critical part of the research process. Research problem definition involves specifying the information needed by management. Unless the problem is properly defined, the information produced by the research process is unlikely to have any value. Coca-Cola Company researchers utilized a very sound research design to collect information on taste preferences. Unfortunately for Coca-Cola, taste preferences are only part of what drives the soft drink purchase decision.

Research problem definition involves four interrelated steps: (1) management problem / opportunity clarification, (2) situation analysis, (3) model development, and (4) specification of information requirements.

The basis goal of problem clarification is to ensure that the decision makers initial description of the management decision is accurate and reflects the appropriate area of concern for research. If the wrong management problem is translated into a research problem, the probability of providing management with useful information is low. Situation Analysis

The situation analysis focuses on the variables that have produced the stated management problem or opportunity. The factors that have led to the problem/opportunity manifestations and the factors that have led to managements concern should be isolated.

A situation analysis of the retail trade outflow problem revealed, among other things, that (1) the local population had grown 25 percent over the previous five years, (2) buying power per capita appeared to be growing at the national rate of 3 percent a year, and (3) local retail sales of nongrocery items had increased approximately 20 percent over the past five years. Thus, the local retailers sales are clearly not keeping pace with the potential in the area. Step 2: Estimate the Value of the Information

A decision maker normally approaches a problem with some information. If the problem is, say, whether a new product should be introduced, enough information will normally have been accumulated through past experience with other decisions concerning the introduction of new products and from various other sources to allow some preliminary judgments to be formed about the desirability of introducing the product in question. There will rarely be sufficient confidence in these judgments that additional information relevant to the decision would not be accepted if it were available without cost or delay. There might be enough confidence, however, that there would be an unwillingness to pay very much or wait very long for the added information.

Step 3: Select the Data Collection Approach

There are three basic data collection approaches in marketing research: (1) secondary data, (2) survey data, and (3) experimental data. Secondary data were collected for some purpose other than helping to solve the current problem, whereas primary data are collected expressly to help solve the problem at hand. Step 4: Select the Measurement Technique

There are four basic measurement techniques used in marketing research: (1) questionnaires, (2) attitude scales, (3) observation, and (4) depth interviews and projective techniques.

Primary Measurement Techniques

I. Questionnaire a formalized instrument for asking information directly from a respondent concerning behavior, demographic characteristics, level of knowledge, and/or attitudes, beliefs, and feelings.

II. Attitude Scales a formalized instrument for eliciting self-reports of beliefs and feelings concerning an object(s).

A. Rating Scales require the respondent to place the object being rated at some point along a numerically valued continuum or in one of a numerically ordered series of categories.

B. Composite Scales require the respondents to express a degree of belief concerning various attributes of the object such that the attitude can be inferred from the pattern of responses.

C. Perceptual maps derive the components or characteristics an individual uses in comparing similar objects and provide a score for each object on each characteristic.

D. Conjoint analysis derive the value an individual assigns to various attributes of a product.

I. Observation the direct examination of behavior, the results of behavior, or physiological changes.

II. Projective Techniques and Depth Interview designed to gather information that respondents are either unable or unwilling to provide in response to direct questioning.

A. Projective Techniques allow respondents to project or express their own feelings as a characteristic of someone or something else.

B. Depth Interviews allow individuals to express themselves without any fear of disapproval, dispute, or advice from the interviewer. Step 5: Select the Sample

Most marketing studies involve a sample or subgroup of the total population relevant to the problem, rather than a census of the entire group. Step 6: Select the Model of Analysis

It is imperative that the researcher select the analytic techniques prior to collecting the data. Once the analytic techniques are selected, the researcher should generate fictional responses (dummy data) to the measurement instrument. These dummy data are then analyzed by the analytic techniques selected to ensure that the results of this analysis will provide the information required by the problem at hand. Step 7: Evaluate the Ethics of the Research

It is essential that marketing researchers restrict their research activities to practices that are ethically sound. Ethically sound research considers the interests of the general public, the respondents, the client and the research profession as well as those of the researcher. Step 8: Estimate Time and Financial Requirements

The program evaluation review technique (PERT) coupled with the critical path method (CPM) offers a useful aid for estimating the resources needed for a project and clarifying the planning and control process. PERT involves dividing the total research project into its smallest component activities, determining the sequence in which these activities must be performed, and attaching a time estimate for each activity. These activities and time estimates are presented in the form of a flow chart that allow a visual inspection of the overall process. The time estimates allow one to determine the critical path through the chart that series of activities whose delay will hold up the completion of the project. Step 9: Prepare the Research Proposal

The research design process provides the researcher with a blueprint, or guide, for conducting and controlling the research project. The blueprint is written in the form of a research proposal. A written research proposal should precede any research project.