Question: What is this?

Presentation Outline
1. What is research in science and technology? 2. What are the types of research? 3. What is research and development? 4. What is a unit of study? 5. What is the typical research lifecycle? 6. What is expected out of research (i.e., result/output)? 7. What are the research methodologies? 8. What are the research project phases? 9. What are the required supporting skills? 10. How to design a research project? 11. How to choose a research problem? 12. How to write a research proposal?

1/12

What is research in science and technology (S&T)?

Research is:

exploratory (investigative)
Research is: work that show evidence of (a) independent inquiry, (b) originality in the methods used and/or conclusions drawn AND (c) must make an appreciable new contribution to knowledge in the field of study; or, work that show evidence of (a) independent inquiry AND/OR (b) originality in either conclusions or method (Source: University Calendar, Trinity College, Dublin) Research is:

innovating (originating, discovering, inventing) - research in developed countries adapting (adjusting, arranging, accommodating) - research in developing countries NOT adopting (taking in, using, employing) - not research just a project NOT copying (imitating, duplicating, reproducing) - plagiarism or piracy

What are the types of research?

Basic research - refers to experimental or theoretical work undertaken primarily to acquire new knowledge of underlying foundations of phenomena and observable facts without immediate or specific use in view (free fundamental research) geared toward the solution of a specific problem that has not been solved before (oriented basic research) Applied research - refers to investigation undertaken in order to verify data/information gathered from basic or fundamental research or to acquire new knowledge directed primarily towards a specific practical aim or objective systematic work, drawing from existing knowledge gained from research and/or practical experience that is directed to producing new materials, products and devices, to installing new processes, systems and services and to improving substantially those already produced or installed (experimental development) innovative activity aimed at gaining experience leading to further technical improvement of the product or the production process and setting the parameters prior to the commercialization process (pilot testing)
(Source: DOST Revised Guidelines Governing the Grants-in-Aid Program, 1997)

2/12

What are the types of research? (contd)

Basic research
fundamental research

2/12

Applied research
experimental development pilot testing

oriented basic research

Academic research

Industry research

Contract research

Industrial research and development

3/12

What is research and development?

Remember that research (R) is not development (D) although D will be necessary in solving your R problem. If you stray to far in D, you will probably be passed by industrial R&D and find your years of work will (sic) be released as a product before you start to write-up.
(Source: A. Bundy, A. et al., The Researchers Bible, University of Edinburgh, 1995)

4/12

What is the unit of study in S&T research?

Generally, the unit of study may be systems, system components, architectures, models, processes, procedures, methods, techniques, theories, etc in a particular field or discipline (~study population in social science research) For example, in computer science study units may be: In Algorithms: algorithms, routines In Artificial intelligence: methods, techniques, algorithms In Programming languages: languages, language components or features In Architecture: instruction sets, memory hierarchies, architectures In Theory: models, theorems, proof techniques In Systems: systems, components of systems, system architectures (Exercise #1: Can you come up with a similar list in your field/discipline? )

Characteristics of study units: hierarchy and diversity

4/12

What is the unit of study in S&T research? (contd)

Units have HIERARCHICAL structure. (important for understanding context/developing framework)
unit

For example:

computer architecture

Level 0

instruction sets
instructions for branching instructions for procedure call

memory hierarchies
support for virtual memory study of caches study of cache coherency

floating point units

Level 1

floating point representation arithmetic algorithms

Level 2

implementation strategies pipeline design

(Exercise #2: Can you come up with a similar unit hierarchy in your field/discipline? )

4/12

What is the unit of study in S&T research? (contd)

Units vary (DIVERSITY) by their SIZE, POTENTIAL VARIETY, COST OF EVALUATION, and so on. (important for defining scope and limitations) Size: - Small units: algorithms, language features, architectural components - Medium-size units: instruction sets, proof techniques - Large units: languages, architectures, machines Potential Variety: - Low variety: sorting algorithms, cache designs - High variety: AI algorithms, languages, architectures Cost of Evaluation: - Low cost: algorithms, artificial intelligence methods - Medium cost: theorems, components of architectures - High cost: languages, architectures, ideal models Generally, smaller units exhibit less variety and lower cost of evaluation, so they are easier to do research on. (Exercise #3: Can you come up with a similar classification in your field/discipline? )

5/12

What is the typical research lifecycle?

Commercialization Definition (1) Initial Solutions (2)

Exploratory theory

Technology Transfer (6)

Tradeoff

Space of Possible Solutions (5) Comparison of Solutions (4)

Evaluation of Initial Solutions (3)

Next

SIDETRACK: Apocalypse of the two elephants

Research

Billion dollar investment (Commercialization) Standards

Activity

Time
Source: David Clark, MIT (as quoted in A. Tanenbaum, Computer Networks, 2e, Prentice Hall, 1988)

1. Definition Phase (Research Lifecycle) Exploratory research defines a new problem, new constraints, new opportunity, or a new approach.

Back

2. Initial solutions Phase - CREATE UNITS (Research Lifecycle) Initial algorithms, designs, theorems, programs are developed.

Back

3. Evaluation of initial solutions Phase - EVALUATE UNITS (Research Lifecycle) Initial solutions are evaluated and refined in isolation.

Back

4. Comparison of solutions Phase - COMPARE UNITS (Research Lifecycle) Solutions are compared to one another and also to ideal solutions.

Back

5. Space of possible solutions Phase - SPACE OF UNITS IDEAL MODEL (Research Lifecycle) Theorems are proved about the limits on any solutions. Existing solutions are placed in a common framework to determine whether all possible solutions have been found.

Back

6. Technology Transfer Phase (Research Lifecycle) Best approaches are transferred to users.

Back

5/12

What is the typical research lifecycle? (contd) Not all of these phases are seen in all areas. For units with high cost of evaluation only relatively weak methods can be applied to evaluate initial solutions and compare solutions. For units with high variety, it is difficult to understand the space of all possible solutions.

6/12

What is expected out of research (i.e., result/output)?

Here are some examples:
a definition of a problem or task;

a unit for solving a problem, performing a task; identification of factors in influencing the cost, effectiveness, or applicability of a unit (perhaps with some idea of the relative importance of the factors); development of an ideal model; a finished unit that can be distributed to users; and, measurement of some properties of a unit: e.g., run time, chip area, representation requirements, reliability, usability, etc.

What are the research methodologies?

(Some generic examples:) Design units Implement units Run units Find and prove theorems Analyze and consolidate Study users Import techniques Read literature Write paper

7/12

There are many possible research methodologies (depending on the field/discipline).

The Research Methodology section in a research proposal or paper is normally an elaboration of one or a combination of these methodologies used in the research as well as the materials/resources to be used/used.

What are the research methodologies? (contd)

Some specific examples:
writing programs writing systems developing architectures developing content architectures (ontologies, knowledge bases, class libraries, graphics toolboxes, etc.) measuring properties of units finding and proving theorems analyzing and consolidating previous research

7/12

interviewing experts, customers performing psychological experiments, surveys, observations building hardware reading literature importing techniques and results from other fields measuring and predicting constraints on future units (e.g., VLSI technology, government regulation, user expectations and requirements) writing papers, monographs, and textbooks

What are the research methodologies? (Example Scenario)

There are many possible methodologies, but you should have one. Many start from different beginnings but merge later. Here is one example of a methodology. Stage 1 Think of a scenario -- i.e. a sample output which would show that your computer program was exhibiting the ability you want it to model. In mathematical reasoning this scenario might be a proof; in natural language a sample dialogue; in vision the recognition of a scene, etc. Stage 2 Hypothesize what processes might achieve such a scenario. Outline the procedures and data structures that might be involved. Try to make these as general as possible. See the problems you encounter as examples of general problems. Do not use ad hoc mechanisms except to overcome problems that are not central to the issue you are addressing. Stage 3 Think of further scenarios. See whether your proposed program or system could cope with them. Use them to refine; generalize; extend and debug it.

What are the research methodologies?

(Example Scenario contd)
Stage 4 When you are satisfied that your proposed solution or project is stable, choose the programming language that fits your needs closest and implement your program. Stage 5 Do thought experiments before subjecting your problem/solution to a program. Use a pencil and paper to see if your mental solution might work before subjecting it to a coding solution. Stage 6 Describe your program using language independent of your particular implementation. Try to draw out any new techniques. Compare them to previous techniques in your area. If time permits, apply your technique to other areas. N.B. Stages 4 and 5 will take longer than you think -- years not months -- so leave plenty of time!
(Source: A. Bundy, A. et al., The Researchers Bible, University of Edinburgh, 1995)

8/12

What are the research project phases?

An individual research project follows phases related to the research life-cycle: Choose research question/problem/tradeoff; Determine current state of knowledge (literature review); Apply appropriate methods to produce research results; and, Write up research results. To address the basic questions/framework for research: What is the problem? (INTRODUCTION) What did you use to tackle it? (MATERIALS AND METHODS) What results followed? (RESULTS AND DISCUSSION) Research is not complete until it is written up!

What are the research project phases?

Research Methods
Find and prove theorems Analyze and consolidate

8/12

Research Phase-Method Matrix

Implement units

Import techniques

Read literature

Design units

Study users

Research Phases
Definition Create units Evaluate units Compare units Space of units ideal model Tradeoff Explanatory theory

X X X X X X X X X X X X X

X X X X X

X X X X X X X

Write paper

Run units

X X

X X X

Next

SIDETRACK: Research and Literature Review

Literature review is fundamental to research!

Back

What are the required supporting skills?

Different research methods require different supporting skills. Some examples of required SKILLS: Programming Design Organization Mathematics Psychological techniques Protocol analysis Experimental manipulations Survey methods Statistics Writing proposals Writing papers Critiquing papers Designing experiments Giving talks

9/12

How to design a research project?

Some factors to consider:

10/12

Impact and significance Required skills Feasibility and competition Inherent interest Phase of research; extendibility Opportunities for learning new skills

11/12

How to choose a research problem?

Questions about yourself Questions about context Questions about the lifecycle

Questions About Yourself

What research skills do you excel at? Where in the lifecycle are those skills most needed? What kind of outcome do you seek? Satisfaction from being a competent researcher? Solving important societal, commercial, or scientific problems? Transforming the way computers are programmed? Transforming the way current users employ computers? Bringing computation to a new kind of user? Starting a company and making a fortune? What research skills do you excel at? What research skills do you excel at? What research contributions do you most admire?

Questions About the Context

What new technologies may drive change? How will these technologies shift tradeoff points? What new problems will they introduce? What kinds of new users are potentially out there? What are their problems? What application areas are very messy and need formalizing?

Questions about the lifecycle: Research Strategies

Technique-Driven Research Primarily interested in a technique (e.g., machine learning), look for applications of it. Much of computer science is here. Problem-Driven Research Primarily interested in a goal (e.g., dynabook", databases, digital library), use whatever methods are appropriate.

Technique-driven - researcher learn about many applications. Problem-driven - researcher learn about many techniques.

12/12

How to write a research proposal?

CONTENT & FORMAT

Proposal Contents (An Example)

Long term goals. These provide the framework and justification for the proposal. Significance. Explain why the proposed work is significant. Who would care if it
succeeded? What difference will it make?

12/12

Specific goals. Specific things to be achieved during the grant period. Methods and Experiments. What methods will you apply to achieve the goals?
What experiments will you perform?

Feasibility. Evidence that the goals are achievable by you. Assessment of the
difficulty of the goals; prior experience with similar goals and methods.

Risks. What could go wrong? How will risks be minimized? Current State of Knowledge. What is currently known about this problem? Timetable. Demonstrates feasibility, especially within time constraints. Budget. How much will it cost? What are the items of expenditure (line item budget) Budget Justification. Every item of the budget should be essential to the success of
the project. Explain this.

Proposal Format (An Example)

Project Summary. 250-word abstract describing the project and its significance.

12/12

Goals and Significance. Describe the long term goals, the short-term goals (briefly), and
their significance. May need to briefly review prior work to set the stage.

Previous Research. Current state of knowledge. Include your own research here to
demonstrate your knowledge and project feasibility. Cite all potential reviewers. May need to include a tutorial on this area.

Specific Goals. May interleave with methods and experiments. Methods and Experiments. Give sufficient detail to assure reviewers of feasibility and of
your ability to do them. Risky outcomes should be accounted for in the design.

Timetable. Brief list of specific tasks and expected time when they will be completed. References. Budget and Budget Justification. This is usually a separate section. Other Materials. CV of investigators; letters of support from and collaborators. Letters
verifying unusual items (e.g., institutional matching funds, etc.)

In summary,
1. Defined research in S&T 2. Identified the types of research 3. Differentiated research vs. development 4. Defined unit of study in research 5. Discussed the research lifecycle 6. Discussed expected results/output of research 7. Discussed (some) of the research methodologies 8. Discuss project phases vs. research methodologies 9. Mapped project to required supporting skills 10. Identified research project design criteria 11. Discussed factors in choosing a research problem 12. Shown example research proposal content and format.

AFIP Fall Joint Conference, 1968

Douglas C. Engelbart

Answer. This is the first mouse.

Quotes:
Copying from ONE source is plagiarism; copying from MORE THAN ONE source is research. (WRONG!) Research is TEN steps forward and NINE steps backwards. Genius is 1% inspiration and 99% perspiration.

The best way to learn about research is to do it.

A Dictionary of Useful Research Phrases

This is the popular tool for reading research papers seen tacked up in offices, labs, and above copying machines everywhere. It has been reported to me that it originally appeared in a scientific journal article, Graham, Jr., C. D., Metal Progress, Volume 71, Number 5, May, 1957 (http://smurman.best.vwh.net/soga/misc/research.html) Research Phrase It has long been known... A definite trend is evident... Of great theoretical and practical importance... While it has not been possible to provide definite answers to these questions... Three of the samples were chosen for detailed study... Typical results are shown... These results will be shown in a subsequent report... The most reliable results are those obtained by Jones... It is believed that... It is generally believed that... It is clear that much additional work will be required before a complete understanding of the phenomenon occurs... Correct within an order of magnitude... It is hoped that this study will stimulate further investigation in this field... Thanks are due to Joe Blotz for assistance with the experiment and to George Frink for valuable discussions... A careful analysis of obtainable data... Translation I didn't look up the original reference. These data are practically meaningless. Interesting to me. An unsuccessful experiment, but I still hope to get it published. The results of the others didn't make any sense. The best results are shown. I might get around to this sometime if I'm pushed. He was my graduate assistant. I think. A couple of other people think so too. I don't understand it. Wrong. This is a lousy paper, but so are all the others on this miserable topic. Blotz did the work and Frink explained to me what it meant. Three pages of notes were obliterated when I knocked over a glass of beer.

Thank you!