Interpreting & Documenting Research &

Findings
Published by the Universities of Edinburgh, Glasgow and
Strathclyde
W.L. Wilson
Acknowledgements
The material from this booklet has been developed from discussion groups
and interviews with the research staff of Glasgow and Strathclyde
Universities
The advice and contributions of Dr Avril Davidson, Mr Keri Davies, Prof
George Gordon, Mrs Janice Reid, Dr Alan Taylor and Mrs Sheila Thompson
are acknowledged.
The advice of the project Steering Group: Prof Michael Anderson,
University of Edinburgh; Dr Nuala Booth, University of Aberdeen; Dr Ian
Carter, University of Glasgow; Ms Jean Chandler, University of Glasgow; Dr
Avril Davidson, University of Glasgow; Prof George Gordon, University of
Strathclyde; Prof Caroline MacDonald, University of Paisley; Prof James
McGoldrick, University of Dundee; Dr Alan Runcie, University of
Strathclyde; Prof Susan Shaw, University of Strathclyde; Dr Alan Taylor,
University of Edinburgh; Prof Rick Trainor, University of Glasgow is also
acknowledged.
The project was funded by the Scottish Higher Education Funding Council.
Other titles in Series
Gaining Funding for Research
Gathering and Evaluating Information from Secondary Sources
Preparing the Research Brief
Universities of Edinburgh, Glasgow and Strathclyde 1999
Cartoons D. Brown & W. L. Wilson
ISBN 0 85261 688 0 Printed by Universities Design and Print

Introduction
This booklet is one of a series of four aimed at researchers in the early
stages of their career life cycle. The comments within the booklet are
based upon information collected at a series of discussion groups and
interviews at Strathclyde and Glasgow Universities. The questions put to
the discussion groups were based broadly upon the performance criteria
and knowledge requirements identified in the report "Draft Occupational
Standards in Research" (Gealy et al, 1997).
The booklet is in two sections. The first section, "Interpreting Research
Results and Findings" considers various aspects concerning the
interpretation of results. Generally the section considers how to confirm
the reliability and analysis of results, the avoidance of bias or overinterpretation of results, and the identification from the results of potential
areas of future research.
Section two, "Documenting Research Results and Findings," examines
methods of presenting research findings, the physical aspects of record
keeping, and what should be recorded within research records both to
ensure their value to the researcher and to ensure that they are legally
and ethically correct.
The booklet is not intended to be read in one fell swoop, but rather to be
dipped into as and when the occasion arises.
Both sections within the booklet are subdivided into subsections each of
which consist of:

Introduction
Points of advice, and examples from experienced researchers to
highlight these points (colour linked). Information for the second
section was collected through a series of interviews and discussion
groups, which were formed from lecturers, PhD students, and
Contract Research Staff (CRS).

Bullet points which highlight the main points. The bullet points refer to the
points and examples preceding them.

The booklet is not intended to be exhaustive or definitive. The issues

raised are those which most exercised the minds of the researchers
providing the comments for its preparation. These comments do offer
interesting contrasts of opinion, either because commentators disagreed
about the way to approach a certain issue, or because researchers from
different subjects took different approaches in their methodology. The
nature of the examples provided in the booklet are a reflection of the
interests of those taking part in the discussions and interviews, and
possess no greater significance than that.

Contents
INTERPRETING RESEARCH RESULTS AND FINDINGS......................4
HOW DO YOU CONFIRM THE RELIABILITY OF YOUR RESULTS?..................................4
Introduction.......................................................................................... 4
Points to Consider................................................................................ 4
HOW DO YOU AVOID GETTING INTO A RUT WITH YOUR ANALYTICAL METHODS?...........6
Introduction.......................................................................................... 6
Points to consider................................................................................. 6
HOW WOULD YOU DEFINE INTERPRETATIVE METHODS?..........................................8
Introduction.......................................................................................... 8
Definitions of "Interpretative"..............................................................8
HOW DO YOU RECOGNISE AND AVOID BIAS IN YOUR INTERPRETATION OF YOUR
RESULTS?.................................................................................................... 9
Introduction.......................................................................................... 9
Points to Consider................................................................................ 9
HOW DO YOU EVALUATE YOUR RESULTS IN THE LIGHT OF THE OBJECTIVES OF YOUR
ORIGINAL PROPOSAL?..................................................................................11
Introduction........................................................................................ 11
Points to Consider.............................................................................. 11
WHEN DO YOU THINK UNCERTAINTY MAY ARISE OVER RESULTS AND THEIR
INTERPRETATION AND, HOW DO YOU ENSURE THAT YOUR CONCLUSIONS ARE FULLY
JUSTIFIED BY THE RESULTS?...........................................................................12
Introduction........................................................................................ 12
Points to Consider.............................................................................. 12
HOW DO YOU IDENTIFY POTENTIAL AREAS OF FURTHER RESEARCH FROM THE RESULTS?
............................................................................................................... 14
Introduction........................................................................................ 14
Points to Consider.............................................................................. 14
DOCUMENTING RESEARCH RESULTS AND FINDINGS....................17
WHAT TECHNIQUES DO YOU USE TO PRESENT YOUR FINDINGS, AND POSSIBLE AREAS OF
FUTURE RESEARCH TO OTHER INTERESTED BODIES?...........................................17
Introduction........................................................................................ 17
Points to Consider.............................................................................. 17
HOW DO YOU RECORD YOUR RESEARCH AND FINDINGS? ARE THERE METHODS OF
RECORDING THAT YOU WOULD AVOID?............................................................18
Introduction........................................................................................ 18
Points to Consider.............................................................................. 18
WHAT DETAILS DO YOU PUT IN YOUR RESEARCH RECORDS? WHAT DETAILS SHOULD
NEVER BE MISSED OUT OF RECORDS, AND WHY?...............................................20
Introduction........................................................................................ 20
Points to Consider.............................................................................. 20
HOW DO YOU CONFIRM THAT YOUR RECORDS MEET ALL RELEVANT LEGAL AND ETHICAL
REQUIREMENTS?......................................................................................... 22
Introduction........................................................................................ 22
Points to Consider.............................................................................. 22

Interpreting Research Results and Findings

How do you confirm the reliability of your results?
Introduction
The exact nature of what is reliable will vary from field to field.
Mathematical proofs, which are unusual in that there is an absolute right,
are usually developed over years. In other fields, e.g. social planning and
architecture, there may be no absolute right or wrong, and the
confirmation, or otherwise, may take 30 years of urban development.
Communication, experimental repetition, alternate approaches, good
background knowledge will all be applicable in some fields, but are
unlikely to be applicable in all fields.

Points to Consider

The most important initial stage is to be aware that your results

may not be reliable. Blind faith does not make for good
investigative research. Results may be misleading for a wide
range of reasons, e.g. an atypical sample, equipment error, or the
simple vagaries of animal behaviour. The latter point is nicely
summed up by the Harvard Law of Animal Behaviour: Given
precisely controlled conditions, the animal will do as it damn well
pleases.
Example: During a study of prostitution habits the researcher found that
it was difficult to obtain reliable data on condom use. She could ask till she
was blue in the face, and in as any different ways as she could think of:
one-on-one interviews, focus groups, whatever. All interviewees reported
100% condom use unless they happened to burst. Yet it was obvious to
the researcher that there were women who were working without
condoms.
Peer review is a basic step in checks of reliability. Asking colleagues who
have a sound knowledge of the field, but have not been as close to the
work as yourself, is an essential and basic check of reliability. Better to
have a colleague pick up a discrepancy at an early stage rather than a
paper or grant referee at a later one.
It is important to ensure that you have an adequate number of
repetitions within your experimental data (allowing for events
such as pseudo-replication). However, repetitions can add new
variables to the process. There is inevitably a balance between
the demands of the objectives and the demands of precision.
Example: The value of repetition was emphasised by one researcher who
remarked that he would not report on any data which had not been
confirmed within his own laboratory. For experimentation which required
statistical analysis the precise number of replications was dependent upon
the expected level of variability within the measurement. In order to
ensure statistical accuracy when it is not possible to run a number of
replicates simultaneously, the researcher reruns the complete experiment.
The precise number of repetitions depends upon the variability between

trials. His recent study examining the rearing of halibut fish highlights this
latter point. The experiment required four different tanks, each tank
providing a different environment. Normally the researcher would aim to
do these in triplicate, providing a total of 12 tanks. However, because the
experiment was within a production style system, the scale of the project
made simultaneous trials impossible, thereby requiring the entire
experiment to be repeated. This unavoidable variability requires an
increased number of repetitions beyond the average. On the other hand,
the big advantage of using a production style system is the avoidance of
the extra variables inherent in scaling from the very small upwards.
Using several techniques on the same sample provides an alternate form
of experimental repetition. Thus the reliability of tests for genetic
mutations in tumours is regularly checked by using three different
techniques on the same tumour sample.
Refer to previously published work and review your results within the
context of previous publications to obtain a feel for general trends. They
are some trends which may be expected to emerge. You must ensure
adequate quality controls to avoid bias, i.e., inadvertently creating the
result expected from trend. Bear in mind when checking reliability in the
light of previous trends that many breakthroughs in science have at first
been regarded as completely implausible. Plausibility is determined by
present knowledge.
It is important to be thoroughly familiar with the background and
content of the project. This is especially important where moving
into new fields, where some less than obvious fact may pass
unnoticed.
Example: Whilst out collecting crabs a postgraduate researcher observed
that some crabs reacted to other individuals of the same species by
rearing up and attacking. Lower shore crabs were more likely to be
aggressive than upper shore crabs. Several years later the researcher
discovered that there were actually two species of crab on those shores,
but that the two species were virtually identical. Fortunately the
researcher had not published the study, and learnt a valuable lesson
cheaply.
One engineer suggests the following summary for his own speciality:
a) Derive from first principles to establish plausibility. This would help to
highlight erroneous results.
b) Meticulous calibration.
c) Error analysis. (Error analysis being the system used to measure the
parameter will consist of different parts, each with an associated
uncertainty. When this uncertainty can be obtained from calibration,then
the uncertainty of the whole should be quantifiable.)
When working with human subjects it is essential to ensure that
the sample is as representative as possible in order to check for a

variety of different responses. One method of checking the

accuracy of responses is to rephrase the question and then
compare the new response with the answer to the earlier
question. It is important to ensure that the analysis of the data is
as inclusive of the varied responses as possible. One technique by
which this can be done is the inductive procedure of deviant case
analysis.
Example: Deviant case analysis proceeds through examination of the
universe of responses provided to a certain topic. If exploring the question
of condom use, a basic hypothesis may be that prostitutes would
encourage their clients to use condoms to ensure their own protection
against HIV. However, there might be women who do not articulate their
use in these terms at all, but refer to other reasons (e.g. they form a
means of distinguishing between the sex they have with their private
partners and the sex they provide to clients). The overall explanation for
condom use as a barrier would still fit but the argument would have to be
modified to incorporate the broader spectrum of responses. If, for example
it was observed that most women report other reasons for condom use
which do not fit within the barrier explanation then the original argument
must either be modified, or discounted entirely.

Bear in mind that your results may not be as you need them.
Check all results thoroughly.
Use alternative techniques to check results.
Examine your results in the light of other work.
Know your background information well.

How do you avoid getting into a rut with your analytical

methods?
Introduction

The best way to avoid becoming stuck in a rut is to remind yourself

regularly of the risk of staying there. Most researchers will develop
favoured techniques, and it is always easier to fall back on well used
comfortable techniques than to seek out new and novel approaches which
require the additional effort of getting up to speed. Communication and
keeping up with the literature (not just in your own field) appear to be the
best ways of remaining fresh.

Points to consider
Keep in touch with the research world around you. It takes some time for
new methods/techniques to appear in the literature. As with so many
aspects of research, networking is vital. Perhaps the most common reason
for the development of new techniques has been where the present
technique was very labour intensive. In these circumstances it is worth
asking around to see what other investigators are doing.
Look to other fields for inspiration.

Example: One research team studying the prostitute population of the red
light area in a large city decided to adopt the biological technique of
capture/mark/recapture. The study required identifiers, so the team
tagged each individually captured subject with a unique ID, then used
these identifiers to model changes in the prostitution population over a
period of time. This is a particularly elegant example as the technique
originated from an 1800s study in Paris in which the number of priests
was used to estimate the total population of the city. Then the technique
was adopted by ecologists to model animal population dynamics. In this
instance the technique has moved from social science research to
biological research and back to social science.
Discuss what you are about to do with your colleagues who may
well contribute some good ideas. Discussion can help you avoid
becoming enmeshed in minutiae and missing the bigger picture. It
may be helpful to brainstorm with a group of your peers on a
regular basis.
Example: One researcher remarked that she had been experimenting for
months with a new technique to identify differentially expressed genes, all
the while going nowhere. Then after several months she discovered that
throughout that period a colleague in the group had been arguing the case
for an alternative technique. Greater efforts at communication would have
saved her several wasted months.
For some fields, such as English Literature, opening a line of
communication with the author you are studying may provide a useful
insight into their work. For other fields, contacting authors of publications
will allow you to discuss new techniques being developed, or perhaps
highlight publications which you may have inadvertently missed.
Look for weakness in the methodology, for instance, is the
present technique of a lower sensitivity than required, can it be
improved? Regular reappraisal of the techniques used, and
consideration of their less satisfactory points, should help avoid
complacency.
Example: A research group was interested in measuring virus-specific
immune responses. When the researcher joined the group there were a
number of techniques available to measure antibody responses to the
virus. However, there were no reliable techniques available to measure the
T-cell response. Unlike antibodies, T-cells recognise virus infected cells, or
tumour cells, and kill them. The researchers first task was to develop such
a technique. He succeeded in developing a technique which was then
adopted globally. There remained concern that the technique was
underestimating the true magnitude of the host T-cell response. The team
is now in the process of designing novel assays to measure virus-specific
T-cells. Using these assays it will not only be possible to verify the data
that they (and other laboratories) have obtained, but the improved
sensitivity afforded by the new techniques will allow detection of T-cells in
circumstances which would otherwise have been overlooked.

There can be problems in securing funding for completely new

approaches.. However attractive a new methodology may appear, it is
important to ensure that the methodology will not discourage funding
bodies if it is included within your grant application. This can be a "catch
22". You want to be adventurous, but cannot move forward because
funding bodies or collaborators will be cautious of the excessive novelty
of your new idea. On the other hand, in the highly competitive world of
research funding, you may need that bit of novelty as an added attraction.
If in doubt it is well worth contacting your prospective funding bodies in
advance. Some funding bodies run schemes to promote "blue skies"
research such as the Research Council, Realising our Potential Awards
(ROPAs). Though original and novel are not one and the same, a pilot run
will move "novel to original" and help you convince the more sceptical
reviewer.

Keep up with the literature.

Networking is essential.
Look to other fields for inspiration.
Cast a critical eye over your methodologies, identify the weak
points, seek alternatives which ameliorate them.

How would you define interpretative methods?

Introduction
It became obvious during the discussions and interviews used in the
creation of this booklet that the definition of interpretative was not
consistent. The question "how would you define interpretative methods?"
was put to participants to try to gain some idea of the definitions of
different fields. In order to avoid interpreting the interpretations of
"interpretative" and inadvertently shifting the definitions towards a
biologists view of the world, this section has been kept in the form of the
original quotations.

Definitions of "Interpretative"

"This would partly be related to the way that the experiments have been
set up - you set up experiments with defined objectives, the interpretation
of which would initially be based on that background information. You
analyse results by plotting them in various ways, carrying out statistical
analysis and comparing them with your expected views from the
experiment."
"As an architect you interpret your model of parts of cities against a set of
criteria which you hope are generally agreed upon. There is a huge
amount of literature on what a sustainable city should look like, although
there is also huge disagreement. However, there are certain consistent
demands upon a city, e.g. public transport, low degrees of pollution and
eliminating congestion. You can set these as the targets for your models
and test your models to see how they influence these criterion, and to
what degree. But as the models are not real, there is no actual physical
proof. Thus there are two sets of interpretations, what criterion should be

used to judge the model and, in the absence of physical proof, how
accurately does the model reflect what would happen in reality. The
difficulty in our field is that everybody has his or her own set of criteria to
judge against, and so we never agree."
"I think if something meets with your understanding of the subject. In my
own research I have interpretative methods that would anticipate my
critics. It can be very objective in the sense that you can interpret
according to the aim of what you are trying to do. Thus in terms of
interpretative methods researchers need to be aware of how the results
would be interpreted, by the media, peers in research and the community
at large. This is important when the research involves some controversial
subject. A further advantage of attempting to foresee how some
arguments will be interpreted is that a prior response can be prepared."
"Interpretation in English Literature often possesses the implicit danger of
interpreting things along the lines of your own preconceived notions. In my
view, that has happened too frequently, with theories transposed onto
(and into) texts, and the resulting criticism has been not so much a
criticism of the text but an expression of the critics own opinions. So
interpretation becomes too greatly bound up with opinion. I think covering
yourself to anticipate your critics is necessary to a certain extent, in the
sense that your thesis must be as logical and consistent as possible, but
this should not be at the expense of your being totally inflexible, and
blinding yourself to any shortcomings in your thesis. By all means go into
your project armed with notions which will challenge the received wisdom
in your chosen area, but be aware of dealing with gaps in your own
argument too!"

How do you recognise and avoid bias in your interpretation of

your results?
Introduction
One researcher remarked that he wanted a particular solution because he
was sure that it was the correct solution, when in fact it was the wrong
solution. It was the interpretation of a brief for a housing development
scheme. He tried to test what was actually meant by the relationship of
the different functional elements within the scheme. In this instance he
thought he knew the answer because he had worked on similar schemes
previously. Thus, when told by a colleague that there was a mistake in his
interpretation he failed to check it. The housing scheme was developed to
the full, and then collapsed because of that mistake. It was, he remarked,
a painful exercise often remembered, never to be repeated!
Few of us can claim to be completely free of such bias, the following
section attempts to identify some of the areas where bias commonly
arises, and outline some techniques for its recognition.

Points to Consider

Bias can arise in the construction of the experiment rather than in

the interpretation. It is important to ensure that the experimental

design, or the behaviour of the researcher, does not introduce

bias long before the interpretative stages are reached.
Example: A team of ethologists were attempting to breed a more
intelligent strain of rat, intelligence being measured by maze learning
abilities. As the project proceeded it appeared that a superior strain of
intellect had been bred. At least that was the conclusion until the
techniques of the workers involved were more closely examined. The
researchers stroked the more intelligent rats before introducing them to
the maze, but did not stroke the supposedly less intelligent rats. Improved
learning was not a function of superior breeding, but rather of more
pleasant handling conditions.
Discuss your results and interpretation of these results with
colleagues. It is especially helpful to seek out colleagues from
different backgrounds and experience. It is important to ensure
that the review of your conclusions will be genuinely critical,
there is little value to be gained from seeking excessively polite
or friendly colleagues.
Example: One researcher remarked that a recently retired professor
tended to think in a different way to most of his colleagues. There were a
few colleagues who, when a document was put in front of them, would
react in a predictable manner. But this professor tended to throw up quite
different points from the document. He had a very different background
from the rest of the group, as well as having a wider range of experience.
He had worked in industry for a number of years, and had had a lot of
experience in vaccine development and trials, and marine biology. The
areas in which the researcher was interested, but from a quite a different
perspective.
In interview-based research, the perception of the interviewer
and his/her experience of life, has considerable potential to
radically colour his/her interpretation of events around them.
Example: The researcher who had been conducting a research project
studying prostitutes on the streets of Glasgow initially assumed that they
would be afraid of the police, when many did not care about the police at
all. The researchers interpretation of how they would react to being
apprehended by the police was influenced by her own background. She
stood to lose a lot by being prosecuted for anything, the prostitutes, on
the other hand, felt that they had little to lose.
Reflexivity is at the core of interviewing. To some extent
interviewing an individual is like looking in a mirror. There is a
strong tendency for the interviewees response to be coloured by
how they perceive the interviewer. It is not only how you perceive
the experimental subject, but how they perceive you.
Example: When interviewing drug addicts about needle sharing, the
addicts rightly construe the interviewer as being someone who thinks that
needle sharing is not a great idea. As a result, they will tend to deny
sharing needles because they do not wish to give a poor impression to the

researcher. Drug users require, in their search for drugs, good

manipulatory skills, and become very skilled social actors. They will often
only give an interviewer as much information as they estimate he/she
already possesses. As the researcher entered more deeply into the field
she gained a relatively deeper understanding of what was going on. In
becoming more aware of the tendency to present selective information,
the researcher established ways of getting beyond the surface
presentation of the facts.
Imagine that you are presenting your conclusions to your worst enemy,
where would they pick flaws, and how would you defend yourself against
their arguments? Anticipate your critics.
Bear in mind that an absolutely objective truth may be unobtainable,
perhaps only another kind of truth will be possible. The focus of your
research is complex, the interpretation more so.
Bias can appear through the unconditional acceptance of previous
work. It is in circumstances such as those described in the
example below which make networking and a broad awareness of
the background vital. Researchers should never accept blindly the
bias of the past, if you think something is wrong - perhaps it is.
Example: During a PhD viva the examiner asked about a certain result
which the research student had quoted from a classic mathematics
textbook, remarking, Of course, you know the proof is wrong. The
researcher recalled having struggled with this proof. He had never
managed to follow the proofs logic, but as it was cited in a classic
textbook, accepted the result as true, and quoted it. It was not a big issue
at the time, but there is an element in mathematics (as in most subjects)
where a theorem can be correct almost through folklore. Even if you go to
the source of the proof you will find reference to some other source, or
simply a statement of the result but no proof. More often than not these
results are correct, but there have been occasions when results have
made it into the folklore of mathematics. It raises the issue of how far back
you should go before you accept that a result is true without having to
iterate the entire proof yourself.
Seek out the counter arguments to your own interpretations, and consider
whether or not you have given the alternates a fair hearing.

Take advice on your interpretation, especially from colleagues with a

different view of things to yourself.
Be aware that your perception of the results is coloured by your life
experiences and expectations.
Equally you must be aware that just as your interpretation is biased
by your experiences, so can the interviewees responses be biased
by their perception of the interviewer.
Do not accept historical wisdoms blindly.

How do you evaluate your results in the light of the objectives

of your original proposal?

Introduction
Adaptation of new methodological techniques was identified as a frequent
source of problems. Either the methodology proves unable to deliver all
that it promised, or it takes too long to train the staff/yourself in the use of
the new techniques. The next most likely cause of failure was simply
setting ones sights too high, a common temptation with the increasing
competitiveness of winning grant funding.

Points to Consider
The use of untested methodologies can result in a lower than expected
level of success.
Example: The researchers were examining the genetic basis of the
production of particular types of toxin by bacteria. This was of interest as
these were toxins which were generally considered to be produced by
algae, but not by bacteria. The methods that the group chose to use were
not as well developed as they had anticipated. They found that the
sensitivity of the methods was not good enough to allow the screening of
bacterial toxin production, although the method worked well for algae,
which produce much more toxin. To effectively screen bacteria the project
required a screening system which could run 1,000 samples rather than
the 50-60 needed for algae, a demand beyond the capabilities of the
technique. In retrospect, overconfidence in an untried methodology
resulted in the projects failure.
It can take longer than anticipated to train or re-train and provide the
necessary level of new experience required for staff (and yourself) to
change fields. Time must be allowed within your proposal for training,
failure to do so can result in projects being less successful than
anticipated.
Were the objectives unattainable? This can occur when relying on
claims made for techniques without any definite evidence that
these claims are accurate. This may occur for a range of reasons
e.g., extrapolation beyond the reasonable range of the original
results, altered conditions, attempting to push the system beyond
its capacity.
Example: The research team had been working on the microbiology of
turbot larvae, which are susceptible to very large losses when they are at
the first feeding stage. A new project was proposed which was to be run in
collaboration with a European company. The funding was granted on the
grounds that the commercial company claimed to possess an improved
technique for preparing water to allow the rearing of larvae in defined
conditions. However, major collaborative trials using the methodology
failed to reach the standards which the company had claimed were

possible in their publications. The reason for the failure was probably that
the research team had inadvertently exceeded the capacity of the system
and extrapolation of the technique to larger systems was impossible.

How certain are you that the claims made by others are accurate?
Ensure an adequate allowance of time for retraining and developing
untested techniques.
Consider that you may not have failed, perhaps the results were not
positive, but that in itself need not constitute failure.
Unexpected results may result in the research progressing along a
different route.

When do you think uncertainty may arise over results and

their interpretation and, how do you ensure that your
conclusions are fully justified by the results?
Introduction
There are a variety of reasons for which interpretation and results may
become confused or over-interpreted. These range from decisions made
regarding modification of the data, to allowing the expectations for the
results to blind the researcher to their actuality. This section offers various
suggestions on how to avoid such problems arising and, as with so many
of the discussions in these booklets, emphasises that consultation with
colleagues is vital.

Points to Consider

Avoid overstating the result: inconclusive results should not result in

conclusive statements.
Present your results and conclusions to colleagues and ask them for
comments. Make certain they understand that you are looking for
constructive criticism, and not just a pat on the head.
A good rule of thumb may be to go for the simplest explanation.
One method of avoiding confusion between interpretation and results is to
leave the work alone for six months. After that period when you have
forgotten some of the background, and are free of any unhelpful habits of
thinking you may have inadvertently fallen into, then you should be better
able to spot any confusion between interpretation and results, or for that
matter bias in interpretation.
When repeated trials provide conflicting results any decisions taken
regarding the relative reliability or accuracy of the various results is an
interpretation of the results, and should be noted as such.
Clearly differentiate between the results per se and your
extrapolation/interpretation of them.

A common error is to confuse a correlation between two variables and an

actual cause and effect. The magnitude, significance and direction of the
correlation is the result: conclusions regarding cause and effect are
interpretation.
It is important not to let your expectations of results
predetermine your view of them. Firmly drawn conclusions should
be sustainable by the data alone, and not reliant on the theories
of previous work.
Example: A researcher collated a large set of data on school childrens
use of drugs and alcohol. This was initially analysed using logistic
regression. The results appeared to fit with ideas which she had previously
expressed in a literature review. Both results and discussion supported the
researchers interpretation of the literature. She later re-analysed the data
using a different series of tests. The explanation of the data she provided
on the basis of these tests, although informed by the literature review,
was not dictated by it. The re-interpretation was much closer to the data.
For example, peer formation is significant in any behaviour but particularly
in relation to drug use. Previously the researcher had been enthusiastic
about theories that suggested that core family background would allow
predictions of negative peer engagement. The literature had provided
fairly clear evidence that this was the case, and although she had not
claimed that the data demonstrated this, she had used the theory as a
possible explanation of the data. However, her re-analysis suggested that
the original conclusions had been an over-interpretation of the data.
Perhaps the most common reason for conclusions and discussion
not being justified by results occurs when the discussion is
extrapolated well beyond the limitation of the results.
Example: A paper described research involving sampling maturing fish
over the course of a year or more and looked at changes in the amount of
fat stored. The paper recorded differences between maturing and
immature fish, especially in the patterns of depletion and re-building of
mesenteric fat stores. The discussion section of the paper considers this in
the light of ecological implications of fasting on maturation rates.
However, more than half of the lengthy discussion proposed a model for
the hormonal control of maturation. Hormonal control had not been
mentioned prior to the discussion, and the research was only tangentially
related to it, leaving the reader with the impression that the authors had
failed to complete a sufficiently comprehensive review article, and simply
added an incomplete review at the end of the paper.

Avoid any confusion in data recording.

Correlation is not proof of causality.
Decisions made regarding the reliability of the results should be
labelled as interpretations not results.
Do not allow your expectations to predetermine your conclusions.

How do you identify potential areas of further research from

the results?
Introduction
Opportunism is one watch word in the identification of potential areas of
new research. Opportunism takes many forms, e.g. capitalising on current
trends and fashions, and spotting weakness in present methodologies. The
second key is to keep an eye on the long term objective. The ability to
avoid becoming lost in the woods can be improved by a regular
interchange of ideas with colleagues.

Points to Consider

Going away to a major conference can be a good way of focusing on the

results of the past year. It both allows you escape the distractions of your
usual routine and, in conversation with others, identify the direction in
which your field is heading.
Example: "At the conference last week, one area of interest was in a
particular fish disease and I had gained quite a lot of information on that.
When you tie it in with existing information you can spot the areas which
are obvious for development. For example, this particular organism is a
rickettsial infection. Rickettsiae are bacteria which must replicate within a
eukaryotic cell so they are rather unusual. This is a fish disease which is
particularly important in Chile but has been reported in Northern Europe
as well and what we are interested in is identifying particular components
of the outer membrane of the bacteria which stimulate an immune
response, so from the meeting I got a clear idea of which antigens to
concentrate on."
There are short term objectives and long term objectives. It is the
examination of the latter which is most likely to indicate the potential for
further research.
Be prepared to shift the focus of your research as the political,
social and scientific priorities of the wider community move. It is
not unheard of for projects which have been rejected for funding
to become, at a later date, greatly sought after by the funding
bodies.
Example: A researchers first project included an examination of the
behaviour of drug injectors. At the time this was just a small part of the
project, but with increasing HIV awareness injecting behaviour gained a
much higher profile politically. It became obvious that this was an area in
which research funding would become available.
The identification of weakness in present techniques, e.g. high
cost of production, or the potential to develop a more efficient
system, will often provide new avenues of research.
Example: The majority of currently available vaccines are based on either
inactivated virus or bacteria, or comprise a synthetic or "recombinant"
protein which has been produced in bacteria. The production costs for

these types of vaccine are high, and some improperly inactivated vaccines
have been responsible for outbreaks of disease. These problems have
encouraged researchers to evaluate the potential of nucleic acid or DNA
vaccination as an alternative. Using this technique the DNA is injected
directly into the animal or person, there is no risk of infection since the
whole virus or bacteria is not used, and there is no costly production and
purification of recombinant protein. To optimise the immune responses
produced following vaccination a chemical called an "adjuvant" is often
included. Recently, the team has pioneered the use of genetic adjuvants in
veterinary medicine. The results have now opened up a whole new area of
research, not only in the application of this technology to other infectious
agents of man and animals, but also in improving our knowledge of the
way the genetic adjuvant is exerting its effect.

Keep an eye focused on the longer term objectives of your project.

Monitor the shift in public and political priorities, the timing of a
proposal can be vital.
Look for weakness in the existing and preferred techniques.
Keep discussing your work, its progress and its potential with your
colleagues.

Documenting Research Results and Findings

What techniques do you use to present your findings, and
possible areas of future research to other interested
bodies?
Introduction
Other interested bodies are a varied group, both in their understanding of
your subject, and their specific interest in it. In order to achieve maximum
impact it is important to vary your approach according to interest and
understanding. The following section considers when it is appropriate to
take different approaches to presenting research findings, and offers
suggestions as to what these alternates may involve.

Points to Consider

Industrial workshops can be a useful way of putting your message across

to potential funders as workshops often allow a much freer exchange of
information than conferences. This occurs, at least in part, because the
presentation is to potential funding bodies, whereas at conference the
presentation is to potential competitors and less recent results are often
presented. You would have more opportunity to describe your capabilities
and past achievements. However, when visiting industrial workshops it
may be wise to take advice on intellectual property rights before making
your presentation. Consult your Research Support staff who can advise on
the institutional policies in this area.
When seeking research funding it is important to make your objectives
absolutely clear. A good technique is to provide a very succinct list of aims
and objectives. When presenting to potential industrial collaborators,
funders or users of your research, a one page A4 summary (in bullet point
form) of your objectives, and the commitments that you require from the
industrialist, can be very helpful.
If you are talking to a group of people who know little of your subject then
it becomes especially important to avoid jargon. Use clear, plain English.
Get a non-expert to review your presentation or paper.
How you dress may be important, the more casual dress code common in
academia will certainly be less acceptable to potential funders from
industry.
Try to target your audiences interests, tailor your presentation
accordingly. Talking to members of the audience will give some
idea of the sort of language they use, what they are likely to be
interested in, and what they will understand.
Example: When a researcher presented data on her project to her Cooperative Award in Science and Engineering (CASE) funding partner, a fish
farming company, she altered the emphasis of the presentations. The
presentations were more or less the same as those she gave at scientific
conference, but with one significant difference. Scientists think in terms of

the length of fish, whereas fish farmers think in terms of the weight. Thus
for presentations to fish farmers she re-analysed her data to take account
of the difference in approach.
For a larger audience one researcher remarked that he would use
Powerpoint and a slide projector, but for a smaller more informal audience,
a board and a pen or overheads. If the lights are on you can better gauge
if your audience is interested and enjoying your presentation. Standing
and writing also has the further advantage/disadvantage of adding to the
informality of the proceedings.
As all who have written a thesis or a major report will know, most people
will never read them in their totality. One solution may be to present an
executive summary of the research. This increases the likelihood of its
being read by focusing all of the ideas into a short and concise section, but
of course it leaves out all the proof, evidence, arguments and counter
arguments. Multi-media productions offer considerable potential in this
area. Although more complex and expensive to produce, they allow
readers to look through your research and pick out what interests them, by
jumping from one point to the another.
When seeking funding consider emphasising the benefits rather than the
features. Thus instead of a fully integrated software package which is
easy to use, highlight the benefits, e.g. minimal training required, financial
savings.

Know your audience

Target the interests of your audience and be prepared to vary your
approach according to those interests.
Attend informal workshops set up by relevant industries.
Always produce executive summaries of large reports.
When presenting to industrial and other end users keep it simple
and straight to the point.
Avoid jargon.

How do you record your research and findings? Are there

methods of recording that you would avoid?
Introduction
The emphasis of this section is centred upon the physical aspects of
record keeping. The second half of the discussion considers the
importance of duplicates, accessibility and longevity of the records, and
when records can be discarded.

Points to Consider

Records must possess longevity. Use good quality paper, which should last
at least 30 years. Do not use pencils or strange coloured inks, the ink must
not be water soluble or solvent reactive, it should not smear and should be
light stable (BTG plc)

Research records should be kept in a form which ensures that their

authenticity can be appropriately defended. Claims of originality and
scientific priority are best supported by records whose provenance and
date are beyond reasonable doubt. This is especially important for the
protection of Intellectual Property Rights (IPR) when negotiating contracts
for the exploitation of research results, and seeking to establish ownership
of background IPR. To fulfil such obligations to maintain accountable and
dependable records, best practice suggests that all experimental data
should be meticulously and permanently recorded, in a bound notebook
with numbered pages, with all entries dated, signed and witnessed.
Computer printouts and instrumental data printouts should be
incorporated permanently into the notebook. Where these are pasted in,
the witness should sign and date across the join.
Arrangements should be made to keep duplicates of all irreplaceable data
records. Important material stored on computer should be systematically
backed up, ideally there should always be at least three copies, one of
which is off site. Loss of experimental records, data, grant applications,
and drafts of publications in fires, floods, or other disasters can vary in
effect from extremely frustrating to catastrophic.
Loose leaf laboratory records can be very useful if the data contains lots of
ancillary documents (e.g. photographic plates, spectrophotometer
printouts or sample interview sheets). Such records are always difficult to
file and a loose leaf folders can serve as a good supplementary, or
replacement, to a laboratory notebook, though a loose leaf folder is less
convincing evidence in any IPR disputes. The indexing of ancillary data,
such as that described above, is critical. At the very least, each item
should be annotated with the data and location in the notebook of the
corresponding experiment. Many items of computer-controlled equipment
provide printouts of instrument settings, as well as date and time. Ensure
that the clock is correctly set, it may be important in future IPR debates.
Make certain that you, or the computer, compensate for leap years and
seasonal time changes(Beynon, 1993).
Keep duplicate records, if you are using electronic records make sure that
you have off-site as well as on-site records. Ideally all computer records
should be in triplicate, the hard disc, one floppy on-site, and one floppy
off-site. Remember though that floppies, if unused for long periods, can
cease to function properly, and data may be lost as a result. It should be
borne in mind that because dates can easily be altered on electronic
records they are poor evidence in the event of IPR disputes.
"It is ironic that many laboratories seem to give more consideration to the
storage of reprints, which are copies of existing literature, than to
notebooks, which are irreplaceable originals" (Beynon, 1993).
Make sure your records are well labelled. It may seem obvious now, but a
year or so down the line the chances of remembering what the data
columns represent are slim.
Consider building your research records like a tree, allowing connecting
ideas to follow through a particular branch of the tree. Once you have

discarded the other branches over the years, the remainder can be
discarded. On the other hand, archiving old material, even if you do not
believe you will return to it, may allow you to refer back to a solution to a
problem which you have had to deal with previously.
Make sure that your record system is accessible, it is of little use if you
have to walk through half the building to access it, or if you have one type
of computer system at home and another in the office. Similarly, try not to
put it on some obscure computing system that is likely to vanish within the
next few years.
Take care of where and how you record your list of things to do.
Consider mentioning your objectives to others at coffee time - in
six months time they may remind you.
Example: One researcher admitted that on moving office recently she
found a list of things to do dated four years earlier - none of which she
had done.

Results and methods should be recorded in a manner which can

leave no doubt as to their authenticity.
All records should be signed, dated, and witnessed.
Keep duplicates.
Record labelling must withstand the test.
Records must possess physical longevity.
That list of things to do must be high profile and visible.

What details do you put in your research records? What

details should never be missed out of records, and why?
Introduction
Consistency is an important aspect of record keeping. Records should be
kept in a consistent manner regardless of the experiment. Failure to do so
is likely to result in records becoming incomprehensible a few years down
the line. Records, if they are to be of real value, are to be kept over a
period of years. A consistent style of record keeping will reduce the risk of
the records appearing incomprehensible if there is a need to examine
them after a prolonged period of neglect. Exactly what should be
incorporated within the records depends primarily upon their end purpose.
Ideally, your main record collection should allow you to repeat any course
of work, be able to secure your IPR and help defend you against unjust
accusations. For further explanations on material on patent law, etc.
please refer to THEROS: Technology Ventures - Intellectual Property
Guidelines (1998). On the whole it is better to err on the side of caution
when deciding what to leave in and what to leave out.

Points to Consider

US and UK patent laws are not identical. Thus in the US evidence of the
date of conception of an invention, and proof of diligence in its reduction
to practice is required for patenting.

"Errors and mistakes should not be erased or obliterated beyond

recognition. Neither should liquid paper be used. Simply crossing out an
error so that it is apparent what the error was should be adequate. Explain
all errors and mistakes as they occur and initial them. Never remove
pages from the notebook." (BTG plc)
Record novel concepts and ideas relating to the work though avoid the
expression of opinions (BTG plc).
Many of the researchers in our discussion group kept a day book in which
they recorded everything they had done that day. The book might include
the chemicals used and in what quantity, and anything that had gone
wrong. Any results which come from a printer should be put into that day
book (if these are perishable printouts, copies must be made). Tables of
results would go into a separate folder.
When recording data it is self-evident which data set is which. Five years
down the line and it is highly likely that you will have forgotten which data
set is which. It is vital to sort and clearly label computer held data from
day one, especially if the computer records will include earlier and later
versions of the same data set. Records should be in a form which can
readily be understood by everybody. This is necessary partly because in
debate over patent rights it is vital that the records should be easily
understood, and partly for the reasons above - you will feel silly if at
sometime in the future you admit you cannot understand your own
records.
Log the incidental. There may be contextual events or activities which
affect the data, climatic conditions, on-going political/newspaper
campaign, delay in sampling (Brown et al., 1995).
Although the comments made in the example below were valid,
the potential for the records to serve more than one purpose
should be borne in mind. The main aims were to ensure that the
work was repeatable, and that the details were adequate for
protecting intellectual property rights (however, see "Diligence").
In this light the criticism of research assistants including
irrelevant details was valid. The records would allow the
researcher to point out that while they accepted that a particular
objective was not met, nonetheless, it was not their
responsibility.
Example: Include anything which is remotely likely to be required or
useful. You come to recognise through experience (which does not take
long to acquire) when you have failed to record points that are going to be
needed in as much detail as possible. However, it is interesting to see the
irrelevant details that some people have recorded. Huge amounts of
irrelevant information, for example, I was away on holiday, or something
has not yet arrived. The critical issues are the date, and a couple of lines
on the objective and methods used. In terms of the details, the
experimental method (especially if it deviated in any way from the
standard protocol), and the results, should go straight into the book which
is the day-to-day record - in the book not just on any piece of paper.

However, BTG would tend to support the researchers method of

recording. "Diligence in the reduction to practice of an invention means
that, as far as possible, generally steady, uninterrupted and constant work
occurred following the conception of an invention. In an interference
action (where IPR is challenged) periods of inactivity could lose the case,
especially in a situation where each day is critical. All activities must be
logged, even if it is only to note that you were waiting for, say, sample
analysis that resulted in delay in the proceedings," (BTG plc).
When recording the results of pilot studies be careful not to be more lax
than is normal in terms of the quality of the information recorded.
Data must be recorded carefully. Resist the temptation to record
the data in rough form and transpose it to your notebook at a
later date - this provides an extra opportunity for the introduction
of errors (assuming you get round to it in the first place). When
recording data from instruments, note the settings on the
instrument panel.
Three examples:
(i)A fluorimeter value of 10.4 units is meaningless and cannot be
rechecked without notes on scale widths, scale expansion factors,
wavelengths and all other machine settings.
(ii) During electrophoresis note the current and voltage; this will allow you
to calculate the resistance of the gel, and spot a buffer of incorrect
conductivity.
(iii) In a chromatography run, note flow rate, column back pressure,
detector settings, column type and, if there is more than one column of
that type in the laboratory, the serial number of the column (Beynon,
1993).

Do not exclude data which you only think may be significant in the
future, err on the side of caution.
Keep record keeping consistent.
Remember that you must be able to recognise data files not just
next year but in three or four years time.
You can use your records not only to record experimental details,
but also to cover yourself against future unfair accusations.

How do you confirm that your records meet all relevant legal
and ethical requirements?
Introduction
The ethical problems of working with human subjects are considered not
only from the perspective of their rights, but whether or not they
understand these rights. The final section of the discussion considers
where sources of ethical advice may be located.

Points to Consider
In ethical terms make certain that your volunteers understand the ethical
promises you have made them. Although your explanation may seem
clear to you, ask the subjects some questions on what you have promised
them, they may not have understood after all.
Example: One researcher working on prostitution and HIV explained to all
her subjects that a double blind system was being used. This meant that
the results of the HIV test could not be identified with the person from
whom the samples had been taken. Despite this, the researcher regularly
received requests from her subjects as to their HIV status, thus it became
obvious that many of the volunteers had not understood the ethical
commitments that the research team had made.
Research upon human beings can carry the added complication of
political overtones. Data, results and conclusions should not be
modified for political purposes, but neither can researchers deny
that their conclusions are liable to be used for such purposes.
Especially where vulnerable groups are involved, consideration
should be given as to how the project will be presented.
Example: Prostitutes are a stigmatised and vulnerable group. The group
the research team tested for HIV prevalence came up with a relatively low
percentage, about 3%. However, had that percentage been 50% then that
would have raised a completely different set of ethical issues.
Animal experimentation is a continuous process. Once begun the animals
(even during non experimental periods) require constant supervision. This
must be taken into account when planning the project. Appropriate
experimental records to meet the requirements for the annual Home
Office returns must be kept. Organisations such as the Ministry for
Agriculture and Fisheries (MAFF) will want to know whether you have
conformed to standards such as ISO 2000 or Good Laboratory Practice
(GLP), ISO being a standard for experimental procedure and the recording
of data. GLP again has particular requirements. These often tend to be in
the form of standard checks. Thus results may have to be confirmed by a
superior who may have to initial a page in a notebook to say that they had
read and checked the records. If there are Home Office requirements then
there is a very clear line of responsibility and there will generally be
someone in your Department responsible for Home Office requirements.
Your institution will have staff who are responsible for such issues. Be sure
to seek out their advice at an early stage of project planning.
It is not unheard of for data to be destroyed when they have not provided
the expected results, or in order to avoid a closer scrutiny of conclusions.
This is unethical. Your institution will have a policy on scientific conduct.
Make yourself familiar with this document. Your Research Support staff will
be able to advise you on such issues.
Plagiarising the work of others is unethical.

Example: One researcher recently published her dissertation - a German

dissertation - but when she went to the examination in Germany
discovered that her work had been published by a member of staff.
You must obtain written permission from respondents to cite extracts from
interviews in publications (even if they have been anonymised).
If you are concerned with legal aspects of animal research your
institution will have an office designated to answer such
enquiries. If they cannot help, you must contact the Home Office.
They are very helpful and will usually give you an answer
immediately.
Example: When the researcher was working on a fish pathogen
(rickettsiae) he telephoned the Scottish Office to clarify the situation
regarding the import and export of these bacteria and whether or not a
licence is required, because it is also the provider of such licences.
For information on safety requirements there will always be
someone within your Department who will be able to provide you
with the necessary information. Your institution may have a
central Health and Safety Office
Example: In microbiology the pathogenic category of the material you are
using should be identified before the project commences. The
pathogenicity will have been categorised one, two, three and so on. Most
institutions will have little difficulty for categories one and two, but
categories three and upwards require special facilities. These categories
tend to be reasonably virulent organisms which require special conditions
for safely recording growth. It is important that you clarify the position
before you begin. This applies to most high risk material, e.g. for ionising
radiation, someone within your Department will be delegated to look after
radiation matters, but your institution will probably have a Radiation
Protection Officer.
If your subject is to be allowed to make an informed decision on whether
they wish to participate in the experiment then you should respect
participant/research subject autonomy, i.e.
doing what you said you would do, and nothing more or less.
The following example from the experiences of Stephen Waters
(Bell, 1993) provides interesting insights into the difficulties of
carrying out research on ones own colleagues. Although the
example is predominately one of gathering the basic data,
nonetheless, there are points to be learnt regarding what
happens to that data after they become secondary published
data.
Example: Stephen Waters was a teacher who decided, as part of an Open
University course, to investigate the role of his own Head of Department.
He went through a fairly prolonged negotiation period to reassure
colleagues as to his trustworthiness before embarking on his programme

of research. Of interest here are some of the comments he later made

regarding the guarantees he had given prior to undertaking the research.
He had promised all participants an opportunity to verify statements prior
to production of the final report. This proved almost impossible, most
participants not having time to read the entire manuscript. Therefore,
lacking time to identify all their comments within the manuscript. He had
further promised all participants a copy of the final report, which
ultimately cost rather more than anticipated. The ethical agreement he
reached with contributors was only made verbally. This created problems
at a later stage, when it transpired that none of the contributors could
precisely recall the conditions agreed upon. In retrospect he regretted not
providing them with a written copy of the agreement. However, it was in
seeking to publish the data that the greatest problems arose. All
contributors had been promised anonymity. A promise which could be met
externally, nobody outwith the school could identify the contributors.
However, as all the contributors came from the same school, it proved
impossible to provide anonymity internally.
Careful consideration should be given to any possible conflict of interest,
or the appearance of such. If this problem arises during the course of your
research, experienced advice should be sought.
Independent work can bring researchers into conflict with their institution.
This may occur if the independent work utilises the results of research
which the institution may regard as being part of its intellectual property.
Conflicts of interest may arise when a person involved in a research
project has the opportunity to influence institutional funding decisions
impinging upon that project. In any of these types of situation it is
essential to get advice from your own Research Support staff.

Make certain that human subjects understand the ethical conditions

under which you are operating.
You have a duty to the people you are working upon, consider how
your results will affect their lives.
Keep a record of any agreements made.
Seek out the person within your department who is responsible for
safety issues.
Do not plagiarise.
Remember that legal and ethical are not one and the same, and
that the absence of a written code does not excuse an absence of
ethical behaviour.
If there are doubts about the legal aspects of animal
experimentation then the Home Office is the place to go.
All researchers will have a range of sources of advice available to
them e.g., funding bodies, hospital/institution/ professional bodies
ethical committees.
Finally ask yourself if the standards you practice are those by which
you would like to be treated?

References
Bell, J. (1993) Doing Your Research Project: A Guide for First-Time
Researchers in Education and Social Science. Open University Press,
Buckingham. 176 pp.
Beynon, R.J. (1993) Postgraduate study in the biological sciences: A
researchers companion. Portland Press, London 151 pp.
Brown, S., McDowell, E., and Race, P. (1995) 500 Tips for Research
Students. Kogan Page Ltd, London. 127 pp.
BTG plc Keeping a Laboratory Notebook. Gulph Mills, USA. 12pp.
Gealy, N. and Clarke, D. (1998) Development of an Interim Workplan for
the Researchers Lead Body. Maloney and Gealy, 24-26 Mossbury Rd.
London. 30 pp.
Gealy, N., Westlake, D., & Clarke, D. (1997) Draft Occupational
Standards In Research. Maloney and Gealy, 24-26 Mossbury Rd. London.
59 pp.
Skelton, F. and Walker, L. (1995) Pilot Study to Assess the Benefits of
Gathering Evidence of Research Competencies for PhD Students to
Improve Their Subsequent Employability. Glasgow University. 21pp.
THEROS: Technology Ventures - Intellectual Property Guidelines
(1998)