Accountability in Research, 14:269–305, 2007

Copyright © Taylor & Francis Group, LLC

ISSN: 0898-9621 print / 1545-5815 online
DOI: 10.1080/08989620701670187

AUTHORSHIP AND PUBLICATION PRACTICES:

1545-5815
0898-9621
GACR
Accountability in Research
Research, Vol. 14, No. 4, Sep 2007: pp. 0–0

EVALUATION OF THE EFFECT OF RESPONSIBLE

CONDUCT OF RESEARCH INSTRUCTION TO
POSTDOCTORAL TRAINEES
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

CAROLYN L. FUNK, KIRSTEN A. BARRETT, and FRANCIS L. MACRINA

Effectiveness
C. L. Funk et of
al. RCR Instruction

Virginia Commonwealth University, Richmond, VA

We have studied postdoctoral trainees funded by NIH F32 fellowship awards

in order to test the effectiveness of responsible conduct of research (RCR) edu-
cation in the areas of authorship and publication practices. We used a 3-wave
telephone and on-line survey design, conducted over a period of two years, in
order to test for individual change before and after completing RCR educa-
tion. Overall the responses of the subjects suggested a clear awareness of stan-
dards and practices in publication. However, our results failed to suggest that
RCR education in this group significantly increased the level of ethically
appropriate behavioral responses measured in the study. Similarly we saw no
significant effect on increasing awareness of or attention to ethical guidelines
about authorship and publication practices. Our interpretation of these null
findings was influenced by the significant publication experience of our cohort
of subjects. We forward possible explanations for these null findings in this
context. Most importantly, we do not suggest that our results argue against
continued instruction in RCR education. Instead, we believe our data rein-
force the importance of careful articulation of course goals and objectives with
attention to the background and experience of the student audience when
developing RCR curricula.

Keywords: authorship and publication, authorship guidelines, NIH F32

postdoctoral fellows, RCR, responsible conduct of research instruction

This research was supported by the Research on Research Integrity Program, an

ORI/NIH collaboration: USPHS Grant NS042494 to Virginia Commonwealth University
(F.L.M., Principal Investigator). We gratefully acknowledge the assistance of William E.
McGarvey and Walter Schaffer, Office of the Director, National Institutes of Health.
Address correspondence to Francis L. Macrina, Office of the Vice President for
Research, Virginia Commonwealth University, Richmond, VA 23298-0568, USA.
E-mail: macrina@vcu.edu

269
 270 C. L. Funk et al.

Introduction

Largely driven by federal policy, instruction in the responsible

conduct of research (RCR) has become commonplace in scientific
training. Since 1990, the National Institutes of Health (NIH)
have required that trainees supported by either T32 institutional
training grants or T34 Minority Access to Research Career
program awards receive formal instruction in responsible con-
duct of research (RCR) (NIH Guide, 1992). In 2000, this require-
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

ment was extended to F32 postdoctoral fellow trainees, F31

predoctoral trainees, and K (career) awardees (NIH Guide,
2000a,b). The magnitude of the total audience receiving training
in RCR at a given point in time is not easily determined. However,
analyzing the effect of such federal policies on trainee exposure
to RCR training can be estimated. Based on NIH data for 2005
(NIH Office of Extramural Research, 2006) there were just over
1,900 active T32 training grants in place at approximately 220
institutions. These awards supported 7,500 predoctoral and 5,612
postdoctoral trainees. There also were 827 trainees supported by
T34 awards. The most recent information (2004) regarding
earned doctorates in the life sciences reported 8,819 graduates
(Chronicle of Higher Education, 2006). If we assume 25% of the
T32 predoctoral trainees in 2005 received their doctorate then
approximately 21% (1,851) of all life sciences graduates in that
year received mandatory RCR instruction at some time in their
predoctoral training (using the 2004 earned doctorate data).
Although such estimates are imperfect and reflect a minimum
number of graduates, they do suggest that RCR training is reach-
ing a significant number of U.S. predoctoral trainees.
Another impact of the federal mandate for RCR instruction
involves curriculum content. Federal agencies like the NIH and
the Office of Research Integrity (U.S. Department of Health
and Human Services) have provided guidance on material to be
covered in such instruction. The most recent iteration of core
topics recommended by the Office of Research Integrity
includes: data acquisition, management, sharing and owner-
ship, conflict of interest and commitment, human subjects, ani-
mal welfare, research misconduct, publication practices and
responsible authorship, mentor–trainee responsibilities, peer
review, and collaborative science. Thus, the federal mandate for
 Effectiveness of RCR Instruction 271

RCR instruction is unique in terms of both its selective compul-

sory nature and its guidance on curriculum content. Such gov-
ernmental mandates have been augmented in recent years with
the Office of Research Integrity providing funding to instruc-
tors and contractors to develop resources for use in RCR
instruction. These are available on-line for use without cost
(Office of Research Integrity, 2006).
Despite the intervention of federal policies requiring RCR
instruction, the relevant agencies have never articulated the
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

educational goals or the expected outcomes of such education nor

have they evaluated the effectiveness of such instruction. Further,
past research has found little evidence supporting the effectiveness
of RCR instruction (Kalichman and Friedman, 1992; Brown and
Kalichman, 1998; Eastwood et al., 1996; Elliott and Stern, 1996).
The most recent report in this area suggests that trainees acquire
new knowledge of value in recognizing, avoiding, and responding
to research misconduct, but that training had a lesser effect on pos-
itive changes in skills and attitudes (Plemmons et al., 2006).
Evaluating the success of an RCR course should be strictly
tied to students achieving well-articulated and reasonable course
objectives (Stern and Elliott, 1997). Although the characteristics
of some RCR courses have been catalogued (Mastroianni and
Kahn, 1999) there is no available consensus of RCR course objec-
tives. Similarly, information on evaluating whether or not objec-
tives are being met is not available. Such information would be
critical in order to establish best practices for RCR instruction.
The Council of Graduate Schools (Tate and Denecke, 2006)
recently published the results of a research and demonstration
project aimed at developing, testing, and assessing strategies for
delivering RCR instruction in the training programs of students
in the behavioral and biomedical sciences. Among their summary
of six best practices was the development of multilevel assessment
strategies to ensure the effectiveness of RCR instruction. This
recommendation affirms the pressing and long-standing need for
proactive and meaningful evaluation.
In this study, we evaluated the effectiveness of RCR instruction
against what we considered to be practical and reasonable short
term goals for RCR education. RCR instruction typically seeks to
raise student awareness of the ethical guidelines and ethically
appropriate behavior choices in a range of research situations.
 272 C. L. Funk et al.

Here, we report the results of a longitudinal panel design that

evaluated the effectiveness of RCR education in a core topic area:
authorship and publication practices.

Methods

Using information provided by the NIH Training Office, we

enrolled 426 postdoctoral fellows who were about to commence
their NIH F32 fellowship training. NIH funding of all participants
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

was initiated during 2002–2003 and eligibility was limited to F32

trainees working at institutions in the United States. Data were col-
lected in three waves at 6-month intervals and included a tele-
phone interview followed by completion of a web-based case study
(Macrina et al., 2004; Barrett et al., 2005). Each wave of data col-
lection continued for approximately one year. Wave 1 telephone
interviews began in January 2003; Wave 3 telephone interviews
were completed in December 2004. This research was carried out
under a human-subject research protocol, reviewed and approved
by the Virginia Commonwealth University Institutional Review
Board (VCU IRB # 2826; approval period, 2003–2005).

Study Procedures

All F32 trainees eligible for the study were sent a pre-notification
letter inviting them to participate in the study. This was followed
by a telephone call and request for participation in the 12–15
minute telephone survey. After completing the survey interview,
participants were sent an e-mail explaining how to access the on-
line case study vignettes. A reminder e-mail was sent if needed.
After completion of the online case study vignette, participants
were sent a check for $30 as a thank you for their time. The same
procedures were followed approximately 6 months and one year
later for the second and third waves of the study. (A pre-notification
letter was sent only once—prior to the first contact.)
A total of 426 F32 trainees participated in the study at Wave 1
and just 30 of the F32 trainees that were contacted and confirmed
to be eligible for the study refused to participate; thus, the simple
cooperation rate was 93% in the first wave of the study.
Cooperation rate was calculated as [426 respondents/(426+30
refusals)] and it represented a maximum estimate of response rate.
 Effectiveness of RCR Instruction 273

A minimum response rate to the Wave 1 telephone survey using

the RR1 formula of the American Association for Public Opinion
Research was 83% [426 completions/(426 completions + 30
refusals + 49 not contacted and not confirmed as eligible + 6 not
contacted but confirmed as eligible)]. Another 18 individuals
were identified as not eligible for the study either because they
declined the F32 award or were located outside the U.S. The
number of participants in the online case study vignette at Wave 1
was 368. As with any longitudinal design, there was some attrition
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

over the three waves of the study. A total of 384 and 338 F32 fel-
lows participated in the second and third wave telephone inter-
views, respectively. Most of our analyses below are limited to the
group of 325 F32 fellows who completed all three waves of the
telephone survey interview.

Measures

The telephone survey interview included: questions about RCR

instruction; other training in responsible research conduct; edu-
cational background; experience with authorship in peer-
reviewed journals; awareness of and attention to guidelines for
authorship and publication practices; and a series of questions
probing behavioral judgments about appropriate authorship and
publication practices in different contexts. See Appendix A for
full details on the telephone survey questionnaire.
The online case study vignettes consisted of a case narrative of a
research study and descriptors of three investigators involved in the
study. The project background was identical in each of the three
waves while the three investigators were different at each wave. Fol-
lowing each description, respondents were asked to rate whether it
was appropriate or inappropriate for each investigator to be listed as
an author on the manuscript being prepared from this research
study. (For the full script of the case study vignettes see Appendix B).

Results

Research, Training, and Authorship Experience

RCR education. Tables 1 and 2 show the timing and character-

istics of RCR training. We found a sizeable number (approximately
 274 C. L. Funk et al.

TABLE 1 Completion and timing of RCR training

Wave 1 Wave 2 Wave 3

(1/22/2003– (7/22/2003– (1/22/2004–
Completed RCR training 12/19/2003) 6/19/2004) 12/19/2004)
requirement? % % %

Yes 37 59 65
No 44 29 24
Don’t know about RCR/RCR 17 9 9
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

training requirement
(volunteered)
Don’t know/No answer 3 2 2
100 100 100
N=426 N=384 N=338
All respondents

Timing of RCR training %

RCR before Wave 1 (and 34

completed other waves)
RCR between Wave 1 and 24
Wave 2
RCR between Wave 2 and 9
Wave 3
No RCR by time of Wave 3 20
Incomplete waves 12
100
N=426

Percentages may not total 100 due to rounding.

TABLE 2 Characteristics of RCR training

Yes No
% %

Seminar series such as a brown bag lunch meeting+ 57 43

Included a great deal or quite a lot of in-person 55 45
discussion groups*
Self-paced tutorial such as a web-based or CD-ROM course 40 60
One- or multi-day workshop or conference 41 59
For credit course 29 71

Based on 307 respondents who completed RCR training.

*Rated on a 5-point scale: great deal, quite a lot, some, not much, not at all.
+
This item based on 292 respondents due to missing data for some cases.
 Effectiveness of RCR Instruction 275

37%) of F32 awardees had completed RCR training prior to our

initial contact with them at the beginning of their trainee period.
Seventeen percent of F32 awardees appeared to be unaware of
either RCR training or the RCR training requirements associated
with their award at this early stage in their trainee period. Approx-
imately one year later, about a third of the F32’s completing the
Wave 3 telephone interview indicated that they had not com-
pleted the RCR training requirements. Over the course of the
study, about a third of our F32 respondents were contacted both
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

before and after completing their RCR training; 34% were con-
tacted after already completing RCR; and 20% did not complete
their training by the time of the third wave of the study, approxi-
mately one year into their fellowship. Another 12% participated in
some but not all waves of the study and thus we could not deter-
mine the timing of their RCR training. The 37% who completed
RCR training at Wave 1 included 12 respondents who did not
complete either the second or third wave of the study. Thus, the
summary measure of those completing RCR before Wave 1 was
reduced by 12 respondents to 34% of the total sample.
RCR training requirements were met in diverse ways. Among
all F32’s participating in the study who completed their RCR
training (n=307), fewer than 3-in-10 (29%) reported taking a
course for credit to satisfy the training requirement while 57%
said their training involved a seminar series such as brown bag
lunch meeting. And, 55% of those who completed RCR training
characterized their training as involving “a great deal’ or “quite a
lot” of in-person discussion groups. About 4-in-10 characterized
their training as involving a self-paced tutorial or a one- to multi-
day workshop or conference.
On the whole, the timing of RCR training was not strongly
related to characteristics of RCR education. We found no signifi-
cant association between the likelihood of completing a course
for credit and timing of RCR training. Similarly, there was no
association between taking a self-paced tutorial and timing of
training or between reports of more in-person discussion groups
and timing of RCR training. On the other two characteristics
there were modest, statistically significant differences. There was
a tendency for those who completed RCR training later in their
fellowship year to describe their training as a one- to multiday
workshop or conference (Chi-sq. (2,299)=10.4 p =.01) and those
 276 C. L. Funk et al.

who completed their RCR training prior to the study’s Wave 1

were slightly less likely to describe their RCR training as a seminar
series such as a brown bag lunch (Chi-sq. (2,299)=7.5, p =.02).

Other training in responsible conduct of research. More than 7-in-

10 (71%) F32 fellows completed some other kind of formal train-
ing in responsible conduct of research apart from meeting the
RCR education requirements by the end of the study (Table 3).
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

About 68% of the F32 fellows who had not completed their RCR
requirements by Wave 3 of the study report having had some
other kind of formal training in the responsible conduct of
research, as did 63% of those who completed RCR training prior
to Wave 1 and better than 80% of those completing RCR training
between Waves 1 and 3 of the study.

Authorship and research experience. F32 trainees came to the

study with at least some research training and authorship experi-
ence already in hand (Table 4). On average, the F32 trainees

TABLE 3 Other training in responsible research conduct

Completed other formal training on Wave Wave Wave 3 All Waves+

the responsible conduct of research? 1% 2% % %

Yes 59 53 57 71
No 39 47 41 29
Don’t know/No answer 1 1 2 0
100 100 100 100
N=426 N=384 N=338 N=426
Had No
Other Other
Training Training

Timing of RCR training % % % N

RCR before Wave 1 63 37 = 100 145

RCR between Wave 1 and Wave 2 81 19 = 100 102
RCR between Wave 2 and Wave 3 85 15 = 100 40
No RCR by time of Wave 3 68 32 = 100 87
Incomplete waves 65 35 = 100 52
+
Summary measure based on responses across all three waves. Those with training
answered “yes” on any of the three waves. Includes those completing only some waves.
Percentages may not total 100 due to rounding.
 Effectiveness of RCR Instruction 277

TABLE 4 Authorship and publication experience

Number of first-
Number of authored
publications publications+

Mean SD N+ Mean SD N+

All respondents at Wave 1 6.41 5.05 424 3.7 5.38 410

Timing of RCR training
RCR before Wave 1 6.1 4.17 144 3.5 2.70 140
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

RCR between Wave 1 and Wave 2 6.3 3.94 102 3.4 2.24 98
RCR between Wave 2 and Wave 3 7.1 8.16 40 3.0 1.86 40
No RCR by time of Wave 3 7.2 5.79 86 4.9 10.73 84
Incomplete waves 5.9 4.79 52 3.2 3.21 48

Based on number of publications/first-authored publications at the time of Wave 1.

+
N omits those who did not know the number of publications/first-authored
publications.

were first contacted about two years after completing their doc-
toral degree. At the start of the study, F32 trainees reported an
average of 8.7 years of experience conducting scientific research,
including time spent in graduate training. Research experience
was similar regardless of timing of RCR; Tukey tests found no
mean pairs were significantly different from each other. The larg-
est paired mean difference showed that those completing their
RCR training prior to Wave 1 had slightly less total research expe-
rience (8.4 years on average) than those who did not complete
their RCR training by the time of Wave 3 (9.3 years on average);
however, this difference did not reach statistical significance at
the p =.05 level.
Further, F32 trainees had considerable experience with
research publications. The mean number of publications among
all F32’s contacted at Wave 1 was 6.4 publications. F32 trainees
were the first authors on an average of 3.7 publications. Forty five
percent of F32 trainees had a manuscript under review at the
time of Wave 1. There was considerable variation in the total
number of publications across trainees (ranging from 0 to 37 with
one possible outlier reporting 54 publications). Ten F32 trainees
had no publications at the time of Wave 1 and 20 with at least one
publication had no experience as the first author on a research
publication.
 278 C. L. Funk et al.

There was a slight tendency for those who completed RCR

training later in their fellowship year to have more experience as
authors however, this relationship was not statistically significant
in a one-way ANOVA and, Tukey tests found no mean pairs were
significantly different from each other. (We also repeated this
analysis after removing the possible outlier with 54 publications
and found the same results.)

Hypothesis 1: Attention to and Awareness of Authorship Guidelines

Downloaded by [University of Auckland Library] at 17:31 06 January 2015

Awareness of authorship guidelines. We hypothesized that RCR

training would increase awareness of and attention to issues
about authorship and publication practices. We asked three
items related to awareness of authorship and publication
guidelines and combined these items into one categorical
measure of awareness. Trainees were characterized as being
aware of and using authorship guidelines if: 1) indicated that
they had “heard of guidelines on authorship and publication
practices published by institutions, professional societies, or
journals;” 2) had ever looked at these guidelines; and 3)
referred to guidelines at least “some of the time” when prepar-
ing a manuscript for publication were categorized as aware of
and users of authorship guidelines. All others were classified as
not aware and/or not users of such guidelines. This was
intended to provide a first approximation of whether RCR
training was related to awareness. These data are shown in
Tables 5 and 6.
There was a small increase in the percentage of F32’s who
were aware of and use authorship guidelines at Wave 3 compared
with Wave 1 (49% and 57%, respectively); however, this occurred
for all groups of F32 fellows, regardless of RCR training status. A
chi-square test found no statistically significant association
between timing of RCR training and awareness. As a further
check, chi-square tests of all those having completed RCR train-
ing versus those who had not completed RCR training at each
wave found no statistically significant association between having
completed RCR training and awareness of authorship guidelines
for any of the three survey waves.
Our hypothesis argues for individual change in awareness of
authorship and publication guidelines before and after RCR
 Effectiveness of RCR Instruction 279

TABLE 5 Awareness of authorship and publication guidelines

Wave 1 Wave 2 Wave 3

Awareness of author guidelines* % % %

Aware of and used author guidelines 49 51 57

Not aware of or not used author guidelines 49 48 42
Don’t know/No answer 2 2 1
100 100 100
N=426 N=384 N=338
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

Aware of and used

guidelines

Wave 1 Wave 2 Wave 3

Timing of RCR training % % % N+

RCR before Wave 1 47 50 55 111

RCR between Wave 1 and Wave 2 55 48 62 87
RCR between Wave 2 and Wave 3 50 43 63 40
No RCR by time of Wave 3 45 48 54 87
+
Based on those completing all three waves.
*Awareness of author guidelines combines responses to three items: have you ever
heard of guidelines on authorship and publication practices, have you ever looked at any
of these guidelines, and how often do you refer to guidelines like these when preparing a
manuscript for submission. Respondents are classified as aware of guidelines if they
report having heard of, looked at, and refer to guidelines “some of the time”, “quite
often” or “all of the time” when preparing a manuscript for submission.
Percentages may not total 100 due to rounding.

training. Overall, about 55% of the respondents stayed in the

same classification over the three waves of the survey. About a
fifth (21%) of the F32 trainees increased their awareness and use
of these kinds of guidelines during the course of the study. The
gain among this group was offset, however, by a nearly equal por-
tion who indicated at Wave 1 that they were aware of and used
authorship guidelines but did not do so at either Wave 2 or 3. A
chi-square test found no significant association between timing of
RCR training and this measure of individual change in awareness
of authorship and publication guidelines.

Attention to authorship and publication practices. We used two mea-

sures to test for differences in attention to authorship and publi-
cation practices. The first was a five-point rating of the frequency
of discussion with others about authorship and publication
 280 C. L. Funk et al.

TABLE 6 Individual changes in awareness of authorship and publication

guidelines: Wave 1 versus Wave 2 or 3

Timing of RCR training

RCR RCR
RCR between between No RCR
before Wave Wave by time of
Awareness of author All+ Wave 1 1&2 2&3 Wave 3
guidelines* % % % % %
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

Aware of and used 32 28 37 38 30

author guidelines at
Wave 1, remained
aware at Wave 2 and 3
Not aware of or not 27 27 25 28 29
used author
guidelines at Wave 1,
remained unaware at
Wave 2 and 3
Not aware or not used 21 23 15 20 24
at Wave 1, GAIN at
Wave 2 or 3
Aware of and used at 16 19 17 13 13
Wave 1, LOSS at
Wave 2 or 3
Don’t know/No answer 4 3 6 2 5
100 100 100 100 100
N=325 N=111 N=87 N=40 N=87
+
Based on those completing all three waves.
*Awareness of author guidelines combines responses to three items: have you ever
heard of guidelines on authorship and publication practices, have you ever looked at any
of these guidelines, and how often do you refer to guidelines like these when preparing a
manuscript for submission. Respondents are classified as aware of guidelines if they report
having heard of, looked at, and refer to guidelines “some of the time”, “quite often” or “all
of the time” when preparing a manuscript for submission.
Percentages may not total 100 due to rounding.

practices. The second was a five-point rating of the amount of

attention paid to new guidelines about authorship and publica-
tion practices.
Mean comparisons over the three waves were expected to
display the following patterns. Those who did not complete RCR
training by the time of the third wave and those who completed
their RCR training prior to the first wave of telephone surveys
should exhibit no change across waves in attention to authorship
 Effectiveness of RCR Instruction 281

and publication practices. Any increase in ethically appropriate

responses among these groups was expected to reflect a practice
effect from having completed the survey. Those who completed
their RCR training between Wave 1 and 2 were predicted to show
an increased attention to authorship issues at Wave 2 which
would remain stable for Wave 3. Those who completed their RCR
training between Wave 2 and 3 should show increased attention
to authorship issues at Wave 3 only, with no change expected for
this group between Waves 1 and 2.
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

We found no significant mean differences on either measure

of attention which followed these expected patterns using a
repeated measures ANOVA test. On the measure of attention to
new guidelines, there was a slight increase in attention at Wave 3
among all groups, regardless of RCR training; there was no signif-
icant change over time in mean ratings of discussion about
authorship and publication practices.

Hypothesis 2: Ethically Appropriate Judgments

Our second hypothesis was that RCR instruction would increase

the likelihood of making ethically appropriate behavioral judg-
ments about authorship and publication practices. In testing this
hypothesis, we included a large number of questions that mea-
sured behavioral judgments about authorship and publication
practices. One set of measures was collected in the context of the
telephone survey interview. Each survey measure asked for judg-
ments about the appropriateness of different behaviors related to
authorship and publication practices; some of these related to
the role of authors and co-authors while others related to copy-
right and sharing information in manuscripts and publications. A
second set of measures was collected on the web case study
vignettes. These measures ask for a behavioral judgment in the
context of a specific case study.
As with mean ratings of attention, we expected those who
did not complete RCR training by the time of the third wave and
those who completed their RCR training prior to the first wave of
telephone surveys to exhibit no change across waves in their
behavioral judgments about authorship issues. Any increase in
ethically appropriate responses among these groups was expected
to reflect a practice effect from having completed the survey and
 282 C. L. Funk et al.

web-based case vignettes multiple times. Those who completed

their RCR training between Wave 1 and 2 should show an
increased likelihood of responding in the ethically appropriate
direction at Wave 2 which remained stable for Wave 3. Those who
completed their RCR training between Wave 2 and 3 should show
an increased likelihood of responding in the ethically appropri-
ate direction at Wave 3 only, with no change expected for this
group between Waves 1 and 2. We use a repeated-measures
ANOVA to test for these patterns across the three time points.
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

Telephone survey measures: Behavioral measures in a general

context. A total of 11 items on the telephone survey question-
naire tapped behavioral judgments about appropriate behaviors
related to authorship. We constructed an index of these mea-
sures based on a simple count of the number of items with the
ethically appropriate response, consistent with the previously
cited authorship guidelines (see questions 26 through 35 and
question 37 in Appendix A; a twelfth item, question 35, was con-
sidered for the index but was not included because of our
understanding that some fields of study hold different standards
in this area). The scores on this index had a possible range from
0 to 11 with higher scores indicating more ethically appropriate
responses.
The mean response for all F32 trainees on this measure was
5.0 (SD=1.74) at Wave 1 and there was virtually no change in
mean response across the three waves, regardless of RCR training
status. There were no significant mean differences on this sum-
mary measure using a repeated-measures ANOVA test. These
data are illustrated in Figure 1.
We also tested this approach with a more “relaxed” summary
measure of what constitutes an ethically appropriate response.
For example, if the ethically appropriate response was “always” on
a given item, we counted “always” and “usually” as an ethically
appropriate response. The mean response on this summary mea-
sure was 7.8 (SD=1.62) at Wave 1. While the possible range was
again 0 to 11, no respondents scored below a 3 and some
achieved the full 11 points by this standard. However, there was
little gain in average scores at Wave 3 (mean 8.0, SD=1.76). And,
there was no significant difference between RCR training groups
on this “relaxed” summary measure. We also tested for mean dif-
 Effectiveness of RCR Instruction 283

11.00

10.00

9.00
Index of Ethically Appropriate Responses

8.00

7.00
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

6.00

5.00

4.00

Between W1 & W2
3.00

RCR before W1
2.00

No RCR by W3
1.00

0.00
Wave 1 Wave 2 Wave 3

Wave 1 Wave 2 Wave 3

Timing of RCR training Mean SD Mean SD Mean SD N+

RCR before Wave 1 4.90 1.80 5.03 1.71 4.89 1.88 111
RCR between Wave 1 and 2 5.06 1.70 5.33 1.85 5.32 1.79 87
RCR between Wave 2 and 3 4.70 1.62 4.50 1.77 4.85 1.61 40
No RCR by time of Wave 3 4.83 1.79 4.83 1.84 4.75 1.63 87
+
Listwise N across the three waves.
Based on a count of ethically appropriate responses across 11 survey items repeated in
each wave. Values have a possible range from 0 to 11 with higher scores indicating more
ethically appropriate responses.

FIGURE 1 Ethically Appropriate Responses by Timing of RCR Instruction.

ferences on each of the 11-items as separate indicators and found

no significant differences by RCR training across the three waves.

Case study vignettes: Behavioral judgments in a specific context.

The judgments required on the telephone survey leave the details
of the situation unspecified. To augment these data, F32 trainees
 284 C. L. Funk et al.

completed a brief online survey containing a case study vignette

with specific details of a research study and three investigators
involved in the study. Respondents rated whether it was appropri-
ate or inappropriate for each of the three investigators to be
listed as an author on the paper from this study on a 4-point scale.
At each wave at least one investigator provided a clear case for
authorship based on criteria commonly found in professional
society guidelines (Claxton, 2005) and at least one provided a
clear case against authorship by these criteria. For further details
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

on the case study vignettes see Appendix B.

We created a summary measure which counted the number
of ethically appropriate responses across the three investigators in
each wave. Values had a possible range from 0 to 3 with higher
scores indicating more ethically appropriate responses. Among
all respondents, the mean score at Wave 1 was 2.2 (SD=.72) and
goes down slightly at Wave 3 (Mean 1.7, SD=.79). A repeated-
measures ANOVA test found no significant learning effect by
RCR training status on these case study vignette measures. These
data are summarized in Figure 2.
We conducted a further test of the case study vignette
responses using a more “relaxed” summary measure of what
constitutes an ethically appropriate response. For example, if the
ethically appropriate classification for a given investigator was
“definitely appropriate” we counted both “definitely appropriate”
and “probably appropriate” as an ethically appropriate judgment.
By this standard, 88% of all respondents classified the authorship
status on all three items in the ethically appropriate direction at
Wave 1, 78% did so at Wave 2, and 69% did so at Wave 3.
Repeated-measures ANOVA tests found no significant improve-
ment in ability to assign authorship credit over the course of the
three waves and no significant differences between RCR training
status groups across waves.
Waves 1 and 2 included two investigators designed to be con-
sidered appropriate as authors on the research study and one
investigator to be considered inappropriate as an author. Mean
responses on Wave 3 may have been a bit lower because, here,
there were two investigators designed to be considered inappro-
priate as authors and just one designed to be seen as appropriate.
It’s possible that making judgments about inappropriate author-
ship is more difficult than judgments about appropriate author-
 Effectiveness of RCR Instruction 285

3.00
Index of Ethically Appropriate Responses

2.00
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

Between W1 & W2
1.00
RCR before W1

No RCR by W3

0.00
Wave 1 Wave 2 Wave 3

Wave 1 Wave 2 Wave 3

Timing of RCR training Mean SD Mean SD Mean SD N+

RCR before Wave 1 2.2 .72 2.0 .69 1.7 .82 73

RCR between Wave 1 and 2 2.3 .63 1.9 .76 1.8 .79 67
RCR between Wave 2 and 3 2.2 .55 1.9 .70 1.8 .76 25
No RCR by time of Wave 3 2.2 .61 1.7 .72 1.5 .76 57
+
Listwise N across the three waves.
Based on a count of ethically appropriate judgments across the three measures in
each wave. Values have a possible range from 0 to 3 with higher scores indicating more
ethically appropriate responses.

FIGURE 2 Ethically Appropriate Responses to Case Study Vignettes by Timing

of RCR Instruction.

ship. If so, it may be that RCR training has more impact when it
comes to recognizing inappropriate behaviors. As a final test of
responses to the case study vignettes we looked at responses to
just the subset of investigators who would not deserve authorship
credit under current norms. About 4-in-10 (42%) study partici-
pants correctly identified the investigator as not deserving author-
ship credit at Wave 1; the corresponding figure at Wave 2 is 29%.
At Wave 3, two thirds of all study participants correctly classified
at least one of two investigators as not deserving authorship
credit. Here, too, there was no significant learning effect by RCR
 286 C. L. Funk et al.

training status across the three waves. This idea was also explored
on the telephone survey measures of behavioral judgments but
was not supported. Repeated-measures ANOVA tests on the 11
individual items comprising the summary measure found no
significant differences by RCR training status on any item.

Discussion

To examine the effectiveness of RCR training we conducted a

Downloaded by [University of Auckland Library] at 17:31 06 January 2015

national, longitudinal study of biomedical, behavioral, and clini-

cal research postdoctoral trainees supported by NIH F32 fellow-
ships. Applications for such support “must include the
candidate’s plans for obtaining instruction in the responsible
conduct of research” (NIH Guide, 2000a). This requirement
allowed us to follow trainees across a two-year period in which it
was probable that they would receive RCR training and allowed
us to evaluate the effects of RCR instruction without concern that
self-selection into RCR education accounted for differences
between groups. Further, our findings were not limited to one
institution’s instructional program but instead speak to the
effectiveness of RCR instruction as it is occurring at institutions
around the nation.
Our study design was a three-wave telephone survey that
probed awareness of, attention to, and behavioral judgments
regarding ethically appropriate authorship and publication prac-
tices. In addition, study participants were asked to complete an
on-line survey at each wave. Each on-line survey tested behavioral
responses to detailed case study vignettes. We limited our study to
one core area of RCR content: authorship and publication prac-
tices. This topic area is one of nine supported by federal agencies
including the Office of Research Integrity. Codes of normative
behavior in authorship and publication have developed substan-
tively in the past three decades providing us with emerging
standards on which we could base our investigation (Claxton,
2005; Scheetz, 2001; and guidelines by the American Chemical
Society, 2006; American Society for Microbiology, 2006; Interna-
tional Committee of Medical Journal Editors, 2006; and Society
for Neuroscience, 1998).
We have published preliminary reports of our findings ana-
lyzed prior to the completion of the three wave survey period.
 Effectiveness of RCR Instruction 287

We found that an unexpectedly sizeable subset of F32 fellows

(37%) were beginning their fellowships having already com-
pleted RCR instruction. Comparisons of those who had and had
not completed RCR training revealed minor associations
between RCR training and greater attention to guidelines on
authorship and publication issues. Similarly, we did not observe
striking differences between those with and without RCR train-
ing in recognizing and making ethically appropriate behavioral
judgments about authorship and publication issues (Macrina
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

et al., 2004). At the Wave 1 data collection, we also observed

that approximately one-half of the participant F32 fellows
reported they were aware of and had referred to guidelines on
authorship and publication practices (e.g., publishers Instruc-
tions to Authors). The remaining half reported no such aware-
ness or guideline use. We found both groups to have similar
publication records, years of research experience, and receipt
of RCR training. However our analysis of awareness and use of
authorship guidelines was, at best, only modestly associated with
more ethically appropriate judgments about author roles and
responsibilities (Barrett et al., 2005).
The findings presented in this article corroborate our pub-
lished preliminary results. Following F32 fellows longitudinally
across a training timeframe that included RCR instruction
allowed us to test two hypotheses regarding observable effects of
RCR education related to authorship and publication practices.
We predicted that after RCR instruction trainees would be more
likely to: 1) demonstrate attention to and awareness of issues
about authorship and publication practices than prior to RCR
instruction; and, 2) make ethically appropriate behavioral judg-
ments about authorship and publication practices than prior to
RCR instruction.
As can be seen in Table 4, F32 fellows had considerable expe-
rience in conducting scientific research and in preparing
research for publication in peer-reviewed journals. Further, most
F32 trainees had completed some kind of formal training in the
responsible conduct of research apart from the mandated RCR
training (Table 3). There was little relationship between timing
of RCR education and type of RCR training, other training in the
responsible conduct of research, years of experience in conduct-
ing research, or number of publications. Thus, there is no reason
 288 C. L. Funk et al.

to believe that these factors provide an alternative explanation for

findings in our hypothesized areas of attention to, awareness of,
and ethically appropriate judgments about authorship and
publication practices.
The data presented in Tables 5 and 6 did not reveal any
statistically significant association between completing RCR train-
ing and guideline awareness. Thus, for this group of subjects our
data did not support that RCR training increased awareness and
usage of authorship and publication guidelines. These longitudi-
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

nal data are in keeping with the restricted analysis of entire group
of trainees prior to completing the three-wave panel design
(Barrett et al., 2005). Table 7 presents data on changes in atten-
tion to authorship and related practices. As with Tables 5 and 6
these measures provided little support for our first hypothesis
that RCR training increased either awareness of or attention to
authorship and publication guidelines and practices.

TABLE 7 Attention to authorship and publication practices

Frequency of discussion with others

Wave 1 Wave 2 Wave 3

Timing of RCR training Mean SD Mean SD Mean SD N+

RCR between Wave 1 and 2 2.8 .93 2.8 .98 2.9 .92 111
RCR between Wave 2 and 3 2.7 .95 2.9 .82 2.8 .93 86
RCR before Wave 1 3.1 .96 3.1 .87 3.1 1.07 39
No RCR by time of Wave 3 2.7 1.05 2.7 1.03 2.7 1.11 86

Values have a possible range from 1 to 5 with higher scores indicating less discussion.

Attention to reports about new guidelines

Wave 1 Wave 2 Wave 3

Timing of RCR training Mean SD Mean SD Mean SD N+

RCR between Wave 1 and 2 3.2 1.16 3.2 1.11 3.0 1.15 110
RCR between Wave 2 and 3 3.2 1.02 3.1 1.16 3.0 1.01 86
RCR before Wave 1 3.3 .99 3.3 .91 3.1 1.09 40
No RCR by time of Wave 3 3.2 1.03 3.1 1.16 2.8 1.10 84

Values have a possible range from 1 to 5 with higher scores indicating less attention.
+
Listwise N across the three waves; N omits those answering don’t know.
 Effectiveness of RCR Instruction 289

Evaluating ethically appropriate judgments was accomplished

using specific measures collected during the telephone survey
interview as well as those collected in response to the case study
vignettes. Figure 1 showed no significant improvement in ability to
make ethically appropriate judgments about authorship and publi-
cation issues using an 11-item index. As seen in Figure 2, there was
no significant improvement in ability of our subjects to appropri-
ately assign authorship credit over the course of the three waves;
here too, there was no significant difference between RCR training
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

status groups across waves using both stringent and relaxed mea-
sures of behavioral judgments in the case study vignettes.
Our focus on one core topic area of RCR instruction limited
our ability to generalize our findings about effectiveness of RCR
instruction to other topics. Our choice of this topic as a first test
was driven by: 1) the undisputed centrality of authorship issues to
scientific research; 2) the likelihood that young scientists would
have some exposure to authorship and publication issues in RCR;
and, 3) the available codes of normative behavior in publishing
that would allow us to evaluate responses against clear standards
of normative behavior.
It should be noted that our findings were also limited to the
ways in which we sought to measure effectiveness. There may be
other kinds of effects of RCR education than those tested here. In
the absence of clear guidelines, we sought to test the effectiveness
of RCR education against what we considered to be practical and
reasonable short term goals for RCR education. To do so, we eval-
uated behavioral judgments rather than behaviors in real-world
situations as a practical proxy for behaviors. And we focused on
bottom-line judgments rather than ethical reasoning. We were
limited to short-term behavioral impact; a longer-term goal such
as reducing the incidence of research misconduct is simply not
practicable.
As expected, the findings of this study show that RCR man-
dated education is implemented in a variety of ways for individual
fellows across institutions. For example, some F32 fellows com-
pleted RCR training by completing self-paced tutorials, others
attended a brown-bag seminar series; in some cases RCR training
involved a great deal of in-person discussion and in others it
involved no discussion (Table 2). Our study was designed to
evaluate the effectiveness of RCR education as it has been
 290 C. L. Funk et al.

implemented across the whole range of institutional and individ-

ual experiences. This is a key strength of the study design. It does
not preclude other studies from finding that specific courses or
instructors are effective at increasing attention to and awareness
of authorship guidelines or making ethically appropriate judg-
ments about authorship and publication practices, however.
Thus, our findings concern the effectiveness of RCR education,
on average, as implemented for F32 fellows across the nation.
While our study results are clear, the implications of these
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

findings are far from clear. Some will be tempted to argue that
our findings show that RCR instruction is not effective and, there-
fore, not indicated. We would disagree with such a position.
Although our data do not provide support for our hypotheses,
there are a number of reasons why it would be premature and,
indeed, inadvisable to interpret our findings in such a way.
Our findings must be viewed in light of the qualifications
and experience of our subjects. The characteristics of our partici-
pating F32 trainees indicated considerable exposure to
manuscript preparation and the publication process. In aggre-
gate, our subjects had an average of 8.7 years of research experi-
ence and had published an average of 6.4 articles. Further, a
majority of subjects also completed some other kind of training
in the responsible conduct of research (Table 3). Thus we tested
for changes attributable to RCR education, per se, among a
group with considerable relevant experience. Such extensive
exposure to authorship and publication practices would be
expected to provide a level of practical training that could
account for our results; that about one-half of our subjects knew
of the existence of publication guidelines at the time of enroll-
ment in the study (Table 5) reinforces this notion. We attempted
to evaluate this notion by comparing F32 fellows who had com-
pleted relatively fewer versus more publications and who had
relatively fewer versus more years of research experience. How-
ever, our analysis revealed no significant differences between
these groups on measures of ethically appropriate judgments or
attention to authorship and publication guidelines. It is possible
that gains in awareness, attention, and ethically appropriate
behavioral judgments would be found for those with no author-
ship experience or other training related to responsible research
conduct. Further analysis of those fellows with almost or no
 Effectiveness of RCR Instruction 291

experience in authorship and publication was not possible due to

the small size of this group, however.
Taking into account the research background and experi-
ence of F32 fellows completing mandated RCR education places
the results of our study in a different light. Namely, it suggests
that RCR curriculum content may need to be adjusted in
response to the audience. The needs of a class comprised of expe-
rienced postdoctoral fellows would be different from those of first
year graduate students. Although further testing would be
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

needed to verify this, the idea is in keeping with the widely held
notion in RCR instruction that “one size does not fit all.” We
believe our data provide an underlying rationale for such
guidance.
There are a number of reasons to continue mandated RCR
education. Our findings suggest that RCR education efforts
would be optimized by recommending or requiring that it occur
early in the predoctoral research training process. On the other
hand, the changing landscape of authorship and publication
practices as well as that of other RCR core areas also argues for
follow up and continuing RCR education for scientists at all levels
of development. The development of substantive publication
guidelines by societies, the growing changes in publisher’s
Instructions to Authors, and the emergence of regulations that
have a bearing on reporting research argue for proactive instruc-
tion in this arena. Further, mandated training in RCR education
provides a safeguard that all researchers, regardless of informal
training and other educational experience, are formally exposed
to the key concepts and standards in the responsible conduct of
authorship and publication practices.

References

American Chemical Society. (2006). Ethical Guidelines to Publication of Chem-

ical Research. Available at http://pubs.acs.org/ethics/ethics.pdf, accessed
October 18, 2007.
American Society for Microbiology. (2006). Instructions to Authors for ASM
Journals. Available at http://www.journals.asm.org/misc/ifora.shtml,
accessed October 18, 2007.
Barrett, K. A., Funk, C. L., and Macrina, F. L. (2005). Awareness of publication
guidelines and the responsible conduct of research, Accountability in Research,
12: 103–206.
 292 C. L. Funk et al.

Brown, S. and Kalichman, M. W. (1998). Effects of training in the responsible

conduct of research: A survey of graduate students in experimental sciences,
Science and Engineering Ethics, 4: 487–498.
Chronicle of Higher Education. (2006). Characteristics of Recipients of Earned
Doctorates, 2004. Available at http://chronicle.com/weekly/almanac/
2006/nation/0102001.htm, accessed October 18, 2007.
Claxton, L. D. (2005). Scientific authorship. Part 2. History, recurring issues,
practices, and guidelines, Mutation Research, 589: 31–45.
Eastwood, S., Derish, P., Leash, E., and Ordway, S. (1996). Ethical issues in bio-
medical research: Perceptions and practices of postdoctoral research fellows
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

responding to a survey, Science and Engineering Ethics, 2: 89–114.

Elliott, D. and Stern, J. (1996). Evaluating teaching and students’ learning of
academic research ethics, Science and Engineering Ethics, 2: 345–366.
International Committee of Medical Journal Editors (ICMJE). (2006) Uniform
Requirements for Manuscripts Submitted to Biomedical Journals: Writing
and Editing for Biomedical Publication. Available at http://www.icmje.org/
index.html, accessed October 18, 2007.
Kalichman, M. W. and Friedman, P. J. (1992). A pilot study of biomedical
trainees’ perceptions concerning research ethics, Academic Medicine, 67:
769–775.
Macrina, F. L., Funk, C. L., and Barrett, K. (2004). Effectiveness of responsible
conduct of research instruction: Initial findings, Journal of Research Administra-
tion, 6: 6–12.
Mastroianni, A. and Kahn, J. (1999). Encouraging accountability in research: A
pilot assessment of training efforts, Accountability in Research, 7: 85–100.
NIH Guide. (1992). Requirement for Instruction in the Responsible Conduct of
Research. In National Research Service Award Institutional Training Grants.
Available at http://grants.nih.gov/grants/guide/notice-files/not92-236.html,
accessed October 18, 2007.
NIH Guide. (2000a). National Research Service Awards for Individual Postdoc-
toral Fellows (F32). Available at http://grants.nih.gov/grants/guide/
pa-files/PA-00-104.html, accessed October 18, 2007.
NIH Guide. (2000b). National Research Service Awards for Individual Predoc-
toral Fellows. Available at http://grants.nih.gov/grants/guide/pa-files/
PA-07-002.html, accessed October 18, 2007.
NIH Office of Extramural Research. (2006). Training and Career Awards. Avail-
able at http://grants1.nih.gov/grants/award/granfell.htm, accessed Octo-
ber 18, 2007.
Office of Research Integrity. (2006). Responsible Conduct of Research. Avail-
able at http://ori.dhhs.gov/education/.
Plemmons, D. K., Brody, S. A., and Kalichman, M. W. (2006). Student percep-
tions of the effectiveness of education in the responsible conduct of research,
Science and Engineering Ethics, 12: 571–582.
Scheetz, M. D. (2001). Promoting Integrity Through Instructions to Authors: A
Preliminary Analysis. USPHS Office of Research Integrity. Available at http://
ori.dhhs.gov/documents/instructions_authors.pdf, accessed October 18,
2007.
 Effectiveness of RCR Instruction 293

Society for Neuroscience. (1998). Guidelines: Responsible Conduct Regarding

Scientific Communication. Available at http://www.sfn.org/index.cfm?page-
name=responsibleConduct, accessed October 18, 2007.
Stern, J. and Elliott, D. (1997). The Ethics of Scientific Research–-A Guidebook for
Course Development. Hanover, NH: University Press of New England.
Tate, P. and Denecke, D. (2006). Graduate Education for the Responsible Conduct of
Research. Washington, DC: Council of Graduate Schools.

APPENDIX A
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

Telephone Survey Questionnaire for Wave 1, Wave 2, and Wave 3

I. Introductory Script

Once speaking with [F32 fellowship trainee]:

INTRODUCTION TO THE SURVEY WAVE 1

Hello, my name is [fill in name], and I am calling from Virginia
Commonwealth University (in Richmond, Virginia). You should
have received a letter telling you about the NIH-funded study we
are doing with NIH F32 fellowship trainees to evaluate beliefs
about and training in the responsible conduct of research. I’m
calling today to conduct the survey portion of this study. If you
don’t have any questions about this study, I’d like to complete the
survey with you now.

INTRODUCTION TO THE SURVEY WAVE 2

Hello, my name is [fill in name], and I am calling from Virginia
Commonwealth University (in Richmond, Virginia). As you may
recall, we are conducting an NIH-funded study with NIH F32
fellowship trainees on the responsible conduct of research. I’m
calling today to conduct the survey portion of this study. If you
don’t have any questions about this study, I’d like to complete the
survey with you now.

INTRODUCTION TO THE SURVEY WAVE 3

Hello, my name is [fill in name], and I am calling from Virginia
Commonwealth University (in Richmond, Virginia). As you may
recall, we are conducting an NIH-funded study with NIH F32
fellowship trainees on the responsible conduct of research. I’m
calling today to conduct the final survey portion of this study. If
 294 C. L. Funk et al.

you don’t have any questions about this study, I’d like to com-
plete the survey with you now.

FOR WAVE 1, WAVE 2, WAVE 3

Your responses to this survey will be confidential. You should
know that this is a voluntary survey. If you do not know the answer
to a question or prefer not to answer, just say so and we’ll skip it.
If you choose to withdraw after we start just let me know. How-
ever, your participation is very important. We’d like to ask you the
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

following questions.

II. RCR Instruction Experience

First, …

1. Have you completed the RCR training requirements for F32

trainees? Yes, No, Never heard of requirement (volunteered)
(Note: RCR stands for the Responsible Conduct of Research)

ASK IF YES in Q1:

2. What month and year did you complete these requirements?

(Enter month/year)
3. To what extent did your training include in-person discussion
groups—a great deal, quite a bit, some, not much, or not at all?

RCR training is conducted in many different ways. For each of the

following please tell me whether this does or does not describe
your training in RCR.

4. A course taken for credit? (Does this describe your training in

RCR?) Yes, No
5. A self-paced tutorial such as a Web-based or CD-ROM course?
(Does this describe your training in RCR?) Yes, No
6. A seminar series such as a brown bag lunch meeting? (Does
this describe your training in RCR?) Yes, No
7. A one- or multi-day workshop or conference? (Does this
describe your training in RCR?) Yes, No
 Effectiveness of RCR Instruction 295

ASK IF NO in Q1:

8. What month and year do you expect to complete these

requirements? (Enter month/year)
9. To what extent do you expect this training will include in-per-
son discussion groups—a great deal, quite a bit, some, not
much, or not at all?

RCR training is conducted in many different ways. For each of the

Downloaded by [University of Auckland Library] at 17:31 06 January 2015

following please tell me whether you expect your training in RCR

to involve this.

10. A course taken for credit? (Do you expect your training in
RCR to involve this?) Yes, No
11. A self-paced tutorial such as a Web-based or CD-ROM
course? (Do you expect your training in RCR to involve
this?) Yes, No
12. A seminar series such as a brown bag lunch meeting? (Do
you expect your training in RCR to involve this?) Yes, No
13. A one- or multi-day workshop or conference? (Do you
expect your training in RCR to involve this?) Yes, No

ASK ALL:

14. Apart from training to complete the RCR requirements,

have you ever received any formal training on responsible
research conduct? Yes, No

ASK IF YES in Q14:

15. What month and year did this other training take place?
(Enter month/year) [Interviewer Note: If both start and end
date are offered, enter ending date]
16. To what extent did that other training include in-person dis-
cussion groups—a great deal, quite a bit, some, not much,
or not at all?

For each of the following please tell me whether this does or does
not describe the other training you received.
 296 C. L. Funk et al.

17. A course taken for credit? (Does this describe your other
training?) Yes, No
18. A self-paced tutorial such as a Web-based or CD-ROM
course? (Does this describe your other training?) Yes, No
19. A seminar series such as a brown bag lunch meeting? (Does
this describe your other training in RCR?) Yes, No
20. A one- or multi-day workshop or conference? (Does this
describe your other training in RCR?) Yes, No
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

III. Attention to Ethical Guidelines, Authorship,

and Publication Practices

21. How often do you have discussions with others about author-
ship and publication practices—very often, fairly often,
some of the time, not too often, or almost never?
22. How much attention do you pay to reports about new guide-
lines for authorship and publication practices—a great deal,
quite a bit, some, not much, or none at all?
23. Have you ever heard of guidelines on authorship and publi-
cation practices published by institutions, professional soci-
eties, or journals OR is this something you are not aware of?
Yes, heard of; No, not aware
24. Have you ever looked at any of these guidelines? Yes, No
25. When preparing a manuscript for publication how often do
you refer to guidelines about authorship and publication
practices like these—all the time, quite often, some of the
time, rarely, or almost never?

IV. Awareness of Ethically Appropriate Behavior on Authorship

and Publication Practices

ASK Q26 and Q27 in RANDOMIZED ORDER

26. When a person makes a purely technical contribution to a

research study, how often would you say he or she deserves
authorship credit on any publications from this research–
always, usually, sometimes, rarely, or never?
27. When a person conceived of an idea, performed a key exper-
iment to test the idea, and makes an interpretation of the
 Effectiveness of RCR Instruction 297

results, how often would you say he or she deserve author-

ship credit on any publications from this research—always,
usually, sometimes, rarely, or never?
28. When a person receives funding from a for-profit
organization with a direct financial interest in the research
findings—how often would say it is appropriate for him or
her to note that funding in any publications from this
research—always, usually, sometimes, rarely, or never?
29. How often would you say it is appropriate for the same
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

data to be published in two different journal publications

when the main audience of each journal represents
different disciplines, such as biochemistry versus genetics
—is this always, usually, sometimes, rarely, or never
appropriate?

Next, suppose you want to post tables and figures from one
of your publications on your lab website. The publisher holds the
copyright to your published article. For each of the following
indicate whether it would or would not be appropriate for you to
post this information on your lab website. First…

ASK Q30 THROUGH Q32 in RANDOMIZED ORDER

30. Without permission from and without acknowledgment of

the publisher. (Is this definitely appropriate, probably
appropriate, somewhat appropriate, probably not appropri-
ate, or definitely not appropriate?)
31. Without permission from, but with acknowledgment of the
publisher. (Is this definitely appropriate, probably appropri-
ate, somewhat appropriate, probably not appropriate, or
definitely not appropriate?)
32. With permission from and with acknowledgement of the
publisher. (Is this definitely appropriate, probably appropri-
ate, somewhat appropriate, probably not appropriate, or
definitely not appropriate?)

For each of the following statements, please tell me whether

you strongly agree, agree, neither agree nor disagree, disagree, or
strongly disagree. First, …
 298 C. L. Funk et al.

ASK Q33 THROUGH Q37 in RANDOMIZED ORDER

33. It is appropriate for a journal reviewer to share a manuscript

in confidence with a colleague without notifying the journal
editor in order to get additional expert information for use
in his or her written review. (Do you strongly agree, agree,
neither agree nor disagree, disagree, or strongly disagree
with this statement?)
34. All co-authors are equally responsible for the accuracy of pub-
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

lished data. (Do you strongly agree, agree, neither agree nor
disagree, disagree, or strongly disagree with this statement?)
35. When authors from multiple disciplines work together on a
research project, only authors with expertise in the area
have responsibility for the accuracy of the published data.
(Do you strongly agree, agree, neither agree nor disagree,
disagree, or strongly disagree with this statement?)
36. When materials described in published research are
requested by others in the field, it is up to the author to
decide whether or not to share those materials. (Do you
strongly agree, agree, neither agree nor disagree, disagree,
or strongly disagree with this statement?)
37. The senior author has more responsibility than other
authors for the accuracy of published data. (Do you strongly
agree, agree, neither agree nor disagree, disagree, or
strongly disagree with this statement?)

V. Background Educational Information (Asked During Wave 1 Only)

Now on a different topic.

38. Do you currently hold a Ph.D. degree? Yes, No

ASK IF YES in Q38:

39. What year did you receive your Ph.D? (Enter year)

ASK ALL:

40. Do you hold an M.D. degree? Yes, No

 Effectiveness of RCR Instruction 299

ASK IF YES in Q40:

41. What year did you receive your M.D.? (Enter year)

ASK ALL:

42. Do you have any other doctoral degrees? Yes, No

ASK IF YES in Q42:

Downloaded by [University of Auckland Library] at 17:31 06 January 2015

43. What degree is that? (Enter degree)

44. What year did you receive that degree? (Enter year)

ASK ALL:

45. Including time spent in graduate school, how many years

have you been involved in conducting scientific research?
(Enter total number of years)
46. Is your primary research area best described as clinical,
behavioral, or biomedical?

VI. Experience with Authorship and Publication

47. Have you been an author on any research publications in a

peer-reviewed journal so far? Yes, No

ASK IF YES in Q47:

48. How many publications do you have? (Enter number)

[Note: Enter total number of publications. Count co-
authored publications and single-authored publications.]
49. Are you the FIRST author on any of (these publications/this
publication)? IF YES: How many publications do you have
where you are the first author? (Enter number)

ASK ALL:

50. Have you been an author on any manuscripts currently

under review by a peer-reviewed journal? Yes, No
 300 C. L. Funk et al.

ASK IF YES in Q50:

51. How many manuscripts do you have under review? (Enter

number)
[Note: Enter total number of manuscripts under review.
Count co-authored and single-authored manuscripts.]
52. Are you the FIRST author on any of (these manuscripts/this
manuscript)? IF YES: How many manuscripts under review
do you have where you are the FIRST author? (Enter
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

number)

ASK ALL:

53. Are you an author on any manuscripts currently being

prepared for submission to a peer-reviewed journal?
Yes, No

ASK IF YES in Q53:

54. How many manuscripts like this do you have? (Enter

number)
[Note: Enter total number of manuscript being prepared for
submission. Count co-authored and single-authored
manuscripts.]
55. Are you the FIRST author on any of (these manuscripts/this
manuscript)? IF YES: How many manuscripts being pre-
pared for submission do you have where you are the FIRST
author? (Enter number)

Closing and Other Background Characteristics (Wave 1 Only)

56. Enter sex of respondent (ask only if necessary)

That’s all my questions….

You will receive an e-mail notice on how to link to the Web-
based case-study portion of this study in a day or two. May I verify
that your e-mail address is: [fill in email]?
Thank you.
 Effectiveness of RCR Instruction 301

APPENDIX B

Online Case Study Vignettes Wave 1, Wave 2, and Wave 3

Project Background

Dr. Lynn Newell, a professor in a university biochemistry labora-

tory, is the principal investigator on a National Institute of
Health) NIH grant to study the properties of enzymes isolated
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

from bacteria that live in extreme environments. A novel bacte-

rium isolated two years ago in Yellowstone National Park has
been studied and a heat resistant form of the enzyme DNA ligase
has been purified from it. This enzyme has been thoroughly char-
acterized. The gene for it has been cloned and over-expressed in
Escherichia coli allowing purification of large amounts of the
enzyme. The nucleotide sequence of the DNA ligase gene has
been determined and analyzed. Preliminary studies suggest this
enzyme may be useful in an assay to achieve the linkage (ligation)
of select stretches of DNA which may be diagnostic for certain
genetic diseases.
At a meeting of all lab personnel, Dr. Newell says it’s time
to prepare a manuscript describing these exciting results and
submit it to the Proceedings of the National Academy of Sciences.
Dr. Newell starts a discussion aimed at deciding whose names
will appear on the author byline of the article. Dr. Newell asks
everyone to describe their involvement in the work in order to
begin a discussion about what contributions merit authorship
on the article.

Wave 1 Investigators

LYNN NEWELL
“I am the principal investigator of the NIH grant that provided
funding for this work. This grant paid for research materials, and
portions of the salaries of personnel involved in the work. The
experimental approaches to look for the heat-resistant ligase were
described in my NIH grant application, but the entire DNA ligase
project was only a minor part of the overall thrust of the proposal.
I did no experimental work on this project, but I will oversee the
writing, editing, and content of the planned manuscript.”
 302 C. L. Funk et al.

Do you think it is appropriate or inappropriate that

Lynn Newell is listed as an author on the article? (Definitely
appropriate, Probably appropriate, Probably inappropriate, Defi-
nitely inappropriate)

ROBIN WILLOW
“I am a program support technician employed by Dr. Newell.
I plan to do copy editing on the manuscript drafts. I will also use a
computer drawing program to prepare the figures needed for the
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

manuscript. I will produce all the photographic quality computer-

generated prints of figures needed to accompany the submitted
manuscript.”
Do you think it is appropriate or inappropriate that Robin
Willow is listed as an author on the article? (Definitely appropri-
ate, Probably appropriate, Probably inappropriate, Definitely
inappropriate)

FRAN MCCLURE
“I am a research associate in Dr. Newell’s lab. I had the original
idea to look for a heat-resistant DNA ligase. I suggested several
strategies for isolating and cultivating bacteria from hot springs.
I designed the enzyme purification scheme, and have mentored
Pat Langella—Dr. Newell’s predoctoral student— in carrying out
this aspect of the work. I helped interpret all data involving
the enzyme isolation and purification. On several occasions,
I suggested new experimental approaches to the enzyme purifica-
tion, all of which proved fruitful.”
Do you think it is appropriate or inappropriate that Fran
McClure is listed as an author on the article? (Definitely appropri-
ate, Probably appropriate, Probably inappropriate, Definitely
inappropriate)

Wave 2 Investigators

PAT LANGELLA
“I am a 4th year predoctoral trainee working in Dr. Newell’s
lab. Although Dr. Newell is my formal academic advisor, others
in the lab have provided me with invaluable mentoring. I puri-
fied and characterized the enzyme with my own hands, and
completed the nucleotide sequence of the gene. I plan to write
 Effectiveness of RCR Instruction 303

the entire first draft of the manuscript, including composing

all the data tables and manuscript drawings. I will do the
literature search needed to critically review the field. Eventu-
ally, this manuscript will become a chapter in my Ph.D.
dissertation.”
Do you think it is appropriate or inappropriate that Pat
Langella is listed as an author on the article? (Definitely appropri-
ate, Probably appropriate, Probably inappropriate, Definitely
inappropriate)
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

BROOK LOVELL
“I am working towards a Master’s degree in Dr. Newell’s lab. I
have a B.S. degree and extensive experience in bioinformatics.
I taught Pat Langella-–Dr. Newell’s predoctoral student—how to
use several computer programs to analyze DNA and protein
sequence information. Pat used this training to do all the com-
puter analyses on the gene and its gene product. My five weeks of
instruction provided to Pat were equivalent to a 2 credit hour
course. I also helped Pat learn a complex computer graphics pro-
grams for illustrating sequence data.”
Do you think it is appropriate or inappropriate that Brook
Lovell is listed as an author on the article? (Definitely appropri-
ate, Probably appropriate, Probably inappropriate, Definitely
inappropriate)

CHRIS EVANS
“I am doing an undergraduate honors project under Dr. Newell’s
supervision. When I was on vacation in Yellowstone two years ago,
Dr. Newell asked me to bring back some water samples from the
hot springs for my honors project. One of the bacterial strains I
cultivated from these samples yielded the heat resistant DNA
ligase used in the project. I did all the necessary classification
work to identify this bacterium at the genus and species level. I
used standard taxonomic tools like evaluating metabolic capabili-
ties of the organism. I also obtained and used a panel of DNA
probes to augment my taxonomic studies. Finally, I determined
the sequence of the 16s ribosomal RNA from this organism in
order to make a definitive identification.”
Do you think it is appropriate or inappropriate that Chris
Evans is listed as an author on the article? (Definitely appropriate,
 304 C. L. Funk et al.

Probably appropriate, Probably inappropriate, Definitely

inappropriate)

Wave 3 Investigators

KIM LEE
“As a postdoctoral fellow in Dr. Newell’s lab, I cloned the DNA
ligase gene as a “side project” during a break in my own research
activities. I did a preliminary characterization of the cloned gene
Downloaded by [University of Auckland Library] at 17:31 06 January 2015

and made milligram amounts of the recombinant plasmid carry-

ing the gene. I gave this plasmid material to Pat Langella—Dr.
Newell’s predoctoral student. Pat performed the nucleotide
sequence analysis of the DNA ligase gene. I also came up with the
idea to use this enzyme in an assay that may be commercially use-
ful as a diagnostic for genetic diseases.”
Do you think it is appropriate or inappropriate that Kim Lee
is listed as an author on the article? (Definitely appropriate,
Probably appropriate, Probably inappropriate, Definitely
inappropriate)

PHIL NEWTON
“I am in charge of the nucleic acid support facility in the depart-
ment of biochemistry. I used an automated DNA synthesizer to
create 42 different oligonucleotides used by Pat Langella—
Dr. Newell’s predoctoral student—in determining the nucleotide
sequence of the DNA ligase gene. I worked closely with Pat in giv-
ing guidance on the design of the primers and their use. Several
times, I helped Pat troubleshoot problems when the DNA
sequencing did not work.”
Do you think it is appropriate or inappropriate that Phil
Newton is listed as an author on the article? (Definitely appropri-
ate, Probably appropriate, Probably inappropriate, Definitely
inappropriate)

CASEY TUCKER
“I have an undergraduate background in biochemistry and am
doing part-time consulting for Dr. Newell’s lab while I complete
my final year of law school. I have expertise in intellectual prop-
erty law. I performed about 100 hours of background research on
the technology transfer implications of this discovery. I am advising
 Effectiveness of RCR Instruction 305

Dr. Newell on the preparation of this manuscript in terms of

intellectual property protection. I will write and submit a provi-
sional patent application for this discovery.”
Do you think it is appropriate or inappropriate that Casey
Tucker is listed as an author on the article? (Definitely appropri-
ate, Probably appropriate, Probably inappropriate, Definitely
inappropriate)
Downloaded by [University of Auckland Library] at 17:31 06 January 2015