ARTICLE

The Rapid Assessment for

Adolescent Preventive
Services (RAAPS): ProvidersÕ
Assessment of Its Usefulness
in Their Clinical Practice
Settings
Cynthia S. Darling-Fisher, PhD, RN, FNP-BC,
Jennifer Salerno, DNP, CPNP, FAANP,
Chin Hwa Y. Dahlem, PhD, FNP-C,
& Kristy K. Martyn, PhD, RN, CPNP-PC, FNP-BC

ABSTRACT adolescent-provider communication. The survey was com-

Introduction: The purpose of this study was to evaluate
health providersÕ use of the Rapid Assessment for Adolescent
Preventive Services (RAAPS) screening tool to identify ado-
lescent high-risk behaviors, its ease of use and efficiency,
and its impact on provider/patient discussions of sensitive
risk behaviors.
Method: This mixed methods descriptive study used an
online survey to assess providersÕ use of the RAAPS and
their perspectives on its implementation and effect on
pleted by providers from a variety of settings across the
United States (N = 201).
Results: Quantitative and qualitative analyses indicated that
the RAAPS facilitated identification of risk behaviors and
risk discussions and provided efficient and consistent assess-
ments; 86% of providers believed that the RAAPS positively
influenced their practice.
Discussion: Adoption of the RAAPS in practice settings could
lead to more effective adolescent preventive services by

Cynthia S. Darling-Fisher, Clinical Assistant Professor, Health licenses issued for use of the Web-based version of the RAAPS
Promotion and Risk Reduction Programs at the University of tool. The paper version of the RAAPS tool is available for download
Michigan, School of Nursing, North Ingalls, Ann Arbor, MI. and clinical use free of charge. Chin Hwa Y. Dahlem reports that as
one of the inventors for the paper-based version of RAAPS tool,
Jennifer Salerno, Director, Regional Alliance for Healthy Schools
that is free and available for clinical use online, she also receives
(RAHS), University of Michigan Health System, and Clinical
royalties from the University of Michigan from licenses issued for
Adjunct Faculty, University of Michigan School of Nursing,
use of the Web-based version of the RAAPS tool. Kristy K. Martyn
North Ingalls, Ann Arbor, MI.
reports no financial interests or potential conflicts of interest.
Chin Hwa Y. Dahlem, Clinical Assistant Professor, Health
Correspondence: Cynthia S. Darling-Fisher, PhD, RN, FNP-BC,
Promotion and Risk Reduction Programs at the University of
University of Michigan School of Nursing, 400 North Ingalls St, Ann
Michigan, School of Nursing, North Ingalls, Ann Arbor, MI.
Arbor, MI 48109-0482; e-mail: darfish@umich.edu.
Kristy K. Martyn, Associate Professor and Chair, Health Promotion
0891-5245/$36.00
and Risk Reduction Programs at the University of Michigan,
School of Nursing, North Ingalls, Ann Arbor, MI. Copyright Q 2014 by the National Association of Pediatric Nurse
Practitioners. Published by Elsevier Inc. All rights reserved.
Conflicts of interest: Cynthia S. Darling-Fisher reports no financial
interests or potential conflicts of interest. Jennifer Salerno reports Published online April 25, 2013.
that as an inventor of the RAAPS Web-based risk screening tool
she receives royalties from the University of Michigan from http://dx.doi.org/10.1016/j.pedhc.2013.03.003

www.jpedhc.org May/June 2014 217

giving providers a tool to systematically assess and identify providers can facilitate adolescent communication dur-
adolescents at risk. Implementation of RAAPS offers health ing health care visits through use of a tool composed of
providers an efficient, consistent, and ‘‘adolescent friendly’’ questions designed to elicit responses (Boekeloo et al.,
way to identify risky behaviors and open the discussion 2003). Such a tool would allow providers to focus
needed to tailor interventions to meet their needs. J Pediatr their limited time more effectively on discussion and
Health Care. (2014) 28, 217-226.
counseling of identified risks (Frankenfeld et al., 2000;
Olsen, Gaffney, Hedberg, & Gladstone, 2009; Yi,
KEY WORDS
Martyn, Salerno, & Darling-Fisher, 2009).
Adolescent risk assessment, risk behavior, adolescent screen-
ing questionnaire
The Guidelines for Adolescent Preventive Services
(GAPS) questionnaire developed by the American
Most health problems among adolescents are due to Medical Association (AMA, 1997) has been considered
risky behaviors rather than biological dysfunction the gold standard screening tool to identify adolescent
(Centers for Disease Control and Prevention [CDC], risk behaviors (Gadomski, Bennett, Young, & Wissow,
2012). In fact, almost 75% of the primary causes of death 2003; Levenberg, 1998). However, the length of this
in the adolescent population are preventable (CDC, comprehensive tool (four pages) and the time needed
2012). Addressing adolescentsÕ risky behaviors in to complete and review it have been barriers to
health care visits is therefore essential to reduce their its use for many health care providers. Additionally, dif-
morbidity and mortality. Current American Academy ferential screening of adolescents based on their age,
of Pediatrics (AAP) and National Prevention Council gender, racial affiliation, socioeconomic level, or per-
(NPC) guidelines recommend routine risk behavior ceived risk behavior status has been identified as a
screening for all adolescents and the use of brief inter- barrier to identifying
vention techniques, as indicated, on an annual basis and addressing adoles- The Rapid
(AAP, 2008; NPC, 2011). cent risk behaviors Assessment for
Despite national recommendations and positive in- (Bethell, Klein, &
tentions, primary care providers report many barriers Peck, 2001; Wiehe, Adolescent
to implementing routine risk screening. The most fre- Rosenman, Wang, Preventive
quently reported concern has been insufficient time Katz, & Fortenberry, Services (RAAPS)
(Cheng, DeWitt, Savageau, & OÕConnor, 1999; Henry- 2011).
Reid et al., 2010; Van Hook et al., 2007; Yarnall, Pollak, The Rapid Assess- risk screening tool
Ostbye, Krause, & Michener, 2003). Although the vast ment for Adolescent was developed to
majority of physicians believe it is their responsibility Preventive Services identify the risk
to educate patients about risk factors, provide support (RAAPS) risk screening
regarding risk behavior issues, and help patients adhere tool was developed to behaviors
to recommended regimens (Schaeuble, Haglund, & identify the risk behav- contributing most
Vukovich, 2010; Wechsler, 1996), only a few believe iors contributing most to adolescent
they have time to provide such interventions (Henry- to adolescent morbid-
Reid et al., 2010). In fact, other research studies confirm ity, mortality, and so- morbidity,
that health care providers rarely ask adolescents specif- cial problems and to mortality, and
ically about their health risk behaviors and little time is provide a more stream- social problems
spent delivering preventive care (Adams, Husting, lined assessment to
Zahnd, & Ozer, 2009; Beebe, Harrion, & Park, 2006; help providers address and to provide
Henry-Reid et al., 2010; Irwin, Adams, Park, & key adolescent risk a more streamlined
Newacheck, 2009; Schaeuble et al., 2010). Furthermore, behaviors in a time- assessment to help
gender of the health care providers has been shown to efficient and user-
affect risk behavior screening, with female health care friendly format. The providers address
providers more likely to identify and respond to risk development and clini- key adolescent risk
behaviors reported by adolescents than their male cal use of the RAAPS behaviors in a time-
counterparts (Epner, Levenberg, & Schoeny, 1998; was described in an
Torkko, Gershman, Crane, Hamman, & Bar on, 2000). earlier publication (Yi efficient and user-
Misperceptions of risk status and/or incomplete elicita- et al., 2009). Research friendly format.
tion of risk information by health care providers leads conducted in several
to missed opportunities for screening and prevention school-based clinic programs demonstrated that
activities. providers using the RAAPS were able to detect
This situation highlights the need for an efficient the major risk behaviors associated with poor
screening tool to help busy health care providers obtain outcomes in adolescents, discuss and document
more complete and systematic risk assessments during these risk behaviors in a single clinic visit, and refer
adolescent visits. Research has shown that health care adolescents for further management when appropriate.

218 Volume 28  Number 3 Journal of Pediatric Health Care

 Strengths of the RAAPS include its concise format,
ease of use, assessment of major risks in a short time,
and establishment of rapport with adolescents (Yi
et al., 2009). The RAAPS is a 21-item questionnaire
that can be completed by adolescents in an average of
5 to 7 minutes. Paper and computerized online versions
of the RAAPS screening tool are available for use (a de-
tailed description of the tool and ways to access it can be
found at www.raaps.org). Further research has demon-
strated the reliability and validity of the RAAPS as a tool
for identifying the adolescent risk behaviors contribut-
ing most to adolescent morbidity, mortality, and social
problems (Salerno, Marshall, & Picken, 2012). Salerno
and colleagues (2012) used psychometric methods to
establish face-, content-, and criterion-related validity
and inter-rater and equivalence reliability of the
paper-based RAAPS. Face validity was established by
focus group consensus. Adolescent expert content val-
idity index scores ranged from 0.825 to 1.0, with inter-
rater content agreement ranging from 0.9 to 1.0.
Criterion-related validity and equivalence between
the RAAPS and GAPS paired questions was demon-
strated by CohenÕs kappa scores ranging from 0.44 to
0.99, with percent agreement ranging from 0.71 to
0.99. FisherÕs exact test showed that all p values were
>.05, indicating no statistically significant differences
in responses to the GAPS- and RAAPS-paired questions
(Salerno et al., 2012).
The purpose of this study was to evaluate the clinical
use of the RAAPS screening tool by surveying health
care providers from a wide variety of clinical settings
and geographic locations who had requested the tool
via the RAAPS Web site: www.raaps.org. Specifically,
providers were surveyed on their current use of the
RAAPS in their practice; their perceptions of its influ-
ence on their practice; their assessment of the RAAPSÕ
performance in terms of its ease of use and efficiency;
its ability to identify adolescent high-risk behaviors;
and its impact on providersÕ communication with ado-
lescents around their risk behaviors.
METHODS
A mixed methods descriptive study was conducted us-
ing an online survey to assess providersÕ use of the
RAAPS and their perspectives on its implementation
and effect on adolescent-provider communication.
The survey included quantitative multiple choice and
Likert-scaled items with qualitative open-ended items
to supplement quantitative data analysis.
Procedure
An online survey was used to obtain providersÕ per-
spectives about the clinical use of the RAAPS tool. Insti-
tutional Review Board approval of the study was
obtained through the University of Michigan. An e-
mail message describing the study with a Web-based
address link to an online Qualtrics survey was sent to
www.jpedhc.org
all providers who had downloaded the paper version
or were currently using the Web-based version of the
RAAPS between 2006 and 2010. All respondents pro-
vided demographic information about themselves
and their practice. The first survey question asked about
current use of the RAAPS. Those responding ‘‘No’’ were
asked to state their reasons for not using the tool and
ended the survey. Individuals who answered ‘‘Yes’’
completed a 16-item survey consisting of multiple
choice and open-ended questions and eight Likert-
scaled statements rating the RAAPS.
The provider survey was evaluated for content valid-
ity by members of the research team who had experi-
ence developing scales for prior funded research on
adolescent risk screening, quality of adolescent care,
and patient-provider communication. The surveyÕs
open-ended questions asked providers to describe
their method of assessing adolescent risk behaviors
prior to using the RAAPS, their current assessment of
the RAAPSÕ influence on their practice with adolescents,
the strengths of the RAAPS, limitations to the RAAPS,
any proposed changes, and whether they would rec-
ommend the RAAPS survey to other health providers,
along with why or why not.
Providers were also asked to respond to eight state-
ments reflecting their experiences with the RAAPS in
terms of its impact on communication with adoles-
cents and its performance in their practice using
a 5-point Likert scale with responses ranging from
strongly agree to strongly disagree (see Table 1).
The statements were selected by the research team
based on review of and experience with existing tools
used in research to assess quality of care, patient-
provider communication, and implementing new
approaches in clinical practice with adolescents
(Woods, Klein, Wingood, Rose, & Wypij, 2006;
Martyn et al., 2012). Statements were then slightly
modified to be more specific to the experience with
the RAAPS. Once responses were obtained, each
statement was examined by the team for the distribu-
tion of responses, its correlation with other items, and
its conceptual meaning. The statements that were
highly correlated with one another (p < .001) and
that shared conceptual similarities were combined
to form composite index scores assessing Perfor-
mance (three items) and Communication (four items).
An overall assessment of experience with the RAAPS
or Overall Experience score, composed of responses
to all eight statements, was also calculated. Internal
consistency for each of the scales and the Overall Ex-
perience score was determined using CronbachÕs co-
efficient alpha with a cutoff of 0.7 as an indicator of
acceptability (Nunnally, 1978). CronbachÕs alpha de-
termines the internal consistency or intercorrelation
of items in a survey instrument to gauge its reliability.
CronbachÕs standardized alphas for the three scales
ranged from 0.81 to 0.91 and are shown in Table 1.
May/June 2014 219
 TABLE 1. Scales assessing providersÕ experiences with the Rapid Assessment for Adolescent
Preventive Services on performance, communication, and overall rating (n = 98)
Scale Statement M SD Range a
Performance RAAPS helped me to document patient encounters related to their risk 10.49 1.77 3-12 0.81
(3 items) behaviors.
RAAPS is easy to use.
RAAPS helped me identify risk behaviors in adolescents.
Communication RAAPS was used to direct risk behavior discussions with my adolescent 13.32 2.47 4-16 0.86
(4 items) patients.
RAAPS helped me to develop rapport with my adolescent patients.
RAAPS helped to improve communication with my adolescent patients.
RAAPS gave adolescents an opening for discussion of their behaviors
and experiences.
Overall experience RAAPS helped me identify risk behaviors in adolescents. 26.84 4.49 10-32 0.91
(8 items) RAAPS was used to direct risk behavior discussions with my adolescent
patients.
RAAPS helped me develop rapport with my adolescent patients.
RAAPS helped to improve communication with my adolescent patients.
RAAPS helped me to document patient encounters related to their risk
behaviors.
RAAPS helped adolescents understand their own risk behaviors and
potential outcomes.
RAAPS gave adolescents an opening for discussion of their behaviors
and experiences.
RAAPS is easy to use.

RAAPS, Rapid Assessment for Adolescent Preventive Services.

Statement responses were scored on a 5-point Likert scale ranging from strongly disagree to strongly agree (scale 0-4).

Data Analysis Use of the RAAPS

SAS 9.2 (SAS Institute Inc., Cary, NC) was used to carry
out statistical tests. Missing values were excluded from
statistical tests. Descriptive statistics were obtained for
the sample and the responses on each of the survey
items. Two-sample t-tests assumed equality of variances
unless otherwise specified. Chi-square and McNemar
tests were used to test associations between variables.
The Fisher exact test was used when the expected cell
count was less than five. Qualitative analyses were per-
formed by the fourth author (an experienced qualitative
researcher) using the constant comparative method to
identify common themes related to use of the RAAPS
from the perspectives of the providers (Glaser, 1992).
The final determination of themes was guided by the
following research question: ‘‘How does the RAAPS
influence adolescent risk assessment and adolescent-
provider communication from providersÕ perspec-
tives?’’ The research team confirmed that the themes
were relevant and reflected the perceptions of the
providers. The common themes were used to enhance
interpretation of the quantitative results.
RESULTS
Descriptive Analyses
Sample characteristics
The provider survey was distributed to 567 providers, of
whom 201 responded, for a response rate of 35%. Re-
sponses came from providers from 26 U.S. states and
three foreign countries (Canada, Korea, and Ireland).
More than half of the respondents (n = 111; 55%) re-
ported they were using the RAAPS in their clinical prac-
tices. Respondents who were not using the RAAPS (n =
90; 45%) had a variety of reasons for not using it. Most
reasons were related to constraints of their health sys-
tem or practice site; other reasons were satisfaction
with their current method of assessment (e.g., GAPS,
HEADSS assessment questions [Home, Education/
employment, Activities, Drugs, Sexuality, and Suicide/
depression], or their perceived skill at interviewing)
and that they were interested in the RAAPS for academic
or research purposes rather than clinical use.
Description of risk assessment practices
Of the persons who responded to this question (n =
107), the majority (n = 92, 86%) were already assessing
adolescent risk behaviors before using the RAAPS. Of
the respondents who provided more detailed informa-
tion (n = 85), most were using the GAPS (n = 45, 53%);
some were doing an informal assessment as part of their
history (n = 18, 21%); the remainder used a variety of
methods, including self-made forms; the HEADSS dur-
ing the history; or Bright Futures Guidelines.
Characteristics of RAAPS users compared with
nonusers
Table 2 presents demographic characteristics of the
RAAPS users and nonusers, as well as the statistically
significant differences between these groups, using
chi-square analyses. The majority of providers who

220 Volume 28  Number 3 Journal of Pediatric Health Care

 TABLE 2. Demographic comparisons between Rapid Assessment for Adolescent Preventive
Services users and nonusers
Current user Yes (%) No (%) v2 p
Provider type (n = 161) 12.7652, df = 2 < .00
Health care provider 64 (75.3) 55 (72.4)
Mental health provider 13 (15.3) 2 (2.6)
Other 8 (9.4) 19 (25.0)
Practice setting (n = 152) 12.7652, df = 1 < .00
Outpatient health clinic 20 (24.1) 36 (52.2)
School-based health clinic 63 (75.9) 33 (47.8)
% Adolescent patients (n = 154) 7.3780, df = 1 .01
# 50% 26 (30.6) 36 (52.2)
> 50% 59 (69.4) 33 (47.8)
Years in practice (n = 157) 6.2597, df = 1 .01
# 5 years 44 (51.8) 23 (31.9)
> 5 years 41 (48.2) 49 (68.1)
U.S. Practice region (n = 151) 29.68, df = 3 < .00
Northeastern United States 13 (15.3) 15 (22.7)
Southern United States 11 (12.9) 22 (33.3)
Midwestern United States 57 (67.1) 16 (24.2)
Western United States 4 (4.7) 13 (19.7)
Race (n = 201) 1.2865, df = 2 .53
Black/African American 11 (9.9) 5 (5.6)
White/Caucasian 66 (59.5) 56 (62.2)
Other 34 (30.6) 29 (32.2)
Provider age in years (n = 145) 4.00, df = 2 .14
20–39 years 21 (25.6) 8 (12.7)
40–49 years 24 (29.3) 19 (30.2)
50+ years 37 (45.1) 36 (57.1)
df, degrees of freedom.

used the RAAPS survey were female (n = 153, 96.5%),
worked as health care providers (n = 161, 75.3%), and
practiced in school-based health centers (n = 152,
75.9%) located in the Midwest (n = 151, 67.1%), in
which the majority of the patient population (greater
than 50%) were adolescents (n = 154, 69.4%). About
45% of RAAPS users (n = 145) were older than 50 years.
Approximately half of the providers who were using
the RAAPS had been in practice for less than 5 years
(n = 157, 51.8%) and predominantly worked with ado-
lescents between the ages of 13 and 15 years (n = 154,
55.3%). Almost half of these providers were practicing
in an urban setting (n = 154, 45.9%). The largest ra-
cial/ethnic group represented (n = 201) was White
(59.5%), followed by African American (9.9%).
Seventy-five percent of the RAAPS users were health
care providers, the majority of whom were nurse prac-
titioners (RAAPS users, n = 57, 67%; RAAPS nonusers,
n = 50, 66%), but the health care provider category
also included physicians (RAAPS users, n = 4, 4.7%;
RAAPS nonusers, n = 5, 6.6%) and physician assistants
(RAAPS users, n = 3, 3.5%; nonusers, n = 0).
Statistically significant differences between charac-
teristics of RAAPS users and nonusers using chi-
square analyses are displayed in Table 2. Statistically
significant differences were noted between RAAPS
users and nonusers with respect to provider types, prac-
tice setting, percent of adolescent patients, years in
practice, and practice region. No statistically significant
demographic differences were found between RAAPS
users and nonusers with respect to race, age, gender
(Fisher exact test, n = 153, p = .47) or practice profile
(urban, suburban, or rural, Fisher exact test, n = 154,
p = .95).
Time spent counseling
Of the 85 providers who provided information about
counseling using the RAAPS, more than half of pro-
viders (n = 45, 53%) spent less than 10 minutes counsel-
ing adolescents on their identified risk behaviors. The
majority of those (n = 37, 43%) spent 6 to 10 minutes
counseling. When compared, providers who spent 11
minutes or more counseling were more likely to be
mental health providers. The longer counseling visits
occurred in practices where more than 50% of the pop-
ulation were adolescents (n = 85, v2 = 3.99, df = 1, p <
.05) and in school-based clinics (n = 83, v2 = 3.99,
df = 1, p < .02).
RAAPS influence on practice
When asked about their use of the RAAPS, 98 providers
responded. Of these, 85% (n = 83) believed that the
RAAPS had influenced their practice and 74 provided
examples of its benefits (refer to qualitative results).

www.jpedhc.org May/June 2014 221

 of the eight statements about their experience with
TABLE 3. Comparison of performance,
the RAAPS were further examined for the participants
communication, and overall experience
who believed that the RAAPS had influenced their prac-
scores between providers who reported that
tice. Each question was coded dichotomously with
Rapid Assessment for Adolescent Preventive
strongly agree/agree in one category and neutral/dis-
Services did and did not have an influence on
agree/strongly disagree in another category. The
practice
McNemarÕs chi-square test was then used to test for an
Influence on Influence on association between favorable perceptions (strongly
practice practice agree/agree responses) and responding that the RAAPS
Yes (n = 83) No (n = 15) had influenced practice (yes/no). Statistically signifi-
Scale Mean SD Mean SD t-test cant differences were found for those who believed
Performance 10.84 1.42 8.53 2.26 3.82* the RAAPS had influenced practice compared with
Communication 13.85 1.93 10.33 2.13 4.33* those who did not on four questions: RAAPS helped
Overall experience 27.82 3.57 21.47 3.89 4.42* me identify risk behaviors (McNemar v2 = 13, df = 1,
*p < .001, df = 16. p = .0003); RAAPS was used to direct risk behavior dis-
cussions with patients (McNemar v2 = 12, df = 1, p =
.0005); RAAPS gave adolescents an opening for discus-
sion of their behaviors and experiences (McNemar v2 =
Of those who believed it had not influenced practice (n 6.23, df = 1, p = .01); and RAAPS is easy to use (McNemar
= 15), 12 provided comments. Most noted that they v2 = 3.77, df = 1, p = .05).
were already addressing risks in their current practice,
and thus it did not change their prior practice. Qualitative Results

Recommendation to other providers ProvidersÕ general perceptions of the RAAPS

When asked if they would recommend the RAAPS to
other providers, 86 responded, and 98% (n = 84) stated
they would recommend the RAAPS. The two most com-
mon reasons cited for their recommendation were for
screening (n = 76, 92%) and identification of risk behav-
iors (n = 75, 90%). Improved communication (n = 52,
63%) and improved documentation (n = 46, 55%) and
increased patient understanding of their risk behaviors
(n = 48, 58%) were also cited by respondents as reasons
to recommend the RAAPS.
ProvidersÕ qualitative comments indicated that the
RAAPS was viewed overall as an effective adolescent as-
sessment and communication tool. One provider
summed up the benefits of the RAAPS by saying that
the RAAPS is:
‘‘Quick, efficient . . . helps get to the heart of the matter
in a way that is engaging to adolescents, using a vehicle
with which they are comfortable (computer) . . . and I
believe that this piece alone helps them share in ways
that they might not otherwise.’’

Assessment of RAAPS: Performance, Identification of risk

communication, and overall experience rating
scales
Results of analyses of the composite scales evaluating
the influence of the RAAPS in terms of its Performance
(ability to identify risks, ease of use, and facilitation of
documentation) and effect on Communication with ad-
olescents, as well as the Overall Experience using the
RAAPS, showed positive ratings in all areas. The aver-
age scores for Performance, Communication, and the
Overall Experience were relatively high for each scale
and are presented in Table 3. However, participants
who indicated that the RAAPS had influenced their
practice had statistically significant higher average rat-
ings on all three scales than did those who indicated
their practice was not influenced by use of this tool
(p < .001; see Table 3).
Assessment of RAAPS: Survey item analysis
To better understand the ways in which the RAAPS
influenced individual practice, the responses on each
Providers pointed out the ease of using the RAAPS to
identify risk behaviors with comments such as, ‘‘It has
helped to identify students at risk and helped me
keep these issues upfront,’’ and ‘‘Improved identifica-
tion of risk factors leading to face to face interview/as-
sessment,’’ and ‘‘Patient acknowledges his/her own
concerns; self-disclosure without pressure.’’ In addi-
tion, providers indicated ‘‘(RAAPS) makes it easier for
me to come right out and ask the questions that I am
thinking . . . for the students to answer honestly too,’’
‘‘I used to dread coming up with a way to discuss risk
behaviors. The RAAPS gives me a way to not only bring
it up, but identify risky behaviors that warrant interven-
tion,’’ and ‘‘Has given important info that would other-
wise go undetected.’’
The RAAPS was also consistently noted as being easy
to use, concise, and comprehensive. Providers de-
scribed the RAAPS as, ‘‘Easy for the children to read
and understand; Easy to quickly see red flags for discus-
sion; Easy to understand questions; comprehensive yet

222 Volume 28  Number 3 Journal of Pediatric Health Care

brief, straightforward, non-judgmental.’’ They also said
it was:
‘‘. . . easy for me to evaluate’’ and ‘‘I like the fact that you
can visually scan the columns and all risk factors are in
a line for easy viewing’’ and ‘‘Documenting interven-
tion on the form makes follow-up easy.’’ In addition,
many liked the Web-based version of the RAAPS, saying
that since the RAAPS is: ‘‘on the computer (not face to
face answering questions), maybe they [teens] feel
more comfortable being honest.’’
Facilitation of risk discussions
Providers believed that the RAAPS facilitated risk dis-
cussions with adolescents in numerous ways. They
noted that the ‘‘(RAAPS) acts as a launching tool for dis-
cussion and intervention regarding risk behaviors,’’ ‘‘It
has given me a starting point with students that I am un-
familiar with,’’ and ‘‘gives this provider justification to
address, educate, and refer adolescents.’’ Many pro-
viders shared that the RAAPS ‘‘opens door for discus-
sion,’’ ‘‘presents opportunities for teens to ask
questions,’’ and ‘‘provides a consistent ÔopeningÕ to ad-
dress risky behaviors, ask about possible issues and
generally address intimate issues.’’ In addition, sensitive
topics are easier to discuss as evidenced by comments
such as, ‘‘It assists in developing a relationship and trust
with my patients,’’ ‘‘It puts sensitive topics right out on
the table for discussion and helps clients feel comfort-
able discussing topics that impact greatly on their over-
all health,’’ ‘‘It makes discussing sensitive subjects less
embarrassing for students and staff,’’ and ‘‘The student
completes the survey, so the topics are on their mind-
s.helps students have a voice for topics they may be
shy of discussing.’’ Lastly, comments such as the RAAPS
‘‘provides more information to help guide intervention
and increases opportunities to provide health guidance
and education’’ show the impact the RAAPS can have
on improving patient outcomes related to their risk
behaviors.
Efficiency and consistency in assessment
Strengths of the RAAPS were identified by many pro-
viders as ‘‘Quick; concise; efficient; fast; easy to inter-
pret,’’ ‘‘The RAAPS definitely speeds the process and
is a very efficient tool,’’ and ‘‘ItÕs quick for adolescents
to fill out in the office’’ and ‘‘Format is quick and easy
to look over.’’ Providers believed that by using the
RAAPS, ‘‘More frequent screenings are possible,’’
and ‘‘(RAAPS) opens the door to risk discussions with-
out having to ask each question verbally.’’ Many pro-
viders also shared the feeling that the RAAPS ‘‘Saves
time.more time-efficient for my practice and yet ef-
fective.’’ In addition, RAAPS helps to reduce dispar-
ities in risk assessment provided to different
adolescent populations. When using the RAAPS: ‘‘All
www.jpedhc.org
adolescents are screened in the same manner by all
.providers here,’’ and ‘‘Because itÕs a standard form
that every student gets. it helps with asking difficult
questions without making kids feel weird about it,’’
and ‘‘Allows for more thorough risk assessment on
EVERY adolescent.’’
Areas for improvement
Some providers listed limitations or areas to change in
the RAAPS, including: it was not as comprehensive as
other assessment tools; not having time to complete
and to address risk behaviors identified; and concerns
that adolescents would not answer honestly. For exam-
ple, some providers noted that they did not use the
RAAPS because they used other assessment forms,
such as the GAPS form. One provider stated: ‘‘We are
using the GAPS form . . . a more in depth form and really
creates an atmosphere for adolescents to expand on
their answers and opens up the lines of communication
for a more detailed educational response.’’ However,
others found the RAAPS to be a more efficient tool
than the GAPS and said: the ‘‘. . . GAPS puts teens off
and starts the visit off on negative note as many were
not happy about filling out such a long form.’’
It was also noted by providers that some history
information was not included on the RAAPS, such
as more details on risk behaviors, general health,
mental health, resiliency, body size, and treatment
for sexually transmitted infection. Health care pro-
viders expressed concerns about needing to clarify
sensitive issues, and one said: ‘‘Many students check
that they are abused because they were in a fight at
school. I have to spend a lot of time trying to deter-
mine if they are abused or not.’’ Another provider
questioned applicability for certain populations
when they said, ‘‘Some questions are not applicable
for inner-city impoverished kids or kids 18 years
and over, like helmet use. Also, in our clinic, diet
concerns just are not a priority when STIs, preg-
nancy, and drug use are so prominent.’’ Issues
were also raised about using the RAAPS with youn-
ger age groups and those with limited literacy. Sev-
eral respondents wished for tools specific to
younger age groups: ‘‘Would like an elementary
age risk assessment please’’ and ‘‘Younger kids
have different risks—more safety and less sexual.’’
Constructive suggestions were also made with regard
to format, such as adding more room for recording,
changing of wording for certain questions, and addi-
tional risk questions to add.
Overall perception of the RAAPS
At the end of the survey, providers were asked, ‘‘Is there
anything else you would like to tell us about the
RAAPS?’’ Providers were very positive, with many ex-
pressing ‘‘Thanks’’ for the development of the survey.
May/June 2014 223
One comment summarized some of the issues raised the RAAPS assisted providers in discussing sensitive
concerning implementation and the need for such topics with their adolescent patients and provided op-
a tool. portunities for teens to ask questions. Providing confi-
I am a fan of the RAAPS survey. I wish it was more dential preventive services through risk assessment
accepted here in (my state) as a screening tool. I and counseling has an additional benefit of establishing
think that more research needs to be conducted trust between the adolescent and the provider. This mu-
with regard to identification of risks and the promo- tual alliance with the teens and their providers allows
tion of positive health behaviors. I think that we also for disclosure of information about teen risk behaviors
need to screen our patientÕs strengths to help them and provides an opportunity to seek advice. Research
understand the positive behaviors and resources has shown that for sensitive topics such as sex, contra-
that they employ. ception, and sexually transmitted infections, adoles-
cents consider their health care providers the most
DISCUSSION reliable sources of information, thus enhancing the ef-
The results of this study provide strong support for use fectiveness of preventive services (Duncan et al.,
of the RAAPS in clinical practice settings. The majority 2007; Elster, 1998; Elster & Levenberg, 1997). For
of the RAAPS users from a wide range of practice sites many adolescents, a health care provider is the primary
were very positive in their assessment of the RAAPS as adult they can confide in about health concerns and
an efficient and easy-to-use tool both to identify high- health risk behaviors (Levenberg, 1998; Schaeuble
risk behaviors of their adolescent patients and to facili- et al., 2010). Identifying and discussing risk behaviors
tate discussion and communication with adolescents can be a comfort to adolescents who may have no other
around these issues. A large proportion of the providers source of information or support (Bartlett, Holditch-
(85%) believed that the Davis, & Belyea, 2007). Furthermore, when providers
RAAPS had positively The majority of the are able to identify and discuss risk behaviors during
influenced their prac- health care visits, teens learn that their behaviors are le-
tice with adolescent RAAPS users from gitimate health care concerns and their health care pro-
patients. The signifi- a wide range of viders are open to talking with them about these topics
cant factors identified practice sites were (Harrison, Beebe, Park, & Rancone, 2003).
as affecting their prac- Using a tool like the RAAPS could also prevent the
tice were: identifica- very positive in their bias that has been identified in the literature both in
tion of risk behaviors; assessment of the terms of assumptions made by providers about who
ability to direct risk be- RAAPS as an should be screened for risk and with respect to practice
havior discussions; giv- variations based on gender or provider experience.
ing the adolescents an efficient and easy- This systematic approach to assessment also facilitates
opening for discussion to-use tool both to implementation of the current national screening rec-
of their behaviors and identify high-risk ommendations in a brief and time-efficient format.
experiences; and ease Thus the use of the RAAPS can help enhance the quality
of use. behaviors of their of care in clinical practice.
Implications for adolescent Further research is needed to understand ways the
practice are significant patients and to RAAPS can be used to identify and then intervene
in terms of work- with at risk youth. Providers commented that the
ing with individual facilitate RAAPS provides an excellent way to identify risk behav-
adolescent patients to discussion and iors and open the discussion, but the actual interven-
develop a better under- communication tion/risk reduction counseling is dependent on the
standing of potential providerÕs skill. Approaches such as Motivational Inter-
group education inter- with adolescents viewing have been shown to be effective in work with
ventions in the practice around these adolescents (Bernstein et al., 2009; Black et al., 2010;
setting and providing issues. DÕAmico, Miles, Stern, & Meredith, 2008; Kiene &
quality care to adoles- Barta, 2006; McCambridge & Strang, 2004; Monti
cent patient popula- et al., 2007; VanVoorhees et al., 2009). Event History
tions. The availability of a tool with demonstrated Calendars are another effective approach to more fully
reliability and validity that is well accepted in the clinical involve high-risk adolescents in the discussion of their
practice setting promotes standardization of care so pro- sexual risk behaviors and to help them focus on their
viders can efficiently and systematically assess all their goals (Martyn, Reifsnider, & Murray, 2006; Martyn
adolescent patients for potential risk behaviors. et al., 2012; Martyn, Saftner, Darling-Fisher, & Schell,
Providers repeatedly stated that the RAAPS helped 2013). Research that uses the RAAPS to identify youth
them initiate communication and identify risky behav- at risk combined with ‘‘adolescent friendly’’ interven-
iors that might otherwise go undetected. For instance, tion strategies to promote behavior change is needed

224 Volume 28  Number 3 Journal of Pediatric Health Care

to determine ways in which providers can best affect
these risk behaviors.
Although the RAAPS is an excellent tool to identify
the major risk behaviors in adolescents, it is not a com-
prehensive assessment tool. As several providers com-
mented, a number of areas are not included in the
RAAPS, such as school performance and general health.
The RAAPS does not take the place of a full history;
however, it does allow all adolescents and their pro-
viders (regardless of their comfort with this population)
to identify risk behaviors in an easy and efficient fash-
ion. Although this study provided input from a wide
range of providers across the United States, it was lim-
ited in that a majority of the providers were nurse prac-
titioners who currently worked with adolescent
patients in school-based settings in the Midwest. Addi-
tional research with physicians and physician assistants
and other providers in outpatient or community clinics
in other areas of the United States may be warranted.
However, the positive response from a significant num-
ber of experienced adolescent providers provides
strong support for broader implementation of the
RAAPS in adolescent health care. In this era of managed
care, health assessment and preventive services are in-
creasingly needed to improve adolescent health and re-
duce costs (Harrison et al., 2003; Institute of Medicine
and National Research Council, 2011). Health care pro-
viders need to consider the interrelatedness of health
risk behaviors among their adolescent patients with
various behavioral risks (Fox, McManus, & Arnold,
2010). Use of the RAAPS as a standard component of
adolescent health care practice can help achieve these
goals.
This study has shown that the RAAPS risk screening
tool was able to identify and facilitate risk behavior dis-
cussions between health care providers and adoles-
cents in a time-efficient manner. Use of the RAAPS
could lead to more effective adolescent preventive ser-
vices by providing a tool to systematically assess and
identify those adolescents in need of specific services.
This may reduce the documented disparities that occur
in identification of risk. This may also allow health care
providers to tailor the education provided to adolescent
patients with the ultimate goal of decreasing their mor-
bidity and mortality and improving their overall health.
REFERENCES
Adams, S. H., Husting, S., Zahnd, E., & Ozer, E. M. (2009). Adoles-
cent services: Rates and disparities in preventive health topics
covered during routine medical care in a California sample.
Journal of Adolescent Health, 44(6), 536-545.
American Academy of Pediatrics, Committee on Adolescence.
(2008). Achieving quality health services for adolescents. Pedi-
atrics, 121(6), 1263-1270.
American Medical Association. (1997). Guidelines for adolescent
preventive services (GAPS). Chicago, IL: American Medical
Association. Retrieved from http://www.ama-assn.org/ama/
upload/mm/39/gapsmono.pdf
www.jpedhc.org
Bartlett, R., Holditch-Davis, D., & Belyea, M. (2007). Problem behav-
iors in adolescents. Pediatric Nursing, 33(1), 13-18.
Beebe, T., Harrison, P., & Park, E. (2006). The effects of data collec-
tion mode and disclosure on adolescent reporting of health be-
havior. Social Science Computer Review, 24(4), 476-488.
Bernstein, E., Edwards, E., Dorfman, D., Heeren, T., Bliss, C., & Bern-
stein, J. (2009). Screening and brief intervention to reduce mari-
juana use among youth and young adults in pediatric emergency
department. Academic Emergency Medicine, 16(11), 1174-1185.
Bethell, C., Klein, J., & Peck, C. (2001). Assessing health system pro-
vision of adolescent preventive services: The young adult health
care survey. Medical Care, 39, 478-490.
Black, M. M., Hager, E. R., Le, K., Anliker, J., Arteaga, S. S., DiCle-
mente, C., . Wang, Y. (2010). Challenge! Health promotion/
obesity prevention mentorship model among urban, black ado-
lescents. Pediatrics, 126(2), 280-288.
Boekeloo, B. O., Bobbin, M. P., Lee, W. I., Worrell, K. D., Hamburger,
E. K., & Russek-Cohen, E. (2003). Effect of patient priming and
primary care provider prompting on adolescent-provider com-
munication about alcohol. Archives of Pediatrics & Adolescent
Medicine, 157(5), 433-439.
Centers for Disease Control and Prevention. (2012). The youth risk
behavior surveillance system—United States 2011. MMWR,
61(4). Retrieved from http://www.cdc.gov/mmwr/pdf/ss/
ss6104.pdf
Cheng, T. L., DeWitt, T. G., Savageau, J. A., & OÕConnor, K. G.
(1999). Determinants of counseling in primary care pediatric
practice: Physician attitudes about time, money, and health
issues. Archives of Pediatrics & Adolescent Medicine, 153,
629-635.
DÕAmico, E., Miles, J. N. V., Stern, S. A., & Meredith, L. S. (2008).
Brief motivational interviewing for teens at risk of substance
use consequences: A randomized pilot study in a primary
care clinic. Journal of Substance Abuse Treatment, 35(1),
53-61.
Duncan, P. M., Garcia, A. C., Frankowski, B. L., Carey, P. A., Kallock,
E. A., Dixon, R. D., & Shaw, J. S. (2007). Inspiring healthy ado-
lescent choices: A rationale for and guide to strength promotion
in primary care. Journal of Adolescent Health, 41(6), 525-535.
Elster, A. B. (1998). Comparison of recommendations for adolescent
clinical preventive services developed by national organizations.
Archives of Pediatrics & Adolescent Medicine, 152(2), 193-198.
Elster, A. B., & Levenberg, P. (1997). Integrating comprehensive ad-
olescent preventive services into routine medicine care: Ratio-
nale and approaches. Pediatric Clinics of North America,
44(6), 1365-1377.
Epner, J. E., Levenberg, P. B., & Schoeny, M. E. (1998). Primary care
providersÕ responsiveness to health-risk behaviors reported by
adolescent patients. Archives of Pediatrics & Adolescent Med-
icine, 152(8), 774-780.
Frankenfield, D. L., Keyl, P. M., Gielen, A., Wissow, L. S., Werthamer,
L., & Baker, S. P. (2000). Adolescent patients—healthy or hurt-
ing? Missed opportunities to screen for suicide risk in primary
care setting. Archives of Pediatrics & Adolescent Medicine,
154(2), 162-168.
Fox, H. B., McManus, M. A., & Arnold, K. N. (2010). Significant multi-
ple risk behaviors among U.S. high school students (Fact Sheet
No. 8). Retrieved from http://www.thenationalalliance.org
Gadomski, A., Bennett, S., Young, M., & Wissow, L. S. (2003).
Guidelines for adolescent preventive services: The GAPS in
practice. Archives of Pediatrics & Adolescent Medicine,
157(5), 426-432.
Glaser, B. G. (1992). Basics of grounded theory analysis. Mill Valley,
CA: Sociology Press.
Harrison, P. A., Beebe, T. J., Park, E., & Rancone, J. (2003). The
adolescent health review: Test of a computerized screening
tool in school-based clinics. Journal of School Health, 73(1),
15-20.
May/June 2014 225
Henry-Reid, L. M., OÕConnor, K. G., Klein, J. D., Cooper, E., Flynn,
P., & Futterman, D. C. (2010). Current pediatrician practices
in identifying high-risk behaviors of adolescents. Pediatrics,
125(40), e741-e747.
Institute of Medicine & National Research Council. (2011). The sci-
ence of adolescent risk-taking workshop report. Washington,
DC: National Academies Press.
Irwin, C. E., Adams, S. H., Park, J., & Newacheck, P. W. (2009). Pre-
ventive care for adolescents: Few get visits and fewer get ser-
vices. Pediatrics, 123(4), e565-e572.
Kiene, S. M., & Barta, W. D. (2006). A brief individualized computer-
delivered sexual risk reduction intervention increases HIV/AIDS
preventive behavior. Journal of Adolescent Health, 39(3),
404-410.
Levenberg, P. B. (1998). GAPS: An opportunity for nurse practi-
tioners to promote the health of adolescents through clinical
preventive services. Journal of Pediatric Health Care, 12(1), 2-9.
Martyn, K. K., Darling-Fisher, C., Pardee, M., Ronis, D. L., Felicetti,
I. L., & Saftner, M. A. (2012). Improving sexual risk communica-
tion with adolescents using event history calendars. Journal of
School Nursing, 28(2), 108-115.
Martyn, K. K., Reifsnider, E., & Murray, A. (2006). Improving adoles-
cent sexual risk assessment with event history calendars: A fea-
sibility study. Journal of Pediatric Health Care, 20(1), 19-26.
Martyn, K. K., Saftner, M. A., Darling-Fisher, C., & Schell, M. (2013).
Sexual risk assessment using event history calendars with male
and female adolescents. Journal of Pediatric Health Care, 27(6),
406-409. http://dx.doi.org/10.1016/j.pedhc.2012.05.002.
McCambridge, J., & Strang, J. (2004). The efficacy of single-session
motivational interviewing in reducing drug consumption and
perceptions of drug-related risk and harm among young peo-
ple: Results from a multi-site cluster randomized trial. Addiction,
99(1), 39-52.
Monti, P. M., Barnett, N. P., Colby, S. M., Gwaltney, C. J., Spirito, A.,
Rohsenow, D. J., & Woolard, R. (2007). Motivational interview-
ing versus feedback only in emergency care for young adult
problem drinking. Addiction, 102(8), 1234-1243.
National Prevention Council. (2011). National prevention strategy.
Washington, DC: U.S. Department of Health and Human Ser-
vices, Office of the Surgeon General.
Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York, NY:
McGraw Hill.
Olsen, A. L., Gaffney, C. A., Hedberg, V. A., & Gladstone, G. R.
(2009). Use of inexpensive technology to enhance adolescent
226 Volume 28  Number 3
health screening and counseling. Archives of Pediatrics & Ado-
lescent Medicine, 163(2), 172-177.
Salerno, J., Marshall, V., & Picken, E. (2012). Rapid assessment for
adolescent preventive services: Validity and reliability of the
RAAPS adolescent risk screening tool. Journal of Adolescent
Health, 50(6), 595-599.
Schaeuble, K., Haglund, K., & Vukovich, M. (2010). AdolescentsÕ
preferences for primary care provider interactions. Journal for
Specialists in Pediatric Nursing, 15, 202-210.
Torkko, K. C., Gershman, K., Crane, L. A., Hamman, R., &
Baro n, A. (2000). Testing for Chlamydia and sexual history
taking in adolescent females: Results from a statewide
survey of Colorado primary care providers. Pediatrics,
106(3), E32.
Van Hook, S., Harris, S. K., Brooks, T., Carey, P., Kossack, R., Kulig,
J., Knight, J. R. & New England Partnership for Substance
Abuse Research. (2007). The ‘‘Six TÕs’’: Barriers to screening
teens for substance use in primary care. Journal of Adolescent
Health, 40(5), 456-461.
VanVoorhees, B. W., Fogel, J., Reinecke, M. A., Gladstone, T.,
Stuart, S., Gollan, J., . Bell, C. (2009). Randomized clinical
trial of an internet-based depression prevention program for
adolescents (PROJECT CATCH-IT) in primary care: 12 week
outcomes. Journal of Developmental & Behavioral Pediatrics,
30(1), 23-27.
Wechsler, H. (1996). The physicianÕs role in health promotion revis-
ited: A survey of primary care practitioners. New England Jour-
nal of Medicine, 334(15), 996-998.
Wiehe, S. E., Rosenman, M. B., Wang, J., Katz, B. P., & Fortenberry,
J. D. (2011). Chlamydia screening among young women: Indi-
vidual- and provider-level differences in testing. Pediatrics,
172(2), e336-e344.
Woods, E. R., Klein, J. D., Wingood, G. M., Rose, E. S., & Wypij, D.
(2006). Development of a new Adolescent Patient-Provider In-
teraction Scale (APPIS) for youth at risk for STDs/HIV. Journal
of Adolescent Health, 38(6), 753.e1-753.e7.
Yarnall, K. S., Pollak, K. I., Ostbye, T., Krause, K. M., & Michener, J. L.
(2003). Primary care is there enough time for prevention. Amer-
ican Journal of Public Health, 93(4), 635-641.
Yi, C. H., Martyn, K., Salerno, J., & Darling-Fisher, C. S. (2009).
Development and clinical use of Rapid Assessment for Adoles-
cent Preventive Services (RAAPS) questionnaire in school
based health centers. Journal of Pediatric Health Care, 23(1),
2-9.
Journal of Pediatric Health Care