Discussion Post

Read two of the articles, then discuss [in detail] at least four (4) things learned about searching and appraising evidence.

https://canvas.polk.edu/courses/3481/files/280452/download?verifier=qkMBQkr9gYOjV9zOb9IcUJ9NzElB2Z0fPeVq5RNB&wrap=1

Practical Tips for Literature Synthesis
Cheryl Westlake, PhD, RN, ACNS-BC

T
he clinical nurse specialist (CNS), as an advanced
practice nurse, has the primary goal of continuous
improvement in patient outcomes and nursing care

by creating clinical practice environments that reflect
evidence-based practices and interventions.1 A synthesis
of the literature may be needed to answer a clinical ques-
tion for implementation in practice, serve as an indepen-
dent paper for publication as a review, or be part of a
larger project such as a research proposal, dissertation,
or data-based publication. In order to achieve this goal,
the CNS needs to be able to systematically review the lit-
erature and synthesize the findings on issues related to
patient outcomes and nursing care.

The synthesis of the literature is intended to do more
than just document or summarize the relevant literature.
Rather, the synthesis of the literature is intended to pro-
vide a detailed analysis and yield conclusions about the
current state of the science and the knowledge gaps
about the topic of interest. An example of such a review
created by Dumoulin and Hay-Smith2 on pelvic floor
muscle training for urinary incontinence in women was
published as a synopsis in the journal Clinical Nurse
Specialist by McLoughlin and McAuley.3 The review serves
as the example cited throughout this article.

While creating and writing a synthesis of the literature
may seem like a daunting task to the CNS who has many
demanding responsibilities and literature sources with
which to maintain currency, the goal of this article is to
provide a simple guide for the process that may be used
easily by the CNS. Specifically, the purpose of this article
is to provide a step-by-step process of how one might
approach the literature synthesis. The specific topics
covered in this article are the steps involved in a synthe-
sis of the literature including recommendations about

how to analyze each individual paper for potential inclu-
sion in the analysis and synthesis, specific processes for
analyzing the collective papers, and suggestions for writ-
ing the final product. Tools one might use to facilitate the
process with examples using the Dumoulin and Hay-
Smith’s2 article are provided to smooth the process and
soothe the mind of the ambitious CNS.

A literature synthesis is a focused review, in-depth eval-
uation, critical analysis, and creative compilation of infor-
mation based on patterns, alternatives, or relationships
gleaned from the process that allow one to come to a
higher level of understanding or knowledge about a spe-
cific topic. ‘‘It works very much like a jigsaw puzzle. The
individual pieces (arguments) must be put together in or-
der to reveal the whole (state of knowledge).’’4

CONSIDER AND REFINE THE QUESTION
The first step in the synthesis of the literature is to care-
fully consider and refine the question the CNS wishes to
ask of the literature. Particular attention to the proposed
concepts, research design, sample selection criteria and
process, measures, and statistical analyses is warranted
and recommended to ease the process. Refinement of the
topic and clarity about the question are critical at the onset
and is enhanced by the expert knowledge base of the
CNS. The eventual quality of the synthesis of the literature
is dependent, in part, on this important first step. Issues
for consideration are provided in Figure 1.

Using the example of urinary incontinence, the posed
question may have been: What is the difference in wom-
en’s outcomes for different interventions for urinary in-
continence including to pelvic floor muscle training?

LITERATURE SEARCH
Next, the CNS needs to consider the focused question
and describe the state of the science related to the CNS’s
potential future as reflected in the available literature. In
addition, the CNS needs to address the knowledge gaps
and the areas where the available literature does not
inform the CNS’s potential future project. The quality of

Author Affiliation: Professor and Associate Dean, International and
Community Programs, School of Nursing, Azusa Pacific University,
California.
The author reports no conflicts of interest.
Correspondence: Cheryl Westlake, PhD, RN, ACNS-BC, School of Nurs-
ing, Azusa Pacific University, 720 E Foothill Blvd, Azusa CA 91702
(ccanary@apu.edu).
DOI: 10.1097/NUR.0b013e318263d766

244 www.cns-journal.com September/October 2012

Clinical Nurse SpecialistA Copyright B 2012
Wolters Kluwer Health | Lippincott Williams & Wilkins

Using Research to Advance
Nursing Practice

Column Editor: Janice Buelow, PhD, RN, FAAN

the literature search is dependent on the thoroughness
and accuracy of the writer’s search. Thus, consultation
with a librarian is recommended as librarians are expert
in search methods and terms.

The refinement of the question and the review of the
literature is an iterative process that is recursive in nature
with the desired end point a fully refined and focused
question with a matching review of the literature. This
second step may require some consideration of the ques-
tion, review and sorting of the literature, and reconsidera-
tion and refinement of the question with another return
to the literature (Figure 2).

With each iteration, the CNS is wise to document the
changes, additions, or deletions made in the search terms
and databases. These changes may be made by hand in a
paper notebook at the side one’s computer, online in a
notebook such as Notepad (http://www.google.com/
notebook/#b=BDQmGSgoQl92H_PAm) Evernote (http://www
.evernote.com/ for personal computer users, or Evernote
for Mac users http://www.evernote.com/), or Notes on the
iPad, or in a simple word document, or using an applica-
tion such as Sticky Notes (http://www.sticky-notes.net/ for
personal computer users and stickynotesapp.com for Mac
users). Whatever the method, the CNS is now well poised
to move forward, and the documented changes will serve
to streamline the third step.

LITERATURE SEARCH METHODS
A complete and comprehensive description of the liter-
ature search in very specific, concrete terms is required
next. Begin with an introduction that outlines the empir-
ical literature reviewed explaining the methods used in
your literature search. Remember to include the search
strategies, keywords used, databases and periods searched,
a description of articles that were excluded and the ra-
tionale for the exclusion. For example, considering the
question about pelvic floor muscle training posed at the

outset, the following might be offered regarding the
search. A search in CINAHL plus full text, Health Source:
Nursing/Academic Edition, and MEDLINE with full text
of English-language, peer-reviewed, research articles of
adult (919 years of age) women using the search terms
pelvic floor muscle training yielded 51 results for from
2002 to 2012 that reduced to 38 by adding urinary incon-
tinence and was further reduced to 32 by adding women.
All articles were read. Six articles were not included in
the final review of the literature. Three were deemed in-
appropriate on the basis of the population,5Y7 a fourth
because the study was an economic evaluation not
a clinical evaluation,8 and 2 additional articles as they
were not clinical research studies.9,10 Thus, 26 articles
were included for the next stet.

ANALYZE INDIVIDUAL, RETAINED PAPERS
Now that the articles for inclusion in the review of the
literature have been identified in a cursory fashion, the
CNS must critically appraise the quality of the selected
empirical studies. Based on the internal (instrument re-
liability and statistics, equivalence of participant charac-
teristics, and control of experience/environment variables)
and external validity characteristics (operations and instru-
ment validity, population validity, and ecological validity)
by Cook and Campbell,11 evaluate each study that re-
mains for consideration in the review of the literature.
This review may be done by the CNS and another col-
league or two or by an external judge panel. Those arti-
cles that are evaluated as being of sufficient quality will
be retained for the review of the literature.

REVIEW AND ORGANIZE THE LITERATURE
Once the relevant, high quality papers have been iden-
tified and located, one needs to read, analyze, and or-
ganize them to begin the synthesis. The first step is to
analyze each individual article and then organize all the

FIGURE 1. Define your research question.

FIGURE 2. Question refinement and literature review.

Clinical Nurse SpecialistA www.cns-journal.com 245

articles into a usable format for review and synthesis col-
lectively. The method frequently used in nursing research
courses is to compile the individual articles into a table
of evidence or matrix table.12,13

An example of a single entry in a table of evidence using
an article from our pelvic floor muscle training muscle re-
view of literature is provided in Table 1.13 The table of
evidence would include an entry with the associated data
included for each of the retained studies.

STRUCTURE AND SUMMARIZE THE LITERATURE
Now, the CNS is ready to reflect about the studies in the
table for the purpose of structuring and summarizing the
information. Review the studies, take notes, sort, and or-
ganize your concepts and their relationships. Seek the
common points or themes that allow the individual arti-
cles to be grouped into specific categories. The created
similarity heuristic will facilitate the development of a ta-
ble of these common points or themes that will serve as
an organizing framework for analyzing, and synthesizing
of the individual articles into a whole. These categories
may be used later to organize the article.

The CNS may wish to organize all the individual articles
using a single table with all the potential relationships in-
dicated and the studies supporting or refuting the relation-
ships cited in the table. Using our pelvic floor muscle
training example and the article of Borello-France et al,13

the table might be constructed as in Table 2. The final
table would include all articles retained in the review of
the literature.

Into each of the boxes where a relationship is de-
scribed in the table of evidence, a plus or minus sign and
the authors/date may be written to indicate studies where
the relationship is supported or refuted. Notes and thoughts
may be written in the margins for reflection in order for
conclusions to be drawn.

Another variation on the similarity heuristic table may
be to create the table using only the interventions as the
common points across which the literature is reviewed.
Now, the CNS might complete the table as indicated
in Table 3 and synthesize the literature by intervention.
The article by Borello-France et al13 is included in the
sample (Table 3). As the original question posed in the
pelvic floor muscle training was about a comparison of
interventions, such a table may be most useful. Similar to
Table 2, all retained articles would be included in the fi-
nal version of Table 3.

Another strategy to synthesize the collective literature
the CNS may wish to consider is the use of a concept map.
Concept maps are tools for organizing and graphically re-
presenting knowledge. The concepts are usually indicated
by circles or boxes, and the relationships among concepts
by colors and between concepts by lines or arrows, if the
direction is known. Words or symbols such as +, j, &, and
% may be placed on the line to describe the relationships
between concepts. Such an organizing tool may provide the
CNS with a tool to visualize the relationships between and
among the concepts. Concept maps have been described
as particularly useful for organizing complex scientific in-
formation14 and creating a literature review (Figure 3).15Y18

Table 1. Table of Evidence

Method
Sample (Age; UI,
SI, UrI, or Mixed)

Treatment (PFMT, None,
Sham, PL, or Other)

Outcome
(SC, SxI, QOL) Risk Bias

& 2-Arm RCT, parallel design & N = 44 & PFMT supine (n = 22) & Primary end point
at 9Y12 wk

& Random sequence
generationa& Comparison: supine

PFMT vs supine and
upright PFMT

& SI & PFMT alternate exercise
between supine, sitting,
and standing positions
(n = 22)

& Primary outcome: not
stated

& Allocation
concealmenta

& No a priori power
calculation

& 1 US site

& Other outcome measures & Blindingb
& 38Y70 y old

) QOL (IIQ) & Incomplete
outcome datac

& Not pregnant

) Urinary diary
& Selective reportingd

& Ambulatory

) Modified 1-h pad test
& Symptoms of SI Q1/wk

) Digital PFM
assessment
(Brink scale)

& Mean age: 51.7 T 8.9 y
vs 53.6 T 8.1 y

Abbreviations: IIQ, Incontinence Impact Questionnaire; PFM, pelvic floor muscle; PFMT, pelvic floor muscle training; PL, placebo; QOL, quality of life; RCT, random-
ized controlled trial; SC, symptomatic cure; SI, stress incontinence; SxI, symptomatic improvement; UI, urinary incontinence; UrI, urge incontinence.
aSelection bias.
bPerformance bias and detection bias.
cAttrition bias.
dReporting bias.
Borello-France et al.13

246 www.cns-journal.com September/October 2012

Using Research to Advance Nursing Practice

The literature review as a concept map for the pelvic floor
muscle training is represented as an example in Figure 4.

The concept map may be created as a rough hand-
drawn diagram, on a computer using a simple document
format or concept mapping software (www.SmartDraw.com
for personal computer users or Inspiration Maps for the
iPad http://www.inspiration.com/ipad),or online-developed
sticky notes or note cards.

SYNTHESIZE THE LITERATURE
At this step, the CNS is challenged with drawing conclu-
sions about the concepts and their relationship. Note re-
peated words, concepts, and instruments, and conceptual,
methodological, and analytical conflicts or contradictions
within and between studies. The identification of the pat-
terns, trends, strengths, and weaknesses of the reviewed
literature may be accomplished by examining the similar-
ities and differences between and among the concepts.
The state of the science about the topic must be de-
scribed, the knowledge gaps illuminated, and the clinical
implications delineated. Keep the original question in the
forefront as the synthesis is conducted and later written.
The synthesis will describe these concepts and their rela-
tionships in a way that is novel and creative and may lay
the foundation for future work or study.

WRITE THE LITERATURE SYNTHESIS PAPER
The final step is for the CNS to write the paper describing
the synthesis of the literature. Begin with the question in
mind and provide the reader with the rationale about
why the synthesis is needed. The background and ratio-
nale for the synthesis of the literature may assist the reader
in understanding the scope of the concepts, literature, syn-
thesis, and implications. Use the steps as the outline for the
written work. Tables and figures created by the CNS may
serve to communicate the process and elucidate the relation-
ships between and among concepts. Describe the similarities
and differences, methods and limitations, and strengths and
weaknesses of the literature as a whole. Use headings and
subheadings to allow the reader totrack andfacilitategreater
understanding of the synthesis of the literature. Focus on
the question the synthesis addresses and the totality of
the literature rather than on the individual papers. Finish
your paper with a concluding paragraph that summarizes
the critical findings offered. The use of bullet points may
make your key findings abundantly obvious.19 Finally, pro-
vide the reader with the important clinical and research
implications of the synthesis in a few sentences. Using
the example of our pelvic floor muscle training question,
Dumoulin and Hay-Smith2 offered:

Table 3. Table Matrix Alternate Form

UI SI

UrI

PFMT SC

SxI

QOL Borello-France et al13

& Effect of exercise position (supine
or supine and upright positions)
– IIQ at baseline and after
treatment

& Exercise position j

& PFMT +

Sham SC

SxI

QOL

Placebo SC

SxI
QOL

Other SC

SxI
QOL

Abbreviations: IIQ, Incontinence Impact Questionnaire; QOL, quality of life;
PFM, pelvic floor muscle; PFMT, pelvic floor muscle training; SC, symptomatic
cure; SI, stress incontinence; SxI, symptomatic improvement; UI, urinary
incontinence; UrI, urge incontinence.

Table 2. Literature Matrix

Age PFMT

SHAM,
Placebo,
Other SC SxI QOL

SI Borello-France
et al13 j +;
no effect of
position

UrI

Age

PFMT X Borello-France
et al13 + IIQ;
no effect of
position

Sham,
placebo,
other

SC X

SxI X

QOL X

Clinical Nurse SpecialistA www.cns-journal.com 247

The review provides support for the widespread
recommendation that PFMT be included in
first-line conservative management programs
for women with stress, urge, or mixed urinary
incontinence. Statistical heterogeneity reflecting
variation in incontinence type, training, and
outcome measurement made interpretation difficult.
The treatment effect seems greater in women with
stress urinary incontinence alone, who participate
in a supervised PFMT program for at least 3 months,
but these and other uncertainties require testing
in further trials.2(p2)

In summary, a synthesis of the literature is a high-level
skill needed often by the CNS in achieving the goals of
continuous improvement in patient outcomes and nurs-

ing care through evidence-based practices and interven-
tions. The skill may be described in a 7-step process:

1. consider and refine the question to be addressed in
the synthesis of the literature;

2. search the literature for relevant studies;
3. analyze the quality of the individual articles;
4. review and organize the literature into a manage-

able form;
5. structure and summarize the collective literature;
6. synthesize the literature addressing the state of the

science and knowledge gaps regarding the original
question; and

7. write the synthesis of the literature.
Tools the CNS might use to facilitate the process such

as a table of evidence, literature matrix, or concept map

FIGURE 3. Concept mapVsorting relevant information. From Steinerová.19

248 www.cns-journal.com September/October 2012

Using Research to Advance Nursing Practice
Copyright @ 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

were offered, and examples of each provided. Hopefully,
the individual CNS reader will feel more empowered to
generate and publish these important literature syntheses
in order that all CNSs might benefit and use the review in
the quest to improve patient outcome\s and nursing care.

Resources
Literature Review: Synthesizing Multiple Sources. http://www.iupui

.edu/~uwc/pdf/Literature%20Review%20and%20Synthesis .
Writing A Literature Review and Using a Synthesis Matrix. http://

www.ncsu.edu/tutorial_center/writespeak/PDF%20Handouts/
synthesis%20matrix .

The Use of Mapping in Literature Review. http://sites.google.com/
site/andycoverdale/texts/the-use-of-mapping-in-literature-review.

Galvan J. Writing Literature Reviews: A Guide for Students of the Be-
havioral Sciences. 3rd ed. Glendale, CA: Pyrczak Publishing; 2006.

References
1. APRN Consensus Work Group & the National Council of State

Boards of Nursing APRN Advisory Committee. July 7, 2008.
Consensus Model for APRN Regulation: Licensure, Accreditation,
Certification & Education. APRN Joint Dialogue Group Report.
http://www.aacn.nche.edu/education-resources/APRNReport
. Accessed April 25, 2012.

2. Dumoulin C, Hay-Smith J. Pelvic floor muscle training versus no
treatment, or inactive control treatments, for urinary incontinence
in women. Cochrane Database Syst Rev. 2010;(1):CD005654.

3. McLoughlin G, McAuley C. Pelvic floor muscle training versus
no treatment, or inactive control treatments, for urinary incon-
tinence in women. Clin Nurse Spec. 2010;25(5):226Y227.

4. NC State University Writing and Speaking Tutorial Service Tu-
tors. 2006. http://www.ncsu.edu/tutorial_center/writespeak/
PDF%20Handouts/synthesis%20matrix . Accessed April 19,
2012.

5. Mason L, Roe B, Wong H, Davies J, Bamber J. The role of ante-
natal pelvic floor muscle exercises in prevention of postpar-
tum stress incontinence: a randomised controlled trial. J Clin
Nurs. 2010;19(19/20):2777Y2786.

6. Dias LAR, Driusso P, Aita DLCC, Quintana SM, B+ K, Ferreira
CHJ. Effect of pelvic floor muscle training on labour and new-
born outcomes: a randomized controlled trial. Braz J Phys Ther.
2011;15(6):487Y493.

7. Batista RL, Franco MM, Naldoni LMV, Duarte G, Oliveira AS,
Ferreira CHJ. Biofeedback and the electromyographic activity
of pelvic floor muscles in pregnant women. Braz J Phys Ther.
2011;15(5):386Y392.

8. Brunenberg DE, Joore MA, Veraart CP, Berghmans BC, van der
Vaart CH, Severens JL. Economic evaluation of duloxetine for
the treatment of women with stress urinary incontinence: a
Markov model comparing pharmacotherapy with pelvic floor
muscle training. Clin Ther. 2006;28(4):604Y618.

9. BL K. Urinary incontinence, pelvic floor dysfunction, exercise
and sport. Sports Med. 2004;34(7):451Y464.

10. Chen S. The development and testing of the pelvic floor mus-
cle exercise self-efficacy scale. J Nurs Res. 2004;12(4):257Y265.

11. Cook TD, Campbell DT. Quasi-Experimentation: Design and
Analysis for Field Settings. Chicago, IL: Rand McNally; 1979.

12. Klopper R, Lubbe S, Rugbeer H. The matrix method of litera-
ture review. Alternation. 2007;14(1):262Y276.

13. Borello-France DF, Zyczynski HM, Downey PA, Rause CR,
Wister JA. Effect of pelvic-floor muscle exercise position on con-
tinence and quality-of-life outcomes in women with stress uri-
nary incontinence. Phys Ther. 2006;86(7):974Y986.

14. Garrard J. Health Sciences Literature Review Made Easy: The
Matrix Method. Gaithersburg, MD: Aspen Publishers; 2004.

15. Novak JD, CaDas AJ. The Theory Underlying Concept Maps and
How to Construct Them; Technical Report IHMC CmapTools
2006-01 Rev 01-2008, Florida Institute for Human and Machine
Cognition, 2008. Available at: http://cmap.ihmc.us/Publications/
ResearchPapers/TheoryUnderlyingConceptMaps . Accessed
April 19, 2012.

16. Alias M, Suradi Z. Concept mapping: a tool for creating a litera-
ture review. In: CaDas AJ, Reiska P, )hlberg M, Novak JD, eds.
Concept Mapping: Connecting Educators: The Third Interna-

tional Conference on Concept Mapping. Tallinn & Helsinki,
Finland; 2008.

17. Carnot MJ. Using concept maps to organize information for
large scale literature reviews and technical reports: two case
studies. In: CaDas AJ, Novak JD, eds. Concept Maps: Theory,
Methodology, Technology Proceedings of the Second Interna-

tional Conference on Concept Mapping. San Jos2, Costa Rica;
2006.

18. Rowley J, Slack F. Conducting a literature review. Manage Res
News. 2004;27(4):31Y39.

19. Steinerová J. Seeking relevance in academic information use.
Information Research. 2008;13(4), paper 380. http://InformationR
.net/ir/13-4/paper380.html. Accessed April 19, 2012.

FIGURE 4. Concept mapVpelvic floor muscle training.
Abbreviations: PFMT, pelvic floor muscle training; SI, stress
incontinence; UI, urinary incontinence; UrI, urge incontinence.

Clinical Nurse SpecialistA www.cns-journal.com 249

By Susan B. Stillwell, DNP, RN, CNE,
Ellen Fineout-Overholt, PhD, RN,

FNAP, FAAN, Bernadette Mazurek
Melnyk, PhD, RN, CPNP/PMHNP,

FNAP, FAAN, and Kathleen M.
Williamson, PhD, RN

In the previous article in this series, our hypothetical nurse, Rebecca R., with the help
of one of her hospital’s expert
evidence-based practice (EBP)
mentors, Carlos A., learned Step
1 of the EBP process—how to
formulate a clinical question.
The impetus behind her desire
to develop her question, as you
may re call in our case scenario,
was that Rebecca’s nurse man-
ager asked her to search for more
evidence to support her idea of
using a rapid response team to
decrease rates of in-hospital car-
diac arrests and unplanned ICU
admissions—both of which were
on the rise on Rebecca’s medical–
surgical unit. She learned of the
idea of a rapid response team
from a study she read on the sub-
ject in Critical Care Medicine.1

Here is the clinical question
Rebecca formulated: “In hospital-
ized adults (P), how does a rapid
response team (I) compared with
no rapid response team (C) affect
the number of cardiac arrests (O)
and unplanned admissions to the
ICU (O) during a three-month
period (T)? Her question, called
a PICOT question, contains

the following elements: patient
population (P), intervention of
interest (I), comparison interven-
tion of interest (C), outcome(s)
of interest (O), and time it takes
for the intervention to achieve
the outcome(s) (T). (To review
PICOT questions and how to
formulate them, see “Asking
the Clinical Question: A Key
Step in Evidence-Based Practice,”
March.)

This month Rebecca begins
Step 2 of the EBP process, search
ing for the evidence. For an over-
view of this step, see How to
Search for Evidence to Answer
the Clinical Question.

THE BEST EVIDENCE TO ANSWER THE
CLINICAL QUESTION
In their next meeting, Carlos
and Rebecca discuss what type
of evidence will best answer her
clinical question. Carlos explains
that knowing the type of PICOT
question you’re asking (for
example, is it an intervention,
etiology, diagnosis, prognosis, or
meaning question?) will help you
determine the best type of study
design to search for. Rebecca’s
PICOT question is an interven-
tion question because it compares
two possible interventions—a
rapid response team versus no
rapid response team.

ajn@wolterskluwer.com AJN ▼ May 2010 ▼ Vol. 110, No. 5 41

Searching for the Evidence
Strategies to help you conduct a successful search.

This is the fourth article in a series from the Arizona State University College of Nursing and Health Innovation’s Center
for the Advancement of Evidence-Based Practice. Evidence-based practice (EBP) is a problem-solving approach to the
delivery of health care that integrates the best evidence from studies and patient care data with clinician expertise and
patient preferences and values. When delivered in a context of caring and in a supportive organizational culture, the
highest quality of care and best patient outcomes can be achieved.

The purpose of this series is to give nurses the knowledge and skills they need to implement EBP consistently, one
step at a time. Articles will appear every two months to allow you time to incorporate information as you work toward
implementing EBP at your institution. Also, we’ve scheduled “Chat with the Authors” calls every few months to provide
a direct line to the experts to help you resolve questions. See details below.

Need Help with Evidence-Based Practice? Chat with
the Authors on May 5!

On May 5 at 1 pm EDT, join the “Chat with the Authors” call. It’s your chance to get personal consultation from the
experts! Dial-in early! U.S. and Canada, dial 1-800-947-5134
(International, dial 001-574-941-6964). When prompted, enter
code 121028#.

Go to www.ajnonline.com and click on “Podcasts” and then
on “Conversations” to listen to our interview with Susan B. Stillwell
and Ellen Fineout-Overholt.

42 AJN ▼ May 2010 ▼ Vol. 110, No. 5 ajnonline.com

Determine the level of evi-
dence. Research evidence, also
called external evidence, can be
viewed from a hierarchical per-
spective. The best external evi-
dence (that which provides the
most reliable information) is at
the top of the list and the least
reliable is at the bottom (see Hi
erarchy of Evidence for Inter
vention Studies2). The level and
quality of the evidence are impor-
tant to clinicians because they
give them the confidence they
need to make clinical decisions.
The research methodology that
provides the best evidence will
differ depending on the type of
clinical question asked. To answer
a question that includes an in-
tervention, such as Rebecca’s
question, a systematic review of

randomized, controlled trials or a
metaanalysis in which studies are
compared using statistical analy-
sis is the best study design.2-5 When
well designed and executed, these
studies provide the strongest evi-
dence, and therefore the most
confidence for clinical decision
making.

“What happens when there
isn’t a metaanalysis or systematic
review available?” Rebecca asks.
Carlos replies that the next-best
evidence would be Level II evi-
dence, the findings of a random-
ized, controlled trial. Carlos
reminds Rebecca that when de-

Cumulative Index to Nursing
and Allied Health Literature

The CDSR and DARE databases
contain systematic reviews and
metaanalyses of randomized,
controlled trials. The reviews
conducted by the Cochrane Col-
laboration are contained in the
CDSR, and abstracts of sys-
tematic reviews not conducted
by Cochrane are indexed in the
DARE. Cochrane reviews are
considered to have the strongest
level of evidence for intervention
questions because they have the
best study designs and are gener-
ally the most rigorous.

To find other types of evidence,
databases other than CDSR and
DARE must be searched. Because
the intervention—rapid response
team—is a multidisciplinary, in-
terprofessional initiative, evidence
to answer Rebecca’s question
may be found in medical as well
as in nursing and allied health
journals. Therefore, the PubMed
database, which contains medical
and life sciences literature, and
the CINAHL database, which
contains nursing and allied health
literature, should be searched.
Abstracts can be reviewed and
accessed free of charge in the
Cochrane Library and PubMed
databases (although a fee may be
required to obtain electronic cop-
ies of reviews or articles), but a
subscription is required to access
CINAHL.

SEARCHING STRATEGIES
Now that Rebecca and Carlos
have decided what databases to
search, they need to select the
keywords they’ll use to begin
their search.

Choose keywords from the
PICOT question. Rebecca and
Carlos identify the following
keywords from her PICOT ques-
tion: hospitalized adults, rapid
response team, cardiac arrests,
and ICU admissions. Lynne

ciding whether to use evidence
to support a practice change, it’s
important to consider both the
level and quality of the evidence
as well as the feasibility of imple-
menting the intervention.

WHERE TO FIND THE EVIDENCE
Rebecca and Carlos set up an
appointment with Lynne Z., the
hospital librarian, to learn how
to begin searching for the evi-
dence. Lynne tells Rebecca and
Carlos that no matter what type
of question is being asked, it’s wise
to search more than one database.
Because databases index different
journals, searching several data-
bases will reduce the possibility of
missing relevant literature.

Select relevant databases to
search. To find evidence to an-

swer Rebecca’s PICOT question,
Lynne recommends searching the
following databases:
• the Cochrane Database of

Systematic Reviews (CDSR)
and the Database of Abstracts
of Reviews of Effects (DARE),
which are found in the Co-
chrane Library and can be ac-
cessed through the Cochrane
Collaboration Web site (www.
cochrane.org)

• PubMed, which includes
MEDLINE (www.ncbi.nlm.
nih.gov/pubmed)

• CINAHL (www.ebscohost.
com/cinahl), an acronym for

How to Search for Evidence to Answer the Clinical Question
1. Identify the type of PICOT question.
2. Determine the level of evidence that best answers the question.
3. Select relevant databases to search (such as the CDSR, DARE, PubMed, CINAHL).
4. Use keywords from your PICOT question to search the databases.
5. Streamline your search with the following strategies:
• Use database controlled vocabulary (such as “MeSH terms”).
• Combine searches by using the Boolean connector “AND.”
• Limit the final search by selecting defining parameters (such as “humans” or

“English”).

ajn@wolterskluwer.com AJN ▼ May 2010 ▼ Vol. 110, No. 5 43

the same search conducted at
different times will likely produce
different numbers of articles.)

Rebecca and Carlos want to
combine their searches because
they’re interested in finding
articles that contain all of the
keywords (hospitalized adults
AND rapid response team AND
cardiac arrests AND ICU admis
sions). After they enter each key-
word into the selected database
and search it individually, they’ll
combine all the searches using
the Boolean connector “AND.”
There’s a chance, however, that
combining the searches may re-
sult in few or even no articles. For
example, the first time Rebecca
searched PubMed using its con-
trolled vocabulary for her PICOT
keywords, and then combined

the searches, the database came
up with only one article. She de-
cided to refocus her search, hoping
that including only the interven-
tion and outcomes keywords,
and not the patient population,
would produce articles relevant
to her clinical issue.

Place limits on the final com-
bined search to further narrow
the results. This strategy can
eliminate articles written in lan-
guages other than English or
those in which animals, and not
hu mans, are the subjects. Other
limits—such as age or sex of
subjects or type of article (such
as clinical trial, editorial, or
review)—are available; however,
placing too many limits on a
search may produce too few or
even no articles.

recommends that in cases when
a database has its own indexing
language, or controlled vocabu-
lary, the search be conducted
with these index terms. In this
way, the search will be the most
inclusive.

Use database controlled
vocabulary. For example, when
the keyword rapid response
team is entered into PubMed,
the PubMed database matches
it to the controlled vocabulary
term “Hospital Rapid Response
Team.” All articles that contain
the topic of hospital rapid re-
sponse teams can be found by
searching with this one index
term. Using controlled vocabu-
lary in a search saves time and
helps prevent the chance of miss-
ing evidence that could answer
the clinical question.

If the index terms matched
by the database aren’t relevant
to the searcher’s keyword, then
the keyword and its synonyms
should be used to search the data-
base. It’s helpful, though rare,
when a keyword and an index
term match perfectly. More
often, the searcher will need
to determine which of several
database index terms is closest in
meaning to the keyword.

Combine searches. Each key-
word in the PICOT question is
searched individually. However,
keyword searches can result in
a large number of articles. For
example, a CINAHL search of
cardiac arrest resulted in more
than 2,700 articles and a search
of rapid response team resulted in
100 articles. But combining the
searches using the Boolean con-
nector “AND” (for example, car
diac arrest AND rapid response
team) yielded a more manageable
12 articles that contained both
concepts and were more likely
to answer the clinical question.
(Note that databases index arti-
cles on a regular basis; therefore,

Hierarchy of Evidence for Intervention Studies2

Type of evidence Level of evidence Description

Systematic review or
metaanalysis

I A synthesis of evidence from all relevant random-
ized, controlled trials.

Randomized, con-
trolled trial

II An experiment in which subjects are randomized
to a treatment group or control group.

Controlled trial with-
out randomization

III An experiment in which subjects are nonrandomly
assigned to a treatment group or control group.

Case-control or
cohort study

IV Case-control study: a comparison of subjects with
a condition (case) with those who don’t have the
condition (control) to determine characteristics that
might predict the condition.

Cohort study: an observation of a group(s) (cohort[s])
to determine the development of an outcome(s)
such as a disease.

Systematic review of
qualitative or descrip-
tive studies

V A synthesis of evidence from qualitative or descrip-
tive studies to answer a clinical question.

Qualitative or de-
scriptive study

VI Qualitative study: gathers data on human behavior
to understand why and how decisions are made.

Descriptive study: provides background information
on the what, where, and when of a topic of
interest.

Opinion or con-
sensus

VII Authoritative opinion of expert committee.

44 AJN ▼ May 2010 ▼ Vol. 110, No. 5 ajnonline.com

may yield additional useful articles.
From the results page, Rebecca

enters rapid response team in the
search field and clicks “Search.”
This search produces over 300
articles (see Figure 6); however,
many of them still don’t appear
to be relevant to the clinical ques-
tion. Lynne reassures Rebecca
that eventually combining her
searches will help weed out the
irrelevant articles. (Because this
search produced so many more
articles than her MeSH term
search, which captured only the
most recent articles, Lynne sug-
gests that when Rebecca com-
bines her searches, she use the
results of her keyword rapid
response team search, not her
“Hospital Rapid Response Team”
search.

Rebecca continues to use the
MeSH database to search her
two remaining keywords. For
each one, she starts back on the
PubMed home page (click on the
PubMed.gov logo on any results
page to get to the home page).

Again, she enters cardiac
arrest on the MeSH database
screen. Of the three MeSH terms
provided she selects “heart
arrest,” which yields over 25,000
articles. Since the keyword ICU
admissions produces no MeSH
terms, Lynne advises Rebecca to
search with the keyword inten
sive care units, which matches
perfectly with the MeSH term
“Intensive Care Units” and
yields more than 40,000 articles.
After searching her keyword
and appropriate MeSH terms,
Rebecca has a total of more than
60,000 articles.

Lynne reassures Rebecca that
she won’t need to read all 60,000
articles. She explains that the next
step, combining the searches,
will eliminate extraneous articles
and focus on the search results
specific to the clinical question.
Combining the searches by using

the Boolean connector “AND”
will produce a list of articles that
contain all three keywords Re-
becca searched.

To combine her searches,
Rebecca selects the “Advanced
Search” tab at the top of any
results page. Each of her searches
now appears on the Advanced
Search page in the “Search
History” box. Lynne reminds
Rebecca to clear the search field
at the top of the page of any key-
words from past searches before
combining the final group of
searches.

Rebecca clicks on the number
assigned to her rapid response
team keyword search and selects
AND from the pull-down “Op-
tions” menu. Lynne shows her
that the number assigned to her
keyword search now appears in
the search field at the top of the
page. Rebecca continues to select
her individual searches and, one
by one, their corresponding num-
bers appear in the field above (see
Figure 7). To run the combined
searches and view the results, Re-
becca selects the “Search” tab.

Her combined search pro-
duces 11 articles (see Figure 8), a
much more manageable number
to review for relevancy to the
clinical question than the more
than 60,000 articles produced by
the individual keyword and con-
trolled vocabulary searches.

Rebecca asks Lynne if she can
request the three free full-text
articles (see “Free Full Text (3)”
under “Filter your results” on the
upper right of the results page;
Figure 8). Lynne informs her that
she can ap ply any number of lim-
its to her search, including “Links
to free full text.” However, the
more limits applied, the narrower
the search, and evidence to an-
swer the clinical question may be
missed.

Lynne shows Rebecca where
“Limits” can be found on the

CONDUCTING THE SEARCH
Rebecca begins to search the
PubMed database for the evidence
to answer her PICOT question.
She and Carlos will be search-
ing the keywords rapid response
team, the intervention of inter-
est, and cardiac arrests and ICU
admissions, the outcomes of
interest. To follow along, access
the PubMed home page at www.
ncbi.nlm.nih.gov/pubmed. (Note
that because new articles are
added to the database regularly,
your search results may not match
those described here.)

Rebecca starts by using
PubMed’s Medical Subject Head-
ing (MeSH) database to search
for the intervention keyword,
rapid response team. From
the PubMed home page, she
clicks on “MeSH Database”
(see Figure 1). On the MeSH
database screen, she types rapid
response team in the search field
and clicks “Go” (see Figure 2).
Rapid response team is a direct
match to the one MeSH term
provided—“Hospital Rapid
Response Team” (see Figure 3).
Rebecca selects this term by click-
ing the box next to it and then
selects “Search Box with AND”
from the pull-down menu. “‘Hos-
pital Rapid Response Team’
[Mesh]” appears in the search
box on the next screen (see Fig-
ure 4); Rebecca clicks on “Search
PubMed.” Her search is per-
formed and results in 19 articles
(see Figure 5). She notes that most
but not all articles appear to be
relevant to the clinical question,
and that they date back only to
2009 because the MeSH term
“Hospital Rapid Response Team”
was recently introduced.

Before Rebecca continues with
her MeSH database searches,
Lynne suggests that she use rapid
response team in a separate search
because the search will be broader
than a MeSH term search and

ajn@wolterskluwer.com AJN ▼ May 2010 ▼ Vol. 110, No. 5 45

Figure 1. Select “MeSH Database” on the PubMed home page.

Figure 2. Type rapid response team in
the search field and click “Go.”

Figure 4. Click on “Search PubMed.”

Figure 3. Select the
MeSH term “Hospital
Rapid Response Team,”
then select “Search Box
with AND” from the
pull-down menu.

Figure 5. The “Hospital Rapid Response
Team” search yields 19 articles.

46 AJN ▼ May 2010 ▼ Vol. 110, No. 5 ajnonline.com

top of the Advanced Search page
(Figure 7). She suggests that Re-
becca consider limiting the ages
of her population to further re-
duce her results. If she eliminates
the pediatric population, for
example, the number of articles
produced by her search should
decrease. But Rebecca thinks that
any articles that include children
may be of interest to the nurses
on the pediatric unit, so she de-
cides to limit her search to only
“Humans” and “English” (Fig-
ure 9). Applying these limits to
Rebecca’s final combined search
reduces the re sults from 11 ar-
ticles to 10.

Rebecca asks Lynne if any of
the articles retrieved in the search
are metaanalyses, which she re-
members is the best study design
to answer her clinical question.
Lynne responds that a quick
way to find out is by going back

to the Limits page and selecting
“Meta-Analysis” (see Figure 9).
Although this didn’t produce
any results, limiting the search to
“Randomized Controlled Trial”
resulted in one article.

As Rebecca’s session in search-
ing PubMed concludes, Lynne
explains to Carlos and Rebecca
that searching is a skill that im-
proves with practice. Moreover,
each database may have its own
controlled vocabulary and limits.
In any search, Lynne emphasizes
the importance of
• searching at least two data-

bases
• searching one keyword at a

time
• using the database’s controlled

vocabulary when available
• combining the searches to yield

articles that are manageable in
number and relate specifically
to the PICOT question

• applying “Humans” and “Eng-
lish” limits to the final search
Rebecca is excited to practice

her searching skills to find the
answer to her clinical question.
She and Carlos set up a time
to search the Cochrane and
CINAHL databases. Carlos
reminds Rebecca that although
considering the level of evidence
when making a clinical decision
is important, it’s not the only fac-
tor. The decision should also be
based on the quality of the evi-
dence, the feasibility of imple-
menting a change in the hospital,
and a consideration of the patients’
values and preferences.

In the next article in this series,
to be published in the July issue
of AJN, Rebecca gathers all the
articles relevant to her PICOT
question and meets with Carlos
to learn how to critically appraise
the evidence. You’re invited to

Figure 6. Type rapid response team in
the search field and click “Search”; this
search results in more than 300 articles.

Figure 7. Combine the individual searches.

Practice Mentorship Program at Ar izona
State University in Phoenix, where Ellen
FineoutOverholt is clinical professor
and director of the Center for the Ad
vancement of EvidenceBased Practice,
Bernadette Mazurek Melnyk is dean and
distinguished foundation professor of
nursing, and Kathleen M. Williamson is
associate director of the Center for the
Advancement of EvidenceBased Prac
tice. Contact author: Susan B. Stillwell,
sstillwell@asu.edu.

REFERENCES
1. Dacey MJ, et al. The effect of a rapid

response team on major clin ical out-
come measures in a community hos-
pital. Crit Care Med 2007;35(9):
2076-82.

2. Melnyk BM, Fineout-Overholt E.
Making the case for evidence-based
practice. In: Melnyk BM, Fineout-
Overholt E, editors. Evidencebased
practice in nursing and healthcare: a
guide to best practice. 1st ed. Phila-
delphia: Lippincott Williams and
Wilkins; 2005. p. 3-24.

3. DiCenso A, et al. Introduction to
evidence-based nursing. In: DiCenso A,
et al., editors. Evidencebased nurs ing:
a guide to clinical practice. St. Louis:
Elsevier Mosby; 2005. p. 3-19.

4. Gibson F, Glenny A. Critical appraisal
of quantitative studies: is the quality
of the study good enough for you to
use the findings? In: Craig JV, Smyth
RL, editors. The evidencebased
practice manual for nurses. 2nd ed.
Edinburgh; New York: Churchill Liv-
ingstone Elsevier; 2007. p. 95-122.

5. Fineout-Overholt E, et al. Finding
relevant evidence. In: Melnyk BM,
Fineout-Overholt E, editors. Evidence
based practice in nursing and health
care: a guide to best practice. 1st ed.
Philadelphia: Lippincott Williams
and Wilkins; 2005. p. 39-69.

this meeting to learn, along with
Rebecca, how to select “keeper”
studies that, when synthesized,
will help determine if a practice
change should be implemented at
her hospital. ▼

Susan B. Stillwell is clinical associate
professor and program coordinator of
the Nurse Educator EvidenceBased

Solutions to Our “Practice Creating a PICOT Question”
Exercise
Did your questions come close to these?

Scenario 1: A meaning question.
How do family caregivers (P) with relatives receiving hospice care
(I) perceive the loss of their relative (O) during end of life (T)?

Scenario 2: An intervention or therapy question.
In patients with dementia who are agitated (P), how does baby
doll therapy (I) compared with risperidone (or antipsychotic drug
therapy) (C) affect behavior outbursts (O) within one month (T)?

ajn@wolterskluwer.com AJN ▼ May 2010 ▼ Vol. 110, No. 5 47

Figure 8. The final results.

Figure 9. Using limits to narrow the search.

Turn in your highest-quality paper
Get a qualified writer to help you with

“ Discussion Post ”

Get high-quality paper

NEW! AI matching with writer

Still stressed with your coursework?

Get quality coursework help from an expert!

Order now