}

PMC PubMed Central® Q

Generate clinical question

Generating clinical questions, the first step

of EBM, is the most difficult and the most
important step. Turning a specific question
raised into an answerable question is the es-
sential technique in EBM [10]. Since there
is plenty of information available around us,
it is rare to fail to solve a certain problem
because of lack of information. We fail to
solve a certain problem mostly because we
do not accurately recognize the necessity of
the problem, we lack the time to search rele-
vant information, or we fail to access exist-
ing information as a result of not being fa-
miliar with search techniques. In addition,
one of the most common reasons why an an-
swer to a question is not obtained is because
the question itself is unanswerable.
Therefore, it is essential to make a question
as answerable as possible for problem
solving.

Generally, a good question is 1) relevant

Back
specific, 2) distinctly communicated, 3)
to
clear in the objective and necessity af in- Top
quiry, and 4) one that will reduce
required to obtain the answer.
 PMC PubMed Central® Q

Generally, a good question is 1) relevant and

specific, 2) distinctly communicated, 3)
clear in the objective and necessity of in-
quiry, and 4) one that will reduce the time
required to obtain the answer.

To generate a good question for EBM, a

clinician should be 1) specific in recognition
of a problem and clarification of the clinical
topic, 2) problem-oriented, 3) patient-fo-
cused, 4) able to determine answers through
literature, and 5) able to consider various as-
pects such as patient selection and subjects
to compare.

Considering these points, it is recommended

to specify a clinical question for EBM as
much as possible by applying the standard
of PICO, where 'P' denotes population’ or
‘patients’, 'T' 'intervention' or 'exposure', 'C'
‘comparison’, and 'O' ‘outcome’

After generating a clinical question, the cat-

egory of the question should be verified in
order to understand the type of information
required by the question. Verification of the
category helps to decide on the ¢
strategy for a literature survey ai
 }
PMC PubMed Central® Q

After generating a clinical question, the cat-

egory of the question should be verified in
order to understand the type of information
required by the question. Verification of the
category helps to decide on the appropriate
strategy for a literature survey and enables
clarification of the component that is not
clearly known and determination of the re-
search design to be considered for the cate-
gory. Common categories of questions are
diagnosis, prognosis, therapy, and risk (fac-
tors), and the type of research for answering
the question is dependent on the category. If
a clinical question belongs to the category
of diagnosis, the optimal research design to
answer the question is not a randomized
controlled trial but a cross-sectional study or
a case-control study. If a clinical question
belongs to the category of prognosis, a co-
hort study is effective because a follow-up
study of patients who have been evaluated
from the early stage of a disease is neces-
sary. If the category is therapy, a random-
ized controlled study or a systematic litera-
ture review of randomized controlled studies
is usually needed. If a clinical question be-
longs to the category of risk (factors), a ran-
domized controlled study, a coh( 5 Feedbac k
 PMC PubMed Central® Q

longs to the category of risk (factors), a ran-

domized controlled study, a cohort study, or
a case-control study is necessary.

Depending on the type of information, study

design is added to PICO to constitute PICO-
SD, which clearly describes the distinct es-
tablishment of a study question.

Table 1 shows an example of PICO-SD

specified for a question established by the
Propofol Task Force Team of the Korean
Society of Anesthesiologists according to
the "Clinical Guidelines of Propofol
Sedation for Non-Anesthesiologists." The
clinical question is, "Can combination ther-
apy with propofol and another sedative
make the risk of adverse effects lower than
that of propofol monotherapy in patients un-
dergoing sedation therapy?"

Table 1

Examples of PICO-SD Selection

PICO Description Example |

P: Patients Information Pat B Feedback

or on what
 PMC PubMed Central® Q

Find the evidence

The conditions for the evidence acquired in

EBM are that the evidence should be attain-
able, obtained externally from research or
from an expert, up-to-date, timely, of high
quality, applicable to individual patients,
and appropriately the best [11].

Identify resources There are various re-

sources available to use to find evidence.
These resources include individual experi-
ence; intuition or rationale; peers' view-
points; publications such as books, reports,
and journals; electronic databases; and the
assistance of expert librarians. It should be
noted that individual experience, intuition,
rationale, or peers’ viewpoints may provide
not information regarding the methods or
practices that should be performed or ap-
plied but rather information regarding either
current methods or practices or those per-
formed or applied in the past. In addition
information derived from these sources nn
Back
not be based on evidence or may not have
to
been verified, or the information may be Top
out-of-date, rather than the most. 1n-to-dare
: . NE (— |,
information. However, clinicians |B Feedback
 PMC PubMed Central® Q
EE a i as

formed or applied in the past. In addition,

information derived from these sources may
not be based on evidence or may not have
been verified, or the information may be
out-of-date, rather than the most up-to-date
information. However, clinicians or their
peers who are experts in their specialized
fields may have information which may not
be found in databases or books, or may be
aware of some important journals that are
not currently included in databases.
Therefore, individual experience and peers’
viewpoints are also very important sources
to utilize in finding the evidence.

Literature survey database International lit-

erature survey databases include
MEDLINE, EMBASE, Cochrane Central
Register of Controlled Trials (CENTRAL),
LILACS, CINAHL, PsycINFO, Google, and
Web of Science. Literature survey databases
available in Korea include KoreaMed,
KMbase, KISS, and RISS.

Beginning with the U.S. National Library of

Medicine (NLM) in 1966, MEDLINE,
which is the initial representative 2
information databases, includes [FB Feedback
 pr =.
PMC PubMed Central® Q

Medicine (NLM) in 1966, MEDLINE,

which is the initial representative of medical
information databases, includes about 4,000
international journals at present. MEDLINE
includes the free PubMed database as well
as the pay databases of Ovid (Ovid
Technologies, Inc.) and WinSPIRS
(SilverPlatter Information) provided by
companies with special search strategies.

It is not that respective databases include

different journals, but a certain journal may
not be included in a certain type of database
and may be included in multiple types of
databases simultaneously. It is known that
the scope of data and the scope of incorpo-
ration of MEDLINE and EMBASE are very
different depending on the topic, but it has
also been reported that about 30% of jour-
nals are redundant with respect to a certain
topic [12]. Therefore, in order not to miss a
piece of literature that should be found, a lit-
erature survey should be performed with the
inclusion of all databases available.
However, it may be virtually impossible to
search all databases since too m
and time
.
are required.
.
Therefore |BBE) Feedback
cocdback
 PMC PubMed Central® Q

~Prece OL nerdaiurc ulat srnouia pe ound, « It-

erature survey should be performed with the

inclusion of all databases available.
However, it may be virtually impossible to
search all databases since too much effort
and time are required. Therefore, the U.S.
NLM provides the scope for a literature sur-
vey, which is the COSI (Core, Standard,
Ideal) model [13] (Fig. 1). The term "core"
means the essence of a literature survey,
which is the minimal database required to
find the best results rapidly and simply.
Therefore, the "core" database is what must
be searched. The group of "core" databases
includes MEDLINE, EMBASE, and
CENTRAL, while it also includes Korean
databases such as KoreaMed, KMbase,
KISS, and NDSL. The term "standard," rep-
resenting the standard scope of a literature
survey, includes a manual search of core
journals and the search of databases that are
not "core" (Web of Science, DARE,
CINAHL, and PsycINFO). The "ideal" part
includes conference proceedings, gray liter-
ature, unpublished articles, and clinical tri-
als currently in process [13].

PubMed, Embase
cochrane CENTRAL
 PMC PubMed Central® Q

Some think that search of the literature out-

side the scope of the "core" with the goal of
increasing the sensitivity may reduce publi-
cation bias, but others think that the overall
bias may be enhanced since the studies out-
side the scope of the "core" have not under-
gone peer review,

Literature survey strategies While there are

countless pieces of information around us,
the effort and time we may put into address-
ing any question are limited. Therefore, it is
necessary to understand various survey
strategies and establish an appropriate sur-
vey strategy not only to identify as many
pieces of information as possible but also to
rapidly detect useful information, and at the
same time exclude baseless or inappropriate
information.

Survey strategies are classified as strategies

to increase sensitivity and strategies to in-
crease specificity. Sensitivity is defined a
the possibility of identifying relevant stuc ™
ies, searching for all the articles relevant * ack
the topic without missing one. A survey 1
with high sensitivity is a comprehensive ¢ Top
vey that necessarily includes irre [Bl Feedback
 PMC PubMed Central® Q

Survey strategies are classified as strategies

to increase sensitivity and strategies to in-
crease specificity. Sensitivity is defined as
the possibility of identifying relevant stud-
ies, searching for all the articles relevant to
the topic without missing one. A survey
with high sensitivity is a comprehensive sur-
vey that necessarily includes irrelevant arti-
cles. Specificity is defined as the possibility
of excluding irrelevant studies, and a strat-
egy to rule out articles that are not relevant
to the topic. A strategy to increase sensitiv-
ity is aimed at incorporating important stud-
ies on a certain topic, and a survey focused
on a research topic is one example.
Harmonizing sensitivity and specificity is
required in a survey strategy. However, a
survey strategy to increase sensitivity is
used for literature surveys for research, sys-
tematic literature reviews, and the develop-
ment of clinical practice guidelines, whereas
a survey strategy to increase specificity is
chosen for literature surveys for knowledge
or EBM [14].

A manual survey is also necessary in litera-

ture surveys in addition to searc
databases. In a manual survey, tl
 PMC PubMed Central® Q

A manual survey is also necessary in litera-

ture surveys in addition to searches using
databases. In a manual survey, the refer-
ences cited by the literature retrieved in the
database survey may be reviewed, or the
Science Citation Index may be used to
search relevant literature. Since an impor-
tant journal in the current field may not be
included in the database to be surveyed, an
expert in the field may be consulted. If an
important journal is not included in the
database, a manual survey should be per-
formed with that journal.

Additionally, surveys may be performed

with gray literature, which means literature
that has not undergone peer review. Gray lit-
erature may include reports and clinical trial
registries. Databases that specialize in gray
literature include Grey, NTS, and
PsycEXTRA [13].

Critical appraisal AN
Back
Once the literature has been surveyed, ap to
Top
propriate literature should be selected anc
 PMC PubMed Central® Q

Critical appraisal

Once the literature has been surveyed, ap-

propriate literature should be selected and
the quality and usefulness of the literature
should be appraised. The quality of a study
is appraised by considering the validity, reli-
ability, and clinical importance of the study.
The level of evidence of a piece of literature
is determined by the quality appraisal,
which in turn affects the strength of
recommendation.

Selection of Literature Since not all the

searched literature is relevant to a particular
study question, and many pieces of irrele-
vant literature are also retrieved by literature
survey, the literature relevant to the study
question is chosen by applying inclusion and
exclusion criteria. Selection of literature
goes through many steps. First, redundancy
of the literature retrieved from various data-
bases is checked. A bibliographical informa-
tion management software program such as
EndNote or Reference Manager is used to
check the redundancy. However, since such
bibliographical information ma
software programs check the red
2 Feedback
 PMC PubMed Central® Q

EndNote or Reference Manager is used to

check the redundancy. However, since such
bibliographical information management
software programs check the redundancy
mechanically, pieces of literatures that are
actually redundant may be considered differ-
ent. Therefore, the redundancy should be
verified manually. A pilot test may be per-
formed before starting the selection and ex-
clusion of the main body of literature selec-
tion to reduce the discrepancy between re-
searchers regarding inclusion and exclusion
as well as trial-and-error.

Objectivity and transparency should be

maintained in the process of literature selec-
tion and exclusion so that the bias of indi-
vidual appraisers may not intervene. The
process of literature selection and exclusion
should be carried out independently by at
least two appraisers. In case of differing
opinions on the part of appraisers, a conflict
resolution strategy should be prepared in ad-
vance, including discussion between ap-
praisers or intervention of a third party or an
expert committee. In addition, reproducibil-
ity should be secured in the process of liter-
ature selection [15]. IB Feedback
 PMC PubMed Central® Q

There is not a standardized format for the

literature selection and exclusion process.
However, generally, a primary selection pro-
cedure is carried out only with the titles and
abstracts, followed by a second selection
procedure to search the full text. When it is
difficult to make decisions using only the ti-
tles and abstracts in the primary selection
procedure, the full text should be searched
to decide whether to include or exclude a
piece of literature. If ambiguous, the litera-
ture should not be arbitrarily excluded by an
appraiser. In the primary exclusion proce-
dure, only the reason of exclusion and the
number of pieces of excluded remain in the
flowchart. A study that has not been ex-
cluded by at least one of the appraisers in
the primary selection and exclusion proce-
dure should undergo the secondary selection
procedure. Selection is made in the sec-
ondary selection procedure on the basis of
the full text of pieces of literature. During
this procedure, reasons for excluding a ce
tain piece of literature should be providec
and the list of excluded pieces of literatur Back
should be presented. Top

Appraisal of quality of literature 3 Feedback

 PMC PubMed Central® Q

Appraisal of quality of literature After the

selection of literature, the quality of the se-
lected pieces of literature should be ap-
praised. Appraisal of the quality of literature
enables decisions to be made regarding
whether to accept the conclusions drawn by
the literature or not, conflicting results from
different pieces of literatures to be inter-
preted and judgments to be made about
them, and the need for additional studies to
be determined. Although there are various
methods of appraising the quality of litera-
ture according to study designs, not all of
them are frequently used for various rea-
sons, including convenience, validity, and
difficulties in the application of relevant
tools. Tools of appraisal are classified as
scoring system and checklist system.

Generally, in the scoring system, scores for

each appraisal item are assigned and the to-
tal score is calculated. However, the mean-
ing of the score is unclear and excessively
simplified, the result is dependent on the
type of tool, and the item regarding blinding
randomized allocation is not considered in
the scoring system. In the scoring system,
the Chalmers scale and the Jada |g pee
Feedba k
 PMC PubMed Central® Q

the scoring system. In the scoring system,

the Chalmers scale and the Jadad scale are
used. In the checklist system, the checklist
developed by the Scottish Intercollegiate
Guidelines Network (SIGN), the Risk of
Bias Assessment tool for Nonrandomized
Studies (RoBANS), and the Risk of Bias
tool developed by Cochrane's group are
used.

The methods of appraising the quality of

randomized controlled studies include the
Chalmers scale, the Jadad scale, the check-
list developed by SIGN, and the Risk of
Bias tool. The methods of appraising the
quality of nonrandomized controlled studies
include the Newcastle-Ottawa scale, the
Deeks criteria, MINORS, and RoBANS.

Table 2 shows the items included in the Risk

of Bias tool developed by Cochrane's group
to appraise the quality of randomized con-
trolled studies. These items are known to
have a significant effect on the results of a
study. While the tool has simplified the
items, it has also reduced the possibility of
subjective and arbitrary answers to the same
items as well as the possibility o
[2 dback
Feedba c
reliability depending on the app =
0 °3 nchbi.nlm.nih.gov/pme/art + (3)

PMC PubMed Central® Q

Table 2 shows the items included in the Risk

of Bias tool developed by Cochrane's group
to appraise the quality of randomized con-
trolled studies. These items are known to
have a significant effect on the results of a
study. While the tool has simplified the
items, it has also reduced the possibility of
subjective and arbitrary answers to the same
items as well as the possibility of lowered
reliability depending on the appraiser's un-
derstanding and skillfulness with respect to
the methodology. In addition, the Risk of
Bias tool is easy to use because it provides
specific descriptions and guidelines for each
item to select from one among "high risk of
bias, No," "low risk of bias, Yes," and "un-
certain risk of bias, unclear” depending on
the information included in the literature
[16].

MN
Table 2 Back
to
Risk of Bias Table Top

Item
 PMC PubMed Central® Q

Bias The quality of literature is represented

by the risk of bias. All measurement values
different from the true values are called er-
ror, and errors are classified as random er-
ror, which is inconsistent, and systematic er-
ror, which has some direction. Random er-
ror is generated regardless of the research
procedure or method, but systematic error,
also called bias, is generated in cases in
which there is a problem in the research pro-
cedure or method. There are various types
of bias (Table 3). A biased study may pro-
vide an erroneous conclusion or may overes-
timate or underestimate results. Therefore, a
study methodology should be strictly ap-
plied to minimize the risk of bias [17]. The
presence of bias itself may not be measured,
and only the risk of bias may be evaluated.
Therefore, the quality of literature may be
appraised by evaluating the risk of bias.
Risk of Bias is also the name of a tool for
appraising the quality of randomized con-
trolled studies.

Table 3

Kinds of Biases
 PMC PubMed Central® Q

Methods of interpreting actual analytical re-

sults in the presence of risk of bias include
the method of analyzing all the studies and
describing the risk of bias, the method of in-
cluding only the studies having a low risk of
bias in the analysis, and the method of pre-
senting various analytical results including
the analytical results from all studies as well
as the analytical results only from the stud-
ies having a low risk of bias in the analysis.

Summary of evidence Data analysis is the

step in which extracted data are analyzed,
summarized, and synthesized. Data synthe-
sis is divided into qualitative synthesis and
quantitative synthesis. When statistical
quantitative synthesis is impossible, only
qualitative synthesis is performed to de-
scribe and present respective results. When
statistical synthesis is possible, quantitative
synthesis is performed.

Systematic Review: Systematic literature re-

view refers to collecting all available studies
by using an objective, systematic, and repro-
ducible methodology to answer a specific
and clearly described study question:
searching and selecting studies [A Feedback
 PMC PubMed Central® Q

Systematic Review: Systematic literature re-

view refers to collecting all available studies
by using an objective, systematic, and repro-
ducible methodology to answer a specific
and clearly described study question;
searching and selecting studies by using a
clear and systematic method to obtain valid
and reliable results with minimal bias; and
presenting the results obtained from the se-
lected studies after appraising the validity of
the results and discussing and analyzing the
results.

The characteristics of systemaltic literature

review, in comparison with narrative review,
are 1) that the study selection criteria are
provided in advance and the objectives are
clearly established, 2) that a clear and repro-
ducible methodology is used, 3) that a sys-
tematic attempt is made to identify all stud-
ies satisfying the selection criteria, 4) that
the validity of the selected studies is mea-
sured, and 5) that the results and character-
istics of the selected studies are systemati-
cally presented and synthesized [18].

Meta-analysis: Meta-analysis refers to a sta-

tistical method of synthesizing p [Bl Feedback
FCC UUdaLh
 PMC PubMed Central® Q

Meta-analysis: Meta-analysis refers to a sta-

tistical method of synthesizing pooled esti-
mates by summarizing estimates from two
or more individual studies. In other words,
meta-analysis is a statistical method of
quantitatively calculating pooled estimates
by integrating results from studies and eval-
uating the effect and efficiency of the pooled
estimates [19]. Therefore, meta-analysis is
also called analysis of analyses. In general,
systematic literature review and meta-analy-
sis are performed simultaneously. However,
it is not necessary to perform a meta-analy-
sis when a systematic literature review is
performed: a meta-analysis may or may not
be performed. For example, if the character-
istics of studies are too heterogeneous,
quantitative synthesis may not be attempted.
In contrast, a meta-analysis, although it is
generally performed following a systematic
literature review, may be performed without
performing a systematic literature review.

The purpose of a meta-analysis is not jus oN

find out summary estimates. If there is a Back
consistent pattern among research results ©
. . . To
is important to determine the meaning. If P
there is not a consistent pattern, [|g Feedback
 }
PMC PubMed Central® Q

The purpose of a meta-analysis is not just to

find out summary estimates. If there is a
consistent pattern among research results, it
is important to determine the meaning. If
there is not a consistent pattern, it is impor-
tant to ascertain what makes the results in-
consistent. The characteristic of being in-
consistent is called heterogeneity. In meta-
analysis, heterogeneity refers to cases in
which the variation observed in the results
integrated from individual studies by a
meta-analysis is greater than the variation of
sampling, so the variation may not be attrib-
uted to chance. Heterogeneity may be
caused by the variety of clinical settings
among studies, the variation in methodolo-
gies, chance, and bias.

The presence and degree of heterogeneity

may be verified by a visual test using a plot
or by a statistical test. The most representa-
tive visual test method using a plot to verify
heterogeneity is to draw a forest plot and
verify whether there is an overlap between
the directionality of the therapeutic effect
values of individual studies and the confi-
dence interval. Other methods of using a
plot employ the L'Abbe plot an yl
 }
PMC PubMed Central® Q
¥ LLL y COYY LAW LEAW] LIA%W dW ALY GALL WY Wl Ete] od Un’ YY Wall

the directionality of the therapeutic effect

values of individual studies and the confi-
dence interval. Other methods of using a
plot employ the L'Abbe plot and the
Galbraith plot. Statistical tests for verifying
heterogeneity include the ¥? test (Q statis-
tics) [20] and Higgins' 12 statistic [21].

Bias in Meta-Analysis: Bias in meta-analy-

sis includes publication bias, language bias,
location bias, and time lag bias. The most
representative bias is publication bias,
which refers to the bias in meta-analysis
caused by the fact that studies showing sta-
tistically significant results are included
more often in meta-analysis because those
studies are more likely to be published than
studies that do not show statistically signifi-
cant results. Bias may cause overestimation
or underestimation of study results or lead
to an erroneous conclusion, resulting in a
wrong interpretation or difficulties in
interpretation.

The presence and degree of publication bias

may be verified by a visual test using a plot
or by a statistical test. The most mannecants
tive method of using a plot is to BR FeFeedback
 }
PMC PubMed Central® Q

The presence and degree of publication bias

may be verified by a visual test using a plot
or by a statistical test. The most representa-
tive method of using a plot is to employ a
funnel plot. In a funnel plot, the y-axis rep-
resents the sample size of a study and the x-
axis represents the effect size. Individual
studies are expressed as points in the scatter
diagram having the shape of an upside-
down funnel, since the top is narrow while
the bottom is wide. The methods based on
statistical tests include Begg and
Mazumdar's rank correlation test [22] and
Egger's test [23].

Levels of evidence Evidence refers to what

is proved by studies conducted according to
the best research methodology, and level of
evidence refers to the degree of confidence
in the effect of intervention on the basis of
the current evidence. Level of evidence is
expressed in various other terms such as
quality of evidence by the Grading of
Recommendations Assessment,
Development and Evaluation (GRADE),
level of uncertainty by the U.S. Preventive
Services Task Force (USPSTF),
eyidence by the Agency for Heacal Bea ck
° PMC PubMed Central® Q

A
FRE SLCLAVINNINFLES NAVAN LWSN I) EAE NEE gs Aaa

Development and Evaluation (GRADE),

level of uncertainty by the U.S. Preventive
Services Task Force (USPSTF), strength of
evidence by the Agency for Healthcare
Research and Quality (AHRQ), and quality
of a body of evidence by Cochrane.

Studies are conducted with various types of

designs, and the level of clinical evidence is
determined by the risk of various biases that
may be present in a particular type of de-
sign. For example, it is considered that a
randomized controlled, double-blind,
placebo-controlled trial conducted with a
homogeneous patient group and completely
followed up provides the least risk of bias
and the strongest evidence. On the contrary,
a case report or an expert opinion is consid-
ered to have a weak level of evidence be-
cause it has a high probability of bias.

Many other factors contribute to the deter-

mination of the level of evidence, including
quality of literature, quantity of evidence,
consistency of evidence, and directness of
evidence. Table 4 displays the level of evi-
dence system provided by SIGN ~~
the scales. This system provides [A Feedback
 PMC PubMed Central® Q

followed up provides the least risk of bias

and the strongest evidence. On the contrary,
a case report or an expert opinion is consid-
ered to have a weak level of evidence be-
cause it has a high probability of bias.

Many other factors contribute to the deter-

mination of the level of evidence, including
quality of literature, quantity of evidence,
consistency of evidence, and directness of
evidence. Table 4 displays the level of evi-
dence system provided by SIGN as one of
the scales. This system provides levels of
evidence on the basis of study design and
quality of literature. The studies located near
the top of the table are expected to have a
low risk of bias and a high level of evidence.
This system, widely used in the past, is not
used currently because it is known that the
system of grading the recommendations in
A, B, C, and D according to the level of evi-
dence is not appropriate. Therefore, SIGN
decided to accept the internationally reco
nized GRADE system in 2013. MN
Back
to
7
Table 4 is
SIGN Level of Evidence top.ars Fes
0 °3 nchbi.nlm.nih.gov/pme/art + (3)

PMC PubMed Central® Q

The strength of evidence may be determined

by the quantity of evidence, consistency of
evidence, and directness of evidence.
Quantity of evidence is determined by the
number of research articles on the topic, the
number of subjects, and the effect size.
Consistency of evidence is determined by
how similar the results of individual studies
are in terms of the direction and size of
effect.

Strength of recommendation The strength of

recommendation of an intervention refers to
the level of confidence in obtaining a de-
sired result by performing the intervention
according to the recommendation. The
strength of recommendation is determined
by considering the level of evidence and the
balance between the benefit and risk of the
intervention. The GRADE working group
prepared a system of strength of recommen-
dation by considering not only the quality of
medical studies but also the viewpoint af-
fecting the reliability of results. In this sys-
tem, the quality of evidence is apnrai
grading into four categories of " [=
 PMC PubMed Central® Q

intervention. The GRADE working group

prepared a system of strength of recommen-
dation by considering not only the quality of
medical studies but also the viewpoint af-
fecting the reliability of results. In this sys-
tem, the quality of evidence is appraised by
grading into four categories of "high,"
"moderate," "low," and "very low," accord-
ing to how an effect observed across the en-
tire body of evidence, not just in individual
research results, may be similar to the actual
effect. The "high" grade means that further
research 1s very unlikely to change the confi-
dence in the estimate of an effect. The
"moderate" grade means that further re-
search is likely to have an important impact
on the confidence in the estimate of an effect
and may change the estimate. The "low"
grade means that further research is very
likely to have an important impact on the
confidence in the estimate of an effect and is
likely to change the estimate. The "very
low" grade means that any estimate of effect
is very uncertain.

Initially, the "high" grade is assigned to a

randomized study, and the "low" grade is as-
signed to an observational stud
Co ’' [2 Feedback
is lowered if there is a risk of bic on
 PMC PubMed Central® Q
AT YY &r [S83 = ITED LIE CLL y TW ILLLIIOLLYY
WI Wal WL

iS very uncertain.

Initially, the "high" grade is assigned to a

randomized study, and the "low" grade is as-
signed to an observational study. The grade
is lowered if there is a risk of bias in the re-
search design, if the evidence lacks direct-
ness, if the accuracy of the research is low,
if there is unexplained heterogeneity, or if
there is a risk of publication bias. The level
of recommendation is elevated if the effect
size is great, if there is a low possibility of
confounders, or if there is a dose-response.
The grade of a randomized controlled study
is not elevated [24].

Apply the evidence

Following literature survey and evidence ap-

praisal, the results determined by evaluation
to be useful should be applied in actual clin-
ical settings. A complicated, well-conducted
study presented in a sophisticated article
will not be helpful if the results are not ap-
plied to patients. In addition, if new discov-
eries from studies are applied in clinical set-
tings too slowly or not applied ar
tential benefit that clinical studie
 }
PMC PubMed Central® Q

Apply the evidence

Following literature survey and evidence ap-

praisal, the results determined by evaluation
to be useful should be applied in actual clin-
ical settings. A complicated, well-conducted
study presented in a sophisticated article
will not be helpful if the results are not ap-
plied to patients. In addition, if new discov-
eries from studies are applied in clinical set-
tings too slowly or not applied at all, the po-
tential benefit that clinical studies may pro-
vide to clinical settings will be removed.
However, it seems that the application of ev-
idence is not receiving sufficient attention.
In fact, there are many studies in which data
analysis was conducted to improve the qual-
ity of medical service and for patient safety,
but no studies exist regarding how to apply
evidence. The application of evidence itself
is very difficult and requires changes in the
medical system, individual clinicians, and
ultimately, the entire culture of the medical
system.

Evidence obtained from EBM may be ap-

plied to various situations, but th-
tion neither always occurs in me
 }
PMC PubMed Central® Q

Evidence obtained from EBM may be ap-

plied to various situations, but that applica-
tion neither always occurs in medical ser-
vices nor results in change [25]. Even when
EBM is applied, the application is often in-
appropriate. According to one report, even
in the situations in which EBM was applied
for patient treatment, 18% of the doctors
changed the patient therapy when recom-
mendations were presented as the results of
a literature survey [26].

If the research has been conducted ade-

quately, EBM may be performed. However,
there are cases in which sufficient research
has not been conducted. In such cases, deci-
sions are made on the basis of evidence that
is not based on research (e.g., expert opinion
or scientific inference) [27]. In these cases,
the clinician, as a researcher, may need to
make efforts to produce evidence.

Since no patients are identical, they have

different values, preferences, expectation! gack
and circumstances. Patients are often en- 10
countered in a situation that is different fi Top
the one that has been searched a | 3 Feedback
 PMC PubMed Central® Q

Since no patients are identical, they have

different values, preferences, expectations,
and circumstances. Patients are often en-
countered in a situation that is different from
the one that has been searched and evalu-
ated. Therefore, applying the "ideal" evi-
dence that was retrieved and evaluated may
not be the best for the patient to be treated.
In this case, decisions should be made by
considering the amassed evidence as well as
the situation of the patient.

In addition, clinicians having different types

of training, experience, and specialties may
prefer different treatment methods.
Therefore, the retrieved evidence may con-
flict with the treatment method prioritized
by each clinician. Conflicts may occur in
such cases. In these cases, EBM may help
the patient make a decision about interven-
tion or treatment.

Evaluate

After the application of evidence, the infor-

mation, intervention, and EBM process are
evaluated. The evaluation is per!pm oo
"A Feedback
respect to the appropriateness of
 PMC PubMed Central® Q

Evaluate

After the application of evidence, the infor-

mation, intervention, and EBM process are
evaluated. The evaluation is performed with
respect to the appropriateness of the quan-
tity and quality of evidence in the informa-
tion, the difficulty of acquiring the evidence
and results, the cost of application, the re-
sponse and the degree of compliance on the
part of patients, the difficulty of application,
the clinical results, the effect of the actual
application, and the changes that the experi-
ence has produced in the thoughts and skills
of clinicians.

A feedback mechanism should be in place

for the knowledge acquired in the process of
actually applying the evidence, so that oth-
ers may perform the process well and the
EBM implementation strategy may be
improved.
Nn
Strategy for Rapid Decision- Golo
Making to
Top

The application of EBM may be

clinical decision should be made
 }
PMC PubMed Central® Q

Strategy for Rapid Decision- Go to: »

Making

The application of EBM may be difficult if a

clinical decision should be made rapidly.
Sacket and Strauss [28] tested whether it is
possible for clinicians, during their rounds,
to use the "evidence cart" containing various
types of evidence to search for and apply
relevant evidence. This study was conducted
due to the concern that actual EBM applica-
tion may be limited because time and effort
are required for a systematic literature ap-
praisal before decision-making. The study
showed that the "evidence cart" was often
applicable as rapidly as medical service may
be provided during team rounds, and that
the "evidence cart" could affect decisions
about diagnosis and treatment in 81% of
cases, among which 91% had successful
outcomes for patient treatment. Therefore, it
was stated that the "evidence cart" is helpful
for busy clinicians to perform rapid and a
N
propriate decision-making. Rapid decisio
Back
making may be helped if there is a protoc
to
such as practice guidelines for quick refer ! Top
ence or if the EBM library is eas?
ble. Therefore, efforts are conti 3 Feedsback
 PMC PubMed Central® Q

~ble. 'I'heretore, eiforts are continuously

made to prepare and implement practice
guidelines, while the portal sites of treat-
ment guidelines also provide practice guide-
lines (e.g., NGC [National Guideline
Clearinghouse, http://www.guideline.gov],
GIN [Guidelines International Network,
http://www.g-i-n.net]).

An EBM library enabling quick access and

search should be convenient to use, accept
up-to-date information, and include elec-
tronic databases. The EBM libraries satisfy-
ing these conditions include medical infor-
mation services such as Medscape and
HDCN and review services such as
Evidence-Based Medicine Review,
Cochrane Library, Best Evidence, and Up to
Date.

Limitations Go to: »

The limitations of EBM are that 1) logice®

and consistent scientific evidence may be 4
sufficient, 2) applying the retrieved evide gy
to the treatment of a specific patient is ofl to
difficult, 3) there is a barrier in applying Top
high-quality medical skills, 4) c [=] Feerbect
ten do not have the skills require.
 PMC PubMed Central® Q

Limitations Go to: »

The limitations of EBM are that 1) logical

and consistent scientific evidence may be in-
sufficient, 2) applying the retrieved evidence
to the treatment of a specific patient is often
difficult, 3) there is a barrier in applying
high-quality medical skills, 4) clinicians of-
ten do not have the skills required for litera-
ture survey or appraisal, 5) busy clinicians
have limited time to master and apply EBM,
and 6) the resources needed to find the evi-
dence are often inadequate in clinical set-
tings [2].

This article has explained the general out-

line and history of EBM, the steps and
methodologies of EBM, the strategy for
rapid decision-making, and the limitations
of EBM.

The term "EBM" is broadly used in the

preparation of practice guidelines and the
education and policy-making of EBM im N
plementation. Recently, the term has beer Back
widely used in the domains where eviden i
is emphasized on the level of the group om
the individual. In addition, EB
 PMC PubMed Central® Q

The term "EBM" is broadly used in the

preparation of practice guidelines and the
education and policy-making of EBM im-
plementation. Recently, the term has been
widely used in the domains where evidence
is emphasized on the level of the group or
the individual. In addition, EBM is continu-
ously undergoing advancements and revi-
sions. Although EBM has been developed in
many areas and applied to actual clinical
settings, it has not been perfectly applied to
medicine, a field in which experience is
highly regarded [29]. Moreover, many clini-
cians feel it is difficult to understand and
perform EBM.

EBM is like a double-edged sword to clini-

cians. While EBM may ensure that the diag-
nosis and therapy provided is based on sci-
entific evidence, it reduces the space in
which individual clinicians may perform di-
agnosis and therapy on the basis of their
clinical judgment. Just as a person who has
mastered how to use a knife in a wrong way
may become a robber but a person who has
mastered its use in a right way may be an
excellent medical doctor, a clinician w
masters EBM rightly and introci |B Feedback
 PMC PubMed Central® Q
TAA ITE ALF BEAN W MALS LLL WR Ws LY TY LIEW LW WAlLAIER

cians. While EBM may ensure that the diag-

nosis and therapy provided is based on sci-
entific evidence, it reduces the space in
which individual clinicians may perform di-
agnosis and therapy on the basis of their
clinical judgment. Just as a person who has
mastered how to use a knife in a wrong way
may become a robber but a person who has
mastered its use in a right way may be an
excellent medical doctor, a clinician who
masters EBM rightly and introduces it will
become an excellent clinician, while the
clinician may do harm to himself or herself
or to patients if he or she does not under-
stand and use EBM appropriately. In addi-
tion, since the global trend is toward the em-
phasis and application of EBM, clinicians
who do not apply it will confront challenges
from others. Therefore, just as an excellent
doctor masters the use of a surgeon's knife,
we need to master the interpretation and ap-
plication of EBM to harmonize our knowl-
edge and capability with the global ad-
vances in knowledge.

References Go to: »

1. Sackett DL. Evidence-based medi | : n