Research Report

Quantitative Analysis Research

Physical Therapy

Background and Purpose. Some physical therapists argue that research will Valma J Robertson
justrh and improve practice in the discipline. Whether these outcomes can be
realized is partly a function of the nature of the research, the research methods
used, and the extent to which clinical treatments are investigated.Subjects
and Metbods. This study analyzed aspects of a sample of 272 articles pub-
lished in Physical Therapy during 1954 through 1 9 3 . Each article was read
and categorized according to the method used and to whether it investigated
clinical treatments. Results. An increasing percentage of the articles sampled
described studies that used group designs, and most of the studies did not in-
vestigate clinical treatments. ConcCusfonand Dfsnrssfon TheseBndings a m
paradoxical given existing criticisms of group designs and eqctations in phys-
ical therapy of 1-esearch.More discussions and evaluations of current research
practices in physical therapy are needed. [Robertson VJ.A quantitative analysis
of research in Physical Therapy. Phys Ther. 1995;75:315327.1

Key Words: Physical therapy profession, Publications and audiovisual material,

Research, Research methods.

Research and interest in research have apy Inc in the United States). Journal Physical therapists appear to anticipate
increased in physical therapy over the editors and ex-editors have high- that research will benefit the profes-
past two decades. Many articles and lighted an increased publication of sion and practice of physical therapy.
books on how to do research have research in the profession.12 New Among the benefits expected from
been published.'-5 Physical therapy English-language journals, including research are scientific credibility1' and
databases have been developed (eg, the Journal of Orthopaedic and S p m the growth and development of the
Physiotherapy Index). Editorials and Physical Therapy and Physiotherapy profession.14J5Research is also ex-
journal policy statements about re- Theory and Practice, have appeared. pmed by some to validate current
search have been pubhhed in most At the same time, established physical practice16 and to satisfy increasing
physical therapy Articles therapy journalsAustralian Joumal demands for the accountability of
encouraging the formation of special of Physiotherapy, Physical Thmpy, professional gr0ups.~71~~ Various
interest groups on research and on Physiotherapy Canada, and Physio- sources expect treatments to be of
obtaining funding for research have t h e r a p have increased the volume high quality and cost effective: and
been published.lO,"In some countries, of research contributions they publish. research is expected to demonstrate
special funds for research in physical Together, these developments demon- that physical therapy interventions are
therapy have been instituted (eg, the strate the increasingly active interest in cost effeai~e~9~~0 and of hlgh quality.21
Physiotherapy Research Foundation in research within physical therapy over Research is also expected to demon-
Australia and in the United Kingdom, the past two decades. strate and improve the effectiveness of
and the Foundation for Physical Ther- physical therapy techniques, interven-
tions, and modalitie~.7.~~.~~.~3The em-
phasis on the benefits of research for
clinical practice reflects the centrality
VJ Robertson, PhD, BAppSdPhty), BA(Hons), is Senior Lecturer, School of Physiotherapy, La
Trobe University, Locked Bag 12, Carlton South, Victoria, Australia 3053.
of practice in physical therapy. An
important question is whether re-
7bis article was submittedjanuury ZG, 1994, and was accepted November 23, 1994.

Physical Therapy / Volume 75, Number 4 / April 1995 313 / 69

search is providing physical therapy s i g n ~ . %Further,
, ~ ~ when used to inves- does research in physical therapy
with the expected benefits. tigate the relatively small effects investigate clinical treatment? Does
typically obtained in clinical practice, research focusing on clinical treat-
Research outcomes are in large part a group designs (experimental and ments use methods different from
function of the suitability and produc- quasi) produce findings with low those used in nonclinical research?
tivity of the research method used. A power and a reduced probability of
research method most conspicuously correctly rejecting null hypotheses.52 Answers to these questions about
reconlrnended in physical therapy, published physical therapy research
both explicitly and hlplicitly, is group- The relevance of many findings from are not available, although some rele-
experimental designs. This is an ex- group designs for practitioners is ques- vant studies69J0have been conducted.
perimental method that compares tionable. The findings are based on These studies indicate that the relative
oulcomes for groups, rather than out- averaged group performance when frequency of studies using group-
comes for individuals or series of clinicians focus on indi~iduals.~~~+55 experimental designs ranges from
individuals (ie, single-subject experi- Another criticism suggests that the 10.7% in Physiotherapy Canada, Phys-
mental designs), following the applica- choice of problems investigated is ical Z%erapy, and Physiotherapr com-
tion of controlled interventions.l(~lR* constrained by amenability to statisti- bined (data reworked70) to 41% in
The method is also known as group- cal analysis48s5%ther than by clinical, Physiotherapy Canada.@ Another
companson experimend(p1'14)and theoretical, or practical significance study71showed that the relative fre-
randomized clinical trials.24-27 If and relevance. Other criticisms of quency of "experimental trials" in-
some requirements of the group- group-experimental designs concern creased from 19??over the period
experimental method are made less the ethical dilemmas raised by the 1980 to 1984 to 27% over the period
rigorous, it is then called a quasi- requirements of the research method. 1985 to 1989 in articles from the jour-
group-comparison design or a clin- Such ethical issues can arise in the nals Physiotherapy,Physical nerapy,
ical trial. Explicit support is appar- random allocation of subjects to treat- Physiotherapy Canada, Australian
ent in the positive advocacy of the ment and control g r o ~ p s ~ and
~ z 5in~ Journal of Physiotherapy,and South
investigatory powera~27,znand gen- the replacement of a therapeutic situa- African Journal of Physiotherapy.The
cia1 dcccptdbility of the group- tion responsive to the changing needs category definitions provided in that
expuhilental irlethtd.26 Inlplicit sup- of the patient by one in which treat- study do not make clear how clinical
port for group-experimental designs is ment is based on previously specified studies using group designs would be
apparent in the extent to which arti- experimental procedures.36 Some of classified and apparently exclude
cles and books aimed at the physical these problems may be obviated by methods such as single-subject experi-
therapy market focus on these de- using the less rigorous quasi-group- mental designs and series of case
signs. Some books and articles that comparison designs. Some of the studies. The definitions provided in
describe how to do research 1 2 4 28-33
I 3 practical problems inherent in clinical other stud1esM,7~~~3 do not differentiate
and how to evaluate resear~h.3~83~ research, such as the recruitment of a group-experimental designs from
focus almost exclusively on group- sufficient number of appropriate sub- quasi-group-comparisondesigns and
experimental designs. jects, however, still tend to remain for other data-based research methods.
users of group designs. The conclusions of some studies of
Published criticisms of group- research methods are limited by the
experimental designs suggest that the The popularity of group- lack of details of the source or the
method might not be optimal for gen- experimental designs in physical method of selection of the articles
eral use in the clinical ~ o n t e x t . l ~ ~ - ~therapy
~ research and the critiques exarninedM~69,~~ and by the reliance on
hnong other things, criticisms of of these designs in clinical research a convenience sample of
group-experimental designs have raise some questions. To what published over a period of relatively
referred to difficulties in obtaining extent does research in physical short duration (10 years maximum) in
suficiently large and appropriately therapy depend on group- a context in which the relative fre-
homogeneous samples of subjects experimental designs? Conversely, quency of the different types and
within the required time period,%-3"n to what extent are alternative re- topics of articles can vary markedly
providing and maintaining the re- search methods used? Possible from year to year.
quired experimental control within the alternative research methods dis-
clinical context,l.39-" in using statisti- cussed in physical therapy journals My study had three speclfic aims: (1)
cal procedures correctly and appropri- include the following: case studies to establish the frequency of use of
ately (eg, in medicine43and orthope- and series of ca~es,~,5~-59 single- group-experimental designs in a longi-
d i c ~ ,psychology
~" and sociology,*5 subject experimental de~igns,3~,60-63 tudinal sample of articles from the
and social worka), in interpreting ethnographic studies,64l65sequential physical therapy literature, (2) to ex-
findings based on Inferential statii- trials design,36,66 and surveys.67 amine the extent to which the same
ti~s,"~,4'-5land in validating findings Books intended for physical thera- sample of articles investigated clinical
and cumulating knowledge from re- pists discuss these and other re- treatments, and (3) to investigate the
search using group-experimental de- search methods.',68 To what extent extent to which clinical research was

Physical Therapy / Volume 75, Number 4 /April 1995

based on methods differing from those "professional education," and "neuro- designs, and quasi-group-comparison
used to examine nonclinical issues. physiology/neuroanatomy" were ex- designs.
cluded, the topics of direct relevance
Method to clinical practice and with the largest The second classification system was
numbers of articles were the knee, the developed to drstinguish articles deal-
Literature Sample back, and electrical stimulation. Given ing with treatment from those that did
the system of cross-referencing used not deal with treatment (Tab. 2). The
In this study, I analyzed a sample of by Physical Theram, each of these criteria were based on existing distinc-
272 articles published from 1754 to topics included articles on subjects as tions between aspects of clinical and
1 9 3 . The sample was derived from diverse as anatomy, equipment, and nonclinical practice and research
Physical i'Berapy, a physical therapy evaluations of treatments. Having two drawn w i t h the physical therapy and
journal described in its purpose state- anatomically defined areas and a treat- psychology literature^.^^^^^^^^ The start-
ment as "a scholarly, refereed journal ment modality broadened the basis for ing point was categories used by Hol-
that contributes to and documents the the study and optimized clinical rele- liday70 in her analysis of research
evolution and expansion of the scien- vance. The prominence of an electrical contributions made by physical thera-
fit and professional body of knowl- modality among the Merent manual pists. The common element in the
edge in physical therapy."74 and electrical treatment techniques categories Holliday called " c h c a l
and types of exercise indexed was surveys" and "clinical trials" was
Physical Therapy was for many years consistent with the findings of an "treatment in a clinical context."70The
the sole physical therapy journal p u b earlier survey of the treatment most term "in a clinical context" was too
lished monthly and indexed in Index commonly provided by physical then- general, however, to provide a suffi-
Medicus and the Science Citation p i ~ t sIf. ~a ~particular article was in- cient drstinction between clinical treat-
Index as well as in physical therapy dexed under two or more of these ment and nonclinical studies, as it did
indexes (eg, Physiotherapy Current three topic areas (ie, back, knee, or not require that the treatment was
Awareness Topic Search, Physiother- electrical stimulation), it was included used to effect a beneficial outcome for
apy Index). Physical irherapy also has in each area for analysis of the topic. a patient. A treatment could equally
a growing impact factor75and is cited well have been tested on asymptom-
more frequently within the profession Classification Systems atic subjects.
than any other physical therapy
Based on distinctions drawn by Gold-
j0urna1.7~3~ Two systems were used to categorize
i a m ~ n dclinical
,~ treatrnent studies
each of the 272 articles. The first clas-
were differentiated from nonclinical
Articles published in Ph.ysica1 fierapy sification system was based on the
studies by the requirement that they
prior to 1954 were not included in this type of method used in each article.
reported an aim of maintaining or
sample for three reasons. First, the The eight method categories used,
improving the function of patients
number of articles published per year shown in Table 1, were (1) group
receiving the treatment. Support for
prior to 1954 was small, especially experimental, (2) quasi-group compar-
this distinction is found in a guest
during World War 2. Second, by 1754, ison, (3) single-subject design, (4)
editorial in PhLysica17herapyentitled
the Journal had changed to its current practical, (5) descriptive, (6) survey,
"Reliability Discussion Required.""'
monthly publication rate (in 1747). (7) case study, and (8) correlational.
The editorial emphasized the impor-
Third, a four-decade period was ex-
tance of selecting a sample that repre-
pected to provide a suficiently long The classification of the method used
sents the population that the treatment
time frame to reliably indicate trends in each article was based on a system
being investigated is intended to affect
in the aspects of the articles investi- described by Ottenbacher and Short,79
and that using unimpaired volunteers
gated in this study. who surveyed 674 research articles
when clinically useful information is
published from 1970 to 1780 in the
being sought is inappropriate. That is,
The sample of 272 articles consisted of Occupational 7herapyJoumal of
subjects can only be said to be partici-
all 106 articles indexed in PF?ysical Research. These authors subsequently
pating in clintcal studies of treatment if
fierapy on the topic "knee," all 79 tested the reliability of their categoriza-
they represent at least some part of
articles indexed on the topic "back," tions and obtained an interobserver
the population that treatment is in-
and all 87 articles indexed on the agreement of .74 using a coefficient of
tended to benefit.
topic "electrical stimulation" between agreement. The wording of their
1754 and 1993. The topics were cho- seven-category definitions was modi- Category Assignment
sen by establishing which indexed fied in my study in an attempt to in-
topics included the largest numbers of crease the precision and the relevance I read each of the 272 articles in the
articles, excluding commentaries, to physical therapy. I also included an sample and assigned each article to
author responses to commentaries, eighth category called "single-subject one of the eight method categories.
and letters to the editor, published in design" to enable the differentiation of Following a second reading, I then
Pbwical7herapy from 1954 to 1 9 3 . single-subject experimental designs categorized each article as either clii-
When the topics "professional issues," from case studies, group-experimental cal treatment or nonclinical.

Physical Therapy /Volume 75, Numbe

-
Table 1 . Method Categories

Category

Group experimental
Description

In group-experimental articles, there was a comparison between an experimental or treatment group and a control or
nonexperimental group, or between two or more alternative experimentalor treatment groups, in which the subjects
were randomly assigned to each group. The article explicitly reported these features and presented a statistical
analysis of the results.
Quasi-group comparison In quasi-group-comparison articles, there were group-comparisons but there was no indication of the use of random
group assignment or nontreatment groups. Both dependent and independent variables were generally clearly
identified, and there was an attempt to manipulate the variables involved. A statistical analysis of the results was
included.
Single-subject design In single-subject design articles, there was use of a single-subject experimental design in which a baseline was
established, repeated measures of patient performance were made, and a reversal to baseline conditions was
reinstated. A graphic presentation of the data was included with or without an accompanying statistical analysis of the
data.
Practical In practical articles, there were reports of the development of new apparatuses; modifications to existing equipment; or
new assessment, measuring, or treatment techniques. The apparatuses, equipment, and techniques reported were
applicable in the clinical environment. Articles included in the practical category did not fit into any of the other
categories above.
Descriptive In descriptive articles, there was a description of a situation, a theory, a condition, a treatment regimen or technique, a
historical perspective, or a philosophical approach, or there was a discussion of anatomy, physiology, and
biomechanics. Review articles were inciuded in this category. Articles containing numerical data or statistical analyses,
or both, were excluded from this category.
Survey In survey articles, there was use of some type of measuring instrument or interview technique to gather information of
either an attitudinal or observational nature. Included were articles about patients, conditions, or treatments and
articles that established norms for different performances (eg, articles surveying the isometric or isokinetic strength of
the quadriceps femoris muscles in college students or evaluating the use of different pain scales for patients following
surgery). No attempt was made to control or manipulate variables of interest.
Case study In case-study articles, there were reports of an intensive investigation of one or more persons. These reports would
usually, although not always, include a description of a therapeutic intervention and the relevant medical history. The
case study was carried out over two or more successive consultations, interviews, or sessions. In these articles,
numerical data may, but need not necessarily, have been presented.
Correlational In correlational articles, an attempt was made to determine the relationship between variables. These articles reported
the use of correlational statistics (eg, Pearson's coefficient of correlation, Goodman and Kruskal's coefficient of ordinal
association, and other such statistics). No attempt was made to vary or control one of the variables, as is done in
experimental research.

D
Table 2. Description of ClmiJicatiott System: Clinical Treatment and Nonclinical Categories

Category Description

Clinical treatment In clinical treatment articles, there were reports of the implementation and the outcomes of treatment techniques applied to
individual patients or to groups of patients. There was an intention of maintaining or improving the function of those patients
receiving the treatment. Reports of methods of treatment were not inciuded in this category unless they involved a report of
the results of the application of those methods.
Nonclinical In nonclinical articles, there was no report and evaluation of treatment techniques intended to improve or maintain the function
of those receiving the treatment. Such articles included descriptions of (1) medical, surgical, and physical therapy techniques
and apparatuses, (2) procedures conducted using unimpaired volunteers not requiring the treatment to improve their
functioning, (3)norms established for different capabilities, (4) theories, and (5) conditions. These articles also included reports
of anatomy, biomechanics, and physiology.
-

Physical Therapy /Volume 75, Number 4 /April 1995

Reliability bined, in each 10-year period exam- The figure shows that increases have
ined (ie, 1954-1963, 1964-1973, occurred in the total numbers of arti-
I checked the reliability of both cate- 1974-1 983, 1984-193). Figure 1 cles published and indexed per de-
gory assignments of the part of the shows that an increased percentage of cade in each topic area since 1954.
sample dated 1954 to 1 W at least 1 articles in the sample described the
year after the initial analyses. Twenty- use of group designs in each succes- Figure 2 shows that of the 272 articles
seven of these articles, slightly over sive 10-year period. In the most recent in the sample, 187 (69%) were catego-
10%of that section of the sample, period (ie, 1984-1!993), group designs rized as nonclinical and 85 (31%) as
were checked. Using a table of ran- were used in the studies repolted in clinical treatment. In the initial period
dom numbers, articles were selected 53% of aU articles in the sample. By examined, 1954 to 1963, more than
fmm each topic subset in turn. The contrast, group designs were used in half (55940) of the articles described
initial assignments of each article to a studies reported in only 12% of the studies that investigated clinical treat-
particular method category and to articles from the period 1954 to 1963. ment. Since then, increasingly more
either the clinical category or the non- nontreatment articles have been pub-
clinical category were then compared Figure 2 shows the number of clinical lished. In 1964 to 1973, 57% of the
with the blinded reassignments of treatment articles (top graph) and articles were nonclinical; in 1974 to
each article. A coefficient of agreement nonclinical articles (lower graph) in 1983, 80% were nonclinical. In the
was used. The resultant level of agree- each of the successive 10-year periods most recent period examined, 19% to
ment was 1.00 for the classification of examined. Figure 2 also shows the 1333, 70% of the articles were classi-
articles as clinical or nonclinical and methods used in both the clinical fied as nonclinical. In 1973, the cumu-
.93 for the methods. This level of treatment and nonclinical categories. lative number of nonclinical articles in
agreemtmt for the methods is com-
parable to the .94 value obtained
by Ottenbacher and Short," the
originators of the methods classifi-
cation system used in this study. I
categorized each article in the re-
mainder of the sample (ie, articles 40
published 1991-1993) and then 35
recategorized each article after a
1-week interval to confirm the reli- 30
ability of the classification. 25

20

15 Experimental
Figure 1 shows the percentage of
articles describing studies using group 10 Quasi-group
designs, group-experimental and
quasi-group-comparisonarticles com- 5 Correlation
cn
a,
7

-
.u
4
0
1954- 1963 1964- 1973 1974- 1983 1984- I 9 9 3
Practical

2
&
. . Single subject
0
Survey

case

[I Descriptive

Figure 1. Percentage of articles in

1954- 1963 1964- 1973 1974- 1983 1984- 1993
the sarnple using group designs (group-
experimental and quasi-group-
comparison designs) for four successive Figure 2. Number of clinical (clinical treatment) and nonclinical articles in each
10-yearpennods. method categoryfor four successive 10-yearperiods.

Physical Therapy / Volume 75, Number 4 /April 1995

 examined, 1954 to 1963, group
designs were used in studies re-
ported in 11% of the clinical arti-
cles and in 13% of the nonclinical
40 articles. The use of group designs
35 increased steadily in the following
Experimental
cn two decades, and, in the most re-
a,
'.-
;5 30 ~uasi-group cent period examined (ie,
4
(ti Correlation 1984-19931, 61% of the studies
2
v-
25 reported in the clinical treatment
0 20 ~ractical articles and 50% of the studies
L
2 15 ... Single subject
reported in the nonclinical articles
used group designs.
E
2 10
survey
The studies reported in the nonclinical
5 case
articles, as shown in Figure 2, used
0 Descriptive both groupexperimental and quasi-
Knee Back Electrical
groupcomparison designs in addtion
Stimulation
to descriptive, survey, practical, and
correlational methods. Nonclinical
articles did not report the use of case
f ig~l'e3. Number of articles in each to@ categoy (ie, knee, back, electrical stim- studies or single-subject experimental
ulation) using each method. designs. By contrast, clinical articles
described the use of case studies,
the sample finally equaled that of the Figure 2 also shows a marked in- single-subject experimental designs, or
treatment articles. Since 1973, the crease has occurred in the use of either of the two types of group
cumulative number of nonclinical both types of group designs (ie, designs.
alricles in the sample has always ex- group-experimental and quasi-
ceeded the cumulative number of group-comparison designs) in both Figure 2 shows changes over time in
clinical treatment articles. the clinical treatment and nonclini- the number of articles reporting the
cal categories. In the initial period use of alternative methods. The use of
case studies was reported in 49% of all
sampled articles in 1954 to 1963 and
in 10% of the articles from 1984 to
193. The percentage of descriptive
studies dropped steadily from 27% in
1954 to 1963 to 6% in 1984 to 1 9 3 ,
except during the decade 1974 to 1983
when it rose to 42%. Special issues on
both the back (August 1979) and the
knee (December 1980) were pub-
lished during this decade and included
a large number of articles classified as
descriptive. These special issues re-
sulted in a marked inflation for the
decade of the number of descriptive
articles, which are, by definition, non-
clinical articles. Two methods, single-
subject designs and correlational stud-
ies, were used only in the most recent
decade surveyed, 1984 to 1 9 3 . Small
numbers of articles were involved; for
example, only two articles described
the use of single-subject designs. No
clear trends are apparent in the use of
the remaining alternative methods.
Figure 4. Percentage of articles categorized as clinical treatment in each topic
category (ie, knee, back, and electrical stimulation) tn each of four successive 10-year Figures 3 and 4 both illustrate the
periods. contribution of the individual topics

Physical Therapy / Volume 75, Number 4 / April 1995

(ie, knee, back, and electrical stimula- These findings must be put in per- profession. These three sets of articles,
tion) comprising the sample of articles spective. Three of a large number of therefore, likely reflect the more gen-
examined in this study. Figure 3 indexed topics were analyzed (ie, eral trends and changes in the re-
shows that group designs (ie, group- knee, back, electrical stimulation), and search pamdigms acceptable within
experimental and quasi-group- all articles on these topics came from the profession, and possibly also influ-
comparison designs) were used more the same physical therapy journal. The ence them. Articles on topics of lesser
frequently than any other methods in results showed some differences general interest and in some of the
the articles on electrical stimulation among the three topics in both the smaller bodies of physical therapy
(56%) and second most frequently in methods used and the percentage of literature are less likely to reflect gen-
articles on the knee (30?/0) and the clinical treatment arricles. The results, eral trends, or to influence them. Fur-
back (27%). More articles on the back however, were similar across all three ther studies of both different topics in
(29?/0)and the knee (27%)were de- topics: a general increase in the use of and different parts of the physical
scriptive. Figure 3 also shows that the group designs and a clearly lower therapy literature would indicate the
articles on electrical stimulation re- percentage of clinical treatment articles extent of generalizability of these
ported the use of few surveys and than nonclinical articles. Further, the findings. Further studies would also
more case studies than articles on number of articles published in and extend the degree of reliable reflection
either the back or the knee. since 1954 on each of these particular possible within the profession on
topics shows they are of major interest trends in relevant published research.
Figure 4 shows the percentage of all to physical therapists and of major
articles on each topic categorized as importance in the profession. Each of The finding of an increasing reliance
clinical treatment in each of the suc- these three topics has direct relevance on group designs in all t h e topic
cessive 10-year periods investigated. for practitioners in what is essentially a areas is consistent with what I believe
The figure shows that in only five clinical profession. Further, all articles is the extent of support in the profes-
instances. (ie, for the knee and back in came from Physical 7berapy, arguably sion for the method. Discussions of
1954-1943, for the back and electrical the most influential and enduring research and research methods in the
stimulation in 1W-1973, and for English-language journal in the profes- different journals and books produced
electrical stimulation in 1974-1983) sion as, among other things, it is the for the physical therapy market tend
has the percentage of clinical treat- only journal in the profession that to advocate use of group designs.
ment articles in any topic area reached both is indexed in major medical, Implicit support for group designs is
or exceeded 50%. In the most recent scientific, and physical therapy in- found in the greater space given them
period examined (ie, 1984-1993), dexes and has been published on a in many of the books and articles that
fewer than 30% of the articles on the monthly basis for decades. describe how to do1,2,4,28-33 and to
knee and back and 40% of the articles evaluateM.35 research and in the un-
on electrical stimulation were cli~lical Arguably, the broader relevance of stated presumptions in many discus-
treatment articles. these findings appears to rest on fur- sions of research that a group design
ther analyses of other major topics will be used. Whether these findings
Discussion within the different parts of the physi- are primarily a function of editorial
cal therapy literature. That is, the gen- policies in the different physical ther-
Over the four-decade period exam- eralizability of the findings of this apy journals, or whether they directly
ined (ie, 1954-19!93), an increasing study would appear to be limited until reflect the beliefs and practices of
percentage of articles on all three replications of the same procedures on readers of and contributors to those
topics analyzed used group designs, articles indexed on other topics and journals, is beyond the scope of this
and the majority of articles on each from other parts of the physical ther- study. This finding is, however, consis-
topic were nonclinical. The context of apy literature are undertaken. If subse- tent with an apparently high level of
this finding is the literature of a clinical quent investigations show the findings support for a particular approach to
professjon that actively and positively of thls study apply to only the three research in much of the physical ther-
promotes research. The profession topics examined and to the journal apy literature.
expects research to validate16 and to Physical ~ e m p ythese
, findings
further develop practice, and to in- would appear to be of limited interest. The finding of an increasing reliance
crease its own scientific credibility." These findings, however, would re- on group designs in both clinical treat-
These findings, however, suggest that main relevant and diicult to easily ment and nonclinical studies is, how-
the direct validation of practice via dismiss. The findings of this study are ever, paradoxical given existing criti-
research may not be occurring as generally consistent with what is said cisms of the method. Physical therapy
frequently as many believe and would about research in physical therapy. is a clinical profession. Many criticisms
like. These findings also suggest that a Further, the articles examined in this of group designs apply particularly
concurrent narrowing, rather than a study comprise a central portion of an when the method, not originally de-
widening, has occurred in the choices important and influential body of the signed for clinical research, is used in
of acceptable research methods. physical therapy literature and concern a clinical ~ o n t e ~ t . For ~ ~exam-
~ 5 ~ ~ ~
topic areas of major importance in the ple, criticisms of group designs con-

Physical Therapy/ Volume 75, Numbe

cem the difFiculties in obtaining SUB- ings that may be obtained as well as and interest. For example, in the pe-
ciently large and appropriate patient for the questions that can be riod investigated, the Journal's highest
samples and the risks of compromises investigated. priorities included the publication of
that may be made.l.36-41 Other criti- clinically relevant research57 and stud-
cisms concern difficulties in maintain- Alternative research methods to group ies on the eficacy of therapeutic inter-
ing experimental control in the clinical designs are available. Among those ventions.*l Additionally, during that
environment and in overcoming the methods explicitly designed for use in period, the current editor of the Jour-
ethical problems consequent upon the the clinical context are single-subject nal on more than one occasion explic-
requirement that treatment is allocated experimental designs and series of itly solicited research articles and case
according to the research design rather case studies. In this study, however, I reports relevant to clinicians.7 Al-
than to the patients' needs. Yet other found few single-subject experimental though requests by the current editor
criticisms question the relevance to designs and increasingly fewer case for articles could not have affected
practitioners, concerned with individu- studies in the topics examined. This most of the articles analyzed here,
als, of findings based on averaged finding is not explained by editorial they are indicative of the emphasis the
group data.36.42The relevance for policy. Editors of all major physical Journal has placed over time on clini-
physical therapy.of these criticisms of therapy journals have published re- cal articles.
group designs becomes apparent quests for57359 or advice on how to
when the clinical nature of the profes- use@-63 these methods. A range of The trend toward relatively fewer
sion, the context of the research, and different methods have also been used clinical treatment articles cannot be
the heavily clinically oriented benefits in the physical therapy literature. The explained by the advent of "new"
expected from research are question of why proportionately so journals. The newer physical therapy
remembered. few studies in the sample used alter- journals, those introduced since 1970,
native methods remains. The impor- commenced publication on a relatively
Little attention has been paid in the tance of this question lies in the role a infrequent basis, and the earliest of
physical therapy literature to the role a research method can play in establish- them, the Journal of Orthopaedic and
research method may play in research ing which questions are investigated, Sports Physical 7berapy,was not pub-
outcomes. Some contributors to the in the possible research outcomes, and lished until 1979. By contrast, the
physical therapy literature have sug- in the expectations in physical therapy vend toward a higher relative fre-
gested that the choice of problems that research will benefit practice. quency of nonclinical articles was well
investigated might be constrained by established by the 1980s, to the extent
amenability to statistical analysis.48~56 The second major finding of this study that their cumulative frequency in the
Inferences based on existing criticisms was that most studies reported in the sample always exceeded that of the
of group designs would support this sample of articles examined did not clinical articles published after 1973.
view. The diaculties inherent in inves- investigate clinical treatments. Clinical That is, the trend away from chnical
tigating questions where sufficiently practice is central to physical therapy, treatment research was well estab-
large and homogeneous patient treat- and physical therapists expect research lished prior to the advent of the spate
ment and nontreatment or alternative to benefit clinical practice. Most sur- of "new" physical therapy journals.
treatment groups are a requirement of veyed articles, however, did not report
the method quite likely do have impli- or evaluate the treatment techniques Questions arise about the clinical
cations for what is researched in phys- or regimens used in physical therapy treatment and nonclinical classification
ical therapy. That is, many questions to maintain or improve patient func- system. In particular, given the lack of
from within the practice domain are tion. Instead, most articles described precedents for such an analysis in
possibly not amenable to research procedures, apparatuses, or tech- physical therapy, does the classifica-
using group designs. Further, prob- niques used on unimpaired subjects tion system used enable meaningful
lems arise when the likely outcomes who explicitly did not require treat- and reliable distinctions? Distinctions
of the research using group designs in ment to improve their hnctioning or between clinical treatment and non-
the clinical context are considered. A provided information or discussed clinical articles were based on those
consequence of the demonstrated low issues of interest to physical therapists. existing within the physical therapy
power of many clinical studies using and psychological literatures, appear
group designs may be the unwar- The prevalence of nonclinical articles valid, and were shown to be reliable.
ranted acceptance of the null hypothe- is not explained by the sample se- Care, however, must be taken in inter-
sis in~estigated.~~ Research conducted lected. All three constituent topics (ie, preting the findings. In particular, the
with larger groups of more homoge- knee, back, and electrical stimulation) division between clinical treatment
neous patients, or research using dif- are highly relevant to practice. All and nonchcal articles does not imply
ferent measures or even a quite Mer- articles are cross-indexed and include that nonclinical articles have no rele-
ent research method, might produce a either areas often treated or methods vance for clinical practice. Clearly,
quite different outcome. That is, the often used for treating patients. Fur- many nonclinical articles describe
selection of a particular research ther, Physical nerapy actively solicits techniques used by clinicians or the
method has implications for the find- articles on topics of clinical relevance anatomy of a region, or they report

Physical Therapy /Volume 75, Number 4 /April 1995

the testing of measurement or treat- References 24 Norton BJ, Strube MJ. Some cautionary
comments on the use of retrospective designs
ment procedures subsequently used to evaluate treatment efficacy. Phys n e r .
and even investigated clinically, mese
I Currier DP. E l m a t s of Reseanh in Ph-ysi-
1988;68:1374-1377.
cal 7berapy. 3rd ed. Baltimore, Md: Williams
articles, however, do not report on the a Wilkins: 1 m . 25 Norton BJ, Strube MJ. Making decisions
im~lementationof treatments. Their 2 Makrides L, Richman I. Research methodol- based on group designs and meta-analysis.
o m and applied statistics. part 1: general prin- Phys 7bm. 1989;69:594-600.
subsequent or potential clinical usehl- 26 Sim J. Methodology and morality in phys-
ciples and basic concepts. Physiotherapy Can-
ness would be very diacult to estab- ada. 1980:32:135-139. iotherapy research. Physiotherapy. 1989;75:
lish and would involve a number of 3 Michels E. Design of Research and Analysis 237-243.
very questionable assumptions. The of Data in the Clinic. Alexandria, Va: Ameri- 27 Van der Linden S, Bouter L, Tugwell P.
can Physical Therapy Association; 1985. What are the minimal methodological and
distinction between nonclical and statistical requirements of a good trial? In:
4 Paycon OD. Research: 7he Validation of
treatment articles was specfi- Practice. 2nd ed. Philadelphia, Pa: FA Davis Schlapbach PB, Gerber NJ, eds. Physiothwapy:
cally made in this study to establish co; 1988. Controlled Trials and Facts. Basel,
Switzerland: S Karger AG, Medical and Scien-
the exrent to which a carefully se- 5 Schlaobach PB. Gerher N1, eds. Phvsiother- tific Publishers; 1991:2.
lected sample of physical thelapy apy: ~ o ~ t r o l l eTrials
d and Facts. ~ a & lSwit-
,
zerland: S Karger AG, Medical and Scientific 28 Makrides L, Richman J. Research method-
literature explicitly investigated clinical publishers:
, -
ology and applied statistics, part 6: ethics in
treatments. As even though human research. Physiotherapy Canada.
6 Twomey L. A growing commitment to re- 1981;33:89-94.
the profession is primarily clinical and search and evaluation. Australian Journal of
Physiotherapy. 1W,36:83. 29 Prince B, Makrides L, Richman J. Research
most ex~ectedoutcomes from re- methodology and applied statistics, part 2: the
7 Rothstein JM. Editor's note: The Journal- literature search. Pbysiotherapy Canada.
search relate to improving and validat- past, present, and future. Phys 7ber. 1989;69:
ing practice, most articles examined 1980;32:201-206.
795-796.
30 Richman J, Makrides L, Prince B. Research
were nonclinical. 8 Physiotherapy research. Physiotherapy. methodology and applied statistics, part 3:
1332;78:356-357. Policy statement. measurement procedures in research. Physio-
The findings of this study, together 9 Dean E. A vision of Physiotherapy Canada: therapy Canada. 1980;32:253257.
All things to all people. Physiotherapy Can- 31 Richman J, Makrides L. Research method-
with the increasing body of criticism ada. 1993:45:5-6. ology and applied statistics, part 4: overcom-
of group designs and the awareness in 10 The Association of Chartered Physiothera- ing statistics anxiety. Physiotherapy Canada.
the ~rofessionof the need for research pists in Research. Physiotherapy. 1983;69:317. 1980;32:321-329.
into clinical treatments, suggest more 11 Miles-Ta~~ing C, Dyck A, Brunham S, 32 Makrides L, Richman J. Research method-
et al. Canadian therapists' priorities for clinical ology and applied statistics, pan 5: experi-
extensive considelation of and reflec- research: a Delphi study, phys ner, mental design. Physiotherapy Canada.
tions on the research published within 19~;70:448-454. 1981;33:6-14.
the physical therapy literature are 12 Rose SJ. Editor's note: Gathering storms. 33 Makrides L, Richman J. Research method-
needed. For example, is the reliance P h y ~7ber. 1989;69:354-355. ology and applied statistics, part 7: writing the
on group designs found in this study 13 Gielen F. Discussion of placebo elfect in research proposal. Physiotherapy Canada.
physiotherapy based on a noncritical review 1981;33:16%168.
also found in the more extended of the literature. Physiotherapy Canada. 34 Domholdt FA, Malone TR. Evaluating re-
physical therapy literature?Why are 1989;41:210-216. search literature: the educated clinician. Phys
apparently so few researchers using 14 List M. Physiotherapy: a mobile profession 'Ther. 1985;65:487-491.
in health care. Physiotherapy. 1986;72: 35 APTA Committee on Research: Guidelines
alternatives to group designs, or hav- 122-124. on critically considering research papers. Phys
ing them published? Do the findings 15 Momssey MC, Kanda LT, Brewster CE. 7ber 1988;68:555-556.
in this study of apparently so few Development of a clinical physical therapy 36 Gomella C. Designs for clinical research.
studies of clinical treatments also ap- research program. Phys 7ber. 1987;67: Pbys 7ber. 1973;53:1276-1283.
1110-1114.
ply to other topic areas? If so, why, 37 Lehmkuhl LD. Mixing one part common
16 Cooper BA, Saarinen-Rahikka HK. Interne- sense with each part statistics in planning the
and how can more research into clini- lationships of theory, clinical models and re- design and reporting the results of clinical
cal treatments be actively promoted search. Physiotherapy Canada. 1986; research in Ph.vsica1 7berapy. Phys 'Ther.
and published within the profession's 38:97-100. 1987;67:1851-1853.
17 Ottenbacher KJ. Emluating Clinical 38 Riddvch J, Lennon S. Evaluation of
literature? Change: Strategiesfor Occupational and practice: the single-case study approach. Phys-
Physical 'Therapists.Baltimore, Md: Williams L iotherapy 7beory and Practice. 1331;7:%11.
Conclusions Wilkins; 1986. 39 Shepard KF. Qualitative and quantitative
18 Zimny NJ, Tandy CJ. Problem-knowledge research in clinical practice. Phys 7ber.
coupling: a tool for physical therapy clinical 1987;67:1891-1894.
The findings of this study question practice. Phys 7ber. 1989;69:155-161. 40 Deyo RA,Walsh NE, Schoenfeld LS,Ra-
whether the benefits expected to ac- 19 Mathews JS. The 1987 Presidential mamurthy S. Can trials of physical treatments
crue from research to physical thelapy Address: Towards a profession of suhstance. be blinded? Am J Phys Med Rehabil. 1990;
can be realized, given current research Phys n e r . 1987;67:1723-1725. 69:6-9.
20 Wood R. Twenty-third Mary McMillan 41 Partridge C. The myth of the medical
practices and emphases. More physical Lecture: Footprints. Phys 7her. 1989;69: model. Physiotherapy. 1982;68:329.
therapists need to be aware of the 975-980. 42 Rosnow RL, Rosenthal R. Statistical proce-
importance of researching our clinical 21 Stockham MJ. Are physiotherapists neces- dures and the justification of knowledge in
treatments and the range of research sary? Physiotherapy. 1988;74:499-501. psychological science. Am Psychol. 19W,
methods available. 22 Grieve E. A fair trial for physiotherapy: the 44:1276-1284.
controlled trial. Physiotherapy. 1993;79:304. 43 Gore SM, Jones IG, Rytter EC. Misuse of
23 Jackson DA. Where is the physiotherapy statistical methods: critical assessment of arti-
profession going? Physiotherapy. 1987;73: cles in BMJfrom January to March 1976. EM].
590-591. 1977;1:85-87.

physical Therapy / Volume 75, Number 4 / ~ ~ r1995

i l
 44 Vrbos LA, Lorenz MA, Peabody EH, model. Am J Phys Med Rehabil. 1990;69: 69 Aufdemkarnpe G. Some comments on
McGregor M. Clinical methodologies and inci- 286-292. single-case studies. Pbysiotherw Theory and
dence of appropriate statistical testing in or- 56 Tamrnivaara J, Shepard KF. Theory: the Practice. 1991;7:63-71.
thopaedic spine literature. Spine. 1993;18: guide to clinical practice and research. Phys 70 Holliday PJ. A survey of research contribu-
1021-1029. Ther. 1990;70:578-582. tions of physiotherapists. Physiotherapy Can-
45 Grichting WL. Psychology and sociology in 57 Lister MJ. Editor's note: Farewell. Phys ada. 1981;33:372-376.
Australia: the published evidence. Australian Ther. 1988;68:183-184. 71 Mitchell J. Research in the 1980s: world
Psychologfi. 1989;24:115-126. trends in physiotherapy research. Physiother-
58 Rothstein JM. Editor's note: Clinical litera-
46 Huxley P. Statistical errors in papers in the ture. Phys Ther. 1989;69:895-8%. apy Theov and Practice. 1%3;9:171-176.
British Journal of Social Work. Soc Work. 72 Dean E. Physiotherapy Canada survey
59 Rothstein JM. Editor's note: The case for
1986;16:64>658. (1973-19821: a trend to research. Physiother-
case reports. Phys Ther. 1993;73:492-293.
47 Carver RP. The case against statistical sig- apy Canada. 1985;37:158-161,
nificance testing. Haruard Educational Re- 60 Cole P. How to . . . single-subject designs
for clinical and research purposes. Australian 73 Jones D. M~nichielloV. A survey of geron-
dm. 1978;48:378-399. tology articles in physiotherapy journals. Aus-
Journal of Physiotherapy. 1991;37:127-128.
48 French S. How significant is statistical sig- tralian Journal of Physiotherapy. 191;37:
61 Gonnella C. Single-subject experimental
nificance? A critique of the use of statistics in 155-161.
paradigm as a clinical decision tool. Phys
research. Physiotherapy. 1988;74:266-268, 74 Instructions for authors. Phys Ther. 1993;
Ther. 1989;69:601-609.
49 Harcum ER. The highly inappropriate cali- 73:58.
62 Riddoch J, Lemon S. Single-case research
brations of statistical significance. Am Psychol. 75 Bohannon RW. Importance of Physical
designs. Physiotherapy Theory and Practice.
1989;44:964-965. Therapy grows. Phys Ther. 1988;68:584.Letter
1991;7:1. Editorial.
50 Oakes M. Statistical Inference;A Commen- to the Editor.
63 Robertson VJ, Lee VL. Some misconcep-
tary for the S o c ~ and
l Behauiouml Sciences. 76 Bohannon RW. Physiotherapy Practice.
tions about single-subject designs in physio-
Chichester, England: John Wiley & Sons Ltd; 1989;5:99-100. Letters page.
therapy. Physiotherapy. 1994;80:762-766.
1986.
64 Jensen GM. Qualitative methods in physi- 77 Bohannon RW. Core journals of physio-
51 Rothstein JM. Editor's note: Much ado therapy. Physiotheram Practice. 1987;3:
cal therapy research: a form of disciplined
about probability. Phys Ther. 1990;70:535-536. 126-128.
inquiry. Phys Ther 1989;69:492-500.
52 Matyas TA, Ottenbacher KJ. Confounds of 78 Dennis JK. Decisions made by physio-
insensitivity and blind luck: statistical conclu-
65 Schmoll BJ. Ethnographic inquiry in clini-
cal settings. Phys mer. 1987;67:1895-1897. therapists: a study of private practitioners in
sion validity in stroke rehabilitation clinical Victoria. AustrallanJoumal of Physiotherapy.
trials. Arch Phy.s Med Rehabif. 1993;74: 66 Stonecipher DR, Catlin PA. The effect of
a forearm strap on wrist extensor strength. 1987; 33:181-191.
559-565.
J M o p Sports Phys Ther. 1984;6:184-189. 79 Ottenbacher KJ, Short MA. Publication
53 Barlow DH. On the relation of clinical re- trends in occupational therapy. Occupational
search to clinical practice: current issues, new 67 Michels E. Using and Understanding
Surveys; An Introductory Manual. Alexandria, TherapyJournal of Research. 1982;2:80-88.
directions. J Consult Clin Psychol. 1981;2:
Va: American Physical Therapy Association; 80 Goldiamond I. Prvtection of human sub-
147-155.
1985. jects and patients. Behauiorism. 1976;l:l-41.
54 Ottenbacher KJ. The significance of power
68 Portney LG, Watkins MP. Foundations of 81 Lamb RL, Bohannon RW, Craik RL., et al:
and the power of signhcance: recommenda-
Clinical Research. East Norwalk, Conn: Apple- Guest editorial: Reliability discussion required.
tions for occupational therapy research. Occu-
ton & Lange; 1993. Phys Ther. 1987;67:501.
pational Therapy- - Journal of Research. 1984;
4:37-50.
55 Ottenbacher KJ. Clinically relevant designs
for rehabilitation research: the idiographic

Invited Commentarv

Dr Robertson provides us with much ClinicaVNonclinical Conclusions therapy techniques, interventions, and
useful information about selected modalities." I h d thts to be a limited
publicationsincluding their methods One of Dr Robertson's major findings and somewhat unrealistic view of
and clinical content--that have ap- is that only 31%of the articles in her research. Because I have not exam-
peared in Physical merapy over the sample investigated clinical treatments. ined the citations that Dr Robertson
last 40 years. She and I differ, how- My first point related to the clinical/ used to substantiate the above quote,
ever, in our views of the framework nonclinical conclusions drawn in this I am uncertain whether my debate is
into which these data should be study concerns Dr Robe~tson'sview of with Dr Robertson's interpretation of
placed and in the conclusions that we the purpose of research. The finding these sources or with the sources
draw from them. In this commentary, of only 31% of clinical articles was themselves. In any event, this view
then, I will place a different spin on viewed as problematic by Dr Robert- that research will "demonsvate effec-
the data that Dr Robertson has care- son because such a low proportion of tiveness" presupposes that physical
fully collected and reported. clinical treatment articles seems un- therapy is effective and that a major
likely to fulfill a mission of research purpose of research is to document
that is "expected to demonstrate and for others what we, as physical thera-
improve the effectiveness of physical pists already convinced of our woah

Physical Therapy /Volume 75, Number 4 / April 1995