Global Journal of Health Science; Vol. 7, No.

7; 2015
ISSN 1916-9736 E-ISSN 1916-9744
Published by Canadian Center of Science and Education

The Effect of Applying Direct Observation of Procedural Skills

(DOPS) on Nursing Students' Clinical Skills: A Randomized Clinical
Trial
Habibi Hengameh1, Raiesifar Afsaneh2, Khaghanizade Morteza3, Mahmudi Hosein4,
Seyed Mazhari Marjan1 & Abbas Ebadi3
1
Faculty of Nursing, Aja University of Medical Science, Tehran, Iran
2
Faculty of Nursing and Midwifery, Tehran University of Medical Sciences, Tehran, Iran
3
Behavioral Sciences Research Center (BSRC), Baqiyatallah University of Medical Sciences, Tehran, Iran
4
Faculty of Nursing, Baqiyatallah University of Medical Sciences, Tehran, Iran
Correspondence: Khaghanizade Morteza, Behavioral Sciences Research Center (BSRC), Baqiyatallah University
of Medical Sciences, Tehran, Iran. Tel: 98-21-2228-9941. Fax: 98-21-2612-7237. E-mail:
[email protected]

Received: December 2, 2014 Accepted: January 28, 2015 Online Published: March 26, 2015
doi:10.5539/gjhs.v7n7p17 URL: http://dx.doi.org/10.5539/gjhs.v7n7p17

Abstract
Background: Remarkable advances in educational measurement have proved need to the implementation of
modern and appropriate methods of clinical evaluation. This study was carried out to compare the effect of
applying direct observation procedural skills and routine evaluation method on clinical skills of nursing students.
Methods: This randomized clinical trial was conducted on students of Nursing Army College, Tehran, Iran.
After obtaining approval from the Ethics Committee of the Baqiyatallah University of Medical Sciences
Research Deputy, all nursing students and instructors who agreed to participate in this study sign the informed
consent. The participants were randomly assigned into intervention and control groups. After the teachers were
trained and an inter-raters reliability test was conducted, evaluation was performed through DOPS in the
intervention group while the control groups were evaluated through the routine method. Assessment checklists
for two procedures (Intra venous catheterization and change dressing) were valid and reliable. Finally data were
analyzed through descriptive and analytical statistics (Chi-square, t-test, Repeated Measure ANOVA) using
SPSS version 16.
Results: No significant difference was observed between the two groups in terms of demographic variables
(P>0.05), but a significant difference was observed between intervention and control scores (P=0.000). In other
words, application of DOPS has improved clinical skills of the students significantly.
Conclusion: Using this new method improved the students’ scores in clinical procedures implementation;
therefore, we suggest that nursing colleges apply this evaluation method for clinical education.
Keywords: DOPS, direct observation of procedural skills, clinical evaluation, nursing education
1. Introduction
Clinical education is a set of facilitators of learning activities in a clinical environment. The purpose of such
education is to create measurable changes in students clinical practice (Abedini, Aghamolaee, Jome Zadeh, &
Kamjo, 2009). More than half of the educational goals of the nursing course are devoted to clinical education
(Ebrahimi & Oudi, 2005). Nursing education managers recognize clinical education as the most important part of
nursing education (Rahimi, 2005). Clinical education is one of the strategies for developing nurses' clinical
competency (Zareiyan Jahromi, 2005). In this regard, one of the important and challenging issues in clinical
education is students' evaluation in the field. Evaluation is the most important duty which the faculties are
confronted with (Crossley Humphris, & Jolly, 2002). Through an appropriate evaluation method, possible
strengths and weaknesses of education can be enumerated. The next step to the recognition of the pros and cons
of the system positive aspects can be improved and shortcomings eradicated, this would be another step toward
renovating of the entire educational system (Smith-Strøm & Nortvedt, 2008). Effective evaluation creates

17
www.ccsenet.org/gjhs Global Journal of Health Science Vol. 7, No. 7; 2015

motivation in students, and also it help the educator with evaluating their activities (Franko et al., 2008) and if it
is accompanied with feedback, it will be effective on promoting the learning of individual skill (Bari, 2010).
As a result of constant changes of approaches in clinical education, the necessity of implementing appropriate
new methods of evaluation has become more succinct than before (Noohi & Haghdoost, 2008). Feedback
oriented clinical evaluation methods will promote learning in addition to evaluating the difficult issues in
traditional students' evaluation (Kariman, 2010). To carry out evaluation in clinical environments, different
methods can be used such as performance observation, 360 degree evaluation, objective structured clinical
examination, Mini-clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills
(DOPS). This latter one, DOPS, is a method for evaluation and providing feedback pertaining to practical skills
such as Intravenous catheterization, ECG taking, change dressing and etc. In this method, observation and
clinical skill are performed through adopting the procedure by a student on a real patient (Crossley et al., 2002).
DOPS is held at three levels with specified time intervals. At the end of each level, evaluator observes the
student during performing the procedure then provides feedback to student and mentions their strengths and
weaknesses. At the end of the third level, the evaluator rates the students' performance by using a structured
checklist and gives them the feedback (Malekanrad, 2006).
Although clinical skill have important role in medical education, success of educated medical groups follow their
memories to some extent. According to the literature in the most of clinical courses, evaluation methods do not
include efficiency in evaluation of clinical skills and students' function as well as inappropriateness with
educational objectives (Nouhi & Foroud, 2003; Rushforth, 2007; Asefzadeh, 2000; Schoonheim‐Klein Walmsley,
Habets, Van Der Velden, & Manogue, 2005). This is while practical skill have had the main role in medical
education and mental knowledge is of secondary importance (Noohi & Haghdoost, 2008). Therefore the current
study was conducted to assess the effect of DOPS method as a new evaluation method on promotion of nursing
students' clinical skills.
2. Methods
This is a randomized clinical study. An approval was obtained from the Ethics Committee of the Baqiyatallah
University of Medical Sciences Research Deputy. The purpose and methods of the study were explained for the
participants, both the educators and students, and then informed consent was obtained. All nursing students, who
were passing medical-surgical and intensive courses in tow hospitals fields, were allocated into two intervention
and control groups randomly. Ten clinical instructors who desired to participate and passing the new evaluation
methods workshop were placed into two intervention and control groups at random. The students and the
instructors were allowed to leave the study whenever they would not prefer to continue. Considered procedures
in this study included change dressing and IV catheterization. The study consisted of two general phases of A)
preparation and training the instructors and B) implementation of evaluation plan. In the first phase, the required
training protocol was prepared for DOPS method then the intervention instructors group was trained within a
four-hour workshop.
In the second phase, control group was evaluated based on the routine evaluation method of Army Nursing
College and intervention group was evaluated based on the DOPS. The routine method of Nursing College
includes a subjective judgment of an instructor about general skills of the student during their clinical course
hence the scoring. In intervention group, clinical activities of students (performance of the aforementioned
procedures) were evaluated through the DOPS based on the checklists. The evaluation steps in intervention
group took place as follow:
A) First stage test: (observation of the skills for 15 minutes and giving feedback for 5 minutes).
B) Second stage test: repeating the first test after two weeks (emphasis on providing feedback on the students'
strength and weakness).
C) Third stage test: repeating the first stage test after four weeks and giving the final scores to the student.
In the control group, two procedures were evaluated only in one stage as suggested by the routine. Final
evaluation of students in each group was performed based on the checklists prepared by the researchers. A zero
score represents observing no appropriate behavior by a student, one is less than expectation, 2 bordering, 3
expected, and 4 is excellent. For assessment of psychometric properties of the checklists, after an in-depth library
and digital search to find out how the procedures is to be implemented, an initial draft for each procedures was
prepared. Next, the draft was submitted to 10 experienced faculty members in the relevant clinical field for
content validity assessment. According to the Low-shed table, content validity rate (CVR) was 0.62, based on
which, the questions that obtained good points were selected. In order to assess the content validity index (CVI)

18
www.ccsenet.org/gjhs Global Journal of Health Science Vol. 7, No. 7; 2015

with relevant formulation, the questions scored more than 0.79 considered as appropriate and those with scores
between 0.70 and 0.79 were later revised by experts. The questions scored less than 0.70 were rejected. To
determine the inter rater reliability, two methods of equivalence and inter rater reliability were used. Three
teachers observed and evaluated five students for each procedure simultaneously; then, inter rater reliability
between assessors was measured by ICC (Intraclass correlation coefficient) and each item was evaluated by
Kappa coefficient test; Kappa coefficient was 0.6 and ICC equaled 0.5. Finally, the data was analyzed by the
SPSS software version 16 within descriptive statistic tests (Frequency, mean and standard deviation) and
analytical statistic tests such as Kolmogorov Smirnov, Chi-square, independent t-test, Repeated Measure
ANOVA, Pearson correlation coefficient based on the type of variable.
3. Results
Homogeneity of the two groups of age and the total last terms means tested by independent t-test, gender, marital
status and distribution of students in hospital units through Chi-square test and there was no meaningful
difference in two groups (Table 1).

Table 1. Distribution of Demographic Variables in Intervention and Control Groups

Intervention group Control group
Demographic variables Test
Mean± SD Mean± SD
Age(year) 23±0.84 22.8±0.58 T=-1.15 P=0.25 Df=68
Total average 15.9±1.59 16.19±1.42 T=-0.8 P=0.42 Df=68
Frequency (%) Frequency (%)
Female 54.3 60
Gender X2= 0.232 P= 0.62
male 45.7 40

Marital Married 31.4 45.7

X2= 1.5 P= 0.22
status Single 68.6 54.3

The independent t-test indicates a significant difference in resulted scores from the two different evaluation
methods (Table 2). Comparison of DOPS scores in three steps shows there is an increasing trend from step one
to step three in both procedures [IV catheterization(p=0.000), change dressing(p<0.014)]. A significant
correlation was observed between total students means in last semester and score of IV catheterization procedure
(r=0.23, p<0.016) by using Pearson correlation coefficient.

Table 2. Comparison of Tow Procedures Final Scores in Intervention and Control Groups
Procedure Intervention group Control group Independent sample t-test
(Mean± SD) (Mean± SD)

IV catheterization 112.34±10.24 86.08±10.92 T=-10.373 Df=68 P=0.000

Change dressing 92.22±8.15 66.62±9.82 T=-11.858 Df=68 P=0.000

4. Discussion
This study was carried out in order to compare the effect of DOPS against the routine method on clinical skills
promotion of nursing students. The results indicate that the DOPS test is more effective on skills level promotion
of the nursing students in comparison with the traditional evaluation method. These findings were confirmed by
the Shahgheibi et al. (2009) study conducted to assess the DOPS effect on the evaluation of clinical skills of
internship course students in maternity units. In their study students' mean scores which were evaluated through
DOPS had significant difference in comparison with the control group (Shahgheibi Sh, 2009). In the present
study, by performing DOPS in three stages, improvement of the students' scores and the development of their
skills level were seen. The reason for this development can be attached to more practice in implementation of
procedure and taking tests in several stages and giving feedback in each stage. Bazrafkan (2009) study indicated

19
www.ccsenet.org/gjhs Global Journal of Health Science Vol. 7, No. 7; 2015

that the students' scores in DOPS test had normal distribution and 87.6% of students earned acceptable scores
(Bazrafkan, 2009). Also in this study there was an ascending trend in two procedures' scores in intervention
group. Vang et al. (2007) cohort study with the purpose of evaluating the clinical skills development of nurses
through DOPS and mini-CEX, indicated that these evaluation methods cause development in skills, learning
strategies, and attitude toward hospital, assessment the ability and performance and improving the quality of
interventions in nursing practices (Wang and Lin, 2007). In this study, no significant correlation was observed
between students’ demographic variables and clinical skills scores but total average of passed last courses had
significant correlation with IV catheterization procedure score also. In Salimi et al. (2005) study no meaningful
difference was seen between male and female students' clinical skill but there was a significant difference
between average of last semester and clinical skills mean scores in intensive units (Salimi, 2005). Therefore, it
seems that better theoretical background and higher average have direct effect on skill learning and increasing
the level of the skill. Sinclair and Cleland (2007) in their study concluded that male medical students, who are
weak in theory, may not use feedback for assessing their learning experiences. Female and cleverer students
follow feedbacks to improve their status (Sinclair & Cleland, 2007). As it can be noticed, in different studies,
there are similar results about significant difference between final score of DOPS and the traditional evaluation
method. Compared with new methods, such as the OSCE which evaluation done in artificial conditions or
log-books which measures the quantity of procudures or acts, DOPS have more validity. Sahebalzamani et al.
(2012) study showed that a high percentage of studied teachers and students are satisfied or very satisfied with
evaluation with DOPS. Thay argued that the students are satisfied when assessed by those with more knowledge
level (Sahebalzamani & Jahantigh, 2012).
5. Conclusion
This study showed that using DOPS improved the students’ scores in clinical procedures implementation. Thus,
it is recommended that in various courses or procedure evaluation, the traditional methods be replaced with the
DOPS. These new evaluation techniques could reduce the most important educational problem that is
theory-practice gap especially in medical sciences fields, which are more sensitive fields. The limitations of this
study were instructors’ resistance and lack of inclination for the new evaluation methods. Also, unfortunately in
this study, the possibility of hawthorn effect from some teachers was not probed. Besides, due to the nature of the
study no pretest could be taken by the participants.
Acknowledgments
We would really appreciate the respective authorities of Nursing College of Army and Baqiyatallah, instructors,
students and particular research manager of Nursing College of Baqiyatallah for they have played important role
in doing this study.
References
Abedini, S. A. S., Aghamolaee, T., Jome Zadeh, A., & Kamjo, A. (2009). Clinical education problems from the
perspective of nursing and midwifery students Hormozgan University of Medical Sciences. Journal of
Hormozgan University ofMedical Sciences, 12, 249-253.
Asefzadeh, S. (2000). A practical guide to assessment for higher specialist training. Qazvin, Qazvin University
of Medical Sciences.
Bari, V. (2010). Direct observation of procedural skills in radiology. American Journal of Roentgenology, 195,
W14-W18. http://dx.doi.org/10.2214/AJR.09.4068
Bazrafkan, L. (2009. Comparison of the Assessment of Dental Students’. Journal of Medical Education, 13.
Crossley, J., Humphris, G., & Jolly, B. (2002). Assessing health professionals. Medical education, 36, 800-804.
http://dx.doi.org/10.1046/j.1365-2923.2002.01294.x
Ebrahimi, H. Z. H. R., & Oudi, D. (2005). Assessment problems of teachers in Clinical evaluation for nursing
students in shahrood University of Medical Sciences. Iranian journal of medical education, 14.
Franko, D. L., Cousineau, T. M., Trant, M., Green, T. C., Rancourt, D., Thompson, D., ... Ciccazzo, M. (2008).
Motivation, self-efficacy, physical activity and nutrition in college students: Randomized controlled trial of
an internet-based education program. Preventive medicine, 47, 369-377.
http://dx.doi.org/10.1016/j.ypmed.2008.06.013
Kariman, N. H. T. (2010). The effect of Portfolio’s evaluation on learning and satisfaction of midwifery students.
Arak Medical University Journal (AMUJ), 12, 81-88.
Malekanrad, E. E. A. (2006). Clinical education and evaluation, Tehran, Tohfeh.

20
www.ccsenet.org/gjhs Global Journal of Health Science Vol. 7, No. 7; 2015

Noohi, E. M. M., & Haghdoost, A. (2008). Clinical Teachers' Viewpoints towards Objective Structured Clinical
Examination in Kerman University of Medical Science. Iranian journal of medical education, 8, 113-119.
Nouhi, S. F. A., & Foroud, A. (2003). Investigation of the problems in clinical evaluation according to clinical
teachers of medicine and dentists in Kerman University of Medical Sciences 6th Medical Education
Congress. Tehran: Shahid Beheshti University of Medical Sciences.
Rahimi, A. A. F. (2005). The obstacles and improving strategies of clinical education from the viewpoints of
clinical instructors in Tehran's Nursing Schools. Iranian Journal of Medical Education, 5, 73-80.
Rushforth, H. E. (2007). Objective structured clinical examination (OSCE): review of literature and implications
for nursing education. Nurse Education Today, 27, 481-490. http://dx.doi.org/10.1016/j.nedt.2006.08.009
Sahebalzamani, M. F. H., & Jahantigh, M. (2012). Validity and reliability of direct observation of procedural
skills in evaluating the clinical skills of nursing students of Zahedan nursing and midwifery school.
Zahedan J Res Med Sci (ZJRMS), 14, 76-81.
Salimi, T. K. H., Shahbazi, L., Dehghan Pour Mh, Hafezieh, A., Parandeh, K., et al. (2005). Evaluation of
clinical skills of final year nursing students in critical care units. Journal of Shahid Sadoughi University of
Medical Sciences and Health Services, 3, 60-66.
Schoonheim‐ Klein, M., Walmsley, A., Habets, L., Van Der Velden, U., & Manogue, M. (2005). An
implementation strategy for introducing an OSCE into a dental school. European Journal of Dental
Education, 9, 143-149. http://dx.doi.org/10.1111/j.1600-0579.2005.00379.x
Shahgheibi Sh, P. A., Bahramrezaie, M., Farhadifar, F., & Khatibi, R. (2009). Evaluation of the Effects of Direct
Observation of Procedural Skills (DOPS) on Clinical Externship Students’ Learning Level in Obstetrics
Ward of Kurdistan University of Medical Sciences. Journal of Medicine Eduction, 13, 29-33.
Sinclair, H. K. & Cleland, J. A. (2007). Undergraduate medical students: who seeks formative feedback?
Medical education, 41, 580-582. http://dx.doi.org/10.1111/j.1365-2923.2007.02768.x
Smith-Strøm, H., & Nortvedt, M. W. (2008). Evaluation of evidence-based methods used to teach nursing
students to critically appraise evidence. The Journal of nursing education, 47, 372-375.
http://dx.doi.org/10.3928/01484834-20080801-08
Wang, M., & Lin, W. (2007). The nurturing and development of nurse practitioners]. Hu li za zhi The journal of
nursing, 54, 11.
Zareiyan Jahromi, A. A. F. (2005). Learning needs assessment in bachelor of nursing: A qualitative research.
Iranian journal of medical education, 5, 81-2.

Copyrights
Copyright for this article is retained by the author(s), with first publication rights granted to the journal.
This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution
license (http://creativecommons.org/licenses/by/3.0/).

21