Disability and Rehabilitation: Assistive Technology

ISSN: 1748-3107 (Print) 1748-3115 (Online) Journal homepage: http://www.tandfonline.com/loi/iidt20

Rehabilitation of activities of daily living in virtual

environments with intuitive user interface and
force feedback

Vico Chung-Lim Chiang, King-Hung Lo & Kup-Sze Choi

To cite this article: Vico Chung-Lim Chiang, King-Hung Lo & Kup-Sze Choi (2016):
Rehabilitation of activities of daily living in virtual environments with intuitive user
interface and force feedback, Disability and Rehabilitation: Assistive Technology, DOI:
10.1080/17483107.2016.1218554

To link to this article: http://dx.doi.org/10.1080/17483107.2016.1218554

Published online: 26 Oct 2016.

Submit your article to this journal

Article views: 25

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=iidt20

Download by: [University of California, San Diego] Date: 01 November 2016, At: 10:25
DISABILITY AND REHABILITATION: ASSISTIVE TECHNOLOGY, 2016
http://dx.doi.org/10.1080/17483107.2016.1218554

ORIGINAL RESEARCH

Rehabilitation of activities of daily living in virtual environments with intuitive

user interface and force feedback
Vico Chung-Lim Chianga, King-Hung Lob and Kup-Sze Choic
a
School of Nursing, The Hong Kong Polytechnic University, Kowloon, Hong Kong; bDepartment of Occupational Therapy, Hong Kong Red Cross
Princess Alexandra School, Kowloon, Hong Kong; cCentre for Smart Health, School of Nursing, The Hong Kong Polytechnic University, Kowloon,
Hong Kong

ABSTRACT ARTICLE HISTORY

Purpose: To investigate the feasibility of using a virtual rehabilitation system with intuitive user interface Received 31 March 2016
and force feedback to improve the skills in activities of daily living (ADL). Revised 23 July 2016
Method: A virtual training system equipped with haptic devices was developed for the rehabilitation of Accepted 26 July 2016
three ADL tasks – door unlocking, water pouring and meat cutting. Twenty subjects with upper limb dis-
abilities, supervised by two occupational therapists, received a four-session training using the system. The KEYWORDS
task completion time and the amount of water poured into a virtual glass were recorded. The perform- Activities of daily living;
ance of the three tasks in reality was assessed before and after the virtual training. Feedback of the partic- computer-assisted rehabili-
ipants was collected with questionnaires after the study. tation; haptic feedback;
Results: The completion time of the virtual tasks decreased during the training (p < 0.01) while the per- occupational therapy;
centage of water successfully poured increased (p ¼ 0.051). The score of the Borg scale of perceived exer- virtual reality
tion was 1.05 (SD ¼ 1.85; 95% CI ¼ 0.18–1.92) and that of the task specific feedback questionnaire was 31
(SD ¼ 4.85; 95% CI ¼ 28.66–33.34). The feedback of the therapists suggested a positive rehabilitation
effect. The participants had positive perception towards the system.
Conclusions: The system can potentially be used as a tool to complement conventional rehabilitation
approaches of ADL.

ä IMPLICATIONS FOR REHABILITATION

 Rehabilitation of activities of daily living can be facilitated using computer-assisted approaches.
 The existing approaches focus on cognitive training rather than the manual skills.
 A virtual training system with intuitive user interface and force feedback was designed to improve the
learning of the manual skills.
 The study shows that system could be used as a training tool to complement conventional rehabilita-
tion approaches.

Introduction shown that task-oriented approaches focusing on the attainment

of specific goals can improve their performance in ADL.[3–5]
Activities of daily living (ADL) refer to the activities carried out by
Conventional task-oriented approach requires disabled people to
people to live an independent life without assistance, e.g. groom-
perform a task repeatedly in a controlled environment, and under
ing, preparing food or eating. Manual dexterity is needed to per-
form ADL properly and effectively. For people with upper limb direct supervision of occupational therapists (OTs) who provides
disabilities, deficiency in the functions of their upper arm, fore- guidance and make assessment during the process. The conven-
arms or hands limit their ability to perform ADL independently, tional approach demands considerable amount of human resource
thereby hindering self care and leading to low self esteem. since the rehabilitation process requires constant and long hours
According to the Centers for Disease Control and Prevention, of one-to-one supervision. The therapists need to prepare the
there are at least 1500 newborns annually suffering from various materials required for the training of ADL, set up, clean up and
forms of upper limb disabilities in the United States of America.[1] reset the environment again and again. In the meantime, they
In Hong Kong, as of 2008, 15% of the physically disabled popula- also evaluate the performance of the patient while making correc-
tion had motor impairment in the upper limbs.[2] This could be tion and providing guidance. The process is critical for effective
an issue to the healthcare system as well as the individuals and rehabilitation but it is also demanding and laborious, which limits
the families concerned. the throughput of the training.
Occupational therapy is an effective form of rehabilitation for The rehabilitation process can potentially be computerized
people with upper limb disabilities to cope with their motor using virtual reality (VR) technology that can offer a solution to
impairment and regain the ability of performing ADL and self complement the conventional approaches and enhance the train-
care. The therapy caters for the specific needs of individuals, and ing effectiveness. In fact, VR has been regarded as an effective
maximizes the function of their affected limbs. Studies have tool for motor and cognitive rehabilitation. VR technology

CONTACT Dr Kup-Sze Choi, PhD [email protected] School of Nursing, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong
ß 2016 Informa UK Limited, trading as Taylor & Francis Group
2 V. C.-L. CHIANG ET AL.

encompasses the use of computer hardware and software to Table 1. Demographic data of the subjects.
simulate real environments using interactive computer-generated Demographics Category N %
audio, visual and haptic feedbacks so that the users can percept Gender Male 12 60
and immerse in the corresponding virtual environments as if they Female 8 40
were real. One of the major advantages of using VR systems in Age (years) <13 11 55
13 to 17 4 20
rehabilitation is that the training tasks can be readily custom-
>17 5 25
designed through computer programming to meet the specific MACS Level 1 0 0
needs of the target users. Besides, the automation, availability and 2 5 25
accessibility of VR training systems enable repeated practice and 3 7 35
self-learning that can help enhancing the proficiency of specific 4 6 30
5 2 10
tasks without excessively increasing the workload of OTs. Besides,
quantitative data can be recorded automatically to analyze user
performance and gain insight into the training progress. For tasks approaches, the proposed platform can lower the barrier to wider
that involve the manipulation of potentially hazardous materials, adoption, thus enabling more people who suffer from upper limb
e.g. sharp objects or hot liquid, safety is guaranteed as training in disabilities to enjoy the benefits. While the haptic device
virtual environments are harmless, and errors could be undone.[6] employed in this study was less powerful (with a maximum out-
Virtual reality systems have been successfully implemented for put force of 3.3 N), it demonstrates the feasibility of producing
the rehabilitation of different target populations, e.g. stroke forces that are involved in various ADL tasks.
patients,[7–9] people with traumatic brain injury [10,11] or chil- To demonstrate the feasibility, the proposed VR system was
dren with developmental disorders.[12] They have been employed implemented for the training of three common tasks of ADL,
to assess the performance of upper limb motor abilities of namely, door unlocking, water pouring and meat cutting. For easy
patients through real-time quantitative measures.[13] In these reference, the three virtual tasks are referred to as Task 1, Task 2
studies, force feedbacks were provided in the training using haptic and Task 3 in this paper. A study was conducted to evaluate the
devices, in conjunction with audio and audio feedbacks to system by recruiting students with upper limb disabilities from a
enhance the realism of the virtual environments. Findings from special school. Details of the study and the findings will be dis-
previous studies consistently showed that users had a high level cussed in the following sections.
of satisfaction towards virtual rehabilitation systems. The advances
demonstrate the potential to adopt such systems as routine train-
ing approaches in occupational therapy.[14] Methods
However, the use of VR for the rehabilitation of ADL receives Participants
relatively less attention. One example is the Virtual Life Skills [15]
project, which attempted to facilitate people with learning disabil- Twenty students with upper limb disabilities were recruited from
ities to learn the knowledge and perform the tasks necessary for a special school through purposive sampling. Those who were
independent survival in the real world, e.g. handling money and visually impaired and the impairment could not be corrected with
dressing, by simulating them with a virtual city. Other examples visual aids; in an unstable medical conditions such as frequent
that leverage VR for the rehabilitation of ADL include the use of a seizure activities; suffered from complete sensory loss of upper
virtual kitchen for training meal preparation tasks, or the use of limbs; had severe arm pain, and manifested behavioral problems,
virtual supermarket for practicing shopping tasks,[16] for people were excluded. The study was approved by the Human Subjects
suffering from traumatic brain injury.[17] The emphasis of these Ethics Committee of the institution. Informed consents were
projects was to enhance the cognitive abilities, where generic obtained from the parents or guardians of all the subjects.
user interface devices like computer mouse or joystick were only Besides, two OTs responsible for the rehabilitation of the subjects
used, and the training of manual skills involved in ADL was there- were also recruited to participate in the study to perform profes-
fore not possible. sional assessment on the performance of the subjects and give
On the other hand, robotics has been used to develop haptic comments on the proposed system.
interface that is essential for the training of manual skills, e.g.
HapticMaster [18,19] and ARMin,[20,21] which are high-perform- Demographics
ance devices, at about human height, that can guide the impaired
arm or hand of users through a robotic arm. In particular, ARMin The demographic data of the 20 subjects is shown in Table 1.
is an exoskeleton robot also equipped with actuator to assist 60% of the subjects were male. The mean age is 16.65 years
shoulder movement. These devices can generate nominal forces (SD ¼ 4.57 years); the youngest subject was 7 years old and the
of 100 N [18] and torques in the range of 2 to 20 Nm.[20] eldest was 22 years old. In addition, the ability of the subjects in
HapticMaster was used for the training of ADL like grooming, handing objects in ADL was identified using the manual ability
drinking, eating and playing tic-tac-toe.[19] ADL tasks like cooking, classification system (MACS).[22] The MACS level, ranging from
cleaning and using a ticket machine were trained using ARMin, level 1 to level 5, was rated by the two OTs responsible for pro-
where the assist-as-needed strategy was adopted to provide guid- viding rehabilitative care to the subjects. The higher the MACS
ing forces when necessary.[20] level, the more severe the disability. In the study, 75% of the sub-
In this study, a VR rehabilitation system was proposed as a jects were at MSCS level 3 or above. On the other hand, the two
platform to facilitate the training of the manual skills in ADL in a OTs are both senior occupational therapists with over 20 years of
safe, versatile and flexible virtual environment. Unlike the robot- working experience in the rehabilitation of disabled students.
assisted rehabilitation approaches discussed above, the platform
was realized using a relatively low cost and handy desktop haptic
The virtual rehabilitation system
device to simulate force feedback interactively during virtual
rehabilitation. The device comes with an intuitive pen-like stylus The proposed system was developed using a desktop personal
and has a small footprint. Given the potential of robotics computer with an Intel Core i7–4770 3.4 GHz CPU, 16GB RAM and
 VIRTUAL ADL REHABILITATION WITH FORCE FEEDBACK 3

Figure 1. The computerized virtual training system for the rehabilitation of ADL: the complete system is shown on the left; the middle column shows the stylus of
the haptic device replaced with 3D-printed key (top), jar (middle), meat and knife (bottom) respectively for Task 1, Task 2 and Task 3; the corresponding virtual envi-
ronments as displayed on the screen are shown on the right.

Figure 2. Trajectories of hand movements during the three virtual tasks: door unlocking (left), water pouring (middle) and meat cutting (right). Bimanual hand move-
ments are shown in the third task, where a piece of virtual meat was being hold by the left hand and sawn by the right.

an NVidia GeForce GTX750Ti display card, running on the position and orientation of the key, jar, meat and knife were ren-
Microsoft Windows 7 operating system. The computer was con- dered interactively on the screen depending how the user maneu-
nected to a 22-inch Viewsonic VA2261 LED monitor, which was vered the tailor-made 3D printed handles that were mounted to
fixed on the desk top using a flexible mount to allow for easy and the haptic devices. In addition, the locality data of the haptic devi-
ergonomic adjustment. The computer was also connected to a ces were recorded during the process of virtual training, which
pair of SensAble Phantom Omni haptic devices. The complete sys- could be used to plot the trajectories of hand movements in 3D
tem was installed on an Ergotron WorkFit-C movable workstation, space. An example of the 3D plots for the three tasks is shown in
as shown in Figure 1. Figure 2.
The haptic devices employed in the study are intuitive 3D user
interface equipped with a pen-like stylus. It provides 6 degrees-of-
Instruments
freedom inputs (3 degrees respectively for the position and orien-
tation of the stylus endpoint) and 3 degrees-of-freedom force out- Four instruments were used to evaluate the computerized virtual
put. In the study, the stylus was detached and replaced by tailor- training system, including (i) the task-specific feedback question-
made handles fabricated using 3D printing technology. The 3D naire (TSFQ), (ii) the Borg scale of perceived exertion (BSPE), (iii)
printed handles included a key for Task 1, a jar for Task 2, and a the questionnaire on performance in simulated real tasks (QSRT),
piece of meat and a knife for Task 3, as shown in the middle col- and (iv) the computer system usability questionnaire (CSUQ). The
umn of Figure 1. 20 subjects were requested to respond to the first two instru-
The software of the system was developed using the C/Cþþ ments, whereas the OTs were asked to complete the last two.
programming language, with the OpenGL and OpenHaptics appli-
cation programming interfaces adopted to render the graphic and Task-specific feedback questionnaire
haptic display interactively in the 3D virtual environments. For The TSFQ is a standardized tool used to evaluate the level of enjoy-
Task 1, a door with door knob and a key were rendered; for Task ment, difficulty and the sense of control perceived by a user after
2, a jar, a glass and the water were displayed; for Task 3, a virtual performing a task in virtual environment.[14] The questionnaire
hand holding a piece of meat and a knife were shown. The virtual consisted of eight items, on a Likert scale from 1 (not at all) to 5
environments are shown in the right column of Figure 1. The (very much). In the study, the subjects were asked to rate their
4 V. C.-L. CHIANG ET AL.

Figure 4. Research protocol.

Figure 3. Simulated real tasks (left) versus the corresponding virtual tasks (right)
of the three ADL in this study: door unlocking (top), water pouring (middle) and Computer system usability questionnaire
meat cutting (bottom). The CSUQ was developed by the IBM Corporation to measure the
overall feeling when they were performing the three virtual tasks, usability of a computer system,[24] on a 7-point Likert scale from
and also the level of realism of the virtual environments they per- 1 (strongly agree) to 7 (strongly disagree). The questionnaire con-
ceived. Considering that some of the subjects were quite young, tains 19 items evaluating various aspects of user perception
emoticons associating with the numerical choices were supplied to towards the effectiveness, interface, expectations, and satisfaction
help them to better appreciate the extent of differences. of a computer system. At the end of the study, the OTs accompa-
nying the subjects during the study were asked to rate the usabil-
ity of the proposed training system using the CUSQ based on
Borg scale of perceived exertion
their professional judgment.
The BSPE measures the extent of exertion during physical exercise
or sport using an 11-point Likert scale, with 0 indicating no exer-
tion at all, to 10 indicating the maximal exertion.[23] In the study, Research design
the subjects were asked to rate the level of physical exertion they A pre-post test was conducted over a period of two months to
perceived during the virtual tasks. Similar to the TSFQ, emoticons evaluate the proposed virtual training system. The research proto-
associating with the numerical choices were supplied to assist the col is shown in Figure 4 and the procedures are described as
subjects in appreciating the differences. follows.

Questionnaire on performance in simulated real tasks Pre-test

The three tasks of ADL concerned in the study are conventionally In the pre-test, the baseline performance of the subjects in ADL
assessed in the special school in simulated real environments was assessed by asking them to perform the three above-men-
designed by the OTs. The term “simulated” was specifically used tioned simulated real tasks one by one. The processes were video-
here to make clear that the assessments were not conducted recorded with a digital camera. For door opening, the task was to
using the actual objects in real ADL due to safety and conveni- pick up the key, insert it into the key hole of the door knob, and
ence consideration. Hence, the door opening task was simulated then make a clockwise rotation to unlock and open the miniature
by asking students to open a customized miniature door put on door (see the top-left photo in Figure 3). For water pouring, the
top of a desk. Water pouring was simulated by pouring dyed task started by lifting a transparent jar of dyed water and pouring
water from a transparent jar into a glass put on a tray, whereas it into an empty glass, until all the water was removed from the
meat cutting was simulated by cutting a piece of therapy putty jar. The purpose of the dye was to allow the researcher and OTs
with a plastic knife. The simulated real tasks are shown in the left to observe the process more clearly (see the middle-left photo in
of Figure 3. Further details are given in the next section. Figure 3). For meat cutting, subjects were requested to hold with
In the study, the performance of the subjects in these three one hand a piece of therapy putty, in the form of a slab, and cut
simulated real tasks was assessed by the OTs, before and after the it with a plastic knife with the other hand by sawing back and
virtual training, in order to evaluate whether the proposed system forth until the putty was separated into two parts (see the bot-
can help improving the skills in reality. To this end, the QSRT was tom-left photo in Figure 3). The video recordings were then used
developed for the OTs to assess the performance of the subjects by the researchers to determine the time taken by each subject to
on the three simulated real tasks. For each task, the OTs would complete the individual tasks, and also reviewed by the OTs to
rate the performance of the subjects from four aspects, i.e., time evaluate the performance of the subjects using the QSRT.
taken to complete the task, the accuracy, completeness, and the
overall competency of the subject. The QSRT consisted of 12 Intervention
items, and adopted a 7-point Likert scale, from 1 (strongly dis- After the pre-test, the subjects received four sessions of virtual
agree) to 7 (strongly agree). training using the proposed rehabilitation system, two sessions per
 VIRTUAL ADL REHABILITATION WITH FORCE FEEDBACK 5

week in four separate days. In each session, a subject used the sys- photo in Figure 3). Using the two 3D-printed models, the subject
tem to perform the three tasks of ADL. For door opening, the sub- maneuvered the virtual knife to cut a piece of virtual meat. The vir-
ject used the 3D-printed key attached to the haptic device to tual task was programmed to contain three steps. In the first step,
control the virtual key (see the top-right photo in Figure 3). The vir- the subjects used the left hand to move the virtual meat to a target
tual key was to be moved towards the door knob and inserted into position on the desk and pressed it downwards. Next, a guideline
the keyhole. Feedback forces were generated and felt by the hand was displayed near the right edge of the virtual meat, as a visual
of the subject whenever the key collided with the door or the door cue to signify the subject to move the virtual knife towards the
knob in the virtual environments. After the key was inserted and guideline and align accordingly. In the third step, the subject used
turned 90 degrees clockwise, an audio clip of the sound of door the virtual knife to saw, along the guideline, back and forth hori-
opening was played and the task was completed. zontally on the meat for 10 times to complete the task. With feed-
For water pouring, the 3D-printed jar was attached to the hap- back forces generated by the haptic device, the subject could feel
tic device as the user interface. The subject held the ear of the jar the knife-meat friction while performing the sawing motion. On-
with one hand and maneuvered it to control the water-filled vir- screen message was displayed when the virtual knife was too close
tual jar displayed on the screen, as if they were holding a real jar to the virtual left hand. An animated audio clip was played when
(see the middle-right photo in Figure 3). Gravity was simulated the knife was sawing on the virtual meat.
such that the subject could feel the weight of the water inside
the virtual jar through the haptic device, which gradually Post-test
decreased as the water flowed out of the jar. In the virtual envir- After the virtual training, the subjects were asked to demonstrate
onment, an empty glass was put on a desk and the subject was their skills in the three simulated real tasks as they had performed
required to move the mouth of the jar on top of the glass, and in the pre-test. Similar to the pre-test, the processes were
then tilt the jar to pour the water into the glass. The amount of recorded by videos, which were subsequently reviewed by the
water successfully poured into the virtual glass was recorded auto- researchers to determine the time spent on each of the three
matically by the system as a measure of accuracy. Interactive simulated real tasks, and by the two OTs to evaluate the perform-
audio cues were provided such that the sound of water pouring ance of the subjects using the QRST. Finally, the subjects were
into the glass was played when the task was being performed also asked to fill in the TSFQ and BSPE, whereas the OTs were
successfully; otherwise, the sound of water spilling over the table asked to complete the CSUQ.
was played instead.
For meat cutting, a pair of haptic devices was employed for sim- Results
ulating the bi-manual operations required in the task. The haptic
Virtual training
stylus of the device on the left was replaced with a 3D-printed slab
to represent a piece of meat, whereas the one on the right was The performance of the subjects during the four practice sessions
replaced with a 3D-printed handle of a knife (see the bottom-right is shown in Figure 5 and the performance in the first and the

Figure 5. Performance of virtual training during the four sessions: the average complete time of Task 1 (top left), Task 2 (bottom left) and Task 3 (top right), and the
average amount of water poured into the virtual glass in Task 2 (bottom right).
6 V. C.-L. CHIANG ET AL.

Table 2. Performance between the first and the fourth session of the virtual Table 3. Performance between the pre-test and post-test on the simulated real
tasks. tasks.
Percentage of water filled Completion time (s)
Completion time (s) (%)
Pre-test Post-test
Session 1 Session 4 Session 1 Session 4
Task Mean SD Mean SD p Values
Task Mean SD Mean SD Mean SD Mean SD p Values Task 1 18.56 15.60 14.00 13.43 0.181
Task 1 110.09 78.54 46.53 31.83 – – – – 0.001 Task 2 14.44 7.76 12.74 5.73 0.225
Task 2 18.73 7.76 13.29 4.15 – – – – 0.007 Task 3 26.50 23.07 20.44 12.44 0.167
Task 2 – – – – 79.71 12.77 85.56 7.95 0.051
Task 3 73.58 54.77 34.61 19.30 – – – – 0.004
Table 4. TSFQ Scores rated by the subjects.
No. Item Mean SD 95% CI
1 Feeling of enjoyment 4.53 0.90 4.09–4.96
2 Sense of being in environment 4.37 0.90 3.94–4.80
3 Successful feeling 4.42 0.77 4.05–4.79
4 Control of the system 4.37 0.96 3.91–4.83
5 Perception of a realistic environment 3.74 1.37 3.08–4.40
6 Comprehension of computer feedback 4.89 0.32 4.74–5.00
7 Level of comfort 2.26 1.91 1.34–3.18
8 Perception of easiness 2.42 1.84 1.54–3.31
Total score 31.00 4.85 28.66–33.34

Table 5. CSUQ Scores rated by the two OTs.

No. Item OT1 OT2 Mean Difference
1 Easiness of use 2 3 2.5 
2 Simplicity of system 2 1 1.5 
3 Effectiveness in task completion 1 4 2.5 
Figure 6. Performance of the simulated real tasks in the pre-post tests. 
4 Completion speed 3 2 2.5
5 Efficiency in task completion 1 3 2 
fourth session is compared in Table 2. In terms of task completion 6 Level of comfort 2 2 2 –
7 Easiness of learning to use 3 1 2 
time, it can be observed from the box-plots that there was appar- 8 Improvements in productivity 1 4 2.5 
ently a decreasing trend in the median value over the four ses- 9 Messages on fixation of errors 3 3 3 –
sions for the three virtual tasks. The decrease was more 10 Recovery rate of system 3 3 3 –
11 Explicitness of information 4 3 3.5 
conspicuous for Task 1 and Task 3. Comparing the performance in 
12 Accessibility of information 2 3 2.5
the first and last session, the results of the t-tests indicated the 13 Easiness in understanding of information 3 3 3 –
decrease was statistically significant, with p values equal to 0.001, 14 Usefulness of information 3 2 2.5 
0.007 and 0.004 for the three tasks, respectively. Besides, regard- 15 Organization of information 2 4 3 
ing the percentage of water successfully poured into the virtual 16 System interface 2 2 2 –
17 Satisfaction towards system interface 2 1 1.5 
glass in Task 2, an increasing trend over the four sessions 18 Fulfilling expectations on functions 3 5 4 
was revealed from the box-plots. The result of the t-test yielded a 19 Overall satisfaction 3 2 2.5 
p value of 0.051 for the observed increase in the amount of water The symbols ,   and  denote a difference of one, two and three points,
filled between the first and the fourth session. respectively, between the scores rated by the two OTs for an item; with dash
indicating no difference.

Simulated real tasks

CI ¼ 0.18–1.92), with “0” indicating no exertion and “10” maximal
The performance of the subjects in the simulated real tasks before exertion.
and after the virtual training is shown in Figure 6 and the per- The proposed virtual training system was evaluated by the two
formance in the pre-test and post-test is compared in Table 3. OTs who supervised the four practice sessions using the CSUQ.
The average completion time for three tasks was found to The results are shown in Table 5. Among the 19 items, the mean
decrease in the post-test but not statistically significant, with scores of 15 items were between 2 and 3 (“1” indicating strongly
p values equal to 0.181, 0.225 and 0.167 for Tasks 1, 2 and 3, agree and “7” strongly disagree). The items regarding “Simplicity of
respectively. system” (item no. 2) and “Satisfaction towards system interface”
(item no. 17) scored the best (1.5) while the item “Fulfilling
expectation on function” (item no. 18) scored less favorably (4.0).
User evaluation
The mean overall satisfaction score is 2.5. Furthermore, the differ-
The statistical results of TSFQ, i.e. mean score, SD and 95% confi- ence in score of 13 items was 1 point, indicating that the rating
dence interval (CI), are summarized in Table 4. The mean total of the two OTs were quite consistent. However, they disagreed in
score was 31 (SD ¼ 4.85; 95% CI ¼ 28.66–33.34) out of 40. The some way on items concerning “Effectiveness in task completion”
mean scores in 5 of the 8 items were above 4 (“1 indicating (item no. 3) and “Improvements in productivity” (item no. 8), with
strongly disagree and “5” strongly agree), whereas the mean scores a difference of 3 points.
of the questions regarding “Level of comfort” (item no. 7) and The two OTs rated the degree of improvement in the perform-
“Perception of easiness” (item no. 8) were lower than 3. On the ance of the subjects in the three simulated real tasks by reviewing
other hand, the mean BSPE score was 1.05 (SD ¼ 1.85; 95% videos taken in the pre-tests and post-tests. As shown in Table 6,
 VIRTUAL ADL REHABILITATION WITH FORCE FEEDBACK 7

the results are positive and the mean scores of the individual Table 6. QSRT Scores rated by the two OTs after examining the videos taken at
items are all within 5 and 6 (“1” indicating strongly disagree and the pre-tests and post-tests for all the subjects.
“7” strongly agree). Task 1 Task 2 Task 3
No. Item Mean SD Mean SD Mean SD
Discussion 1 Improvement in completion time 5.25 1.62 5.30 1.34 5.50 1.47
2 Improvement in accuracy 5.45 1.39 5.65 1.46 5.60 1.60
In this study, it is hypothesized that the proposed virtual training 3 Improvement in completeness 5.50 1.47 5.60 1.57 5.65 1.60
approach can improve the learning curve of people with upper 4 Improvement in competency 5.45 1.54 5.70 1.56 5.70 1.56
limb disability in the acquisition of skills of ADL. The results sug-
gest that the approach has the potential to achieve this goal. In
the virtual training, the subjects were able to complete the three
ADL tasks faster in the fourth practice session as compared to the in the virtual environment by using shadows, lighting or other
first session (Table 2). For the water pouring task, there was a rendering effects in computer graphics. On the other hand, some
modest reduction in completion time. This is due to the fact that, subjects reflected that since Task 3 was a bi-manual operation
among the three virtual tasks, the design and user interface of that required a higher level of coordination of both hands with
water pouring were considered to be most intuitive by both the the eyes, they found it relatively less comfortable and easy to per-
subjects and the OTs. Provision of instructions to the subjects was form it than the single-handed tasks.
almost unnecessary. The subjects could all immerse into the vir- The positive results of the CSUQ scores suggest that the tow
tual reality easily and quickly, naturally grasping the 3D printed OTs, from the perspectives of the subjects, were satisfied with the
jar and pour water into the glass shown on the computer screen proposed computer-based virtual training system (Table 5). For
as if they were doing so in reality. Indeed, the action of water the items that were rated relatively low, i.e. “Explicitness of
pouring was also relatively simple and involved less dexterous
information” (item no. 11) and “Fulfilling expectations on
manipulations when compared with the other two tasks. As a
functions” (item no. 18), while the OTs appreciated the efforts
result, they could already complete the task using a short amount
expended to mimic the setting and situation of the real ADL tasks,
of time in the first session and the task completion time did not
they recommended further improvements to enrich the informa-
improve much further in the following sessions. Similar situation
was also observed from the percentage of water successfully tion and functions provided by the system and their manifesta-
poured into the virtual glass, which increased moderately by tions. For example, the door could be made to open after
about 10% over the four training sessions. unlocking, sounds of collisions between key and door knob could
Regarding the pre-post tests on the simulated real tasks, while be simulated. Wider variety of instructions, motivation messages
there was a decrease in completion time after the virtual training, or alerts, presented using texts, graphics or audio speech com-
the difference was not significant (Table 3). This could be mands, could be helpful cues to guide the users. Also, because of
explained by the limitation that the virtual environments and user complexity in computer programming, Task 3 was divided into
interface of the computer-based training were not exactly identi- three separate steps to simply the design, and the subjects were
cal to the setting of that in the simulated real tasks. For example, constrained to make linear sawing motion only in the last step. It
the real key used for door opening was shorter than the 3D- was suggested that the system could be equipped with functions
printed key; the tactile feeling of cutting therapy putty was differ- to allow users to make suboptimal motions, which can then be
ent from that rendered in the virtual task. Some adaptation was detected to produce warring signals.
required to transfer the manual skills and experience gained from The OTs also observed the issue that some of the subjects
the virtual reality to the simulated real tasks. Also, the number of could not “immerse” into the virtual environments while perform-
virtual training sessions may need to be increased in order to ing the virtual tasks. During the virtual training of Task 3, they
observe a more significant improvement in performance. looked alternately at the 3D-printed handles attached to the hap-
Nevertheless, the subjects were mentally better prepared for the tic devices and the virtual objects displayed on the screen. The
real ADL after going through the four sessions of virtual training. frequent change in gaze direction would indeed adversely affect
This is also supported by the positive results obtained from the the performance of the subjects and the training effectiveness. In
QSRT which will be discussed in the following text. some cases, a few subjects even thought that they would use the
In fact, the subjects were quite satisfied with the proposed 3D-printed knife model to cut the 3D-printed “meat” model in
training system, as shown by the TSFQ scores (Table 4). However,
reality, instead of performing the actions in the virtual environ-
the “Level of comfort” (item no. 7) and “Perception of easiness”
ment. To circumvent this situation, the setup will be designed to
(item no. 8) were rated relatively low, which, from the comments
cover the haptic user interface and the 3D-printed handed so that
of the subjects, were due to the settings of Task 1 and Task 3. In
the users can focus on the virtual environments displayed on the
Task 1, they found it difficult to aim the virtual key at the keyhole.
screen. In the meantime, virtual hands will be rendered on the
Some even considered it more difficult than the real task.
Although the virtual environment was rendered with 3D objects, screen to serve as proxy of the real hands (currently only the
stereoscopic effect was not available and thereby leading to the hand holding the knife was rendered in Task 3), so as to improve
absence of depth cues. Without depth information, it was indeed the sense of presence of the users in the virtual environments
difficult to judge the relative position between the key and the and raise the level of immersion.
keyhole, even for able-bodied people. While the incorporation of As discussed previously, the pre-post test results of the com-
stereoscopic vision in the virtual training is technically possible, it pletion time of the simulated real tasks before and after the vir-
may turn out to be creating confusion, if not an additional hurdle, tual training did not arrive at conclusive findings. However, the
to the subjects who would need to wear a pair of goggles and responses of the OTs to the QRST, based on their judgment by
adapt to the artificially generated stereoscopic 3D environment. examining the videos recorded during the pre-tests and post-tests,
This is particularly non-trivial for those with multiple disabilities, as show that they agreed that the subjects had improved their per-
cautioned by the OTs. Nevertheless, depth cues can be provided formance of all the three simulated real tasks after the virtual
8 V. C.-L. CHIANG ET AL.

training, in terms of completion time, accuracy, completeness and Available from: http://www.statistics.gov.hk/pub/
competency. B11301482008XXXXB0100.pdf
One attractive feature of using virtual reality and haptic devices [3] French B, Leathley M, Sutton C, et al. A systematic review
for rehabilitation is the automatic recording of the movements of repetitive functional task practice with modeling of
made by the users, which allows for objective performance assess- resource use, costs and effectiveness. Health Technol
ment in terms of quantitative metrics. In addition to the task com- Assess 2008;12:iii, ix–x, 1–117.
pletion time measured in all the three virtual tasks and the [4] Ahl LE, Johansson E, Granat T, et al. Functional therapy for
amount of water filled in Task 2, other metrics gauging the accur- children with cerebral palsy: an ecological approach. Dev
acy and dexterity in performing the tasks could be included to Med Child Neurol 2005;47:613–619.
conduct a more comprehensive evaluation. For example, as shown [5] Blundell SW, Shepherd RB, Dean CM, et al. Functional
in Figure 2, the proposed system was able to record the position strength training in cerebral palsy: a pilot study of a group
of the haptic stylus, which made possible the analysis of hand circuit training class for children aged 4-8 years. Clin
movement trajectories. Kinematic parameters such as trajectory Rehabil 2003;17:48–57.
straightness, mean distance from target, path length and jerkiness [6] Rizzo AS, Kim GJ. A SWOT analysis of the field of
can be estimated to gain insights into the degree of paresis or virtual reality rehabilitation and therapy. Presence
loss of fractionated movement.[25] Besides, other parameters like 2005;14:119–146.
speed and acceleration of hand movements and the amount of [7] Broeren J, Dixon M, Sunnerhagen KS, et al. Rehabilitation
forces applied, as well as the number of collisions between the after stroke using virtual reality, haptics (force feedback)
key and the door knob or the door in Task 1 and the number of and telemedicine. Stud Health Technol Inform 2006;124:
times that the hand holding the knife is too close to the hand 51–56.
holding the meat in Task 3 can also be logged. Further study will [8] Henderson A, Korner-Bitensky N, Levin M. Virtual reality in
be conducted using these quantitative parameters to perform stroke rehabilitation: a systematic review of its effectiveness
kinematic assessment in the rehabilitation of ADL in the virtual for upper limb motor recovery. Top Stroke Rehabil
training. 2007;14:52–61.
[9] Sampson M, Shau YW, King MJ. Bilateral upper limb trainer
with virtual reality for post-stroke rehabilitation: case series
Conclusion report. Disabil Rehabil Assist Technol 2012;7:55–62.
The present study provides insights into the use of VR technology [10] Mumford N, Duckworth J, Thomas PR, et al. Upper-limb vir-
for the teaching and learning of ADL for people with upper limb tual rehabilitation for traumatic brain injury: a preliminary
disability. The positive results suggest that the proposed com- within-group evaluation of the elements system. Brain Inj
puter-based training system has the potential to be used as a tool 2012;26:166–176.
to complement the existing approaches in occupational therapy. [11] Levin MF, Weiss PL, Keshner EA. Emergence of virtual real-
Invaluable comments and feedback were collected from both the ity as a tool for upper limb rehabilitation: incorporation of
subjects and the OTs, based on which further effort will be made motor control and motor learning principles. Phys Ther
2015;95:415–425.
to improve the current system. In addition, the current study was
[12] Takahashi Y, Ito Y, Inoue K, et al. Haptic device system for
a pre-post test with 20 subjects all received the virtual training
upper limb and cognitive rehabilitation — application for
with a relatively small number of practice sessions. A randomized-
development disorder children. In: El Saddik A, editor.
controlled trial with a larger sample size will be conducted to
Haptics rendering and applications. InTech; 2012. doi:
investigate the performance of the subjects with and without the
10.5772/25450.
virtual training. Experiment with different training dosage, strati-
[13] Bardorfer A, Munih M, Zupan A, et al. Upper limb motion
fied by the MACS level, will also be administered to study the
analysis using haptic interface. IEEE/ASME Trans
amount of training required for people of different degrees of dis-
Mechatronics 2001;6:253–260.
ability to yield the optimal rehabilitation effect. Furthermore, the
[14] Crosbie JH, Lennon S, McNeill MD, et al. Virtual reality in
system will be extended for the training of other ADL.
the rehabilitation of the upper limb after stroke: the user's
perspective. Cyberpsychol Behav 2006;9:137–141.
Disclosure statement [15] Cobb SVG, Neale HR, Reynolds H. Evaluation of Virtual
Learning Environments. The 2nd Euro. Conf. Disability,
The authors report no conflicts of interest. € vde, Sweden:
Virtual Reality & Assoc. Tech (ECDVRAT); Sko
The University of Reading; 1998.
Funding [16] Lee J, Ku J, Cho W, et al. A virtual reality system for the
assessment and rehabilitation of the activities of daily liv-
Hong Kong Polytechnic University, [G-UC93, G-YBKX], Quality
ing. Cyberpsychol Behav 2003;6:383–388.
Education Fund of the Hong Kong Education Bureau [2011/0162].
[17] Zhang L, Abreu BC, Seale GS, et al. A virtual reality environ-
ment for evaluation of a daily living skill in brain injury
References rehabilitation: reliability and validity. Arch Phys Med
Rehabil 2003;84:1118–1124.
[1] Birth Defects [Internet]. Centers for Disease Control and [18] Van der Linde RQ, Lammertse P, Frederiksen E, et al. The
Prevention. [updated 2014 July 21; cited 2015 July 21]. HapticMaster, a new high-performance haptic interface. In:
Available from: http://www.cdc.gov/ncbddd/birthdefects/ul- Proceedings of the Eurohaptics Conference, 8–10 July 2002,
limbreductiondefects.html Edinburgh, UK. p. 1–5.
[2] Persons with disabilities and chronic diseases [Internet]. [19] Johnson MJ, Wisneski KJ, Anderson J, et al. Development of
Census and Statistics Department Hong Kong. 2008 July 22. ADLER: The Activities of Daily Living Exercise Robot. The
In Special Topics Report No. 48. [cited 2015 July 22]. First IEEE/RAS-EMBS International Conference on Biomedical
 VIRTUAL ADL REHABILITATION WITH FORCE FEEDBACK 9

Robotics and Biomechatronics, 2006. BioRob 2006; 2006 [22] Eliasson AC, Krumlinde-Sundholm L, Rosblad B, et al. The
Feb 20–22; Pisa, Italy. Manual Ability Classification System (MACS) for children
[20] Guidali M, Duschau-Wicke A, Broggi S, et al. A robotic sys- with cerebral palsy: scale development and evidence of val-
tem to train activities of daily living in a virtual environ- idity and reliability. Dev Med Child Neurol 2006;48:549–554.
ment. Medical Biol Eng Comput 2011;49:1213–1223. [23] Borg G. Borg's perceived exertion and pain scale.
[21] Nef T, Guidali M, Klamroth-Marganska V, et al. ARMin — Champaign, IL: Human Kinetics; 1998.
Exoskeleton robot for stroke rehabilitation. In: Do €ssel O, [24] Lewis JR. IBM computer usability satisfaction question-
Schlegel WC, editors. World Congress on Medical Physics naires: Psychometric evaluation and instructions for use. Int
and Biomedical Engineering, September 7–12, 2009, J Human-Computer Interact. 1995;7:57–78.
Munich, Germany: Vol. 25/9 Neuroengineering, Neural sys- [25] Nordin N, Xie SQ, Wu €nsche B. Assessment of movement
tems, rehabilitation and prosthetics. Berlin, Heidelberg: quality in robot- assisted upper limb rehabilitation after
Springer Berlin Heidelberg; 2009. p. 127–130. stroke: a review. J NeuroEng Rehabil 2014;11:1–23.