-
Design, Development, and Evaluation of an Interactive Personalized Social Robot to Monitor and Coach Post-Stroke Rehabilitation Exercises
Authors:
Min Hun Lee,
Daniel P. Siewiorek,
Asim Smailagic,
Alexandre Bernardino,
Sergi Bermúdez i Badia
Abstract:
Socially assistive robots are increasingly being explored to improve the engagement of older adults and people with disability in health and well-being-related exercises. However, even if people have various physical conditions, most prior work on social robot exercise coaching systems has utilized generic, predefined feedback. The deployment of these systems still remains a challenge. In this pap…
▽ More
Socially assistive robots are increasingly being explored to improve the engagement of older adults and people with disability in health and well-being-related exercises. However, even if people have various physical conditions, most prior work on social robot exercise coaching systems has utilized generic, predefined feedback. The deployment of these systems still remains a challenge. In this paper, we present our work of iteratively engaging therapists and post-stroke survivors to design, develop, and evaluate a social robot exercise coaching system for personalized rehabilitation. Through interviews with therapists, we designed how this system interacts with the user and then developed an interactive social robot exercise coaching system. This system integrates a neural network model with a rule-based model to automatically monitor and assess patients' rehabilitation exercises and can be tuned with individual patient's data to generate real-time, personalized corrective feedback for improvement. With the dataset of rehabilitation exercises from 15 post-stroke survivors, we demonstrated our system significantly improves its performance to assess patients' exercises while tuning with held-out patient's data. In addition, our real-world evaluation study showed that our system can adapt to new participants and achieved 0.81 average performance to assess their exercises, which is comparable to the experts' agreement level. We further discuss the potential benefits and limitations of our system in practice.
△ Less
Submitted 12 May, 2023;
originally announced May 2023.
-
Enabling AI and Robotic Coaches for Physical Rehabilitation Therapy: Iterative Design and Evaluation with Therapists and Post-Stroke Survivors
Authors:
Min Hun Lee,
Daniel P. Siewiorek,
Asim Smailagic,
Alexandre Bernardino,
Sergi Bermúdez i Badia
Abstract:
Artificial intelligence (AI) and robotic coaches promise the improved engagement of patients on rehabilitation exercises through social interaction. While previous work explored the potential of automatically monitoring exercises for AI and robotic coaches, the deployment of these systems remains a challenge. Previous work described the lack of involving stakeholders to design such functionalities…
▽ More
Artificial intelligence (AI) and robotic coaches promise the improved engagement of patients on rehabilitation exercises through social interaction. While previous work explored the potential of automatically monitoring exercises for AI and robotic coaches, the deployment of these systems remains a challenge. Previous work described the lack of involving stakeholders to design such functionalities as one of the major causes. In this paper, we present our efforts on eliciting the detailed design specifications on how AI and robotic coaches could interact with and guide patient's exercises in an effective and acceptable way with four therapists and five post-stroke survivors. Through iterative questionnaires and interviews, we found that both post-stroke survivors and therapists appreciated the potential benefits of AI and robotic coaches to achieve more systematic management and improve their self-efficacy and motivation on rehabilitation therapy. In addition, our evaluation sheds light on several practical concerns (e.g. a possible difficulty with the interaction for people with cognitive impairment, system failures, etc.). We discuss the value of early involvement of stakeholders and interactive techniques that complement system failures, but also support a personalized therapy session for the better deployment of AI and robotic exercise coaches.
△ Less
Submitted 31 December, 2022; v1 submitted 15 June, 2021;
originally announced June 2021.
-
Designing Personalized Interaction of a Socially Assistive Robot for Stroke Rehabilitation Therapy
Authors:
Min Hun Lee,
Daniel P. Siewiorek,
Asim Smailagic,
Alexandre Bernardino,
Sergi Bermúdez i Badia
Abstract:
The research of a socially assistive robot has a potential to augment and assist physical therapy sessions for patients with neurological and musculoskeletal problems (e.g. stroke). During a physical therapy session, generating personalized feedback is critical to improve patient's engagement. However, prior work on socially assistive robotics for physical therapy has mainly utilized pre-defined c…
▽ More
The research of a socially assistive robot has a potential to augment and assist physical therapy sessions for patients with neurological and musculoskeletal problems (e.g. stroke). During a physical therapy session, generating personalized feedback is critical to improve patient's engagement. However, prior work on socially assistive robotics for physical therapy has mainly utilized pre-defined corrective feedback even if patients have various physical and functional abilities. This paper presents an interactive approach of a socially assistive robot that can dynamically select kinematic features of assessment on individual patient's exercises to predict the quality of motion and provide patient-specific corrective feedback for personalized interaction of a robot exercise coach.
△ Less
Submitted 13 July, 2020;
originally announced July 2020.
-
Trans-NIH/Interagency Workshop on the Use and Development of Assistive Technology for the Aging Population and People with Chronic Disabilities
Authors:
Elizabeth Mynatt,
Alice Borrelli,
Sara Czaja,
Erin Iturriaga,
Jeff Kaye,
Wendy Nilsen,
Dan Siewiorek,
John Stankovic
Abstract:
The first baby boomer born in 1946 turned 65 in 2011 and the last baby boomer will turn 65 in 2029. By then, the total U.S. population over 65 is projected to be 71.5 million (compared with 44.6 million in 2013). The current (2014) median cost of a nursing home is $226 a day ($82,490 per year), while assisted living is $3500 a month ($42,000 per year). While the elderly population continues to get…
▽ More
The first baby boomer born in 1946 turned 65 in 2011 and the last baby boomer will turn 65 in 2029. By then, the total U.S. population over 65 is projected to be 71.5 million (compared with 44.6 million in 2013). The current (2014) median cost of a nursing home is $226 a day ($82,490 per year), while assisted living is $3500 a month ($42,000 per year). While the elderly population continues to get larger and costs will continue to rise, nearly ninety percent (90%) of people want to grow old in their own home and community and remain out of the hospital, nursing home or other institutional setting. New technologies could potentially allow older adults and people with disabilities to remain in their homes longer, reduce health care costs and enhance the quality of life.
In summary, there is a need for a new generation of research that addresses the complexity of supporting the quality of life and independence of a vast, diverse, and aging population. While there are common themes and needs in this research that we describe shortly, we must start by recognizing that there is more than one needed path and approach to meet these diverse needs. One path includes the tight integration of chronic disease management in the home with existing acute healthcare systems. Another path embraces comprehensive home health for improving nutrition and social connectedness while combating physical, cognitive and psychological ailments. Yet another path emphasizes wellness, consumer technologies and removing basic barriers to meaningful community participation. These paths will intersect in interesting ways for individuals, families, healthcare providers, and communities. However research is critically needed to illuminate these paths and to make measurable strides in our care and support for over 15% of our nation's citizens.
△ Less
Submitted 27 April, 2020;
originally announced April 2020.
-
Opportunities of a Machine Learning-based Decision Support System for Stroke Rehabilitation Assessment
Authors:
Min Hun Lee,
Daniel P. Siewiorek,
Asim Smailagic,
Alexandre Bernardino,
Sergi Bermúdez i Badia
Abstract:
Rehabilitation assessment is critical to determine an adequate intervention for a patient. However, the current practices of assessment mainly rely on therapist's experience, and assessment is infrequently executed due to the limited availability of a therapist. In this paper, we identified the needs of therapists to assess patient's functional abilities (e.g. alternative perspective on assessment…
▽ More
Rehabilitation assessment is critical to determine an adequate intervention for a patient. However, the current practices of assessment mainly rely on therapist's experience, and assessment is infrequently executed due to the limited availability of a therapist. In this paper, we identified the needs of therapists to assess patient's functional abilities (e.g. alternative perspective on assessment with quantitative information on patient's exercise motions). As a result, we developed an intelligent decision support system that can identify salient features of assessment using reinforcement learning to assess the quality of motion and summarize patient specific analysis. We evaluated this system with seven therapists using the dataset from 15 patient performing three exercises. The evaluation demonstrates that our system is preferred over a traditional system without analysis while presenting more useful information and significantly increasing the agreement over therapists' evaluation from 0.6600 to 0.7108 F1-scores ($p <0.05$). We discuss the importance of presenting contextually relevant and salient information and adaptation to develop a human and machine collaborative decision making system.
△ Less
Submitted 2 March, 2020; v1 submitted 27 February, 2020;
originally announced February 2020.
-
Attention Filtering for Multi-person Spatiotemporal Action Detection on Deep Two-Stream CNN Architectures
Authors:
João Antunes,
Pedro Abreu,
Alexandre Bernardino,
Asim Smailagic,
Daniel Siewiorek
Abstract:
Action detection and recognition tasks have been the target of much focus in the computer vision community due to their many applications, namely, security, robotics and recommendation systems. Recently, datasets like AVA, provide multi-person, multi-label, spatiotemporal action detection and recognition challenges. Being unable to discern which portions of the input to use for classification is a…
▽ More
Action detection and recognition tasks have been the target of much focus in the computer vision community due to their many applications, namely, security, robotics and recommendation systems. Recently, datasets like AVA, provide multi-person, multi-label, spatiotemporal action detection and recognition challenges. Being unable to discern which portions of the input to use for classification is a limitation of two-stream CNN approaches, once the vision task involves several people with several labels. We address this limitation and improve the state-of-the-art performance of two-stream CNNs. In this paper we present four contributions: our fovea attention filtering that highlights targets for classification without discarding background; a generalized binary loss function designed for the AVA dataset; miniAVA, a partition of AVA that maintains temporal continuity and class distribution with only one tenth of the dataset size; and ablation studies on alternative attention filters. Our method, using fovea attention filtering and our generalized binary loss, achieves a relative video mAP improvement of 20% over the two-stream baseline in AVA, and is competitive with the state-of-the-art in the UCF101-24. We also show a relative video mAP improvement of 12.6% when using our generalized binary loss over the standard sum-of-sigmoids.
△ Less
Submitted 21 July, 2019;
originally announced July 2019.
-
Weighted Multisource Tradaboost
Authors:
João Antunes,
Alexandre Bernardino,
Asim Smailagic,
Daniel Siewiorek
Abstract:
In this paper we propose an improved method for transfer learning that takes into account the balance between target and source data. This method builds on the state-of-the-art Multisource Tradaboost, but weighs the importance of each datapoint taking into account the amount of target and source data available. A comparative study is then presented exposing the performance of four transfer learnin…
▽ More
In this paper we propose an improved method for transfer learning that takes into account the balance between target and source data. This method builds on the state-of-the-art Multisource Tradaboost, but weighs the importance of each datapoint taking into account the amount of target and source data available. A comparative study is then presented exposing the performance of four transfer learning methods as well as the proposed Weighted Multisource Tradaboost. The experimental results show that the proposed method is able to outperform the base method as the number of target samples increase. These results are promising in the sense that source-target ratio weighing may be a path to improve current methods of transfer learning. However, against the asymptotic conjecture, all transfer learning methods tested in this work get outperformed by a no-transfer SVM for large number on target samples.
△ Less
Submitted 26 March, 2019;
originally announced March 2019.
