Domain-Specific Cognitive Prosthesis for Face Memory and Recognition
Abstract
:1. Introduction
2. Materials and Methods
2.1. Subjects
2.2. Study Framework and Evaluation Criteria
- Efficacy: (must-have) Improved face recognition accuracy and reaction time >50% compared with baseline; (nice-to-have) improved face recognition accuracy and reaction time >75% compared with baseline.
- Safety: (must-have) No serious adverse event.
- Usability: (must-have) Systemic usability scale ≥65; (nice to have) systemic usability scale ≥85.
- Performance: (must-have) Precision ≥90% and recall ≥90% with average training image set <10 images; (nice-to-have) precision ≥95% and recall ≥95% with average training image set <10 images.
2.3. The Cognitive Prosthesis Device
2.4. Study Procedures
2.5. Efficacy Evaluation
2.5.1. Acquaintance Face Recognition Test
2.5.2. Taiwan Face Memory Test, Quality of Life, and Other Neuropsychological Assessments
2.6. Safety, Usability, and Machine Learning Performance Evaluation
2.7. Functional Study to Clarify the Mechanism of Action
2.8. Statistical Analysis
3. Results
3.1. Subject Profiles
3.2. Device Efficacy Assessment
3.2.1. Taiwan Face Memory Test
- Stage 1: 72% (13/18) (healthy subjects: 94 ± 5%).
- Stage 2: 50% (15/30) (healthy subjects: 72 ± 16%).
- Stage 3: 42% (10/24) (healthy subjects: 71 ± 15%).
- Total: 52.78% (healthy subjects: 77 ± 12%).
3.2.2. Quality of Life and Other Neuropsychological Assessments
3.3. Safety and Usability Assessment
3.4. Machine Learning Performance Assessment
3.5. Functional MRI Study Results
4. Discussion
4.1. Face Memory and Recognition in Healthy Subjects
4.2. Efficacy Criteria
4.2.1. Acquaintance Face Recognition Test and Taiwan Face Memory Test
4.2.2. Quality of Life and Other Neuropsychological Assessments
4.3. Safety Criteria
4.4. Usability Criteria
4.5. Machine Learning Performance Criteria
4.6. Mechanism of Action of Cognitive Prosthesis Based on Functional Study
4.7. Planned Changes to the Prototype
4.8. Limitations and Future Directions
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Sachdev, P.S.; Blacker, D.; Blazer, D.G.; Ganguli, M.; Jeste, D.V.; Paulsen, J.S.; Petersen, R.C. Classifying neurocognitive disorders: The DSM-5 approach. Nat. Rev. Neurol. 2014, 10, 634–642. [Google Scholar] [CrossRef]
- Langa, K.M.; Levine, D.A. The diagnosis and management of mild cognitive impairment: A clinical review. JAMA 2014, 312, 2551–2561. [Google Scholar] [CrossRef]
- Sun, J.H.; Tan, L.; Yu, J.T. Post-stroke cognitive impairment: Epidemiology, mechanisms and management. Ann. Transl. Med. 2014, 2, 80. [Google Scholar] [CrossRef]
- de Freitas Cardoso, M.G.; Faleiro, R.M.; de Paula, J.J.; Kummer, A.; Caramelli, P.; Teixeira, A.L.; de Souza, L.C.; Miranda, A.S. Cognitive Impairment Following Acute Mild Traumatic Brain Injury. Front. Neurol. 2019, 10, 198. [Google Scholar] [CrossRef]
- Walter, E.J.; Carraretto, M. The neurological and cognitive consequences of hyperthermia. Crit. Care 2016, 20, 199. [Google Scholar] [CrossRef]
- Logsdon, R.G.; Gibbons, L.E.; McCurry, S.M.; Teri, L. Assessing quality of life in older adults with cognitive impairment. Psychosom. Med. 2002, 64, 510–519. [Google Scholar] [CrossRef]
- Gorgoraptis, N.; Zaw-Linn, J.; Feeney, C.; Tenorio-Jimenez, C.; Niemi, M.; Malik, A.; Ham, T.; Goldstone, A.P.; Sharp, D.J. Cognitive impairment and health-related quality of life following traumatic brain injury. NeuroRehabilitation 2019, 44, 321–331. [Google Scholar] [CrossRef]
- Seidel, D.; Thyrian, J.R. Burden of caring for people with dementia—Comparing family caregivers and professional caregivers. A descriptive study. J. Multidiscip. Healthc. 2019, 12, 655–663. [Google Scholar] [CrossRef]
- Patterson, C. World Alzheimer Report 2018: The State of the Art of Dementia Research: New Frontiers; Alzheimer’s Disease International: London, UK, 2018. [Google Scholar]
- Arendash, G.; Cao, C.; Abulaban, H.; Baranowski, R.; Wisniewski, G.; Becerra, L.; Andel, R.; Lin, X.; Zhang, X.; Wittwer, D.; et al. A Clinical Trial of Transcranial Electromagnetic Treatment in Alzheimer’s Disease: Cognitive Enhancement and Associated Changes in Cerebrospinal Fluid, Blood, and Brain Imaging. J. Alzheimers Dis. 2019, 71, 57–82. [Google Scholar] [CrossRef]
- García-Casal, J.A.; Loizeau, A.; Csipke, E.; Franco-Martín, M.; Perea-Bartolomé, M.V.; Orrell, M. Computer-based cognitive interventions for people living with dementia: A systematic literature review and meta-analysis. Aging Ment. Health 2017, 21, 454–467. [Google Scholar] [CrossRef]
- Lieder, F.; Chen, O.X.; Krueger, P.M.; Griffiths, T.L. Cognitive prostheses for goal achievement. Nat. Hum. Behav. 2019, 3, 1096–1106. [Google Scholar] [CrossRef]
- Lienkämper, R.; Dyck, S.; Saif-ur-Rehman, M.; Metzler, M.; Ali, O.; Klaes, C. Quantifying the alignment error and the effect of incomplete somatosensory feedback on motor performance in a virtual brain–computer-interface setup. Sci. Rep. 2021, 11, 4614. [Google Scholar] [CrossRef]
- Fletcher, M.D.; Zgheib, J. Haptic sound-localisation for use in cochlear implant and hearing-aid users. Sci. Rep. 2020, 10, 14171. [Google Scholar] [CrossRef]
- Vermette, L.; Chilana, P.; Terry, M.; Fourney, A.; Lafreniere, B.; Kerr, T. CheatSheet: A contextual interactive memory aid for web applications. In Proceedings of the 41st Graphics Interface Conference, Halifax, NS, Canada, 3–5 June 2015; pp. 241–248. [Google Scholar]
- King, A.C.; Dwan, C. Electronic memory aids for people with dementia experiencing prospective memory loss: A review of empirical studies. Dementia 2019, 18, 1994–2007. [Google Scholar] [CrossRef]
- Hodges, S.; Williams, L.; Berry, E.; Izadi, S.; Srinivasan, J.; Butler, A.; Smyth, G.; Kapur, N.; Wood, K. SenseCam: A retrospective memory aid. In Proceedings of the International Conference on Ubiquitous Computing, Orange County, CA, USA, 17–21 September 2006; Springer: Berlin/Heidelberg, Germany, 2006; pp. 177–193. [Google Scholar]
- Silva, A.R.; Pinho, M.S.; Macedo, L.; Moulin, C.J.A. A critical review of the effects of wearable cameras on memory. Neuropsychol. Rehabil. 2018, 28, 117–141. [Google Scholar] [CrossRef]
- Wieser, I.; Toprak, S.; Grenzing, A.; Hinz, T.; Auddy, S.; Karaoğuz, E.C.; Chandran, A.; Remmels, M.; El Shinawi, A.; Josifovski, J. A Robotic Home Assistant with Memory Aid Functionality. In Proceedings of the Joint German/Austrian Conference on Artificial Intelligence (Künstliche Intelligenz), Bamberg, Germany, 21–25 September 2019; pp. 102–115. [Google Scholar]
- Neto, L.B.; Grijalva, F.; Maike, V.R.M.L.; Martini, L.C.; Florencio, D.; Baranauskas, M.C.C.; Rocha, A.; Goldenstein, S. A kinect-based wearable face recognition system to aid visually impaired users. IEEE Trans. Hum. Mach. Syst. 2016, 47, 52–64. [Google Scholar] [CrossRef]
- Chaudhry, S.; Chandra, R. Design of a mobile face recognition system for visually impaired persons. arXiv 2015, arXiv:1502.00756. [Google Scholar]
- Krishna, S.; Little, G.; Black, J.; Panchanathan, S. A wearable face recognition system for individuals with visual impairments. In Proceedings of the 7th international ACM SIGACCESS Conference on Computers and Accessibility, New York, NY, USA, 9–12 October 2005; pp. 106–113. [Google Scholar]
- Cheng, Y.-H.; Shyi, G.C.-W.; Cheng, K.-H. Age Differences in Face Memory and Face Processing Between Younger and Older Adults in Taiwan. Chin. J. Psychol. 2016, 58, 233–262. [Google Scholar]
- Kinsel, D. Design control requirements for medical device development. World J. Pediatr. Congenit. Heart Surg. 2012, 3, 77–81. [Google Scholar] [CrossRef]
- Yock, P.G. Biodesign: The Process of Innovating Medical Technologies, 2nd ed.; Cambridge University Press: Cambridge, UK, 2015. [Google Scholar]
- Schroff, F.; Kalenichenko, D.; Philbin, J. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 815–823. [Google Scholar]
- Séguin, J.A.; Scharff, A.; Pedersen, K. Triptech: A Method for Evaluating Early Design Concepts. In Proceedings of the Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, Scotland, UK, 4–9 May 2019. [Google Scholar]
- Wechsler, D. Wechsler Adult Intelligence Scale—Fourth Edition (WAIS–IV); NCS Pearson: San Antonio, TX, USA, 2008; Volume 22, pp. 816–827. [Google Scholar]
- Shyu, Y.I.; Yip, P.K. Factor structure and explanatory variables of the Mini-Mental State Examination (MMSE) for elderly persons in Taiwan. J. Med. Assoc. 2001, 100, 676–683. [Google Scholar]
- Nelson, H.E. A modified card sorting test sensitive to frontal lobe defects. Cortex A J. Devoted Study Nerv. Syst. Behav. 1976, 12, 313–324. [Google Scholar] [CrossRef]
- Wechsler, D. Wechsler, D. Wechsler Memory Scale. In Wechsler Memory Scale; Psychological Corporation: San Antonio, TX, USA, 1945. [Google Scholar]
- Benton, A.L.; Varney, N.R.; Hamsher, K.D. Visuospatial judgment. A clinical test. Arch. Neurol. 1978, 35, 364–367. [Google Scholar] [CrossRef]
- Benton, A.L.; deS Hamsher, K.; Sivan, A.B. Multilingual Aphasia Examination; AJA associates: Iowa City, IA, USA, 1994. [Google Scholar]
- Ware, J.E., Jr.; Sherbourne, C.D. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med. Care 1992, 30, 473–483. [Google Scholar] [CrossRef] [PubMed]
- Weinger, M.B.; Gardner-Bonneau, D.J.; Wiklund, M.E. Handbook of Human Factors in Medical Device Design; CRC Press: Boca Raton, FL, USA, 2010. [Google Scholar]
- Brooke, J. SUS-A quick and dirty usability scale. Usability Eval. Ind. 1996, 189, 4–7. [Google Scholar]
- Sokolova, M.; Lapalme, G. A systematic analysis of performance measures for classification tasks. Inf. Process. Manag. 2009, 45, 427–437. [Google Scholar] [CrossRef]
- Duchaine, B.; Nakayama, K. The Cambridge Face Memory Test: Results for neurologically intact individuals and an investigation of its validity using inverted face stimuli and prosopagnosic participants. Neuropsychologia 2006, 44, 576–585. [Google Scholar] [CrossRef]
- Taft, C.; Karlsson, J.; Sullivan, M. Do SF-36 summary component scores accurately summarize subscale scores? Qual. Life Res. Int. J. Qual. Life Asp. Treat. Care Rehabil. 2001, 10, 395–404. [Google Scholar] [CrossRef]
- Card, S.K.; Newell, A.; Moran, T.P. The Psychology of Human-Computer Interaction; L. Erlbaum Associates Inc.: Hillsdale, NJ, USA, 1983. [Google Scholar]
- Marotta, J.J.; Genovese, C.R.; Behrmann, M. A functional MRI study of face recognition in patients with prosopagnosia. Neuroreport 2001, 12, 1581–1587. [Google Scholar] [CrossRef]
- Kuo, W.J.; Yeh, T.C.; Duann, J.R.; Wu, Y.T.; Ho, L.T.; Hung, D.; Tzeng, O.J.; Hsieh, J.C. A left-lateralized network for reading Chinese words: A 3 T fMRI study. Neuroreport 2001, 12, 3997–4001. [Google Scholar] [CrossRef] [Green Version]
Without Device Assistance | With Device Assistance | |||
---|---|---|---|---|
Accuracy (%) | Reaction Time (s) | Accuracy (%) | Reaction Time (s) | |
Attempt 1 | 92.38 ± 4.41 | 1.27 ± 0.12 | 80.48 ± 6.23 | 2.11 ± 0.20 ** |
Attempt 2 | 99.05 ± 0.60 | 1.14 ± 0.11 | 81.43 ± 5.94 * | 2.00 ± 0.21 * |
Baseline | Week 1 | Week 2 | Week 3 | Week 4 | |
---|---|---|---|---|---|
Acquaintance face recognition test (while not using the device) | |||||
Attempt 1 | |||||
| 74.29% | 100% | 85.71% | 97.14% | 100% |
| 6.65 | 3.54 | 4.50 | 1.29 | 1.23 |
Attempt 2 | |||||
| 68.57% | 100% | 100% | 100% | 100% |
| 8.62 | 2.19 | 3.53 | 1.82 | 1.55 |
Other quality of life and neuropsychological assessments | |||||
SF-36 PCS | 50.47 | - | - | - | 33.86 |
SF-36 MCS | 48.54 | - | - | - | 49.76 |
MMSE | 27 | - | - | - | 30 |
TFMT | -Stage 1: 13 -Stage 2: 15 -Stage 3: 24 -Total: 52.78% | - | - | - | -Stage 1: 16 -Stage 2: 14 -Stage 3: 7 -Total: 51.39% |
Benton’s facial recognition test | 44 | - | - | - | 45 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Tani, J.; Yang, Y.-H.; Chen, C.-M.; Siow, C.Y.; Chang, T.-S.; Yang, K.; Yao, J.; Hu, C.-J.; Sung, J.-Y. Domain-Specific Cognitive Prosthesis for Face Memory and Recognition. Diagnostics 2022, 12, 2242. https://doi.org/10.3390/diagnostics12092242
Tani J, Yang Y-H, Chen C-M, Siow CY, Chang T-S, Yang K, Yao J, Hu C-J, Sung J-Y. Domain-Specific Cognitive Prosthesis for Face Memory and Recognition. Diagnostics. 2022; 12(9):2242. https://doi.org/10.3390/diagnostics12092242
Chicago/Turabian StyleTani, Jowy, Yao-Hua Yang, Chao-Min Chen, Co Yih Siow, Tsui-San Chang, Kai Yang, Jack Yao, Chaur-Jong Hu, and Jia-Ying Sung. 2022. "Domain-Specific Cognitive Prosthesis for Face Memory and Recognition" Diagnostics 12, no. 9: 2242. https://doi.org/10.3390/diagnostics12092242