Dysphagia (2023) 38:1224–1237

https://doi.org/10.1007/s00455-022-10547-w

ORIGINAL ARTICLE

A Systematic and Universal Artificial Intelligence Screening Method

for Oropharyngeal Dysphagia: Improving Diagnosis Through Risk
Management
Alberto Martin‑Martinez1,4 · Jaume Miró1,2 · Cristina Amadó1 · Francisco Ruz3 · Antonio Ruiz3 · Omar Ortega1,4 ·
Pere Clavé1,4

Received: 5 July 2022 / Accepted: 12 December 2022 / Published online: 28 December 2022
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022

Abstract
Oropharyngeal dysphagia (OD) is underdiagnosed and current screening is costly. We aimed: (a) to develop an expert system
(ES) based on machine learning that calculates the risk of OD from the electronic health records (EHR) of all hospital-
ized older patients during admission, and (b) to implement the ES in a general hospital. In an observational, retrospective
study, EHR and swallowing assessment using the volume-viscosity swallow test for OD were captured over 24 months in
patients > 70 yr admitted to Mataró Hospital. We studied the predictive power for OD of 25,000 variables. ES was obtained
using feature selection, the final prediction model was built with non-linear methods (Random Forest). The database included
2809 older patients (mean age 82.47 ± 9.33 yr), severely dependent (Barthel Index 47.68 ± 31.90), with multiple readmis-
sions (4.06 ± 7.52); 75.76% had OD. The psychometrics of the ES built with a non-linear model were: Area under the ROC
Curve of 0.840; sensitivity 0.940; specificity, 0.416; Positive Predictive Value 0.834; Negative Predictive Value 0.690; posi-
tive likelihood ratio (LH), 1.61 and negative LH, 0.146. The ES screens in 6 s all patients admitted to a 419-bed hospital,
identifies patients at greater risk of OD, and shows the risk for OD in the clinician’s workstation. It is currently in use at our
institution. Our ES provides accurate, systematic and universal screening for OD in real time during hospital admission of
older patients, allowing the most appropriate diagnostic and therapeutic strategies to be selected for each patient.

Keywords Dysphagia · Swallowing disorders · Machine learning · Aging · Screening methods · Diagnostics

Introduction

Definition, Demographic Data and Complications

Managing older patients is one of the most pressing chal-

lenges for healthcare systems. It is expected that by 2050,
* Alberto Martin‑Martinez one in six people will be over 65, while those over 80 will
[email protected] triple [1]. Oropharyngeal dysphagia (OD) is a symptom of
* Pere Clavé a swallowing disorder defined by the difficulty in forming
[email protected] or moving the alimentary bolus safely/effectively from the
1 oral cavity to the esophagus. OD is a condition recognized
Gastrointestinal Physiology Laboratory, CIBERehd
CSdM‑UAB, Hospital de Mataró, Universitat Autònoma de by the WHO [2] and classified in the International Clas-
Barcelona, Carretera de Cirera 230, 08304 Mataró, Spain sification Diseases (ICD) with codes 787.2 (ICD-9), R13
2
Fundació Salut Consorci Sanitari del Maresme, Mataró, (ICD-10) and MD93 (ICD-11). It has also been recognized
Spain as a geriatric syndrome by 2 European societies (European
3
IT Department, Consorci Sanitari del Maresme, Mataró, Society for Swallowing Disorders and European Union Geri-
Spain atric Medicine Society) [3]. OD is very prevalent among
4
Centro de Investigación Biomédica en Red de Enfermedades several phenotypes of patients: 38–78% in post-stroke,
Hepáticas Y Digestivas (CIBERehd), Barcelona, Spain between 18–28% in Parkinson disease, 47.5% in hospitalized

13
Vol:.(1234567890)
A. Martin-Martinez et al.: A Systematic and Universal Artificial Intelligence Screening Method 1225

older patients [4], 80% in Alzheimer [5], 85% in patients Diagnosis of OD—Three Steps (Screening, Clinical
with dementia admitted to an intermediate care hospital [6] Assessment and Instrumental Assessment)
and rises to 91% in patients hospitalized with community-
acquired pneumonia (CAP) [7]. The main complications The procedure to establish a diagnosis of OD consists of
of OD are malnutrition (MN) and dehydration, respiratory three steps: (1) clinical screening, (2) clinical and (3) instru-
infections such as aspiration pneumonia (AP), readmissions, mental assessment (Fig. 1). The screening phase aims to
institutionalization and morbimortality, increased healthcare detect patients at risk of OD and need further clinical and
costs and reduced quality of life [4, 8, 9]. The prevalence instrumental assessment. It should be quick, easy, cheap, low
of OD among patients over 65 years is comparable to that risk, and applicable at the first line of care of older patients
of diabetes [10], although awareness is much lower despite admitted to healthcare centers by nurses or nursing assis-
the fact that an estimated 30, 16 and 10 million European, tants without specific training in OD [3]. Depending on the
USA and Japanese citizens, respectively, suffered OD at the country, health professionals have different roles in the mul-
beginning of the twenty-first century [11]. tidisciplinary team involved in the diagnostic and therapeutic
Appropriate management of OD is still a major chal- management of OD.
lenge for healthcare systems and poor treatment can lead To date, the screening for OD consisted of a specific
to high rates of complications [4]. In addition, health- anamnesis (swallowing difficulty, choking, cough during
economic studies have explored the costs associated with meals, a sensation of residue in the pharynx, increased
OD, estimating that each undetected hospitalized patient mealtime and recent weight loss) and the use of specific
has an increased cost of 40.36% and length of hospital stay validated questionnaires that aim to screen for OD risk such
of 8.42 days [12]. In a cost of illness study carried out at as: (1) EAT-10, a self-reported questionnaire on the symp-
the Mataró Hospital, OD was independently associated toms associated with OD [13, 14]; (2) Sydney Swallowing
with higher costs during hospitalization (p < 0.011) and at Questionnaire (SSQ), which assesses the severity of OD in
3 months follow-up. Patients with dysphagia and malnour- patients with OD; [15] and finally, (3) the Swallowing Dis-
ished who suffer from respiratory infections had higher costs turbance Questionnaire, a self-administered 15-item ques-
compared to those without dysphagia at 12 months follow- tionnaire on swallowing disturbances [16]. Failure to detect
up (€19,817.58 vs. €7,242.8, p < 0.0004) [8]. patients at risk of OD by screening results in decreased rates

Fig. 1  Diagnostic algorithm for oropharyngeal dysphagia: screening, clinical assessment and instrumental assessment. Team specialist of the
multidisciplinary team who performs the examination in Europe. ENT ear, nose, and throat medical doctor

13
1226 A. Martin-Martinez et al.: A Systematic and Universal Artificial Intelligence Screening Method

of clinical and instrumental diagnosis, and increased clinical untreated [5, 26]. In a retrospective study where the rela-
risks and healthcare costs associated with undetected OD tionship between OD and frailty was analyzed based on
[17]. Anamnesis’ results require time and patient knowledge the International Statistical Classification of Diseases and
(normally unaware of their OD symptoms) and the involve- Related Health Problems version 9 (ICD-9) in more than 6
ment of carers and relatives. Questionnaires and screening million hospitalized North American patients over 50 years
tests have modest psychometric characteristics and are time- of age, it was established that only 2.88% of them presented
consuming [18] and are seldom used in general hospitals OD, 4.44% being over 80 years of age according to hospital
because: (a) nurses are not specifically trained to perform discharge coding. [27]. It is well known that real prevalence
them; (b) speech language therapist (SLTs) are usually not is much higher for example in hospitalized older patients
available in the wards; and (c) awareness of OD among (47%) and those with community-acquired pneumonia
healthcare professionals is still very low in many hospitals. (55%–91.7%) and admitted for stroke (51%–78%) [5]. Many
It is well known that older adults living within the commu- patients are diagnosed when safety impairment leads to res-
nity admitted to acute hospitals and those living in a nursing piratory infections and pneumonia requiring hospitalization
home may have undetected or underdiagnosed swallowing [28]. A study of older patients with community-acquired
impairments that are not systematically screened for [19]. pneumonia found that 9 out of 10 patients had OD and sug-
If the screening process is positive, a clinical assessment gested it should be considered an independent risk factor for
is carried out with the following elements: (a) specific swal- developing this pathology [7]. Given the high prevalence
low patient history, (b) assessment of cognition and com- described, especially among the older age groups, we must
munication, (c) evaluation of oral, laryngeal, and pharyngeal consider under-diagnosis both clinically and in the coding
physiology, anatomy, and functioning with special focus of the pathology.
on cranial nerve examination, and (d) oral intake assess-
ment [20]. OD diagnosis aims to evaluate two deglutition Risk Factors for OD in Older People
characteristics: (a) efficacy of swallow, the ability to ingest
the calories and fluid needed to be correctly nourished and The pathophysiology of oropharyngeal dysphagia in older
hydrated, and; (b) safety of swallow, or the capacity to take people is characterized by an impairment of both biome-
fluids and food without risking respiratory complications. chanical and neurophysiological swallow responses. The
The Volume-Viscosity Swallow Test (V-VST) is a clinical first involves delayed times to laryngeal vestibule closure
diagnostic method we developed for the clinical diagnosis and to upper esophageal sphincter opening, which leads
of OD that uses an algorithm with different volumes and to a high prevalence of swallowing safety and efficiency
viscosities to identify signs that affect swallowing efficiency impairment signs, respectively [29]. The neurophysiologi-
(such as lip seal, oral and pharyngeal residue) and also cal alteration is characterized by a delay in the conduc-
swallowing safety (cough, wet voice, and oxygen desatu- tion and integration velocity of sensory inputs, reduced
ration between 3 and 5%) [21]. When used by adequately activation of brain areas related to swallowing control
trained personnel, the V-VST showed a 93.17% sensitivity and decreased pharyngeal sensitivity [29–31]. Impaired
and 81.39% specificity for the clinical diagnosis of OD; and swallow function in older people is also associated to loss
86.07 and 68.47% for the clinical diagnosis of impaired of muscle mass and function (sarcopenia), a reduction of
safety of swallow [22]. The V-VST also identifies the opti- tissue elasticity, changes in the cervical spine, reduction
mal bolus viscosity and volume needed for each patient [21]. of saliva production, poor dental status, reduced oral and
The third phase, the instrumental assessment, consists of pharyngeal sensitivity, and reduced olfactory and gusta-
the use of gold standard techniques to objectively evaluate tory function [32]. Over the last 15 years, our group has
deglutition such as videofluoroscopy (VFS), fiberoptic endo- generated evidence that OD in the elderly is related to poor
scopic evaluation of swallowing (FEES), and more recently functional status, frailty, sarcopenia, the severity of the
high resolution pharyngo-oesophageal manometry associ- acute and chronic stroke, neurodegenerative pathologies
ated with impedance [23–25]. These techniques detect OD and dementia, nutritional status, and other comorbidities
and assess the specific mechanisms of swallow dysfunction. and we found these factors are systematically repeated in
All of them enable us to understand the pathophysiology of hospitalized, institutionalized and community-dwelling
OD including aspiration mechanisms and alteration of safety elderly people with OD. [6, 28, 33–37]. Another relevant
and swallowing efficiency in each patient (Fig. 1). factor is the effect of medication on deglutition as sev-
eral drugs have been related with impaired swallowing
OD Underdiagnosis function like antipsychotics, sedatives and neuroleptics,
particularly in older patients [38]. These and many other
OD affects several population groups causing serious com- potential risk factors for OD, such as age, fractures, surgi-
plications, however most patients are undiagnosed and cal procedures, readmissions, and consultancy to primary

13
A. Martin-Martinez et al.: A Systematic and Universal Artificial Intelligence Screening Method 1227

care or emergency rooms are recorded in the patient's elec- Aim of the Study
tronic health records (EHR) through the ICD [39]and the
Anatomical Therapeutic Chemical (ATC) classification OD is rarely systematically screened despite its prevalence
system for drugs [40]. and complications, and most hospitalized patients with dys-
phagia are not treated or even diagnosed. To prevent the
severe complications related to OD, an automatic and sys-
Artificial Intelligence and Machine Learning Based tematic tool is needed to help detect OD at an early stage and
on EHR Applied to Clinical Care to identify patients who are at high risk for OD and might
develop impaired safety of swallow. It should be remem-
The creation of complex algorithms and the unstoppable bered that all patients suffering OD have the right to be
revolution in computing power has enabled the evolution diagnosed and treated for this condition, and that healthcare
of a new branch of computer science, artificial intelligence systems have the mission to provide the appropriate, simi-
(AI). AI is not globally defined, but a good approximation lar and state of the art care to all these patients. The aim of
is offered by Andreas Kaplan and Michael Haenlein as”the this study was to develop and implement an expert system
ability of a system to correctly interpret external data, to (ES) based on machine learning that calculates the risk of
learn from that data and to use that knowledge to achieve OD from the EHR of all hospitalized older patients during
specific tasks and goals through flexible adaptation”[41]. admission at the Mataró Hospital, and to assess its clinical
Machine learning is a branch of AI and can be defined as: utility and psychometrics with linear and non-linear models.
“A computer program is said to learn from experience with
respect to some class of tasks T and performance measure P,
if its performance at tasks in T, as measured by P, improves Methods
with experience E” [42]. Machine learning has been used
in numerous fields. In the food industry, computer vision Study Design
techniques are used to assess the quality of foodstuffs [43].
Significant advances have also been made in natural lan- This was an observational, retrospective study where each
guage, called Natural Language Processing [44]. Even in one patient’s clinical information was captured from EHR
of today's notable challenges, climate prediction, machine over the 24 months prior to the swallowing assessment.
learning techniques are used because of the non-linear The study was performed on older patients consecutively
dynamic complexity of the Earth and the high dimensional- admitted for acute diseases to Hospital de Mataró and its
ity of observational data sets and models [45]. In medicine, intermediate care hospital (IMCH) Hospital St. Jaume i Sta.
machine learning has been used to create classification mod- Magdalena, of the Consorci Sanitari del Maresme (CSdM),
els to detect and diagnose diseases [46]. Among the fields in Spain, between 1st January 2013 and 31 December 2018.
healthcare, some notable uses are: (a) personalized medicine The CSdM has a catchment area of 275,000 inhabitants and
[47], (b) epidemiological analysis of large databases [48, makes more than 21,000 admissions per year. Both hospitals
49], and (c) clinical decision support systems through the work in coordination and have a total of 419 beds, of which
development of classifiers that predict the risk of a patient 316 are for acute hospitalization and 103 for IMCH. Expert
suffering or developing a disease [47, 50, 51]. nurses specifically trained performed the V-VST to deter-
Machine learning algorithms can be widely grouped mine the presence of OD and signs of impaired safety and
into linear and non-linear models [52]. Linear mod- efficacy of swallow during hospitalization [22].
els predict a target variable based on linear relationships
between one or more predictors. The assumption of the Workstation and Software to Collect all EHR
linear models is that the relationship between the target
variable yi and( the) predictors x(j is) given by the equation The software containing the EHR of the patients in the
yi = 𝛽0 + 𝛽1 𝜑1 x1 + ⋯ + 𝛽n 𝜑n xn + 𝜀i, where 𝛽j are coef- CSdM is TESISHCE version 2022.1.0 (Nexus/sisinf S.L,
ficients, 𝜑j may be a non-linear function and 𝜀i is an inde- Sabadell, Spain). This software has been in use since 2012.
pendent noise term. Known examples are linear regression The anonymized clinical information is extracted from
[53] and logistic regression [54]. In contrast, non-linear TESISHCE and stored in.csv format for exploitation.
models can fit non-linear relationships between variables,
represented by yi = 𝜑i (x1 , x2 , ⋅ ⋅ ⋅, xn , 𝛽1 , 𝛽2 , ⋅ ⋅ ⋅, 𝛽j ) + 𝜀i . Database
Among non-linear models a widely used one is Random
Forest [55]. Predictive modeling, based on machine learning, In line with the published literature and our team's expert
with EHR can lead to improvements in healthcare quality knowledge of OD in older people, more than 25,000 poten-
and cost-effectiveness [56, 57]. tial variables were selected as being directly or indirectly

13
1228 A. Martin-Martinez et al.: A Systematic and Universal Artificial Intelligence Screening Method

related to OD. These consisted of the presence or absence variables. Numerical variables were standardized for linear
of pathologies described in patients’ EHR in the 24 months and logistic regression.
prior to admission, examples being diabetes mellitus
(E08.0), cerebral vascular accidents (I63) and renal insuf- Linear Model
ficiency (N18). Of those codes, only 279 were diagnosed
in our database. To account for dysphagia, we have used For the linear model, starting from the processed database,
the ICD code (R13) that were diagnosed as a result of the we selected risk factors for OD through a bivariate corre-
test V-VST [21]. Additionally, we included the anatomical lation analysis. We used the Chi-square test to assess the
therapeutic chemical codes (ATC) of dispensed medication relationship of different categorical variables or the Fisher’s
and sociodemographic variables, such are age, sex, hospital exact test for small sample sizes, i.e., if any expected value
readmissions during previous 2 years, and length of stay of in the contingency tables was smaller than five. For the con-
each hospitalization. Finally we accounted for other clini- tinuous variables, we used Student's t-test when the distribu-
cal variables related to functionality and frailty, such as the tion was normal and homoscedastic. However, in the case of
Barthel Index. Table 1 shows a summary of the variables normality but heteroscedasticity, we used Welch's correction
included in the ES. and Mann–Whitney’s U test when normality could not be
ensured. To find independent OD risk factors, we built a
logistic regression with variables that had a significant cor-
Machine Learning Approach
relation with OD (p-value < 0.05), except those that were not
clinically relevant according to our expertise. Finally, vari-
This section explains the two main strategies presented in
ables with a significant coefficient in the logistic regression
this paper for predicting dysphagia: the linear and the non-
were used to build the final linear model.
linear approach. In the first case, we used the traditional
procedure of finding independent risk factors of OD and
Non‑Linear Model
used those variables to create a logistic regression model to
predict the risk of OD [54]. In the non-linear case, we used
For the non-linear model, we used a Random Forest. Before
a purely data-driven approach. We selected the variables and
the training, we performed a feature selection step: we used
created a predictive model without any previous assumptions
a recursive feature elimination algorithm based on random
of correlation and linearity, and only accounted for the com-
forests to find the combination of predictive variables which
bination of them that led to higher performance metrics [58].
led to higher accuracy [59]. The idea behind the algorithm
consists in creating a large sample of models through boot-
Data Pre‑Processing strap and recursive feature steps to find a model with high
performance and keep the variables used to build it. We used
The database was split randomly into two datasets, one for this model to assemble an ES, as described in subsection
training and one of testing. The training had 20% of the orig- Artificial Intelligence Massive Screening—Oropharyngeal
inal database (582 cases) and the training kept 80% (2326 Dysphagia (AIMS-OD).
cases). The split was done ensuring that the prevalence of
OD was as similar as possible in both datasets. Regarding Evaluation Metrics
the categorical variables, those with a ratio between the
most common and the second most common value of 95 To quantify the predictive performance of machine learning
to 5 were discarded. Variables with a prevalence smaller algorithms for detecting the risk of OD in the study popula-
than 10% were also discarded. For categorical variables with tion, we used area under curve receiver operating charac-
more than two values, we encoded them into binary dummy teristics (AUC ROC) [60], sensitivity [61], specificity [61],

Table 1  Variables used in the Data type Description N

study
Demographic data Sex and age 2
Diagnosis codes ICD-10 codes 279
Validated scales and indexes Functionality and frailty scores 12
Administrative data Admissions, readmissions and medical visits to emergency 3
room
Dispensed medication Medication potentially associated with dysphagia 22

ICD indicates international code of diseases; N depicts the number of variables included in the algorithm

13
A. Martin-Martinez et al.: A Systematic and Universal Artificial Intelligence Screening Method 1229

positive predictive value (PPV) [62], negative predictive API, which enables any center that has an individualized
value (NPV) [63] and positive and negative likelihood ratio EHR to ask for their patients’ risk of OD. The web ser-
(LHR) [63].We discarded other standard metrics such as vice offered has some steps and implications: (a) first the
Cohens Kappa, due to their pitfalls for machine learning healthcare center anonymizes the clinical data from EHR
models [64] (See Fig. 2) and sends a query through the HyperText Transfer Protocol
Secure (HTTPS) using JavaScript Object Notation (JSON)
Statistical Analysis international standard; (b) our API receives the query and
checks the authorisation of the user (hospital, rehabilitation
We have used the bivariate analysis described in the linear center, etc.) and whether it contains the required fields and
model section to estimate the risk factors for OD. security (c) finally, the API sends the query to the internal
ES which predicts the risk of OD. The risk is returned as a
Artificial Intelligence Massive Screening—Oropharyngeal number between 0 and 1, to the consultant healthcare center
Dysphagia via HTTPS (Fig. 3). The ES is composed of a patented
algorithm that uses non-linear machine learning methods to
We built the ES, called AIMS-OD, after performing experi- predict the risk of OD. The ES and the API are built using
ments using the processed database to establish the best open-source software and are fully scalable, as the system
model between linear and non-linear. includes several API-based microservices. This architecture
enables an immediate response to any number of consulting
Technical Issues and Implementation AIMS-OD was cre- hospitals.
ated as a service for acute hospitals, rehabilitation centers,
and nursing homes to measure the risk of suffering from Innovation, Valorization and Intellectual Property The
dysphagia based on anonymised data (age, sex, Barthel technology transfer process for the ES started in 2019 with
Index, ICD code, …). In countries with a personal EHR for the participation in the Mentor in Health Innovation Pro-
each citizen such as in Catalonia, Spain, it can also estimate gram (Consorci Sanitari del Maresme—TecnoCampus;
the risk for OD in all the population with an EHR including https://​mentor.​csdm.​cat/). Subsequently, the innovation was
primary care centers. The risk estimation service is offered chosen for participation in two accelerator programs during
through an application programming interface (API) or web 2020, StartHealth (TecnoCampus; https://​www.​tecno​cam-
pus.​cat/​en/​accel​eracio-​de-​negoc​is/​progr​ama-​start​health),
and Caixa Impulse Validate (“LaCaixa” Research Foun-
dation; https://​funda​cionl​acaixa.​org/​es/​convo​cator​ia-​caixa​
resea​rch-​valid​ate-​descr​ipcion-​progr​ama). Researchers have
received training and mentoring in these several programs
to help validate assets and define a valorisation plan. This
project has been awarded the Creative Awards for the best
business initiative in technology and innovation in 2021.
Finally, in 2022, AIMS-Medical S.L. was created, the spin-
off to which the asset has been licensed to bring it to the
market. AIMS-OD, is the subject of an international patent
application—PCT/ES2020/070723; OEPM-P201931028—
with a priority date of November 2019

Ethical and Legal Considerations

The study was conducted following the recommendations

contained in the Declaration of Helsinki, the WHO recom-
mendations, and current Spanish legislation. The Ethics
Review Board of Hospital de Mataró approved the study
protocol (code 40/17). The hospital's Information Technol-
ogy Department carried out irreversible anonymization after
the data was transferred and the ES does not store any query
information once the risk is returned. Following the Regu-
Fig. 2  Graphical representation of under-screening of oropharyngeal
dysphagia among hospitalized older people leading to underdiagnosis lation (EU) 2016/679 of the European Parliament and of
of this condition the Council of 27 April 2016 on the protection of natural

13
1230 A. Martin-Martinez et al.: A Systematic and Universal Artificial Intelligence Screening Method

Fig. 3  Graphical representation of the process of sending, receiving technology; Icd International code disease; JSON javascript object
and answering the query (what is the risk of OD of each hospital- notation; HTTPS hypertext transfer protocol secure; AI artificial intel-
ized patient?) by the hospital and the prediction server. This banner ligence
has been designed with resources from Flaticon.com IT information

persons concerning the processing of personal data and the and 446 (20.95%) needed high viscosity (> 800 m.Pa.s). The
free movement of such data, we performed a Data Protec- most prevalent sign of impaired efficacy of swallow was
tion Impact Assessment (DPIA) [65]. It was approved by the oropharyngeal residue, present in 512 (24.06%) at medium
data protection officer of the Catalan Department of Health, viscosity and 587 (27.54%) at high viscosity.
resulting in “low risk” of data processing for individuals
in the use of new technologies, nature, scope, context and Main Risk Factors Associated with OD
purpose of the processing.
The primary health conditions and pathologies significantly
associated with OD after a bivariate analysis in the database
Results study sample were old age (p < 0.0001) , poor functional
status (p < 0.0001), chronic kidney pathology (p < 0.0001),
Demographic, Clinical Characteristics and Swallow neurodegenerative disease (p < 0.0001), delirium (p <
Capacity of Patients Included in the Database 0.0001), chronic respiratory disease (p = 0.0098), diabe-
tes mellitus (p < 0.0001) and malnutrition (p < 0.0001)
The expert database includes 2809 patients, 1539 (54.79%) (Table 2). In addition, older people with OD showed more
women, with a mean age of 82.47 ± 9.33. Of these, 1459 acute hospital admissions in the previous 24 months (p =
(51.97%) were admitted to the acute hospital and the rest 0.0178) and a higher rate of bronchoaspirations (p < 0.0001)
to an intermediate care hospital. Older patients included in and lower tract respiratory infections (LTRI) (p = 0.0181)
the database have a high prevalence of comorbidities. Main than those without OD. No significant differences were
diagnoses were chronic respiratory disease (25.45%), dia- observed in the rates of diagnosis of pneumonia between
betes mellitus (22.96%), chronic kidney disease (21.72%), the two groups (Table 2).
neurodegenerative diseases (19.01%) and cerebral vascu-
lar accident (18.55%). The majority of individuals showed Machine Learning Approach for Predicting Risk
severe dependence (61.30% Barthel Index 21–60, with a of OD
mean score of 47.66 ± 31.89). The main clinical character-
istics of the sample are presented in Table 2. The linear model is composed of 31 variables that showed
According to the V-VST, 2,128 (75.76%) of patients statistical significance after bivariate analysis. After multi-
showed clinical signs of OD, 1,740 (81.76%) presented variate analysis and logistic regression only age remained
efficacy impairment, 1,464 (68.80%) safety impairment, significant. This ES showed AUCROC of 0.734 (95% CI
and 1,082 (50.85%) suffered both. Of the individuals who 0.713—0.755), with a sensitivity of 0.964, specificity of
had OD, only 824 (38.72%) could swallow liquid viscosity 0.191, PPV of 0.788, and NPP of 0.628 to detect OD in our
(< 50 m.Pa.s) safely, and 1,115 (61.28%) required thickened database.
products, 699 (32.85%) with medium viscosity (250 m.Pa.s)

13
A. Martin-Martinez et al.: A Systematic and Universal Artificial Intelligence Screening Method 1231

Table 2  Clinical conditions Total OD NO OD p-value

of database sample and
comparison between individuals N % (n) 2809 75.76 (2128) 24.24 (681) –
with and without dysphagia
Sex (Female) % (n) 54.79 (1539) 55.03 (1171) 54.04 (368) p = 1.000
with the information recorded in
the EHR for the 2 years prior to Age (Years) mean ± SD 82.47 ± 9.33 83.38 ± 8.51 79.61 ± 11.05 p < 0.0001
the V-VST clinical assessment Acute admissions previous 24 months 4.06 ± 7.52 4.39 ± 7.96 3.02 ± 5.81 p = 0.0178
mean ± SD
Functional capacity (Barthel) 47.66 ± 31.89 43.49 ± 30.81 60.77 ± 31.73 p < 0.0001
Independent (100–95) %(n) 12.14 (341) 8.22 (175) 24.38 (166) p < 0.0001
Mild dependence 26.52 (745) 25.42 (541) 29.96 (204) p < 0.0001
(94–60) % (n)
Severe dependence 61.30 (1,722) 66.31 (1411) 45.67 (311) p < 0.0001
(< 60) % (n)
Swallowing evaluation (V-VST)
Efficacy impairment % (n) 81.76 (1,740) – –
Safety impairment % (n) 68.80 (1,464)​ – –
Acute respiratory events
Bronchoaspiration % (n) 4.34 (122) 5.22 (111) 1.62 (11) p < 0.0001
LTRI % (n) 12.64 (355) 13.20 (281) 10.87 (74) p = 0.0181
Pneumonia % (n) 1.28 (36) 1.32 (28) 1.17 (8) p = 1.000
Comorbidities
Heart disease % (n) 13.24 (372) 13.44 (286) 12.63 (86) p = 1.000
Chronic hepatopathy % (n) 5.27 (148) 5.64 (120) 4.11 (28) p = 0.1388
Chronic kidney disease 21.72 (610) 23.31 (496) 16.74 (114) p < 0.0001
% (n)
Cerebrovascular disease 18.55 (521) 18.56 (395) 18.50 (126) p = 1.000
% (n)
Neurodegenerative disease 19.01 (534) 21.66 (461) 10.72 (73) p < 0.0001
% (n)
Delirium % (n) 16.30 (458) 18.66 (397) 8.96 (61) p < 0.0001
Chronic respiratory disease % (n) 25.45 (715) 26.83 (571) 21.15 (144) p = 0.0098
Diabetes mellitus % (n) 22.96 (645) 23.73 (504) 20.56 (140) p < 0.0001
Malnutrition % (n) 11.78 (331) 12.55 (267) 9.40 (64) p < 0.0001

Bold values indicate statistical differences (p-values)

OD indicates oropharyngeal dysphagia; V-VST, volume-viscosity swallow Test; LTRI, lower tract respira-
tory infection

The non-linear model or ES uses 103 variables. After

selecting the variables and building the prediction model
with Random Forest, we achieved the AUCROC of 0.840
(95% CI: 0.829–0.867), with a sensitivity of 0.940, specific-
ity of 0.416, PPV of 0.834, and NPP of 0.690 (Fig. 4).
When comparing the psychometrics between both ES we
found statistically significant differences in several param-
eters: AUCROC (p < 0.0001), sensitivity (p = 0.0464), speci-
ficity (p < 0.0001), PPV (p < 0.0001) and NPV (p < 0.0001).

ES Risk Management Capacity

The resultant ES allows the user (hospital management) to Fig. 4  ROC curves of the linear and non-linear models to detect OD
with respect to the V-VST findings. Blue curve depicts the AUCROC
decide the OD risk thresholds to be displayed to develop
of the linear model (multivariate logistic regression analysis) and the
strategies for risk management, both in diagnosis and thera- red curve depicts the AUCROC of the non-linear model (Random
peutic approach. As shown in Table 3, as the user increases Forest)

13
1232 A. Martin-Martinez et al.: A Systematic and Universal Artificial Intelligence Screening Method

Table 3  Psychometrics of ES according to the several risk cut-offs The result can be seen in several computer systems, such as
between 0.3 and 0.8 the EHR, the drug prescription and administration software
Pre-defined Sensitivity Specificity PPV NPV and the diet prescription software (Fig. 5).
risk

0.3 0.975 0.182 0.788 0.701

0.4 0.964 0.231 0.796 0.671
Discussion
0.5 0.950 0.304 0.810 0.661
The main result of this study is that it is possible to develop
0.6 0.923 0.413 0.831 0.631
an ES that provides accurate, systematic and universal
0.7 0.847 0.565 0.859 0.542
screening for OD in real time during hospital admission of
0.8 0.665 0.752 0.893 0.418
all older patients admitted to a general hospital, allowing the
PPV indicates positive predictive value; NPV negative predictive most appropriate diagnostic and therapeutic strategies to be
value selected for each patient. Our study describes the process
used to develop this ES. Information from the EHR of 2807
the risk threshold, showing those patients at higher risk of patients was used, taking into account up to 25,000 variables
OD, the specificity and PPV increase. As the user lowers the during the training and feature selection process. Based on
risk threshold, showing all patients at risk, sensitivity and Random Forest for both feature selection and training, the
NPV increase. The ability to pre-determine the risk thresh- best model provides an AUCROC of 0.840 and allows the
old enables the ES user to adapt the sensitivity and speci- systematic screening of a 400 + bed general hospital in 6 s.
ficity of the system according to the needs and resources Therefore AIMS-OD fits the qualities for screening tools
available at any given time leading to an increase in the in older patients with OD: easy to use, quick, inexpensive,
number of at-risk patients detected as well as increasing the accurate, without risk for the patient, and usable by care
cost-efficiency of each new case. providers without specific knowledge of OD [3]. The com-
bined strategy of massive screening with AIMS-OD, clinical
diagnosis with V-VST, and therapeutic management of older
Utility and Feasibility ES Properties patients with dysphagia with minimal/optimal massive inter-
ventions could be a paradigm shift in providing universal
As described in methods, the ES automatically screens all clinical care for all older patients with OD.
patients admitted in both our acute and intermediate care For many years, we have developed studies assessing the
hospital with 419 beds in 6 s, and provides the risk of each risk factors for OD in older persons [28]. We also identified
patient at the clinician’s workstation. The ES allows the these risk factors in our ES training database, as we observed
consulting hospital to query as many times as they want that individuals with OD compared to those without OD
by providing real-time data. It shows a color-coded alert were significantly older, had more hospital readmissions,
with icons and the individual’s risk is green when low risk worse functional capacity, neurodegenerative pathologies,
(< 0.45), orange when medium risk (0.46–0.70) and red and delirium as well as a higher rate of malnutrition. These
when high risk (> 0.71). The CSdM pilot implementation data have also been described in the scientific literature by
performs an hourly query on all inpatients. After weeks of several authors and corroborate that the learning database
implementation, the ES is stable in determining risk, with used by AIMS-OD to establish the risk of OD describes
a daily percentage of high risk patients of 10.55% ± 0.61; the reality of older hospitalized patients with OD. [4, 6, 28,
medium risk, 45.43% ± 2.14, and low risk, 44.02% ± 2.22. 33–36, 66]. Regarding complications related to impaired

Fig. 5  Information shown on

the clinicians' workstation, with
a value between 0 and 1. This
banner has been designed with
resources from Flaticon.com. IT
information technology

13
A. Martin-Martinez et al.: A Systematic and Universal Artificial Intelligence Screening Method 1233

swallow safety, we observed that OD was significantly asso- who did not have these codes in the electronic medical
ciated with a higher rate of broncho-aspirations and LTRI record were randomized and selected (n = 21,716). The best
in our sample, as previously described by Almirall et al. in model chosen by the authors was the Random Forest, which
a similar sample of older patients [7]. In contrast to what was fed by clinical variables (ICD-10, procedures, labora-
was expected, OD patients had a similar rate of pneumonia tory data, nursing protocols, etc.) demographic variables
to those without OD. The term pneumonia collected in the (age, sex), and medication related to dysphagia. It presented
EHR in the 24 months prior to the V-VST assessment of the an AUCROC of 0.94 with a sensitivity and specificity of
study sample is composed of a set of up to 46 CIM10 codes, 0.88 [70]. However, given the existing under-diagnosis and
including CAP, pneumonia of viral origin and also AP. Most under-coding of OD and AP, relying exclusively on ICD-10
of these respiratory infections are not related to OD. AP to determine the presence or absence of the condition can
occurs when there is radiological evidence of pulmonary easily lead to false negatives and false positives. This under-
condensation caused by the entry of oropharyngeal secre- diagnosis is shown in the study by Cohen S et al., which
tions contaminated by pathogenic bacteria into the bronchial describes only 4.44% of hospitalized patients over 80 years
tree in patients with swallow dysfunction [67]. It has been of age in North America had diagnosis codes related to OD
estimated that up to 50% of older patients with OD will in their EHR [27]. This prevalence differs greatly from that
present an oropharyngeal aspiration, and from those, 50% described by Cabré et al. (47%) [28] in hospitalized older
will develop an AP with an associated mortality of up to people. It is also very far from that described by Martino in
50% [68]. patients with acute (51–55%) and chronic (25–45%) stroke
The EAT-10 developed and validated by Belafsky et al. [71]. In both cases, the evaluation of swallowing was per-
suggests that with this screening tool for OD, a score equal formed by clinical examination. Our ES takes as a reference
to or higher than 3 points can be considered a positive result for OD diagnosis the clinical evaluation carried out using
[13]. Later, Rofes et al., by setting the cut-off point at 2 or the V-VST clinical test to establish a clinical diagnosis of
more points, showed an AUCROC of 0.89 with outstand- OD, which presents sensitivity and specificity of 0.93 and
ing sensitivity and specificity (0.85 and 0.82, respectively) 0.81, respectively [22]. Taking as a reference for training and
when studying a population with a prevalence of OD of 87% modeling patients clinically evaluated allows us to get closer
[17]. Other studies found the EAT-10 demonstrated good to determining true positives and negatives.
discriminant ability to accurately identify ALS penetrator/ We believe that the solution to the under-diagnosis
aspirators (PAS ≥ 3) with a cut off score of 3 (AUC: 0.77, of OD in hospitalized older patients is the combination
sensitivity: 88%, specificity: 57%) [69]. The SSQ also has of AIMS-OD system screening with clinical assessment
good sensitivity and specificity for detecting OD in differ- using the V-VST (93.17% sensitivity and 81.39% spec-
ent aetiologies (0.73 and 0.793, respectively) [16]. Despite ificity) in patients at risk for OD [22]. The V-VST has
their good psychometrics, the added value of AIMS-OD lies high diagnostic sensitivity and high PPV to detect OD,
in the ability to screen a large number of patients in a few impaired safety, and aspirations (including silent aspira-
seconds, thus, universalizing OD screening by detecting tions), clearly showing a high discriminating ability [21,
those patients who should be clinically assessed for OD. 22]. In a recent systematic review, we found that more
This increases efficiency in carrying out clinical tests like than a decade from its description and initial validation
the V-VST by a healthcare professional in patients with a by our team [21], the V-VST is now used internationally
positive screening. Unlike self-administered tests, our sys- for clinical screening and clinical diagnosis of OD, to
tem does not require any participation of healthcare staff or select the most appropriate bolus volume and viscosity in
patients or their caregivers, an advantage given the difficulty patients with OD, to determine the prevalence of the con-
of communication that can exist with older patients and, on dition, and to assess the clinical outcome and the effect of
many occasions, with neurodegenerative pathologies. More- treatments applied to patients with OD. The two reviews
over, being a machine learning-based technology, each new included in this manuscript showed very good psychomet-
case and each new clinical or instrumental evaluation will ric properties of the V-VST for OD and impaired safety
improve the predictive capacity of the model. and efficacy of swallow, and good reliability when applied
Other authors have conducted studies developing predic- by trained and experienced professionals [22]. The V-VST
tive models based on machine learning to detect patients at should be administered by trained healthcare profession-
risk for OD prior to hospitalization. Lienhart et al. developed als at all medical facilities and can be repeated according
a predictive model based on information collected from the to the natural progression of the disease. With the use
medical records of more than 33,000 hospitalized individu- of AIMS-OD, nurses, who spend most of their time with
als. Those who during the study period had had an ICD-10 the patient, and physicians will have real-time informa-
codification for dysphagia (R13) or AP (J69) (n = 12,068) tion to determine the patients who are at high risk for OD
were determined to be positive. As a control group, those and should be prioritized, explored with the V-VST and

13
1234 A. Martin-Martinez et al.: A Systematic and Universal Artificial Intelligence Screening Method

receive treatment for OD. Treatment in older hospitalized highly prevalent condition. Moreover the use of our ES
patients following diagnosis is feasible and well defined will help reduce OD-associated complications, improve
with minimal-massive interventions (MMI) aimed at treat- patient quality of life and reduce healthcare-associated
ing the maximal number of patients with cost-effective and costs [8].
simple interventions such as fluid and texture adaptation,
nutritional supplementation, and oral hygiene [66]. A pilot Limitations
study with the MMI concluded that the functional and
nutritional status of patients who received the interven- In general, in cases where the information on frailty and
tion improved and there was a reduction in hospital read- functionality of patients was not recorded in the EHR, the
missions, LTRI incidence, and mortality after 6 months model presented difficulties in predicting OD. Not having
follow-up versus the control group without the MMI. the validation of the ES vs the gold standard for diagnosis
Recently we have improved the MMI and developed the of OD was also a limitation in predicting the risk of this
optimal massive intervention (OMI) (Clinical trial identi- condition. Ideally, in order to improve the tool's sensitiv-
fier NCT04581486) by intensifying nutritional support by ity and specificity, the individuals in the training database
providing patients with recipes, video recipes, and culi- should have been more evenly distributed between those
nary training, explaining step by step how to make triple with and without swallowing disorders, as the population
adapted diets (rheological, caloric-protein, and organolep- included in the database of the study presented a high
tic) for patients with OD. Oral nutritional supplements prevalence of OD (75.74%). In an ideal situation, this
are also included for patients with poor nutritional status. population should mimic the clinical and demographic
Regarding oral hygiene improvement in the OMI, profes- characteristics of the population where the screening sys-
sional dental cleaning is proposed during admission and tem will be used [60]. Finally, although the study sample
in the first month of follow-up, as well as personalized was from an acute and an intermediate care hospital, the
recommendations to patients and caregivers (Clinical trial population was collected from a single healthcare institu-
identifier NCT04581486). The combination of AIMS-OD, tion, and there may be a bias in the management and clini-
V-VST and OMI offers a feasible and effective treatment cal outcomes of these patients.
solution for all hospitalized patients with OD (Fig. 6).
Finally, AIMS-OD allows risk management for the
screening of OD. AIMS-OD gives the clinician a risk Future Work
value between 0 and 1. Once installed in the hospital EHR,
the ES allows healthcare managers to pre-determine the Although the ES uses the V-VST to determine whether
risk boundary at which to highlight, for example, high- or not the patient has OD, an improvement on previous
risk patients and make decisions by adjusting sensitivity evidence-based diagnostic coding, clinical validation
and specificity for patient detection. They will have the of AIMS-OD against the gold standard for OD diagno-
option to improve the diagnosis process efficiency by: (a) sis (VFS or FEES) is necessary as the next step for the
detecting the same number of patients with fewer clinical validation of the system. In addition, once it has been
tests; or (b) detecting a higher number of patients with OD implemented in an acute and intermediate care hospital,
with the same clinical tests. This proposal for systematic the impact of the improved diagnostic process on: (a) the
screening, clinical assessment with V-VST, and treatment number of patients diagnosed; (b) the clinical outcomes of
with MMI will allow many hospitalized older patients with patients, and (c) the healthcare costs of hospital admission
OD to be identified, treated through cost-effective policies in older patients with OD should be evaluated. In addi-
and to democratize the diagnosis and treatment of this tion, incorporating new variables, in particular those of a

Fig. 6  Proposed diagram with the combination of a systematic screening tool, clinical assessment and compensatory treatment for all older peo-
ple with oropharyngeal dysphagia admitted to a healthcare center. AIMS-OD artificial intelligence massive screening—oropharyngeal dysphagia

13
A. Martin-Martinez et al.: A Systematic and Universal Artificial Intelligence Screening Method 1235

non-linear nature, and an incremental number of patients References

into the database will enable continuous improvement of
the AIMS-OD prediction model. It will also be neces- 1. United Nations, Department of Economic and Social Affairs,
sary in the future for professionals to be trained in the use Population Division. World Population Ageing 2019: Highlights
(ST/ESA/SER.A/430). 2019.
of clinical decision support systems in standard clinical 2. International Statistical Classification of Diseases and Related
practice. Health Problems (ICD). Available from: https://​www.​who.​int/​
stand​ards/​class​ifica​tions/​class​ifica​tion-​of-​disea​ses. Accessed 07
April 2022.
3. Baijens LWJ, Clavé P, Cras P, Ekberg O, Forster A, Kolb GF,
Conclusions et al. European society for swallowing disorders - European
union geriatric medicine society white paper: oropharyn-
In this study, an ES with good psychometric characteristics geal dysphagia as a geriatric syndrome. Clin Interv Aging.
has been developed and implemented in the information 2016;11:1403–28. https://​doi.​org/​10.​2147/​CIA.​S1077​50.
4. Carrión S, Cabré M, Monteis R, Roca M, Palomera E, Serra-Prat
systems of a general hospital. This software, AIMS-OD, M, et al. Oropharyngeal dysphagia is a prevalent risk factor for
provides systematic and universal screening for OD in real malnutrition in a cohort of older patients admitted with an acute
time during hospital admission, allowing the most appropri- disease to a general hospital. Clin Nutr. 2015;34:436–42. https://​
ate diagnostic and therapeutic strategies to be selected for doi.​org/​10.​1016/j.​clnu.​2014.​04.​014.
5. Clavé P, Shaker R. Dysphagia: current reality and scope of the
each patient. The results of this screening process are used to problem. Nat Rev Gastroenterol Hepatol. 2015;12(5):259–70.
select and prioritize patients at high risk for OD to undergo https://​doi.​org/​10.​1038/​nrgas​tro.​2015.​49.
the V-VST for clinical diagnosis and subsequent treatment 6. Espinosa-Val C, Martín-Martínez A, Graupera M, Arias O, Elvira
with OMI for OD [Clinical trial identifier NCT04581486]. A, Cabré M, et al. Prevalence, risk factors, and complications of
oropharyngeal dysphagia in older patients with dementia. Nutri-
As a result, more OD patients will be diagnosed and treated ents. 2020;12(3):863. https://​doi.​org/​10.​3390/​nu120​30863.
and thus, we believe there will be a reduction in the main 7. Almirall J, Rofes L, Serra-Prat M, Icart R, Palomera E, Arreola
complications associated with the disease (malnutrition, AP, V, et al. Oropharyngeal dysphagia is a risk factor for community-
and mortality), an improvement in the quality of life of hos- acquired pneumonia in the elderly. Eur Respir J. 2013;41:923–8.
https://​doi.​org/​10.​1183/​09031​936.​00019​012.
pitalized older patients with dysphagia and an amelioration 8. Marin S, Serra-Prat M, Ortega O, AudouardFericgla M, Valls J,
of healthcare-associated costs. Palomera E, et al. Healthcare costs of post-stroke oropharyngeal
dysphagia and its complications: malnutrition and respiratory
Acknowledgements We would like to thank the IT Department of the infections. Eur J Neurol. 2021;28:3670–81. https://​doi.​org/​10.​
Consorci Sanitari del Maresme for their support and collaboration dur- 1111/​ene.​14998.
ing the study. We thank Mrs Jane Lewis for reviewing the English of 9. Leow LP, Huckabee ML, Anderson T, Beckert L. The impact of
the manuscript. dysphagia on quality of life in ageing and parkinson’s disease as
measured by the swallowing quality of life (SWAL-QOL) ques-
Author contributions AMM, JM, PC: conceptualization. AMM, JM, tionnaire. Dysphagia. 2010;25:216–20. https://​doi.​org/​10.​1007/​
PC: methodology. JM, CA: formal analysis and investigation. AMM, s00455-​009-​9245-9.
JM: writing—original draft preparation. OO, PC: writing—review 10 Sinclair A, Saeedi P, Kaundal A, Karuranga S, Malanda B, Wil-
and editing. AMM: funding acquisition. FR, AR: resources. PC: liams R. Diabetes and global ageing among 65–99-year-old adults:
supervision. findings from the international diabetes federation diabetes atlas
9th edition. Diabetes Res Clin Pract. 2020;162:108078. https://​
Funding Financial support received from: a) The Acadèmia de Cièn- doi.​org/​10.​1016/j.​diabr​es.​2020.​108078.
cies Mèdiques de Catalunya i Balears, filial del Maresme, b) by the 11. Robbins J, Langmore S, Hind JA, Erlichman M. Dysphagia
Centro de Investigación Biomédica en Red en el Área de Enfermedades research in the 21st century and beyond: proceedings from Dys-
Hepáticas y Digestivas (CIBERehd), Instituto de Salud Carlos III, c) phagia Experts Meeting, August 21, 2001. J Rehabil Res Dev.
The project leading to these rules has received funding and support 2002;39:543–7.
from “la Caixa” Foundation, under the CaixaImpulse Grant (CI20- 12 Attrill S, White S, Murray J, Hammond S, Doeltgen S. Impact of
00158), d) The Territorial Competitiveness Specialization Project oropharyngeal dysphagia on healthcare cost and length of stay
(PECT) of Mataro-Maresme (PRE/161/2019) financed by the Govern- in hospital: a systematic review. BMC Health Serv Res. 2018.
ment of Catalunya-Generalitat de Catalunya within the framework of https://​doi.​org/​10.​1186/​s12913-​018-​3376-3.
the European Regional Developments Funds of Catalonia Operational 13. Belafsky PC, Mouadeb DA, Rees CJ, Pryor JC, Postma GN, Allen
Program 2014–2020 and Strategic Plan for Research and Innovation J, Leonard RJ. Validity and reliability of the eating assessment
in Health (PERIS), Generalitat de Catalunya (intensification grant of tool (EAT-10). Ann Otol Rhinol Laryngol. 2008;117(12):919–24.
AM; SLT017/10/000219). https://​doi.​org/​10.​1177/​00034​89408​11701​210.
14. Rofes L, Arreola V, Mukherjee R, Swanson J, Clavé P. The effects
Declarations of a xanthan gum-based thickener on the swallowing function of
patients with dysphagia. Aliment Pharmacol Ther. 2014;39:1169–
Conflict of interest The authors declare no financial conflicts of inter- 79. https://​doi.​org/​10.​1111/​apt.​12696.
est. 15. Wallace KL, Middleton S, Cook IJ. Development and validation
of a self-report symptom inventory to assess the severity of oral-
pharyngeal dysphagia. Gastroenterology. 2000;118(4):678–87.
https://​doi.​org/​10.​1016/​s0016-​5085(00)​70137-5.

13
1236 A. Martin-Martinez et al.: A Systematic and Universal Artificial Intelligence Screening Method

16. Cohen JT, Manor Y. Swallowing disturbance questionnaire for 32. Muhle P, Wirth R, Glahn J, Dziewas R. Schluckstörungen im alter:
detecting dysphagia. Laryngoscope. 2011;121:1383–7. https://d​ oi.​ physiologie und pathophysiologie. Nervenarzt Springer Verlag.
org/​10.​1002/​lary.​21839. 2015;86:440–51. https://​doi.​org/​10.​1007/​s00115-​014-​4183-7.
17. Rofes L, Arreola V, Mukherjee R, Clavé P. Sensitivity and speci- 33 Arreola V, Vilardell N, Ortega O, Rofes L, Muriana D, Palomeras
ficity of the eating assessment tool and the volume-viscosity E, et al. Natural history of swallow function during the three-
swallow test for clinical evaluation of oropharyngeal dysphagia. month period after stroke. Geriatr. 2019. https://​doi.​org/​10.​3390/​
Neurogastroenterol Motil. 2014;26:1256–65. https://​doi.​org/​10.​ geria​trics​40300​42.
1111/​nmo.​12382. 34. Cabib C, Ortega O, Vilardell N, Mundet L, Clavé P, Rofes L.
18 Kertscher B, Speyer R, Palmieri M, Plant C. Bedside screening to Chronic post-stroke oropharyngeal dysphagia is associated with
detect oropharyngeal dysphagia in patients with neurological dis- impaired cortical activation to pharyngeal sensory inputs. Eur J
orders: an updated systematic review. Dysphagia. 2014;29:204– Neurol. 2017;24:1355–62. https://​doi.​org/​10.​1111/​ene.​13392.
12. https://​doi.​org/​10.​1007/​s00455-​013-​9490-9. 35. Rofes L, Muriana D, Palomeras E, Vilardell N, Palomera E, Alva-
19. Abu-Ghanem S, Chen S, Amin MR. Oropharyngeal dysphagia rez-Berdugo D, et al. Prevalence, risk factors and complications of
in the elderly: evaluation and prevalence. Curr Otorhinolaryngol oropharyngeal dysphagia in stroke patients: a cohort study. Neu-
Rep. 2020. https://​doi.​org/​10.​1007/​s40136-​020-​00258-​xGERI​ rogastroenterol Motil. 2018. https://​doi.​org/​10.​1111/​nmo.​13338.
ATRIC. 36. Serra-Prat M, Palomera M, Gomez C, Sar-Shalom D, Saiz A,
20 Speyer R. Oropharyngeal dysphagia: screening and assessment. Montoya JG, et al. Oropharyngeal dysphagia as a risk factor for
Otolaryngol Clin North Am. 2013;46:989–1008. https://​doi.​org/​ malnutrition and lower respiratory tract infection in independently
10.​1016/j.​otc.​2013.​08.​004. living older persons: a population-based prospective study. Age
21. Clavé P, Arreola V, Romea M, Medina L, Palomera E, Serra-Prat Ageing. 2012;41:376–81. https://​doi.​org/​10.​1093/​ageing/​afs006.
M. Accuracy of the volume-viscosity swallow test for clinical 37. Carrión S, Cabré M, Monteis R, Roca M, Palomera E, Serra-Prat
screening of oropharyngeal dysphagia and aspiration. Clin Nutr. M, et al. Oropharyngeal dysphagia is a prevalent risk factor for
2008;27:806–15. https://​doi.​org/​10.​1016/j.​clnu.​2008.​06.​011. malnutrition in a cohort of older patients admitted with an acute
22. Riera SA, Marin S, Serra-Prat M, Tomsen N, Arreola V, Ortega disease to a general hospital. Clin Nutr Churchill Livingstone.
O, et al. A systematic and a scoping review on the psychometrics 2015;34:436–42. https://​doi.​org/​10.​1016/j.​clnu.​2014.​04.​014.
and clinical utility of the volume-viscosity swallow test (V-vst) in 38. Miarons Font M, Rofes SL. Antipsychotic medication and oro-
the clinical screening and assessment of oropharyngeal dysphagia. pharyngeal dysphagia: systematic review. Eur J Gastroenterol
2021. Foods. https://​doi.​org/​10.​3390/​foods​10081​900. Hepatol. 2017;29(12):1332–9. https://​doi.​org/​10.​1097/​MEG.​
23. Rosenbek JC, Robbins JA, Roecker EB, Coyle JL, Wood JL. A 00000​00000​000983.
penetration-aspiration Scale dysphagia. Dysphagia. 1996. https://​ 39 Harrison JE, Weber S, Jakob R, Chute CG. ICD-11: an inter-
doi.​org/​10.​1007/​BF004​17897. national classification of diseases for the twenty-first century.
24. Logemann JA. Dysphagia: evaluation and treatment. Folia Pho- BMC Med Inform Decis Mak. 2021. https://​doi.​org/​10.​1186/​
niatr Logop. 1995;47(3):140–64. https://​doi.​org/​10.​1159/​00026​ s12911-​021-​01534-6.
6348. 40. WHO. Anatomical Therapeutic Chemical (ATC) Classification
25. Langmore SE. Evaluation of oropharyngeal dysphagia: which [Internet]. Available from: https://​www.​who.​int/​tools/​atc-​ddd-​
diagnostic tool is superior? Curr Opin Otolaryngol Head Neck toolk​it/​atc-​class​ifica​tion. Accessed 14 february 2022
Surg. 2003. https://d​ oi.o​ rg/1​ 0.1​ 097/0​ 00208​ 40-2​ 00312​ 000-0​ 0014. 41. Kaplan A, Haenlein M. Siri, Siri, in my hand: Who’s the fairest
26. Barczi SR, Sullivan PA, Robbins J. How should dysphagia care in the land? On the interpretations, illustrations, and implications
of older adults differ? establishing optimal practice patterns. of artificial intelligence. Bus Horiz. 2019;62(1):15–25.
Semin Speech Lang. 2000;21(4):347–61. https://​doi.​org/​10.​ 42. Mitchell TM. Machine Learning. New York: MCGraw-Hill; 1997.
1055/s-​2000-​8387. 43. Du CJ, Sun DW. Learning techniques used in computer vision for
27. Cohen SM, Lekan D, Risoli T, Lee HJ, Misono S, Whitson food quality evaluation: a review. J Food Eng. 2006;72:39–55.
HE, et al. Association between dysphagia and inpatient out- 44. Chowdhury GG. Natural language processing. Annu. Rev. Inf.
comes across frailty level among patients ≥ 50 years of age. Sci. Technol. 2003. Available from: http://​eprin​ts.​cdlr.​strath.​ac.​
Dysphagia Springer. 2020;35:787–97. https://​doi.​org/​10.​1007/​ uk/​2611/
s00455-​019-​10084-z. 45. Tibau X-A, Reimers C, Requena-Mesa C, Runge J. Spatio-tempo-
28. Cabré M, Serra-Prat M, Force L, Almirall J, Palomera E, Clavé ral Autoencoders in Weather and Climate Research. Deep Learn
P. Oropharyngeal dysphagia is a risk factor for readmission for Earth Sci A Compr Approach Remote Sens, Climate Sci Geosci.
pneumonia in the very elderly persons: observational prospec- 2021. https://​doi.​org/​10.​1002/​97811​19646​181.​ch13.
tive study. J Gerontol - Ser A Biol Sci Med Sci. 2014;69:330–7. 46. Foster KR, Koprowski R, Skufca JD. Machine learning, medi-
https://​doi.​org/​10.​1093/​gerona/​glt099. cal diagnosis, and biomedical engineering research - commen-
29 Rofes L, Arreola V, Romea M, Palomera E, Almirall J, Cabré M, tary. Biomed Eng Online. 2014;13:1–9. https://​doi.​org/​10.​1186/​
et al. Pathophysiology of oropharyngeal dysphagia in the frail 1475-​925X-​13-​94.
elderly. Neurogastroenterol Motil. 2010. https://d​ oi.o​ rg/1​ 0.1​ 111/j.​ 47. Wu G, Yang P, Xie Y, Woodruff HC, Rao X, Guiot J, et al. Devel-
1365-​2982.​2010.​01521.x. opment of a clinical decision support system for severity risk pre-
30 Tomsen N, Ortega O, Nascimento W, Carrión S, Clavé P. Oro- diction and triage of COVID-19 patients at hospital admission: an
pharyngeal dysphagia in older people is associated with reduced international multicentre study. Eur Respir J. 2020. https://d​ oi.o​ rg/​
pharyngeal sensitivity and low substance P and CGRP concentra- 10.​1183/​13993​003.​01104-​2020.
tion in saliva. Dysphagia. 2022;37:48–57. https://d​ oi.o​ rg/1​ 0.1​ 007/​ 48. Schneeweiss S, Avorn J. A review of uses of health care utiliza-
s00455-​021-​10248-w. tion databases for epidemiologic research on therapeutics. J Clin
31. Rofes L, Ortega O, Vilardell N, Mundet L, Clavé P. Spatiotem- Epidemiol. 2005;58:323–37. https://​doi.​org/​10.​1016/j.​jclin​epi.​
poral characteristics of the pharyngeal event-related potential in 2004.​10.​012.
healthy subjects and older patients with oropharyngeal dysfunc- 49. Wiens J, Shenoy ES. Machine learning for healthcare: on the
tion. Neurogastroenterol Motil. 2017;29:1–11. https://​doi.​org/​10.​ verge of a major shift in healthcare epidemiology. Clin Infect Dis.
1111/​nmo.​12916. 2018;66:149–53. https://​doi.​org/​10.​1093/​cid/​cix731.

13
A. Martin-Martinez et al.: A Systematic and Universal Artificial Intelligence Screening Method 1237

50. Myszczynska MA, Ojamies PN, Lacoste AMB, Neil D, Saffari 64. Delgado R, Tibau XA. Why Cohen’s kappa should be avoided as
A, Mead R, et al. Applications of machine learning to diagnosis performance measure in classification. PLoS ONE. 2019;14(9):
and treatment of neurodegenerative diseases. Nat Rev Neurol. e0222916. https://​doi.​org/​10.​1371/​journ​al.​pone.​02229​16.
2020;16(8):440–56. 65. TicSalut [Internet]. [cited 2022 May 5]. Available from: https://​
51. Gultepe E, Green JP, Nguyen H, Adams J, Albertson T, Tagko- ticsa​lutso​cial.​cat/​dpd-​salut/​eina-​dpia/
poulos I. From vital signs to clinical outcomes for patients with 66. Martín A, Ortega O, Roca M, Arús M, Clavé P. Effect of a
sepsis: a machine learning basis for a clinical decision support minimal-massive intervention in hospitalized older patients
system. J Am Med Informatics Assoc. 2014;21:315–25. https://​ with oropharyngeal dysphagia: a proof of concept study. J Nutr
doi.​org/​10.​1136/​amiaj​nl-​2013-​001815. Health Aging. 2018;22(6):739–47. https://​d oi.​o rg/​1 0.​1 007/​
52. Hastie T, Tibshirani R, Friedman J. The Elements of Statistical s12603-​018-​1043-3.
Learning Data Mining, Inference, and Prediction. New York: 67. Tuomanen EI, Austrian R, Masure HR. Pathogenesis of pneumo-
Springer Series in Statistics; 2009. coccal infection. N Engl J Med. 1995;332(19):1280–4. https://d​ oi.​
53. Seber GA, Lee AJ. Linear regression analysis. 2nd ed. New York: org/​10.​1056/​NEJM1​99505​11332​1907.
John Wiley & Sons; 2012. 68 Almirall J, Cabré M, Clavé P. Neumonía aspirativa [Aspiration
54. Goutam S. Applying logistic regression model to the examination pneumonia]. Med Clin. 2007;129(11):424–32. https://​doi.​org/​10.​
results data. J Reliab Stat Stud. 2011;4(2):105–17. 1157/​13110​467.
55. Breiman L. Random forests. Mach Learn. 2001;45(1):5–32. 69. Plowman EK, Tabor LC, Robison R, Gaziano J, Dion C, Watts
56 Rajkomar A, Oren E, Chen K, Dai AM, Hajaj N, Hardt M, et al. SA, Vu T, Gooch C. Discriminant ability of the eating assess-
Scalable and accurate deep learning with electronic health records. ment Tool-10 to detect aspiration in individuals with amyotrophic
npj Digit Med. 2018. https://d​ oi.o​ rg/1​ 0.1​ 038/s​ 41746-0​ 18-0​ 029-1. lateral sclerosis. Neurogastroenterol Motil. 2016;28(1):85–90.
57. Bates DW, Saria S, Ohno-Machado L, Shah A, Escobar G. https://​doi.​org/​10.​1111/​nmo.​12700. (Epub 2015 Oct 28).
Big data in health care: using analytics to identify and man- 70. Lienhart AM, Kramer D, Jauk S, Gugatschka M, Leodolter W,
age high-risk and high-cost patients. Health Aff Project HOPE. Schlegl T. Multivariable risk prediction of dysphagia in hospital-
2014;33:1123–31. https://​doi.​org/​10.​1377/​hltha​ff.​2014.​0041. ized patients using machine learning. Stud Health Technol Inform.
58 Gotz D, Borland D. Data-driven healthcare: challenges and oppor- 2020. https://​doi.​org/​10.​3233/​SHTI2​00071.
tunities for interactive visualization. IEEE Comput Graph Appl. 71. Martino R, Foley N, Bhogal S, Diamant N, Speechley M, Teasell
2016;36(3):90–6. https://​doi.​org/​10.​1109/​MCG.​2016.​59. R. Dysphagia after stroke: Incidence, diagnosis, and pulmonary
59. Svetnik V, Liaw A, Tong C, Wang T. LNCS 3077 - Application of complications. Stroke. 2005;36:2756–63. https://d​ oi.o​ rg/1​ 0.1​ 161/​
Breiman’s Random Forest to Modeling Structure-Activity Rela- 01.​STR.​00001​90056.​76543.​eb.
tionships of Pharmaceutical Molecules. Springer, New York:
International workshop on multiple Classifier systems; 2004. Publisher's Note Springer Nature remains neutral with regard to
60 Hanley JA, McNeil BJ. The meaning and use of the area under a jurisdictional claims in published maps and institutional affiliations.
receiver operating characteristic (ROC) curve. Radiology. 1982.
https://​doi.​org/​10.​1148/​radio​logy.​143.1.​70637​47. Springer Nature or its licensor (e.g. a society or other partner) holds
61. Pepe PMS, M. S. The statistical evaluation of medical tests for exclusive rights to this article under a publishing agreement with the
classification and prediction. USA: Oxford University Press; 2003. author(s) or other rightsholder(s); author self-archiving of the accepted
62. Hardesty LA, Klym AH, Shindel BE, Chough DM, Sumkin JH, manuscript version of this article is solely governed by the terms of
Gur D. Is maximum positive predictive value a good indicator of such publishing agreement and applicable law.
an optimal screening mammography practice? AJR Am J Roent-
genol. 2005;184(5):1505–7. https://​doi.​org/​10.​2214/​ajr.​184.5.​
01841​505. Alberto Martin‑Martinez MD, PhD
63. Deeks JJ, Altman DG. Diagnostic tests 4: likelihood ratios. BMJ.
2004;329(7458):168–9. https://​doi.​org/​10.​1136/​bmj.​329.​7458.​ Pere Clavé MD, PhD
168.

13