Received April 11, 2022, accepted April 27, 2022, date of publication May 9, 2022, date of current version

May 18, 2022.

Digital Object Identifier 10.1109/ACCESS.2022.3173289

User Experience Design Using Machine

Learning: A Systematic Review
ABDALLAH M. H. ABBAS , KHAIRIL IMRAN GHAUTH , AND CHOO-YEE TING
Faculty of Computing and Informatics, Multimedia University, Cyberjaya 63100, Malaysia
Corresponding author: Khairil Imran Ghauth ([email protected])
This work was supported by the Multimedia University IR Fund under Grant MMUI/220066.

ABSTRACT User experience (UX) is the key to increased productivity by enhancing the usability and
interactivity of the product. Machine learning (ML) solutions have raised user and academic awareness of
technical innovation. As a result, ML is becoming increasingly popular to improve the quality of UX. Several
investigations have highlighted a potential lack of studies on the overall challenges and recommendations for
UX using ML. Therefore, more attention should be paid to ML’s existence and potential applications across
various applications to get the most out of ML techniques to improve the UX design process. To this objective,
a systematic review of the literature was performed as to determine the challenges faced by UX designers
when incorporating ML in their design process. Recommendations that help UX designers incorporate ML
into UX design will be highlighted. Furthermore, the PRISMA approach is used (a process that has been
established in the literature), to restrict the chance of bias at the selection stage. Relevant articles in the
following four databases were searched: IEEE Xplore, Scopus, Web of Science, and ACM. The findings
revealed that the number of publications on issues linked to UX with ML had advanced exponentially.
This review highlights the challenges, recommendations, tools, algorithms, techniques and datasets used
in different studies. In addition, suggestions are given for future investigations.

INDEX TERMS User experience, experience design, UX, ED, machine learning, ML, HCI, UX design, user
interaction, user behavior.

I. INTRODUCTION has demonstrated its ability to enhance the UX in today’s

Over the last few decades, user experience (UX) and machine services and products, researchers suggest that the UX design
learning (ML) are relatively new topics that have made sig- practice underpins complex challenges when using UX as a
nificant progress in website designs. As these two disciplines design material (e.g., [7], [8], [9]).
grow more relevant and become widely used throughout L. M. Policarpo et al. [6] gave an overview of the obstacles
industries and applications, they open a multitude of research UX designers face when using ML as a design element. One
opportunities. Within the realm of human-computer inter- example of the obstacles is the struggle UX designers go
action (HCI), UX is considered a critical factor to devel- through when collaborating with data scientists in a proactive
oping successful, efficient and pleasant solutions. The use manner. Another obstacle is the lack of the tools and abilities
of ML to improve UX is becoming more common [1], [2]. needed to sketch or prototype when using ML as a design
Conversion rates in applications with excellent UX design can material. It is implied that existing UX design education
increase by 400 per cent, whereas conversion rates in appli- and practice are unprepared to work with ML as a design
cations with inadequate user interfaces (UI) only increase by resource [10]. For example, UX designers may find ML too
200 per cent [3]. technically challenging design material [11], and therefore
As for ML, researchers and digital designers recognize that they may decide to shift the current design process due to their
the ML trend has become especially interesting since it opens unawareness of design tools and methodologies to verify the
many new design opportunities for UX designers [4], [5]. The viability of ML-powered solutions. As UX designers in [8]
authors of [6], viewed ML as the new UX. Even though ML surveyed their experiences working with ML, they claimed
their obstacles. In particular, they were worried when devel-
The associate editor coordinating the review of this manuscript and oping something ethical and purposeful. Even though ML is
approving it for publication was Ahmed Farouk . predicted to bring concrete benefits to designers, it is not clear

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
VOLUME 10, 2022 51501
 A. M. H. Abbas et al.: User Experience Design Using Machine Learning: Systematic Review

how to incorporate it into design processes [8], resulting in follows; the next section, Section II, describes our methods
untapped potential [8]. for locating academic and non-academic materials, empha-
Technology-assisted creativity, such as Simon’s sizing the need of avoiding bias. Section III is the results
optimization-based design [12], Tiedemann’s ‘‘creativity section where we present our findings and their catego-
support tools,’’ [13], and Horvitz’s ‘‘mixed-initiative user rization. Discussion and analysis of the results is presented
interface,’’ [14] can all be linked to today’s interest in employ- in Section IV. Limitations of the findings are discussed in
ing ML in order to assist human creativity. Recent related Section V. Suggestions for future research as some research
research presents some inspiring illustrations of ML-based gaps are identified are shown in Section VI. The conclusion
UX design tools. For example, such tools can use formal is given in the last section, Section VII where we state impli-
models to optimize graphical user interface (GUI) layouts in cations of our findings.
order to meet objective performance criteria [15].
An ML automatically vectorizes existing digital GUI II. METHODOLOGY
designs (using computer vision) to quickly transfer them to This research was motivated by the following three research
new projects [16]. It also enables quantifiable evaluations of questions: RQ1 - ‘What are the challenges facing UX design-
given GUIs by utilizing a set of user perception and attention ers while incorporating ML algorithms of UX design?’,
models [17]. Furthermore, Chen et al. [18] recently used RQ2 - ‘What are the recommendations for how ML’s func-
deep learning to translate digital user interface (UI) mockups tionality can be incorporated into the UX design process?’.
into UI specifications (e.g., Android layout XML). Infrequent Finally, RQ3 – ‘How can ML’s algorithms be used to help
design tools assist the automated shift from paper to digital make better UX design decisions?’. We looked at contem-
material: Microsoft [19] and Airbnb [20] are experiment- porary literature as a critical source of answers to these
ing with converting paper sketches directly into GUI code, study questions in order to completely achieve the research
bypassing much of the digital wireframing phase. They also objectives.
claim that UX designers’ difficulties when dealing with ML
limit their ability to innovate and think outside the box. A. RELATED STUDIES SEARCH METHOD
There has been continuing study on how to approach UX First, in early October 2021, we searched electronic databases
design practice while working with ML as a design material the Web of Science (WoS), Scopus, IEEE Xplore, and ACM.
([8], [10], [21]). According to [22], ML ‘‘will force us to We chose the mentioned database previously since it aids
rethink, restructure, and reassess what is feasible in practi- as the entry point for all Computer Science and Social and
cally every experience we create’’. [10] warned, designers Behavior Science [24]. Then, we designed a search string
will not be able to fully exploit ML’s strong potential if they using our understanding and knowledge of the UX design
do not understand the technical side of the technology, which and ML domains and consulting relevant UX design and
involves statistics, data analysis, and programming in the first ML search strings. Table 1 shows the search string for all
place. Secondly, dealing with ML as a design material is like databases.
the last challenge in that a lack of awareness of ML’s full
potential can lead to setbacks in employing it as a design B. STUDY SELECTION
material. The articles that were discovered during the search were
Despite ML’s growing importance in the field of UX saved in Microsoft Excel. Duplicates and relevance to the
design, several recent studies have found that many UX study title were checked against the article list. Any publi-
designers are unaware of its possibilities and limits ([5], [23], cations that were not related to UX and ML were rejected.
[10]). The main reason for this lack of knowledge is that The titles and abstracts of all full-text articles retrieved were
current models of UX design learning fail to suitably inspire examined by two researchers (AMA and KIG) to exclude
design learners to work with ML [8]. those that were unrelated to the research topic. If the two
A wide range of products and services, particularly in the researchers differed on the papers’ relevance, they would
digital sphere, employ ML in various crucial ways. One of discuss it with the third reviewer (TCY).
the most sought-after and valuable applications of ML in UX The chosen studies were divided into two categories:
design is its capacity to provide users with new levels of Screening was completed first to reject articles based on title,
personalization. L. M. Policarpo et al. [6] described ML as keyword, and abstract modifications, followed by a full-text
an untapped possibility for UX designers and that it has yet evaluation. The query’s results that had nothing to do with
to be fully realized as a design material. Algorithms that learn UX or ML, such as coincidences with other UI and UCD that
from underlying data sources are frequently used to improve weren’t part of the ‘‘User Interface’’ or ‘‘UI’’ concept, were
UX [7]. also removed.
This review aimed to systemically report UX designers’
challenges during incorporating ML in UX design. In addi- C. CONDUCTING THE REVIEW
tion, this review presents an evaluation of the effectiveness In the end of January 2022, the systematic literature review
of the available written approach in enhancing the UX using detailed in this paper was completed. We have performed a
ML. The structure of this review paper is organized as comprehensive text review of each of the 18 articles listed

51502 VOLUME 10, 2022

A. M. H. Abbas et al.: User Experience Design Using Machine Learning: Systematic Review

in Figure. 1. For systematic reviews, we adopted the PRISMA TABLE 1. Search string for databases.
statement, which was introduced in [25]. The PRISMA dia-
gram for this review is shown in Figure 1.

D. IDENTIFICATION
The following databases were searched: IEEE Xplore, WoS,
ACM Digital Library, and Scopus. The search engines of
the four libraries were configured to execute search queries
both in the metadata and in the full text of articles. A filter
was added to the questions to ensure that the results did not
include any articles released before 2017, the year in which
the first study utilizing machine learning in user experience
was published.
The IEEE, WoS, ACM, and Scopus now have a second
filter that excludes articles authored in languages other than
English. Articles in other languages were manually elimi-
nated throughout the screening process in the other libraries.
As illustrated in the upper level of the PRISMA diagram in
Figure 1, this query returned 1,044 publications from the four
digital libraries.

library did not give the entire text and results from textbooks
were excluded from this review.
The results of the query that had nothing to do with UX
and ML were also deleted, including, for example, coinci-
dences with other Usability and UX Evaluation concepts.
In total, 922 results were eliminated in this screening, none
of which were subjected to a full-text examination. There
FIGURE 1. PRISMA flow diagram for this systematic literature review. were 18 papers for full-text review currently. As a result of
this phase, 922 articles were rejected, leaving only 18 for the
qualitative synthesis.
E. SCREENING
One hundred four articles were initially removed from F. ELIGIBILITY
the original 1,044 because they were found in multiple We evaluated the 18 publications for eligibility. According
libraries. The screening procedure then began, with one of to the criteria outlined in Section B Study Selection, the
the researchers going over the title, abstract, and keywords of researchers discarded a set of papers. A cross-review
each of the 1,044 publications. Those papers for which the of the discarded documents was performed to ensure

VOLUME 10, 2022 51503

 A. M. H. Abbas et al.: User Experience Design Using Machine Learning: Systematic Review

no contradictions. The researchers also pitched some pub- problems; they did not hand over fully developed designs to a
lications that were irrelevant to UX design using machine technical team. Instead of providing abstractions, exemplars,
learning. and new tools and methods to support collaborating with data
These are works that propose educational tools using scientists, a new way to inspire will benefit UX designers
machine learning, for example [26]. In addition, the study using ML.
by [27], calculates this number in linear time to find a number Comprehending the limitations of what an ML system can
that summarizes all interactions between an object and a user. learn, experimenting with different formulations of an ML
problem, and assessing the performance of ML models in the
G. INCLUDED context of their unique application are formed the challenges
As can be seen in the lower portion of Figure 1, the study facing UX designers. Translating real-world problems into
comprised a total of 18 articles that applied machine learn- learnable tasks, optimizing the design of the ML model rather
ing in UX design and so answered the research question. than aiming to enhance performance by adding more training
A researcher conducted a cross-review of this list, and the data, evaluating model performance, and correcting for bias
discrepancies discovered were resolved. Table 2 summarizes and overfitting are all challenges in this area. The stated
the papers reviewed and included in each review step, orga- issues have a lot of overlap, with all of them relating to
nized by the digital library to which they belong, year of understanding ML capabilities and mapping them to practical
publication, and citation count. applications and the capacity to analyze how well a trained
model performs in a particular setting.
TABLE 2. Summary of reviewed papers. Explicitly addressing the issues that UX designers face,
instead of being considered throughout the process, UX is
frequently an afterthought. Available data limits UX ideas,
and designers struggle to collaborate with data scientists
proactively. It’s challenging to prototype ML, and it doesn’t
play well with designers’ ‘‘fail early, often fail’’ strategy.
Designers have a hard time grasping the capabilities of ML in
the context of UX. Several issues are related to development
methods and communication in mixed project teams. Others
are more basic, highlighting a mismatch between ML as a
design material and the context in which they were found.
The results of our review summarize the challenges faced
by UX designers; hence, we assume that the best fits of
each challenge faced by the UX designers provided as shown
in Table 3.
According to [33] introduce an approach that suggests user
experience (UX) is an interactive ML, which the term IML
refers to ML applications that rely on ongoing user engage-
ment. Some surveys to learn more about each participant
and understand if they have ever been participated in ML
III. RESULTS practices and design [8]. Dove et al. (2017) investigated how
A. CHARACTERIZATION OF THE RESULTS to make UX design and ML experts collaborate. At the same
The results of full-text research of the collected articles time, the study by [34] identifying interesting directions for
are shown in this section, organized by the analyzed topic. the application of ML to UX. As [36] points out involves
We analyze the reports in three main categories: surveys and the iterative generation of design artefacts and experiential
interviews, tools, and datasets. ways that assist designers in the growth of their knowledge
abilities through applying concepts with ML. How do UX
1) GENERAL CHARACTERISTICS OF RQ1 practitioners predict the synergies between UX work and ML
Based on the findings of the associated articles for challeng- techniques? The authors conducted 13 semi-structured inter-
ing facing UX designers during incorporating ML. Designers views with UX experts. In addition, surveyed 49 practitioners
admitted that they had a limited understanding of how ML experienced in UX, ML. [36] claimed that most respondents
worked and was not a priority. Instead, they used designer believe that ML and UX will become increasingly inter-
abstractions and well-known exemplars to describe ML and twined in the future. Also, the authors mentioned that 13
convey design ideas among themselves. The most effective respondents currently see the minimal overlap. In addition,
strategy to help designers connect with ML as a design mate- eight respondents expect the overlap to increase in the future.
rial is to tell them how it works. ML projects take longer Respondents were asked to evaluate their present perceptions
to plan and execute than other design projects. In addition, of the relationship between ML and UX. As well as how they
designers worked closely with data scientists to solve their believe it will change in the future. Only nine respondents

51504 VOLUME 10, 2022

A. M. H. Abbas et al.: User Experience Design Using Machine Learning: Systematic Review

believe ML and UX intersect to some or a significant amount techniques that demand collaboration between UX designers,
in the current situation. However, 23 believe that ML and UX data scientists, and software engineers.
will converge to some extent in the future. In total, 35 of the This collaborative, creative approach would be beneficial
49 survey respondents believe that the interaction between to UX designers. The discussion between UX designers and
the two disciplines will grow in the future. In addition, [34] data scientists was centered on determining a design goal
interviewed 13 UX experts from industry and academia in that was worthwhile to pursue. Our findings revealed many
semi-structured interviews to learn how they envision ML recommendations to incorporate ML functionality into the
technologies enhancing or influencing their UX processes. UX design process, as illustrated in Table 4.
The authors pointed out that many participants, so unsu-
pervised ML has a lot of potential in user segmentation. TABLE 4. Summary of recommendations.
For example, data logs can automatically identify unique
user groups using clustering approaches. Another area of
focus was assisting design decisions by analyzing and rec-
ommending UI solutions based on previous user behavior or
preferences [36]. While [34] proposed term is called Interac-
tion Design. Interaction Design develops mappings between
factors of UX and those of IML, which contains four ele-
ments that influence UX (Artifact, context, epistemology, and
collaboration).

TABLE 3. Summary of challenges faced by the UX designers.

[8] claim that the UX designer needs to collaborate with

data scientists. When using ML techniques with the help
of tools and processes, UX designers should respect and
comprehend the complexity and richness of a design issue.
Because it is vital to specify the design criteria, UX designers
should know what tools and procedures to utilize to integrate
ML techniques into the UX design [8], [10].
For example, to satisfy a commercial value or achieve
the users’ aim. Therefore, a UX designer’s use of ML tech-
niques during the design process can substantially impact the
UX [35]. A UX designer’s profession and education entail
using ML algorithms to investigate links between various
design solutions, including different design materials [34].
According to [36], design practice is always unique in
some way, whether in a new context, the type of historical
data of user behavior, or user preferences. ML’s ability to
empower the UX in today’s more cost-effective option for
boosting UX design [35]. Furthermore, other researchers
claim that a new understanding of ML techniques in organi-
zational and procedural contexts can significantly impact the
UX design process [10], [35].
[7] recommend that UX and HCI teachers develop more
ML-related courses to attract cross-disciplinary UX design-
ers. For example, methods demand collaboration between
users, designers, data scientists, and software engineers. This
2) GENERAL CHARACTERISTICS OF RQ2 collaborative, creative approach might be beneficial to UX
Many recommendations draw the path to incorporate ML into designers.
the UX design process. For example, integrate UX design-
ers with data science engineers to develop ideas, creative 3) GENERAL CHARACTERISTICS OF RQ3
experience, and inspirational content for designing a prod- Incorporating ML algorithms and techniques with UX design
uct or service using ML capabilities. In addition, UX and are still misunderstood due to the complications of UX design
HCI instructors originate more ML-related courses geared to using ML and lack of studies. We are increasingly noticing
attract UX designers from various disciplines. For examples, a shift from experiences ‘powered by ‘ML’ to ML as the

VOLUME 10, 2022 51505

 A. M. H. Abbas et al.: User Experience Design Using Machine Learning: Systematic Review

experience itself in the previous years. Understanding how TABLE 5. Summary of survey questions.
and what ML models do can help us approach design thinking
for ML in new and creative ways to help UX designers.
We must consider ML as a cultural and historical artifact, both
the labelled data entering the model and the predictions that
result.
How can UX designers take advantage of their unique ML
capabilities while changing their process design over time?
According to [37] that current UX and ML algorithms can
forecast final user satisfaction, which is essential for users’
decisions about further use or whether they recommend a
product or service to others or not. [37] reveal that the ML
process can help predict last user satisfaction in at least two
contexts: Experiment I, service usage, and Experiment II,
product usage [37].
[41] reveal that the BN method is valuable for creating
and manipulating probabilistic models for dealing with uncer-
tainty in context-aware systems. [38] proposed Bayesian UM
uses contextual data to identify four degrees of accurate,
original, diversified, and popular publications for users [38].
In addition, [38] suggest in their research the parameters were
computed by employing the bn.fit function using bnlearn
package, which operates the network data to evaluate their
highest likelihood.
Therefore, this study seeks to provide a UX model that uses
the Bayesian network (BN) method to recognize the most
relevant contextual data to make appropriate paper recom-
mendations for scholars [38]. While the study by [40] argue
that a forward feature selection technique was used to find the
optimal collection of features for each type of model.
A program that uses ML techniques to detect users’ emo-
tions using user interaction data from websites is demon-
strated. It has a lot of promise: the predictive models outlined
can be readily included in a script that web developers may
use to record users’ interactions with a web page and infer
their emotions [40].
[31] designed a novel framework to evaluate the busi-
ness value of a coupon targeting model to enhance UX.
The authors adjust classification models utilizing random
search and 10-fold cross-validation [31]. Furthermore, cross- as a design material. New methods and tools are needed
validation for benchmark models ensures that all page views to understand the work with ML effectively. For example,
relating to a user session appear in the same fold and preserve we sketched ideas concerning ML as a design material during
their original order. Avoid scenarios where earlier page views the UX design process.
of a user session show up in the holdout fold, but later page At the same time, the lack of competency demonstrates
views are used for training [31]. that most UX designers do not have the necessary knowledge
or tools to operate with ML. The existing design process
B. QUESTIONNAIRE AND SURVEYS may alter due to the lack of design tools and methodologies
This section reviews and summarizes the survey questions to verify the viability of ML-powered solutions, as ML is
used in the questionnaires. Table 5 summarizes the questions too technically knowledge-demanding design material for
for the UX designers’ backgrounds of using ML capabilities. UX designers. The following Table 6 reviewed the related
tools drawn by different articles. [39] claim a separate step;
C. TOOLS participants draw ideas for a design challenge for roughly
ML is not yet systematically integrated into design pat- three minutes. Second, each participant is given one minute
terns, design education, or prototyping tools. In this section, to offer their opinion to the group.
we review prototyping tools that incorporated. The preva- This procedure is done three to four times to allow par-
lence of the designers represented that working with ML ticipants to build on each other’s ideas and iterate on the

51506 VOLUME 10, 2022

A. M. H. Abbas et al.: User Experience Design Using Machine Learning: Systematic Review

TABLE 6. Summary of ML tools. to those elements to discover future design opportunities.

However, because the current ML system cannot promise that
the outputs are always right, the ML system’s reactions are
designed to be inconspicuous.

TABLE 7. Summary of precise ML-related UX concerns.

designs. As a trade-off between swiftly prototyping an ML-

based application and precise predictions, the authors picked
Microsoft Custom Vision 9, an established ML platform [39]. Therefore, another issue while dealing with ML is estab-
This platform was used to develop and test a model for lishing tools and approaches. For example, [12] emphasizes
detecting GUI elements in photographs of paper sketches. the difficulties of sketching or prototyping with massive
Microsoft Custom Vision already comes with a high-quality datasets and the need for computational power, time, and
foundation model trained on a large amount of generic image reliance on data scientists [43]. After our restructuring
data. The authors generated a series of photographs of GUI and merging review process, we obtained six groups, each
drawings and manual labels for further instruction for our describing one of the typical difficulties illustrated in Table 6.
specific instance. Each issue has several related themes. Transparency, for
The Sketch plugin called this model to detect GUI element example, raises various topics, including explainable AI [43].
kinds and positions. Then the digital wireframe is generated In addition, the lack of prototyping using real datasets or
by detecting the relevant GUI components in Sketch Finally, including datasets at any phase of process design. Next
the study’s overall design was like that of the pre-study. section, we present the articles that used datasets in their
The authors adopted a within-subject strategy (manual vs research.
Paper2Wire). Qualitative/subjective metrics to acquire com-
prehensive insights into the concept’s potential integration in D. DATASETS
a practical setting focus on the practitioner’s UX. 1) METHODS OF DATA COLLECTION
In comparison, the study by [29] demonstrates that design- Predicting user behavior momentary UX data and ML tech-
ers can use the ML-Process Canvas to gather essential data niques is a solution for using datasets. This section presents
throughout the ML process without changing their usual the articles that use datasets in their work. The paper we
design activities. The tool highlights the individual UX chal- summarize shows more than methods to build a dataset from a
lenges created by ML and then describes the user, scenario, questionnaire, sessions, pageview, and collecting users’ data
and ML system elements that may affect those UX chal- in a period.
lenges. In the conceptual phase of UX design, Canvas allows Table 8 display the methods of collecting data from dif-
UX designers to organize and show their findings linked ferent sources. According to Table 8, there are four methods

VOLUME 10, 2022 51507

 A. M. H. Abbas et al.: User Experience Design Using Machine Learning: Systematic Review

for creating datasets using ML algorithms to enhance UX were requested to use the WoS bibliographic database to
design. [37] applied two experiments with two sample sizes conduct relevant publications for their present work in a nat-
to check final user satisfaction. In addition, [44] describe uralistic setting. The participants were asked to provide their
the number of samples per class for small which specific current contexts/situations, such as task and pre-knowledge,
algorithms. Some studies have shown that using a smaller in Step 1 of the data collection process. Step 2 required the
sample size for building a classic machine learning model scholars to choose the most relevant paper to their present
improves performance. needs and score it on a 5-point Likert scale for innovation,
[45] used sampling method is accepted from designing new correctness, popularity, and diversity. Finally, participants
data or a current original dataset and is used to create a new had to submit or upload their paper ID (identity paper created
classification model using ML methods. [32] reveal that a in WOS) [38].
video camera or a webcam continuously captured use face [31] collected data is provided by a shop that specializes
images of each user while they were utilizing products or in selling fashion items and wishes to remain anonymous.
services, and the data were collected. The following preprocessing steps are used to prepare the
data for analysis. The dataset includes five sessions totaling
TABLE 8. Summary of methods for data collections. over 400 page views. Then, due to the significant number of
page views indicates that the sessions were produced by bots,
remove them from the data collection.
According to [40], a total of 12 participation volunteers
of various ages (mean age 32.3) and genders were recruited
(6 women). They were requested to install a plugin we
created for 30 days on their browser (Chrome or Firefox
was necessary). This plugin takes a photo of the user’s face
from the webcam every 2 seconds and mouse and keyboard
logs. A dataset of user emotions and interactions from real
users who interacted with genuine websites ‘‘in the wild’’
for 30 days is created. Second, the findings of compar-
ing four commonly employed machine-learning methods to
detect emotions are presented. Third, the result classifica-
tion models can predict users’ moods in real-time during
interactions [40].

2) ALGORITHMS AND RESULTS

The section briefly introduces the algorithms were used in
different articles and the results for each article. Table 9 iden-
tifies corresponding studies with an algorithm in the datasets
and the accuracy of the results. Summary of classification
algorithms data is provided in Table 8.
According to [32], the SVM-SMOTE oversampling tech-
nique was applied to reduce the problem of class imbal-
ance. Seven ML approaches, including K Nearest Neighbor
(KNN), Support Vector Machine (SVM), SVM with the sig-
moid kernel, SVM with linear kernel, SVM with the poly-
nomial kernel, SVM with radial bias, Logistics Regression,
and Neural Net. Table 9 compared for model evaluation. Each
In addition, the user’s gender and age are gathered. After classification model was validated using leave-one-out cross-
using the items or services, the user reviews them on a validation during the development process.
5-point scale (1 to 5 stars) to determine final user satisfaction. The results reveal that the best accuracy was achieved using
The classification model was validated using leave-one-out a combination of SVM-SMOTE oversampling and SVM with
cross-validation during the development process. While [38] a polynomial kernel. The highest level of cross-validation
prepare a suitable dataset for the BN modelling. A web- accuracy was 86%. While the study by [38] revealed that the
based application was created to collect the data. A large- KFCV approach revealed that the EL of the GS algorithm is
scale questionnaire survey was used to collect data for the (ρ(a) = 0.341486) and the MMHC is (ρ(a) = 2.350423),
study. this suggests that the GS method generates a BN model
Participants should have prior expertise working with the better suited to the data in this investigation than the MMHC
Web of Science (WoS) bibliographic database and a mini- approach. As a result, GS outperforms the MMHC algorithm
mum of 30 minutes to complete the survey. The researchers and is better suited to the dataset used in this study for

51508 VOLUME 10, 2022

A. M. H. Abbas et al.: User Experience Design Using Machine Learning: Systematic Review

TABLE 9. Summary of algorithms.

TABLE 10. Summary of classification methods.

BN modelling. [40] claimed that the best average accuracy Finally, [31] argue that only two RNN-based designs
between the four ML algorithms is a random forest with an outperform the naive model across the board. The GRU
accuracy of 47%. appears to perform slightly better than the LSTM of these

VOLUME 10, 2022 51509

 A. M. H. Abbas et al.: User Experience Design Using Machine Learning: Systematic Review

two architectures. Despite this, determining which RNN TABLE 11. Themes of experience.
model better predicts the possible order values of non-
purchase sessions are challenging due to minor changes in
error measurements.

IV. DISCUSSION
ML is becoming increasingly vital in developing new goods
and services to provide a better UX. Understanding ML
capability, imagining new goods and services, and effec-
tively interacting with data scientists are all UX designers
face today. Our study summarizes the challenges and rec-
ommendations for UX designers using ML in their products,
services, and websites.
The results of the full-text research for the collected arti-
cles are shown in this section. We discovered that these UX
designers do not consider themselves ML experts and that
learning more about ML will not help them become better
ML designers. Instead, participants seemed to be the most
successful when they worked with data scientists regularly to
help them imagine what they should produce and adopted a UX designers, for example, must be able to identify design
data-centric culture and became proactive in their use of data criteria for an experience that prioritizes the user. Hence,
in their design practice. to move forward with potentially beneficial ideas to cus-
Below, we discuss how these findings depend on our tomers and users. One possible result is that the UX design
research questions. We also consider other design research practice should be better prepared with better preparation
opportunities which can better inform the HCI community support measures.
about the existing divergence between UX practice and inno- One possible outcome is that the UX design practice should
vation via ML. be better prepared with better preparation support measures.
The previous studies consistently [17] assert that research
A. SUMMARY OF FINDINGS AND DISCUSSION must affect UX designers’ work. The characterization of
1) THEMES OF EXPERIENCE the consumer was expressed to be relevant in approaching
The themes of experiences highlight the attitudes and chal- a design project, as mature vs immature customers imply.
lenges that designers face while working with ML in the UX The current review advocates for more profound research
design process and why we need to learn more about ML as a into customers that employ machine learning in their digital
design material. In addition, the experience themes different solutions.
features to enhance UX design practice and draw on studies Furthermore, the absence of support for ethical consider-
on designers using ML as a design material. The following ations suggests that designers working with ML face sce-
Table 11 illustrates the experience themes. narios where they have difficulty expressing and presenting
Even poorly organized and fuzzy design projects may be assessment criteria [33]. This research highlights the need
rigorous and disciplined with the correct tools and method- for supporting tools and methodologies to promote ethical
ologies. However, this study implies that the UX design arguments in UX design practice, bolstering the designer’s
practice lacks the competencies to handle scenarios incorpo- confidence and actions. UX designers must be ready for
rating ML effectively. In contrast to earlier research on ML action and arrive with the appropriate toolkit for the project
as a design material in UX design practice (e.g [3], [12], at hand [33].
[21], [36]), these studies adds to what we need to know about The results show that most UX designers have lacked a
ML in UX practice from the perspective of a UX designer. fair understanding of what problems are difficult for ML to
This research [3], [12], [21], [36] helps us to understand solve, which is especially important for customizable appli-
the need to learn more about UX designers’ motivations to cations that use transfer learning to contextualize ML models.
learn more about ML as a design material. In addition, it could However, the results show that most UX designers have a fair
help the UX design profession understand how and why understanding of what problems are difficult for ML to solve.
understanding more about ML as a design material would Since this only refers to the specific set of ML capabilities
be beneficial. Moreover, it could also help UX designers used when working with datasets, it does put into perspective
determine whether this should be approached differently than claims in the previous sections in this review. UX designers
other design materials or deemed a new design material. frequently have misconceptions about ML functionality [8]
According to [3], [12], [21], [36], a lack of ML expertise or have difficulty understanding the restriction of what ML
in UX design practice can have unsettling repercussions. can learn [39].

51510 VOLUME 10, 2022

A. M. H. Abbas et al.: User Experience Design Using Machine Learning: Systematic Review

2) TOOLS gap between conceptual frameworks and detailed instructions

As a result of our research approach, we argue in this review for designing tools to improve the UX design process is
that ML tools should not be ‘‘pushed at creative minds’’ necessary.
simply because they are technically possible to build. Instead,
they should be informed and produced through a thorough 3) ALGORITHMS AND DATASETS
examination of existing design processes. As a result, intro- Data classification is essential since it instructs the system
ducing new tools may alter how designers think about and on classifying users depending on their behaviors. ML algo-
approach their work. rithms can categorize user behavior, which can aid UX
Tools focus on the changeable nature of ML-enabled UX designers in indicating the use of declared terms in prior user
design and allow designers to contribute to the material’s sessions. However, knowing how ML systems work is not the
growth. Unfortunately, ML has remained underutilized to same as knowing where and how to include them into the UX
assist designers, and it has yet to be fully integrated into designs process because ‘‘many of these algorithms have poor
design patterns, education, and prototype tools [12]. Within linkages to varied user experiences after repeated use’’.
the concrete design process used by an industrial partner, UX designers must recognize unexpected user requests that
[39] studied integrating ML into early GUI design stages. are theoretically satisfactory using ML. Due to the increased
[39] reported that an in-depth development process for an usage of ML in the design process, the data scientists are
ML-based tool concept for UI/UX designers. the following UX designers. ML can be considered a tool
Microsoft Custom Vision was used to develop and test a to improve user experience because of its capabilities and
model for detecting GUI elements in photographs of paper issues. However, ML is a difficult medium to build for or with
sketches as a compromise between quickly developing an due to its many complexities. The performance of several
ML-based app and making accurate predictions. Microsoft ML capabilities was scored in a wide range of ways by UX
Custom Vision already comes with a high-quality foundation designers; though, above-par mean ratings for all abilities
model trained on a large amount of generic image data. show that most designers are positive about ML capabilities.
The main limitation in the study of [39] was prototype’s Implying that these issues may implement minimal to
ML model was trained on a small set of sketches. At the well-defined generic ML problems UX designers may have
same time, real-world applications necessitate more in-depth heard of or read about, such as object recognition or face
instruction in a broader range of concepts and more complex interpretation. This type of debate has yet to occur in the
user interface designs. design of ML. What are the unique qualities of ML in terms
Therefore, the current implementation of [39] is limited of increasing UX? What are the differences in its experience
to a prototype level. Still, it is incorporated into a well- features among algorithms or datasets? Answers to these
known tool (Sketch), allowing us to explore practitioners’ issues would allow for a more thoughtful and disciplined.
perspectives within the framework of a real-world design How to make ML available to UX designers and enable
process used by an industrial partner. design-led ML innovation? It would also help UX experts
According to [48], attempts to integrate the ML develop- determine whether and how ML instruction for UX designers
ment process into traditional conceptual design processes, should differ from non-ML professionals in general. Accord-
in which designers consider ML, users, and the situation as ing to Table 6, the data collection methods are Video Capture,
a whole. The Canvas tools it’s only good for the conceptual Survey Log Data, and User Sessions. Video capture is a
stage and can’t be utilized for prototyping. This tool is also method that uses 72 videos captured for facial expressions
limited to UX designers; it can unite professionals from all was chunks to 1-minute video. Then system analyzed the cap-
fields. tured videos as 72 facial expressions to numeric data to build
Therefore, tools are still in the early stages. Finally, it can- the dataset [32]. After that classification model building, each
not cover all conceivable questions, and the question list may model was validated by leave-one-out cross-validation. The
limit creative design activities. In addition, we found that all results show that the combination of SVM-SMOTE over-
developed tools are still in the conceptual stage and cannot sampling and SVM with polynomial kernel provided the
undoubtedly be used in prototyping. Also, we claimed that the best accuracy. However, one limitation of this study is the
tools were not designed with the participants of UX designers. small number of samples. Therefore, larger sample sizes are
For that reason, tools are still in the early stages. required for further validation of the approaches.
Furthermore, we claim that the tools were not created in At the same time, the second method to build a dataset
collaboration with UX designers. However, because current is a survey [38]. The data collection survey was conducted
conceptual design tools aren’t specialized to the ML context, for 1.5 years and resulted in 1121 records. After that, the
there isn’t yet a design process or device that can help design- 675 participants entered the paper IDs, and 446 participants
ers think of new and practical ways to use ML. The lack of uploaded the PDF files of the relevant papers, according to the
research encourages designers to engage in the expanding data collected. However, ten records of users’ data were found
circle of ML by developing design methods that consider ML, to be invalid, and 58 PDF files were found to be meaningless
users, and the situation. As a result, a strategy that bridges the and irrelevant. As a result, 68 invalid records were removed

VOLUME 10, 2022 51511

 A. M. H. Abbas et al.: User Experience Design Using Machine Learning: Systematic Review

from the dataset, leaving 1053 records in total. Nevertheless, Baumann et al. (2019), who utilize this approach to assess
the sample size of a dataset was small, leading to less accurate order values [50]. The features considered in the study of [50]
results [38]. observe a matching logic of value estimation methods. Fol-
Log data is another method used to prepare a dataset by lowing [40], compared several learning methods, including
creating a plugin that users can install in their browser [40]. those that had previously been employed, such as binary
This plugin takes a photo of the user’s face from the webcam decision trees, random forests, AdaBoost, and Multi-Layer
every 2 seconds and mouse and keyboard logs. The critical Perceptron.
shortcoming is that it is impossible to maintain track of abso- A forward feature selection technique was used to find
lute test reliability: users may choose to ‘‘rush’’ the test or the optimal collection of features for each type of model.
be influenced by the testing environment (a problem that also Because emotions are essential aspects of UX, some authors
affects ‘‘traditional’’ approaches), causing the final findings are experimenting with ML algorithms ([51], [52], [53]) to
to be distorted [40]. recognize users’ emotions by monitoring their interactions
User session is the last method in this review [31]. The with the system (e.g., mouse movements).
data was acquired between May 20th and July 20th, 2018,
covering user sessions from an online shop’s website. The V. LIMITATIONS
dataset did not include information on redemption possi- Our investigation demonstrated the associations between UX
bilities. In addition, Koehn et al. 2020 acknowledge that design using machine learning. Previous research suggests
research has a more general drawback, which derives from that design practice lacks understanding of how to work
our decision to use conversion categorization as a vehicle for with ML as a design material effectively and how it can
e-coupon targeting [31]. add value to their practice. Some research is still in its
Finally, it is worth mentioning that no studies used a early stages, with inaccurate results. Other studies are still
benchmark dataset where the results will be more accurate in the conceptual stage and cannot be used for prototyping.
than other studies. Analyzing clickstreams of user sessions, Also, the implementation in a working environment has been
extracting useful information, and making predictions about investigated yet.
their interaction behavior are critical solutions to improve UX However, a more accurate prediction model would be
depending on each user behavior. Incororating benchmark preferable, which might be achieved by increasing the num-
datasets to enhance UX design using ML techniques is still ber of samples utilized in the training phase or the number
missing. All the studies are confirmed that the fields of ML of participants in the data-collection phase. In addition, the
and UX design have still lacked in the following: sample size is limited to selective groups, genders, and distri-
• Lack of awareness among UX designers of the impor- butions. Additional validation of the methods requires investi-
tance of using ML algorithms in improving UX design gations with larger instance extents. Tools in section 3.2 were
and challenges facing UX designers when incorporating trained on a bit of a set of drawings tailored.
ML techniques [8], [12], [36], [49].
• Lack of research on ML algorithms and UX, especially VI. FUTURE DIRECTIONS
in envisioning how ML might improve UX [10]. The results of this peer review revealed a significant amount
• In addition, the studies emphasize the importance of of evidence and recurrent concerns about UX design issues
developing and testing a model to collaborate both ML and recommended strategies using ML, the impact of these
and UX [7], [49], [30]. perceived issues, and the effectiveness of strategies to incor-
porate ML into UX design. This knowledge will be an invalu-
4) ALGORITHMS AND TECHNIQUES able insight into fully understanding the scope of machine
The term ‘‘technique’’ covers a wide range of approaches that learning’s capabilities and techniques to express UX design-
can be utilized to achieve progress on various issues. Because ers’ concerns about their effectiveness in their actual efforts
it’s so broad, it’s usually not specific enough to the specifics to access, execute, or communicate the UX design process.
of any given problem to offer a single answer. Instead, the Further research is needed to explore these areas to pro-
solution to an issue may necessitate a combination of different mote and improve our understanding of ML capabilities
approaches. Koehn et al. (2020) reveal that all possible com- and improve design issues related to the UX design pro-
binations of hyperparameters are evaluated in random order. cess. In addition, developing web-based tools using users’
These 200 distinct classifiers are then applied to each method. behaviors and sessions data integrated with ML algorithms
Next, the area under the receiver operating characteristic to enhance UX design is a promising research area with more
curve (AUC) on the current holdout fold is used to evaluate challenges and directions.
each fit. Finally, the combination of hyper-parameters of the
model with the highest average AUC overall ten folds is VII. CONCLUSION
selected for subsequent comparisons on the test set [31]. This study presented a systematic review indicating the UX
It is essential to mention that cross-validation tuning design using ML from UX designers and experts’ perspec-
demand value estimation methods avoids details leak- tives. The majority of the UX designers who took part in
age from the test set. The benchmark is inspired by this research had no expertise with ML as a design tool.

51512 VOLUME 10, 2022

A. M. H. Abbas et al.: User Experience Design Using Machine Learning: Systematic Review

This suggests some of these issues may arise because of [17] A. Oulasvirta, S. D. Pascale, J. Koch, T. Langerak, J. Jokinen, K. Todi,
the novelty of using ML as a design material. On the other M. Laine, M. Kristhombuge, Y. Zhu, A. Miniukovich, G. Palmas, and
T. Weinkauf, ‘‘Aalto interface metrics (AIM): A service and codebase for
hand, we discovered valuable academic work already with computational GUI evaluation,’’ in Proc. 31st Annu. ACM Symp. User
ML for UX. Interface Softw. Technol. Adjunct, Oct. 2018, pp. 16–19.
We believe that creating such a study will positively impact [18] C. Chen, T. Su, G. Meng, Z. Xing, and Y. Liu, ‘‘From UI design image to
GUI skeleton: A neural machine translator to bootstrap mobile GUI imple-
UX design, benefiting both users and designers. The implica- mentation,’’ in Proc. 40th Int. Conf. Softw. Eng., May 2018, pp. 665–676.
tions for ML development, testing, and feedback collection [19] M. AI. (2018). Transform Sketches into HTML Using AI. [Online]. Avail-
could be huge: not only would more models improve the able: https://sketch2code.azurewebsites.net
[20] B. Wilkins. (2018). Sketching Interfaces. Airbnb.Design. [Online]. Avail-
quality and breadth of feedback. Furthermore, the develop- able: https://airbnb.design/sketching-interfaces
ment of said rougher models would save significant time [21] S. Amershi, D. Weld, M. Vorvoreanu, A. Fourney, B. Nushi, P. Collisson,
in the early stages of app development (where feedback is J. Suh, S. Iqbal, P. N. Bennett, K. Inkpen, J. Teevan, R. Kikin-Gil, and
E. Horvitz, ‘‘Guidelines for human-AI interaction,’’ in Proc. CHI Conf.
most important), pushing the polishing of the user interface Hum. Factors Comput. Syst., May 2019, pp. 1–13.
to the later stages. This could have influenced the outcome, [22] J. Lovejoy. (2018). The UX of AI. Google Design. Accessed: Oct. 26, 2021.
implying the need for a follow-up investigation in the future. [Online]. Available: https://design.google/library/ux-ai/
[23] Q. Yang, J. Zimmerman, A. Steinfeld, and A. Tomasic, ‘‘Planning adaptive
mobile experiences when wireframing,’’ in Proc. ACM Conf. Designing
REFERENCES Interact. Syst., Jun. 2016, pp. 565–576.
[1] C.-J. Wu, D. Brooks, K. Chen, D. Chen, S. Choudhury, M. Dukhan, [24] T. Pedersen, C. Johansen, and A. Jøsang, ‘‘Behavioural computer science:
K. Hazelwood, E. Isaac, Y. Jia, B. Jia, and T. Leyvand, ‘‘Machine learning An agenda for combining modelling of human and system behaviours,’’
at Facebook: Understanding inference at the edge,’’ in Proc. IEEE Int. Hum.-Centric Comput. Inf. Sci., vol. 8, no. 1, pp. 1–20, Dec. 2018.
Symp. High Perform. Comput. Archit. (HPCA), Feb. 2019, pp. 331–344. [25] A. Liberati, D. G. Altman, J. Tetzlaff, C. Mulrow, P. C. Gotzsche,
J. P. A. Ioannidis, M. Clarke, P. J. Devereaux, J. Kleijnen, and D. Moher,
[2] O. Nikiforova, V. Zabiniako, J. Kornienko, M. Gasparoviča-Asīte, and
‘‘The PRISMA statement for reporting systematic reviews and meta-
A. Silina, ‘‘Mapping of source and target data for application to machine
analyses of studies that evaluate healthcare interventions: Explanation and
learning driven discovery of IS usability problems,’’ Appl. Comput. Syst.,
elaboration,’’ Brit. Med. J., vol. 339, Dec. 2009, Art. no. b2700, doi:
vol. 26, no. 1, pp. 22–30, May 2021.
10.1136/bmj.b2700.
[3] D. I. Permatasari, F. F. Hardiansyah, M. A. Wakhidah, and M. B. A. Rasyid,
[26] Y. Sasaki, M. Fukui, J. Hagikura, J. Moriyama, and T. Hirashima, ‘‘Devel-
‘‘UX design documentation application using the five planes method,’’ in
opment of an interactive educational tool to experience machine learning
Proc. 6th Int. Conf. Sustain. Inf. Eng. Technol., Sep. 2021, pp. 29–32.
with image classification,’’ in Proc. IEEE 9th Global Conf. Consum.
[4] L. E. Holmquist, ‘‘Intelligence on tap: Artificial intelligence as
Electron. (GCCE), Oct. 2020, pp. 78–80.
a new design material,’’ Interactions, vol. 24, no. 4, pp. 28–33,
[27] P. Szabo and B. Genge, ‘‘Efficient conversion prediction in E-Commerce
Jun. 2017.
applications with unsupervised learning,’’ in Proc. Int. Conf. Softw.,
[5] Q. Yang, A. Steinfeld, C. Rosé, and J. Zimmerman, ‘‘Re-examining
Telecommun. Comput. Netw. (SoftCOM), Sep. 2020, pp. 1–6.
whether, why, and how human-AI interaction is uniquely difficult to
[28] J. Cruz-Benito, A. Vázquez-Ingelmo, J. C. Sánchez-Prieto, R. Therón,
design,’’ in Proc. CHI Conf. Hum. Factors Comput. Syst., Apr. 2020,
F. J. García-Peñalvo, and M. Martín-González, ‘‘Enabling adaptability in
pp. 1–13.
web forms based on user characteristics detection through A/B testing and
[6] L. M. Policarpo, D. E. da Silveira, R. da Rosa Righi, R. A. Stoffel, machine learning,’’ IEEE Access, vol. 6, pp. 2251–2265, 2018.
C. A. da Costa, J. L. V. Barbosa, R. Scorsatto, and T. Arcot, ‘‘Machine [29] Z. Zhou, Q. Gong, Z. Qi, and L. Sun, ‘‘ML-process canvas: A design tool
learning through the lens of e-commerce initiatives: An up-to-date to support the UX design of machine learning-empowered products,’’ in
systematic literature review,’’ Comput. Sci. Rev., vol. 41, Aug. 2021, Proc. Extended Abstr. CHI Conf. Hum. Factors Comput. Syst., May 2019,
Art. no. 100414. pp. 1–6.
[7] Q. Yang, N. Banovic, and J. Zimmerman, ‘‘Mapping machine learning [30] V. Johnston, M. Black, J. Wallace, M. Mulvenna, and R. Bond, ‘‘A frame-
advances from HCI research to reveal starting places for design innova- work for the development of a dynamic adaptive intelligent user interface
tion,’’ in Proc. CHI Conf. Hum. Factors Comput. Syst., Apr. 2018, pp. 1–11. to enhance the user experience,’’ in Proc. 31st Eur. Conf. Cognit. Ergonom.,
[8] G. Dove, K. Halskov, J. Forlizzi, and J. Zimmerman, ‘‘UX design innova- Sep. 2019, pp. 32–35.
tion: Challenges for working with machine learning as a design material,’’ [31] D. Koehn, S. Lessmann, and M. Schaal, ‘‘Predicting online shopping
in Proc. CHI Conf. Hum. Factors Comput. Syst., May 2017, pp. 278–288. behaviour from clickstream data using deep learning,’’ Expert Syst. Appl.,
[9] W. Xu, ‘‘Toward human-centered AI: A perspective from human-computer vol. 150, Jul. 2020, Art. no. 113342.
interaction,’’ Interactions, vol. 26, no. 4, pp. 42–46, Jun. 2019. [32] K. Koonsanit and N. Nishiuchi, ‘‘Classification of user satisfaction using
[10] Q. Yang, A. Scuito, J. Zimmerman, J. Forlizzi, and A. Steinfeld, facial expression recognition and machine learning,’’ in Proc. IEEE
‘‘Investigating how experienced UX designers effectively work with REGION Conf. (TENCON), Nov. 2020, pp. 561–566.
machine learning,’’ in Proc. Designing Interact. Syst. Conf., Jun. 2018, [33] S. S. Chivukula, C. R. Watkins, R. Manocha, J. Chen, and C. M. Gray,
pp. 585–596. ‘‘Dimensions of UX practice that shape ethical awareness,’’ in Proc. CHI
[11] N. Li, J. Mayes, and P. Yu, ML Tools for the Web: A Way for Rapid Conf. Hum. Factors Comput. Syst., Apr. 2020, pp. 1–13.
Prototyping and HCI Research BT—Artificial Intelligence for Human [34] M. Ghajargar, J. Persson, J. Bardzell, L. Holmberg, and A. Tegen, ‘‘The
Computer Interaction: A Modern Approach, Y. Li and O. Hilliges, Eds. UX of interactive machine learning,’’ in Proc. 11th Nordic Conf. Hum.-
Cham, Switzerland: Springer, 2021, pp. 315–343. Comput. Interact., Shaping Experiences, Shaping Soc., Oct. 2020, pp. 1–3.
[12] Q. Yang, ‘‘The role of design in creating machine-learning-enhanced user [35] M. Winter and P. Jackson, ‘‘Flatpack ML: How to support designers in
experience,’’ in Proc. Assoc. Adv. Artif. Intell., Mar. 2017, pp. 1–6. creating a new generation of customizable machine learning applications,’’
[13] B. Shneiderman, ‘‘User interfaces for creativity support tools,’’ in Proc. Tech. Rep., 2020, pp. 175–193, doi: 10.1007/978-3-030-49760-6_12.
3rd Conf. Creativity Cognition (C&C), 1999, pp. 15–22. [36] M. Chromik, F. Lachner, and A. Butz, ‘‘ML for UX?—An inventory and
[14] E. Horvitz, ‘‘Principles of mixed-initiative user interfaces,’’ in Proc. predictions on the use of machine learning techniques for UX research,’’
SIGCHI Conf. Hum. Factors Comput. Syst. CHI Limit (CHI), 1999, in Proc. 11th Nordic Conf. Hum.-Comput. Interact., Shaping Experiences,
pp. 159–166. Shaping Soc., Oct. 2020, pp. 1–11.
[15] K. Todi, D. Weir, and A. Oulasvirta, ‘‘Sketchplore: Sketch and explore [37] K. Koonsanit and N. Nishiuchi, ‘‘Predicting final user satisfaction using
with a layout optimiser,’’ in Proc. ACM Conf. Designing Interact. Syst., momentary UX data and machine learning techniques,’’ J. Theor. Appl.
Jun. 2016, pp. 543–555. Electron. Commerce Res., vol. 16, no. 7, pp. 3136–3156, Nov. 2021.
[16] A. Swearngin, M. Dontcheva, W. Li, J. Brandt, M. Dixon, and A. J. Ko, [38] Z. D. Champiri, B. Fisher, and C. Y. Chong, ‘‘A contextual Bayesian user
‘‘Rewire: Interface design assistance from examples,’’ in Proc. CHI Conf. experience model for scholarly recommender systems,’’ in Proc. Int. Conf.
Hum. Factors Comput. Syst., Apr. 2018, pp. 1–12. Hum. Comput. Interact., 2021, pp. 139–165.

VOLUME 10, 2022 51513

 A. M. H. Abbas et al.: User Experience Design Using Machine Learning: Systematic Review

[39] D. Buschek, C. Anlauff, and F. Lachner, ‘‘Paper2Wire—A case study of ABDALLAH M. H. ABBAS received the bache-
user-centred development of machine learning tools for UX designers,’’ lor’s degree in information technology from the
I-Com, vol. 20, no. 1, pp. 19–32, 2021. Department of Computer Science and Information
[40] G. Desolda, A. Esposito, R. Lanzilotti, and M. F. Costabile, ‘‘Detecting Technology (CSIT), University College of Science
emotions through machine learning for automatic UX evaluation,’’ in Proc. and Technology (UCST), Gaza Strip, Palestine,
IFIP Conf. Hum.-Comput. Interact., 2021, pp. 270–279. in 2007, and the master’s degree in informa-
[41] R. Rim, M. M. Amin, and M. Adel, ‘‘Bayesian networks for user modeling: tion technology (industrial computing) from the
Predicting the user’s preferences,’’ in Proc. 13th Int. Conf. Hybrid Intell.
Faculty of Information Science and Technology,
Syst. (HIS), Dec. 2013, pp. 144–148.
National University of Malaysia, in 2015. He is
[42] D. Wang, Q. Yang, A. Abdul, and B. Y. Lim, ‘‘Designing theory-driven
user-centric explainable AI,’’ in Proc. CHI Conf. Hum. Factors Comput. currently pursuing the Ph.D. degree with the Fac-
Syst., May 2019, pp. 1–15. ulty of Computing and Informatics, Multimedia University. Two years after,
[43] Q. Yang, ‘‘Machine learning as a UX design material: How can we imagine he began work at the Department of Computer Science and Information,
beyond automation, recommenders, and reminders?’’ in Proc. AAAI Spring UCST, as an Assistant Lecturer in computer science. His current research
Symp. Ser., 2018. Proc. interests include green technology, supply chain management, sustainability,
[44] M. Khondoker, R. Dobson, C. Skirrow, A. Simmons, and D. Stahl, ‘‘A com- web technology, web data mining, and machine learning.
parison of machine learning methods for classification using simulation
with multiple real data examples from mental health studies,’’ Stat. Meth-
ods Med. Res., vol. 25, no. 5, pp. 1804–1823, Oct. 2016.
[45] C. Beleites, U. Neugebauer, T. Bocklitz, C. Krafft, and J. Popp, ‘‘Sam- KHAIRIL IMRAN GHAUTH received the M.IT.
ple size planning for classification models,’’ Anal. Chim. Acta, vol. 760, degree from The University of Melbourne and the
pp. 25–33, Jan. 2013. Ph.D. degree from the University of Malaya. He is
[46] F. Pedregosa, S. Varoquaux, A. Gramfort, V. Michel, and B. Thirion, currently a Senior Lecturer with the Faculty of
‘‘Scikit-learn: Machine learning in Python,’’ J. Mach. Learn. Res., vol. 12,
Computing and Informatics, Multimedia Univer-
pp. 2825–2830, Dec. 2011.
sity. He has authored or coauthored over 20 ref-
[47] S. Haykin, Neural Networks and Learning Machines, 3/E. London, U.K.:
Pearson, 2010. ereed publications in international journals, book
[48] Z. Zhou, L. Sun, Y. Zhang, X. Liu, and Q. Gong, ‘‘ML lifecycle canvas: chapters, and conferences in the area of informa-
Designing machine learning-empowered UX with material lifecycle think- tion retrieval, recommender systems, and big data.
ing,’’ Hum. Comput. Interact., vol. 35, nos. 5–6, pp. 362–386, Nov. 2020.
[49] A. Srivastava, Developing Functional Literacy of Machine Learning
Among UX Design Students. Cincinnati, OH, USA: Univ. Cincinnati, 2021.
[50] A. Baumann, J. Haupt, F. Gebert, and S. Lessmann, ‘‘The price of privacy,’’
Bus. Inf. Syst. Eng., vol. 61, no. 4, pp. 413–431, 2019.
CHOO-YEE TING is a Professor with the Faculty
[51] P. Zimmermann, S. Guttormsen, B. Danuser, and P. Gomez, ‘‘Affective
of Computing and Informatics, Multimedia Uni-
computing—A rationale for measuring mood with mouse and keyboard,’’
Int. J. Occupational Saf. Ergonom., vol. 9, no. 4, pp. 539–551, Jan. 2003. versity. He is an Active Researcher in the areas of
[52] S. Salmeron-Majadas, O. C. Santos, and J. G. Boticario, ‘‘An evaluation of Bayesian networks, location analytics, and health
mouse and keyboard interaction indicators towards non-intrusive and low insurance analytics.
cost affective modeling in an educational context,’’ Proc. Comput. Sci.,
vol. 35, pp. 691–700, Jan. 2014.
[53] L. Vea and M. M. Rodrigo, ‘‘Modeling negative affect detector of novice
programming students using keyboard dynamics and mouse behavior,’’ in
Proc. Pacific Rim Int. Conf. Artif. Intell., 2016, pp. 127–138.

51514 VOLUME 10, 2022