See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/378458769

Recommender System in Academic Choices of Higher Education: A Systematic

Review

Article in IEEE Access · January 2024

DOI: 10.1109/ACCESS.2024.3368058

CITATION READS

1 442

5 authors, including:

Nabila Kamal Arifur Rahman

United International University Khulna University of Engineering and Technology
3 PUBLICATIONS 10 CITATIONS 8 PUBLICATIONS 1 CITATION

SEE PROFILE SEE PROFILE

Khondaker A. Mamun
University of Toronto
106 PUBLICATIONS 735 CITATIONS

SEE PROFILE

All content following this page was uploaded by Arifur Rahman on 12 March 2024.

The user has requested enhancement of the downloaded file.

Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.
Digital Object Identifier 10.1109/ACCESS.2023.0322000

Recommender System in Academic

Choices of Higher Education: A
Systematic Review
NABILA KAMAL1 , FARHANA SARKAR2 , ARIFUR RAHMAN 1
, SAZZAD HOSSAIN3 ,
KHONDAKER A. MAMUN1,2,4 ,
1
Advanced Intelligent Multidisciplinary Systems Lab (AIMS Lab), IRIIC, United International University, Dhaka, Bangladesh
2
Education Research Foundation (ERF), Dhaka, Bangladesh
3
University Grant Commission (UGC), Government of Bangladesh, Bangladesh
4
United International University, Department of Computer Science and Engineering, Madani Avenue, Dhaka-1212, Bangladesh
Corresponding author: Khondaker A. Mamun ([email protected])
This work was supported by the ICT Division, Ministry of Posts, Telecommunications, and Information Technology, Bangladesh and
Institutite of Advanced Research (IAR), United International University under grant number IAR-2024-Pub-012. We express gratitude to
our colleagues from the Advanced Intelligent Multidisciplinary Systems Lab (AIMS Lab), United International University (UIU),
Bangladesh, whose insights and expertise greatly enriched this study.

ABSTRACT Recommender systems have gained significant attention as powerful tools for supporting
decision-making processes in various domains. However, the understanding of their impact and application
in the field of academic choices in higher education remains limited. This systematic review aims to provide
a comprehensive summary of the current knowledge regarding recommender systems utilized in the context
of academic choices and advising in higher education. The study is based on the systematic analysis of
a set of primary studies (N = 56 out of 1578, published between 2011 and 2023) included according to
defined criteria. The articles were categorized based on specific criteria, and their findings were analyzed
and synthesized. Results show that the hybrid strategy has been the most effective method for producing
recommendations. Evaluation measures such as offline experiments and case-study validation were promi-
nently observed in the empirical studies, providing insights into the effectiveness of recommender systems.
The findings reveal that the design of recommender systems in higher education is context-specific, with
researchers considering various parameters to tailor recommendations to individual needs. However, most
of the selected articles relied on lab-based studies rather than real-world applications, indicating a need
for further research in practical settings. This systematic review also identifies future research directions,
including the incorporation of deep learning technologies and the analysis of personality traits. By providing
a comprehensive overview of the current state of recommender systems for academic choices in higher
education, this review offers valuable insights for researchers and practitioners, guiding the development of
more effective and personalized recommendation systems to unlock the full potential of individuals in their
academic journey.

INDEX TERMS Academic choices, higher education, recommendation systems, course recommendation
systems, holland code assessment, systematic literature review.

I. INTRODUCTION in areas such as e-commerce [1], [2] entertainment

In recent years, recommender systems have (YouTube [3]), (Netflix [4]), and personalized con-
emerged as powerful tools for aiding decision- tent recommendations. However, their potential im-
making processes across various domains. These pact and application in the field of academic choices
systems have been extensively studied and applied and advising in higher education have received

VOLUME 11, 2023 1

 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

limited attention. Understanding how recommender ficulties arise from differences in learner’s edu-
systems can support students’ academic decision- cational interests and needs [8]. These challenges
making processes is crucial for providing effec- include data sparsity, cold start [7], privacy con-
tive guidance and enhancing educational outcomes. cerns, ensuring diversity in recommendations, and
Recommender algorithms can be used to make addressing biases that may arise from the data used
smart decisions in complex information systems and to train the system. The approach utilized to produce
help the users decide upon useful materials [5]. The recommendations is also an example of difficulties.
purpose of this systematic review is to provide a For instance, the way content-based recommender
comprehensive summary of the current knowledge systems manage data is inextricably linked to over-
regarding the utilization of recommender systems specialization [9], [10]. Accurately identifying user
in the context of academic choices and advising expectations and recommendations is one of the
in higher education. The review categorizes the main challenges [11]. Differences in learners’ ed-
selected articles based on specific criteria, allowing ucational preferences and needs lead to challenges
for a structured analysis of the various dimensions [8]. There are questions about how to assess the
and approaches adopted in the design and imple- effectiveness of RS from an educational perspec-
mentation of recommender systems for academic tive. Applying the classic recommender evaluation
choices. Notably, the articles primarily focus on methodologies is a common strategy to assess the
three distinct domains of academic choices, with quality of educational recommenders [12]. This
course recommendation being the most prevalent method evaluates the performance characteristics
area of interest. of the system, such as its precision and prediction
Students’ often get confused while choosing the accuracy. However, system effectiveness in the ed-
most suitable course in higher education that meets ucational setting must take into account students’
their requirements. This confusion arises due to the learning progress. This feature adds significant chal-
vast array of available courses and the complexity lenges to how to assess RS effectively from an
of educational systems. To address this challenge, educational perspective. The scientific community
recommender systems have emerged as valuable has become increasingly interested in RS [13], and
tools for supporting students in making informed in recent years, substantial study has been done
decisions about their academic choices in higher to solve these concerns [6], [14], [15], [16]. Data
education. Recommender systems utilize advanced mining, information filtering, education and infor-
algorithms and techniques, such as machine learn- mation technologies, machine learning, and other
ing and data mining [6], to analyze relevant data computational approaches are only a few examples
and generate personalized recommendations for stu- of how RS has evolved into an area of application
dents. By considering factors such as students’ pref- [6] in education [17].
erences, academic performance, career aspirations, Despite these challenges, recommender systems
and feedback from previous students, these systems hold great potential to revolutionize the course se-
can assist in identifying the most suitable courses lection process in higher education. The efficient
that align with individual requirements. The benefits design of the recommender system in education
of recommender systems in academic choices are will help the students by generating the appropriate
manifold. Firstly, they save students time and effort recommendations [18]. As technology advances and
by providing them with tailored recommendations, more data becomes available, it is expected that
eliminating the need for extensive manual research. recommender systems will become even more accu-
Secondly, they enhance the accuracy of decision- rate and effective in providing personalized recom-
making processes by considering a wide range of mendations to students. In addition, recommender
relevant information and filtering out irrelevant op- systems offer valuable support to students facing the
tions. Moreover, these systems promote personal- daunting task of choosing the most suitable courses
ized learning experiences, enabling students to ex- in higher education. By leveraging advanced algo-
plore courses that align with their interests and rithms and analyzing relevant data, these systems
goals, Recommender System (RS) can contribute can alleviate confusion, save time, and enhance
to their academic performance and motivation by the accuracy of decision-making. As research and
indicating personalized learning content [7]. development in this field continue to progress, rec-
However, there are challenges associated with the ommender systems will play an increasingly vi-
development and implementation of recommender tal role in helping students navigate the multitude
systems in the context of academic choices. Dif- of course options [19], ensuring they make well-
2 VOLUME 11, 2023
Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

TABLE 1: Research questions that are the focus of this systematic review

ID Research Questions Rationale

RQ-1 What are the various purposes of RS in Overview of the key areas in higher educational
higher educational admission? advising. Thus, possible findings from this question will
recognize areas of academic advising where RS is
applied.
RQ-2 What approaches are utilized in This research question seeks to get insights into the most
developing higher education admission commonly used techniques, methods, or algorithms in
RS? recommendation systems in educational choices.
RQ-3 What types of parameters are used in This question aims to explore the specific parameters
the design of RS? used in designing each RS.
RQ-4 Which platform is used for the This question aims to find whether the RS has been
deployment of recommender system? deployed in a web/mobile environment or proposed
model/framework.
RQ-5 What types of evaluation strategies are The goal of this query is to analyze validation criteria
applied to RS? utilized in this domain.
RQ-6 What are the outcomes of this It is crucial to explore whether the proposed RS has any
recommender system in education? educational outcome.
RQ-7 What parameter’s hierarchy should The parameters hierarchy ensures a nuanced and
follow in the design and development personalized approach, aligning academic suggestions
of course recommendation system? with individual preferences, strengths, and aspirations
for an effective and satisfactory user experience.
RQ-8 What are the significance and Understanding the significance and relevance of each
relevance of each parameter in the parameter in the course recommendation system is
course recommendation System? essential to tailor recommendations based on diverse
factors, ensuring a comprehensive and personalized
approach that meets the unique needs and preferences of
individual users.
RQ-9 How holland tratis can help in course Holland traits can aid in a course recommendation
recommendation system? system lies in leveraging vocational preferences to
provide tailored academic suggestions, aligning
educational paths with individual strengths and
inclinations for an enhanced and personalized learning
experience.

informed choices that align with their academic and II. LITERATURE REVIEW
professional aspirations. Recommender systems (RSs) have gained signifi-
This review article is organized as follows: In Sec- cant interest in education as technologies support-
tion II, related works are presented. Section III ing personalized teaching and learning experiences.
describes the protocol applied to conduct the sys- Over the past 12 years, research efforts have focused
tematic literature review. Section IV presents a syn- on mapping and summarizing various aspects of
thesis of the important results guided by research RSs in education. By exploring RSs in education,
questions. Section V highlights the comprehensive researchers aim to enhance teaching and learning by
outcome of this review and identifies some research tailoring educational content to individual learners.
gaps and possible solutions. Section VI describes The scientific community’s attention to RSs reflects
future research direction in this field. Section VII a growing interest in utilizing technology to improve
represents the practical uses of course recommen- educational experiences through personalized rec-
dation system. Finally, Section VIII concludes this ommendations.
review. In [20] a comprehensive review of technology-
enhanced learning recommender systems was car-
ried out. The authors analyzed 82 recommender
VOLUME 11, 2023 3
 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

systems from 35 different countries published from ular research questions [24], [25]. Although con-
2000 to 2014 and provided an overview of the area. ventional reviews aim to summarize the findings
This study explores various aspects of recommender of several studies, systematic reviews use precise
methods, information sources, and assessments in and strict guidelines to select, critically analyze,
education and information technologies. It catego- and summarize all research on a specific topic. We
rizes selected publications using a provided frame- have performed a systematic literature review by
work. considering the methods outlined in [24] to obtain
In [21], Rivera et al. conducted a systematic a synopsis of recommendation systems regarding
mapping to present a comprehensive overview of higher educational academic choices. Our pipeline
the ERS (Educational Recommender Systems) do- involves several stages to conduct the systematic
main. Their study covered a broad range of pa- literature review, divided into three phases, each
pers and aimed to identify global characteristics in producing an output [25] and summarized in Fig.
ERS research. Similarly, Pinho, Barwaldt, Espín- 2. Details concerning the steps of our systematic
dola, Torres, Pias, Topin, Borba, and Oliveira (2019) review are described in the following subsections:
performed a systematic review of ERS, focusing
on different questions and utilizing different repos- A. REVIEW PLANNING
itories. Both works shared a common concern in The first phase involves defining the underlying
providing insights into the evaluation methods of need for systematic review, formulating research
these systems and the main techniques employed in questions (RQs) to guide the work, developing
the recommendation process. the research protocol, which involves identifying
In their research, [22] focused on course recom- the keywords and operators, exploring literature
mendation systems and conducted a comprehensive sources, defining inclusion and exclusion criteria,
review of techniques and parameters used in this and establishing a search strategy.
type of Educational Recommender Systems (ERS).
Additionally, they defined a taxonomy of the factors 1) Search, Selection, and Extraction
considered in the course recommendation process. The second phase includes searching for and identi-
On the other hand, in [23], the authors conducted fying relevant research, selecting research articles,
a review on affectivity based ERS. Their study analyzing quality by applying criteria, retrieving
presented a macro analysis, identifying key authors and monitoring data, synthesizing observations, and
and research trends while summarizing various as- identifying possibilities for future research.
pects of recommender systems related to affectivity.
These aspects included the techniques employed in B. RESEARCH QUESTION
affectivity analysis, the sources of affectivity data Formulating the research question(s) (RQ) is the
collection, and the methods used to model emotions. critical component of any systematic review, as it
In [16] the authors present a systematic literature focuses the study, determines the methodology, and
review on recommender systems in the educational guides all the stages of analysis and reporting [24].
domain. They analyzed 16 out of 756 primary Our research purpose is to examine an overview of
studies, published from 2015 to 2020. The review the related work in using recommender systems re-
reveals the dominance of the hybrid approach for garding higher educational course advising systems.
recommendation production and the limited focus Therefore, to achieve a better understanding of the
on accuracy in evaluation studies. It emphasizes the current literature, it is vital to formulate a series
need for multidimensional evaluation frameworks of research questions, each one addressing various
to assess the pedagogical effectiveness of recom- facets of RS in higher educational choices. Table
menders. The paper also identifies and discusses key I outlines all the research questions of this review
limitations, highlighting areas for future research to along with their rationale.
enhance recommender systems in education.
C. SEARCH STRATEGY
III. METHODOLOGY In this systematic literature review, we employed a
A systematic literature review (SLR) primarily in- well-structured search strategy in two stages. Firstly,
cludes a comprehensive and rigorous plan and we identified digital repositories for searches, in-
search strategy to minimize bias by defining, as- cluding ACM Digital Library, IEEE Xplore, Science
sessing, and categorizing all related research studies Direct, and Semantic Scholar, conducted through
on a specific topic, providing answers to partic- Google Scholar. Secondly, we defined keywords and
4 VOLUME 11, 2023
Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

FIGURE 1: Detailed workflow of the systematic search, based on the PRISMA workflow guidelines

TABLE 2: Definition of inclusion and exclusion criteria

Criteria Sl. No Definition
IC1 Papers supporting formal educational choices and advising by using a
recommendation system.
Inclusion Criteria IC2 Full papers, academic journals, and conference proceedings studies.
IC3 Literature published in the field of recommender system, last 13 years
(2011 – 2023)
EC1 Any other literature review regarding academic choices and admission is
excluded from this review
Exclusion Criteria EC2 Papers that are focused on other areas of educational recommender
system
EC3 Studies include non-formal education (e.g., Virtual learning environ-
ments, MOOCs)
EC4 Publications that do not clearly state the algorithm/approaches being
used are excluded
EC5 Papers that are not written in English

combined them with Boolean operators to create D. SELECTION CRITERIA & SCREENING
search strings. The list of keywords and correspond- PROCESS
ing search strings used for each digital library is To ensure that selected research articles meet the
presented in Table II and Table III, respectively. The scope of this systematic review, we applied an
search resulted in an initial set of 1578 potential inclusion-exclusion criterion to the initially ob-
primary studies for assessment of eligibility and tained studies to determine whether a paper should
inclusion. be included in the final review. Therefore, the in-
clusion and exclusion criteria are defined as Table
2. We performed preliminary selection, defined the
inclusion and exclusion criteria, and appraised the
initially selected papers. Paper titles, abstracts, and
VOLUME 11, 2023 5
 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

[28], the authors exert grade forecasting, which

specifies the success rate for future course selection
and thus recommends the best possible courses.
Areas of higher education advising are not con-
fined to course recommendations and go beyond ad-
vising suitable universities, colleges, or even degree
programs for the students. According to extracted
data, another significant use of the recommender
system in 12 studies is to provide recommendations
on specific or best-fitted colleges or universities.
FIGURE 2: Stages undertaken for crafting the Most studies [29], [30], [31], [32], [33], [34], [35],
systematic literature review (SLR). [36], [37], [38] predict a set of most suitable uni-
versities or colleges for the admission of the new
students. Only one study [39] determines the educa-
keywords were screened by the authors to apply tional institution befitting university students.
the inclusion criteria. A total of 1578 studies were However, a few studies, representing 8.9% of all
included for full paper reading, where we applied primary ones, have been focusing on suggesting the
the exclusion criteria. Among the screened publi- candidate’s most suitable degree programs. Included
cations, 56 research articles were sorted for this studies either recommend a suitable study track or
study. Fig. 1 demonstrates the detail of the initially department [31], [40], [41] or provide a list of prob-
found studies from each digital library and selected able majors depending on a student’s interest [30],
primary studies. [32]. Besides, the following two studies have fo-
E. DATA EXTRACTION AND ANALYSIS cused on multiple areas: [35] proposes an automated
We extracted data by splitting each research ques- multidimensional framework for recommending a
tion into precise parameters for determining a range suitable program, relevant courses, and appropriate
of potential scopes. instructor for each student, and [41] provides a
Selected papers have been thoroughly read, ana- conceptual ontology-based recommendation frame-
lyzed, and categorized based on the various scopes work to help students select both universities and
of each criterion. Table IV delineates possible an- majors that conform to their preferences.
swers to each research question along with the pos-
sible features extracted from the criteria. B. RECOMMENDER SYSTEM APPROACHES
The purpose of research question 2 is to classify
IV. RESULT the nature of the approaches to the recommendation
A. AREAS IN HIGHER EDUCATION systems yielded in these literatures.
Although the demands of the recommendation sys-
tems span across several domains of education, a 1) Hybrid Approaches
few areas are addressed in terms of the academic From Table VI, it is observed that 24 papers, com-
choice sector. RQ1 seeks to know the main points prising 43% of the reviewed literature, are based
of interest in the research on higher educational ad- on the combination of different recommender types,
vising. After extracting studies from Table III based known as hybridization. Decision trees are fre-
on the features of RQ1, the purpose of the reviewed quently employed in hybrid approaches. The au-
papers is to provide recommendations mainly in thors of [29] use hybrid methods of a decision tree
three areas of educational choices. This includes and association rules to recommend a specific insti-
predicting an institute (college, university, or gradu- tute. In [31] and [39] the authors propose a hybrid
ate school), an academic discipline or degree pro- framework for university admission by integrating
gram, and lastly, an academic course suitable for the back-propagation neural network algorithm and
a student. From Table III, it is apparent that the the C4.5 decision tree. Similarly, [37] focuses on a
predominant area is course recommender systems; combined method of random forest and Multivariate
37 papers, comprising 66.07% of the reviewed arti- Adaptive Regression Splines (MARS) to predict a
cles, recommend appropriate academic compulsory list of the best colleges. Karzai [41] proposes a hy-
or elective-specific courses in an undergraduate or brid web recommender system by combining neural
graduate program according to the student’s goals, networks (NN) and decision tree (DT). [61] focuses
success rate, or preferences. In some studies [26]– on recommending an appropriate elective course
6 VOLUME 11, 2023
Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

TABLE 3: List of search strings

Digital Library Search String
(acmdlTitle: (“Recommender System/s” “Recommendation System/s”) OR
recordAbstract: (“Recommender System/s” “Recommendation System/s”))
AND ((acmdlTitle: (“higher education”) OR recordAbstract: (“higher education”))
ACM Digital Library
AND ((acmdlTitle: (“university”) OR recordAbstract: (“university”)) AND
((acmdlTitle: (“program”) OR recordAbstract: (“program”)) AND ((acmdlTitle:
(“course”) OR recordAbstract: (“course”))
(“Recommender System/s” OR “Recommendation System/s”)
IEEE Xplore
AND (“higher education”) AND (“university” OR “program” OR “course”)
(“Recommender System/s” OR “Recommendation System/s”)
Science Direct
AND (“higher education”) AND (“university” OR “program” OR “course”)
(“Recommender System/s” OR “Recommendation System/s”) AND
Semantic Scholar
(“higher education”) AND (“university” OR “program” OR “course”)
Web of Science recommender system in higher education for course selection

TABLE 4: Criteria and features extracted from each RS

RQ ID Criteria Scopes Description
Recommendation of
Feature describes if the study recommends a/or list of institutes
C1. Areas in higher college/university
RQ1
education Recommendation of
Feature describes if the study recommends a/list of suitable degree/s
degree program
Recommendation of Feature describes if the study recommends suitable primary/elective
course/curriculum courses
Clustering based Feature determines if the proposed solution of the primary study is
recommendation based on model (clustering) based recommendation approach
Feature determines if the proposed solution of the primary study is
RQ2 C2. RS approach Classification/Rule
based on model (classification/Rule based/Neural network) based
based / Neural Network
recommendation approach
Feature determines if the proposed solution of the primary study is
Collaborative Filtering
composed of collaborative filtering recommendation approach
Hybrid Approaches Feature determines if the proposed solution of the primary study is
(combination of the above) composed of collaborative filtering recommendation approach
Content based Feature determines if the proposed solution of the primary study
recommender system proposes a content based recommendation approach
C3. Identifying and categorizing Explore parameters for a This research question seeks to delineate the specific types of
RQ3 specific parameters detailed understanding. parameters employed in the design of university course
recommendation systems, contributing to a nuanced comprehension
of the system’s architecture and functionality.
Feature denotes whether the study proposes an algorithm or series of
Algorithms
algorithms
RQ4 C4. RS development Feature denotes whether the study proposes a series of procedures for
Methods
recommendation
System Feature denotes whether the study is deployed in a platform (web, App)
Framework This feature denotes that if the study purposes has a framework
Feature implies that the proposed solution has been assess with some
Experiment
experiments
C5. RS empirical
RQ5 Feature implies that the proposed solution has been analyze with academic
validation Academic
case study
Feature implies that the proposed solution has been evaluate with online/pen
Survey
paper survey
Not validated Feature implies that the proposed solution has not been validated
Feature determines if the primary study has an educational outcome, i.e.
C6. RS educational
RQ6 Yes study has been used in real life environment to recommend educational
outcome
choices
C7. Establishing an effective Defining a structured hierarchy for This research question aims to determine a hierarchical arrangement
RQ7 parameter hierarchy to course recommendation system of parameters in course recommendation systems for personalized
align course recommendations and user-centric academic suggestions.
C8. Assessing the importance Evaluating the significance This research question aims to highlight the importance of each
RQ8 of each parameter and relevance of parameters parameter in the course recommendation system for tailoring personalized
recommendations to diverse user needs and preferences.
C9. Exploring the contribution Investigating how leveraging This research question explores using Holland traits in a course
RQ9 of Holland traits Holland traits enhances recommendation system to enhance personalized learning by aligning
educational path alignment suggestions with individual strengths and inclinations.

by determining a successful relationship between students.

previous courses taken by Computer Engineering
The system analyzed transcripts of 100 students
VOLUME 11, 2023 7
 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

TABLE 5: Areas of academic choices

Scopes: Domain Publication Total
[29], [30], [31], [32], [33], [34], [35], [36], [37], [38],
Recommendation of college/university 12
[39], [42]
Recommendation of degree/program [37], [41], [42], [43], [44] 5
[26], [27], [28], [45], [46], [47], [48], [49], [50], [51],
[52], [53], [54], [55], [56], [57], [58], [59], [60], [61],
Recommendation of course/curriculum 37
[62], [63], [64], [65], [66], [67], [68], [69], [70], [71].
[72], [73], [74], [75], [76], [77]

to categorize mandatory and elective courses. It cal- single score denoting the degree of suitability of a
culated the effect rate between courses and extracted course for target students using Backpropagation.
rules using the C5.0 decision tree, finally developing Furthermore, [60] utilizes an ontology-based hybrid
a fuzzy logic model based on these rules. filtering framework comprising CB and CBF filter-
In article [30], the authors combine a hybrid ing, with CBF measuring similarities between user
approach of multiclass Support Vector Machine preferences and courses, and CF observing user-
(SVM) and K-Nearest Neighbor (KNN) to rec- user similarities in the system. The recommendation
ommend appealing graduate programs. A. Ragab list is then generated by combining the scores of CF
proposes in [34] and [35] a recommender system & CBF. Besides, model-based CF is another choice
comprising two cascading recommenders and a pre- for implementing the RS framework. [27] applies a
dictor to provide university recommendations and CF RS using K-means clustering to cluster similar
predictions. Both track and college recommenders students and find similarity between target students
use knowledge discovery rules, while the predictor and cluster groups by utilizing the N-nearest neigh-
compares current and previous student data avail- borhood technique, and generates a list of elective
able in the system, forecasting the most fitting col- course recommendations along with the expected
lege for a student. grade by applying the association rule mining algo-
Another choice of hybridization technique in- rithm. [66] aims to provide a 4-year study strategy
volves combining collaborative filtering (CF) and by considering multiple constraints. The authors
content-based filtering (CBF). In [40], CF and CBF propose a hybrid model in which the min-cost-max-
techniques are combined, where CF computes user flow algorithm is used to solve credit constraints
similarities using Euclidean distance, and CBF cal- and evaluate the usefulness of courses by combin-
culates student interest in academic tracks to rec- ing the scores of Course registration possibility by
ommend a suitable diploma track. [26] presents a matrix factorization, student performance prediction
two-stage collaborative filtering approach, employ- by CBF, career interest ratings, and interest levels on
ing an Artificial Immune System to predict course skills thus generating a learning plan by sorting a list
grades and make recommendations. [49] combines of courses into a directed graph and determining the
Alternating Least Square (ALS), a model-based CF priorities of courses.
algorithm, with TF*IDF, a popular content-based
filtering approach, to recommend suitable courses 2) Collaborative Filtering
for college students. In [58], personalized elective Collaborative filtering is a traditional filtering ap-
course recommendations are proposed, with two proach for information filtering, which represents
preference estimations calculated based on student 20% of the total of the studies. Paper [36] rep-
and course information separately, and a genetic resents Multi-Criteria Collaborative Filtering (MC-
algorithm configures the relevance of each crite- CF) along with Dimensionality Reduction tech-
rion to provide user-specific suggestions. In [59], niques to yield university or college recommen-
a course recommendation system measures simi- dations. [47] proposes a web-based course advis-
larity of course topics, matches descriptions and ing system using model-based CF (K-means algo-
tags between courses and users’ profiles and level, rithm), whereas study [48] represents a framework
and analyzes course sentiments using Linear dis- for university elective courses utilizing Pearson Re-
criminant analysis (LDA), word-correlation factors, lationship Coefficient and Alternating Least Square
and SentiWordNet score respectively. It predicts (ALS). The authors in [50] also recommend a course
ratings using Matrix Factorization and generates a with CF based Bayesian Personal Ranking Matrix
8 VOLUME 11, 2023
Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

Factorization (BPRMF) algorithm, whereas the au- behaviors, followed by converting variables into
thors in [52] recommend suitable study programs fuzzy variables and mining fuzzy association rules
using item-based CF algorithms. [54] outlines a in each cluster, yielding course selection rules to
design of a recommendation system based on the recommend courses along with the predicted scores
28 graduating attributes (developing values) of stu- for a student.
dents. Lastly, [56] predicts master’s course remarks,
thus recommending a suitable course to students’ 5) Expert Systems
using singular value decomposition (SVD) and CF. A few studies, comprising 5% of the total papers,
focus on developing expert systems for course ad-
3) Classification / Rule-based / Neural Network vising. For instance, [78] suggests a rule-based ex-
Supervised learning methods such as classification pert system by employing Oracle Policy Automa-
and rule-based schemes are also employed to rec- tion (OPA) software to assist undergraduate stu-
ommend. Seven papers, representing 16% of the dents in academic course selection. [63] proposes
total studies, are based on different classification an educational advisory system by employing fuzzy
and rule-based techniques. [33] implements the K- logic into an expert system, and [65] presents the
nearest neighbor (KNN) algorithm to recommend a Course Advisory Expert System (CAES), consist-
list of graduate schools to the users. Moreover, [38] ing of rule-based reasoning (RBR) and case-based
uses K-nearest Neighbor, Random Forest, SVM reasoning (CBR).
separately on the training data and founds SVM
performs better than the other two. Similarly, [57] 6) Swarm Intelligence
compares the performance of the Linear Regression Among all the studies only one study [64] compares
Model, Naïve Bayes, Support Vector Machine, K- five different swarm intelligence algorithms, e.g.,
Nearest Neighbor, and Decision Tree Classifiers Ant Colony Optimization (ACO), Particle Swarm
to identify appropriate courses goes with students’ Optimization (PSO), Intelligent Weed Optimization
grade. However, [45] involves extracting rules from (IWO), Bee Colony Optimization (BCO), and Bat
the preferences of previous students and thus rec- Algorithm (BA) for student course recommenda-
ommending by checking the similarity between the tion. Among them, ACO applies a hybrid CF and
courses pursued by the students and the precedents content-based filtering approach, whereas others de-
of the rules. Besides, Neural networks are also pop- pend on CF.
ular, and their usage is reported to be promising
for both hybrid and single methods. Specifically, 7) Semantic Analysis and Graph-Based Approach
the proposed framework, intelligent recommenda- Alternatively, [79] involves a series of algorithms,
tion system (IRS) [51] employs multi-layered feed including TF-IDF, word2vec, latent semantic analy-
forward NN to recommend the appropriate courses sis (LSI), and Doc2vec utilized for similarity anal-
where success chance is higher, and [55] recom- ysis based course recommendation. Course Rank
mends courses at the graduate level by applying [80], provides course recommendations based on
feed-forward neural networks. the Ford-Fulkerson max-flow algorithm with con-
straints (e.g., prerequisites, requirements, etc.) fo-
4) Clustering cused on satisfying the degree program require-
A few studies—less than 0.7%—refer to using a ments at each semester. Moreover, [81] proposes
clustering-based approach. [32] designs a college a Personalized Forecasting Model (N-PSEF and
recommender system by utilizing a weighted clus- BPSEF), a personalized recommendation system for
tering process, WCLUSTER. The authors in [41] academic curriculum paths based on performance
propose an ontology-based framework to distin- traits, learning styles, and cognitive traits for each
guish students’ interests and skills in order to pro- individual.
vide a recommendation for a university. However, C. TYPES OF DATA SOURCES USED RS
the proposed framework needs to be evaluated. RQ3 seeks to know the types of input parameters
Moreover, a framework to recommend course en- that are used in developing RS. As seen in Table
rollment based on clustering techniques is explored VII, various input feature groups are used in the
in [44]. Apart from that, [28] applies a K-means design of RS. All the research articles comprised
clustering strategy with different numbers of clus- at least two groups of parameters. Predominantly,
ters using Principal Component Analysis (PCA) 23 studies, representing 41% of the total studies,
to identify students with similar preferences and combine more than one type of data in design-
VOLUME 11, 2023 9
 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

TABLE 6: Approaches used in developing recommendation system

Scopes: Approach Publication Total
Clustering [32], [42], [44] 3
Classification/ Rule based [33], [38], [43], [45], [51], [55], [57], [67], [82], [83], [84], [85],
13
Neural Network [77], [86]
Collaborative Filtering [36], [46], [47], [48], [50], [52], [53], [54], [56] 9
Similarity Analysis [79], [74] 2
Expert Systems [62], [63], [65] 3
Swarm Intelligence [64] 1
Graph - Based [80], [87], [76] 3
Hybrid Approaches [29], [39], [30], [31], [34], [35], [40], [37], [41], [26], [49], [58],
(Combination of the above) [59], [60], [61], [27], [28], [66], [88], [89], [72], [74], 24
[73], [90]

ing RS. Academic data comprises university/college entrepreneurial spirit & service orientation, demo-
major, GPA, course grades, and entrance score. De- graphic data, study preferences, goal orientation,
mographic Data such as age, gender, and ethnicity communication style, multilingual proficiency, tech-
are proven to be crucial parameters in these stud- nology adoption, cultural preferences, time man-
ies. Moreover, 12 primary studies use one or more agement, prioritization skills, course data. In Table
demographic parameters. Furthermore, the course 7 and 8 we have shown each of these parameters
recommender system mainly comprises course data, significance and relevance in the establishment of
e.g., name, type, credit, department, description, course recommendation system. In the previous
instructor information, and ratings of each course studies each of these parent parameters was splitted
enrolled or preferences. These data categories rep- into several child parameters. For each of those child
resent 36% and 14%, respectively. parameters a test was conducted. The parent param-
Recommenders for graduate school are mainly eter’s score was the summation of corresponding
taking into account Performance data (standard test child parameters scores.
scores (GRE, TOEFL), research publication, work Let’s denote:
experience) and Institute Profile (location, type, P as the parent parameter’s score,
ranking, safety, facility, admission requirements), Ci as the score of the ith child parameter.
which comprise only 3 studies. In some studies The mathematical equation representing the sum-
(around 13% of all studies), RS are designed based mation of the child parameters’ scores for the parent
on ratings of user interest (personal, academic, or parameter would be:
professional) and several aspects of an institute’s X
profile, such as location, facilities, and faculties. P = Ci (1)
A few studies utilized parameters from more than i
two categories. [43] uses user interest, GPA, and This equation expresses that the parent parameter’s
test scores. [36] combines personality type with score (P) is the sum of the scores of all correspond-
other parameters, and [18] consists of data like ing child parameters (Ci ).
the skills of learners, family income, and institute In Table 9, 10, 11, 12, 13, 14 and 15 we have
profile. In [28], the authors utilize age, gender, represented each of the parent parameter’s corre-
high school GPA, and the score of seven subjects sponding child parameters.
on the university entrance exam, along with the
university elective course description and grade. In D. DEVELOPMENT PLATFORM OF RS
this study, we have observed that, total 24 parent RQ4 seeks information on whether the proposed so-
parameters was used in the course recommendation lutions in primary studies are an algorithm/series of
system. The parent parameters are: self-regulatory algorithms, a web- or mobile-based application, or a
learning strategies, user interest, approach towards framework. The analysis of RQ4 in Table VIII illus-
learning, performance data, psychosocial contextual trates that most reviewed papers propose systems,
factors, institute profile, personality dimensions, which represent 39.3% of the total of the studies,
skills of learners, motivational factors, academic followed by studies corresponding to methods, with
data, cognitive preferences, ratings, learning styles, 38.81% then
10 VOLUME 11, 2023
Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

TABLE 7: Significance and relevance of each parameter in the course recommendation System
Parameters Significance Relevance
Self-Regulatory Indicates the student’s ability to regulate their Courses that encourage self-directed learning
Learning own learning processes, including goal-setting, and provide opportunities for reflection may be
Strategies self-monitoring, and self-reflection. recommended.
Approach Reflects the student’s general attitude and style Helps in recommending courses that align
Towards Learning in acquiring knowledge and skills. with the preferred learning style, whether it’s
through hands-on experiences, theoretical anal-
ysis, or creative exploration.
Psychosocial Takes into account the social and psychological Recommends courses that consider the stu-
Contextual aspects influencing learning, including family dent’s overall well-being and align with their
Factors support, peer relationships, and external stres- social and psychological context.
sors.
Ratings Ratings data gauges quality and satisfaction, Ratings data shapes a dynamic system, en-
informing future recommendations in a course hancing precision and relevance for improved
system. learning experiences in recommendations.
Skills of learners Learners’ skills feature is significant for precise Skills of learners contribute to a personalized,
alignment in course recommendations. skill-focused system for an effective educa-
tional journey.
Personality Assesses various personality traits that can im- Courses can be recommended based on how
Dimensions pact academic preferences and performance. well they match the student’s personality, en-
suring a better fit for their individual strengths
and preferences.
Performance data Performance data, including test scores and Performance data shapes a targeted course rec-
work experience, is significant for precise uni- ommendation system based on individual apti-
versity course recommendations. tude and experiences.
Motivational Fac- Identifies what drives the student to learn, Recommends courses that align with the stu-
tors whether it’s intrinsic passion or extrinsic re- dent’s motivations, whether they are driven by
wards. personal interest or future career prospects.
Institute data Institute data tailors course recommendations Institute data shapes a context-aware recom-
based on unique attributes of institutions. mendation system, aligning suggestions with
institution features for informed decision-
making.
Cognitive Prefer- Highlights the student’s preferred thinking and Recommends courses that cater to the student’s
ences problem-solving approaches. cognitive strengths, such as analytical or cre-
ative thinking.
Learning Styles Identifies how the student best absorbs infor- Recommends courses with teaching methods
mation, whether visually, auditorily, or kines- that align with the student’s preferred learning
thetically. style.
Demographic Demographic data tailors academic sugges- Demographic data contributes to a personalized
data tions based on individual backgrounds, enhanc- and inclusive course recommendation system,
ing the system’s adaptability. aligning with unique user needs and socio-
economic contexts.
Entrepreneurial Indicates the student’s inclination towards in- Recommends courses that foster
Spirit & Service- novation, risk-taking, or a desire to make a entrepreneurial skills or align with service-
Orientation social impact. oriented professions.
Academic data Academic data shapes recommendations based Academic data shapes a precise course sys-
on strengths and study preferences in a course tem, aligning with study programs, academic
system. achievements, and preferences.
Study Preferences Reflects whether the student prefers indepen- Recommends courses that match the stu-
dent study or collaborative research. dent’s study preferences, promoting a con-
ducive learning environment.

VOLUME 11, 2023 11

 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

TABLE 8: Significance and relevance of each parameter in the course recommendation System
Parameters Significance Relevance
Goal Orientation Identifies the student’s short-term and long- Recommends courses that contribute to the
term goals. achievement of the student’s goals, whether
they are immediate objectives or future aspira-
tions.
User interest data User interest data tailors suggestions, fostering User interest data creates a personalized, user-
engagement and satisfaction in a course sys- centric system for an engaging educational ex-
tem. perience.
Communication Assesses the student’s comfort with verbal and Recommends courses that align with the stu-
Style written communication. dent’s communication strengths, be it through
spoken or written expression.
Multilingual Pro- Highlights the student’s language skills and Recommends language-related courses or
ficiency interest in multicultural communication. courses that involve cross-cultural perspectives.
Technology Assesses the student’s comfort with technology Recommends courses that leverage technology
Adoption and adaptability to technological changes. or match the student’s preference for traditional
or online learning.
Course data Course data shapes tailored suggestions align- Course data ensures a comprehensive recom-
ing with academic offerings in a recommenda- mendation system, finely tuned to details and
tion system. contributing to informed decision-making for
students.
Cultural Identifies the student’s cultural sensitivity, Recommends courses that incorporate diverse
Preferences global awareness, and interest in cultural stud- cultural perspectives or align with the student’s
ies. cultural interests.
Time Assesses the student’s ability to manage time Recommends courses that consider the stu-
Management effectively. dent’s time management skills and preferences
for flexible schedules.
Prioritization Reflects the student’s ability to prioritize tasks Recommends courses that align with the stu-
Skills and manage multiple responsibilities. dent’s prioritization skills, ensuring a balanced
workload.

Motivational Factors Cognitive Preferences

Parent Parameter Child Parameters Parent Parameter Child Parameters
Intrinsic Motivation Passion for the subject Analytical Thinking Problem-solving capa-
bility
Personal interest Logical reasoning
Love for learning Critical thinking
Extrinsic Motivation Career prospects Creative Thinking Out-of-the-box thinking
Financial incentives Idea generation
External rewards Innovation
TABLE 9: Parameters hierarchy table of motivational factors, cognitive preferences fators for course
recommendation system.

studies comprised frameworks, with 19.7%, and in [29], [33], [49] the authors deployed their system
finally algorithms, with 19% of the total primary into Android apps.
studies. Some of the studies have deployed the
recommendation systems on specific platforms. The
E. EMPIRICAL VALIDATION OF RS
web is the most widely adopted platform for de-
veloping a recommendation system. 11 studies de- RQ5 examines the validation criteria used in these
ployed the RS framework on a web platform, and selected studies, which may be theoretical or exper-
imental. Among the validation categories, 4 distinct
12 VOLUME 11, 2023
Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

Personality Dimensions Learning Styles

Parent Parameter Child Parameters Parent Parameter Child Parameters
Openness to Experi- Curiosity Independent Study Self-directed learning
ence
Imagination Autonomy
Aesthetic appreciation Research interests
Interest in Leadership Desire for managerial Visual Learner Preference for visual
roles aids
Leadership potential Graphic learning
Decision-making pref- Diagrams and charts
erences
Conscientiousness Organization Auditory Learner Preference for listening
Detail-oriented Verbal instructions
Dependability Group discussions
Extraversion Sociability Kinesthetic Learner Hands-on activities
Assertiveness Physical involvement
Energy level Experiential learning
Agreeableness Cooperation Entrepreneurial Spirit Risk-taking propensity
Empathy Innovation
Patience Business acumen
Neuroticism Emotional stability Service-Oriented Desire to make a social
impact
Stress tolerance Helping professions
Resilience Volunteerism
TABLE 10: Parameters hierarchy table of personality dimensions, learning styles factors for course
recommendation system.

Time Management Cultural Preferences

Parent Parameter Child Parameters Parent Parameter Child Parameters
Time Management Ability to meet dead- Cultural Sensitivity Appreciation for diver-
Skills lines sity
Time allocation for Global perspectives
studies
Punctuality Cross-cultural commu-
nication
Flexible Schedule Desire for flexibility Local vs. International Interest in local vs. in-
Preferences Focus ternational issues
Part-time vs. full-time Global awareness
studies
Work-study balance Cultural immersion
Prioritization Skills Ability to prioritize Interest in Cultural Courses related to cul-
tasks Studies ture
Juggling multiple re- Anthropology
sponsibilities
Sociology
TABLE 11: Parameters hierarchy table of time management, cultural preferences factors for course
recommendation system.

classes have been identified, and 86% of all the stud- conducted experiments to assess the efficacy of a
ies are validated in some way. Table IX depicts that recommender system.
22 papers, which represent 50% of the total studies,

VOLUME 11, 2023 13

 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

Goal Orientation Technology Adoption Communication Style

Parent Parameter Child Parameters Parent Parameter Child Parameters Parent Parameter Child Parameters
Long-Term Vision for the Tech-Savvy Comfort with Verbal Com- Comfort
Goals future digital tools munication with spoken
communication
Career Online learning Public speaking
aspirations preferences
Graduate studies Technical skills Verbal
expression
Short-Term Immediate Traditional Preference Written Com- Writing skills
Goals objectives Learner for traditional munication
classrooms
Skill Paper-based re- Essay
development sources preferences
Certifications Face-to-face in- Documentation
teractions abilities
Task- Focus on com- Adaptability to Willingness to Multilingual Number of lan-
Orientation pleting tasks Technological embrace new Proficiency guages spoken
Changes technologies
Focus on achiev- Resistance to Interest in
ing goals technological learning new
change languages
Balance Tech curiosity Multicultural
between task communication
orientation &
goal orientation
TABLE 12: Parameters hierarchy table of goal orientation, technology adoption and communication style
factors for course recommendation system.

Parent Parameter Child Parameters Parent Parameter Child Parameters Parent Parameter Child Parameters
Age Study Program Course Level
Gender GPA Course Duration
Ethnicity Academic Scores Prerequisites
in Specific
Subjects
Location Research Course Format
Experience
Family Income Internship/Work Credit Hours
Experience
Educational Academic Instructor
Demographic data Academic data Course data
Background Achievements Experience
First-generation Extracurricular Student
College Status Activities Reviews
Language Study Abroad Textbook/Reading
Proficiency Experience Materials
Accessibility Specialized Skills Lab or Practical
Requirements Component
Parental Professional Course Syllabus
Educational Certifications
Background
TABLE 13: Parameters hierarchy table of demographic, academic and course data factors for course
recommendation system.

14 VOLUME 11, 2023

Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

Parent Parameter Child Parameters Parent Parameter Child Parameters Parent Parameter Child Parameters
Standardized Test Ranking in Personal Interests
Scores Breakdown Specific Programs
Subject-Specific Facility Academic
Test Scores Breakdown Preferences
Entrance Exam Location Career
Section Scores Attributes Aspirations
Research Teacher Profiles Preferred
Publications Learning Style
Work Experience Program-Specific Extracurricular
Details Requirements Involvement
Performance data Institute data User interest
Internship Details Extracurricular Preferred
Opportunities Teaching Methods
Professional Internship Professional
Certifications Placement Record Development Goals
Project Alumni Success Industry Interests
Contributions Stories
Leadership Roles Collaborative Preferred Work
Research Initiatives Environment
Conference Accreditation Technology
Presentations Information Interests
TABLE 14: Parameters hierarchy table of performance, institute data and user interest factors for course
recommendation system.

Parent Parameter Child Parameters Parent Parameter Child Parameters

Course Ratings Presentation Skills
Teacher Ratings Technical Skills
Institution Ratings Soft Skills
Course Difficulty Analytical Skills
Ratings
Relevance Ratings Language Proficiency
Feedback on Research Skills
Ratings Assignments Skills of learners
Accessibility Ratings Creativity
Interactive Component Leadership Skills
Ratings
Collaborative Project Adaptability
Ratings
Recommendation Project Management Skills
Effectiveness
TABLE 15: Parameters hierarchy table of ratings and skills of learners factors for course recommendation
system.

1) Experiment among the proposed classifier, back-propagation,

and c4.5 algorithms. The authors in [30] compare
For validating models through experiments, some performance measurements, e.g., precision, recall,
studies consider comparing models with related f-measure, and accuracy, with the values of the
or previous models. [29] measures instrument per- other four admission recommendation systems and
formance by showing accuracy, precision, and re- also conduct an online user study. Moreover, [32]
call. [39] and [31] evaluate their proposed hybrid compares the K-means clustering algorithm with
classifier by using real student data for accuracy the proposed W-clustering algorithm in terms of
and time performance and also compare error rates
VOLUME 11, 2023 15
 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

TABLE 16: Types of features used in designing recommendation system.

Scopes: Features Publications Total
Demographic data [29], [39], [43], [45], [47], [30], [33], [36], [28], [89], [91] 11
[39], [31], [33], [40], [38], [41], [43], [46], [26], [47], [48], [79], [52], [55],
Academic data 21
[56], [57], [63], [28], [65], [92], [70]
Course data [45], [50], [79], [53], [54], [59], [60], [61], [62], [63], [27], [28], [65], [66] 14
Performance data [33], [38], [46], [81], [69], [70], [71] 7
Institute profile [29], [30], [32], [36], [51] 5
User interest [41], [42], [44], [55], [67], [82], [69], [70], [89] 9
Ratings [49], [54], [56], [58], [59], [60] 7
Skills of learners [29], [81], [67], [82], [69], [70], [84] 7

TABLE 17: Types of features used in designing recommendation system.

Scopes: Features Publications Total
Algorithm [32], [33], [52], [54], [81], [70], [91], [85], [72], [74], [77], [90] 12
[29], [33], [34], [35], [36], [41], [45], [47], [49], [57], [60], [62], [63], [65],
System 18
[80], [81], [82], [89]
Framework [39], [31], [40], [46], [26], [50], [79], [51], [58], [66], [92] 11
[30], [37], [38], [43], [44], [48], [53], [55], [56], [59], [61], [27], [64], [28],
Method 19
[67], [84], [73], [75], [76]

execution time and number of clusters. The results square error (MSE) to define the accuracy of the
show that the W-clustering algorithm is scalable to models, whereas [52] measures the performance
the maximum extent. In [36], the authors compare of the rating matrix using domain knowledge with
their proposed HOSVD PCA algorithm with MC- precision. [56] uses a statistical accuracy metric, the
IB CF and Higher-order singular value decomposi- mean absolute error (MAE) between the predicted
tion (HOSVD) algorithms by measuring precision, and the real values, to analyze the error in predic-
recall, F1 metric, and execution time and observe tion. [57] measures the accuracies and RMSE of
that the model performs better among them as well the four different classification models. In [58], the
as handles scalability issues efficiently. [43] evalu- authors examine the model by comparing it with
ates several classification models (with or without related work. In [61], the performance of the model
feature selection) based on accuracy. In [45], the is measured using ROC analysis, and using the pa-
authors compares their course recommendation sys- rameters of the analysis, accuracy (.73), sensitivity
tem, RARE, with the other two course recommen- (.68), and specificity (.88) values are computed. The
dation systems (SCR and AACORN) for evaluation root mean squared error (RMSE) is used to evaluate
and finds that their system performs better under the the models in [81].
cold start problem compared to the two previous
systems. In [59], the authors compared their pro- 2) Academic Case-study
posed model and other recommendation approaches
Another important category is validating a proposed
by average precision and mean reciprocal rank
method using an academic case study, which repre-
(MRR). Moreover, [60] compares the traditional
sents 26.7% of all the studies. [46] uses evaluation
CBF, CF, and proposed OPCR algorithms by three
based on real data from students with respective
performance measure metrics: recovery, accuracy of
letter grades for courses. Also, [26] validates the
relevance, and rank accuracy for online evaluation
system through a case study using MAE and con-
to assess the results obtained from the participants.
fusion matrix analysis. In [48], the authors evaluate
Conversely, [33] evaluates the system by measuring
the performance of their proposed solution using
its accuracy with the variation of training and test
a dataset of academic records of university stu-
data. Similarly, [47] performs descriptive analysis
dents. Using Area under the ROC Curve (AUC)
on the experiment results, while [49] evaluates their
as a performance metric, [50] compares their pro-
mobile-based college recommender with RMSE and
posed BPR-MF solutions with four different sets
accuracy metrics. Studies [51] and [54] utilize mean
of models, such as baseline, memory-based, graph-
16 VOLUME 11, 2023
Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

based, and ensemble of different types of solutions, a mean satisfaction level of 3.89 out of 5.0, which
using a real-world course registration dataset. In indicates 77.8% user satisfaction.
[53], the authors use an enrollment dataset from a However, a number of studies [37], [38], [40],
university to compare the accuracy of the proposed [42], [44], [79] did not validate their proposed so-
Markov-based model with item-based and matrix lutions.
factorization-based course recommenders. The eval-
uation of this proposed model in [27] is performed F. OUTCOME OF RS
using a real course dataset of graduate electrical RQ6 aims to discover studies that are not only
engineering students by computing the precision implemented, but also assist students in making
and recall of different variations of the following informed decisions by providing real-life academic
parameters: minimum confidence, minimum match, recommendations. Some studies within the course
minimum specified grade, and minimum support. recommendation domain are successful platforms.
Conversely, Janusz Sobecki [64] compares the per- They are used by several universities to provide
formance of the 5 SI algorithm by utilizing a course- students with personalized and appropriate recom-
grade dataset using the following metrics: Mean mendations. They require planning their academic
Absolute error (MAE), Normalized Mean Absolute path. For example, RARE [45] an association rule-
Error (NMAE), and prediction accuracy (PA). ACO based course recommender system, is used at the
performed best with 0.88 PA. [66] uses the RMSE University de Montréal. Also, CourseRank [80], a
score to evaluate the student performance predic- personalized Stanford University Curriculum, now
tion model and the Simple Matching Coefficient provides course-related services, e.g., choosing the
(SMC) as the study-path recommendation model’s right courses for each student at many universities
efficiency evaluation. In [80], the authors involve throughout the United States. Another one is a per-
558 undergraduate students at Stanford to evaluate sonalized forecasting model, a performance predic-
their package recommendations based on Precision. tion system, and a course recommendation system
More specifically, we can say that the researchers used at CanTho University [81].
around the world are interested in educational rec-
ommendation system, mostly in USA, India, China, V. RESULT SYNTHESIS
Saudi Arabia, and Thailand.
Within the educational domain, recommender sys-
tems serve numerous purposes by generating mean-
3) Survey ingful recommendations from an abundance of in-
Some studies—around 17.6% of all studies—validate formation. In this study, we surveyed the state-of-
the RS through a survey. For instance, [34] and [35] the-art within the domain of mainstream educational
conducted a survey where students compared their choice recommendation systems over the last ten
admitted college results with the proposed system years. Applying systematic review methodology, a
prediction. Also, [46] conducted a survey to deter- total of 56 research papers were identified and ex-
mine the satisfaction of the students upon seeing the amined from an initial set of 1578 studies. This
recommendation to measure the effectiveness of the section presents the empirical findings of this re-
system. Some studies not only used performance view and provides insights based on the overall
metrics to evaluate the systems but also validated analysis of the selected papers. The findings have
them using surveys. The effectiveness of [59] was outlined three distinct vital axes of empirical re-
measured using 1,000 test cases by comparing the search on academic choices: university selection,
Top-3 courses generated by their proposed recom- program selection, and course selection. We have
mender with three other popular courses. Another inspected the frequency of educational choice ar-
study, [60], was also evaluated by a group of uni- ticles and domains from Table VI and found that
versity students. Following the evaluation, students among the disciplines of educational preferences
ranked user satisfaction level and recommendation discussed in this review, course recommendation is
quality. Besides, in [62], the authors "Rule-based a demanding research area, as extensive research
expert systems for supporting university students" is has been carried out in this field [34–59]. How-
tested by Oracle Policy Automation (OPA). Finally, ever, it is noted that research work towards rec-
evaluation of the intelligent advisor (CAES) [65] ommending universities or study programs is min-
is carried out by human advisors, who rate the imal; only 7 studies were found over the last 13
CAES recommendation on a Likert scale of 0–5 to years. Therefore, major/program recommendation
determine the degree of reliability. Results illustrate after high school/college needs a significant focus.
VOLUME 11, 2023 17
 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

FIGURE 3: Geographical distribution of selected articles

TABLE 18: Evaluation criteria’s of RS

Scopes: Evaluation Publications Total
[29], [39], [30], [31], [32], [33], [36], [43], [45], [47], [49], [51], [18], [54],
Experiment [56], [57], [58], [59], [60], [61], [62], [81], [82], [54], [89], [73], [74], [75], 29
[76], [77]
Academic [30], [46], [26], [48], [67], [50], [53], [27], [64], [66], [80], [93], [85], [72] 13
Survey [34], [35], [46], [59], [60], [65], [84] 7
Not validated [40], [37], [38], [42], [44], [79] 6

probe the approaches applied in this field to develop

better systems. We observed that hybrid recom-
mender systems are well suited since these mech-
anisms enable systems to accommodate changes
as well as resolve problems like sparsity and cold
start [6]. The literature indicates that predominantly
hybridization of CBCBF [27], [40], [49], [58], [59],
[60] or combinations of machine learning and data
mining approaches [28], [29], [30], [31], [34], [35],
[37], [39], [41], [61] are used in this domain. Several
studies proposed classification-based recommenda-
tion approaches such as Decision Tree, SVM, KNN,
NN, Linear Regression [33], [38], [47], [51], [55],
clustering (k-means) [31], [41], [43], and rule-based
FIGURE 4: Number of selected papers by year. approaches [45]. In some studies, authors proposed
Neighborhood-based CF [36], [79] or model-based
CF (Matrix Factorization [48], [50], SVD [56],
Another crucial objective of this review was to
18 VOLUME 11, 2023
Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

achieves an average of 67.9% accuracy. Besides,

several studies validate their system using MAE,
NMAE, and RMSE as performance metrics with the
aim of enhancing the RS model. The review process
also yielded some articles that conducted online ex-
periments and surveys to substantiate the efficiency
of their proposed systems. Studies examined in this
review mostly propose a framework, method, or
algorithm, comprising a total of 68% of all articles,
rather than employing them on platforms. Only a
few studies implemented their proposed system on
web or mobile platforms. Therefore, only some
FIGURE 5: Number of selected papers by studies have an impact on real-life students’ suc-
approaches. cessful learning outcomes [79], [80], [81] and pro-
vide a recommendation service to enable students to
make informed academic decisions.
Markov chain [34], Non-parametric approach [47]) Additionally, the findings of this study have dis-
based models. Few studies include rule-based ex- closed some limitations in the existing literature and
pert systems [62], [63], [65], swarm intelligence outlined some research paths that may help foster
[64], or graph-based [80], [81] algorithms. Another research on this topic. For example, this review
prime aspect was exploring the parameters utilized observed a notable paucity of empirical research
in the design of RS. For recommending courses or in the academic program recommendation domain.
curriculum, studies prioritize academic subjects and To date, program/major recommendation has not
rating parameters, i.e., ratings of different courses, been extensively studied. Assisting students to make
user feedback, academic grades, and subject inter- an informed decision to determine suitable higher
ests. Academic and institute profiles were taken into study options means encouraging students to ex-
consideration when recommending a university or plore and determine strategies befitting their overall
academic field. Only two studies focus on learners’ career and educational goals. Therefore, researchers
skills [29], [81]. Nevertheless, input attributes, such should consider conducting extensive research in
as socio-environmental factors and psychometrics, this particular domain. Researchers have integrated
are not addressed, which prove to be crucial pa- a broad range of techniques, such as machine learn-
rameters for producing tailored recommendations ing, information filtering algorithms, data mining,
in other domains [94]. The majority of studies and others, for designing educational recommender
used real-life academic cases in training and testing engines. Yet, deep learning is a promising alterna-
recommender systems. Prediction accuracy, recall, tive for enhancing performance and managing the
and precision were used as evaluation metrics in uncertainties of preference modeling. Finally, the
about 39% of the reviewed papers. These papers majority of the studies provide a conceptual method
are grouped according to the recommender’s ap- or framework rather than deploying it on a platform.
proaches. They are compared by the average ac- Developing a system within a dynamic real-world
curacy (calculated from the lowest to the highest environment can provide valuable feedback on cur-
range of accuracy) achieved by each group to un- rent approaches while assisting prospective students
derstand the performance. Table X illustrates the in making well-informed academic choices.
average group accuracy achieved by these studies
based on their approaches. While comparing the
classification accuracy of model-based recommen-
dation approaches, we noted that Hybrid techniques VI. FUTURE RESEARCH DIRECTION
yield better accuracy over 88% and scalability than
all other approaches. After Hybrid methods, var- Despite the advancements in recommender systems
ious collaborative filtering algorithms were used for academic advising in higher education, several
in domains of academic preferences and achieved research gaps and future directions remain to be
about 81% average accuracy. CF-based recommen- explored.
dations achieved an average of 85%. However, a
simple classification-based recommendation system
VOLUME 11, 2023 19
 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

TABLE 19: Comparative accuracy analysis for various approaches

Approaches Studies Average accuracy
Hybrid Approach of machine
learning algorithms (Decision
Tree, Random Forest, Regression [29], [39], [30], [82], [31], [41], [61], [70], [72], [73], 88.25%
Analysis, Association rules, [74], [75], [76], [77] (70%-98%)
SVM, KNN, Fuzzy Logic, Neural
Network)
Hybrid Approach of collaborative and 81%
[26], [60], [93]
content-based filleting (76.8%-84.5%)
[33], [38], [43], 67.9%
Classification [45], [55], [57] (50%-90%)
Collaborative Filtering [36], [48] 85%

A. APPLICATION OF HOLLAND CODE (I), Artistic (A), Social (S), Enterprising (E), and
ASSESSMENT IN UNIVERSITY COURSE Conventional (C) categories of the RIASEC model.
RECOMMENDATION SYSTEM This assessment can take the form of an online ques-
After meticulously examining all of the research tionnaire or a traditional paper-based tool, offering
papers explored in this study, we have observed that a comprehensive approach to capturing the diverse
all of these course recommendation system follows facets of a student’s personality. After completion
some non-standard systems relying on individual of the assessment, each student’s responses can
characteristics testing. As a standardized approach, be transformed into a mathematical representation
Holland code assessment can be employed to rec- known as a vector. In this context, a vector is a
ommend courses. multidimensional entity where each dimension cor-
The integration of the Holland Code assessment responds to one of the RIASEC categories. For each
in course recommendation system, offers a com- question, the system captures the student’s response
prehensive solution to the challenge of selecting and updates the corresponding dimension in the
perfect courses. Through a meticulous mathematical vector. For example, the vector representation like
analysis, the system can provide tailored recom- [R, I, A, S, E, C].
mendations, empowering students to make informed Vector normalization for Consistency – Vector nor-
decisions about their academic paths. Applying Hol-
land Code assessments in university course recom-
mendation system involves of assessment outcomes
with the individual student’s academic and career
preferences. The Holland Code, also known as the
RIASEC model, categorizes individuals into six FIGURE 6: Matrix to map RIASEC factor values
personality types: to corresponding courses.
R Realistic
I Investigative malization ensures the vectors maintain consistent
A Artistic scales and do not introduce biases due to variations
S Social in response intensity. The process involves adjust-
E Enterprising ing the values in each vector to a standard range
C Conventional (between 0 and 1). This guarantees that the overall
In below, we have explained the steps how the magnitude of the vector does not influence the sub-
Holland Code assessment could be use in university sequent analyses. The Normalized Value (NV) can
course recommendation system. be calculated using the below equation:
Max Value − Min Value
1) Administer the Assessment NV =
Original Value (OV) − Min Value
In the initial phase of the Holland Code Assess-
ment process, students are presented with a set of 2) Course Mapping
carefully crafted questions intended to discern their Align Courses with Holland Codes – In the course
preferences across the Realistic (R), Investigative mapping phase, the objective is to establish a con-
20 VOLUME 11, 2023
Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

TABLE 20: Significance and relevance of Holland Code Assessment’s RIASEC factors in course
recommendation system
Factor name Significance Relevance
Realistic Realistic traits favor hands-on activities, en- In course recommendations, Realistic traits
suring recommendations suit those inclined guide towards programs with tangible
towards applied and skill-based courses. skills, like vocational training or technical
courses.
Investigative Investigative traits emphasize analytics, Investigative traits are pertinent for sug-
guiding recommendations for courses that gesting programs in scientific research, data
cultivate critical thinking and problem- analysis, or disciplines requiring analytical
solving skills. thinking.
Artistic Artistic traits favor creativity, enhancing the Artistic traits guide towards programs nur-
system’s significance by suggesting courses turing creative talents and artistic expres-
in fine arts, design, or creative disciplines. sion.
Social Social traits stress interpersonal Social traits are relevant for suggesting pro-
skills, holding significance in grams in fields like psychology, sociology,
recommending courses involving or any discipline requiring strong interper-
teamwork, communication, and community sonal skills.
engagement.
Enterprising Enterprising traits signify leadership and Enterprising traits are pertinent for suggest-
business acumen, adding significance by ing programs in business, management, or
recommending courses aligned with en- entrepreneurship.
trepreneurial pursuits and leadership devel-
opment.
Conventional Conventional traits stress organization and Conventional traits are relevant for suggest-
detail orientation, providing significance ing programs in fields like finance, admin-
in recommending courses with structured istration, or any discipline requiring organi-
tasks and attention to detail. zational skills.

nection between each course in the curriculum and 3) Feedback Loop

the relevant Holland Codes. This alignment is cru-
In the dynamic landscape of a university course
cial for understanding the compatibility between the
recommendation system, user feedback plays a
content of a course and the corresponding personal-
pivotal role in refining and enhancing the accuracy
ity types identified by the Holland Code assessment.
of the system over time. This process involves
actively encouraging users to provide feedback on
Relevance Score: Assigning Scores for Alignment
the recommended courses. The feedback gathered
– As part of this process, a relevance score is
is then systematically integrated into the system
assigned to each course for each Holland Code. This
to adjust weights, relevance scores, and prediction
score serves as a quantitative measure of how well
models, fostering continuous improvement. This
a particular course aligns with the characteristics
iterative optimization process will fine-tune various
associated with each Holland Code. Typically, this
components of the recommendation system.
score is normalized to a scale from 0 to 1, where
0 indicates no alignment, and 1 signifies a perfect
Adjusting Weights: – For each trait category in
match.
the Holland Code assessment, adjust the weights
The implementation involves constructing a matrix
based on the feedback received. Here Wi represents
where each row represents a course, each column
the weight for the i-th trait category. The updated
corresponds to a Holland Code, and the entries
weight Wi′ can be calculated as:
signify the relevance scores. If M is a matrix,
where Mij represents the relevance score for the i- Wi′ = Wi + α × Feedbacki
th course and j-th Holland Code.
Here, α is a learning rate, and Feedbacki is the
feedback received for the i-th trait category.
VOLUME 11, 2023 21
 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

E. FAIRNESS AND BIAS MITIGATION

Relevance Score Adjustment: – Modify the rele- As recommender systems play an influential role in
vance scores for courses based on user feedback. shaping students’ academic decisions, it is essential
This adjustment can be implemented by updating to address issues related to fairness and bias. Future
the relevance scores using the feedback. research can focus on developing fairness-aware
Here, Rij be the relevance score for the i-th course algorithms and mitigation strategies to ensure eq-
and j-th Holland Code. The updated relevance score uitable and unbiased recommendations for students
′
Rij can be computed using the below equation: from diverse backgrounds [93], [98].
′
Rij = Rij + β × UserRatingij
F. INCORPORATING USER FEEDBACK AND
Here, β is a learning rate, and UserRatingij is the EXPLAINABILITY
user’s rating or feedback for the alignment of the i- Enhancing the transparency and interpretability of
th course with the j-th Holland Code. recommender systems is critical for building trust
The integration of the Holland Code assessment and acceptance among users. Future research can
in course recommendation system, offers a com- explore ways to incorporate user feedback into the
prehensive solution to the challenge of selecting recommendation process and provide meaningful
perfect courses. Through a meticulous mathematical explanations for the recommendations, enabling stu-
analysis, the system can provide tailored recom- dents to understand and trust the system’s sugges-
mendations, empowering students to make informed tions [99].
decisions about their academic paths.
G. PERSONALIZED LEARNING PATHWAYS
B. ADDRESSING COLD START PROBLEM Going beyond course recommendations, future re-
One critical research gap is finding effective strate- search can explore the development of recom-
gies to address the "cold start" problem, where mender systems that support personalized learning
recommender systems struggle to provide accurate pathways for students. These systems can provide
recommendations for new students with limited or tailored learning resources, study materials, and
no historical data. Future research can focus on skill development opportunities based on individual
developing innovative approaches, such as knowl- learning styles and preferences [100].
edge transfer techniques, to tackle this challenge and
enhance the usability of recommender systems for H. EFFECTIVENESS ACROSS DIVERSE
all students [95]. EDUCATIONAL SETTINGS
Most existing research focuses on recommender
C. INTEGRATING INTERDISCIPLINARY DATA systems in traditional higher education settings. Fu-
Most existing recommender systems in higher ed- ture research can investigate the effectiveness of
ucation primarily focus on academic data, such as these systems in diverse educational contexts, such
course histories and grades. However, to provide as online learning platforms, vocational training
truly holistic and personalized recommendations, programs, and lifelong learning environments [20].
future research can explore the integration of in-
terdisciplinary data, such as co-curricular activities, I. HYBRID MODELS WITH USER
extracurricular interests, and career aspirations. This COLLABORATION
will enable recommender systems to cater to the While hybrid recommender systems show promis-
diverse needs and goals of students beyond their ing results, there is a research gap in understanding
academic pursuits [96]. how user collaboration and input can be effectively
integrated into the hybrid models. Future research
D. LONG-TERM IMPACT ASSESSMENT can explore innovative ways to leverage user feed-
Evaluating the long-term impact of recommender back and preferences to improve the accuracy and
systems on students’ academic performance, career relevance of recommendations [71].
outcomes, and overall learning experience is a cru- In conclusion, addressing the identified research
cial research area. Future studies can employ lon- gaps and exploring the suggested future research
gitudinal data and conduct follow-up assessments directions will advance the field of academic ad-
to understand how students’ choices and academic vising through recommender systems, leading to
trajectories are influenced by recommender system more accurate, personalized, and meaningful rec-
recommendations over time [97]. ommendations for students in higher education.
22 VOLUME 11, 2023
Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

Moreover, considering the ever-evolving educa- in the material, they are more likely to actively par-
tional landscape, continuous research and innova- ticipate in class discussions, complete assignments,
tion are essential to ensuring that recommender and seek out additional learning opportunities.
systems remain relevant, trustworthy, and support-
ive tools in empowering students to make well- C. RETENTION IMPROVEMENT
informed choices for their educational and profes- Course recommendation systems can contribute to
sional success. higher student retention rates by guiding students
towards courses that match their academic strengths
VII. PRACTICAL USES OF COURSE and interests. When students feel supported in their
RECOMMENDATION SYSTEM academic journey and see a clear path towards their
Course recommendation systems are essential tools goals, they are less likely to drop out.
for individuals navigating the complex landscape
of career path selection. These systems provide D. REDUCED DROPOUT RATES
personalized guidance by leveraging advanced data Decreasing dropout rates is a key benefit of
analytics and machine learning algorithms to ana- course recommendation systems. By helping stu-
lyze individual preferences, skills, and career aspi- dents choose courses that align with their abilities
rations. By offering tailored recommendations for and interests, these systems can prevent academic
courses and educational pathways, they assist users overwhelm and increase the likelihood of successful
in exploring diverse career options and identifying course completion.
the most suitable educational opportunities. Further-
more, course recommendation systems help bridge
E. COURSE DIVERSITY PROMOTION
the gap between individuals’ current skills and the
requirements of various industries by suggesting Promoting diversity in course selection is essential
courses that align with emerging trends and work- for providing a well-rounded education. Course rec-
force demands. This proactive approach ensures that ommendation systems can achieve this by suggest-
users acquire the necessary expertise to succeed in ing a variety of options from different disciplines,
their chosen fields and remain competitive in the job cultures, and perspectives, encouraging students to
market. Additionally, these systems facilitate life- explore new subjects and broaden their horizons.
long learning by encouraging users to continuously
update their skills and knowledge, enabling them to F. ADDRESSING CURRICULUM GAPS
adapt to evolving job requirements and pursue ca- Course recommendation systems can identify gaps
reer advancements effectively. Overall, course rec- in the curriculum and suggest supplementary or
ommendation systems play a vital role in empow- complementary courses to fill those gaps. This en-
ering individuals to make informed decisions about sures that students receive a comprehensive educa-
their career paths, facilitating professional growth, tion that covers all necessary topics and prepares
and enhancing overall career satisfaction and suc- them for future academic or professional endeavors.
cess. Here, we have discussed various practical uses
of course recommendation system in personalized G. OPTIMIZING COURSE LOAD
career path selection process. Helping students optimize their course load is cru-
cial for academic success. Course recommendation
A. PERSONALIZED LEARNING PATHS systems can suggest a balanced mix of core, elec-
Tailoring course recommendations based on indi- tive, and prerequisite courses, taking into account
vidual student preferences and academic goals en- students’ schedules, academic goals, and extracur-
ables students to pursue a curriculum aligned with ricular commitments.
their interests and career aspirations. This fosters a
sense of ownership over their education, increasing H. ADAPTIVE LEARNING SUPPORT
motivation and engagement. Adaptive learning environments require personal-
ized instruction tailored to each student’s needs.
B. IMPROVED STUDENT ENGAGEMENT Course recommendation systems play a key role
By suggesting courses that align with students’ in supporting adaptive learning by recommending
interests, learning styles, and career goals, course courses that adapt to students’ evolving knowledge
recommendation systems can enhance student en- and skill levels, ensuring they receive appropriate
gagement. When students are genuinely interested challenges and support.
VOLUME 11, 2023 23
 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

I. CAREER PATH PLANNING O. SUPPORT FOR TRANSFER STUDENTS

Course recommendation systems can assist students Transfer students often face unique challenges when
in planning their career paths by recommending transitioning to a new institution. Course recom-
courses relevant to their chosen professions or in- mendation systems can assist transfer students by
dustries. By aligning coursework with future career recommending courses that align with their prior
goals, these systems help students make informed coursework and academic background, ensuring a
decisions about their education and career trajec- smooth transition and maximizing credit transfer
tory. opportunities.

J. GUIDANCE FOR COURSE PREREQUISITES P. CUSTOMIZED DEGREE PLANNING

Prerequisite courses are essential for building foun- Customized degree planning is essential for helping
dational knowledge and skills in a particular subject students navigate the complexities of higher edu-
area. Course recommendation systems can recom- cation and achieve their academic goals. Course
mend prerequisite courses necessary for students to recommendation systems can recommend courses
succeed in advanced or specialized coursework, en- that fulfill specific degree requirements and elective
suring they have the necessary background knowl- preferences, empowering students to create a per-
edge to excel. sonalized academic pathway tailored to their inter-
ests and aspirations.
K. EXPLORING NEW SUBJECT AREAS
Encouraging students to explore new subject areas Q. CROSS-INSTITUTIONAL COLLABORATION
or interdisciplinary fields is essential for fostering
Course recommendation systems can facili-
creativity, critical thinking, and innovation. Course
tate cross-institutional collaboration by sharing
recommendation systems can achieve this by sug-
anonymized course recommendation data. By pool-
gesting relevant introductory courses outside stu-
ing data from multiple institutions, these systems
dents’ usual areas of study, sparking curiosity and
can improve the accuracy and effectiveness of
encouraging intellectual exploration.
course recommendations, benefiting students across
different educational contexts.
L. ALIGNMENT WITH LEARNING OBJECTIVES
Course recommendation systems must align with
R. INSTRUCTOR COURSE ASSIGNMENT
institutional learning objectives and academic stan-
dards to ensure that students receive a high-quality Assigning instructors to courses is a critical task for
education. By recommending courses that meet academic institutions. Course recommendation sys-
these objectives, these systems contribute to the tems can assist in this process by matching instruc-
overall effectiveness and reputation of the institu- tors’ expertise and preferences with course needs,
tion. ensuring a well-qualified and motivated teaching
staff.
M. INTEGRATION WITH ACADEMIC ADVISING
Integrating course recommendation systems with S. CONTINUOUS IMPROVEMENT
academic advising services enhances the guidance Course recommendation systems must continuously
and support available to students. Academic advi- evolve and improve to meet the changing needs
sors can use course recommendation data to pro- of students and academic institutions. By leverag-
vide personalized advice, helping students make ing feedback mechanisms and performance metrics,
informed decisions about their course selection and these systems can identify areas for improvement
academic pathway. and refine their algorithms and models accordingly.

N. EARLY WARNING SYSTEMS T. ENHANCED LEARNING ANALYTICS

Early warning systems leverage course recommen- Course recommendation data provides valuable in-
dation data to identify students at risk of academic sights into student learning behaviors, preferences,
underperformance and provide timely interventions. and academic outcomes. By analyzing this data,
By analyzing students’ course selections, perfor- academic institutions can generate actionable in-
mance metrics, and engagement levels, these sys- sights to inform instructional design, curriculum
tems can flag potential issues and connect students development, and institutional improvement initia-
with the resources and support they need to succeed. tives.
24 VOLUME 11, 2023
Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

U. FACULTY DEVELOPMENT INITIATIVES By providing seamless access to course recommen-

Course recommendation systems can support fac- dations within the LMS interface, these systems
ulty development initiatives by identifying areas improve usability and accessibility for all users.
where additional training or resources may be
needed. By analyzing course recommendation data, . ACCESSIBILITY AND INCLUSIVITY
institutions can identify trends and patterns that Course recommendation systems can promote ac-
may indicate areas for improvement in teaching and cessibility and inclusivity by recommending courses
learning practices. that accommodate diverse learning needs and pref-
erences. By considering factors such as language
V. PREDICTIVE ANALYTICS FOR ENROLLMENT support, accommodation services, and alternative
MANAGEMENT formats, these systems ensure that all students have
Predictive analytics techniques can be applied to equal access to educational opportunities.
course recommendation data to forecast future en-
rollment trends and student demand. By analyzing . FEEDBACK MECHANISMS FOR
historical enrollment data and course recommenda- IMPROVEMENT
tion patterns, institutions can make more informed Course recommendation systems should incorpo-
decisions about course offerings and resource allo- rate feedback mechanisms to gather input from
cation. students, faculty, and administrators. By soliciting
feedback on course recommendations, these sys-
W. PERSONALIZED LEARNING MATERIALS tems can identify areas for improvement and refine
In addition to recommending courses, course rec- their algorithms and models to better meet the needs
ommendation systems can also suggest personal- of users.
ized learning materials, such as textbooks, articles,
videos, and online resources. By curating a selection . LONGITUDINAL TRACKING OF STUDENT
of relevant materials tailored to each student’s needs PROGRESS
and preferences, these systems support self-directed
Course recommendation data can be used to track
learning and independent study.
students’ progress over time and identify trends
in course selection and academic performance. By
X. ALIGNMENT WITH STUDENT LEARNING
analyzing longitudinal data, institutions can gain
OBJECTIVES
insights into student behavior and outcomes, in-
Course recommendation systems can ensure that
forming strategic planning and decision-making.
recommended courses align with students’ learning
objectives and educational goals. By considering
. ETHICAL CONSIDERATIONS AND PRIVACY
factors such as learning style, academic interests,
PROTECTIONS
and career aspirations, these systems help students
make choices that are meaningful and relevant to Course recommendation systems must adhere to
their personal and professional development. ethical guidelines and privacy regulations to pro-
tect students’ sensitive information. By implement-
Y. ALIGNMENT WITH INSTITUTIONAL ing robust data security measures and transparency
RESOURCES practices, institutions can build trust with users and
Course recommendation systems must take into ac- ensure the responsible use of course recommenda-
count institutional resources and constraints when tion data.
making course recommendations. By considering
factors such as class size, faculty availability, and . COST-EFFECTIVENESS AND RESOURCE
facilities, these systems ensure that recommended OPTIMIZATION
courses are feasible and practical for both students Course recommendation systems can help institu-
and the institution. tions optimize resource allocation and minimize
costs by recommending courses that maximize stu-
Z. INTEGRATION WITH LEARNING dent enrollment and faculty utilization. By ana-
MANAGEMENT SYSTEMS lyzing historical data and forecasting future de-
Integrating course recommendation systems with mand, these systems support efficient planning and
learning management systems (LMS) streamlines decision-making.
the course selection process for students and faculty.
VOLUME 11, 2023 25
 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

. ALIGNMENT WITH INDUSTRY NEEDS faculty can identify areas for improvement and share
Course recommendation systems can align course best practices with colleagues, fostering a culture of
offerings with industry needs and workforce de- continuous improvement and innovation.
mands. By analyzing labor market trends and em-
ployer feedback, institutions can identify areas of . ALIGNMENT WITH INSTITUTIONAL MISSION
growth and opportunity and tailor course recom- AND VALUES
mendations accordingly. Course recommendation systems should align with
the institutional mission and values to ensure that
. SUPPORT FOR ACADEMIC PLANNING recommended courses reflect the institution’s ed-
COMMITTEES ucational philosophy and goals. By incorporating
Academic planning committees rely on course rec- institutional priorities and priorities into course rec-
ommendation data to make decisions about curricu- ommendations, these systems contribute to the over-
lum development, program evaluation, and resource all coherence and integrity of the academic program.
allocation. By providing timely and accurate course
recommendations, these systems support the work . SUPPORT FOR TRANSFER ARTICULATION
of academic planning committees and help institu- AGREEMENTS
tions achieve their strategic goals. Course recommendation systems can support trans-
fer articulation agreements by recommending
. INTEGRATION WITH STUDENT SUPPORT courses that meet transfer requirements and articu-
SERVICES lation guidelines. By facilitating the transfer process
Course recommendation systems can integrate with for students, these systems promote seamless path-
student support services, such as tutoring, advising, ways to degree completion and academic success.
and counseling, to provide holistic support to stu-
dents. By identifying students who may benefit from . COMMUNITY ENGAGEMENT AND OUTREACH
additional assistance and connecting them with the Course recommendation systems can engage the
appropriate resources, these systems contribute to broader community, including alumni, employers,
student success and well-being. and industry partners, in the educational process. By
soliciting input and feedback from external stake-
. INTERNATIONALIZATION AND GLOBAL holders, institutions can ensure that course offerings
ENGAGEMENT are relevant, responsive, and aligned with commu-
Course recommendation systems can support in- nity needs and priorities.
ternationalization efforts by recommending courses These are some practical uses highlighting the mul-
that reflect diverse perspectives and global trends. tifaceted impact of course recommendation sys-
By exposing students to a variety of cultural and tems on student success, institutional effectiveness,
linguistic experiences, these systems prepare them and educational innovation. By leveraging data-
to thrive in an increasingly interconnected world. driven insights and personalized recommendations,
these systems support informed decision-making,
. ALIGNMENT WITH ACCREDITATION enhance learning experiences, and promote equi-
STANDARDS table access to educational opportunities for all stu-
Course recommendation systems can help insti- dents.
tutions demonstrate compliance with accreditation
standards and quality assurance measures. By en- VIII. CONCLUSION
suring that recommended courses meet established In recent years, the field of tertiary education has
criteria for rigor, relevance, and effectiveness, these witnessed a notable surge in interest in develop-
systems support accreditation processes and institu- ing recommender systems for making informed de-
tional accountability. cisions. This systematic review aimed to investi-
gate the trends and techniques employed in rec-
. FACULTY COLLABORATION AND ommendation systems within the context of higher
PROFESSIONAL DEVELOPMENT educational academic advising. By analyzing 56
Course recommendation systems can facilitate fac- selected studies, we gained valuable insights into
ulty collaboration and professional development by the utilization of recommender systems in six key
providing insights into teaching and learning prac- aspects driven by research questions, including the
tices. By analyzing course recommendation data, purpose, development approach, incorporated fea-
26 VOLUME 11, 2023
Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

tures, deployment, validation criteria, and educa- dation: a review of ontology- based recommender systems for
tional outcomes. The findings revealed that course e-learning, Artificial intelligence review 50 (2018) 21–48.
[7] S. Garcia-Martinez, A. Hamou-Lhadj, Educational recom-
recommendations emerged as the most prominent mender systems: A pedagogical- focused perspective, Multi-
area of focus, accounting for approximately 53% of media services in intelligent environments: Recommendation
the papers. Notably, hybrid strategies constituted the services (2013) 113–124.
[8] K. Verbert, N. Manouselis, X. Ochoa, M. Wolpers, H. Drach-
primary development technique across the analyzed
sler, I. Bosnic, E. Duval, Context- aware recommender systems
studies. However, it was observed that there is no for learning: a survey and future challenges, IEEE transactions
one-size-fits-all generic model or framework for on learning technologies 5 (4) (2012) 318–335.
recommending educational choices, as each recom- [9] S. Khusro, Z. Ali, I. Ullah, Recommender systems: issues,
challenges, and research op- portunities, in: Information sci-
mender system is tailored to its specific context and ence and applications (ICISA) 2016, Springer, 2016, pp.
data type. As a conclusion, this systematic review 1179–1189.
sheds light on the growing importance of recom- [10] L. Iaquinta, M. De Gemmis, P. Lops, G. Semeraro, M. Fi-
lannino, P. Molino, Introducing serendipity in a content-based
mender systems in academic advising in higher recommender system, in: 2008 eighth international conference
education. The identified trends and techniques can on hybrid intelligent systems, IEEE, 2008, pp. 168–173.
guide future researchers in developing innovative [11] S. C. Cazella, P. A. Behar, D. Schneider, K. K. da Silva, R.
Freitas, Developing a learning objects recommender system
approaches to unlock the full potential of academic
based on competences to education: Experience report, in:
advising and enhance student learning experiences. New Perspectives in Information Systems and Technologies,
By effectively leveraging recommender systems, Volume 2, Springer, 2014, pp. 217–226.
educational institutions can offer personalized guid- [12] M. Erdt, A. Fernandez, C. Rensing, Evaluating recommender
systems for technology en- hanced learning: a quantitative
ance to students, leading to more informed and survey, IEEE Transactions on Learning Technologies 8 (4)
successful academic decisions. The comprehensive (2015) 326–344.
understanding gained from this review can pave [13] J. Zhong, H. Xie, F. L. Wang, The research trends in recom-
mender systems for e-learning: A systematic review of ssci
the way for the continuous improvement and im- journal articles from 2014 to 2018, Asian Association of Open
plementation of recommender systems in tertiary Universities Journal 14 (1) (2019) 12–27.
education, contributing to the advancement of the [14] N. Manouselis, H. Drachsler, R. Vuorikari, H. Hummel, R.
Koper, Recommender systems in technology enhanced learn-
field and ultimately benefiting students’ academic
ing, Recommender systems handbook (2011) 387–415.
journeys. [15] S. Dietze, H. Drachsler, D. Giordano, A survey on linked data
and the social web as facilitators for tel recommender systems,
Recommender Systems for Technology Enhanced Learning:
ACKNOWLEDGMENT Research Trends and Applications (2014) 47–75.
This work was supported by the ICT Division, Min- [16] F. L. da Silva, B. K. Slodkowski, K. K. A. da Silva, S.
istry of Posts, Telecommunications, and Informa- C. Cazella, A systematic literature review on educational
recommender systems for teaching and learning: research
tion Technology, Bangladesh and Institutite of Ad- trends, limitations and opportunities, Education and Informa-
vanced Research (IAR), United International Uni- tion Technologies 28 (3) (2023) 3289–3328.
versity under grant number IAR-2024-Pub-012. We [17] K.-K. Chu, M. Chang, Y.-T. Hsia, Designing a course recom-
mendation system on web based on the students’ course selec-
express gratitude to our colleagues from the Ad- tion records, in: EdMedia+ Innovate Learning, Association for
vanced Intelligent Multidisciplinary Systems Lab the Advancement of Computing in Education (AACE), 2003,
(AIMS Lab), United International University (UIU), pp. 14–21.
Bangladesh, whose insights and expertise greatly [18] V. Vaidhehi, R. Suchithra, A systematic review of recom-
mender systems in education, Inter- national Journal of En-
enriched this study. gineering & Technology 7 (1) (2018).
[19] S. A. A. Maria, S. C. Cazella, P. A. Behar, Sistemas de
recomendac ao: conceitos e t ecnicas de aplicac ao, Recomen-
REFERENCES dac ao Pedag ogica em Educac ao a Dist^ancia (2019) 19–47.
[1] R. Rafeh, Recommender systems in ecommerce, 2017. [20] H. Drachsler, K. Verbert, O. C. Santos, N. Manouselis,
[2] G. Linden, B. Smith, J. York, Amazon. com recommendations: Panorama of recommender systems to support learning, Rec-
Item-to-item collaborative filtering, IEEE Internet computing 7 ommender systems handbook (2015) 421–451.
(1) (2003) 76–80. [21] A. C. Rivera, M. Tapia-Leon, S. Lujan-Mora, Recommenda-
[3] P. Covington, J. Adams, E. Sargin, Deep neural networks for tion systems in education: A systematic mapping study, in:
youtube recommendations, in: Proceedings of the 10th ACM Proceedings of the International Conference on Information
conference on recommender systems, 2016, pp. 191–198. Technology & Systems (ICITS 2018), Springer, 2018, pp.
[4] C. Gomez-Uribe, N. Hunt, The netflix recommender system: 937–947.
Algorithms, business value, and innovation, in “acm transac- [22] E. Ashraf, S. Manickam, S. Karuppayah, A comprehensive
tion on management information systems”, vol. 6 (2016). review of course recommender systems in e-learning., Journal
[5] M.-I. Dascalu, C.-N. Bodea, M. N. Mihailescu, E. A. Tanase, P. of Educators Online 18 (1) (2021).
Ordo nez de Pablos, Ed- ucational recommender systems and [23] C. Salazar, J. Aguilar, J. Monsalve-Pulido, E. Montoya, Af-
their application in lifelong learning, Behaviour & information fective recommender systems in the educational field. a sys-
technology 35 (4) (2016) 290–297. tematic literature review, Computer Science Review 40 (2021)
[6] J. K. Tarus, Z. Niu, G. Mustafa, Knowledge-based recommen- 100377.

VOLUME 11, 2023 27

 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

[24] B. Kitchenham, O. P. Brereton, D. Budgen, M. Turner, J. study, International Journal of e-Education, e-Business, e-
Bailey, S. Linkman, Systematic literature reviews in software Management and e-Learning 4 (5) (2014) 329.
engineering–a systematic literature review, Information and [42] C. Obeid, I. Lahoud, H. El Khoury, P.-A. Champin, Ontology-
software technology 51 (1) (2009) 7–15. based recommender system in higher education, in: Compan-
[25] P. Brereton, B. A. Kitchenham, D. Budgen, M. Turner, M. ion proceedings of the the web conference 2018, 2018, pp.
Khalil, Lessons from applying the systematic literature review 1031–1034.
process within the software engineering domain, Journal of [43] M. Vukicevic, M. Jovanovic, B. Delibasic, M. Suknovic, Rec-
systems and software 80 (4) (2007) 571–583. ommender system for selection of the right study program
[26] P.-C. Chang, C.-H. Lin, M.-H. Chen, A hybrid course rec- for higher education students, RapidMiner: Data Mining Use
ommendation system by integrating collaborative filtering and Cases and Business Analytics Applications (2013) 145.
artificial immune systems, Algorithms 9 (3) (2016) 47. [44] Q. E. Booker, A student program recommendation system
[27] A. Al-Badarenah, J. Alsakran, An automated recommender prototype, Issues in Information Systems (2009) 544–551.
system for course selection, International Journal of Advanced [45] N. Bendakir, E. A ımeur, Using association rules for course
Computer Science and Applications 7 (3) (2016) 166– 175. recommendation, in: Proceedings of the AAAI workshop on
[28] S. Asadi, S. Jafari, Z. Shokrollahi, Developing a course recom- educational data mining, Vol. 3, Citeseer, 2006, pp. 1–10.
mender by combining clustering and fuzzy association rules, [46] A. Slim, D. Hush, T. Ojha, C. Abdallah, G. Heileman, G. El-
Journal of AI and Data mining 7 (2) (2019) 249–262. Howayek, An automated framework to recommend a suitable
[29] K. Pupara, W. Nuankaew, P. Nuankaew, An institution recom- academic program, course and instructor, in: 2019 IEEE Fifth
mender system based on student context and educational insti- International Conference on Big Data Computing Service and
tution in a mobile environment, in: 2016 International Com- Applications (BigDataService), IEEE, 2019, pp. 145–150.
puter Science and Engineering Conference (ICSEC), IEEE, [47] K. Ganeshan, X. Li, An intelligent student advising system
2016, pp. 1–6. using collaborative filtering, in: 2015 IEEE Frontiers in Ed-
[30] A. Baskota, Y.-K. Ng, A graduate school recommendation ucation Conference (FIE), IEEE, 2015, pp. 1–8.
system using the multi-class support vector machine and knn
[48] K. Bhumichitr, S. Channarukul, N. Saejiem, R. Jiamthap-
approaches, in: 2018 IEEE International Conference on Infor-
thaksin, K. Nongpong, Recom- mender systems for university
mation Reuse and Integration (IRI), IEEE, 2018, pp. 277–284.
elective course recommendation, in: 2017 14th international
[31] S. Fong, R. P. Biuk-Aghai, An automated university admission joint conference on computer science and software engineering
recommender system for secondary school students, in: The (JCSSE), IEEE, 2017, pp. 1–5.
6th international conference on information technology and
[49] Z. Khoja, S. Shetty, Hybrid recommender system for college
applications, 2009, p. 42.
courses, in: 2017 International Conference on Computational
[32] M. Y. S. Reddy, P. Govindarajulu, College recommender sys-
Science and Computational Intelligence (CSCI), IEEE, 2017,
tem using stu- dent’preferences/voting: A system development
pp. 1167–1171.
with empirical study, Int. J. Comput. Sci. Netw. Secur 18
(2018) 87–98. [50] E. L. Lee, T.-T. Kuo, S.-D. Lin, A collaborative filtering-based
two stage model with item dependency for course recommen-
[33] M. Hasan, S. Ahmed, D. M. Abdullah, M. S. Rahman, Gradu-
dation, in: 2017 IEEE international conference on data science
ate school recommender system: Assisting admission seekers
and advanced analytics (DSAA), IEEE, 2017, pp. 496–503.
to apply for graduate studies in appropriate graduate schools,
in: 2016 5th International Conference on Informatics, Elec- [51] K. Kongsakun, C. C. Fung, Neural network modeling for an
tronics and Vision (ICIEV), IEEE, 2016, pp. 502–507. intelligent recommendation system supporting srm for univer-
[34] A. H. M. Ragab, A. F. S. Mashat, A. M. Khedra, Hrspca: sities in thailand, WSEAS transactions on Computers 11 (2)
Hybrid recommender system for predicting college admission, (2012) 34–44.
in: 2012 12th International conference on intelligent systems [52] V. Vaidhehi, R. Suchithra, An enhanced approach using col-
design and applications (ISDA), IEEE, 2012, pp. 107–113. laborative filtering for generating under graduate program
[35] A. H. M. Ragab, A. F. S. Mashat, A. M. Khedra, Design recommendations, in: 2019 Second International Conference
and implementation of a hybrid recommender system for pre- on Advanced Computational and Communication Paradigms
dicting college admission, International Journal of Computer (ICACCP), IEEE, 2019, pp. 1–6.
Information Systems and Industrial Management Applications [53] E. S. Khorasani, Z. Zhenge, J. Champaign, A markov chain
6 (2014) 35–44. collaborative filtering model for course enrollment recommen-
[36] D. K. Bokde, S. Girase, D. Mukhopadhyay, An approach to a dations, in: 2016 IEEE International Conference on Big Data
university recommendation by multi-criteria collaborative fil- (Big Data), IEEE, 2016, pp. 3484–3490.
tering and dimensionality reduction techniques, in: 2015 IEEE [54] B. Bakhshinategh, G. Spanakis, O. Zaiane, S. ElAtia, A course
International Symposium on Nanoelectronic and Information recommender system based on graduating attributes, in: Inter-
Systems, IEEE, 2015, pp. 231–236. national Conference on Computer Supported Education, Vol.
[37] Z. Khanam, S. Alkhaldi, An intelligent recommendation en- 2, SCITEPRESS, 2017, pp. 347–354.
gine for selecting the university for graduate courses in ksa: [55] J. Britto, S. Prabhu, A. Gawali, Y. Jadhav, A machine learn-
Sars student admission recommender system, in: Inventive ing based approach for recom- mending courses at graduate
Computation Technologies 4, Springer, 2020, pp. 711–722. level, in: 2019 International Conference on Smart Systems and
[38] J. Manley, S. Krishnakumar, University recommender system Inventive Technology (ICSSIT), IEEE, 2019, pp. 117–121.
for graduate studies in usa. [56] F. O. G. Carballo, Masters’ courses recommendation: Explor-
[39] S. Fong, Y.-W. Si, R. P. Biuk-Aghai, Applying a hybrid model ing collaborative filtering and singular value decomposition
of neural network and decision tree classifier for predicting with student profiling, no. November (2014).
university admission, in: 2009 7th international conference on [57] M. Isma’il, U. Haruna, G. Aliyu, I. Abdulmumin, S. Adamu,
information, communications and signal processing (ICICS), An autonomous courses recom- mender system for undergrad-
IEEE, 2009, pp. 1–5. uate using machine learning techniques, in: 2020 international
[40] G. Meryem, K. Douzi, S. Chantit, Toward an e-orientation conference in mathematics, computer engineering and com-
platform: Using hybrid recom- mendation systems, in: 2016 puter science (ICMCECS), IEEE, 2020, pp. 1–6.
11th International Conference on Intelligent Systems: Theories [58] A. Esteban, A. Zafra, C. Romero, Helping university students
and Applications (SITA), IEEE, 2016, pp. 1–6. to choose elective courses by using a hybrid multi-criteria rec-
[41] K. Wakil, B. Akram, N. Kamal, A. Safi, Web recommender ommendation system with genetic optimization, Knowledge-
system for private universities’ admission in iraq: Uhd case Based Systems 194 (2020) 105385.

28 VOLUME 11, 2023

Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

[59] Y.-K. Ng, J. Linn, Crsrecs: a personalized course recommen- [76] Z. Wu, Q. Liang, Z. Zhan, Course recommendation
dation system for college stu- dents, in: 2017 8th International based on enhancement of meta-path embedding in
Conference on Information, Intelligence, Systems & Appli- heterogeneous graph, Applied Sciences (2023). URL
cations (IISA), IEEE, 2017, pp. 1–6. https://api.semanticscholar.org/CorpusID:256880819
[60] M. E. Ibrahim, Y. Yang, D. L. Ndzi, G. Yang, M. Al-Maliki, [77] M. Gao, Y. Luo, X. Hu, Online course rec-
Ontology-based personalized course recommendation frame- ommendation using deep convolutional neural
work, IEEE Access 7 (2018) 5180–5199. network with negative sequence mining, Wireless
[61] M. F. Adak, N. Yumusak, H. Taskin, An elective course sug- Communications and Mobile Computing (2022). URL
gestion system developed in computer engineering department https://api.semanticscholar.org/CorpusID:251421701
using fuzzy logic, in: 2016 International Conference on Indus- [78] M. Rajabi, S. Hossani, F. Dehghani, A literature review on
trial Informatics and Computer Systems (CIICS), IEEE, 2016, current approaches and applica- tions of fuzzy expert systems,
pp. 1–5. arXiv preprint arXiv:1909.08794 (2019).
[62] G. Engin, B. Aksoyer, M. Avdagic, D. Bozanlı, U. Hanay, D. [79] H. Ma, X. Wang, J. Hou, Y. Lu, Course recommendation
Maden, G. Ertek, Rule-based expert systems for supporting based on semantic similarity analysis, in: 2017 3rd IEEE
university students, Procedia Computer Science 31 (2014) International Conference on Control Science and Systems
22–31. Engineering (ICCSSE), IEEE, 2017, pp. 638–641.
[63] M. H. Goodarzi, V. Rafe, Educational advisor system imple- [80] A. Parameswaran, P. Venetis, H. Garcia-Molina, Recommen-
mented by web-based fuzzy expert systems (2012). dation systems with complex constraints: A course recommen-
[64] J. Sobecki, Comparison of selected swarm intelligence al- dation perspective, ACM Transactions on Information Sys-
gorithms in student courses rec- ommendation application, tems (TOIS) 29 (4) (2011) 1–33.
International Journal of Software Engineering and Knowledge [81] N. Thai-Nghe, T. Horv, L. Schmidt-Thieme, et al., Personal-
Engineering 24 (01) (2014) 91–109. ized forecasting student perfor- mance, in: 2011 IEEE 11th In-
[65] O. Daramola, O. Emebo, I. Afolabi, C. Ayo, Implementation ternational Conference on Advanced Learning Technologies,
of an intelligent course advisory expert system, International IEEE, 2011, pp. 412–414.
Journal of Advanced Research in Artificial Intelligence 3 (5) [82] M. S. Sulaiman, A. S. A. Tamizi, M. R. Shamsudin, A. Azmi,
(2014) 6–12. Course recommendation system using fuzzy logic approach,
[66] T. Vuong, T. Trinh, H. Ha, X. Phan, T. Tran, A personalized Indonesian Journal of Electrical Engineering and Computer
course recommender system for undergraduate students, Inter- Science 17 (2020) 365–371.
national Journal of Learning and Teaching ijlt (2019) 181–190. [83] N. Torres, Recommender systems for education: A case of
[67] M. R. Sundari, G. Shreya, T. Jawahar, Course recommendation study using formative assessments, 2022 41st International
system, International Journal of Computer Applications 175 Conference of the Chilean Computer Science Society (SCCC)
(2020) 13–16. (2022) 1–6.
[68] F. L. Roux, E. Ranjeet, V. Ghai, Y. Gao, J. Lu, A [84] V. Sankhe, J. Shah, T. Paranjape, R. Shankarmani, Skill based
course recommender system using multiple criteria course recommendation system, 2020 IEEE International Con-
decision making method, in: Proceedings of the 2007 ference on Computing, Power and Communication Technolo-
International Conference on Intelligent Systems and gies (GUCON) (2020) 573–576.
Knowledge Engineering (ISKE 2007), Atlantis Press,
[85] S. P. Perumal, K. Arputharaj, G. Sannasi, Fuzzy family tree
2007/10, pp. 1407–1411. doi:10.2991/iske.2007.238. URL
similarity based effective e- learning recommender system,
https://doi.org/10.2991/iske.2007.238
2016 Eighth International Conference on Advanced Comput-
[69] H. Bydzovsk a, Course enrollment recommender system, in:
ing (ICoAC) (2017) 146–150.
Educational Data Mining, 2016.
[86] S. Ghosh, A hybrid programming course recommenda-
[70] V. Anupama, M. S. Elayidom, Course recommendation sys-
tion system using fuzzy logic and xdeepfm, 2023 Interna-
tem: Collaborative filtering, machine learning and topic mod-
tional Conference on Intelligent Systems, Advanced Com-
elling, 2022 8th International Conference on Advanced Com-
puting and Communication (ISACC) (2023) 1–8. URL
puting and Communication Systems (ICACCS) 1 (2022)
https://api.semanticscholar.org/CorpusID:257935912
1459–1462.
[87] Q. Guo, F. Zhuang, C. Qin, H. Zhu, X. Xie, H. Xiong, Q. He,
[71] N. D. Lynn, A. W. R. Emanuel, A review on recommender
A survey on knowledge graph-based recommender systems,
systems for course selection in higher education, IOP Con-
IEEE Transactions on Knowledge and Data Engineering 34
ference Series: Materials Science and Engineering 1098 (3)
(2020) 3549–3568.
(2021) 032039. doi:10.1088/1757-899X/1098/3/032039. URL
https://dx.doi.org/10.1088/1757-899X/1098/3/032039 [88] V. G. M. Murillo, D. E. P. Avenda no, F. R. Lopez, J. M.
[72] K. K. Jena, S. K. Bhoi, T. K. Malik, K. S. Sahoo, N. Z. G. Calleros, A systematic literature review on the hybrid ap-
Jhanjhi, S. Bhatia, F. Amsaad, E- learning course recom- proaches for recommender systems, Computaci on y Sistemas
mender system using collaborative filtering models, Elec- (2022).
tronics 12 (1) (2023). doi:10.3390/electronics12010157. URL [89] W. Kaiss, K. Mansouri, F. Poirier, Personalized e-learning
https://www.mdpi.com/2079-9292/12/1/157 recommender system based on a hybrid approach, 2022 IEEE
[73] G. George, A. M. Lal, A personalized approach to Global Engineering Education Conference (EDUCON) (2022)
course recommendation in higher educa- tion, Int. 1621–1627.
J. Semantic Web Inf. Syst. 17 (2021) 100–114. URL [90] G. Deepak, I. Trivedi, A hybridized deep learning
https://api.semanticscholar.org/CorpusID:233988383 strategy for course recommendation, International
[74] Y. Shen, H. Li, Z. Liao, Online education course recom- Journal of Adult Education and Technology (2023). URL
mendation algorithm based on path factors, 2022 IEEE 5th https://api.semanticscholar.org/CorpusID:258282835
International Conference on Information Systems and Com- [91] S. Sharma, A. T. Sharma, Y. Sharma, M. K. Bhatia, Rec-
puter Aided Education (ICISCAE) (2022) 257–260. URL ommender system using hybrid approach, 2016 International
https://api.semanticscholar.org/CorpusID:253271353 Conference on Computing, Communication and Automation
[75] P. Mishra, V. Jain, Course recommendation (ICCCA) (2016) 219–223.
system using content-based filtering, 2023 7th [92] O. Mazhoud, A. Kalboussi, A. H. Kacem, Educational recom-
International Conference on Trends in Electronics mender system based on learner’s annotative activity, Int. J.
and Informatics (ICOEI) (2023) 1431– 1436. URL Emerg. Technol. Learn. 16 (2021).
https://api.semanticscholar.org/CorpusID:258869504 [93] Z. Gulzar, A. A. Leema, G. Deepak, Pcrs: Personalized course

VOLUME 11, 2023 29

 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

recommender system based on hybrid approach, Procedia DR. FARHANA SARKER received
Computer Science 125 (2018) 518–524. her PhD degree in Computer Science
[94] M. Tkalcic, L. Chen, Personality and recommender systems, from the University of Southampton,
in: Recommender systems handbook, Springer, 2015, pp. UK. Before her PhD study, she worked
715–739. as a researcher in the University of
[95] M. Zhang, J. Tang, X. Zhang, X. Xue, Addressing cold start Southampton, UK and in King Saud
in recommender systems: A semi-supervised co-training algo- University, Riyadh, Saudi Arabia. Dr.
rithm, SIGIR ’14, Association for Computing Machinery, New
Sarker has been an Assistant Professor
York, NY, USA, 2014, p. 73–82.
at Department of Computer Science
[96] I. Palomares, S. V. Kovalchuk, Multi-view data approaches in
recommender systems: an overview: (invited paper), Procedia and Engineering, University of Liberal
Computer Science 119 (2017) 30–41, 6th International Arts Bangladesh (ULAB), Bangladesh from 2015 to 2023. She
Young Scientist Conference on Computational Science, is also a co-founder of CMED Health Ltd, a health tech startup
YSC 2017, 01-03 November 2017, Kotka, Finland. in Bangladesh. She has diversified research background in the
doi:https://doi.org/10.1016/j.procs.2017.11.157. URL area of computer science and engineering. Her current research
https://www.sciencedirect.com/science/article/pii/S187705091 includes education, technology-enhanced teaching and learning,
7323670 linked data, health, and data science. She published a number of
[97] L. Xue, P. Zhang, A. Zeng, Enhancing the long-term per- peer-reviewed journal and conference papers.
formance of recommender system, ArXiv abs/1904.00672
(2019).
[98] M. Fang, J. Liu, M. Momma, Y. Sun, Fairroad: Achieving
fairness for recommender systems with optimized antidote
data, Proceedings of the 27th ACM on Symposium on Access
Control Models and Technologies (2022).
[99] B. P. Knijnenburg, M. C. Willemsen, Z. Gantner, H. Soncu, C.
Newell, Explaining the user experience of recommender sys-
tems, User Modeling and User-Adapted Interaction 22 (2012) ARIFUR RAHMAN received the
441–504. B.Sc. degree in computer science and
[100] B. Cheng, Y. Zhang, D. Shi, Ontology-based personal- engineering from the Khulna Uni-
ized learning path recommendation for course learning, 2018 versity of Engineering and Technol-
9th International Conference on Information Technology in ogy (KUET), Bangladesh, in 2023.
Medicine and Education (ITME) (2018) 531–535. He is currently engaged as an NLP
Research Engineer serving at AIMS
LAB, United International University
with support from the Bill & Melinda
Gates Foundation and ICT Division,
Bangladesh. Besides, he was lead author in several manuscripts
in various conferences. His research interests include deep
learning, bioinformatics, natural language processing, computer
vision and the Internet of Things (IoT).

DR. SAZZAD HOSSAIN received his

PhD degree in Electrical and Computer
Engineering from the Portland State
University, Oregon, USA. Currently he
is working as a full-time member of
University Grants Commission, Gov-
ernment of Bangladesh. Dr. Hossain
NABILA KAMAL received the B.Sc. is a masterly instructor in ICT field
degree in Computer Science and En- with enormous knowledge on subject
gineering from Ahsanullah University matter. He joined as a Professor and
of Science & Technology, Bangladesh. Head of the Department of Computer Science and Engineering
She is currently pursuing the M.Sc. de- at University of Liberal Arts Bangladesh in December 2009. His
gree in Computer Science & Engineer- research interest focuses on quantum computer, de-coherence,
ing at United International University, natural computing, fault-tolerant computing, theoretical com-
Bangladesh. At present, she is working puter science, artificial intelligent, machine learning, big data,
as a Research Assistant at Advanced education, and human robot interaction. He has published more
Intelligent Multidisciplinary Systems than 50+ peer reviewed international journals and conference
(AIMS) Lab within the Department of Computer Science Engi- proceedings. He is also the author of many books.
neering, United International University. Her research interests
encompass technology-enhanced learning, data mining, machine
learning, recommender systems.

30 VOLUME 11, 2023

 Nabila et al.: Recommender System in Academic Choices of Higher Education: A Systematic Review

DR. KHONDAKER A. MAMUN re-

ceived his PhD and Postdoc from Uni-
versity of Southampton, UK and Uni-
versity of Toronto, Canada respec-
tively. At present, he is serving as a
Professor at the Department of Com-
puter Science and Engineering, and
Director of the Advanced Intelligent
Multidisciplinary Systems Lab (AIMS
Lab), United International University
(UIU), Bangladesh. He has research expertise on the applications
of 4IR technologies like, AI, big data and IoT in healthcare,
disabilities, brain computer interface (BCI), and education. He
has published more than 125 peer reviewed international journal
and conference articles. He has invented and implemented a
number of digital healthcare services, intelligent and mobile
based solutions for developing countries. He also holds a number
of patents. His contribution in the innovation and sustainable de-
velopment is featured in local and international news outlets, and
recognized internationally and by the government of Bangladesh.
Currently, Prof. Mamun is leading more than 10 national and
international research projects in AIMS Lab. Prof. Mamun is
the founder of CMED Health (www.cmed.com.bd), a startup that
commercialized from research and focuses on health inclusion
through digital platform and currently serving over 3 million
people in Bangladesh. Prof. Mamun is also the contributor in the
policy making in health sector of Bangladesh and topic driver of
WHO/ITU Focus Group on AI for Health (FG-AI4H).

VOLUME 11, 2023 31

View publication stats