IEEE EDUCATION SOCIETY SECTION

Received 30 April 2023, accepted 16 May 2023, date of publication 22 May 2023, date of current version 30 May 2023.
Digital Object Identifier 10.1109/ACCESS.2023.3278370

Analysis of Learning Behavior Characteristics and

Prediction of Learning Effect for Improving
College Students’ Information Literacy
Based on Machine Learning
YONG SHI 1, FANG SUN2 , HONGKUN ZUO3 , AND FEI PENG1
1 School of Computer Science, Huainan Normal University, Huainan 232038, China
2 Institute of Education, Huainan Normal University, Huainan 232038, China
3 School of Finance and Mathematics, Huainan Normal University, Huainan 232038, China

Corresponding author: Yong Shi ([email protected])

This work was supported in part by the Scientific Research Project of China Anhui Provincial Department of Education under Grant
2020jyxm1733 and Grant 2020qkl38, in part by the Key Research Project of Huainan Normal University under Grant 2022XJZD027.

ABSTRACT Information literacy is a basic ability for college students to adapt to social needs at present,
and it is also a necessary quality for self-learning and lifelong learning. It is an effective way to reveal the
information literacy teaching mechanism to use the rich and diverse information literacy learning behavior
characteristics to carry out the learning effect prediction analysis. This paper analyzes the characteristics of
college students’ learning behaviors and explores the predictive learning effect by constructing a predictive
model of learning effect based on information literacy learning behavior characteristics. The experiment used
320 college students’ information literacy learning data from Chinese university. Pearson algorithm is used to
analyze the learning behavior characteristics of college students’ information literacy, revealing that there is a
significant correlation between the characteristics of information thinking and learning effect. The supervised
classification algorithms such as Decision Tree, KNN, Naive Bayes, Neural Net and Random Forest are
used to classify and predict the learning effect of college students’ information literacy. It is determined that
the Random Forest prediction model has the best performance in the classification prediction of learning
effect. The value of Accuracy is 92.50%, Precision is 84.56%, Recall is 94.81%, F1-Score is 89.39%, and
Kapaa coefficient is 0.859. This paper puts forward differentiated intervention suggestions and management
decision-making reference in the information literacy teaching process of college students, with a view to
adjusting the information literacy teaching behavior, improving the information literacy teaching quality,
optimizing educational decision-making, and promoting the sustainable development of high-quality and
innovative talents in the information society.Our work involving research of the thinking and direction of the
sustainable development of information literacy training proved to be encouraging.

INDEX TERMS Machine learning, information literacy, learning behavior characteristics, learning effect,
innovative talents.

I. INTRODUCTION technology, computers and the Internet have been widely

With the rapid development of information technology repre- used in various fields of society. Information plays an increas-
sented by computer, network technology and communication ingly important role in the development of human society
and increasingly becomes one of the most active and decisive
The associate editor coordinating the review of this manuscript and factors in all fields of society. Information literacy, critical
approving it for publication was James Harland. thinking and creativity are the core skills that college students
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.
For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
VOLUME 11, 2023 50447
 Y. Shi et al.: Analysis of Learning Behavior Characteristics and Prediction of Learning Effect

must master in the 21st century [1]. In the information age, experience based on the characteristics of learning behavior
information literacy is an important part of college students’ before and after learning AlShammari et al. [8].The predic-
core literacy. Information literacy is a kind of adaptability to tion of learning results includes prediction theoretical model,
the information society. The information literacy of college empirical research of prediction model, comparison of algo-
students is directly related to the sustainable development rithms, development of algorithms, research of early warning
of future talents and the cultivation of innovative talents factors and literature review, etc. The prediction of students’
[2], [3].Information literacy is a part of cultural literacy and learning performance and learning effect is carried out using
overall quality. Cultivating college students’ information lit- regression analysis, neural network, Bayes and other methods
eracy has already become an important issue facing contem- Gaihua and Gangshan [9]. UNESCO’s 2019 report, Artificial
porary higher education. Intelligence in Education: Challenges and Opportunities for
Information literacy includes the basic knowledge and Sustainable Development, explores how artificial intelligence
skills of information and information technology, the ability technologies can help education systems use data to to pro-
to use information technology to learn, cooperate, communi- mote equity and quality in education [10].Using educational
cate and solve problems, as well as information awareness data mining technology and machine learning technology to
and social ethics. At present, information literacy educa- build learning effect prediction model through data-driven
tion has received the attention of people from all walks of way, that is, automatically learning from data to build predic-
life. The education departments and libraries in the United tion model, which is the current research focus and research
States, the United Kingdom, Australia and other countries trend.
have carried out information literacy education to differ- This study links multiple specific behavioral data together
ent degrees.In 2022, the Ministry of Education and other to create an integrated data link based on college students’
four departments of China jointly issued the ‘‘key points of learning behaviors in information literacy courses. The pre-
improving the digital literacy and skills of the whole people dictive analysis and evaluation of different machine learning
in 2022’’. Students’ information literacy and digital literacy classification models are used to classify and predict the
are expected to be further improved in the next few years learning effect of college students. This study focuses on the
[4].In recent years, due to the influence of online teaching and following questions.
hybrid teaching, and the development of artificial intelligence (1) Which indicators of information literacy learning
technology, information literacy has also received more and behavioral characteristics of college students have better pre-
more research attention. Many colleges and universities at dictive ability for learning effect?
home and abroad have opened information literacy courses (2) Which machine learning models have better predictive
through various ways to carry out targeted information lit- performance and efficacy based on the study sample?
eracy education. For example, on the MOOC platform of (3) What diagnostic observations for use in learning rec-
the University of China, Tsinghua University has opened ommendations and instructional interventions were derived
‘‘Information Literacy: A Compulsory Course for Academic in conjunction with the study findings?
Research’’, Wuhan University has opened ‘‘Information Lit-
eracy and Practice - A Pair of Academic Eyes’’, Sun Yat-sen II. LITERATURE REVIEW
University’s ‘‘Information Literacy General Course - A Com- This study conducted a literature study on the analysis of
pulsory Course for Digital Survival’’, and Sichuan Normal learning behavior characteristics and prediction of learning
University’s ‘‘Information Literacy and Lifelong Learning effect to improve college students’ information literacy.
(Autonomous Mode)’’ [5].In view of the existing information The literature data mainly comes from the common
literacy education for college students, many problems have databases for international paper retrieval such as Web of
emerged. Science, Scopus, Ei Compendex, etc., and is mainly based
In the field of education big data, learning prediction is a on the relevant research in the past three years.
very meaningful topic. Learning effect prediction is one of Since there are many professional terms and machine
the core issues in the field of learning analysis. Its essence learning algorithm terms in the reference documents, they are
is to use various data generated by learners in the learning uniformly described, as shown in Table 1.
process, and use the method represented by machine learning
to predict the learning effect. According to the prediction A. INFORMATION LITERACY LEARNING BEHAVIOR
results, teachers can know the learners’ learning status in ANALYSIS AND LEARNING EFFECT EVALUATION
time and intervene in the learning process in time. Such as Scholars have carried out research on information literacy
improving learners’ learning habits, adjusting teaching strate- from different angles. Specific literature analysis and com-
gies, etc. Wufati and Hao [6].Learning analysis technology parison are as follows:
has developed from principle exploration and application Literature comparison in research domain: In terms of
value to application in learning behavior analysis, data visu- information literacy learning behavior and learning effect,
alization and learning prediction Hang et al. [7]. Learning many studies in recent years have focused on the evalu-
prediction is based on learning achievement, learning goals, ation framework of information literacy effect; Strategies
and learning ability, and predicts learning effect and learning for improving information literacy learning behavior; The
50448 VOLUME 11, 2023
Y. Shi et al.: Analysis of Learning Behavior Characteristics and Prediction of Learning Effect

TABLE 1. Abbreviation of professional terms. research, learning quality analysis and curriculum evaluation
research.
Comparison of literatures on research methodology: Most
research methods are mainly machine learning algorithms,
including neural networks, decision trees, support vector
machines and other algorithms.The tick marks in Table 3 rep-
resent the use of machine learning. Comprehensive analysis
and quantitative research were used in individual studies.
Comparison of literature on finding: There are many
research results based on machine learning algorithm. For
example, performance prediction model; Students’ willing-
ness analysis model; Prediction of classroom teaching effect;
Learning behavior diagnosis model, etc.
Scholars have carried out research on learning behavior
analysis and learning effect from different angles. Specific
literature analysis and comparison are as follows:
Literature comparison in research domain: The research
mainly focuses on teaching effect prediction model research,
learning performance model research, teaching model effect
research, learning quality analysis and curriculum evaluation
research.
Comparison of literatures on research methodology: Most
research methods are mainly machine learning algorithms,
including neural networks, decision trees, support vector
machines and other algorithms; Comprehensive analysis and
cultivation of specific ability of information literacy; Online
quantitative research were used in individual studies.
courses or the relationship between intelligent environment
Comparison of literature on finding: There are many
and information literacy.
research results based on machine learning algorithm. For
Literature comparison on research methodology: Most
example, performance prediction model; Students’ willing-
scholars mainly use quantitative research, qualitative
ness analysis model; Prediction of classroom teaching effect;
research, questionnaire survey, data mining, and factory
Learning behavior diagnosis model, etc.
quality-experience, while some scholars use machine learn-
To sum up, the current research on information liter-
ing technology. The tick marks in Table 2 represent the use
acy is mainly based on theoretical deduction and expe-
of machine learning. According to the literature, scholars
rience, establishing the hypothesis that some factors are
mainly use traditional research methods in research methods,
related to academic performance, and then collecting data
and seldom use machine learning methods to carry out rele-
through questionnaires and interviews to analyze and verify
vant research.
the hypothesis. This method can only prove the correlation
Literature comparison in the research of finding: Scholars
between selected factors and academic achievement, but it
have made fruitful exploration results in the research. For
is difficult to determine the quantitative relationship between
example, some scholars establish a correct teaching mode,
selected factors and academic achievement. Machine learning
some scholars explore new teaching methods of informa-
and data mining technology are rarely used, and data intel-
tion literacy, and some scholars build prediction models to
ligence analysis research of information literacy education
enhance the use of information literacy. The literature shows
is lacking. Some researchers use decision tree, neural net-
that there are few achievements in the analysis of information
work and other algorithms to establish academic achievement
literacy learning behavior and the construction of learning
prediction models, but lack of information literacy learning
effect prediction model.The detailed comparative studies are
effect prediction research.
summarized in Table 2.
With the continuous development and maturity of intelli-
gent technologies such as data mining, emotion analysis and
B. LEARNING BEHAVIOR ANALYSIS AND LEARNING pattern recognition, especially the combination of machine
EFFECT PREDICTION BASED ON MACHINE LEARNING learning technology and education field, it provides strong
Scholars have carried out research on learning behavior anal- technical support for learning prediction research. Although
ysis and learning effect from different angles. Specific litera- some studies have pointed out the negative impact of artificial
ture analysis and comparison are as follows: intelligence on educational research, the use of educational
Literature comparison in research domain: The research data mining and other technologies is still the current research
mainly focuses on teaching effect prediction model research, trend. Therefore, it is an urgent problem to build an infor-
learning performance model research, teaching model effect mation literacy learning behavior characteristic analysis and

VOLUME 11, 2023 50449

 Y. Shi et al.: Analysis of Learning Behavior Characteristics and Prediction of Learning Effect

TABLE 2. Research on information literacy learning behavior and learning effect.

learning effect prediction model for college students with establishment of information literacy standards should not
strong usability, easy operation and good prediction perfor- be limited to general standards [40].The research team has
mance, as well as differential recommendation and interven- built the evaluation index system of college students’ infor-
tion based on the prediction results. mation literacy in the previous research [41].The index sys-
tem provides a basic reference tool for this paper.Based on
III. MATERIALS AND METHODS this evaluation index, the research team observed, measured,
A. RESEARCH TOOLS extracted and described the information literacy learning
Common learning prediction tools include Weka, SPSS, behavior characteristics of college students, and formed the
Python, Rapidminer and other tools. In this study, SPSS and information literacy learning behavior characteristics obser-
Rapidminer analysis tools are mainly used in data preprocess- vation scale for college students. The scale includes aware-
ing, feature set selection and classification prediction, and ness and attitude, knowledge and skills, application and
model performance evaluation. Rapidminer is mainly used in innovation, ethics and responsibility. Awareness and atti-
machine learning. Rapidminer is the world’s mainstream data tude mainly focus on the understanding of the importance
mining and machine learning software. It provides functions of information technology. Knowledge and skills mainly
such as data preprocessing and visualization, predictive anal- focus on the knowledge and skills of information technology.
ysis and statistical modeling, evaluation and deployment, and Application and innovation mainly examine the cognitive
has rich machine learning algorithms [39]. thinking and innovative application of information technol-
ogy. Morality and responsibility mainly focus on information
B. RESEARCH OBJECT AND DATA SOURCE laws, regulations and moral concepts. There are 4 first-level
Due to regional factors, different regions have differ- indicators, 9 second-level indicators and 28 third-level indi-
ent requirements for information literacy. Therefore, the cators. In order to measure the learning effect of students,

50450 VOLUME 11, 2023

Y. Shi et al.: Analysis of Learning Behavior Characteristics and Prediction of Learning Effect

TABLE 3. Learning behavior analysis and learning effect prediction based on machine learning.

this study divides the students’ learning scores into five Identify and classify information (IPC1); Using the Web to
categories: excellent (5), good (4), medium (3), qualified find, filter, and judge information (IPC2); Determine the
(2) and unqualified (1). Each three-level indicator of infor- correctness and reliability of information sources (IPC3);
mation literacy learning behavior characteristics of college Using information technology related knowledge and meth-
students corresponds to Likert’s five-level scale: ‘‘1=never’’; ods to solve problems (IAC1); Using information technology
‘‘2=seldom’’; ‘‘3=sometimes’’; ‘‘4=often’’; ‘‘5=always’’. tools such as mind mapping tools to assist learning (IAC2);
Table 4 describes the observed indicators of information Leveraging Information Technology to support Lifelong
literacy learning behavioral characteristics of college stu- Learning (LLC1); Using Information Technology to Support
dents. In conjunction with the above, the four areas of learn- Professional and Personal Development (LLC2).
ing behavior are described in terms of learning behavior in Learning behavior in knowledge and skills:Mainly includ-
consciousness and attitude, learning behavior in knowledge ing Information science knowledge(ISK) and Information
and skills, learning behavior in application and innovation, application Skills (IAS). Specific behaviors include: Under-
and learning behavior in morality and responsibility. stand all kinds of operating systems, word processing soft-
Learning behavior in consciousness and attitude:Mainly ware, graphics and image processing software, video and
including Information perception consciousness (IPC), audio processing software operation method (ISK1); Under-
Information application consciousness(IAC) and Lifelong stand the development history, basic status and future trend of
learning consciousness (LLC). Specific behaviors include: information technology (ISK2); Master the basic knowledge

VOLUME 11, 2023 50451

 Y. Shi et al.: Analysis of Learning Behavior Characteristics and Prediction of Learning Effect

TABLE 4. Observation scale of information literacy learning behavior characteristics of college students.

and technology of information retrieval and evaluation, infor- and methods (IAS3); Create valuable information resources
mation classification and storage method (ISK3); Master the based on specific teaching content or around specific teaching
basic scientific knowledge of information literacy, data lit- topics (IAS4).
eracy, visual literacy and other multi-literacy (ISK4); Use Learning behavior in application and innovation:Mainly
various search engines and network platforms to find the including Information thinking (IT) and Information behav-
required information (IAS1); Classify the information and ior (IB). Specific behaviors include: Define and identify
present the information in a tabular form (IAS2); Identifica- implicit assumptions in information, and deduce informa-
tion and analysis of information through various approaches tion (IT1); Carry out targeted information-based instructional

50452 VOLUME 11, 2023

Y. Shi et al.: Analysis of Learning Behavior Characteristics and Prediction of Learning Effect

design and implement effective instructional activities (IT2); literacy learning behavior questionnaire data and information
Using information technology to support services and man- literacy course performance data of 320 junior students in
agement (IT3); Construct problem solutions by integrat- Huainan Normal University in 2020. Data was collected by
ing resources and using reasonable algorithms (IT4); Use means of a web-based questionnaire administered in batches
collaborative tools to create and manage content,such as to students in each class. A pre-survey was conducted before
project management systems, shared documents, etc. (IB1); the questionnaire was distributed to test whether the questions
Use advanced communication tools to communicate with were fully understood by the subjects, whether the expression
people (e.g. video conferencing, data sharing, application was appropriate and the degree of cooperation, so the overall
sharing) (IB2); Developing innovative teaching applications recall quality of the questionnaire was very high. The data
(IB3); To carry out information technology cooperation and presents positive distribution, with little difference between
exchange (IB4). the data, and good reliability and validity.
Learning behavior in morality and responsibility: Mainly
including Information ethics (IE),Information laws and reg- 2) DATA PREPROCESSING
ulations (ILR). Specific behaviors include: Healthy and cor-
Figure 2 shows a descriptive statistical overview.The hori-
rect use of learning resources to create a good information
zontal axis represents variables, and the vertical axis repre-
learning environment (IE1); Restrain one’s own information
sents numerical values. It gives some indication of the data
ethical behavior and supervise others’ information behavior
results for each variable.The descriptive statistics revealed
(IE2); Abide by the network civilization convention, purify
a few missing values and outliers.The Min, Max, Average
the network language, civilized and polite learning and com-
and Deviation of each feature subset are shown in the figure.
munication (IE3); Impart knowledge of laws, regulations and
Average ranks in the top 3 for IAC1, IPC3 and IPC2, while
ethics related to technology utilization (ILR1); Be clear about
IB4, IB2 and IB3 rank in the bottom 3; IE1, IPC1 and IAC1
equal access and access to information and respect for others’
rank in the bottom 3 for Deviation, while IB4, ISK2 and ISK3
intellectual property rights (ILR2).
rank in the top 3.
To ensure the quality of the classification learning model
C. RESEARCH METHOD construction, data preprocessing was performed on the col-
According to the general process of learning analysis and lected learning behavior characteristics and performance
machine learning, this study mainly includes data preprocess- data, including operations such as missing value processing,
ing, feature extraction, algorithm selection, model training, abnormal data processing and data transformation.Outlier
performance evaluation and result analysis. processing. SPSS was used to remove the null data and other
The main technical route of this study is shown in Figure 1: abnormal data present in the training set. Then the outliers
(1)The correlation between learning behavior characteris- of the data are removed by box plot.After data cleaning,
tics and learning effect is calculated on the basis of data clean- 315 recorded data were finally retained.
ing; Observe and analyze the relationship between predictive Data transformation. In order to enable the machine
variables and learning effect, and establish the feature subset learning model to achieve better recognition, the data in
participating in model construction. the collected training set needs to be transformed opera-
(2) The five models are trained and tested using the cross- tionally. The attribute of learning effect should be defined
validation method of ten-fold. as ‘‘nominal attribute’’. The transformation level attributes
(3) On the basis of a single prediction model, the prediction are: ‘‘5=Excellent’’, ‘‘4=Good’’, ‘‘3=Medium’’, ‘‘2=Pass’’,
effect of the model is improved by optimizing the algorithm ‘‘1=Fail’’.
parameters several times.
(4) Carry out prediction effect evaluation and compara-
IV. RESULTS AND DISCUSSION
tive analysis, and establish the optimal prediction algorithm
A. CORRELATION ANALYSIS OF LEARNING BEHAVIOR
model.
CHARACTERISTICS AND LEARNING EFFECT
Modeling feature subset selection can be achieved through
D. DATA COLLECTION AND PREPROCESSING correlation analysis of learning behavior characteristics and
1) DATA COLLECTION learning effect. Correlation analysis is the analysis of two or
The research data comes from the ‘‘Special Survey on Infor- more elements of variables that are related as a measure of
mation Literacy of College Students’’ implemented by the their degree of association. The related elements must have
research group of the ‘‘Research on Information Literacy some kind of association or likelihood in order for correlation
of College Students Supported by Smart Campus’’, a teach- analysis to be performed.
ing quality project in Anhui Province, China, in 2022. The If two variables have a strong interdependence, then we
study takes into account the impact of scattered, random can say that the two variables have a high correlation. If the
and representative data on the student population, involv- values of both groups increase at the same time, they are said
ing students from a variety of disciplinary and professional to be positively correlated; if the value of one group increases,
backgrounds.The data were collected from the information then the value of the other group decreases, which is called

VOLUME 11, 2023 50453

 Y. Shi et al.: Analysis of Learning Behavior Characteristics and Prediction of Learning Effect

FIGURE 1. Technical road map.

FIGURE 2. Dataset descriptive statistics.

a negative correlation. Pearson’s algorithm is used here to the diagonal has a symmetric relationship with the part
calculate the correlation. Pearson’s correlation coefficient is below.
an important measure of the interrelationship between two The Pearson correlation coefficient between each variable
variables, and it has a correlation between -1 and 1. If there and the learning effect was calculated to measure the linear
are P related variables and the correlation coefficient of the correlation between the existing variables. The correlation
two variables needs to be found, the number of correlation coefficients between the variables are shown in Figure 4. The
coefficients obtained is as follows: intersection of the two variables in the rows and columns is
the significance plot, and the color knob at the bottom corre-
RP×P = p(p − 1)/2 (1)
sponds to the correlation coefficient. The correlation between
If the variables are arranged into a numerical square the predictor variables and the learning effect is shown in
in order of their numbering, this square is the corre- Figure 3. R takes values between -1 and +1. If r>0, it means
lation matrix.There are two identical variables on the that the two variables are positively correlated, i.e., the larger
diagonal from the top left to the bottom right, both of the value of one variable, the larger the value of the other
which have a value of 1; the correlation coefficient above variable; if r<0, it means that the two variables are negatively

50454 VOLUME 11, 2023

Y. Shi et al.: Analysis of Learning Behavior Characteristics and Prediction of Learning Effect

FIGURE 3. Correlation matrix data.

1) ANALYSIS OF HIGH CORRELATION LEARNING BEHAVIOR

CHARACTERISTICS
The top five variables are IT1, IT4, IAS1, IT2, and IT3, and it
can be seen that the relationship between Information think-
ing (IT) and learning effect in Application and innovation is
the most significant. The top three variables with the highest
correlation with learning effect were IT1 with 0.810, IT4 with
0.768, and IAS1 with 0.766.
IT1 is defining and identifying implicit assumptions in
information and inferring information. IT1 is concerned with
information mining and targeted critical cognition. IT4 is
constructing problem solutions by integrating resources and
FIGURE 4. Correlation between predictor variables and learning effect.
using rational algorithms. IT4 is concerned with informa-
tion creation, focusing on specific solutions to problems and
correlated, i.e., the larger the value of one variable, the smaller emphasizing the development of creativity. ias1 is using var-
the value of the other variable. The larger the absolute value of ious search engines and web platforms to find the required
r, the stronger the correlation; the smaller the absolute value information. It is an information application skill. This learn-
of r, the weaker the correlation [42].The linear correlation ing behavior focuses on the ability of college students to
between the existing variables was measured by calculating access information. The results of the analysis suggest the
Pearson’s correlation coefficients between the variables of development of application and innovation skills, and the
information literacy of college students and learning effect. increasing of information acquisition skills in knowledge and
It was concluded that the vast majority of the predictor skills.Integrating critical thinking methods into the informa-
variables showed some positive correlation with learning tion literacy education system Zhuozhuo et al. [43] is an
outcomes. As the correlations reflect some variability, this important strategy to improve the innovation and creativity
provides support for the analysis of learning behavioural ability of college students and break through the bottleneck
characteristics. of information literacy education. At the same time, attention

VOLUME 11, 2023 50455

 Y. Shi et al.: Analysis of Learning Behavior Characteristics and Prediction of Learning Effect

should still be paid to cultivating and improving students’ The Precision is the ratio of the number of positive cases
information literacy knowledge and skills, especially the correctly predicted by the classification model to the number
improvement of information acquisition ability in the infor- of all positive cases predicted by the classification model,
mation age. i.e., the proportion of true positive cases among all results
predicted as positive.
2) ANALYSIS OF LOW CORRELATION LEARNING BEHAVIOR
Precision = TP/(TP + FP) (3)
CHARACTERISTICS
The three variables with the lowest correlations with learning The Recall is the ratio of the number of positive samples
effect were ILR2 with 0.430, LLC2 with 0.484, and IPC1 correctly predicted by the classification model to the actual
with 0.486. ILR2 is learning equal access to information and number of positive samples in the entire test set, i.e., the
respecting the intellectual property rights of others; LLC2 proportion of true positive cases that are found by the clas-
is using information technology to support professional and sification model.
personal development; IPC1 is identifying and classifying
information. From the perspective of learning behavior pre- Recall = TP/(TP + FN ) (4)
sentation space, these three learning behaviors are abstract in The F1-Score is a comprehensive metric that combines
nature; from the perspective of learning behavior presentation Precision and Recall. Since Precision and Recall are a pair
time, these three learning behaviors are less integrated with of contradictory measures, and different problems focus on
college students’ study, life and existing learning environ- different criteria, F1-Score is a good comprehensive evalua-
ment than other learning behavior characteristics. This sug- tion metric, and the larger the value of this metric, the better.
gests a reference for later pedagogical improvement, which
requires more streamlined and effective learning behaviors F1 − Score = 2 × Precision × Recall/(Precision + Recall)
for college students in response to these indicators. (5)
The kappa (KIA) coefficient is a measure of classifica-
3) ESTABLISHMENT OF A SUBSET OF LEARNING BEHAVIOR tion accuracy. kia is an index that enables the calculation of
CHARACTERISTICS overall consistency and classification consistency. The KIA
In order to construct better prediction models and achieve is used to perform an assessment of the accuracy of a multi-
better prediction results with fewer features, three learned classification model. The higher the value of this coefficient,
behavioral features with correlations below 0.500 were not the higher the classification accuracy achieved by the model.
involved in the prediction model construction, namely IPC1 kappa coefficient can be calculated as follows. Po denotes
(0.486), LLC2 (0.484), and ILR2 (0.430). The gender vari- the proportion of observation accuracy or consistency cells.
able was also not involved in the prediction model construc- Pc indicates the proportion of cells that are contingently
tion because its degree of relationship with learning effect consistent or expected to be contingently consistent.
was -0.103.
To sum up, a subset of 25 information literacy learn- KIA = (Po − Pc)/(1 − Pc) (6)
ing behavior characteristics of college students is currently
In terms of prediction model selection, the main focus is
retained, specifically IT1, IT4, IAS1, IT2, IT3, ILR1, IAS4,
on predicting college students’ learning effect levels through
IB1, ISK4, IE1, IE3, IAC1, ISK3, IPC2, IAS3, IB2, LLC1,
their information literacy behavioral performance indicators.
IE2, IAS2, IPC3, ISK1, IB3, ISK2, IB4, IAC2.
This is a typical classification problem, so the classical
machine learning classification algorithm is used to compare
B. CLASSIFICATION MODEL PREDICTION OF LEARNING the prediction performance of different models separately
EFFECT for this study sample. In the following, the five models are
The performance evaluation metrics of the binary classifi- trained and tested using a ten-fold cross validation method.
cation model include Accuracy, Precision, Recall, F1-Score The dataset is first divided into ten parts, then nine of them
(F1), etc. [44].TP indicates the number of positive samples are rotated as training data and the remaining one is used
whose learning effect was correctly predicted; TN indicates as test data, and finally the model training is performed by
the number of negative samples whose learning effect was maximizing the use of samples by averaging the correct rate
correctly predicted; FP indicates the number of samples each time as the evaluation value of the algorithm accuracy.
that were incorrectly predicted as positive; and FN indicates
the number of samples that were incorrectly predicted as 1) DECISION TREE
negative. Decision tree is a greedy algorithm that classifies instances
Classification Accuracy is the percentage of the number based on features and performs recursive binary partitioning
of correct samples that can be predicted in the classification on the feature space. Starting from the root node of the tree,
model and reflects the accuracy of the overall classification. the sample data is compared with the feature nodes in the
decision tree, and the branches at the next level are selected
Accuracy = (TP + TN )/(TP + TN + FP + FN ) (2) to continue the comparison based on the judgment result,

50456 VOLUME 11, 2023

Y. Shi et al.: Analysis of Learning Behavior Characteristics and Prediction of Learning Effect

FIGURE 5. The selection process of Decision Tree hyperparameters. FIGURE 6. The selection process of KNN hyperparameters.

and the final leaf node is the classification result [45].The

advantage of decision trees is that they are more readable and
faster to classify [46].The C4.5 decision tree algorithm uses
the ‘‘gain ratio’’ to select the optimal partitioning attribute.
By training and optimizing the Decision Tree parameters,
the best recognition effect of the model is obtained when the
core parameter Maximum depth is set to 8, the minimum leaf
size is 2 and the confidence is 0.1. The experimental process
of obtaining Decision Tree is shown in Figure 5. Observe the
results of multiple experiments and get the optimal accuracy
rate of 84.17%.

2) K-NEAREST NEIGHBOR
The K-Nearest Neighbor (KNN) algorithm is an algorithm FIGURE 7. The selection process of Naive Bayes hyperparameters.
based on statistical classification. The advantage of this
algorithm is that it does not need to partition the vector
space consisting of all data records, and the classifica- 4) NEURAL NET
tion is better by training the model data to find K simi- Neural network is a mathematical model that simulates bio-
lar vectors, and the disadvantage is that it is insensitive to logical neural networks for information processing, and neu-
outliers [47]. ral networks are applied in classification problems with good
By training and optimizing the KNN parameters, the model results [49]. Neural networks are mainly composed of: input
recognition effect is best when the core parameter K is set to layer, hidden layer, and output layer.
6. The experimental procedure of KNN is obtained as shown By training and optimizing the Neural Net parameters, the
in Figure 6. The optimal accuracy rate of 90.83% is obtained best recognition effect of the model is obtained when the
by observing the results of multiple experiments. core parameters momentum is set to 0.9, training cycles to
200, and learning rate to 0.01. The experimental process of
3) NAIVE BAYES obtaining Neural Net is shown in Figure 8. Observing the
Naive Bayes is a data detection and classification algorithm results of multiple experiments, the optimal accuracy rate is
based on probability theory. The algorithm can relate the prior obtained as 91.67%.
and posterior probabilities of events and use sample data with
prior information to determine the posterior probability of 5) RANDOM FOREST
events. Its advantage is that the model is simple to construct Random Forest utilizes random sampling of data samples
and has high efficiency and stability [48]. and features to train multiple tree classifiers, avoiding the
By training and optimizing the Naive Bayes parameters, learning of all samples and all features per tree, thus increas-
the model recognition effect is best when the core parameter ing randomness, avoiding overfitting, and integrating the
minimum bandwidth is set to 0.2. The experimental proce- results of a single decision tree according to the rules of
dure of obtaining Naive Bayes is shown in Figure 7. The Bagging [50].The training sample data are sampled with put-
optimal accuracy rate of 90.00% is obtained by observing the back to generate K classification regression trees; assume that
results of multiple experiments. there are n features in the feature space and m features are

VOLUME 11, 2023 50457

 Y. Shi et al.: Analysis of Learning Behavior Characteristics and Prediction of Learning Effect

TABLE 5. Prediction results of the classification model.

FIGURE 8. The selection process of Neural Net hyperparameters.

FIGURE 10. Performance comparison of learning effect prediction

FIGURE 9. The selection process of Random Forest hyperparameters. models.

randomly selected at the nodes of each tree, requiring m < n; Neural Net, KNN; the highest Precision is Naive Bayes,
make each tree grow maximally without any pruning; form reaching 93.06%, followed by Random Forest, KNN; the
a forest by multiple trees, and the classification results are highest Recall is Random, reaching 94.81%, followed by
determined by how many tree classifiers vote. KNN, Decision Tree; the highest F1-Score is Random For-
By training and optimizing the parameters of the Random est, reaching 89.39%, followed by KNN, Naive Bayes; the
Forest model, the best recognition effect of the model was highest Kapaa is Random Forest, reaching 0.859, followed
obtained when the number of trees parameter was set to by Neural Net The results of all indicators show that Random
150 and the criterion was set to gain_ratio. The experimental Forest prediction model has the best performance and can
results of the Random Forest are shown in Figure 9 after be used to enhance the learning effect prediction of college
repeated execution for several times. Observe the results of students’ information literacy.
the multiple experiments and get the optimal accuracy rate of
92.50%. C. DISCUSSION
After the parameter tuning of each model, the prediction Machine learning research is not simply to pursue the high
results of each model are obtained as shown in Table 5. accuracy of the machine learning prediction model, but
The visual illustration of the prediction models is shown more importantly to explore the characteristics that can be
in Figure 10. The range of kappa taking values represents explained and trusted for optimizing the teaching process
different degrees. 0.1∼0.2: slight; 0.2∼0.4: fair; 0.4∼0.6: [52].Through the correlation study, the intervention measures
moderate; 0.6∼0.8: substantial; 0.8∼1.0: almost perfect [51]. needed to improve the teaching quality of college students’
The kappa of each model kappa indicates that the overall information literacy are clarified. Pearson algorithm is used
consistency and classification consistency of each model are to analyze the learning behavior characteristics of college
normal and basically meet the requirements. students’ information literacy, revealing that there is a more
From the analysis of the prediction results, the highest significant correlation between information thinking and
Accuracy is Random Forest, reaching 92.50%, followed by information application skills and learning effect, especially

50458 VOLUME 11, 2023

Y. Shi et al.: Analysis of Learning Behavior Characteristics and Prediction of Learning Effect

information thinking.Information literacy education should learning. This study provides a more reliable data base for
focus on cultivating students’ innovative spirit and practical educational administrators to analyze the potential connec-
ability.Some universities try to cultivate critical thinking from tions between information literacy education phenomena and
the perspective of knowledge transfer [53]; some scholars outcomes, thereby increasing the success rate of educational
propose the integration model of information literacy educa- decisions.
tion with professional courses that integrates critical think- In conclusion, this study proposes an effective machine
ing [54], adopt the information literacy education model that learning approach to characterize the learning behaviors and
integrates innovation and entrepreneurship training of college predict the learning effects of information literacy among
students to cultivate students’ innovative thinking by using college students. This study uses a data-driven thinking to
the ‘‘holistic thinking approach’’ [55], and improve critical promote teachers and students to optimize their learning paths
reflection ability based on the ‘‘163’’ information literacy and improve the effectiveness of information literacy instruc-
education system of ‘‘Internet+’’ [56].However, there is a tion. It also strongly supports the implementation of differen-
lack of a complete training system for critical thinking ability tiated teaching decision-making [59] and the construction of a
and innovative thinking ability. long-term mechanism for differentiated educational decisions
Combining the four metrics of accuracy, precision, recall, through a data-driven approach.
F1 value, and kappa, the performance of the classifier Although this study has conducted some exploration, there
obtained by the Random Forest algorithm is optimal in terms are still limitations. While machine learning approaches work
of prediction performance for all types of models. The higher to some extent, the study suggests that technological tools
prediction accuracy proves the effect of machine learning need to be applied according to the specific teaching and
algorithms applied to learning effect prediction modeling, learning situation. Further research is proposed in the follow-
which is basically in line with the findings of related studies ing areas at a later stage.(1)Learning effect prediction has not
such as Juan et al. [57] and Faqin et al. [58]. By using the been able to cover other possible factors in learning scenarios,
Random Forest algorithm model for predicting the learning which puts higher demands on the quality of the learning
effect of college students’ information literacy, we can predict behavior trait scale. We will explore the learning behavior
the learning effect of college students in information literacy characteristics in more scenarios, evolve the learning behav-
education more accurately, guide the adjustment of teaching ior characteristics scale, improve the universality, and form a
behaviors and allocation of teaching resources, and effec- closed loop of textbook development of teaching experiment,
tively guarantee teaching quality. teaching research, and teaching practice.(2)In this study, only
five supervised classification algorithms are used, such as
V. CONCLUSION Decision Tree, KNN, Naive Bayes, Neural Net, and Random
The results prove that the prediction model proposed in this Forest. In subsequent studies, the adopted algorithms can be
paper has a significant effect on the cultivation of informa- collectively improved to achieve better prediction results.
tion literacy of college students. On the one hand, Algo-
rithmic analysis of the learning behaviour characteristics of ACKNOWLEDGMENT
college students’ information literacy reveals a more signif- The author would like to thank everyone who made every
icant correlation between information thinking, information effort to improve the content of this research paper. They also
application skills and learning effect. Emphasis should be like to thank the anonymous reviewers and editors for their
placed on the cultivation of information thinking, while not work on the publication of this paper.
neglecting the cultivation of information acquisition ability.
Universities should understand the importance and urgency REFERENCES
of information literacy education for college students from [1] Z. Changhai, ‘‘Research on the information literacy education model
the height of sustainable development of talents and culti- of Chinese college students based on critical thinking and creativ-
ity,’’ J. China Library, vol. 4, no. 15, pp. 15–16, Aug. 2016, doi:
vation of high-quality and innovative talents. It is necessary 10.13530/j.cnki.jlis.164008.
to make full use of network and multimedia technologies [2] S. Hui, ‘‘Information literacy education for college students,’’ Educ. The-
to provide intelligent learning tools and learning environ- ory Pract., vol. 30, no. 10, pp. 38–39, Oct. 2008.
[3] D. Yan and G. Li, ‘‘A heterogeneity study on the effect of digital education
ments conducive to independent, cooperative and research technology on the sustainability of cognitive ability for middle school
learning, to incorporate critical thinking methods into the students,’’ Sustainability, vol. 15, no. 3, pp. 2784–2786, Feb. 2023, doi:
information literacy education system, to establish a long- 10.3390/su15032784.
[4] Y. Xiaohong, Z. Xin, and Z. Jing, ‘‘Research on the Chinese experi-
term assessment mechanism oriented to the cultivation of ence of the ‘epidemic’ of the education war—Online education perspec-
critical thinking, and to actively promote the critical thinking tive,’’ China Distance Educ., vol. 43, no. 1, pp. 1–11, Jan. 2023, doi:
cognition and knowledge creation skills of college students. 10.13541/j.cnki.chinade.2023.01.002.
[5] G. Shan and F. Shan, ‘‘Digital literacy practice and enlightenment of
On the other hand, it is necessary to further optimize the the University of Queensland library in Australia,’’ Library Sci. Res.,
learning methods in such aspects as information laws and vol. 7, no. 10, pp. 95–101, Jul. 2019, doi: 10.15941/j.cnki.issn1001-
regulations, streamline the learning contents in such aspects 0424.2019.10.015.
[6] T. H. Wufati, ‘‘Mining the characteristics of meaningful learning behavior:
as information perception awareness, and continuously instill A framework for predicting learning outcomes,’’ Open Educ. Res., vol. 25,
and guide college students to establish the concept of lifelong no. 6, pp. 75–82, Jan. 2019, doi: 10.13966/j.cnki.kfjyyj.2019.06.008.

VOLUME 11, 2023 50459

 Y. Shi et al.: Analysis of Learning Behavior Characteristics and Prediction of Learning Effect

[7] H. Hang, L. Yaxin, L. Qi’e, Y. Hairu, Z. Qiuhua, and C. Yifan, ‘‘The occur- [26] Y. Jia and E. Wang, ‘‘Research on information anxiety of college stu-
rence process, design model and mechanism explanation of deep learn- dents under the background of information overloaded based on sup-
ing,’’ China’s Distance Educ., vol. 5, no. 1, pp. 54–61, Mar. 2020, doi: port vector machine optimization alogrithm,’’ in Proc. 2nd Int. Conf.
10.13541/j.cnki.chinade.2020.01.005. Inf. Sci. Educ. (ICISE-IE), Nov. 2021, pp. 484–487, doi: 10.1109/ICISE-
[8] I. A. AlShammari, M. Aldhafiri, and Z. Al-Shammari, ‘‘A meta-analysis of IE53922.2021.00117.
educational data mining on improvements in learning outcomes,’’ College [27] C. Pei, ‘‘The construction of a prediction model for the teaching
student J., vol. 47, no. 2, pp. 326–333, Jun. 2013. effect of two courses education in colleges and universities based
[9] W. Gaihua and F. Gangshan, ‘‘Prediction of online learning behav- on machine learning algorithms,’’ Wireless Commun. Mobile Com-
ior and achievement and design of learning intervention model,’’ put., vol. 2022, Aug. 2022, Art. no. 1167454, doi: 10.1155/2022/
China’s Distance Educ., vol. 2, no. 2, pp. 39–48, Mar. 2019, doi: 1167454.
10.13541/j.cnki.chinade.20181214.007. [28] H. Xu, ‘‘GBDT-LR: A willingness data analysis and prediction model
[10] UNESCO. Accessed: Apr. 1, 2023. [Online]. Available: https://www. based on machine learning,’’ in Proc. IEEE Int. Conf. Adv. Electr.
unesco.org/en/articles/challenges-and-opportunities-artificial- Eng. Comput. Appl. (AEECA), no. 8, Aug. 2022, pp. 396–401, doi:
intelligence-education 10.1109/AEECA55500.2022.9919013.
[11] G. Yang, B. Wen, and W. Lin, ‘‘Research status, hot spots and enlight- [29] T. Li, ‘‘students’ numeracy and literacy aptitude analysis and prediction
enment of college students’ information literacy: Based on bibliometric using machine learning,’’ J. Comput. Commun., vol. 10, no. 8, pp. 90–103,
analysis of CNKI from 2000 to 2021,’’ in Proc. 4th World Symp. Softw. Aug. 2022, doi: 10.4236/jcc.2022.108006.
Eng., Sep. 2022, pp. 161–166, doi: 10.1145/3568364.3568389. [30] X. Shen and C. Yuan, ‘‘A college student behavior analysis and man-
[12] J. Li, ‘‘Machine learning-based evaluation of information literacy enhance- agement method based on machine learning technology,’’ Wireless Com-
ment among college teachers,’’ Int. J. Emerg. Technol. Learn., vol. 17, mun. Mobile Comput., vol. 2021, Aug. 2021, Art. no. 3126347, doi:
no. 22, pp. 116–131, Nov. 2022, doi: 10.3991/ijet.v17i22.35117. 10.1155/2021/3126347.
[31] R. Wang, ‘‘Research on effectiveness of college English blended teaching
[13] L. Yu, D. Wu, H. H. Yang, and S. Zhu, ‘‘Smart classroom preferences and
mode under small private online course based on machine learning,’’ Social
information literacy among college students,’’ Australas. J. Educ. Technol.,
Netw. Appl. Sci., vol. 5, no. 2, pp. 1–13, Feb. 2023, doi: 10.1007/s42452-
vol. 38, no. 2, pp. 144–163, Feb. 2022, doi: 10.14742/ajet.7081.
023-05278-y.
[14] Y. Ying, ‘‘Research on college students’ information literacy based on big
[32] E. Rusdiana, Q. Violinda, C. Pramana, R. Y. Purwoko, D. D. Chamidah,
data,’’ Cluster Comput., vol. 22, no. S2, pp. 3463–3470, Mar. 2019, doi:
N. Rahmah, Y. J. Prihanto, F. Y. Hasnawati, R. Susanti, A. Y. Haimah,
10.1007/s10586-018-2193-0.
R. Purwahida, F. Arkiang, M. A. Equatora, and A. Mujib, ‘‘Col-
[15] Z. Mian, Y. Bai, and R. K. Ur, ‘‘Research on college computer-computing lege students’ perception of electronic learning during COVID-19 pan-
and information literacy online course based on MOOC: Taking the north demic in Indonesia: A cross-sectional study,’’ J. Higher Educ. Theory
minzu university as an example,’’ in Proc. IEEE 3rd Int. Conf. Com- Pract., vol. 10, no. 13, pp. 29–44, Oct. 2022, doi: 10.33423/jhetp.v22i13.
put. Sci. Educ. Inf. (CSEI), Jun. 2021, vol. 6, no. 7, pp. 300–306, doi: 5505.
10.1109/CSEI51395.2021.9477751.
[33] X. Liu and C. Yang, ‘‘Analysis of college students’ practical teaching
[16] M. S. Haider and C. Ya, ‘‘Assessment of information literacy skills and effect based on machine learning correlation analysis algorithm: Take the
information-seeking behavior of medical students in the age of technology: software technology course as an example,’’ in Proc. 8th Int. Conf. Educ.,
A study of Pakistan,’’ Inf. Discovery Del., vol. 49, no. 1, pp. 84–94, Manage., Inf. Manage. Soc. (EMIM), 2018, vol. 8, no. 1, pp. 782–788, doi:
Feb. 2021, doi: 10.1108/IDD-07-2020-0083. 10.2991/emim-18.2018.159.
[17] X. Ouyang, Y. Xiao, and J. Zhong, ‘‘Research on the influencing factors [34] H. Akram, A. H. Abdelrady, A. S. Al-Adwan, and M. Ramzan, ‘‘Teachers’
and the promotion measures of college students’ information literacy,’’ in perceptions of technology integration in teaching-learning practices: A
Proc. 8th Int. Conf. Inf. Technol. Med. Educ. (ITME), Dec. 2016, vol. 12, systematic review,’’ Frontiers Psychol., vol. 13, Jun. 2022, Art. no. 920317,
no. 12, pp. 728–732, doi: 10.1109/ITME.2016.0169. doi: 10.3389/fpsyg.2022.920317.
[18] T. Nishikawa and G. Izuta, ‘‘The information technology literacy level of [35] M. Hussain, W. Zhu, W. Zhang, and S. M. R. Abidi, ‘‘student engage-
newly enrolled female college students in Japan,’’ Hum. Social Sci. Rev., ment predictions in an e-learning system and their impact on student
vol. 7, no. 1, pp. 1–10, Mar. 2019, doi: 10.18510/hssr.2019.711. course assessment scores,’’ Comput. Intell. Neurosci., vol. 2018, Oct. 2018,
[19] R. Diani, A. Susanti, N. Lestari, M. Saputri, and D. Fujiani, ‘‘The influence Art. no. 6347186, doi: 10.1155/2018/6347186.
of connecting, organizing, reflecting, and extending (CORE) learning [36] V. G. Karthikeyan, P. Thangaraj, and S. Karthik, ‘‘Towards developing
model toward metacognitive abilities viewed from students’ information hybrid educational data mining model (HEDM) for efficient and
literacy in physics learning,’’ J. Phys., Conf. Ser., vol. 1796, no. 1, pp. 1–9, accurate student performance evaluation,’’ Soft Comput., vol. 24,
Feb. 2021, doi: 10.1088/1742-6596/1796/1/012073. no. 24, pp. 18477–18487, Dec. 2020, doi: 10.1007/s00500-020-
[20] F. Liu and Q. Zhang, ‘‘A new reciprocal teaching approach for infor- 05075-4.
mation literacy education under the background of big data,’’ Int. J. [37] H. Hang, D. Shuang, L. Jiarou, and K. Zhonglin, ‘‘Construction of learning
Emerg. Technol. Learn, vol. 16, no. 3, pp. 246–260, Feb. 2021, doi: performance prediction model: From big data analysis of learning behav-
10.3991/ijet.v16i03.20459. ior,’’ China’s Distance Educ., vol. 41, no. 4, pp. 8–20, May 2021, doi:
[21] M. K. Masko, K. Thormodson, and K. Borysewicz, ‘‘Using case-based 10.13541/j.cnki.chinade.2021.04.002.
learning to teach information literacy and critical thinking skills in under- [38] M. Zhijia and W. Fati, ‘‘MOOC learning results prediction index explo-
graduate music therapy education: A cohort study,’’ Music Therapy Per- ration and learning group characteristics analysis,’’ Res. Mod. Distance
spect., vol. 38, no. 2, pp. 143–149, Oct. 2020, doi: 10.1093/mtp/miz025. Educ., vol. 147, no. 3, pp. 58–66, Jun. 2017, doi: 10.3969/j.issn.1009-
[22] R. Xu, C. Wang, Y. Hsu, and X. Wang, ‘‘Research on the influence of DNN- 5195.2017.03.008.
based cross-media data analysis on college students’ new media literacy,’’ [39] Rapidminer Official Website. Accessed: Feb. 1, 2023. [Online]. Available:
Comput. Intell. Neurosci., vol. 2022, Aug. 2022, Art. no. 9224834, doi: https://rapidminer.com/why-rapidminer/
10.1155/2022/9224834. [40] Q. Dongming, Z. Yumeng, L. Baige, and C. Zhiyun, ‘‘Research on the eval-
[23] M. Aydın, ‘‘A multilevel modeling approach to investigating factors uation criteria of college students’ information literacy—Taking Shanghai
impacting computer and information literacy: ICILS Korea and Finland as an example,’’ China Higher Educ. Res., vol. 349, no. 9, pp. 53–59,
sample,’’ Educ. Inf. Technol., vol. 27, no. 2, pp. 1675–1703, Aug. 2021, Oct. 2022, doi: 10.16298/j.cnki.1004-3667.2022.09.10.
doi: 10.1007/s10639-021-10690-1. [41] Y. Shi, F. Peng, and F. Sun, ‘‘A blended learning model based
[24] Y. Sun, Z. Tan, Z. Li, and S. Long, ‘‘Predicting and analyzing college stu- on smart learning environment to improve college students’ infor-
dents’ performance based on multifaceted data using machine learning,’’ mation literacy,’’ IEEE Access, vol. 10, pp. 89485–89498, 2022, doi:
in Proc. 4th Int. Conf. Adv. Comput. Technol., Inf. Sci. Commun. (CTISC), 10.1109/ACCESS.2022.3201105.
Apr. 2022, pp. 1–6, doi: 10.1109/CTISC54888.2022.9849815. [42] S. Fang, B. Tang, Z. Fu, C. Luo, and J. Hu, ‘‘A study of the fac-
[25] J. Guo and T. Xu, ‘‘IT monitoring and management of learning quality of tors influencing Pearson’s correlation coefficient in the development of
online courses for college students based on machine learning,’’ Mobile higher education—Taking the Yangtze River delta as an example,’’ Adv.
Inf. Syst., vol. 2022, Sep. 2022, Art. no. 5501322, doi: 10.1155/2022/ Social Sci., vol. 12, no. 01, pp. 49–58, 2023, doi: 10.12677/ASS.2023.
5501322. 121007.

50460 VOLUME 11, 2023

Y. Shi et al.: Analysis of Learning Behavior Characteristics and Prediction of Learning Effect

[43] L. Zhuozhuo, M. Miaoer, and Z. Kang, ‘‘Cultivating critical thinking YONG SHI was born in 1984. He received the
in information literacy education—Investigation and experience of for- master’s degree from Anhui Normal University.
eign university libraries,’’ Inf. Data Work, vol. 43, no. 4, pp. 103–112, He is currently pursuing the Ph.D. degree in infor-
Sep. 2022, doi: 10.12154/j.qbzlgz.2022.04.011. mation technology with St. Paul University Philip-
[44] R. Yacouby and D. Axman, ‘‘Probabilistic extension of precision, recall, pines. He is a Teacher with the School of Computer
and F1 score for more thorough evaluation of classification models,’’ Proc. Science, Huainan Normal University, a Certified
1st Workshop Eval. Comparison NLP Syst., Nov. 2020, vol. 11, no. 1, MOOC Teacher with the China School Online,
pp. 79–91, doi: 10.18653/v1/2020.eval4nlp-1.9.
an Innovation and Entrepreneurship Instructor for
[45] A. Weipeng and S. Jiaze, ‘‘Improvement and analysis of decision tree
college students in Anhui, and an advanced indi-
C4.5 algorithm,’’ Comput. Eng. Appl., vol. 55, no. 12, pp. 169–173,
vidual with the Anhui Education and Research
Jun. 2019, doi: 10.3778/j.issn.1002-8331.1805-0482.
[46] Y. Liyuan, ‘‘The application of decision tree algorithm in college perfor- Network. He has five projects with the national and provincial level, guided
mance analysis,’’ J. Weifang Univ., vol. 8, no. 4, pp. 57–58, Dec. 2008, doi: college students to participate in various competitions and won more than
10.3969/j.issn.1671-4288. 30 awards at the national provincial level. He has published more than
[47] Z. Yu, H. Chen, J. Liu, J. You, H. Leung, and G. Han, ‘‘Hybrid k-nearest 20 articles, and has a number of soft works and patents. His research interests
neighbor classifier,’’ IEEE Trans. Cybern., vol. 46, no. 6, pp. 1263–1275, include information education and data mining.
Jun. 2015, doi: 10.1109/TCYB.2015.2443857.
[48] M. S. Roobini, K. Babu, J. Joseph, and G. Ieshwarya, ‘‘Predicting
FANG SUN was born in October 1978.
stock price using data science technique,’’ in Proc. 2nd Int. Conf. Artif.
He received the master’s degree in educational
Intell. Smart Energy (ICAIS), no. 2, Feb. 2022, pp. 1013–1020, doi:
10.1109/ICAIS53314.2022.9742772. technology from Beijing Normal University. He is
[49] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and currently an Associate Professor with the School
R. Salakhutdinov, ‘‘Dropout: A simple way to prevent neural networks of Education, Huainan Normal University. He is
from overfitting,’’ J. Mach. Learn. Res., vol. 15, pp. 1929–1958, Jun. 2014, the author of The Essence of Flash Course-
doi: 10.5555/2627435.2670313. ware Making Technology and other books and
[50] C. Zhan, Y. Zheng, H. Zhang, and Q. Wen, ‘‘Random-forest-bagging founded the influential educational technology
broad learning system with applications for COVID-19 pandemic,’’ IEEE Forum (www.etthink.com) and other resources
Internet Things J., vol. 8, no. 21, pp. 15906–15918, Nov. 2021, doi: (www.qiexing.com). His research interests include
10.1109/JIOT.2021.3066575. information teaching, application of educational technology, development of
[51] H.-J. Park, S. S. Kim, S.-Y. Lee, N.-H. Park, E.-C. Chung, M.-H. Rho, digital teaching resources, modern educational technology, and multimedia
H.-J. Kwon, and S.-H. Kook, ‘‘A practical MRI grading system for courseware making.
cervical foraminal stenosis based on oblique sagittal images,’’ Brit.
J. Radiol., vol. 86, no. 1025, May 2013, Art. no. 20120515, doi:
10.1259/bjr.20120515. HONGKUN ZUO was born in 1976. He received
[52] L. Sannvya, L. Shengyingjie, and S. Jianwen, ‘‘Several key issues in the the master’s degree from Anhui University. He is
development of intelligent education,’’ China’s Distance Educ., vol. 555, currently pursuing the Ph.D. degree with the Nan-
no. 4, pp. 1–7, May 2021, doi: 10.13541/j.cnki.chinade.2021.04.001. jing University of Aeronautics and Astronautics.
[53] S. Jinfeng, ‘‘Critical thinking education and enlightenment at Stanford He is an Associate Professor with the School of
University,’’ Higher Educ. Forum, vol. 259, no. 5, pp. 122–124, Jul. 2021, Finance and Mathematics, Huainan Normal Uni-
doi: 10.3969/j.issn.1671-9719. versity. His research interests include teaching the-
[54] Z. Changhai, ‘‘Research on the information literacy education model ory and methodology of mathematical modeling,
of Chinese college students based on critical thinking and creativ- as well as financial quantitative investment anal-
ity,’’ J. China Library, vol. 42, no. 4, pp. 102–116, Apr. 2016, doi: ysis and financial engineering practice teaching,
10.13530/j.cnki.jlis.164008. and numerical solutions of nonlinear dynamic systems and differential equa-
[55] H. Hongmei, ‘‘Research on the optimization of information literacy train-
tions. He has guided students to win one second prize in the National
ing mode in colleges and universities under the environment of ‘innova-
Undergraduate Mathematical Modeling Competition, three second prizes in
tion,’’’ Library Construct., vol. 296, no. 2, pp. 142–147, Feb. 2019, doi:
10.19764/j.cnki.tsgjs.20182378.
the competition area, and two third prizes in the competition area.
[56] J. Haixia, ‘‘Research on the construction of information literacy education
system based on ‘internet plus ‘thinking,’’’ Mod. Inf., vol. 37, no. 2, p. 5, FEI PENG was born in 1976. He received the
Mar. 2017, doi: 10.3969/j.issn.1008-0821. master’s degree from East China Normal Uni-
[57] W. Juan, Z. Dongwei, L. Falun, Z. Xianglong, and X. Wei, ‘‘Anal- versity. He is currently an Associate Professor
ysis of academic behavior and performance prediction of online and the Vice President of the Computer School,
learners in open education based on data mining,’’ China Adult Huainan Normal University. He has published
Educ., vol. 13, no. 3, pp. 29–33, Sep. 2022, doi: 10.3969/j.issn.
many articles. He has been outstanding in instruct-
1004-6577.
ing college students in information technology
[58] S. Faqin and F. Rui, ‘‘Research on the influencing factors of online
academic achievement based on learning analysis,’’ China’s Audio-
skills competitions. He presided over a number of
Vis. Educ., vol. 24, no. 3, pp. 48–54, Apr. 2019, doi: 10.3969/j.issn. provincial and ministerial teaching and scientific
1006-9860. research projects, and his courses include graphic
[59] Z. Xuebo and L. Shubing, ‘‘Data-driven differentiated teaching and image processing, web design, and UI design. His research interest
decision-making: Crux, logic and mechanism,’’ Res. Mod. Distance includes computer educational applications. He won the Teaching Achieve-
Educ., vol. 34, no. 3, pp. 48–57, May 2022, doi: 10.3969/j.issn. ment Award of Anhui Province for many times.
1009-5195.

VOLUME 11, 2023 50461