Volume 6, Issue 7, July – 2021 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165

Lecturer Research Performance Model Evaluation

using Machine Learning Approach
Ahmad Sanmorino
Faculty of Computer Science
Universitas Indo Global Mandiri
Palembang, Indonesia

Abstract:- Through this study, the author evaluates the

lecturer's research performance model built on previous
research. This model consists of seven independent
variables and one dependent variable. The seven
independent variables that construct the model are
Scientific Article, H-Score, College Type, Journal Cluster,
Research Grant, Research Collaboration, Research
Interest, while the dependent variable is research
performance. Based on the results of the evaluation using
the machine learning approach, a good accuracy score was
obtained for each classifier, for Random Forest at 93
percent, Multi-layer Perceptron at 90 percent, Decision
Tree at 97 percent, and Linear Discriminant Analysis at 93 Fig. 1. Conceptual Model
percent. The results of this evaluation show that the
proposed research performance model of the lecturer This model consists of seven independent variables and
meets the author's expectations and is relevant to the one dependent variable (See Fig. 1). The author has not
conditions of higher learning institutions. conducted feature selection and evaluation of the proposed
model. In this study, only evaluation of the model is carried
Keywords:- Research Performance Model; Lecturer; out, for feature selection will be carried out in other studies.
Evaluation; Machine Learning.
Other related studies published by Wichian et al. [3],
I. INTRODUCTION investigated the factors that influence research productivity in
public universities. Research Management, Research Funding,
This study is a continuation of the previous related study. Communication, Networking-Teamwork, Age, Academic
In the previous study, the author has built a lecturer research Position, Thinking, Research Mind, Volition-Control,
performance model. In this study, an evaluation of the proposed International Meeting, Research Skill-Techniques, Institutional
model was carried out. Evaluation of the model involves four Policy, and Library Expenditure are the variables used in this
machine learning classifiers. The model building is expected to study. The empirical data support the research productivity
be a reference for research managers in assessing and model (Chi-Square at 80.007). The Chi-Square function is used
improving the research performance of lecturers at higher to calculate the degree of relationship between variables [4].
learning institutions. The research performance which is judged Back Propagation Neural Networks are used by the authors to
from the quality and quantity of research output is still not analyze the factors that influence research performance [5].
optimal, with the model that the author proposes to be an
alternative solution for the problem. III. METHODOLOGY
II. RELATED STUDY The steps that the author uses as a guide in conducting
this study are shown in research design in Fig. 2:
In previous study [1][2], the author has built a lecturer
research performance model shown in Fig. 1:

IJISRT21JUL1244 www.ijisrt.com 1167

Volume 6, Issue 7, July – 2021 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
LDA 23 % 0% 73.67 % 3.33 %

Confusion-Matrix using Random Forest classifier, the

number of correctly identified lecturers who did not meet the
research performance target was 63.33 percent. 6.67 percent of
lecturers were incorrectly identified as not meeting the research
performance target. 30 percent of lecturers are correctly
identified that they met the research performance target. 0
percent of lecturers were incorrectly identified as meeting the
research performance target.

Confusion-Matrix using a Multi-layer Perceptron

classifier, the number of correctly identified lecturers who did
not meet the research performance target was 70 percent. 0
percent of lecturers were incorrectly identified as not meeting
the research performance target. 20 percent of lecturers are
correctly identified that they met the research performance
target. 10 percent of lecturers were incorrectly identified as
meeting the research performance target.

Confusion-Matrix using Decision Tree classifier, the

number of correctly identified lecturers who did not meet the
research performance target was 70 percent. 3.33 percent of
lecturers were incorrectly identified as not meeting the research
performance target. 26.67 percent of lecturers are correctly
Fig. 2. Research Design
identified that they met the research performance target. 0
percent of lecturers were incorrectly identified as meeting the
This study begins with study literature publications research performance target.
related to lecturer research performance, followed by the
definition of variables that are used as the proposed construct Confusion-Matrix using Linear Discriminant Analysis
model. Then collect data based on the variables that have been classifier, the number of correctly identified lecturers who did
determined, followed by the development of a lecturer research
not meet the research performance target was 73.67 percent.
performance model. Determination of variables, data
3.33 percent of lecturers were incorrectly identified as not
collection, and model development has been carried out by the
meeting the research performance target. 23 percent of lecturers
authors in previous research [1][2], so that this study enters the
are correctly identified that they met the research performance
next stage, the stage of evaluating the model that has been built. target. 0 percent of lecturers were incorrectly identified as
This evaluation phase involves four machine learning
meeting the research performance target.
classifiers [6][7], Random Forest (RF) [8], Multi-layer
Perceptron (MLP) [9][10], Decision Tree (DT) [11], and Linear The results of the evaluation using the Random Forest
Discriminant Analysis (LDA) [12]. The classification results (RF) method are shown in TABLE II:
are evaluated using several mechanisms, confusion-matrix
[13][14], accuracy [15], precision, recall [16][17], f1-score TABLE II. RANDOM FOREST E VALUTION REPORT
[18], and AUC (area under the curve)[19][20]. In the last
Precision Recall f1-Score
section, the author compares the results of the evaluation of
0 0.83 1.00 0.90
each classifier, to find out whether the proposed model is
relevant for use in higher learning institutions or vice versa. 1 1.00 0.90 0.95
Accuracy 0.93
IV. RESULT AND DISCUSSION Macro avg 0.91 0.95 0.93
Weighted 0.95 0.93 0.94
The evaluation results of the proposed model consist of avg
confusion-matrix, accuracy score, precision, recall, f1-measure,
misclassification rate, and others (See TABLE I – TABLE V). Random Forest (RF) produced 83 percent of lecturers
The comparison of True Positive, False Positive, True who did not meet the research performance target of all
Negative, and False Negative scores are shown in TABLE I: lecturers who were predicted to fail. Of all lecturers predicted
to meet the research performance target, RF produces 100
TABLE I. CONFUSION MATRIX REPORT percent who can actually meet it. In comparison to all lecturers
Classifier True False True False who do not meet the research performance target, RF produces
Positive Positive Negative Negative 100 percent of lecturers who are predicted not to meet the
RF 30 % 0% 63.33 % 6.67 % research performance target. In comparison to all lecturers who
MLP 20 % 10 % 70 % 0% actually meet the research performance target, RF produces 90
percent of those who are predicted to meet it. RF generates a
DT 26.67 % 0% 70 % 3.33 %
comparison of average precision and recall for lecturers who do

IJISRT21JUL1244 www.ijisrt.com 1168

Volume 6, Issue 7, July – 2021 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
not meet the 90 percent research performance target. RF generates a comparison of the average precision and recall for
generates a comparison of the average precision and recall for lecturers who meet the 98 percent research performance target
lecturers who meet the 95 percent research performance target (f-measure). DT produced 97 percent of lecturers who were
(f-measure). RF produced 93 percent of lecturers who were correctly predicted to meet the research performance target but
correctly predicted to meet the research performance target but did not meet the overall lecturers' research performance target
did not meet the overall lecturers' research performance target (accuracy). Furthermore, the evaluation using the Linear
(accuracy). Furthermore, the evaluation using the Multi-layer Discriminant Analysis (LDA) method is shown in TABLE V.
Perceptron (MLP) method is shown in TABLE III:
Linear Discriminant Analysis (LDA) produced 78 percent
TABLE III. MULTI-LAYER PERCEPTRON E VALUTION REPORT of lecturers who did not meet the research performance target
Precision Recall f1-Score of all lecturers who were predicted to fail. Of all lecturers
0 1.00 0.67 0.80 predicted to meet the research performance target, LDA
1 0.88 1.00 0.93 produces 100 percent who can actually meet it. In comparison
Accuracy 0.90 to all lecturers who do not meet the research performance
Macro avg 0.94 0.83 0.87 target, LDA produces 100 percent of lecturers who are
Weighted 0.91 0.90 0.89 predicted not to meet the research performance target.
avg
TABLE V. LINEAR DISCRIMINANT ANALYSIS EVALUTION
Multi-layer Perceptron (MLP) produced 100 percent of REPORT
lecturers who did not meet the research performance target of Precision Recall f1-Score
all lecturers who were predicted to fail. Of all lecturers 0 0.78 1.00 0.88
predicted to meet the research performance target, MLP 1 1.00 0.91 0.95
produces 88 percent who can actually meet it. In comparison to Accuracy 0.93
all lecturers who do not meet the research performance target, Macro avg 0.89 0.96 0.91
MLP produces 67 percent of lecturers who are predicted not to Weighted 0.95 0.93 0.94
meet the research performance target. In comparison to all avg
lecturers who actually meet the research performance target,
MLP produces 100 percent of those who are predicted to meet In comparison to all lecturers who actually meet the
it. MLP generates a comparison of average precision and recall research performance target, LDA produces 91 percent of those
for lecturers who do not meet the 80 percent research who are predicted to meet it. LDA generates a comparison of
performance target. MLP generates a comparison of the average precision and recall for lecturers who do not meet the
average precision and recall for lecturers who meet the 93 88 percent research performance target. LDA generates a
percent research performance target (f-measure). MLP comparison of the average precision and recall for lecturers
produced 90 percent of lecturers who were correctly predicted who meet the 95 percent research performance target (f-
to meet the research performance target but did not meet the measure). LDA produced 93 percent of lecturers who were
overall lecturers' research performance target (accuracy). correctly predicted to meet the research performance target but
Furthermore, the evaluation using the Decision Tree (DT) did not meet the overall lecturers' research performance target
method is shown in TABLE IV: (accuracy).

TABLE IV. DECISION TREE EVALUTION REPORT The comparison of accuracy and misclassification rate for
Precision Recall f1-Score each classifier is shown in TABLE VI:
0 0.89 1.00 0.94
1 1.00 0.95 0.98 TABLE VI. THE COMPARISON OF ACCURACY AND
Accuracy 0.97 MISCLASSIFICATION RATE
Macro avg 0.94 0.98 0.96 Classifier Accuracy AUC Misclassification
Weighted 0.97 0.97 0.97 RF 93 % 95 % 7%
avg MLP 90 % 83 % 10 %
DT 97 % 98 % 3%
Decision Tree (DT) produced 89 percent of lecturers who LDA 93 % 96 % 7%
did not meet the research performance target of all lecturers
who were predicted to fail. Of all lecturers predicted to meet The highest accuracy score is Decision Tree for 97
the research performance target, DT produces 100 percent who percent, followed by 93 percent for Random Forest and Linear
can actually meet it. In comparison to all lecturers who do not Discriminant Analysis, the last, Multi-layer Perceptron at 90
meet the research performance target, DT produces 100 percent percent. The goal of this evaluation is not to find the best
of lecturers who are predicted not to meet the research accuracy score, but to see if the variables that comprise the
performance target. In comparison to all lecturers who actually model can pass the test phase with expected results. A good or
meet the research performance target, DT produces 95 percent relevant result is one with an accuracy score of more than 70%.
of those who are predicted to meet it. DT generates a The wider the area under the curve (AUC), the better the
comparison of average precision and recall for lecturers who do qualification results, the AUC of Decision Tree has the highest
not meet the 94 percent research performance target. DT score compared to other classifiers.

IJISRT21JUL1244 www.ijisrt.com 1169

Volume 6, Issue 7, July – 2021 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
V. CONCLUSION [14]. J. Xu, Y. Zhang, and D. Miao, “Three-way confusion
matrix for classification : A measure driven view,” Inf.
The test results of the proposed lecturer research Sci. (Ny)., no. xxxx, 2019, doi:
performance model obtained a high score for each classifier. 10.1016/j.ins.2019.06.064.
Evaluation using confusion-matrix, accuracy, precision, recall, [15]. P. Galdi and R. Tagliaferri, “Data mining: Accuracy and
and f-measure shows good results. The accuracy score for error measures for classification and prediction,” Encycl.
Random Forest is 93 %, Multi-layer Perceptron is 90%, Bioinforma. Comput. Biol. ABC Bioinforma., vol. 1–3,
Decision Tree is 97 %, and Linear Discriminant Analysis is 93 pp. 431–436, 2018, doi: 10.1016/B978-0-12-809633-
%. The results of this evaluation show that the proposed model 8.20474-3.
is relevant to real conditions in higher learning institutions. In [16]. L. Derczynski, “Complementarity , F-score , and NLP
future work, the author will add variables that construct the Evaluation,” pp. 261–266, 2013.
model, and perform testing with different combinations of https://doi.org/10.1016/B978-0-12-809633-8.20351-8.
machine learning classifiers. [17]. D. Berrar, “Performance measures for binary
classification,” Encycl. Bioinforma. Comput. Biol. ABC
REFERENCES Bioinforma., vol. 1–3, no. 1, pp. 546–560, 2018.
[18]. I. Bairamov, “Order Statistics 3rd.pdf,” pp. 281–284,
[1]. A. Sanmorino, Ermatita, Samsuryadi, and D. P. Rini, 2010.
“Building Research Productivity Framework in Higher [19]. R. Tauler and M. Maeder, Multivariate Curve
Education Institution,” Int. Journal of Advanced Resolution—Error in Curve Resolution, Second Edition.,
Computer Science and Applications, vol. 12, no. 6, pp. vol. 2. Elsevier, 2020.
1–9, 2021. [20]. M. Castelli, L. Vanneschi, and Á. R. Largo, “Supervised
[2]. A. Sanmorino, Ermatita, and Samsuryadi, “The learning: Classification,” Encycl. Bioinforma. Comput.
preliminary results of the kms model with additional Biol. ABC Bioinforma., vol. 1–3, no. Ml, pp. 342–349,
elements of gamification to optimize research output in a 2018, doi: 10.1016/B978-0-12-809633-8.20332-4.
higher education institution,” Int. J. Eng. Adv. Technol.,
vol. 8, no. 5, pp. 554–559, 2019.
[3]. S. Na Wichian, S. Wongwanich, and S. Bowarnkitiwong,
“Factors affecting research productivity of faculty
members in government universities: LISREL and
Neural Network analyses,” Kasetsart J. - Soc. Sci., vol.
30, no. 1, pp. 67–78, 2009.
[4]. K. Molugaram and G. S. Rao, Chi-Square Distribution.
2017.
[5]. F. Allegrini and A. C. Olivieri, Chemometrics and
statistics | neural networks, 3rd ed. Elsevier Inc., 2019.
[6]. S. Theodoridis, Classification: a Tour of the Classics.
2020.
[7]. H. Anns, Basic classification concepts 13. 2018.
[8]. A. Urso et al., “Data Mining: Prediction Methods,” pp.
1–18, 2018, doi: 10.1016/B978-0-12-809633-8.20462-7.
[9]. A. Urso, A. Fiannaca, M. La Rosa, V. Ravì, and R.
Rizzo, “Data mining: Classification and prediction,”
Encycl. Bioinforma. Comput. Biol. ABC Bioinforma.,
vol. 1–3, pp. 384–402, 2018, doi: 10.1016/B978-0-12-
809633-8.20461-5.
[10]. M. Guarascio, G. Manco, and E. Ritacco, “Deep
learning,” Encycl. Bioinforma. Comput. Biol. ABC
Bioinforma., vol. 1–3, no. Dl, pp. 634–647, 2018, doi:
10.1016/B978-0-12-809633-8.20352-X.
[11]. M. Fratello, “Decision Trees and Random Forests,” pp.
1–10, 2018, doi: 10.1016/B978-0-12-809633-8.20337-3.
[12]. R. G. Brereton, Chemometrics and Statistics:
Multivariate Classification Techniques. Elsevier Inc.,
2013.
[13]. V. Kotu, and B. Deshpande, “Predictive Analytics and
Data Mining - Chapter3: Data Exploration. Predictive
Analytics and Data Mining,” 2015, doi: 10.1016/B978-0-
12-801460-8.00010-0.

IJISRT21JUL1244 www.ijisrt.com 1170