electronics

Article
A Job Recommendation Model Based on a Two-Layer
Attention Mechanism
Yu Mao 1,2, *, Shaojie Lin 1 and Yuxuan Cheng 1

1 School of Computer Science, Minnan Normal University, Zhangzhou 363000, China; [email protected] (S.L.);
[email protected] (Y.C.)
2 Key Laboratory of Data Science and Intelligence Application, Minnan Normal University,
Zhangzhou 363000, China
* Correspondence: [email protected]

Abstract: In the field of job recruitment, traditional recommendation methods only rely on users’
rating data of positions for information matching. This simple strategy has problems such as low
utilization of multi-source heterogeneous data and difficulty in mining relevant information between
recruiters and applicants. Therefore, this paper proposes a recurrent neural network model based on
a two-layer attention mechanism. The model first improves the entity representation of recruiters and
applicants through user behavior, company-related knowledge and other information. The entities
and their combinations are then mapped to the vector space using one-hot and TransR methods, and
a recurrent neural network with a two-layer attention mechanism is used to obtain their potential
interests from the click sequence, and then a recommendation list is generated. The experimental
results show that this model achieves better results than previous models.

Keywords: attention mechanism; recurrent neural network; job recommendation; heterogeneous data

1. Introduction
In recent years, the number of college graduates has continued to grow, constantly
impacting the talent market. How to improve the efficiency of human resource allocation,
upgrade and optimize the existing industrial structure, and use information data as a new
Citation: Mao, Y.; Lin, S.; Cheng, Y. A driving force for social development has become a problem that needs to be solved in
Job Recommendation Model Based on both academia and industry. At the same time, although many companies spend a lot of
a Two-Layer Attention Mechanism. manpower and material resources on recruitment, they cannot find suitable candidates.
Electronics 2024, 13, 485. https:// Therefore, in the big data environment, there is a huge demand for building an online
doi.org/10.3390/electronics13030485 recruitment analysis system based on the respective characteristics of users and enter-
Academic Editor: Hideaki Iiduka prises [1,2]. However, among the existing online job recruitment platforms at home and
abroad (such as Jobsite, ParisJob, Zhaopin Recruitment and Boss Direct Recruitment), the
Received: 18 December 2023 online recruitment model basically matches the results according to job requirements and
Revised: 13 January 2024
then displays the corresponding job information to users, or uses a statistical method to
Accepted: 19 January 2024
obtain the popularity of the position by recording the frequency of each position being
Published: 24 January 2024
consulted, and finally presents the more popular positions on the user end [3]. Obviously,
for the domestic job recruitment field with 750 million Internet users and 80 million compa-
nies, the existing job search and recruitment platforms use the above-mentioned general
Copyright: © 2024 by the authors.
methods to only provide users and companies with an information integration platform
Licensee MDPI, Basel, Switzerland. and do not really effectively use these data to mine the deeper intentions of users and
This article is an open access article enterprises and the provision of customized services, ultimately wasting the advantages of
distributed under the terms and massive data information brought by the huge user scale.
conditions of the Creative Commons With the advent of the big data era, artificial intelligence is being used in various fields,
Attribution (CC BY) license (https:// such as medical, social and engineering applications and movies [4–10]. In the field of
creativecommons.org/licenses/by/ recruitment, many companies have begun to develop their own precise recommendation
4.0/). platforms [11,12]. For example, Netflix has established a personalized recommendation

Electronics 2024, 13, 485. https://doi.org/10.3390/electronics13030485 https://www.mdpi.com/journal/electronics

Electronics 2024, 13, 485 2 of 12

system based on user behavior data and basic movie information, providing hundreds of
millions of users around the world with their own personalized movie recommendations
and visual materials, thus achieving great commercial success. In addition, large Internet
companies such as Amazon [13], Alibaba [14], Meituan [15] and Boss Direct Pin [16] have
carried out their own research on recommendation prediction algorithms in recent years.
Although these companies have achieved certain results in recruitment recommendation
research, their research content is basically to build personalized recommendation systems
based on users’ text information and behavior sequences, and they have not fully and
systematically utilized heterogeneous data to build recommendation models.
In view of the low utilization rate of massive heterogeneous data in the field of job
search and recruitment, and the difficulty in mining relevant information between recruiters
and applicants, this chapter uses deep neural networks to build a two-way recommendation
system in the field of job recruitment to provide more accurate and personalized information
services to applicants and companies. To this end, this paper proposes a recurrent neural
network model (RNN-TLA) based on a double-layer attention mechanism to intelligently
recommend positions and candidates. First, the model optimizes the entity representation
of recruiters and applicants through information such as subscriptions, retrieval records,
and enterprise-related knowledge graphs. Then, the above entities and their combinations
are mapped to vector space using one-hot and TransR [17] methods, and a recurrent neural
network with a two-layer attention mechanism is used to obtain their potential interests
from the click sequence, and then a recommendation list is generated. The contributions of
this paper mainly include the following two points:
(1) Improve entity information on the job recruitment platform through user behavior
and enterprise-related knowledge graphs, thereby optimizing the modeling results of
applicants and recruiters.
(2) Utilize a recurrent neural network with a two-layer attention mechanism to obtain
their potential intentions from user behavior sequences and enterprise behavior
sequences, and calculate the mutual attention weight between the candidate list,
the target to be recommended, and the upper layer’s attention value to improve
recommendation performance.
The organizational structure of this article is as follows: Section 2 introduces the
current research status in the field of job recommendation; Section 3 introduces the model
and recommendation method proposed in this chapter; Section 4 gives the recommendation
experimental results based on this method and other related model comparison results and
discussion; Section 5 concludes the paper.

2. Related Work
The recommendation system in the automated recruitment platform allows human
resources agents to advertise their job positions on relevant platforms. The purpose of
this recommendation mechanism is to attract the best candidates in a short period of
time. In fact, there are already some similar platforms at home and abroad, such as
Monste, iquesta, jobsite, parisjob, Zhaopin Recruitment and Boss Direct Recruitment.
Currently, retrieving and matching candidates for job roles is still seen as a challenging and
daunting task. To achieve this goal, several recommendation systems have been proposed
in academia and industry. These systems are usually divided into three major categories:
text recommendation systems, collaborative filtering recommendation systems and hybrid
recommendation systems.
Polato et al. [18] proposed a collaborative filtering job search recommendation frame-
work based on multiple predictor variables. The framework can map out multiple pre-
dictors and merge information from multiple texts in a collaborative manner. In order
to better combine this information, the author uses the method of linear combination of
predictor variables to calculate the weight of the predictor variables in the model, thereby
verifying the recommendation performance of the combined collaboration framework.
Liu et al. [19] proposed a system filtering recommendation framework based on college
Electronics 2024, 13, 485 3 of 12

students’ personal data to address the problems of newly graduated college students who
lack work experience and are unfamiliar with the job market. This framework analyzes the
personal situation of students who have obtained job opportunities in the past few years,
mines their user characteristics, and then performs similarity calculations with job-seeking
users to provide a list of potential positions for graduating college students using a collab-
orative filtering algorithm. Almalis et al. [20] aimed at the problem of difficult matching
of job seekers and positions, extended and improved the traditional Minkowski distance
calculation method, and proposed a content-based job search recommendation algorithm.
The algorithm uses a four-dimensional recommendation method to more flexibly quantify
job seekers’ resume information and job introduction information, thereby extracting more
features from these structured data and ultimately improving the effectiveness of job rec-
ommendations. Shivam et al. [21] proposed a content-based recommendation engine that
automatically recommends suitable jobs to job seekers by matching their interests and skills
with job information attributes. During the matching process, the algorithm combines text
filters and feature similarity measures, using the n-gram algorithm and topic models as
elements of the feature vector to calculate similarity, thereby matching the user’s interests,
preferences and skills with positions for recommendations.
In recommendation systems, it is usually difficult to avoid some problems when using
a single traditional recommendation method. Therefore, some scholars have applied the
hybrid recommendation model to the field of job recruitment and achieved better results.
Zhang et al. [22] proposed a job search recommendation method that integrates
content-based and collaborative filtering algorithms. This method believes that the advan-
tage of a collaborative filtering algorithm is that it can find patterns in job search user rating
data and can handle complex objects well. Content-based algorithms can recommend
jobs based on the similarity of the content and associate job descriptions with job seekers’
interests. Therefore, this method takes advantage of both and integrates the two algorithms
to improve the performance of the job recommendation system. Yang et al. [23] proposed
a model-based hybrid recommendation system applied to the field of job recruitment.
This model combines collaborative filtering and content-based methods using statistical
relationship learning methods, learns the features and weighs in project content data, and
builds a hybrid work recommendation model based on statistical relationship learning
methods. This method first learns the personal information of job-seeking users, extracts
the user’s preferences and historically related information, and calculates the probability of
the user matching the position. Then, an enhanced model is built, which contains multiple
relationship regression trees, and cumulative regression values are set at the nodes of each
branch path to capture the matching strength and weakness between the target user and the
candidate position. Finally, the matching values between job-seeking users and candidate
positions are calculated, and then the recommendation results are obtained.
In recent years, with the widespread application of deep learning methods, some
scholars have also used deep learning methods in the field of job recruitment to improve
the performance of recommendation models. Qin et al. [24] proposed an end-to-end
capability-aware human-post fitting neural network model in view of the difficulty in
measuring “personnel fit” in the current recruitment market and the need for review by a
large number of human resource experts. This model uses a large amount of historical job
application information from job seekers to construct a word-level semantic representation
network and designs four hierarchical ability awareness attention strategies to measure
the importance of different job requirements to semantic representation and to measure
the differential contribution of each work experience to specific competency requirements.
Zhu et al. [25] proposed an end-to-end data-driven model based on convolutional neural
networks to match job seekers’ skills with job requirements. The model is based on a
bidirectional neural network that effectively learns the job seeker’s job suitability from
historical job applications to recommend suitable job information to job seekers.
In the job recommendation system, job recruitment activities are a long-term interac-
tion process between job seekers and recruiting units. Therefore, recruitment data often
Electronics 2024, 13, 485 4 of 12

contains a large amount of implicitly related information, such as the potential interests
and hobbies of job seekers, demand characteristics in job information, and click behavior
information of job seekers. At present, some scholars have added time series-based rec-
ommendation methods to their research in the recommendation field. A time series is a
sequence of consecutive observations of the same phenomenon at different time points.
Depending on the observation time, the time in the time series can be in years, quarters,
months or any other time format, and the data change over time. Existing recommendation
models based on time series usually analyze the time series, and their main purpose is to
predict the future based on existing historical data.
Today, the best-performing short-term sequence methods tend to be deep network
models based on RNN. RNN is a type of recursive neural network that takes sequence data
as input, performs recursion in the evolution direction of the sequence, and connects all
nodes in a chain. Different from the traditional neural network model, this network model
not only considers the input at the previous moment but also gives the network a “memory”
function for the previous content; that is, the current output of a sequence is related to
the previous output. Hidasi et al. [26] applied RNN to a session-based recommendation
system for the first time (i.e., session-based recommendation sessions indicate certain
specific time periods), and designed the training, evaluation method and ranking loss
function of RNN for this task. Tan et al. [27] used a recurrent neural network (GRU unit)
to conduct a session-based recommendation system, mainly using two techniques: data
augmentation and input data transformation measurement methods. Hidasi et al. [28]
explored how to add candidate attribute information (such as text and image) into the
RNN framework, and proposed several model frameworks for fusion attributes. Bogina
et al. [29] consider the length of time a user stays on a candidate item in a specific session
as one of the features; that is, the longer the user stays on the item, the higher the level of
interest. Li et al. [30] used two GRU encoders with attention mechanisms to monitor the
macro overall information and micro purpose in user behavior data, and then integrated
the two vector results together. The similarity of the candidate vectors is finally calculated
through the softmax layer to calculate the final score. Quadrana et al. [31] proposed a
hierarchical RNN model to characterize the changes in users’ personal interests during the
session, and ultimately achieve personalized session recommendations for users.
To sum up, in terms of job search recommendation research, most current work focuses
on text information about job seekers and job postings. At the same time, some scholars
have also conducted some analysis on the click sequence of job search users. However,
with massive heterogeneous data as the research target, the relationship between the
various behaviors of job-seeking users in the recruitment platform (user click sequence,
user browsing time, user search records, etc.) and the recruitment unit and recruitment
positions has not been clearly understood. In particular, how to use massive heterogeneous
data for entity modeling analysis, and how to deeply consider the difference in attention of
each element in the behavior sequence to obtain deeper user characteristics for accurate
recommendations is still a problem to be solved. Therefore, this article explores solutions
for building an intelligent recommendation system in the field of job recruitment by using
a recurrent neural network combined with an attention mechanism.

3. Job Recommendation Model Based on Two-Layer Attention Mechanism

3.1. Feature Input
By applying the model to build a real online job recruitment platform, this paper
is able to verify the performance of the proposed model with real data. First of all, for
applicants, when they register on this platform, the platform will ask them to fill in basic
information and upload their resume. This allows corporate recruiters to find talents that
match the company’s business more accurately and quickly. Applicants’ basic information
includes gender, major, expertise, award records, educational background and published
papers. Most of these can be obtained from the user’s basic information, and missing
items can be further extracted through the personal resume. In the process of interacting
Electronics 2024, 13, 485 5 of 12

with the platform, users will generate a series of interactive information, such as login
records, personalized subscription records, retrieval records and click records. In-depth
and extensive mining of this information can better improve user portraits.
From an enterprise perspective, it is crucial for users to click on the job information
content provided by the enterprise, and it is also indispensable for building a comprehen-
sive user portrait and corporate portrait. The company’s position information includes
industry, work nature orientation, working location, household registration information,
work pressure status, work content orientation and salary. The above basic information
is generally provided when companies publish recruitment information, and useful infor-
mation is extracted into the database through entity matching and information extraction
technologies. If no valid information is provided, it is set to the default value.

3.2. Enterprise Related Knowledge Map

The enterprise-related knowledge graph is a huge network of relationships between
related entities such as enterprises, departments, executives, locations and positions. Estab-
lishing a knowledge graph can help to better understand the relationship between positions
and related corporate departments and other entities, and can improve the accuracy of
personalized recommendations to a certain extent.
In the knowledge graph structure of this article, different entities are connected to
each other through correspondence relationships, and similar entities of the same type
are very similar in semantics and real-life situations, for example, “Baidu Search Division”
and “Smart Car Division”. Likewise, the positions under these two departments are closer
together than they are to positions in other departments. There are many methods to vector-
ize the nodes in the knowledge graph [32,33]. According to the experimental performance,
this article uses the TransR [17] method to vectorize each node in the knowledge graph,
which is used to input the neural network model to predict clicks.

3.3. Model Framework

The input of this model is mainly divided into two parts: recruitment positions and
information representation of job-seeking users. Specifically, the recruitment position entity
is represented by its basic information and the result of vectorization of its knowledge
graph. Among them, the basic information of the position includes industry, work nature
orientation, working location, household registration information, work pressure status,
work content orientation and salary. The vectorized representation of the knowledge graph
is implemented using the TransR method. The applicant entity is represented by its basic
information plus one-hot encoding of subscription records and retrieval records. Among
them, the applicant’s basic information includes gender, major, expertise, award records,
educational background and published papers. In the recurrent layer, since Hidasi et al. [26]
proved that GRU is often better than the long short-term memory network (LSTM) [34] in
the specific task of sequence recommendation, this article uses bidirectional GRU [35] as
the basic unit.
The model framework is shown in the Figure 1. The basic principle of this model is
as follows: First, through the input layer and RNN bidirectional loop layer, the implicit
semantics of browsing recruitment position entities are obtained. Second, the first attention
layer is used to obtain the part of the sequence that the user is actually interested in while
weakening the influence of noise to obtain the potential interest representation vector c.
Then, it is combined with the candidate item set of the target to be recommended and
the representation set of the user to be recommended, and the second attention layer is
used to obtain the user’s matching degree to the recruitment position in the candidate item
set. Finally, the output is processed using feedforward layers and softmax layers, and the
results are predicted.
 while weakening the influence of noise to obtain the potential interest representation vec-
tor c. Then, it is combined with the candidate item set of the target to be recommended
and the representation set of the user to be recommended, and the second attention layer
is used to obtain the user’s matching degree to the recruitment position in the candidate
Electronics 2024, 13, 485 item set. Finally, the output is processed using feedforward layers and softmax layers, and
6 of 12
the results are predicted.

Figure1.1.Model
Figure Modelframework.
framework.

In
In order to capture
order to capture the
theuser’s
user’sdifferent
differentpreference
preference levels
levels in in
thethe current
current sequence,
sequence, the
the model first introduces a first-layer attention mechanism, which allows
model first introduces a first-layer attention mechanism, which allows the model the model to
to dy-
dynamically select and linearly combine different parts in the input sequence. The
namically select and linearly combine different parts in the input sequence. The attention attention
vector
vectorisiscalculated
calculatedasasshown
shownin inthe
theFormula
Formula(1).
(1).
t
∑∑j=1 αtj𝛼h jℎ
c =c = (1)
(1)
The factor 𝛼 determines which part of the input sequence should be emphasized or
The factor α determines which part of the input sequence should be emphasized
ignored when making recommendations. ℎ represents the hidden state of the entity after
or ignored when making recommendations. h represents the hidden state of the entity
passing through the loop layer. The size of 𝛼 is affected by the hidden state. More pre-
after passing through the loop layer. The size of α is affected by the hidden state. More
cisely, the weighted distribution coefficient 𝛼 models the degree of match between the
precisely, the weighted distribution coefficient αtj models the degree of match between the
input
input around positionj 𝑗and
aroundposition andthethe output
output at at position
position 𝑡. Function
t. Function 𝑠 specifically
s specifically calculates
calculates the
the similarity between the final hidden state ℎ and the representation
similarity between the final hidden state ht and the representation of the previous of the previous
target
target
item h .item ℎ . Its calculation
Its calculation is shown is in
shown in Formula
Formula (2). (2).
j
𝛼 = 𝑠 ℎ ,
ℎ = 𝑣 𝜎 𝐴 ℎ 𝐴 ℎ (2)
αtj = s ht , h j = v T σ A1 ht + A2 h j


(2)

Here, σ is an activation function (such as the sigmoid function), matrix A1 is used to

transform ht into a latent space, and A2 plays the same role for h j . Matrices A1 , A2 and
vector v are all items to be trained.
After that, the result vector of attention layer 1 and the target to be recommended are
combined into a matrix U ∈ Rd×J . During the training process, the model calculates the
attention of the matrix H ∈ Rd×T composed of this matrix and the candidate list. First, it is
necessary to calculate the similarity matrix S ∈ RT ×J between them, where Stj represents
the relationship between the t-th candidate item and the j-th target item (i.e., the result of
 Electronics 2024, 13, 485 7 of 12

attention layer 1 or the target to be recommended). The similarity is calculated as shown in

Formula (3).

Stj = β H:t , U:j (3)
where β is a scalar function used to measure the similarity between two vectors. This
article uses β(h, u) = w T [h; u; h ◦ u] , where w ∈ R3d is a vector to be trained, the
symbol ◦represents the multiplication of the corresponding positions of the vector, and
[; ] represents the splicing of vectors. Attention layer 2 aims to decide which items in
the candidate set should be enhanced and which items should be ignored. This article
uses at ∈ R J to represent the weight of the t-th position candidate for the target item. In
general, ∑ atj = 1, the attention weight calculation method is at = softmax(St: ); this is
used to calculate the weight vector of each target item U:t′ = ∑ j atj U:j , the matrix U ′ ∈ Rd×T
represents the attention between the candidate item and the target item. Then, the results
are input into the fully connected layer, and finally, the corresponding score of the candidate
can be obtained through the softmax layer.

4. Results
In this section, the performance of the model is experimentally demonstrated and
compared with some baseline models. In addition, this section will also explore the impact
of the attention mechanism on model performance.

4.1. Vectorized Representation of Entities

This experiment uses one-hot encoding, TransR mapping and feature combination to
represent entities. In addition to basic personal information, applicants also add data for
personalized subscription records and retrieval records. This section compares the School
of Computer Science (BIT-CS) and School of Law (BIT-L) of Beijing Institute of Technology
and the School of Law (BNU-L) and the School of Psychology (BNU-P) of Beijing Normal
University. The Figure 2a shows the results of dimensionality reduction using the t-SNE [36].
It can be seen that there is a certain but subtle similarity between students from different
colleges in the same school, and the figure also shows that individual points appear in the
representative areas of other colleges. The reason for this phenomenon is that students
from different colleges have different knowledge and the courses they have studied lead
to different job-hunting intentions. In addition, when comparing different schools in the
same college, it can be found that their regional overlap has increased significantly, which
Electronics 2024, 13, x FOR PEER REVIEW 8 of 13
shows that there are often strong similarities in job search intentions between students in
similar majors.

(a) Example of applicant mapping (b) Example of job entity mapping

Figure 2. Figure
Example of mapping
2. Example of applicants
of mapping and job entities
of applicants and jobinentities
three-dimensional space. space.
in three-dimensional

When visualizing recruitment positions, this section also integrates the results of en-
tities in the knowledge graph represented by the TransR model. Figure 2b compares the
vector representation results of four companies: Baidu, Tencent, CASIC and PwC, show-
ing the vector space mapping relationship between them. As can be seen from the figure,
the areas of Baidu and Tencent overlap significantly, which shows that there is great sim-
ilarity in the positions between them. The regional distance between these two Internet
Electronics 2024, 13, 485 8 of 12

When visualizing recruitment positions, this section also integrates the results of
entities in the knowledge graph represented by the TransR model. Figure 2b compares the
vector representation results of four companies: Baidu, Tencent, CASIC and PwC, showing
the vector space mapping relationship between them. As can be seen from the figure, the
areas of Baidu and Tencent overlap significantly, which shows that there is great similarity
in the positions between them. The regional distance between these two Internet companies
and CASIC and PwC is relatively long, showing the differences between these companies.
It is worth mentioning that there are still very few overlaps between these companies. By
reviewing the recruitment information of the four companies, it can be seen that CASIC and
PwC also put forward the demand for students majoring in computers, electronics, data
mining and other related majors during the recruitment process to support their businesses,
and these are related to those of Baidu and Tencent. There are also certain similarities in
job requirements.

4.2. Model Performance Metrics and Comparison Models

Since the recommendation system proposed in this chapter aims to recommend a list
of potentially interesting items to users, this section uses the following four indicators to
measure the model’s recommendation performance for the top-N recommendation problem.
(1) Recall@N: This article sets Recall as the main indicator, which represents the
probability that the recommended user likes the project. It is defined as the ratio between
the items in the recommended list and all the items that the user actually likes.
(2) MRR@N: Although the single Recall@N indicator effectively measures whether
a real project appears in the recommendation list, it does not consider ranking factors.
Other metrics are therefore needed to measure the performance of the ordering of items
in the list recommended by the recommendation system. Mean reciprocal rank (MRR) is
an important indicator commonly used to measure the effectiveness of evaluation ranking
algorithms. Its core idea is to calculate the average of the reciprocal rankings of desired
items. If the rank is greater than N or the item is not in the recommended list, the reciprocal
rank is set to zero.
(3) NDCG@N: In addition to using the MRR indicator to evaluate the recommendation
list, this section also uses the normalized discount cumulative gain.
Normalized discount cumulative gain (NDCG) is used to measure the quality of the
ranking list. When calculating the cumulative gain, a position attenuation factor needs to
be added.
(4) Novelty@N: The above three evaluation indicators can measure the similarity be-
tween the recommended results and the real state. But in recommendation systems, popular
items usually have higher priority. Therefore, in this section of the experiment, the Novelty
metric is added to indicate the average unpopularity of recommended positions [37].
In addition, this article also adds the results of these models to actual scenario ap-
plications and further measures the quality of the models using the real click-through
rate (CR).
In order to verify the effectiveness of the model method, this paper compares the
proposed RNN-TLA model with four other methods, which are two traditional methods
and two classic recruitment recommendation methods in recent years.
(1) ItemKNN: ItemKNN [38] is a traditional and popular collaborative filtering al-
gorithm. The similarity of items is measured by their frequency of co-occurrence in the
hit list.
(2) FMPC: FPMC [39] is a hybrid model for predicting next-click recommendations,
which contains a common Markov chain and normal matrix factorization model.
(3) BRM: BRM [40] mainly makes recommendations through the matching of bilateral
relationships between recruitment and applicants, plus Bayesian estimation.
(4) EC-DNN: The EC-DNN [41] model first uses Doc2Vec to vectorize the recruitment
text, and then combines a deep recurrent neural network to predict the likelihood of future
clicks in the candidate list based on the user’s click sequence.
Electronics 2024, 13, 485 9 of 12

4.3. Model Comparison Results

In order to evaluate the performance of the RNN-TLA model proposed in this paper,
experiments were conducted to compare the model with four other methods. The data used
in the experiment is the actual online job recruitment platform-“Re Zhao” in 2018. As part
of the recruitment data, the platform comprises a job recruitment system that integrates job
posting, resume delivery, job retrieval, collection and subscription and other functions. In
the experiment, job recruitment information data were first obtained from the platform,
and specific fields related to the enterprise were extracted from this information, such
as work location, salary, skill requirements and click contact behavior, and a knowledge
map related to the enterprise was gradually constructed. Then, the data are preprocessed,
and a candidate item set is generated after target matching, which is then passed into
the two-way recommendation engine together with the user representation, and finally, a
recommendation list is generated.
Figure 3 shows the results of the comparison of performance indicators of experi-
mental methods when N values are 5, 10, 15, 20 and 25 in the experiment for the Top-N
recommendation problem. The dropout rate is set to 0.3 to avoid the overfitting. Table 1
shows the detailed experimental results of performance indicators when N is set to 10. As
can be seen from Table 1, the model RNN-TLA proposed in this chapter achieved the best
results in all four performance indicators. Taking the recall rate as an example, compared
with the two traditional models ItemKNN and FMPC, the model RNN-TLA increased
by 20.44% and 18.3%, respectively. The reason is that these two traditional models only
calculate the similarity of positions by the number of job seekers clicking on common posi-
tions. In addition, although the models build an overall portrait of users to a certain extent,
they do not consider their short-term behavioral factors. Judging from the experimental
results of recall rate, the BRM model has the worst effect. The reason may be that the model
Electronics 2024, 13, x FOR PEER REVIEW
only considers the matching degree of bilateral conditions and does not conduct a 10 of 13
deeper
exploration of other features.

(a) Recall (b) MRR

(c) NDCG (d) Novelty

Figure 3.
Figure 3. Model performance comparison.
Model performance comparison.

Table 1. Model performance comparison.

Recall@10 MRR@10 NDCG@10 Novelty@10

ItemKNN 0.2321 0.1238 0.1691 0.1792
FMPC 0.2535 0.1498 0.1823 0.2165
BRM 0.2241 0.1359 0.1528 0.2062
EC-DNN 0.4043 0.2354 0.3154 0.3223
Electronics 2024, 13, 485 10 of 12

Table 1. Model performance comparison.

Recall@10 MRR@10 NDCG@10 Novelty@10

ItemKNN 0.2321 0.1238 0.1691 0.1792
FMPC 0.2535 0.1498 0.1823 0.2165
BRM 0.2241 0.1359 0.1528 0.2062
EC-DNN 0.4043 0.2354 0.3154 0.3223
RNN-TLA 0.4365 0.2807 0.3771 0.4301

As can be seen from the overall experimental results in Figure 3, the models RNN-TLA,
EC-DNN and IPJF-JT are significantly better than the other three models in terms of overall
performance. The reason is that these three models all add the click behavior sequence of
job seekers and combine it with a deep recurrent neural network to predict the candidate
list. Compared with the methods used by the other three models, these methods can mine
deeper user characteristics and achieve better results.
In addition, although model EC-DNN and model IPJF-JT also achieved good results
in the data set, their recall rates are still 3.22% and 4.21% lower than the models proposed
in this paper. The model RNN-TLA proposed in this paper represents entities based on
knowledge graphs for both job-seeking users and recruiters, and combines a two-layer
attention mechanism to obtain potential interests. Therefore, RNN-TLA is not only more
comprehensive and detailed than these two models in terms of the overall portrait of the
user but also more in-depth in mining the characteristics of users and positions.
In addition to the recall rate, the experimental results also include the results of the
indicators MRR and NDCG. As can be seen from the results, the results for these two
metrics are similar to the results for recall, which indicates that the comparison results of
each model will not change much even if the ranking factor is added to the recommendation
metric. In addition, it can be seen from the results of the Novelty indicator that the effect
of the model RNN-TLA is significantly better than other recommendation models. This
shows that the RNN-TLA model can recommend more unpopular positions to job seekers.
A phenomenon can also be seen from the figure. As the number of recommended positions
increases, the results of all indicators show a downward trend. This shows that within a
certain period of time, the number of positions that job seekers are really interested in will
not continue to increase.

5. Conclusions
This paper conducts data mining on the current online recruitment system and pro-
poses a recurrent neural network model with a double-layer attention mechanism to
provide applicants and recruiters with a better recommendation service experience. First,
applicants’ retrieval and subscription records are collected and an enterprise-related knowl-
edge graph is constructed. Second, the TransR method is used to obtain the vector rep-
resentation of job entities, thereby integrating the basic information and the additional
information to obtain a complete entity modeling result. Finally, they are trained and pre-
dicted by inputting them into a specific recurrent neural network with a two-layer attention
mechanism. The experimental results show that the model proposed in this article has
good results. The model has achieved better results than previous models in various perfor-
mance indicators of the recommendation system. Future work will include the following
directions: First, the company-related fields extracted from company information are not
rich enough, and it is difficult to obtain some industry-related fields using a small sample
size. Therefore, learning with a small number of samples should be further investigated.
Second, the experimental data are based on user data from previous years. In the future, we
can try to deploy the model on a real-time online recruitment platform. Finally, in the field
of job recommendation, the future trend should be to utilize multi-source heterogeneous
data in online recruitment platforms for high-quality job recommendations.
Electronics 2024, 13, 485 11 of 12

Author Contributions: Conceptualization, Y.M.; methodology, Y.M.; software, Y.M.; validation, Y.M.
and S.L.; formal analysis, Y.M.; investigation, S.L. and Y.C.; resources, Y.M.; writing—original draft
preparation, Y.M.; writing—review and editing, Y.C.; visualization, Y.M. All authors have read and
agreed to the published version of the manuscript.
Funding: This research was funded by the Natural Science Foundation of Fujian Province (No. 2022J01914).
Data Availability Statement: The datasets generated and analyzed during the current study are
available from the corresponding author upon reasonable request.
Conflicts of Interest: The authors declare no conflicts of interest.

References
1. Bersini, H. The immune recruitment mechanism: A selective evolutionary strategy. In Proceedings of the 4th International
Conference on Genetic Algorithms, San Diego, CA, USA, 13–16 July 1991; pp. 520–526.
2. Reiter-Palmon, R.; Brown, M.; Sandall, D.L.; Buboltz, C.; Nimps, T. Development of an O* NET web-based job analysis and its
implementation in the US Navy: Lessons learned. Hum. Resour. Manag. Rev. 2006, 16, 294–309.
3. Selden, S.; Orenstein, J. Government E-Recruiting Web Sites: The influence of e-recruitment content and usability on recruiting
and hiring outcomes in US state governments. Int. J. Sel. Assess. 2011, 19, 31–40. [CrossRef]
4. Baek, K.; Shin, S.; So, H. Decoupling thermal effects in GaN photodetectors for accurate measurement of ultraviolet intensity
using deep neural network. Eng. Appl. Artif. Intell. 2023, 123, 106309. [CrossRef]
5. Abdou, M.A. Literature review: Efficient deep neural networks techniques for medical image analysis. Neural Comput. Appl. 2022,
34, 5791–5812. [CrossRef]
6. Shin, S.; Baek, K.; So, H. Rapid monitoring of indoor air quality for efficient HVAC systems using fully convolutional network
deep learning model. Build. Environ. 2023, 234, 110191. [CrossRef]
7. Bang, J.; Baek, K.; Lim, J.; Han, Y.; So, H. Deep Neural Network Regression-Assisted Pressure Sensor for Decoupling Thermal
Variations at Different Operating Temperatures. Adv. Intell. Syst. 2023, 5, 2300186. [CrossRef]
8. Shin, S.; Kwon, M.; Kim, S.; So, H. Prediction of Equivalence Ratio in Combustion Flame Using Chemiluminescence Emission
and Deep Neural Network. Int. J. Energy Res. 2023, 2023, 3889951. [CrossRef]
9. Feng, X.; Liu, Z.; Wu, W.; Zuo, W. Social recommendation via deep neural network-based multi-task learning. Expert Syst. Appl.
2022, 206, 117755. [CrossRef]
10. Shin, S.; Ko, B.; So, H. Noncontact thermal mapping method based on local temperature data using deep neural network
regression. Int. J. Heat Mass Transf. 2022, 183, 122236. [CrossRef]
11. Furlan, J.C.; Singh, J.; Hsieh, J.; Fehlings, M.G. Methodology of systematic reviews and recommendations. J. Neurotrauma 2011, 28,
1335–1339. [CrossRef]
12. Hendrick, D.; Lanphear, D.; Mahfoud, R.; Megraw, R. Recommendations System. US Patent 9,922,360, 20 March 2018.
13. Wang, N.; Wang, H.; Jia, Y.; Yin, Y. Explainable recommendation via multi-task learning in opinionated text data. In Proceedings
of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, Ann Arbor, MI, USA,
8–12 July 2018; pp. 165–174.
14. Chen, Q.; Zhao, H.; Li, W.; Huang, P.; Ou, W. Behavior sequence transformer for e-commerce recommendation in alibaba. In
Proceedings of the 1st International Workshop on Deep Learning Practice for High Dimensional Sparse Data, Anchorage, Alaska,
5 August 2019; pp. 1–4.
15. Wang, Z.; Gao, Y.; Chen, H.; Yan, P. Session-based item recommendation with pairwise features. In Proceedings of the Workshop
on ACM Recommender Systems Challenge, Copenhagen, Denmark, 20 September 2019; pp. 11–15.
16. Yan, R.; Le, R.; Song, Y.; Zhang, T.; Zhang, X.; Zhao, D. Interview choice reveals your preference on the market: To improve
job-resume matching through profiling memories. In Proceedings of the 25th ACM SIGKDD International Conference on
Knowledge Discovery & Data Mining, Anchorage, AK, USA, 4–8 August 2019; pp. 914–922.
17. Lin, Y.; Liu, Z.; Sun, M.; Liu, Y.; Zhu, X. Learning entity and relation embeddings for knowledge graph completion. In Proceedings
of the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, USA, 25–30 January 2015; pp. 2181–2187.
18. Polato, M.; Aiolli, F. A preliminary study on a recommender system for the job recommendation challenge. In Proceedings of the
Recommender Systems Challenge, Boston, MA, USA, 15 September 2016; pp. 1–4.
19. Liu, R.; Rong, W.; Ouyang, Y.; Xiong, Z. A hierarchical similarity based job recommendation service framework for university
students. Front. Comput. Sci. 2017, 11, 912–992. [CrossRef]
20. Almalis, N.D.; Tsihrintzis, G.A.; Karagiannis, N.; Strati, A.D. FoDRA—A new content-based job recommendation algorithm for
job seeking and recruiting. In Proceedings of the 2015 6th International Conference on Information, Intelligence, Systems and
Applications (IISA), Corfu, Greece, 6–8 July 2015; pp. 1–7.
21. Bansal, S.; Srivastava, A.; Arora, A. Topic modeling driven content based jobs recommendation engine for recruitment industry.
Procedia Comput. Sci. 2017, 122, 865–872. [CrossRef]
22. Zhang, C.; Cheng, X. An ensemble method for job recommender systems. In Proceedings of the Recommender Systems Challenge,
Boston, MA, USA, 15 September 2016; pp. 5–8.
Electronics 2024, 13, 485 12 of 12

23. Yang, S.; Korayem, M.; AlJadda, K.; Grainger, T.; Natarajan, S. Combining content-based and collaborative filtering for job
recommendation system: A cost-sensitive Statistical Relational Learning approach. Knowl. Based Syst. 2017, 136, 37–45. [CrossRef]
24. Qin, C.; Zhu, H.; Xu, T.; Zhu, C.; Jiang, L.; Chen, E.; Xiong, H. Enhancing person-job fit for talent recruitment: An ability-aware
neural network approach. In Proceedings of the 41st International ACM SIGIR Conference on Research & Development in
Information Retrieval, Ann Arbor, MI, USA, 8–12 July 2018; pp. 25–34.
25. Zhu, C.; Zhu, H.; Xiong, H.; Ma, C.; Xie, F.; Ding, P.; Li, P. Person-job fit: Adapting the right talent for the right job with joint
representation learning. ACM Trans. Manag. Inf. Syst. (TMIS) 2018, 9, 1–17. [CrossRef]
26. Hidasi, B.; Karatzoglou, A.; Baltrunas, L.; Tikk, D. Session-based recommendations with recurrent neural networks. arXiv 2015,
arXiv:1511.06939.
27. Tan, Y.K.; Xu, X.; Liu, Y. Improved recurrent neural networks for session-based recommendations. In Proceedings of the 1st
Workshop on Deep Learning for Recommender Systems, Boston, MA, USA, 15 September 2016; pp. 17–22.
28. Hidasi, B.; Quadrana, M.; Karatzoglou, A.; Tikk, D. Parallel recurrent neural network architectures for feature-rich session-based
recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems, Boston, MA, USA, 15–19 September
2016; pp. 241–248.
29. Bogina, V.; Kuflik, T. Incorporating Dwell Time in Session-Based Recommendations with Recurrent Neural Networks. In
Proceedings of the RecTemp@ RecSys, Como, Italy, 27 August 2017; pp. 57–59.
30. Li, J.; Ren, P.; Chen, Z.; Ren, Z.; Lian, T.; Ma, J. Neural attentive session-based recommendation. In Proceedings of the 2017 ACM
on Conference on Information and Knowledge Management, Singapore, 6–10 November 2017; pp. 1419–1428.
31. Quadrana, M.; Karatzoglou, A.; Hidasi, B.; Cremonesi, P. Personalizing session-based recommendations with hierarchical
recurrent neural networks. In Proceedings of the Eleventh ACM Conference on Recommender Systems, Como, Italy, 27–31
August 2017; pp. 130–137.
32. Bruna, J.; Zaremba, W.; Szlam, A.; LeCun, Y. Spectral networks and locally connected networks on graphs. arXiv 2013,
arXiv:1312.6203.
33. Kipf, T.N.; Welling, M. Semi-supervised classification with graph convolutional networks. arXiv 2016, arXiv:1609.02907.
34. Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput. 1997, 9, 1735–1780. [CrossRef]
35. Chung, J.; Gulcehre, C.; Cho, K.; Bengio, Y. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv
2014, arXiv:1412.3555.
36. Van der Maaten, L.; Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res. 2008, 9, 2579–2605.
37. Kaminskas, M.; Bridge, D. Diversity, serendipity, novelty, and coverage: A survey and empirical analysis of beyond-accuracy
objectives in recommender systems. ACM Trans. Interact. Intell. Syst. (TiiS) 2017, 7, 1–42. [CrossRef]
38. Linden, G.; Smith, B.; York, J. Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet Comput. 2003, 7,
76–80. [CrossRef]
39. Rendle, S.; Freudenthaler, C.; Schmidt-Thieme, L. Factorizing personalized markov chains for nextbasket recommendation. In
Proceedings of the 19th International Conference on World Wide Web, Raleigh North, CA, USA, 26–30 April 2010; pp. 811–820.
40. Malinowski, J.; Keim, T.; Wendt, O.; Weitzel, T. Matching people and jobs: A bilateral recommendation approach. In Proceedings
of the 39th Annual Hawaii International Conference on System Sciences (HICSS’06), Kauai, HI, USA, 4–7 January 2006; p. 137c.
41. Benabderrahmane, S.; Mellouli, N.; Lamolle, M. On the predictive analysis of behavioral massive job data using embedded
clustering and deep recurrent neural networks. Knowl.-Based Syst. 2018, 151, 95–113. [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.