1794 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL.

17, 2024

Knowledge-Graph-Driven Mind Mapping for

Immersive Collaborative Learning: A Pilot Study in
Edu-Metaverse
Ye Jia , Xiangzhi Eric Wang , Zackary P. T. Sin , Chen Li , Member, IEEE, Peter H. F. Ng , Xiao Huang ,
George Baciu , Senior Member, IEEE, Jiannong Cao , Fellow, IEEE, and Qing Li , Fellow, IEEE

Abstract—One of the promises of edu-metaverse is its ability to to engage in various learning activities remotely and collabo-
provide a virtual environment that enables us to engage in learning ratively in a digital realm. Aside from being able to transfer
activities that are similar to or on par with reality. The digital en- learners into another place of being [2], a virtual environment is
hancements introduced in a virtual environment contribute to our
increased expectations of novel learning experiences. However, de- also easier to manipulate for a learning environment [3]. It may
spite its promising outcomes, there appears to be limited adoption also have the potential to uplift remote learning to be, perhaps, an
of the edu-metaverse for practical learning at this time. We believe effective substitute for face-to-face learning [4]. Thus, it is worth
this can be attributed to the fact that there is a lack of investigation pursuing edu-metaverse as it can improve the overall learning
into learners’ behavior given a social learning environment. This experience over a multitude of dimensions.
lack of investigation is critical, as without behavioral insight, it
hinders the development of education material and the direction of However, a lingering issue for edu-metaverse is a lack of
an edu-metaverse. Upon completing our work with the pilot user understanding of how learners will behave in a virtual learning
studies, we provide the following insights: 1) compared to Zoom, environment of edu-metaverse. Although a previous study has
a typical video conferencing and remote collaboration platform, explored the differences between conventional video confer-
learners in the edu-metaverse demonstrate heightened involvement encing platforms and edu-metaverse environments in a multi-
in learning activities, particularly when drawing mind mapping
aided by the embedded knowledge graph, and this copresence learner setting [5], the study is focused on investigating learner-
significantly boosts learner engagement and collaborative contri- to-learner conversations in edu-metaverse, which is a small
bution to the learning tasks; and 2) the interaction and learning subset of learner behaviors. An essential question arises: Can
activity design within the edu-metaverse, especially concerning the the edu-metaverse support not only simple conversations but
use of MM. also more complicated interactions involved in various learning
Index Terms—Collaborative learning (CL), constructivism, activities? Addressing this question is crucial, as without such
immersive learning (IL), knowledge graph (KG), mind map, virtual insights, there will be a lack of direction on how to design future
reality (VR), education metaverse. learning materials. In this work, we have conducted a pilot study
aimed at providing general insights, particularly focusing on
collaborative learning (CL) and mind mapping (MM).
I. INTRODUCTION
Our decision to conduct our pilot study on general learner
ETAVERSE has been an important focus in the e-
M learning community in the past few years. So much so
that a new term, edu-metaverse, has been proposed to specify
behaviors in edu-metaverse with collaborative MM is based on
our previous theoretical work on a constructivist edu-metaverse
framework [6]; we called for edu-metaverse learning environ-
a class of metaverse that is reserved for the purpose of educa- ments that enable social learning and the feature to visualize
tion [1]. Its main appeal is that edu-metaverse can enable learners knowledge in deconstructed knowledge graph (KG) form. Thus,
our work on investigating CL and MM may assist in providing
the foundation for an edu-metaverse.
Manuscript received 16 February 2024; revised 5 May 2024; accepted 20
May 2024. Date of publication 28 May 2024; date of current version 25 June Briefly, constructivism is a teaching methodology that aims
2024. This work was supported by the Hong Kong Polytechnic University, to facilitate the learners to internalize and organize knowledge
Strategic Importance Scheme under Grant 1-ZE2N. (Corresponding author: themselves. Our focus areas, CL and MM, are effective tools
Ye Jia.)
This work involved human subjects or animals in its research. Approval for assisting learners in internalizing and organizing knowl-
of all ethical and experimental procedures and protocols was granted by the edge. CL involves multiple learners working together to achieve
Institutional Review Board of The Hong Kong Polytechnic University under a learning goal. It has a constructivist founding because so-
Application No. HSEARS20240207002.
The authors are with the Hong Kong Polytechnic University, Hong Kong cial constructivism believes that knowledge can be constructed
(e-mail: [email protected]; [email protected]; through interaction, correction, and agreement among learners.
[email protected]; [email protected]; peter.nhf@ In addition, both socialization, which facilitates interpersonal
polyu.edu.hk; [email protected]; [email protected];
[email protected]; [email protected]). interactions among learners, and social learning, which involves
Digital Object Identifier 10.1109/TLT.2024.3406638 acquiring knowledge through these interactions, are expected

© 2024 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see
https://creativecommons.org/licenses/by/4.0/
JIA et al.: KNOWLEDGE-GRAPH-DRIVEN MIND MAPPING FOR IMMERSIVE COLLABORATIVE LEARNING 1795

to be key components of the edu-metaverse supporting many 2) A pilot study that shows the generalized potential of using
concurrent learners. Therefore, exploring CL, which inherently edu-metaverse for various learning activities through CL
incorporates both aspects, is a vital area of investigation. and MM is conducted.
Similarly, as a much-used tool for learning, MM also has a The rest of this article is organized as follows. Section II
constructivist root; this is related to the fact that an MM can presents related works. The pedagogical literature to discuss
effectively help us deconstruct and organize knowledge. Since the benefit of CL and MM in the context of edu-metaverse is
MM is a tool that can be applied to various educational domains, revisited in Section III. The technical overview and interactive
constructing it is believed to be an ideal learning task for our details of our system are presented in Section IV. Sections V
general investigation of CL in edu-metaverse. We extend our and VI present user study design. We present the result in
discussion on the details in pedagogical terms in Section III to Section VII and discuss its implication in Section VIII. Section
better lay the theoretical motivation. IX presents limitations and future work. Finally, Section X
In previous work [6], [7], we developed a graph visualization concludes this article.
system called K-Cube, designed for a compact representation of
knowledge concepts. We have since extended this system to a
virtual reality (VR) platform, now referred to as K-Cube VR [6]. II. RELATED WORK
In this work, we further enhance K-Cube VR by integrating This section discusses some previous related work or systems
multilearner and MM functionalities to experiment with CL on VR, CL, MM, and edu-metaverse.
and MM in the edu-metaverse setting. The core of K-Cube VR First, there is a growing amount of work investigating how
is essentially academic yellow pages of connected educational to apply VR for e-learning. Learning with VR is also referred
concepts visualized as an explorable graph. Previously, we have to as immersive learning (IL) [8] due to the immersiveness
shown its success as an immersive environment for course brought forth by VR. These works primarily focus on the use of
understanding [6]. Here, we transform it into an immersive en- extended reality (XR) for educational purposes, with evidence
vironment for a collaborative workspace for drawing 3-D MMs. of its effectiveness in various educational settings. XR has been
Learners are embodied in avatars in the virtual environment, and found to enhance traditional classroom education [9], aid in
they can communicate with one another through speech. The science education [10], [11], [12], raise awareness of social
MM in this variant of K-Cube VR can be drawn through various issues in an engaging manner [13], and simulate natural disaster
features, such as adding, moving, deleting, and stylizing (i.e., protocols for public safety [14]. There is also ongoing research
shape, color, label, etc.). The system also supported a speech- on improving the effectiveness of IL through the use of peda-
to-text and keyboard for typing the keyword for each node on the gogical agents [15]. However, the literature suggests that there
MM. K-Cube VR is embedded with a KG of relevant concepts. has been limited research on learning outcomes, intervention
We also explored using the KG to provide recommendations characteristics, and assessment measures associated with XR in
when adding new nodes on the MM. We believe that this can education [16], [17], [18]. How VR can enhance engagement in
help provide guidance to the learner, particularly in the case learning is also a well-discussed topic [6], [19], [20]. Last, there
when the learner is not familiar with the subject in question. is also an attempt to frame VR systems for educational needs
In order to extract user behavior given the collaborative via a cognitive approach [21].
MM learning task, we have conducted a user study with our CL is a methodology in education. As such, many platforms
edu-metaverse prototype. The learners were asked to use the and systems have been developed throughout recent times.
MM to learn together in designing a game. In order to compare Foremost, there is a survey paper that discusses platforms for
the behaviorial difference from standard streaming platforms, CL [22]. Similarly, Li et al. [5] compare different VR platforms
we have conducted a similar CL MM task with Zoom as well. for CL. There is a work that has explored the benefits of using
After the experiments, we conducted a focus group study and a CAVE system for IL [23]; based on this CAVE system, an
extracted both the conversation and the MMs. Furthermore, we extended work has also proposed design and implementation for
conducted another user study focused on the useability of our a CAVE-based e-learning system [24]. A CL e-learning system
proposed prototype. The result indicates that our prototype can oriented on user perceptions has also been investigated in [25];
better create a sense of copresence and spatial presence among they pinpointed pivotal elements to emphasize the importance
the learners, allowing them to collaborate more effectively and of integrating user perspectives in improving collaborations
be more engaged in the learning activity. This is in contrast to on e-learning platforms. Gamification approaches can also be
the conventional video conferencing and collaboration platform utilized when designing CL, as shown in previous works that
(i.e., Zoom), where learners may find it easier to disengage from aim to teach computational thinking [26] and improve student
the learning activity. engagement [27]. Several studies have explored innovative ap-
To summarize, this article proposed a system for collaborative proaches in the realm of CL enhancements through technology.
MM in edu-metaverse, which can provide insights into learner For instance, Xie et al. [28] utilized a smart conversational
behavior during remote CL. The contributions of our work are agent to facilitate CL within a VR environment. Similarly,
highlighted as follows. Harley et al. [29] introduced a system that provides prompts and
1) A prototype that enables collaborative MM with good feedback to guide participants and enhance learning outcomes
usability is proposed. in CL settings. More recently, VoRtex [30] has been proposed
1796 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

as a gamified metaverse platform for e-learning. KG has been saying, “the fundamental feature is the course of this construc-
investigated in the context of cooperative learning [31]. It has tion: Nothing is given at the start except some limiting points on
been shown that MMs can be used to drive KG construction [32]. which all the rest is based” [47].
Designing an e-learning CL system that can cater to the emo- The key issue, of course, is how this process of knowledge
tional aspect of learners has also been discussed in [33]. construction comes to be within the mental realm of the learner.
CL in virtual environments has been studied before [34], In our previous work [48], we have proposed eight metaverse
[35]; however, by comparison, CL work exclusively on im- learning principles and how they can act as a guideline to drive
mersive VR seems somewhat limited. What is the effect on constructivist learning in edu-metaverse. These principles are
symmetric/asymmetric pair learning has also been investigated rooted in constructivism and are believed to assist in learners’
for VR [36]. However, it is limited to the learning behavior when internal knowledge construction. In this article, we only focus on
the two actors are one teacher and one student only. There are four of the eight principles. Specifically, they are (M1) Present
investigations of CL on XR, in general, [37], while, specifically, Knowledge as Visualized Construct, (M2) Associate the Knowl-
there is a CL work that investigated a 3-D tablet device [38]. edge, (M5) Engage the Student in XR, and (M8) Embody Social
In addition, there is a discussion on how CL can be applied to Learning Avatars. These four principles are particularly related
VR [39]. Overall, the investigation of CL in VR environments to CL and MM. M1 and M2 are related to MM, and M8 is related
seems limited. However, there is a recent work that focuses to CL, while an edu-metaverse learning environment naturally
on comparing the feeling of presence and engagement among drives M5.
conventional video conferencing platforms and VR platforms for In order for a virtual learning environment to facilitate M5,
CL [5], which is similar to ours. However, it should be pointed it needs to be engaging, and a key appeal of VR and, by
out that our work focuses on an interactive task (i.e., MM). extension, edu-metaverse, is that it can bring us a feeling of
Like CL, MM is also a hot topic in education as it can “being there.” Defined by Gibson [49], this feeling is called
visually assist learners in organizing their understanding and presence, “the sense of being in an environment” and it is
thoughts. Some VR systems have been developed for MM. First, induced by immersion, which is generated by technological
VERITAS [40] is a MM VR application, of which the interaction means [50] (e.g., head-mounted display (HMD) and tracked
design has been demonstrated to be effective. The most similar controllers). Presence can be further deconstructed into more
work in the context of VR to ours is MMVR [41], but their focus detailed components, which we will discuss soon.
is limited to the development of a multiuser system and the eval-
uation of its usability. Both VERITAS and MMVR have yet to
dip into collaborative MM and, therefore, do not discuss learner B. Collaborative Learning
behavior in said context. There was, however, an investigation to CL is an educational approach that involves groups of learners
use MMs as an assessment tool [42]. Combining MM with CL working together to solve a problem or complete a learning
has also been attempted to drive the implementation of flipped task. This pedagogical strategy is highly related to the social
classroom [43]. Related to MM, concept mapping has been used constructivist theory, which posits that knowledge is constructed
in a VR setting to drive CL [44], [45]. There is a recent work through social interaction and negotiation [51]. During CL,
that investigated a CL MM reported in [46], but it is focused on learning becomes a shared experience among a group of learners,
using sentiment analysis to measure the outcome. To the best fostering a sense of community and collective responsibility
of our knowledge, there is generally a lack of investigation on for learning [52]. It can encourage active participation, inter-
learner behavior during the completion of a CL MM learning action, and the exchange of ideas, thereby promoting critical
task in edu-metaverse. thinking, the development of problem-solving skills, and the
co-construction of knowledge. It has been shown that CL can
enhance academic achievement [53], improve motivation [54],
III. PEDAGOGICAL DISCUSSIONS
and increase engagement [55].
It is crucial to apply a set of pedagogy to drive the design of The effectiveness of CL is influenced by various factors,
edu-metaverse. Thus, in our previous work, we have proposed including group dynamics, task, and engagement. As such, for
a pedagogical framework that advocates for a constructivist CL to work, one key is ensuring that the learners are engaged and
methodology when designing learning in edu-metaverse [6]. In attentive. Student engagement is widely recognized as crucial
this section, we will discuss constructivism, CL, and MM in for achieving learning outcomes; therefore, interaction among
detail to further highlight the motivation and rationale of our members of a learner group is equally, if not more, vital for
study from the pedagogical perspective. CL [56]. If some students are not engaged in the learning process,
it is easy to see that the learning will collapse to individual
learning. It has been recognized that presence plays a crucial
A. Constructivism in Edu-Metaverse role in ensuring effective collaboration [57]. In addition, a study
It can be said that the core of many educational discussions proved that social presence is associated with the cognitive
comes from constructivism. Its key tenet comes from the idea engagement model [58]. Particularly, in the context of CL in
that learning is not a process of absorbing knowledge passively virtual environments (which is under the umbrella of computer-
but a process of internally constructing knowledge actively. supported CL [59]), it is essential to facilitate and induce the
Crucially, this pillar of pedagogy can be encapsulated by the sense of copresence (i.e., learner feels the presence of other
JIA et al.: KNOWLEDGE-GRAPH-DRIVEN MIND MAPPING FOR IMMERSIVE COLLABORATIVE LEARNING 1797

learners) and cognitive presence (i.e., learner feels the presence another. From the related work and discussion above, however,
of the content) in a virtual world [60]. it seems that, currently, there is a lack of investigation on MM
and CL in the VR realm of edu-metaverse, particularly in the
C. Mind Mapping realms of VR and edu-metaverse. Thus, this article aims to dip
into this research gap and study learner behaviors during CL in
MM is a process of using a visual tool called MMs to structure, edu-metaverse.
organize, and represent information, ideas, or concepts. It is A particular point of investigation lies in the necessity of
characterized by a central node from which branches radiate immersive edu-metaverse learning. As revealed by previous
into more nodes, representing subtopics or related ideas. It has studies, conventional video conferencing and collaboration plat-
been used as a tool for personal note-taking [61]. According forms, such as Zoom, suffer from issues such as poor engage-
to visual literacy [62], the use of colors, symbols, images, and ment [80] and low attentiveness [81], making them not ideal
spatial arrangement in MMs can help learners think visually for CL. Meanwhile, edu-metaverse is well positioned to fill
and thus structurally [63], and this may benefit their knowledge this role. It has already been known that in a conversational
construction. Furthermore, with visual hierarchy and proxim- setting, edu-metaverse can induce a better sense of presence and
ity [64], MM may particularly help with organizing knowledge stronger engagement [5]. Also, VR technology enhances learn-
as it can visualize different levels of abstraction as well. ing by promoting active engagement and practical application,
In the context of education, MM has been considered an providing clear advantages [82]. Thus, the focus of this study
effective tool to help learners. It has been shown to be able is to introduce an interactive learning task in the form of MM
to deliver benefits in many forms. Foremost, as mentioned, it and further investigate the specific learner behaviors that are
is useful for abstraction [65] and can help learners encapsulate associated with the outcomes of this improved immersiveness.
the intricacy of relationships and focus on the overall picture.
MMs have also been known to assist in memorizing learning
IV. K-CUBE VR: VIRTUAL SPACE FOR CL
materials [66]. It has also been shown that MM can foster critical
thinking [67], problem solving [68], and creativity [69] as well. To facilitate KG-driven MM for immersive CL in edu-
Due to all these reasons, MM has been considered a powerful metaverse, we developed a 3-D MM editor operational on HMD
and general tool in education [66] that can be applied to a broad devices. It features a subsystem we call the KG-based recom-
range of domains [70]. mendation system, which assists users in creating new MM
Particularly for our work, MM also serves a unique purpose nodes using straightforward nontraditional strategies for gener-
in mixing with CL, as it has been suggested that MMs can ating recommendations. As an application of edu-metaverse, our
facilitate CL [71]. MM can serve as a shared visual language for system also supports real-time communications, virtual avatars,
learners to exchange ideas, negotiate meanings, and co-construct and collaborative MM. The detailed description of our system
knowledge when used in group activities [69], [72], [73]. is divided into the following subsections.

D. Knowledge Graph A. Overall Design

A KG is a collection of nodes that are connected by edges. The players enter the game room wearing VR helmets with
Its main purpose is to represent the associative relationships a specified task, i.e., brainstorm to conceptualize a video game
between concepts for knowledge reasoning [74]. Many appli- with others and utilize the properties of the nodes to construct a
cation domains have investigated its use, and education is no brief overview of the consolidated proposal for their video game.
exception [75]. Foremost, how to construct an educational KG Starting from the root node named “Game Design,” players
has been demonstrated [76], [77], and there is previous work that extend and expand the related topics stepwise upon it, with the
visualizes educational events with KG and news [78]. With a KG, suggestive guidance of our unique recommendation system. The
relationship analysis has also been done; KnowEdu is a system production of this MM represents the results of the players’
that can not only generate KGs automatically but also provide discussions and collaborations, and every player has the right to
analysis of the relationships of students’ assessment data [79]. access and adjust any nodes in this game room. This leads to two
In general, it seems that most previous studies of KG in main requirements in our system: 1) a VR-based 3-D MM editor
education focus on the construction for the purpose of data supporting multiplayer collaborations; and 2) a recommendation
analysis. It is noted that KG and MMs can both organize in- subsystem that suggests the “next-node” based on the node being
formation visually; however, KG is tailored for computational extended to with the pre-defined KG. Fig. 1 demonstrates our
reasoning [74], whereas MMs facilitate personal knowledge system’s overall interfaces and how the MM appears from a
organization for learners [66]. player’s perspective.

E. Collaborative MM in Edu-Metaverse B. Collaborative MM

CL and MM are both foundational constructivist pedagogical As the most basic feature of our system, the 3-D MM ed-
methodologies that are general and can be applied in a wide array itor is capable of displaying to the player and supporting the
of domains. Furthermore, MM can benefit CL by providing a adjustments by different user inputs. The MM in our system is
medium for learners to communicate or understand with one developed based on the concept, that is, starting from a central
1798 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

Fig. 1. Overview of our KG-driven MM editor in an edu-metaverse setting. (a) Players collaborating on an MM. (b) Root node and interaction ray between the
controller and a target node. (c) Semitransparent dummy nodes for recommendations. (d)–(g) UI features: the text editor, color picker, voice-to-text, and shape
selector. (h) Buttons and triggers bindings on controllers to operate in the system.

node, extending the graph by nodes with color, text, and spatial a node, the player must press and hold for 2 s when hovering
arrangements [62] in MMs to enhance memory recall [66], over the node. The node to be deleted will keep flashing as a
problem solving [68], and stimulate creative thinking [69]. The reminder. The MM is initialized with a root node named “game
MM uses a graph-like setup with nodes and edges. The nodes design” [see Fig. 1(b)], and it is unchangeable on the shape
appear as shapes of cube, sphere, star, or cone with user-specified and cannot be deleted. The root node is much bigger than other
colors and label names. The edges are the lines connecting pairs regular nodes for easy identity, and the construction of the MM
of nodes in a gradient form. The players can freely add, remove, should start with it. Locomotion helps players to view the 3-D
move, or change the properties of the nodes. MM from different perspectives using the left-hand controller.
To select a node, we provide an interaction ray hovering over Players are free to navigate around the scene without moving
the node when using the right-hand controller to navigate in their real bodies for convenience and safety. We adopt grid-based
close proximity to the node. The ray connects the player’s virtual movements that suddenly move the player’s virtual camera by
right hand and the most “potential” node, if any, which depends a fixed distance to avoid dizziness. The controllers with usages
on the virtual hand’s point direction [see Fig. 1(b)]. Through are demonstrated in Fig. 1(h).
hovering and selecting, the player can add new nodes, connect Real-time network communication among different end de-
existing nodes, and accept a node recommendation by dragging vices is vital to facilitate the immersive experience of multiple
the controller in specific directions. To modify other properties players aside from the VR system. The communication elements
of a node, the user opens up a user interface (UI) and operates on in our system consist of MM synchronization, avatar appearance
the interface. The UI provides functionalities including a color and movements, and voice communication. A shared client
picker [see Fig. 1(e)], keyboard [see Fig. 1(d)], voice-based text mode is adopted as the network settings, that is, the server is
input [see Fig. 1(f)], and shape picker [see Fig. 1(g)]. To delete only responsible for forwarding messages and assigning the
JIA et al.: KNOWLEDGE-GRAPH-DRIVEN MIND MAPPING FOR IMMERSIVE COLLABORATIVE LEARNING 1799

This recommendation subsystem differs from traditional rec-

ommendation systems but in a more straightforward way. When
the player adds a new node to the selected node, the subsystem
starts work by taking the selected node’s label name, comparing
it with the keys, and finding the matches by calculating the
Levenshtein distance between them. The values corresponding
to the closet distance key will be picked and returned as the rec-
ommended results. If there are no sufficient values, the values of
Fig. 2. Node control authority management of our system in a timeline chart:
the second closest key will be fetched. Such progress continues
The virtual room allocates the authority in the FIFS manner. The clients send until the recommendation list for the node label is filled with
requests to the room applying for authority and will be rejected if other players six words. The recommendation list is regarded as the potential
take over the node.
word list for the label names of the new node.
The recommendations are displayed as semitransparent
dummy nodes [see Fig. 1(c)], which are evenly scattered around
control privilege upon shared objects in the game. In other words, the selected node and have already connected with it. If one
the server does not maintain the data and objects in the scene, of the dummy nodes fits the player’s expectations, she/he can
but the clients do. A virtual room, which stands as the context drag the interact ray to the expected one, and this dummy node
manager and communication media in the server, will be created will become a real node. When more than one recommendation
when the first player goes online and is destroyed after the fits the player’s expectation, the steps can be repeated to add
last player quits. As long as the virtual room exists, the MM all the expected. The accepted recommendation will not appear
will be synchronized when other players join, and rejoin is also twice. Moreover, to minimize the network traffic, the dummy
possible. In our system, only the control rights of the nodes are node would not be synchronized to other users, only the final
varied over time, in the manner of first-in first-served (FIFS). new node will be scattered over the Internet. In this manner, the
Once the player is done with modifying a node, the authority recommendation is set locally on each end device but efficient
is released and waiting for the next authority request. As for enough to run.
synchronization, the properties of the nodes are immediately
spread to other players if the player has made modifications,
and only the one who has the control authority can perform V. METHODS
the modification. We force the player to keep the control rights A. Experimental Design
during adding, moving, removing, and editing before releasing
This study was meticulously designed to evaluate the efficacy
it to prevent multiple access to a single node. We present the
of our KG-driven recommendation system in the context of CL
control rights management in Fig. 2. The voices of the players are
utilizing MM. To facilitate a comprehensive comparison, we
recorded by the microphone on the VR helmets and transmitted
incorporated three distinct modalities. The first modality com-
to anyone else so that the players can hear others’ sounds
prised the Zoom group, which functioned as the control group.
and talk with each other. The rotation and movements of the
The second modality, referred to as the MM group, consisted of
player’s virtual avatar body and hands are synchronized as well,
participants who utilized the VR MM tool without access to the
which helps the players to feel the appearance of other team
KG-based recommendations. The third modality, the MM+KG
members [see Fig. 1(a)]. The rotation of the MM nodes is always
group, also operated within a VR environment but, unlike the
designed to face the individual player for better readability and
second, was augmented with the KG-driven recommendation
is not synchronized across users. However, all other changes
system.
to the nodes, such as updates or deletions, are synchronized to
The research involved two user studies. The first was a
ensure that everyone in the multiuser environment sees the same
between-subject study design that evaluated the three aforemen-
content.
tioned modalities to gauge the impact of each on the learning pro-
cess. The second user study was supplementary, which adopted
C. KG-Based Recommendation System the same setting from MM+KG, focusing specifically on the
usability of our KG-driven MM recommendation system within
To take advantage of KG and stand as a reference helping a CL framework.
the players add new MM nodes, our unique recommendation The objective of the two studies was to explore how KG
subsystem is designed to provide guidance based on our pre- recommendations influence both the process of CL and the
defined knowledge base and drive MM. The knowledge base is effectiveness of MM.
single-track word mappings stored in a dictionary as key values
ready for querying. The values are semantically the subtopics of
B. Participants
the key, for example, an entry of the dictionary can be {Game
Design: Genre, Target Audience, Game AI,... }. The values For the first user study, all participants were invited from
are possibly to be the key in other entries (e.g., {Genre: FPS, the campus and possess a background in science, technology,
Adventure, Simulation,... }), which implicitly presents the KG engineering and mathematics (STEM) . Initially, 37 participants
with recursive operations. responded to the recruitment, of which seven participants were
1800 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

TABLE I
DEMOGRAPHICS OF THE PARTICIPANTS

Fig. 3. MMs from the three modalities. (a) Zoom group. (b) MM group. (c) MM+KG group.

excluded due to not studying in STEM-related disciplines and six

participants due to time constraints or other private reasons that
prevented them from participating in the experiment. Finally, a
total number of 24 participants attended the experiment. Among
them, some are currently pursuing their postgraduate degrees,
while the rest are undergraduate students. They were randomly
allocated to one of three experimental conditions.
For the second user study, referred to as the usability group,
we recruited 24 participants from a Metaverse technology master
course. These individuals possess expertise in VR and Metaverse
technologies, making them well-suited for assessing the usabil-
ity of our KG-driven MM recommendation system.
Each experimental group consisted of four participants who
engaged in the collaborative tasks. In the comparative user study,
across each modality, there were a total of two groups. As
for the usability group, there are six groups. To enhance the Fig. 4. Overview of the experiment environment. (a) VR groups environment.
experiment’s reliability, participants were asked to record their (b) Zoom groups environment.
game experiences. Comprehensive demographic information
about the participants is detailed in Table I.
Informed consent was obtained from all participants. For were permitted to review the task instructions within VR or
the first user study, participants were compensated with an Zoom meetings as needed.
HK$100 supermarket voucher, while those in the second user The study was conducted within the confines of the PolyU
study received an HK$50 voucher as a token of appreciation library. We booked two types of group rooms for the experiment.
for their time and contribution. The study was approved by The room for the Zoom modality was approximately 2 × 2 m,
the Institutional Review Board of the Hong Kong Polytechnic suitable for single online meetings. In contrast, for the VR
University (Application Number: HSEARS20240207002). modalities, which necessitate bodily movement, a larger space
was provided, measuring 4 × 4 m. The layout of these rooms is
depicted in Fig. 4.
C. Task Materials and Environment As detailed in Section IV, for the VR setting, participants were
The participants were instructed to perform MM for a collab- equipped with a VR-based MM tool capable of constructing
orative task adapted from the source cited in [69], which pertains 3-D MMs. This tool offered comprehensive and collaborative
to team-based brainstorming in the field of game development. MM functions comparable to those in the Zoom setting, such
To ensure the task’s relevance and suitability for the students, it as managing authority, communication, editing, node deletion,
was reviewed and validated by two experts in game development. and modifying properties, among others.
Details of the collaboration task can be found in Appendix A. We Three instances of participants employing the MM for collab-
provided both Chinese and English versions of the task materials orative tasks are showcased in Fig. 3. While the MMs generated
to accommodate participants who may not be familiar with cer- for the CL tasks varied slightly, they all retained the essential
tain terms in English. Throughout the experiment, participants elements of the CL tasks.
JIA et al.: KNOWLEDGE-GRAPH-DRIVEN MIND MAPPING FOR IMMERSIVE COLLABORATIVE LEARNING 1801

TABLE II
RESULT OF THE CONVERSATION FOR CL OF FIRST USER STUDY

VI. MEASURES AND INSTRUMENTS Use” pertains to the participants’ manipulation of the MM tool
during their discussions. Finally, the category “Other” accounts
A. Measures
for instances when participants were distracted or engaged in
1) Group Conversation: The group conversation refers to the activities unrelated to the task at hand.
talking of a group of participants during the CL task that was 2) Usability: The System Usability Scale (SUS) was em-
audio recorded and transcribed for analysis of the engagement. ployed to evaluate our proposed KG-driven MM in CL [83].
To gain a comprehensive understanding of the conversational
dynamics in CL across three modalities, we build upon the
findings of Sun et al. [69]. We adopted a similar approach
B. Procedure
to group interactions in our study, as summarized in Table II.
This table delineates five principal levels of group conversation: The overview of the experiment is illustrated in Fig. 5 . The
Deliberative Thinking, Idea Generation, Social Communication, experiment began with participants being guided by a team of
Technology Use, and Others. four assistants into separate rooms. Initially, they completed a
The term “Deliberative Thinking” denotes the cognitive pro- demographic survey to collect background information. Subse-
cesses underlying how ideas are conceived and developed. quently, they watched an instructional video on MM to ensure
“Social Communication” encapsulates the exchange of infor- that they grasped the fundamental concepts of organizing knowl-
mation within the context of social interactions. “Technology edge using this technique.
1802 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

C. Apparatus
For the Zoom modalities, all four laptops were outfitted
with high-definition cameras and microphones to facilitate clear
communication. The HMDs used in this study were four Meta
Quest 3,1 each verified to support a maximum refresh rate of
120 Hz.

D. Focus Group
We are interested in the user feedback for the CL using MM in
VR. For the two VR modalities’ groups in the first user study and
the usability group, a semistructured interview was conducted
for each group. The interview questions were asked sequentially
in Appendix B.

E. Data Analysis

Fig. 5. Procedure of two user studies.

Coding of group conversation: Group conversations were
recorded and transcribed in mixed with Chinese and English.
To ensure the reliability of the analysis, after the transcript,
we translated all Chinese into English. All conversations were
Each group was then introduced to the MM tools by the segmented into separate turns of students’ talk for the analysis.
assistants. This introduction included a demonstration of a de-
tailed MM, adapted from the result of previous research [69], VII. RESULTS
to provide a clear example of the expected outcome. Once the
participants felt comfortable with the tools, as indicated by their A. Group Conversation
acknowledgment of familiarity, they were presented with the Chi-square tests were conducted to evaluate the presence of
specific tasks of the experiment. They were informed that they significant differences across three modalities in various cate-
could refer back to the instructions at any point during the gories of interaction. The significance level was set at 0.05. The
session. results revealed a significant association between the modalities
The participants then moved into the collaborative spaces and the categories of conversation, χ2 (26, N = 456) = 146.85,
tailored to their respective experimental conditions and started p < 0.001. This indicates a strong statistical significance, sug-
the experiment. For the Zoom group, this space was a shared gesting distinct patterns of communication across the modal-
whiteboard, while for the VR groups, it was a virtual environ- ities. Further detailed analysis was carried out on Cognition
ment modified from our previous edu-metaverse [6]. and Social Communication, our study focus. For Cognition, no
If participants in any group did not initiate conversation, the statistically significant relationship was detected, as evidenced
assistants would prompt them once to begin. Apart from this, the by χ2 (8, N = 120) = 3.46, p > 0.05. Conversely, for Social
assistants refrained from offering further guidance, except when Communication, the analysis demonstrated a statistically sig-
requested for technical assistance. The participants were not nificant relationship, with a Chi-square statistic of χ2 (10, N =
allowed to have any unnecessary interaction with the assistants 411) = 48.50, p < 0.001.
to prevent the experiment from deviating from its original plan Furthermore, in the experimental group using Zoom, the
and objectives. total number of utterances was notably lower compared to
Observations were conducted discreetly and unobtrusively. both the MM and MM+KG groups. There was an almost com-
For the Zoom group, remote monitoring was enabled through plete absence of discussion pertaining to “Technology Use.”
the meeting platform, with assistants exiting the experiment Furthermore, the proportion of participants focusing on “Reg-
room post-initiation to preserve the integrity of the experimental ulation” differed significantly from that of the VR group.
conditions. In the VR groups, the HMDs inherently isolated the It was also observed that in terms of “Social Communica-
participants from the physical presence of the assistants, who tion,” the Zoom group had a propensity for more elabora-
remained inside the experimental space to monitor the proceed- tive communication. In contrast, the two VR groups exhib-
ings, ensuring that the experiment’s immersive experience was ited similar patterns, except for differences in “Questions”
left undisturbed. and “Technology Use.” The number of utterances from the
After 20 min, the assistants concluded the experiment. For MM+KG group was marginally higher than that from the MM
the two VR modalities, a focus group was conducted to elicit group.
user feedback on the MM CL. Upon the completion of the
focus group, all participants from all conditions signed a receipt
to acknowledge their participation and exited the experiment
rooms. 1 [Online]. Available: https://www.meta.com/us/quest/quest-3/
JIA et al.: KNOWLEDGE-GRAPH-DRIVEN MIND MAPPING FOR IMMERSIVE COLLABORATIVE LEARNING 1803

B. Usability VR makes me feel more collaborative” and another participant

The mean score for the SUS is 56.25, with a standard devia- mentioned feeling “connected to each other when using VR.” As
tion of 11.59. This suggests that our KG-driven MM generally one participant said: “(In Zoom), I feel less connected to each
achieves an “OK” level of usability in CL, as defined by Bangor other.” An additional insight gleaned from participant feedback
et al. [84]. suggests that the heightened sense of copresence experienced
during CL tasks could be a contributing factor. This observation
resonates with our theoretical framework, which posits that a
VIII. DISCUSSION
strong sense of copresence is pivotal when engaging in joint
A. Higher Group Involvement educational activities. The perceived increase in copresence
among participants likely facilitated a more cohesive and IL en-
Based on the observations and results from our group con-
vironment, in line with the principles underpinning our approach
versations, it seems that people in the Zoom group are less
(refer to Section III-A).
inclined to engage in conversations compared to the VR group.
Together with the conversation result, we can see that the VR
In our experiment, we noticed that in each Zoom group, there
group shows a consistent degree of collaboration between learn-
were one or two participants who remained silent and focused
ers. This user behavior and feedback show that the copresence
on manipulating their MM nodes, appearing quite disturbed.
of an edu-metaverse learning environment has the potential to
Another observation worth mentioning is that when we started
be more involved in a CL task as it may help with copresence.
the experiment and opened the camera on Zoom, all participants
subconsciously either covered the camera with their hands or hid
their faces from the camera. This behavior might indicate that C. Realistic Avatar is Not a Must
when they are faced with unfamiliar individuals and forced into a Another interesting finding is that it seems that a realistic
collaborative environment, they feel anxiety. As argued by Guo avatar is not a must. Despite previous studies suggesting the
et al. [85], when people interact with others without a mask, significance of a realistic pedagogical agent for teaching [15], it
they experience social anxiety. In this context, it could be the appears that for collaborative learners, the realism of an avatar
reason why the VR group with avatars, which function as masks holds less weight. Following the discussion of our previous
to cover people’s faces, had more conversation utterances. This point, learners provided feedback that they feel stay together
observation is further supported by the feedback from the Focus even though the avatar is very less realistic, but still, now
Group, where one participant from the MM group stated that “VR participants said: “The avatar does not move a lot, but we still
is the ‘gospel’ for social anxiety, as it provides anonymity.” From feel like to be in the same room.”
this perspective, using the MM modality in VR with avatars Thus, it seems that simply placing the learners in the VR envi-
can be an effective way to increase involvement. Although the ronment is already sufficient to generate a feeling of copresence
MM and MM+KG groups are quite similar, the MM+KG groups within them. Regardless, it is believed that a realistic avatar can
had more conversations regarding questions and off-task discus- help learners to better engage, but, if that is not possible, the
sions. The frequencies of questions and off-task discussions are edu-metaverse system may consider focusing resources other
[see Table II(f)], respectively, 75 and 47 for MM+KG, and 47 than optimizing the avatar.
and 33 for MM. This might be attributed to the fact that our
recommendation system successfully provided the users with D. Concurrent Work in Edu-Metaverse
the necessary information, allowing them to overcome techno-
The advent of VR technology brings a transformative poten-
logical hurdles and saving more time for asking and answering
tial to collaborative educational environments, often referred to
questions. This positive outcome indicates that our KG-based
as the edu-metaverse. A notable advantage of VR over traditional
recommendation system can further enhance user involvement
desktop displays is its expansive field of view, offering users a
and provide more discussion opportunities, building upon the
more extensive virtual space. This can facilitate a sense of free-
normal VR MM modality.
dom and spatial abundance conducive to collaborative efforts. In
an edu-metaverse setting, the ability to accommodate numerous
B. Feeling of Copresence
participants within a single shared virtual environment is a
It has been known that a VR environment can provide a greater key feature, enabling synchronous collaborative activities and
sense of presence compared to typical streaming platforms [5]. interactions that closely mimic physical presence.
But what does that entail, given an interactive learning task such However, this enlarged virtual arena also presents certain
as MM? Based on user behavior, it appears that a stronger sense challenges. The copious space can lead to increased distractions
of presence can facilitate greater engagement among learners. as users may find themselves more susceptible to environmental
This is evident in the Cognition-Idea Generation categories stimuli. In addition, the complexity of functions within such an
[see Table II(b)], where participants in the VR group were ob- immersive space can lead to a dilution of attention. For instance,
served to generate double the number of building ideas compared in the case of the MM+KG groups, there was a noticeable uptick
to those in the Zoom group; the frequency for Building on in off-task conversation. This trend suggests that participants’
Ideas is 22 and 27 for MM and MM+KG, while it is 10 for focus could be diverted by the intricate and perhaps alluring
Zoom. This finding is further supported by the feedback from features of the virtual environment, such as sophisticated rec-
the focus group, where one participant expressed that “Using ommendation systems.
1804 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

Despite these potential distractions, users appear to retain G. Difference of MMs Among Three Modalities
a preference for the extensive collaborative space offered by
Our study uncovered a notable divergence in the MM for-
VR. This sentiment was encapsulated during a focus group
malization across the three distinct modalities we examined, as
discussion, where one participant emphasized the benefits of VR
illustrated in Fig. 3. From our observation, in the Zoom groups,
by stating, “VR provides more space to work on compared to a
a dominant pattern emerged where a single individual tended to
screen.” Such feedback underscores the enduring appeal of VR’s
guide the discussion, with the MM predominantly shaped by this
immersive qualities, suggesting that the benefits of increased
leader. This dynamic led to seemingly intricate MM outcomes
spatiality may outweigh the drawbacks associated with environ-
when compared to those of the VR groups.
mental distractions. Future designs of Edu-metaverse platforms
Within the VR environment, different patterns were observed.
might need to balance the allure of expansive virtual spaces
In the standard VR group with MM, all members contributed to
with mechanisms that help maintain user focus and task-oriented
the MM, closely aligning the content with the designated tasks.
collaboration.
Conversely, in the MM+KG group, we noted a dynamic evolu-
tion of the MM. Initially, the MM conformed to the structure
E. Focused Environment suggested by our KG-driven system; subsequently, participants
adapted and pruned nodes in response to task requirements.
Another benefit of performing CL tasks in VR is that it Although the Zoom group’s MMs appeared more complex,
provides a focused environment that is less distracting. As the they may not fully represent collective consensus, as not all
desktop, they can see the environment around them; thus, it can members were equally engaged in the MM development process.
be very easy to distribute. One participant said “In reality, there In contrast, the VR groups’ MMs, while simpler, reflected a
may be things that distract us. In (edu-metaverse), we can just broader participation in their creation, suggesting a more unified
focus on the area. We can hear the sounds of our teammates. We group understanding and problem-solving approach.
can do it more (focused).”
Although it can be considered as a disadvantage for a
VR/metaverse system, the HMD may act as some sort of
H. Usability Needs to Be Improved
constraint to isolate the learner from reality and focus on the
learning material. This user feedback may also provide insight Usability is a critical component that significantly affects
on why a VR environment can provide a better sense of presence user experience. Although our KG-driven MM has achieved
compared to streaming platforms as a platform such as Zoom an “OK” level of usability, there remains substantial scope for
can be easily disengaged. enhancement. We gathered feedback from the usability group’s
focus group, which highlighted several areas for improvement.
A predominant concern relates to the authenticity of the exper-
F. KG Guidance imental environment. One user pointed out, “The setting of the
The principal objective in the development of a KG-based rec- environment needs to be close to reality, whether it is from the
ommendation system is to assist users in discovering potential light effect or, slightly more serious environment.” This factor
discussion topics. It is acknowledged that participants engaged could potentially impact learning efficiency. Another notable
in collaborative activities may lack the necessary background issue arises as the MM becomes increasingly complex, making
knowledge, potentially hindering their ability to effectively ini- it challenging for users to manage effectively. A participant
tiate discussions. Often, this leads to the creation of MMs that remarked, “For example, if multiple people cooperate, in a 3D
reflect only the immediate task or personal experiences. scene you see such a complex node will be a little messy, not
Our KG-driven recommendation system is intentionally as good as the automatic layout function of some PC software.”
crafted to steer participants toward meaningful reflection on the We will keep working on improving these issues in our future
subject matter, which, in turn, promotes more profound engage- work.
ment in dialogue. This benefit is supported by a testimonial from
a participant within the MM+KG group, who said “We first use
the element given by (the system) in the recommendation.” Fur- IX. LIMITATIONS AND FUTURE WORK
thermore, the recorded conversations demonstrate an increase
A. Avatar
in social interactions within MM+KG groups [see Table II(d)],
suggesting that such an environment fosters a greater propensity In prior work, we advocated for avatars optimized for com-
for communication. munication over photorealism. We proposed gaze tracking to
In addition, it is noteworthy that despite the recommendation visualize user focus and enhance avatar communication. How-
system’s directive nature, the generation of new ideas flourished ever, in this study, examining a KG-driven recommendation
within the MM+KG groups [see Table II(b)]. This indicates system for multimodal conversational tasks, we have not yet
that our recommendation system does not stifle the cognitive implemented gaze tracking. Instead, we only track limited hand
process of ideation. These observations affirm the system’s gestures and head turns. This approach has limitations, as it does
capability not just to anchor participants’ comprehension but not capture full nonverbal cues essential for engagement. We aim
also to enhance the caliber of collaborative discourse by ensuring to later incorporate gaze tracking and advanced motion tracking
that discussions are rooted in pertinent and substantial material. to enable avatars to convey natural engaging nonverbal cues.
JIA et al.: KNOWLEDGE-GRAPH-DRIVEN MIND MAPPING FOR IMMERSIVE COLLABORATIVE LEARNING 1805

B. Gender Balance (FQ2) How do you collaboratively decide to add a new word or
Most of the participants who responded to our recruit- not?
ment were predominantly female, despite attempts to recruit (FQ3) What can further assist you in adding a new word or not?
a gender-balanced sample. This imbalance is noteworthy as (FQ4) Compare to drawing a mind map via Zoom, what is your
communication styles differ between genders, as highlighted expectation of the difference between VR and Zoom?
by Burleson [86]. The overrepresentation of females could thus (FQ5) What concerns/issues do you have in a VR e-learning
influence the study’s outcomes and its applicability to a broader session?
population. (FQ6) Do you think VR can make learning more engaging or
enjoyable? Why or why not?
X. CONCLUSION
(FQ7) How would you feel about group work or collaborative
Investigating user behavior in CL tasks is crucial for realizing projects in a VR environment?
an edu-metaverse. One of the issues to look into is what benefits (FQ8) What is the difference between a 2-D and 3-D mind map?
can edu-metaverse bring when conducting learning activities (FQ9) How well have you been interacting with the mind map?
compared to streaming platforms such as Zoom. Based on our Is it natural, if not, how can it be improved?
previous study, we have selected MM as a CL task because it is (FQ10) What is your thinking process during the session?
both generally applicable to many learning scenarios and also an (FQ11) Will you find yourself to be less distracted?
important component of constructivist teaching methodology. (FQ12) Let’s start by sharing our thoughts about VR as a tool
Furthermore, we utilize the KG from K-Cube VR, our edu- for learning together. What do you see as the most significant
metaverse prototype, to provide guidance during the MM de- benefits of using VR for CL, based on your own experiences
signing session. Our work indicates that in contrast to platforms or expectations? Potential drawbacks of CL in VR.
such as Zoom, conducting collaborative tasks in edu-metaverse, (FQ13) How do you feel the experience of CL in VR compares
particularly MM, can have several advantages. The conversation to more traditional learning environments, such as classrooms
analysis shows that students in edu-metaverse may be more or online forums without immersive elements? Suggestions
engaged as a group, in contrast to that of Zoom, which may be for improvement.
prone to detachment. Furthermore, the edu-metaverse enhances (FQ14) While we’ve considered the positives, it’s also important
the feeling of copresence among students, which contributes to to look at the other side. What drawbacks or challenges
a more effective virtual environment for CL. might arise when using VR for CL sessions? Comparative
experiences with traditional learning methods.
APPENDIX A (FQ15) Considering both the pros and cons we’ve discussed,
CL TASK what improvements or features would you suggest could en-
The collaborative learning task is shown in Fig. 6. hance the experience of CL in VR? As for the MM+KG group,
we also asked the following question to receive participants’
feedback.
(FQ16) What do you feel about the suggestion nodes? When you
see the recommendation, did you change your understanding?

ACKNOWLEDGMENT
The authors would like to thank Junnan Dong, Qinggang
Zhang, Wentao Li, and Yaowei Wang for their useful discussions
and feedback. The authors would also like to thank Yufei Lu,
Yao Wang, Guang Chen, Xinran Xu, Xinrui Bai, Wujie Gao, and
Yixin Dai for their assistance in User Study.

REFERENCES
[1] M. Wang, H. Yu, Z. Bell, and X. Chu, “Constructing an edu-metaverse
ecosystem: A new and innovative framework,” IEEE Trans. Learn. Tech-
nol., vol. 15, no. 6, pp. 685–696, Dec. 2022.
[2] L. Hejtmanek, M. Starrett, E. Ferrer, and A. D. Ekstrom, “How much
of what we learn in virtual reality transfers to real-world navigation?,”
Multisensory Res., vol. 33, nos. 4/5, pp. 479–503, 2020.
Fig. 6. Collaborative learning task panel. [3] S. Jang, J. M. Vitale, R. W. Jyung, and J. B. Black, “Direct manipulation
is better than passive viewing for learning anatomy in a three-dimensional
virtual reality environment,” Comput. Educ., vol. 106, pp. 150–165, 2017.
[4] H. Lee, D. Woo, and S. Yu, “Virtual reality metaverse system supplement-
APPENDIX B ing remote education methods: Based on aircraft maintenance simulation,”
FOCUS GROUP QUESTIONS Appl. Sci., vol. 12, no. 5, 2022, Art. no. 2667.
[5] C. Li et al., “Collaborative learning in the edu-metaverse era: An empirical
(FQ1) What is your first thought when immersed in this envi- study on the enabling technologies,” IEEE Trans. Learn. Technol., vol. 17,
ronment? pp. 1107–1119, 2024.
1806 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

[6] Z. P. T. Sin et al., “Toward an edu-metaverse of knowledge: Immersive [30] A. Jovanović and A. Milosavljević, “Vortex metaverse platform for gami-
exploration of university courses,” IEEE Trans. Learn. Technol., vol. 16, fied collaborative learning,” Electronics, vol. 11, no. 3, 2022, Art. no. 317.
no. 6, pp. 1096–1110, Dec. 2023. [31] Z. Yang, W. Pan, and Y. Wang, “Group cooperative learning model and
[7] Q. Li et al., “KCUBE: A knowledge graph university curriculum frame- its application based on group knowledge graph,” in Proc. IEEE 3rd Int.
work for student advising and career planning,” in Proc. Int. Conf. Blended Conf. Comput. Sci. Educ. Inf., 2021, pp. 324–327.
Learn., 2022, pp. 358–369. [32] D. Kudryavtsev, T. Gavrilova, I. Leshcheva, A. Begler, M. Kubelskiy, and
[8] M. A. Kuhail, A. ElSayary, S. Farooq, and A. Alghamdi, “Exploring O. Tushkanova, “Mind mapping and spreadsheets in collaborative design
immersive learning experiences: A survey,” Informatics, vol. 9, 2022, of knowledge graphs,” in Proc. Joint BIR Forum, Workshops Doctoral
Art. no. 75. Consortium, 2018, pp. 82–93.
[9] S. D. Piovesan, L. M. Passerino, and A. S. Pereira, “Virtual reality as a [33] K. L. Nitin, L. L. N. Rao, and N. K. Sivaraman, “An emotional system for
tool in the education,” in Proc. Int. Conf. Cogn. Explor. Learn. Digit. Age, effective and collaborative e-learning,” in Proc. 4th Int. Conf. Adv. Comput.
2012, pp. 295–298. Hum. Interact., 2011, pp. 260–266.
[10] T. Weymuth and M. Reiher, “Immersive interactive quantum mechanics for [34] T. Monahan, G. McArdle, and M. Bertolotto, “Virtual reality for collabo-
teaching and learning chemistry,” CHIMIA Int. J. Chem., vol. 75, nos. 1/2, rative e-learning,” Comput. Educ., vol. 50, no. 4, pp. 1339–1353, 2008.
pp. 45–49, 2021. [35] M. H. Kim, “Effects of collaborative learning in a virtual environment on
[11] Q. Cao, B. T. Png, Y. Cai, Y. Cen, and D. Xu, “Interactive virtual reality students’ academic achievement and satisfaction,” J. Digit. Convergence,
game for online learning of science subject in primary schools,” in Proc. vol. 19, no. 4, pp. 1–8, 2021.
Int. Conf. Eng., Technol, Educ., 2021, pp. 383–389. [36] T. Drey et al., “Towards collaborative learning in virtual reality: A com-
[12] F. Grivokostopoulou, I. Perikos, K. Kovas, M. Paraskevas, and I. Hatzi- parison of co-located symmetric and asymmetric pair-learning,” in Proc.
lygeroudis, “Utilizing virtual reality to assist students in learning physics,” CHI Conf. Hum. Factors Comput. Syst., 2022, pp. 1–19.
in Proc. Int. Conf. Teaching, Assessment, Learn. Eng., 2017, pp. 486–489. [37] S. W. Greenwald et al., “Technology and applications for collaborative
[13] D. M. Markowitz and J. N. Bailenson, “Virtual reality and the psychology learning in virtual reality,” in Proc. Int. Soc. Learn. Sci., Philadelphia, PA,
of climate change,” Curr. Opin. Psychol., vol. 42, pp. 60–65, 2021. USA, 2017, pp. 719–726.
[14] Z. Feng et al., “An immersive virtual reality serious game to enhance earth- [38] T. Shibata, E. Drago, T. Araki, and T. Horita, “Encouraging collaborative
quake behavioral responses and post-earthquake evacuation preparedness learning in classrooms using virtual reality techniques,” in Proc. EdMedia:
in buildings,” Adv. Eng. Informat., vol. 45, 2020, Art. no. 101118. World Conf. Educ. Media Technol., 2018, pp. 1577–1582.
[15] G. B. Petersen, A. Mottelson, and G. Makransky, “Pedagogical agents in [39] E. Prasolova-Førland, S. McCallum, and J. G. Estrada, “Collaborative
educational VR: An in the wild study,” in Proc. ACM Conf. Hum. Factors learning in VR for cross-disciplinary distributed student teams,” in Proc.
Comput. Syst., 2021, pp. 1–12. IEEE Conf. Virtual Reality 3D User Interfaces Abstr. Workshops, 2021,
[16] D. Hamilton, J. McKechnie, E. Edgerton, and C. Wilson, “Immersive pp. 320–325.
virtual reality as a pedagogical tool in education: A systematic literature [40] R. Sims and A. Karnik, “VERITAS: Mind-mapping in virtual reality,” in
review of quantitative learning outcomes and experimental design,” J. Proc. 7th Int. Conf. Immersive Learn. Res. Netw., 2021, pp. 1–8.
Comput. Educ., vol. 8, no. 1, pp. 1–32, 2021. [41] M. Miyasugi, H. Akaike, Y. Nakayama, and H. Kakuda, “Implementation
[17] G. Billingsley, S. Smith, S. Smith, and J. Meritt, “A systematic literature and evaluation of multi-user mind map authoring system using virtual re-
review of using immersive virtual reality technology in teacher education,” ality and hand gestures,” in Proc. IEEE 6th Glob. Conf. Consum. Electron.,
J. Interact. Learn. Res., vol. 30, no. 1, pp. 65–90, 2019. 2017, pp. 1–5.
[18] C. Li and J. Liu, “Collaborative virtual environment for distant and blended [42] R. Zvauya, S. Purandare, N. Young, and M. Pallan, “The use of mind maps
learning in the higher education setting: A systematic review,” in Proc. Int. as an assessment tool in a problem based learning course,” Creative Educ.,
Conf. Blended Learn., 2022, pp. 135–146. vol. 8, no. 11, pp. 1782–1793, 2017.
[19] M. Hmoud, H. Swaity, O. Karram, H. Shibli, S. Swaity, and W. Daher, [43] X. Zheng, T. E. Johnson, and C. Zhou, “A pilot study examining the impact
“High school students’ engagement in biology in the context of XR of collaborative mind mapping strategy in a flipped classroom: Learning
technology,” IEEE Access, vol. 11, pp. 137053–137066, 2023. achievement, self-efficacy, motivation, and students’ acceptance,” Educ.
[20] S. Mystakidis, E. Berki, and J.-P. Valtanen, “Deep and meaningful e- Technol. Res. Develop., vol. 68, pp. 3527–3545, 2020.
learning with social virtual reality environments in higher education: A [44] R. L. Jackson and E. Fagan, “Collaboration and learning within immersive
systematic literature review,” Appl. Sci., vol. 11, no. 5, 2021, Art. no. 2412. virtual reality,” in Proc. 3rd Int. Conf. Collaborative Virtual Environ., 2000,
[21] A. Sanchez, J. María Barreiro, and V. Maojo, “Design of virtual reality pp. 83–92.
systems for education: A cognitive approach,” Educ. Inf. Technol., vol. 5, [45] M. Webb et al., “Haptic-enabled collaborative learning in virtual reality
pp. 345–362, 2000. for schools,” Educ. Inf. Technol., vol. 27, pp. 937–960, 2022.
[22] A. Scavarelli, A. Arya, and R. J. Teather, “Virtual reality and augmented [46] D. Kozachek, M. A. Yaqin, K. Prasad, and S.-M. Wang, “Evaluating
reality in social learning spaces: A literature review,” Virtual Reality, the outcome of collaborative VR mind mapping sessions with sentiment
vol. 25, pp. 257–277, 2021. analysis and emotional intelligence,” in Proc. Int. Conf. Hum.-Comput.
[23] T. T. de Back, A. M. Tinga, P. Nguyen, and M. M. Louwerse, “Benefits Interact., 2023, pp. 245–263.
of immersive collaborative learning in cave-based virtual reality,” Int. J. [47] G. Trajkovski, An Imitation-Based Approach to Modeling Homogenous
Educ. Technol. Higher Educ., vol. 17, pp. 1–18, 2020. Agents Societies. Hershey, PA, USA: IGI Global, 2007, ch. 5, p. 41.
[24] T. T. De Back, A. M. Tinga, and M. M. Louwerse, “Learning in immersed [48] Z. P. T. Sin et al., “Towards a metaverse of knowledge: A constructivist
collaborative virtual environments: Design and implementation,” Interact. proposition on immersive learning and visual literacy,” in Proc. Int. Conf.
Learn. Environ., vol. 31, no. 8, pp. 5364–5382, 2023. Web-Based Learn., 2022, pp. 214–225.
[25] S.-S. Liaw and H.-M. Huang, “Developing a collaborative e-learning [49] J. J. Gibson, The Ecological Approach to Visual Perception: Classic
system based on users’ perceptions,” in Proc. Int. Conf. Comput. Supported Edition. London, U.K.: Psychology Press, 2014.
Cooperative Work Des., 2006, pp. 751–759. [50] J. J. Cummings and J. N. Bailenson, “How immersive is enough? A meta-
[26] L.-K. Lee, T.-K. Cheung, L.-T. Ho, W.-H. Yiu, and N.-I. Wu, “Learning analysis of the effect of immersive technology on user presence,” Media
computational thinking through gamification and collaborative learning,” Psychol., vol. 19, no. 2, pp. 272–309, 2016.
in Blended Learning: Educational Innovation for Personalized Learning. [51] B. Kim, “Social constructivism,” in Emerging Perspectives on Learning,
Cham, Switzerland: Springer, 2019, pp. 339–349. Teaching, and Technology. Athens, GA, USA: Univ. Georgia, 2001.
[27] C. Li, Z. Dong, R. H. Untch, and M. Chasteen, “Engaging computer [52] S. R. Hiltz, “Collaborative learning in asynchronous learning networks:
science students through gamification in an online social network-based Building learning communities,” in Proc. WebNet 98 World Conf. WWW,
collaborative learning environment,” Int. J. Inf. Educ. Technol., vol. 3, Internet, Intranet, no. ED 427 705 IR 019 256, 1998.
no. 1, pp. 72–77, 2013. [53] M. Davis, “How collaborative learning leads to student success,” 2012.
[28] T. Xie, R. Liu, Y. Chen, and G. Liu, “MOCA: A motivational online Accessed: May 23, 2024. [Online]. Available: https://www.edutopia.org/
conversational agent for improving student engagement in collaborative stw-collaborative-learning-college-prep
learning,” IEEE Trans. Learn. Technol., vol. 14, no. 5, pp. 653–664, [54] P. Quinn, “Cooperative learning and student motivation,” Unpublished
Oct. 2021. master.s thesis, Dept. Educ. Hum. Develop., State Univ. New York, Brock-
[29] J. M. Harley, M. Taub, R. Azevedo, and F. Bouchet, “Let’s set up some port, NY, USA, 2006. [Online]. Available: http://www.brockport.edu/ehd/
subgoals: Understanding human-pedagogical agent collaborations and [55] X. Zhou and C.-W. Tsai, “The effects of socially shared regulation of
their implications for learning and prompt and feedback compliance,” learning on the computational thinking, motivation, and engagement in
IEEE Trans. Learn. Technol., vol. 11, no. 1, pp. 54–66, Jan.–Mar. 2017. collaborative learning by teaching,” Educ. Inf. Technol., vol. 28, no. 7,
pp. 8135–8152, 2023.
JIA et al.: KNOWLEDGE-GRAPH-DRIVEN MIND MAPPING FOR IMMERSIVE COLLABORATIVE LEARNING 1807

[56] A. Lazareva, “Factors affecting student engagement in online collaborative [81] J. T. Wong, A. Mesghina, E. Chen, N. A. Yeung, B. S. Lerner, and L. E.
learning courses,” in Teaching and Learning in a Digital World. Cham, Richland, “Zooming in or zoning out: Examining undergraduate learning
Switzerland: Springer, 2018, pp. 349–359. experiences with zoom and the role of mind-wandering,” Comput. Educ.
[57] J. Casanueva and E. Blake, “The effects of group collaboration on presence Open, vol. 4, 2023, Art. no. 100118.
in a collaborative virtual environment,” in Proc. Eurographics Workshop [82] M. Conrad, D. Kablitz, and S. Schumann, “Learning effectiveness of
Virtual Environ., 2000, pp. 85–94. immersive virtual reality in education and training: A systematic review
[58] I. D. Dunmoye, A. Rukangu, D. May, and R. P. Das, “An exploratory study of findings,” Comput. Educ.: X Reality, vol. 4, 2024, Art. no. 100053.
of social presence and cognitive engagement association in a collaborative [83] J. Brooke, “SUS—A quick and dirty usability scale,” Usability Eval. Ind.,
virtual reality learning environment,” Comput. Educat.: X Reality, vol. 4, vol. 189, no. 194, pp. 4–7, 1996.
2024, Art. no. 100054. [84] A. Bangor, P. Kortum, and J. Miller, “Determining what individual SUS
[59] E. Lehtinen, K. Hakkarainen, L. Lipponen, M. Rahikainen, and H. scores mean: Adding an adjective rating scale,” J. Usability Stud., vol. 4,
Muukkonen, “Computer supported collaborative learning: A review,” no. 3, pp. 114–123, 2009.
JHGI Giesbers Reports Educ., vol. 10, 1999, Art. no. 1999. [85] K. Guo, A. Hare, and C. H. Liu, “Impact of face masks and viewers’
[60] M. G. Moore, “Three types of interaction,” Amer. J. Distance Educ., vol. 3, anxiety on ratings of first impressions from faces,” Perception, vol. 51,
no. 2, pp. 1–7, 1989, doi: 10.1080/08923648909526659. no. 1, pp. 37–50, 2022.
[61] M. J. Eppler, “A comparison between concept maps, mind maps, concep- [86] B. R. Burleson, A. W. Kunkel, W. Samter, and K. J. Working, “Men’s and
tual diagrams, and visual metaphors as complementary tools for knowl- women’s evaluations of communication skills in personal relationships:
edge construction and sharing,” Inf. Visual., vol. 5, no. 3, pp. 202–210, When sex differences make a difference and when they don’t,” J. Social
2006. Pers. Relationships, vol. 13, no. 2, pp. 201–224, 1996.
[62] L. J. Ausburn and F. B. Ausburn, “Visual literacy: Background, theory and
practice,” Programmed Learn. Educ. Technol., vol. 15, no. 4, pp. 291–297,
1978.
[63] J. Kedra and R. Žakevičiutė, “Visual literacy practices in higher education:
What, why and how?,” J. Vis. Lit., vol. 38, nos. 1/2, pp. 1–7, 2019.
[64] S. Roth, “Visual literacy and the design of digital media,” ACM SIGGRAPH
Comput. Graph., vol. 29, no. 4, pp. 45–47, 1995.
[65] E. Choporova, “Application of mind map-based abstracting technique in
Ye Jia received the B.Eng. degree in robotics engi-
pedagogical strategy for esp teaching/learning,” J. Educ., Culture, Soc.,
neering from the Chongqing University of Arts and
vol. 5, no. 1, pp. 110–123, 2014.
Science, Chongqing, China, in 2022.
[66] T. Buzan, Mind Map Mastery: The Complete Guide to Learning and Using
He is currently a Research Assistant with the De-
the Most Powerful Thinking Tool in the Universe. London, U.K.: Watkins
partment of Computing, Hong Kong Polytechnic Uni-
Media Limited, 2018.
versity, Hong Kong. His research interests include
[67] R. Sari, S. Sumarmi, I. Astina, D. Utomo, and R. Ridhwan, “Increasing
human–computer interaction and extended reality.
students critical thinking skills and learning motivation using inquiry mind
map,” Int. J. Emerg. Technol. Learn., vol. 16, no. 3, pp. 4–19, 2021.
[68] M. N. Ismail, N. A. Ngah, and I. N. Umar, “The effects of mind mapping
with cooperative learning on programming performance, problem solving
skill and metacognitive knowledge among computer science students,” J.
Educ. Comput. Res., vol. 42, no. 1, pp. 35–61, 2010.
[69] M. Sun, M. Wang, R. Wegerif, and J. Peng, “How do students generate
ideas together in scientific creativity tasks through computer-based mind
mapping?,” Comput. Educ., vol. 176, 2022, Art. no. 104359.
[70] A. Erdem, “Mind maps as a lifelong learning tool,” Universal J. Educ. Xiangzhi Eric Wang received the B.Sc. degree in
Res., vol. 5, no. n12A, pp. 1–7, 2017. computing from Hong Kong Polytechnic University,
[71] D. Koznov and M. Pliskin, “Computer-supported collaborative learning Hong Kong, in 2023.
with mind-maps,” in Proc. Int. Symp. Leveraging Appl. Formal Methods, He is currently a Research Assistant with the
Verification Validation, 2008, pp. 478–489. Department of Computing, Hong Kong Polytechnic
[72] ö. Polat, T. Sezer, and N. A. Akyol, “Collaborative learning with mind University. His research interests include computer
mapping in the development of social skills of children,” Participatory graphics, human–computer interaction, and immer-
Educ. Res., vol. 9, no. 1, pp. 463–480, 2022. sive technologies, such as virtual and augmented
[73] L. Zheng, M. Long, B. Chen, and Y. Fan, “Promoting knowledge elabo- reality, with a particular interest in metaverse appli-
ration, socially shared regulation, and group performance in collaborative cations.
learning: An automated assessment and feedback approach based on
knowledge graphs,” Int. J. Educ. Technol. Higher Educ., vol. 20, no. 1,
p. 46, 2023.
[74] X. Chen, S. Jia, and Y. Xiang, “A review: Knowledge reasoning over
knowledge graph,” Expert Syst. Appl., vol. 141, 2020, Art. no. 112948.
[75] X. Zou, “A survey on application of knowledge graph,” J. Phys.: Conf.
Ser., vol. 1487, 2020, Art. no. 012016.
[76] F.-R. Dang, J.-T. Tang, K.-Y. Pang, T. Wang, S.-S. Li, and X. Li,
“Constructing an educational knowledge graph with concepts linked to Zackary P. T. Sin received the B.Sc. and Ph.D.
Wikipedia,” J. Comput. Sci. Technol., vol. 36, pp. 1200–1211, 2021. degrees in computing from Hong Kong Polytechnic
[77] Y. Shen, Z. Chen, G. Cheng, and Y. Qu, “CKGG: A Chinese knowledge University (PolyU), Hong Kong, in 2016 and 2023,
graph for high-school geography education and beyond,” in Proc. Int. respectively.
Semantic Web Conf., 2021, pp. 429–445. He is currently a Research Assistant Professor with
[78] K. Sun, Y. Liu, Z. Guo, and C. Wang, “EDUVIS: Visualization for PolyU. He is an experienced Multimedia Researcher
education knowledge graph based on web data,” in Proc. Int. Symp. Vis. and Developer with publications in IEEE Virtual Re-
Inf. Commun. Interact., 2016, pp. 138–139. ality Conference, SIGGRAPH Symposium on Inter-
[79] P. Chen, Y. Lu, V. W. Zheng, X. Chen, and B. Yang, “KnowEdu: A active 3D Graphics and Games, and Annual Sym-
system to construct knowledge graph for education,” IEEE Access, vol. 6, posium on Computer–Human Interaction in Play. He
pp. 31553–31563, 2018. has authored or coauthored papers in IEEE TRANSAC-
[80] K. Wheaton, “Improved zoom engagement by adopting flipped class strate- TIONS ON LEARNING TECHNOLOGIES, International Conference on Web-Based
gies to the curriculum of a biology of aging course during the COVID-19 Learning, and International Conference on the Bioscience of Lipids. His main
pandemic,” Biochem. Mol. Biol. Educ., vol. 51, pp. 451–454, 2023. research interests include metaverse, computer graphics, and human–computer
interaction, as well as enabling novel learning experiences in an edu-metaverse.
1808 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

Chen Li (Member, IEEE) received the B.Sc. degree George Baciu (Senior Member, IEEE) received the
in computer science and technology from Nanjing bachelor’s degree in computer science and applied
University, Nanjing, China, in 2008, and the M.Sc. mathematics and the Ph.D. degree in systems engi-
and Ph.D. degrees in computer science from the City neering from the University of Waterloo, Waterloo,
University of Hong Kong, Hong Kong, in 2011 and ON, Canada, 1985, 1987, and 1992, respectively.
2018, respectively. He was a Member of the Waterloo Computer
He is currently an Assistant Professor with the Graphics Lab and the Pattern Analysis and Machine
Department of Applied Social Sciences and the De- Intelligence Lab, University of Waterloo. He was the
partment of Computing, Hong Kong Polytechnic Uni- Founding Director of the GAME Lab, Hong Kong
versity, Hong Kong. His research interests include University of Science and Technology, Hong Kong, in
virtual reality, augmented reality, Metaverse, spatial 1993, and the Graphics and Multimedia Applications
user interface, serious games, persuasive games, and educational technology. Lab, Department of Computing, Hong Kong Polytechnic University, in 2000.
He is currently a Professor with the Department of Computing, Hong Kong
Polytechnic University. He has published extensively in computer graphics,
image processing, and virtual reality (VR) journals and conferences. His research
interests include information visualization, cognitive computing, cloud comput-
ing, distributed machine learning, VR, and computer graphics, with applications
to cognitive digital agents, digital twins, motion synthesis, animation, collision
detection, geometric modeling, and image analysis.
Dr. Baciu has served as the Chair of international conferences, such as
Game Technology Conference, Pacific Conference on Computer Graphics and
Applications, ACM Symposium on Virtual Reality Software and Technology
(and ACM International Conference on Virtual Reality Continuum and Its Appli-
cations), Annual Conference of the European Association for Computer Graph-
ics, Computer Graphics International Conference, International Conference on
Computer-Aided Design and Computer Graphics, and Computer Animation and
Social Agents, and IEEE International Conference on Cognitive Informatics
Peter H. F. Ng received the B.Sc. (Hons.) degree and Cognitive Computing. He is a Member of the Association for Computing
in internet and multimedia technologies, the M.Sc. Machinery.
degree in multimedia and entertainment technology,
and the Ph.D. degree in computing from Hong Kong
Polytechnic University (PolyU), Hong Kong, in 2004,
2007, and 2013, respectively.
He is currently an Assistant Professor with the
Department of Computing and Rehabilitation Sci-
ences, PolyU, where he is also in charge of the Game Jiannong Cao (Fellow, IEEE) received the M.Sc. and
Lab and a Deputy Programmer Leader of the FT Ph.D. degrees in computer science from Washington
Broad Discipline of Computing (EIS Steam) with the State University, Pullman, WA, USA, in 1986 and
Department of Computing. His research interests include artificial intelligence, 1990, respectively.
health application, application (APP), games, and extended reality development. He is currently a Chair Professor with the Depart-
Dr. Ng received one of the Best anime, comics, games (ACG)+ Capital Teams ment of Computing, Hong Kong Polytechnic Univer-
Award from the Hong Kong Home Affairs Department in 2017, Merit Awards sity, where he is also the Dean of Graduate School,
of Teaching from the PolyU Faculty of Engineering (FENG) in 2017 and 2021, the Director of the Research Institute of Artificial
Team Awards of Teaching from the PolyU FENG in 2021, and Merit Awards Intelligent of Things and the Internet and Mobile
of Service from the PolyU FENG in 2019. He was the Gold Winner for the Computing Lab, and the Vice Director of the Uni-
Community Outreach Award (eLFA2021) in 2021. He also received the Best versity’s Research Facility in Big Data Analytics.
Ten Teachers in the Greater Bay Area STEM Excellence Award in 2022. He has coauthored five books, coedited nine books, and published more than
500 papers in major international journals and conference proceedings. His
research interests include distributed systems and blockchain, wireless sensing
and networking, big data and machine learning, and mobile cloud and edge
computing.
Dr. Cao is a Member of Academia Europaea and a Distinguished Member of
the Association for Computing Machinery.

Qing Li (Fellow, IEEE) received the B.Eng. degree

Xiao Huang received the B.S. degree in engineer- from Hunan University, Changsha, China, in 1982,
ing from Shanghai Jiao Tong University, Shanghai, and the M.Sc. and Ph.D. degrees from the University
China, in 2012, the M.S. degree in electrical engineer- of Southern California, Los Angeles, CA, USA, in
ing from the Illinois Institute of Technology, Chicago, 1985 and 1988, respectively, all in computer science.
IL, USA, in 2015, and the Ph.D. degree in computer He is currently a Chair Professor with the Depart-
engineering from Texas A&M University, College ment of Computing, Hong Kong Polytechnic Univer-
Station, TX, USA, in 2020. sity, Hong Kong. He has authored or coauthored more
He is currently an Assistant Professor with the than 500 publications. He is actively involved in the
Department of Computing, Hong Kong Polytechnic research community. His research interests include
University, Hong Kong. His research interests in- multimodal data management, conceptual data mod-
clude large language models, graph neural networks, eling, social media, web services, and e-learning systems.
knowledge graphs, question answering, and recommender systems. Dr. Li is a Fellow of the Asia-Pacific Artificial Intelligence Association and the
Dr. Huang was the recipient of the Best Paper Award Honorable Mention at Institution of Engineering and Technology (U.K.) and a Distinguished Member
SIGIR 2023. of the China Computer Federation.