2023

2023 13th
13th International
InternationalConference
Conferenceon
onInformation
InformationTechnology
Technologyin
inMedicine
Medicineand
andEducation
Education(ITME)
(ITME)

)LQHWXQLQJ/DUJH/DQJXDJH0RGHOVLQ(GXFDWLRQ
Fine-tuning Large Language Models in Education


<RQJ&KHQ+RQJSHQJ&KHQ6RQJ]KL6X
Yong Chen, Hongpeng Chen, Songzhi Su *
2023 13th International Conference on Information Technology in Medicine and Education (ITME) | 979-8-3503-1915-6/23/$31.00 ©2023 IEEE | DOI: 10.1109/ITME60234.2023.00148

,QVWLWXWHRI$UWLILFLDO,QWHOOLJHQFH
Institute of Artificial Intelligence
;LDPHQ8QLYHUVLW\
Xiamen University
;LDPHQ&KLQD
Xiamen, China
HPDLOVV]#[PXHGXFQ
e-mail: [email protected]


$EVWUDFW²,QUHFHQW\HDUVODUJHODQJXDJHPRGHOV
Abstract-In recent years, large language models (LLMs) //0V KDYHhave 6WDEOH
Stable 'LIIXVLRQ
Diffusion >@ [4] DQG
and *37
GPT-4 >@[ 5 ] EDVHG
based RQ
o n WKH
the IXVLRQ
fusion RI
of
EHHQ
been Da KRW
hot WRSLF
topic LQ
in DUWLILFLDO
artificial LQWHOOLJHQFH
intelligence UHVHDUFK
research, SURIRXQGO\
profoundly PDVVLYH
massive WH[W
text DQG
and LPDJH
image GDWDdata KDYH
have IXUWKHU
further DFKLHYHG
achieved FURVV
cross­
LPSDFWLQJ
impacting PDQ\
many ILHOGV
fields, LQFOXGLQJ
including HGXFDWLRQ
education. //0V
LLMs VKRZFDVH
showcase PRGDO
modal FRPSUHKHQVLRQ
comprehension DQG and JHQHUDWLRQ
generation, HQKDQFLQJ
enhancing WKHLU their
SRZHUIXO
powerful FDSDELOLWLHV
capabilities LQ
in QDWXUDO
natural ODQJXDJH
language FRPSUHKHQVLRQ
comprehension DQG JHQHUDOL]DWLRQ
and generalization FDSDELOLWLHV
capabilities DFURVVacross PXOWLSOH
multiple GRPDLQV
domains DQG and
JHQHUDWLRQ
generation, GHPRQVWUDWLQJ
demonstrating LPSUHVVLYH
impressive GRPDLQ
domain JHQHUDOL]DWLRQ WDVNV
generalization
tasks. +RZHYHU
However, GXH due WRto WKH
the PDVVLYH
massive WUDLQLQJ
training GDWD
data, UHODWLYHO\
relatively
DIWHU
after GRZQVWUHDP
downstream WDVNVtasks ILQHWXQLQJ
fine-tuning. 7KHUHIRUH
Therefore, ILQHWXQLQJ FRPSOH[
//0V
fine-tuning
complex LQWHUQDO
internal VWUXFWXUH
structure, DQG and QXPHURXV
numerous SDUDPHWHUV
parameters LQ in
LLMs KDYHhave DWWUDFWHG
attracted PXFK
much DWWHQWLRQ
attention IURP
from HGXFDWLRQ
education, DQG and //0VWKHWUDLQLQJDQGXVDJHFRVWVDUHYHU\KLJKIRUDVLQJOH
UHVHDUFK RQ WKHLU HGXFDWLRQDO DSSOLFDWLRQV KDV EHJXQ WR HQWHU LLMs, the training and usage costs are very high for a single
research on their educational applications has begun to enter RUDIHZWDVNVDQGWKHGHOD\FDXVHGE\PRGHOFDOFXODWLRQLV
or a few tasks, and the delay caused by model calculation is
WKH
the SXEOLF
public YLHZ
view. 7KLV
This SDSHU
paper XQGHUWRRN
undertook D a FRPSUHKHQVLYH
comprehensive
H[DPLQDWLRQ DQG DQDO\VLV RI WKH ILQHWXQLQJ //0V DQG WKHLU
DOVR
also FRQVLGHUDEOH
considerable. :LWK With WKH the SUHWUDLQLQJ
pre-training DQG and ILQHWXQLQJ
fine-tuning
examination and analysis of the fine-tuning LLMs and their
SRWHQWLDO PHWKRG
method, VLJQLILFDQW
signifi c ant SHUIRUPDQFH
performance LPSURYHPHQWV
improvements FDQ
can EHbe
potential DSSOLFDWLRQV
applications LQin WKH
the ILHOG
field RI
of HGXFDWLRQ
education. 2QOn WKLV
this EDVLV
basis,
ZH H[SORUH WKH FRPPRQ LVVXHV FXUUHQWO\ HQFRXQWHUHG LQ ILQH DFKLHYHG
achieved E\
by ILQHWXQLQJ
fin e-tuning WKH
the SUHWUDLQHG
pre-trained PRGHOV
models XVLQJ
using WDVN
task­
we explore the common issues currently encountered in fine­
WXQLQJ//0VFRQVLGHULQJERWKWKHDGYDQWDJHVDQGOLPLWDWLRQV VSHFLILF
specific ODEHOHG
labeled GDWD data, VLJQLILFDQWO\
significantly UHGXFLQJ
reducing WKH the FRVW
cost RI
of
tuning LLMs, considering both the advantages and limitations
RI ILQHWXQLQJ //0V DQG WKHLU SRWHQWLDO
of fine-tuning LLMs and their potential to enhance the WR HQKDQFH WKH WUDLQLQJ
training VSHFLDOL]HG
specialized //0V
LLMs LQ
in VSHFLILF
specifi c GRPDLQV
domains. 7KHUHIRUH
Therefore,
HIILFLHQF\
efficiency DQG
and HIIHFWLYHQHVV
effectiveness RI of HGXFDWLRQ
education. )LQDOO\
Finally, WRto KHOS
help
WKLV
this PRUH
more HIILFLHQW
efficient SUHWUDLQLQJ
pre-training DQG and ILQHWXQLQJ
fine-tuning PHWKRG
method KDVhas
HGXFDWRUV
educators PDNH
make LQIRUPHG
informed GHFLVLRQV
decisions DQG and IRVWHU
foster LQQRYDWLRQ
innovation LQ in
JUDGXDOO\
gradually EHFRPH
become WKH the PDLQVWUHDP
mainstream SDUDGLJPparadigm LQ in DSSO\LQJ
applying
HGXFDWLRQ
education IRU
for WKH
the EHWWHU
better VHUYLFH
service RI
of KXPDQ
human HGXFDWLRQ
education, ZHwe ORRN
look //0VLQYDULRXVILHOGV
LLMs in various fields.
LQWRWKHIXWXUHWUHQGVDQGDSSOLFDWLRQVRIILQHWXQLQJ//0VLQ
into the future trends and applications of fine-tuning LLMs in 7KH
The UHVHDUFK
research DQG and DSSOLFDWLRQ
application RI ofILQHWXQLQJ
fine-tuning //0VLLMs KDYH
have
HGXFDWLRQ
education. PDGH
made VLJQLILFDQW
significant SURJUHVV
progress LQ in PDQ\
many YHUWLFDOV
verticals, LQFOXGLQJ
including
KHDOWKFDUH
healthcare, ODZ
law, ILQDQFH
finance, DQG and WKH
the DUWV
arts. +RZHYHU
However, //0VLLMs DUHare
.H\ZRUGVODUJH
Keywords-large ODQJXDJH
language PRGHOV
models; ILQHWXQLQJ
fine-tuning; VXUYH\
survey; VWLOOLQWKHLULQIDQF\LQHGXFDWLRQDQGWKHUHLVDQXUJHQWQHHG
still in their infancy in education, and there is an urgent need
HGXFDWLRQDOWHFKQRORJ\HGXFDWLRQDODSSOLFDWLRQ
educational technology; educational application IRUUHOHYDQWEDVLFUHVHDUFKDQGDSSOLHGLQQRYDWLRQ7KHPRVW
for relevant basic research and applied innovation. The most
VLJQLILFDQW
significant DGYDQWDJH
advantage RI of ILQHWXQLQJ
fine-tuning //0V LLMs LV is WKHLU
their
,
I. ,INTRODUCTION
1752'8&7,21 FRQYHUVDWLRQDO
conversational LQWHUDFWLYLW\
interactivity DQG and SHUIRUPDQFH
performance FRPSDUDEOH
comparable WR to
:LWKWKHUDSLGGHYHORSPHQWRI$,WHFKQRORJ\WKHLPSDFW WKHKXPDQOHYHOLQFRJQLWLYHWDVNVLQYDULRXVILHOGVLQFOXGLQJ
the human level in cognitive tasks in various fi e lds, including
With the rapid development of AI technology, the impact HGXFDWLRQ7KHULVH
RIODUJHODQJXDJHPRGHOV //0V ZKLFKLVHSRFKPDNLQJLQ education. The rise RI ofILQHWXQLQJ
fine-tuning //0V LLMs KDV has JUHDWSRWHQWLDO
great potential
of large language models (LLMs), which is epoch-making in WR LPSURYH WKH HIILFLHQF\ DQG HIIHFWLYHQHVV RI HGXFDWLRQDO
$, to improve the efficiency and effectiveness of educational
AI, KDV
has JUDGXDOO\
gradually HPHUJHG
emerged LQ in YDULRXV
various LQGXVWULHV
industries GXHdue WR
to LWV
its ZRUN
EUHDNWKURXJKV LQ QDWXUDO ODQJXDJH SURFHVVLQJ FRPSXWHU work, SURYLGLQJ
providing QHZ new GHYHORSPHQW
development LGHDV ideas IRU
for XSJUDGLQJ
upgrading
breakthroughs in natural language processing, computer HGXFDWLRQDOLQWHOOLJHQFH
YLVLRQ URERWLFV DQG RWKHU WHFKQLFDO ILHOGV &XUUHQWO\ WKH educational intelligence.
vision, robotics, and other technical fields. Currently, the
GLJLWDOWUDQVIRUPDWLRQDQGLQWHOOLJHQWXSJUDGLQJRIHGXFDWLRQ
digital transformation and intelligent upgrading of education ,, %BASIC
$6,&&
II. OF)FINE-TUNING
21&(3762)
CONCEPTS ,1(781,1*//0
LLMS6
DUH
are DFFHOHUDWLQJ
accelerating LQ in HGXFDWLRQ
education. $V As Da UHVXOW
result, //0V
LLMs DUHare
JUDGXDOO\
gradually being applied to critical aspects VXFK
EHLQJ DSSOLHG WR FULWLFDO DVSHFWV such DV $
as A. 'HYHORSPHQWRI//0V
Development ofLLMs
FRQVWUXFWLQJ
constructing HGXFDWLRQDO
educational HQYLURQPHQWV
environments, VXSSRUWLQJ
supporting WHDFKLQJ
teaching
SURFHVVHV $
A ODQJXDJH
language PRGHO
model LVis Da PRGHOLQJ
modeling RI of WKH
the SUREDELOLW\
probability
processes, DFFXUDWHO\
accurately HYDOXDWLQJ
evaluating WHDFKLQJ
teaching, DQG
and HIILFLHQWO\
efficiently GLVWULEXWLRQRIQDWXUDOODQJXDJH,QDJLYHQFRQWH[WODQJXDJH
PDQDJLQJ
managing HGXFDWLRQ
education. //0V
LLMs KDYH
have EHJXQ
begun Da FRPSUHKHQVLYH
comprehensive distribution of natural language. In a given context, language
DQG PRGHOVFDQHVWLPDWHWKHSUREDELOLW\RIDVHQWHQFHDSSHDULQJ
models can estimate the probability of a sentence appearing,
and GHHS
deep LQWHJUDWLRQ
integration ZLWK
with DOO
all DVSHFWV
aspects RIof HGXFDWLRQ
education DQG
and ZKLFK
WHDFKLQJ
teaching, FUHDWLQJ
creating Da QHZ
new IRUP
form RIof LQWHOOLJHQW
intelligent HGXFDWLRQ
education ZLWK
with which LVis XVHG
used WR
to PHDVXUH
measure WKHthe OLQJXLVWLF
linguistic UDWLRQDOLW\
rationality RI
of Da
LQWHOOLJHQW VHQWHQFH
sentence.
intelligent WHFKQRORJ\
technology FRYHULQJ
covering DOO
all DVSHFWV
aspects RI
ofHGXFDWLRQ
education DQG
and 7KH
SURPRWLQJ
promoting WKH the GHYHORSPHQW
development RI of XELTXLWRXV
ubiquitous OHDUQLQJ
learning DQG
and The GHYHORSPHQW
development RI ofODQJXDJH
language PRGHOV
models KDV
has JRQH
gone WKURXJK
through
SHUVRQDOL]HGOHDUQLQJ JUDPPDU
grammar UXOH
rule ODQJXDJH
language PRGHOV
models, VWDWLVWLFDOODQJXDJH
statistical language PRGHOV
models,
personalized learning. DQGQHXUDOODQJXDJHPRGHOV*UDPPDUUXOHODQJXDJHPRGHOV
//0VDUH$,V\VWHPVWUDLQHGRQPDVVLYHGDWDZLWKODUJH
LLMs are AI systems trained on massive data with large­ and neural language models. Grammar rule language models
VFDOHSDUDPHWHUVIRUQDWXUDOODQJXDJHSURFHVVLQJWDVNV7KH\ DUHEDVHGRQOLQJXLVWLFDQGGRPDLQNQRZOHGJHE\PDQXDOO\
are based on linguistic and domain knowledge by manually
scale parameters for natural language processing tasks. They GHVLJQLQJOLQJXLVWLF
DUH
are XVXDOO\
usually WUDLQHG
trained RQ
on ODUJHVFDOH
large-scale FRUSRUD
corpora VXFK
such DV
as ERRNV
books, designing linguistic JUDPPDUVEXW
grammars, but WKH\
they DUHGLIILFXOWWRGHDO
are difficult to deal
DUWLFOHV ZLWK
with ODUJHVFDOH
large-scale WH[WV
texts >@
[6]. $PRQJ
Among VWDWLVWLFDO
statistical ODQJXDJH
language
articles, DQG
and ,QWHUQHW
Internet FRQWHQW
content. /00V
LMMs, DOVRalso NQRZQ
known DV as PRGHOV
)RXQGDWLRQ0RGHOV>@FDQEHDSSOLHGWRYDULRXV$,PRGHOV
Foundation Models [ 1 ], can be applied to various AI models models, WKH
the PRVW
most UHSUHVHQWDWLYH
representative RQH
one LV
is QJUDP
n-gram >@
[7] , ZKRVH
whose
IRU PDLQ
main LGHD
idea LVis WR
to XVH
use VWDWLVWLFDO
statistical PHWKRGV
methods WR to SUHGLFW
predict WKH
the
for GLIIHUHQW
different WDVNV
tasks, VXFK
such DV
as *37
OPT -3 >@[2] DQG
and //D0D
LLaMa2 >@
[3], SUREDELOLW\RIWKHQH[WZRUGDSSHDULQJLQWKHWH[W
ZKLFK
which KDYH
have DFKLHYHG
achieved JUHDW
great VXFFHVV
success LQin WKH
the ILHOG
field RI
of QDWXUDO
natural probability of the next word appearing in the text.
ODQJXDJHSURFHVVLQJ,QDGGLWLRQPXOWLPRGDO//0VVXFKDV
language processing. In addition, multimodal LLMs such as

2474-3828/23/$31.00
2474-3828/23/$31 .00©2023
©2023IEEE
IEEE 718
718
DOI
DOl10.1109/ITME60234.2023.00148
1 0. 1 109/ITME60234.2023.00148
Authorized licensed use limited to: UNIVERSITY OF STRATHCLYDE. Downloaded on February 13,2025 at 19:03:36 UTC from IEEE Xplore. Restrictions apply.
 :LWK
With DGYDQFHG
advanced GHHSdeep OHDUQLQJ
learning, QHXUDO
neural QHWZRUNV
networks KDYHhave IRFXVHVRQDGDSWLQJGRZQVWUHDPWDVNVE\DGGLQJDOHDUQDEOH
focuses on adapting downstream tasks by adding a learnable
JUDGXDOO\
gradually been introduced into language model PRGHOLQJ
EHHQ LQWURGXFHG LQWR ODQJXDJH PRGHO modeling, QHWZRUNPRGXOHDGDSWHUWRWKHSUHWUDLQHGPRGHOEDVHGRQ
network module, adapter, to the pre-trained model based on
RSHQLQJ
opening Da QHZ
new SKDVH
phase NQRZQ
known DV as 1HXUDO
Neural /DQJXDJH
Language 0RGHOV
Models. WKH
the 7UDQVIRUPHU
Transformer DUFKLWHFWXUH
architecture. 7KH The DGDSWHU
adapter QHWZRUN
network LV is
7KH
The :RUG
Word (PEHGGLQJ
Embedding PHWKRG
method SURSRVHG
proposed E\by %HQJLR
Bengio >@
[8], VWUXFWXUHG
structured DV as D a ERWWOHQHFN
bottleneck, ZKLFK which GHFUHDVHV
decreases WKH the LQSXW¶V
input's
ZKLFKPDSVWKHXQLTXHRQHKRWHQFRGLQJRIZRUGVLQWRDORZ
which maps the unique one-hot encoding of words into a low GLPHQVLRQDOLW\
dimensionality, DSSOLHV applies Da QRQOLQHDU
nonlinear WUDQVIRUPDWLRQ
transformation, DQG and
GLPHQVLRQDO
dimensional GHQVH dense UHDO
real QXPEHU
number YHFWRU
vector E\
by FRQVWUXFWLQJ
constructing Da VXEVHTXHQWO\
subsequently UHFRQVWUXFWV
reconstructs WKH the LQSXW
input WRto LWV
its RULJLQDO
original KLJKhigh
VKDOORZ
shallow QHXUDO
neural QHWZRUN
network, KDV has SURIRXQGO\
profoundly LPSDFWHG
impacted WKH the GLPHQVLRQDOLW\
dimensionality. )LQDOO\ Finally, WKH the UHVLGXDOV
residuals DUH are DGGHG
added WR to WKH
the
GHYHORSPHQWRIODQJXDJHPRGHOV6LQFHWKHQQHXUDOQHWZRUN
development of language models. Since then, neural network XOWLPDWH
ultimate output via a residual connection. The adapter LV
RXWSXW YLD D UHVLGXDO FRQQHFWLRQ 7KH DGDSWHU is
DSSURDFKHV
approaches VXFK such DV
as 511
RNN DQG and /670
LSTM >@[9], ZKLFK
which XVHuse LQVHUWHG
inserted WZLFH
twice LQWRinto HYHU\
every 7UDQVIRUPHU
Transformer OD\HUlayer: RQFH
once DIWHU
after WKH
the
GLVWULEXWHGZRUGYHFWRUVWRPRGHOFRQWH[WXDOUHODWLRQVKLSVLQ
distributed word vectors to model contextual relationships in PXOWLKHDGDWWHQWLRQDOPDSSLQJDQGDJDLQIROORZLQJWKHWZR
multi-head attentional mapping and again following the two­
ODQJXDJH
language, KDYHhave EHJXQ
begun WR to HPHUJH
emerge. ,Q In UHFHQW
recent \HDUV
years, WKH
the OD\HU
layer IHHGIRUZDUG
feedforward QHXUDO neural QHWZRUN
network. 'XULQJ
During ILQHWXQLQJ
fine-tuning RQ on
H[SORVLYH
explosive growth of data, improved performance RI
JURZWK RI GDWD LPSURYHG SHUIRUPDQFH of VSHFLILF
specific GDWD
data, RQO\only WKHthe SDUDPHWHUV
parameters ZLWKLQwithin WKH
the DGDSWHU
adapter DUH are
KDUGZDUH
hardware GHYLFHV
devices, DQG
and WKH
the GHYHORSPHQW
development RI of VHOIVXSHUYLVHG
self-supervised XSGDWHG
updated, ZKLOH while WKH the SDUDPHWHUV
parameters RI of WKH
the SUHWUDLQHG
pre-trained PRGHOmodel
OHDUQLQJ>@WHFKQLTXHVKDYHPDGHLWSRVVLEOHWRWUDLQVXSHU
learning [10] techniques have made it possible to train super­ UHPDLQ
remain XQFKDQJHG
unchanged. 7KH The DGDSWHU
adapter WXQLQJ
tuning PHWKRG
method LV is ZLGHO\
widely
ODUJH
large VFDOH
scale QHXUDO
neural QHWZRUNEDVHG
network-based ODQJXDJH
language PRGHOV
models. (/0R
ELMo XVHG
used, DQGand VHYHUDO
several YDULDQWV
variants KDYH have EHHQ
been SURSRVHG
proposed. :DQJ
Wang HW et DO
al.
>@
[ 1 1 ] RSHQV
opens WKH
the GRRU
door WR
to SUHWUDLQHG
pre-trained PHWKRGV
methods IRUfor ODQJXDJH
language DSSOLHGWKHDGDSWHUWRWUDQVIHUOHDUQLQJDQGSURSRVHGWKH.
applied the adapter to transfer learning and proposed the K­
PRGHOV
models. 7KH The DGYHQW
advent RI of ODUJHVFDOH
large-scale SUHWUDLQHG
pre-trained ODQJXDJH
language DGDSWHU
adapter PHWKRGmethod WR to VROYH
solve WKH the SUREOHP
problem RI of FDWDVWURSKLF
catastrophic
PRGHOV
models such as BERT [ 1 2] and GPT-4 EDVHG
VXFK DV %(57 >@ DQG *37 based RQ
on WKH
the IRUJHWWLQJ
forgetting during new knowledge injection [ 1 5 ] . 3IHLIIHU
GXULQJ QHZ NQRZOHGJH LQMHFWLRQ >@ Pfeiffer HW et
7UDQVIRUPHUDUFKLWHFWXUH>@KDVPDGHWKHSUHWUDLQLQJDQG
Transformer architecture [ 1 3] has made the pre-training and DOSURSRVHGWKH$GDSWHU)XVLRQPHWKRGWRDFKLHYHPD[LPXP
al. proposed the AdapterFusion method to achieve maximum
ILQHWXQLQJ
fine-tuning SDUDGLJP
paradigm Da PDLQVWUHDP
mainstream UHVHDUFK
research GLUHFWLRQ
direction. 7KH
The WDVNWUDQVIHUEHWZHHQPXOWLSOHDGDSWHUPRGXOHV>@
task transfer between multiple adapter modules [ 1 6] .
SUHWUDLQLQJ DSSURDFK LQYROYHV WUDLQLQJRQ D
pre-training approach involves training on a vast amount ofYDVWDPRXQWRI 3URPSWWXQLQJ
Prompt-tuning >@ [ 1 7 ] , SURSRVHG
proposed E\ b y /HVWHU
Lester, $O5IRX
Al-Rfou, DQG and
GDWD
data WRto KHOS
help WKH
the PRGHO
model OHDUQ
learn KRZ
how WRto H[WUDFW
extract IHDWXUHV
features. 7KHQ
Then, &RQVWDQWIRUOHDUQLQJVRIWSURPSWVLQYROYHVVSOLFLQJDWDVN
Constant for learning soft prompts, involves splicing a task­
WKH
the PRGHO
model LV
is ILQHWXQHG
fine-tuned EDVHG
based RQon WKH
the VSHFLILF
specific REMHFWLYHV
obj ectives RI
of VSHFLILFFRQWLQXRXVDQGOHDUQDEOHSUHIL[WHQVRUDWWKHIURQW
specific, continuous, and learnable prefix tensor at the front
WKH
the WDVN
task. 7KLV
This PHDQV
means WKDW
that WKH
the SUHWUDLQHG
pre-trained PRGHO
model LVis WUDLQHG
trained HQG
end RI of WKH
the LQSXW
input HPEHGGLQJ
embedding RI of Da SUHWUDLQHG
pre-trained //0LLM. 7KLV This
ZLWKODEHOHGGDWDWKDWLVVSHFLILFWRWKHWDVNZKLFKDOORZVIRU
with labeled data that is specific to the task, which allows for DSSURDFK
approach HQDEOHVenables DGDSWDWLRQ
adaptation IRU for GRZQVWUHDP
downstream WDVNVtasks. 'XULQJ
During
WKHWUDQVIHURINQRZOHGJHIURPWKHSUHWUDLQHGPRGHOWRWKH
the transfer of knowledge from the pre-trained model to the WUDLQLQJ
training, WKH the JUDGLHQW
gradient GHVFHQWdescent DOJRULWKP
algorithm RSWLPL]HV
optimizes WKH the
GRZQVWUHDPWDVNLQDQHIIHFWLYHPDQQHU
downstream task in an effective manner. OHDUQDEOH
learnable prefix tensor on downstream WDVNV
SUHIL[ WHQVRU RQ GRZQVWUHDP tasks ZKLOH
while
PDLQWDLQLQJ
maintaining WKH the SUHWUDLQHG
pre-trained //0V¶ LLMs' SDUDPHWHUV
parameters LQ in Da IUR]HQ
frozen
%
B. 'LIIHUHQW$SSURDFKHVWR)LQHWXQLQJ
Different Approaches to Fine-tuning VWDWH
state. 3UHIL[WXQLQJ
Prefix-tuning, GHVLJQHG designed E\ by /L
Li DQG
and /LDQJ
Liang >@
[ 1 8], LV
is DQ
an
6XSHUYLVHG
Supervised ILQHWXQLQJ
fine-tuning, DOVR
also FDOOHG
called LQVWUXFWLRQ
instruction WXQLQJ
tuning, LPSURYHGPHWKRGRISURPSWWXQLQJZKLFKIXUWKHUVSOLFHVWKH
improved method of prompt-tuning, which further splices the
LQYROYHV
involves XVLQJ
using WDVNVSHFLILF
task-specific ODEHOHG
labeled GDWD
data WR
to ILQHWXQH
fine-tune Da SUH
pre­ OHDUQDEOHSUHIL[WHQVRUDWWKHIURQWHQGRIDOOKLGGHQVWDWHVLQ
learnable prefix tensor at the front end of all hidden states in
WUDLQHGPRGHOZKLFKDOORZVWKHPRGHOWRIROORZLQVWUXFWLRQV
trained model, which allows the model to follow instructions WKHPRGHOWRLPSURYHWKHVWDELOLW\RIWUDLQLQJ
the model to improve the stability of training.
DFFXUDWHO\
accurately. ,QVWUXFWLRQ
Instruction WXQLQJ
tuning LVis Da FRPPRQ
common PHWKRG
method IRUfor +RZHYHU
However, DOO all RIof WKH the DERYH
above PHWKRGV
methods KDYH have VRPH some
DGDSWLQJSUHWUDLQHG//0VWRGRZQVWUHDPWDVNV
adapting pre-trained LLMs to downstream tasks. GUDZEDFNV$GDSWHUWXQLQJDGGVDGGLWLRQDOPRGHOSDUDPHWHUV
drawbacks. Adapter tuning adds additional model parameters
7KHVXUJLQJQXPEHURISDUDPHWHUVLQ//0VVLJQLILFDQWO\
The surging number of parameters in LLMs significantly WKDW
that OHDG
lead WRto WKH
the LQIHUHQFH
inference ODWHQF\
latency SUREOHP
problem LQ in WKH
the LQIHUHQFH
inference
LQFUHDVHV
increases WKHthe PRGHO¶V
model 's SHUIRUPDQFH
performance DQG and JHQHUDWHV
generates DQ an SKDVH3URPSWWXQLQJLVGLIILFXOWWRRSWLPL]HLWVSHUIRUPDQFH
phase. Prompt tuning is difficult to optimize; its performance
HQRUPRXVGHPDQGIRUFRPSXWDWLRQDOUHVRXUFHV)RUH[DPSOH
enormous demand for computational resources. For example, YDULHV
varies QRQOLQHDUO\
nonlinearly ZLWK with WKH the VL]H
size RIof WKH
the WUDLQLQJ
training SDUDPHWHUV
parameters
*37
OPT-3 SURSRVHG
proposed E\ by 2SHQ$,
OpenAI KDV has %
17 5B SDUDPHWHUV
parameters DQGand >@
[ 1 9] . )XQGDPHQWDOO\
Fundamentally, SUHIL[HV prefixes UHGXFH
reduce WKHthe OHQJWK
length RI of WKH
the
UHTXLUHVDWOHDVWH)/23VRIFRPSXWDWLRQDQG*%
requires at least 3 . 1 4e23 FLOPs of computation and 700GB VHTXHQFHV
sequences used to process downstream tasks. Hu HW
XVHG WR SURFHVV GRZQVWUHDP WDVNV +X et DO
al.
RIJUDSKLFVPHPRU\VSDFHIRUWUDLQLQJXQGHUVLQJOHSUHFLVLRQ
of graphics memory space for training under single precision K\SRWKHVL]HG
hypothesized WKDW that WKH
the XSGDWHG
updated YDOXHV
values RI of PRGHO
model ZHLJKWV
weights
>@
[2]. 7KH
The WUDLQLQJ
training RI of *37
OPT -3 UHTXLUHV
requires SDUDOOHOL]DWLRQ
parallelization RQon DFWXDOO\
actually KDYHhave Da ORZHU
lower LQWULQVLF
intrinsic UDQN
rank DQG
and WKXV
thus SURSRVHG
proposed Da
WKRXVDQGV
thousands RI of KLJKSHUIRUPDQFH
high-performance *38V OPUs, FRVWLQJ
costing PLOOLRQV
millions RIof PHWKRG
method called LoRA [ 1 9], which adds a branch network WR
FDOOHG /R5$ >@ ZKLFK DGGV D EUDQFK QHWZRUN to
GROODUV
dollars >@
[2]. ,W
It FDQ
can EH
be VHHQ
seen WKDW
that WKH
the KLJK
high H[SHQVH
expense RIof IXOO\
fully DOO
all IXOO\
fully FRQQHFWHG
connected OD\HUV layers LQ in WKH
the SUHWUDLQHG
pre-trained //0V
LLMs. 7KLV This
WUDLQLQJ
training //0V
LLMs LV is GLIILFXOW
difficult WR
to XQGHUWDNH
undertake IRU
for PRVW
most HQWHUSULVHV
enterprises QHWZRUN
network XWLOL]HV
utilizes WKH the SURGXFW
product RI of WZR
two UDQN
rank GHFRPSRVLWLRQ
decomposition
DQGODERUDWRULHV
and laboratories. PDWULFHV
matrices to approximate the updated YDOXHV
WR DSSUR[LPDWH WKH XSGDWHG values RI of IXOO\
fully
7UDGLWLRQDOWUDQVIHUOHDUQLQJPHWKRGVUHTXLUHILQHWXQLQJ
Traditional transfer learning methods require fine-tuning FRQQHFWHG
connected OD\HU layer ZHLJKWV
weights LQ in GRPDLQ
domain DGDSWDWLRQ
adaptation. 7KHUHIRUH
Therefore,
DOO
all SDUDPHWHUV
parameters RI of WKH
the SUHWUDLQHG
pre-trained PRGHO
model, FDOOHG
called IXOO
full ILQH
fine­ RQO\
only WKHthe SDUDPHWHUV
parameters LQ in WKLV
this EUDQFK
branch QHWZRUN
network QHHG need WR to EH
be
WXQLQJ
tuning. +RZHYHU
However, WKLV this PHWKRG¶V
method's FRPSXWDWLRQDO
computational FRVW cost LVis XSGDWHG
updated GXULQJ
during WUDLQLQJ
training. 7KH The SURSRVDO
proposal RI of /R5$
LoRA DWWUDFWHG
attracted
H[FHHGLQJO\KLJKIRUWKHODUJHQXPEHURI//0VSDUDPHWHUV
exceedingly high for the large number of LLMs parameters. PXFK
much DWWHQWLRQ
attention, DFFRUGLQJ
according WR to ZKLFK
which UHVHDUFKHUV
researchers KDYH have
,QFRQWUDVWSDUDPHWHUHIILFLHQWILQHWXQLQJ
In contrast, parameter-efficient fine-tuning (PEFT) 3()7 RQO\ILQH
only fine­ VXFFHVVLYHO\
successively SURSRVHG proposed PHWKRGV methods VXFK such DVas $GD/R5$
AdaLoRA >@ [20],
WXQHV
tunes a small or additional number of model SDUDPHWHUV
D VPDOO RU DGGLWLRQDO QXPEHU RI PRGHO parameters, 4/R5$>@DQG,QFUH/R5$>@
QLoRA [21], and IncreLoRA [22] .
IL[HV
fixes PRVW
most SUHWUDLQHG
pre-trained SDUDPHWHUV
parameters, JUHDWO\
greatly UHGXFHV
reduces %DVHG
Based RQ on WKH
the DQDO\VLV
analysis DERYHabove, 3()7PEFT FDQcan VLJQLILFDQWO\
significantly
FRPSXWDWLRQDODQGVWRUDJHFRVWVDQGH[SDQGVWKHDSSOLFDWLRQ
computational and storage costs, and expands the application UHGXFH
reduce WKH the WUDLQLQJ
training FRVWcost RI of //0V
LLMs, DFKLHYLQJ
achieving SHUIRUPDQFH
performance
UDQJHRISUHWUDLQHG//0V,QDGGLWLRQWKHDGYDQFHG3()7
range of pre-trained LLMs. In addition, the advanced PEFT FRPSDUDEOH
comparable WR to IXOO
full ILQHWXQLQJ
fine-tuning ZKLOH while RQO\
only UHTXLULQJ
requiring Da VPDOO
small
WHFKQRORJ\FDQDOVRDFKLHYHSHUIRUPDQFHFRPSDUDEOHWRIXOO
technology can also achieve performance comparable to full QXPEHU
number RI of DGGLWLRQDO
additional SDUDPHWHUV
parameters WR to DFKLHYH
achieve GRPDLQ
domain
ILQHWXQLQJ
fine-tuning. DGDSWDWLRQ
adaptation. ,Q In DGGLWLRQ
addition, 3()7 PEFT FDQ can DOOHYLDWH
alleviate WKH
the FDWDVWURSKLF
catastrophic
7KH
The FRPPRQO\
commonly XVHG used 3()7
PEFT PHWKRGV
methods LQFOXGH
include DGDSWHUV
adapters, IRUJHWWLQJLVVXHRINQRZOHGJHUHVXOWLQJIURPIXOOILQHWXQLQJ
forgetting issue of knowledge resulting from full fine-tuning,
VRIW
soft SURPSWV
prompts, DQG and ORZUDQN
low-rank DGDSWDWLRQ /R5$  $GDSWHU
adaptation (LoRA). Adapter WKHUHE\
thereby LPSURYLQJ
improving JHQHUDOL]DWLRQ
generalization. 1RWDEO\ Notably, 3()7PEFT LV is Da
WXQLQJ
tuning ZDV
was SURSRVHG
proposed E\ by +RXOVE\
Houlsby HWDO
et al. LQ
in >@
20 1 9 [ 14], ZKLFK
which

719
719
Authorized licensed use limited to: UNIVERSITY OF STRATHCLYDE. Downloaded on February 13,2025 at 19:03:36 UTC from IEEE Xplore. Restrictions apply.
IOH[LEOH
flexible DQG
and JHQHUDO
general SXUSRVH
purpose, ZKLFK
which FDQ
can DGDSW
adapt ZHOO
well WR
to %
B. +XPDQ$,,QWHUDFWLYH/HDUQLQJ
Human-A! Interactive Learning
GLIIHUHQWGRZQVWUHDPWDVNV
different downstream tasks. +XPDQ$,,QWHUDFWLYH/HDUQLQJLVJUDGXDOO\EHFRPLQJDQ
Human-A! Interactive Learning is gradually becoming an
LPSRUWDQWIRUPDQGFRPSRQHQWRIWHDFKLQJDFWLYLWLHVDQGWKH
important form and component of teaching activities, and the
,,,
III. ( '8&$7,21$/$
EDUCATIONAL OF )
33/,&$7,212)
APPLICATION INE-781,1*
F,1( TUNING //0 6
LLMS
ELJJHVW
biggest DGYDQWDJH
advantage RI of //0V
LLMs OLHVlies LQ
in WKHLU
their GLUHFW
direct LQWHUDFWLRQ
interaction
7RPHHWYDULRXVDSSOLFDWLRQUHTXLUHPHQWVLQHGXFDWLRQLW
To meet various application requirements in education, it FDSDELOLW\$QLPSRUWDQWDGYDQWDJHRIFRPELQLQJLQWHUDFWLRQ
capability. An important advantage of combining interaction
LVQHFHVVDU\ILUVWWRFRQVWUXFWJHQHUDOSXUSRVH//0V7KHVH
is necessary first to construct general-purpose LLMs. These ZLWK
with WHDFKLQJ
teaching LV is WKDW
that SHUVRQDOL]HG
personalized HOHPHQWV
elements FDQcan EH
be EHWWHU
better
//0VDOORZILQHWXQLQJRQGRZQVWUHDPWDVNVWRIRUPWKUHH
LLMs allow fine-tuning on downstream tasks to form three LQWHJUDWHGLQWRWKHOHDUQLQJSURFHVV
integrated into the learning process.
W\SLFDO
typical DSSOLFDWLRQV
applications: DXWRPDWLF
automatic JHQHUDWLRQ
generation RI
of WHDFKLQJ
teaching 3DWDUDQDXWDSRUQ
Pataranautaporn HW et DO
al., EDVHG
based RQ on WKH
the *$1
GAN DUFKLWHFWXUH
architecture,
UHVRXUFHV
resources, KXPDQ$,
human-A! LQWHUDFWLYH
interactive OHDUQLQJ
learning, DQG
and WHDFKLQJ
teaching XVHG
used $,JHQHUDWHG
AI-generated DQLPDWHG
animated FKDUDFWHUV
characters WR to LQWHUDFW
interact ZLWK
with
LQWHOOLJHQW
intelligent DVVLVWDQFH
assistance. 7KH
The SURFHVV
process LQYROYHV
involves FROOHFWLQJ
collecting OHDUQHUV
learners >@
[29] . 'RQJ
Dong HW
et DO
al. XVHG
used //0
LLM DV as D
a SDUWQHU
partner WR
to WDFNOH
tackle
PDVVLYH
massive DPRXQWV
amounts RI of GDWD
data DQG
and NQRZOHGJH
knowledge IURP
from ERWK
both JHQHUDO
general FRPSOH[
complex VFLHQWLILF
scientific FKDOOHQJHV
challenges, LQWURGXFLQJ
introducing D a IUDPHZRUN
framework
DQG
and HGXFDWLRQDO
educational ILHOGV
fields, VXFK
such DVas VXEMHFW
subject NQRZOHGJH
knowledge, FDOOHG
called 6RFUDWLF
Socratic UHDVRQLQJ
reasoning DQG
and SURSRVLQJ
proposing D a SDUDGLJP
paradigm QDPHG
named
DVVLJQPHQWV
assignments, H[DP
exam SDSHUV
papers, 022&V
MOOCs, WHDFKLQJ
teaching WKHRULHV
theories, HWF
etc., //0IRU6FLHQFH>@,QHGXFDWLRQILQHWXQLQJ//0VZLWK
LLM for Science [30]. In education, fine-tuning LLMs with
DQGWKHQXVLQJVHOIVXSHUYLVHGOHDUQLQJWRSUHWUDLQRQWKHVH
and then using self-supervised learning to pre-train on these D
a 6RFUDWLF
Socratic WXWRULQJ
tutoring PRGH
mode FDQ can HQFRXUDJH
encourage VWXGHQWV
students WRto WKLQN
think
GDWD7KHILQHWXQLQJ//0VREWDLQHGLQWKLVZD\FDQGHHSO\
data. The fine-tuning LLMs obtained in this way can deeply LQGHSHQGHQWO\
independently DQG and JXLGH
guide WKHP
them WRto ILQG
find DQVZHUV
answers E\ by DVNLQJ
asking
XQGHUVWDQG
understand WKHthe WKUHH
three HGXFDWLRQDO
educational HOHPHQWV
elements RI
of WHDFKLQJ
teaching DSSURSULDWH
appropriate TXHVWLRQV
questions. $OWKRXJK
Although DSSURSULDWH
appropriate TXHVWLRQV
questions DUHare
UHVRXUFHV
resources, WHDFKLQJ
teaching REMHFWV
objects, DQG
and WHDFKLQJ
teaching SURFHVVHV
processes WR
to VHUYH
serve HVVHQWLDO
essential, HYDOXDWLQJ
evaluating KRZ
how VWXGHQWV
students UHVSRQG
respond WR to DQG
and LQWHUDFW
interact
DQGVXSSRUWHGXFDWLRQDOSDUWLFLSDQWVEHWWHU
and support educational participants better. ZLWK
with WKHVH
these TXHVWLRQV
questions LVis DOVR
also QHFHVVDU\
necessary. $EGHOJKDQL
Abdelghani HW et DO
al.
$ VWXGLHGWKHLPSDFWRISURPSWOHDUQLQJRQWKHTXHVWLRQDVNLQJ
studied the impact of prompt learning on the question-asking
A. $XWRPDWLF*HQHUDWLRQRI7HDFKLQJ5HVRXUFHV
Automatic Generation of Teaching Resources
EHKDYLRU
behavior RI of VWXGHQWV
students. 7KH\
They IRXQG
found WKDW
that VXFK
such DXWRPDWLF
automatic
7KH
The FRQFHSW
concept RI of XVLQJ
using DUWLILFLDO
artificial LQWHOOLJHQFH
intelligence V\VWHPV
systems WR to SURPSWV
prompts JHQHUDOO\
generally KDYH
have SRVLWLYH
positive HIIHFWV
effects, VXFK
such DVas HQKDQFLQJ
enhancing
DXWRPDWLFDOO\JHQHUDWHHGXFDWLRQDOUHVRXUFHVLVQRWQHZDQG
automatically generate educational resources is not new, and VWXGHQWV¶FXULRVLW\DQGOHDUQLQJLQLWLDWLYH>@
students ' curiosity and learning initiative [3 1 ] .
GLVFXVVLRQVRQDOJRULWKPJHQHUDWHGOHDUQLQJPDWHULDOVFDQEH
discussions on algorithm-generated learning materials can be ,Q
In VXP
sum, ILQHWXQLQJ
fine-tuning //0V
LLMs LQ in VSHFLILF
specific ILHOGV
fields WKURXJK
through
WUDFHG
traced EDFN
back WR to WKH
the V
1 970s >@
[23 ] . ,Q
In UHFHQW
recent \HDUV
years, WKH
the UDSLG
rapid KXPDQ$,
human-A! LQWHUDFWLRQ
interaction FDQ
can SURYLGH
provide SHUVRQDOL]HG
personalized OHDUQLQJ
learning
GHYHORSPHQW
development RI of //0V
LLMs KDV has EURXJKW
brought SRZHUIXO
powerful JHQHUDWLYH
generative H[SHULHQFHV,QWKLVUHJDUGUROHSOD\LQJDXWRPDWLFIHHGEDFN
experiences. In this regard, role-playing, automatic feedback,
DELOLWLHV
abilities, PDNLQJ
making LW it Da WRRO
tool IRU
for VXSSOHPHQWLQJ
supplementing WHDFKLQJ
teaching VRFLDO
social LQWHUDFWLRQ
interaction, DQGand RWKHU
other PHWKRGV
methods FDQ can HQFRXUDJH
encourage
UHVRXUFHVDQGEURDGHQLQJWKHDSSOLFDWLRQRIUHODWHGILHOGV
resources and broadening the application of related fields. H[SORUDWLRQDQGTXHVWLRQLQJIRFXVLQJRQOHDUQHUV¶FRJQLWLYH
exploration and questioning, focusing on learners ' cognitive
5HJDUGLQJ
Regarding WKH the DXWRPDWLF
automatic JHQHUDWLRQ
generation RI of WHDFKLQJ
teaching VWDWHLQWHQWLRQVDQGWHDFKLQJRULHQWHGLQWHUDFWLRQV7KLVZLOO
state, intentions, and teaching-oriented interactions. This will
UHVRXUFHV
resources, LQQRYDWLYH
innovative WHFKQRORJLHV
technologies EDVHG based RQ
on //0V
LLMs KDYH
have KHOS
help OHDUQHUV
learners HQKDQFH
enhance WKHLU
their NQRZOHGJH
knowledge DQG and DFKLHYH
achieve DQ an
GHPRQVWUDWHG
demonstrated FHUWDLQ
certain FDSDELOLWLHV
capabilities. ,PDJH
Image JHQHUDWLRQ
generation PRGHOV
models, HIILFLHQWKXPDQ$,OHDUQLQJSURFHVV
efficient human-A! learning process.
VXFKDV6WDEOH'LIIXVLRQFDQJHQHUDWHDUWWHDFKLQJUHVRXUFHV
such as Stable Diffusion, can generate art teaching resources
ZLWK
with YDULRXV
various VW\OHV
styles, QRYHOW\
novelty, XQLTXHQHVV
uniqueness, DQG and DHVWKHWLFV
aesthetics E\by &
C. 7HDFKLQJ,QWHOOLJHQW$VVLVWDQFH
Teaching Intelligent Assistance
LQSXWWLQJ
inputting WH[W
text GHVFULSWLRQV
descriptions RI of LPDJHV
images EDVHG
based RQ on WHDFKLQJ
teaching 7KHFXUUHQWEDVLF//0VDOUHDG\SRVVHVVVWURQJSUREOHP
The current basic LLMs already possess strong problem­
QHHGV
needs. ,Q
In WHUPV
terms RI of WH[W
text UHVRXUFH
resource JHQHUDWLRQ
generation, WKHUH
there LV
is OLWWOH
little VROYLQJ
solving FDSDELOLWLHV
capabilities DQG
and FDQcan EHbe IXUWKHU
further ILQHWXQHG
fine-tuned LQin
GLIIHUHQFH
difference EHWZHHQ
between DEVWUDFWV
abstracts JHQHUDWHG
generated E\ by //0V
LLMs DQGand WKRVH
those HGXFDWLRQ
education WRH[SDQG
to expand WKHLUDELOLW\
their ability WRto DVVLVW
assist LQWHDFKLQJ
in teaching. )LQH
Fine­
SURGXFHG
produced E\ by SHRSOH
people. 'HHS0LQG
DeepMind DQG and 6WDQIRUG
Stanford 8QLYHUVLW\
University WXQLQJ
tuning //0V
LLMs FDQcan SOD\
play D a VLJQLILFDQW
significant UROH
role, HVSHFLDOO\
especially LQ
in
SURSRVHG'UDPDWURQ>@DWH[WJHQHUDWLRQPRGHOZKLFKFDQ
proposed Dramatron [24], a text-generation model which can DSSO\LQJ
applying HGXFDWLRQDO
educational WKHRU\
theory, DXWRPDWLFDOO\
automatically JUDGLQJ
grading WHVW
test
JHQHUDWH
generate VSHFLILF
specific DQG and YLYLG
vivid VFULSW
script FRQWHQW
content. ,QIn DGGLWLRQ
addition, TXHVWLRQVDQGPDQDJLQJWHDFKLQJSURFHVVHV
questions, and managing teaching processes.
*RRJOH¶V
Google's 0XVLF/0
MusicLM >@ [25] FDQ
can JHQHUDWH
generate KLJKTXDOLW\
high-quality PXVLF
music =L\XH
Ziyue LV
is DQ
an HGXFDWLRQDO
educational //0
LLM UHOHDVHGE\
released by 1<6('$2
NYSE: DAO,
FOLSV
clips GLUHFWO\
directly EDVHG
based RQ on QDWXUDO
natural ODQJXDJH
language GHVFULSWLRQV
descriptions. )LQH
Fine­ HTXLSSHG
equipped ZLWKwith YDULRXV
various IXQFWLRQV
functions VXFK
such DVas //0VEDVHG
LLMs-based
WXQLQJ
tuning //0V
LLMs DUH are DOVR
also PDNLQJ
making FRQWLQXRXV
continuous SURJUHVV
progress LQ in WUDQVODWLRQ
translation, YLUWXDO
virtual RUDO
oral FRDFKLQJ
coaching, DQG
and $,
AI HVVD\
essay JXLGDQFH
guidance WR
to
DSSOLFDWLRQV
applications WKDW
that UHTXLUH
require VWURQJ
strong ORJLFDO
logical UHDVRQLQJ
reasoning DELOLWLHV
abilities. DVVLVW
assist OHDUQLQJ
learning. .KDQ
Khan $FDGHP\
Academy LV is D
a QRQSURILW
non-profit HGXFDWLRQDO
educational
5HVHDUFKHUV
Researchers DW at 5LFH
Rice 8QLYHUVLW\
University SURSRVH
propose JHQHUDWLQJ
generating LQVWLWXWLRQ
institution DFWLYHO\
actively UHVHDUFKLQJ
researching KRZ how WRto DSSO\
apply //0V
LLMs LQin
PXOWLGLVFLSOLQDU\
multidisciplinary DQG and KLJKTXDOLW\
high-quality TXHVWLRQV
questions WKDW
that FDQ
can EH be .KDQPLJRWRRSWLPL]HRQOLQHWHDFKLQJ(GX&KDWLVDQ//0V
Khanmigo to optimize online teaching. EduChat is an LLMs­
GLUHFWO\
directly DSSOLHG
applied WR to WHDFKLQJ
teaching, XWLOL]LQJ
utilizing *37
OPT -3 DQGand SURPSW
prompt EDVHGHGXFDWLRQDOFKDWURERWV\VWHPGHYHORSHGE\'DQHWDO
based educational chat robot system developed by Dan et al.
ILQHWXQLQJ
fine-tuning >@
[26] . ,Q
In WKH
the SURJUDPPLQJ
programming FRXUVH course, WKH
the PRVW
most >@
[32] . 7KLV
This FKDW
chat URERW
robot V\VWHP
system LV is JXLGHG
guided E\by SV\FKRORJ\
psychology DQG
and
DGYDQFHG
advanced //0 LLM, &RGH[ Codex, FDQ can JHQHUDWH
generate UHDVRQDEOH
reasonable HGXFDWLRQDO
educational WKHRULHV
theories. ,W
It OHDUQV
learns GRPDLQVSHFLILF
domain-specific NQRZOHGJH
knowledge
SURJUDPPLQJ
programming H[HUFLVHV
exercises IRU for VWXGHQWV
students DQGand SURYLGH
provide H[DPSOHV
examples WKURXJK
through SUHWUDLQLQJ
pre-training RQ
on HGXFDWLRQDO
educational FRUSRUD
corpora DQG
and ILQHWXQLQJ
fine-tuning
DQG
and DFFXUDWH
accurate FRGH code H[SODQDWLRQV
explanations >@ [27] . 7KH
The HGXFDWLRQDO
educational RQ
on GHVLJQHG
designed V\VWHP
system SURPSWV
prompts DQG and LQVWUXFWLRQV
instructions. 7KLV
This IXUWKHU
further
WHFKQRORJ\
technology GHYHORSHG
developed WKURXJK through ILQHWXQLQJ
fine-tuning &RGH[
Codex DOVR
also HQKDQFHV
enhances HGXFDWLRQDO
educational IXQFWLRQV
functions VXFK
such DVas RSHQ
open TXHVWLRQ
question
GHPRQVWUDWHV
demonstrates WKH the DELOLW\
ability WR to VROYH
solve 
81% RI of DGYDQFHG
advanced DQVZHULQJ
answering, HVVD\essay DVVHVVPHQW
assessment, 6RFUDWLF
Socratic WHDFKLQJ
teaching, DQG
and
PDWKHPDWLFVSUREOHPV>@
mathematics problems [28]. HPRWLRQDOVXSSRUW
emotional support.
,Q
In VXPPDU\
summary, EDVHG based RQ on H[LVWLQJ
existing //0V
LLMs, ILQHWXQLQJ
fine-tuning IRUfor )LQHWXQLQJ//0VKDVVKRZQJUHDWSRWHQWLDOLQWHDFKLQJ
Fine-tuning LLMs has shown great potential in teaching
DXWRPDWLF
automatic JHQHUDWLRQ
generation RI of WHDFKLQJ
teaching UHVRXUFHV
resources LVis H[SHFWHG
expected WR to LQWHOOLJHQW
intelligent DVVLVWDQFH
assistance, DQG
and WKHUH
there LVis VWLOO
still PXFK
much URRP
room IRU
for
PDNHFRQWLQXRXVSURJUHVVLQIXQFWLRQDOLW\DQGSHUIRUPDQFH
make continuous progress in functionality and performance. H[SORUDWLRQLQERWKWHFKQRORJ\DQGDSSOLFDWLRQGRPDLQV
exploration in both technology and application domains.
(VSHFLDOO\
Especially LQ in WHUPV
terms RI of SHUVRQDOL]DWLRQ
personalization, LQVSLUDWLRQ
inspiration,
PXOWLPRGDOLW\
multimodality, DQG and LQWHUGLVFLSOLQDULW\
interdisciplinarity, LW it ZLOO
will SURYLGH
provide PRUH
more
SRVVLELOLWLHVIRUSURPRWLQJWKHGLJLWL]DWLRQDQGLQWHOOLJHQFHRI
possibilities for promoting the digitization and intelligence of
HGXFDWLRQ
education.

720
720

Authorized licensed use limited to: UNIVERSITY OF STRATHCLYDE. Downloaded on February 13,2025 at 19:03:36 UTC from IEEE Xplore. Restrictions apply.
 ,9 CHALLENGES
IV. &+$//(1*(62) ),1(781,1*LLMS
OF FINE-TUNING //06,1 ('8&$7,21
IN EDUCATION ZKLFKthey
which WKH\can
FDQobj
REMHFWLYHO\ DGMXVWthe
ectively adjust WKHteaching
WHDFKLQJplan.SODQThese
7KHVH
IDFWRUV ZLOO GLPLQLVK WKH TXDOLW\ RI WHDFKLQJ
factors will diminish the quality of teaching in numerous LQ QXPHURXV
$ Technical
A. 7HFKQLFDO&KDOOHQJHV
Challenges ZD\V and
ways DQG directly
GLUHFWO\ impact
LPSDFW the WKH conventional
FRQYHQWLRQDO educational
HGXFDWLRQDO
,QWKHSURFHVVRIJUDGXDOO\DSSO\LQJILQHWXQLQJ//0VWR SURFHVVDQGV\VWHP
process and system.
In the process of gradually applying fine-tuning LLMs to
WKHeducation
HGXFDWLRQfiILHOG WKHUHare DUHsome
VRPHcorresponding
FRUUHVSRQGLQJtechnical
WHFKQLFDO 9DULRXV factors,
Various IDFWRUV including
LQFOXGLQJ training
WUDLQLQJ data,
GDWD influence
LQIOXHQFH the WKH
the eld, there
FKDOOHQJHVDue 'XHto WRthe
WKHtraining
WUDLQLQJmechanism
PHFKDQLVPof RILLMs,
//0Vthey WKH\ ELDV of
bias RI LLMs.
//0V Gebru,
*HEUX aD renowned
UHQRZQHG expert H[SHUW in LQ artifi
DUWLILFLDO
cial
challenges.
UDUHO\ involve
LQYROYH comprehension-level
FRPSUHKHQVLRQOHYHO content FRQWHQW and DQG exhibit
H[KLELW LQWHOOLJHQFH ethics,
intelligence HWKLFV has
KDV pointed
SRLQWHG out RXW thatWKDW itLW isLV almost
DOPRVW
rarely
VLJQLILFDQW OLPLWDWLRQVin LQsome
VRPHrespects,
UHVSHFWVsuch
VXFKas DVreasoning,
UHDVRQLQJ LPSRVVLEOHWRFRPSOHWHO\HOLPLQDWHFHUWDLQVRFLDOELDVHVVXFK
impossible to completely eliminate certain social biases, such
signifi cant limitations
VHOIDZDUHQHVV emotions,HPRWLRQV intuition,
LQWXLWLRQ responsibility,
UHVSRQVLELOLW\ and DQG DVWKRVHUHODWHGWRSROLWLFVHWKQLFLW\JHQGHUFODVVDQGDJH
as those related to politics, ethnicity, gender, class, and age,
self-awareness,
PRUDOLW\)XUWKHUPRUHNQRZOHGJHLQWKHILHOGRIHGXFDWLRQLV IURP training
from WUDLQLQJ data
GDWD and
DQG models
PRGHOV [33 >@ 7KHUHIRUH in
] . Therefore, LQ the
WKH
morality. Furthermore, knowledge in the field of education is
FRQVWDQWO\ evolving.
HYROYLQJ Since 6LQFH LLMs//0V do GR not
QRW have
KDYH real-time
UHDOWLPH DSSOLFDWLRQ RI HGXFDWLRQ LW LV FUXFLDO
application of education, it is crucial for educators to IRU HGXFDWRUV WR
constantly
LQWHUQHW access,
DFFHVV they WKH\ cannot
FDQQRW learnOHDUQ the
WKH most
PRVW up-to-date
XSWRGDWH VXSHUYLVH and
supervise DQG define
GHILQH the
WKH scope
VFRSH of RI the
WKH model'
PRGHO¶V XVDJH to
s usage WR
internet
LQIRUPDWLRQ Consequently,
&RQVHTXHQWO\ their WKHLU knowledge
NQRZOHGJH base EDVH remains
UHPDLQV SUHYHQW any
prevent DQ\ adverse
DGYHUVH impact
LPSDFW on RQ learners
OHDUQHUV¶ LQGHSHQGHQW
' independent
information.
IXQGDPHQWDOO\ OLPLWHG //0V DUH PDLQO\ SUHWUDLQHG RQ WKLQNLQJ and
thinking DQG cognitive
FRJQLWLYH processes.
SURFHVVHV The 7KH problems
SUREOHPV in LQ
fundamentally limited. LLMs are mainly pre-trained on
PDVVLYH unlabeled
XQODEHOHG data, GDWD andDQG even
HYHQ after
DIWHU fine-tuning,
ILQHWXQLQJ itLW isLV JHQHUDWLQJ teaching
generating WHDFKLQJ resources
UHVRXUFHV by E\ fiILQHWXQLQJ
ne-tuning LLMs //0V may PD\
massive
GLIILFXOW to WR avoid
DYRLG inherent
LQKHUHQW issuesLVVXHV such
VXFK as DV data
GDWD bias,
ELDV KDUPVWXGHQWV¶VRFLDOFRJQLWLRQDQGHWKLFV7HDFKHUVZKRXVH
harm students' social cognition and ethics. Teachers who use
difficult
LQWHOOHFWXDO SURSHUW\ DQG NQRZOHGJH DFFXUDF\ ,Q IDFW WKHVHUHVRXUFHVZLOODOVRHQFRXQWHUQHZSUHVVXUHVDQGULVNV
these resources will also encounter new pressures and risks.
intellectual property, and knowledge accuracy. In fact,
GLYHUVHandDQGhigh-quality
KLJKTXDOLW\trainingWUDLQLQJdataGDWDisLVUHODWLYHO\ VFDUFHin LQ 1DWXUDOlanguage
Natural ODQJXDJHdata GDWDused
XVHGby E\LLMs
//0Vfor IRUfiILQHWXQLQJ
ne-tuning in LQ
diverse relatively scarce
HGXFDWLRQ and DQG theWKH accuracy
DFFXUDF\ of RI training
WUDLQLQJ data
GDWD cannot
FDQQRW be EH HGXFDWLRQDOapplications
educational DSSOLFDWLRQVmay PD\contain
FRQWDLQSHUVRQDO
personal and DQGsensitive
VHQVLWLYH
education,
HIIHFWLYHO\ ensured.
HQVXUHG Consequently,
&RQVHTXHQWO\ fiILQHWXQLQJ //0V may PD\ LQIRUPDWLRQabout
information DERXW individuals'
LQGLYLGXDOV¶private
SULYDWH lives
OLYHVand
DQG identities.
LGHQWLWLHV
effectively ne-tuning LLMs
SURYLGH incorrect
LQFRUUHFW answers.
DQVZHUV Learners/HDUQHUV lacking
ODFNLQJ specialized
VSHFLDOL]HG 7KLVZLOOHDVLO\OHDGWRSULYDF\DQGGDWDVHFXULW\LVVXHVVXFK
This will easily lead to privacy and data security issues such
provide
NQRZOHGJH may PD\ be EH unable
XQDEOH to WR find
ILQG and
DQG correct
FRUUHFW these
WKHVH DVleakage,
as OHDNDJHunauthorized
XQDXWKRUL]HGaccess,
DFFHVVand DQGdata
GDWDabuse.
DEXVHTherefore,
7KHUHIRUH
knowledge
SUREOHPV OHDGLQJ WR SRWHQWLDO PLVJXLGDQFH :RUVH WKH HIIHFWLYHPHDVXUHVPXVWEHWDNHQWRHQVXUHVHFXULW\
effective measures must be taken to ensure security.
problems, leading to potential misguidance. Worse, the
KDOOXFLQDWLRQ problem
hallucination SUREOHP has KDV been
EHHQ widely
ZLGHO\ observed
REVHUYHG in LQ fiILQH
ne­ 9 DIRECTIONS
V. ',5(&7,216)25 )8785(LLMS
FOR FUTURE //06,1 ('8&$7,21
IN EDUCATION
WXQLQJ//0VZKHUHLQDFFXUDWHLQIRUPDWLRQLVIDEULFDWHGDQG
tuning LLMs, where inaccurate information is fabricated and
HVSRXVHG lucidly.
espoused OXFLGO\ These
7KHVH errors,
HUURUV biases,
ELDVHV and
DQG hallucinations
KDOOXFLQDWLRQV 7KH scaling
The VFDOLQJlaw ODZ of
RIlanguage
ODQJXDJH models
PRGHOV suggests
VXJJHVWVthatWKDWtheWKH
ZLOOmake
will PDNH itLWdifficult
GLIILFXOW for IRUlearners
OHDUQHUVto WRdiscern
GLVFHUQwhether
ZKHWKHUthey WKH\ PRGHO¶V
model SHUIRUPDQFH demonstrates
's performance GHPRQVWUDWHV aD linear
OLQHDU improvement
LPSURYHPHQW
KDYHEHHQDFFXUDWHO\LPSDUWHGNQRZOHGJH
have been accurately imparted knowledge. ZLWKH[SRQHQWLDOLQFUHDVHVLQWKHQXPEHURISDUDPHWHUVWKH
with exponential increases in the number of parameters, the
//0V are
LLMs DUH typically
W\SLFDOO\black EODFN box ER[ models
PRGHOV whose
ZKRVH internal
LQWHUQDO DPRXQWRIGDWDDQGWKHWUDLQLQJWLPH>@$FFRUGLQJWRWKLV
amount of data, and the training time [34]. According to this
GHFLVLRQPDNLQJSURFHVVHVDUHGLIILFXOWWRH[SODLQPDNLQJLW
decision-making processes are difficult to explain, making it ODZincreasing
law, LQFUHDVLQJthe WKHamount
DPRXQWof RItraining
WUDLQLQJdata
GDWDand
DQGexpanding
H[SDQGLQJ
FKDOOHQJLQJWRJDLQSHRSOH¶VWUXVWWUXO\&RQVHTXHQWO\QRQH
challenging to gain people' s trust truly. Consequently, none WKH VFDOH RI ODUJH PRGHOV LV D VWUDLJKWIRUZDUG
the scale of large models is a straightforward approach to DSSURDFK WR
RI the
of WKH existing
H[LVWLQJ fine-tuning
ILQHWXQLQJ LLMs //0V can FDQ beEH considered
FRQVLGHUHG AI $, LPSURYLQJLLMs'
improving //0V¶performance.
SHUIRUPDQFHCurrently,
&XUUHQWO\the
WKHtraining
WUDLQLQJdataGDWD
V\VWHPV that
systems WKDW areDUH totally
WRWDOO\ transparent
WUDQVSDUHQW to WR educational
HGXFDWLRQDO IRU//0VLVW\SLFDOO\DWWKH7%OHYHODQGPD\SURJUHVVWRWKH
for LLMs is typically at the TB level and may progress to the
VWDNHKROGHUV Additionally,
stakeholders. $GGLWLRQDOO\ real-time UHDOWLPH interaction
LQWHUDFWLRQ and DQG 3%OHYHOLQWKHIXWXUH,WLVDQWLFLSDWHGWKDWIXWXUH//0VZLOO
PB level in the future. It is anticipated that future LLMs will
IHHGEDFN JUHDWO\ LPSDFW OHDUQHUV¶
feedback greatly impact learners' initiative and learningLQLWLDWLYH DQG OHDUQLQJ DFKLHYHbreakthroughs
achieve EUHDNWKURXJKVin LQcapabilities
FDSDELOLWLHVthrough
WKURXJKthe WKH surge
VXUJHin LQ
HIIHFWin
effect LQthe
WKHteaching
WHDFKLQJprocess.SURFHVVHowever,
+RZHYHUdue GXHto WRreasoning
UHDVRQLQJ WUDLQLQJ data.
training GDWD In ,Q the
WKH fine-tuning
ILQHWXQLQJ process
SURFHVV ofRI educational
HGXFDWLRQDO
GHOD\fiILQHWXQLQJ
delay, ne-tuning LLMs //0V still VWLOOhave
KDYHcertain
FHUWDLQdefects
GHIHFWVin LQreal­
UHDO DSSOLFDWLRQV data
applications, GDWD quality,
TXDOLW\ structure,
VWUXFWXUH andDQG diversity
GLYHUVLW\ are DUH
WLPHLQWHUDFWLRQ7KHH[LVWLQJILQHWXQLQJ//0VDUHGLIILFXOW
time interaction. The existing fine-tuning LLMs are difficult LPSRUWDQWIDFWRUVDIIHFWLQJWKHPRGHO¶VSHUIRUPDQFH,QWKLV
important factors affecting the model' s performance. In this
WR effectively
to HIIHFWLYHO\ help KHOS learners
OHDUQHUV in LQ learning
OHDUQLQJ motivation
PRWLYDWLRQ and DQG UHJDUGwe
regard, ZHcan FDQcontinue
FRQWLQXHto WRexplore
H[SORUHdata
GDWDmining
PLQLQJtechniques
WHFKQLTXHV
HPRWLRQDO support.
emotional VXSSRUW More0RUH fundamentally,
IXQGDPHQWDOO\ in LQthe
WKH interactive
LQWHUDFWLYH DQG extract
and H[WUDFWmeaningful
PHDQLQJIXOand DQGvaluable
YDOXDEOHinformation
LQIRUPDWLRQ for IRU fiILQH
ne­
WHDFKLQJ between
teaching EHWZHHQ teachers
WHDFKHUV and DQG students,
VWXGHQWV teachers
WHDFKHUV and DQG WXQLQJ//0VIURPHGXFDWLRQDOELJGDWD
tuning LLMs from educational big data.
VWXGHQWVwill
students ZLOOalso
DOVRshare
VKDUHan DQemotional
HPRWLRQDOexperience
H[SHULHQFHand DQGmoral
PRUDO &XUUHQWLLMs
Current //0Vare DUHprimarily
SULPDULO\basedEDVHGonRQthe
WKHTransformer
7UDQVIRUPHU
UHVRQDQFH while
resonance ZKLOH imparting
LPSDUWLQJ knowledge.
NQRZOHGJH In ,Q this
WKLV regard,
UHJDUG the WKH UHJDUGLQJalgorithm
regarding DOJRULWKPand DQG model
PRGHOarchitecture.
DUFKLWHFWXUHIn ,Qthe
WKH future,
IXWXUH
JHQHUDO intelligence
general LQWHOOLJHQFH embodied
HPERGLHG by E\ LLMs
//0V isLV still
VWLOO far
IDU from
IURP LPSURYHPHQWVWRWKH7UDQVIRUPHURUWKHHPHUJHQFHRIRWKHU
improvements to the Transformer or the emergence of other
JHQXLQHKXPDQLQWHOOLJHQFH
genuine human intelligence. VXSHULRUDUFKLWHFWXUHVZLOOIXUWKHUHQKDQFHWKHFDSDELOLWLHVRI
superior architectures will further enhance the capabilities of
//0V During
LLMs. 'XULQJ training,
WUDLQLQJ innovations
LQQRYDWLRQV and DQG breakthroughs
EUHDNWKURXJKV in LQ
% Ethical
B. (WKLFDO,VVXHV
Issues VHOIVXSHUYLVHGOHDUQLQJDQGILQHWXQLQJDOJRULWKPVZLOODOVR
self-supervised learning and fine-tuning algorithms will also
3RZHUIXO technology
Powerful WHFKQRORJ\ isLV often
RIWHQ aD double-edged
GRXEOHHGJHG sword.
VZRUG LPEXH LLMs
imbue //0V with ZLWK new
QHZ capabilities.
FDSDELOLWLHV A $ possible
SRVVLEOH research
UHVHDUFK
,PSURSHU or
Improper RU misuse
PLVXVH of
RI LLMs
//0V will ZLOO lead
OHDG to
WR negative
QHJDWLYH GLUHFWLRQLVH[SORULQJZD\VWRHIIHFWLYHO\UHGXFHWKHVFDOHRI
direction is exploring ways to effectively reduce the scale of
DSSOLFDWLRQeffects.
application HIIHFWVWhen
:KHQfiILQHWXQLQJ
ne-tuning LLMs //0Vare DUHapplied
DSSOLHGin LQ //0VZKLOHPDLQWDLQLQJWKHLUSHUIRUPDQFHRQVSHFLILFWDVNV
LLMs while maintaining their performance on specific tasks,
HGXFDWLRQitLWisLV essential
education, HVVHQWLDOto
WRconduct
FRQGXFWrisk
ULVNassessments
DVVHVVPHQWVfrom
IURP E\OHYHUDJLQJWKHLUFDSDELOLWLHVLQIHZVKRWOHDUQLQJGRPDLQ
by leveraging their capabilities in few-shot learning, domain
YDULRXVdimensions
various GLPHQVLRQVsuch VXFKasDVscientifi
VFLHQWLILFLW\ IDLUQHVVaccuracy,
city, fairness, DFFXUDF\ JHQHUDOL]DWLRQ and
generalization, DQG task
WDVN generalization.
JHQHUDOL]DWLRQ Additionally,
$GGLWLRQDOO\
DQGYDOXHV)LQHWXQLQJ//0VFDQTXLFNO\DQVZHUTXHVWLRQV
and values. Fine-tuning LLMs can quickly answer questions, GHVLJQLQJ DOJRULWKPV WKDW FRQVXPH OHVV
designing algorithms that consume less on computility to RQ FRPSXWLOLW\ WR
JHQHUDWH papers,
generate SDSHUV writeZULWH code,
FRGH and
DQG compose
FRPSRVH scripts,
VFULSWV and
DQG DFKLHYHEUHDNWKURXJKVLQFRPSXWLOLW\LVDSDWKIRUHQKDQFLQJ
achieve breakthroughs in computility is a path for enhancing
HGXFDWRUVmay
educators PD\be EHunable
XQDEOHtoWRidentity
LGHQWLI\them.
WKHPIf,Imisused,
PLVXVHGthey
WKH\ UHDOWLPH interaction.
real-time LQWHUDFWLRQ About
$ERXW reinforcement
UHLQIRUFHPHQW learning
OHDUQLQJ fromIURP
FDQeasily
can HDVLO\become
EHFRPHaDtool WRROfor
IRUcheating
FKHDWLQJin LQeducation.
HGXFDWLRQOn
2QtheWKH KXPDQIHHGEDFN//0VLQHGXFDWLRQFDQXVHWKHLQWHUDFWLRQ
human feedback, LLMs in education can use the interaction
RQHKDQGLWZLOOFDXVHXQIDLUQHVVLQWHJULW\FULVLVDQGRWKHU
one hand, it will cause unfairness, integrity crisis, and other DQGfeedback
and IHHGEDFNinformation
LQIRUPDWLRQwith ZLWKlearners
OHDUQHUVduring
GXULQJthe WKHlearning
OHDUQLQJ
SUREOHPVOn
problems. 2Qthe WKHother
RWKHUhand,
KDQGitLWmay
PD\prevent
SUHYHQWteachers
WHDFKHUVfrom
IURP SURFHVVWRIXUWKHUILQHWXQHWKHPRGHOEDVHGRQSHUVRQDOL]HG
process to further fine-tune the model based on personalized
WUXO\ XQGHUVWDQGLQJ VWXGHQWV  OHDUQLQJ VWDWXVHV
truly understanding students' learning statuses, according to DFFRUGLQJ WR LQVWUXFWLRQVfrom
instructions IURP learners.
OHDUQHUVThis
7KLV enables
HQDEOHVLLMs
//0Vto WRimprove
LPSURYH

721
721
Authorized licensed use limited to: UNIVERSITY OF STRATHCLYDE. Downloaded on February 13,2025 at 19:03:36 UTC from IEEE Xplore. Restrictions apply.
WKHLU
their FDSDELOLWLHV
capabilities FRQWLQXRXVO\
continuously DQG and SURYLGHV
provides PRUH
more >@
[7] 3 P. )
F. %URZQ
Brown, 9V. -
J. 'HOOD
Della 3LHWUD
Pietra, 3
P. 9
V. 'HVRX]D
Desouza, -&
J. C. /DL
Lai, DQG
and 5
R. /
L.
SHUVRQDOL]HGWHDFKLQJVHUYLFHV 0HUFHU
Mercer, ³&ODVVEDVHG
"Class-based QJUDP n-gram PRGHOV
models RI of QDWXUDO
natural ODQJXDJH´
language,"
personalized teaching services. &RPSXWDWLRQDOOLQJXLVWLFVYROQRSS
Computational linguistics, vol. 1 8, no. 4, pp. 467-480, 1 992.
,Q
In HGXFDWLRQDO
educational DSSOLFDWLRQV
applications, //0V
LLMs FDQ can EH
be FRQQHFWHG
connected WR
to
>@
[8] Y. %HQJLR
< Bengio, 5 R. 'XFKDUPH
Ducharme, DQG and 3
P. 9LQFHQW
Vincent, ³$ "A QHXUDO
neural SUREDELOLVWLF
probabilistic
WKH
the ,QWHUQHW
Internet WR
to DFFHVV
access WKH
the ODWHVW
latest WHDFKLQJ
teaching UHVRXUFHV
resources DQG
and ODQJXDJH
language PRGHO´
model," $GYDQFHV
Advances LQ in QHXUDO
neural LQIRUPDWLRQ
information SURFHVVLQJ
processing
LQIRUPDWLRQ
information, HQVXULQJ
ensuring WKH
the DFFXUDF\
accuracy DQG
and FUHGLELOLW\
credibility RI
of WKH
the V\VWHPVYRO
systems, vol. 13, 2000.
JHQHUDWHG
generated WHDFKLQJ
teaching FRQWHQW
content. 0RUHRYHU
Moreover, SUHFLVH
precise XVHU
user SURILOHV
profiles >@
[9] $ A. 6KHUVWLQVN\
Sherstinsky, ³)XQGDPHQWDOV
"Fundamentals RI of UHFXUUHQW
recurrent QHXUDO
neural QHWZRUN
network (RNN) 511 
FDQ
can EH
be FUHDWHG
created EDVHG
based RQ
on HGXFDWLRQUHODWHG
education-related LQIRUPDWLRQ
information DQG
and DQG
and ORQJ
long VKRUWWHUP
short-term PHPRU\
memory (LSTM)/670  QHWZRUN´
network," 3K\VLFD
Physica ' D:
UHFRPPHQGDWLRQ
recommendation DOJRULWKPV
algorithms WR to JHQHUDWH
generate SHUVRQDOL]HG
personalized 1RQOLQHDU3KHQRPHQDYROSS
Nonlinear Phenomena, vol. 404, pp. 132306, 2020.
OHDUQLQJ
learning UHVRXUFHV
resources, HQDEOLQJ
enabling //0V
LLMs WR to VKRZFDVH
showcase GLIIHUHQW
different >@
[!OJ $-DLVZDO$5%DEX0==DGHK'%DQHUMHHDQG)0DNHGRQ
A. Jaiswal, A. R. Babu, M. Z. Zadeh, D. Banerjee, and F. Makedon,
WHDFKLQJ
teaching VW\OHV
styles IRU
for OHDUQHUV
learners. ,Q
In WKH
the WHDFKLQJ
teaching SURFHVV
process, WKH
the ³$
"A VXUYH\
survey RQon FRQWUDVWLYH
contrastive VHOIVXSHUYLVHG
self-supervised OHDUQLQJ´
learning," 7HFKQRORJLHV
Technologies,
YROQRSS
vol. 9, no. I , pp. 2, 2020.
DFFXUDWHFRPSUHKHQVLRQRIPXOWLPRGDOGDWDDQGDXWRQRPRXV
accurate comprehension of multimodal data and autonomous
XWLOL]DWLRQRIWHDFKLQJWRROVDUHFUXFLDOFDSDELOLWLHVWKDWILQH >@
[ I I ] -
J. 6DU]\QVND:DZHU
Sarzyuska-Wawer, $ A. :DZHU
Wawer, $ A. 3DZODN
Pawlak, - J. 6]\PDQRZVND
Szymanowska, , I.
utilization of teaching tools are crucial capabilities that fine­ 6WHIDQLDN
Stefaniak, 0M. -DUNLHZLF]
Jarkiewicz, DQG and /
L. 2NUXV]HN
Okruszek, ³'HWHFWLQJ
"Detecting IRUPDO
formal
WXQLQJ
tuning //0V
LLMs LQ in HGXFDWLRQ
education DUHare H[SHFWHG
expected WR to DFKLHYH
achieve WKRXJKW
thought GLVRUGHU
disorder E\ by GHHS
deep FRQWH[WXDOL]HG
contextualized ZRUGword UHSUHVHQWDWLRQV´
representations,"
EUHDNWKURXJKV
breakthroughs LQ in WKH
the IXWXUH
future. 7KLV
This ZLOO
will UHQGHU
render HGXFDWLRQDO
educational 3V\FKLDWU\5HVHDUFKYROSS
Psychiatry Research, vol. 304, pp. 1 14135, 2021 .
//0V
LLMs PRUHmore LQWHOOLJHQW
intelligent DQG
and KDYH
have JUHDWHU
greater GHYHORSPHQW
development >@
[12] -J . 'HYOLQ
Devlin, 0:
M.-W. &KDQJ
Chang, . K. /HH
Lee, DQG
and .
K. 7RXWDQRYD
Toutanova, ³%HUW
"Bert: 3UHPre­
SRWHQWLDO
potential. WUDLQLQJ
training RI of GHHS
deep ELGLUHFWLRQDO
bidirectional WUDQVIRUPHUV
transformers IRU for ODQJXDJH
language
XQGHUVWDQGLQJ´DU;LYSUHSULQWDU;LY
understanding," arXiv preprint arXiv : l 8 10.04805, 2018.
9,
VI. & 21&/86,21
CONCLUSION >@
[13] $A. 9DVZDQL
Vaswani, 1 N. 6KD]HHU
Shazeer, 1 N. 3DUPDU
Parmar, -
J. 8V]NRUHLW
Uszkoreit, / L. -RQHV
Jones, $A. 1
N.
*RPH]
Gomez, à L. .DLVHU
Kaiser, DQG
and ,
I. 3RORVXNKLQ
Polosukhin, ³$WWHQWLRQ
"Attention LVis DOO
all \RX
you QHHG´
need,"
0RWLYDWHG
Motivated E\ by WKH
the RQJRLQJ
ongoing GLJLWDO
digital WUDQVIRUPDWLRQ
transformation LQ in $GYDQFHVLQQHXUDOLQIRUPDWLRQSURFHVVLQJV\VWHPVYRO
Advances in neural information processing systems, vol. 30, 20 17.
HGXFDWLRQ
education, WKLV
this SDSHU
paper H[SORUHV
explores WKH
the DSSOLFDWLRQ
application RIof DGYDQFHG
advanced >@
[14] 1N. +RXOVE\
Houlsby, $A. *LXUJLX
Giurgiu, 6 S. -DVWU]HEVNL
Jastrzebski, % B. 0RUURQH
Morrone, 4 Q. 'H
De
ILQHWXQLQJ//0VLQHGXFDWLRQ)LUVWO\WKLVSDSHULQWURGXFHV
fine-tuning LLMs in education. Firstly, this paper introduces /DURXVVLOKH
Laroussilhe, $ A. *HVPXQGR
Gesmuudo, 0 M. $WWDUL\DQ
Attariyan, DQG
and 6
S. *HOO\
Gelly, 3DUDPHWHU
"Parameter­
WKH
the WHFKQLFDO
technical EDFNJURXQG
background NQRZOHGJH
knowledge RI of ILQHWXQLQJ
fine-tuning //0V
LLMs, HIILFLHQWWUDQVIHUOHDUQLQJIRU1/3SS
efficient transfer learning for NLP." pp. 2790-2799.
SURYLGLQJDQRYHUYLHZRIWKHPRGHODUFKLWHFWXUHDQGWUDLQLQJ
providing an overview of the model architecture and training >@
[ 1 5] 5:DQJ'7DQJ1'XDQ=:HL;+XDQJ*&DR'-LDQJDQG
R. Wang, D. Tang, N. Duan, Z. Wei, X. Huang, G. Cao, D. Jiang, and
SURFHVV
process. 6XEVHTXHQWO\
Subsequently, WKHthe DSSOLFDWLRQ
application RI
of ILQHWXQLQJ
fine-tuning //0V
LLMs 0
M. =KRX
Zhou, ³.DGDSWHU
"K-adapter: ,QIXVLQJ
Infusing NQRZOHGJH
knowledge LQWRinto SUHWUDLQHG
pre-trained PRGHOV
models
WHFKQRORJ\LQHGXFDWLRQZDVH[HPSOLILHGSULPDULO\IRFXVLQJ
technology in education was exemplified, primarily focusing ZLWKDGDSWHUV´DU;LYSUHSULQWDU;LY
with adapters," arXiv preprint arXiv:2002.01 808, 2020.
RQ
on SXEOLVKHG
published FDVHV
cases. )LQDOO\
Finally, ZH
we VXPPDUL]H
summarize WKH the PDLQ
main >@
[16] -J. 3IHLIIHU
Pfeiffer, $A. .DPDWK
Kamath, $ A. 5FNOp
Ruckle, .K. &KR
Cho, DQG
and ,I. *XUHY\FK
Gurevych,
FKDOOHQJHV ³$GDSWHU)XVLRQ
"AdapterFusion: 1RQGHVWUXFWLYH
Non-destructive WDVN task FRPSRVLWLRQ
composition IRU for WUDQVIHU
transfer
challenges LQ in ILQHWXQLQJ
fine-tuning //0V
LLMs LQ in HGXFDWLRQ
education DQG
and RIIHU
offer OHDUQLQJ´DU;LYSUHSULQWDU;LY
learning," arXiv prepriut arXiv:2005.00247, 2020.
SURVSHFWV
prospects IRU
for WKHLU
their IXWXUH
future GHYHORSPHQW
development. )LQHWXQLQJ
Fine-tuning //0V
LLMs,
>@
[17] %B. /HVWHU
Lester, 5
R. $O5IRX
Al-Rfou, DQG and 1
N. &RQVWDQW
Constant, ³7KH
"The SRZHU
power RI of VFDOH
scale IRU
for
ZKLFKDUHFXUUHQWO\RQHRIWKHFXWWLQJHGJHUHVHDUFKGRPDLQV
which are currently one of the cutting-edge research domains SDUDPHWHUHIILFLHQWSURPSWWXQLQJ´DU;LYSUHSULQWDU;LY
parameter-efficient prompt tuning," arXiv preprint arXiv:2104.0869 1 ,
LQDUWLILFLDOLQWHOOLJHQFHZLOOFRQWLQXHWRGHYHORSUDSLGO\DQG
in artificial intelligence, will continue to develop rapidly and 
202 1 .
SURIRXQGO\
profoundly LQIOXHQFH
influence HGXFDWLRQ
education LQ
in WKH future. +RZHYHU
the IXWXUH However, ZH we >@
[ 1 8] ;
X . /
L . /L
Li, DQG
and 3P . /LDQJ
Liang, ³3UHIL[WXQLQJ
"Prefix-tuuing: 2SWLPL]LQJ
Optimizing FRQWLQXRXV
continuous
VKRXOG
should DOVRUHPDLQ
also remain YLJLODQW
vigilant LQ
in DGGUHVVLQJ
addressing HWKLFDO
ethical, OHJDODQG
legal, and SURPSWVIRUJHQHUDWLRQ´DU;LYSUHSULQWDU;LY
prompts for generation," arXiv preprint arXiv:21 01 .00190, 2021 .
VRFLDO
social LVVXHV
issues UHODWHG
related WR
to WKH
the DSSOLFDWLRQ
application RI
of ILQHWXQLQJ
fine-tuning //0V
LLMs >@
[19] (-+X<6KHQ3:DOOLV=$OOHQ=KX</L6:DQJ/:DQJ
E . J . Hu, Y . Shen, P . Wallis, Z . Allen-Zhu, Y . Li, S . Wang, L . Wang,
LQHGXFDWLRQHQVXULQJWKDWWKHLUDSSOLFDWLRQVDOZD\VFRPSO\
in education, ensuring that their applications always comply DQG:&KHQ³/RUD/RZUDQNDGDSWDWLRQRIODUJHODQJXDJHPRGHOV´
and W. Chen, "Lora: Low-rank adaptation of large language models,"
ZLWKHWKLFDOVWDQGDUGVDQGUHJXODWRU\UHTXLUHPHQWV
with ethical standards and regulatory requirements. DU;LYSUHSULQWDU;LY
arXiv preprint arXiv:21 06.09685, 2021 .
>@
[20] 4=KDQJ0&KHQ$%XNKDULQ3+H<&KHQJ:&KHQDQG7
Q . Zhang, M. Chen, A. Bukharin, P . He, Y . Cheng, W. Chen, and T.
5()(5(1&(6
REFERENCES =KDR
Zhao, ³$GDSWLYH
"Adaptive EXGJHW
budget DOORFDWLRQ
allocation IRUfor SDUDPHWHUHIILFLHQW
parameter-efficient ILQH fine­
WXQLQJ´DU;LYSUHSULQWDU;LY
tuning," arXiv preprint arXiv:2303 . I 0512, 2023.
 >@
[21] 7T. 'HWWPHUV
Dettmers, $ A. 3DJQRQL
Paguoni, $ A. +ROW]PDQDQG
Holtzman, and / L. =HWWOHPR\HU
Zettlemoyer, ³4ORUD
"Qlora:
>@
[!] 5%RPPDVDQL'$+XGVRQ($GHOL5$OWPDQ6$URUD6YRQ
R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von (IILFLHQW
Efficient ILQHWXQLQJ
frnetuuing RI of TXDQWL]HG
quantized OOPV´
llms," DU;LYarXiv SUHSULQW
preprint
$U[06%HUQVWHLQ-%RKJ$%RVVHOXWDQG(%UXQVNLOO³2QWKH
Arx, M. S. Bernstein, J. Bohg, A. Bosselut, and E. Brunskill, "On the DU;LY
arXiv:2305. 1 43 1 4, 2023.
RSSRUWXQLWLHV
opportunities DQGand ULVNV
risks RI
of IRXQGDWLRQ
foundation PRGHOV´
models," DU;LY
arXiv SUHSULQW
preprint
DU;LY >@
[22] ) F. =KDQJ
Zhang, / L. /L
Li, -J. &KHQ
Chen, = Z. -LDQJ
Jiang, %
B. :DQJ
Wang, DQG and <Y. 4LDQ
Qian,
arXiv:2108.07258, 202 1 .
³,QFUH/R5$
"IucreLoRA: ,QFUHPHQWDO
Incremental 3DUDPHWHU
Parameter $OORFDWLRQ
Allocation 0HWKRGMethod IRU for
>@
[2] 7
T . %URZQ
Brown, % B. 0DQQ
Mann, 1 N . 5\GHU
Ryder, 0
M . 6XEELDK
Subbiah, -J. '
D . .DSODQ
Kaplan, 3
P. 3DUDPHWHU(IILFLHQW
Parameter-Efficient )LQHWXQLQJ´
Fine-tuning," DU;LY
arXiv SUHSULQW
preprint DU;LY
arXiv:2308. 12043,
'KDULZDO
Dhariwal, $ A. 1HHODNDQWDQ
Neelakantan, 3P. 6K\DP
Shyam, * G. 6DVWU\
Sastry, DQG
and $
A. $VNHOO
Askell, 
2023.
³/DQJXDJH
"Language PRGHOV
models DUH
are IHZVKRW
few-shot OHDUQHUV´
learners," $GYDQFHV
Advances LQ in QHXUDO
neural
LQIRUPDWLRQSURFHVVLQJV\VWHPVYROSS >@
[23] &C. *XDQ
Guan, -J. 0RX
Mou, DQG
and =
Z. -LDQJ
Jiang, ³$UWLILFLDO
"Artificial LQWHOOLJHQFH
intelligence LQQRYDWLRQLQ
innovation in
information processing systems, vol. 33, pp. 1 877-190 1 , 2020.
HGXFDWLRQ
education: $ A WZHQW\\HDU
twenty-year GDWDGULYHQ
data-driven KLVWRULFDO
historical DQDO\VLV´
analysis,"
>@
[3] +7RXYURQ/0DUWLQ.6WRQH3$OEHUW$$OPDKDLUL<%DEDHL
H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, ,QWHUQDWLRQDO-RXUQDORI,QQRYDWLRQ6WXGLHVYROQRSS
International Journal of Innovation Studies, vol. 4, no. 4, pp. 1 34-147,
1%DVKO\NRY6%DWUD3%KDUJDYDDQG6%KRVDOH³/ODPD2SHQ
N. Bashlykov, S. Batra, P. Bhargava, and S. Bhosale, "Llama 2: Open 
2020.
IRXQGDWLRQ
foundation DQG and ILQHWXQHG
frne-tuned FKDW
chat PRGHOV´
models," DU;LY
arXiv SUHSULQW
preprint
DU;LY >@
[24] P. 0LURZVNL
3 Mirowski, . K. :
W. 0DWKHZVRQ
Mathewson, - J. 3LWWPDQ
Pittman, DQG
and 5R. (YDQV
Evans, &R"Co­
arXiv:2307.09288, 2023.
:ULWLQJ
Writing 6FUHHQSOD\V
Screenplays DQG and 7KHDWUH
Theatre 6FULSWV
Scripts ZLWK
with /DQJXDJH
Language 0RGHOV
Models:
>@
[4] 5
R. 5RPEDFK
Rombach, $ A. %ODWWPDQQ
Blattmarrn, '
D. /RUHQ]
Lorenz, 3P. (VVHU
Esser, DQG
and %
B. 2PPHU
Ommer, (YDOXDWLRQE\,QGXVWU\3URIHVVLRQDOVSS
Evaluation by Industry Professionals." pp. 1-34.
+LJKUHVROXWLRQ
"High-resolution LPDJH
image V\QWKHVLV
synthesis ZLWK
with ODWHQW
latent GLIIXVLRQ
diffusion PRGHOV
models." SS
pp.
 >@
[25] $ A. $JRVWLQHOOL
Agostinelli, 7T. ,I. 'HQN
Denk, =Z. %RUVRV
Borsos, -J. (QJHO
Engel, 0M. 9HU]HWWL
Verzetti, $ A.
I 0684-1 0695.
&DLOORQ
Caillon, 4Q. +XDQJ
Huang, $ A. -DQVHQ
Jansen, $ A. 5REHUWV
Roberts, DQG
and 0
M. 7DJOLDVDFFKL
Tagliasacchi,
>@
[5] 6
S. %XEHFN
Bubeck, 9V. &KDQGUDVHNDUDQ
Chandrasekaran, 5
R. (OGDQ
Eldan, -
J. *HKUNH
Gehrke, (
E. +RUYLW]
Horvitz, (
E. ³0XVLFOP
"Musiclm: *HQHUDWLQJ
Generating PXVLF mus1c IURPfrom WH[W´
text," DU;LY
arXiv SUHSULQW
preprint
.DPDU
Kamar, 3 P. /HH
Lee, <
Y. 7
T. /HH
Lee, <
Y. /L
Li, DQG
and 6
S. /XQGEHUJ
Luudberg, ³6SDUNV
"Sparks RI
of DU;LY
arXiv:230 1 . 1 1325, 2023.
DUWLILFLDO
artificial JHQHUDO
general LQWHOOLJHQFH
intelligence: (DUO\
Early H[SHULPHQWV
experiments ZLWK
with JSW´
gpt-4," DU;LY
arXiv
SUHSULQWDU;LY >@
[26] =:DQJ-9DOGH]'%DVX0DOOLFNDQG5*%DUDQLXN7RZDUGV
Z. Wang, J. Valdez, D. Basu Mallick, and R. G. Baranink, "Towards
preprint arXiv:2303 . 1 27 12, 2023.
KXPDQOLNH
human-like HGXFDWLRQDO
educational TXHVWLRQ
question JHQHUDWLRQ
generation ZLWKwith ODUJH
large ODQJXDJH
language
>@
[6] -*LOOHWWDQG::DUG$ODQJXDJHPRGHOFRPELQLQJWULJUDPVDQG
J. Gillett, and W. Ward, "A language model combining trigrams and PRGHOVSS
models." pp. 1 53-166.
VWRFKDVWLFFRQWH[WIUHHJUDPPDUV
stochastic context-free grammars."
>@
[27] 6 S. 6DUVD
Sarsa, 3P. 'HQQ\
Denny, $ A. +HOODV
Hellas, DQG
and -
J. /HLQRQHQ
Leinonen, $XWRPDWLF
"Automatic
JHQHUDWLRQ
generation RI of SURJUDPPLQJ
programming H[HUFLVHV
exercises DQG
and FRGH
code H[SODQDWLRQV
explanations XVLQJusing
ODUJHODQJXDJHPRGHOVSS
large language models." pp. 27-43.

722
722

Authorized licensed use limited to: UNIVERSITY OF STRATHCLYDE. Downloaded on February 13,2025 at 19:03:36 UTC from IEEE Xplore. Restrictions apply.
>@
[28] ,'URUL6=KDQJ56KXWWOHZRUWK/7DQJ$/X(.H./LX/
I. Drori, S. Zhang, R. Shuttleworth, L. Tang, A. Lu, E. Ke, K. Liu, L. WUDLQ
train FKLOGUHQ¶V
children's FXULRXVTXHVWLRQDVNLQJ
curious question-asking VNLOOV´
skills," ,QWHUQDWLRQDO
International -RXUQDO
Journal
&KHQ67UDQDQG1&KHQJ³$QHXUDOQHWZRUNVROYHVH[SODLQVDQG
Chen, S. Tran, and N. Cheng, "A neural network solves, explains, and RI$UWLILFLDO,QWHOOLJHQFHLQ(GXFDWLRQSS
of Artificial Intelligence in Education, pp. 1 -36, 2023.
JHQHUDWHV
generates XQLYHUVLW\
university PDWK
math SUREOHPV
problems E\by SURJUDP
program V\QWKHVLV
synthesis DQG
and IHZ
few­ >@
[32] <'DQ=/HL<*X</L-<LQ-/LQ/<H=7LH<=KRX
Y. Dan, Z. Lei, Y. Gu, Y. Li, J. Yin, J. Lin, L. Ye, Z. Tie, Y. Zhou,
VKRWOHDUQLQJDWKXPDQOHYHO´3URFHHGLQJVRIWKH1DWLRQDO$FDGHP\
shot learning at human level," Proceedings of the National Academy DQG
and <Y. :DQJ
Wang, ³(GX&KDW
"EduChat: $ A /DUJH6FDOH
Large-Scale /DQJXDJH
Language 0RGHOEDVHG
Model-based
RI6FLHQFHVYROQRSSH
of Sciences, vol. 1 19, no. 32, pp. e21 23433 1 19, 2022. &KDWERW
Chatbot 6\VWHP
System IRUfor ,QWHOOLJHQW
Intelligent (GXFDWLRQ´
Education," DU;LY
arXiv SUHSULQW
preprint
>@
[29] 33DWDUDQXWDSRUQ9'DQU\-/HRQJ33XQSRQJVDQRQ'1RY\3
P. Pataranutaporn, V. Danry, J. Leong, P. Punpongsanon, D. Navy, P. DU;LY
arXiv:2308.02773, 2023.
0DHV
Maes, DQG
and 0 M. 6UD
Sra, ³$,JHQHUDWHG
"AI-generated FKDUDFWHUV
characters IRU
for VXSSRUWLQJ
supporting >@
[33] (0%HQGHU7*HEUX$0F0LOODQ0DMRUDQG66KPLWFKHOO2Q
E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell, "On
SHUVRQDOL]HG
personalized OHDUQLQJ
learning DQG
and ZHOOEHLQJ´
well-being," 1DWXUH
Nature 0DFKLQH
Machine ,QWHOOLJHQFH
Intelligence, WKHGDQJHUVRIVWRFKDVWLFSDUURWV&DQODQJXDJHPRGHOVEHWRRELJ""
the dangers of stochastic parrots: Can language models be too big??."
YROQRSS
vol. 3, no. 12, pp. 1 0 1 3-1022, 202 1 . SS
pp. 6 1 0-623.
>@
[30] 4
Q. 'RQJ
Dong, /
L . 'RQJ
Dong, .
K. ;X
Xu, *
G . =KRX
Zhou, <
Y . +DR
Hao, =
Z . 6XL
Sui, DQG
and )
F . :HL
Wei, >@
[34] -
J. .DSODQ
Kaplan, 6
S. 0F&DQGOLVK
McCandlish, 7T. +HQLJKDQ
Henighan, 7
T. %
B. %URZQ
Brown, %
B. &KHVV
Chess, 5
R.
³/DUJH
"Large /DQJXDJH0RGHOIRU6FLHQFH$6WXG\RQ3YV13͇DU;LY
Language Model for Science: A Study on P vs. NP," arXiv &KLOG6*UD\$5DGIRUG-:XDQG'$PRGHL³6FDOLQJODZVIRU
Child, S. Gray, A. Radford, J. Wu, and D. Amodei, "Scaling laws for
SUHSULQWDU;LY
preprint arXiv:2309.05689, 2023. QHXUDOODQJXDJHPRGHOV´DU;LYSUHSULQWDU;LY
neural language models," arXiv preprint arXiv:200 1 .083 6 1 , 2020.
>@
[3 1] 5
R. $EGHOJKDQL
Abdelghani, <+
Y.-H. :DQJ
Wang, ;
X. <XDQ
Yuan, 7
T. :DQJ
Wang, 3 P. /XFDV
Lucas, +
H. 
6DX]pRQ
Sauzeon, DQG
and 3<
P.-Y. 2XGH\HU
Oudeyer, ³*37GULYHQ
"GPT-3-driven SHGDJRJLFDO
pedagogical DJHQWV
agents WR
to


723
723

Authorized licensed use limited to: UNIVERSITY OF STRATHCLYDE. Downloaded on February 13,2025 at 19:03:36 UTC from IEEE Xplore. Restrictions apply.