Exploring the Use of ChatGPT as a Tool for Learning and Assessment in Undergraduate

Computer Science Curriculum: Opportunities and Challenges

Basit Qureshi
Department of Computer Science, Prince Sultan University, Saudi Arabia
[email protected]

Abstract— The application of Artificial intelligence for complete assignments. Additionally, it can also grade and
teaching and learning in the academic sphere is a trending provide feedback on assignments, giving teachers more time
subject of interest in the computing education. ChatGPT, as an to create engaging lesson plans and focus more attention on
AI-based tool, provides various advantages, such as heightened their students. However, it is important to control ChatGPT
student involvement, cooperation, accessibility and availability. use to avoid potential negative impacts, especially for
This paper addresses the prospects and obstacles associated students. Teachers and lecturers are concerned that the
with utilizing ChatGPT as a tool for learning and assessment in chatbot may increase plagiarism and spread academic
undergraduate Computer Science curriculum in particular to dishonesty, which could negatively impact academic
teaching and learning fundamental programming courses.
integrity [3, 10]. While ChatGPT can provide a clear formula
Students having completed the course work for a Data
Structures and Algorithms (a sophomore level course)
for producing written work, it may not necessarily improve
participated in this study. Two groups of students were given critical thinking and problem-solving skills. Nevertheless,
programming challenges to solve within a short period of time. these potential dangers can be mitigated by providing
The control group (group A) had access to text books and notes students with tools and resources for identifying and
of programming courses, however no Internet access was avoiding inaccurate information.
provided. Group B students were given access to ChatGPT and Computer Science educators generally believe that
were encouraged to use it to help solve the programming
hands-on experience in a computer lab environment, help
challenges. The challenge was conducted in a computer lab
environment using Programming Contest Control (PC2)
students gain valuable experience [6-7]. In addition to
environment which is widely used in ACM International effective book/classroom learning, practical work provides
Collegiate Programming Contest (ICPC). Each team of students with insight into the methods of science, research
students address the problem by writing executable code that practice, and experimental methodology. The incorporation
satisfies certain number of test cases. Student teams were of scientific research into schools through pedagogical
scored based on their performance in terms of number of principles, rather than strict scientific formalism, is often
successful passed testcases. Results show that students using categorized into three main learning goals: learning science,
ChatGPT had an advantage in terms of earned scores, however learning about science, and learning to do science, which
there were inconsistencies and inaccuracies in the submitted involves engaging in and developing expertise in scientific
code consequently affecting the overall performance. After a inquiry and problem-solving [8]. Practical work has been
thorough analysis, the paper's findings indicate that shown to promote students' positive attitudes and enhance
incorporating AI in higher education brings about various motivation for effective learning [9-10]. New approaches for
opportunities and challenges. Nonetheless, universities can teaching and assessing scientific inquiry and practices are
efficiently manage these apprehensions by adopting a proactive essential for guiding students to make informed decisions.
and ethical stance towards the implementation of such tools.
Prince Sultan University's College of Computer and
Keywords—ChatGPT, Academic assessment, programming Information Sciences features three undergraduate programs,
concepts, Data Structures and Algorithms, Academic assessment. namely Computer Science, Information Systems, and
Software Engineering, all of which are accredited by ABET.
I. INTRODUCTION These programs mandate students to complete 124 credit
hours of study [11]. Additionally, students who choose to
The progress made in machine learning and natural enroll in any of these programs must take three consecutive
language processing (NLP) has led to a surge in research courses - CS101 Programming I, CS102 Programming II,
interest for chatbots, which have been employed in a diverse and CS210 Data Structures and Algorithms, during the first
range of commercial and non-commercial applications [1-3]. two years of study. These courses provide a comprehensive
The emergence of ChatGPT and other AI-powered text understanding of Java programming language concepts and
generation technologies has brought them into the spotlight structures. The CS101 course comprises a lab component in
for educators and scholars. The availability of ChatGPT and which students must attend weekly sessions to learn and
similar technologies has raised many questions for apply problem-solving skills using a prevalent IDE.
educational professionals, policymakers, and researchers [4- Conversely, CS102 and CS210 adopt a project-based
5]. Using ChatGPT in computer science education has the approach to learning, where students collaborate on real-life
potential to offer several benefits. scenarios and devise solutions to address them. The main
In the context of informatics education, ChatGPT can be emphasis of the CS102 project is to implement an object-
a valuable tool for teaching and learning fundamental oriented approach to storing data in a structured manner. The
concepts of computing such as programming languages, data primary objective of the CS210 project is to utilize
structures, algorithms etc. It can help students to review and appropriate data structures and algorithms to solve the
study, especially when they struggle to find study partners or problem while adhering to specific time and space
don't have time to attend tutoring. The technology has the limitations. Once a student completes the CS210 course,
potential to streamline the research and learning process, he/she may take advanced courses in computing/engineering
making it easier for students to access information and within their program of study.
 This study aims to scrutinize the efficacy of ChatGPT in enthusiasm for this technology. ChatGPT, developed by
enhancing students' learning capabilities in the initial OpenAI [17], is a large language model with remarkable
programming courses of a Computing curriculum. One of the abilities in comprehending and producing language that
principal objectives of this study is to gauge the effectiveness closely resembles human speech. Its impressive performance
of ChatGPT in the students' learning process. Additionally, in answering questions, engaging in conversations, and
we seek to explore how effectively this technology can be generating coherent and contextually relevant responses has
employed to evaluate students' performance in online project marked a significant milestone in the advancement of
assessments. With this objective in mind, students who were conversational Artificial Intelligence. Despite its advanced
undertaking the CS210 - Data Structures and Algorithms capabilities, ChatGPT is essentially a complex chatbot that
course were motivated to take part in a computing challenge. can trace its roots in the early stages of Long Short-Term
During the programming challenge, two sets of students Memory (LSTM) development [14] and have been
were given a limited time to solve programming problems. extensively employed in various fields, including automated
The control group (Group A) was granted access to customer service support, E-Commerce, Healthcare, and
textbooks and programming notes but was not given internet Education.
access. On the other hand, Group B was allowed to use
OpenAI's release of the GPT-3 model family has raised
ChatGPT to help tackle the programming problems. The
challenge was conducted in a computer lab environment, the bar considerably for its main rivals, Google and
employing the Programming Contest Control (PC2) system, Facebook, and represents a significant milestone in the
which is widely used in the ACM International Collegiate advancement of natural language processing (NLP) models
Programming Contest (ICPC). Each team of students tackled [12]. The largest GPT-3 configuration, which consists of 175
the problem by crafting executable code that meets specific billion parameters, 96 attention layers, and a batch size of 3.2
test case requirements. The performance of the student teams million training samples, was trained using 300 billion
was graded based on the number of successful test cases. The tokens, typically sub-words [15]. The training process of
outcomes demonstrate that students who used ChatGPT had GPT-3 draws on the successful strategies employed by its
an edge in terms of earned scores; however, inconsistencies predecessor, GPT-2, such as modified initialization, pre-
and inaccuracies in the submitted code negatively impacted normalization, and reverse tokenization. However, GPT-3
their overall performance. also introduces a new refinement based on alternating dense
and sparse attention patterns. GPT-3 is designed as an
We make the following contributions in this work autoregressive framework that can achieve task-agnostic
goals through a few-shot learning paradigm [15]. With its
• In the experimental study, two sets of students ability to adapt to various tasks with minimal training data, it
participating in the challenge were observed, and their is a versatile and powerful tool for NLP applications.
performance with and without ChatGPT was recorded,
resulting in a dataset for further research. All GPT (Generative Pre-trained Transformer) models,
including the most recent GPT-4 model, are built based on
• A comprehensive examination of the impact of ChatGPT the core technology of Transformers. The Transformer
on this challenge is provided, discussing the advantages architecture was first introduced in the seminal paper
and disadvantages of its use in education. "Attention is All You Need" by Vaswani et al. in 2017 [16],
The paper is structured as follows: Section 2 contains a which has significantly impacted the deep learning research
related works section that describes the ChatGPT technology community. Transformers have revolutionized the processing
and its recent applications in the education sector. Section 3 of sequence-to-sequence models, surpassing traditional
outlines the methodology used in this research, followed by models based on recurrent neural networks by a significant
section 4, which presents the results and outcomes. Section 5 margin. While Transformers follow the classical encoder-
provides a discussion analyzing the results and offering decoder architecture, they differ dramatically by integrating
recommendations for educators and researchers. Finally, self-attention modules that excel in capturing long-term
Section 6 concludes the paper. dependencies between input sequence elements (i.e., tokens).
By leveraging this information, it efficiently determines the
importance of each element in the input sequence. The self-
II. RELATED WORKS attention mechanism computes a weight for each element
Although ChatGPT is a recent addition to the public based on its relevance to other tokens in the sequence. This
domain, it has already captured significant research interest. enables Transformers to handle variable-length sequences
In the following section, we present recent studies that more effectively and capture complex relationships between
describe the technology and explore its potential applications the sequence elements, thus improving performance on
for research. various natural language processing tasks. Trained on a vast
dataset of 570 GB of Internet data, ChatGPT possesses
A. What is ChatGPT? several features that make it stand out among its peers,
Machine learning, a subfield of artificial intelligence particularly in terms of accurate natural language generation
(AI), refers to the ability of computer systems to learn from [17]. These features are a significant advantage for ChatGPT.
experience without being explicitly programmed. With
advancements in computing power, more data availability, B. Applications of ChatGPT in education and research
and algorithmic improvements, deep learning has emerged as ChatGPT, a cutting-edge chatbot developed by OpenAI,
a high-performing predictive tool [1-4]. The field of Natural has already captured the interest of researchers despite its
Language Processing (NLP) has witnessed significant recent release to the public domain. In this section, we will
growth over the past few years. However, the emergence of delve into recent studies that explore the numerous
ChatGPT (Chat Generative Pre-trained Transformer) in applications and research possibilities of ChatGPT.
November 2022 has generated a renewed interest and
 In [13] Wang et.al., conducted an assessment of the passing threshold, achieving 60% accuracy without any
ChatGPT's ability to support various design, manufacturing, specialized input from humans.
and engineering education tasks. Their findings highlight
ChatGPT's impressive capacity to provide information, Pardos and Bhandari [23] conducted a study to determine
generate structured content, and propose initial solutions in a the effectiveness of ChatGPT in providing hints to students
comprehensive and creative manner, demonstrating its in elementary and intermediate Algebra topics. They found
potential to support tasks involving summarization, that 70% of the hints offered by ChatGPT led to positive
synthesis, and creation. They note that it struggles to learning gains for students. Basic et al. [24] conducted a
comprehend questions accurately and lacks the ability to study with nine students in the control group and nine in the
utilize knowledge effectively to generate appropriate experimental group that used ChatGPT to determine whether
solutions. In some instances, ChatGPT may even create non- it improves essay quality. Surprisingly, the control group
existent rules or equations to produce solutions. Additionally, outperformed the experimental group in most criteria,
ChatGPT lacks a profound understanding of the underlying leading the authors to conclude that ChatGPT does not
concepts of its responses, and its answers may not always be necessarily improve essay quality. In the field of software
trustworthy, traceable, or verifiable. development, ChatGPT has been evaluated for its
capabilities in tutoring and software testing. Jalil et al. [25]
In their study, Khan et al. [18] explored various medical presented ChatGPT with textbook questions to evaluate its
education applications that could benefit from ChatGPT, ability in software testing. Unfortunately, ChatGPT's
such as creating personalized learning experiences and performance in this case was not promising, as it was only
generating case studies. Additionally, the authors highlighted able to provide correct answers to 37.5% of the questions.
the potential of ChatGPT for clinical management purposes,
such as documentation and decision support. Jianning Li et. In a nutshell, ChatGPT is a disruptive technology for its
Al., in [19] authored a systematic review on ChatGPT in potential to fundamentally change the way academia and
Healthcare. They note that ChatGPT's interface is designed research are conducted. ChatGPT can generate personalized
for QA, allowing it to be seamlessly integrated into existing learning materials and case studies, improving the quality of
clinical workflows, and providing real-time feedback. In education for students. However, there are also potential
addition to providing direct answers to specific questions, drawbacks to the use of ChatGPT in academia and research.
ChatGPT also offers justifications for its responses. There is a risk that the use of ChatGPT could lead to a loss of
Interestingly, these justifications and answers sometimes creativity and critical thinking skills, as researchers may rely
contain novel insights and perspectives, which could inspire too heavily on the tool's generated content. Additionally,
new research ideas. Even open-ended questions can yield there are concerns around the accuracy and reliability of
valuable information thanks to ChatGPT's unique ChatGPT's responses, particularly in cases where it may
capabilities. ChatGPT is not equipped to handle queries that fabricate non-existing rules or equations.
require image interpretation. Unlike expert systems, Overall, ChatGPT represents a disruptive technology that
ChatGPT lacks the ability to "reason" and the justifications it has the potential to significantly alter the paradigm for
provides are a byproduct of its ability to predict the next academia and research. While there are both potential
word based on probabilities. The accuracy of ChatGPT's benefits and drawbacks to its use, it is clear that ChatGPT
responses is heavily reliant on the quality of the data it was has already begun to have a significant impact on these fields
trained on, as the training data determines how ChatGPT and is likely to continue to do so in the years to come.
responds to a given question. Kostick-Quenet and Gerke's in
[20] emphasizes the importance of evaluating the real-world III. METHODOLOGY
impact of ChatGPT and other language learning models
(LLMs) to prevent any potential negative consequences Research Question: Is there any statistical difference
resulting from their misuse. Although ChatGPT and other between the academic attainment level of students using
LLMs are innovative and revolutionary tools, their impact ChatGPT to complete a programming challenge and the
can be severely damaging if they are not used properly. students who do not have the opportunity to do so.
Thus, it is crucial to assess their practical applications and Research Design: Quasi-experimental research [26] is
potential risks before widespread implementation. typically carried out in real-world settings where it's either
According to Fauzi et al. [21], ChatGPT can play a impossible or not feasible to randomly assign participants to
significant role in enhancing student productivity in higher groups. This type of research is commonly employed to
education. The authors suggest that ChatGPT can provide assess the efficacy of a treatment or an educational
valuable resources and information, improve language skills, intervention. To ensure the comparability of the control and
promote collaboration, enhance time management, boost experimental groups, all students who participated in the
motivation, and increase efficiency and effectiveness. study were assessed with a pre-test to determine their
However, they also caution that ChatGPT should not be proficiency in programming concepts. Additionally, all
viewed as a replacement for human interaction or students' students had previously completed the three-programming
effort in achieving their academic goals. Instead, its role course taught at Prince Sultan University CS101, CS102, and
should be considered as a complement to these efforts. Kung CS210 courses with a minimum grade of C+.
et al. [22] conducted a study to determine if ChatGPT is • CS101- Programming I is a 4 credit hours course
capable of answering medical exam questions. They that covers topics including Java Programming Language
evaluated ChatGPT's performance on the US medical Syntax, Language Structure, Flow control, Loops, Methods,
licensing exam, which includes three standardized tests that Arrays, File reading and writing.
are mandatory for obtaining medical licensure in the US. The
study excluded any questions containing visual information • CS102 – Programming II is a 3 credit hours course
such as medical images. ChatGPT performed at a level near covering topics such as Object-Oriented Programming in
 Figure 1: Overview of research methodology

Java, Inheritance, Polymorphism, Java Collections limitations. To encourage efficient problem-solving, each
Framework, Array Lists, Sets, Maps, Multi- Dimensional problem has prescribed memory and CPU time limitations,
Arrays. thereby motivating students to use optimized data structures
in their solutions.
• CS210 – Data Structure & Algorithms is a 3 credit
hours course that covers Runtime analysis of Algorithms, The challenge was conducted in a computer lab
Lists, Stacks, Queues, Trees, BST, AVL Trees, Heaps, environment using Programming Contest Control (PC2) [32]
Priority Queues, Sorting Algorithms, Graphs traversal environment which is widely used in ACM International
algorithms, Shortest Path, and Minimum Spanning Trees. Collegiate Programming Contest (ICPC). Each team of
students comprising of 2 students addressed the challenge-
After finishing the pre-test, the students were separated problems by writing executable code that satisfies certain
into two groups: a control group (group A) and an number of test cases. Student teams were scored based on
experimental group (group B). Each group was composed of their performance in terms of number of successful passed
six teams, and each team had two members. The study testcases. Time taken by all teams for each problem was also
involved a total of 24 students. The purpose of this was to recorded for statistical evaluation.
ensure that both groups were homogeneous with respect to
academic level and pre-test scores. The intervention or
challenge was then administered, and the post-test scores of TABLE 1: SET OF PROBLEMS USED
the students were used to measure their achievement levels. Prob. Data Structure / Algorithm # of Test Time, Memory Req.
The resulting data was collected and statistically analyzed to No Cases
identify any significant differences in the mean scores P1 Lists traversal 10 O(n), O(n)
between the control and experimental groups. Figure 1 P2 AVL trees 8 O(log n), O(n)
shows an overview of the research design. P3 Sorting Arrays 14 O(n log n), O(n)
P4 Graph topological sorting 8 O(V+E), O(V+E)
Instrumentation: The challenge is specifically crafted to P5 Shortest Path / Dijkstra Algo 12 O((V+E) log V), O(V)
evaluate the participants' algorithmic and programming P6 MST using Prim’s algorithm 13 O(E log V), O(V+E)
capabilities, as well as their problem-solving skills. During
the competition, teams are presented with a series of Table 1 shows the data-structure used for each of the six
programming problems, each assigned a point value based problems, with the number of test-cases defined in the
on its difficulty level. Teams earn points for every problem system. The time and memory requirements are also given.
they solve correctly within the stipulated time limit. Notably, Here problems P1-P3 are considered easy, P4-6 increase in
each problem in the challenge is accompanied by a set of difficulty.
hidden test-cases, with the number of earned points
depending on the number of test-cases a student-team can Data collection process: The control panel in PC2
successfully complete. The challenge comprises six provides a detailed view on students scores earned, time
programming problems, each with its own statement spend in each problem, the number of times a student
outlining its requirements, input/output conditions, and submitted their work, number of test-cases passed etc. In this
 TABLE 2: PERFORMANCE OF GROUP A STUDENT TEAMS
P# A1 A2 A3 A4 A5 A6
P1 100 80 100 90 70 100
P2 87.5 62.5 75 50 62.5 100
P3 78.5 42.8 64.3 57.1 71.4 85
P4 62.5 37.5 50 25 25 75
P5 58.3 25 42.6 16.6 8.33 83.3
P6 53.8 30.7 7.69 0 0 38.4
Total 440.60 278.50 339.59 238.70 237.23 481.70
Max 100.00 80.00 100.00 90.00 71.40 100.00
Min 53.80 25.00 7.69 0.00 0.00 38.40
Avg 73.43 46.42 56.60 39.78 39.54 80.28
Std Dev 18.24 20.90 31.35 32.45 32.31 22.76

TABLE 3: PERFORMANCE OF GROUP B STUDENT TEAMS USING CHATGPT

P# B1 B2 B3 B4 B5 B6
P1 100 70 100 100 100 100
P2 50 62.5 87.5 100 100 87.5
P3 100 71.4 100 64.3 75 100
P4 75 0 87.5 75 87.5 87.5
P5 83.3 16.6 0 66.6 58.3 91.6
P6 69.2 0 7.69 38.4 61.53 76.9
Total 477.50 220.50 382.69 444.30 482.33 543.50
Max 100.00 71.40 100.00 100.00 100.00 100.00
Min 50.00 0.00 0.00 38.40 58.30 76.90
Avg 79.58 36.75 63.78 74.05 80.39 90.58
Std Dev 19.24 34.86 46.83 23.53 18.40 8.77

TABLE 4: MEANS AND STANDARD DEVIATIONS FOR P1-P6 FOR GROUPS A AND B

Problems Group A Group B

MEAN STD MEAN STD
P1 95.00 12.65 100.00 12.25
P2 68.75 18.40 87.50 20.54
P3 67.85 15.27 87.50 16.66
P4 43.75 20.41 81.25 34.23
P5 33.80 28.24 62.45 36.75
P6 19.20 22.49 49.97 32.54

study, the results of interest were the team scores earned and management resulted in low scores earned in P3-P6. Teams
the time taken by a team to submit their work. B1 to B6 used ChatGPT to help write code.

IV. RESULTS
Table 4 shows the mean and SD for scores earned by
groups A and B for problems P1 to P6. It is a common
practice to breakdown numerical data in order to provide a
visual representation that can highlight any variations in
scores and deviations. The mean score for Group B is
significantly higher than Group A for all problems. Both
groups had fairly similar SDs for Problems P1, P2 and P3,
however the difference is significant for P4 – P6. Figure 2: Average team scores for group A and B
Figure 2 shows the average team scores for group A and
B for the challenge. Overall Group B teams score better Observations were made that indicated teams B2 to B5
compared to group A teams. utilized code generated by ChatGPT, but faced challenges in
Figure 3 shows the time taken by teams to complete the passing all the test cases within the PC2 environment. This
challenges P1 to P6. Group A teams took more time to was due to the fact that some of the generated code was too
complete challenges P1 to P4. However, Group B teams took lengthy and exceeded ChatGPT's output size limits, which
more time to complete challenges P5 and P6. Group A teams resulted in truncation. As a result, more time was spent
were not able to score 100% in any of P3 to P6 problems. debugging the code to resolve these issues. Additionally,
Significant amount of time was used in debugging and fixing multiple queries were made to ChatGPT, leading to
errors in problems P3 and P4, consequently poor time inconsistencies. On the other hand, teams B1 and B6 also
used ChatGPT to develop their code, but they did not solely
rely on the generated code. They employed the suggested may suffice in producing the desired output, significant effort
algorithms to write their programs. However, none of the is required to obtain the desired outcome.
Group B teams were able to score 100% on problems P4 to
P6. As a result, the majority of teams in group B spent time Students from Group B were urged to copy/save prompts
on resolving errors by debugging their code. used to generate code. Some prompts used by students:
• Write java code to address this problem (problem
statement).
• This code did not compile, write a better code.
• This code still did not compile, fix the errors.
• This error was generated, fix this error and re-write code.
• Your code works but it does not pass testcases.
• This code is too slow, write a faster program.
• This code runs out of memory. Write a better program that
takes O(n) time.
• This program fails this test case. Fix the program so that it
Figure 3: Time taken by teams to complete the challenge passes (test case input with desired output pasted at
prompt).
V. DISCUSSION
The use of AI applications such as ChatGPT directed
Our findings suggest that ChatGPT is remarkable in towards students has tremendous potential in enhancing
producing coherent and well-structured code, and proposing intelligent student support systems and individualized
optimized solutions against the problem statement. As a learning. ChatGPT serves as an Intelligent tutoring system
generative model, ChatGPT generates promising outputs and (ITS) that can help provide customized instructions to
a comprehensive explanation for students to address the enhance learning process. Figure 4 shows an example of
given problem. If prompted correctly, it gives step by step tailored learning when the ChatGPT is prompted: “Visually
guidance explaining the underlying algorithm to be used in explain how an AVL tree attains O(log n) runtime?”
the solution with code snippets to help understanding.
Students prompted it to write complete code for the given As education researchers continue to explore new
problem, which it did, however executing this code in the learning models, opportunities in the field are expanding, and
IDE gave mixed results. personalization is growing. With the aid of AI algorithms,
ITSs can now imitate the support provided by a tutor, such as
We also identified limitations of ChatGPT in writing offering tailored assistance in problem-solving. It is expected
correct code/program. It faces challenges in comprehending that the advancements in big data technology and learning
questions and lacks the proficiency to utilize knowledge to analytics will lead to a groundbreaking paradigm of adaptive,
generate accurate solutions. In some cases, the generated individualized learning.
code snippet did not compile using the IDE after several
prompts to fix the code, causing frustration. Additionally, Typically, when innovative educational technologies are
inaccuracies were observed with the output of the generated introduced in the classroom [27-31], it can pose several
code which led in failing the test-cases. This was particularly challenges to the traditional teaching and learning methods.
noticeable as the problems difficulty level increased for the Managing these challenges becomes the responsibility of
students (P4 onwards). In addition, because of the education practitioners and policymakers [33]. If left
restrictions on the length of the generated text, it was unaddressed, these challenges can potentially reveal
occasionally impossible to obtain the entire program. As a deficiencies in pedagogical practices. In the context of
result, students had to prompt the ChatGPT multiple times to teaching and learning programming in a computing
acquire the desired output, which led to compile errors. curriculum, it is essential to see students learn by doing and
gain experience writing code. Our findings suggest that
For a student, having a thorough grasp of the inner students can easily generate working code in any
workings of the algorithms is unnecessary, but programming language of their choice using ChatGPT. In
comprehending the variables that will influence the result is most cases, the code generated, executes successfully after
crucial. It is important to inquire about the corpus of data that several prompts. Eventually the technology would catch up
was utilized to train the AI, including any omissions and producing better results in fewer prompts.
selection criteria. Additionally, it is important to consider the
possible implicit biases and assumptions that the tool may To prevent misuse of ChatGPT as a tool for learning
have. programming, there are several strategies that can be
employed:
To generate the most useful content, students must be
proficient in providing precise prompts to the AI bot. In this • Clear expectations: Educators should communicate
sense, large language models are comparable to search clearly to students that ChatGPT is not a suitable resource
engines since they require appropriate prompts to yield for learning programming and should not be used for that
relevant results. Crafting effective prompts necessitates a purpose.
deep understanding of the tool and the content that underlies • Encourage hands-on learning: Emphasize the importance
the prompt, as well as critical thinking. While a basic prompt of hands-on learning and encourage students to practice
 Figure 4: A sample of tailored response to a query

programming on their own using legitimate programming feasible for large classes or assignments with many
resources, such as textbooks, online courses, and coding students.
platforms.
• Monitor usage: Instructors need to be vigilant and keep VI. CONCLUSIONS
an eye on how students are using ChatGPT and intervene The integration of AI in higher education teaching,
if there is evidence that it is being used inappropriately. learning, and assessment, presents both opportunities and
This can be done by monitoring students' online activity challenges. An experimental study was conducted to assess
or by checking their programming work for signs of the effectiveness of using ChatGPT for solving programming
plagiarism or excessive reliance on external resources. problems. The study involved 24 students who were divided
• Assess learning outcomes: Incorporate assessments that into two groups control and experimental. According to the
test students' ability to apply programming concepts and findings, the group that utilized ChatGPT in the experiment
skills, rather than just regurgitating information. This will performed better than the control group, achieving higher
encourage students to engage in active learning and scores in less time. However, the experimental group was not
discourage them from relying on ChatGPT for answers. able to achieve perfect scores due to inaccuracies or
inconsistencies in the code generated by ChatGPT.
• Plagiarism detection tools: Moss (Measure of Software
Similarity) is a plagiarism detection tool developed by Overall, it is important to create a learning environment
Stanford University. It is designed to detect similarities that emphasizes the importance of active, hands-on learning
between programming assignments submitted by and discourages students from relying on external resources
students, and it can be used to help instructors identify such as ChatGPT for their programming education. As AI
cases of potential plagiarism. There are several other tools for solving programming challenges continue to
tools that can be used to getect plagiarism including emerge and improve in accuracy, students may be tempted to
JPlag, Codequiry etc. cheat if they do not fully understand the value of their
learning. It is the responsibility of educators to clearly
• Interviews can be a useful tool for detecting plagiarism in explain the significance of their work and the benefits of
cases where software tools may not be effective, such as learning through practice. At the institutional level, academic
when the student has modified code to make it appear integrity policies and honor codes should be revised to
original. However, this method requires significant time address the use of AI tools. Clear and simple guidelines for
and effort on the part of the instructor and may not be the proper use of language models in teaching and learning
should be developed, and consequences for cheating should
be clearly outlined. Faculty should be trained to adopt AI Materials & Continua, vol. 71, no.2, pp.2347-2361, 2022,
tools to augment their teaching practices. Additionally, https://doi.org/10.32604/cmc.2022.021447
[15] Tom Brown et.al., (2020) Language Models are Few-Shot Learners,
students should receive training on academic integrity to arXiv:2005.14165, https://doi.org/10.48550/arXiv.2005.14165.
ensure they fully understand the importance of maintaining [16] Ashish Vaswani, (2017). Attention Is All You Need, arXiv
ethical standards in their work. Preprint:1706.03762, https://doi.org/10.48550/arXiv.1706.03762
[17] Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever,
Improving Language Understanding by Generative Pre-Training,
ACKNOWLEDGMENT Accessible online, IRL: https://cdn.openai.com/research-
The author would like to acknowledge the role of covers/language-unsupervised/language_understanding_paper.pdf
[18] Khan, R. A., Jawaid, M., Khan , A. R., & Sajjad, M. (2023). ChatGPT
ChatGPT in the development of this paper. The author - Reshaping medical education and clinical management. Pakistan
utilized the tool/service to enhance readability of his/her own Journal of Medical Sciences, 39(2).
sentences, and summarize and clarify text from related works https://doi.org/10.12669/pjms.39.2.7653
section. The author takes full responsibility for reviewing [19] Jianning Li, Amin Dada, Jens Kleesiek, Jan Egger , (2023) ChatGPT in
and editing the content as necessary prior to publication. Healthcare: A Taxonomy and Systematic Review, MedRxiv:
https://doi.org/10.1101/2023.03.30.23287899
[20] Kostick-Quenet, K.M.; Gerke, S. (2022), AI in the hands of imperfect
REFERENCES users. npj Nature Digital Medicine. 2022, 5, 197.
[21] Fauzi, F., Tuhuteru, L., Sampe, F., Ausat, A., & Hatta, H. (2023).
[1] van Dis EAM, Bollen J, Zuidema W, van Rooij R, Bockting CL.
Analysing the Role of ChatGPT in Improving Student Productivity in
(2023). ChatGPT: five priorities for research. Nature. 2023
Higher Education. Journal on Education, 5(4), 14886-14891.
Feb;614(7947):224-226. https://doi.org/10.1038/d41586-023-00288-7
https://doi.org/10.31004/joe.v5i4.2563
[2] Sallam, Malik. (2023). ChatGPT Utility in Healthcare Education,
[22] Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C,
Research, and Practice: Systematic Review on the Promising
et al. (2023) Performance of ChatGPT on USMLE: Potential for AI-
Perspectives and Valid Concerns. Healthcare 11, no. 6: 887.
assisted medical education using large language models. PLOS Digit
https://doi.org/10.3390/healthcare11060887.
Health 2(2): e0000198. https://doi.org/10.1371/journal.pdig.0000198
[3] Debby R. E. Cotton, Peter A. Cotton & J. Reuben Shipway (2023)
[23] Zachary A. Pardos, Shreya Bhandari, (2023). Learning gain
Chatting and cheating: Ensuring academic integrity in the era of
differences between ChatGPT and human tutor generated algebra
ChatGPT, Innovations in Education and Teaching International,
hints. arXiv Preprint: https://doi.org/10.48550/arXiv.2302.06871.
https://doi.org/10.1080/14703297.2023.2190148.
[24] Zeljana Basic, Ana Banovac, Ivana Kruzic, Ivan Jerkovic (2023).
[4] Li, C., Xing, W. (2021). Natural Language Generation Using Deep
Better by you, better than me, chatgpt3 as writing assistance in
Learning to Support MOOC Learners. Int J Artif Intell Educ 31, 186–
students essays. arXiv Preprint
214. https://doi.org/10.1007/s40593-020-00235-x
https://doi.org/10.48550/arXiv.2302.04536.
[5] Sakib Shahriar, Kadhim Hayawi, (2023). Let's have a chat! A
[25] Sajed Jalil, Suzzana Rafi, Thomas D. LaToza, Kevin Moran, Wing
Conversation with ChatGPT: Technology, Applications, and
Lam (2023),. ChatGPT and Software Testing Education: Promises &
Limitations, arXiv arXiv:2302.13817,
Perils, arXiv Preprint https://doi.org/10.48550/arXiv.2302.03287.
https://doi.org/10.48550/arXiv.2302.13817.
[26] Howard White, Shagun Sabarwal, (2014). Quasi-Experimental Design
[6] Teo Susnjak, (2022) ChatGPT: The End of Online Exam Integrity?
and Methods: Methodological Briefs - Impact Evaluation No. 8, Papers
arXiv Preprint. https://doi.org/10.48550/arXiv.2212.09292.
innpub753, Methodological Briefs.
[7] Sshana, Z.J., & Abulibdeh, E.S. (2020). Science practical work and its
[27] Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021).
impact on students’ science achievement. Journal of Technology and
Conceptualizing AI literacy: An exploratory review. Computers and
Science Education, 10(2), 199-215. https://doi.org/10.3926/jotse.888
Education: Artificial Intelligence, 2, 100041.
[8] Jhonattan Miranda, et.al. (2021), The core components of education
[28] Basit Qureshi, Yasir Javed, Anis Koubâa, Mohamed-Foued Sriti,
4.0 in higher education: Three case studies in engineering education,
Maram Alajlan, (2016). Performance of a Low Cost Hadoop Cluster
Computers & Electrical Engineering, Volume 93, 2021, 107278,
for Image Analysis in Cloud Robotics Environment, Procedia
https://doi.org/10.1016/j.compeleceng.2021.107278.
Computer Science, Volume 82, 90-98,
[9] Taylor, E., Breed, M., Hauman, I., & Homann, A. (2013). Choosing
https://doi.org/10.1016/j.procs.2016.04.013.
Learning Methods Suitable for Teaching and Learning in Computer
[29] Qureshi B, Koubaa A. On Energy Efficiency and Performance
Science. IADIS International Conference on e-Learning, Prague,
Evaluation of Single Board Computer Based Clusters: A Hadoop Case
Czech Republic, Jul 23-26, 2013.
Study. Electronics. 2019; 8(2):182.
[10] Kamrul Ahsan, Suraiyah Akbar & Booi Kam (2022) Contract cheating
https://doi.org/10.3390/electronics8020182.
in higher education: a systematic literature review and future research
[30] B. Qureshi, K. Kawlaq, A. Koubaa, B. Saeed and M. Younis, "A
agenda, Assessment & Evaluation in Higher Education, 47:4, 523-539,
Commodity SBC-Edge Cluster for Smart Cities," 2019 2nd
https://doi.org/10.1080/02602938.2021.1931660
International Conference on Computer Applications & Information
[11] The College of Computer and Information Science at Prince Sultan
Security (ICCAIS), Riyadh, Saudi Arabia, 2019, pp. 1-6, doi:
University. (2023) Last accessed: April 12, 2023. URL:
10.1109/CAIS.2019.8769500.
https://www.psu.edu.sa/en/CCIS/cs-cd
[31] Tate, T. P., Doroudi, S., Ritchie, D., Xu, Y., & uci, m. w. (2023,
[12] Koubaa, A.; Boulila, W.; Ghouti, L.; Alzahem, A.; Latif, S. Exploring
January 10). Educational Research and AI-Generated Writing:
ChatGPT Capabilities and Limitations: A Critical Review of the NLP
Confronting the Coming Tsunami.
Game Changer. Preprints.org 2023, 2023030438.
https://doi.org/10.35542/osf.io/4mec3
https://doi.org/10.20944/preprints202303.0438.v1
[32] PC² (the Programming Contest Control system: Last accessed April
[13] Xingzhi Wang, Nabil Anwer, Yun Dai, Ang Liu, (2023) ChatGPT for
2023, URL: https://pc2ccs.github.io/
design, manufacturing, and education.
[33] Stokel-Walker, C. (2022). AI bot ChatGPT writes smart essays —
http://dx.doi.org/10.13140/RG.2.2.35077.22244
Should professors worry? Nature, d41586-022-04397-7. doi:
[14] Malini M Patil, Rekha PM, Arun Solanki, A Nayyar and B Qureshi,
https://doi.org/10.1038/d41586-022-04397-7
(2022) Performing Big data analytics using swarm-based Long short-
term memory neural network for temperature forecasting Computers,
.