GPTutor: a ChatGPT-powered

programming tool for code explanation

Eason Chen1, Ray Huang2, Han-Shin Chen3, Yuen-Hsien Tseng1 and Liang-Yi Li1
1
National Taiwan Normal University, Taipei, Taiwan
2
KryptoCamp, Taipei, Taiwan
3
University of Toronto, Toronto, Canada
[email protected]

Abstract. Learning new programming skills requires tailored guidance. With the
emergence of advanced Natural Language Generation models like the ChatGPT
API, there is now a possibility of creating a convenient and personalized tutoring
system with AI for computer science education. This paper presents GPTutor, a
ChatGPT-powered programming tool, which is a Visual Studio Code extension
using the ChatGPT API to provide programming code explanations. By integrat-
ing Visual Studio Code API, GPTutor can comprehensively analyze the provided
code by referencing the relevant source codes. As a result, GPTutor can use de-
signed prompts to explain the selected code with a pop-up message. GPTutor is
now published at the Visual Studio Code Extension Marketplace, and its source
code is openly accessible on GitHub. Preliminary evaluation indicates that GPTu-
tor delivers the most concise and accurate explanations compared to vanilla
ChatGPT and GitHub Copilot. Moreover, the feedback from students and teach-
ers indicated that GPTutor is user-friendly and can explain given codes satisfac-
torily. Finally, we discuss possible future research directions for GPTutor. This
includes enhancing its performance and personalization via further prompt pro-
gramming, as well as evaluating the effectiveness of GPTutor with real users.

Keywords: ChatGPT, Tutoring System, Developer Tool, Prompt Engineering,

Natural Language Generation.

1 Introduction

Lately, there has been a rise in the need for skilled programmers, and as a result, many
individuals are opting to learn coding and pursue lucrative software-related careers. At
school, students are crowded in programming courses [1]. Moreover, the gap between
learning and practical application requires students to continue learning after entering
the workforce. For example, in 2020, 42% of beginner-level technology workers joined
the US job market via the Coding Boot Camp [2]. Because of the strong demand for
coding education, there is a shortage of teachers, which makes it difficult to provide
personalized learning in these classrooms. Some students may feel frustrated. While
self-studying and using Google to find solutions to problems can be helpful, there are
times when students may require assistance when reading documents or examples for
an unfamiliar programming language. Furthermore, it is especially challenging when
novice people onboarding a new job and need to catch up by reading others’ codes [3].
2

The code could include domain-specific business logics, which might be unfamiliar to
them, and may be uncomment, poorly maintained, or even unclean.
This paper presents GPTutor as a remedy to relieve programmers from aforemen-
tioned issues. GPTutor is a plugin for Visual Studio Code that uses ChatGPT to provide
detailed explanations of source code. With GPTutor, students can conveniently receive
personalized explanations for coding problems they encounter. Additionally, those
seeking to learn a new programming language can use GPTutor to understand example
code. Finally, new employees needing to quickly familiarize themselves with a code-
base can use GPTutor to gain insights into the business logic behind each line of code.
In sum, the main contributions of this paper are:
1. We developed GPTutor, a Visual Studio Code extension that utilizes the OpenAI
ChatGPT API to provide detailed explanations of the given source code.

2. We demonstrated and explained why GPTutor surpasses other code explain applica-
tions, such as vanilla ChatGPT or GitHub Copilot, by advanced prompt designs.

3. We discussed potential applications, limitations, and future research directions on

programming code explain applications like GPTutor.

2 Background
2.1 Natural Language Generation
Natural Language Generation (NLG) is a subfield of artificial intelligence (AI) that uses
computer algorithms to produce human-like language output from the given input [4].
NLG aims to generate coherent and contextually appropriate language indistinguisha-
ble from human-writing language.
NLG applications may appear to provide intelligent responses to given questions,
but in reality, they just guess next words based on the vast amount of data they read [5].
For example, in Figure 1, if the NLG model uses the Artificial Intelligence in Education
Conference websites as its training data and receives the prompt input "International
Conference on Artificial Intelligence in". In that case, the NLG model may deem "Ed-
ucation" as a more possible follow-up after the given input than other words. As a result,
the NLG model will complete the word with "Education" then continued to generate
possible follow-up texts such as "will take place July 3-7, 2023 in Tokyo". The model
may also produce results such as "July 27-31, 2022 in Durham" or even a fictitious
outcome like "July 20-24, 1969 on the Moon".
By providing additional contextual information in the prompt, the likelihood of the
desired text being generated increases. Figure 1 demonstrates this phenomenon. When
we include the prompt "The 24th" to the beginning of the prompt input, the model will
be more inclined to generate "July 3-7, 2023 in Tokyo" as output since the website
stated that the 24th AIED is held at 2023 in Tokyo. The technique of designing proper
prompts to get the desired output is known as prompt programming [6].
 3

Fig. 1. Example of probability on generating different outputs during NLG process.

2.2 Using NLG for Programming Code Explanation
We could use prompt programming to employ large language models, such as GPT-3,
as a tutor to answer question based on the context [4]. For example, if the NLG model
was trained with lots of document about programming code and its comments/explana-
tions, the model will be able to explain the given code like the example in Figure 2.

Fig. 2. Example of input and output on using NLG model as a code explainer.
Many existing applications, such as GPT-3, ChatGPT, and GitHub Copilot, can per-
form the NLG explanation as shown above in Figure 2. Nevertheless, these applications
still have three main limitations, as presented in Figure 3.
First, existing NLG code explainers are superficial, as they can only offer insights
based on the code present on the current file. Consequently, they may overlook or spec-
ulate domain logics behind the function. This issue becomes particularly noteworthy
when analyzing code with object-oriented design that imports objects from other files.
Second, existing NLG code explainers tend to offer excessive, irrelevant, or even
fictitious information. For instance, if a user asking on a line of code with GitHub Co-
pilot, it may explain the entire code from top to bottom, which is often unnecessary.
4

Lastly, existing NLG code explainers may not be up to date. For example, ChatGPT
was only trained with data until 2021 and, therefore, may perform well with popular
libraries which had a lot of training data at that time. However, it may not provide a
satisfactory explanation when dealing with new, unpopular, or private libraries.
GPTutor was developed to surpass the aforementioned limitations, as shown in Fig-
ure 3. It offers the most concise and accurate explanations. Additionally, it can provide
a comprehensive analysis of the provided code by examining the function's source code.

Fig. 3. Example Code and the comparison of the explanation from ChatGPT, GitHub
Copilot, and the GPTutor.
 5

3 Implementation of GPTutor
In this section, we will first describe how we built the GPTutor Extension with Visual
Studio Code API. Then, we discuss how we enhance its' performance in ChatGPT API.
3.1 Building GPTutor as a Visual Studio Code Extension
We built GPTutor in the Visual Studio Code extension development environment in
Typescript. During the initial setup, the extension will ask users to provide their
OpenAI API key, which will be stored in the extension's global state.
Then, when users request an explanation of code through the GPTutor extension by
command or hot key, the extension will perform the following steps:
1. Use the "editor.document.languageId" API to determine the language of the file.
2. Use the "editor.document.getText" API to obtain the code for the current file.
3. If the cursor is positioned on a function, the GPTutor will additionally use the
"editor.action.revealDefinition" API to retrieve the source code behind the function.
3.2 Getting answer by ChatGPT API with prompt programming
Using the data obtained from the above steps, the GPTutor will create the prompt shown
in Figure 4 for the gpt-3.5-turbo model via the OpenAI API, which was just released
on March 1, 2023. We tried several prompts and found the following formatted in Fig-
ure 4 yielded the most favorable results.

Fig. 4. The prompts GPTutor used to feed into the gpt-3.5-turbo model.

4 Current Results
GPTutor has been published on the Visual Studio Code Extension Marketplace at
https://marketplace.visualstudio.com/items?itemName=gptutor.gptutor, and its source
code is openly accessible at https://github.com/GPTutor/gptutor-extension. Preliminary
user interview with students, programming teachers, and coding boot camp tutors indi-
cated that GPTutor is user-friendly and can explain any given code satisfactorily.
GPTutor especially impresses users with its remarkable ability to incorporate relevant
source codes behind functions into prompts to provide a thorough explanation.
6

5 Discussion and Future Works

5.1 Enhance performance and personalization by prompt programming
GPTutor's superior performance compared to other similar applications can be at-
tributed to its use of more relevant code in its prompts. This enables the NLG model to
provide more desirable answers. We will continue to enhance GPTutor's performance
by optimizing prompts. One possible way is by using heuristic search to identify rele-
vant code in the code base. Then, after transforming the codes into many possible
prompts, GPTutor could provide various explanations [7] to find users preference and
then offer them personalized explanations and a better user experience.
5.2 Evaluate the effectiveness of using GPTutor in the real world
We will investigate the impact of GPTutor on students' comprehension of programming
by observing how they interact with it to complete programming assignments. To assess
the effectiveness of GPTutor, we will collaborate with coding course lecturers and uti-
lize the Between-Subjects Design and the Interrupted Time Series Analysis to measure
the relationship between the student grades and the frequency of the use of GPTutor.

6 Conclusion
We created GPTutor, an extension for Visual Studio Code that leverages ChatGPT to
provide programming code explanations. GPTutor collects relevant code and utilizes
the OpenAI ChatGPT API to explain the chosen code. Comparisons indicate that GPTu-
tor delivers the most concise and accurate explanations compared to Vanilla ChatGPT
and GitHub Copilot. We believe that GPTutor can enhance computer science education
and offer each student a convenient and personalized learning experience in the future.
Acknowledgement. This work was supported by the Ministry of Science and Technol-
ogy of Taiwan (R.O.C.) under Grants 109-2410-H-003-123-MY3 and 110-2511-H-
003-031-MY2. We thank the KryptoCamp for the use cases and preliminary evaluation.
References
1. UCAS, UCAS Undergraduate sector-level end of cycle data resources 2020. Undergraduate
Statistics & Reports, 2021.
2. Seibel, S., et al. Reflections on Educational Choices Made by Coding Bootcamp and
Computer Science Graduates. in Proceedings of the 53rd ACM Technical Symposium on
Computer Science Education V. 2. 2022.
3. Grossman, K.W., The Onboarding Battle and Beyond: Predator or Alien VS. You, in Tech
Job Hunt Handbook: Career Management for Technical Professionals. 2012, Springer. p.
193-203.
4. Brown, T., et al., Language models are few-shot learners. Advances in neural information
processing systems, 2020. 33: p. 1877-1901.
5. Holtzman, A., et al., The curious case of neural text degeneration. arXiv preprint
arXiv:1904.09751, 2019.
6. Reynolds, L. and K. McDonell. Prompt programming for large language models: Beyond
the few-shot paradigm. in Extended Abstracts of the 2021 CHI Conference on Human
Factors in Computing Systems. 2021.
7. Chen, E. and Y.-H. Tseng. A Decision Model for Designing NLP Applications. in
Companion Proceedings of the Web Conference 2022. 2022.