Constructing Depression Prediction Model using

ChatGPT and Machine Learning Algorithms

2023 12th International Conference on Awareness Science and Technology (iCAST) | 979-8-3503-2469-3/23/$31.00 ©2023 IEEE | DOI: 10.1109/iCAST57874.2023.10359295

Long-Sheng Chen Zi-Jie Luo Venkateswarlu Nalluri*

Dept. of Information Management Dept. of Information Management Dept. of Information Management
Chaoyang University of Technology Chaoyang University of Technology Chaoyang University of Technology
Taichung 413310, Taiwan Taichung 413310, Taiwan Taichung 413310, Taiwan
[email protected] [email protected] [email protected]

Abstract—Depression is currently one of the main causes of self- media has been widely used in disaster management [7], drug
harm in human society. However, the insufficient diagnosis of abuse [8], and suicide prevention [9], where depression and
depression has been a long-standing problem. Traditional suicide are strongly associated. In conclusion, there is an
diagnostic methods heavily rely on the patient’s current increasing need to develop intelligent systems that can
emotions, and patients often hesitate to seek help, which effectively address early risk detection issues on social media,
frequently delays intervention. With the deep expansion of social
media, individuals with suicidal thoughts often express their
such as early depression detection, early rumor detection,
perspectives and thoughts on these platforms. Relevant studies crime prevention, or identification of sexual offenders [10].
have also found that people with depression are more likely to The ability to detect these diseases is directly related to
disclose their condition. Therefore, there is a potential to enhance the ability to intervene. If we can analyze the posts made by
the identification of users at risk of suicide by analyzing social social media users and identify their depression at an early
media posts. Although machine learning has been successfully stage, and take necessary measures, we can prevent social
applied to build depression prediction models, the brevity of media users from developing depression-related illnesses or,
social media posts often affects the classification performance of in the best case scenario, prevent them from engaging in self-
these models. Hence, this study aims to introduce ChatGPT to harming behaviors.
automatically generate experimental corpus using information
Although machine learning has been successfully applied
from short comments. Natural language processing (NLP),
Support Vector Machines (SVM), and Naive Bayes (NB) to build depression prediction models, the brevity of social
classifiers are then employed to enhance the classification media posts often affects the classification performance of
accuracy of the depression prediction model. The experimental these models. Hence, this study aims to introduce ChatGPT to
results validate the effectiveness of the proposed method. automatically generate experimental corpus using information
from short comments. Natural language processing (NLP),
Keywords—Depression prediction, Social media, Natural language support vector machines (SVM), and Naive Bayes (NB)
processing, ChatGPT, Machine learning. classifiers are then employed to enhance the classification
accuracy of the depression prediction model. The
I. INTRODUCTION experimental results validate the effectiveness of the proposed
Social media has become people's preferred means of method.
communication, allowing them to express their thoughts,
feelings, and emotions. Unfortunately, the use of social II. LITERATURE REVIEW
networking sites seems to be an additional risk factor for A. Depression prediction in social media
adolescent depression [1]. Despite spending more time on
The world health organization discovered that depression
social media every day, people are increasingly concerned
accounts for the majority of years spent with a handicap
about its impact on their mental health [2]. This has also raised
globally [11]. The goal of early detection is crucial since it can
concerns about the potential for self-harm risks associated
lead to better therapy and results. People can openly
with social media use [3].
communicate with others and share their ideas on online social
Researchers are attempting to utilize user-generated
media sites. These platforms produce a lot of data on the
content from social media for non-clinical mental health
online activities of their users, which can be used to create
assessment and monitoring [4]. Previous studies have found
technology that can identify mental health illnesses. For
that adolescents frequently use social media to share their
instance, it has been demonstrated that language indicators
health and psychological issues [5]. By analyzing the
from Facebook posts are more accurate at predicting
information posted by individuals with severe depression on
depression diagnoses than current screening surveys [12]. For
social media platforms, we can predict whether they have or
a better understanding of the evolution of users’ mental health
may have depression [6]. Researchers analyze user-generated
states, it is particularly interesting to track the language of
data from social media to detect and diagnose signs of such
users through time. For systems to be able to recognize early
mental disorders in individuals [6]. In recent years, social
warning indicators of risk and suggest actions, access to

979-8-3503-2469-3/23/$31.00 ©2023 IEEE 233

Authorized licensed use limited to: Amrita School of Engineering. Downloaded on October 09,2024 at 02:57:03 UTC from IEEE Xplore. Restrictions apply.
lengthy temporal sequences of writings is essential. ChatGPT to help generate content ideas, draft posts, or even
Additionally, knowing and assessing the severity of the write captions for their posts. ChatGPT can be a valuable tool
various symptoms that underlie a mental health condition can for content creators seeking inspiration and efficiency in their
be beneficial for focused strategies to0 choose the proper workflow. Businesses and marketers may use ChatGPT to
course of intervention. analyze social media trends, conduct sentiment analysis, and
The act of recognizing and understanding the connections identify potential target audiences. This information can then
and information flow between individuals, teams, firms, and be used to create more effective social media marketing
other networked information entities is known as social strategies. ChatGPT can be used to monitor social media
network analysis [6]. Moreover, a medical ailment has an conversations and analyze trending topics and sentiments.
impact on a user’s feelings, thoughts, and actions. Depression This data can be valuable for businesses, researchers, and
can cause a variety of emotions, including melancholy and a marketers to gain insights into public opinions and
loss of interest in past hobbies, which can make it harder to preferences.
perform at home and at work. According to the American It is important to note that while AI language models like
Psychiatric Association, it affects 7.2% of individuals overall ChatGPT offer various benefits in the context of social media,
each year. How depression impacts 9.17% of persons will they also raise concerns related to users’ behaviors, content
encounter something throughout their lifetime. The current sharing, and privacy. This is a new direction for research to
body of research examines the relationship between social classify the user’s depression through social media tweets and
media and the evaluation of depressive posts. ChatGPT comments.
In order to categorize sentiment levels, such as neutral,
negative, or positive feelings, researchers use postings and III. METHODOLOGY
comments on social media sites about depression. Social The used methodology has 7 steps which are clarified as
media platforms like Facebook and Twitter are two instances below [13].
of those that are becoming helpful for supporting those who
Step 1: Data collection
need extra care or attention in relation to mental assistance.
They are also employed in network formation and link This study used DepressionTweets dataset on the Kaggle
communication. Numerous depression support groups on website
Facebook are helpful for giving victims emotional assistance. (https://www.kaggle.com/datasets/samrats/depressiontweets).
This dataset contains 62,000 tweets. Among them, it includes
In the study, researchers want to collect postings and
30,000 normal tweet and 32,000 depression tweets.
comments regarding sadness, encode them into a compressed
lexical database, and assess the sentiment levels from each Step 2: Natural language processing
occurrence. [4] asserted that the severe mental disease of Step 2.1: Delete no relevant information
depression has an effect on a person’s psychological, physical,
and emotional well-being. Despite the fact that many Remove unrelated information, e.g., null values, special
academics have looked at different approaches for predicting characters, etc.,
depression, they typically rely simply on looking at facial Step 2.2: Tokenization and delete stop words
expressions. However, although minute and subtle, they might
still have an impact on the model’s performance. Therefore, In this step, we using the Natural Language Toolkit (NLTK) of
Python language for textual tokenization. Then, we remove
accuracy might be improved by classifying the social media
stop words.
posts using NLP analysis and SVM methods in this study.
Step 2.3: Lemmatization
B. The link between ChatGPT and social media
ChatGPT and social media are interconnected in various We simplify word type to the basic form. For instances, “sung”
ways [2]. Social media platforms and ChatGPT can interact and “sang” will be transformed into “sing”.
and complement each other to provide enhanced user Step 3: Generate comments using ChatGPT
experiences and support various functionalities. Here are some Due to the short length of tweets, it can affect the performance
of the key links between ChatGPT and social media. The of classification. Therefore, after natural language processing,
chatbots on social media platforms like Facebook, Twitter, we took the first 20 characters of each tweet (padding zeros for
and Instagram allow businesses to deploy chatbots powered by those shorter than 20 characters) and used ChatGPT to generate
AI language models like ChatGPT. These chatbots can tweets within 50 characters. For the AI-generated new tweets,
interact with users, answer questions, provide product they underwent the same procedure as in step two and
information, and assist with various tasks, improving user proceeded to step four.
engagement and satisfaction. In addition, social media
Step 4: Build TDM
platforms often employ AI, including language models like
ChatGPT, to personalize users' feeds and content The TF-IDF (Term Frequency-Inverse Document Frequency)
recommendations. By understanding users' preferences and weights are the multiplication of the TF and IDF values for
behaviors, these platforms can curate content that is more each word in each document. The TF-IDF shown in equation
relevant and engaging to individual users. In content creation, (1) is constructed into a Term-Document Matrix (TDM).
some social media users leverage AI language models like

234

Authorized licensed use limited to: Amrita School of Engineering. Downloaded on October 09,2024 at 02:57:03 UTC from IEEE Xplore. Restrictions apply.
 (1) the Jupyter Notebook software environment. The
parameter settings for each model are shown in Table 2.
is the ith term and is the jth document. N( ) indicates the
total number of documents with occurrences of terms, where B. Eempoyed Data Set
a document represents a comment.
The dataset used in this study is from the
Step 5: Build the classifier DepressionTweets dataset on the Kaggle website
This study employs Support Vector Machine (SVM) and (https://www.kaggle.com/datasets/samrats/depressiontweets).
Naive Bayes (NB) to build classifiers. In SVM, we choose This dataset contains 62,000 tweets from Twitter along with
the RBF kernel function as equation (2) information on whether the user has depression. It includes
30,000 tweets from users without depression and 32,000
(2)
tweets from users with depression.
As this study is exploratory in nature, we initially
Step 6: Performance evaluation
conducted experiments using a sample of 25 tweets each
In this study, we used overall accuracy and F1 to from non-depressed and depressed users. Table 3 illustrates
effectively evaluate the classification performance. an example of depressed tweets, where it can be seen that the
Step 7: Making conclusions original tweets were too short. Therefore, using ChatGPT, we
generated comments within 50 characters, which then
Finally, based on the results of the SVM and NB became the new left-hand side tweets in Table 3.
classifiers, we can make some conclusions. Potential direction
of future works also could be provided. Table 3 An example of original and AI generated (ChatGPT)
depression comments
IV. EXPERIMENT RESULTS Original comment AI generated (ChatGPT) comment
A. Experiment Settings I absolutely hate how my life has been
This section describes the training model environment. in the past. The memories haunt me
Table 1 shows the hardware specifications used in this relentlessly, reminding me of
experiment, which include an Intel Core i7-11800H processor everything I've done wrong. It feels like
with 8 cores (CPU), an RTX 3060 Laptop GPU accelerator, there's a constant storm inside me, and I
and 16GB DDR4-3200 RAM. twitter wording priv can't escape it. I see others on Twitter,
absolutely hate done living their best lives, and it makes me
Table 1 Experiment settings past goddamn life feel even worse. The weight of my
Operation Systems Windows 11 deserve mistakes and regrets is suffocating. I
CPU I7-11800 H(2.30 GHz, max can't help but think that I don't deserve
4.60 GHz) anything good in life. Every word I say
GPU RTX 3060 Laptop(6GB) feels wrong, and I wish I could
disappear, away from the judgment and
RAM DDR4-3200 (16GB)
disappointment.
Table 2 Parameter settings
Subsequently, we performed natural language processing
Model Parameter settings
procedures. The original comments were transformed into a
Original comments:
term-document matrix (TDM) of size 50 (number of data
alpha=10, class_prior=None,
points) × 500 (499 word count + 1 label for depression status).
fit_prior=True
NB After generating comments using ChatGPT and applying
AI generated comments (Chat GPT):
natural language processing, a TDM of size 50 × 835 (834
alpha=0.00001, class_prior=None,
word count + 1 label for depression status) was created. Next,
fit_prior=True
we randomly split the dataset, using 70% as the training set
Original comments:
and 30% as the test set. After conducting five repetitions of
kernel=rbf, C=10, gamma=0.1
SVM the experiment, we compared the results.
AI generated comments (Chat GPT):
kernel=rbf, C=10, gamma=0.1
C. Results
This study was conducted in the Jupyter Notebook Table 4 shows the results of the Naive Bayes (NB)
software environment using the Python programming classifier. If we create a depression prediction model using the
language for data preprocessing. The Naive Bayes original tweet text, we can observe that the average accuracy
classifier and SVM algorithm were also implemented in is only 53.34%, and the average F1 score is only 41.34%.

235

Authorized licensed use limited to: Amrita School of Engineering. Downloaded on October 09,2024 at 02:57:03 UTC from IEEE Xplore. Restrictions apply.
Some experiments even yielded an F1 score of zero, indicating sample size and considering the adoption of other classifiers to
poor classification performance. validate the effectiveness of the proposed method outlined in
On the other hand, when using the depression prediction this paper.
model generated by ChatGPT, the average accuracy improved
from 53.34% to 69.34%, and the F1 score significantly ACKNOWLEDGMENT
increased from 41.34% to 57.18%. This work was supported in part by the Ministry of Science
and Technology of Taiwan, R.O.C. (Grant No NSTC 112-
Table 4 Results of NB 2410-H-324-004). We also express our thanks for supports
AI generated from Chaoyang University of Technology.
Original comments
comments
Experiment REFERENCES
Accuracy F1 Accuracy F1
(%) (%) (%) (%)
[1] Li, J.-B., Lau, J.T.F., Feng, L.-F., Zhang, X., Li, J.-H., Mai, J.-C., Chen,
#1 60 70 73.3 60 Y.-X., Mo, P.K.H., 2022, Associations of intensity and emotional
#2 53.3 66.7 60 40 connection related to online social networking use on the risk of incident
#3 46.7 0 80 76.9 depression among Chinese adolescents: A prospective cohort study,
Journal of Affective Disorders, 308, pp. 116-122.
#4 46.7 0 66.7 54.5 [2] Hassan, H., Hsbollah, H. M., Mohamad, R., 2022, Examining the
#5 60 70 66.7 54.5 interlink of social media use, purchase behavior, and mental health,
Procedia Computer Science, 196, pp. 85-92.
Mean 53.34 41.34 69.34 57.18
[3] Macrynikola, N., Auad, E., Menjivar, J., Miranda, R., 2021, Does social
Standard media use confer suicide risk? A systematic review of the evidence,
5.948 33.775 6.79 11.886
Deviation Computers in Human Behavior Reports, 3, 100094.
[4] Malhotra, A., Jindal, R. 2022, Deep learning techniques for suicide and
Table 5 presents the results of the Support Vector depression detection from online social media: A scoping review,
Applied Soft Computing, 130, 109713.
Machine (SVM) classifier. As we tuned the parameters to their
[5] Zaghouani, W., 2018, A Large-Scale Social Media Corpus for the
optimal values, we only observed a slight improvement in the Detection of Youth Depression (Project Note), Procedia Computer
classification performance. Science, 142, pp. 347-351.
[6] Ortega-Mendoza, R. M., D. I. Hernández-Farías, M. Montes-y-Gómez,
Table 5 Results of SVM L. Villaseñor-Pineda, 2022, Revealing traces of depression through
personal statements analysis in social media, Artificial Intelligence in
AI generated Medicine, 123, 102202.
Original comments
comments [7] Ogie, R. I., S. James, A. Moore, T. Dilworth, M. Amirghasemi, J.
Experiment
Accuracy F1 Accuracy F1 Whittaker, 2022, Social media use in disaster recovery: A systematic
(%) (%) (%) (%) literature review, International Journal of Disaster Risk Reduction, 70,
102783.
#1 93.3 93.3 86.7 87.5
[8] Jenhani, F., M. S. Gouider, L. B. Said, 2019, Hybrid System for
#2 93.3 93.3 86.7 88.9 Information Extraction from Social Media Text: Drug Abuse Case
#3 93.3 93.3 86.7 88.9 Study, Procedia Computer Science, 159, pp. 688-697.
#4 86.7 85.7 93.3 94.1 [9] Dogrucu, A., A. Perucic, A. Isaro, D. Ball, E. Toto, E. A. Rundensteiner,
E. Agu, R. Davis-Martin, E. Boudreaux, 2020, Moodable: On feasibility
#5 80 80 93.3 94.1 of instantaneous depression assessment using machine learning on voice
Mean 89.32 89.12 89.34 90.7 samples with retrospectively harvested smartphone and social media
data, Smart Health, 17, 100118.
Standard
5.315 5.427 3.233 2.823 [10] Burdisso, S. G., M. Errecalde, M. Montes-y-Gómez, 2019, A text
Deviation classification framework for simple and effective early depression
detection over social media streams, Expert Systems with Applications,
133, pp. 182-197.
V. CONCLUSSIONS [11] Keerthiga, M., Abisha, D., Kalaiselvi, P., & Shenbagalakshmi, S. (2023,
April). Machine Learning-based Depression Prediction using Social
It has been previously established that building a Media Feeds. In 2023 International Conference on Inventive
depression prediction model based on social media posts is Computation Technologies (ICICT) ( pp . 863-869). IEEE.
feasible. However, due to the shorter length of social media [12] Nalluri, V., Chang, JR, Chen, LS, & Chen, JC (2023). Building
user posts, it often affects the classification performance. This prediction models and discovering important factors of health insurance
study proposes combining natural language processing with fraud using machine learning methods. Journal of Ambient Intelligence
and Humanized Computing , 14 ( 7 ) , 9607-9619.
ChatGPT and SVM/NB classifiers, successfully demonstrating
[13] Wang, Y.-Y., Luo, Z.-J., Chen, M.-C., Chen, L.-S., 2023, Classifying
that adopting the proposed method can indeed improve the Game Reviews by Using Natural Language Processing and Support
accuracy of depression prediction models under NB and SVM Vector Machines with SMOTE-Tomek Algorithm, 14th International
classifiers. Congress on Advanced Applied Informatics, July 8-13, 2023, Koriyama,
Japan.
Since this study only used a sample of 50 instances for
preliminary research, future work can involve increasing the

236

Authorized licensed use limited to: Amrita School of Engineering. Downloaded on October 09,2024 at 02:57:03 UTC from IEEE Xplore. Restrictions apply.