Using Text Classification to Estimate the Depression Level of Reddit Users

Sergio Gastón Burdisso; Marcelo Errecalde; Manuel Montes-y-Gómez

doi:10.24215/16666038.21.e1

Authors

Sergio Gastón Burdisso UNSL/CONICET
Marcelo Errecalde UNSL https://orcid.org/0000-0001-5605-8963
Manuel Montes-y-Gómez https://orcid.org/0000-0002-7601-501X

DOI:

https://doi.org/10.24215/16666038.21.e1

Keywords:

Beck's Depression Inventory, CLEF eRisk 2019, Depression Level Estimation, SS3, Text Classification

Abstract

Psychologists have used tests and carefully designed survey questions, such as Beck's Depression Inventory (BDI), to identify the presence of depression and to assess its severity level.
On the other hand, methods for automatic depression detection have gained increasing interest since all the information available in social media, such as Twitter and Facebook, enables novel measurement based on language use.
These methods learn to characterize depression through natural language use and have shown that, in fact, language usage can provide strong evidence in detecting depressive people.
However, not much attention has been paid to measuring finer grain relationships between both aspects, such as how is connected the language usage with the severity level of depression.
The present study is a first step towards that direction.
We train a binary text classifier to detect ``depressed'' users and then we use its confidence value to estimate the user's clinical depression level.
In order to do that, our system has to be able to fill the standard BDI depression questionnaire on users' behalf, based only on their posts in Reddit.
Our proposal was publicly tested in the eRisk 2019 task obtaining the best and second-best performance among the other 13 submitted models.

Downloads

Download data is not yet available.

References

World Health Organization, Depression and other common mental disorders: global health estimates. WHO, 2017.

World Health Organization, Preventing suicide: a global imperative. WHO, 2014.

National Center for Health Statistics, “Mortality in the United States, 2017.” https://www.cdc.gov/nchs/products/databriefs/db328.htm, 2019. [Online; accessed 13-April-2019].

A. T. Beck, C. H. Ward, M. Mendelson, J. Mock, and J. Erbaugh, “An inventory for measuring depression,” Archives of general psychiatry, vol. 4, no. 6, pp. 561– 571, 1961.

D. E. Losada and F. Crestani, “A test collection for research on depression and language use,” in International Conference of the Cross-Language Evaluation Forum for European Languages, pp. 28–39, Springer, 2016.

D. E. Losada, F. Crestani, and J. Parapar, “erisk 2017: Clef lab on early risk prediction on the internet: Experimental foundations,” in International Conference of the Cross-Language Evaluation Forum for European Languages, pp. 346–360, Springer, 2017.

D. E. Losada, F. Crestani, and J. Parapar, “Overview of erisk: Early risk prediction on the internet,” in International Conference of the Cross-Language Evaluation Forum for European Languages, pp. 343–361, Springer, 2018.

S. G. Burdisso, M. Errecalde, and M. M. y G´omez, “Towards measuring the severity of depression in social media via text classification,” in Actas del XXV Congreso Argentino de Ciencias de la Computaci´on (CACIC 2019), pp. 577–588, 2019.

D. E. Losada, F. Crestani, and J. Parapar, “Overview of eRisk 2019: Early Risk Prediction on the Internet,” in Experimental IR Meets Multilinguality, Multimodality, and Interaction. 10th International Conference of the CLEF Association, CLEF 2019, (Lugano, Switzerland), Springer International Publishing, 2019.

S. G. Burdisso, M. Errecalde, and M. M. y G´omez, “A text classification framework for simple and effective early depression detection over social media streams,” Expert Systems with Applications, vol. 133, pp. 182 –197, 2019.

S. G. Burdisso, M. Errecalde, and M. Montes-y G´omez, “t-SS3: A text classifier with dynamic n-grams for early risk detection over text streams,” Pattern Recognition Letters, vol. 138, pp. 130 – 137, 2020.

D. G. Funez, M. J. G. Ucelay, M. P. Villegas, S. G. Burdisso, L. C. Cagnina, M. Montes-y G´omez, and M. L. Errecalde, “UNSL’s participation at erisk 2018 lab,” in Experimental IR Meets Multilinguality, Multimodality, and Interaction. 9th International Conference of the CLEF Association, CLEF 2018, (Avignon, France), Springer International Publishing, 2018.

M. L. Errecalde, M. P. Villegas, D. G. Funez, M. J. G. Ucelay, and L. C. Cagnina, “Temporal variation of terms as concept space for early risk prediction.,” in Experimental IR Meets Multilinguality, Multimodality, and Interaction. 8th International Conference of the CLEF Association, CLEF 2017, (Dublin, Ireland), Springer International Publishing, 2017.

P. Abed-Esfahani, D. Howard, M. Maslej, S. Patel, V. Mann, S. Goegan, and L. French, “Transfer learning for depression: Early detection and severity prediction from social media postings,” inWorking Notes of CLEF 2019 - Conference and Labs of the Evaluation Forum, (Lugano, Switzerland), 2019.

A. Trifan and J. L. Oliveira, “Bioinfo@ uavr at erisk 2019: delving into social media texts for the early detection of mental and food disorders,” in Working Notes of CLEF 2019 - Conference and Labs of the Evaluation Forum, (Lugano, Switzerland), 2019.

P. van Rijen, D. Teodoro, N. Naderi, L. Mottin, J. Knafou, M. Jeffryes, and P. Ruch, “A data-driven approach for measuring the severity of the signs of depression using reddit posts,” in Working Notes of CLEF 2019 - Conference and Labs of the Evaluation Forum, (Lugano, Switzerland), 2019.

T. Wilson, J. Wiebe, and P. Hoffmann, “Recognizing contextual polarity in phrase-level sentiment analysis,” in Proceedings of human language technology conference and conference on empirical methods in natural language processing, pp. 347–354, 2005.

G. A. Miller, WordNet: An electronic lexical database. MIT press, 1998.

A. Kraskov, H. St¨ogbauer, and P. Grassberger, “Estimating mutual information,” Physical review E, vol. 69, no. 6, p. 066138, 2004.

K. Ethayarajh, “Unsupervised random walk sentence embeddings: A strong but simple baseline,” in Proceedings of The Third Workshop on Representation Learning for NLP, pp. 91–100, 2018.

A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving language understanding by generative pre-training,” URL https://s3-us-west-2. amazonaws. com/openaiassets/researchcovers/languageunsupervised/languageunderstanding paper. pdf, 2018.

J.W. Pennebaker, R. L. Boyd, K. Jordan, and K. Blackburn, “The development and psychometric properties of LIWC2007,” 2007. LIWC, Austin, Texas.