Original article

Med Lav 2023; 114 (1): e2023009

DOI: 10.23749/mdl.v114i1.13918

Psychometric Evaluation of GHQ-12 as a Screening

Tool for Psychological Impairment of Healthcare
Workers Facing COVID-19 Pandemic
Anna Comotti1*, Alice Fattori2, Francesca Greselin3, Lorenzo Bordini1,
Paolo Brambilla4,5, Matteo Bonzini1,2
1
Occupational Health Unit, Foundation IRCCS Ca’ Granda Ospedale Maggiore Policlinico, Milan, Italy
2
Department of Clinical Science and Community Health, University of Milan, Milan, Italy
3
Department of Statistics and Quantitative Methods, University of Milano-Bicocca, Milan, Italy
4
Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy
5
Department of Neurosciences and Mental Health, Foundation IRCCS Ca’ Granda Ospedale Maggiore Policlinico, Milan, Italy

Keywords: General Health Questionnaires (GHQ-12); Occupational stress; Health-Care Workers; Item
Response Theory

Abstract
Background: The General Health Questionnaire (GHQ) is a widely used tool in clinical and research settings
due to its brevity and easy administration. Researchers often adopt a dichotomous measurement method, consider-
ing a total score above or below a certain threshold, leading to an extreme simplification of the gathered data and,
therefore, the loss of clinical details. In a multistep evaluation study aimed at assessing health care workers’ mental
health during the COVID-19 pandemic, GHQ-12 proved to be the most effective tool to detect psychological distress
compared to other scales. These results deepened the understanding of GHQ-12 properties through a statistical study
focusing on items’ properties and characteristics. Methods: GHQ-12 responses were analyzed using Item Response
Theory (IRT), a suitable method for scale assessment. Instead of considering the single overall score, in which each
item accounts equally, it focuses on individual items’ characteristics. Moreover, IRT models were applied combined
with the latent class (LC) analysis, aiming to determine subgroups of individuals according to their level of psycho-
logical distress. Results: GHQ-12 was administered to 990 healthcare workers, and responses were scored using
the binary method (0-0-1-1). We applied the two-parameter logistic (2-PL) model, finding that the items showed
different ways of responses and features. The latent class analysis classified subjects into three sub-groups according
to their responses to GHQ-12 only: 47% of individuals with general well-being, 38% expressing signs of discomfort
without severity, and 15% of subjects with a high level of impairment. This result almost reproduces the subjects’ clas-
sification obtained after administering the six questionnaires of the study protocol. Conclusions: Accurate statistical
techniques and a deep understanding of the latent factors underlying the GHQ-12 resulted in more effective usage of
such a psychometric questionnaire – i.e., a more refined gathering of data and significant time and resource efficiency.
We underlined the need to maximize the extraction of data from questionnaires and the necessity of them being less
lengthy and repetitive.

Received 09.11.2022 - Accepted 23.01.23

Corresponding Author: Anna Comotti, Occupational Health Unit, Foundation IRCCS Ca’ Granda Ospedale Maggiore Policlinico,
Milan, Italy; E-mail: [email protected]
2 Comotti et al

1. Introduction We proposed to analyze GHQ-12 data through

Item Response Theory (IRT)-based methods as
The General Health Questionnaire (GHQ) aims they provide more details about individual items.
to provide information about an individual’s mental IRT is a specific statistical model for evaluating
well-being by identifying distressing symptoms [1]. questionnaires, and it is a more suitable tool than the
Its shorter version (GHQ 12-item) has become one usual methodologies based on Classical Test Theory
of the most widely used scales for assessing psycho- (CTT), whose use is still prevalent in the psycho-
logical distress and short-term changes in mental metric field. The strength of such a technique lies
health, and its popularity can be mainly attributable in its focus on items rather than individual scores,
to its brevity and easy administration [2]. while in the CTT, the evaluation of test properties
GHQ-12 has shown strong psychometric prop- and item characteristics is not included.
erties and it is recommended as screening tool to From a statistical point of view, if the results of
detect common mental disorders as depressive, anx- a test are reported as a single score, it is implicitly
iety and somatic disorders [3, 4]. assumed that all the items are measuring the same
Several analyses explored its characteristics, espe- trait equally therefore losing the complexity of un-
cially the factor structure, mostly identifying a two- derlying traits in psychological testing. IRT allows
factor solution through Confirmatory Factor Analysis to evaluate individual-level distress and to describe
(CFA) techniques. The two factors commonly pro- the performances of the items on the questionnaire
posed were the Depression/Anxiety construct (re- simultaneously may providing a better clinical in-
lated to the emotional component of psychological sight on symptoms detected and associations with
distress) and the Social Dysfunction construct (re- potential underlying mental disorders.
lated to the social functioning component of the in- To our knowledge, few authors proposed analyz-
dividual experiencing the distress) [5]. Other authors ing the GHQ-12 scale via an IRT approach. In some
proposed a three-dimensional solution, comprising cases, IRT was applied as a suitable tool to deter-
anxiety (4-item), social dysfunction (6-item), and loss mine the factor structure of the scale [16, 17], and
of confidence (2-item) dimensions [6]. more recently through the multidimensional version
The use of GHQ-12 to measure the mental of IRT [18]. For instance, the IRT approach used by
health status in healthcare populations is frequent, Smith et al. [16] explored the fact that item phrasing,
and several recent contributions gave examples of its item variance and levels of respondents’ distress affect
application in analyzing psychological well-being the factor structure observed for the GHQ-12 and
during the pandemic [7-14]. In all these cases, the may perhaps explain why different factor structures of
screening of the psychological status through GHQ- the instrument have been found in different popula-
12 was determined according to its total score. The tions. Other uses of IRT on GHQ-12 regard comput-
scores typically used are the binary scale (0-0-1-1) erized adaptive testing [19] or Mokken analysis [20].
and the 4-point Likert-type scale (0-1-2-3). Re- However, no studies have studied the perfor-
sponses to all items are summed up to a total score mance of GHQ-12 with IRT in an occupational
ranging from 0 to 12 (binary scale) or 0 to 36 (Lik- setting during a pandemic. Therefore, this work aims
ert scale), with higher scores indicating more severe to perform an IRT-based analysis on GHQ-12, in-
impairment. A score above a specific cut-off (3/4 for vestigating the methodological and clinical benefits
bimodal and 13/14 for the Likert scale) indicates of such an approach.
psychological distress and suggests further investi-
gation for potential mental disorders [15]. 2. Methods
A possible difference in items contribution can be
lost through such a measurement method, in which 2.1 Population Study
each item counts the same. Indeed, every single item
may have a different weight, expressing different se- We conducted a multistep epidemiological study
verity of the psychological impairment measured by within occupational health surveillance to system-
the test. atically assess healthcare workers’ mental well-being
 Psychometric Evaluation of GHQ-12 3

during the COVID-19 pandemic in a large Hospi- Six hundred and twenty-seven workers (63%) did
tal in Milan (Italy). not show signs of psychological impairment; 363
GHQ-12 was administered, jointly with the Im- (37%) presented signs of psychological impairment
pact of Event Scale (IES-R; post-traumatic distress, at the first screening level (i.e., with scores above
[21]) and General Anxiety Disorder Scale (GAD-7; the cut-off in at least one scale among GHQ-12,
anxiety, [22]) questionnaires, to assess the psy- IES-, R and GAD-7) and underwent the second
chological impact of the pandemic and to identify level assessment. Among these, 231 (67%; 23%
possible signs of impairment, further investigated of the total sample) scored above the cut-off in at
through psycho-diagnostic questionnaires and spe- least one scale among PHQ-9, DES, and SCL-90.
cialist evaluation. As a result, we were able to classify participants
The steps of such assessment were fully presented into three sub-groups, according to their scorings:
in a previous study [23]: for each worker, the psycho- a group with no evidence of psychological distress
logical well-being was screened in three steps. The after first-level screening (Group 1, N=627), work-
first-level questionnaire collected several personal ers who expressed distress without severe symptoms
information and data from three tests (i.e., GHQ- (Group 2, N=132), and subjects who expressed signs
12, IES-R, and GAD-7). Workers who scored above of impairment and received psychological and/or
the cut-off in at least one scale were further inves- psychiatric support (Group 3, N=231).
tigated by the second-level questionnaire composed Out of the 363 subjects who showed psycho-
of psycho-diagnostic scales to assess depressive logical impairment at first-level screening, almost
symptoms (Patient Health Questionnaire-9; [24]), all (91%) scored above the cut-off (equal to 4) of
dissociative symptoms (Dissociative Experiences GHQ-12, while about half of them over-passed the
Scale –II; [25]) and other psychological symptoms cut-off of IES-R and GAD-7 scales (53% and 56%
(Symptoms Checklist-90 [26]). If the second level respectively). This result suggested that GHQ-12
showed psychological impairments, an individual could determine the transition to the second level
specialized treatment with a psychiatrist and psy- more effectively than the other scales.
chologist (third-level) was offered. Results obtained from the analysis of risk factors
In this framework, GHQ-12 (binary version) for psychological impairment were presented in de-
proved to be an effective screening tool, deserving a tail in a previous paper [27].
deeper investigation [27].
The occupational medicine unit, where workers 2.2 Item Response Theory (IRT)
underwent the periodical health surveillance al-
ready prescribed by the current Italian legislation, The basic assumption of IRT models is that a per-
proposed the study protocol to all workers since son’s interactions with test items can be represented
July 2020. By July 2021, 990 subjects out of a total according to probabilistic relations, containing a
population of 1,610 had been enrolled in the study. single parameter to describe the individual’s char-
The participation rate was 62%. In detail, 220 (13%) acteristics θ. The power of IRT is that it estimates
workers did not answer our calls or were unavailable item characteristics through some item parameters,
and 400 (25%) refused to participate. which permit the calculation of the expected score
Participants were predominantly female (70%) at the item level (e.g., probability of 1 or correct an-
with a mean age of 45 years (sd=11); nurses (42.5%) swer if responses are binary or dichotomous) and at
was the most prominent job category, followed the test level. In addition, the person’s latent trait
by physicians (23.5%), administrative staff (12%), θi for an individual i is also estimated, considering
health assistants (6.5%) and other roles (16%). Four specific item characteristics and how the person an-
hundred and forty-six (45%) participants had the swers to each item.
experience of working in a COVID-19 area: 25% We applied the so-called two parameters logis-
were working with COVID-19 patients during data tic (2-PL), suitable for binary data, which uses two
collection, and 20% had worked in a COVID-19 parameters to describe each item j, corresponding
department before enrollment. to its “difficulty” and its “discrimination”. The item
4 Comotti et al

difficulty represents the level of latent trait for level of psychological distress). Table 1 presents the
which one has a 50% probability of responding ‘cor- twelve questionnaire items with the distribution of
rectly’ (or 1) to that item. In other words, if θi=βj, the respective answers.
then P(Yij=1)=0.5. If θi>βj, then P(Yij=1)>0.5 and if Item parameters estimation (difficulty and dis-
θi<βj, then P(Yij=1)<0.5. The discriminating param- crimination) of the 2-PL model are reported and
eter for item j, λj, estimates the capacity of the item graphically represented in Figure 1, which shows
to distinguish between subjects with different latent the ICCs.
trait levels. Item 5 (followed by Item 7) has the lowest
Two plots are typically employed in the IRT threshold parameter, while Item 3 (“feeling useless”)
framework to visualize the analysis results. Item and Item 11 (“thinking of yourself as a worthless
Characteristic Curves (ICCs) show the probability person”) are the most ‘difficult’ ones. Item 5 has a lo-
of answering 1 to an item at varying levels of the cation (or difficulty) parameter equal to 0, meaning
latent trait, specifying how well an item discrimi- that a person with a latent trait θ at level 0 has the
nates between respondents at various levels of the same probability of answering 0 or 1 to Item 5. On
latent trait. The “easier” items functions are on the the contrary, a level of latent trait θ equal to 1.73 is
left side of the plot, in the lower regions of the la- needed for having an equal probability of answering
tent trait scale, while the more “difficult” items are less than/same as usual or more/much more than
on the right (in our case, they are the items that un- usual to Item 11. Concerning the discrimination pa-
derlie more severe impairment in mental health). rameter, Item 12 (feeling reasonably happy, all things
The discrimination parameter represents the slope, considered) has the highest value, much greater than
which refers to how well the item response options the others. On the other hand, Item 3 has the lowest
discriminate (or differentiate) between subjects with discrimination parameter. The item provides sample
high and low latent trait levels. information about differences across individuals
IICs show how well and precisely each item when discrimination is high. Item 5 and Item 7 have
measures the latent trait at various attribute levels. the leftmost lines represented in the plot, while the
Item Information measures the strength of the rela- curve of Item 3, which is also the less steep, is plot-
tionship between an item and the latent trait. Some ted on the right. The curve of Item 12 is indeed the
items may provide more information at low levels of steepest one. In the IICs plot in Figure 2, the curve
the attribute, while others may provide more infor- of Item 12 gives much information around a value
mation at higher levels of the attribute.
We applied the IRT model in one of its discrete
versions, based on the so-called Latent Class (LC) Table 1. GHQ-12 answers distribution.
analysis [28, 29], whose assumption is that the pop- 0 1
ulation under study is composed of homogeneous Item 1 - Able to concentrate 75% 25%
classes of individuals who have very similar unob- Item 2 - Loss of sleep over worry 64% 36%
servable characteristics [30, 31]. Data were collected Item 3 - Playing a useful part 86% 14%
through a computerized database generated by
Item 4 - Capable of making decisions 85% 15%
REDCap [32], which was subsequently analyzed by
Item 5 - Felt constantly under strain 49% 51%
R software [33].
Item 6 - Could not overcome difficulties 78% 22%
3. Results Item 7 - Able to enjoy day-to-day activities 55% 45%
Item 8 - Able to face problems 79% 21%
Based on the dichotomous scored version of Item 9 - Feeling unhappy and depressed 68% 32%
GHQ-12, we calculated Cronbach’s α equal to 0.87, Item 10 - Losing confidence 83% 17%
indicating good internal consistency. The mean score
Item 11 - Thinking of self as worthless 92% 8%
was 3.31 (SD=3.45), with 37% of subjects scoring
Item 12 - Feeling reasonably happy 73% 27%
above the cut-off equal to 4 (indicating a general
 Psychometric Evaluation of GHQ-12 5

1.00
Item 5 , β = -0.02 , λ = 2.34
Item 7 , β = 0.18 , λ = 2.34
Item 2 , β = 0.46 , λ = 2.5
0.75
Item 9 , β = 0.54 , λ = 3.59
Item 12 , β = 0.67 , λ = 4.03
P (θ)

0.50 Item 1 , β = 0.91 , λ = 1.93

Item 6 , β = 0.93 , λ = 2.67
Item 8 , β = 0.97 , λ = 2.74
0.25 Item 10 , β = 1.09 , λ = 3.01
Item 4 , β = 1.54 , λ = 1.58
Item 11 , β = 1.7 , λ = 2.43
0.00 Item 3 , β = 1.91 , λ = 1.18
0 2 4
θ

Figure 1. Item Characteristic Curves for GHQ-12 questionnaire.

Item 1
4

Item 2
Item 3
Item 4
3

Item 5
Item 6
l(θ)

Item 7
2

Item 8
Item 9
Item 10
1

Item 11
Item 12
0

-4 -2 -0 2 4
θ
Figure 2. Item Information Curves for GHQ-12 questionnaire.

of θ between 0 and 1, while Item 3 (with the low- We compared the three latent classes with the
est discrimination parameter) gives more or less the three groups resulting from questionnaire scorings
same (low) information over a broader range. (Group 1, Group 2, Group 3) in Table 2. Almost all
The latent class model reaches the best fit (97%) subjects belonging to Class 1 did not undergo
­(calculated through BIC) with three latent classes. the second-level screening, i.e., they did not express
The weights and levels of the latent trait for each any sign of discomfort through GHQ-12, IES-R,
dimension and latent class are in Table 2. The la- and GAD-7. On the contrary, most (85%) of those
tent model estimates three values for the θ finding assigned to Class 3 needed psychological support,
support points equal to -3.3 (low level of distress) while only one-third of Class 2 required psychologi-
with weight equal to 0.47, a medium level (around cal therapy.
0) for 38% of subjects, and a higher level of distress Table 3 shows the percentage of answers equal to
(θ=1.15) with weight 0.15. 1 for each item, according to the latent class.
6 Comotti et al

Table 2. Latent class model results and percentage of subjects by multistep evaluation.
Group 1 Group 2 Group 3
θ % (no distress) (psychological distress) (psychological impairment)
Class 1 -3.3 47% 97% 1% 2%
Class 2 -0.8 38% 44% 20% 36%
Class 3     1.2 15% 1% 14% 85%

Table 3. Probability of answering equally to 1 by class.

ITEM
1 2 3 4 5 6 7 8 9 10 11 12
Class 1 3% 4% 4% 2% 17% 1% 16% 1% 1% - - -
Class 2 31% 52% 17% 17% 76% 24% 64% 21% 45% 15% 6% 33%
Class 3 76% 92% 38% 50% 96% 81% 90% 81% 95% 76% 41% 97%

Participants assigned to Class 1 answered 0 for overall score, in which each item accounts equally,
almost all the items, with the highest percentages the item-based analysis produced interesting results
of the answer 1 occurring for Items 5 and 7 (but by identifying specific items able to detect psycho-
much less than in the general distribution). On the logical impairment effectively.
contrary, considering such items (Items 5 and 7), Such considerations are similar to those ob-
almost everyone who belongs to Class 3 answered tained in previous analyses on GHQ-12 based on
0. In addition, for the group with more severe signs IRT methods in other frameworks. For example,
of psychological distress, percentages of answer 1 the approach of Smith and colleagues [18], within
were much higher (than the general distribution) the multi-dimensionality assessment of GHQ-12,
up to the items found to be the most “difficult” showed how the use of the summated scores for the
(Item 3 and Item 11). For the second class, the dis- GHQ-12 could potentially lead to an incorrect as-
tribution was more balanced, and more than half of sessment of patients’ psychiatric morbidity.
the participants answered 1 only to Item 2, Item 5, The focus on items characteristics allowed us to
and Item 7. deeply investigate how the mental health status was
captured by GHQ-12 in our population, identifying
4. Discussion different levels of severity (given by item difficulty)
and quantifying the impact each item had on the
The GHQ-12 is frequently used among different measurement of general distress. We further specify
settings and populations, and its assessment meth- that participants were healthcare workers involved
ods adopt predominantly a dichotomous scoring, in a disruptive pandemic, which imposed them un-
which may contribute to lose potential differences precedented and heavy workloads coupled with lack
in items contribution; this rationale motivated us to pf preparation to cope with such demands. In such
a psychometric analysis to better clarify the meth- circumstances, questions about utility, capacity to
odological and clinical quality of this tool. The anal- make decisions, loss of trust and confidence showed
ysis was carried out within a study aimed to evaluate peculiar responses, affecting in different way the
psychological well-being of healthcare workers in a psychological wellbeing; feeling useless (Item 3) and
large Hospital in Milan (Italy) facing COVID-19 thinking of yourself as a worthless person (Item 11)
pandemic. caused more severe impairments than, for instance,
In our scenario, the IRT was a suitable tool for feeling constantly under strain (Item 5) or being un-
scale assessment. Instead of considering the single able to enjoy day-to-day activities (Item 7).
 Psychometric Evaluation of GHQ-12 7

The analysis on the item response patterns also al- pandemic in Italy (e.g., more than 50% above cut-
lowed the classification of subjects according to dif- off equal to 3 in Del Piccolo et al. [10]) and in other
ferent impairment levels: the first class with almost countries (e.g., 39% of subjects above cut-off equal
all responses equal to 0 (subjects without distress), to 3 in Dai et al. [8]).
the second class where percentages of responses
equal to 1 were high only for Item 3, Item 5 and 5. Conclusions
Item 9 (subjects with psychological distress) and the
third class with huge percentages of responses equal The GHQ-12 is commonly analyzed and in-
to 1 (subjects with psychological impairment). terpreted according to CTT rules and we decided
Such a classification agreed with previous results to complement it by performing an analysis based
obtained by administering several other psycho- on IRT. As outlined in our work, drawing on the
logical questionnaires. Indeed, through an item- strengths of IRT as an alternative to CTT analy-
based latent class analysis, we could determine the ses supported the development of rigorous meas-
screening outcome without considering the other ures and valuable interpretations. It was possible to
questionnaires, previously part of the first-level classify the degree of severity of psychological im-
evaluation (IES-R, GAD-7). pairment by administering only GHQ-12 question-
IRT was a helpful tool for identifying clinically naire and according to response patterns, focusing
meaningful subgroups in our population, recogniz- on the way in answering each question more than
ing distinct patient profiles, and tailoring effective the scored obtained as a sum of “positive responses”.
interventions, whose importance was already under- In light of these results, our approach may suggest
lined in previous works [34]. simplifying the multistep protocol for evaluating
Thus, subjects’ classification based only on re- mental health in occupational settings, recommend-
sponses to GHQ-12 could potentially simplify ing using GHQ-12 as a single measurement tool
workers evaluation. Results show that one step of the to be the most effective. Such a method may also
evaluation (i.e. second-level) is redundant and may meet the need for resources and time reduction
be skipped. According to symptoms’ severity, imme- when conducting studies and assessments involving
diate access to specialist evaluation can be planned workers.
for those with psychological impairment (Class 3), Through such analysis, we gave an example of the
without testing them through second-level scales; utility of IRT in psychometric studies conducted
subjects with less severity, i.e. psychological distress among workers populations. The application of ap-
(Class 2), will be instead be monitored with a check propriate methodological tools to support the inter-
evaluation after a certain period of time. pretation of questionnaires could sensibly discover
Our study is prone to potential biases as their potential in simplifying the screening frame-
self-selection of respondents [35]. We managed to work and saving one of the most important workers’
minimize that risk grounding our investigation on resources: their time. Even in questionnaire-based
the occupational physician health surveillance, ob- epidemiological studies, in many cases, less is more.
taining a very high participation rate and minimiz-
ing the risk of untrue or uncompleted answers. Funding: The full cost of the study was covered by the
We know that our results cannot be generalized, Foundation IRCCS Ca’ Granda Ospedale Maggiore Poli-
and neither are they comparable with results ob- clinico (internal funding) and by a charitable donation to the
University of Milan by the “Fondazione Romeo ed Enrica
tained in different scenarios. The pandemic’s con-
Invernizzi”.
sequences directly affected our population, and this
exceptional situation should be carefully considered. Institutional review board statement: The study was
However, in terms of psychological assessment, approved by the Hospital Ethical Committee (Milan Area
our results agree with findings obtained in similar 2 Ethical Committee, n.652_2020 of July 21, 2020) and was
populations of healthcare workers who expressed conducted in compliance with all local legal and regulatory
high levels of GHQ-12 during the COVID-19 requirements, Good Clinical Practice, the International
8 Comotti et al

Conference on Harmonisation document and the Declara- 9. Shoja E, Aghamohammadi V, Bazyar H, et al. Covid-19
tion of Helsinki. effects on the workload of Iranian healthcare workers.
BMC Public Health. 2020;20(1):1636. Published 2020
Informed consent statement: The participation Nov 2. Doi:10.1186/s12889-020-09743-w
was voluntary; each subject read and signed an extended 10. Del Piccolo L, Donisi V, Raffaelli R, et al. The
­informed-consent to participate in the study. ­Psychological Impact of COVID-19 on Healthcare
Providers in Obstetrics: A Cross-Sectional Survey
Study. Front Psychol. 2021;12:632999. Published 2021
Acknowledgments: The Authors thank all workers Apr 9. Doi:10.3389/fpsyg.2021.632999
who agreed to participate and undergo our multistep eval- 11. Feng J, Xu J, Xu S, et al. Psychological Impact Dur-
uation for giving us their precious time. Further, authors ing the First Outbreak of COVID-19 on Frontline
thank Ezio Belleri, Laura Chiappa, Dario Laquintana and Health Care Workers in Shanghai. Front Public Health.
all the IRCCS Cà Granda Fondazione Hospital manag- 2021;9:646780. Published 2021 May 17. Doi:10.3389
ing team for their support in different phases of study de- /fpubh.2021.646780
sign and implementation; Antonio Minolfi and Stefania 12. Mascayano F, van der Ven E, Moro MF, et al. The im-
Pazzi from the Pavia (Italy) Consorzio di Bioingenieria pact of the COVID-19 pandemic on the mental health
e Informatica Medica (CBIM) for the data collection of healthcare workers: study protocol for the COVID-19
software and their technical support; Chiara Moltrasio, HEalth caRe wOrkErS (HEROES) study. Soc Psychia-
MariaLuna Sandri and Francesca Maddalena for their try Psychiatr Epidemiol. 2022;57:633-645. Doi:10.1007
/s00127-021-02211-9
technical support in contacting study subjects and arrang-
13. Mediavilla R, Fernández-Jiménez E, Martínez-Alés G,
ing workers’ interview. et al. Role of access to personal protective equipment,
treatment prioritization decisions, and changes in job
Declaration of interest: The authors declare no con- functions on health workers’ mental health outcomes
flict of interest. during the initial outbreak of the COVID-19 pan-
demic. J Affect Disord. 2021;295:405-409. Doi:10.1016/j​
.jad.2021.08.059
References 14. Tanaka K, Tahara M, Mashizume Y, Takahashi K. Ef-
fects of Lifestyle Changes on the Mental Health of
1. Baldwin JA. The Detection of Psychiatric Illness by Healthcare Workers with Different Sense of Coherence
Questionnaire. By DP Goldberg. Maudsley Mono- Levels in the Era of COVID-19 Pandemic. Int J Envi-
graph No. 21. Oxford University Press. 1972. ron Res Public Health. 2021;18(6):2801. Published 2021
2. Goldberg DP. General Health Questionnaire-12. Mar 10. Doi:10.3390/ijerph18062801
Aust J Psychol. 1978. Doi: https://doi.org/10.1037/t00​ 15. Goldberg DP, Gater R, Sartorius N, et al. The valid-
297-000 ity of two versions of the GHQ in the WHO study
3. Goldberg D. GHQ and psychiatric case. Br J Psychiatry. of mental illness in general health care. Psychol Med.
1979;134:446-447. 1997;27(1):191-197. Doi:10.1017/s0033291796004242
4. Goldberg DP. The validity of two versions of the GHQ 16. Smith AB, Fallowfield LJ, Stark DP, Velikova G, Jen-
in the WHO study of mental illness in general health kins V. A Rasch and confirmatory factor analysis of
care. Psychol. Med. 1997;27(1):191-197. the general health questionnaire (GHQ)-12. Health
5. Andrich D, van Schoubroeck L. The General Health Qual Life Outcomes. 2010;8:45. Published 2010 Apr 30.
Questionnaire: a psychometric analysis using latent trait Doi:10.1186/1477-7525-8-45
theory. Psychol Med. 1989;19(2):469-485. Doi:10.1017​ 17. Böhnke JR, Croudace TJ. Calibrating well-being, qual-
/s0033291700012502 ity of life and common mental disorder items: psycho-
6. Graetz B. Multidimensional properties of the G ­ eneral metric epidemiology in public mental health research.
Health Questionnaire. Soc Psychiatry ­Psychiatr Epide- Br J Psychiatry. 2016;209(2):162-168. Doi:10.1192/bjp​
miol. 1991;26(3):132-138. Doi:10.1007/bf00782952 .bp.115.165530
7. Marton G, Vergani L, Mazzocco K, Garassino MC, 18. Nouri F, Feizi A, Roohafza H, Sadeghi M, Sarrafzade-
Pravettoni G. 2020s Heroes Are Not Fearless: The gan N. How different domains of quality of life are
­Impact of the COVID-19 Pandemic on Wellbeing and associated with latent dimensions of mental health
Emotions of Italian Health Care Workers During Italy measured by GHQ-12. Health Qual Life Outcomes.
Phase 1. Front Psychol. 2020;11:588762. Published 2020 2021;19(1):255. Published 2021 Nov 14. Doi:10.1186​
Oct 15. Doi:10.3389/fpsyg.2020.588762 /s12955-021-01892-9
8. Dai Y, Hu G, Xiong H, Qiu H, Yuan X. Psychological 19. Stochl J, Böhnke JR, Pickett KE, Croudace TJ. An
impact of the coronavirus disease 2019 (COVID-19) evaluation of computerized adaptive testing for gen-
outbreak on healthcare workers in China. bioRxiv. Pub- eral psychological distress: combining GHQ-12 and
lished online 2020. Doi:10.1101/2020.03.03.20030874 ­Affectometer-2 in an item bank for public mental health
 Psychometric Evaluation of GHQ-12 9

research. BMC Med Res Methodol. 2016;16:58. Pub- in Italy. Front Psychiatry. 2022;13:834753. Published
lished 2016 May 20. Doi:10.1186/s12874-016-0158-7 2022 Mar 10. Doi:10.3389/fpsyt.2022.834753
20. Stochl J, Jones PB, Croudace TJ. Mokken scale analy- 28. Lazarsfeld PF. Latent Structure Analysis. Boston:
sis of mental health and well-being questionnaire item Houghton Mifflin Company; 1968.
responses: a non-parametric IRT method in empiri- 29. Goodman LA. Exploratory latent structure a­nalysis
cal research for applied health researchers. BMC Med using both identifiable and unidentifiable m­ odels. Bi-
Res Methodol. 2012;12:74. Published 2012 Jun 11. ometrika. 1974;61(2):215-231. Doi:10.1093/biomet/61​
Doi:10.1186/1471-2288-12-74 .2.215
21. Weiss DS, Marmar CR. Impact of event scale--revised. 30. Bartolucci F. A class of multidimensional IRT models
PsycTESTS Dataset. 1997. Doi:10.1037/t12199-000 for testing unidimensionality and clustering items. Psy-
22. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief chometrika. 2007;72(2):141-157. Doi:10.1007/s11336​
measure for assessing generalized anxiety disorder: the -005​-1376-9
GAD-7. Arch Intern Med. 2006;166(10):1092-1097. 31. Bartolucci F, Bacci S, Gnaldi M. Statistical Analysis of
Doi:10.1001/archinte.166.10.1092 Questionnaires: A Unified Approach Based on R and
23. Fattori A, Cantù F, Comotti A, et al. Hospital workers Stata. Chapman & Hall Book/CRC; 2015.
mental health during the COVID-19 pandemic: meth- 32. Harris PA, Taylor R, Minor BL, et al. The ­REDCap
ods of data collection and characteristics of study sam- consortium: Building an international ­ community of
ple in a university hospital in Milan (Italy). BMC Med ­software platform partners. J Biomed Inform. 2019;​
Res Methodol. 2021;21(1):163. Published 2021 Aug 10. 95:103208. Doi:10.1016/j.jbi.2019.103208
Doi:10.1186/s12874-021-01355-1 33. R Core Team. A Language and Environment for
24. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: va- ­S tatistical Computing. R Foundation for Statistical
lidity of a brief depression severity measure. J Gen Intern Computing, Vienna, Austria (2020). Available online at:
Med. 2001;16(9):606-613. Doi:10.1046/j.1525-1497.2001​ https://www.R-project.org/
.016009606.x 34. Prenovost KM, Fihn SD, Maciejewski ML, Nelson
25. Bernstein EM, Putnam FW. Development, reliability, K, Vijan S, Rosland AM. Using item response the-
and validity of a dissociation scale. J Nerv Ment Dis. ory with health system data to identify latent groups
1986;174(12):727-735. Doi:10.1097/00005053-1986​ of patients with multiple health conditions. PLoS
12000-00004 One. 2018;13(11):e0206915. Published 2018 Nov 26.
26. Derogatis LR, Lipman RS, Covi L. SCL-90: an out- Doi:10.1371​/journal.pone.0206915
patient psychiatric rating scale--preliminary report. 35. Nguyen LH, Drew DA, Graham MS, et al. Risk of
­Psychopharmacol Bull. 1973;9(1):13-28. COVID-19 among front-line health-care workers and
27. Bonzini M, Comotti A, Fattori A, et al. One Year Facing the general community: a prospective cohort study. Lan-
COVID. Systematic Evaluation of Risk Factors Associ- cet Public Health. 2020;5(9):e475-e483. Doi:10.1016/
ated With Mental Distress Among Hospital Workers S2468-2667(20)30164-X