1

The assessment of cyberchondria: Instruments for

assessing problematic online health-related research

Vladan Starcevic1*, David Berle2,3, Sandra Arnáez4, Matteo Vismara5 and Naomi A.

Fineberg6,7,8

1
University of Sydney, Faculty of Medicine and Health, Sydney Medical School, Nepean

Clinical School, Discipline of Psychiatry, Sydney, NSW, Australia.

2
University of Technology Sydney, Graduate School of Health, Discipline of Clinical

Psychology, Ultimo, NSW, Australia.

3
University of New South Wales, School of Psychiatry, Sydney, NSW, Australia.
4
University of Valencia, Facultad de Psicología, Departamento de Personalidad, Evaluación

y Tratamientos Psicológicos, Valencia, Spain.

5
University of Milan, Department of Biomedical and Clinical Sciences Luigi Sacco,

Department of Mental Health, Milan, Italy.

6
University of Hertfordshire, Hatfield, United Kingdom.
7
Hertfordshire Partnership University NHS Foundation Trust, Welwyn Garden City, United

Kingdom.
8
University of Cambridge, School of Clinical Medicine, Cambridge, United Kingdom.

* Corresponding author

Address: Vladan Starcevic; Department of Psychiatry; Nepean Hospital; PO Box 63; Penrith

NSW 2751; Australia

Email: [email protected]

Telephone: +61 2 4734 2585; Fax: +61 2 4734 3343

 2

Abstract

Purpose of review: Cyberchondria is a problematic, i.e., distressing or anxiety-increasing

pattern of online health information seeking. The development of psychometrically sound

instruments for the assessment of cyberchondria is imperative for better understanding of this

construct. The aim of the present article is to provide a systematic literature review of

cyberchondria instruments.

Recent findings: Although several measures of cyberchondria have been developed, the

Cyberchondria Severity Scale (CSS) has been used most often. The CSS is based on a solid

theoretical framework, with very good to excellent reliability and validity. It has been

translated into several languages. Modifications of the original version of the CSS have been

introduced to refine its conceptual foundation and improve its utility by making it shorter.

Summary: Further improvement of the CSS may boost the quality of cyberchondria

research. There remains a need to test the theoretical underpinnings of the CSS and consider

alternative models of cyberchondria.

Keywords:

Cyberchondria; online health information seeking; online health research; assessment;

measurement; Cyberchondria Severity Scale

 3

Introduction

The Internet and digital technology have revolutionised searches for health-related

information. Instead of relying on medical books and encyclopaedias and consulting medical

professionals, most people now turn to the Internet for any enquiry about health- or illness-

related matters. Although several factors seem to have contributed to this change, it is likely

due to the ease with which health information can be accessed via the Internet.

Online health research (OHR) allows millions of people worldwide to quickly obtain

information that they need, with a generally empowering effect. Some individuals, however,

report a higher level of anxiety or distress during or after OHR. This outcome, coupled with a

pattern of excessive or repetitive OHR, came to be known as cyberchondria [1, 2].

Cyberchondria quickly became an object of interest to clinicians and researchers, but

studies were hampered by a lack of instruments assessing the construct. That changed in

2014, when the first cyberchondria scale appeared [3]. Years that followed have seen

flourishing of cyberchondria research, along with efforts to improve the original instrument

and introduce others.

The present article focuses on the assessment of cyberchondria and aims to present

and critically review the psychometric properties of instruments developed thus far to assess

cyberchondria. The article will also highlight the shortcomings of the present instruments,

identify the gaps in the relevant literature and offer suggestions for further research.

Methods

We conducted a systematic review in accordance with the recommended PRISMA guidelines

[4]. We searched PubMed, Web of Science, ScienceDirect, Scopus, PsycINFO and Google
 4

Scholar for the relevant publications published until February 2020. Search terms included

combinations of the following text words: cyberchondria AND scale OR measure OR

instrument OR test OR questionnaire OR inventory OR self-report OR development OR

validation OR reliability OR validity OR psychometric. Publications were identified on the

basis of two or more of these terms, whereby one of them had to be cyberchondria. We did

not search for unpublished studies.

The following criteria were used to identify publications for possible inclusion in this

systematic review: 1) Full-length, peer-reviewed journal articles or book chapters published

in English; 2) Publications reporting original data; 3) Use of an instrument to assess

cyberchondria; 4) Reporting of at least one psychometric property (e.g., internal consistency)

of the instrument used to assess cyberchondria. Abstracts and review articles were excluded.

The following data were extracted from the selected publications: authors and year of

publication, instrument(s) used in the publication, country, sample size, basic demographic

characteristics of the sample, type of sample, method of recruitment, health status of study

participants and main psychometric properties of the instruments. The latter included the

factor structure; intercorrelations between subscales; correlations between subscales and total

scale scores; internal consistency; and correlations between total scale scores and subscales

on one hand and other scales of relevance on the other. Additional information was extracted

for the specific scales and/or for specific purposes, e.g., whether there were any data on cut-

off points.

Results
 5

Figure 1 shows that of the 33 articles identified through the searches, 24 met our inclusion

criteria and were included in the systematic review. Figure 1 also shows reasons for

excluding articles.

[FIGURE 1 ABOUT HERE]

Characteristics of the articles and samples

Table 1 presents the main characteristics of the articles and samples included in the present

systematic review. All identified instruments are based on self-report. Of the 24 articles, 21

reported on the Cyberchondria Severity Scale (CSS) and 4 reported on four other

cyberchondria instruments. Of the 21 articles addressing the CSS, 13 reported on the full, 33-

item version, 10 reported on the 30-item version (CSS-30) and 1 article each reported on the

15-item version (CSS-15), 15-item version modified to a 12-item version (CSS-15-Revised)

and 12-item version (CSS-12). Several articles reported findings based on more than one

sample [5-9], whereas 2 articles were based on the same sample [10, 11], resulting in 29

samples across 24 articles.

Almost one half of articles were based on samples from the USA (10 articles) and

Puerto Rico/USA (1). The remaining articles used samples from Turkey (4), Poland (2), UK

(2), Croatia (1), Germany (1), Iran (1) and Australia (1). One study published by the

Australian and Swiss authors restricted participation to English-speaking individuals from

Australia, Canada, Ireland, New Zealand, the UK and USA [12]. Sample sizes across the

articles varied from 86 to 1200 (mean = 413). The majority of participants were women in 21

of the 28 samples for which the data on gender were available, and the proportion of women

across all 28 studies ranged from 35.5% to 87.2%. The mean age of participants ranged from

19 to 39 years across 26 samples, with the mean age in 16 samples being between 31 and 39

years.
 6

The majority of studies were conducted in community samples (6 samples) and online

community samples (7 samples), with some studies including only community subjects using

the Internet for health searches (3 samples) or online community subjects using the Internet

for health searches (5 samples). The remaining samples consisted of university students,

including undergraduate students (6 samples), university employees (1 sample) and

treatment-seeking individuals with the DSM-5 illness anxiety disorder or DSM-5 somatic

symptom disorder (1 sample). The method of recruitment was described for 15 of the 29

samples and involved use of an online crowdsourcing platform to recruit 10 samples. Health

status was taken into consideration or noted when recruiting 12 of the 29 samples, with 3

studies excluding subjects with medical conditions and 3 studies excluding subjects with

current or recently diagnosed severe mental disorders.

[TABLE 1 ABOUT HERE]

Cyberchondria Severity Scale

The CSS [3], developed in the UK, has been the most widely used instrument in

cyberchondria research. It has been translated into several languages, but the psychometric

properties have been published only for the German [5], Polish [13*] and two Turkish [14,

15] versions of the CSS. The Croatian version only reported data on internal consistency [9]

and the Iranian version only reported total CSS score correlations with other scales [16].

The basis for the development of the CSS was the definition of cyberchondria as an

“increase in anxiety about one’s own health status, as a result of excessive reviews of online

health information” [3, p. 259] and the conceptualisation of cyberchondria as a “multi-

dimensional construct, reflecting both anxiety and an element of compulsiveness” [3, p. 260].

The stated aim of the CSS is to measure anxiety as a result of online searches for health

information. It was not developed as a screening tool for cyberchondria and corresponding
 7

cut-off scores were not established. The CSS initially consisted of 43 items that were

generated by means of a review of the cyberchondria literature and conceptually related to

anxiety disorders.

The original CSS (“full CSS”) consists of 33 items that are scored on a 5-point scale

[3]. Scores on each item are summed up to provide a total CSS score. Exploratory factor

analysis of the CSS revealed 5 factors: Compulsion, Distress, Excessiveness, Reassurance (or

Reassurance Seeking) and Mistrust of Medical Professionals (MMP). These factors represent

separate subscales and scores on each subscale can be calculated separately. Compulsion was

conceptualised to reflect “an unwanted, compulsive element” of cyberchondria and refers to

ways in which OHR interrupts activities. Distress denotes negative emotional states (e.g.,

anxiety) associated with OHR. Excessiveness involves repetitious and time-consuming OHR,

often relying on numerous online sources. Reassurance reflects a need to seek reassurance

about health-related matters from a suitably qualified person such as medical professionals.

Finally, MMP was originally conceptualised as a “conflict” as to whether one should trust

medical professionals or the results of one’s own OHR, with greater confidence in the latter.

[TABLE 2 ABOUT HERE]

Table 2 shows psychometric properties of the CSS across 13 studies. The original

study by McElroy and Shevlin [3] used exploratory factor analysis to identify the 5-factor

structure of the CSS. Additional six studies identified the same 5-factor structure, one of them

using exploratory factor analysis, one using principal component analysis and four using

confirmatory factor analysis. Using confirmatory factor analysis, one study reported bifactor

modelling with a General Cyberchondria Factor and Specific Factors that comprise

Compulsion, Distress, Excessiveness, Reassurance and MMP [10]. Except for the MMP

subscale, the intercorrelations between the CSS subscales ranged between 0.26 and 0.80,
 8

while their correlations with the total CSS score ranged from 0.65 to 0.89. These correlations

support conceptual coherence of the CSS.

Internal consistency for the total CSS was excellent, with Cronbach α ranging

between 0.91 and 0.96 in 10 studies and only one study reporting Cronbach α of 0.89.

Likewise, internal consistency was excellent for the Compulsion subscale (Cronbach α

ranging between 0.90 and 0.96 in 6 studies, with one study reporting Cronbach α of 0.88) and

Distress subscale (Cronbach α ranging between 0.92 and 0.95 in 6 studies, with one study

reporting Cronbach α of 0.87). Internal consistency figures for the Excessiveness and

Reassurance subscales were generally good: for the Excessiveness subscale, Cronbach α

ranged between 0.85 and 0.91 in 5 studies, with one study reporting Cronbach α of 0.74,

whereas for the Reassurance subscale, Cronbach α ranged between 0.80 and 0.89 in 5 studies,

with one study reporting Cronbach α of 0.76.

Test-retest reliability of the CSS was assessed in 2 studies using translated versions.

For one of the Turkish translations, a reliability figure in 66 participants over a 2-week period

was 0.65 for the total CSS, while the corresponding figures for the CSS subscales ranged

from 0.53 to 0.71 [14]. The reported test-retest reliability figures for the CSS subscales of the

Polish version over a period of 3 months in 59 participants ranged from 0.58 to 0.76 [13*].

Convergent validity of an instrument can be assessed by examining the strength of

correlations between measures of the two conceptually related constructs. Close relationships

have been postulated theoretically or based on research not relying on the CSS between

cyberchondria and health anxiety [1], problematic Internet use [17] and obsessive-compulsive

disorder (OCD) [18]. Therefore, convergent validity of the CSS could be examined via its

correlations with measures of health anxiety, problematic Internet use and OCD. Table 2

shows that convergent validity of the CSS is solid because the correlations between the total

CSS and measures of health anxiety such as the Short Health Anxiety Inventory and
 9

Modified Version of the Short Health Anxiety Inventory ranged between 0.53 and 0.59. In

one study, however, the correlation between the total CSS and the Health Anxiety Inventory

was only 0.23 [15]. With regards to problematic Internet use, only one study reported a

correlation of 0.45 between the total CSS and the Internet Addiction Test [15], suggesting

good convergent validity of the CSS. Although somewhat lower, the correlations between the

total CSS and measures of OCD such as the total Dimensional Obsessive-Compulsive Scale

(rs = 0.38 and 0.49) and the subscales of Maudsley Obsessive-Compulsive Inventory (rs =

0.27-0.40) also support a solid convergent validity of the CSS.

Divergent validity of an instrument is assessed by examining the strength of

correlations between measures of the constructs that are conceptually unrelated or less

related. Cyberchondria is expected to be less related to depression and correlations between

the measures of these two constructs are therefore expected to be relatively low. Indeed, the

correlations between the total CSS and measures of depression, such as the Depression

subscale of the Depression, Anxiety and Stress Scale – 21-Item Version (rs = 0.22 and 0.24)

and the Center for Epidemiologic Studies Depression Scale (r = 0.31) were generally weaker.

This provides some support to the divergent validity of the CSS.

[TABLE 3 ABOUT HERE]

The MMP subscale of the CSS consists of 3 items, which are the only reverse-scored

items of the CSS. The performance of the MMP subscale was different from other CSS

subscales, as shown in Table 3. Its correlations with other CSS subscales were lower or

negative (ranging from -0.16 to 0.44), as were the correlations with the total CSS (ranging

between 0.12 and 0.41). Internal consistency of the MMP subscale was considerably lower

compared to the other CSS subscales, with Cronbach α ranging between 0.62 and 0.89 across

9 studies and being lower than 0.69 in 4 studies. Correlations between the MMP subscale and

measures of health anxiety, problematic Internet use, OCD, other anxiety-related variables
 10

and depression were generally weak, often nonsignificant and at times negative. These

findings suggest that the MMP subscale may not assess the same overarching construct of

cyberchondria as the other CSS subscales and that it may not belong to the CSS. It has been

suggested that the MMP score might still have some value, but that it should be calculated

separately from the CSS score [13*, 19]. Consequently, several authors proposed a removal

of the MMP subscale items from the CSS [5, 10, 14]. This led to the development of shorter

and modified versions of the CSS.

30-item version of the Cyberchondria Severity Scale

Given the problems with the MMP subscale, several authors have used the CSS without 3

items of the MMP subscale. The psychometric properties of this 30-item version of the CSS

(CSS-30) are reported in Table 4.

In two studies [13*, 20], a confirmatory factor analysis identified 4 factors/subscales

of the CSS-30: Compulsion, Distress, Excessiveness and Reassurance. In terms of the

correlations between these 4 CSS-30 subscales (ranging between 0.42 and 0.78) and the

correlations between the 4 CSS-30 subscales and the total CSS-30 score (ranging from 0.73

to 0.90), the CSS-30 performs similarly to the full CSS (if the MMP subscale is not taken into

account).

Internal consistency for the total CSS-30 was excellent (Cronbach α = 0.95 and 0.96

across 7 studies) and therefore somewhat better than internal consistency for the total full

CSS. Similarly, internal constancy values for the Compulsion (Cronbach α = 0.95 and 0.96

across 2 studies), Distress (Cronbach α = 0.95 across 2 studies), Excessiveness (Cronbach α =

0.87 across 2 studies) and Reassurance (Cronbach α = 0.85 and 0.88 across 2 studies)

subscales of the CSS-30 were excellent to good and somewhat better than internal
 11

consistency values for the same subscales of the full CSS. There are no published data on

test-retest reliability of the CSS-30.

Convergent validity indices for the CSS-30 were similar to those for the full CSS,

with strongest correlations reported with measures of health anxiety or measures related to

health anxiety, such as the Short Health Anxiety Inventory (rs = 0.52-0.58), Whiteley

Index/Whiteley Index-6 (rs = 0.56-0.67) and Metacognitions Questionnaire – Health Anxiety

(rs = 0.32-0.66). The correlation with a measure of problematic Internet use (Problematic

Internet Use Questionnaire) was also robust (r = 0.59), whereas the correlations with a

measure of OCD (total Dimensional Obsessive-Compulsive Scale) were somewhat lower (rs

= 0.38-0.49). Divergent validity of the CSS-30 has not been examined, as no study reported

the correlations with measures of depression or other conceptually unrelated constructs.

[TABLE 4 ABOUT HERE]

15-item and 12-item versions of the Cyberchondria Severity Scale

A general tendency to shorten self-report instruments has played a role in several attempts to

decrease the number of items of the CSS. The first such attempt was made by Barke et al. [5]

in their German version of the scale. They conducted a principal components analysis of the

full CSS and retained the 3 best-performing items from each of the 5 CSS subscales based on

their factor loadings, thereby forming a 15-item version (CSS-15). A confirmatory factor

analysis identified 5 factors/subscales, but their intercorrelations and the correlations with the

total CSS-15 score were much lower than those reported for the full CSS and CSS-30 (Table

4). Internal consistency values for the total CSS-15 (Cronbach α = 0.82) and for the CSS-15

subscales (Cronbach α = 0.67-0.86) were also lower. Indices of convergent and divergent

validity of the CSS-15 were good (Table 4).

 12

Another shorter version of the CSS was based on the CSS-15, except for 3 items of

the MMP subscale. This resulted in a 12-item instrument, referred to as CSS-15-Revised [7].

Internal consistency for the total CSS-15-Revised (Cronbach α = 0.88) was better than that

for the CSS-15, with solid convergent validity (Table 4). Divergent validity was not assessed.

The authors of the original CSS developed an abbreviated version of the CSS

containing 12 items (CSS-12) [21**]. They first conducted an exploratory factor analysis of

the full CSS and selected for inclusion in the CSS-12 3 items from each factor/subscale

except for the MMP subscale. This selection was made on the basis of several criteria: factor

loadings, endorsement rates, impact on subscale internal consistency, length and content. The

CSS-12 thus consists of 4 factors/subscales. Internal consistency for the total CSS-12 was

excellent (Cronbach α = 0.90), but lower than internal consistency for the full CSS and CSS-

30 (Table 4). Likewise, internal consistency figures for the CSS-12 subscales were lower

(Cronbach α = 0.73-0.87). Total score of the CSS-12 correlated more strongly with a measure

of health anxiety (r = 0.53) than with a measure of general anxiety (r = 0.30), suggesting a

good convergent validity. Divergent validity was not assessed.

Other translations of the Cyberchondria Severity Scale

The CSS has been translated into several other languages and used in several non-English

speaking samples. However, data on psychometric properties of these translated versions of

the CSS are not available in peer-reviewed journals published in English. Therefore, the

corresponding articles have not been included in the present systematic review. Still, it should

be noted that the CSS has been translated into Brazilian Portuguese [22] and that Indonesian

version of the CSS was used in one study [23]. The CSS was also used in published studies

conducted in Pakistan [24, 25], India [26, 27] and Sri Lanka [28]. The original English
 13

version of the CSS was administered in some of these studies [25, 27], while other studies did

not provide details of translations into local languages [24, 26, 28].

Other cyberchondria instruments

Four additional cyberchondria instruments have been developed in recent times:

Cyberchondria Scale (CS) [8], Short Cyberchondria Scale (SCS) [9], Brief Cyberchondria

Scale (BCS) [29] and Cyberchondria Tendency Scale (CTS) [30]. This is partly a

consequence of a need for measures that would be more “culturally specific”, that is, suitable

for various non-English speaking populations. Hence, two of these scales were developed in

Turkey (CS and CTS), one was developed in Croatia (SCS) and another in Puerto Rico for

Hispanics/Spanish-speaking people (BCS). Two measures reflect a tendency to further

shorten cyberchondria instruments, with SCS consisting only of 4 items and BCS having 10

items. Another reason for introducing new cyberchondria measures can be found in different

theoretical frameworks for cyberchondria and different purposes of these instruments.

Although the CS and CSS are both based on the notion that cyberchondria is a

multidimensional construct, these dimensions overlap, but are not the same. Consequently,

the aim of the CS is to measure the level of cyberchondria focusing on information-seeking

behaviour (or OHR) and characteristics of online health-related material that both increase

and decrease anxiety. In contrast, the SCS is based on the premise that the “core element” of

cyberchondria is anxiety amplification following OHR. The BCS was developed on the

assumption that cyberchondria represents OHR driven by health anxiety. The stated purpose

of the CTS is to assess Internet users’ “tendency to cyberchondria”, i.e., the extent to which

they use health information obtained online and seek a “solution” online when having health

problems.

[TABLE 5 ABOUT HERE]

 14

Table 5 presents the psychometric properties of the CS, SCS, BCS and CTS. Their

factor structure differs from one scale to another, which is a consequence of their different

conceptual underpinnings. Some of the factors/subscales resemble those of the CSS, while

others are unrelated to them. Internal consistency for all the instruments was excellent

(Cronbach α = 0.92, 0.93 and 0.94 for total BCS, CS and CTS, respectively), except for the

SCS (Cronbach α = 0.73). Correlations with measures of health anxiety were generally

robust, suggesting a relatively good convergent validity for the BCS, CS and SCS. Unlike

any version of the CSS, the BCS and CTS provide instructions for scoring and for

distinguishing between various degrees of severity of “cyberchondria symptoms” (BCS) and

“cyberchondria tendencies” (CTS).

Discussion

The CSS has dominated cyberchondria research. This is mainly due to two reasons. First, the

CSS is based on a credible theoretical conceptualisation of cyberchondria, according to which

it is a multidimensional construct that comprises excessive OHR, corresponding distress or

anxiety, interference with activities and reassurance seeking from offline sources, usually

medical professionals. These 4 components of cyberchondria are assessed via the

corresponding subscales of the CSS.

Second, the CSS has very good to excellent psychometric properties. Internal

consistency for the total CSS is excellent, which is particularly important considering the

postulated multidimensional nature of the cyberchondria construct. Moreover, internal

consistency of the CSS subscales that assess distress and interference with activities is also

excellent. Internal consistency for the subscales assessing the excessive nature of OHR and
 15

reassurance seeking is acceptable to good and could be improved. This also suggests that

excessive OHR and reassurance seeking from medical professionals as components of the

cyberchondria construct may require further conceptual elaboration. The CSS has a solid

convergent validity, as demonstrated by strong correlations with measures of health anxiety

and to a lesser extent, measures of problematic Internet use. Correlations with measures of

OCD have been generally weaker than those with health anxiety, suggesting that the CSS

reflects the relationship between cyberchondria and health anxiety more than the one between

cyberchondria and OCD. The CSS appears to have a solid divergent validity, but there is a

need for further research examining the correlations between the CSS and measures of the

constructs conceptually unrelated to cyberchondria such as depression, social anxiety or

impulsivity.

The validity of the CSS is also supported by numerous studies that have confirmed the

overall factor structure. This provides confidence in each of the five subscales as reflective of

specific domains of cyberchondria, notwithstanding the less convincing findings for the

MMP subscale. Further research should take into account bifactor modelling of the CSS

reported by Norr et al. [10] and reconsider the notion that cyberchondria is a unitary

construct, best described as including both a factor which incorporates all items, as well as

the separate factors. Although the interpretation of CSS total scores as indicative of a unitary

concept of cyberchondria is generally assumed to be correct, this has not always been

supported. For example, Fergus [19] did not find that CSS subscales “tapped” a general

cyberchondria factor, leaving open the possibility that CSS subscales reflect relatively

independent, separate constructs.

The full version of the CSS may not be practical for routine use because of its length

(33 items). Another shortcoming of the CSS is the lack of a clear scoring system. A

performance of the MMP subscale that was incongruent with that of the other CSS subscales
 16

has been addressed by omitting the 3 MMP subscale items from the scale (as in the CSS-30)

or by calculating the score on this subscale separately from the total CSS score. The

psychometric properties of the CSS-30 are comparable to those of the full CSS. The same

generally holds true for even shorter versions of the CSS: CSS-15, CSS-15-Revised and CSS-

12. Lower internal consistency values for the shorter versions of the CSS are possibly a

consequence of fewer scale items. However, the approach taken to construct shorter forms of

the CSS may not necessarily allow these instruments to “capture” the full extent of the

construct as the original CSS [31]. Therefore, modifications of the CSS call for further

examination of their reliability and validity.

The development of other cyberchondria instruments reflects a growing interest in

cyberchondria. These measures are difficult to compare with the CSS because their

theoretical foundations and purposes are different. Furthermore, they have been developed

mainly for use in certain non-English speaking countries. Despite their generally solid

psychometric properties, it does not seem very likely that these instruments will be used

across different countries, perhaps except for the BCS, which may be used in various

Spanish-speaking countries.

Future studies of the CSS and other cyberchondria instruments need to be conducted

in treatment-seeking or clinical samples to ascertain their responsiveness to changes with

treatment. Only one study so far [32**] has been conducted in a treatment-seeking sample.

Further research should control for individuals who were medically or psychologically

unwell at the time of assessment. While internal consistency, convergent and divergent

validity of the CSS and its variants are well established, there is a need for further detailed

psychometric examination. For instance, additional examination of test-retest reliability will

determine whether these measures are consistent in capturing the concept of cyberchondria

across time. Studies should ideally investigate the invariance of the factor structure across
 17

different samples and population groups and the item response characteristics for each item

of the scale. More work also remains to be done in terms of the interpretation of the CSS

scores. It would be useful, for example, to have guidance on classifying cyberchondria cases

into mild, moderate and severe based on their CSS scores. This is of particular importance for

use of the CSS in clinical practice and for public health purposes. Another issue is the

conceptual status of cyberchondria and the question of what the CSS is assessing: a distinct

entity or several interrelated phenomena and behaviours.

The near-ubiquitous use of the CSS in cyberchondria research has been helpful for

comparing findings across studies. However, this fact also carries the limitation that almost

everything that we know about cyberchondria is based on use of the CSS in research. The

model of cyberchondria upon which the CSS was constructed is not necessarily accurate,

while CSS-based research findings implicitly (or even explicitly) endorse such a model. This

situation calls for development of other approaches to cyberchondria, which would be

compared with the CSS-based model. Such approaches might introduce different theoretical

frameworks and associated assessment tools, including clinician-administered interviews.

This endeavour would then allow a more adequate appraisal of the validity of the CSS.

Conclusion

To the best of our knowledge, this is the first published systematic review of cyberchondria

instruments. To date, the cyberchondria literature has relied heavily on a single measure – the

CSS. It is critically important that the CSS and its derivatives have robust psychometric

properties as our understanding of the construct of cyberchondria and its place in the

landscape of conceptually similar constructs hinges upon valid and reliable measurement.
 18

This is particularly the case for understanding the relationships between cyberchondria,

health anxiety, problematic Internet use and OCD. While the CSS has met the standard

thresholds for psychometric validation, there remains considerable scope for refinement.

Compliance with Ethics Guidelines

Conflict of Interest

The authors declare no conflicts of interest in the production of this work.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of

the authors.
 19

References

1. Starcevic V, Berle D. Cyberchondria: Towards a better understanding of excessive

health-related Internet use. Exp Rev Neurotherap. 2013;13:205-13.

doi:10.1586/ERN.12.162

2. Starcevic V. Cyberchondria: Challenges of problematic online searches for health-related

information. Psychother Psychosom. 2017;86:129-33. doi:10.1159/000465525

3. McElroy E, Shevlin M. The development and initial validation of the Cyberchondria

Severity Scale (CSS). J Anxiety Disord. 2014;28:259-65.

https://doi.org/10.1016/j.janxdis.2013.12.007

4. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred reporting

items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med.

2009;6:e1000097. https://doi.org/10.1371/journal.pmed.1000097.

5. Barke A, Bleichhardt G, Rief W, Doering BK. The Cyberchondria Severity Scale (CSS):

German validation and development of a Short Form. Int J Behav Med. 2016;23:595-

605. https://doi.org/10.1007/s12529-016-9549-8

6. Fergus TA, Spada MM. Cyberchondria: Examining relations with problematic Internet

use and metacognitive beliefs. Clin Psychol Psychother. 2017;24:1322-30.

https://doi.org/10.1002/cpp.2102

7. Fergus TA, Spada MM. Moving toward a metacognitive conceptualization of

cyberchondria: Examining the contribution of metacognitive beliefs, beliefs about rituals,

and stop signals. J Anxiety Disord. 2018;60:11-9.

https://doi.org/10.1016/j.janxdis.2018.09.003
 20

8. Durak-Batigun A, Gor N, Komurcu B, Senkal-Erturk I. Cyberchondria Scale (CS):

development, validity and reliability study. Dusunen Adam The Journal of Psychiatry

and Neurological Sciences 2018;31:148-62. https://doi.org/10.5350/DAJPN2018310203

9. Jokić-Begić N, Mikac U, Čuržik D, Jokić CS. The development and validation of the

Short Cyberchondria Scale (SCS). J Psychopathol Behav Assess. 2019;41:662-76.

https://doi.org/10.1007/s10862-019-09744-z

10. Norr AM, Allan NP, Boffa JW, Raines AM, Schmidt NB. Validation of the

Cyberchondria Severity Scale (CSS): Replication and extension with bifactor modeling.

J Anxiety Disord. 2015;31:58-64. https://doi.org/10.1016/j.janxdis.2015.02.001

11. Norr AM, Albanese BJ, Oglesby ME, Allan NP, Schmidt NB. Anxiety sensitivity and

intolerance of uncertainty as potential risk factors for cyberchondria. J Affect Disord.

2015;174:64-9. http://dx.doi.org/10.1016/j.jad.2014.11.023

12. Starcevic V, Baggio S, Berle D, Khazaal Y, Viswasam K. Cyberchondria and its

relationships with related constructs: A network analysis. Psychiatr Q. 2019;90:491-505.

https://doi.org/10.1007/s11126-019-09640-5

13. * Bajcar B, Babiak J, Olchowska-Kotala A. Cyberchondria and its measurement. The

Polish adaptation and psychometric properties of the Cyberchondria Severity Scale CSS-

PL. Psychiatr Pol. 2019;53:49-60. https://doi.org/10.12740/PP/81799 This is an

example of a well-conducted validation study of the translated version of the

Cyberchondria Severity Scale.

14. Uzun SU, Zencir M. Reliability and validity study of the Turkish version of

Cyberchondria Severity Scale. Curr Psychol. 2018. https://doi.org/10.1007/s12144-018-

0001-x
 21

15. Selvi Y, Turan SG, Sayin AA, Boysan M, Kandeger A. The Cyberchondria Severity

Scale (CSS): Validity and reliability study of the Turkish version. Sleep Hypn.

2018;20:241-6. https://dx.doi.org/10.5350/Sleep.Hypn.2018.20.0157

16. Zangoulechi Z, Yousefi Z, Keshavarz N. The role of anxiety sensitivity, intolerance of

uncertainty, and obsessive-compulsive symptoms in the prediction of cyberchondria.

Advances in Bioscience and Clinical Medicine (ABCmed) 2018;6(4):1-6.

http://dx.doi.org/10.7575/aiac.abcmed.v.6n.4p.1

17. Singh K, Brown RJ. Health‐related Internet habits and health anxiety in university

students. Anxiety Stress Coping 2014;27:542-54.

https://doi.org/10.1080/10615 806.2014.888061

18. Vismara M, Caricasole V, Starcevic V, Cinosi E, Dell’Osso B, Martinotti G, Fineberg

NA. Is cyberchondria a new transdiagnostic digital compulsive syndrome? A systematic

review of the evidence. Compr Psychiatry 2020;99:152167.

https://doi.org/10.1016/j.comppsych.2020.152167

19. Fergus TA. The Cyberchondria Severity Scale (CSS): An examination of structure and

relations with health anxiety in a community sample. J Anxiety Disord. 2014;28:504-10.

https://doi.org/10.1016/j.janxdis.2014.05.006

20. Mathes BM, Norr AM, Allan NP, Albanese BJ, Schmidt NB. Cyberchondria: Overlap

with health anxiety and unique relations with impairment, quality of life, and service

utilization. Psychiatry Res. 2018;261:204-11.

https://doi.org/10.1016/j.psychres.2018.01.002

21. ** McElroy E, Kearney M, Touhey J, Evans J, Cooke Y, Shevlin M. The CSS-12:

Development and validation of a Short-Form Version of the Cyberchondria Severity

Scale. Cyberpsychol Behav Soc Netw. 2019;22:330-5.

https://doi.org/10.1089/cyber.2018.0624 This study demonstrates how the identified

 22

shortcomings of the Cyberchondria Severity Scale were addressed to create a

shorter and more conceptually coherent version of the instrument, without

sacrificing its solid psychometric properties.

22. da Silva FG, Andrade R, Silva I, Cardoso A. Cross-cultural adaptation of the

Cyberchondria Severity Scale for Brazilian Portuguese. Trends Psychiatry Psychother.

2016;38:90-5. http://dx.doi.org/10.1590/2237-6089-2015-0063

23. Aulia A, Marchira CR, Supriyanto I, Pratiti B. Cyberchondria in first year medical

students of Yogyakarta. J Consum Health Internet 2020;24:1-9.

https://doi.org/10.1080/15398285.2019.1710096

24. Malik MN, Mustafa MAT, Yaseen M, Ghauri SK, Javaeed A. Assessment of

cyberchondria among patients presenting to the emergency department of three hospitals

in Islamabad, Pakistan. South Asian Journal of Emergency Medicine (SAJEM)

2019;2:19-23. doi:10.5455/sajem.020207

25. Akhtar M, Fatima T. Exploring cyberchondria and worry about health among individuals

with no diagnosed medical condition. J Pak Med Assoc. 2020;70:90-95.

https://doi.org/10.5455/JPMA.8682

26. Makarla S, Gopichandran V, Tondare D. Prevalence and correlates of cyberchondria

among professionals working in the information technology sector in Chennai, India: A

cross‑sectional study. J Postgrad Med. 2019;65:87-92. doi:10.4103/jpgm.JPGM_293_18

27. Dagar D, Kakodkar P, Shetiya SH. Evaluating the cyberchondria construct among

computer engineering students in Pune (India) using Cyberchondria Severity Scale (CSS-

15). Indian J Occup Environ Med. 2019;23:117-20. doi:10.4103/ijoem.IJOEM_217_19

28. Wijesinghe CA, Liyanage ULNS, Kapugama KGCL, Warsapperuma WANP, Williams

SS, Kuruppuarachchi KALA, Rodrigo A. “Muddling by googling” – Cyberchondria

 23

among outpatient attendees of two hospitals in Sri Lanka. SL J Psychiatry 2019;10:11-5.

http://doi.org/10.4038/sljpsyc.v10i1.8202

29. González-Rivera JA, Santiago-Olmo KL, Cruz-Rodríguez AS, Pérez-Ojeda RJ, Torres-

Cuevas H. Development and validation of the Brief Cyberchondria Scale in Puerto Rico.

International Journal of Recent Scientific Research 2020;11:36734-7.

http://dx.doi.org/10.24327/ijrsr.2020.1101.4984

30. Tatli Z, Tatli O, Kokoc M. Development and validity of Cyberchondria Tendency Scale.

World Journal on Educational Technology: Current Issues 2019;11:001-9.

31. Smith GT, McCarthy DM, Anderson KG. On the sins of short-form development.

Psychol Assess. 2000;12:102-11.

32. ** Newby JM, McElroy E. The impact of internet-delivered cognitive behavioural

therapy for health anxiety on cyberchondria. J Anxiety Disord, 2020;69:102150.

https://doi.org/10.1016/j.janxdis.2019.102150 This is the first study in which the

Cyberchondria Severity Scale was used to monitor changes in cyberchondria

during cognitive-behavioural therapy for health anxiety. The levels of

cyberchondria decreased significantly in patients whose levels of health anxiety

were reduced in the course of therapy, reaffirming the relationship between

cyberchondria and health anxiety in treatment-seeking individuals.

33. Fergus TA. Anxiety sensitivity and intolerance of uncertainty as potential risk factors for

cyberchondria: A replication and extension examining dimensions of each construct. J

Affect Disord. 2015;184:305-9. http://dx.doi.org/10.1016/j.jad.2015.06.017

34. Norr AM, Oglesby ME, Raines AM, Macatee RJ, Allan NP, Schmidt NB. Relationships

between cyberchondria and obsessive-compulsive symptom dimensions. Psychiatry Res.

2015;230:441-6. http://dx.doi.org/10.1016/j.psychres.2015.09.034
 24

35. Fergus TA, Russell LH. Does cyberchondria overlap with health anxiety and obsessive-

compulsive symptoms? An examination of latent structure and scale interrelations. J

Anxiety Disord. 2016;38:88-94. http://dx.doi.org/10.1016/j.janxdis.2016.01.009

36. Gibler RC, Jastrowski Mano KE, O’Bryan EM, Beadel JR, McLeish AC. The role of

pain catastrophizing in cyberchondria among emerging adults. Psychol Health Med.

2019;24:1267-76. https://doi.org/10.1080/13548506.2019.1605087

37. Bajcar B, Babiak J. Self-esteem and cyberchondria: The mediation effects of health

anxiety and obsessive-compulsive symptoms in a community sample. Curr Psychol.

2019. https://doi.org/10.1007/s12144-019-00216-x
 25

Figure 1. PRISMA flowchart: Selection of the original studies reporting on the psychometric
properties of measurement instruments for cyberchondria.

Records identified through database searching

(Web of Science, PubMed, ScienceDirect,
PsychINFO, Scopus, Google Scholar)
Identification

N = 33

Records excluded
N=8
Reasons for exclusion:
Screening

• Review articles (n = 2)
• Not using a cyberchondria
instrument (n = 6)

Full-text articles assessed for eligibility

N = 25

Full-text article excluded

Eligibility

N=1
Reason for exclusion:
• Not reporting on any psychometric
property of the cyberchondria
instrument used in the study
Inclusion

Full-text articles included in the review

N = 24
 26

Table 1. Characteristics of the articles (N=24) and samples (N=29) in which the Cyberchondria Severity Scale (CSS), modified versions of the CSS and other cyberchondria
scales were tested or in which the psychometric properties of these instruments were described.
Article and sample Instrument Country Sample % Mean age Age range Sample characteristics Recruitment method Health status taken into
size females (years) (years) consideration or noted during
recruitment
McElroy & CSS UK 208 63.9 24.2±8.2 18-60 University undergraduate NR No
Shevlin, 2014 [3] students
Fergus, 2014 [19] CSS, USA 539 43.4 31.3±9.9 18-67 Online sample using the Online crowdsourcing Yes (individuals with medical
CSS-30 Internet for health searches platform - MTurk conditions were excluded)
Norr et al., 2015 CSS USA 526 69.2 34.9±12.4 18-72 Online community sample Online crowdsourcing No
[10] & Norr et al., platform - MTurk
2015 [11]
McElroy et al., CSS, UK 661 73.0 22.2±5.9 NR University undergraduate NR No
2019 [21**] CSS-12 students
Starcevic et al., CSS Australia & 751 NR NR NR Online sample using the Online crowdsourcing No
2019 [12] Switzerland† Internet for health searches platform - PROL
during previous 3 months
Newby & CSS Australia 86 87.2 30±12 18-65 Treatment-seeking individuals NR Yes (individuals with psychosis,
McElroy, 2020 with illness anxiety disorder bipolar affective disorder and
[32**] (DSM-5) or somatic symptom severe depression were excluded)
disorder (DSM-5)
Barke et al., 2016 CSS Germany 500 73.6 29.1±10.4 NR Online community sample NR Yes (but individuals with current
[5] – Sample A illness were not excluded)
Bajcar et al., 2019 CSS, Poland 380 53.4 26.5±11.1 19-68 Community sample NR No
[13*] CSS-30
Uzun & Zencir, CSS Turkey 335 35.5 38.2±8.5 19-61 University employees NR No
2018 [14]
Selvi et al., 2018 CSS Turkey 337 55.8 21.8±5.2 16-55 University students NR No
[15]
Zangoulechi et al., CSS Iran 177 72.9 NR NR University students NR No
2018 [16]
Fergus, 2015 [33] CSS-30 USA 578 43.7 31.2±9.8 NR Online community sample Online crowdsourcing Yes (individuals with medical
platform - MTurk conditions were excluded)
Norr et al., 2015 CSS-30 USA 468 71.6 35.4±12.5 18-72 Online community sample Online crowdsourcing No
[34] platform - MTurk
Fergus & Spada, CSS-30 USA 337 50.1 33.3±11.5 18-65 Online sample using the Online crowdsourcing Yes (25.5% had medical
2017 [6] – Sample Internet for health searches platform - MTurk conditions)
1
 27

Fergus & Spada, CSS-30 USA 260 40.8 32.9±9.2 19-69 Online sample using the Online crowdsourcing Yes (29.2% had medical
2017 [6] – Sample Internet for health searches platform - MTurk conditions)
2
Fergus & Spada, CSS-30 USA 330 66.6 19.4±2.1 18-47 University undergraduate NR No
2018 [7] – Sample students
1
Fergus & Russell, CSS-30 USA 375 47.3 31.6±10.2 19-64 Online community sample Online crowdsourcing Yes (individuals with medical
2016 [35] platform - MTurk conditions were excluded)
Mathes et al., 2018 CSS-30 USA 462 64.3 36.6±12.9 18-77 Online community sample Online crowdsourcing No
[20] platform - MTurk
Gibler et al., 2019 CSS-30 USA 221 70.6 19.2±1.7 19-33 Undergraduate students NR Yes (percentage with various
[36] types of pain and related problems
was noted)
Bajcar & Babiak, CSS-30 Poland 207 58.9 31.5±13.0 19-64 Community sample Snowball method No
2019 [37]
Barke et al., 2016 CSS-15 Germany 292 76.4 24.2±4.1 NR Online community sample NR Yes (but individuals with current
[5] – Sample B illness were not excluded)
Fergus & Spada, CSS-15- USA 331 53.5 38.7±10.4 22-74 Online sample using the Online crowdsourcing No
2018 [7] – Sample Revised Internet for health searches platform – MTurk
2
Durak-Batigun et CS Turkey 250 49.6 36.6±14.3 NR Community sample NR Yes (individuals with
al., 2018 [8] – current/recently diagnosed mental
Sample 1 illness were excluded)
Durak-Batigun et CS Turkey 360 61.1 36.5±13.6 NR Community sample NR Yes (individuals with
al., 2018 [8] – current/recently diagnosed mental
Sample 2 illness were excluded)
Jokić-Begić et al., SCS-7 Croatia 507 72.2 33.2±12.3 18-71 Community sample using the Link to survey on No
2019 [9] – Sample Internet for health searches social networks, use of
1 mailing lists and
snowball method
Jokić-Begić et al., SCS-7 Croatia 379 75.5 31.8±10.9 18-71 Community sample using the Link to survey on No
2019 [9] – Sample Internet for health searches social networks, use of
2 mailing lists and
snowball method
Jokić-Begić et al., CSS, Croatia 594 83.8 30.3±10.4 18-73 Community sample using the Link to survey on No
2019 [9] – Sample SCS-4 Internet for health searches social networks, use of
3 mailing lists and
snowball method
 28

González-Rivera et BCS Puerto Rico/ 320 71.3 37.3±12.1 NR Community sample Advertisement paid on No
al., 2020 [29] USA social networks
Tatli et al., 2019 CTS Turkey 1200 42.2 NR 18-35 Community sample NR Yes (presence of “any health
[30] problem in the previous 6
months” was an inclusion
criterion)
NR: Not reported.

† Participation was restricted to English-speaking individuals from Australia, Canada, Ireland, New Zealand, the United Kingdom and the United States of America.

MTurk: Amazon’s Mechanical Turk; PROL: Prolific.

UK: United Kingdom; USA: United States of America.

BCS: Brief Cyberchondria Scale; CS: Cyberchondria Scale; CSS: Full, 33-item version of the Cyberchondria Severity Scale; CSS-30: A 30-item version of the
Cyberchondria Severity Scale; CSS-15: A 15-item version of the Cyberchondria Severity Scale; CSS-15-Revised: A 15-item version of the Cyberchondria Severity Scale
(CSS-15) modified to a 12-item version; CSS-12: A 12-item version of the Cyberchondria Severity Scale; CTS: Cyberchondria Tendency Scale; SCS-7: A preliminary 7-item
version of the Short Cyberchondria Scale; SCS-4: Final 4-item version of the Short Cyberchondria Scale.
 29

Table 2. Psychometric properties of the Cyberchondria Severity Scale (CSS) across the studies.
Article/study Factor structure (identification Correlations Correlations Internal consistency Total CSS score correlations with Correlations of the CSS subscales
of factors/subscales) between the between the CSS (Cronbach α), not other scales (excluding MMP subscale) with other
CSS subscales subscales reporting for the MMP scales
(excluding (excluding MMP subscale
MMP subscale) and
subscale) total CSS score
McElroy & Exploratory factor analysis: 0.52-0.67 NR - Total: 0.94 - DASS-21 Total: 0.40 - DASS-21 Total: 0.29-0.46
Shevlin, 1. Compulsion - Compulsion: 0.95 - DASS-21 Anxiety: 0.43 - DASS-21 Anxiety: 0.29-0.49
2014 [3] 2. Distress - Distress: 0.92 - DASS-21 Stress: 0.37 - DASS-21 Stress: 0.27-0.36
3. Excessiveness - Excessiveness: 0.85 - DASS-21 Depression: 0.24 - DASS-21 Depression: 0.20-0.34
4. Reassurance - Reassurance: 0.89
5. MMP
Fergus, 2014 Confirmatory factor analysis: 0.52-0.68 0.75-0.88 - Total: 0.95 - SHAI: 0.59 - SHAI: 0.33-0.61
[19] 1. Compulsion - Compulsion: 0.95 - DOCS: 0.49 - DOCS: 0.27-0.50
2. Distress - Distress: 0.95
3. Excessiveness - Excessiveness: 0.87
4. Reassurance - Reassurance: 0.88
5. MMP
Norr et al., Confirmatory factor analysis: 0.63-0.80 NR - Total: 0.95 NR - SHAI Thought Intrusion: 0.34-0.49
2015 [10] Bifactor modelling - Subscales: 0.86-0.95 - SHAI Fear of Illness: 0.36-0.52
1. General Cyberchondria
Factor
2. Specific Factors
(Compulsion, Distress,
Excessiveness, Reassurance,
MMP)
Norr et al., NR NR NR - Total: 0.95 - SHAI: 0.53 NR
2015 [11] - Subscales: 0.86-0.95 - ASI-3: 0.57
- IUS-12 Prospective: 0.38
- IUS-12 Inhibitory: 0.50
McElroy et Exploratory factor analysis: 0.34-0.54 NR - Total: NR NR NR
al., 2019 1. Compulsion - Compulsion: 0.93
[21**] 2. Distress - Distress: 0.92
3. Excessiveness - Excessiveness: 0.88
4. Reassurance - Reassurance: 0.82
5. MMP
 30

Starcevic et NR NR NR - Total: 0.95 NR NR

al., 2019 - Subscales: NR
[12]
Newby & NR NR NR - Total: 0.96 NR NR
McElroy, - Compulsion: 0.96
2020 [32**] - Distress: 0.95
- Excessiveness: 0.91
- Reassurance: 0.85
Barke et al., Principal component analysis: 0.26-0.59 0.65-0.85 - Total: 0.93 - mSHAI: 0.59 - mSHAI: 0.28-0.65
2016 [5] – 1. Compulsion - Compulsion: 0.90 - PHQ-15: 0.40 - PHQ-15: 0.23-0.37
German 2. Distress - Distress: 0.87 - CES-D: 0.31 - CES-D: 0.09-0.33
version 3. Excessiveness - Excessiveness: 0.74 - HCU: 0.29 - HCU: 0.18-0.25
4. Reassurance - Reassurance: 0.76
5. MMP
Bajcar et al., Confirmatory factor analysis: 0.61-0.75 0.78-0.89 - Total: 0.95 - SHAI Total: 0.56 - SHAI Total: 0.36-0.59
2019 [13*] † 1. Compulsion - Compulsion: 0.88 - SHAI Illness Likelihood: 0.53 - SHAI Illness Likelihood: 0.36-0.56
– Polish 2. Distress - Distress: 0.92 - SHAI Negative Consequences of - SHAI Negative Consequences of
version 3. Excessiveness - Excessiveness: 0.87 Illness: 0.33 Illness: 0.19-0.39
4. Reassurance - Reassurance: 0.80 - DOCS Total: 0.38 - DOCS Total: 0.23-0.43
5. MMP - DOCS Contamination: 0.21 - DOCS Contamination: 0.17-0.22
- DOCS Responsibility: 0.44 - DOCS Responsibility: 0.29-0.48
- DOCS Unacceptable Thoughts: - DOCS Unacceptable Thoughts:
0.29 0.14-0.32
- DOCS Symmetry: 0.30 - DOCS Symmetry: 0.17-0.34
Uzun & Confirmatory factor analysis: NR NR - Total: 0.89 - DASS-21 Total: 0.33 - DASS-21 Total: 0.17-0.33
Zencir, 2018 1. Compulsion - Subscales: 0.65-0.85 - DASS-21 Anxiety: 0.31 - DASS-21 Anxiety: 0.15-0.30
[14] †† – 2. Distress (Cronbach α value for - DASS-21 Stress: 0.33 - DASS-21 Stress: 0.19-0.33
Turkish 3. Excessiveness the MMP subscale is - DASS-21 Depression: 0.22 - DASS-21 Depression: 0.08-0.23
version 4. Reassurance not reported, but is
5. MMP included in this range)
Selvi et al., Confirmatory factor analysis: 0.53-0.80 0.72-0.83 - Total: 0.91 - IAT: 0.45 - IAT: 0.34-0.43
2018 [15] – 1. Compulsion - Subscales: 0.78-0.87 - HAI: 0.23 - HAI: 0.07-0.32
Turkish 2. Distress - ASI-3: 0.39 - ASI-3: 0.25-0.40
version 3. Excessiveness
4. Reassurance
5. MMP
Zangoulechi NR NR NR NR - MOCI subscales: 0.27-0.40 NR
et al., 2018 - ASI-R subscales: 0.17-0.38
 31

[16] – - IUS Prospective: 0.34

Iranian - IUS Inhibitory: 0.39
version
Jokić-Begić NR NR NR - Total: 0.91 NR NR
et al., 2019 - Compulsion: 0.91
[9] – - Distress: 0.91
Croatian - Excessiveness: NR
version - Reassurance: NR
NR: Not reported.

† Test-retest reliability figures for CSS subscales ranging from 0.58 to 0.76 reported over a period of 3 months in 59 participants.
†† Test-retest reliability figures for CSS subscales ranging from 0.53 to 0.71 and for total CSS score of 0.65 reported over a period of 2 weeks in 66 participants.

MMP: Mistrust of Medical Professionals.

ASI-3: Anxiety Sensitivity Index-3; ASI-R: Anxiety Sensitivity Index Revised; CES-D: Center for Epidemiologic Studies Depression Scale; DASS-21: Depression, Anxiety
and Stress Scale – 21-Item Version; DOCS: Dimensional Obsessive-Compulsive Scale; HAI: Health Anxiety Inventory; HCU: Health Care Utilization Questionnaire; IAT:
Internet Addiction Test; IUS: Intolerance of Uncertainty Scale; IUS-12: Intolerance of Uncertainty Scale – Short Form; MOCI: Maudsley Obsessive-Compulsive Inventory;
mSHAI: Modified Version of the Short Health Anxiety Inventory; PHQ-15: Patient Health Questionnaire; SHAI: Short Health Anxiety Inventory.
 32

Table 3. Status of the Mistrust of Medical Professionals (MMP) subscale of the Cyberchondria Severity Scale (CSS).
Article/study Internal Correlations Correlations Correlations between the MMP subscale and Recommendations/suggestions/comments about the
consistency of the between the MMP between the MMP other scales of relevance for validity suitability of the MMP subscale
MMP subscale subscale and other subscale and total
(Cronbach α) CSS subscales CSS score
McElroy & 0.75 -0.04 to 0.23 NR - DASS-21 Total: 0.15 - None.
Shevlin, 2014 - DASS-21 Anxiety: 0.14
[3] - DASS-21 Stress: 0.16
- DASS-21 Depression: 0.09
Fergus, 2014 0.87 -0.01 to 0.27 0.26 - SHAI: 0.21 - It does not assess the same construct as the other 4 CSS
[19] - DOCS: 0.11 subscales.
Norr et al., 2015 NR <0.01 to 0.28 NR - SHAI Thought Intrusion: 0.01 - Consider as a construct distinct from cyberchondria as
[10] - SHAI Fear of Illness: -0.08 measured by the CSS.
- Consider removal from the CSS.
Norr et al., 2015 0.81 NR 0.12† - SHAI: -0.02 - “Future research should attempt to further clarify the
[11] - ASI-3: -0.001 role of the MMP factor with regard to cyberchondria as a
- IUS-12 Prospective: 0.07 construct.”
- IUS-12 Inhibitory: -0.10
McElroy et al., 0.64 -0.16 to 0.26 NR NR - Items from this subscale were excluded from the short
2019 [21**] form of the CSS due to low internal consistency and
nonsignificant and weak correlations with all of the other
four subscales of the CSS.
Newby & 0.81 NR NR NR - Reductions in health anxiety after treatment were partly
McElroy, 2020 mediated by reductions in cyberchondria severity, except
[32**] for scores on the MMP subscale.
Barke et al., 0.89 0.03-0.21 0.34 - mSHAI: 0.13 - It does not seem to be a part of the CSS construct.
2016 [5] – - PHQ-15: 0.10 - Propose to “drop” it from the CSS.
German version - CES-D: 0.11
- HCU: 0.09
Bajcar et al., 0.62 -0.08 to 0.08 0.15 - SHAI Total: 0.08 - Excluded from the cyberchondria model.
2019 [13*] – - SHAI Illness Likelihood: 0.07 - Potentially theoretically related to cyberchondria.
Polish version - SHAI Negative Consequences of Illness: 0.06 - May be considered a “control variable”.
- DOCS Total: 0.07
- DOCS Contamination: 0.02
- DOCS Responsibility: 0.05
- DOCS Unacceptable Thoughts: 0.06
- DOCS Symmetry: 0.08
 33

Uzun & Zencir, NR NR NR - DASS-21 Total: 0.09 - “Seems unnecessary to the CSS construct”.
2018 [14] – - DASS-21 Anxiety: 0.14 - Consider removal from the CSS.
Turkish version - DASS-21 Stress: 0.02
- DASS-21 Depression: 0.09
Selvi et al., 0.64 -0.05 to 0.44 0.41 - IAT: -0.07 - None.
2018 [15] – - HAI: 0.01
Turkish version - ASI-3: 0.02
Jokić-Begić et 0.68 NR NR NR - None.
al., 2019 [9] –
Croatian version
NR: Not reported.

† Total CSS score does not include items of the MMP subscale.

ASI-3: Anxiety Sensitivity Index-3; CES-D: Center for Epidemiologic Studies Depression Scale; DASS-21: Depression, Anxiety and Stress Scale – 21-Item Version; DOCS:
Dimensional Obsessive-Compulsive Scale; HAI: Health Anxiety Inventory; HCU: Health Care Utilization Questionnaire; IAT: Internet Addiction Test; IUS-12: Intolerance
of Uncertainty Scale – Short Form; mSHAI: Modified Version of the Short Health Anxiety Inventory; PHQ-15: Patient Health Questionnaire; SHAI: Short Health Anxiety
Inventory.
 34

Table 4. Psychometric properties of the shorter/modified versions of the Cyberchondria Severity Scale (CSS) across the studies.
Article/study Correlations Correlations between the CSS Internal consistency Total CSS score correlations with other scales Correlations of the CSS subscales
between the CSS subscales and total CSS score (Cronbach α) with other scales
subscales
CSS-30 NR 0.78-0.88 - Total: 0.96 - SHAI: 0.58 NR
(Fergus, 2014) - DOCS: 0.49
[19]
CSS-30 0.45-0.66 0.78-0.88 - Total: 0.95 - Whiteley Index: 0.62 - Whiteley Index: 0.38-0.65
(Fergus, 2015) - Compulsion: 0.95 - ASI-3-Physical: 0.55 - ASI-3-Physical: 0.33-0.59
[33] - Distress: 0.95 - ASI-3-Cognitive: 0.55 - ASI-3-Cognitive: 0.35-0.53
- Excessiveness: 0.87 - ASI-3-Social: 0.40 - ASI-3-Social: 0.21-0.39
- Reassurance: 0.88 - IUS-12-Inhibitory: 0.47 - IUS-12-Inhibitory: 0.24-0.49
- IUS-12-Prospective: 0.33 - IUS-12-Prospective: 0.17-0.36
CSS-30 0.61-0.78 NR -Total and subscales: NR - SHAI: 0.39-0.60
(Norr et al., 2015) 0.86-0.97 - DOCS-Contamination: 0.33-0.45
[34] - DOCS-Harm Avoidance: 0.36-0.55
- DOCS-Unacceptable Thoughts:
0.28-0.41
- DOCS-Symmetry: 0.29-0.40
- PANAS-NA: 0.23-0.43
CSS-30 NR NR - Total: 0.95 - PIUQ: 0.59 NR
(Fergus & Spada, - Subscales: NR - Whiteley Index - 6: 0.67
2017) [6] - ASI-3-Physical: 0.64
- ASI-3-Cognitive: 0.62
- ASI-3-Social: 0.52
- IUS-12-Inhibitory: 0.52
- IUS-12-Prospective: 0.44
- PANAS-NA: 0.58
- MCQ-HA-U: 0.66
- MCQ-HA-B: 0.58
- MCQ-HA-C: 0.49
CSS-30 NR NR - Total: 0.95 - Whiteley Index - 6: 0.56 NR
(Fergus & Spada, - Subscales: NR - ASI-3-Physical: 0.37
2018) [7] - ASI-3-Cognitive: 0.44
- ASI-3-Social: 0.35
- IUS-12-Inhibitory: 0.31
- IUS-12-Prospective: 0.24
- PANAS-NA: 0.34
 35

- MCQ-HA-U: 0.51
- MCQ-HA-B: 0.49
- MCQ-HA-C: 0.32
- BARI: 0.49
- SSQ: 0.33
CSS-30 0.42-0.61 0.73-0.86 - Total: NR - MIHT-Affective: 0.57 - MIHT-Affective: 0.27-0.62
(Fergus & - Subscales: 0.86-0.96 - MIHT-Cognitive: 0.46 - MIHT-Cognitive: 0.22-0.46
Russell, 2016) - MIHT-Perceptual: 0.13 - MIHT-Perceptual: -0.05-0.28
[35] - MIHT-Behavioural: 0.32 - MIHT-Behavioural: 0.15-0.32
- DOCS-Contamination: 0.41 - DOCS-Contamination: 0.22-0.43
- DOCS-Responsibility: 0.44 - DOCS-Responsibility: 0.18-0.48
- DOCS-Thoughts: 0.32 - DOCS-Thoughts: 0.08-0.35
- DOCS-Symmetry: 0.26 - DOCS-Symmetry: 0.14-0.27
- PANAS-NA: 0.31 - PANAS-NA: 0.14-0.30
CSS-30 0.50-0.75 NR - Total: 0.96 NR - SHAI Thought Intrusion: 0.34-0.49
(Mathes et al., - Compulsion: 0.96 - SHAI Fear of Illness: 0.33-0.52
2018) [20] † - Distress: 0.95 - WHOQOL-Physical Health:
- Excessiveness: 0.87 -0.24 to -0.17
- Reassurance: 0.85 - WHOQOL-Psychological Health:
-0.24 to -0.10
- WHOQOL-Social Relationships:
-0.20 to -0.05
- WHOQOL-Environment:
-0.22 to -0.10
- SDS Occupational Functioning:
0.42-0.70
- SDS Social Functioning: 0.41-0.70
- SDS Family Functioning: 0.45-0.73
- Physical Health Care Utilisation:
0.19-0.33
- Mental Health Care Utilisation:
0.11-0.25
CSS-30 0.64-0.71 0.82-0.89 -Total and subscales: - SHAI: 0.58 - SHAI: 0.43-0.55
(Gibler et al., 0.83-0.96 - PCS Total: 0.52 - PCS Total: 0.38-0.50
2019) [36] - PANAS-NA: 0.44 - PANAS-NA: 0.30-0.43
CSS-30 – Polish NR 0.80-0.90 - Total: 0.95 - SHAI Total: 0.56 NR
version - Subscales: NR - SHAI Illness Likelihood: 0.52
- SHAI Negative Consequences of Illness: 0.33
 36

(Bajcar et al., - DOCS Total: 0.38

2019) [13*] †† - DOCS Contamination: 0.22
- DOCS Responsibility: 0.44
- DOCS Unacceptable Thoughts: 0.28
- DOCS Symmetry: 0.30
CSS-30 – Polish NR NR - Total: 0.95 - SHAI: 0.52 NR
version - Subscales: NR - DOCS: 0.41
(Bajcar & Babiak, - RSES: -0.25
2019) [37]
CSS-15 – German 0.02-0.45 0.30-0.76 - Total: 0.82 - mSHAI: 0.57 - mSHAI: 0.18-0.61
version - Compulsion: 0.86 - PHQ-15: 0.35 - PHQ-15: 0.08-0.31
(Barke et al., - Distress: 0.83 - CES-D: 0.28 - CES-D: 0.08-0.31
2016) [5] ††† - Excessiveness: 0.71
- Reassurance: 0.67
- Mistrust of Medical
Professionals: 0.69
CSS-15-Revised NR NR - Total: 0.88 - Whiteley Index - 6: 0.61 NR
(Fergus & Spada, - Subscales: NR - DOCS: 0.56
2018) [7] - MCQ-HA-U: 0.64
- MCQ-HA-B: 0.47
- MCQ-HA-C: 0.40
- BARI: 0.58
- SSQ: 0.36
- BFI-2-XS-N: 0.41
CSS-12 NR NR - Total: 0.90 - SHAI: 0.53 NR
(McElroy et al., - Compulsion: 0.87 - GAD-7: 0.30
2019) [21**] - Distress: 0.87
†††† - Excessiveness: 0.83
- Reassurance: 0.73
NR: Not reported.

† The factor structure of the 30-item version of the CSS (Mathes et al., 2018) comprised 4 factors/subscales identified by means of confirmatory factor analysis: 1)
Compulsion; 2) Distress; 3) Excessiveness; 4) Reassurance. However, bifactor modelling with General Cyberchondria factor and Specific factors was the best-fitting model.
†† The factor structure of the 30-item version of the CSS (Bajcar et al., 2019) comprised 4 factors/subscales identified by means of confirmatory factor analysis: 1)
Compulsion; 2) Distress; 3) Excessiveness; 4) Reassurance.
††† The factor structure of the CSS-15 comprised 5 factors/subscales identified by means of confirmatory factor analysis: 1) Compulsion; 2) Distress; 3) Excessiveness; 4)
Reassurance; 5) Mistrust of Medical Professionals.
 37

††††† The factor structure of the CSS-12 comprised 4 factors/subscales identified by means of exploratory factor analysis: 1) Compulsion; 2) Distress; 3) Excessiveness; 4)
Reassurance.

CSS-30: 30-item version of the Cyberchondria Severity Scale; CSS-15: 15-item version of the Cyberchondria Severity Scale; CSS-15-Revised: 15-item version (CSS-15) of
the Cyberchondria Severity Scale modified to a 12-item version; CSS-12: 12-item version of the Cyberchondria Severity Scale.

ASI-3: Anxiety Sensitivity Index-3; BARI: Beliefs about Rituals Inventory; BFI-2-XS-N: Big Five Inventory-2-Extra Short Neuroticism Scale; CES-D: Center for
Epidemiologic Studies Depression Scale; DOCS: Dimensional Obsessive-Compulsive Scale; GAD-7: Generalized Anxiety Disorder 7-Item Scale; IUS-12: Intolerance of
Uncertainty Scale – Short Form; MCQ-HA-B: Metacognitions Questionnaire – Health Anxiety – Biased Thinking; MCQ-HA-C: Metacognitions Questionnaire – Health
Anxiety – Thoughts Can Cause Illness; MCQ-HA-U: Metacognitions Questionnaire – Health Anxiety – Thoughts Are Uncontrollable; MIHT: Multidimensional Inventory of
Hypochondriacal Traits; mSHAI: Modified Version of the Short Health Anxiety Inventory; PANAS-NA: Positive and Negative Affect Schedule – Negative Affect; PCS:
Pain Catastrophizing Scale; PHQ-15: Patient Health Questionnaire; PIUQ: Problematic Internet Use Questionnaire; RSES: Rosenberg Self-Esteem Scale; SDS: Sheehan
Disability Scale; SHAI: Short Health Anxiety Inventory; SSQ: Stop Signals Questionnaire; WHOQOL: World Health Organisation Quality of Life Assessment.
 38

Table 5. Cyberchondria scales other than the Cyberchondria Severity Scale and its modifications.
Scale Country of Number Response Cut-off point and/or scoring Factor structure (identification of Internal consistency (Cronbach α) Correlations with other
origin of items format details factors/subscales) scales
CS [8] Turkey 27 5-point NR Exploratory factor analysis and - Total: 0.93 Total score correlations:
confirmatory factor analysis: - Anxiety-Increasing Factors: 0.88 - INAS: 0.43 (subscale
1. Anxiety-Increasing Factors - Compulsion/Hypochondria: 0.83 correlations: 0.25-0.44)
2. Compulsion/Hypochondria - Anxiety-Reducing Factors: 0.80 - HAI: 0.53 (subscale
3. Anxiety-Reducing Factors - Physician-Patient Interaction: 0.80 correlations: 0.23-0.64)
4. Physician-Patient Interaction - Non-Functional Internet Usage: 0.84 - BSI: 0.33 (subscale
5. Non-Functional Internet Usage correlations: 0.10-0.41)
SCS Croatia 4 5-point NR Exploratory factor analysis and - 0.73 - CSS: NR
[9] confirmatory factor analysis: - HAQ: 0.48*; 0.53*
One factor/dimension - ASI: 0.35*; 0.51*
BCS Puerto 10 5-point 0-7: Minimal symptoms Exploratory factor analysis: - Total: 0.92 Total score correlations:
[29] Rico/USA 8-13: Mild symptoms 1. Online Health Information Search - Subscales: NR - IAS: 0.68
14-24: Moderate symptoms 2. Health Anxiety - GAD-7: 0.43
25-40: Severe symptoms
CTS Turkey 30 5-point 30-60: Low level of cyber- Exploratory factor analysis and - Total: 0.94 NR
[30] chondria tendencies (CT) confirmatory factor analysis: - Reflection: 0.94
60-90: Moderate level of CT 1. Reflection - Information seeking: 0.87
90-120: High level of CT 2. Information Seeking
NR: Not reported.

* These correlations were based on the longer (7-item) version of the Short Cyberchondria Scale.

BCS: Brief Cyberchondria Scale; CS: Cyberchondria Scale; CTS: Cyberchondria Tendency Scale; SCS: Short Cyberchondria Scale.

ASI: Anxiety Sensitivity Index; BSI: Brief Symptom Inventory; CSS: Cyberchondria Severity Scale; GAD-7: Generalized Anxiety Disorder 7-Item Scale; HAI: Health
Anxiety Inventory; HAQ: Health Anxiety Questionnaire; IAS: Illness Anxiety Scale; INAS: Internet Addiction Scale.
39